L01 · Security Principles I (Threat Models, Human Factors, Economics)

CS 161, Lesson 1. It covers threat models, attacker assumptions, human factors, security economics, and the principle of detecting what you cannot prevent, across 20 slides anchored to textbook sections 1.1 to 1.4 and to Saltzer & Schroeder (1975).

Subject: Computer Security · 39 slides · applied lesson

Open the interactive version of this deck · Homework for this lesson

What this lesson covers

The lesson, slide by slide

1. Thinking Like an Attacker

Title

CS 161 · Lesson 1 of 45

Threat models · human factors · security economics · detect-and-respond

2. By the end of this lesson you can…

Objectives

  1. State the textbook's six attacker assumptions and apply the 'persistent and lucky' rule to a concrete system.
  2. Build a threat model for a login page: name the assets, the attacker, and their capabilities.
  3. Explain why usable security is a socio-technical requirement, not a UI afterthought.
  4. Decide how much to spend on a control using the expected-loss test (security is economics).
  5. Say when a system should stop trying to prevent and instead detect and respond.

3. Threat Models

Section

Part 1 · §1.1

4. A threat model names your attacker

Concept

Before you can secure anything you must answer: who attacks, why, from where, and with what resources? That answer is your threat model.

Threat model — A model of who your attacker is and what resources they have. Every security claim is relative to one — 'secure' is meaningless without 'against whom'.

Cited: textbook §1.1 Know your threat model; the formal design principles trace to Saltzer & Schroeder (1975).

5. Break it if you can: A threat model names your attacker

Counterexample

Discussion prompt

Before you can secure anything you must answer: who attacks, why, from where, and with what resources? That answer is your threat model.

That is stated as though it always holds. Do one of two things: produce a case where it fails, or say precisely what rules such a case out. "It just does" is not on the menu.

Hint: Hunt at the extremes first — zero, one, negative, empty, equal. If every extreme survives, the reason they survive is the proof.

Answer:

Cited: textbook §1.1 Know your threat model; the formal design principles trace to Saltzer & Schroeder (1975).

6. Assume the attacker is persistent and lucky

Intuition

The textbook tells you to grant the attacker more than you'd expect. The most-missed assumption:

If an attack succeeds 1 in 1,000,000 times, assume the attacker runs it 1,000,000 times. Rare is not safe — rare just means slow.

Ask yourself: which of these assumptions does a 'nobody would bother attacking my app' mindset quietly violate?

7. By analogy: Assume the attacker is persistent and lucky

Analogy

Discussion prompt

Explain Assume the attacker is persistent and lucky by analogy to something with no Computer Security in it at all — a queue, a recipe, a map, a bank balance, whatever fits. Then say where your analogy breaks.

Hint: An analogy that never breaks is not an analogy, it is the same idea wearing a hat. Find the seam — that is the part that is actually new.

Answer:

The textbook tells you to grant the attacker more than you'd expect. The most-missed assumption:

8. What has to happen first: Model the threat to a login page

Ranking

Put in order

Put the moves of Model the threat to a login page into the order they have to happen.

  1. Name the asset
  2. Name the attacker and motive
  3. Enumerate capabilities
  4. Derive the requirement

Why: These are the moves of the worked example in the order it makes them, and each one is set up by the one before it. The thing of value here is the set of user credentials and the authenticated session they unlock — not 'the page'.

9. Model the threat to a login page

Worked example

Name the asset

Why: The thing of value here is the set of user credentials and the authenticated session they unlock — not 'the page'.

Name the attacker and motive

Why: A criminal seeking account takeover for resale, plus a credential-stuffing bot reusing leaked passwords. Different motives, same door.

Enumerate capabilities

Why: Can send unlimited requests, can read any error message you return, knows your framework — by the §1.1 assumptions, assume all three.

Derive the requirement

Why: Because the attacker is persistent (one-in-a-million → a million tries), a weak-but-rare-to-guess password is not enough; you need rate limiting and a slow hash. The threat model produced the control.

10. Draw the shape of it: Model the threat to a login page

Blank canvas

Draw it

Draw what Model the threat to a login page just did — the shape of it, not the line-by-line working. One picture, labels only where you need them. Then check it against the steps: anything you could not draw is a step you followed rather than understood.

11. Something is wrong here: 'attackers only do the economically rational thing'

Anomaly

Predict first

A student writes this, and it looks reasonable:

Modeling a small blog with no payment data.

It is wrong. Say what breaks — and say it before you turn the page.

Correct: Treats the attacker as a rational investor who only targets high-value systems.

Modeling a small blog with no payment data.

Why: Treats the attacker as a rational investor who only targets high-value systems.

12. Trap: 'attackers only do the economically rational thing'

Trap

The trap

Modeling a small blog with no payment data.

Conclude: 'no one would spend effort attacking this', so skip patching

Why: Treats the attacker as a rational investor who only targets high-value systems.

The fix

Modeling a small blog with no payment data.

Conclude: it will be attacked anyway — for its compute, its mailing list, or just for the dare

Why: §1.1: every system is a potential target, and the attacker is persistent. Low value ≠ no target (the fish-tank thermometer had no 'value' either).

13. Humans & Economics

Section

Part 2 · §1.2–1.3

14. Security is a socio-technical problem

Concept

A control that people route around is not a control. Programmers are human and make mistakes; users chase convenience and will subvert security that gets in their way.

Cited: textbook §1.2 Consider human factors; Saltzer & Schroeder (1975) call this psychological acceptability.

15. Teach it back: Security is a socio-technical problem

Explain it

Discussion prompt

Explain Security is a socio-technical problem to a student a year behind you. No notation, no jargon they have not met — and it still has to be true.

Hint: If your explanation needs a symbol they have never seen, you are describing the notation rather than the idea.

Answer:

A control that people route around is not a control. Programmers are human and make mistakes; users chase convenience and will subvert security that gets in their way.

16. Usability IS security

Intuition

The 'remind me later' button on a security update is a vulnerability surface: every delayed patch is extra exposure time, and most users delay.

Contrast the NSA crypto token shaped like an ordinary door key: insert and turn. An 18-year-old in the field uses it correctly with near-zero training — the secure path is the easy path.

Ask yourself: in your own design, is the secure action also the lowest-effort action? If not, humans will defeat it.

17. Teach it back: Usability IS security

Explain it

Discussion prompt

Explain Usability IS security to a student a year behind you. No notation, no jargon they have not met — and it still has to be true.

Hint: If your explanation needs a symbol they have never seen, you are describing the notation rather than the idea.

Answer:

Contrast the NSA crypto token shaped like an ordinary door key: insert and turn. An 18-year-old in the field uses it correctly with near-zero training — the secure path is the easy path.

18. Security is economics

Concept

No system is 100% secure against all attacks. You defend against a level of attack, and the expected benefit of a defense should scale to the expected cost of the attack.

The one-liner: there is no point putting a $100 lock on a $1 item. Cited: textbook §1.3 Security is economics.

19. By analogy: Security is economics

Analogy

Discussion prompt

Explain Security is economics by analogy to something with no Computer Security in it at all — a queue, a recipe, a map, a bank balance, whatever fits. Then say where your analogy breaks.

Hint: An analogy that never breaks is not an analogy, it is the same idea wearing a hat. Find the seam — that is the part that is actually new.

Answer:

No system is 100% secure against all attacks. You defend against a level of attack, and the expected benefit of a defense should scale to the expected cost of the attack.

20. What has to happen first: When is the $3,000 safe worth it?

Ranking

Put in order

Put the moves of When is the $3,000 safe worth it? into the order they have to happen.

  1. Read the ratings
  2. Estimate expected loss
  3. Apply the test
  4. Flip the inputs

Why: These are the moves of the worked example in the order it makes them, and each one is set up by the one before it. A TL-15 safe resists common tools for 15 minutes (~$3,000); a TL-30 resists for 30 minutes and costs more.

21. When is the $3,000 safe worth it?

Worked example

Read the ratings

Why: A TL-15 safe resists common tools for 15 minutes (~$3,000); a TL-30 resists for 30 minutes and costs more. Rating = attacker-time purchased.

Estimate expected loss

Why: If the contents and max liability are $2,000, you are protecting $2,000 of value — that is the ceiling on rational spend.

Apply the test

Why: Spending $3,000 to protect $2,000 fails the economics test; the lock costs more than the item. Buy down value or accept the risk instead.

Flip the inputs

Why: Now protecting $5M of bearer bonds against a 25-minute-capable crew: the TL-30 is cheap insurance. Same math, opposite decision — the threat model and the value both moved.

22. Draw the shape of it: When is the $3,000 safe worth it?

Blank canvas

Draw it

Draw what When is the $3,000 safe worth it? just did — the shape of it, not the line-by-line working. One picture, labels only where you need them. Then check it against the steps: anything you could not draw is a step you followed rather than understood.

23. Something is wrong here: 'maximum security is always the goal'

Anomaly

Predict first

A student writes this, and it looks reasonable:

A $4,000-value internal database.

It is wrong. Say what breaks — and say it before you turn the page.

Correct: Optimizes security in isolation, ignoring what is actually at stake.

A $4,000-value internal database.

Why: Optimizes security in isolation, ignoring what is actually at stake.

24. Trap: 'maximum security is always the goal'

Trap

The trap

A $4,000-value internal database.

Buy $50,000/yr of controls because 'you can never be too secure'

Why: Optimizes security in isolation, ignoring what is actually at stake.

The fix

A $4,000-value internal database.

Cap spend near the expected loss; put the saved budget where value is higher

Why: §1.3: defense scales to expected attack cost. Over-spending here is itself a failure — it starves higher-value assets.

25. Which of these survive contact with L01 · Security Principles I (Threat Models…?

Two truths and a lie

Sort into buckets

Some of these hold up and some are the exact mistakes this lesson is built to prevent. Sort them.

Holds up
Before you can secure anything you must answer: who attacks, why, from where, and with what resources? That answer is your threat model.; The textbook tells you to grant the attacker more than you'd expect. The most-missed assumption:; A control that people route around is not a control. Programmers are human and make mistakes; users chase convenience and will subvert security that gets in their way.
Breaks
Modeling a small blog with no payment data.; A $4,000-value internal database.
sound
These are stated as this lesson states them — each one survives the edge cases L01 · Security Principles I (Threat Models, Human Factors, Economics) puts it through.
flawed
Each of these is lifted from a trap in this deck: reasonable-sounding, and wrong in a way that only shows up once you rely on it.

26. Detect & Respond

Section

Part 3 · §1.4

27. Detect if you can't prevent

Concept

Because economics (§1.3) means you cannot prevent everything, mature systems assume some attacks land — and invest in detecting them and responding fast, not only in walls.

This is the seed of logging, intrusion detection, and incident response — we cash it out in Lesson 45. Cited: textbook §1.4 Detect if you can't prevent.

Ask yourself: for the login page in our worked example, what would you log so that a breach is noticed — not just hopefully prevented?

28. Break it if you can: Detect if you can't prevent

Counterexample

Discussion prompt

Because economics (§1.3) means you cannot prevent everything, mature systems assume some attacks land — and invest in detecting them and responding fast, not only in walls.

That is stated as though it always holds. Do one of two things: produce a case where it fails, or say precisely what rules such a case out. "It just does" is not on the menu.

Hint: Hunt at the extremes first — zero, one, negative, empty, equal. If every extreme survives, the reason they survive is the proof.

Answer:

Ask yourself: for the login page in our worked example, what would you log so that a breach is noticed — not just hopefully prevented?

29. Rebuild the recipe: The Lesson-1 checklist

Ranking

Put in order

These are the steps of The Lesson-1 checklist, scrambled. Put them back in order before the next slide shows you.

  1. Threat model first (§1.1): who, why, from where, with what resources — grant the six assumptions.
  2. Persistent and lucky (§1.1): one-in-a-million → a million tries; every system is a target.
  3. Human factors (§1.2): make the secure path the easy path, or humans defeat it.
  4. Economics (§1.3): spend proportional to expected loss — no $100 lock on a $1 item.
  5. Detect if you can't prevent (§1.4): assume some attacks land; log and respond.

Why: This is the order the recipe itself gives. Recalling the sequence without the slide in front of you is the difference between recognising the method and being able to run it — most of what goes wrong in practice is a step done out of turn.

30. The Lesson-1 checklist

Pattern

  1. Threat model first (§1.1): who, why, from where, with what resources — grant the six assumptions.
  2. Persistent and lucky (§1.1): one-in-a-million → a million tries; every system is a target.
  3. Human factors (§1.2): make the secure path the easy path, or humans defeat it.
  4. Economics (§1.3): spend proportional to expected loss — no $100 lock on a $1 item.
  5. Detect if you can't prevent (§1.4): assume some attacks land; log and respond.

31. Where does it stop working: The Lesson-1 checklist

Edge cases

Discussion prompt

The Lesson-1 checklist works on the cases you have just seen. Push it to the edge: what is the most degenerate input it still handles — empty, zero, one item, everything equal — and what is the first case where it stops being true? Name the case, not just "it breaks".

Hint: Try the smallest legal input, then the largest, then the one where two things collide. Methods are specified at their edges; the middle takes care of itself.

Answer:

  1. Threat model first (§1.1): who, why, from where, with what resources — grant the six assumptions.
  2. Persistent and lucky (§1.1): one-in-a-million → a million tries; every system is a target.
  3. Human factors (§1.2): make the secure path the easy path, or humans defeat it.
  4. Economics (§1.3): spend proportional to expected loss — no $100 lock on a $1 item.
  5. Detect if you can't prevent (§1.4): assume some attacks land; log and respond.

32. Rule out three: Checkpoint 1 — attacker assumptions

Elimination

Eliminate the wrong options

Which statement best matches the textbook's attacker model for this blog?

3 of these 4 are wrong. Strike them one at a time, and say what rules each one out before you strike the next. The survivor is the answer.

  • A. It's too low-value to be worth attacking, so a basic threat model isn't needed.
  • B. If an exploit works one time in a million, assume the attacker runs it a million times.
  • C. You may assume the attacker does not know which CMS or server software you run.
  • D. A single attacker can only target one machine at a time, so chaining isn't a concern.

Survives elimination: B

Why: §1.1 tells you to assume the attacker is persistent and lucky: a one-in-a-million success rate just means they try a million times. Low value does not remove you as a target — every system is a potential target (the fish-tank thermometer).

33. Checkpoint 1 — attacker assumptions

Check

You're modeling threats to a hobby blog with no logins and no payment data. Decide on paper first.

Check your understanding

Which statement best matches the textbook's attacker model for this blog?

  • A. It's too low-value to be worth attacking, so a basic threat model isn't needed.
  • B. If an exploit works one time in a million, assume the attacker runs it a million times. (correct)
  • C. You may assume the attacker does not know which CMS or server software you run.
  • D. A single attacker can only target one machine at a time, so chaining isn't a concern.

Answer: B

Why: §1.1 tells you to assume the attacker is persistent and lucky: a one-in-a-million success rate just means they try a million times. Low value does not remove you as a target — every system is a potential target (the fish-tank thermometer).

Why A tempts people
Underestimates persistence and the 'every system is a target' assumption — low value ≠ no target.
Why C tempts people
Backwards: the model assumes the attacker DOES know your OS and likely software vulnerabilities.
Why D tempts people
Backwards: the model assumes the attacker can coordinate and chain attacks across your whole network at once.

34. Rule out three: Checkpoint 2 — which principle, and the fix?

Elimination

Eliminate the wrong options

Which principle does this violate, and what is the right move?

3 of these 4 are wrong. Strike them one at a time, and say what rules each one out before you strike the next. The survivor is the answer.

  • A. Complete mediation — start checking every access to the dataset.
  • B. Security is economics — cap spend near the expected loss and redeploy the rest.
  • C. Defense in depth — add even more layers of control on top.
  • D. Fail-safe defaults — switch the dataset to deny-by-default access.

Survives elimination: B

Why: §1.3: the expected benefit of a defense should scale to the expected cost of the attack. Spending $50,000 to protect $4,000 is a $100 lock on a $1 item — over-spending is itself a failure because it starves higher-value assets.

35. Checkpoint 2 — which principle, and the fix?

Check

A team spends $50,000/yr protecting a dataset whose total worth and maximum breach liability is $4,000.

Check your understanding

Which principle does this violate, and what is the right move?

  • A. Complete mediation — start checking every access to the dataset.
  • B. Security is economics — cap spend near the expected loss and redeploy the rest. (correct)
  • C. Defense in depth — add even more layers of control on top.
  • D. Fail-safe defaults — switch the dataset to deny-by-default access.

Answer: B

Why: §1.3: the expected benefit of a defense should scale to the expected cost of the attack. Spending $50,000 to protect $4,000 is a $100 lock on a $1 item — over-spending is itself a failure because it starves higher-value assets.

Why A tempts people
Complete mediation is about checking every access — a real principle, but it doesn't address the cost-vs-value mismatch here.
Why C tempts people
Adding layers increases spend in the wrong direction; the problem is too much spend for the value, not too little.
Why D tempts people
Fail-safe defaults concern what happens on error or by default, not how much budget to allocate.

36. Primary sources & where to read more

Concept

37. Where does each piece belong: L01 · Security Principles I (Threat Models…

Sorting

Sort into buckets

These are the pieces of L01 · Security Principles I (Threat Models, Human Factors, Economics), out of order. Put each one back under the part of the lesson it belongs to.

Threat Models
A threat model names your attacker; Assume the attacker is persistent and lucky; Model the threat to a login page
Humans & Economics
Security is a socio-technical problem; Usability IS security; Security is economics
Detect & Respond
Detect if you can't prevent; The Lesson-1 checklist; Primary sources & where to read more
s1
Threat Models is where L01 · Security Principles I (Threat Models, Human Factors, Economics) puts A threat model names your attacker, Assume the attacker is persistent and lucky, Model the threat to a login page. Knowing which part of the lesson a problem belongs to is most of knowing which method to reach for.
s2
Humans & Economics is where L01 · Security Principles I (Threat Models, Human Factors, Economics) puts Security is a socio-technical problem, Usability IS security, Security is economics. Knowing which part of the lesson a problem belongs to is most of knowing which method to reach for.
s3
Detect & Respond is where L01 · Security Principles I (Threat Models, Human Factors, Economics) puts Detect if you can't prevent, The Lesson-1 checklist, Primary sources & where to read more. Knowing which part of the lesson a problem belongs to is most of knowing which method to reach for.

38. Connect it up: L01 · Security Principles I (Threat Models, Human Factors, Economics)

Connect it up

Draw it

One page, no notation unless you need it: draw how these connect — Threat Models · Humans & Economics · Detect & Respond. Put an arrow wherever one of them is what makes another possible, and label the arrow with why.

39. Recap — Lesson 1

Recap

You can now build a threat model, grant the six attacker assumptions, apply the economics test, and say when to detect rather than prevent.

Principle§The one thing to remember
Threat model1.1Secure means nothing without 'against whom'
Persistent & lucky1.11-in-a-million → a million tries
Human factors1.2Secure path must be the easy path
Economics1.3Spend ∝ expected loss
Detect ≥ prevent1.4Assume some attacks land; log + respond

Sources

  1. CS 161 Computer Security Textbook §1.1–1.4 (Threat model, Human factors, Economics, Detect if you can't prevent) — Wagner, Weaver, Kao, Shakir, Law & Ngai, UC Berkeley
  2. The Protection of Information in Computer Systems — J. Saltzer & M. Schroeder, Proceedings of the IEEE 63(9), 1975 — the seminal source for security design principles
  3. Casino hacked through a fish-tank thermometer — Reported by Darktrace, 2017 — illustrates 'every system is a potential target'

Want this taught 1-on-1? Alexander tutors Computer Security — $55/session, free consultation.

Book on Wyzant · Text (657) 465-8108