CS 161, Lesson 2, in 50 slides. It covers the nine remaining design principles: defense in depth, least privilege, separation of responsibility, complete mediation, Shannon's Maxim, fail-safe defaults, designing security in from the start, the Trusted Computing Base, and TOCTTOU races. It is anchored to textbook sections 1.5 to 1.13 and to Saltzer & Schroeder (1975).
Subject: Computer Security · 64 slides · applied lesson
Open the interactive version of this deck · Homework for this lesson
Title
CS 161 · Lesson 2 of 45
Defense in depth · least privilege · mediation · Shannon · fail-safe · TCB · TOCTTOU
Objectives
Warm-up
Discussion prompt
Before we open L02 · Security Principles II (the remaining nine: Defense in Depth → TOCTTOU): without looking back, what was the main idea of L01 · Security Principles I (Threat Models, Human Factors, Economics), and what could you do by the end of it that you could not do before?
Hint: One sentence for the idea, one for the skill. If the second one is blank, that is the part to revisit.
Answer:
CS 161, Lesson 1. It covers threat models, attacker assumptions, human factors, security economics, and the principle of detecting what you cannot prevent, across 20 slides anchored to textbook sections 1.1 to 1.4 and to Saltzer & Schroeder (1975).
Concept
Six of today's nine principles were named in a single 1975 paper. They recur in every later unit — memory safety, crypto, web, network.
Matching
Match the pairs
From Where these came from — match each one to what it actually does. The descriptions have been shuffled.
Why: Layer & minimize, Mediate & open, Default & design are easy to tell apart while they are sitting next to their descriptions and much harder afterwards, which is what this checks.
Section
Part 1 · §1.5–1.7
Concept
Layer multiple kinds of defense so an attacker must breach all of them, not just one.
Defense in depth — Independent layers stacked so a single failure is not catastrophic — high walls, then a moat, then inner walls.
But it is not foolproof (siege cannons ignore walls), and it has diminishing returns — the 101st wall rarely justifies its cost. That caveat is just §1.3 economics again.
Counterexample
Discussion prompt
Layer multiple kinds of defense so an attacker must breach all of them, not just one.
That is stated as though it always holds. Do one of two things: produce a case where it fails, or say precisely what rules such a case out. "It just does" is not on the menu.
Hint: Hunt at the extremes first — zero, one, negative, empty, equal. If every extreme survives, the reason they survive is the proof.
Ranking
Put in order
Put the moves of §1.5 Two detectors: series vs parallel into the order they have to happen.
Why: These are the moves of the worked example in the order it makes them, and each one is set up by the one before it. A detector trades these two error types off against each other; composing them lets you steer the trade-off.
Worked example
Set up two detectors D1, D2, each with a false-alarm rate and a missed-attack rate
Why: A detector trades these two error types off against each other; composing them lets you steer the trade-off.
Wire them in PARALLEL (either alert fires a response)
Why: Fewer attacks slip past (lower miss rate) but more false alarms — you catch more, and cry wolf more.
Wire them in SERIES (both must alert)
Why: Far fewer false alarms, but more misses — you only act when both agree, so quiet attacks get through.
Choose by cost
Why: There is no free lunch: the right wiring depends on whether a missed attack or a false alarm costs you more — economics decides the layering.
Blank canvas
Draw it
Draw what §1.5 Two detectors: series vs parallel just did — the shape of it, not the line-by-line working. One picture, labels only where you need them. Then check it against the steps: anything you could not draw is a step you followed rather than understood.
Concept
Give each program exactly the access it needs to do its job — and nothing more.
It does not lower the probability of failure; it lowers the expected cost of failure. Less privilege ⇒ less harm when (not if) the program is subverted.
Analogy
Discussion prompt
Explain §1.6 Least privilege by analogy to something with no Computer Security in it at all — a queue, a recipe, a map, a bank balance, whatever fits. Then say where your analogy breaks.
Hint: An analogy that never breaks is not an analogy, it is the same idea wearing a hat. Find the seam — that is the part that is actually new.
Answer:
Give each program exactly the access it needs to do its job — and nothing more.
Intuition
On Unix every program you launch inherits all your user's privileges. A text editor opening one file can also read, modify, or delete every other file you own — far more than it needs. Windows: 'just as lousy.'
Mobile does better: each app is sandboxed and cannot touch other apps' data, so one compromised app does limited harm.
Ask yourself: when a buffer overflow (next unit) fully takes over a process, what determines the blast radius? The privileges that process was holding.
Explain it
Discussion prompt
Explain §1.6 Unix is 'pretty lousy' at this to a student a year behind you. No notation, no jargon they have not met — and it still has to be true.
Hint: If your explanation needs a symbol they have never seen, you are describing the notation rather than the idea.
Answer:
Mobile does better: each app is sandboxed and cannot touch other apps' data, so one compromised app does limited harm.
Anomaly
Predict first
A student writes this, and it looks reasonable:
A media server has an exploitable bug.
It is wrong. Say what breaks — and say it before you turn the page.
Correct: Confuses reducing impact with reducing likelihood — the bug is just as reachable.
A media server has an exploitable bug.
Why: Confuses reducing impact with reducing likelihood — the bug is just as reachable.
Trap
A media server has an exploitable bug.
Claim: running it as a low-privilege user PREVENTS the compromise
Why: Confuses reducing impact with reducing likelihood — the bug is just as reachable.
A media server has an exploitable bug.
Claim: it still gets compromised, but the attacker inherits only the server's tiny privilege set
Why: §1.6: least privilege does not change the probability of failure, only the cost. The exploit lands; the damage is contained.
Concept
Split privilege so no single person or program has complete power. Require more than one party to approve a sensitive action.
A nuclear silo needs two launch officers to agree. The bar: it is far likelier that one party is malicious than that all parties collude.
Intuition
You pay a cashier for a ticket; ten feet later a different employee tears it and drops half in a lockbox. Why two people for one transaction?
Insider fraud. With one employee, they could wave a friend in for free. Splitting 'sell' and 'admit' across two people means a free entry now requires both to collude — much harder.
Ask yourself: this is the human-world version of which later crypto idea? (Threshold / multi-signature schemes.)
Section
Part 2 · §1.8–1.9
Concept
Check every access to every object — no exceptions, no caching a decision past the point it can go stale.
Reference monitor — A single, unbypassable point through which all access must pass — so getting that one point right protects everything behind it.
Definition probe
Sort into buckets
Every line below is part of the definition of Defense in depth or of Reference monitor — one or the other, never both. Put each where it belongs.
Intuition
An airport funnels every passenger through a few security checkpoints. Get those checkpoints right and there is no way around them, and you protect hundreds of gates with a handful of controls.
The failure mode to watch: any path that reaches a gate without passing a checkpoint. In software, a cached 'already authorized' flag that outlives the authorization is exactly that bypass — which sets up TOCTTOU (Part 4).
Concept
Assume the attacker knows the system — its algorithms, hardware, and defenses. Never rely on the design being secret.
The related Kerckhoff's Principle (crypto unit): a system must stay secure even when everything except the key is public — and keys are easy to rotate when leaked, unlike a whole codebase.
Explain it
Discussion prompt
Explain §1.9 Shannon's Maxim to a student a year behind you. No notation, no jargon they have not met — and it still has to be true.
Hint: If your explanation needs a symbol they have never seen, you are describing the notation rather than the idea.
Answer:
The related Kerckhoff's Principle (crypto unit): a system must stay secure even when everything except the key is public — and keys are easy to rotate when leaked, unlike a whole codebase.
Anomaly
Predict first
A student writes this, and it looks reasonable:
A vendor: 'Only ~100 people understand our protocol, so attackers won't bother.'
It is wrong. Say what breaks — and say it before you turn the page.
Correct: Self-defeating: success makes the system popular, which raises the incentive to reverse-engineer it — and then the obscurity is gone.
A vendor: 'Only ~100 people understand our protocol, so attackers won't bother.'
Why: Self-defeating: success makes the system popular, which raises the incentive to reverse-engineer it — and then the obscurity is gone.
Trap
A vendor: 'Only ~100 people understand our protocol, so attackers won't bother.'
Treat secrecy of design as the security mechanism
Why: Self-defeating: success makes the system popular, which raises the incentive to reverse-engineer it — and then the obscurity is gone.
A vendor: 'Only ~100 people understand our protocol, so attackers won't bother.'
Assume the design is public and rest security on a rotatable secret key
Why: §1.9 / Kerckhoff: security must survive full knowledge of the system. Obscurity may be a thin extra layer, never the foundation.
Section
Part 3 · §1.10–1.11
Concept
When a mechanism fails or crashes, it should fail toward the secure state, not away from it. Default to deny; allow only what is explicitly permitted.
Analogy
Discussion prompt
Explain §1.10 Fail-safe defaults by analogy to something with no Computer Security in it at all — a queue, a recipe, a map, a bank balance, whatever fits. Then say where your analogy breaks.
Hint: An analogy that never breaks is not an analogy, it is the same idea wearing a hat. Find the seam — that is the part that is actually new.
Answer:
When a mechanism fails or crashes, it should fail toward the secure state, not away from it. Default to deny; allow only what is explicitly permitted.
Step zero
Discussion prompt
§1.10 Firewall: fail-safe vs fail-open — before any calculation: what is the plan? Name the moves in order, in plain English, without doing the arithmetic.
Hint: It starts with: Adopt a default-deny policy
Answer:
Worked example
Adopt a default-deny policy
Why: The firewall must explicitly decide to forward each packet; anything not permitted is dropped.
Now crash the firewall
Why: With default-deny, a crashed firewall forwards nothing — it fails closed. Inconvenient, but safe.
Contrast a fail-open design
Why: If a crash let all packets through, the attacker's whole job is to induce a crash, then 'the fort is wide open.' Fail-open hands the attacker a one-step win.
Concept
Retrofitting security onto a finished system is painful: you are stuck with an architecture that may not permit least privilege, complete mediation, or a clean TCB.
Backwards compatibility makes it worse — you can inherit the worst insecurities of every prior version. Bake the decomposition in while the architecture is still soft.
Sorting
Sort into buckets
These are the pieces of L02 · Security Principles II (the remaining nine: Defense in Depth → TOCTTOU), out of order. Put each one back under the part of the lesson it belongs to.
Section
Part 4 · §1.12–1.13
Concept
TCB — The portion of a system that must operate correctly for the security goals to hold. Everything outside it can misbehave — even maliciously — without defeating those goals.
Make the TCB unbypassable (no path around it), tamper-resistant (nothing outside can modify its code or state), and verifiable (simple enough to actually check).
Counterexample
Discussion prompt
Make the TCB unbypassable (no path around it), tamper-resistant (nothing outside can modify its code or state), and verifiable (simple enough to actually check).
That is stated as though it always holds. Do one of two things: produce a case where it fails, or say precisely what rules such a case out. "It just does" is not on the menu.
Hint: Hunt at the extremes first — zero, one, negative, empty, equal. If every extreme survives, the reason they survive is the proof.
Intuition
Goal: only authorized users may SSH into the server. Walk the dependency chain.
| Component | In TCB? | Why |
|---|---|---|
| SSH daemon | Yes | It makes the auth decision — a bug here lets in unauthorized users |
| Operating system | Yes | It can tamper with the daemon's memory, so it must be correct |
| CPU | Yes | We rely on it to execute the daemon's instructions faithfully |
| Web browser on same host | No (hopefully) | OS memory protection isolates it from the daemon — its bugs can't grant SSH access |
Ask yourself: if the browser can reach the daemon's memory, what just happened to your TCB? (It silently grew — and the 'tamper-resistant' property failed.)
Discrimination
Sort into buckets
Sort these by In TCB?, from memory, without looking back at §1.12 What's in the TCB for 'only authorized SSH…. Telling them apart on the spot is the skill; the table is only where the answer happens to be written down.
Ranking
Put in order
Put the moves of §1.12 Why a small TCB wins: the defect-rate math into the order they have to happen.
Why: These are the moves of the worked example in the order it makes them, and each one is set up by the one before it. An empirical baseline — more code means proportionally more latent bugs.
Worked example
Take the industry error rate: 1–5 defects per 1,000 lines of code
Why: An empirical baseline — more code means proportionally more latent bugs.
A 1,000-line TCB → about 1–5 defects
Why: Few enough that you can plausibly find and remove the security-relevant ones.
A 100,000-line TCB → about 100–500 defects
Why: Now auditing every exploitable flaw is hopeless. (Windows XP put ~40 million lines in the TCB — a virtual certainty of vulnerabilities.)
Conclusion: shed code
Why: Decompose so as much code as possible lives OUTSIDE the TCB. Smaller TCB ⇒ fewer defects to chase ⇒ better odds of a secure system.
Blank canvas
Draw it
Draw what §1.12 Why a small TCB wins: the defect-rate math just did — the shape of it, not the line-by-line working. One picture, labels only where you need them. Then check it against the steps: anything you could not draw is a step you followed rather than understood.
Concept
A time-of-check-to-time-of-use bug: state is validated at the check, then changes before the use, so the stale check authorizes something it shouldn't. It is the classic failure of complete mediation under concurrency.
Ranking
Put in order
Put the moves of §1.13 The $100 → $200 ATM withdrawal into the order they have to happen.
Why: These are the moves of the worked example in the order it makes them, and each one is set up by the one before it. The check (b ≥ w) has passed, but the use (set balance) has not happened yet — the decision is now stale-able.
Worked example
procedure withdraw(amount w) {
// contact central server to get balance
1. let b := balance
2. if b < w, abort
// contact central server to set the balance
3. set balance := b - w
4. give w dollars to the user
}Balance is $100; you request w = $100 at ATM-1, and pause it right after step 2
Why: The check (b ≥ w) has passed, but the use (set balance) has not happened yet — the decision is now stale-able.
Walk to ATM-2 and withdraw $100 to completion
Why: ATM-2 reads balance $100, passes its own check, sets balance to $0, dispenses $100. Account is now empty.
| Step | ATM-1 sees | ATM-2 sees | Real balance |
|---|---|---|---|
| ATM-1 reads b | b=100 | — | 100 |
| ATM-1 check passes, PAUSE | 100 ≥ 100 ✓ | — | 100 |
| ATM-2 full withdraw | (paused) | 100→0, pays $100 | 0 |
| ATM-1 resumes step 3–4 | sets 100−100=0, pays $100 | — | 0 (but $200 paid) |
Unpause ATM-1; it completes steps 3–4 on its stale check
Why: ATM-1 never re-checked the balance after the pause, so it pays a second $100. You withdrew $200 from a $100 account — the check and the use disagreed.
Discrimination
Sort into buckets
Sort these by ATM-2 sees, from memory, without looking back at §1.13 The $100 → $200 ATM withdrawal. Telling them apart on the spot is the skill; the table is only where the answer happens to be written down.
Anomaly
Predict first
A student writes this, and it looks reasonable:
Fix attempt: move the balance read closer to the write.
It is wrong. Say what breaks — and say it before you turn the page.
Correct: A smaller race window is still a race window — under enough tries (persistent attacker) it still hits.
Fix attempt: make check-and-use one indivisible step.
Why: A smaller race window is still a race window — under enough tries (persistent attacker) it still hits.
Trap
Fix attempt: move the balance read closer to the write.
Shrink the gap between check and use, call it fixed
Why: A smaller race window is still a race window — under enough tries (persistent attacker) it still hits.
Fix attempt: make check-and-use one indivisible step.
Use an atomic compare-and-set / a lock / a transaction so no other withdraw can interleave
Why: §1.13: the only real fix removes the interleaving entirely — check and use must be one atomic operation, restoring complete mediation.
Two truths and a lie
Sort into buckets
Some of these hold up and some are the exact mistakes this lesson is built to prevent. Sort them.
Pattern
Edge cases
Discussion prompt
The nine-principle checklist works on the cases you have just seen. Push it to the edge: what is the most degenerate input it still handles — empty, zero, one item, everything equal — and what is the first case where it stops being true? Name the case, not just "it breaks".
Hint: Try the smallest legal input, then the largest, then the one where two things collide. Methods are specified at their edges; the middle takes care of itself.
Answer:
Elimination
Eliminate the wrong options
Which principle most directly limits how much harm the attacker can do, and why?
3 of these 4 are wrong. Strike them one at a time, and say what rules each one out before you strike the next. The survivor is the answer.
Survives elimination: A
Why: §1.6: least privilege does not reduce the probability of compromise, but it bounds the cost. A sandboxed reader hands the attacker only the reader's minimal privileges, so the blast radius is small.
Check
A PDF reader is taken over by a memory-corruption exploit while a user opens an email attachment.
Check your understanding
Which principle most directly limits how much harm the attacker can do, and why?
Answer: A
Why: §1.6: least privilege does not reduce the probability of compromise, but it bounds the cost. A sandboxed reader hands the attacker only the reader's minimal privileges, so the blast radius is small.
Prediction
Predict first
To double-withdraw, between which two steps must the attacker stall ATM-1?
Answer it in your own words, now, with nothing to choose from. The options are on the next slide — and picking the right one off a list is an easier skill than producing it.
Correct: Between step 2 and step 3 — after the check passes but before the balance is updated.
Why: The race is the gap between the check (step 2) and the use (step 3). Stalling there lets ATM-2 complete a full withdrawal, so ATM-1 later acts on a balance value that is no longer true.
Check
Re-read the withdraw() procedure: (1) read balance, (2) abort if too low, (3) set balance, (4) dispense.
Check your understanding
To double-withdraw, between which two steps must the attacker stall ATM-1?
Answer: B
Why: The race is the gap between the check (step 2) and the use (step 3). Stalling there lets ATM-2 complete a full withdrawal, so ATM-1 later acts on a balance value that is no longer true.
Prediction
Predict first
Which component is (hopefully) NOT in the TCB for this goal?
Answer it in your own words, now, with nothing to choose from. The options are on the next slide — and picking the right one off a list is an easier skill than producing it.
Correct: An unprivileged web browser installed on the same machine.
Why: §1.12: the TCB is whatever must be correct for the goal to hold. The daemon, OS, and CPU all can directly defeat the goal, so they're in. A correctly isolated, unprivileged browser is kept out by OS memory protection — its bugs cannot grant SSH access.
Check
Security goal: only authorized users may log into a Linux server over SSH.
Check your understanding
Which component is (hopefully) NOT in the TCB for this goal?
Answer: C
Why: §1.12: the TCB is whatever must be correct for the goal to hold. The daemon, OS, and CPU all can directly defeat the goal, so they're in. A correctly isolated, unprivileged browser is kept out by OS memory protection — its bugs cannot grant SSH access.
Elimination
Eliminate the wrong options
Why does §1.10 prefer design X even though it can lock out legitimate traffic on failure?
3 of these 4 are wrong. Strike them one at a time, and say what rules each one out before you strike the next. The survivor is the answer.
Survives elimination: B
Why: §1.10 fail-safe defaults: a mechanism should fail toward the secure state. Default-deny fails closed, so inducing a crash gains the attacker nothing; default-allow fails open, turning a crash into a one-step breach.
Check
Two firewall designs: (X) default-deny, drops anything not explicitly allowed; (Y) default-allow, forwards anything not explicitly blocked.
Check your understanding
Why does §1.10 prefer design X even though it can lock out legitimate traffic on failure?
Answer: B
Why: §1.10 fail-safe defaults: a mechanism should fail toward the secure state. Default-deny fails closed, so inducing a crash gains the attacker nothing; default-allow fails open, turning a crash into a one-step breach.
Concept
Concept
Concept
Connect it up
Draw it
One page, no notation unless you need it: draw how these connect — Layer & Minimize · Mediate & Open · Default & Design · The TCB & TOCTTOU. Put an arrow wherever one of them is what makes another possible, and label the arrow with why.
Recap
You can now place each of the nine principles, name what it actually buys you, find a TOCTTOU window, and argue for a small, verifiable TCB.
| Principle | § | Buys you |
|---|---|---|
| Defense in depth | 1.5 | Survive one failed layer |
| Least privilege | 1.6 | Smaller blast radius |
| Separation | 1.7 | No single point of betrayal |
| Complete mediation | 1.8 | No unchecked access path |
| Shannon's Maxim | 1.9 | Security without secret designs |
| Fail-safe defaults | 1.10 | Crashes fail closed |
| Design from start | 1.11 | Architecture that allows the above |
| Minimal TCB | 1.12 | Fewer defects to trust |
| TOCTTOU-aware | 1.13 | Atomic check-and-use |
Want this taught 1-on-1? Alexander tutors Computer Security — $55/session, free consultation.