CS 161, Lesson 4, in 50 slides and code mode. It covers binary, hexadecimal, and two's-complement representation, the toolchain that turns source into a running process, the ELF sections (supplemental), and the four-section 32-bit C memory layout - code, static, heap, and stack - with the direction each one grows. It is anchored to textbook sections 2.1 to 2.3, and all conversions were verified.
Subject: Computer Security · 49 slides · code lesson
Open the interactive version of this deck · Homework for this lesson
Title
CS 161 · Lesson 4 of 45 · Systems unit begins
binary & hex · two's complement · the toolchain · 32-bit memory layout
Objectives
Warm-up
Discussion prompt
Before we open L04 · Numbers, the Toolchain, and 32-bit C Memory Layout: without looking back, what was the main idea of L03 · Threat Modeling (STRIDE/DFD) + TCB & TOCTTOU Labs, and what could you do by the end of it that you could not do before?
Hint: One sentence for the idea, one for the skill. If the second one is blank, that is the part to revisit.
Answer:
CS 161, Lesson 3 and Quiz 3, in 48 slides. It covers structured threat modeling with STRIDE and data-flow diagrams, marked as supplemental, then a TCB-identification exercise from section 1.12 and the classic access()/open() symlink TOCTTOU lab from section 1.13. It applies all 13 principles from Lessons 1 and 2.
Concept
The textbook fixes this convention: 'unless otherwise stated we'll be using 32-bit systems.' That means 4-byte words, addresses from 0x00000000 to 0xFFFFFFFF, and the registers eip/ebp/esp (next lesson).
| System | Address size | Address space |
|---|---|---|
| 32-bit (this course) | 32 bits = 4 bytes | 2^32 bytes = 4 GiB |
| 64-bit (real machines) | 64 bits = 8 bytes | 2^64 bytes |
Every offset we compute in Weeks 2–5 assumes 4-byte words. Memorize that now — it's the #1 source of wrong exam answers when students drift to 64-bit.
Comparison
Comparison matrix
From We are on a 32-bit system from here on: refill the Address size column from what you know. The rest of the table is as it appeared.
| System | Address size | Address space |
|---|---|---|
| 32-bit (this course) | 32 bits = 4 bytes | 2^32 bytes = 4 GiB |
| 64-bit (real machines) | 64 bits = 8 bytes | 2^64 bytes |
Section
Part 1 · §2.1
Concept
Hex is base-16. Each digit 0–F encodes exactly 4 bits, so two hex digits = one byte. That is the entire reason memory dumps use hex — one byte is always two clean characters.
binary hex decimal
0000 0 0
1010 A 10
1111 F 15
10110101 B5 181| Binary | Group into nibbles | Hex | Decimal |
|---|---|---|---|
| 10110101 | 1011 | 0101 | B5 | 181 |
| 00101010 | 0010 | 1010 | 2A | 42 |
| 11111111 | 1111 | 1111 | FF | 255 |
Trade off
Comparison matrix
From One hex digit = one nibble = 4 bits: every row here is a choice with a cost. Fill the Decimal column, then say which row you would actually pick and what you give up for it.
| Binary | Group into nibbles | Hex | Decimal |
|---|---|---|---|
| 10110101 | 1011 | 0101 | B5 | 181 |
| 00101010 | 0010 | 1010 | 2A | 42 |
| 11111111 | 1111 | 1111 | FF | 255 |
Ranking
Put in order
Put the moves of Convert 0b10110101 by hand into the order they have to happen.
Why: These are the moves of the worked example in the order it makes them, and each one is set up by the one before it. 128 + 32 + 16 + 4 + 1 = 181. Only the columns with a 1 contribute.
Worked example
1 0 1 1 0 1 0 1
128 64 32 16 8 4 2 1Sum the set bits
Why: 128 + 32 + 16 + 4 + 1 = 181. Only the columns with a 1 contribute.
Group into nibbles for hex
Why: 1011 = B (11), 0101 = 5. So 0b10110101 = 0xB5.
| Form | Value |
|---|---|
| binary | 10110101 |
| hex | 0xB5 |
| decimal | 181 |
Check by reversing
Why: 0xB5 = 11×16 + 5 = 176 + 5 = 181. Agrees — conversion verified.
Pattern
Step through it
Step through Convert 0b10110101 by hand one row at a time. What is driving the change, and what would the row after the last one be?
Concept
A signed 32-bit int stores negatives in two's complement: to get −x, invert all bits of x and add 1. The top bit is the sign.
1 = 0x00000001
~1 = 0xFFFFFFFE (invert)
+1 = 0xFFFFFFFF (add one) => -1| Value | 32-bit hex | Note |
|---|---|---|
| 0 | 0x00000000 | all zeros |
| -1 | 0xFFFFFFFF | all ones |
| INT_MAX | 0x7FFFFFFF | sign bit 0, rest 1 = 2147483647 |
| INT_MIN | 0x80000000 | sign bit 1, rest 0 = -2147483648 |
Comparison
Comparison matrix
From Two's complement: how negatives are stored: refill the Note column from what you know. The rest of the table is as it appeared.
| Value | 32-bit hex | Note |
|---|---|---|
| 0 | 0x00000000 | all zeros |
| -1 | 0xFFFFFFFF | all ones |
| INT_MAX | 0x7FFFFFFF | sign bit 0, rest 1 = 2147483647 |
| INT_MIN | 0x80000000 | sign bit 1, rest 0 = -2147483648 |
Anomaly
Predict first
A student writes this, and it looks reasonable:
The 4 bytes 0xFFFFFFFF are in memory.
It is wrong. Say what breaks — and say it before you turn the page.
Correct: Reads the bits as unsigned without checking the variable's type — the same bits mean different numbers.
The 4 bytes 0xFFFFFFFF are in memory.
Why: Reads the bits as unsigned without checking the variable's type — the same bits mean different numbers.
Trap
The 4 bytes 0xFFFFFFFF are in memory.
Assume they always mean 4,294,967,295
Why: Reads the bits as unsigned without checking the variable's type — the same bits mean different numbers.
The 4 bytes 0xFFFFFFFF are in memory.
Read them as the TYPE says: signed int → −1, unsigned int → 4,294,967,295
Why: Bits carry no sign on their own; the declared type decides interpretation. This exact ambiguity becomes the integer-conversion bug in Lesson 9.
Break the constraint
Discussion prompt
The rule this trap just fixed:
Bits carry no sign on their own; the declared type decides interpretation. This exact ambiguity becomes the integer-conversion bug in Lesson 9.
Now break it on purpose. Build a case that violates it and follow the consequences until something visibly fails. Where does the failure first show up — and would you have noticed it if you had not been looking?
Hint: The dangerous rules are the ones whose violation still produces an answer. If yours fails loudly, try to find one that fails quietly.
Answer:
Reads the bits as unsigned without checking the variable's type — the same bits mean different numbers.
Section
Part 2 · §2.2
Concept
C does not run directly. A pipeline transforms text into a running process, and each stage has one job.
source.c
-> preprocessor (expand #include, #define)
-> compiler (C -> assembly)
-> assembler (assembly -> object file, raw bytes)
-> linker (resolve symbols across .o + libraries)
-> executable
-> loader (map into virtual memory)
-> process| Stage | Input → Output | Resolves |
|---|---|---|
| Preprocessor | text → text | macros, includes |
| Compiler | C → assembly | syntax, types, codegen |
| Assembler | asm → object (.o) | mnemonics → machine bytes |
| Linker | .o + libs → exe | cross-file symbol addresses |
| Loader | exe → process | map sections into memory |
Counterexample
Discussion prompt
C does not run directly. A pipeline transforms text into a running process, and each stage has one job.
That is stated as though it always holds. Do one of two things: produce a case where it fails, or say precisely what rules such a case out. "It just does" is not on the menu.
Hint: Hunt at the extremes first — zero, one, negative, empty, equal. If every extreme survives, the reason they survive is the proof.
Intuition
The compiler sees one file at a time. When main.c calls helper() defined in util.c, the compiler emits a placeholder — it cannot know helper's final address.
The linker sees all the object files at once and patches every placeholder to a real address (symbol resolution). That's why a missing function is a link error, not a compile error.
Ask yourself: if you get 'undefined reference to helper', which stage failed — and which stage was perfectly happy?
Analogy
Discussion prompt
Explain Why the linker is its own stage by analogy to something with no Computer Security in it at all — a queue, a recipe, a map, a bank balance, whatever fits. Then say where your analogy breaks.
Hint: An analogy that never breaks is not an analogy, it is the same idea wearing a hat. Find the seam — that is the part that is actually new.
Answer:
The compiler sees one file at a time. When main.c calls helper() defined in util.c, the compiler emits a placeholder — it cannot know helper's final address.
Concept
⊕ Supplemental — beyond the textbook's four-section model. A real ELF executable splits the program's static parts into named sections:
| Section | Holds | Writable? |
|---|---|---|
| .text | machine code | no (executable) |
| .rodata | string literals, const data | no |
| .data | initialized globals/statics | yes |
| .bss | zero-initialized globals/statics | yes (no file bytes) |
The exam-relevant model is still the book's four regions (next part). Know .text/.data/.bss/.rodata for real binaries, but don't expect them on a CS 161 stack diagram.
Trade off
Comparison matrix
From ⊕ ELF sections (real-world detail): every row here is a choice with a cost. Fill the Holds column, then say which row you would actually pick and what you give up for it.
| Section | Holds | Writable? |
|---|---|---|
| .text | machine code | no (executable) |
| .rodata | string literals, const data | no |
| .data | initialized globals/statics | yes |
| .bss | zero-initialized globals/statics | yes (no file bytes) |
Section
Part 3 · §2.3
Concept
At runtime the OS hands the process one contiguous address space. The book divides it into four regions, lowest address to highest:
| Region | Holds | Grows |
|---|---|---|
| Code | executable instructions | fixed |
| Static | globals, static vars, constants | fixed |
| Heap | malloc'd data | UP ↑ (toward higher addr) |
| Stack | locals, call frames | DOWN ↓ (toward lower addr) |
Heap and stack grow toward each other from opposite ends of the space. The stack starts high and grows down; the heap starts low and grows up.
Discrimination
Sort into buckets
Sort these by Grows, from memory, without looking back at Four regions, low address to high. Telling them apart on the spot is the skill; the table is only where the answer happens to be written down.
Ranking
Put in order
Put the moves of Place each variable in its region into the order they have to happen.
Why: These are the moves of the worked example in the order it makes them, and each one is set up by the one before it. Globals and statics live for the whole program, so they sit in the fixed static section (g initialized, s zero-initialized).
Worked example
int g = 7; // A
static int s; // B
const char *msg = "hi"; // C: pointer + literal
void f(void) {
int local; // D
int *p = malloc(16); // E: pointer D-area, block heap
}g and s → static region
Why: Globals and statics live for the whole program, so they sit in the fixed static section (g initialized, s zero-initialized).
the literal "hi" → static (read-only); the pointer msg → wherever msg is declared
Why: String literals are constants in static/.rodata; the pointer variable itself follows its own scope.
| Var | Declared as | Region |
|---|---|---|
| g | global int | static |
| s | static int | static |
| "hi" | string literal | static (read-only) |
| local | function local | stack |
| *p block | malloc(16) | heap |
Verify the split
Why: local and the pointer p sit on the stack; only the 16 bytes from malloc live on the heap. The pointer is on the stack; the pointee is on the heap — a distinction we exploit in Lesson 10.
Comparison
Comparison matrix
From Place each variable in its region: refill the Region column from what you know. The rest of the table is as it appeared.
| Var | Declared as | Region |
|---|---|---|
| g | global int | static |
| s | static int | static |
| "hi" | string literal | static (read-only) |
| local | function local | stack |
| *p block | malloc(16) | heap |
Intuition
Each new function call allocates its frame at a lower address than its caller. So a local buffer sits at a lower address than the saved return address that lives above it.
But a write into that buffer (e.g. copying a string) moves from low to high addresses — upward, toward the return address. Growth direction (down) and write direction (up) are opposite. Hold that thought: it is the buffer overflow in one sentence.
Ask yourself: if locals are below the return address and writes go upward, what does an over-long write eventually reach?
Sorting
Sort into buckets
These are the pieces of L04 · Numbers, the Toolchain, and 32-bit C Memory Layout, out of order. Put each one back under the part of the lesson it belongs to.
Anomaly
Predict first
A student writes this, and it looks reasonable:
A buffer is filled by copying bytes into it.
It is wrong. Say what breaks — and say it before you turn the page.
Correct: Conflates frame ALLOCATION direction with data WRITE direction — they are not the same thing.
A buffer is filled by copying bytes into it.
Why: Conflates frame ALLOCATION direction with data WRITE direction — they are not the same thing.
Trap
A buffer is filled by copying bytes into it.
Assume bytes are written from high address to low because 'the stack grows down'
Why: Conflates frame ALLOCATION direction with data WRITE direction — they are not the same thing.
A buffer is filled by copying bytes into it.
Frames are allocated downward, but bytes are written upward (low → high) within the buffer
Why: §2.3: allocating a new frame decrements the pointer (down), but a string copy fills buf[0], buf[1], … at increasing addresses (up). The overflow reaches the return address precisely because writes go up.
Two truths and a lie
Sort into buckets
Some of these hold up and some are the exact mistakes this lesson is built to prevent. Sort them.
int stores negatives in two's complement: to get −x, invert all bits of x and add 1. The top bit is the sign.; C does not run directly. A pipeline transforms text into a running process, and each stage has one job.0xFFFFFFFF are in memory.; A buffer is filled by copying bytes into it.Concept
Every process believes it owns the full 0x00000000–0xFFFFFFFF space. The OS gives each one that illusion via virtual memory — two processes can both 'have' address 0x08048000 mapped to different physical RAM.
We treat addresses as virtual throughout this unit; the physical mapping (CS 61C / 162) doesn't change any exploit arithmetic.
Counterexample
Discussion prompt
We treat addresses as virtual throughout this unit; the physical mapping (CS 61C / 162) doesn't change any exploit arithmetic.
That is stated as though it always holds. Do one of two things: produce a case where it fails, or say precisely what rules such a case out. "It just does" is not on the menu.
Hint: Hunt at the extremes first — zero, one, negative, empty, equal. If every extreme survives, the reason they survive is the proof.
Ranking
Put in order
These are the steps of The Lesson-4 recipe, scrambled. Put them back in order before the next slide shows you.
Why: This is the order the recipe itself gives. Recalling the sequence without the slide in front of you is the difference between recognising the method and being able to run it — most of what goes wrong in practice is a step done out of turn.
Pattern
Edge cases
Discussion prompt
The Lesson-4 recipe works on the cases you have just seen. Push it to the edge: what is the most degenerate input it still handles — empty, zero, one item, everything equal — and what is the first case where it stops being true? Name the case, not just "it breaks".
Hint: Try the smallest legal input, then the largest, then the one where two things collide. Methods are specified at their edges; the middle takes care of itself.
Answer:
Check
A 32-bit signed int variable holds the bytes 0xFFFFFFFF. Solve on paper first.
Check your understanding
What value does it hold, and why?
Answer: A
Why: For a signed 32-bit int, 0xFFFFFFFF is two's-complement −1 (invert 1 → 0xFFFFFFFE, add 1 → 0xFFFFFFFF). The unsigned reading would be 4,294,967,295, but the type here is signed.
Elimination
Eliminate the wrong options
Which toolchain stage produced this error?
3 of these 4 are wrong. Strike them one at a time, and say what rules each one out before you strike the next. The survivor is the answer.
Survives elimination: C
Why: The declaration satisfied the compiler (it knew helper's signature), so compilation succeeded. The error appears when the linker tries to resolve the call to a definition across all object files and finds none — a classic link-time symbol-resolution failure.
Check
Your build prints: 'undefined reference to helper'. helper() is declared in a header and called in main.c, but never defined anywhere.
Check your understanding
Which toolchain stage produced this error?
Answer: C
Why: The declaration satisfied the compiler (it knew helper's signature), so compilation succeeded. The error appears when the linker tries to resolve the call to a definition across all object files and finds none — a classic link-time symbol-resolution failure.
Elimination
Eliminate the wrong options
The stack 'grows down,' so in which direction do the copied bytes land, and what do they threaten?
3 of these 4 are wrong. Strike them one at a time, and say what rules each one out before you strike the next. The survivor is the answer.
Survives elimination: B
Why: Frame allocation goes down, but a string copy fills buf[0], buf[1], … at increasing addresses — upward. Since the saved return address sits above buf, a long-enough copy overwrites it. That is the entire mechanism of the stack overflow in Lesson 8.
Check
A local char buf[8] is filled by copying a long string into it.
Check your understanding
The stack 'grows down,' so in which direction do the copied bytes land, and what do they threaten?
Answer: B
Why: Frame allocation goes down, but a string copy fills buf[0], buf[1], … at increasing addresses — upward. Since the saved return address sits above buf, a long-enough copy overwrites it. That is the entire mechanism of the stack overflow in Lesson 8.
Concept
Concept
0xDEADBEEF typed as \xef\xbe\xad\xde.eip/ebp/esp, little-endian in detail, and push/pop.Concept
gcc -m32, run readelf -S to see real sections, and objdump -d to see .text.Connect it up
Draw it
One page, no notation unless you need it: draw how these connect — Numbers & Hex · The Toolchain · C Memory Layout. Put an arrow wherever one of them is what makes another possible, and label the arrow with why.
Recap
You can convert across bases, read signed vs unsigned bits, order the toolchain, place any variable in its region, and explain why the stack growing down sets up every overflow to come.
| Idea | § | Keep this |
|---|---|---|
| Hex nibble | 2.1 | 1 hex digit = 4 bits |
| Two's complement | 2.1 | −1 = 0xFFFFFFFF (32-bit) |
| Toolchain | 2.2 | Missing function = LINK error |
| Memory regions | 2.3 | Globals→static, locals→stack, malloc→heap |
| Directions | 2.3 | Alloc down, writes up |
Want this taught 1-on-1? Alexander tutors Computer Security — $55/session, free consultation.