CS 161, Lesson 7, in 50 slides and code mode. You learn to read gcc -S -O0 output, recognize the prologue, body, and epilogue, and decode MOV, LEA, ADD, SUB, CMP, JE, JNE, and JMP along with the AT&T addressing modes. You then annotate assembly instruction by instruction and step through it in gdb using stepi, info registers, and x/8xw $esp, closing on the unbounded C strcpy copy loop that sets up the overflow in Lesson 8. It includes several full trace tables and is anchored to textbook section 2.9.
Subject: Computer Security · 88 slides · code lesson
Open the interactive version of this deck · Homework for this lesson
Title
CS 161 · Lesson 7 of 45
gcc -S · prologue/body/epilogue · mov vs lea · stepi · x/8xw $esp · the C→memory bridge
Objectives
gcc -S -O0 output and split a function into prologue, body, and epilogue.mov, lea, add, sub, cmp, and je/jne/jmp into pseudocode — and say exactly which one dereferences.%eax, $16, (%esp), 8(%ebp)) to register / immediate / memory.esp/ebp/eip/flags/memory.break, stepi, info registers, x/8xw $esp, disas — to verify the frame the assembly builds.Warm-up
Discussion prompt
Before we open L07 · Reading x86 Assembly + GDB (and the C→memory bridge): without looking back, what was the main idea of L06 · The 11-Step Calling Convention & Stack-Frame Anatomy, and what could you do by the end of it that you could not do before?
Hint: One sentence for the idea, one for the skill. If the second one is blank, that is the part to revisit.
Answer:
CS 161 Lesson 6 / Quiz 6 (50 slides, code mode): the full 11-step x86 cdecl call/return, the prologue/epilogue, ebp-relative addressing (args at ebp+8), and the foo(1,2) assembly with leave/ret. Two full call-trace tables.
Section
Part 1 · §2.9
Concept
gcc -m32 -O0 -S foo.c emits the 32-bit AT&T assembly the compiler would assemble — and -O0 means no optimization, so every C variable gets a real stack slot and the structure is easy to read.
Conventions we never leave: 32-bit x86, registers eip/ebp/esp, 4-byte words, AT&T syntax (destination LAST), and cdecl (arguments on the stack).
| Flag | Effect |
|---|---|
| -m32 | 32-bit code: eip/ebp/esp, 4-byte words |
| -S | stop after compiling: emit .s assembly, don't assemble |
| -O0 | no optimization: every local spills to a stack slot |
Comparison
Comparison matrix
From What -O0 buys you: literal, unoptimized assembly: refill the Effect column from what you know. The rest of the table is as it appeared.
| Flag | Effect |
|---|---|
| -m32 | 32-bit code: eip/ebp/esp, 4-byte words |
| -S | stop after compiling: emit .s assembly, don't assemble |
| -O0 | no optimization: every local spills to a stack slot |
Intuition
Think of a function's assembly like a play in three acts. Set the stage (prologue), do the work (body), strike the set (epilogue). The first and last acts are nearly identical in every function — only the middle act differs.
Matching
Match the pairs
From Every -O0 function has the same three-act shape — match each one to what it actually does. The descriptions have been shuffled.
Why: Prologue, Body, Epilogue are easy to tell apart while they are sitting next to their descriptions and much harder afterwards, which is what this checks.
Concept
The prologue is steps 4–6 and looks the same at the top of essentially every -O0 C function: save the caller's frame pointer, anchor our own, then drop esp to reserve locals.
push %ebp # step 4: save old ebp (sfp)
mov %esp, %ebp # step 5: ebp = esp (anchor frame)
sub $16, %esp # step 6: esp -= 16 (reserve locals)| Instruction | Step | Effect on register |
|---|---|---|
| push %ebp | 4 | esp -= 4; *(esp) = old ebp (sfp) |
| mov %esp,%ebp | 5 | ebp = esp (frame anchored) |
| sub $16,%esp | 6 | esp -= 16 (16 bytes of locals) |
Trade off
Comparison matrix
From The prologue fingerprint: push / mov / sub: every row here is a choice with a cost. Fill the Step column, then say which row you would actually pick and what you give up for it.
| Instruction | Step | Effect on register |
|---|---|---|
| push %ebp | 4 | esp -= 4; *(esp) = old ebp (sfp) |
| mov %esp,%ebp | 5 | ebp = esp (frame anchored) |
| sub $16,%esp | 6 | esp -= 16 (16 bytes of locals) |
Concept
The epilogue is steps 8–10. mov %ebp,%esp ; pop %ebp is exactly what the single instruction leave abbreviates; then ret pops the saved return address into eip.
mov %ebp, %esp # step 8: esp = ebp (discard locals)
pop %ebp # step 9: ebp = old ebp (restore sfp)
# steps 8+9 together == the single instruction leave
ret # step 10: pop rip -> eip (return)| Instruction | Step | Effect on register |
|---|---|---|
| mov %ebp,%esp | 8 | esp = ebp (locals gone) |
| pop %ebp | 9 | ebp = *(esp); esp += 4 (restore sfp) |
| leave | 8–9 | the two above, combined |
| ret | 10 | eip = *(esp); esp += 4 (pop rip) |
Counterexample
Discussion prompt
The epilogue is steps 8–10. mov %ebp,%esp ; pop %ebp is exactly what the single instruction leave abbreviates; then ret pops the saved return address into eip.
That is stated as though it always holds. Do one of two things: produce a case where it fails, or say precisely what rules such a case out. "It just does" is not on the menu.
Hint: Hunt at the extremes first — zero, one, negative, empty, equal. If every extreme survives, the reason they survive is the proof.
Ranking
Put in order
Put the moves of Annotate the textbook's foo(1,2) into the order they have to happen.
Why: These are the moves of the worked example in the order it makes them, and each one is set up by the one before it. push %ebp ; mov %esp,%ebp ; sub $16,%esp — steps 4–6.
Worked example
main:
push $2 # caller: arg2
push $1 # caller: arg1
call foo # push rip; eip = &foo
add $8, %esp # caller cleanup: drop the 2 args
foo:
push %ebp # prologue
mov %esp, %ebp # prologue
sub $16, %esp # prologue
# ... body ...
leave # epilogue (mov %ebp,%esp; pop %ebp)
ret # epilogue (pop rip)Find the prologue
Why: push %ebp ; mov %esp,%ebp ; sub $16,%esp — steps 4–6. The 16 is the compiler's chosen local-allocation size for foo.
Find the epilogue
Why: leave ; ret — steps 8–10. leave collapses the frame; ret pops the rip back into eip.
Notice the caller's cleanup
Why: add $8,%esp runs back in main (step 11) to remove the two 4-byte arguments — cdecl makes the caller responsible.
| Instruction | Step / phase | Effect on esp / ebp / eip |
|---|---|---|
| push $2 / push $1 | 1 (caller) | esp -= 8 (args pushed) |
| call foo | 2–3 (caller) | esp -= 4 (rip); eip = &foo |
| push %ebp | 4 (prologue) | esp -= 4 (sfp) |
| mov %esp,%ebp | 5 (prologue) | ebp = esp |
| sub $16,%esp | 6 (prologue) | esp -= 16 (locals) |
| leave | 8–9 (epilogue) | esp = ebp; pop ebp (esp += 4) |
| ret | 10 (epilogue) | eip = *(esp); esp += 4 |
| add $8,%esp | 11 (caller) | esp += 8 (args removed) |
Error analysis
Annotate
Walk the callouts on Annotate the textbook's foo(1,2). Each one is a place this is easy to get subtly wrong.
push %ebp ; mov %esp,%ebp ; sub $16,%esp — steps 4–6. The 16 is the compiler's chosen local-allocation size for foo.leave ; ret — steps 8–10. leave collapses the frame; ret pops the rip back into eip.add $8,%esp runs back in main (step 11) to remove the two 4-byte arguments — cdecl makes the caller responsible.Concept
Once you can spot the prologue and epilogue fingerprints, the body is simply everything in between. You never have to memorize the body — it's the only part that differs per function.
Reading strategy: bracket off push %ebp … sub $N,%esp at the top and leave ; ret at the bottom, then read the middle as ordinary instructions operating on args (ebp+8↑) and locals (ebp-4↓).
| Region | How to recognize it | What it does |
|---|---|---|
| prologue | push %ebp / mov %esp,%ebp / sub $N | build the frame |
| body | everything between the fingerprints | the C logic |
| epilogue | leave (or mov/pop) then ret | tear down + return |
Analogy
Discussion prompt
Explain The body is whatever's between the fingerprints by analogy to something with no Computer Security in it at all — a queue, a recipe, a map, a bank balance, whatever fits. Then say where your analogy breaks.
Hint: An analogy that never breaks is not an analogy, it is the same idea wearing a hat. Find the seam — that is the part that is actually new.
Answer:
Once you can spot the prologue and epilogue fingerprints, the body is simply everything in between. You never have to memorize the body — it's the only part that differs per function.
Anomaly
Predict first
A student writes this, and it looks reasonable:
Read it left-to-right: "move ebp into esp", so esp = ebp
It is wrong. Say what breaks — and say it before you turn the page.
Correct: That is Intel order (destination FIRST).
Decode mov %esp, %ebp.
Why: That is Intel order (destination FIRST). In AT&T it gets the data flow exactly backwards.
Trap
Decode mov %esp, %ebp.
Read it left-to-right: "move ebp into esp", so esp = ebp
Why: That is Intel order (destination FIRST). In AT&T it gets the data flow exactly backwards.
Decode mov %esp, %ebp.
Destination is LAST in AT&T: ebp = esp
Why: gdb and gcc both speak AT&T. The % source is %esp, the % destination is %ebp; the value flows right. Same instruction in Intel syntax would read mov ebp, esp.
Section
Part 2 · §2.9 + Intel SDM
Concept
Six instruction families cover nearly all of a simple -O0 function. Learn their pseudocode meaning once and you can read most listings.
mov src, dst # dst = src (copy a value)
lea m, dst # dst = address-of(m) (NO dereference)
add src, dst # dst = dst + src
sub src, dst # dst = dst - src
cmp a, b # compute b - a, set flags, DISCARD result
jmp L # eip = L (unconditional)
je / jne L # branch on the zero flag| Opcode | Pseudocode meaning |
|---|---|
| mov src, dst | dst = src |
| lea m, dst | dst = &m (effective address; no load) |
| add src, dst | dst = dst + src |
| sub src, dst | dst = dst - src |
| cmp a, b | flags = (b - a); result thrown away |
| jmp / je / jne L | eip = L (unconditional / if ZF / if !ZF) |
Explain it
Discussion prompt
Explain The working set: mov, lea, add, sub, cmp, jmp to a student a year behind you. No notation, no jargon they have not met — and it still has to be true.
Hint: If your explanation needs a symbol they have never seen, you are describing the notation rather than the idea.
Answer:
Six instruction families cover nearly all of a simple -O0 function. Learn their pseudocode meaning once and you can read most listings.
Intuition
Picture a numbered mailbox. mov 8(%ebp), %eax opens the mailbox at ebp+8 and copies what's inside into eax. lea 8(%ebp), %eax writes down the mailbox's address (ebp+8) into eax — it never opens the box.
Same 8(%ebp) operand, opposite meaning: one dereferences, one computes an address. This single distinction is the most common assembly-reading error students make.
Estimation
Predict first
Suppose ebp = 0xbffff000, and the 4 bytes at memory address 0xbffff008 hold the value 42.
Commit before you compute: what does mov vs lea, side by side come out to? A rough magnitude and the right form is enough — the point is to have something concrete to be wrong about.
Correct: Resolve the lea
Why: A prediction you can defend turns the computation into a check rather than a leap of faith — and an answer that contradicts it is caught on the spot. lea computes the effective address of the operand without touching memory: eax receives ebp+8 = 0xbffff008.
Worked example
Suppose ebp = 0xbffff000, and the 4 bytes at memory address 0xbffff008 hold the value 42.
mov 8(%ebp), %eax # eax = *(ebp+8) = the VALUE 42
lea 8(%ebp), %eax # eax = (ebp+8) = the ADDRESS 0xbffff008Resolve the mov
Why: 8(%ebp) is a memory operand. mov dereferences it: eax receives the value stored there, 42.
Resolve the lea
Why: lea computes the effective address of the operand without touching memory: eax receives ebp+8 = 0xbffff008.
| Instruction | Reads memory? | eax becomes |
|---|---|---|
| mov 8(%ebp), %eax | yes — loads *(ebp+8) | 42 (the value) |
| lea 8(%ebp), %eax | no — computes ebp+8 | 0xbffff008 (the address) |
| mov %ebp, %eax | no — register copy | 0xbffff000 (ebp itself) |
Reverse engineer
Discussion prompt
Work backwards. The example finished here:
Resolve the lea
What was it asked to do, and what must it have been given? Reconstruct the problem from its answer.
Hint: Every quantity in the result had to enter somewhere. Account for each one.
Answer:
Suppose ebp = 0xbffff000, and the 4 bytes at memory address 0xbffff008 hold the value 42.
Concept
cmp a, b performs the subtraction b - a purely to set the flags (notably the zero flag, ZF). It does not store b - a anywhere — b is unchanged.
cmp $0, %eax # flags reflect (eax - 0); eax UNCHANGED
je done # if ZF=1 (eax == 0) jump to done
jne loop # if ZF=0 (eax != 0) jump to loop| Sequence | Branches to L when |
|---|---|
| cmp $0,%eax ; je L | eax == 0 (ZF = 1) |
| cmp $0,%eax ; jne L | eax != 0 (ZF = 0) |
| cmp %ebx,%eax ; je L | eax == ebx (eax - ebx == 0) |
Intuition
jmp L is an unconditional goto L — it just sets eip to L. je/jne are the if part of a C if/while: they only set eip when the zero flag from the preceding cmp says so.
A C if (x == 0) { ... } compiles to roughly: cmp $0, x ; jne skip ; <body> ; skip: — the branch jumps over the body when the condition is false.
Explain it
Discussion prompt
Explain jmp is GOTO; je/jne are conditional GOTOs to a student a year behind you. No notation, no jargon they have not met — and it still has to be true.
Hint: If your explanation needs a symbol they have never seen, you are describing the notation rather than the idea.
Answer:
jmp L is an unconditional goto L — it just sets eip to L. je/jne are the if part of a C if/while: they only set eip when the zero flag from the preceding cmp says so.
Missing information
Discussion prompt
add and sub are pure register arithmetic when both operands are registers/immediates. Start with eax = 10, ebx = 3 and run a short sequence.
What do you need to know — or decide — before the first line can be written? List everything the problem has to hand you.
Hint: Anything you would have to invent to get started is a thing the problem must supply.
Answer:
add $5 makes eax = 15; sub %ebx makes eax = 15 - 3 = 12; add %eax,%eax makes eax = 12 + 12 = 24.
Worked example
add and sub are pure register arithmetic when both operands are registers/immediates. Start with eax = 10, ebx = 3 and run a short sequence.
add $5, %eax # eax = eax + 5
sub %ebx, %eax # eax = eax - ebx
add %eax, %eax # eax = eax + eax (doubles it)Apply each with destination last
Why: add $5 makes eax = 15; sub %ebx makes eax = 15 - 3 = 12; add %eax,%eax makes eax = 12 + 12 = 24.
| Instruction | eax before | eax after |
|---|---|---|
| add $5,%eax | 10 | 15 |
| sub %ebx,%eax | 15 | 12 |
| add %eax,%eax | 12 | 24 |
Pattern
Step through it
Step through Trace add/sub on registers one row at a time. What is driving the change, and what would the row after the last one be?
Anomaly
Predict first
A student writes this, and it looks reasonable:
After cmp %ebx, %eax, what is in eax?
It is wrong. Say what breaks — and say it before you turn the page.
Correct: Treats cmp like sub. But then the comparison would destroy the very value you were comparing.
After cmp %ebx, %eax, what is in eax?
Why: Treats cmp like sub. But then the comparison would destroy the very value you were comparing.
Trap
After cmp %ebx, %eax, what is in eax?
Assume eax now holds eax - ebx
Why: Treats cmp like sub. But then the comparison would destroy the very value you were comparing.
After cmp %ebx, %eax, what is in eax?
eax is unchanged; only the flags moved
Why: cmp = sub-with-no-writeback. It sets ZF/SF/CF from (eax - ebx) and discards the number. Use sub when you actually want the difference stored.
Section
Part 3 · §2.9
Concept
Every operand is one of four kinds. The %, $, and parentheses tell you which: % = register, $ = immediate constant, parentheses = dereference memory, and a leading number is an offset added before the dereference.
%eax # register: the contents of eax
$16 # immediate: the constant 16 (not memory!)
(%esp) # memory: *(esp)
8(%ebp) # base+offset: *(ebp + 8)| Operand | Kind | Means |
|---|---|---|
| %eax | register | the value in eax |
| $16 | immediate | the constant 16 |
| (%esp) | memory | *(esp) — value at address esp |
| 8(%ebp) | base + offset | *(ebp + 8) — value at ebp+8 |
| -4(%ebp) | base + offset | *(ebp - 4) — a local |
Estimation
Predict first
Apply the rules to a less familiar instruction. xor is dst = dst XOR src; the source here is a base+offset memory operand.
Commit before you compute: what does Read xorl 4(%esi), %eax come out to? A rough magnitude and the right form is enough — the point is to have something concrete to be wrong about.
Correct: Apply the opcode with destination last
Why: A prediction you can defend turns the computation into a check rather than a leap of faith — and an answer that contradicts it is caught on the spot. AT&T puts the destination last, so eax is updated: eax = eax XOR *(esi+4).
Worked example
Apply the rules to a less familiar instruction. xor is dst = dst XOR src; the source here is a base+offset memory operand.
xorl 4(%esi), %eax # eax = eax ^ *(esi + 4)Resolve the source operand
Why: 4(%esi) is base+offset: it dereferences the address esi+4, yielding the 4-byte value stored there.
Apply the opcode with destination last
Why: AT&T puts the destination last, so eax is updated: eax = eax XOR *(esi+4).
| Piece | Resolves to |
|---|---|
| 4(%esi) | *(esi + 4) — a memory value |
| %eax (dest) | current contents of eax |
| whole instruction | eax = eax ^ *(esi + 4) |
Comparison
Comparison matrix
From Read xorl 4(%esi), %eax: refill the Resolves to column from what you know. The rest of the table is as it appeared.
| Piece | Resolves to |
|---|---|
| 4(%esi) | *(esi + 4) — a memory value |
| %eax (dest) | current contents of eax |
| whole instruction | eax = eax ^ *(esi + 4) |
Trap
Decode sub $16, %esp.
Read $16 as "the value at memory location 16"
Why: Drops the $. Without it you'd be subtracting whatever lives at address 16 — nonsense here.
Decode sub $16, %esp.
$16 is the literal constant 16: esp = esp - 16
Why: $ marks an immediate. 16 (no $) would be a memory operand; (%esp) would dereference esp. The $/parens distinction is the whole game in addressing modes.
Section
Part 4 · §2.9
Concept
Here is a tiny function with one local int x and a compare/branch. At -O0 the local x lives at ebp-4 (the first local slot), and the if becomes a cmp plus a conditional jump.
int f(int a) {
int x = a + 1;
if (x == 0)
return 7;
return x;
}| C name | Where it lives at -O0 |
|---|---|
| a (parameter) | ebp + 8 (first argument) |
| x (local) | ebp - 4 (first local) |
| return value | eax |
Step zero
Discussion prompt
Full annotation of f(int a) — before any calculation: what is the plan? Name the moves in order, in plain English, without doing the arithmetic.
Hint: It starts with: Lines 1–3 build the frame
Answer:
Worked example
f:
push %ebp # 1 prologue: save sfp
mov %esp, %ebp # 2 prologue: anchor frame
sub $16, %esp # 3 prologue: reserve locals
mov 8(%ebp), %eax # 4 eax = a (load arg)
add $1, %eax # 5 eax = a + 1
mov %eax, -4(%ebp) # 6 x = eax (spill to local)
cmp $0, -4(%ebp) # 7 flags = (x - 0)
jne .Lret_x # 8 if x != 0 -> return x
mov $7, %eax # 9 eax = 7
jmp .Ldone # 10
.Lret_x:
mov -4(%ebp), %eax # 11 eax = x (load local)
.Ldone:
leave # 12 epilogue
ret # 13 epilogue: pop ripLines 1–3 build the frame
Why: The standard prologue: save sfp, set ebp = esp, drop esp by 16 for locals. x's slot at ebp-4 now exists.
Lines 4–6 compute x = a + 1
Why: Load arg a from ebp+8 into eax, add 1, then store back to the local at ebp-4. At -O0 the value is spilled to memory, not kept in a register.
Lines 7–8 are the if
Why: cmp sets ZF from (x - 0); jne jumps to .Lret_x when x != 0 — i.e. it skips the return 7 path.
Lines 12–13 tear it down
Why: leave collapses the frame, ret pops the rip into eip. eax already holds the return value.
| Line | Instruction | Effect on esp / ebp / eip / flags / memory |
|---|---|---|
| 1 | push %ebp | esp -= 4; mem[esp] = sfp |
| 2 | mov %esp,%ebp | ebp = esp (anchored) |
| 3 | sub $16,%esp | esp -= 16 (locals) |
| 4 | mov 8(%ebp),%eax | eax = a (read mem ebp+8) |
| 5 | add $1,%eax | eax = a + 1 |
| 6 | mov %eax,-4(%ebp) | mem[ebp-4] = x (spill) |
| 7 | cmp $0,-4(%ebp) | ZF set from (x - 0); x unchanged |
| 8 | jne .Lret_x | eip = .Lret_x if ZF = 0 |
| 12 | leave | esp = ebp; pop ebp |
| 13 | ret | eip = mem[esp]; esp += 4 |
Cost model
Annotate
In Full annotation of f(int a), before reading the notes: mark where the time actually goes. Which line dominates?
return 7 path.Concept
Notice line 6 stored x to ebp-4 and line 7/11 re-read it. At -O0 the compiler does not keep x in a register between statements — it writes it to its stack slot and reloads it. That's why every local has a memory home you can examine in gdb.
This matters for security: every local — including a char buf[] — is a real, addressable region on the stack, sitting at a known offset from ebp, right below the sfp and rip.
Anomaly
Predict first
A student writes this, and it looks reasonable:
Just before call foo, esp points at the last pushed argument. Where does esp point right after call?
It is wrong. Say what breaks — and say it before you turn the page.
Correct: Then you'd miscount the frame by 4 and place the saved ebp where the rip actually is.
Just before call foo, esp points at the last pushed argument. Where does esp point right after call?
Why: Then you'd miscount the frame by 4 and place the saved ebp where the rip actually is.
Trap
Just before call foo, esp points at the last pushed argument. Where does esp point right after call?
Assume call just sets eip, so esp is unchanged
Why: Then you'd miscount the frame by 4 and place the saved ebp where the rip actually is.
Just before call foo, esp points at the last pushed argument. Where does esp point right after call?
call PUSHES the 4-byte return address first, so esp -= 4
Why: call = push %eip ; jmp target. The rip now sits at the new top of stack. Miss this and your esp count — and your overflow math in Lesson 8 — is off by one word.
Break the constraint
Discussion prompt
The rule this trap just fixed:
Just before call foo, esp points at the last pushed argument.
Now break it on purpose. Build a case that violates it and follow the consequences until something visibly fails. Where does the failure first show up — and would you have noticed it if you had not been looking?
Hint: The dangerous rules are the ones whose violation still produces an answer. If yours fails loudly, try to find one that fails quietly.
Answer:
Then you'd miscount the frame by 4 and place the saved ebp where the rip actually is.
Section
Part 5 · GDB manual
Concept
gdb lets you stop the program and step one machine instruction at a time, watching the registers and stack change. Five commands cover the assembly-reading workflow.
| gdb command | What it does / shows |
|---|---|
| break f | set a breakpoint at the start of function f |
| run | start the program; stop at the breakpoint |
| stepi | execute exactly ONE machine instruction |
| info registers | print all registers (esp, ebp, eip, eax, eflags…) |
| x/8xw $esp | examine 8 hex words (4 bytes each) starting at esp |
| disas | disassemble the current function (AT&T) |
Intuition
stepi is like advancing a film one frame: exactly one instruction runs, so you can watch esp/ebp/eip move on each push, mov, and sub. (next/step move by C statement; stepi moves by instruction.)
x/8xw $esp is your microscope on the stack: 8 units, x = hex format, w = word (4 bytes), starting at the address in $esp. gdb prints them as little-endian words, so a saved address reads naturally left-to-right.
Concept
The x command's slash-spec is x/<count><format><size>. Read it as a sentence: "examine N items, shown in this format, this many bytes each."
| Spec piece | Meaning |
|---|---|
| 8 | count: show 8 units |
| x | format: hexadecimal |
| w | size: word = 4 bytes |
| $esp | start address (the stack pointer) |
| x/8xw $esp | → 8 hex 4-byte words from the top of stack |
Analogy
Discussion prompt
Explain Decoding the x/ format letters by analogy to something with no Computer Security in it at all — a queue, a recipe, a map, a bank balance, whatever fits. Then say where your analogy breaks.
Hint: An analogy that never breaks is not an analogy, it is the same idea wearing a hat. Find the seam — that is the part that is actually new.
Answer:
The x command's slash-spec is x/<count><format><size>. Read it as a sentence: "examine N items, shown in this format, this many bytes each."
Ranking
Put in order
Put the moves of Stepping the prologue in gdb into the order they have to happen.
Why: These are the moves of the worked example in the order it makes them, and each one is set up by the one before it. Execution halts at f's first instruction (push %ebp) before the frame is built — the ideal place to watch it form.
Worked example
(gdb) break f
(gdb) run
Breakpoint 1, f ()
=> push %ebp
(gdb) info registers esp ebp eip
esp 0xbffff03c ebp 0xbffff058 eip <f+0>
(gdb) stepi # runs: push %ebp
(gdb) info registers esp
esp 0xbffff038 # esp dropped by 4 (sfp pushed)
(gdb) stepi # runs: mov %esp,%ebp
(gdb) info registers ebp
ebp 0xbffff038 # ebp now equals esp (anchored)
(gdb) stepi # runs: sub $16,%esp
(gdb) info registers esp
esp 0xbffff028 # esp dropped by 16 (locals)Break and run
Why: Execution halts at f's first instruction (push %ebp) before the frame is built — the ideal place to watch it form.
stepi past push %ebp
Why: One instruction runs; esp drops from 0xbffff03c to 0xbffff038 (−4). The sfp is now on the stack.
stepi past mov and sub
Why: ebp becomes 0xbffff038 (= esp, anchored), then sub $16 drops esp to 0xbffff028. The frame is fully built.
| After stepi over… | esp | ebp |
|---|---|---|
| (break, before push) | 0xbffff03c | 0xbffff058 |
| push %ebp | 0xbffff038 | 0xbffff058 |
| mov %esp,%ebp | 0xbffff038 | 0xbffff038 |
| sub $16,%esp | 0xbffff028 | 0xbffff038 |
Discrimination
Sort into buckets
Sort these by esp, from memory, without looking back at Stepping the prologue in gdb. Telling them apart on the spot is the skill; the table is only where the answer happens to be written down.
Ranking
Put in order
Put the moves of Reading the stack with x/8xw $esp into the order they have to happen.
Why: These are the moves of the worked example in the order it makes them, and each one is set up by the one before it. ebp = 0xbffff038. The word printed there, 0xbffff058, is the saved frame pointer (sfp) — the caller's ebp.
Worked example
With the frame built, examine the stack. The dump below reads from esp (low address, left/top) upward toward the sfp and rip.
(gdb) x/8xw $esp
0xbffff028: 0x00000000 0x00000000 0x00000000 0x00000005
0xbffff038: 0xbffff058 0x08048461 0x00000003 0x00000000
# ^sfp(old ebp) ^rip(ret) ^arg a=3Locate ebp's row
Why: ebp = 0xbffff038. The word printed there, 0xbffff058, is the saved frame pointer (sfp) — the caller's ebp.
Find the rip at ebp+4
Why: One word higher, 0xbffff03c holds 0x08048461 — the saved return address. This is the value ret will pop into eip.
Find the first argument at ebp+8
Why: 0xbffff040 holds 0x00000003 — the argument a = 3, exactly where the ebp+8 rule predicts.
| Address | Offset from ebp | Contents | Meaning |
|---|---|---|---|
| 0xbffff028 | ebp − 16 | 0x00000000 | uninitialized local space (esp) |
| 0xbffff034 | ebp − 4 | 0x00000005 | the local x |
| 0xbffff038 | ebp + 0 | 0xbffff058 | sfp (saved ebp) |
| 0xbffff03c | ebp + 4 | 0x08048461 | rip (saved return addr) |
| 0xbffff040 | ebp + 8 | 0x00000003 | arg a = 3 |
Error analysis
Annotate
Walk the callouts on Reading the stack with x/8xw $esp. Each one is a place this is easy to get subtly wrong.
ret will pop into eip.Concept
info registers (or i r) dumps all registers; pass names to narrow it. The three that describe the frame are esp (bottom), ebp (top, anchor), and eip (next instruction). eflags carries the zero flag the branches read.
(gdb) info registers esp ebp eip
esp 0xbffff028 0xbffff028
ebp 0xbffff038 0xbffff038
eip 0x8048456 0x8048456 <f+6>| Register | Role | Watch it on |
|---|---|---|
| esp | stack bottom (low addr) | every push/pop, sub/add |
| ebp | frame anchor (high addr) | mov %esp,%ebp; pop %ebp |
| eip | next instruction | every stepi; ret |
Concept
disas (or disassemble) prints the current function's instructions in AT&T with addresses, and marks the current eip with =>. It's the fastest way to see the prologue/body/epilogue split on a real binary.
(gdb) disas f
0x08048450 <+0>: push %ebp
0x08048451 <+1>: mov %esp,%ebp
0x08048453 <+3>: sub $0x10,%esp
=> 0x08048456 <+6>: mov 0x8(%ebp),%eax
...
0x0804846f <+31>: leave
0x08048470 <+32>: ret| disas marker | Tells you |
|---|---|
| <+0> push %ebp | prologue begins here |
| => arrow | the instruction eip is about to execute |
| leave / ret at the bottom | the epilogue — function exit |
Anomaly
Predict first
A student writes this, and it looks reasonable:
You want to watch esp change on each push in the prologue.
It is wrong. Say what breaks — and say it before you turn the page.
Correct: step (and next) advance by C source LINE, often running the whole prologue at once — you blow right past the individual register moves.
You want to watch esp change on each push in the prologue.
Why: step (and next) advance by C source LINE, often running the whole prologue at once — you blow right past the individual register moves.
Trap
You want to watch esp change on each push in the prologue.
Use step and expect to stop after each instruction
Why: step (and next) advance by C source LINE, often running the whole prologue at once — you blow right past the individual register moves.
You want to watch esp change on each push in the prologue.
Use stepi to advance exactly one machine instruction
Why: stepi stops after every single instruction, so esp/ebp/eip update one move at a time. Use step/next only when you care about C lines, not the frame mechanics.
Section
Part 6 · sets up Lesson 8
Concept
Watch strcpy(buf, src) at three altitudes — C, pointer logic, and assembly. The whole point: at no altitude is there a length check. The copy stops only when it hits a NUL byte in the source.
char buf[8];
strcpy(buf, src); // copy src into buf until src's NUL terminator| Altitude | How the copy is expressed | Bound? |
|---|---|---|
| C | strcpy(buf, src) | none — stops at NUL only |
| pointer | while (*src) *buf++ = *src++; | none |
| assembly | movb %al,(%edi); inc %edi; loop | none |
Counterexample
Discussion prompt
Watch strcpy(buf, src) at three altitudes — C, pointer logic, and assembly. The whole point: at no altitude is there a length check. The copy stops only when it hits a NUL byte in the source.
That is stated as though it always holds. Do one of two things: produce a case where it fails, or say precisely what rules such a case out. "It just does" is not on the menu.
Hint: Hunt at the extremes first — zero, one, negative, empty, equal. If every extreme survives, the reason they survive is the proof.
Concept
An array name decays to a pointer: buf is &buf[0]. Indexing is just pointer arithmetic plus a dereference — buf[i] is exactly *(buf + i). There is no bounds check in this: buf[100] computes *(buf+100) and writes it, in bounds or not.
| C expression | Equivalent | Checks i < 8? |
|---|---|---|
| buf | &buf[0] (an address) | — |
| buf[i] | *(buf + i) | NO |
| buf[100] | *(buf + 100) | NO — writes past the array |
Estimation
Predict first
A naive byte-copy loop reads a byte from the source (esi), writes it to the destination (edi), advances both, and stops only when the byte copied was 0 (the NUL). edi (the destination) just keeps walking upward.
Commit before you compute: what does The copy loop in assembly has no limit come out to? A rough magnitude and the right form is enough — the point is to have something concrete to be wrong about.
Correct: Follow edi past the buffer
Why: A prediction you can defend turns the computation into a check rather than a leap of faith — and an answer that contradicts it is caught on the spot. If src is longer than buf (8 bytes), edi marches from buf into the sfp, then the rip, then the caller's frame — overwriting control data with attacker bytes.
Worked example
A naive byte-copy loop reads a byte from the source (esi), writes it to the destination (edi), advances both, and stops only when the byte copied was 0 (the NUL). edi (the destination) just keeps walking upward.
.Lcopy:
movb (%esi), %al # al = *src
movb %al, (%edi) # *dst = al <- NO compare against a limit
inc %esi # src++
inc %edi # dst++ <- destination walks upward
test %al, %al # was the byte 0?
jne .Lcopy # if not NUL, keep copyingSpot the missing guard
Why: There is no cmp of edi against any end-of-buffer address. The only stop condition is the NUL byte in the SOURCE — the destination size is never consulted.
Follow edi past the buffer
Why: If src is longer than buf (8 bytes), edi marches from buf into the sfp, then the rip, then the caller's frame — overwriting control data with attacker bytes.
| After byte # | edi points at | Slot being overwritten |
|---|---|---|
| 1–8 | buf[0] … buf[7] | the local buffer (legitimate) |
| 9–12 | ebp + 0 | sfp (saved frame pointer) |
| 13–16 | ebp + 4 | rip (saved return address!) |
| 17+ | ebp + 8 and up | args / caller's frame |
Trade off
Comparison matrix
From The copy loop in assembly has no limit: every row here is a choice with a cost. Fill the edi points at column, then say which row you would actually pick and what you give up for it.
| After byte # | edi points at | Slot being overwritten |
|---|---|---|
| 1–8 | buf[0] … buf[7] | the local buffer (legitimate) |
| 9–12 | ebp + 0 | sfp (saved frame pointer) |
| 13–16 | ebp + 4 | rip (saved return address!) |
| 17+ | ebp + 8 and up | args / caller's frame |
Anomaly
Predict first
A student writes this, and it looks reasonable:
char buf[8]; strcpy(buf, src); with a 20-byte src.
It is wrong. Say what breaks — and say it before you turn the page.
Correct: There is no length parameter anywhere — buf's size is invisible to strcpy.
char buf[8]; strcpy(buf, src); with a 20-byte src.
Why: There is no length parameter anywhere — buf's size is invisible to strcpy. This false sense of a limit is exactly how overflows ship.
Trap
char buf[8]; strcpy(buf, src); with a 20-byte src.
Assume strcpy copies at most 8 bytes to fit buf
Why: There is no length parameter anywhere — buf's size is invisible to strcpy. This false sense of a limit is exactly how overflows ship.
char buf[8]; strcpy(buf, src); with a 20-byte src.
strcpy copies until src's NUL — all 20 bytes, overrunning buf
Why: The copy loop's only stop condition is the NUL in the source. 12 bytes spill past buf, through the sfp and into the rip. That overwritten rip is the hijack target of Lesson 8.
Two truths and a lie
Sort into buckets
Some of these hold up and some are the exact mistakes this lesson is built to prevent. Sort them.
8(%ebp) operand, opposite meaning: one dereferences, one computes an address. This single distinction is the most common assembly-reading error students make.cmp %ebx, %eax, what is in eax?Section
Reference only · not assessed
Concept
Reference only — we assess in 32-bit. On real 64-bit systems the registers are rip/rbp/rsp, slots are 8 bytes, and the first six integer arguments arrive in registers, not on the stack: RDI, RSI, RDX, RCX, R8, R9. Everything you read in gdb on a 64-bit box uses these names.
| 32-bit (this course) | x86-64 (sidebar) | |
|---|---|---|
| pointers | eip / ebp / esp | rip / rbp / rsp |
| word size | 4 bytes | 8 bytes |
| first 6 int args | on the stack (cdecl) | RDI,RSI,RDX,RCX,R8,R9 |
| return value | eax | rax |
We mention it once so a 64-bit gdb session doesn't surprise you. Every quiz, exam, and worked example in CS 161 stays 32-bit.
Comparison
Comparison matrix
From x86-64 sidebar (this course assesses in 32-bit): refill the 32-bit (this course) column from what you know. The rest of the table is as it appeared.
| 32-bit (this course) | x86-64 (sidebar) | |
|---|---|---|
| pointers | eip / ebp / esp | rip / rbp / rsp |
| word size | 4 bytes | 8 bytes |
| first 6 int args | on the stack (cdecl) | RDI,RSI,RDX,RCX,R8,R9 |
| return value | eax | rax |
Section
Part 7
Constraint
Discussion prompt
Run How to read any -O0 function (and verify it in gdb) with this step confiscated:
Resolve operands: %r=register, $k=constant, (%r)=(r), n(%r)=(r+n). Parentheses dereference; a $ does not.
Is it still possible? If it is, say what takes its place and what it costs you. If it is not, say exactly what that step was providing that nothing else does.
Hint: A step you can drop for free was never load-bearing. If you cannot drop it, name the thing that goes wrong the moment it is gone.
Answer:
push %ebp / mov %esp,%ebp / sub $N,%esp prologue and the leave / ret epilogue; everything between is the body.mov=copy value, lea=copy address, cmp=subtract-and-discard for flags, jmp/je/jne=set eip.%r=register, $k=constant, (%r)=(r), n(%r)=(r+n). Parentheses dereference; a $ does not.ebp+8, ebp+12, …; locals at ebp-4, ebp-8, …; rip at ebp+4, sfp at ebp+0.break, run, stepi through the prologue, info registers for esp/ebp/eip, and x/8xw $esp to read the frame you just predicted.Pattern
push %ebp / mov %esp,%ebp / sub $N,%esp prologue and the leave / ret epilogue; everything between is the body.mov=copy value, lea=copy address, cmp=subtract-and-discard for flags, jmp/je/jne=set eip.%r=register, $k=constant, (%r)=(r), n(%r)=(r+n). Parentheses dereference; a $ does not.ebp+8, ebp+12, …; locals at ebp-4, ebp-8, …; rip at ebp+4, sfp at ebp+0.break, run, stepi through the prologue, info registers for esp/ebp/eip, and x/8xw $esp to read the frame you just predicted.Edge cases
Discussion prompt
How to read any -O0 function (and verify it in gdb) works on the cases you have just seen. Push it to the edge: what is the most degenerate input it still handles — empty, zero, one item, everything equal — and what is the first case where it stops being true? Name the case, not just "it breaks".
Hint: Try the smallest legal input, then the largest, then the one where two things collide. Methods are specified at their edges; the middle takes care of itself.
Answer:
push %ebp / mov %esp,%ebp / sub $N,%esp prologue and the leave / ret epilogue; everything between is the body.mov=copy value, lea=copy address, cmp=subtract-and-discard for flags, jmp/je/jne=set eip.%r=register, $k=constant, (%r)=(r), n(%r)=(r+n). Parentheses dereference; a $ does not.ebp+8, ebp+12, …; locals at ebp-4, ebp-8, …; rip at ebp+4, sfp at ebp+0.break, run, stepi through the prologue, info registers for esp/ebp/eip, and x/8xw $esp to read the frame you just predicted.Elimination
Eliminate the wrong options
What value does lea 8(%ebp), %eax put in eax?
3 of these 4 are wrong. Strike them one at a time, and say what rules each one out before you strike the next. The survivor is the answer.
Survives elimination: A
Why: lea computes the EFFECTIVE ADDRESS of its operand without dereferencing. The operand 8(%ebp) addresses ebp+8 = 0x1000 + 8 = 0x1008, so eax = 0x1008. lea never reads memory; that's the whole difference from mov.
Check
ebp = 0x1000, and the 4 bytes at memory address 0x1008 hold the value 0x2A. Work it on paper before clicking.
Check your understanding
What value does lea 8(%ebp), %eax put in eax?
Answer: A
Why: lea computes the EFFECTIVE ADDRESS of its operand without dereferencing. The operand 8(%ebp) addresses ebp+8 = 0x1000 + 8 = 0x1008, so eax = 0x1008. lea never reads memory; that's the whole difference from mov.
mov 8(%ebp), %eax would do — mov dereferences and loads the value 0x2A. lea loads the address instead.lea (%ebp), %eax (no offset) would give ebp = 0x1000, but the operand here is ebp+8.Concept
mov vs lea — mov n(%r),dst loads the VALUE at r+n; lea n(%r),dst loads the ADDRESS r+n (no dereference).call pushes the rip — call = push %eip ; jmp target; esp drops 4 before the callee even starts.mov %esp,%ebp means ebp = esp, not the reverse.cmp stores its result — cmp only sets flags; the operands are unchanged. Use sub to keep the difference.Concept
x/8xw $esp is how you'll confirm the smashed return address in L08 — predict the rip's slot, then read it in gdb.stepi + info registers eip lets you watch ret load an attacker-controlled value into eip — the hijack, observed live.Concept
stepi), examining memory (x/Nfu), info registers, disas.gcc -m32 -O0 -S f.c, match each line to mov/lea/cmp/jmp; then gdb it — break f ; run ; stepi while watching $esp $ebp $eip and x/8xw $esp.Connect it up
Draw it
One page, no notation unless you need it: draw how these connect — Reading gcc -S Output · Instruction Semantics · Addressing Modes · Annotating a Function · GDB at the Machine Level · The C → Memory Bridge. Put an arrow wherever one of them is what makes another possible, and label the arrow with why.
Recap
You can split a -O0 function into prologue/body/epilogue, decode mov/lea/cmp/jmp with destination-last, resolve every addressing mode, annotate a listing line-by-line, and verify the frame in gdb with stepi and x/8xw $esp.
| Skill | The move |
|---|---|
| Split a function | find push/mov/sub … leave/ret |
| value vs address | mov = *(r+n); lea = r+n |
| cmp + jcc | subtract-and-discard for flags, branch on ZF |
| find the data | args ebp+8↑, locals ebp-4↓, rip ebp+4 |
| verify live | stepi; info registers; x/8xw $esp |
Want this taught 1-on-1? Alexander tutors Computer Security — $55/session, free consultation.