CS 161, Lesson 5, in 50 slides and code mode. It covers little-endian byte order, the three special 32-bit registers eip, ebp, and esp, and push and pop as decrement-then-write and read-then-increment, with a full esp trace table. It is anchored to textbook sections 2.4 to 2.6.
Subject: Computer Security · 43 slides · code lesson
Open the interactive version of this deck · Homework for this lesson
Title
CS 161 · Lesson 5 of 45
little-endian bytes · eip / ebp / esp · push and pop, mechanically
Objectives
Warm-up
Discussion prompt
Before we open L05 · Little-Endian, the Registers eip/ebp/esp, and Push/Pop: without looking back, what was the main idea of L04 · Numbers, the Toolchain, and 32-bit C Memory Layout, and what could you do by the end of it that you could not do before?
Hint: One sentence for the idea, one for the skill. If the second one is blank, that is the part to revisit.
Answer:
CS 161 Lesson 4 (50 slides, code mode): binary/hex/two's-complement, the source→process toolchain, ELF sections (⊕), and the four-section 32-bit C memory layout — code/static/heap/stack with grow directions. Anchored to textbook §2.1–2.3.
Section
Part 1 · §2.4
Concept
x86 is little-endian: when a 4-byte word is stored, its least-significant byte goes to the lowest address and the most-significant byte to the highest.
word = 0x44332211 (stored at addresses 0x1000..0x1003)
addr byte
0x1000 0x11 <- least significant (lowest addr)
0x1001 0x22
0x1002 0x33
0x1003 0x44 <- most significant (highest addr)| Address | Byte | Significance |
|---|---|---|
| 0x1000 | 0x11 | least |
| 0x1001 | 0x22 | — |
| 0x1002 | 0x33 | — |
| 0x1003 | 0x44 | most |
Discrimination
Sort into buckets
Sort these by Significance, from memory, without looking back at Least-significant byte at the lowest address. Telling them apart on the spot is the skill; the table is only where the answer happens to be written down.
Intuition
Bytes are physically little-endian, but on stack diagrams we usually write the whole word 0x44332211 in one 4-byte row to stay readable — the little-endianness is abstracted away.
Networking is the opposite: many protocols are big-endian ('network byte order'). When we reach the network unit, byte order flips — keep the distinction filed.
Ask yourself: when you craft exploit input later, will you type the target address most-significant-byte first, or least-significant-byte first?
Counterexample
Discussion prompt
Bytes are physically little-endian, but on stack diagrams we usually write the whole word 0x44332211 in one 4-byte row to stay readable — the little-endianness is abstracted away.
That is stated as though it always holds. Do one of two things: produce a case where it fails, or say precisely what rules such a case out. "It just does" is not on the menu.
Hint: Hunt at the extremes first — zero, one, negative, empty, equal. If every extreme survives, the reason they survive is the proof.
Answer:
Networking is the opposite: many protocols are big-endian ('network byte order'). When we reach the network unit, byte order flips — keep the distinction filed.
Ranking
Put in order
Put the moves of Read a little-endian value from a dump into the order they have to happen.
Why: These are the moves of the worked example in the order it makes them, and each one is set up by the one before it. 0xEF is at the lowest address, so it is the LSB; 0xDE at the highest is the MSB.
Worked example
Memory dump (low -> high address):
0x2000: EF BE AD DELowest address is the least-significant byte
Why: 0xEF is at the lowest address, so it is the LSB; 0xDE at the highest is the MSB.
Assemble most-significant first to read the word
Why: Reverse the byte order on read: DE | AD | BE | EF → 0xDEADBEEF.
| Address | Byte | Word reading |
|---|---|---|
| 0x2000 | EF | (LSB) |
| 0x2001 | BE | |
| 0x2002 | AD | |
| 0x2003 | DE | (MSB) → 0xDEADBEEF |
Verify by storing it back
Why: Storing 0xDEADBEEF little-endian puts EF at the lowest address — matches the dump. This is exactly how we'll write a return address in Lesson 8.
Trade off
Comparison matrix
From Read a little-endian value from a dump: every row here is a choice with a cost. Fill the Byte column, then say which row you would actually pick and what you give up for it.
| Address | Byte | Word reading |
|---|---|---|
| 0x2000 | EF | (LSB) |
| 0x2001 | BE | |
| 0x2002 | AD | |
| 0x2003 | DE | (MSB) → 0xDEADBEEF |
Anomaly
Predict first
A student writes this, and it looks reasonable:
Goal: write 0xDEADBEEF over a 4-byte slot.
It is wrong. Say what breaks — and say it before you turn the page.
Correct: Types the value in 'reading' order — but memory is little-endian, so this stores the bytes reversed and the slot holds 0xEFBEADDE.
Goal: write 0xDEADBEEF over a 4-byte slot.
Why: Types the value in 'reading' order — but memory is little-endian, so this stores the bytes reversed and the slot holds 0xEFBEADDE.
Trap
Goal: write 0xDEADBEEF over a 4-byte slot.
Send bytes DE AD BE EF (as written, left to right)
Why: Types the value in 'reading' order — but memory is little-endian, so this stores the bytes reversed and the slot holds 0xEFBEADDE.
Goal: write 0xDEADBEEF over a 4-byte slot.
Send bytes EF BE AD DE (least-significant first)
Why: §2.4: the lowest address takes the LSB. Input bytes go to increasing addresses, so the LSB must come first to land 0xDEADBEEF correctly.
Section
Part 2 · §2.5
Concept
Registers store a word on the CPU, have no memory address, and are named. Three matter for this unit:
| Register | Name | Holds / points to |
|---|---|---|
| eip | instruction pointer | address of the instruction now executing (code region) |
| ebp | base pointer | TOP (highest addr) of the current stack frame |
| esp | stack pointer | BOTTOM (lowest addr) of the current stack frame |
The e means 'extended' — these are the 32-bit forms. On a 64-bit CPU they become rip/rbp/rsp, but we stay 32-bit.
Comparison
Comparison matrix
From Three special registers: refill the Name column from what you know. The rest of the table is as it appeared.
| Register | Name | Holds / points to |
|---|---|---|
| eip | instruction pointer | address of the instruction now executing (code region) |
| ebp | base pointer | TOP (highest addr) of the current stack frame |
| esp | stack pointer | BOTTOM (lowest addr) of the current stack frame |
Intuition
Because the stack grows down, the 'top of the frame' (where it began) is the highest address — that's ebp. The 'bottom' (the growing edge) is the lowest address — that's esp.
high addr
| ...caller...
| [ ebp ] ---> top of frame
| local
| local
| [ esp ] ---> bottom of frame
low addr| Register | Edge of frame | Address |
|---|---|---|
| ebp | top | higher |
| esp | bottom | lower |
Sanity check from the book: which region do eip, ebp, esp point into? eip → code; ebp and esp → stack.
Counterexample
Discussion prompt
Because the stack grows down, the 'top of the frame' (where it began) is the highest address — that's ebp. The 'bottom' (the growing edge) is the lowest address — that's esp.
That is stated as though it always holds. Do one of two things: produce a case where it fails, or say precisely what rules such a case out. "It just does" is not on the menu.
Hint: Hunt at the extremes first — zero, one, negative, empty, equal. If every extreme survives, the reason they survive is the proof.
Answer:
Sanity check from the book: which region do eip, ebp, esp point into? eip → code; ebp and esp → stack.
Concept
Six more general registers appear in our assembly: eax, ebx, ecx, edx, esi, edi. For this unit you mostly need to recognize them and know eax conventionally carries return values.
| Register | Typical use |
|---|---|
| eax | return value / accumulator |
| esi / edi | source / destination index (string copies) |
| ebx, ecx, edx | general scratch |
Comparison
Comparison matrix
From The general-purpose registers: refill the Typical use column from what you know. The rest of the table is as it appeared.
| Register | Typical use |
|---|---|
| eax | return value / accumulator |
| esi / edi | source / destination index (string copies) |
| ebx, ecx, edx | general scratch |
Section
Part 3 · §2.6
Concept
Storing a value on the stack is two micro-steps: first allocate space by decrementing esp by 4, then write the value there.
push %eax is equivalent to:
esp = esp - 4 # allocate 4 bytes (stack grows down)
*(esp) = eax # write the value into the new slot| Step | esp | Action |
|---|---|---|
| before | S | — |
| allocate | S − 4 | make room |
| write | S − 4 | *(esp) = value |
Trade off
Comparison matrix
From push = decrement, then write: every row here is a choice with a cost. Fill the esp column, then say which row you would actually pick and what you give up for it.
| Step | esp | Action |
|---|---|---|
| before | S | — |
| allocate | S − 4 | make room |
| write | S − 4 | *(esp) = value |
Concept
Removing a value reverses it: read the value at esp into a register, then deallocate by incrementing esp by 4.
pop %eax is equivalent to:
eax = *(esp) # read the top value into a register
esp = esp + 4 # deallocate (the value is now BELOW esp)| Step | esp | Action |
|---|---|---|
| before | S | value at *(S) |
| read | S | eax = *(S) |
| deallocate | S + 4 | old value now below esp = undefined |
Note: pop does not erase the bytes — it just moves esp above them. Anything below esp is 'undefined' and may be overwritten by the next push.
Comparison
Comparison matrix
From pop = read, then increment: refill the esp column from what you know. The rest of the table is as it appeared.
| Step | esp | Action |
|---|---|---|
| before | S | value at *(S) |
| read | S | eax = *(S) |
| deallocate | S + 4 | old value now below esp = undefined |
Step zero
Discussion prompt
Full trace: two pushes and a pop — before any calculation: what is the plan? Name the moves in order, in plain English, without doing the arithmetic.
Hint: It starts with: push $0xAA
Answer:
Worked example
; start with esp = 0xbffff020
push $0xAA
push $0xBB
pop %eaxpush $0xAA
Why: esp = 0xbffff020 − 4 = 0xbffff01c, then *(0xbffff01c) = 0xAA.
push $0xBB
Why: esp = 0xbffff01c − 4 = 0xbffff018, then *(0xbffff018) = 0xBB.
pop %eax
Why: eax = *(0xbffff018) = 0xBB, then esp = 0xbffff018 + 4 = 0xbffff01c.
| After instruction | esp | eax | memory written |
|---|---|---|---|
| start | 0xbffff020 | — | — |
| push $0xAA | 0xbffff01c | — | *(0xbffff01c)=0xAA |
| push $0xBB | 0xbffff018 | — | *(0xbffff018)=0xBB |
| pop %eax | 0xbffff01c | 0xBB | (0xBB now below esp) |
Verify the end state
Why: Final esp = 0xbffff01c, eax = 0xBB. The 0xAA we pushed first is still in memory at 0xbffff01c but is now the live top-of-stack slot again — confirm by tracing every 4-byte step.
Discrimination
Sort into buckets
Sort these by esp, from memory, without looking back at Full trace: two pushes and a pop. Telling them apart on the spot is the skill; the table is only where the answer happens to be written down.
Anomaly
Predict first
A student writes this, and it looks reasonable:
Write *(0x1000) = eax, THEN set esp = 0x0FFC
It is wrong. Say what breaks — and say it before you turn the page.
Correct: Writes at the OLD esp, clobbering the current top-of-stack value before allocating.
push %eax with esp = 0x1000.
Why: Writes at the OLD esp, clobbering the current top-of-stack value before allocating.
Trap
push %eax with esp = 0x1000.
Write *(0x1000) = eax, THEN set esp = 0x0FFC
Why: Writes at the OLD esp, clobbering the current top-of-stack value before allocating.
push %eax with esp = 0x1000.
Decrement first: esp = 0x0FFC, THEN write *(0x0FFC) = eax
Why: §2.6: allocate before you write. Order matters — decrement-then-write protects the existing top, write-then-decrement would corrupt it.
Two truths and a lie
Sort into buckets
Some of these hold up and some are the exact mistakes this lesson is built to prevent. Sort them.
Ranking
Put in order
These are the steps of The Lesson-5 recipe, scrambled. Put them back in order before the next slide shows you.
*(esp) = value.reg = *(esp), then esp += 4 (old value now below esp = undefined).Why: This is the order the recipe itself gives. Recalling the sequence without the slide in front of you is the difference between recognising the method and being able to run it — most of what goes wrong in practice is a step done out of turn.
Pattern
*(esp) = value.reg = *(esp), then esp += 4 (old value now below esp = undefined).Edge cases
Discussion prompt
The Lesson-5 recipe works on the cases you have just seen. Push it to the edge: what is the most degenerate input it still handles — empty, zero, one item, everything equal — and what is the first case where it stops being true? Name the case, not just "it breaks".
Hint: Try the smallest legal input, then the largest, then the one where two things collide. Methods are specified at their edges; the middle takes care of itself.
Answer:
*(esp) = value.reg = *(esp), then esp += 4 (old value now below esp = undefined).Check
A 4-byte slot at 0x3000 contains the bytes (low→high): 00 10 04 08. Solve on paper.
Check your understanding
What 32-bit value is stored there?
Answer: A
Why: Little-endian: the lowest address holds the least-significant byte. Bytes low→high are 00,10,04,08, so the MSB (0x08) is last. Reading most-significant first gives 08 04 10 00 = 0x08041000.
Check
esp starts at 0xbffff100. Execute: push, push, push, pop. (Values don't matter here.)
Check your understanding
What is esp afterward?
Answer: A
Why: Three pushes lower esp by 3×4 = 12 (0xbffff100 → 0xbffff0f4), and one pop raises it by 4 (→ 0xbffff0f8). Net is −8 from the start.
Elimination
Eliminate the wrong options
Which register points to the HIGHEST address of the current frame, and what is it called?
3 of these 4 are wrong. Strike them one at a time, and say what rules each one out before you strike the next. The survivor is the answer.
Survives elimination: A
Why: §2.5: ebp (base pointer) marks the top of the frame, which — because the stack grows down — is the highest address. esp marks the bottom (lowest address), and eip is the instruction pointer into the code region.
Check
You're reading a stack diagram drawn with high addresses at the top.
Check your understanding
Which register points to the HIGHEST address of the current frame, and what is it called?
Answer: A
Why: §2.5: ebp (base pointer) marks the top of the frame, which — because the stack grows down — is the highest address. esp marks the bottom (lowest address), and eip is the instruction pointer into the code region.
Concept
Concept
Concept
info registers esp ebp eip, then x/4xb $esp to see four raw bytes in little-endian order.Connect it up
Draw it
One page, no notation unless you need it: draw how these connect — Little-Endian Bytes · The Registers · Push & Pop. Put an arrow wherever one of them is what makes another possible, and label the arrow with why.
Recap
You can read and write little-endian words, name what eip/ebp/esp point to, and trace push (down-then-write) and pop (read-then-up) to an exact esp and register value.
| Idea | § | Keep this |
|---|---|---|
| Little-endian | 2.4 | Lowest address = least-significant byte |
| eip | 2.5 | current instruction (code region) |
| ebp | 2.5 | top of frame = highest address |
| esp | 2.5 | bottom of frame = lowest address |
| push / pop | 2.6 | −4 then write / read then +4 |
Want this taught 1-on-1? Alexander tutors Computer Security — $55/session, free consultation.