CS 161, Lesson 13, in 54 slides and code mode. It covers W^X and NX pages and the NX bit in the page table, why the buffer-injection attack from Lesson 8 now faults, and the code-reuse insight that defeats NX. It then builds the 32-bit ret2libc stack layout, with its fake return address and argument pointer, shows how to find the libc base with ASLR off and why you need a leak with ASLR on, and chains libc calls toward ROP. The examples are toys running in a sandbox and the addresses are kept symbolic. It is anchored to textbook sections 4.5 to 4.6.
Subject: Computer Security · 87 slides · code lesson
Open the interactive version of this deck · Homework for this lesson
Title
CS 161 · Lesson 13 of 45
W^X · the NX bit · why injection faults · return-to-libc · &system on the stack · finding libc · toward ROP
Objectives
ret into it traps.&system, then a fake return address, then a pointer to the argument string.Warm-up
Discussion prompt
Before we open L13 · Non-Executable Pages & Return-to-libc: without looking back, what was the main idea of L12 · Exploit Mitigations Overview: NX, Stack Canaries, ASLR, and what could you do by the end of it that you could not do before?
Hint: One sentence for the idea, one for the skill. If the second one is blank, that is the part to revisit.
Answer:
CS 161 Lesson 12 (50 slides, code mode): why mitigations exist when you're stuck in C, non-executable pages (NX/W^X/DEP) stopping shellcode injection, stack canaries detecting contiguous rip overwrites, ASLR randomizing absolute addresses but not relative offsets, the consolidated mitigation→attack→bypass table, and why combining all three is synergistic. Overview only — deep bypasses are Week 5.
Concept
Every example here is a toy in a sandbox, for authorized security education — exactly how the textbook teaches it. You learn how a defense is bypassed so you can layer the next defense correctly.
Addresses stay symbolic (&system, &"/bin/sh"); we never write real shellcode bytes. The skill being built is reasoning about a stack layout, not weaponizing one.
Counterexample
Discussion prompt
Addresses stay symbolic (&system, &"/bin/sh"); we never write real shellcode bytes. The skill being built is reasoning about a stack layout, not weaponizing one.
That is stated as though it always holds. Do one of two things: produce a case where it fails, or say precisely what rules such a case out. "It just does" is not on the menu.
Hint: Hunt at the extremes first — zero, one, negative, empty, equal. If every extreme survives, the reason they survive is the proof.
Concept
Recall Lesson 8: overflow char buf[8], cross the 8 + 4 = 12-byte offset, overwrite the rip, and point it at shellcode you injected into the buffer. The CPU ran your bytes.
This lesson is the first defense that breaks that exact attack — and then the first attacker move that breaks the defense. Attack and defense, one move each.
Analogy
Discussion prompt
Explain Where we are: the L8 attack worked by analogy to something with no Computer Security in it at all — a queue, a recipe, a map, a bank balance, whatever fits. Then say where your analogy breaks.
Hint: An analogy that never breaks is not an analogy, it is the same idea wearing a hat. Find the seam — that is the part that is actually new.
Answer:
Recall Lesson 8: overflow char buf[8], cross the 8 + 4 = 12-byte offset, overwrite the rip, and point it at shellcode you injected into the buffer. The CPU ran your bytes.
Section
Part 1 · §4.5
Concept
The defense is a single rule on every memory page: it may be writable or executable, but never both at once.
W^X (W xor X) — An invariant enforced per memory page: a page is either writable or executable, exclusively — code pages can't be written, writable pages can't be executed.
Same idea, three names you'll meet: W^X (OpenBSD), DEP = Data Execution Prevention (Windows), NX = No-eXecute (the hardware feature).
Explain it
Discussion prompt
Explain Writable XOR executable to a student a year behind you. No notation, no jargon they have not met — and it still has to be true.
Hint: If your explanation needs a symbol they have never seen, you are describing the notation rather than the idea.
Answer:
The defense is a single rule on every memory page: it may be writable or executable, but never both at once.
Concept
Each page-table entry carries a permission bit. Set it, and the CPU refuses to fetch instructions from that page — any attempt to execute there raises a hardware fault.
NX bit / XD bit — A per-page hardware permission bit (AMD calls it NX 'No-eXecute', Intel calls it XD 'eXecute-Disable') that marks a page as non-executable in the page table.
| Name | Vendor / OS | What it marks |
|---|---|---|
| NX bit | AMD hardware | page is non-executable |
| XD bit | Intel hardware | page is execute-disabled |
| DEP | Windows | the OS policy using NX/XD |
| W^X | OpenBSD / *BSD | the writable-xor-executable invariant |
Comparison
Comparison matrix
From Hardware enforces it: the NX bit: refill the Vendor / OS column from what you know. The rest of the table is as it appeared.
| Name | Vendor / OS | What it marks |
|---|---|---|
| NX bit | AMD hardware | page is non-executable |
| XD bit | Intel hardware | page is execute-disabled |
| DEP | Windows | the OS policy using NX/XD |
| W^X | OpenBSD / *BSD | the writable-xor-executable invariant |
Intuition
Think of memory as a building. Code pages are the locked instruction vault: read-and-run, never edited at runtime. Data pages — the stack, the heap — are scratch paper: written constantly, never meant to be run.
A normal program never needs a page that is both scratch paper and runnable. Injected shellcode needs exactly that: a page you can write to and execute. W^X removes that page from existence.
Intuition
The whole defense compresses to a slogan worth memorizing: don't let eip ever hold the address of a non-executable page.
The L8 attack violated that on purpose — it set eip to point inside buf, a data page. W^X makes that pointer a dead end: the moment the CPU tries to fetch from there, it traps instead of running attacker code.
Estimation
Predict first
Walk the address space of a typical program and ask, per region: writable? executable? could injected shellcode live and run here?
Commit before you compute: what does Which pages can hold shellcode? come out to? A rough magnitude and the right form is enough — the point is to have something concrete to be wrong about.
Correct: Code regions are X, not W
Why: A prediction you can defend turns the computation into a check rather than a leap of faith — and an answer that contradicts it is caught on the spot. .text and libc are executable so the CPU can fetch instructions — so they are marked read-only.
Worked example
Walk the address space of a typical program and ask, per region: writable? executable? could injected shellcode live and run here?
stack -> local buffers, saved rip (you write here via overflow)
heap -> malloc'd data
.data -> global variables
.text -> the program's own code
libc -> shared library code (printf, system, ...)Data regions are W, not X
Why: Stack, heap, and .data are writable so the program can store values — so under W^X they are marked non-executable. Shellcode written there cannot run.
Code regions are X, not W
Why: .text and libc are executable so the CPU can fetch instructions — so they are marked read-only. You cannot write shellcode into them.
| Page / region | Writable? | Executable? | Can hold runnable shellcode? |
|---|---|---|---|
| stack (buf) | yes | no | no — write OK, but fetch faults |
| heap | yes | no | no — same as stack |
| .data (globals) | yes | no | no |
| .text (program code) | no | yes | no — can't write to it |
| libc (shared lib) | no | yes | no — can't write to it |
Discrimination
Sort into buckets
Sort these by Writable?, from memory, without looking back at Which pages can hold shellcode?. Telling them apart on the spot is the skill; the table is only where the answer happens to be written down.
Ranking
Put in order
Put the moves of Re-derive: why the L8 injection now faults into the order they have to happen.
Why: These are the moves of the worked example in the order it makes them, and each one is set up by the one before it. The stack page is writable, so gets happily copies the shellcode bytes into buf.
Worked example
Take the exact L8 Strategy-2 payload — inject shellcode into buf, point rip at buf — and run it on a machine with NX on. Trace what the CPU does.
payload = [shellcode (8B)] # written into buf (stack page = writable)
+ [4 garbage bytes] # over the sfp
+ [&buf] # rip <- start of bufThe write succeeds
Why: The stack page is writable, so gets happily copies the shellcode bytes into buf. Nothing stops the overflow itself — NX is not about writing.
ret loads &buf into eip
Why: The overwritten return address is honored as before; eip now points into the stack, into the bytes we wrote.
The instruction fetch faults
Why: buf lives on a writable stack page, so its NX bit is set. The CPU refuses to fetch an instruction from a non-executable page and raises a fault — the process crashes instead of running shellcode.
| Step | Before NX | With NX on |
|---|---|---|
| write shellcode to buf | OK | OK (stack is writable) |
| ret sets eip = &buf | OK | OK |
| fetch instruction at buf | runs shellcode | FAULT — buf page is non-executable |
Trade off
Comparison matrix
From Re-derive: why the L8 injection now faults: every row here is a choice with a cost. Fill the With NX on column, then say which row you would actually pick and what you give up for it.
| Step | Before NX | With NX on |
|---|---|---|
| write shellcode to buf | OK | OK (stack is writable) |
| ret sets eip = &buf | OK | OK |
| fetch instruction at buf | runs shellcode | FAULT — buf page is non-executable |
Intuition
Notice what NX did not do. The buffer overflow still happens. The sfp and rip are still overwritten. The attacker still controls where eip goes.
All NX changed is the menu of valid destinations: eip may now only land on an executable page. The bug is identical; the set of useful targets shrank.
Anomaly
Predict first
A student writes this, and it looks reasonable:
Does turning on NX fix the buffer overflow bug?
It is wrong. Say what breaks — and say it before you turn the page.
Correct: NX changes page permissions, not the program.
Does turning on NX fix the buffer overflow bug?
Why: NX changes page permissions, not the program. gets(buf) still overflows, still corrupts the sfp and rip — the memory-safety bug is untouched. NX only blocks one consequence: running injected code.
Trap
Does turning on NX fix the buffer overflow bug?
Conclude the program is now memory-safe
Why: NX changes page permissions, not the program. gets(buf) still overflows, still corrupts the sfp and rip — the memory-safety bug is untouched. NX only blocks one consequence: running injected code.
Does turning on NX fix the buffer overflow bug?
The bug remains; NX only blocks injected-code execution
Why: §4.5: W^X is an exploit mitigation, not a fix. The overflow still happens and still hijacks eip — NX merely denies eip a writable landing zone. Memory safety must come from the code (bounds checks) or the language.
Trap
With NX on, can the attacker still take control?
Assume NX stops all control-flow hijacking
Why: NX only forbids executing non-executable pages. It says nothing about jumping to pages that are already executable — and a program is full of those. The hijack of eip still works; only the destination is constrained.
With NX on, can the attacker still take control?
NX only stops INJECTED code; existing executable code is fair game
Why: §4.5–4.6: the rip overwrite still works. The attacker just can't point it at their own bytes — so they point it at code that's already marked executable. That pivot is return-to-libc.
Concept
W^X is not a 'stack' feature — it is a per-page rule, so it also kills heap-injection and global-buffer-injection in one stroke: any page you can write to is, by construction, non-executable.
That's the strength of doing it in the page table: one bit per page, checked by hardware on every fetch, covers every data region at once with no per-program work.
Estimation
Predict first
When the hijacked ret jumps into a non-executable page, the CPU does not 'run garbage' — it raises a precise fault. Trace what the OS sees.
Commit before you compute: what does Tracing the fault the CPU reports come out to? A rough magnitude and the right form is enough — the point is to have something concrete to be wrong about.
Correct: A fault, not silent corruption
Why: A prediction you can defend turns the computation into a check rather than a leap of faith — and an answer that contradicts it is caught on the spot. Unlike the original overflow (silent), the NX violation is loud: the hardware traps and the OS delivers a fatal signal.
Worked example
When the hijacked ret jumps into a non-executable page, the CPU does not 'run garbage' — it raises a precise fault. Trace what the OS sees.
ret ; pops attacker's &buf into eip
fetch @ buf ; CPU checks page-table NX bit for buf's page
; NX = 1 -> raise fault (no instruction fetched)
OS signal ; SIGSEGV / access violation -> process killedThe check happens at fetch time
Why: Before executing a single byte of buf, the CPU consults the page's NX bit. A set bit means 'never fetch instructions here'.
A fault, not silent corruption
Why: Unlike the original overflow (silent), the NX violation is loud: the hardware traps and the OS delivers a fatal signal. The exploit attempt becomes a crash.
| Phase | What the CPU does | Outcome |
|---|---|---|
| ret | loads eip = &buf | control hijacked |
| instruction fetch | reads NX bit of buf's page | NX=1 → fault |
| OS handler | delivers SIGSEGV | process terminated |
Error analysis
Annotate
Walk the callouts on Tracing the fault the CPU reports. Each one is a place this is easy to get subtly wrong.
Section
Part 2 · §4.6
Concept
Read the NX rule precisely: it stops the CPU from executing a non-executable page. It does nothing to code that is already in memory and already marked executable.
So the attacker stops trying to bring code and starts reusing code that the program legitimately loaded. Don't inject — redirect.
Concept
Almost every C program links libc, the C standard library: printf, malloc, strcpy, system, execv, and thousands more. It is mapped into the process, executable and non-writable — perfect for W^X, and perfect for reuse.
That's thousands to millions of executable instructions already present, that the program may legitimately call. The attacker doesn't need to write a single new byte of code — the useful code is already there.
Intuition
NX is a metal detector at the door: it stops you smuggling your own weapon (shellcode) into the building. Clever — but the building is an armory. The walls are lined with weapons (libc functions) that are supposed to be there.
So you walk in empty-handed and just grab one already mounted on the wall. The metal detector never fires, because you brought nothing. You only had to reach the right one.
Anomaly
Predict first
A student writes this, and it looks reasonable:
NX blocks injected shellcode. So the exploit is dead, right?
It is wrong. Say what breaks — and say it before you turn the page.
Correct: Equates 'attacker code' with 'attacker's own bytes'.
NX blocks injected shellcode. So the exploit is dead, right?
Why: Equates 'attacker code' with 'attacker's own bytes'. But the attacker can run the program's code — libc — instead. Control-flow hijack never required injection; it required controlling eip, which still works.
Trap
NX blocks injected shellcode. So the exploit is dead, right?
Conclude no injection means no exploit
Why: Equates 'attacker code' with 'attacker's own bytes'. But the attacker can run the program's code — libc — instead. Control-flow hijack never required injection; it required controlling eip, which still works.
NX blocks injected shellcode. So the exploit is dead, right?
Redirect eip into existing executable code (libc)
Why: §4.6: this is code reuse. You still overwrite the rip; you just aim it at a libc function already mapped executable. No injection needed — the metal detector finds nothing to flag.
Section
Part 3 · §4.6
Concept
return-to-libc (ret2libc) — A code-reuse attack: overwrite the saved return address with the address of a libc function, so that ret 'returns' into that function with attacker-chosen arguments — no injected code.
The textbook's example is execv, which starts executing another program — give it a filename like "/bin/sh" and it launches a shell. (The closely-related system("/bin/sh") does the same.)
Definition probe
Sort into buckets
Every line below is part of the definition of W^X (W xor X) or of return-to-libc (ret2libc) — one or the other, never both. Put each where it belongs.
ret 'returns' into that function with attacker-chosen argumentsret 'returns' into that function with attacker-chosen arguments — no injected code.Concept
execv and system don't run themselves — they take an argument: a pointer to the filename string, e.g. "/bin/sh". To make the call do anything useful, the attacker must deliver that argument.
Here is the key fact that makes it possible on 32-bit x86: arguments are passed on the stack (cdecl), not in registers. And the stack is exactly what an overflow controls.
Intuition
When any function starts, the 32-bit calling convention says: the return address sits at the top of its frame, and the first argument sits in the slot just above that. The function blindly reads its argument from [esp+4] on entry — relative to ebp, that's ebp+8.
| On entry to a function, at… | the function expects… |
|---|---|
| top of stack (esp) | the return address |
| esp + 4 (later ebp+8) | first argument |
| esp + 8 (later ebp+12) | second argument |
ret2libc weaponizes this: the attacker hand-places the 'return address' and 'first argument' that the libc function will read — because all of it lives on the stack the overflow already owns.
Sorting
Sort into buckets
These are the pieces of L13 · Non-Executable Pages & Return-to-libc, out of order. Put each one back under the part of the lesson it belongs to.
Concept
Crucial for beating NX: the "/bin/sh" string and its pointer are data the attacker writes onto the stack. They are never executed — system reads them as a parameter.
So NX has nothing to object to. The only thing that executes is system itself, which lives in libc — an executable, non-writable page. Writable data + executable libc = NX is satisfied at every step.
Picture it
Figure (svg): 32-bit ret2libc stack from high to low: pointer to /bin/sh at rip+8, a fake return address at rip+4, &system overwriting the saved rip at ebp+4, then sfp and buf below as garbage filler.
Discussion prompt
Read the picture before the words. What is this showing, and what is the one thing it is built to make obvious? Commit to an answer, then read on.
Hint: Name the parts, then say what changes between them — and if nothing changes, say what is being held still.
Answer:
Overflow vulnerable() and, instead of an injected-code address, write the three magic words above the buffer: &system, a fake return address, and a pointer to "/bin/sh".
Worked example
Overflow vulnerable() and, instead of an injected-code address, write the three magic words above the buffer: &system, a fake return address, and a pointer to "/bin/sh".
Figure (svg): 32-bit ret2libc stack from high to low: pointer to /bin/sh at rip+8, a fake return address at rip+4, &system overwriting the saved rip at ebp+4, then sfp and buf below as garbage filler.
12 garbage bytes fill buf + sfp
Why: Same 8 + 4 offset from L8: cross the 8-byte buffer and the 4-byte saved frame pointer to reach the rip.
Overwrite the rip with &system
Why: On return, ret pops this into eip — execution enters system, which is in libc, an executable page. NX is satisfied.
Above the rip: a fake return address, then the arg pointer
Why: When system begins, it sees the slot at rip+4 as its own return address and the slot at rip+8 (ebp+8) as its first argument — the pointer to "/bin/sh". The attacker placed both.
| Stack slot (low→high) | Attacker writes | Role once system() runs |
|---|---|---|
| buf + sfp (12 B) | garbage | spacer to reach the rip |
| rip (ebp+4) | &system | ret jumps here → system executes |
| rip+4 | fake return address | system's saved return addr (where it goes after) |
| rip+8 | &"/bin/sh" | system's 1st argument (read as data) |
Comparison
Comparison matrix
From The 32-bit ret2libc stack layout: refill the Role once system() runs column from what you know. The rest of the table is as it appeared.
| Stack slot (low→high) | Attacker writes | Role once system() runs |
|---|---|---|
| buf + sfp (12 B) | garbage | spacer to reach the rip |
| rip (ebp+4) | &system | ret jumps here → system executes |
| rip+4 | fake return address | system's saved return addr (where it goes after) |
| rip+8 | &"/bin/sh" | system's 1st argument (read as data) |
Intuition
It looks odd that the argument is at rip+8, not rip+4. Remember the convention: when system starts, the very top slot is its return address. The slot above that is its first argument.
So the layout has to mimic a normal call: [ &system ] [ where system returns to ] [ system's argument ]. The middle slot is the fake return address — where execution flows when system itself finishes. Often it's set to &exit for a clean exit, or junk if you don't care.
Anomaly
Predict first
A student writes this, and it looks reasonable:
ret2libc puts "/bin/sh" and pointers on the stack. Doesn't NX block that?
It is wrong. Say what breaks — and say it before you turn the page.
Correct: Confuses placing data on the stack with executing the stack.
ret2libc puts "/bin/sh" and pointers on the stack. Doesn't NX block that?
Why: Confuses placing data on the stack with executing the stack. The string and pointers are never run as instructions — so the stack's NX bit is irrelevant to them.
Trap
ret2libc puts "/bin/sh" and pointers on the stack. Doesn't NX block that?
Assume the stack must be executable for ret2libc to work
Why: Confuses placing data on the stack with executing the stack. The string and pointers are never run as instructions — so the stack's NX bit is irrelevant to them.
ret2libc puts "/bin/sh" and pointers on the stack. Doesn't NX block that?
Only libc executes; the stack data is just read
Why: §4.6: the one thing that runs is system, in executable libc. The stack holds data (a string, pointers) that system reads. NX never fires — which is the whole point of ret2libc.
Trap
We want system("/bin/sh"). Where should the overwritten rip point?
Set rip = &"/bin/sh" (the string)
Why: Aims eip at the data. Even if the string were on an executable page, its bytes aren't system's code — and on the stack it just faults under NX. The string is the argument, not the target.
We want system("/bin/sh"). Where should the overwritten rip point?
Set rip = &system; put &"/bin/sh" two slots higher
Why: rip must hold the function's entry address so ret enters system. The string pointer goes at ebp+8 where system reads its first argument. Function in rip, argument above the fake return address.
Step zero
Discussion prompt
Reading the payload as a call — before any calculation: what is the plan? Name the moves in order, in plain English, without doing the arithmetic.
Hint: It starts with: ret pops &system into eip
Answer:
Worked example
It helps to read the overwritten region as the call system("/bin/sh") spelled out in stack slots. Line up what the attacker writes against what a real call system would have produced.
payload = ['A' * 12] # buf(8) + sfp(4): reach the rip
+ [&system] # rip -> system's entry
+ [&exit] # rip+4 -> fake return addr
+ [&"/bin/sh"] # rip+8 -> system's argumentret pops &system into eip
Why: vulnerable() returns 'into' system. The CPU is now executing libc code on an executable page — NX has no complaint.
system reads its argument at ebp+8
Why: Per cdecl, system's first argument is the slot two above the entry — our &"/bin/sh". It opens a shell.
When system finishes, it 'returns' to &exit
Why: system's own return address (the fake one we planted at rip+4) sends control to exit(), ending the process cleanly instead of crashing.
| Slot | Value | Mimics part of… |
|---|---|---|
| rip | &system | the call target |
| rip+4 | &exit | return address pushed by a real call |
| rip+8 | &"/bin/sh" | the first argument pushed before a call |
Error analysis
Annotate
Walk the callouts on Reading the payload as a call. Each one is a place this is easy to get subtly wrong.
Section
Part 4 · §4.6
Concept
L8 injection only needed the address of your own buffer. ret2libc needs the address of a libc function — &system, &execv. You can't 'return into libc' without knowing where libc is.
Whether that's trivial or hard depends entirely on one defense from L12: ASLR (Address Space Layout Randomization).
Ranking
Put in order
Put the moves of ASLR off (toy / sandbox): libc base is constant into the order they have to happen.
Why: These are the moves of the worked example in the order it makes them, and each one is set up by the one before it. With randomization off, ldd or /proc/<pid>/maps reports the same base every time, so the address is reusable across runs.
Worked example
In a lab with ASLR disabled, libc loads at the same base address every run. So you can simply look up where system lives and hard-code it.
$ ldd ./vuln # shows libc's load address
libc.so.6 => /lib/.../libc.so.6 (0xb7e00000)
$ cat /proc/<pid>/maps # the mapping while it runs
b7e00000-b7fb0000 r-xp ... libc.so.6 # r-x = exec, no w
# &system = libc_base + offset_of_system_within_libcRead the constant libc base
Why: With randomization off, ldd or /proc/<pid>/maps reports the same base every time, so the address is reusable across runs.
Add system's offset within libc
Why: Each function sits at a fixed offset from the library's base; base + offset = &system, which you bake straight into the payload.
Note the r-x permission
Why: The maps line shows libc as r-xp: readable, executable, NOT writable — exactly a W^X code page. That's why returning into it satisfies NX.
| With ASLR… | libc base | How you learn &system |
|---|---|---|
| OFF (toy) | constant every run | ldd / /proc/<pid>/maps, then + offset |
| ON (real) | randomized per run | you can't read it directly — need a leak |
Cost model
Annotate
In ASLR off (toy / sandbox): libc base is constant, before reading the notes: mark where the time actually goes. Which line dominates?
ldd or /proc/<pid>/maps reports the same base every time, so the address is reusable across runs.r-xp: readable, executable, NOT writable — exactly a W^X code page. That's why returning into it satisfies NX.Intuition
NX says 'you may only return to executable pages.' Fine — the attacker returns to libc, an executable page. NX never said anything about libc being at a predictable location.
ASLR fixes that gap: it moves libc to a random base every run. Now 'return to &system' fails because the attacker doesn't know the number to write. NX + ASLR are a pair — NX bans injection, ASLR hides the reuse target.
Explain it
Discussion prompt
Explain ASLR is the lock NX forgot to add to a student a year behind you. No notation, no jargon they have not met — and it still has to be true.
Hint: If your explanation needs a symbol they have never seen, you are describing the notation rather than the idea.
Answer:
NX says 'you may only return to executable pages.' Fine — the attacker returns to libc, an executable page. NX never said anything about libc being at a predictable location.
Step zero
Discussion prompt
ASLR on (real systems): you need a leak first — before any calculation: what is the plan? Name the moves in order, in plain English, without doing the arithmetic.
Hint: It starts with: First obtain ONE real libc pointer at runtime
Answer:
Worked example
Turn ASLR on and the libc base changes every run. Now ret2libc has a missing ingredient: the runtime value of &system. You must leak it before you can use it.
step 1: trigger an info leak -> read some libc pointer at runtime
(e.g. a format-string %p, L15)
step 2: leaked_ptr - known_offset = libc_base # de-randomize
step 3: &system = libc_base + offset_of_system
step 4: now build the ret2libc payload with the real &systemFirst obtain ONE real libc pointer at runtime
Why: A separate bug (often a format-string or out-of-bounds read, L15) discloses a live pointer into libc — defeating the randomization for this run.
Subtract its known offset to recover the base
Why: If you know what that pointer normally points to, leaked − offset = this run's libc base. Everything in libc is now locatable again.
Only THEN build the ret2libc payload
Why: With the de-randomized &system in hand, the Part-3 layout works. This is the Week-5 lesson: you now need two bugs — a leak plus the overflow.
| Have | ASLR off | ASLR on |
|---|---|---|
| the overflow bug | yes | yes |
| &system known? | yes (constant) | no — randomized |
| info-leak bug needed? | no | yes (L15) |
Trade off
Comparison matrix
From ASLR on (real systems): you need a leak first: every row here is a choice with a cost. Fill the ASLR on column, then say which row you would actually pick and what you give up for it.
| Have | ASLR off | ASLR on |
|---|---|---|
| the overflow bug | yes | yes |
| &system known? | yes (constant) | no — randomized |
| info-leak bug needed? | no | yes (L15) |
Anomaly
Predict first
A student writes this, and it looks reasonable:
I have a clean buffer overflow. Can I ret2libc a shell on a modern, ASLR-on box?
It is wrong. Say what breaks — and say it before you turn the page.
Correct: ret2libc needs the literal address &system.
I have a clean buffer overflow. Can I ret2libc a shell on a modern, ASLR-on box?
Why: ret2libc needs the literal address &system. With ASLR on, that address is randomized every run — write a stale value and the 'return' lands on garbage and faults. The overflow gives control of eip but not the number to put in it.
Trap
I have a clean buffer overflow. Can I ret2libc a shell on a modern, ASLR-on box?
Assume the overflow alone is enough
Why: ret2libc needs the literal address &system. With ASLR on, that address is randomized every run — write a stale value and the 'return' lands on garbage and faults. The overflow gives control of eip but not the number to put in it.
I have a clean buffer overflow. Can I ret2libc a shell on a modern, ASLR-on box?
Not without an info leak to find libc first
Why: §4.6 + L12: you need the libc base. ASLR randomizes it, so you must leak a libc pointer (a second bug, L15), de-randomize, then ret2libc. NX + ASLR together force you to chain bugs.
Section
Part 5 · §4.6
Concept
Recall the slot at rip+4 — the fake return address we planted as system's own return target. We used it for a clean &exit. But it can point anywhere executable — including a second libc function.
So you can run more than one call: system("/bin/sh"), and when it returns, flow into another function instead of exiting. ret2libc chains.
Ranking
Put in order
Put the moves of Chaining two libc calls into the order they have to happen.
Why: These are the moves of the worked example in the order it makes them, and each one is set up by the one before it. rip = &system, argument at rip+8 — system("/bin/sh") executes, same as before.
Worked example
Arrange the stack so that when the first libc function returns, control flows straight into a second one. The first function's 'return address' slot is just the entry of the next function.
payload = ['A' * 12]
+ [&system] # rip: call #1 -> system
+ [&exit] # rip+4: system's return -> call #2
+ [&"/bin/sh"] # rip+8: system's argument
# when system() returns, it 'returns' into exit()Function #1 runs with its argument
Why: rip = &system, argument at rip+8 — system("/bin/sh") executes, same as before.
Its return slot is the entry of function #2
Why: The fake return address at rip+4 is &exit. When system returns, control flows into exit() — a second chosen call, no injection.
Generalize: stack a sequence of functions
Why: Each function's return slot points at the next function's entry, with its arguments laid out above. You drive a whole sequence of existing-code calls from the stack.
| Order | Stack slot | Effect |
|---|---|---|
| call #1 | rip = &system | runs system("/bin/sh") |
| link | rip+4 = &exit | system returns into exit |
| arg #1 | rip+8 = &"/bin/sh" | system's argument |
Comparison
Comparison matrix
From Chaining two libc calls: refill the Effect column from what you know. The rest of the table is as it appeared.
| Order | Stack slot | Effect |
|---|---|---|
| call #1 | rip = &system | runs system("/bin/sh") |
| link | rip+4 = &exit | system returns into exit |
| arg #1 | rip+8 = &"/bin/sh" | system's argument |
Intuition
Chaining whole libc functions is powerful but coarse. The next generalization — L14 — chains tiny fragments of existing code instead: short instruction sequences that each end in ret, called gadgets.
Return-Oriented Programming (ROP) — Generalized code reuse: chain many short 'gadgets' (existing instruction sequences ending in ret) so that the stack itself drives a custom computation — all from code already marked executable.
Same core idea as ret2libc — the stack is a list of return addresses into existing executable code — just with finer-grained pieces. ret2libc is ROP's gentle first step.
Analogy
Discussion prompt
Explain From whole functions to tiny fragments: ROP by analogy to something with no Computer Security in it at all — a queue, a recipe, a map, a bank balance, whatever fits. Then say where your analogy breaks.
Hint: An analogy that never breaks is not an analogy, it is the same idea wearing a hat. Find the seam — that is the part that is actually new.
Answer:
Same core idea as ret2libc — the stack is a list of return addresses into existing executable code — just with finer-grained pieces. ret2libc is ROP's gentle first step.
Intuition
Step back and see what the attacker is really doing: each stack slot they wrote is a return address, and ret walks them one after another. The stack has stopped being data the program returns through and become a script the attacker wrote.
ret2libc's script has a few coarse lines (whole functions). ROP's script has many fine lines (gadgets). Either way, the instruction stream is composed entirely from code already on executable pages — which is precisely why NX can never see it coming.
Anomaly
Predict first
A student writes this, and it looks reasonable:
To run system() then exit(), don't you need a little injected stub to jump between them?
It is wrong. Say what breaks — and say it before you turn the page.
Correct: Any injected glue would sit on a writable page and fault under NX — defeating the purpose.
To run system() then exit(), don't you need a little injected stub to jump between them?
Why: Any injected glue would sit on a writable page and fault under NX — defeating the purpose. It also misreads how returns work.
Trap
To run system() then exit(), don't you need a little injected stub to jump between them?
Assume you must inject 'glue' to sequence the calls
Why: Any injected glue would sit on a writable page and fault under NX — defeating the purpose. It also misreads how returns work.
To run system() then exit(), don't you need a little injected stub to jump between them?
The 'glue' is just the next function's address in the return slot
Why: §4.6: each function's saved return address IS the entry of the next call. The sequencing is data on the stack, not injected code — ret provides the jump for free.
Two truths and a lie
Sort into buckets
Some of these hold up and some are the exact mistakes this lesson is built to prevent. Sort them.
&system, &"/bin/sh"); we never write real shellcode bytes. The skill being built is reasoning about a stack layout, not weaponizing one.; The defense is a single rule on every memory page: it may be writable or executable, but never both at once.; Each page-table entry carries a permission bit. Set it, and the CPU refuses to fetch instructions from that page — any attempt to execute there raises a hardware fault.Concept
You can't NX libc away — programs must be able to run their own library code. As long as executable code exists in the address space, an attacker who controls eip can try to reuse it.
That's why defense shifted from 'stop injection' (NX) to 'hide the targets' (ASLR) and later 'verify control flow' (CFI). The arms race moved up a level — which is exactly the L14–L15 story.
Counterexample
Discussion prompt
You can't NX libc away — programs must be able to run their own library code. As long as executable code exists in the address space, an attacker who controls eip can try to reuse it.
That is stated as though it always holds. Do one of two things: produce a case where it fails, or say precisely what rules such a case out. "It just does" is not on the menu.
Hint: Hunt at the extremes first — zero, one, negative, empty, equal. If every extreme survives, the reason they survive is the proof.
Section
Part 6
Constraint
Discussion prompt
Run Pattern: build a ret2libc payload with this step confiscated:
Overwrite the rip with &system (or &execv) — typed little-endian — so ret enters libc.
Is it still possible? If it is, say what takes its place and what it costs you. If it is not, say exactly what that step was providing that nothing else does.
Hint: A step you can drop for free was never load-bearing. If you cannot drop it, name the thing that goes wrong the moment it is gone.
Answer:
char buf[8]: 8 + 4 = 12.ldd / /proc/<pid>/maps, then add the function's offset. ASLR on → leak a libc pointer first…&system (or &execv) — typed little-endian — so ret enters libc.&exit), then a pointer to the argument string (&"/bin/sh") at the slot the function reads…Pattern
char buf[8]: 8 + 4 = 12.ldd / /proc/<pid>/maps, then add the function's offset. ASLR on → leak a libc pointer first (L15), then de-randomize.&system (or &execv) — typed little-endian — so ret enters libc.&exit), then a pointer to the argument string (&"/bin/sh") at the slot the function reads (ebp+8).Edge cases
Discussion prompt
Pattern: build a ret2libc payload works on the cases you have just seen. Push it to the edge: what is the most degenerate input it still handles — empty, zero, one item, everything equal — and what is the first case where it stops being true? Name the case, not just "it breaks".
Hint: Try the smallest legal input, then the largest, then the one where two things collide. Methods are specified at their edges; the middle takes care of itself.
Answer:
char buf[8]: 8 + 4 = 12.ldd / /proc/<pid>/maps, then add the function's offset. ASLR on → leak a libc pointer first…&system (or &execv) — typed little-endian — so ret enters libc.&exit), then a pointer to the argument string (&"/bin/sh") at the slot the function reads…Elimination
Eliminate the wrong options
With NX enabled, why does a return-to-libc attack still succeed where injected shellcode fails?
3 of these 4 are wrong. Strike them one at a time, and say what rules each one out before you strike the next. The survivor is the answer.
system, which lives in libc — an executable, non-writable page — and supplies "/bin/sh" as data on the stack, so nothing non-executable is ever executed."/bin/sh" string and pointers can run as code.system.Survives elimination: A
Why: NX only forbids executing non-executable pages. system() is in libc, which is executable and non-writable, so jumping there is fine. The "/bin/sh" string and its pointer are data the function reads, never executed — so NX is satisfied at every step. The only requirement is knowing &system.
Check
NX (W^X) is on: every writable page is non-executable. An attacker still wants system("/bin/sh") via a buffer overflow. Think it through before clicking.
Check your understanding
With NX enabled, why does a return-to-libc attack still succeed where injected shellcode fails?
system, which lives in libc — an executable, non-writable page — and supplies "/bin/sh" as data on the stack, so nothing non-executable is ever executed. (correct)"/bin/sh" string and pointers can run as code.system.Answer: A
Why: NX only forbids executing non-executable pages. system() is in libc, which is executable and non-writable, so jumping there is fine. The "/bin/sh" string and its pointer are data the function reads, never executed — so NX is satisfied at every step. The only requirement is knowing &system.
Concept
Concept
ret into shellcode faults.&system.Concept
vulnerable() with gcc -m32 -fno-stack-protector (NX on by default), disable ASLR for the toy case, and in gdb watch the rip become &system while the stack stays non-executable.Connect it up
Draw it
One page, no notation unless you need it: draw how these connect — W^X & the NX Bit · The Insight That Breaks NX · Return-to-libc Mechanics · Finding the libc Address · Chaining Calls → ROP · Consolidate. Put an arrow wherever one of them is what makes another possible, and label the arrow with why.
Recap
You can state W^X and the NX/XD bit, explain why injecting into buf now faults, build a 32-bit ret2libc stack (rip = &system, fake return address, then the argument pointer), and tell when you need an info leak — ASLR off vs. on — to find libc.
| Idea | The one fact |
|---|---|
| W^X / NX bit | a page is writable XOR executable; hardware enforces it |
| Why injection faults | buf's page is writable → non-executable → ret into it traps |
| The bypass | reuse executable code already in memory (libc), don't inject |
| ret2libc layout (32-bit) | rip = &system | fake ret addr | &"/bin/sh" (arg at ebp+8) |
| Finding libc | ASLR off: read it (ldd/maps); ASLR on: leak it first (L15) |
| Big picture | NX → code reuse; NX + ASLR → you need multiple bugs |
Want this taught 1-on-1? Alexander tutors Computer Security — $55/session, free consultation.