x86 Function Calls in Assembly

Discussion prompt

Assembly is a line-by-line record of tiny machine actions: move this value, reserve these bytes, compare this number, jump there if the comparison says so.

That is stated as though it always holds. Do one of two things: produce a case where it fails, or say precisely what rules such a case out. "It just does" is not on the menu.

Hint: Hunt at the extremes first — zero, one, negative, empty, equal. If every extreme survives, the reason they survive is the proof.

Discussion prompt

Read the picture before the words. What is this showing, and what is the one thing it is built to make obvious? Commit to an answer, then read on.

Hint: Name the parts, then say what changes between them — and if nothing changes, say what is being held still.

Discussion prompt

Explain The stack is a workbench with a moving edge by analogy to something with no Computer Architecture in it at all — a queue, a recipe, a map, a bank balance, whatever fits. Then say where your analogy breaks.

Hint: An analogy that never breaks is not an analogy, it is the same idea wearing a hat. Find the seam — that is the part that is actually new.

Comparison matrix

From The four registers to watch first: refill the plain meaning column from what you know. The rest of the table is as it appeared.

Comparison matrix

From The command that makes a .s file: every row here is a choice with a cost. Fill the job column, then say which row you would actually pick and what you give up for it.

Predict first

The table runs: a | first int argument | 2 · b | second int argument | 3 · sum | local variable | 5 · doubled | local variable | 10

In The C function we will compile, given the rows so far: what is the next one — the row where source thing is return?

Correct: return | function result | 11

Why: The relationship between the columns, not the individual numbers, is what generates the next row. a and b are parameters. sum and doubled are local variables that the compiler will place somewhere in the frame at -O0.

Annotate

Walk the callouts on The C function we will compile. Each one is a place this is easy to get subtly wrong.

• a and b are parameters. sum and doubled are local variables that the compiler will place somewhere in the frame at -O0.
• Assembly is easier when you already know what the C code is trying to do. For score(2,3), the branch is the true branch because doubled becomes 10.

Discussion prompt

Explain The compiler output has three jobs to a student a year behind you. No notation, no jargon they have not met — and it still has to be true.

Hint: If your explanation needs a symbol they have never seen, you are describing the notation rather than the idea.

Predict first

A student writes this, and it looks reasonable:

Tempting mistake: read movl %edi, -20(%rbp) as if the left side is the destination.

It is wrong. Say what breaks — and say it before you turn the page.

Correct: That is Intel-style thinking applied to AT&T text.

Correct AT&T read: source first, destination second.

Why: That is Intel-style thinking applied to AT&T text. It reverses the data flow.

Discussion prompt

Read the picture before the words. What is this showing, and what is the one thing it is built to make obvious? Commit to an answer, then read on.

Hint: Name the parts, then say what changes between them — and if nothing changes, say what is being held still.

Put in order

Put the moves of Prologue: line-by-line state changes into the order they have to happen.

1. pushq %rbp saves the caller's frame pointer
2. movq %rsp, %rbp makes a stable anchor
3. subq $24, %rsp allocates local stack space

Why: These are the moves of the worked example in the order it makes them, and each one is set up by the one before it. The CPU subtracts 8 from RSP, then writes the old RBP at the new top of stack.

Predict first

At -O0, GCC often writes parameters to stack slots so the debugger has stable locations for source variables.

Commit before you compute: what does Spilling arguments into the frame come out to? A rough magnitude and the right form is enough — the point is to have something concrete to be wrong about.

Correct: Store the second parameter into its stack slot

Why: A prediction you can defend turns the computation into a check rather than a leap of faith — and an answer that contradicts it is caught on the spot. ESI holds b. The store copies that 32-bit value into memory at RBP-24.

Discussion prompt

Work backwards. The example finished here:

Store the second parameter into its stack slot

What was it asked to do, and what must it have been given? Reconstruct the problem from its answer.

Hint: Every quantity in the result had to enter somewhere. Account for each one.

Discussion prompt

Body: compute sum — before any calculation: what is the plan? Name the moves in order, in plain English, without doing the arithmetic.

Hint: It starts with: Load a into EDX

Draw it

Draw what Body: compute sum just did — the shape of it, not the line-by-line working. One picture, labels only where you need them. Then check it against the steps: anything you could not draw is a step you followed rather than understood.

Predict first

A student writes this, and it looks reasonable:

Tempting mistake: after loading memory into EAX, assume EAX stays connected to that stack slot.

It is wrong. Say what breaks — and say it before you turn the page.

Correct: addl $1,%eax changes EAX. It does not write back to memory unless a later store does that.

Correct: the first line copies the value; the second line changes only the register.

Why: addl $1,%eax changes EAX. It does not write back to memory unless a later store does that.

Discussion prompt

Body: compute doubled — before any calculation: what is the plan? Name the moves in order, in plain English, without doing the arithmetic.

Hint: It starts with: Load sum into EAX

Sort into buckets

Every line below is part of the definition of state or of effective address — one or the other, never both. Put each where it belongs.

state

The current contents of registers, stack memory, flags, and the instruction pointer.; Reading assembly means updating this state one instruction at a time.

effective address

The numeric address produced by an addressing expression such as RBP - 8.; lea loads this address itself, not the value stored at that address.

b1

The current contents of registers, stack memory, flags, and the instruction pointer. Reading assembly means updating this state one instruction at a time.

b2

The numeric address produced by an addressing expression such as RBP - 8. lea loads this address itself, not the value stored at that address.

Predict first

A student writes this, and it looks reasonable:

Tempting mistake: treat lea like a memory load because its operand looks like a memory operand.

It is wrong. Say what breaks — and say it before you turn the page.

Correct: That annotation describes a mov load, not lea.

Correct: lea performs address arithmetic.

Why: That annotation describes a mov load, not lea.

Discussion prompt

The rule this trap just fixed:

The result is a pointer/address. To read the int stored there, use a mov from that address.

Now break it on purpose. Build a case that violates it and follow the consequences until something visibly fails. Where does the failure first show up — and would you have noticed it if you had not been looking?

Hint: The dangerous rules are the ones whose violation still produces an answer. If yours fails loudly, try to find one that fails quietly.

Put in order

Put the moves of Branch trace: equal case into the order they have to happen.

1. Assume doubled is 10
2. cmp sees equality, so ZF becomes 1
3. jne is not taken
4. EAX becomes doubled + 1

Why: These are the moves of the worked example in the order it makes them, and each one is set up by the one before it. For score(2,3), sum is 5 and doubled is 10.

Comparison matrix

From je, jne, and jmp: refill the condition column from what you know. The rest of the table is as it appeared.

Predict first

A student writes this, and it looks reasonable:

Tempting mistake: read subq $24,%rsp as if it creates variables and fills them with zero.

It is wrong. Say what breaks — and say it before you turn the page.

Correct: Subtracting from RSP only moves the pointer.

Correct: stack space exists after the pointer moves; stores give slots meaningful values.

Why: Subtracting from RSP only moves the pointer. It does not clear the memory.

Sort into buckets

Sort these by does RSP change?, from memory, without looking back at Which instructions modify RSP?. Telling them apart on the spot is the skill; the table is only where the answer happens to be written down.

yes

pushq %rbp; subq $24,%rsp; leave; ret

no

movq %rsp,%rbp; movl %edi,-20(%rbp)

g1

does RSP change? is "yes" for pushq %rbp, subq $24,%rsp, leave, ret — that is what the table on "Which instructions modify RSP?" records, and it is the single property separating this group from the rest.

g2

does RSP change? is "no" for movq %rsp,%rbp, movl %edi,-20(%rbp) — that is what the table on "Which instructions modify RSP?" records, and it is the single property separating this group from the rest.

Discussion prompt

Annotate a 15-line listing — before any calculation: what is the plan? Name the moves in order, in plain English, without doing the arithmetic.

Hint: It starts with: First pass: bracket prologue and epilogue

Draw it

Draw what Annotate a 15-line listing just did — the shape of it, not the line-by-line working. One picture, labels only where you need them. Then check it against the steps: anything you could not draw is a step you followed rather than understood.

Discussion prompt

Explain GDB lets you watch the state change live to a student a year behind you. No notation, no jargon they have not met — and it still has to be true.

Hint: If your explanation needs a symbol they have never seen, you are describing the notation rather than the idea.

Predict first

The table runs: score(2,3) | function under inspection | returns 11 · printf | visible confirmation | prints 11

In A full sandbox program, given the rows so far: what is the next one — the row where part is return 0?

Correct: return 0 | normal program exit | exit status 0

Why: The relationship between the columns, not the individual numbers, is what generates the next row. The main function calls score(2,3), so the equal branch returns 11 and printf prints 11.

Discussion prompt

GDB: break and start — before any calculation: what is the plan? Name the moves in order, in plain English, without doing the arithmetic.

Hint: It starts with: Set a breakpoint at score

Annotate

Walk the callouts on GDB: break and start. Each one is a place this is easy to get subtly wrong.

• break score tells GDB to stop when the function begins.
• run starts the program. Execution pauses at or near the beginning of score.
• disassemble /r score prints the function's instructions, with raw bytes if GDB supports /r.

Put in order

Put the moves of GDB: step one instruction into the order they have to happen.

1. stepi executes exactly one machine instruction
2. info registers shows the live register state
3. x/8gx $rsp displays stack memory

Why: These are the moves of the worked example in the order it makes them, and each one is set up by the one before it. Use it on the prologue so the student can see RSP drop after pushq %rbp.

Annotate

In GDB: step one instruction, before reading the notes: mark where the time actually goes. Which line dominates?

• Use it on the prologue so the student can see RSP drop after pushq %rbp.
• Ask for the small set you care about first: stack pointer, frame pointer, instruction pointer, return register, and argument registers.
• x examines memory; 8g means eight giant words, each 8 bytes; x means print them in hexadecimal.

Discussion prompt

Explain How to read x/8gx $rsp by analogy to something with no Computer Architecture in it at all — a queue, a recipe, a map, a bank balance, whatever fits. Then say where your analogy breaks.

Hint: An analogy that never breaks is not an analogy, it is the same idea wearing a hat. Find the seam — that is the part that is actually new.

Discussion prompt

The key habit is prediction before stepping: ask the student what should change, then let GDB check the prediction.

That is stated as though it always holds. Do one of two things: produce a case where it fails, or say precisely what rules such a case out. "It just does" is not on the menu.

Hint: Hunt at the extremes first — zero, one, negative, empty, equal. If every extreme survives, the reason they survive is the proof.

Predict first

A student writes this, and it looks reasonable:

Tempting mistake: compile with aggressive optimization, then expect the beginner stack-frame diagram to match exactly.

It is wrong. Say what breaks — and say it before you turn the page.

Correct: At higher optimization levels, the compiler may keep values in registers, fold arithmetic, omit RBP, or remove branches.

Correct for this lesson: compile for readability first.

Why: At higher optimization levels, the compiler may keep values in registers, fold arithmetic, omit RBP, or remove branches.

Sort into buckets

Some of these hold up and some are the exact mistakes this lesson is built to prevent. Sort them.

Holds up

Assembly is a line-by-line record of tiny machine actions: move this value, reserve these bytes, compare this number, jump there if the comparison says so.; A function call borrows a piece of the stack for temporary work. RSP marks the moving bottom edge of the borrowed area; RBP often marks a stable reference point inside it.; On x86-64, EAX is the low 32 bits of RAX. A function returning an int normally leaves that return value in EAX.

Breaks

Tempting mistake: read movl %edi, -20(%rbp) as if the left side is the destination.; Tempting mistake: after loading memory into EAX, assume EAX stays connected to that stack slot.

sound

These are stated as this lesson states them — each one survives the edge cases x86 Function Calls in Assembly puts it through.

flawed

Each of these is lifted from a trap in this deck: reasonable-sounding, and wrong in a way that only shows up once you rely on it.

Discussion prompt

Outside this lesson: where does x86 Function Calls in Assembly actually turn up? Name one concrete situation — a job, a piece of software someone ships, a decision somebody has to make — and say which part of The assembly annotation recipe is doing the work in it.

Hint: Vague is the failure mode here. "Engineering" is not a situation; "deciding whether this build is fast enough to ship" is.

Eliminate the wrong options

Which instruction directly subtracts 24 from RSP?

3 of these 4 are wrong. Strike them one at a time, and say what rules each one out before you strike the next. The survivor is the answer.

• A. pushq %rbp
• B. movq %rsp, %rbp
• C. subq $24, %rsp
• D. movl %edi, -20(%rbp)

Survives elimination: C

Why: subq $24, %rsp has RSP as the destination, so it directly updates RSP to RSP - 24. pushq also changes RSP, but by 8 and for a different purpose: saving old RBP.

Check your understanding

Which instruction directly subtracts 24 from RSP?

• A. pushq %rbp
• B. movq %rsp, %rbp
• C. subq $24, %rsp (correct)
• D. movl %edi, -20(%rbp)

Answer: C

Why: subq $24, %rsp has RSP as the destination, so it directly updates RSP to RSP - 24. pushq also changes RSP, but by 8 and for a different purpose: saving old RBP.

Why A tempts people

pushq %rbp does change RSP, but it subtracts 8 and stores old RBP. The question asks which instruction subtracts 24.

Why B tempts people

movq %rsp, %rbp copies the current RSP into RBP. RSP is the source, not the destination, so RSP does not change.

Why D tempts people

movl %edi, -20(%rbp) writes to a stack memory slot. It uses RBP to form an address but does not change RSP.

Step through it

Step through Check: which line changes RSP? one row at a time. One of these columns never changes — find it, and say why it cannot.

1. Step 1: line is 1
2. Step 2: line is 2
3. Step 3: line is 3
4. Step 4: line is 4

Predict first

In movl -8(%rbp), %eax, what does -8(%rbp) mean?

Answer it in your own words, now, with nothing to choose from. The options are on the next slide — and picking the right one off a list is an easier skill than producing it.

Correct: The memory at address RBP - 8

Why: -8(%rbp) is an AT&T memory addressing expression. It means compute the address RBP - 8, then read the 4-byte value stored at that address into EAX.

Check your understanding

In movl -8(%rbp), %eax, what does -8(%rbp) mean?

• A. The literal number -8
• B. The memory at address RBP - 8 (correct)
• C. The register named RBP minus the register named 8
• D. The return address

Answer: B

Why: -8(%rbp) is an AT&T memory addressing expression. It means compute the address RBP - 8, then read the 4-byte value stored at that address into EAX.

Why A tempts people

A literal immediate would use $-8 in AT&T syntax. Parentheses around %rbp make this an address expression.

Why C tempts people

There is no register named 8. The 8 is a displacement subtracted from the base register RBP.

Why D tempts people

The return address is normally above RBP at 8(%rbp) after the standard prologue, not below it at -8(%rbp).

Comparison matrix

From Check: what does this operand name?: every row here is a choice with a cost. Fill the meaning column, then say which row you would actually pick and what you give up for it.

Predict first

If -8(%rbp) holds 10, what happens at jne .L2?

Answer it in your own words, now, with nothing to choose from. The options are on the next slide — and picking the right one off a list is an easier skill than producing it.

Correct: It does not jump because ZF is 1

Why: Comparing 10 with 10 sets the zero flag to 1 because the values are equal. jne means jump if not equal, which requires ZF = 0, so this jump is not taken.

Check your understanding

If -8(%rbp) holds 10, what happens at jne .L2?

• A. It jumps because the values are equal
• B. It does not jump because ZF is 1 (correct)
• C. It always jumps
• D. It changes RSP before jumping

Answer: B

Why: Comparing 10 with 10 sets the zero flag to 1 because the values are equal. jne means jump if not equal, which requires ZF = 0, so this jump is not taken.

Why A tempts people

je would jump when the values are equal. jne is the opposite condition.

Why C tempts people

jmp always jumps. jne is conditional and reads the flags set by the previous compare.

Why D tempts people

jne changes control flow by updating RIP if taken. It does not push, pop, or adjust RSP.

Eliminate the wrong options

Which instruction gives RAX the address RBP - 8 without reading the value stored there?

3 of these 4 are wrong. Strike them one at a time, and say what rules each one out before you strike the next. The survivor is the answer.

• A. leaq -8(%rbp), %rax
• B. movl -8(%rbp), %eax
• C. cmpl $10, -8(%rbp)
• D. ret

Survives elimination: A

Why: leaq -8(%rbp), %rax computes the effective address RBP - 8 and stores that address in RAX. movl -8(%rbp), %eax would read the 4-byte value from that address instead.

Check your understanding

Which instruction gives RAX the address RBP - 8 without reading the value stored there?

• A. leaq -8(%rbp), %rax (correct)
• B. movl -8(%rbp), %eax
• C. cmpl $10, -8(%rbp)
• D. ret

Answer: A

Why: leaq -8(%rbp), %rax computes the effective address RBP - 8 and stores that address in RAX. movl -8(%rbp), %eax would read the 4-byte value from that address instead.

Why B tempts people

movl -8(%rbp), %eax dereferences the memory operand and loads the stored value, not the address.

Why C tempts people

cmpl $10, -8(%rbp) reads the value only to set flags for a later branch. It does not put the address in RAX.

Why D tempts people

ret returns to the caller by popping a return address into RIP. It is not an address-arithmetic instruction.

Comparison matrix

From Check: lea or mov?: refill the instruction column from what you know. The rest of the table is as it appeared.

Draw it

One page, no notation unless you need it: draw how these connect — The assembly annotation recipe · Assembly is the compiler's diary · The stack is a workbench with a moving edge · The four registers to watch first · The command that makes a .s file. Put an arrow wherever one of them is what makes another possible, and label the arrow with why.