Lesson 114: Phase 3 Mock Exam — Transformers, NLP, and Computer Vision

Discussion prompt

Before we open Lesson 114: Phase 3 Mock Exam — Transformers, NLP, and Computer Vision: without looking back, what was the main idea of CLIP — Contrastive Language-Image Pretraining, and what could you do by the end of it that you could not do before?

Hint: One sentence for the idea, one for the skill. If the second one is blank, that is the part to revisit.

Comparison matrix

From Attention derivation — the four steps: refill the operation column from what you know. The rest of the table is as it appeared.

Put in order

Put the moves of Attention trace — Q, K, V ∈ R^{3×4}, d_k = 4 into the order they have to happen.

1. Compute raw score matrix S = QK^T
2. Scale by 1/√d_k = 1/2 → S_scaled[0] = [1.000, 0.000, 0.500]
3. Softmax row 0: exp([1.0, 0.0, 0.5]) / sum → [0.5065, 0.1863, 0.3072]
4. Output row 0 = 0.5065·V[0] + 0.1863·V[1] + 0.3072·V[2] = [4.2029, 5.2029, 6.2029, 7.2029]

Why: These are the moves of the worked example in the order it makes them, and each one is set up by the one before it. Dot product of each query row with every key row.

Comparison matrix

From Attention trace — Q, K, V ∈ R^{3×4}, d_k = 4: every row here is a choice with a cost. Fill the attn weights column, then say which row you would actually pick and what you give up for it.

Discussion prompt

Verified: nn.MultiheadAttention(embed_dim=64, num_heads=4, bias=False) reports 16,384 parameters — matches the formula exactly.

That is stated as though it always holds. Do one of two things: produce a case where it fails, or say precisely what rules such a case out. "It just does" is not on the menu.

Hint: Hunt at the extremes first — zero, one, negative, empty, equal. If every extreme survives, the reason they survive is the proof.

Predict first

A student writes this, and it looks reasonable:

Exam question: total MHA params for d_model=64, n_heads=4, bias=False?

It is wrong. Say what breaks — and say it before you turn the page.

Correct: Stops after computing W_Q, W_K, W_V for all heads.

Exam question: total MHA params for d_model=64, n_heads=4, bias=False?

Why: Stops after computing W_Q, W_K, W_V for all heads. Misses the output projection W_O that recombines head outputs back to d_model.

Discussion prompt

The rule this trap just fixed:

W_O is a d_model × d_model matrix that merges the concatenated head outputs back into d_model dimensions — always present, always d^2.

Now break it on purpose. Build a case that violates it and follow the consequences until something visibly fails. Where does the failure first show up — and would you have noticed it if you had not been looking?

Hint: The dangerous rules are the ones whose violation still produces an answer. If yours fails loudly, try to find one that fails quietly.

Step through it

Step through Sinusoidal positional encoding — what the values look like one row at a time. What is driving the change, and what would the row after the last one be?

1. Step 1: pos is 0
2. Step 2: pos is 1
3. Step 3: pos is 2
4. Step 4: pos is 3
5. Step 5: pos is 4

Discussion prompt

Explain BERT fine-tuning vs LoRA — parameter efficiency by analogy to something with no Machine Learning in it at all — a queue, a recipe, a map, a bank balance, whatever fits. Then say where your analogy breaks.

Hint: An analogy that never breaks is not an analogy, it is the same idea wearing a hat. Find the seam — that is the part that is actually new.

Put in order

Put the moves of BLEU-4 by hand — ref vs hypothesis into the order they have to happen.

1. Set up: ref = 'the cat sat on the mat', hyp = 'the cat sat on a mat'
2. Compute clipped n-gram precisions: P_1=5/6, P_2=3/5, P_3=2/4, P_4=1/3
3. Brevity penalty: BP = 1 since |hyp|=|ref|=6
4. BLEU-4 = BP · exp((1/4) · Σ log P_n) = exp((log(5/6)+log(3/5)+log(2/4)+log(1/3))/4) = 0.5373

Why: These are the moves of the worked example in the order it makes them, and each one is set up by the one before it. One word differs: 'a' replaces 'the'.

Comparison matrix

From BLEU-4 by hand — ref vs hypothesis: refill the hyp n-grams column from what you know. The rest of the table is as it appeared.

Discussion prompt

Explain ROUGE-N — recall-oriented complement to BLEU by analogy to something with no Machine Learning in it at all — a queue, a recipe, a map, a bank balance, whatever fits. Then say where your analogy breaks.

Hint: An analogy that never breaks is not an analogy, it is the same idea wearing a hat. Find the seam — that is the part that is actually new.

Discussion prompt

NMS result: boxes 0 (score 0.9) and 2 (score 0.7) kept; box 1 suppressed because IoU(box0, box1) = 0.8546 > 0.50.

That is stated as though it always holds. Do one of two things: produce a case where it fails, or say precisely what rules such a case out. "It just does" is not on the menu.

Hint: Hunt at the extremes first — zero, one, negative, empty, equal. If every extreme survives, the reason they survive is the proof.

Put in order

Put the moves of Implement multi-head attention from scratch into the order they have to happen.

1. Project X into Q, K, V using linear layers, then reshape to (B, n_heads, T, d_k)
2. Compute scaled dot-product attention per head: attn = softmax(Q @ K^T / sqrt(d_k)) @ V
3. Merge heads: transpose back to (B, T, heads, d_k), reshape to (B, T, d_model), apply W_O

Why: These are the moves of the worked example in the order it makes them, and each one is set up by the one before it. Each of the n_heads heads attends to a d_k-dimensional subspace.

Annotate

Walk the callouts on Implement multi-head attention from scratch. Each one is a place this is easy to get subtly wrong.

• Each of the n_heads heads attends to a d_k-dimensional subspace. Reshape from (B, T, d_model) → (B, T, n_heads, d_k) then transpose dims 1 and 2.
• Broadcasting: Q and K are (B, heads, T, d_k); Q @ K.transpose(-2,-1) gives (B, heads, T, T). Softmax on dim=-1 (over keys). Result (B, heads, T, d_k).
• .transpose(1,2).contiguous().view(B, T, C) — contiguous() is required before view because transpose creates a non-contiguous tensor.

Sort into buckets

These are the pieces of Lesson 114: Phase 3 Mock Exam — Transformers, NLP, and Computer Vision, out of order. Put each one back under the part of the lesson it belongs to.

Section 1 — Transformer Theory

Attention derivation — the four steps; Attention trace — Q, K, V ∈ R^{3×4}, d_k = 4; Causal masking in decoder self-attention

Section 2 — NLP: BERT, Generation, Metrics

Sinusoidal positional encoding — what the values look like; BERT fine-tuning vs LoRA — parameter efficiency; Greedy decoding vs top-k sampling — trace

Section 3 — Coding & Computer Vision

NMS — non-maximum suppression with IoU; Implement multi-head attention from scratch; Cross-entropy loss and perplexity in language modelling

s1

Section 1 — Transformer Theory is where Lesson 114: Phase 3 Mock Exam — Transformers, NLP, and Computer Vision puts Attention derivation — the four steps, Attention trace — Q, K, V ∈ R^{3×4}, d_k = 4, Causal masking in decoder self-attention. Knowing which part of the lesson a problem belongs to is most of knowing which method to reach for.

s2

Section 2 — NLP: BERT, Generation, Metrics is where Lesson 114: Phase 3 Mock Exam — Transformers, NLP, and Computer Vision puts Sinusoidal positional encoding — what the values look like, BERT fine-tuning vs LoRA — parameter efficiency, Greedy decoding vs top-k sampling — trace. Knowing which part of the lesson a problem belongs to is most of knowing which method to reach for.

s3

Section 3 — Coding & Computer Vision is where Lesson 114: Phase 3 Mock Exam — Transformers, NLP, and Computer Vision puts NMS — non-maximum suppression with IoU, Implement multi-head attention from scratch, Cross-entropy loss and perplexity in language modelling. Knowing which part of the lesson a problem belongs to is most of knowing which method to reach for.

Discussion prompt

Run Phase 3 exam strategy — 6-step recipe with this step confiscated:

ROUGE: compute n-gram overlap; recall = overlap / ref_ngrams; precision = overlap / hyp_ngrams; F1 = harmonic mean.

Is it still possible? If it is, say what takes its place and what it costs you. If it is not, say exactly what that step was providing that nothing else does.

Hint: A step you can drop for free was never load-bearing. If you cannot drop it, name the thing that goes wrong the moment it is gone.

Discussion prompt

Phase 3 exam strategy — 6-step recipe works on the cases you have just seen. Push it to the edge: what is the most degenerate input it still handles — empty, zero, one item, everything equal — and what is the first case where it stops being true? Name the case, not just "it breaks".

Hint: Try the smallest legal input, then the largest, then the one where two things collide. Methods are specified at their edges; the middle takes care of itself.

Eliminate the wrong options

A single-head attention module has Q ∈ R^{8×64}, K ∈ R^{6×64}, V ∈ R^{6×32}. What is the shape of the attention output?

3 of these 4 are wrong. Strike them one at a time, and say what rules each one out before you strike the next. The survivor is the answer.

• A. (8, 32)
• B. (8, 64)
• C. (6, 32)
• D. (8, 6)

Survives elimination: A

Why: QK^T gives (8,64)×(64,6) = (8,6) score matrix. After softmax (8,6). Multiply by V ∈ (6,32): (8,6)×(6,32) = (8,32). Output shape matches V's last dimension, rows match Q's sequence length.

Check your understanding

A single-head attention module has Q ∈ R^{8×64}, K ∈ R^{6×64}, V ∈ R^{6×32}. What is the shape of the attention output?

• A. (8, 32) (correct)
• B. (8, 64)
• C. (6, 32)
• D. (8, 6)

Answer: A

Why: QK^T gives (8,64)×(64,6) = (8,6) score matrix. After softmax (8,6). Multiply by V ∈ (6,32): (8,6)×(6,32) = (8,32). Output shape matches V's last dimension, rows match Q's sequence length.

Why B tempts people

Confuses output with the query key-dimension d_k=64. The output dimension comes from d_v (V's last dim=32), not d_k.

Why C tempts people

Uses K/V sequence length (6) instead of Q sequence length (8). Output sequence length always matches Q's row count.

Why D tempts people

Returns the score matrix shape (8,6) before the weighted sum with V. This is an intermediate step, not the final output.

Predict first

Reference length r=10, hypothesis length c=7. What is the BLEU brevity penalty?

Answer it in your own words, now, with nothing to choose from. The options are on the next slide — and picking the right one off a list is an easier skill than producing it.

Correct: exp(1 - 10/7) ≈ 0.654

Why: BP = exp(1 - r/c) when c < r. Here c=7 < r=10 so BP = exp(1 - 10/7) = exp(-0.4286) ≈ 0.6514. This penalises short hypotheses — a one-word hypothesis with a perfect 1-gram match would still get a near-zero BLEU.

Check your understanding

Reference length r=10, hypothesis length c=7. What is the BLEU brevity penalty?

• A. exp(1 - 10/7) ≈ 0.654 (correct)
• B. 1.000
• C. 7/10 = 0.700
• D. exp(1 - 7/10) ≈ 1.350

Answer: A

Why: BP = exp(1 - r/c) when c < r. Here c=7 < r=10 so BP = exp(1 - 10/7) = exp(-0.4286) ≈ 0.6514. This penalises short hypotheses — a one-word hypothesis with a perfect 1-gram match would still get a near-zero BLEU.

Why B tempts people

BP=1 only when c ≥ r. Here c=7 < r=10, so the hypothesis is too short and must be penalised.

Why C tempts people

Simple ratio c/r is not the formula. The exponential form ensures BP is always ≤ 1 and approaches 0 as the hypothesis shrinks.

Why D tempts people

Inverted the ratio: used exp(1 - c/r) = exp(0.3) ≈ 1.35, which is > 1. BP is capped at 1 and can never exceed it.

Eliminate the wrong options

A 12-layer transformer has d_model=768. LoRA adapters (rank r=8, no bias) are inserted into W_Q and W_V of every layer. How many trainable parameters does LoRA add?

3 of these 4 are wrong. Strike them one at a time, and say what rules each one out before you strike the next. The survivor is the answer.

• A. 294,912
• B. 147,456
• C. 589,824
• D. 12,288

Survives elimination: A

Why: Each LoRA adapter = A (r×d) + B (d×r) = 2×768×8 = 12,288 params. Two adapters per layer (Q and V): 24,576. Times 12 layers: 294,912. Verified with the formula 12 × 2 × 2 × 768 × 8.

Check your understanding

A 12-layer transformer has d_model=768. LoRA adapters (rank r=8, no bias) are inserted into W_Q and W_V of every layer. How many trainable parameters does LoRA add?

• A. 294,912 (correct)
• B. 147,456
• C. 589,824
• D. 12,288

Answer: A

Why: Each LoRA adapter = A (r×d) + B (d×r) = 2×768×8 = 12,288 params. Two adapters per layer (Q and V): 24,576. Times 12 layers: 294,912. Verified with the formula 12 × 2 × 2 × 768 × 8.

Why B tempts people

Counts only Q adapters (12 layers × 1 adapter × 2 matrices × 768×8 = 147,456). Misses the V adapters, which are identical.

Why C tempts people

Doubles the correct answer — perhaps by counting A and B as separate matrices twice each, or mistakenly adding a third adapter per layer.

Why D tempts people

Computes only a single adapter's parameters (2×768×8=12,288) without multiplying by the 12 layers or 2 adapter locations per layer.

Predict first

The table runs: output shape | (3, 4) | (3, 4) · weight row sums | [1.000, 1.000, 1.000] | [1.0000, 1.0000, 1.0000] · causal row 0 | [1.0, 0.0, 0.0] | [1.0000, 0.0000, 0.0000] · causal row 1 | [0.269, 0.731, 0.0] | [0.2689, 0.7311, 0.0000]

In Coding challenge — solution walkthrough, given the rows so far: what is the next one — the row where check is output[0,0]?

Correct: output[0,0] | 4.2029 | 4.2029

Why: The relationship between the columns, not the individual numbers, is what generates the next row. Upper-triangular mask (diagonal=1) zeros out future positions before softmax.

Comparison matrix

From Coding challenge — solution walkthrough: every row here is a choice with a cost. Fill the actual column, then say which row you would actually pick and what you give up for it.

Fill in the blanks

From Show-it-off — SelfAttention module — one line has had its right-hand side removed. Put it back.

class SelfAttention(nn.Module):
def __init__(self, d_model):
super().__init__()
self.Wq = nn.Linear(d_model, d_model, bias=False)
self.Wk = nn.Linear(d_model, d_model, bias=False)
self.Wv = nn.Linear(d_model, d_model, bias=False)
self.Wo = nn.Linear(d_model, d_model, bias=False)

def forward(self, x, causal=False):
Q, K, V = self.Wq(x), self.Wk(x), self.Wv(x)
out, weights = scaled_dot_product_attention(Q, K, V, causal)
return self.Wo(out), weights

sa = SelfAttention(d_model=16)
x = torch.randn(2, 5, 16) # batch=2, seq=5, d=16
out, w = sa(x, causal=True)
print('output:', out.shape) # (2, 5, 16)
print('weights:', w.shape) # (2, 5, 5)
params = sum(p.numel() for p in sa.parameters())
print('params:', params) # 1024

Why: x is what everything below it consumes, so the wrong expression here fails later and somewhere else. The four projection matrices are the only trainable parameters.

Comparison matrix

From Show-it-off — SelfAttention module: refill the derivation column from what you know. The rest of the table is as it appeared.

Draw it

One page, no notation unless you need it: draw how these connect — Section 1 — Transformer Theory · Section 2 — NLP: BERT, Generation, Metrics · Section 3 — Coding & Computer Vision · Your Turn — Coding Problem. Put an arrow wherever one of them is what makes another possible, and label the arrow with why.