Lesson 19: PCA Implementation

Discussion prompt

Before we open Lesson 19: PCA Implementation: without looking back, what was the main idea of Full Backpropagation, and what could you do by the end of it that you could not do before?

Hint: One sentence for the idea, one for the skill. If the second one is blank, that is the part to revisit.

Discussion prompt

The trick: rotate to axes along which the data varies the most. Those axes are the principal components.

That is stated as though it always holds. Do one of two things: produce a case where it fails, or say precisely what rules such a case out. "It just does" is not on the menu.

Hint: Hunt at the extremes first — zero, one, negative, empty, equal. If every extreme survives, the reason they survive is the proof.

Discussion prompt

Explain Variance is the signal by analogy to something with no Machine Learning in it at all — a queue, a recipe, a map, a bank balance, whatever fits. Then say where your analogy breaks.

Hint: An analogy that never breaks is not an analogy, it is the same idea wearing a hat. Find the seam — that is the part that is actually new.

Comparison matrix

From One running dataset for the whole lesson: refill the feature 1 column from what you know. The rest of the table is as it appeared.

Discussion prompt

Explain The plan: two routes to the same components to a student a year behind you. No notation, no jargon they have not met — and it still has to be true.

Hint: If your explanation needs a symbol they have never seen, you are describing the notation rather than the idea.

Predict first

Column means: feature 1 averages (2+3+5+7+8)/5 = 5; feature 2 averages (0+1+1+3+5)/5 = 2. Subtract [5, 2] from every row:

Commit before you compute: what does Center the data, row by row come out to? A rough magnitude and the right form is enough — the point is to have something concrete to be wrong about.

Correct: Each centered column now sums to 0

Why: A prediction you can defend turns the computation into a check rather than a leap of faith — and an answer that contradicts it is caught on the spot. Column 0: −3−2+0+2+3 = 0. Column 1: −2−1−1+1+3 = 0.

Comparison matrix

From Center the data, row by row: every row here is a choice with a cost. Fill the f1 − 5 column, then say which row you would actually pick and what you give up for it.

Discussion prompt

X.mean(0) averages down each column; broadcasting subtracts that row-vector from every row. This block runs on its own:

What do you need to know — or decide — before the first line can be written? List everything the problem has to hand you.

Hint: Anything you would have to invent to get started is a thing the problem must supply.

Step through it

Step through Center it in NumPy one row at a time. What is driving the change, and what would the row after the last one be?

1. Step 1: row of Xc is row 1
2. Step 2: row of Xc is row 2
3. Step 3: row of Xc is row 3
4. Step 4: row of Xc is row 4
5. Step 5: row of Xc is row 5

Put in order

Put the moves of Build XᶜᵀXᶜ entry by entry into the order they have to happen.

1. Top-left = c₀·c₀ = 9+4+0+4+9 = 26
2. Off-diagonal = c₀·c₁ = 6+2+0+2+9 = 19
3. Bottom-right = c₁·c₁ = 4+1+1+1+9 = 16

Why: These are the moves of the worked example in the order it makes them, and each one is set up by the one before it. Sum of squared entries of centered feature 1.

Annotate

From Build XᶜᵀXᶜ entry by entry — read this one piece at a time. What is each part doing?

On: \( X_c^\top X_c = \begin{bmatrix} 26 & 19 \\ 19 & 16 \end{bmatrix} \)

• Sum of squared entries of centered feature 1.
• (−3)(−2)+(−2)(−1)+(0)(−1)+(2)(1)+(3)(3) = 6+2+0+2+9 = 19. Appears in both off-diagonal slots by symmetry.
• Sum of squared entries of centered feature 2.

Predict first

The table runs: C[0,0] | var(feature 1) | 6.5 · C[1,1] | var(feature 2) | 4.0

In Divide by n−1 to get C, given the rows so far: what is the next one — the row where C entry is C[0,1]?

Correct: C[0,1] | cov(f1, f2) | 4.75

Why: The relationship between the columns, not the individual numbers, is what generates the next row. Both diagonal variances are positive and the covariance 4.75 is large relative to them — the features are strongly correlated, exactly as the scatter suggested.

Discussion prompt

Work backwards. The example finished here:

C = [[6.5, 4.75], [4.75, 4.0]]

What was it asked to do, and what must it have been given? Reconstruct the problem from its answer.

Hint: Every quantity in the result had to enter somewhere. Account for each one.

Draw it

Draw what Characteristic polynomial of C just did — the shape of it, not the line-by-line working. One picture, labels only where you need them. Then check it against the steps: anything you could not draw is a step you followed rather than understood.

Put in order

Put the moves of Solve the quadratic for the eigenvalues into the order they have to happen.

1. Discriminant = tr² − 4·det = 10.5² − 4(3.4375) = 110.25 − 13.75 = 96.5
2. λ₁ = (10.5 + 9.8234)/2 = 10.1617
3. λ₂ = (10.5 − 9.8234)/2 = 0.3383

Why: These are the moves of the worked example in the order it makes them, and each one is set up by the one before it. Positive, so two distinct real eigenvalues — as it must be for a symmetric matrix.

Draw it

Draw what Solve the quadratic for the eigenvalues just did — the shape of it, not the line-by-line working. One picture, labels only where you need them. Then check it against the steps: anything you could not draw is a step you followed rather than understood.

Draw it

Translate both ways. First write the expression above as a sentence with no symbols in it at all. Then cover it, and write your sentence back as notation. If the two versions disagree, the disagreement is the thing to fix.

Fill in the blanks

From Confirm the eigendecomposition in NumPy — one line has had its right-hand side removed. Put it back.

import numpy as np
X = np.array([[2.,0.],[3.,1.],[5.,1.],[7.,3.],[8.,5.]])
Xc = X - X.mean(0)
C = Xc.T @ Xc / (len(X) - 1)
lam, V = np.linalg.eigh(C) # ascending
order = np.argsort(lam)[::-1] # -> descending
lam, V = lam[order], V[:, order]
print(lam.round(4)) # [10.1617 0.3383]
print(V.round(4)) # columns are the components

Why: C is what everything below it consumes, so the wrong expression here fails later and somewhere else. The eigenvector column printed as [-0.792, -0.6105]: the SAME direction as our (0.792, 0.6105), just flipped in sign.

Predict first

A student writes this, and it looks reasonable:

np.linalg.eigh returned PC1 as [−0.792, −0.6105], but I computed [+0.792, +0.6105] by hand — one of us made an error and the answers disagree.

It is wrong. Say what breaks — and say it before you turn the page.

Correct: There is no error. If v is a unit eigenvector then so is −v: C(−v) = −Cv = −λv = λ(−v).

Treat v and −v as the same component — a principal axis is a line, not an arrow.

Why: There is no error. If v is a unit eigenvector then so is −v: C(−v) = −Cv = −λv = λ(−v). Solvers pick the sign by an internal convention; libraries even disagree with each other.

Fill in the blanks

From Explained-variance ratio from the eigenvalues — finish the line. Write what belongs on the right of the equals sign before you look.

\text\frac{\lambda_i}{\sum_j \lambda_j}_i = ___

Why: Producing the right-hand side unprompted is the difference between recognising this line and being able to use it. 10.1617/10.5 = 0.9678. The five points really do live almost entirely along one line, so 2D → 1D loses only 3.2% of the spread.

Predict first

Substitute Xᶜ = UΣVᵀ into the covariance. Since U is orthonormal (UᵀU = I), the U's cancel:

Commit before you compute: what does Prove the two routes are one: σᵢ² = (n−1)λᵢ come out to? A rough magnitude and the right form is enough — the point is to have something concrete to be wrong about.

Correct: Numeric check: σ₁² = 6.3755² = 40.6469, and (n−1)λ₁ = 4·10.1617 = 40.6469 ✓

Why: A prediction you can defend turns the computation into a check rather than a leap of faith — and an answer that contradicts it is caught on the spot. σ₂² = 1.1632² = 1.3531 = 4·0.3383 too.

Annotate

From Prove the two routes are one: σᵢ² = (n−1)λᵢ — read this one piece at a time. What is each part doing?

On: \( \lambda_i = \frac{\sigma_i^2}{\,n-1\,} \quad\Longleftrightarrow\quad \sigma_i^2 = (n-1)\,\lambda_i \)

• XᶜᵀXᶜ = (UΣVᵀ)ᵀ(UΣVᵀ) = VΣᵀUᵀUΣVᵀ = VΣ²Vᵀ. So C's eigenvectors ARE V's columns, and its eigenvalues are σᵢ²/(n−1).
• σ₂² = 1.1632² = 1.3531 = 4·0.3383 too. The singular values squared are exactly (n−1) times the covariance eigenvalues — the two routes are algebraically identical.

Comparison matrix

From EVR from singular values equals EVR from eigenvalues: refill the EVR (verified) column from what you know. The rest of the table is as it appeared.

Predict first

A student writes this, and it looks reasonable:

Eigendecomposing C and taking the SVD of Xᶜ are separate algorithms, so I should expect different components and different explained variance.

It is wrong. Say what breaks — and say it before you turn the page.

Correct: Wrong premise. C = VΣ²Vᵀ/(n−1), so C's eigenvectors are exactly V's columns and its eigenvalues are σᵢ²/(n−1).

They are the same decomposition. Pick the route by numerical stability, not by expecting a different answer.

Why: Wrong premise. C = VΣ²Vᵀ/(n−1), so C's eigenvectors are exactly V's columns and its eigenvalues are σᵢ²/(n−1). The outputs are the same up to sign.

Discussion prompt

The rule this trap just fixed:

Verified equal to 6 decimals. SVD works on Xᶜ directly (never squares the condition number); eig is cheaper to reason about by hand. Choose the tool, not the answer.

Now break it on purpose. Build a case that violates it and follow the consequences until something visibly fails. Where does the failure first show up — and would you have noticed it if you had not been looking?

Hint: The dangerous rules are the ones whose violation still produces an answer. If yours fails loudly, try to find one that fails quietly.

Fill in the blanks

From Compute the scores — one line has had its right-hand side removed. Put it back.

import numpy as np
X = np.array([[2.,0.],[3.,1.],[5.,1.],[7.,3.],[8.,5.]])
Xc = X - X.mean(0)
U, S, Vt = np.linalg.svd(Xc, full_matrices=False)
Z = Xc @ Vt.T # scores: cols = PC1, PC2
print(Z.round(4))
print('var PC1 =', round(Z[:,0].var(ddof=1), 4)) # 10.1617
print('lambda1 =', round(S[0]**2/(len(X)-1), 4)) # 10.1617

Why: Z is what everything below it consumes, so the wrong expression here fails later and somewhere else. The variance of the projected data along PC1 IS the first eigenvalue — the whole reason unit-length components matter.

Step through it

Step through Compute the scores one row at a time. What is driving the change, and what would the row after the last one be?

1. Step 1: sample is 1
2. Step 2: sample is 2
3. Step 3: sample is 3
4. Step 4: sample is 4
5. Step 5: sample is 5

Fill in the blanks

Reconstruct from k = 1 and measure the loss, with the scaffolding fading: two lines are gone now — fill both.

import numpy as np
X = np.array([[2.,0.],[3.,1.],[5.,1.],[7.,3.],[8.,5.]])
mu = X.mean(0); Xc = X - mu
U, S, Vt = np.linalg.svd(Xc, full_matrices=False)
v1 = Vt[0] # top component
z1 = Xc @ v1 # PC1 scores
Xrec = mu + np.outer(z1, v1) # rebuild from 1 component
print('SSE =', round(np.sum((X - Xrec)2), 4)) # 1.3531
print('sigma2^2 =', round(S[1]2, 4)) # 1.3531

Why: Reproducing these unaided, rather than reading them, is what tells you the method has transferred. The error of dropping PC2 equals its squared singular value (= (n−1)λ₂ = 4·0.3383).

Discussion prompt

Work backwards. The example finished here:

Reconstruction SSE = 1.3531 = σ₂² exactly

What was it asked to do, and what must it have been given? Reconstruct the problem from its answer.

Hint: Every quantity in the result had to enter somewhere. Account for each one.

Fill in the blanks

EVR of the digits data, with the scaffolding fading: two lines are gone now — fill both.

import numpy as np
from sklearn.datasets import load_digits
X = load_digits().data # (1797, 64)
Xc = X - X.mean(0)
U, S, Vt = np.linalg.svd(Xc, full_matrices=False)
evr = S2 / np.sum(S2)
print(evr[:5].round(4)) # [0.1489 0.1362 0.1179 0.0841 0.0578]

Why: Reproducing these unaided, rather than reading them, is what tells you the method has transferred. PC1 captures ~15% of the variance; the spectrum then falls off gradually — a typical natural-image profile with no single dominant axis.

Step through it

Step through EVR of the digits data and watch the numbers move. Now imagine the input ten times bigger: which column is the one that stops this being practical?

1. Step 1: component is PC1
2. Step 2: component is PC2
3. Step 3: component is PC3
4. Step 4: component is PC4
5. Step 5: component is PC5

Fill in the blanks

From The scree curve: where does it flatten? — one line has had its right-hand side removed. Put it back.

import numpy as np
from sklearn.datasets import load_digits
X = load_digits().data
Xc = X - X.mean(0)
S = np.linalg.svd(Xc, full_matrices=False)[1]
evr = S2 / np.sum(S2)
for i in range(8):
print(i+1, round(evr[i], 4))

Why: Xc is what everything below it consumes, so the wrong expression here fails later and somewhere else. 0.149 -> 0.136 -> 0.118 are close, then 0.084, 0.058, and the tail flattens.

Step through it

Step through The scree curve: where does it flatten? one row at a time. What is driving the change, and what would the row after the last one be?

1. Step 1: component is PC1
2. Step 2: component is PC2
3. Step 3: component is PC3
4. Step 4: component is PC4
5. Step 5: component is PC5

Discussion prompt

Accumulate the EVR with cumsum, then find the first index that crosses the threshold. searchsorted gives the insertion point; add 1 for a count:

What do you need to know — or decide — before the first line can be written? List everything the problem has to hand you.

Hint: Anything you would have to invent to get started is a thing the problem must supply.

Step through it

Step through How many components reach 95%? and watch the numbers move. Now imagine the input ten times bigger: which column is the one that stops this being practical?

1. Step 1: cumulative target is 50%
2. Step 2: cumulative target is 90%
3. Step 3: cumulative target is 95%
4. Step 4: cumulative target is 99%
5. Step 5: cumulative target is 100%

Predict first

Keep the first two component rows of Vᵀ and project. The result is (1797, 2) — ready to scatter by label:

Commit before you compute: what does Project digits to 2D come out to? A rough magnitude and the right form is enough — the point is to have something concrete to be wrong about.

Correct: Z is (1797, 2) — 64D compressed to a plottable plane

Why: A prediction you can defend turns the computation into a check rather than a leap of faith — and an answer that contradicts it is caught on the spot. Each row is one digit's coordinates on PC1/PC2.

Step through it

Step through Project digits to 2D one row at a time. What is driving the change, and what would the row after the last one be?

1. Step 1: sample is 1
2. Step 2: sample is 2
3. Step 3: sample is 3

Discussion prompt

Explain Why a spiral breaks it to a student a year behind you. No notation, no jargon they have not met — and it still has to be true.

Hint: If your explanation needs a symbol they have never seen, you are describing the notation rather than the idea.

Predict first

A student writes this, and it looks reasonable:

PCA is a powerful reducer, so it will uncover whatever structure is present — curved or not. Just run PCA on the Swiss roll to unroll it.

It is wrong. Say what breaks — and say it before you turn the page.

Correct: PCA only takes linear combinations of features.

Use PCA for linear structure and quick looks; switch tools when the manifold is curved.

Why: PCA only takes linear combinations of features. A curved manifold can't be flattened by rotation + projection, so PCA overlaps the folds and destroys the very structure you wanted.

Sort into buckets

Some of these hold up and some are the exact mistakes this lesson is built to prevent. Sort them.

Holds up

The trick: rotate to axes along which the data varies the most. Those axes are the principal components.; PCA's core bet: the directions where the data spreads out carry the structure, and the directions where it barely moves are mostly noise or redundancy.; The two features clearly rise together — strongly correlated. PCA should find one dominant direction along that trend and a tiny second one across it.

Breaks

np.linalg.eigh returned PC1 as [−0.792, −0.6105], but I computed [+0.792, +0.6105] by hand — one of us made an error and the answers disagree.; Eigendecomposing C and taking the SVD of Xᶜ are separate algorithms, so I should expect different components and different explained variance.

sound

These are stated as this lesson states them — each one survives the edge cases Lesson 19: PCA Implementation puts it through.

flawed

Each of these is lifted from a trap in this deck: reasonable-sounding, and wrong in a way that only shows up once you rely on it.

Put in order

These are the steps of The PCA recipe, scrambled. Put them back in order before the next slide shows you.

1. Center: subtract the column means → Xᶜ (never skip this)
2. Decompose: SVD of Xᶜ (preferred) — or eigendecompose C = XᶜᵀXᶜ/(n−1); both give the same V
3. EVR: σᵢ²/Σσⱼ² (= λᵢ/Σλⱼ); accumulate with cumsum for variance kept
4. Choose k: scree elbow, or smallest k with cumulative EVR ≥ target (95% ⇒ 29 of 64 on digits)
5. Project: Z = Xᶜ V[:, :k]; reconstruction error = Σⱼ₌ₖ₊₁ σⱼ². Remember it is linear-only

Why: This is the order the recipe itself gives. Recalling the sequence without the slide in front of you is the difference between recognising the method and being able to run it — most of what goes wrong in practice is a step done out of turn.

Discussion prompt

The PCA recipe works on the cases you have just seen. Push it to the edge: what is the most degenerate input it still handles — empty, zero, one item, everything equal — and what is the first case where it stops being true? Name the case, not just "it breaks".

Hint: Try the smallest legal input, then the largest, then the one where two things collide. Methods are specified at their edges; the middle takes care of itself.

Eliminate the wrong options

The explained-variance ratio of principal component i is:

3 of these 4 are wrong. Strike them one at a time, and say what rules each one out before you strike the next. The survivor is the answer.

• A. σᵢ² / Σⱼ σⱼ²
• B. σᵢ / Σⱼ σⱼ
• C. σᵢ² (the raw squared singular value)
• D. 1 / k for each of the k kept components

Survives elimination: A

Why: Variance along PC i is proportional to σᵢ² (since λᵢ = σᵢ²/(n−1)), so its share of the total is σᵢ²/Σⱼσⱼ². These ratios sum to 1 — on our toy they were [0.9678, 0.0322].

Check your understanding

The explained-variance ratio of principal component i is:

• A. σᵢ² / Σⱼ σⱼ² (correct)
• B. σᵢ / Σⱼ σⱼ
• C. σᵢ² (the raw squared singular value)
• D. 1 / k for each of the k kept components

Answer: A

Why: Variance along PC i is proportional to σᵢ² (since λᵢ = σᵢ²/(n−1)), so its share of the total is σᵢ²/Σⱼσⱼ². These ratios sum to 1 — on our toy they were [0.9678, 0.0322].

Why B tempts people

Variance scales with the SQUARE of the singular value, not the singular value itself. Using σ instead of σ² gives the wrong shares (and they wouldn't correspond to variance at all).

Why C tempts people

σᵢ² is the unnormalized variance (times n−1). The RATIO divides by the total so the shares sum to 1 and are comparable across datasets.

Why D tempts people

Equal 1/k shares would only hold for perfectly isotropic data. Real components capture unequal variance — PC1 held 96.8% on the toy, not 50%.

Predict first

How do the covariance eigenvalues λᵢ relate to the singular values σᵢ of the centered data Xᶜ (n samples)?

Answer it in your own words, now, with nothing to choose from. The options are on the next slide — and picking the right one off a list is an easier skill than producing it.

Correct: σᵢ² = (n−1)·λᵢ, so both routes give identical explained-variance ratios

Why: Xᶜ = UΣVᵀ gives C = XᶜᵀXᶜ/(n−1) = VΣ²Vᵀ/(n−1), so λᵢ = σᵢ²/(n−1), i.e. σᵢ² = (n−1)λᵢ. On the toy: σ₁² = 40.6469 = 4·10.1617. The (n−1) cancels in the EVR, so both routes report the same shares.

Check your understanding

How do the covariance eigenvalues λᵢ relate to the singular values σᵢ of the centered data Xᶜ (n samples)?

• A. σᵢ² = (n−1)·λᵢ, so both routes give identical explained-variance ratios (correct)
• B. σᵢ = λᵢ exactly
• C. They are unrelated — eig and SVD are different algorithms with different answers
• D. λᵢ = (n−1)·σᵢ²

Answer: A

Why: Xᶜ = UΣVᵀ gives C = XᶜᵀXᶜ/(n−1) = VΣ²Vᵀ/(n−1), so λᵢ = σᵢ²/(n−1), i.e. σᵢ² = (n−1)λᵢ. On the toy: σ₁² = 40.6469 = 4·10.1617. The (n−1) cancels in the EVR, so both routes report the same shares.

Why B tempts people

Off by a square and a factor: σ is a singular value of Xᶜ, λ is an eigenvalue of the covariance. σ₁ = 6.3755 but λ₁ = 10.1617 — clearly not equal.

Why C tempts people

They are the SAME decomposition. C = VΣ²Vᵀ/(n−1), so C's eigenvectors are V's columns. Verified equal to 6 decimals in the lesson.

Why D tempts people

The (n−1) and the square are on the wrong sides. It is σᵢ² that equals (n−1)λᵢ, not the reverse.

Predict first

On digits, 29 of 64 components reach 95% cumulative explained variance. This means:

Answer it in your own words, now, with nothing to choose from. The options are on the next slide — and picking the right one off a list is an easier skill than producing it.

Correct: Projecting to 29 dims keeps 95% of the total variance — a >2× compression

Why: The CUMULATIVE EVR of the top 29 components is ≈0.9548, the first value ≥ 0.95. So a 64→29 projection retains 95% of the variance while more than halving the dimension.

Check your understanding

On digits, 29 of 64 components reach 95% cumulative explained variance. This means:

• A. Projecting to 29 dims keeps 95% of the total variance — a >2× compression (correct)
• B. Each of the 29 components individually explains 95% of the variance
• C. The data has exactly 29 true dimensions
• D. 29 components are needed to reach 100% of the variance

Answer: A

Why: The CUMULATIVE EVR of the top 29 components is ≈0.9548, the first value ≥ 0.95. So a 64→29 projection retains 95% of the variance while more than halving the dimension.

Why B tempts people

Each component explains its own small share (PC1 only ~15%). It is the SUM of the top 29 shares that reaches 95%, not each one.

Why C tempts people

Variance is continuous, not a hard rank. 95% is a chosen threshold; at 99% you'd keep 41, at 90% only 21. There is no single 'true' 29.

Why D tempts people

100% needs all 64 components. 29 reaches 95%; the remaining 35 hold the final 5% of variance.

Eliminate the wrong options

Your data lies on a curved 2-D manifold (a Swiss roll) embedded in 3-D. Running PCA down to 2-D will:

3 of these 4 are wrong. Strike them one at a time, and say what rules each one out before you strike the next. The survivor is the answer.

• A. Fail to unroll it — PCA captures only linear structure and overlaps the folds
• B. Perfectly unroll the manifold into a flat sheet
• C. Cluster the points by their class label
• D. Return 3 components instead of 2

Survives elimination: A

Why: PCA only takes linear projections — rotate and flatten. Unrolling a curved manifold is a nonlinear operation it cannot perform, so far-apart folds collapse onto each other. Use t-SNE/UMAP for curved manifolds.

Check your understanding

Your data lies on a curved 2-D manifold (a Swiss roll) embedded in 3-D. Running PCA down to 2-D will:

• A. Fail to unroll it — PCA captures only linear structure and overlaps the folds (correct)
• B. Perfectly unroll the manifold into a flat sheet
• C. Cluster the points by their class label
• D. Return 3 components instead of 2

Answer: A

Why: PCA only takes linear projections — rotate and flatten. Unrolling a curved manifold is a nonlinear operation it cannot perform, so far-apart folds collapse onto each other. Use t-SNE/UMAP for curved manifolds.

Why B tempts people

Unrolling a curl is inherently NONLINEAR; a single straight projection can't do it. This is exactly the case PCA fails on.

Why C tempts people

PCA is unsupervised and label-agnostic — it maximizes variance, it does not know or use class labels to cluster.

Why D tempts people

Projecting to 2-D returns 2 components by construction. The problem is the quality of that projection, not the number of components.

Discussion prompt

Explain Project: a PCA class from scratch by analogy to something with no Machine Learning in it at all — a queue, a recipe, a map, a bank balance, whatever fits. Then say where your analogy breaks.

Hint: An analogy that never breaks is not an analogy, it is the same idea wearing a hat. Find the seam — that is the part that is actually new.

Comparison matrix

From Milestone 3 — verify & choose k: every row here is a choice with a cost. Fill the value (verified) column, then say which row you would actually pick and what you give up for it.

Predict first

The table runs: EVR top 3 | [0.1489, 0.1362, 0.1179] · k @ 95% | 29

In The full PCA class, given the rows so far: what is the next one — the row where output is 2D shape?

Correct: 2D shape | (1797, 2)

Why: The relationship between the columns, not the individual numbers, is what generates the next row. fit() stores the mean and components; transform() re-centers with the SAME mean and projects onto k components.

Comparison matrix

From The full PCA class: refill the value (verified) column from what you know. The rest of the table is as it appeared.

Discussion prompt

Slides closed, out loud: explain (1) why we center before decomposing, (2) why the SVD route avoids forming XᶜᵀXᶜ, and (3) what σᵢ² = (n−1)λᵢ says about the two routes being one.

That is stated as though it always holds. Do one of two things: produce a case where it fails, or say precisely what rules such a case out. "It just does" is not on the menu.

Hint: Hunt at the extremes first — zero, one, negative, empty, equal. If every extreme survives, the reason they survive is the proof.

Draw it

One page, no notation unless you need it: draw how these connect — What PCA is for · Step 1: center · Route A: covariance eigendecomposition · Route B: SVD · Step 2: project · Choosing k at scale. Put an arrow wherever one of them is what makes another possible, and label the arrow with why.