Lesson 13: Positive (Semi)Definite Matrices

Discussion prompt

Before we open Lesson 13: Positive (Semi)Definite Matrices: without looking back, what was the main idea of MAP, Priors & Regularization, and what could you do by the end of it that you could not do before?

Hint: One sentence for the idea, one for the skill. If the second one is blank, that is the part to revisit.

Discussion prompt

Only the symmetric part of a matrix survives in the quadratic form. The antisymmetric part contributes exactly zero, so a non-symmetric matrix and its symmetrization give the same xᵀAx.

That is stated as though it always holds. Do one of two things: produce a case where it fails, or say precisely what rules such a case out. "It just does" is not on the menu.

Hint: Hunt at the extremes first — zero, one, negative, empty, equal. If every extreme survives, the reason they survive is the proof.

Predict first

Take a non-symmetric C = [[2,4],[0,2]]. Its form must equal that of its symmetric part (C + Cᵀ)/2. Confirm on x = [1.3, −0.7]. Self-contained:

Commit before you compute: what does Symmetrization leaves the form unchanged come out to? A rough magnitude and the right form is enough — the point is to have something concrete to be wrong about.

Correct: Both forms print 0.72; symmetric part is [[2,2],[2,2]]

Why: A prediction you can defend turns the computation into a check rather than a leap of faith — and an answer that contradicts it is caught on the spot. The off-diagonal 4 and 0 average to 2 and 2.

Sort into buckets

Sort these by value, from memory, without looking back at Symmetrization leaves the form unchanged. Telling them apart on the spot is the skill; the table is only where the answer happens to be written down.

0.72

x @ C @ x (non-sym); x @ S @ x (sym part)

[[2, 2], [2, 2]]

S = (C+Cᵀ)/2

[0, 4] → PSD

eigvalsh(S)

g1

value is "0.72" for x @ C @ x (non-sym), x @ S @ x (sym part) — that is what the table on "Symmetrization leaves the form unchanged" records, and it is the single property separating this group from the rest.

g2

value is "[[2, 2], [2, 2]]" for S = (C+Cᵀ)/2 — that is what the table on "Symmetrization leaves the form unchanged" records, and it is the single property separating this group from the rest.

g3

value is "[0, 4] → PSD" for eigvalsh(S) — that is what the table on "Symmetrization leaves the form unchanged" records, and it is the single property separating this group from the rest.

Discussion prompt

Explain The quadratic form xᵀAx by analogy to something with no Machine Learning in it at all — a queue, a recipe, a map, a bank balance, whatever fits. Then say where your analogy breaks.

Hint: An analogy that never breaks is not an analogy, it is the same idea wearing a hat. Find the seam — that is the part that is actually new.

Put in order

Put the moves of Expand xᵀAx entry by entry into the order they have to happen.

1. Inner product first: Ax
2. Now xᵀ(Ax): dot x with that vector
3. Distribute and collect

Why: These are the moves of the worked example in the order it makes them, and each one is set up by the one before it. Row i of A dotted with x. Row 0 is [2,1], row 1 is [1,2].

Annotate

From Expand xᵀAx entry by entry — read this one piece at a time. What is each part doing?

On: \( A x = \begin{bmatrix} 2 & 1 \\ 1 & 2 \end{bmatrix}\begin{bmatrix} x_1 \\ x_2 \end{bmatrix} = \begin{bmatrix} 2x_1 + x_2 \\ x_1 + 2x_2 \end{bmatrix} \)

• Row i of A dotted with x. Row 0 is [2,1], row 1 is [1,2].
• xᵀ times a column is x₁·(row 0 result) + x₂·(row 1 result).
• x₁x₂ appears twice (from the two off-diagonal 1s) and combines into 2x₁x₂. This is the general rule: the off-diagonal entry aᵢⱼ shows up on BOTH sides, so its coefficient in the form is 2aᵢⱼ.

Discussion prompt

Our hand expansion says xᵀAx = 2x₁² + 2x₁x₂ + 2x₂². Plug in a messy x = [1.3, −0.7] both ways and confirm they match. Runnable as written:

What do you need to know — or decide — before the first line can be written? List everything the problem has to hand you.

Hint: Anything you would have to invent to get started is a thing the problem must supply.

Comparison matrix

From Verify the form numerically: refill the printed value column from what you know. The rest of the table is as it appeared.

Discussion prompt

Explain Positive definite (PD) to a student a year behind you. No notation, no jargon they have not met — and it still has to be true.

Hint: If your explanation needs a symbol they have never seen, you are describing the notation rather than the idea.

Discussion prompt

Derive the eigenvalue test — set up — before any calculation: what is the plan? Name the moves in order, in plain English, without doing the arithmetic.

Hint: It starts with: Start from the spectral theorem

Draw it

Draw what Derive the eigenvalue test — set up just did — the shape of it, not the line-by-line working. One picture, labels only where you need them. Then check it against the steps: anything you could not draw is a step you followed rather than understood.

Discussion prompt

Derive the eigenvalue test — finish — before any calculation: what is the plan? Name the moves in order, in plain English, without doing the arithmetic.

Hint: It starts with: Λ is diagonal, so yᵀΛy is a plain weighted sum of squares

Predict first

The table runs: np.linalg.eigvalsh(A) | [1. 3.] | all > 0 → PD · np.linalg.det(A) | 2.9999999999999996 | ≈ 3 = λ₁·λ₂ (float noise)

In Test A by eigenvalues, given the rows so far: what is the next one — the row where quantity is np.trace(A)?

Correct: np.trace(A) | 4.0 | = λ₁ + λ₂

Why: The relationship between the columns, not the individual numbers, is what generates the next row. Both strictly positive, so by the test A ≻ 0.

Comparison matrix

From Test A by eigenvalues: every row here is a choice with a cost. Fill the reads as column, then say which row you would actually pick and what you give up for it.

Predict first

Prove A ≻ 0 by hand: complete the square is about to be worked. State your hypothesis first: which rule or definition decides this one, and what is the first move it forces? Then watch whether the example agrees with you.

Correct: Pull the x₁ terms together

Why: Group the two terms containing x₁: 2x₁² + 2x₁x₂ = 2(x₁² + x₁x₂). We'll complete the square inside.

A hypothesis you wrote down is falsifiable; a vague sense of how it will go is not. If the example opens somewhere else, that gap is the thing worth chasing.

Discussion prompt

Work backwards. The example finished here:

Combine the leftover x₂² terms

What was it asked to do, and what must it have been given? Reconstruct the problem from its answer.

Hint: Every quantity in the result had to enter somewhere. Account for each one.

Fill in the blanks

From Check the completed square numerically — one line has had its right-hand side removed. Put it back.

import numpy as np
A = np.array([[2., 1.], [1., 2.]])
x = np.array([1.3, -0.7])
print(round(x @ A @ x, 4)) # matrix form
print(round(2(x[0] + x[1]/2)2 + 1.5x[1]**2, 4)) # completed square

Why: x is what everything below it consumes, so the wrong expression here fails later and somewhere else. Since both weights (2 and 1.5) are positive, the form can never be negative — an algebraic proof of PD that needs no eigenvalue solver.

Predict first

A student writes this, and it looks reasonable:

The determinant is the product of eigenvalues, so det > 0 must mean all eigenvalues are positive — hence PD.

It is wrong. Say what breaks — and say it before you turn the page.

Correct: Two NEGATIVE eigenvalues also multiply to a positive determinant.

Check every leading principal minor (Sylvester) or all the eigenvalues — never just the full determinant.

Why: Two NEGATIVE eigenvalues also multiply to a positive determinant. det > 0 fixes only the sign of the PRODUCT, not each factor. This matrix is negative definite — its bowl points straight down.

Draw it

Translate both ways. First write the expression above as a sentence with no symbols in it at all. Then cover it, and write your sentence back as notation. If the two versions disagree, the disagreement is the thing to fix.

Put in order

Put the moves of Test 4 — Sylvester's leading minors on A into the order they have to happen.

1. First leading minor: top-left 1×1 = 2
2. Second leading minor: full 2×2 determinant = 3
3. Both leading minors > 0 → A is PD

Why: These are the moves of the worked example in the order it makes them, and each one is set up by the one before it. det = 2·2 − 1·1 = 4 − 1 = 3 > 0.

Step through it

Step through Test 4 — Sylvester's leading minors on A one row at a time. What is driving the change, and what would the row after the last one be?

1. Step 1: leading minor is D₁ (1×1)
2. Step 2: leading minor is D₂ (2×2 det)
3. Step 3: leading minor is verdict

Sort into buckets

These are the pieces of Lesson 13: Positive (Semi)Definite Matrices, out of order. Put each one back under the part of the lesson it belongs to.

The quadratic form

Our running matrix; Why symmetry is required; Symmetrization leaves the form unchanged

PSD and PD, defined

Positive definite (PD); Positive semidefinite (PSD); Why we care in ML

Five tests, one matrix

The five equivalent tests; Test 4 — Sylvester's leading minors on A; Test 5 — the Cholesky decomposition

s1

The quadratic form is where Lesson 13: Positive (Semi)Definite Matrices puts Our running matrix, Why symmetry is required, Symmetrization leaves the form unchanged. Knowing which part of the lesson a problem belongs to is most of knowing which method to reach for.

s2

PSD and PD, defined is where Lesson 13: Positive (Semi)Definite Matrices puts Positive definite (PD), Positive semidefinite (PSD), Why we care in ML. Knowing which part of the lesson a problem belongs to is most of knowing which method to reach for.

s3

Five tests, one matrix is where Lesson 13: Positive (Semi)Definite Matrices puts The five equivalent tests, Test 4 — Sylvester's leading minors on A, Test 5 — the Cholesky decomposition. Knowing which part of the lesson a problem belongs to is most of knowing which method to reach for.

Discussion prompt

Compute Cholesky of A by hand — before any calculation: what is the plan? Name the moves in order, in plain English, without doing the arithmetic.

Hint: It starts with: Top-left: ℓ₁₁² = 2 → ℓ₁₁ = √2

Fill in the blanks

Confirm Cholesky in NumPy, with the scaffolding fading: two lines are gone now — fill both.

import numpy as np
A = np.array([[2., 1.], [1., 2.]])
L = np.linalg.cholesky(A)
print(L.round(6))
print((L @ L.T).round(6)) # rebuilds A

Why: Reproducing these unaided, rather than reading them, is what tells you the method has transferred. The solver returns [[1.414214, 0],[0.707107, 1.224745]] — our √2, 1/√2, √1.5 — and multiplying back reconstructs [[2,1],[1,2]].

Discussion prompt

M's form is a perfect square — before any calculation: what is the plan? Name the moves in order, in plain English, without doing the arithmetic.

Hint: It starts with: Expand xᵀMx

Draw it

Draw what M's form is a perfect square just did — the shape of it, not the line-by-line working. One picture, labels only where you need them. Then check it against the steps: anything you could not draw is a step you followed rather than understood.

Fill in the blanks

From Verify M is PSD-only in NumPy — one line has had its right-hand side removed. Put it back.

import numpy as np
M = np.array([[4., 2.], [2., 1.]])
print(np.linalg.det(M)) # 0.0
print(np.linalg.eigvalsh(M)) # [0. 5.]
x = np.array([1., -2.])
print(round(x @ M @ x, 6)) # 0.0 (flat direction)

Why: x is what everything below it consumes, so the wrong expression here fails later and somewhere else. The zero eigenvalue makes M singular (det = 0) and gives the flat direction.

Predict first

Because Cholesky needs a positive diagonal, it raises on a merely-PSD matrix. Catch the error instead of crashing. Runnable:

Commit before you compute: what does Cholesky refuses M come out to? A rough magnitude and the right form is enough — the point is to have something concrete to be wrong about.

Correct: Prints: not PD: Matrix is not positive definite

Why: A prediction you can defend turns the computation into a check rather than a leap of faith — and an answer that contradicts it is caught on the spot. The final pivot ℓ₂₂² would be 1 − (2/2)² = 0, not positive, so no real positive diagonal exists.

Comparison matrix

From Cholesky refuses M: refill the cholesky column from what you know. The rest of the table is as it appeared.

Predict first

A student writes this, and it looks reasonable:

Sylvester says 'all leading minors > 0 ⟹ PD'. So to test PSD, just relax to 'all leading minors ≥ 0'.

It is wrong. Say what breaks — and say it before you turn the page.

Correct: But xᵀBx = −x₂² < 0 for x = [0,1]!

For PSD you must check all principal minors (every diagonal block, not only the top-left ones) — or just use eigenvalues.

Why: But xᵀBx = −x₂² < 0 for x = [0,1]! B has eigenvalue −1 — it is NOT PSD. Leading minors ≥ 0 is not enough: they miss the bottom-right block.

Discussion prompt

The rule this trap just fixed:

The leading-minor shortcut certifies PD only. For PSD, one negative principal minor (or one negative eigenvalue) rules it out. When in doubt, run eigvalsh.

Now break it on purpose. Build a case that violates it and follow the consequences until something visibly fails. Where does the failure first show up — and would you have noticed it if you had not been looking?

Hint: The dangerous rules are the ones whose violation still produces an answer. If yours fails loudly, try to find one that fails quietly.

Discussion prompt

Explain The freebies you'll meet everywhere to a student a year behind you. No notation, no jargon they have not met — and it still has to be true.

Hint: If your explanation needs a symbol they have never seen, you are describing the notation rather than the idea.

Put in order

Put the moves of Prove XᵀX is always PSD into the order they have to happen.

1. Write the quadratic form of XᵀX
2. Recognize xᵀXᵀ = (Xx)ᵀ
3. A squared norm is never negative

Why: These are the moves of the worked example in the order it makes them, and each one is set up by the one before it. Sandwich XᵀX between xᵀ and x. Group as (Xx) using associativity.

Annotate

From Prove XᵀX is always PSD — read this one piece at a time. What is each part doing?

On: \( \boxed{\,x^\top (X^\top X) x = \lVert X x \rVert^2 \ge 0 \quad \Rightarrow \quad X^\top X \succeq 0\,} \)

• Sandwich XᵀX between xᵀ and x. Group as (Xx) using associativity.
• Transpose reverses a product: (Xx)ᵀ = xᵀXᵀ. So the form is (Xx)ᵀ(Xx) — a vector dotted with itself.
• ‖Xx‖² = Σ(Xx)ᵢ² ≥ 0 for EVERY x, no matter what X is. Therefore XᵀX is PSD for any real matrix X — always.

Fill in the blanks

From The ‖Xx‖² identity, numerically — one line has had its right-hand side removed. Put it back.

import numpy as np
np.random.seed(0)
X = np.random.randn(5, 2)
G = X.T @ X
v = np.array([0.8, -0.5])
print(round(v @ G @ v, 6)) # x^T (X^T X) x
print(round((X @ v) @ (X @ v), 6)) # ||Xv||^2 -- equal
print(np.linalg.eigvalsh(G).round(4))

Why: G is what everything below it consumes, so the wrong expression here fails later and somewhere else. The form is a literal squared length, so it matches ‖Xv‖² to the last digit.

Predict first

The table runs: rank(Xd) | 1 (of 2 cols) | rank-deficient · eigvalsh(XᵀX) | [0, 70] | PSD, not PD

In Collinear columns → PSD, not PD, given the rows so far: what is the next one — the row where quantity is det(XᵀX)?

Correct: det(XᵀX) | 0.0 | singular

Why: The relationship between the columns, not the individual numbers, is what generates the next row. The redundant column gives a null direction x with Xx = 0, so ‖Xx‖² = 0 for that nonzero x — a zero eigenvalue.

Step through it

Step through Collinear columns → PSD, not PD one row at a time. What is driving the change, and what would the row after the last one be?

1. Step 1: quantity is rank(Xd)
2. Step 2: quantity is eigvalsh(XᵀX)
3. Step 3: quantity is det(XᵀX)

Discussion prompt

The sample covariance is Σ = XᶜᵀXᶜ / n where Xᶜ is the mean-centered data. That's a Gram matrix over √(1/n)·Xᶜ, so the same ‖·‖² ≥ 0 argument makes it PSD. Verify against np.cov:

What do you need to know — or decide — before the first line can be written? List everything the problem has to hand you.

Hint: Anything you would have to invent to get started is a thing the problem must supply.

Predict first

A student writes this, and it looks reasonable:

Covariance matrices are PSD and we always invert Σ⁻¹ (Mahalanobis distance, the Gaussian). So they must be positive definite.

It is wrong. Say what breaks — and say it before you turn the page.

Correct: If features are collinear, or you have fewer samples than dimensions (n < d), Σ has a zero eigenvalue — PSD but SINGULAR.

Covariance is always PSD; it is PD only with full column rank and enough independent samples.

Why: If features are collinear, or you have fewer samples than dimensions (n < d), Σ has a zero eigenvalue — PSD but SINGULAR. Inverting it divides by ~0 and blows up (the exact failure of Lessons 7 & 10).

Sort into buckets

Some of these hold up and some are the exact mistakes this lesson is built to prevent. Sort them.

Holds up

The central object is a single number built from a vector x and the matrix A: sandwich A between xᵀ on the left and x on the right.; A symmetric matrix A is positive definite if its quadratic form is strictly positive for every nonzero direction x.; A PSD-but-not-PD matrix has a flat direction — a nonzero x with xᵀAx = 0, a trough in the bowl. Every PD matrix is PSD, but not the reverse: PD ⟹ PSD, one-way.

Breaks

The determinant is the product of eigenvalues, so det > 0 must mean all eigenvalues are positive — hence PD.; Sylvester says 'all leading minors > 0 ⟹ PD'. So to test PSD, just relax to 'all leading minors ≥ 0'.

sound

These are stated as this lesson states them — each one survives the edge cases Lesson 13: Positive (Semi)Definite Matrices puts it through.

flawed

Each of these is lifted from a trap in this deck: reasonable-sounding, and wrong in a way that only shows up once you rely on it.

Predict first

Take the singular M = [[4,2],[2,1]] (eigenvalues [0, 5]). Adding λI shifts every eigenvalue up by λ, so λ = 1 turns the deadly 0 into 1 and the matrix becomes PD:

Commit before you compute: what does Ridge lifts every eigenvalue by λ come out to? A rough magnitude and the right form is enough — the point is to have something concrete to be wrong about.

Correct: Eigenvalues [0,5] → [1,6]; Cholesky now succeeds

Why: A prediction you can defend turns the computation into a check rather than a leap of faith — and an answer that contradicts it is caught on the spot. Why the shift works: if Av = λv then (A+λ_r I)v = (λ + λ_r)v — same eigenvectors, every eigenvalue raised by λ_r.

Discussion prompt

Work backwards. The example finished here:

Eigenvalues [0,5] → [1,6]; Cholesky now succeeds

What was it asked to do, and what must it have been given? Reconstruct the problem from its answer.

Hint: Every quantity in the result had to enter somewhere. Account for each one.

Discussion prompt

Explain Sum of two PSD matrices is PSD by analogy to something with no Machine Learning in it at all — a queue, a recipe, a map, a bank balance, whatever fits. Then say where your analogy breaks.

Hint: An analogy that never breaks is not an analogy, it is the same idea wearing a hat. Find the seam — that is the part that is actually new.

Put in order

These are the steps of The definiteness toolkit, scrambled. Put them back in order before the next slide shows you.

1. Symmetric first. PSD/PD are only defined for symmetric A — symmetrize with (A + Aᵀ)/2 if needed
2. Fast yes/no (PD): np.linalg.cholesky(A) succeeds ⟺ PD; wrap it in try/except
3. Full picture: np.linalg.eigvalsh(A) — all ≥ 0 ⟹ PSD, all > 0 ⟹ PD, any < 0 ⟹ indefinite
4. By hand (PD): Sylvester — all leading minors > 0; for PSD you need all principal minors, so prefer eigenvalues
5. Recognize the freebies: any XᵀX and any covariance is PSD (‖Xx‖² ≥ 0); PD iff full column rank; ridge +λI forces PD

Why: This is the order the recipe itself gives. Recalling the sequence without the slide in front of you is the difference between recognising the method and being able to run it — most of what goes wrong in practice is a step done out of turn.

Discussion prompt

The definiteness toolkit works on the cases you have just seen. Push it to the edge: what is the most degenerate input it still handles — empty, zero, one item, everything equal — and what is the first case where it stops being true? Name the case, not just "it breaks".

Hint: Try the smallest legal input, then the largest, then the one where two things collide. Methods are specified at their edges; the middle takes care of itself.

Eliminate the wrong options

A symmetric matrix has eigenvalues {0, 3, 5}. It is:

3 of these 4 are wrong. Strike them one at a time, and say what rules each one out before you strike the next. The survivor is the answer.

• A. PSD but not PD
• B. PD
• C. Neither PSD nor PD
• D. Negative definite

Survives elimination: A

Why: All eigenvalues are ≥ 0, so xᵀAx = Σλᵢyᵢ² ≥ 0 for all x — PSD. But one eigenvalue is exactly 0 (a flat direction, and the matrix is singular), so it fails the strict > 0 test for PD.

Check your understanding

A symmetric matrix has eigenvalues {0, 3, 5}. It is:

• A. PSD but not PD (correct)
• B. PD
• C. Neither PSD nor PD
• D. Negative definite

Answer: A

Why: All eigenvalues are ≥ 0, so xᵀAx = Σλᵢyᵢ² ≥ 0 for all x — PSD. But one eigenvalue is exactly 0 (a flat direction, and the matrix is singular), so it fails the strict > 0 test for PD.

Why B tempts people

PD requires ALL eigenvalues strictly positive. The zero eigenvalue gives a nonzero x with xᵀAx = 0 (the corresponding eigenvector), which violates PD and makes the matrix singular.

Why C tempts people

It IS PSD — every eigenvalue is non-negative, so the form is never negative. 'Neither' would require at least one negative eigenvalue.

Why D tempts people

Negative definite needs all eigenvalues < 0. Here they are 0, 3, 5 — the opposite sign, so the bowl opens upward, not downward.

Predict first

For a symmetric 2×2 matrix A, det(A) > 0 guarantees that A is:

Answer it in your own words, now, with nothing to choose from. The options are on the next slide — and picking the right one off a list is an easier skill than producing it.

Correct: either positive definite or negative definite — det > 0 only fixes the sign of the product

Why: det = λ₁λ₂ > 0 means the two eigenvalues have the SAME sign — both positive (PD) or both negative (ND). It cannot tell them apart. You must also check a leading minor (e.g. A₁₁ > 0 for PD) or the eigenvalues themselves.

Check your understanding

For a symmetric 2×2 matrix A, det(A) > 0 guarantees that A is:

• A. either positive definite or negative definite — det > 0 only fixes the sign of the product (correct)
• B. positive definite
• C. positive semidefinite but not definite
• D. indefinite

Answer: A

Why: det = λ₁λ₂ > 0 means the two eigenvalues have the SAME sign — both positive (PD) or both negative (ND). It cannot tell them apart. You must also check a leading minor (e.g. A₁₁ > 0 for PD) or the eigenvalues themselves.

Why B tempts people

[[−1,0],[0,−1]] has det = 1 > 0 but is negative definite. det > 0 alone never certifies PD — it's the classic trap.

Why C tempts people

A singular (PSD-not-PD) matrix has a zero eigenvalue, so det = λ₁λ₂ = 0, not > 0. det > 0 actually rules out the semidefinite-only case.

Why D tempts people

An indefinite 2×2 has eigenvalues of opposite sign, so det = λ₁λ₂ < 0. det > 0 rules indefinite OUT, not in.

Predict first

For an arbitrary real matrix X, the Gram matrix XᵀX is always…

Answer it in your own words, now, with nothing to choose from. The options are on the next slide — and picking the right one off a list is an easier skill than producing it.

Correct: PSD, and PD iff X has full column rank

Why: xᵀ(XᵀX)x = (Xx)ᵀ(Xx) = ‖Xx‖² ≥ 0 for any X, so XᵀX is always PSD. It is PD exactly when ‖Xx‖² = 0 forces x = 0, i.e. Xx = 0 has only the trivial solution — X has full column rank.

Check your understanding

For an arbitrary real matrix X, the Gram matrix XᵀX is always…

• A. PSD, and PD iff X has full column rank (correct)
• B. always PD
• C. PSD only when X is square
• D. indefinite in general

Answer: A

Why: xᵀ(XᵀX)x = (Xx)ᵀ(Xx) = ‖Xx‖² ≥ 0 for any X, so XᵀX is always PSD. It is PD exactly when ‖Xx‖² = 0 forces x = 0, i.e. Xx = 0 has only the trivial solution — X has full column rank.

Why B tempts people

Not always PD: if X has collinear (linearly dependent) columns, some nonzero x gives Xx = 0, hence ‖Xx‖² = 0 — a zero eigenvalue, so PSD but not PD.

Why C tempts people

The ‖Xx‖² argument holds for ANY shape of X, tall, wide, or square. Squareness is irrelevant to the sign of a squared norm.

Why D tempts people

A squared norm is never negative, so xᵀ(XᵀX)x ≥ 0 always — XᵀX can never be indefinite; it is at least PSD.

Eliminate the wrong options

What happens at np.linalg.cholesky(np.array([[4.,2.],[2.,1.]]))?

3 of these 4 are wrong. Strike them one at a time, and say what rules each one out before you strike the next. The survivor is the answer.

• A. It raises LinAlgError — M is PSD but not positive definite
• B. It returns L = [[2,0],[1,0]] with a zero on the diagonal
• C. It returns the eigenvalues [0, 5]
• D. It succeeds — M is symmetric, which is all Cholesky needs

Survives elimination: A

Why: Cholesky needs a strictly positive diagonal; the last pivot for M would be 1 − (2/2)² = 0, not positive. NumPy raises LinAlgError('Matrix is not positive definite'), which is precisely the PD test — a raise means 'not PD'.

Check your understanding

What happens at np.linalg.cholesky(np.array([[4.,2.],[2.,1.]]))?

• A. It raises LinAlgError — M is PSD but not positive definite (correct)
• B. It returns L = [[2,0],[1,0]] with a zero on the diagonal
• C. It returns the eigenvalues [0, 5]
• D. It succeeds — M is symmetric, which is all Cholesky needs

Answer: A

Why: Cholesky needs a strictly positive diagonal; the last pivot for M would be 1 − (2/2)² = 0, not positive. NumPy raises LinAlgError('Matrix is not positive definite'), which is precisely the PD test — a raise means 'not PD'.

Why B tempts people

NumPy does not return a degenerate L with a zero pivot; it raises before producing one. A zero diagonal would make the factor non-unique and the routine rejects it.

Why C tempts people

cholesky computes a triangular factor, not eigenvalues. Eigenvalues come from eigvalsh; cholesky either returns L or raises.

Why D tempts people

Symmetry is necessary but not sufficient. M is symmetric yet only PSD, so Cholesky still fails. It succeeds only for strictly positive-definite matrices.

Discussion prompt

Build rules: type every line yourself, catch LinAlgError (never let it crash), and always include a matrix you KNOW is only PSD (M) and one that is indefinite (N) — not just easy PD cases.

That is stated as though it always holds. Do one of two things: produce a case where it fails, or say precisely what rules such a case out. "It just does" is not on the menu.

Hint: Hunt at the extremes first — zero, one, negative, empty, equal. If every extreme survives, the reason they survive is the proof.

Comparison matrix

From Milestone 3 — prove they agree: every row here is a choice with a cost. Fill the agree column, then say which row you would actually pick and what you give up for it.

Comparison matrix

From The full program: refill the printed output column from what you know. The rest of the table is as it appeared.

Draw it

One page, no notation unless you need it: draw how these connect — The quadratic form · PSD and PD, defined · Five tests, one matrix · PSD-only and indefinite · Where PSD comes from · Your turn: PD checker. Put an arrow wherever one of them is what makes another possible, and label the arrow with why.