This deck sets up three classic sequence dynamic programs from scratch: Longest Increasing Subsequence, Longest Common Subsequence, and Edit Distance. It emphasizes defining the state in words, choosing the correct base row and column, writing the match-or-mismatch recurrence, and tracing the grid to read back the actual answer. It targets confusing a subsequence with a substring, mis-initializing the Edit Distance base case to zero, tracing the answer back from the wrong cell or in the wrong direction, and thinking that LIS requires contiguous elements.
Subject: CS3000 Algorithms · 131 slides · symbolic lesson
Open the interactive version of this deck · Homework for this lesson
Objectives
Today you'll build three related dynamic-programming algorithms that all compare or order sequences. By the end you can:
Warm-up
Discussion prompt
Before we open Dynamic Programming III: LIS, LCS & Edit Distance: without looking back, what was the main idea of Dynamic Programming II: Knapsack, Coin Change & Rod Cutting, and what could you do by the end of it that you could not do before?
Hint: One sentence for the idea, one for the skill. If the second one is blank, that is the part to revisit.
Answer:
0/1 knapsack, minimum-coin and combination-counting coin change, and rod cutting, taught with the setup made explicit every time: define the state in words, write the take-or-skip (or best-choice) recurrence, pin the base case, fill the table in dependency order, and reconstruct the actual items, coins, or cuts chosen. Targets four real misconceptions: using a best-ratio greedy strategy on 0/1 knapsack, accidentally reusing a single-copy item, using a greedy coin heuristic on a coin system where it overshoots the true minimum, and an off-by-one in the table's capacity or amount dimension.
Concept
Before any new material: cover the screen.
You have named 13 reusable moves so far. Say as many as you can out loud, by number, from memory.
Do not advance until you have actually tried. Getting four of eight is information; skipping the exercise is not.
Here they are. Score yourself.
Today adds no new moves. Every proof in this lesson is built out of the list above. That is the whole point of the list.
The question that starts every proof from here on is not how do I begin. It is which of these applies here?
Counterexample
Discussion prompt
You have named 13 reusable moves so far. Say as many as you can out loud, by number, from memory.
That is stated as though it always holds. Do one of two things: produce a case where it fails, or say precisely what rules such a case out. "It just does" is not on the menu.
Hint: Hunt at the extremes first — zero, one, negative, empty, equal. If every extreme survives, the reason they survive is the proof.
Answer:
Do not advance until you have actually tried. Getting four of eight is information; skipping the exercise is not.
Concept
A problem is a good fit for dynamic programming when it has two properties: overlapping subproblems (the same smaller question gets asked over and over) and optimal substructure (the best answer to the big question is built from the best answers to smaller ones).
Instead of recomputing a subproblem every time it comes up, we solve each one once, store it in a table, and reuse it. All three algorithms in this lesson are exactly that: a table of small answers that builds up to one big answer.
Concept
A subsequence of a list is what's left after you delete zero or more elements, without changing the order of what remains. The elements you keep do not need to be next to each other.
subsequence — A sequence obtained by deleting some (possibly zero) elements from another sequence, without reordering the ones that remain. For example, "ACE" is a subsequence of "ABCDE": keep positions 1, 3, and 5, and drop the rest.
Analogy
Discussion prompt
Explain What a subsequence is by analogy to something with no CS3000 Algorithms in it at all — a queue, a recipe, a map, a bank balance, whatever fits. Then say where your analogy breaks.
Hint: An analogy that never breaks is not an analogy, it is the same idea wearing a hat. Find the seam — that is the part that is actually new.
Answer:
A subsequence of a list is what's left after you delete zero or more elements, without changing the order of what remains. The elements you keep do not need to be next to each other.
Picture it
Animation
Shows: Subsequence is not substring — a rendered Manim animation.
Rendered with Manim.
Takeaway: LCS allows gaps, which is why a table beats a scan.
Anomaly
Predict first
A student writes this, and it looks reasonable:
A student is asked whether "ACE" is a subsequence of "ABCDE", and instead checks whether "ACE" appears as a contiguous block of letters inside "ABCDE".
It is wrong. Say what breaks — and say it before you turn the page.
Correct: This tests for a substring - a contiguous chunk - which is a stricter, different requirement than a subsequence.
Check the subsequence condition instead: can you find A, then C, then E, each appearing later in the string than the last one, without requiring them to touch?
Why: This tests for a substring - a contiguous chunk - which is a stricter, different requirement than a subsequence.
Trap
A student is asked whether "ACE" is a subsequence of "ABCDE", and instead checks whether "ACE" appears as a contiguous block of letters inside "ABCDE".
Search for "ACE" as a run of adjacent letters
Why: This tests for a substring - a contiguous chunk - which is a stricter, different requirement than a subsequence.
Conclude "ACE" is not there
Why: "ABCDE" contains no contiguous run of letters spelling "ACE" (the letters B and D sit between them), so a substring search wrongly reports failure.
Check the subsequence condition instead: can you find A, then C, then E, each appearing later in the string than the last one, without requiring them to touch?
Scan left to right, matching one target letter at a time
Why: A is at position 1, C is at position 3 (after A), and E is at position 5 (after C). Order is preserved even though B and D are skipped.
Conclude "ACE" is a valid subsequence
Why: Every letter of "ACE" was found in order, just not adjacently. Substrings must be contiguous; subsequences only need to preserve order.
Concept
All three algorithms today - Longest Increasing Subsequence, Longest Common Subsequence, and Edit Distance - reason about subsequences, not substrings. Every recurrence you're about to see is built on the idea of skipping elements freely while keeping order.
If you accidentally think in terms of contiguous runs, the recurrences will look wrong and the trace tables won't match what the DP is actually counting.
Explain it
Discussion prompt
Explain Why this distinction matters for today's three DPs to a student a year behind you. No notation, no jargon they have not met — and it still has to be true.
Hint: If your explanation needs a symbol they have never seen, you are describing the notation rather than the idea.
Answer:
If you accidentally think in terms of contiguous runs, the recurrences will look wrong and the trace tables won't match what the DP is actually counting.
Concept
Longest Increasing Subsequence: finding the longest stretch of improving values buried in noisy data, like the best run of increasing days in a stock's price history.
Longest Common Subsequence: the basis of file-diff tools (what changed between two versions of a file) and comparing DNA sequences for similarity.
Edit Distance: spell-checkers and search-suggestion systems use it to measure how many keystrokes separate what you typed from a real word.
Concept
You're given a list of numbers. You want the longest subsequence whose values are strictly increasing from left to right.
\[ a_1, a_2, \ldots, a_n \]
Symbols: a is the array; a with a subscript is one of its numbers, and n is how many numbers there are, read in the given order.
For example, in the list 5, 2, 8, 6, 3, 6, 9, 7 one increasing subsequence is 2, 3, 6, 9 - each later value is bigger than the one before it, even though they are not next to each other in the original list.
Intuition
A list of n numbers has an enormous number of subsequences - each element is either kept or dropped, independently of every other. Checking every single one for being increasing, then keeping the longest, blows up impossibly fast as n grows.
\[ 2^{n} \text{ subsequences to check} \]
Intuition
A tempting shortcut: scan left to right and greedily keep a number whenever it's bigger than the last one you kept. This fails, because an early greedy pick can block a much longer chain later.
For example, keeping 5 first (since it's the very first number) can lock you out of the longer chain 2, 3, 6, 9, which only works by skipping 5 entirely. You cannot know which number to commit to without looking ahead.
Intuition
Instead of guessing which numbers to commit to, answer a smaller question for every position: if my increasing subsequence has to end exactly at this number, how long can it be?
Once you know that answer for every earlier position, extending to the current position is easy: look back at every smaller, earlier number and build onto the best one.
Concept
Define one small answer per position. Let i be a position in the array, and let dp[i] be the length of the longest increasing subsequence that ends exactly at position i - it must use a with subscript i as its last element.
\[ \text{dp}[i] = \text{length of the longest increasing subsequence ending at position } i \]
This is the single most important setup choice: dp[i] is not 'the best answer using the first i numbers' - it is pinned to end exactly at position i.
Concept
Every proof of this kind has the same five or six moves in the same order. The order is not something you rediscover each time.
It is on the right. It will stay on the right through the worked examples that follow.
Why this matters: the structure is now handled. You are not spending working memory on what comes next — you are spending all of it on the one hard step.
Step 1 is where every failed DP fails. A subproblem you cannot say in one sentence is a subproblem you cannot write a recurrence for, and steps 2 through 6 will not rescue it.
Intuition
What feels wrong about this?
Two candidate sentences for the LIS table entry:
\[ (a) \;\; dp[i] = \text{longest increasing subsequence ending exactly at } i \]
\[ (b) \;\; dp[i] = \text{longest increasing subsequence among the first } i \text{ numbers} \]
Definition (b) answers the question you actually care about. Definition (a) does not.
_Plain English only. No notation, no algebra. Just say what bothers you._
The feeling: definition (b) tells you a length but not what the sequence ends with, and without that you cannot tell whether the next number is allowed to extend it.
That feeling is the proof. It is not a substitute for the proof — it is the thing the proof writes down.
This is the same trap as Kadane's best-ending-here. The entry that answers the final question directly is usually the one you cannot recurse on — you want the entry that carries the fact the next step needs.
Anomaly
Predict first
A student writes this, and it looks reasonable:
A student defines dp[i] as "the length of the longest increasing subsequence using any of the first i numbers," and expects dp[i] to always be at least as large as dp[i-1].
It is wrong. Say what breaks — and say it before you turn the page.
Correct: If dp[i] means 'best among the first i,' then having one more candidate number can never make the best answer worse.
dp[i] is pinned to subsequences that end exactly at position i. It can go up or down compared to dp[i-1]; there's no rule that dp[2] has to be at least dp[1].
Why: If dp[i] means 'best among the first i,' then having one more candidate number can never make the best answer worse.
Trap
A student defines dp[i] as "the length of the longest increasing subsequence using any of the first i numbers," and expects dp[i] to always be at least as large as dp[i-1].
Assume dp[i] only ever grows as i increases
Why: If dp[i] means 'best among the first i,' then having one more candidate number can never make the best answer worse.
Try to fill dp[2] this way for the array 5, 2, 8, ...
Why: Under this wrong definition, dp[2] would just copy dp[1] whenever a with subscript 2 doesn't extend anything, blurring together answers that actually end at different positions.
dp[i] is pinned to subsequences that end exactly at position i. It can go up or down compared to dp[i-1]; there's no rule that dp[2] has to be at least dp[1].
Check the real values for 5, 2, 8, ...
Why: dp[1] = 1 (just the number 5, alone). dp[2] = 1 too (just the number 2, alone) - not because it 'copies' dp[1], but because no earlier number is smaller than 2.
Keep the two questions separate
Why: The answer to the WHOLE problem is the maximum over all dp[i]; each individual dp[i] only answers the narrower 'ending here' question.
Intuition
What move should we make next?
The sentence, fixed:
\[ dp[i] = \text{length of the longest increasing subsequence ending exactly at index } i \]
The last element of that subsequence is forced — it is the number at index i.
So what is the last decision? Careful: unlike the stairs, this one does not have two cases.
_Look at your toolkit. Say a move number out loud before this slide advances._ A wrong guess is useful. A silent guess is not.
Concept
To build a subsequence ending at position i, look at every earlier position j whose value is smaller than a with subscript i. Any increasing subsequence ending at j can be extended by tacking a with subscript i onto the end.
\[ \text{dp}[i] = 1 + \max_{\substack{j < i \\ a_j < a_i}} \text{dp}[j] \]
In words: dp[i] is one more than the best dp[j] among all earlier positions j whose value is strictly smaller than a with subscript i.
Fill the middle
Fill in the blanks
From A case split with many branches, not two — finish the line. Write what belongs on the right of the equals sign before you look.
dp[i] = 1 + \max\{\, dp[j] \;:\; j < i \text{ and } A[j] < A[i] \,\}
Why: Producing the right-hand side unprompted is the difference between recognising this line and being able to use it. Any earlier index j whose value is smaller is a legal predecessor.
Concept
The move: #12 (Case-split on the last decision), then #13 (Name the subproblem).
The last decision is which element came immediately before index i
Why: Any earlier index j whose value is smaller is a legal predecessor. That is not two cases — it is up to i cases, and you take the best.
\[ dp[i] = 1 + \max\{\, dp[j] \;:\; j < i \text{ and } A[j] < A[i] \,\} \]
The base case is the empty maximum
Why: If no earlier element is smaller, the maximum is over an empty set and dp[i] is just 1 — the element standing alone.
The answer is not dp of n
Why: Because entries mean ending exactly here, the answer is the largest entry anywhere in the table. Forgetting this is the single most common LIS error.
Case splits are not always binary. Take-or-skip had two branches; this one has as many branches as there are legal predecessors. The move is the same either way: split on the last decision, take the best.
Notation
Annotate
From A case split with many branches, not two — read this one piece at a time. What is each part doing?
On: \( dp[i] = 1 + \max\{\, dp[j] \;:\; j < i \text{ and } A[j] < A[i] \,\} \)
ending exactly here, the answer is the largest entry anywhere in the table. Forgetting this is the single most common LIS error.Concept
If no earlier number is smaller than a with subscript i, position i cannot extend anything - it starts its own one-element chain.
\[ \text{dp}[i] = 1 \quad \text{if no } j < i \text{ has } a_j < a_i \]
This is really the same rule as the general recurrence: when there is no valid j to extend, the 'best dp[j]' contributes nothing, so dp[i] falls back to 1.
Ranking
Put in order
Put the moves of Worked example: LIS - start filling the table into the order they have to happen.
Why: These are the moves of the worked example in the order it makes them, and each one is set up by the one before it. Position 1 has nothing before it, so it starts its own chain of length 1.
Worked example
Find the longest increasing subsequence of this array.
\[ a:\quad 5,\ 2,\ 8,\ 6,\ 3,\ 6,\ 9,\ 7 \qquad (\text{positions } 1 \text{ to } 8) \]
Compute dp[1]
Why: Position 1 has nothing before it, so it starts its own chain of length 1.
\[ \text{dp}[1] = 1 \]
Compute dp[2]
Why: Value at position 2 is 2. The only earlier value is 5, which is not smaller than 2, so position 2 cannot extend anything.
\[ \text{dp}[2] = 1 \]
Compute dp[3]
Why: Value at position 3 is 8. Both earlier values (5 and 2) are smaller than 8, so position 3 can extend either chain. The better of dp[1]=1 and dp[2]=1 is 1, plus one more for position 3 itself.
\[ \text{dp}[3] = 1 + \max(\text{dp}[1], \text{dp}[2]) = 1 + 1 = 2 \]
Compute dp[4]
Why: Value at position 4 is 6. Earlier smaller values are at positions 1 (value 5, dp=1) and 2 (value 2, dp=1); position 3's value 8 is excluded since it is not smaller than 6.
\[ \text{dp}[4] = 1 + \max(\text{dp}[1], \text{dp}[2]) = 1 + 1 = 2 \]
Check these four values by hand against the array
Why: dp[3]=2 should reflect a genuine 2-element increasing run ending at 8, such as 5 then 8, or 2 then 8 - both are real runs, confirming the value.
| i | value | dp[i] |
|---|---|---|
| 1 | 5 | 1 |
| 2 | 2 | 1 |
| 3 | 8 | 2 |
| 4 | 6 | 2 |
Worked example
Compute dp[5]
Why: Value at position 5 is 3. The only earlier smaller value is at position 2 (value 2, dp=1); positions with values 5, 8, and 6 are all bigger than 3, so they don't count.
\[ \text{dp}[5] = 1 + \text{dp}[2] = 1 + 1 = 2 \]
Compute dp[6]
Why: Value at position 6 is 6. Earlier smaller values are at positions 1 (dp=1), 2 (dp=1), and 5 (dp=2); positions with values 8 and 6 are excluded (8 is bigger, and 6 is not strictly smaller than 6). The best of these is dp[5]=2.
\[ \text{dp}[6] = 1 + \max(\text{dp}[1], \text{dp}[2], \text{dp}[5]) = 1 + 2 = 3 \]
Compute dp[7]
Why: Value at position 7 is 9. Every earlier value (5, 2, 8, 6, 3, 6) is smaller than 9, so all six dp values are candidates. The largest of them is dp[6]=3.
\[ \text{dp}[7] = 1 + \max(\text{dp}[1..6]) = 1 + 3 = 4 \]
Compute dp[8]
Why: Value at position 8 is 7. Earlier smaller values are at positions 1, 2, 4, 5, and 6 (position 3's value 8 is excluded, and position 7 comes after). The best among those dp values is dp[6]=3.
\[ \text{dp}[8] = 1 + \max(\text{dp}[1], \text{dp}[2], \text{dp}[4], \text{dp}[5], \text{dp}[6]) = 1 + 3 = 4 \]
Verify the answer is the maximum entry in the table
Why: The largest dp value in the completed table is 4, reached at both position 7 and position 8, so the longest increasing subsequence has length 4.
| i | value | dp[i] |
|---|---|---|
| 1 | 5 | 1 |
| 2 | 2 | 1 |
| 3 | 8 | 2 |
| 4 | 6 | 2 |
| 5 | 3 | 2 |
| 6 | 6 | 3 |
| 7 | 9 | 4 |
| 8 | 7 | 4 |
Picture it
Animation
Shows: The table holds the answer; traceback holds the solution — a rendered Manim animation.
Rendered with Manim.
Takeaway: Most exam questions want the second, and it costs one extra pass.
Step zero
Discussion prompt
Worked example: LIS - read back the actual subsequence — before any calculation: what is the plan? Name the moves in order, in plain English, without doing the arithmetic.
Hint: It starts with: Start at position 7 (dp=4, value 9)
Answer:
Worked example
The dp array tells us the best length is 4, but not which numbers form it. Trace backward from a position that achieved the max.
Start at position 7 (dp=4, value 9)
Why: This is one of the positions achieving the maximum; any increasing subsequence achieving the max is a valid longest one.
Find which earlier dp value produced dp[7]
Why: dp[7] = 1 + dp[6], and dp[6]=3 is the value that was used, so the chain continues at position 6 (value 6).
Continue back from position 6
Why: dp[6] = 1 + dp[5], and dp[5]=2 is the value that was used, so the chain continues at position 5 (value 3).
Continue back from position 5
Why: dp[5] = 1 + dp[2], and dp[2]=1 is the value that was used, so the chain continues at position 2 (value 2), which is a base case with no predecessor.
Verify the reconstructed subsequence is strictly increasing
Why: Reading the chain forward: position 2 (value 2), position 5 (value 3), position 6 (value 6), position 7 (value 9). Each value is bigger than the last one, confirming a genuine length-4 increasing subsequence.
\[ 2 < 3 < 6 < 9\ \checkmark \]
Reverse engineer
Discussion prompt
Work backwards. The example finished here:
Verify the reconstructed subsequence is strictly increasing
What was it asked to do, and what must it have been given? Reconstruct the problem from its answer.
Hint: Every quantity in the result had to enter somewhere. Account for each one.
Answer:
The dp array tells us the best length is 4, but not which numbers form it. Trace backward from a position that achieved the max.
Concept
The dp table only guarantees the correct length. Several different subsequences can share that same longest length, and the table doesn't prefer one over another.
In our array, position 8 (value 7) also has dp[8] = 4, giving a second valid answer: 2, 3, 6, 7. Both this and 2, 3, 6, 9 are correct longest increasing subsequences.
Anomaly
Predict first
A student writes this, and it looks reasonable:
A student sees "increasing subsequence" and only checks consecutive runs of the array, sliding a window and testing whether each window is increasing.
It is wrong. Say what breaks — and say it before you turn the page.
Correct: This checks contiguous stretches only - a fundamentally different, smaller search space than all subsequences.
Elements of an increasing subsequence can skip over any number of positions, as long as their values still increase left to right.
Why: This checks contiguous stretches only - a fundamentally different, smaller search space than all subsequences.
Trap
A student sees "increasing subsequence" and only checks consecutive runs of the array, sliding a window and testing whether each window is increasing.
Scan for the longest run of adjacent increasing numbers
Why: This checks contiguous stretches only - a fundamentally different, smaller search space than all subsequences.
Report length 2 for 5, 2, 8, 6, 3, 6, 9, 7
Why: The longest adjacent increasing run is just 2 numbers long (for example 2 then 8, or 3 then 6, or 6 then 9) - this badly undercounts the true answer.
Elements of an increasing subsequence can skip over any number of positions, as long as their values still increase left to right.
Allow skipped positions between chosen elements
Why: 2 (position 2), 3 (position 5), 6 (position 6), 9 (position 7) skips positions 1, 3, and 4 entirely, and is still perfectly valid.
Report the true length: 4
Why: Matching the dp table's answer confirms that dropping the 'adjacent' requirement is exactly what the problem calls for.
Break the constraint
Discussion prompt
The rule this trap just fixed:
Matching the dp table's answer confirms that dropping the 'adjacent' requirement is exactly what the problem calls for.
Now break it on purpose. Build a case that violates it and follow the consequences until something visibly fails. Where does the failure first show up — and would you have noticed it if you had not been looking?
Hint: The dangerous rules are the ones whose violation still produces an answer. If yours fails loudly, try to find one that fails quietly.
Answer:
This checks contiguous stretches only - a fundamentally different, smaller search space than all subsequences.
Concept
Filling dp[i] requires scanning every earlier position j to look for smaller values. Doing this for every position i gives a nested loop over pairs of positions.
\[ O(n^{2}) \]
A cleverer method using binary search can bring this down to n log n, but the quadratic version is the one to master first, since it directly matches the recurrence.
Picture it
Animation
Shows: Cost of the sequence DPs — a rendered Manim animation.
Rendered with Manim.
Takeaway: Quadratic in the input lengths, and for long DNA that matters.
Pattern
1. Define the state in plain words first
Why: Say exactly what smaller question each table entry answers, before writing any formula - for example, 'the longest increasing subsequence ending exactly at position i.'
2. Write the match/mismatch or extend/don't-extend recurrence
Why: Identify the handful of ways a bigger answer can be built from smaller ones, and write one formula per case.
3. Nail down the base row, column, or case
Why: Decide the smallest inputs directly - don't assume they're zero. Some bases really are zero (LCS); others count something real (Edit Distance).
4. Know how you will trace back the actual answer
Why: A number alone (a length or a cost) isn't the full answer. Decide, before you code, which direction you'll walk through the table to recover the real sequence.
Real world
Discussion prompt
Outside this lesson: where does Dynamic Programming III: LIS, LCS & Edit Distance actually turn up? Name one concrete situation — a job, a piece of software someone ships, a decision somebody has to make — and say which part of The DP setup checklist is doing the work in it.
Hint: Vague is the failure mode here. "Engineering" is not a situation; "deciding whether this build is fast enough to ship" is.
Answer:
Setting up three classic sequence DPs from scratch: Longest Increasing Subsequence, Longest Common Subsequence, and Edit Distance. Emphasizes defining the state in words, choosing the correct base row/column, writing the match/mismatch recurrence, and tracing the grid to read back the actual answer.
Elimination
Eliminate the wrong options
What is dp[5], the length of the longest increasing subsequence ending at position 5 (value 5)?
3 of these 4 are wrong. Strike them one at a time, and say what rules each one out before you strike the next. The survivor is the answer.
Survives elimination: C
Why: dp[5] extends from the best earlier position whose value is smaller than 5. All four earlier positions (3, 1, 4, 1) qualify, with dp values 1, 1, 2, 1. The largest is dp[3]=2 (the chain 3 then 4, or 1 then 4), so dp[5] = 1 + 2 = 3, giving a chain like 1, 4, 5 or 3, 4, 5.
Check
Consider the array below.
\[ a:\quad 3,\ 1,\ 4,\ 1,\ 5 \qquad (\text{positions } 1 \text{ to } 5) \]
Check your understanding
What is dp[5], the length of the longest increasing subsequence ending at position 5 (value 5)?
Answer: C
Why: dp[5] extends from the best earlier position whose value is smaller than 5. All four earlier positions (3, 1, 4, 1) qualify, with dp values 1, 1, 2, 1. The largest is dp[3]=2 (the chain 3 then 4, or 1 then 4), so dp[5] = 1 + 2 = 3, giving a chain like 1, 4, 5 or 3, 4, 5.
Prediction
Predict first
Which statement correctly completes the LIS recurrence: dp[i] equals...
Answer it in your own words, now, with nothing to choose from. The options are on the next slide — and picking the right one off a list is an easier skill than producing it.
Correct: 1 plus the maximum dp[j] among all j less than i with value at j less than value at i (or 1 if no such j exists).
Why: The recurrence must scan every earlier position (not just the immediate neighbor), require a strictly smaller value to preserve the increasing property, and add 1 to account for including position i itself. Any looser or tighter version breaks on some input.
Check
Recall the setup: dp[i] is the length of the longest increasing subsequence ending exactly at position i.
Check your understanding
Which statement correctly completes the LIS recurrence: dp[i] equals...
Answer: A
Why: The recurrence must scan every earlier position (not just the immediate neighbor), require a strictly smaller value to preserve the increasing property, and add 1 to account for including position i itself. Any looser or tighter version breaks on some input.
Concept
You're given two sequences (think of them as strings). You want the length - and eventually the actual letters - of the longest subsequence that appears in BOTH of them, in order, in each.
\[ X = x_1 x_2 \cdots x_m, \qquad Y = y_1 y_2 \cdots y_n \]
Symbols: X and Y are the two strings; m and n are their lengths. We compare a prefix of X against a prefix of Y and grow both prefixes.
Intuition
Instead of comparing the whole strings at once, ask a smaller question: what is the longest common subsequence of just the first i characters of X and the first j characters of Y?
Grow i and j one step at a time. Once you know the answer for every smaller pair of prefixes, the answer for the full strings falls out of the same rule applied one more time.
Concept
Let i range over prefix lengths of X (from 0 to m) and j range over prefix lengths of Y (from 0 to n). Define dp[i][j] as the length of the longest common subsequence between the first i characters of X and the first j characters of Y.
\[ \text{dp}[i][j] = \text{LCS length of } x_1 \cdots x_i \text{ and } y_1 \cdots y_j \]
Notice this state takes two indices, one per string - unlike LIS, which only needed one, because here we're tracking progress through two sequences at once.
Picture it
Animation
Shows: LCS is what diff computes — a rendered Manim animation.
Rendered with Manim.
Takeaway: Which is why a good diff highlights so little.
Concept
Because i and j both range starting from 0 (the empty prefix) up through the full length, the table needs one extra row and one extra column beyond the string lengths.
\[ \text{table size: } (m+1) \times (n+1) \]
Forgetting that extra row and column is a common setup mistake - row 0 and column 0 represent comparing against an empty string, and they must exist before you can fill in row 1 or column 1.
Concept
Row 0 represents using zero characters of X; column 0 represents using zero characters of Y. The longest common subsequence with an empty string is always empty.
\[ \text{dp}[0][j] = 0 \ \text{ for all } j, \qquad \text{dp}[i][0] = 0 \ \text{ for all } i \]
So the entire top row and entire left column of the table are zero - there is nothing to match yet.
Concept
If the i-th character of X equals the j-th character of Y, that shared character can be the new last character of a common subsequence - extend the best answer from one row up and one column left.
\[ x_i = y_j \ \Rightarrow \ \text{dp}[i][j] = \text{dp}[i-1][j-1] + 1 \]
This is the only case where the answer grows: matching characters are the entire reason LCS length increases.
Picture it
Animation
Shows: The two cases of LCS — a rendered Manim animation.
Rendered with Manim.
Takeaway: Match walks diagonally; mismatch drops one character from one string.
Intuition
What move should we make next?
Comparing prefixes of two strings, and the characters at the ends differ:
\[ dp[i][j] \;\text{ with }\; X[i] \ne Y[j] \]
They cannot both be in the common subsequence, since the subsequence would have to end with both at once.
If at least one of them has to go, how many cases is that, and what do you do with the results?
_Look at your toolkit. Say a move number out loud before this slide advances._ A wrong guess is useful. A silent guess is not.
Concept
If the two characters differ, this pair of positions itself contributes nothing new. The best you can do is fall back to whichever neighbor - dropping the current character of X, or dropping the current character of Y - already has the better answer.
\[ x_i \neq y_j \ \Rightarrow \ \text{dp}[i][j] = \max\big(\text{dp}[i-1][j],\ \text{dp}[i][j-1]\big) \]
No character is added in this case - you're just copying forward the better of two smaller answers you already computed.
Concept
Every entry dp[i][j] depends only on entries with a smaller row (i-1) or, within the same row, an earlier column (j-1). So if you fill row 0 first, then row 1 left to right, then row 2 left to right, and so on, every value you need is already sitting in the table when you need it.
This fill order isn't just convenient bookkeeping - it's forced by the recurrence itself: you can never compute a cell before the cells it depends on exist.
Concept
One table, one comparison, two branches. Every cell looks at three neighbours and never anywhere else, which is what makes the row-by-row sweep safe.
LCS(X, Y)
for i = 0 to X.length
dp[i][0] = 0
for j = 0 to Y.length
dp[0][j] = 0
for i = 1 to X.length
for j = 1 to Y.length
if X[i] == Y[j]
dp[i][j] = dp[i-1][j-1] + 1
else
dp[i][j] = max(dp[i-1][j], dp[i][j-1])
return dp[X.length][Y.length]On a match you move diagonally and add one. On a mismatch you do not add anything — you just take the better of dropping one character from either string. The diagonal is the only direction that ever grows the answer.
Notation
Every line of LCS says one thing. Read the line, then read what it does — not the other way round.
Annotate
Invariant
Each cell holds the length of the longest common subsequence of the first i characters of X and the first j characters of Y. Say that before reading any number.
Step through it
For each cell, say whether it came from the diagonal or from a neighbour, and why.
Picture it
Animation
Shows: LCS executing: the current line of pseudocode is highlighted while the data it touches changes.
Rendered with Manim.
Takeaway: Match moves diagonally and adds one; mismatch takes the better neighbour and adds nothing.
Estimation
Predict first
Find the longest common subsequence of X = "AC" (m=2 characters) and Y = "GAC" (n=3 characters). Compare prefixes of each, growing the table one row at a time.
Commit before you compute: what does Worked example: LCS - build the table for X="AC", Y="GAC" come out to? A rough magnitude and the right form is enough — the point is to have something concrete to be wrong about.
Correct: Check the bottom-right entry against the rules used
Why: A prediction you can defend turns the computation into a check rather than a leap of faith — and an answer that contradicts it is caught on the spot. dp[2][3] = 2 was built purely from the match/mismatch rules above with no shortcuts, so the table is internally consistent before we read off the answer.
Worked example
Find the longest common subsequence of X = "AC" (m=2 characters) and Y = "GAC" (n=3 characters). Compare prefixes of each, growing the table one row at a time.
Row 0 and column 0 are all zero
Why: Comparing anything against an empty prefix always gives a common subsequence of length 0.
\[ \text{dp}[0][j] = 0, \quad \text{dp}[i][0] = 0 \]
Fill row 1 (X prefix = "A")
Why: Compare A against each of G, A, C: mismatch with G gives max(0,0)=0; match with A gives dp[0][1]+1 = 0+1 = 1; mismatch with C gives max(dp[0][3], dp[1][2]) = max(0,1) = 1.
\[ \text{dp}[1][1]=0,\ \text{dp}[1][2]=1,\ \text{dp}[1][3]=1 \]
Fill row 2 (X prefix = "AC")
Why: Compare C against G, A, C: mismatch with G gives max(dp[1][1], dp[2][0]) = max(0,0)=0; mismatch with A gives max(dp[1][2], dp[2][1]) = max(1,0)=1; match with C gives dp[1][2]+1 = 1+1 = 2.
\[ \text{dp}[2][1]=0,\ \text{dp}[2][2]=1,\ \text{dp}[2][3]=2 \]
The completed table:
| "" (j=0) | G (j=1) | A (j=2) | C (j=3) | |
|---|---|---|---|---|
| "" (i=0) | 0 | 0 | 0 | 0 |
| A (i=1) | 0 | 0 | 1 | 1 |
| AC (i=2) | 0 | 0 | 1 | 2 |
Check the bottom-right entry against the rules used
Why: dp[2][3] = 2 was built purely from the match/mismatch rules above with no shortcuts, so the table is internally consistent before we read off the answer.
Picture it
Animation
Shows: The LCS table — a rendered Manim animation.
Rendered with Manim.
Takeaway: A match takes the diagonal plus one. A mismatch takes the better neighbour.
Step zero
Discussion prompt
Worked example: LCS - read the answer and trace back the subsequence — before any calculation: what is the plan? Name the moves in order, in plain English, without doing the arithmetic.
Hint: It starts with: Read the final answer
Answer:
Worked example
The bottom-right entry, dp[2][3], is the length of the longest common subsequence of the full strings.
Read the final answer
Why: dp[2][3] = 2, so X="AC" and Y="GAC" share a common subsequence of length 2 - but the table alone doesn't say which two characters.
\[ \text{dp}[2][3] = 2 \]
Trace back from the bottom-right corner
Why: X's 2nd character is C and Y's 3rd character is C - they match, so this character belongs to the subsequence. A match always steps diagonally, to dp[1][2].
Continue the trace from dp[1][2]
Why: X's 1st character is A and Y's 2nd character is A - they match, so this character also belongs to the subsequence. Step diagonally again, to dp[0][1], which is a base-case zero - the trace stops.
Verify the recovered subsequence
Why: Reading the matches in the order they were found (C, then A) and reversing gives "AC" - which is exactly X itself, and it does appear in order inside Y="GAC" (at positions 2 and 3).
\[ \text{LCS}(\text{"AC"},\ \text{"GAC"}) = \text{"AC"}\ \checkmark \]
Picture it
Animation
Shows: Each line of the worked example "LCS - read the answer and trace back the subsequence", appearing one at a time.
The same working the example does, in the order a tutor would write it.
Takeaway: Reading the matches in the order they were found (C, then A) and reversing gives "AC" - which is exactly X itself, and it does appear in order inside Y="GAC" (at positions 2 and 3).
Anomaly
Predict first
A student writes this, and it looks reasonable:
A student wants the LCS length for the full strings X="AC", Y="GAC", and reads the value out of dp[0][0] - the corner where the table started - instead of the corner where it finished.
It is wrong. Say what breaks — and say it before you turn the page.
Correct: dp[0][0] = 0 by the base case - it represents comparing two empty prefixes, not the full strings.
The answer to the full problem always sits in the cell that has compared the FULL length of both strings - the bottom-right corner, dp[m][n].
Why: dp[0][0] = 0 by the base case - it represents comparing two empty prefixes, not the full strings.
Trap
A student wants the LCS length for the full strings X="AC", Y="GAC", and reads the value out of dp[0][0] - the corner where the table started - instead of the corner where it finished.
Read dp[0][0]
Why: dp[0][0] = 0 by the base case - it represents comparing two empty prefixes, not the full strings.
Conclude the strings share no common subsequence
Why: This wrongly reports length 0, when the correct table clearly builds up to a shared subsequence of length 2 by the time both full strings have been compared.
The answer to the full problem always sits in the cell that has compared the FULL length of both strings - the bottom-right corner, dp[m][n].
Read dp[2][3] (m=2, n=3)
Why: This cell has compared all of X against all of Y, so it holds the true answer: length 2.
Trace back starting from that same bottom-right cell
Why: Reconstruction must also start where the answer lives and walk backward toward dp[0][0], never the other way around - walking forward from the start does not follow the choices the recurrence actually made.
Concept
The table has one entry per pair of prefixes, and each entry takes constant work to fill once its neighbors are known.
\[ O(mn) \text{ time}, \quad O(mn) \text{ space} \]
Both string lengths matter here, unlike LIS - doubling the length of either string roughly doubles the total work.
Picture it
Animation
Shows: The length is easy; the sequence needs the table — a rendered Manim animation.
Rendered with Manim.
Takeaway: Space optimisation costs you the traceback. Decide which you need first.
Hypothesis
Predict first
Worked example: LCS - a second trace with more mismatches is about to be worked. State your hypothesis first: which rule or definition decides this one, and what is the first move it forces? Then watch whether the example agrees with you.
Correct: Base row and column are zero
Why: Same rule as before: comparing against an empty prefix always gives length 0.
A hypothesis you wrote down is falsifiable; a vague sense of how it will go is not. If the example opens somewhere else, that gap is the thing worth chasing.
Worked example
Find the longest common subsequence of X = "ABCB" (m=4) and Y = "BDCB" (n=4).
Base row and column are zero
Why: Same rule as before: comparing against an empty prefix always gives length 0.
\[ \text{dp}[0][j] = 0, \quad \text{dp}[i][0] = 0 \]
Fill row 1 (X prefix = "A")
Why: A matches none of B, D, C, B, so every entry in row 1 falls back to its neighbor - which is always 0 here.
\[ \text{dp}[1][1..4] = 0,\ 0,\ 0,\ 0 \]
Fill row 2 (X prefix = "AB")
Why: The new character is B. It matches Y's 1st character (B) and Y's 4th character (B). dp[2][1] = dp[1][0]+1 = 1. The mismatches with D and C carry the best neighbor forward, giving dp[2][2]=dp[2][3]=1, and the second match gives dp[2][4] = dp[1][3]+1 = 1.
\[ \text{dp}[2][1..4] = 1,\ 1,\ 1,\ 1 \]
Fill row 3 (X prefix = "ABC")
Why: The new character is C, matching Y's 3rd character: dp[3][3] = dp[2][2]+1 = 2. The mismatches before and after take the best neighbor: dp[3][1]=1, dp[3][2]=1, and dp[3][4]=max(dp[2][4], dp[3][3])=max(1,2)=2.
\[ \text{dp}[3][1..4] = 1,\ 1,\ 2,\ 2 \]
Check the completed table with row 4 filled in
Why: Row 4 (X prefix "ABCB") is filled the same way: the two matches with B give dp[4][1]=dp[3][0]+1=1 and dp[4][4]=dp[3][3]+1=3, while the two mismatches falling back to their best neighbor give dp[4][2]=1 and dp[4][3]=2 - all consistent with the earlier rows.
| "" (j=0) | B (j=1) | D (j=2) | C (j=3) | B (j=4) | |
|---|---|---|---|---|---|
| "" (i=0) | 0 | 0 | 0 | 0 | 0 |
| A (i=1) | 0 | 0 | 0 | 0 | 0 |
| AB (i=2) | 0 | 1 | 1 | 1 | 1 |
| ABC (i=3) | 0 | 1 | 1 | 2 | 2 |
| ABCB (i=4) | 0 | 1 | 1 | 2 | 3 |
Picture it
Animation
Shows: Each line of the worked example "LCS - a second trace with more mismatches", appearing one at a time.
The same working the example does, in the order a tutor would write it.
Takeaway: Row 4 (X prefix "ABCB") is filled the same way: the two matches with B give dp[4][1]=dp[3][0]+1=1 and dp[4][4]=dp[3][3]+1=3, while the two mismatches falling back to their best neighbor give dp[4][2]=1 and dp[4][3]=2 - all consistent with the earlier rows.
Ranking
Put in order
Put the moves of Worked example: LCS - trace back the second example into the order they have to happen.
Why: These are the moves of the worked example in the order it makes them, and each one is set up by the one before it. X's 4th character is B and Y's 4th character is B - they match, so B belongs to the subsequence.
Worked example
The bottom-right entry, dp[4][4] = 3, says X="ABCB" and Y="BDCB" share a common subsequence of length 3.
Trace back from dp[4][4]
Why: X's 4th character is B and Y's 4th character is B - they match, so B belongs to the subsequence. Step diagonally to dp[3][3].
Continue from dp[3][3]
Why: X's 3rd character is C and Y's 3rd character is C - they match, so C belongs to the subsequence too. Step diagonally to dp[2][2].
Continue from dp[2][2]
Why: X's 2nd character is B and Y's 2nd character is D - they mismatch. dp[2][2]=1 came from the better neighbor, dp[2][1]=1 (not dp[1][2]=0), so step left to dp[2][1] - no character is added here.
Finish at dp[2][1]
Why: X's 2nd character is B and Y's 1st character is B - they match, so a second B belongs to the subsequence. Step diagonally to dp[1][0], a base case - the trace stops.
Verify the recovered subsequence
Why: Reading the matches in the order found (B, C, B) and reversing gives "BCB": in X="ABCB" the letters B, C, B appear at positions 2, 3, 4 in order; in Y="BDCB" they appear at positions 1, 3, 4 in order.
\[ \text{LCS}(\text{"ABCB"},\ \text{"BDCB"}) = \text{"BCB"}\ \checkmark \]
Reverse engineer
Discussion prompt
Work backwards. The example finished here:
Verify the recovered subsequence
What was it asked to do, and what must it have been given? Reconstruct the problem from its answer.
Hint: Every quantity in the result had to enter somewhere. Account for each one.
Answer:
The bottom-right entry, dp[4][4] = 3, says X="ABCB" and Y="BDCB" share a common subsequence of length 3.
Check
Suppose X has 4 characters and Y has 6 characters.
Check your understanding
What value does dp[0][5] hold in the LCS table, and why?
Answer: A
Why: Row 0 represents using zero characters of X. No matter how many characters of Y you compare against an empty string, there is nothing to match, so every entry in row 0, including dp[0][5], is 0 by the base case.
Commit first
Predict first
Which value should dp[i][j] take in this mismatch case?
Commit to an answer, then rate it — certain, fairly sure, or guessing — and write the rating down before you turn the page.
Correct: The maximum of dp[i-1][j] and dp[i][j-1].
Why: A mismatch adds no new shared character, so dp[i][j] can't grow past what's already known. The best available answer is whichever neighbor - dropping the current character of X, or dropping the current character of Y - already has the larger LCS length, hence the maximum of dp[i-1][j] and dp[i][j-1].
The rating matters as much as the answer: confident-and-wrong is the combination that survives revision, because nothing about it feels like it needs revisiting.
Check
Suppose X's i-th character and Y's j-th character are different, so this is a mismatch cell.
\[ x_i \neq y_j \]
Check your understanding
Which value should dp[i][j] take in this mismatch case?
Answer: A
Why: A mismatch adds no new shared character, so dp[i][j] can't grow past what's already known. The best available answer is whichever neighbor - dropping the current character of X, or dropping the current character of Y - already has the larger LCS length, hence the maximum of dp[i-1][j] and dp[i][j-1].
Concept
You're given two strings. You want the minimum number of single-character edits - insertions, deletions, or replacements - needed to turn the first string into the second.
\[ X = x_1 \cdots x_m \ \longrightarrow \ Y = y_1 \cdots y_n \]
This is also called Levenshtein distance. Unlike LCS, which only asks what characters two strings share, edit distance asks how much work it takes to transform one into the other.
Picture it
Animation
Shows: Edit distance, filling in — a rendered Manim animation.
Rendered with Manim.
Takeaway: Three moves compete in every cell: insert, delete, substitute.
Intuition
Line the two strings up character by character. At each position you have three moves available: delete a character from X, insert a character to match Y, or replace a character of X with the one Y needs. Matching characters need no move at all.
The DP's job is to find the cheapest combination of these moves that turns all of X into all of Y.
Intuition
If the two strings were the same length, you might be tempted to just line them up position by position and count how many positions differ. But real strings can differ in length, and a single insertion or deletion shifts every character after it.
That's why edit distance needs a full grid of subproblems, not a single left-to-right scan: it has to consider every way the two strings could be lined up, including ones with extra characters inserted or removed.
Concept
Let dp[i][j] be the minimum number of edits needed to turn the first i characters of X into the first j characters of Y.
\[ \text{dp}[i][j] = \text{min edits to turn } x_1 \cdots x_i \text{ into } y_1 \cdots y_j \]
This looks just like the LCS state - two indices, one per string - but the number it stores means something different: a cost to minimize, not a length to maximize.
Picture it
Animation
Shows: Where edit distance actually gets used — a rendered Manim animation.
Rendered with Manim.
Takeaway: Changing the cost of each operation changes the application, not the algorithm.
Intuition
Edit Distance's table looks identical to LCS's - same two indices, same grid, same fill order. But where LCS's recurrence rewards agreement (matches add to a growing length), Edit Distance's recurrence penalizes disagreement (mismatches add to a growing cost).
Keeping these two goals straight - maximizing shared characters versus minimizing edit cost - is what keeps the two recurrences, and their very different base cases, from blurring together.
Picture it
Animation
Shows: Tracing back through the table — a rendered Manim animation.
Rendered with Manim.
Takeaway: Diagonal moves are the matches — following them backwards spells the answer.
Concept
Turning i characters of X into an empty string takes exactly i deletions - delete everything. Turning an empty string into j characters of Y takes exactly j insertions - insert everything.
\[ \text{dp}[i][0] = i, \qquad \text{dp}[0][j] = j \]
This is the opposite of LCS's base case. LCS's base row and column are zero because there's nothing to MATCH against an empty string. Edit distance's base row and column count real work, because turning into or out of an empty string still costs one edit per character.
Anomaly
Predict first
A student writes this, and it looks reasonable:
A student copies the LCS base case out of habit and initializes dp[i][0] = 0 and dp[0][j] = 0 for the Edit Distance table.
It is wrong. Say what breaks — and say it before you turn the page.
Correct: Copying the LCS pattern treats an empty target as free - as if turning "CAT" into nothing costs nothing.
Base cells must count the real work of pure insertions or pure deletions - they are not free.
Why: Copying the LCS pattern treats an empty target as free - as if turning "CAT" into nothing costs nothing.
Trap
A student copies the LCS base case out of habit and initializes dp[i][0] = 0 and dp[0][j] = 0 for the Edit Distance table.
Set dp[3][0] = 0 for X="CAT", Y=""
Why: Copying the LCS pattern treats an empty target as free - as if turning "CAT" into nothing costs nothing.
Get a nonsensical answer
Why: Turning "CAT" into the empty string clearly takes 3 deletions, not 0. Every downstream cell that depends on this wrong base inherits the error.
Base cells must count the real work of pure insertions or pure deletions - they are not free.
Set dp[3][0] = 3 for X="CAT", Y=""
Why: Turning "CAT" into the empty string requires deleting all 3 characters, one edit each.
Set dp[0][j] = j the same way
Why: Turning the empty string into a target of length j requires inserting all j characters. The base row and column are literally i and j, counted from the corner outward.
Two truths and a lie
Sort into buckets
Some of these hold up and some are the exact mistakes this lesson is built to prevent. Sort them.
Explain it to yourself
Discussion prompt
In Process: guessing the edit-distance base row this move is made:
It gives a wrong answer immediately
Why is that legal? Name the rule or definition it rests on before you read on.
Hint: If you can only say "because that is what you do", the rule is the thing to go and find.
Answer:
Turning the empty string into a string of length 3 takes three insertions, not zero. A zero there claims the work is free, and every entry that depends on it inherits the error.
Intuition
Watch me not know the answer. This is what the first two minutes actually look like.
Setting up the edit distance table, where the entry is the fewest edits to turn the first i characters of X into the first j characters of Y.
Try zeros along the top row and left column, the way LCS does it
Why: LCS starts at zero when either string is empty, and this table looks identical in shape. Copy it.
It gives a wrong answer immediately
Why: Turning the empty string into a string of length 3 takes three insertions, not zero. A zero there claims the work is free, and every entry that depends on it inherits the error.
Dead end. Not a mistake — a move that was worth trying and did not pay off. This happens in most proofs.
Back up. Read the sentence, not the shape
Why: The entry counts edits. With one string empty, the only way across is to insert every character of the other — so the base row is 0, 1, 2, 3 and the base column likewise.
Same table shape, opposite base cases, because LCS maximizes something you can have none of, while edit distance minimizes something you must pay for. The shape of the table never tells you the base case. The sentence does.
The expert does not see the whole path in advance. The expert tries something, reads the result, and adjusts. That is the skill.
Picture it
Animation
Shows: The base row and column — a rendered Manim animation.
Rendered with Manim.
Takeaway: Two similar tables with different borders — get them wrong and everything shifts.
Concept
If the i-th character of X already equals the j-th character of Y, that position needs no edit at all - just carry forward the cost of matching everything before it.
\[ x_i = y_j \ \Rightarrow \ \text{dp}[i][j] = \text{dp}[i-1][j-1] \]
No plus-one here - a real match is free, unlike LCS, where a match adds one to the running length.
Picture it
Animation
Shows: An alignment, read off the table — a rendered Manim animation.
Rendered with Manim.
Takeaway: The path through the table IS the edit script.
Concept
If the characters differ, you must pay for one edit, and you get to choose the cheapest of three options: delete the current character of X, insert the character Y needs, or replace one character for the other.
\[ x_i \neq y_j \ \Rightarrow \ \text{dp}[i][j] = 1 + \min\big(\text{dp}[i-1][j],\ \text{dp}[i][j-1],\ \text{dp}[i-1][j-1]\big) \]
Each of the three neighbors corresponds to one operation: the cell above is a deletion, the cell to the left is an insertion, and the diagonal cell is a replacement.
Step zero
Discussion prompt
Worked example: Edit Distance - set up X="SEA", Y="EAT" — before any calculation: what is the plan? Name the moves in order, in plain English, without doing the arithmetic.
Hint: It starts with: Fill the base row and column
Answer:
Worked example
Find the edit distance between X = "SEA" (m=3) and Y = "EAT" (n=3).
Fill the base row and column
Why: Turning i characters of X into nothing costs i deletions; turning nothing into j characters of Y costs j insertions.
\[ \text{dp}[0][0..3] = 0,\ 1,\ 2,\ 3 \qquad \text{dp}[0..3][0] = 0,\ 1,\ 2,\ 3 \]
Fill row 1 (X prefix = "S")
Why: S mismatches every character of Y (E, A, T), so each cell costs 1 plus the minimum of its three neighbors: dp[1][1]=1+min(0,1,1)=1; dp[1][2]=1+min(1,2,1)=2; dp[1][3]=1+min(2,3,2)=3.
\[ \text{dp}[1][1..3] = 1,\ 2,\ 3 \]
Fill row 2 (X prefix = "SE")
Why: E matches Y's 1st character: dp[2][1]=dp[1][0]=1 (no added cost). The rest mismatch: dp[2][2]=1+min(dp[1][1],dp[1][2],dp[2][1])=1+min(1,2,1)=2; dp[2][3]=1+min(dp[1][2],dp[1][3],dp[2][2])=1+min(2,3,2)=3.
\[ \text{dp}[2][1..3] = 1,\ 2,\ 3 \]
Check row 3 (X prefix = "SEA") and the completed table
Why: A matches Y's 2nd character: dp[3][2]=dp[2][1]=1 (no added cost). The mismatches cost one plus the best neighbor: dp[3][1]=1+min(dp[2][0],dp[2][1],dp[3][0])=1+min(2,1,3)=2; dp[3][3]=1+min(dp[2][2],dp[2][3],dp[3][2])=1+min(2,3,1)=2.
| "" (j=0) | E (j=1) | A (j=2) | T (j=3) | |
|---|---|---|---|---|
| "" (i=0) | 0 | 1 | 2 | 3 |
| S (i=1) | 1 | 1 | 2 | 3 |
| E (i=2) | 2 | 1 | 2 | 3 |
| A (i=3) | 3 | 2 | 1 | 2 |
Picture it
Animation
Shows: Weighting the operations — a rendered Manim animation.
Rendered with Manim.
Takeaway: The algorithm does not change; only the numbers in the min do.
Estimation
Predict first
The bottom-right entry, dp[3][3], is the minimum number of edits to turn the full string X into the full string Y.
Commit before you compute: what does Worked example: Edit Distance - read the answer and trace… come out to? A rough magnitude and the right form is enough — the point is to have something concrete to be wrong about.
Correct: Verify the total cost matches
Why: A prediction you can defend turns the computation into a check rather than a leap of faith — and an answer that contradicts it is caught on the spot. Reading the moves in order (delete S, keep E, keep A, insert T) turns "SEA" into "EA" and then into "EAT" using exactly 1 deletion plus 1 insertion, or 2 edits total, matching dp[3][3].
Worked example
The bottom-right entry, dp[3][3], is the minimum number of edits to turn the full string X into the full string Y.
Read the final answer
Why: dp[3][3] = 2, so "SEA" can become "EAT" in as few as 2 edits.
\[ \text{dp}[3][3] = 2 \]
Trace back from dp[3][3]
Why: X's 3rd character is A and Y's 3rd character is T - they mismatch. Its value, 2, equals 1 + dp[3][2] (the left neighbor, value 1) - the smallest of the three candidates - so this step is an INSERT of the character T.
Continue from dp[3][2]
Why: X's 3rd character is A and Y's 2nd character is A - they match, so no edit here. Step diagonally to dp[2][1] for free.
Continue from dp[2][1]
Why: X's 2nd character is E and Y's 1st character is E - they match, so no edit here either. Step diagonally to dp[1][0] for free.
Finish at dp[1][0]
Why: This is a base-case cell (j=0) with value 1: turning the single character S into nothing costs one DELETION. The trace stops here.
Verify the total cost matches
Why: Reading the moves in order (delete S, keep E, keep A, insert T) turns "SEA" into "EA" and then into "EAT" using exactly 1 deletion plus 1 insertion, or 2 edits total, matching dp[3][3].
\[ \text{"SEA"} \xrightarrow{\text{delete S}} \text{"EA"} \xrightarrow{\text{insert T}} \text{"EAT"}\ \checkmark \]
Picture it
Animation
Shows: Each line of the worked example "Edit Distance - read the answer and trace back the edits", appearing one at a time.
The same working the example does, in the order a tutor would write it.
Takeaway: Reading the moves in order (delete S, keep E, keep A, insert T) turns "SEA" into "EA" and then into "EAT" using exactly 1 deletion plus 1 insertion, or 2 edits total, matching dp[3][3].
Concept
Each direction you step during traceback corresponds to exactly one kind of edit.
\[ \text{diagonal} = \text{match (free) or replace (cost 1)}, \quad \text{up} = \text{delete}, \quad \text{left} = \text{insert} \]
A diagonal step is free when the characters already match, and costs one edit when they don't (a replacement). Knowing this turns the bare numbers in the table into an actual list of edits.
Concept
Just like LCS, every one of the (m+1) by (n+1) cells takes constant work once its neighbors are known.
\[ O(mn) \text{ time}, \quad O(mn) \text{ space} \]
The only difference from LCS is what each cell computes - a minimum cost instead of a maximum length - the shape of the computation is identical.
Picture it
Animation
Shows: Three edits, three neighbours — a rendered Manim animation.
Rendered with Manim.
Takeaway: The geometry of the table is the algorithm.
Step zero
Discussion prompt
Worked example: Edit Distance - a second example with two replacements — before any calculation: what is the plan? Name the moves in order, in plain English, without doing the arithmetic.
Hint: It starts with: Fill the base row and column
Answer:
Worked example
Find the edit distance between X = "AB" (m=2) and Y = "BA" (n=2).
Fill the base row and column
Why: dp[i][0]=i counts pure deletions; dp[0][j]=j counts pure insertions.
\[ \text{dp}[0][0..2] = 0,\ 1,\ 2 \qquad \text{dp}[0..2][0] = 0,\ 1,\ 2 \]
Fill row 1 (X prefix = "A")
Why: A mismatches Y's 1st character (B): dp[1][1]=1+min(dp[0][0],dp[0][1],dp[1][0])=1+min(0,1,1)=1. A matches Y's 2nd character (A): dp[1][2]=dp[0][1]=1 (no added cost).
\[ \text{dp}[1][1..2] = 1,\ 1 \]
Check row 2 (X prefix = "AB") and the completed table
Why: B matches Y's 1st character (B): dp[2][1]=dp[1][0]=1 (no added cost). B mismatches Y's 2nd character (A): dp[2][2]=1+min(dp[1][1],dp[1][2],dp[2][1])=1+min(1,1,1)=2.
| "" (j=0) | B (j=1) | A (j=2) | |
|---|---|---|---|
| "" (i=0) | 0 | 1 | 2 |
| A (i=1) | 1 | 1 | 1 |
| B (i=2) | 2 | 1 | 2 |
Picture it
Animation
Shows: Each line of the worked example "Edit Distance - a second example with two replacements", appearing one at a time.
The same working the example does, in the order a tutor would write it.
Takeaway: B matches Y's 1st character (B): dp[2][1]=dp[1][0]=1 (no added cost). B mismatches Y's 2nd character (A): dp[2][2]=1+min(dp[1][1],dp[1][2],dp[2][1])=1+min(1,1,1)=2.
Ranking
Put in order
Put the moves of Worked example: Edit Distance - trace back the second example into the order they have to happen.
Why: These are the moves of the worked example in the order it makes them, and each one is set up by the one before it. X's 2nd character is B and Y's 2nd character is A - they mismatch.
Worked example
The bottom-right entry, dp[2][2] = 2, says "AB" needs at least 2 edits to become "BA".
Trace back from dp[2][2]
Why: X's 2nd character is B and Y's 2nd character is A - they mismatch. Its value, 2, equals 1 + dp[1][1] (the diagonal neighbor, value 1), so this step is a REPLACE of B with A.
Continue from dp[1][1]
Why: X's 1st character is A and Y's 1st character is B - they mismatch. Its value, 1, equals 1 + dp[0][0] (the diagonal neighbor, value 0), so this step is also a REPLACE, of A with B.
Verify the total cost matches
Why: Replacing A with B and B with A turns "AB" directly into "BA" using exactly 2 replacements, matching dp[2][2] = 2. No cheaper combination of insert, delete, or replace exists here, since both characters must change.
\[ \text{"AB"} \xrightarrow{\text{replace both}} \text{"BA"}\ \checkmark \]
Notation
Annotate
From Worked example: Edit Distance - trace back the second… — read this one piece at a time. What is each part doing?
On: \( \text{"AB"} \xrightarrow{\text{replace both}} \text{"BA"}\ \checkmark \)
Prediction
Predict first
What is dp[5][0], and what does it represent?
Answer it in your own words, now, with nothing to choose from. The options are on the next slide — and picking the right one off a list is an easier skill than producing it.
Correct: 5, because turning 5 characters into nothing requires 5 deletions.
Why: dp[i][0] is defined as the cost to turn i characters of X into the empty string. Every character must be deleted separately, one edit each, so dp[5][0] = 5 - five real deletions, not a free base case.
Check
Suppose X has 5 characters and Y is the empty string.
Check your understanding
What is dp[5][0], and what does it represent?
Answer: A
Why: dp[i][0] is defined as the cost to turn i characters of X into the empty string. Every character must be deleted separately, one edit each, so dp[5][0] = 5 - five real deletions, not a free base case.
Intuition
What move should we make next?
LIS, LCS and edit distance are all done. Close the deck.
For each: the subproblem sentence, the last decision, and the base case. Nine answers.
This is the same drill as last lesson, and it will be the same drill next lesson. The recurrences are worth nothing to you if the procedure that produces them is not automatic.
_Look at your toolkit. Say a move number out loud before this slide advances._ A wrong guess is useful. A silent guess is not.
Explain it
Discussion prompt
Explain Decision point: three sequence DPs, from memory to a student a year behind you. No notation, no jargon they have not met — and it still has to be true.
Hint: If your explanation needs a symbol they have never seen, you are describing the notation rather than the idea.
Answer:
LIS, LCS and edit distance are all done. Close the deck.
Picture it
Animation
Shows: Two sequences means two indices — a rendered Manim animation.
Rendered with Manim.
Takeaway: One index per sequence — the table's shape follows from the state.
Concept
All three DPs share the same shape: define a state in words, write a recurrence with a matching case and a non-matching case, nail down a base, and know how to trace back the real answer. Only the details change.
| LIS | LCS | Edit Distance | |
|---|---|---|---|
| State | dp[i]: longest run ending at i | dp[i][j]: LCS length of two prefixes | dp[i][j]: min edits between two prefixes |
| Base | dp[i]=1 (alone) | dp[0][j]=dp[i][0]=0 (empty match) | dp[i][0]=i, dp[0][j]=j (pure edits) |
| Match/extend case | extend from a smaller earlier value | +1 from the diagonal | 0 cost from the diagonal |
| Non-match case | skip that earlier position | max of the two neighbors | 1 + min of three neighbors |
Comparison
Comparison matrix
From The three setups, side by side: refill the LIS column from what you know. The rest of the table is as it appeared.
| LIS | LCS | Edit Distance | |
|---|---|---|---|
| State | dp[i]: longest run ending at i | dp[i][j]: LCS length of two prefixes | dp[i][j]: min edits between two prefixes |
| Base | dp[i]=1 (alone) | dp[0][j]=dp[i][0]=0 (empty match) | dp[i][0]=i, dp[0][j]=j (pure edits) |
| Match/extend case | extend from a smaller earlier value | +1 from the diagonal | 0 cost from the diagonal |
| Non-match case | skip that earlier position | max of the two neighbors | 1 + min of three neighbors |
Concept
One sequence, looking for the best run by some ordering property (like increasing values): that's an LIS-shaped problem, with one index into one sequence.
Two sequences, asking what they share in order: that's an LCS-shaped problem, with two indices, one per sequence, and a recurrence that rewards matches.
Two sequences, asking the cost to transform one into the other: that's an Edit-Distance-shaped problem, with two indices and a recurrence that penalizes mismatches instead of rewarding matches.
Analogy
Discussion prompt
Explain How to recognize which DP a problem calls for by analogy to something with no CS3000 Algorithms in it at all — a queue, a recipe, a map, a bank balance, whatever fits. Then say where your analogy breaks.
Hint: An analogy that never breaks is not an analogy, it is the same idea wearing a hat. Find the seam — that is the part that is actually new.
Answer:
One sequence, looking for the best run by some ordering property (like increasing values): that's an LIS-shaped problem, with one index into one sequence.
Intuition
Brute force for any of these three problems means examining an exponential number of candidate subsequences or alignments. But every candidate's evaluation reuses the same small set of prefix comparisons over and over.
By storing each prefix comparison's answer exactly once - in a table with only a polynomial number of cells - we replace exponentially many repeated computations with one polynomial-sized table, filled once, left to right, top to bottom.
Picture it
Animation
Shows: Two rows are often enough — a rendered Manim animation.
Rendered with Manim.
Takeaway: You lose traceback in exchange, so keep the full table when you need the path.
Elimination
Eliminate the wrong options
Which of today's three DPs directly fits this problem, and why?
3 of these 4 are wrong. Strike them one at a time, and say what rules each one out before you strike the next. The survivor is the answer.
Survives elimination: A
Why: The problem asks for the longest subsequence common to two given sequences - exactly the Longest Common Subsequence setup: two indices, one per strand, and a recurrence that rewards shared bases. Its length is the direct answer.
Check
Consider this problem: given two DNA strands (sequences of the letters A, C, G, T), find the length of the longest sequence of bases that appears in both strands, in order, allowing gaps.
Check your understanding
Which of today's three DPs directly fits this problem, and why?
Answer: A
Why: The problem asks for the longest subsequence common to two given sequences - exactly the Longest Common Subsequence setup: two indices, one per strand, and a recurrence that rewards shared bases. Its length is the direct answer.
Concept
Moves added today: none.
That is a result, not a gap. Everything in this lesson was proved with moves you already owned.
Moves you reused today:
Fourth lesson in a row with no new move. If the skeleton still feels like something you are reading rather than something you own, that is the signal to drill it, not to learn something new.
Full toolkit so far: #1 through #13.
Next session opens with you naming every one of these from memory, before any new material.
Counterexample
Discussion prompt
Fourth lesson in a row with no new move. If the skeleton still feels like something you are reading rather than something you own, that is the signal to drill it, not to learn something new.
That is stated as though it always holds. Do one of two things: produce a case where it fails, or say precisely what rules such a case out. "It just does" is not on the menu.
Hint: Hunt at the extremes first — zero, one, negative, empty, equal. If every extreme survives, the reason they survive is the proof.
Answer:
Next session opens with you naming every one of these from memory, before any new material.
Connect it up
Draw it
One page, no notation unless you need it: draw how these connect — The DP setup checklist · Toolkit check-in: name them before you look · Recap: what makes a problem dynamic programming · What a subsequence is · Why this distinction matters for today's three DPs. Put an arrow wherever one of them is what makes another possible, and label the arrow with why.
Recap
You now have the setup recipe for three sequence DPs that show up constantly in this course and beyond.
| Step | What to do |
|---|---|
| 1. State | Say what each table entry means, in words, before writing a formula |
| 2. Recurrence | Write the matching case and the non-matching case separately |
| 3. Base | Work out the smallest cases by hand - never assume they're zero |
| 4. Traceback | Walk from the answer cell backward, one real move at a time |
Want this taught 1-on-1? Alexander tutors CS3000 Algorithms — $55/session, free consultation.