This deck explains what a greedy algorithm is and traces two of them: interval scheduling by earliest finish time, and fractional knapsack by highest value-to-weight ratio. It then gives the two standard ways to prove a greedy algorithm optimal, greedy-stays-ahead and the exchange argument. It targets the beginner's biggest gap, which is getting a proof started, along with the misconceptions that greedy always works, that any scheduling criterion is as good as another, that a few working examples count as a proof, and that an exchange can be made carelessly.
Subject: CS3000 Algorithms · 127 slides · symbolic lesson
Open the interactive version of this deck · Homework for this lesson
Objectives
Greedy correctness proofs are one of the core skills tested in this course. This lesson gives you the moves for starting one, not just the vocabulary. By the end you can:
Warm-up
Discussion prompt
Before we open Greedy Algorithms & Exchange Arguments: without looking back, what was the main idea of Dynamic Programming III: LIS, LCS & Edit Distance, and what could you do by the end of it that you could not do before?
Hint: One sentence for the idea, one for the skill. If the second one is blank, that is the part to revisit.
Answer:
Setting up three classic sequence DPs from scratch: Longest Increasing Subsequence, Longest Common Subsequence, and Edit Distance. Emphasizes defining the state in words, choosing the correct base row/column, writing the match/mismatch recurrence, and tracing the grid to read back the actual answer.
Concept
Before any new material: cover the screen.
You have named 13 reusable moves so far. Say as many as you can out loud, by number, from memory.
Do not advance until you have actually tried. Getting four of eight is information; skipping the exercise is not.
Here they are. Score yourself.
Today adds one move to this list. Everything else you will need is already above.
The question that starts every proof from here on is not how do I begin. It is which of these applies here?
Counterexample
Discussion prompt
You have named 13 reusable moves so far. Say as many as you can out loud, by number, from memory.
That is stated as though it always holds. Do one of two things: produce a case where it fails, or say precisely what rules such a case out. "It just does" is not on the menu.
Hint: Hunt at the extremes first — zero, one, negative, empty, equal. If every extreme survives, the reason they survive is the proof.
Answer:
Do not advance until you have actually tried. Getting four of eight is information; skipping the exercise is not.
Concept
A greedy algorithm builds a solution one piece at a time. At each step it makes whichever choice looks best right now, using only the information available at that moment.
greedy algorithm — An algorithm that, at every step, makes the locally best available choice and moves on. It never revisits or undoes an earlier choice once made.
Analogy
Discussion prompt
Explain What a greedy algorithm is by analogy to something with no CS3000 Algorithms in it at all — a queue, a recipe, a map, a bank balance, whatever fits. Then say where your analogy breaks.
Hint: An analogy that never breaks is not an analogy, it is the same idea wearing a hat. Find the seam — that is the part that is actually new.
Answer:
A greedy algorithm builds a solution one piece at a time. At each step it makes whichever choice looks best right now, using only the information available at that moment.
Picture it
Animation
Shows: Greedy versus dynamic programming — a rendered Manim animation.
Rendered with Manim.
Takeaway: Greedy is faster and needs a proof. DP is slower and needs none.
Intuition
Picture climbing a staircase in the dark, where at every step you feel around and take whichever next step feels sturdiest, then never think about it again.
That is the entire personality of a greedy algorithm: fast, simple, and utterly committed to each choice the instant it is made.
Explain it
Discussion prompt
Explain Climbing without looking back to a student a year behind you. No notation, no jargon they have not met — and it still has to be true.
Hint: If your explanation needs a symbol they have never seen, you are describing the notation rather than the idea.
Answer:
Picture climbing a staircase in the dark, where at every step you feel around and take whichever next step feels sturdiest, then never think about it again.
Concept
The defining feature is commitment: once a greedy algorithm picks an item, it does not later undo that pick to make room for something else, even if hindsight would suggest a different order.
This is what makes greedy algorithms fast — usually one pass after a sort — and also what makes them risky. Committing early is only safe if the locally best choice is guaranteed to fit into some globally best solution.
Picture it
Animation
Shows: When greedy is guaranteed to work — a rendered Manim animation.
Rendered with Manim.
Takeaway: A deep reason why Kruskal works, and why coin change does not.
Intuition
Imagine choosing a hiking path by always taking whichever fork climbs fastest right now. That greedy rule can strand you on a false summit, short of the true highest peak, because it never looks past the very next step.
Some greedy rules genuinely do reach the true peak every time. Others do not. The whole content of this lesson is learning to tell which is which, and how to prove it.
Socratic
Discussion prompt
Some greedy rules genuinely do reach the true peak every time. Others do not. The whole content of this lesson is learning to tell which is which, and how to prove it.
Suppose that were not true. What is the first thing in Greedy Algorithms & Exchange Arguments that would stop working?
Hint: Follow it one step downstream. The answer is whatever was quietly relying on it.
Concept
A greedy algorithm can be short, elegant, and completely wrong. Being simple to describe says nothing about whether it always reaches the best possible answer.
So the claim you must actually establish has a precise shape: for every valid input, the greedy algorithm's output is at least as good as any other valid solution. That is a claim about infinitely many inputs, not just the ones you happened to try.
Intuition
What feels wrong about this?
A greedy rule for interval scheduling, tried on six inputs, correct on all six.
\[ 6 \text{ inputs tested}, \quad 6 \text{ optimal answers} \]
_Plain English only. No notation, no algebra. Just say what bothers you._
The feeling: six is nothing. The inputs were chosen by the person who wrote the rule, and the bad case is exactly the one they would not think to try.
That feeling is the proof. It is not a substitute for the proof — it is the thing the proof writes down.
You have met this shape before: it is a for all inputs claim. Which means proving it needs an argument over all inputs, and disproving it needs exactly one constructed counterexample.
Picture it
Animation
Shows: The right problem, the wrong greedy rule — a rendered Manim animation.
Rendered with Manim.
Takeaway: Same greedy shape, different criterion, different answer.
Concept
Running the algorithm on three or four sample inputs and getting the right answer each time feels convincing, but it establishes nothing about the input you have not tried yet.
This is the single biggest jump beginners have to make: from 'I checked some cases' to 'I have an argument that covers every case at once.' The rest of this lesson is two reusable templates for building that argument.
Concept
When a greedy rule can be proved correct, it is usually also the fastest correct algorithm available: sort the input once by the right key, then sweep through it a single time making one decision per item.
That is the payoff for the extra proof work: instead of trying every combination, you get a fast, simple procedure, backed by an argument that it never needs to look back.
Concept
You are given a set of requests to use a single shared resource, like one classroom or one machine. Each request wants the resource for a specific stretch of time, and two requests can share the resource only if their time stretches do not overlap.
request — One candidate booking, described by a start time and a finish time. It occupies the resource for that whole stretch and cannot be split.
\[ \text{request } i \;\to\; (s_i, f_i), \qquad s_i = \text{start time}, \quad f_i = \text{finish time} \]
compatible — Two requests are compatible if one finishes at or before the other starts, so their time stretches do not overlap. A compatible set is a set of requests that are pairwise compatible.
\[ i, j \text{ compatible} \iff f_i \le s_j \ \text{or} \ f_j \le s_i \]
The goal: choose the largest possible compatible set of requests, so the resource is used by as many non-overlapping requests as it can hold.
Definition probe
Sort into buckets
Every line below is part of the definition of greedy algorithm or of request — one or the other, never both. Put each where it belongs.
Picture it
Animation
Shows: Scheduling to minimise lateness — a rendered Manim animation.
Rendered with Manim.
Takeaway: The exchange argument again, on a different ordering.
Picture it
Figure (svg): Four request bars A, B, C, D drawn along a shared time axis. A and B overlap in time, B and C overlap, but A and C do not.
Discussion prompt
Read the picture before the words. What is this showing, and what is the one thing it is built to make obvious? Commit to an answer, then read on.
Hint: Name the parts, then say what changes between them — and if nothing changes, say what is being held still.
Answer:
Drawing each request as a horizontal bar over a shared time axis turns compatibility into a picture: two requests are compatible exactly when their bars do not touch along the time axis.
Intuition
Figure (svg): Four request bars A, B, C, D drawn along a shared time axis. A and B overlap in time, B and C overlap, but A and C do not.
Drawing each request as a horizontal bar over a shared time axis turns compatibility into a picture: two requests are compatible exactly when their bars do not touch along the time axis.
Concept
Sort all requests by finish time. Walk through them in that order. Take a request whenever it is compatible with everything already taken; otherwise skip it and move on.
\[ \text{sort by } f_1 \le f_2 \le \cdots \le f_n; \quad \text{take } i \text{ iff } s_i \ge f_{\text{last taken}} \]
That is the entire algorithm: one sort, one pass, one comparison per request. No request, once accepted or skipped, is ever revisited.
Concept
Sort once, then scan once. The greedy rule lives entirely on line 2, and everything after it is a single compatibility test repeated.
INTERVAL-SCHEDULE(jobs)
sort jobs by finish time, ascending
S = empty set
last = minus infinity
for each job in sorted order
if job.start >= last
add job to S
last = job.finish
return SLine 4 holds the finish time of the last job accepted, and it is the only memory the algorithm has. It never looks back at S, never reconsiders, and never removes anything — that is what greedy means.
Notation
Every line of INTERVAL-SCHEDULE says one thing. Read the line, then read what it does — not the other way round.
Annotate
Invariant
After each accepted job, the set S is compatible and finishes no later than any other set of the same size could. That second half is the greedy-stays-ahead claim, and it is what the proof has to establish.
Step through it
At each job, decide accept-or-reject yourself before stepping.
Picture it
Animation
Shows: INTERVAL-SCHEDULE executing: the current line of pseudocode is highlighted while the data it touches changes.
Rendered with Manim.
Takeaway: Sort by finish time and take every job that still fits — one scan, no reconsidering, and provably optimal.
Intuition
A request that finishes early hands the rest of the timeline back to you as soon as possible. It is the choice that closes off the least future opportunity, no matter how long or short it is.
Picture it
Figure (svg): Timeline of six requests P, Q, R, S, T, U by start and finish time; greedy picks P, S, and U, shown in green, skipping Q, R, and T, shown in gray, because each overlaps the most recently picked request.
Discussion prompt
Read the picture before the words. What is this showing, and what is the one thing it is built to make obvious? Commit to an answer, then read on.
Hint: Name the parts, then say what changes between them — and if nothing changes, say what is being held still.
Answer:
Six requests, given as (start, finish): P (1, 4), Q (3, 5), R (0, 6), S (5, 7), T (3, 9), U (8, 11).
Worked example
Six requests, given as (start, finish): P (1, 4), Q (3, 5), R (0, 6), S (5, 7), T (3, 9), U (8, 11).
Figure (svg): Timeline of six requests P, Q, R, S, T, U by start and finish time; greedy picks P, S, and U, shown in green, skipping Q, R, and T, shown in gray, because each overlaps the most recently picked request.
Sort by finish time
Why: The greedy rule needs requests in finish-time order before it can sweep through them once.
\[ P(1,4),\ Q(3,5),\ R(0,6),\ S(5,7),\ T(3,9),\ U(8,11) \]
Take P: it is the first request, always compatible
Why: With nothing accepted yet, there is no conflict to check. Last-taken finish time becomes 4.
\[ \text{last finish} = 4 \]
Skip Q and R
Why: Q starts at 3 and R starts at 0; both start before the last accepted finish of 4, so both overlap P.
\[ s_Q = 3 < 4, \quad s_R = 0 < 4 \ \Rightarrow \text{skip both} \]
Take S
Why: S starts at 5, which is at or after the last accepted finish of 4, so S is compatible. Last-taken finish time updates to 7.
\[ s_S = 5 \ge 4 \ \Rightarrow \text{take}; \quad \text{last finish} = 7 \]
Skip T, then take U
Why: T starts at 3, before 7, so it overlaps S and is skipped. U starts at 8, at or after 7, so U is compatible and taken.
\[ s_T = 3 < 7 \ (\text{skip}); \quad s_U = 8 \ge 7 \ (\text{take}) \]
Verify no larger compatible set exists
Why: Greedy returns {P, S, U}, size 3. Every other request overlaps at least one member of this set (Q and R overlap P; T overlaps both S and P), so no fourth request can be added anywhere, confirming 3 is the best possible.
\[ \{P, S, U\}, \ \text{size } 3 \ \checkmark \]
Notation
Annotate
From Trace earliest-finish-time greedy — read this one piece at a time. What is each part doing?
On: \( P(1,4),\ Q(3,5),\ R(0,6),\ S(5,7),\ T(3,9),\ U(8,11) \)
Picture it
Animation
Shows: Greedy by earliest finish time — a rendered Manim animation.
Rendered with Manim.
Takeaway: Finishing early leaves the most room for everything after.
Anomaly
Predict first
A student writes this, and it looks reasonable:
A tempting rule: always take the shortest remaining request first, on the idea that short requests block the least time.
It is wrong. Say what breaks — and say it before you turn the page.
Correct: B runs from 9 to 11, length 2, shorter than A's length 10 and C's length 10, so shortest-first takes it first.
Earliest finish time is what actually matters, not length.
Why: B runs from 9 to 11, length 2, shorter than A's length 10 and C's length 10, so shortest-first takes it first.
Trap
A tempting rule: always take the shortest remaining request first, on the idea that short requests block the least time.
\[ \text{requests: } A(0,10),\ B(9,11),\ C(10,20) \]
Shortest-first picks B, the request of length 2
Why: B runs from 9 to 11, length 2, shorter than A's length 10 and C's length 10, so shortest-first takes it first.
\[ \text{length}(B) = 2 < \text{length}(A) = \text{length}(C) = 10 \]
Taking B blocks both A and C
Why: A overlaps B from 9 to 10, and C overlaps B from 10 to 11, so once B is accepted neither A nor C can be added. Shortest-first ends with just {B}, size 1.
\[ \{B\}, \ \text{size } 1 \]
Earliest finish time is what actually matters, not length.
\[ \text{same requests: } A(0,10),\ B(9,11),\ C(10,20) \]
Sort by finish time: A finishes at 10, B at 11, C at 20
Why: Earliest-finish-time greedy takes A first, since it finishes soonest, regardless of its length.
\[ f_A = 10 < f_B = 11 < f_C = 20 \]
Take A, then take C
Why: A is taken first. B starts at 9, before A's finish of 10, so B is skipped. C starts at 10, at or after A's finish, so C is taken. Result {A, C}, size 2 — strictly better than shortest-first's size 1.
\[ \{A, C\}, \ \text{size } 2 \ \checkmark \]
Anomaly
Predict first
A student writes this, and it looks reasonable:
Another tempting rule: always take whichever remaining request starts earliest, since it seems to get the resource moving right away.
It is wrong. Say what breaks — and say it before you turn the page.
Correct: A starts before every other request, so earliest-start-first takes it immediately.
Earliest finish time again gives the better answer.
Why: A starts before every other request, so earliest-start-first takes it immediately.
Trap
Another tempting rule: always take whichever remaining request starts earliest, since it seems to get the resource moving right away.
\[ \text{requests: } A(0,12),\ B(1,2),\ C(3,4),\ D(5,6) \]
Earliest-start picks A, since it starts at 0
Why: A starts before every other request, so earliest-start-first takes it immediately.
\[ s_A = 0 < s_B = 1 < s_C = 3 < s_D = 5 \]
Taking A blocks everything else
Why: A runs from 0 to 12, so B, C, and D — all nested inside that span — are every one of them incompatible with A. Result: {A}, size 1.
\[ \{A\}, \ \text{size } 1 \]
Earliest finish time again gives the better answer.
\[ \text{same requests: } A(0,12),\ B(1,2),\ C(3,4),\ D(5,6) \]
Sort by finish time: B finishes at 2, C at 4, D at 6, A at 12
Why: Earliest-finish-time greedy takes B first, the request that frees the timeline soonest.
\[ f_B = 2 < f_C = 4 < f_D = 6 < f_A = 12 \]
Take B, C, and D in turn; skip A
Why: Each of B, C, D starts after the previous one's finish, so all three are compatible and accepted. A starts at 0, before B's finish of 2, so A is skipped. Result {B, C, D}, size 3 — strictly better than size 1.
\[ \{B, C, D\}, \ \text{size } 3 \ \checkmark \]
Translation
\( \text{same requests: } A(0,12),\ B(1,2),\ C(3,4),\ D(5,6) \)
Draw it
Translate both ways. First write the expression above as a sentence with no symbols in it at all. Then cover it, and write your sentence back as notation. If the two versions disagree, the disagreement is the thing to fix.
Concept
As greedy sweeps left to right, it only ever needs to remember one number: the finish time of the most recently accepted request.
last-taken finish time — The finish time of whichever accepted request finishes latest so far. A new request is compatible exactly when its start time is at or after this number.
\[ \text{take } i \iff s_i \ge f_{\text{last taken}} \]
Intuition
Because requests are sorted by finish time and greedy only compares each new start time to a single remembered number, it never needs to revisit an earlier decision. One sweep left to right is the whole algorithm.
Concept
Sorting the requests by finish time dominates the cost; the single left-to-right scan afterward touches each request only once.
\[ \Theta(n \log n) \ \text{for the sort}, \quad \Theta(n) \ \text{for the scan} \]
The scan itself is proportional to the number of requests, with no nested loops and no revisiting — exactly the speed a correctness proof is buying you.
Concept
If two requests finish at exactly the same time, any consistent way of breaking the tie works: the proof coming up does not depend on which one is processed first among equal finish times.
Concept
One way to prove a greedy algorithm optimal is to show it is never behind any other valid solution, one choice at a time. This proof style is called greedy stays ahead.
greedy stays ahead — A proof technique that compares greedy's choices to any other solution's choices, position by position, and shows greedy's i-th choice is always at least as good as the other solution's i-th choice.
Socratic
Discussion prompt
One way to prove a greedy algorithm optimal is to show it is never behind any other valid solution, one choice at a time. This proof style is called greedy stays ahead.
Suppose that were not true. What is the first thing in Greedy Algorithms & Exchange Arguments that would stop working?
Hint: Follow it one step downstream. The answer is whatever was quietly relying on it.
Answer:
greedy stays ahead: A proof technique that compares greedy's choices to any other solution's choices, position by position, and shows greedy's i-th choice is always at least as good as the other solution's i-th choice.
Picture it
Animation
Shows: Spotting a greedy problem — a rendered Manim animation.
Rendered with Manim.
Takeaway: If the choice might need undoing later, you want DP.
Intuition
Picture two racers, greedy and a rival, each making a sequence of moves. Greedy stays ahead means that at every single checkpoint, greedy has covered at least as much ground as the rival — never once trailing.
If greedy is never behind at any checkpoint, it cannot possibly finish behind at the end. That single fact, proved checkpoint by checkpoint, is the whole argument.
Concept
Before writing a single justified line, name every object the proof will talk about. This is the step beginners skip, and skipping it is why proofs feel impossible to start.
Let greedy's accepted requests, in the order accepted, be called its sequence. Let any other compatible set's requests, sorted by finish time, be called a competing sequence.
\[ g_1, g_2, \ldots, g_k = \text{greedy's sequence}; \qquad o_1, o_2, \ldots, o_m = \text{a competing sequence} \]
The claim to prove: greedy's i-th pick never finishes later than the competitor's i-th pick, for every position both sequences reach.
\[ \text{Claim: } f(g_i) \le f(o_i) \quad \text{for every } i \le \min(k, m) \]
Concept
Like ordinary induction, greedy stays ahead needs exactly two pieces: a base case establishing the claim at the first position, and an inductive step showing the claim at one position forces it at the next.
\[ \text{Base: } f(g_1) \le f(o_1). \qquad \text{Step: } f(g_i) \le f(o_i) \ \Rightarrow\ f(g_{i+1}) \le f(o_{i+1}) \]
Picture it
Animation
Shows: The staying-ahead argument — a rendered Manim animation.
Rendered with Manim.
Takeaway: A cousin of the exchange argument, often easier to write.
Intuition
What move should we make next?
The claim on the table:
\[ \text{earliest-finish-time greedy selects a maximum number of compatible intervals} \]
You cannot test your way to this. You cannot compute your way to it either.
It is a for-all claim about every input. Which move opens it, and what object does that move put on the table?
_Look at your toolkit. Say a move number out loud before this slide advances._ A wrong guess is useful. A silent guess is not.
Ranking
Put in order
Put the moves of Greedy stays ahead: earliest-finish-time is optimal into the order they have to happen.
Why: These are the moves of the worked example in the order it makes them, and each one is set up by the one before it. Greedy's first pick, g_1, is whichever request has the smallest finish time among ALL requests, since greedy processes requests in finish-time order and nothing can block the very first choice.
Worked example
Claim: for interval scheduling, earliest-finish-time greedy's output is at least as large as any compatible set. Let greedy's sequence be g_1 through g_k; let any other compatible set's requests, sorted by finish time, be o_1 through o_m.
\[ \text{Show: } f(g_i) \le f(o_i) \ \text{for all } i \le \min(k,m), \ \text{hence } k \ge m \]
Base case: compare the first pick of each
Why: Greedy's first pick, g_1, is whichever request has the smallest finish time among ALL requests, since greedy processes requests in finish-time order and nothing can block the very first choice.
\[ f(g_1) = \min_i f_i \ \Rightarrow\ f(g_1) \le f(o_1) \ \text{for any } o_1 \]
Inductive hypothesis: assume the claim at position i
Why: Assume f(g_i) is at most f(o_i). This is the one rung the inductive step is allowed to lean on.
\[ \text{Assume } f(g_i) \le f(o_i) \]
Inductive step: show the claim at position i+1
Why: Since o_{i+1} is compatible with o_i in the competing set, its start time is at or after f(o_i). By the hypothesis f(o_i) is at least f(g_i), so o_{i+1}'s start time is also at or after f(g_i) — meaning o_{i+1} was an available, compatible choice at the moment greedy picked g_{i+1}.
\[ s(o_{i+1}) \ge f(o_i) \ge f(g_i) \ \Rightarrow\ o_{i+1} \text{ was available to greedy} \]
Conclude the step
Why: Greedy always takes the earliest-finishing available request. Since o_{i+1} was available and greedy instead chose g_{i+1}, greedy's choice must finish no later than o_{i+1} does.
\[ f(g_{i+1}) \le f(o_{i+1}) \]
Verify the conclusion on the earlier trace
Why: In the six-request trace, greedy's sequence is P (finish 4), S (finish 7), U (finish 11). Any competing compatible sequence's first pick finishes at 4 or later (for example Q at 5, R at 6), matching the claim at position 1, and greedy's size of 3 was already shown to be unbeatable.
\[ f(g_1)=4 \le f(o_1) \ \text{for every alternative } o_1\ \checkmark \]
Picture it
Animation
Shows: Each line of the worked example "Greedy stays ahead: earliest-finish-time is optimal", appearing one at a time.
The same working the example does, in the order a tutor would write it.
Takeaway: In the six-request trace, greedy's sequence is P (finish 4), S (finish 7), U (finish 11). Any competing compatible sequence's first pick finishes at 4 or later (for example Q at 5, R at 6), matching the claim at position 1, and greedy's size of 3 was already shown to be unbeatable.
Explain it to yourself
Discussion prompt
In Trap: claiming optimality with no proof this move is made:
Stop after checking three examples
Why is that legal? Name the rule or definition it rests on before you read on.
Hint: If you can only say "because that is what you do", the rule is the thing to go and find.
Answer:
Three correct outputs say nothing about the input the grader will actually test. A single untested input where greedy fails would sink this claim, and nothing here rules that out.
Trap
A student runs earliest-finish-time greedy on three different sets of requests, gets the right-looking answer each time, and writes 'therefore the algorithm is optimal.'
Stop after checking three examples
Why: Three correct outputs say nothing about the input the grader will actually test. A single untested input where greedy fails would sink this claim, and nothing here rules that out.
\[ \text{3 examples checked} \ \not\Rightarrow\ \forall \text{ inputs, greedy is optimal} \]
Examples build confidence and can suggest the right claim, but the claim itself needs an argument that covers every possible input at once.
Use greedy stays ahead (or an exchange argument) instead
Why: Both techniques argue about an arbitrary competing solution, not a specific example, which is exactly what 'for every input' requires.
\[ \text{arbitrary competing sequence } o_1,\ldots,o_m \ \Rightarrow\ \text{argument covers all inputs} \]
Break the constraint
Discussion prompt
The rule this trap just fixed:
Both techniques argue about an arbitrary competing solution, not a specific example, which is exactly what 'for every input' requires.
Now break it on purpose. Build a case that violates it and follow the consequences until something visibly fails. Where does the failure first show up — and would you have noticed it if you had not been looking?
Hint: The dangerous rules are the ones whose violation still produces an answer. If yours fails loudly, try to find one that fails quietly.
Answer:
Three correct outputs say nothing about the input the grader will actually test. A single untested input where greedy fails would sink this claim, and nothing here rules that out.
Intuition
Greedy builds its solution one request at a time, in order. A claim about 'every position in that sequence' is exactly the shape induction is built for: prove it at the first position, then show each position forces the next.
Concept
A second way to prove greedy optimal: assume some optimal solution disagrees with greedy, find the first point where they differ, and swap in greedy's choice there. Show the swap keeps the solution valid and does not make it worse.
exchange argument — A proof technique that starts from an assumed optimal solution, locates the earliest place it departs from greedy's choices, replaces that one choice with greedy's, and shows the result is still feasible and at least as good — contradicting the assumption unless the two solutions already matched.
Picture it
Animation
Shows: The exchange argument — a rendered Manim animation.
Rendered with Manim.
Takeaway: You never construct the optimum — you show yours can replace it.
Concept
Every proof of this kind has the same five or six moves in the same order. The order is not something you rediscover each time.
It is on the right. It will stay on the right through the worked examples that follow.
Why this matters: the structure is now handled. You are not spending working memory on what comes next — you are spending all of it on the one hard step.
Steps 3 and 5 carry the proof. Step 5 is where a botched exchange hides — showing the swap is legal is not the same as showing it does not lose anything, and you need both.
Explain it to yourself
Discussion prompt
In The move that runs the rest of this course this move is made:
Assume a rival optimal solution and compare it to greedy
Why is that legal? Name the rule or definition it rests on before you read on.
Hint: If you can only say "because that is what you do", the rule is the thing to go and find.
Answer:
Do not try to prove greedy is optimal directly. Put a rival next to it and find the first index where they disagree — everything before that point is identical, so it can be ignored.
Concept
The move: #14 (Exchange argument).
Assume a rival optimal solution and compare it to greedy
Why: Do not try to prove greedy is optimal directly. Put a rival next to it and find the first index where they disagree — everything before that point is identical, so it can be ignored.
Swap greedy's choice into the rival at that one spot
Why: This is the whole idea. You are not rebuilding the rival; you are editing it in one place.
Show the edit is legal and loses nothing
Why: Legal means the result is still a valid solution. Loses nothing means its value did not go down. Both halves are required — skipping the second is the classic botched exchange.
Then the rival is one step closer to greedy, and still optimal
Why: Repeat, or induct on the number of differences. Greedy is therefore optimal too.
This move reappears in Huffman, in Kruskal, and in Prim. When you meet it there, the correct reaction is not this is new — it is #14 (Exchange argument), third time.
Intuition
Picture debating someone who claims to have a better plan than greedy's. Instead of arguing in the abstract, you take their plan, find the very first place it disagrees with greedy, and replace just that one piece with greedy's choice.
If you can always make that one swap without breaking anything or losing value, then their supposedly better plan can be nudged, step by step, into greedy's plan without ever getting worse — so greedy was just as good all along.
Concept
Let an optimal solution be called OPT — chosen, among all optimal solutions, to be the one that agrees with greedy's sequence for the longest matching prefix.
\[ \text{OPT} = o_1, o_2, \ldots, o_m, \quad \text{sorted by finish time} \]
Suppose OPT is not identical to greedy's sequence. Let position i+1 be the first place they differ, so the first i choices already match.
\[ o_1 = g_1, \ldots, o_i = g_i, \quad \text{but } o_{i+1} \ne g_{i+1} \]
Define the swap: build a new set OPT-prime identical to OPT except that o_{i+1} is replaced by g_{i+1}.
\[ \text{OPT}' = \text{OPT} \setminus \{o_{i+1}\} \cup \{g_{i+1}\} \]
Concept
Every exchange argument has to discharge exactly two obligations about the new solution: it must still be feasible, and it must not be worse than before.
\[ \text{(1) OPT}' \text{ is still a valid solution.} \qquad \text{(2) size or value of OPT}' \ge \text{OPT} \]
Feasibility usually follows from the greedy choice finishing no later, or costing no more, than what it replaced. The no-worse comparison usually follows from the same fact. Both must be checked explicitly — neither is free.
Intuition
What move should we make next?
Earliest-finish-time greedy has now been proved optimal once, by staying ahead.
We are about to prove the same fact again, a different way.
Before we start: what is the opening line of the exchange version, and how does it differ from the opening line of the stays-ahead version?
_Look at your toolkit. Say a move number out loud before this slide advances._ A wrong guess is useful. A silent guess is not.
Step zero
Discussion prompt
Exchange argument: earliest-finish-time is optimal — before any calculation: what is the plan? Name the moves in order, in plain English, without doing the arithmetic.
Hint: It starts with: Compare the two candidates at position i+1
Answer:
Worked example
Assume OPT is an optimal compatible set that disagrees with greedy at the first index i+1, so o_1 through o_i equal g_1 through g_i, but o_{i+1} is different from g_{i+1}.
\[ o_1=g_1,\ldots,o_i=g_i; \quad o_{i+1} \ne g_{i+1} \]
Compare the two candidates at position i+1
Why: Both o_{i+1} and g_{i+1} are compatible with the shared prefix g_1 through g_i. Greedy always picks the earliest-finishing compatible request available, so greedy's pick finishes no later than any other compatible candidate, including o_{i+1}.
\[ f(g_{i+1}) \le f(o_{i+1}) \]
Build OPT-prime by swapping in g_{i+1}
Why: Replace o_{i+1} with g_{i+1} in OPT, keeping every other request the same. This is a one-for-one swap, so OPT-prime has exactly the same number of requests as OPT.
\[ \text{OPT}' = (\text{OPT} \setminus \{o_{i+1}\}) \cup \{g_{i+1}\} \]
Check feasibility of the swap
Why: g_{i+1} is compatible with the shared prefix (greedy chose it to be). Every request after o_{i+1} in OPT starts at or after f(o_{i+1}), which is at or after f(g_{i+1}), so g_{i+1} is also compatible with everything that came after o_{i+1}. OPT-prime is a valid compatible set.
\[ f(g_{i+1}) \le f(o_{i+1}) \le s(\text{later requests in OPT}) \ \Rightarrow\ \text{feasible} \]
Conclude and repeat
Why: OPT-prime is feasible and the same size as OPT, so it is also optimal, and it now agrees with greedy on one more position than OPT did — contradicting that OPT was chosen to agree with greedy the longest. So no such first disagreement can exist: OPT must match greedy entirely, and greedy is optimal.
\[ |\text{OPT}'| = |\text{OPT}|, \ \text{agrees longer} \Rightarrow \text{contradiction} \Rightarrow \text{greedy optimal} \]
Verify the swap mechanics on a small case
Why: In the earlier trace, greedy's first pick P finishes at 4. Any optimal set whose first pick instead finished at 5 or later can have that pick swapped for P without breaking anything after it, since P frees the timeline no later. Check: replacing a hypothetical first pick with finish 5 by P (finish 4) only ever loosens the start-time requirement for what follows.
\[ 4 \le 5 \ \Rightarrow \ \text{swap only loosens later constraints} \ \checkmark \]
Picture it
Animation
Shows: Each line of the worked example "Exchange argument: earliest-finish-time is optimal", appearing one at a time.
The same working the example does, in the order a tutor would write it.
Takeaway: g_{i+1} is compatible with the shared prefix (greedy chose it to be). Every request after o_{i+1} in OPT starts at or after f(o_{i+1}), which is at or after f(g_{i+1}), so g_{i+1} is also compatible with everything that came after o_{i+1}. OPT-prime is a valid compatible set.
Anomaly
Predict first
A student writes this, and it looks reasonable:
A student tries an exchange argument but swaps in the wrong direction: replacing greedy's choice with the competitor's choice, instead of the other way around.
It is wrong. Say what breaks — and say it before you turn the page.
Correct: This replaces the request that finishes at 7 with one that finishes at 12.
The exchange only ever goes one direction: replace the competitor's choice with greedy's choice, never the reverse.
Why: This replaces the request that finishes at 7 with one that finishes at 12. Anything that was compatible with the finish-at-7 pick, starting anywhere from 7 up to 11, is now overlapped by the finish-at-12 request. The swap can make later requests infeasible.
Trap
A student tries an exchange argument but swaps in the wrong direction: replacing greedy's choice with the competitor's choice, instead of the other way around.
\[ \text{requests: } g_{i+1} = (5,7), \quad o_{i+1} = (5, 12) \]
Swap g_{i+1} out and o_{i+1} in
Why: This replaces the request that finishes at 7 with one that finishes at 12. Anything that was compatible with the finish-at-7 pick, starting anywhere from 7 up to 11, is now overlapped by the finish-at-12 request. The swap can make later requests infeasible.
\[ \text{a later request starting at } 9 \text{ was compatible with 7, but not with 12} \]
The exchange only ever goes one direction: replace the competitor's choice with greedy's choice, never the reverse.
\[ \text{swap } o_{i+1} \ \text{out}, \ g_{i+1} \ \text{in} \quad (\text{not the other way}) \]
Swap in the direction that only loosens constraints
Why: Greedy's choice finishes no later than the competitor's at that position, by greedy's own selection rule. Swapping it IN can only free up more room for whatever comes after, never less.
\[ f(g_{i+1}) \le f(o_{i+1}) \ \Rightarrow \ \text{later requests stay feasible} \]
Always name the direction before swapping
Why: Write down explicitly which element leaves and which enters, and check the finish-time (or ratio) inequality actually points the way that keeps things feasible, before claiming the swap is safe.
Concept
Greedy stays ahead and the exchange argument are not competing truths — they are two different routes to the same conclusion, and either one is a fully valid proof by itself.
Greedy stays ahead compares whole sequences position by position. The exchange argument focuses on a single assumed optimal solution and repairs it one swap at a time until it becomes greedy's.
Intuition
Think of greedy stays ahead as watching two racers cross checkpoints together, and the exchange argument as editing a rival's already-finished race report until it reads exactly like greedy's — both prove greedy was never beaten.
Step zero
Discussion prompt
Numeric warm-up: a single swap by hand — before any calculation: what is the plan? Name the moves in order, in plain English, without doing the arithmetic.
Hint: It starts with: Locate the first disagreement
Answer:
Worked example
Requests: X (1, 3), Y (2, 4), Z (4, 6). A proposed optimal set is {Y, Z}. Greedy's earliest-finish-time choice at the first position is X, since X finishes at 3, earlier than Y's finish of 4.
\[ \{Y, Z\}: \ f_Y = 4, \ s_Z = 4 \ (\text{feasible, size } 2) \]
Locate the first disagreement
Why: Greedy's first pick is X (finish 3). The proposed set's first pick, sorted by finish time, is Y (finish 4). They disagree at position 1.
\[ g_1 = X \ (f=3), \quad o_1 = Y \ (f=4) \]
Swap Y out, X in
Why: X finishes at 3, no later than Y's finish of 4, so X is compatible with everything Y was compatible with — in particular, with Z, which starts at 4.
\[ \{X, Z\}: \ f_X = 3 \le s_Z = 4 \ (\text{feasible}) \]
Verify the swap kept the same size and stayed feasible
Why: The new set {X, Z} has 2 requests, same as {Y, Z}, and X starts at 1, ends at 3, while Z starts at 4, ends at 6 — no overlap. The swap produced a feasible set of equal size, exactly as the general argument predicts.
\[ |\{X,Z\}| = |\{Y,Z\}| = 2, \ \text{both feasible} \ \checkmark \]
Concept
You have a set of items, each with a value and a weight, and a bag with a fixed weight capacity. Unlike a suitcase, you are allowed to take any fraction of an item, not only the whole thing.
item — One good with a fixed value and weight. Its value-to-weight ratio is its value divided by its weight.
\[ \text{item } i \to (v_i, w_i), \quad \text{ratio}_i = \frac{v_i}{w_i} \]
The goal: choose an amount of each item, between none and all of it, so total weight does not exceed the bag's capacity, and total value is as large as possible.
\[ 0 \le x_i \le 1, \quad \sum_i x_i w_i \le W, \quad \text{maximize } \sum_i x_i v_i \]
Picture it
Animation
Shows: Where greedy provably wins — a rendered Manim animation.
Rendered with Manim.
Takeaway: The exchange needs to be possible, and fractions are what allow it.
Picture it
Figure (svg): A capacity bar filled from left to right: the item with the highest value-to-weight ratio goes in first, then the next highest, until the bag's capacity runs out, leaving only a partial fraction of the last item taken.
Discussion prompt
Read the picture before the words. What is this showing, and what is the one thing it is built to make obvious? Commit to an answer, then read on.
Hint: Name the parts, then say what changes between them — and if nothing changes, say what is being held still.
Answer:
Fill the bag the way you would fill a shopping cart on a strict budget: buy whichever item gives the most value per pound first, keep buying it until it runs out or the bag is full, then move to the next best value per pound.
Intuition
Figure (svg): A capacity bar filled from left to right: the item with the highest value-to-weight ratio goes in first, then the next highest, until the bag's capacity runs out, leaving only a partial fraction of the last item taken.
Fill the bag the way you would fill a shopping cart on a strict budget: buy whichever item gives the most value per pound first, keep buying it until it runs out or the bag is full, then move to the next best value per pound.
Concept
Sort items by value-to-weight ratio, from highest to lowest. Take as much of the highest-ratio item as the remaining capacity allows — all of it if it fits, otherwise only the fraction that fits — then move to the next item.
\[ \text{ratio}_1 \ge \text{ratio}_2 \ge \cdots \ge \text{ratio}_n \]
Picture it
Animation
Shows: Greedy without a proof is a guess — a rendered Manim animation.
Rendered with Manim.
Takeaway: The proof is not ceremony; it is the only way to know.
Intuition
A very valuable item that is also very heavy might be a poor use of capacity, and a very light item with tiny value wastes little capacity but also gains little. Only the ratio of value to weight actually measures value earned per unit of capacity spent.
Ranking
Put in order
Put the moves of Trace highest-ratio-first on fractional knapsack into the order they have to happen.
Why: These are the moves of the worked example in the order it makes them, and each one is set up by the one before it. Item 1's ratio of 6 is the highest, then Item 2 at 5, then Item 3 at 4.
Worked example
Capacity is 50. Items: Item 1 (weight 10, value 60), Item 2 (weight 20, value 100), Item 3 (weight 30, value 120).
| item | weight | value | ratio |
|---|---|---|---|
| Item 1 | 10 | 60 | 6 |
| Item 2 | 20 | 100 | 5 |
| Item 3 | 30 | 120 | 4 |
Sort by ratio, highest first
Why: Item 1's ratio of 6 is the highest, then Item 2 at 5, then Item 3 at 4.
\[ \text{Item 1} \ (6) \ \to\ \text{Item 2} \ (5) \ \to\ \text{Item 3} \ (4) \]
Take all of Item 1
Why: Item 1 weighs 10, well under the capacity of 50, so take the whole thing. Remaining capacity drops to 40.
\[ \text{take } 10, \ \text{value} = 60, \quad \text{remaining capacity} = 40 \]
Take all of Item 2
Why: Item 2 weighs 20, still under the remaining capacity of 40, so take the whole thing. Remaining capacity drops to 20.
\[ \text{take } 20, \ \text{value} = 100, \quad \text{remaining capacity} = 20 \]
Take a fraction of Item 3
Why: Item 3 weighs 30, more than the remaining capacity of 20, so take only the fraction that fits: 20 out of 30, which is two thirds of the item.
\[ \text{fraction} = \frac{20}{30} = \frac{2}{3}, \quad \text{value} = \frac{2}{3}\times 120 = 80 \]
Verify the total against the capacity
Why: Total weight used is 10 plus 20 plus 20, exactly 50, matching the capacity with none wasted. Total value is 60 plus 100 plus 80, equal to 240.
\[ 10+20+20 = 50 = W; \quad 60+100+80 = 240 \ \checkmark \]
Picture it
Animation
Shows: Each line of the worked example "Trace highest-ratio-first on fractional knapsack", appearing one at a time.
The same working the example does, in the order a tutor would write it.
Takeaway: Total weight used is 10 plus 20 plus 20, exactly 50, matching the capacity with none wasted. Total value is 60 plus 100 plus 80, equal to 240.
Step zero
Discussion prompt
Exchange argument: highest-ratio-first is optimal — before any calculation: what is the plan? Name the moves in order, in plain English, without doing the arithmetic.
Hint: It starts with: Suppose OPT differs from greedy at the first item i
Answer:
Worked example
Items are sorted so ratio_1 is at least ratio_2, at least ratio_3, and so on. Let OPT be an optimal way of filling the bag, chosen among all optimal solutions to agree with greedy the longest, item by item in ratio order.
\[ \text{ratio}_1 \ge \text{ratio}_2 \ge \cdots; \quad \text{OPT takes amount } y_i \text{ of item } i \]
Suppose OPT differs from greedy at the first item i
Why: Greedy fills item i to the maximum the remaining capacity allows. If OPT took strictly less of item i than greedy did, that leftover capacity in OPT must be spent on some later item j, with a strictly lower or equal ratio, since items before i already match greedy exactly.
\[ y_i < x_i \ (\text{greedy's amount}), \quad \text{extra capacity spent on item } j > i, \ \text{ratio}_j \le \text{ratio}_i \]
Swap a small amount from item j into item i
Why: Move a weight amount delta from item j to item i, where delta is the smaller of (greedy's amount of item i minus OPT's amount) and (OPT's amount of item j). Total weight used is unchanged, so the bag's capacity constraint still holds.
\[ \delta = \min(x_i - y_i, \ y_j); \quad \text{total weight unchanged} \Rightarrow \text{feasible} \]
Compare the value before and after the swap
Why: Moving weight delta from item j to item i changes total value by delta times the ratio of item i minus delta times the ratio of item j. Since ratio_i is at least ratio_j, this change is zero or positive — the swap never loses value.
\[ \Delta \text{value} = \delta(\text{ratio}_i - \text{ratio}_j) \ge 0 \]
Conclude and repeat
Why: The new solution is feasible and at least as valuable as OPT, so it is also optimal, and it agrees with greedy on strictly more of the ratio-ordered items than OPT did — contradicting how OPT was chosen. So no such disagreement survives, and greedy's allocation matches some optimal solution exactly.
\[ \text{value}(\text{OPT}') \ge \text{value}(\text{OPT}), \ \text{agrees longer} \ \Rightarrow \ \text{greedy optimal} \]
Verify the direction of the inequality on the earlier example
Why: In the trace above, Item 1's ratio of 6 is at least Item 2's ratio of 5, which is at least Item 3's ratio of 4, so every swap toward the higher-ratio item (never away from it) can only raise or hold value steady, matching the direction the proof requires.
\[ 6 \ge 5 \ge 4 \ \Rightarrow \ \text{swaps toward higher ratio never lose value} \ \checkmark \]
Picture it
Animation
Shows: Each line of the worked example "Exchange argument: highest-ratio-first is optimal", appearing one at a time.
The same working the example does, in the order a tutor would write it.
Takeaway: In the trace above, Item 1's ratio of 6 is at least Item 2's ratio of 5, which is at least Item 3's ratio of 4, so every swap toward the higher-ratio item (never away from it) can only raise or hold value steady, matching the direction the proof requires.
Anomaly
Predict first
A student writes this, and it looks reasonable:
A student sorts items by value alone, taking the biggest-value item first, regardless of its weight.
It is wrong. Say what breaks — and say it before you turn the page.
Correct: Item A is taken first for its higher raw value.
Sort by value-to-weight ratio, not value alone.
Why: Item A is taken first for its higher raw value. It uses 40 of the 50 capacity, leaving only 10 remaining.
Trap
A student sorts items by value alone, taking the biggest-value item first, regardless of its weight.
\[ \text{capacity } 50; \quad \text{Item A: weight } 40, \text{value } 100; \quad \text{Item B: weight } 10, \text{value } 90 \]
Value-first takes Item A completely, since 100 is greater than 90
Why: Item A is taken first for its higher raw value. It uses 40 of the 50 capacity, leaving only 10 remaining.
\[ \text{take Item A: weight } 40, \text{value } 100; \quad \text{remaining capacity } 10 \]
Only a fraction of Item B fits, giving a poor total
Why: Item B weighs 10, and only 10 of capacity remains, so all of Item B fits too: value adds 90. Total is 190. But look at the ratios: A's ratio is 100/40 = 2.5, B's ratio is 90/10 = 9. Sorting by value alone got lucky here only because both items fit; with tighter capacity it would not.
\[ \text{ratio}_A = 2.5, \quad \text{ratio}_B = 9 \ (\text{B is actually the better use of capacity}) \]
Sort by value-to-weight ratio, not value alone.
\[ \text{ratio}_B = 9 > \text{ratio}_A = 2.5 \]
Take Item B first: it has the higher ratio
Why: B weighs 10, value 90, taken completely. Remaining capacity drops to 40.
\[ \text{take Item B: weight } 10, \text{value } 90; \quad \text{remaining capacity } 40 \]
Take Item A next; both orders happen to reach 190 here, but ratio order is the one guaranteed correct in general
Why: Ratio order is what the exchange-argument proof actually establishes as optimal for every instance — value-alone order has no such guarantee and fails as soon as the higher-value item does not fully fit.
\[ \text{ratio order is optimal for every capacity}, \ \text{value order is not}\ \checkmark \]
Notation
Annotate
From Trap: sorting by total value instead of ratio — read this one piece at a time. What is each part doing?
On: \( \text{ratio order is optimal for every capacity}, \ \text{value order is not}\ \checkmark \)
Concept
In the exchange-argument proof above, the swap moved an arbitrary small amount, delta, of weight between two items. That move is only possible because items can be split into any fraction.
\[ \delta = \min(x_i - y_i, \ y_j) \quad \text{can be any real number between 0 and the smaller amount} \]
Being able to choose delta exactly is what lets the swap always land precisely on a feasible, no-worse solution. Take that ability away, and the proof's key step no longer goes through.
Intuition
Fractional knapsack hands you a knife that slices any item to precisely the weight you need. The exchange argument leans on that knife at the exact moment it needs to trade a tiny bit of one item for a tiny bit of another. Remove the knife, and the trade may simply not be available.
Concept
The 0/1 knapsack problem uses the exact same setup — items with value and weight, a capacity — but each item must be taken whole or not at all. No fractions are allowed.
\[ x_i \in \{0, 1\} \quad (\text{instead of } 0 \le x_i \le 1) \]
Socratic
Discussion prompt
The 0/1 knapsack problem uses the exact same setup — items with value and weight, a capacity — but each item must be taken whole or not at all. No fractions are allowed.
Suppose that were not true. What is the first thing in Greedy Algorithms & Exchange Arguments that would stop working?
Hint: Follow it one step downstream. The answer is whatever was quietly relying on it.
Anomaly
Predict first
A student writes this, and it looks reasonable:
Same items as before, capacity 50: Item 1 (weight 10, value 60, ratio 6), Item 2 (weight 20, value 100, ratio 5), Item 3 (weight 30, value 120, ratio 4). Apply highest-ratio-first greedy, but now each item must be taken whole or skipped entirely.
It is wrong. Say what breaks — and say it before you turn the page.
Correct: Item 1 (weight 10) and Item 2 (weight 20) together use 30 of the 50 capacity, leaving 20.
Check the alternative of skipping the highest-ratio item entirely.
Why: Item 1 (weight 10) and Item 2 (weight 20) together use 30 of the 50 capacity, leaving 20. Item 3 needs 30 and cannot be split, so it is skipped entirely, wasting the remaining 20 of capacity.
Trap
Same items as before, capacity 50: Item 1 (weight 10, value 60, ratio 6), Item 2 (weight 20, value 100, ratio 5), Item 3 (weight 30, value 120, ratio 4). Apply highest-ratio-first greedy, but now each item must be taken whole or skipped entirely.
| item | weight | value | ratio |
|---|---|---|---|
| Item 1 | 10 | 60 | 6 |
| Item 2 | 20 | 100 | 5 |
| Item 3 | 30 | 120 | 4 |
Greedy takes Item 1, then Item 2, then cannot fit Item 3
Why: Item 1 (weight 10) and Item 2 (weight 20) together use 30 of the 50 capacity, leaving 20. Item 3 needs 30 and cannot be split, so it is skipped entirely, wasting the remaining 20 of capacity.
\[ \text{greedy: Item 1} + \text{Item 2} = 30 \text{ weight}, \ 160 \text{ value}, \ 20 \text{ capacity wasted} \]
Check the alternative of skipping the highest-ratio item entirely.
\[ \text{Item 2} + \text{Item 3} = 20+30 = 50 \text{ weight (fits exactly)} \]
Item 2 plus Item 3 uses the full capacity and beats greedy's total
Why: Item 2 and Item 3 together weigh exactly 50, using all the capacity, for a value of 100 plus 120, equal to 220 — strictly more than greedy's 160, even though it skips the highest-ratio item completely.
\[ \text{value} = 100+120 = 220 > 160 \ \checkmark \]
Pattern
Step through it
Step through Trap: the natural greedy fails on 0/1 knapsack one row at a time. What is driving the change, and what would the row after the last one be?
Intuition
Watch me not know the answer. This is what the first two minutes actually look like.
Highest-ratio-first is provably optimal for fractional knapsack. Try the same proof for 0/1.
Try the same swap: replace the rival's lower-ratio item with greedy's higher-ratio one
Why: It worked five slides ago on the fractional version, and the setup looks identical.
The swap is not always legal
Why: In the fractional version you take a fraction of the higher-ratio item, exactly filling the space. In 0/1 you cannot cut it — the higher-ratio item may not fit in the space the removed item vacated, so the result is not a valid solution.
Dead end. Not a mistake — a move that was worth trying and did not pay off. This happens in most proofs.
Back up, and read the failure as information
Why: The exchange breaking is not a proof gap to patch. It is telling you the theorem is false, and pointing at the counterexample: make the higher-ratio item slightly too big for the hole.
A failed exchange argument is a counterexample generator. The place the swap becomes illegal is the place to build the input that breaks the rule — which is move #2, arriving from an unexpected direction.
The expert does not see the whole path in advance. The expert tries something, reads the result, and adjusts. That is the skill.
Explain it
Discussion prompt
Explain Process: trying to exchange your way through 0/1 knapsack to a student a year behind you. No notation, no jargon they have not met — and it still has to be true.
Hint: If your explanation needs a symbol they have never seen, you are describing the notation rather than the idea.
Answer:
Highest-ratio-first is provably optimal for fractional knapsack. Try the same proof for 0/1.
Concept
The exchange-argument swap for fractional knapsack moved an exact amount, delta, between two items — any real number of weight, however small. In 0/1 knapsack every item's amount is locked to exactly zero or exactly its full weight.
\[ \delta \ \text{must equal an entire item's weight, or nothing at all} \]
There is no guarantee that the leftover capacity after taking the higher-ratio items exactly matches some other whole item's weight, so the swap that made the fractional proof work often has nowhere to land.
Intuition
After greedy takes Item 1 and Item 2 above, 20 of capacity remains, but the only other item, Item 3, needs 30 — it simply does not fit. In the fractional world you would slice off exactly 20 of Item 3 and gain more value. In the 0/1 world that slice does not exist, and the capacity goes to waste.
Ranking
Put in order
Put the moves of Verify the 0/1 counterexample by full enumeration into the order they have to happen.
Why: These are the moves of the worked example in the order it makes them, and each one is set up by the one before it. Only Item 1 plus Item 2 plus Item 3 exceeds the capacity of 50, using 60 of weight, so that subset alone is ruled out.
Worked example
With only 3 items, there are 2 to the power of 3, that is 8, possible subsets. Check every one directly against the capacity of 50.
| subset | total weight | total value | feasible |
|---|---|---|---|
| none | 0 | 0 | yes |
| Item 1 | 10 | 60 | yes |
| Item 2 | 20 | 100 | yes |
| Item 3 | 30 | 120 | yes |
| Item 1 + Item 2 | 30 | 160 | yes |
| Item 1 + Item 3 | 40 | 180 | yes |
| Item 2 + Item 3 | 50 | 220 | yes |
| Item 1+2+3 | 60 | 240 | no |
Discard the one infeasible subset
Why: Only Item 1 plus Item 2 plus Item 3 exceeds the capacity of 50, using 60 of weight, so that subset alone is ruled out.
\[ 60 > 50 \ \Rightarrow \ \{1,2,3\} \text{ infeasible} \]
Find the maximum value among the remaining 7 feasible subsets
Why: Scanning the feasible rows, the values are 0, 60, 100, 120, 160, 180, and 220. The largest is 220, achieved by Item 2 plus Item 3.
\[ \max\{0,60,100,120,160,180,220\} = 220 \]
Verify greedy's answer is strictly below this maximum
Why: Greedy's highest-ratio-first selection was Item 1 plus Item 2, value 160. Since 160 is less than the true maximum of 220 found by checking every feasible subset, greedy is confirmed not optimal for 0/1 knapsack on this instance.
\[ 160 < 220 \ \Rightarrow \ \text{greedy is not optimal here} \ \checkmark \]
Concept
A problem is safe for greedy when two conditions both hold: making the locally best choice first never rules out reaching a globally optimal solution, and an optimal solution to the whole problem is built from optimal solutions to what remains after that choice.
greedy-choice property — A problem has this property if some globally optimal solution can always be reached by making the locally best choice at the very first step.
optimal substructure — A problem has this property if an optimal solution to it contains, within it, an optimal solution to the smaller problem left over after the first choice.
Picture it
Animation
Shows: The greedy choice property — a rendered Manim animation.
Rendered with Manim.
Takeaway: Not that greedy is optimal — that its first move is survivable.
Concept
Before investing in a full proof, try to break the proposed greedy rule with a small, deliberately adversarial example — a handful of items chosen specifically to create a leftover-capacity gap, or a request order designed to block the most future options.
If a small counterexample turns up quickly, as it did for 0/1 knapsack, the rule is not optimal and no amount of proof-writing will fix it. If the rule survives several serious attempts to break it, that is a good sign a real proof is worth attempting.
Intuition
Treat a new greedy rule the way you would treat a stranger's alibi: assume nothing, and actively look for the case that breaks it before you spend time building a case for why it must be true.
Analogy
Discussion prompt
Explain Every greedy rule is a suspect until proven innocent by analogy to something with no CS3000 Algorithms in it at all — a queue, a recipe, a map, a bank balance, whatever fits. Then say where your analogy breaks.
Hint: An analogy that never breaks is not an analogy, it is the same idea wearing a hat. Find the seam — that is the part that is actually new.
Answer:
Treat a new greedy rule the way you would treat a stranger's alibi: assume nothing, and actively look for the case that breaks it before you spend time building a case for why it must be true.
Concept
Earliest-finish-time greedy is optimal for interval scheduling, and highest-ratio-first greedy is optimal for fractional knapsack, because both have the greedy-choice property and optimal substructure, and both were proved by an argument covering every input.
0/1 knapsack fails only because the fractional flexibility that made the exchange argument's swap always land safely has been removed — the underlying idea of ratio order is still meaningful, it is simply no longer guaranteed optimal.
Pattern
1. State the greedy rule precisely
Why: Write exactly what greedy picks at each step and why that choice is locally best, before attempting any proof of it.
2. Choose a proof template
Why: Pick greedy stays ahead when it is natural to compare two whole sequences position by position; pick the exchange argument when it is easier to start from one assumed optimal solution and repair it.
3. Name every symbol before writing a justified line
Why: Greedy's sequence, a competing or optimal sequence, and the index of the first difference must all be named explicitly. This single habit is what turns a blank page into a proof you can actually start.
4. Discharge the technique's specific obligations
Why: For greedy stays ahead: prove the base case, then prove the inductive step. For an exchange argument: prove the swap keeps the solution feasible, and prove it does not decrease size or value.
5. Verify against a concrete instance, and pre-check with a counterexample attempt
Why: Before trusting the proof, try to break the greedy rule on a small adversarial example; after completing the proof, check that every general inequality actually holds on a specific worked case.
Real world
Discussion prompt
Outside this lesson: where does Greedy Algorithms & Exchange Arguments actually turn up? Name one concrete situation — a job, a piece of software someone ships, a decision somebody has to make — and say which part of The general recipe for proving greedy optimal is doing the work in it.
Hint: Vague is the failure mode here. "Engineering" is not a situation; "deciding whether this build is fast enough to ship" is.
Answer:
That deck explains what a greedy algorithm is and traces two of them: interval scheduling by earliest finish time, and fractional knapsack by highest value-to-weight ratio. It then gives the two standard ways to prove a greedy algorithm optimal, greedy-stays-ahead and the exchange argument. It targets the beginner's biggest gap, which is getting a proof started, along with the misconceptions that greedy always works, that any scheduling criterion is as good as another, that a few working examples count as a proof, and that an exchange can be made carelessly.
Picture it
Animation
Shows: What greedy buys you — a rendered Manim animation.
Rendered with Manim.
Takeaway: You trade a guarantee for a large speedup.
Elimination
Eliminate the wrong options
Which greedy criterion is guaranteed to produce the largest possible compatible set?
3 of these 4 are wrong. Strike them one at a time, and say what rules each one out before you strike the next. The survivor is the answer.
Survives elimination: A
Why: Earliest finish time is the only criterion proved optimal by both greedy stays ahead and the exchange argument: taking the request that frees the timeline soonest leaves the most room for everything still to come.
Check
You are asked to design a greedy algorithm for interval scheduling on a new set of requests.
Check your understanding
Which greedy criterion is guaranteed to produce the largest possible compatible set?
Answer: A
Why: Earliest finish time is the only criterion proved optimal by both greedy stays ahead and the exchange argument: taking the request that frees the timeline soonest leaves the most room for everything still to come.
Prediction
Predict first
What is the key fact the inductive step uses to conclude that greedy's pick at position i+1 finishes no later than the competing solution's pick at position i+1?
Answer it in your own words, now, with nothing to choose from. The options are on the next slide — and picking the right one off a list is an easier skill than producing it.
Correct: Because f(o_i) is at least f(g_i), the competitor's next request is also available to greedy, and greedy always takes the earliest-finishing available request.
Why: The step shows the competitor's (i+1)-th request starts at or after f(o_i), which by the inductive hypothesis is at or after f(g_i) — so it was an available, compatible choice for greedy at that moment. Since greedy always takes the earliest-finishing available request, greedy's actual choice finishes no later.
Check
In the greedy-stays-ahead proof for interval scheduling, the inductive step assumes the claim holds at position i and must establish it at position i+1.
Check your understanding
What is the key fact the inductive step uses to conclude that greedy's pick at position i+1 finishes no later than the competing solution's pick at position i+1?
Answer: A
Why: The step shows the competitor's (i+1)-th request starts at or after f(o_i), which by the inductive hypothesis is at or after f(g_i) — so it was an available, compatible choice for greedy at that moment. Since greedy always takes the earliest-finishing available request, greedy's actual choice finishes no later.
Commit first
Predict first
What is the correct first move in setting up the exchange argument?
Commit to an answer, then rate it — certain, fairly sure, or guessing — and write the rating down before you turn the page.
Correct: Assume some optimal solution disagrees with greedy, and locate the earliest position where the two disagree.
Why: The exchange argument starts by assuming an optimal solution differs from greedy, then finds the first point of disagreement — that specific position is what the swap will target next, so nothing else can proceed before it is named.
The rating matters as much as the answer: confident-and-wrong is the combination that survives revision, because nothing about it feels like it needs revisiting.
Check
You want to prove, by an exchange argument, that a greedy rule is optimal for some problem.
Check your understanding
What is the correct first move in setting up the exchange argument?
Answer: A
Why: The exchange argument starts by assuming an optimal solution differs from greedy, then finds the first point of disagreement — that specific position is what the swap will target next, so nothing else can proceed before it is named.
Prediction
Predict first
Why does highest-ratio-first greedy fail to find the optimal answer here?
Answer it in your own words, now, with nothing to choose from. The options are on the next slide — and picking the right one off a list is an easier skill than producing it.
Correct: Greedy fills the capacity with Item 1 and Item 2 for value 160, wasting 20 of capacity, while Item 2 plus Item 3 uses the full capacity for value 220.
Why: Greedy takes Item 1 and Item 2 (weight 30, value 160), leaving 20 of capacity that Item 3 (weight 30) cannot use, since it cannot be split. Item 2 plus Item 3 uses the entire 50 of capacity for value 220, strictly beating greedy — confirmed by checking all 7 feasible subsets.
Check
Consider the same three items from the knapsack examples — Item 1 (weight 10, value 60), Item 2 (weight 20, value 100), Item 3 (weight 30, value 120) — with capacity 50, but now under the 0/1 rule: whole items only.
Check your understanding
Why does highest-ratio-first greedy fail to find the optimal answer here?
Answer: A
Why: Greedy takes Item 1 and Item 2 (weight 30, value 160), leaving 20 of capacity that Item 3 (weight 30) cannot use, since it cannot be split. Item 2 plus Item 3 uses the entire 50 of capacity for value 220, strictly beating greedy — confirmed by checking all 7 feasible subsets.
Elimination
Eliminate the wrong options
Which swap is guaranteed to keep the solution feasible and no worse?
3 of these 4 are wrong. Strike them one at a time, and say what rules each one out before you strike the next. The survivor is the answer.
Survives elimination: A
Why: The exchange always swaps the competing solution's request OUT and greedy's earlier-finishing request IN. Since 6 is no later than 9, everything that was compatible with the finish-at-9 request remains compatible with the finish-at-6 request, so the swap only loosens constraints on what follows.
Check
In an exchange argument for interval scheduling, at the first position where an assumed optimal solution and greedy disagree, greedy's request finishes at time 6 and the optimal solution's request at that position finishes at time 9.
Check your understanding
Which swap is guaranteed to keep the solution feasible and no worse?
Answer: A
Why: The exchange always swaps the competing solution's request OUT and greedy's earlier-finishing request IN. Since 6 is no later than 9, everything that was compatible with the finish-at-9 request remains compatible with the finish-at-6 request, so the swap only loosens constraints on what follows.
Concept
Moves added today:
Moves you reused today:
Move #14 is built out of moves you already had: it assumes a rival (move #7), finds the first difference, and then inducts (moves #5 and #6). Naming it saves you from re-deriving the structure every time.
Full toolkit so far: #1 through #14.
Next session opens with you naming every one of these from memory, before any new material.
Counterexample
Discussion prompt
Next session opens with you naming every one of these from memory, before any new material.
That is stated as though it always holds. Do one of two things: produce a case where it fails, or say precisely what rules such a case out. "It just does" is not on the menu.
Hint: Hunt at the extremes first — zero, one, negative, empty, equal. If every extreme survives, the reason they survive is the proof.
Connect it up
Draw it
One page, no notation unless you need it: draw how these connect — The general recipe for proving greedy optimal · Toolkit check-in: name them before you look · What a greedy algorithm is · Climbing without looking back · Greedy never reconsiders. Put an arrow wherever one of them is what makes another possible, and label the arrow with why.
Recap
A greedy algorithm commits to the locally best choice at every step and never looks back — fast, but only correct when you can prove it.
| Situation | The one move |
|---|---|
| Comparing two whole sequences | Greedy stays ahead: base case, then i implies i+1 |
| Repairing one assumed-optimal solution | Exchange argument: first difference, swap, feasible and no worse |
| Interval scheduling | Earliest finish time first |
| Fractional knapsack | Highest value-to-weight ratio first |
Want this taught 1-on-1? Alexander tutors CS3000 Algorithms — $55/session, free consultation.