This deck covers Big-O, Big-Omega, and Big-Theta notation as CS3000 uses them. It explains why we measure growth rather than exact time, gives the formal definitions in terms of a constant and a threshold, shows how to construct a Big-O proof by exhibiting a specific c and n0, and lays out the standard growth-rate hierarchy from constant to factorial. It targets four real misconceptions: treating Big-O as a synonym for worst case, treating constant-factor multiples as different classes, reporting a loose Big-O when a tight Big-Theta is actually known, and assuming that an algorithm with a larger Big-O is always slower on the small inputs you actually run.
Subject: CS3000 Algorithms · 127 slides · symbolic lesson
Open the interactive version of this deck · Homework for this lesson
Objectives
Comparing algorithms in this course is never about seconds on a stopwatch — it is about how running time grows as the input gets bigger. By the end of this lesson you can:
Warm-up
Discussion prompt
Before we open Asymptotic Notation (Big-O, Big-Omega, Big-Theta): without looking back, what was the main idea of Reading Math Notation, Sets & Functions, and what could you do by the end of it that you could not do before?
Hint: One sentence for the idea, one for the skill. If the second one is blank, that is the part to revisit.
Answer:
How to read the symbolic vocabulary the rest of CS3000 is written in: sets and set-builder notation, element-of vs. subset-of, union and intersection, ordered pairs and the Cartesian product, functions with domain and codomain, floor vs.
Concept
Before we compare running times, we need to agree on what we are measuring against. We will always measure running time as a function of one quantity: the size of the input.
input size — Usually written as a single letter, most often n. It might be the number of elements in a list, the number of nodes in a graph, or the number of digits in a number — whatever quantity naturally grows when the problem gets bigger.
Everything in this lesson describes how running time behaves as that one quantity grows toward infinity. We are never talking about running time for one fixed input — we are talking about a whole family of inputs, indexed by their size.
Concept
Running time itself is a function of n. Call it f of n: the number of basic operations an algorithm performs on an input of size n.
\[ f(n) = \text{number of basic operations on an input of size } n \]
Two different algorithms solving the same problem can have two very different functions. We want a systematic way to compare those functions that does not depend on the specific computer, compiler, or programming language used to run them.
Counterexample
Discussion prompt
Running time itself is a function of n. Call it f of n: the number of basic operations an algorithm performs on an input of size n.
That is stated as though it always holds. Do one of two things: produce a case where it fails, or say precisely what rules such a case out. "It just does" is not on the menu.
Hint: Hunt at the extremes first — zero, one, negative, empty, equal. If every extreme survives, the reason they survive is the proof.
Intuition
Picture two runners. One gets a thirty-second head start. Over a hundred-meter dash, that head start decides the race. Over a twenty-six mile marathon, it barely matters — the runner's pace dominates the outcome.
Constant amounts of extra work behave like that head start. A fixed setup cost, a fixed number of extra operations, a faster processor — these are all head starts. What actually decides which algorithm wins for large inputs is how the running time scales, not any fixed extra amount tacked on.
Concept
asymptotic — Describing behavior as the input size grows without bound, focusing on what eventually happens for large n rather than what happens for any one specific, possibly small, n.
Every claim in this lesson comes with a hidden phrase attached: eventually, once n is big enough. A claim can be completely false for tiny inputs and still be a true asymptotic statement, because asymptotic statements are only required to hold from some starting point onward.
Intuition
Imagine plotting two running-time functions on a graph and zooming out further and further. Small wiggles, one-time costs, and fixed multipliers shrink to invisible slivers. What remains visible at that zoomed-out scale is the overall shape of the curve — flat, a shallow rising line, a bowl, or a curve that shoots upward.
That shape is exactly what big-O, big-omega, and big-theta notation capture. They are a deliberate zoom-out: a way of talking about the shape of growth while ignoring the details that vanish once the input is large enough.
Picture it
Animation
Shows: Constants dominate until they do not — a rendered Manim animation.
Rendered with Manim.
Takeaway: Below the crossover the quadratic looks better. Above it, forever worse.
Ranking
Put in order
Put the moves of Compare 100 times n and n squared: find the crossover into the order they have to happen.
Why: These are the moves of the worked example in the order it makes them, and each one is set up by the one before it. Starting concrete keeps the comparison honest before reasoning about trends.
Worked example
Two running times are on the table for the same problem: one algorithm takes a time proportional to one hundred times n, the other takes a time proportional to n squared. Which one is better for large inputs?
\[ f(n) = 100n \qquad g(n) = n^2 \]
Evaluate both at a small input, n = 10
Why: Starting concrete keeps the comparison honest before reasoning about trends.
\[ f(10) = 100(10) = 1000 \qquad g(10) = 10^2 = 100 \]
Evaluate both right at the crossover, n = 100
Why: Setting 100n equal to n squared and dividing both sides by n gives 100 equals n, so the two functions cross exactly there.
\[ f(100) = 100(100) = 10{,}000 \qquad g(100) = 100^2 = 10{,}000 \]
Evaluate both well past the crossover, n = 1000
Why: Past the crossover point, the two functions should start to separate, with the squared term pulling ahead.
\[ f(1000) = 100{,}000 \qquad g(1000) = 1{,}000{,}000 \]
Verify the trend continues forever past the crossover
Why: For any n greater than 100, dividing n squared by 100n gives n over 100, which keeps growing without bound as n grows. So n squared does not just exceed 100n once — it pulls further and further ahead forever. This is exactly what it means for n squared to eventually dominate 100n, regardless of the constant multiplier of 100.
\[ \frac{n^2}{100n} = \frac{n}{100} \ \to \ \infty \ \text{as } n \to \infty\ \checkmark \]
Picture it
Animation
Shows: Each line of the worked example "Compare 100 times n and n squared: find the crossover", appearing one at a time.
The same working the example does, in the order a tutor would write it.
Takeaway: For any n greater than 100, dividing n squared by 100n gives n over 100, which keeps growing without bound as n grows. So n squared does not just exceed 100n once — it pulls further and further ahead forever. This is exactly what it means for n squared to eventually dominate 100n, regardless of the constant multiplier of 100.
Anomaly
Predict first
A student writes this, and it looks reasonable:
A student says: this algorithm is big-O of n squared, so that must be its worst-case running time. They treat big-O and worst case as the very same idea.
It is wrong. Say what breaks — and say it before you turn the page.
Correct: This mixes up two independent questions: which input are we analyzing (best, worst, or average case), and what bound are we stating about the resulting function (big-O, big-omega, or big-theta).
Case (best, worst, average) and bound (big-O, big-omega, big-theta) are two separate labels you attach to the same running-time analysis. You need both.
Why: This mixes up two independent questions: which input are we analyzing (best, worst, or average case), and what bound are we stating about the resulting function (big-O, big-omega, or big-theta). Big-O says nothing by itself about which case is being described.
Trap
A student says: this algorithm is big-O of n squared, so that must be its worst-case running time. They treat big-O and worst case as the very same idea.
\[ \text{Claimed: Big-O} \ \equiv \ \text{worst case} \]
Use big-O as if it already picks out a case
Why: This mixes up two independent questions: which input are we analyzing (best, worst, or average case), and what bound are we stating about the resulting function (big-O, big-omega, or big-theta). Big-O says nothing by itself about which case is being described.
\[ \text{Big-O bounds a FUNCTION; it does not choose WHICH function} \]
Case (best, worst, average) and bound (big-O, big-omega, big-theta) are two separate labels you attach to the same running-time analysis. You need both.
Name the case first, then the bound
Why: For example, an algorithm can have a worst-case running time that is big-theta of n squared, and a completely different best-case running time that is big-theta of n. Big-O, big-omega, and big-theta can each be applied to the best case, the worst case, or the average case.
\[ \text{worst-case time} = \Theta(n^2), \quad \text{best-case time} = \Theta(n) \]
Always use both labels together
Why: Saying 'the worst-case running time is big-O of n squared' is a complete, correct statement. Saying only 'it is big-O of n squared,' with no case named, is dropping information — people often mean the worst case by default, but that is a convention, not a definition.
Notation
Annotate
From Trap: Big-O means "worst case" — read this one piece at a time. What is each part doing?
On: \( \text{Claimed: Big-O} \ \equiv \ \text{worst case} \)
Concept
There are three related pieces of notation. Each one makes a different kind of claim about how a function grows.
\[ f(n) = O(g(n)) \]
Say this aloud as: f of n is big-oh of g of n. It claims g of n is an upper bound on the growth of f of n.
\[ f(n) = \Omega(g(n)) \]
Say this aloud as: f of n is big-omega of g of n. It claims g of n is a lower bound on the growth of f of n.
\[ f(n) = \Theta(g(n)) \]
Say this aloud as: f of n is big-theta of g of n. It claims g of n is a tight bound — both an upper bound and a lower bound at once — on the growth of f of n.
Analogy
Discussion prompt
Explain Meet the three symbols: reading O, Omega, Theta aloud by analogy to something with no CS3000 Algorithms in it at all — a queue, a recipe, a map, a bank balance, whatever fits. Then say where your analogy breaks.
Hint: An analogy that never breaks is not an analogy, it is the same idea wearing a hat. Find the seam — that is the part that is actually new.
Answer:
Say this aloud as: f of n is big-oh of g of n. It claims g of n is an upper bound on the growth of f of n.
Concept
Here is the precise definition mathematicians and computer scientists actually use for big-O. Read it slowly the first time through.
\[ f(n) = O(g(n)) \ \text{ if } \ \exists\, c > 0,\ n_0 > 0 \ \text{ such that } \ 0 \le f(n) \le c \cdot g(n) \ \text{ for all } n \ge n_0 \]
In plain words: f is big-oh of g if you can find some fixed positive number c, and some fixed starting point called n-naught, so that from n-naught onward, f of n never exceeds c times g of n. The backwards-E symbol means 'there exists' — read it as 'there is some.'
Notice the two jobs baked into this definition: c gives you room for a constant multiplier, and n-naught lets you ignore everything before some threshold. Both are yours to choose — you just have to make the inequality true from that point on, forever.
Picture it
Animation
Shows: 100n versus n squared — a rendered Manim animation.
Rendered with Manim.
Takeaway: The quadratic loses until n = 100, then loses forever after.
Intuition
Think of c as a volume knob and n-naught as a starting line. The constant c lets f of n be up to c times bigger than g of n and the claim still holds — you are allowed some slack, as long as it is a fixed amount of slack, not a growing one.
The threshold n-naught lets you say 'ignore everything before this point.' Maybe the inequality is false for n equal to one, two, or three — that is fine. Big-O only promises the inequality holds from n-naught onward, forever after.
Intuition
Small inputs are often irregular. A function might have a one-time setup cost, an edge case, or some quirky behavior only for the first few values of n. None of that should be allowed to sink an otherwise-true growth claim.
That is exactly the job of the threshold. By requiring the inequality to hold only from n-naught onward, the definition quietly says: we do not care what happens before the threshold, only what happens forever after it.
Concept
When you are asked to prove that one function is big-oh of another, you are not being asked to explain why it feels true. You are being handed a very specific, mechanical job.
Job 1: propose a specific number for c
Why: Not a variable, not 'some constant' — an actual number, like 5 or 10, that you commit to.
Job 2: propose a specific number for n0
Why: Again an actual number — the point past which you promise the inequality holds.
Job 3: show the inequality actually holds for every n at or above n0
Why: Usually with a short chain of algebra, bounding each piece of f by a piece of c times g.
\[ f(n) \le c \cdot g(n) \quad \text{for all } n \ge n_0 \]
Explain it
Discussion prompt
Explain Your job when proving a Big-O claim to a student a year behind you. No notation, no jargon they have not met — and it still has to be true.
Hint: If your explanation needs a symbol they have never seen, you are describing the notation rather than the idea.
Answer:
When you are asked to prove that one function is big-oh of another, you are not being asked to explain why it feels true. You are being handed a very specific, mechanical job.
Step zero
Discussion prompt
Prove 3n squared + 5n + 2 is Big-O of n squared — before any calculation: what is the plan? Name the moves in order, in plain English, without doing the arithmetic.
Hint: It starts with: Bound the lower-order terms using n squared itself
Answer:
Worked example
Claim: the function three n squared plus five n plus two is big-oh of n squared.
\[ f(n) = 3n^2+5n+2, \qquad g(n) = n^2 \]
Bound the lower-order terms using n squared itself
Why: For any n at least 1, n is at most n squared, so 5n is at most 5 times n squared. Also 2 is at most 2 times n squared once n is at least 1, since n squared is then at least 1.
\[ n \ge 1 \ \Rightarrow \ 5n \le 5n^2 \ \text{and} \ 2 \le 2n^2 \]
Add the bounds to the leading term
Why: Replacing 5n and 2 by their larger bounds only makes the right side bigger, which is fine since we want an upper bound.
\[ 3n^2+5n+2 \ \le \ 3n^2+5n^2+2n^2 \ = \ 10n^2 \]
Read off the witnesses c and n0
Why: The chain of inequalities used c equal to ten and n0 equal to one — those are the two numbers we commit to.
\[ c = 10, \quad n_0 = 1 \]
Verify the inequality at the threshold and one point beyond it
Why: At n equal to one: three plus five plus two is ten, and ten times one squared is ten, so ten is at most ten — true. At n equal to two: three times four plus ten plus two is twenty-four, and ten times four is forty, so twenty-four is at most forty — true. The proof holds.
\[ n=1:\ 10 \le 10\ \checkmark \qquad n=2:\ 24 \le 40\ \checkmark \]
Picture it
Animation
Shows: Each line of the worked example "Prove 3n squared + 5n + 2 is Big-O of n squared", appearing one at a time.
The same working the example does, in the order a tutor would write it.
Takeaway: At n equal to one: three plus five plus two is ten, and ten times one squared is ten, so ten is at most ten — true. At n equal to two: three times four plus ten plus two is twenty-four, and ten times four is forty, so twenty-four is at most forty — true. The proof holds.
Anomaly
Predict first
A student writes this, and it looks reasonable:
A student computes that an algorithm takes exactly two n steps, and writes that its running-time class is its own separate class, twice as large as an algorithm that takes exactly n steps.
It is wrong. Say what breaks — and say it before you turn the page.
Correct: This ignores that big-O already has a constant, c, built into its own definition.
A constant multiplier in front of n does not create a new class. Two n and n describe the exact same growth shape — a straight line — just with different steepness, and big-O was built specifically to ignore steepness differences like that.
Why: This ignores that big-O already has a constant, c, built into its own definition. Any fixed multiplier can be absorbed into that same constant — it never needs a notation of its own.
Trap
A student computes that an algorithm takes exactly two n steps, and writes that its running-time class is its own separate class, twice as large as an algorithm that takes exactly n steps.
\[ \text{Claimed: } O(2n) \ne O(n), \ \text{treated as two different classes} \]
Treat the factor of two as changing the class
Why: This ignores that big-O already has a constant, c, built into its own definition. Any fixed multiplier can be absorbed into that same constant — it never needs a notation of its own.
\[ 2n \le c\cdot n \ \text{for } c = 2 \ \Rightarrow \ 2n = O(n) \]
A constant multiplier in front of n does not create a new class. Two n and n describe the exact same growth shape — a straight line — just with different steepness, and big-O was built specifically to ignore steepness differences like that.
Show 2n is big-oh of n directly
Why: Choose c equal to two and n0 equal to one: two n is at most two times n for every n, trivially, since they are equal.
\[ 2n \le 2\cdot n \ \text{for all } n \ge 1 \]
Show n is big-oh of 2n as well, so they are the same class
Why: Choose c equal to one: n is at most one times two n for every positive n, since n is at most twice itself. Both directions hold, so they describe the same big-O class.
\[ n \le 1\cdot 2n \ \text{for all } n \ge 1 \ \Rightarrow \ O(2n) = O(n) \]
Translation
\( 2n \le 2\cdot n \ \text{for all } n \ge 1 \)
Draw it
Translate both ways. First write the expression above as a sentence with no symbols in it at all. Then cover it, and write your sentence back as notation. If the two versions disagree, the disagreement is the thing to fix.
Concept
Nothing in the definition of big-O requires the bound to be the best possible one. A true upper bound is a true upper bound, even if a much smaller, tighter one also exists.
This surprises people the first time: you can correctly say a function is big-oh of something much bigger than necessary, and the statement is still technically true. It is just not the most informative statement you could make.
Picture it
Animation
Shows: Big-O is an upper bound, and only that — a rendered Manim animation.
Rendered with Manim.
Takeaway: A correct bound can still be a bad answer.
Estimation
Predict first
Claim: n by itself is big-oh of n squared — a deliberately loose bound, since n obviously grows slower than n squared.
Commit before you compute: what does Prove n is Big-O of n squared come out to? A rough magnitude and the right form is enough — the point is to have something concrete to be wrong about.
Correct: Verify at the threshold and beyond
Why: A prediction you can defend turns the computation into a check rather than a leap of faith — and an answer that contradicts it is caught on the spot. At n equal to one: one is at most one — true.
Worked example
Claim: n by itself is big-oh of n squared — a deliberately loose bound, since n obviously grows slower than n squared.
\[ f(n) = n, \qquad g(n) = n^2 \]
Propose c = 1 and n0 = 1
Why: Guess the simplest possible constants first; if they work, there is no need for anything fancier.
Check the inequality n is at most one times n squared
Why: This is the same as asking whether n is at most n squared, which is true exactly when n is at least one.
\[ n \le n^2 \ \iff \ 1 \le n \]
Verify at the threshold and beyond
Why: At n equal to one: one is at most one — true. At n equal to five: five is at most twenty-five — true, with plenty of room to spare. The bound holds, even though it is far from tight.
\[ n=1:\ 1\le 1\ \checkmark \qquad n=5:\ 5 \le 25\ \checkmark \]
Picture it
Animation
Shows: Each line of the worked example "Prove n is Big-O of n squared", appearing one at a time.
The same working the example does, in the order a tutor would write it.
Takeaway: This is the same as asking whether n is at most n squared, which is true exactly when n is at least one.
Intuition
A big-O claim is like a speed limit sign. A road posted at sixty miles per hour tells you cars will not go faster than that — it says nothing about whether cars actually go that fast, or much slower.
Saying a function is big-oh of n squared is posting a speed limit of n squared. The function is guaranteed not to grow faster than that. It might actually grow much more slowly, exactly like a quiet side street that happens to sit on a road posted for highway speeds.
Concept
Big-omega is the mirror image of big-O: instead of capping growth from above, it guarantees a floor from below.
\[ f(n) = \Omega(g(n)) \ \text{ if } \ \exists\, c>0,\ n_0>0 \ \text{such that} \ 0 \le c\cdot g(n) \le f(n) \ \text{for all } n \ge n_0 \]
In plain words: f is big-omega of g if you can find a fixed positive c and a fixed starting point n-naught so that, from n-naught onward, f of n never falls below c times g of n. Your job has the same shape as before — propose c and n0 — but now you show f stays above the bound instead of below it.
Picture it
Animation
Shows: Big-O, stated exactly — a rendered Manim animation.
Rendered with Manim.
Takeaway: A constant and a threshold. Produce both and you have a proof.
Intuition
If big-O is a speed limit sign capping how fast you can go, big-omega is a floor you cannot fall below — like a minimum-speed sign on a highway. Past the threshold, the function is guaranteed to be at least that big, forever after.
Worked example
Claim: n squared plus three n is big-omega of n squared.
\[ f(n) = n^2+3n, \qquad g(n) = n^2 \]
Propose c = 1 and n0 = 1
Why: The extra term, three n, is never negative for positive n, so dropping it can only make the left side smaller — meaning the inequality gets easier to satisfy, not harder.
Show n squared plus 3n is at least one times n squared
Why: Subtracting n squared from both sides leaves three n is at least zero, which is true for every non-negative n.
\[ n^2+3n \ge n^2 \ \iff \ 3n \ge 0 \]
Verify at the threshold and beyond
Why: At n equal to one: one plus three is four, and one times one squared is one, so four is at least one — true. At n equal to ten: one hundred thirty is at least one hundred — true. The lower bound holds.
\[ n=1:\ 4 \ge 1\ \checkmark \qquad n=10:\ 130 \ge 100\ \checkmark \]
Picture it
Animation
Shows: Each line of the worked example "Prove n squared + 3n is Big-Omega of n squared", appearing one at a time.
The same working the example does, in the order a tutor would write it.
Takeaway: At n equal to one: one plus three is four, and one times one squared is one, so four is at least one — true. At n equal to ten: one hundred thirty is at least one hundred — true. The lower bound holds.
Concept
Big-theta combines both directions at once: it says f grows at exactly the same rate as g, up to constant factors — not just an upper bound, and not just a lower bound, but both simultaneously.
\[ f(n) = \Theta(g(n)) \ \text{ if } \ \exists\, c_1, c_2 > 0,\ n_0 > 0 \ \text{such that} \ c_1\cdot g(n) \le f(n) \le c_2\cdot g(n) \ \text{for all } n \ge n_0 \]
In plain words: f is big-theta of g if you can sandwich f of n between two fixed constant multiples of g of n, from some threshold onward. Equivalently — and this is a handy shortcut — f is big-theta of g exactly when f is both big-oh of g and big-omega of g.
Intuition
Picture two lines drawn on the graph, both shaped exactly like g of n, one sitting a little above f and one a little below it, both eventually trapping f between them and never letting go. That is big-theta: a floor and a ceiling with the same shape as g, squeezing f from both sides.
Missing information
Discussion prompt
Claim: five n squared plus two n plus one is big-theta of n squared. Since big-theta needs both directions, this proof has two halves.
What do you need to know — or decide — before the first line can be written? List everything the problem has to hand you.
Hint: Anything you would have to invent to get started is a thing the problem must supply.
Answer:
For n at least one, two n is at most two n squared, and one is at most one n squared, so adding these bounds to the leading term gives an upper bound with constant eight.
Worked example
Claim: five n squared plus two n plus one is big-theta of n squared. Since big-theta needs both directions, this proof has two halves.
\[ f(n) = 5n^2+2n+1, \qquad g(n) = n^2 \]
Upper half: bound the lower-order terms
Why: For n at least one, two n is at most two n squared, and one is at most one n squared, so adding these bounds to the leading term gives an upper bound with constant eight.
\[ 5n^2+2n+1 \ \le \ 5n^2+2n^2+n^2 \ = \ 8n^2 \quad (n \ge 1) \]
Lower half: drop the non-negative extra terms
Why: Since two n plus one is always positive for positive n, removing it can only shrink the left side, so the inequality with just five n squared still holds.
\[ 5n^2+2n+1 \ \ge \ 5n^2 \quad (n \ge 0) \]
Read off all three witnesses
Why: The lower constant is five, the upper constant is eight, and the shared threshold is one — three numbers that make both halves of the sandwich true at once.
\[ c_1 = 5, \quad c_2 = 8, \quad n_0 = 1 \]
Verify the full sandwich at two values
Why: At n equal to one: five is at most eight is at most eight, since five times one is 5, f of one is 8, and eight times one is also 8 — true. At n equal to two: five times four is twenty, f of two is twenty-five, eight times four is thirty-two, so twenty is at most twenty-five is at most thirty-two — true. Both bounds hold together.
\[ n=1:\ 5 \le 8 \le 8\ \checkmark \qquad n=2:\ 20 \le 25 \le 32\ \checkmark \]
Picture it
Animation
Shows: Little-o is strictly smaller — a rendered Manim animation.
Rendered with Manim.
Takeaway: Little-o rules out the same growth rate; big-O permits it.
Anomaly
Predict first
A student writes this, and it looks reasonable:
After proving that five n squared plus two n plus one is bounded both above and below by constant multiples of n squared — a full big-theta result — a student reports only the weaker upper-bound half.
It is wrong. Say what breaks — and say it before you turn the page.
Correct: This throws away real information.
State the strongest true claim you have actually earned. If you have proven both the upper and lower bound, say so — report the tight bound, big-theta, not just one half of it.
Why: This throws away real information. If both halves of the sandwich are already established, saying only the upper half is technically true, but it hides the fact that the bound is actually tight.
Trap
After proving that five n squared plus two n plus one is bounded both above and below by constant multiples of n squared — a full big-theta result — a student reports only the weaker upper-bound half.
\[ \text{Reported: } f(n) = O(n^2) \ \text{only} \]
Stop after the upper bound, even though the lower bound was already proven
Why: This throws away real information. If both halves of the sandwich are already established, saying only the upper half is technically true, but it hides the fact that the bound is actually tight.
State the strongest true claim you have actually earned. If you have proven both the upper and lower bound, say so — report the tight bound, big-theta, not just one half of it.
Report the full Big-Theta result
Why: Since both the c1 times g(n) at most f(n) half and the f(n) at most c2 times g(n) half were proven, the honest, complete, most useful statement is that f is big-theta of n squared, not merely big-oh of n squared.
\[ f(n) = \Theta(n^2) \ \text{(the strongest true claim)} \]
Reserve plain Big-O for when the lower bound genuinely is not known
Why: Big-O is the right, honest choice when you have only established an upper bound and have not proven, or do not yet believe, a matching lower bound. It is not a weaker default you fall back to out of habit when a tight bound is sitting right there.
Break the constraint
Discussion prompt
The rule this trap just fixed:
State the strongest true claim you have actually earned. If you have proven both the upper and lower bound, say so — report the tight bound, big-theta, not just one half of it.
Now break it on purpose. Build a case that violates it and follow the consequences until something visibly fails. Where does the failure first show up — and would you have noticed it if you had not been looking?
Hint: The dangerous rules are the ones whose violation still produces an answer. If yours fails loudly, try to find one that fails quietly.
Answer:
This throws away real information. If both halves of the sandwich are already established, saying only the upper half is technically true, but it hides the fact that the bound is actually tight.
Concept
Sometimes you need to show a claimed bound is false — that one function is not big-oh of another. The standard tool is a proof by contradiction: assume the bound holds, then derive something impossible.
Assume the Big-O relationship holds
Why: Suppose, for the sake of contradiction, that some fixed c and n0 exist making the inequality true for all n past the threshold.
\[ \text{Assume: } f(n) \le c\cdot g(n) \ \text{for all } n \ge n_0 \]
Manipulate the inequality until it forces something impossible
Why: Usually this means showing that c would have to be at least as large as some quantity that grows without bound as n grows — which no fixed constant can be.
Notation
Annotate
From Disproving a bound: assume it, then contradict it — read this one piece at a time. What is each part doing?
On: \( \text{Assume: } f(n) \le c\cdot g(n) \ \text{for all } n \ge n_0 \)
Intuition
If you claim a fixed multiplier c can always keep one function under another, you are betting that a single, unmoving number can out-race a quantity that keeps climbing. Pick any c you like — a runner whose lead grows every single lap will eventually pass a stationary marker, no matter how far out you place it.
That is the heart of every disproof in this lesson: name the fixed constant your opponent offers, then point to an input large enough that the constant can no longer keep up.
Step zero
Discussion prompt
Prove n squared is NOT Big-O of n — before any calculation: what is the plan? Name the moves in order, in plain English, without doing the arithmetic.
Hint: It starts with: Divide both sides by n
Answer:
Worked example
Claim to disprove: n squared is big-oh of n. Suppose, for contradiction, that this were true.
\[ \text{Assume: } n^2 \le c\cdot n \ \text{for all } n \ge n_0, \ \text{for some fixed } c, n_0 \]
Divide both sides by n
Why: Since n is positive for every n we care about, dividing preserves the direction of the inequality and simplifies it a great deal.
\[ n^2 \le c\cdot n \ \Rightarrow \ n \le c \quad \text{for all } n \ge n_0 \]
Find an n that breaks the inequality
Why: The inequality claims n stays at most c forever past the threshold. But c is a single fixed number, while n can be chosen as large as we like — so pick n larger than both c and n0.
\[ \text{Choose } n = c + n_0 + 1 \ \Rightarrow \ n > c \]
Verify the contradiction with concrete numbers
Why: Say someone offers c equal to one million. Then n equal to one million and one gives n squared far bigger than one million times n, breaking the claimed inequality. Any fixed c fails the same way, so no valid witness exists, and n squared is not big-oh of n.
\[ c = 1{,}000{,}000, \ n = 1{,}000{,}001 \ \Rightarrow \ n > c \ \text{(contradiction)}\ \checkmark \]
Picture it
Animation
Shows: Each line of the worked example "Prove n squared is NOT Big-O of n", appearing one at a time.
The same working the example does, in the order a tutor would write it.
Takeaway: Say someone offers c equal to one million. Then n equal to one million and one gives n squared far bigger than one million times n, breaking the claimed inequality. Any fixed c fails the same way, so no valid witness exists, and n squared is not big-oh of n.
Concept
When running time does not depend on the input size at all — the same fixed number of steps no matter how big n gets — we describe it with the simplest possible growth rate: constant.
\[ f(n) = 7 \ \Rightarrow \ f(n) = O(1) \]
Read the right-hand side aloud as: big-oh of one. The one here is not literally a single operation — it stands for 'some fixed amount, not depending on n at all.'
Concept
Running times fall into a small number of standard growth families. From slowest-growing to fastest-growing, here is the ladder every algorithms course relies on.
\[ O(1) \ \lt\ O(\log n) \ \lt\ O(n) \ \lt\ O(n \log n) \ \lt\ O(n^2) \ \lt\ O(n^3) \ \lt\ O(2^n) \ \lt\ O(n!) \]
Read these aloud, left to right: constant, logarithmic, linear, linearithmic (that is, n times the logarithm of n), quadratic, cubic, exponential, and factorial. Each one, past some threshold, is eventually overtaken by everything to its right.
Picture it
Animation
Shows: The ladder of growth rates — a rendered Manim animation.
Rendered with Manim.
Takeaway: Constants matter early and stop mattering permanently.
Intuition
The ladder can feel abstract until you plug in an actual number. Here is what each growth rate produces at an input size of just ten.
| Growth rate | Value at n = 10 |
|---|---|
| constant | 1 |
| logarithmic | about 3.3 |
| linear | 10 |
| linearithmic | about 33 |
| quadratic | 100 |
| cubic | 1,000 |
| exponential | 1,024 |
| factorial | 3,628,800 |
Pattern
Step through it
Step through Concrete numbers make the ladder real one row at a time. What is driving the change, and what would the row after the last one be?
Intuition
Exponential growth means the function multiplies by some fixed factor every time n increases by one. Factorial growth means the function gets multiplied by n itself at every step, so the multiplier keeps growing too — that is even faster.
\[ 7! = 5040, \qquad 2^7 = 128 \]
At n equal to seven, factorial has already pulled far ahead of exponential, and the gap only widens from here — by n equal to ten, factorial is over three thousand times bigger than two raised to that same power.
Picture it
Animation
Shows: The line that actually matters — a rendered Manim animation.
Rendered with Manim.
Takeaway: Tractable and intractable — every polynomial is on one side of this.
Worked example
Order these five functions from slowest-growing to fastest-growing, for large n: n squared, the logarithm of n, n, n times the logarithm of n, and two to the n.
\[ n^2, \quad \log n, \quad n, \quad n\log n, \quad 2^n \]
Place the two extremes first
Why: The logarithm grows the slowest of any of these — slower even than n itself — while two to the n, an exponential, eventually outpaces every one of the others.
Order the three remaining middle terms
Why: n times the logarithm of n is n multiplied by something that grows without bound, so it eventually exceeds plain n; and n squared is n multiplied by itself, which eventually exceeds n multiplied by a mere logarithm.
\[ \log n \ \lt\ n \ \lt\ n\log n \ \lt\ n^2 \ \lt\ 2^n \]
Verify the order with actual numbers at n = 16
Why: The logarithm base two of sixteen is four, n itself is sixteen, n times its logarithm is sixteen times four equals sixty-four, n squared is two hundred fifty-six, and two to the sixteen is sixty-five thousand five hundred thirty-six. Those five numbers are already in increasing order, matching the claimed ranking.
\[ 4 \ \lt\ 16 \ \lt\ 64 \ \lt\ 256 \ \lt\ 65{,}536\ \checkmark \]
Picture it
Animation
Shows: Each line of the worked example "Rank five functions by growth", appearing one at a time.
The same working the example does, in the order a tutor would write it.
Takeaway: The logarithm base two of sixteen is four, n itself is sixteen, n times its logarithm is sixteen times four equals sixty-four, n squared is two hundred fifty-six, and two to the sixteen is sixty-five thousand five hundred thirty-six. Those five numbers are already in increasing order, matching the claimed ranking.
Anomaly
Predict first
A student writes this, and it looks reasonable:
A student is choosing between two algorithms for a school project that will only ever run on lists of about twenty items. One algorithm is big-oh of n log n; the other is big-oh of n squared. They immediately pick the first one, assuming it must run faster.
It is wrong. Say what breaks — and say it before you turn the page.
Correct: Big-O only describes what happens once n is large enough, past whatever threshold made the proof work.
Big-O compares long-run growth, not performance at one particular, possibly small, input size. For the actual sizes you will run, measure — or at least account for the constants — rather than assume the asymptotically better class always wins.
Why: Big-O only describes what happens once n is large enough, past whatever threshold made the proof work. For a small, fixed n like twenty, the algorithm with the worse Big-O class can easily have a smaller constant factor and actually finish first.
Trap
A student is choosing between two algorithms for a school project that will only ever run on lists of about twenty items. One algorithm is big-oh of n log n; the other is big-oh of n squared. They immediately pick the first one, assuming it must run faster.
\[ \text{Algorithm A: } O(n\log n) \qquad \text{Algorithm B: } O(n^2) \]
Assume the asymptotically better class wins at every single n
Why: Big-O only describes what happens once n is large enough, past whatever threshold made the proof work. For a small, fixed n like twenty, the algorithm with the worse Big-O class can easily have a smaller constant factor and actually finish first.
Big-O compares long-run growth, not performance at one particular, possibly small, input size. For the actual sizes you will run, measure — or at least account for the constants — rather than assume the asymptotically better class always wins.
Consider the hidden constants
Why: Suppose algorithm A actually takes fifty times n times the logarithm of n steps, while algorithm B takes exactly n squared steps. At n equal to twenty, A takes about four thousand three hundred steps while B takes only four hundred — B is faster here, even though A has the better asymptotic class.
\[ n=20:\ A \approx 50(20)(\log_2 20) \approx 4{,}322, \quad B = 20^2 = 400 \]
Use Big-O for what it is built for: comparing growth as n keeps increasing
Why: Past a large enough n, A's smaller growth rate will eventually win no matter the constants — but 'eventually' can be a very large number, well beyond the sizes you actually care about. Big-O is a promise about the far future of the curve, not a guarantee about today's inputs.
Two truths and a lie
Sort into buckets
Some of these hold up and some are the exact mistakes this lesson is built to prevent. Sort them.
Concept
Every term in a running-time formula except the fastest-growing one becomes irrelevant once n is large enough — that fastest-growing term eventually swamps all the others combined.
\[ a_k n^k + a_{k-1}n^{k-1} + \cdots + a_1 n + a_0 \ = \ O(n^k) \]
For any polynomial like this, no matter how many lower-order terms it has or how large their coefficients are, the whole expression is big-oh of just its highest power of n. The proof strategy is always the same one used earlier: bound every smaller term by the leading power, once n is at least one.
Explain it
Discussion prompt
Explain Why lower-order terms vanish inside a Big-O claim to a student a year behind you. No notation, no jargon they have not met — and it still has to be true.
Hint: If your explanation needs a symbol they have never seen, you are describing the notation rather than the idea.
Answer:
Every term in a running-time formula except the fastest-growing one becomes irrelevant once n is large enough — that fastest-growing term eventually swamps all the others combined.
Picture it
Animation
Shows: Why lower-order terms disappear — a rendered Manim animation.
Rendered with Manim.
Takeaway: Bounding each small term by the big one is the whole technique.
Estimation
Predict first
Simplify this running-time expression down to the tightest simple Big-O class you can justify: seven n cubed, plus twenty n squared times the logarithm of n, plus fifteen n, plus one hundred.
Commit before you compute: what does Simplify a messy expression to one Big-O class come out to? A rough magnitude and the right form is enough — the point is to have something concrete to be wrong about.
Correct: Verify the bound at n = 2
Why: A prediction you can defend turns the computation into a check rather than a leap of faith — and an answer that contradicts it is caught on the spot. At n equal to two, the logarithm base two of two is one, so f of two equals seven times eight plus twenty times four times one plus fifteen times two plus one hundred, which is fifty-six plus eighty plus thirty plus one hundred, equal to two hundred sixty-six. One hundred forty-two times two cubed is one thousand one hundred thirty-six. Two hundred sixty-six is indeed at most one thousand one hundred thirty-six.
Worked example
Simplify this running-time expression down to the tightest simple Big-O class you can justify: seven n cubed, plus twenty n squared times the logarithm of n, plus fifteen n, plus one hundred.
\[ f(n) = 7n^3 + 20n^2\log n + 15n + 100 \]
Compare the two candidate leading terms
Why: n cubed divided by n squared times the logarithm of n equals n divided by the logarithm of n, which grows without bound as n grows. So n cubed eventually outgrows n squared times the logarithm of n, making n cubed the true leading term.
\[ \frac{n^3}{n^2\log n} = \frac{n}{\log n} \ \to \ \infty \]
Bound every other term by n cubed
Why: Since the logarithm of n is at most n for every n at least one, twenty n squared times the logarithm of n is at most twenty n cubed. Also fifteen n is at most fifteen n cubed, and one hundred is at most one hundred n cubed, all once n is at least one.
\[ 20n^2\log n \le 20n^3, \quad 15n \le 15n^3, \quad 100 \le 100n^3 \quad (n \ge 1) \]
Add up the bounds
Why: Summing the leading term and every bound gives a single constant, one hundred forty-two, multiplying n cubed.
\[ f(n) \le 7n^3+20n^3+15n^3+100n^3 = 142n^3, \quad c = 142, \ n_0 = 1 \]
Verify the bound at n = 2
Why: At n equal to two, the logarithm base two of two is one, so f of two equals seven times eight plus twenty times four times one plus fifteen times two plus one hundred, which is fifty-six plus eighty plus thirty plus one hundred, equal to two hundred sixty-six. One hundred forty-two times two cubed is one thousand one hundred thirty-six. Two hundred sixty-six is indeed at most one thousand one hundred thirty-six.
\[ f(2) = 56+80+30+100 = 266 \ \le \ 142(8) = 1136\ \checkmark \]
Picture it
Animation
Shows: Each line of the worked example "Simplify a messy expression to one Big-O class", appearing one at a time.
The same working the example does, in the order a tutor would write it.
Takeaway: At n equal to two, the logarithm base two of two is one, so f of two equals seven times eight plus twenty times four times one plus fifteen times two plus one hundred, which is fifty-six plus eighty plus thirty plus one hundred, equal to two hundred sixty-six. One hundred forty-two times two cubed is one thousand one hundred thirty-six. Two hundred sixty-six is indeed at most one thousand one hundred thirty-six.
Concept
There is a shortcut hiding inside the last several proofs: when you add running times from different parts of an algorithm together, the sum is dominated by whichever part grows fastest.
\[ f(n) = O(h(n)), \ \ g(n) = O(h(n)) \ \Rightarrow \ f(n)+g(n) = O(h(n)) \]
If two pieces of work are each big-oh of the same bound, their total is still big-oh of that bound — adding two things that are each capped by a shared bound gives something still capped by that same bound, just with a bigger constant.
Ranking
Put in order
Put the moves of Apply the sum rule to a multi-part runtime formula into the order they have to happen.
Why: These are the moves of the worked example in the order it makes them, and each one is set up by the one before it. Among two n squared, fifty n, and ten, the quadratic term grows fastest, so it should anchor the bound.
Worked example
An algorithm runs in three phases. Phase one costs two n squared steps, phase two costs fifty n steps, and phase three costs a flat ten steps. Find the overall Big-O class of the total running time.
\[ \text{total}(n) = 2n^2 + 50n + 10 \]
Identify the dominant term
Why: Among two n squared, fifty n, and ten, the quadratic term grows fastest, so it should anchor the bound.
Bound each smaller phase by the leading term
Why: For n at least one, fifty n is at most fifty n squared, and ten is at most ten n squared.
\[ 50n \le 50n^2, \quad 10 \le 10n^2 \quad (n \ge 1) \]
Add the bounds
Why: Two n squared plus fifty n squared plus ten n squared is sixty-two n squared, giving a witness constant of sixty-two.
\[ 2n^2+50n+10 \ \le \ 62n^2, \quad c = 62, \ n_0 = 1 \]
Verify at n = 1
Why: At n equal to one, the total is two plus fifty plus ten equals sixty-two, and sixty-two times one squared is also sixty-two — the two sides meet exactly at the threshold, confirming the bound is valid starting there.
\[ \text{total}(1) = 62 \ \le \ 62(1)^2 = 62\ \checkmark \]
Picture it
Animation
Shows: Asymptotics are not milliseconds — a rendered Manim animation.
Rendered with Manim.
Takeaway: Which is why insertion sort beats merge sort on tiny arrays.
Intuition
Fluency with this notation is mostly about having a full spoken sentence ready for each symbol, instead of stalling out on the shapes.
\[ f(n) = O(n^2) \]
Say: f of n is big-oh of n squared — meaning f does not grow faster than n squared, from some point on.
\[ f(n) = \Omega(n) \]
Say: f of n is big-omega of n — meaning f does not grow slower than n, from some point on.
\[ f(n) = \Theta(n\log n) \]
Say: f of n is big-theta of n log n — meaning f grows at exactly the rate of n times its own logarithm, no faster and no slower, from some point on.
Analogy
Discussion prompt
Explain Reading a bound aloud, full sentences by analogy to something with no CS3000 Algorithms in it at all — a queue, a recipe, a map, a bank balance, whatever fits. Then say where your analogy breaks.
Hint: An analogy that never breaks is not an analogy, it is the same idea wearing a hat. Find the seam — that is the part that is actually new.
Answer:
Fluency with this notation is mostly about having a full spoken sentence ready for each symbol, instead of stalling out on the shapes.
Concept
Counting operations in code follows a simple multiplication rule: when one loop runs inside another, the total number of times the innermost work happens is the product of how many times each loop runs.
\[ \text{outer loop: } n \text{ times} \quad \times \quad \text{inner loop: } n \text{ times} \quad = \quad n\cdot n = n^2 \ \text{total operations} \]
Concept
You do not need to count exactly to get a Big-O bound. You need to know how many times the innermost line runs, and that is a question about the loop headers alone.
COUNT-PAIRS(A, n)
count = 0
for i = 1 to n
for j = i + 1 to n
if A[i] + A[j] == 0
count = count + 1
return countThe inner loop does not run n times. It runs n minus i times, so the total is the sum of those, which is about half of n squared. Half of n squared is still quadratic, because Big-O throws the constant away.
Notation
Every line of COUNT-PAIRS says one thing. Read the line, then read what it does — not the other way round.
Annotate
Invariant
Watch the comparison counter. The claim is not that it reaches n times n — it reaches roughly half that. Watch it get there, and notice that halving a quadratic leaves a quadratic.
Step through it
Before each step, predict how many comparisons the current value of i will add.
Picture it
Animation
Shows: COUNT-PAIRS executing: the current line of pseudocode is highlighted while the data it touches changes.
Rendered with Manim.
Takeaway: Count how often the innermost line runs. Here it is about half of n squared, and Big-O discards the half.
Hypothesis
Predict first
Analyze a nested-loop algorithm's Big-O is about to be worked. State your hypothesis first: which rule or definition decides this one, and what is the first move it forces? Then watch whether the example agrees with you.
Correct: Count total inner-loop executions
Why: The inner loop body runs once for every combination of an outer-loop pass and an inner-loop pass, so the count is n choices for the outer loop times n choices for the inner loop.
A hypothesis you wrote down is falsifiable; a vague sense of how it will go is not. If the example opens somewhere else, that gap is the thing worth chasing.
Worked example
An algorithm has an outer loop that runs n times, and for every single pass of the outer loop, an inner loop that also runs n times, doing one unit of constant work each time. Find its running time's growth rate.
Count total inner-loop executions
Why: The inner loop body runs once for every combination of an outer-loop pass and an inner-loop pass, so the count is n choices for the outer loop times n choices for the inner loop.
\[ \text{total operations} = n \times n = n^2 \]
Recognize this is exact, not just an upper bound
Why: The loops always run exactly n times each, with no early exits — so the running time is not just capped by n squared, it is also never any less than n squared. Both directions hold with the same constant.
\[ c_1 = c_2 = 1, \quad n_0 = 1 \ \Rightarrow \ \text{running time} = \Theta(n^2) \]
Verify with a small trace
Why: For n equal to three, the outer loop runs three times, and each time the inner loop runs three times, giving nine total executions of the inner body — exactly three squared, matching the tight bound.
\[ n=3:\ 3 \times 3 = 9 = 3^2\ \checkmark \]
Picture it
Animation
Shows: Arithmetic inside big-O — a rendered Manim animation.
Rendered with Manim.
Takeaway: Sums keep the larger; products multiply. Sequential code sums, nested code multiplies.
Concept
Revisit the very first trap from this lesson, now with the formal tools in hand. An algorithm's running time can differ depending on which specific input of size n you feed it — that is the case. Big-O, big-omega, and big-theta describe the growth of whichever function you have chosen to analyze.
So a complete, precise statement always names both pieces: which case, and which bound. 'The worst-case running time is big-theta of n squared' is a complete sentence. 'It's big-oh of n squared,' with no case named, leaves ambiguous which running-time function is even being discussed.
Picture it
Animation
Shows: Case and bound are independent — a rendered Manim animation.
Rendered with Manim.
Takeaway: Naming one does not name the other. State both.
Intuition
Imagine searching for your keys in a messy room by checking one spot at a time. If your keys happen to be in the very first spot you check, you finish almost immediately. If they are in the very last spot — or missing entirely — you end up checking every single spot in the room.
Those are two different functions of the room's size: the quick case and the exhaustive case. Each one gets its own growth-rate analysis; neither one is 'the' running time of searching a room.
Step zero
Discussion prompt
Linear search: tight bound worst case, upper bound best case — before any calculation: what is the plan? Name the moves in order, in plain English, without doing the arithmetic.
Hint: It starts with: Analyze the worst case
Answer:
Worked example
An algorithm scans a list of n items one at a time, looking for a target value, and stops the moment it finds a match, or after checking every item if there is no match. Analyze its worst-case and best-case running times.
Analyze the worst case
Why: If the target is the very last item, or is not present at all, the algorithm must inspect every one of the n items before it can stop — exactly n comparisons, no more and no fewer, so this case is bounded both above and below by n.
\[ \text{worst-case running time} = \Theta(n) \]
Analyze the best case
Why: If the target happens to be the very first item, the algorithm stops after just one comparison, regardless of how large n is — a constant amount of work.
\[ \text{best-case running time} = \Theta(1) \]
Verify with a list of five items
Why: Searching for the last of five items takes all five comparisons, matching big-theta of n at n equal to five; searching for the first item takes exactly one comparison regardless of the list holding five items, matching big-theta of one.
\[ n=5:\ \text{worst} = 5 \text{ comparisons}, \quad \text{best} = 1 \text{ comparison}\ \checkmark \]
Picture it
Animation
Shows: Each line of the worked example "Linear search: tight bound worst case, upper bound best case", appearing one at a time.
The same working the example does, in the order a tutor would write it.
Takeaway: Searching for the last of five items takes all five comparisons, matching big-theta of n at n equal to five; searching for the first item takes exactly one comparison regardless of the list holding five items, matching big-theta of one.
Concept
Real writing rarely says 'big-oh' or 'big-theta' outright — it uses everyday phrases. Learning to translate those phrases into the right symbol is its own skill.
A phrase like 'takes no more than roughly such-and-such time' is asking for an upper bound — big-O. A phrase like 'takes at least roughly such-and-such time' is asking for a lower bound — big-omega. A phrase like 'takes exactly on the order of such-and-such time, both ceiling and floor' is asking for a tight bound — big-theta.
Counterexample
Discussion prompt
Real writing rarely says 'big-oh' or 'big-theta' outright — it uses everyday phrases. Learning to translate those phrases into the right symbol is its own skill.
That is stated as though it always holds. Do one of two things: produce a case where it fails, or say precisely what rules such a case out. "It just does" is not on the menu.
Hint: Hunt at the extremes first — zero, one, negative, empty, equal. If every extreme survives, the reason they survive is the proof.
Fill the middle
Fill in the blanks
From Translate plain-English claims into symbols — finish the line. Write what belongs on the right of the equals sign before you look.
f(n) = O(n\log n)
Why: Producing the right-hand side unprompted is the difference between recognising this line and being able to use it. The phrase 'never takes more than' is an upper-bound promise — exactly what big-oh captures.
Worked example
Translate each plain-English running-time claim below into the correct notation, for a function f of n.
Translate: this sorting method never takes more than roughly n log n time
Why: The phrase 'never takes more than' is an upper-bound promise — exactly what big-oh captures.
\[ f(n) = O(n\log n) \]
Translate: this method always takes at least roughly n time, no shortcuts possible
Why: The phrase 'at least,' with no upper cap mentioned, is a lower-bound promise — big-omega.
\[ f(n) = \Omega(n) \]
Verify by translating a tight claim: this method's time is on the order of n squared, both a floor and a ceiling
Why: Naming both a floor and a ceiling with the same shape is exactly the definition of big-theta, so the correct translation is f of n equals big-theta of n squared. Checking each translation against its defining phrase confirms all three match their definitions.
\[ f(n) = \Theta(n^2) \]
Picture it
Animation
Shows: Three bounds, three claims — a rendered Manim animation.
Rendered with Manim.
Takeaway: Case and bound are independent labels — you can bound a best case.
Missing information
Discussion prompt
Claim: the logarithm base two of n is big-oh of n — logarithms grow strictly slower than a straight line.
What do you need to know — or decide — before the first line can be written? List everything the problem has to hand you.
Hint: Anything you would have to invent to get started is a thing the problem must supply.
Answer:
Try the simplest constants first, since logarithmic growth is famously much slower than linear growth.
Worked example
Claim: the logarithm base two of n is big-oh of n — logarithms grow strictly slower than a straight line.
\[ f(n) = \log_2 n, \qquad g(n) = n \]
Propose c = 1 and n0 = 1
Why: Try the simplest constants first, since logarithmic growth is famously much slower than linear growth.
Confirm the logarithm never exceeds n itself
Why: This is a standard fact: doubling n only adds one more step to its base-two logarithm, while n itself grows by the full doubling — so the logarithm can never catch up to n, for any n at least one.
\[ \log_2 n \le n \quad \text{for all } n \ge 1 \]
Verify at two values
Why: At n equal to one, the logarithm base two of one is zero, which is at most one. At n equal to eight, the logarithm base two of eight is three, which is at most eight. The bound holds comfortably both times.
\[ n=1:\ 0\le 1\ \checkmark \qquad n=8:\ 3 \le 8\ \checkmark \]
Picture it
Animation
Shows: Why the base of a log never appears — a rendered Manim animation.
Rendered with Manim.
Takeaway: Which is why nobody writes the base inside a complexity class.
Estimation
Predict first
Claim: n times the logarithm of n is big-oh of n squared.
Commit before you compute: what does Prove n log n is Big-O of n squared come out to? A rough magnitude and the right form is enough — the point is to have something concrete to be wrong about.
Correct: Verify at n = 8
Why: A prediction you can defend turns the computation into a check rather than a leap of faith — and an answer that contradicts it is caught on the spot. At n equal to eight, n times its logarithm base two is eight times three equals twenty-four, and n squared is sixty-four.
Worked example
Claim: n times the logarithm of n is big-oh of n squared.
\[ f(n) = n\log_2 n, \qquad g(n) = n^2 \]
Reuse the previous result
Why: We already showed the logarithm base two of n is at most n for every n at least one. Multiplying both sides of that inequality by n, which is positive, preserves the direction.
\[ \log_2 n \le n \ \Rightarrow \ n\log_2 n \le n\cdot n = n^2 \quad (n \ge 1) \]
Read off the witnesses
Why: The inequality above already has the exact form needed, with a constant of one in front.
\[ c = 1, \quad n_0 = 1 \]
Verify at n = 8
Why: At n equal to eight, n times its logarithm base two is eight times three equals twenty-four, and n squared is sixty-four. Twenty-four is indeed at most sixty-four.
\[ n=8:\ 8(3) = 24 \ \le \ 64\ \checkmark \]
Picture it
Animation
Shows: Each line of the worked example "Prove n log n is Big-O of n squared", appearing one at a time.
The same working the example does, in the order a tutor would write it.
Takeaway: At n equal to eight, n times its logarithm base two is eight times three equals twenty-four, and n squared is sixty-four. Twenty-four is indeed at most sixty-four.
Concept
For a fast, informal gut-check before writing a full proof, compare how each function reacts to doubling n: does it double as well, does it grow by a fixed amount, does it square, or does it multiply by a fixed factor?
A function that multiplies by a fixed factor greater than one every time n increases by one — exponential growth — will always eventually overtake a function that only multiplies by a fixed factor whenever n itself doubles or increases by one in a polynomial way. This heuristic is a sanity check, not a substitute for the formal definition when a real proof is required.
Step zero
Discussion prompt
Apply the heuristic: n cubed versus two to the n — before any calculation: what is the plan? Name the moves in order, in plain English, without doing the arithmetic.
Hint: It starts with: Apply the heuristic
Answer:
Worked example
Which grows faster in the long run: n cubed, or two raised to the n?
\[ n^3 \quad \text{vs.} \quad 2^n \]
Apply the heuristic
Why: n cubed is a polynomial: doubling n multiplies its value by roughly eight, a fixed factor. Two to the n is exponential: increasing n by just one at all multiplies its value by two, over and over, compounding endlessly. Exponentials with a fixed growth factor eventually beat any fixed power.
Check the trend with growing numbers
Why: At n equal to twenty, n cubed is eight thousand while two to the n is over one million. At n equal to thirty, n cubed is only twenty-seven thousand while two to the n has passed one billion. The exponential is pulling away, not catching up.
\[ n=20:\ 8{,}000 \ \text{vs.} \ 1{,}048{,}576 \qquad n=30:\ 27{,}000 \ \text{vs.} \ 1{,}073{,}741{,}824 \]
Verify the conclusion: n cubed is Big-O of two to the n, but not the reverse
Why: The ratio of n cubed to two to the n keeps shrinking toward zero as n grows, confirming n cubed is eventually dominated. Two to the n is not big-oh of n cubed, since no fixed constant multiple of n cubed can ever catch back up once the exponential has pulled ahead for good.
\[ n^3 = O(2^n), \qquad 2^n \ne O(n^3) \]
Picture it
Animation
Shows: Each line of the worked example "Apply the heuristic: n cubed versus two to the n", appearing one at a time.
The same working the example does, in the order a tutor would write it.
Takeaway: At n equal to twenty, n cubed is eight thousand while two to the n is over one million. At n equal to thirty, n cubed is only twenty-seven thousand while two to the n has passed one billion. The exponential is pulling away, not catching up.
Ranking
Put in order
Put the moves of Prove a constant function is Big-O of 1 into the order they have to happen.
Why: These are the moves of the worked example in the order it makes them, and each one is set up by the one before it. Since f never changes no matter what n is, the constant just needs to be big enough to cover its fixed value once and for all.
Worked example
Claim: the constant function seven is big-oh of one — the simplest possible Big-O statement.
\[ f(n) = 7, \qquad g(n) = 1 \]
Propose c = 7 and n0 = 1
Why: Since f never changes no matter what n is, the constant just needs to be big enough to cover its fixed value once and for all.
Check the inequality
Why: Seven is at most seven times one, for literally every value of n, since neither side depends on n at all.
\[ 7 \le 7\cdot 1 \quad \text{for all } n \]
Verify at any two values of n
Why: At n equal to one and at n equal to one thousand, f of n is still seven both times, and seven is at most seven both times — the bound holds trivially and forever, exactly as expected for a claim with no dependence on n.
\[ n=1:\ 7\le 7\ \checkmark \qquad n=1000:\ 7 \le 7\ \checkmark \]
Picture it
Animation
Shows: Each line of the worked example "Prove a constant function is Big-O of 1", appearing one at a time.
The same working the example does, in the order a tutor would write it.
Takeaway: Seven is at most seven times one, for literally every value of n, since neither side depends on n at all.
Intuition
It helps to picture big-oh of some bound not as a single formula but as a whole bucket, containing every function that stays under that bound past some threshold. Many different, unrelated-looking functions can share the very same bucket.
Three n squared plus five n plus two, seven n squared, and even plain n itself all belong to the same bucket, big-oh of n squared — they are all guaranteed not to outgrow a constant multiple of n squared. The bucket does not care how far below the ceiling a function actually sits, only that it never crosses it.
Pattern
1. Write down the two functions and which relationship you are proving
Why: Name f of n, g of n, and whether you are showing an upper bound, a lower bound, or both.
2. Bound every lower-order or extra term using the leading term
Why: Use the fact that for n past some small threshold, smaller powers and constants are dominated by the biggest power present.
3. Solve for a specific constant c (or c1 and c2 for a tight bound)
Why: Add up the coefficients from the bounding step to land on one committed number.
4. Solve for a specific threshold n0
Why: Find the point past which every inequality you used in step two actually holds — often n0 = 1 is enough.
5. Verify the inequality at n0 and at least one point beyond it
Why: Plug in real numbers to confirm the chain of algebra actually produces a true statement, catching arithmetic slips before you commit to the proof.
6. State the tightest true claim you actually proved
Why: If you proved both directions, report big-theta, not just big-oh — do not undersell a tight result.
Real world
Discussion prompt
Outside this lesson: where does Asymptotic Notation (Big-O, Big-Omega, Big-Theta) actually turn up? Name one concrete situation — a job, a piece of software someone ships, a decision somebody has to make — and say which part of The reusable recipe for any O, Omega, or Theta proof is doing the work in it.
Hint: Vague is the failure mode here. "Engineering" is not a situation; "deciding whether this build is fast enough to ship" is.
Answer:
That deck covers Big-O, Big-Omega, and Big-Theta notation as CS3000 uses them. It explains why we measure growth rather than exact time, gives the formal definitions in terms of a constant and a threshold, shows how to construct a Big-O proof by exhibiting a specific c and n0, and lays out the standard growth-rate hierarchy from constant to factorial. It targets four real misconceptions: treating Big-O as a synonym for worst case, treating constant-factor multiples as different classes, reporting a loose Big-O when a tight Big-Theta is actually known, and assuming that an algorithm with a larger Big-O is always slower on the small inputs you actually run.
Picture it
Animation
Shows: Theta is what you usually mean — a rendered Manim animation.
Rendered with Manim.
Takeaway: Say Theta when you know it; say O when you only have a ceiling.
Elimination
Eliminate the wrong options
What does the statement 'this algorithm's running time is big-oh of n squared' actually claim?
3 of these 4 are wrong. Strike them one at a time, and say what rules each one out before you strike the next. The survivor is the answer.
Survives elimination: A
Why: Big-O is purely a statement about an upper bound on growth: there exist a constant c and threshold n0 such that the running time is at most c times n squared for all n past n0. It says nothing about which case is being measured, or about exact equality.
Check
Suppose you are told an algorithm's running time is big-oh of n squared.
Check your understanding
What does the statement 'this algorithm's running time is big-oh of n squared' actually claim?
Answer: A
Why: Big-O is purely a statement about an upper bound on growth: there exist a constant c and threshold n0 such that the running time is at most c times n squared for all n past n0. It says nothing about which case is being measured, or about exact equality.
Prediction
Predict first
How should the two running times, two n and n, be classified using Big-O?
Answer it in your own words, now, with nothing to choose from. The options are on the next slide — and picking the right one off a list is an easier skill than producing it.
Correct: Both are big-oh of n; the factor of two is a constant multiplier absorbed into the definition's constant c.
Why: Big-O already allows any fixed positive constant multiplier inside its own definition, so a function that is a constant multiple of another, like two n compared to n, falls in exactly the same Big-O class rather than a separate, larger one.
Check
Two algorithms are timed on the same machine. One takes exactly two n steps to run; the other takes exactly n steps.
Check your understanding
How should the two running times, two n and n, be classified using Big-O?
Answer: A
Why: Big-O already allows any fixed positive constant multiplier inside its own definition, so a function that is a constant multiple of another, like two n compared to n, falls in exactly the same Big-O class rather than a separate, larger one.
Check
You want to prove that four n plus seven is big-oh of n, using the specific constant c equal to five.
\[ 4n+7 \ \le \ 5n \ ? \]
Check your understanding
Using c equal to five, what is the smallest threshold n0 for which four n plus seven is at most five n for every n at or above it?
Answer: A
Why: Solving four n plus seven is at most five n gives seven is at most n, so the inequality first becomes true exactly at n equal to seven, and stays true for every n from seven onward. Seven is the smallest valid threshold.
Commit first
Predict first
Which lists these four functions correctly from slowest-growing to fastest-growing for large n?
Commit to an answer, then rate it — certain, fairly sure, or guessing — and write the rating down before you turn the page.
Correct: log n, n, n log n, n squared
Why: The logarithm grows slowest, then plain n, then n multiplied by its own growing logarithm, then n squared, which is n multiplied by all of itself rather than just a logarithm — each step in this order eventually overtakes the one before it.
The rating matters as much as the answer: confident-and-wrong is the combination that survives revision, because nothing about it feels like it needs revisiting.
Check
Consider these four functions of n.
\[ n\log n, \quad \log n, \quad n^2, \quad n \]
Check your understanding
Which lists these four functions correctly from slowest-growing to fastest-growing for large n?
Answer: A
Why: The logarithm grows slowest, then plain n, then n multiplied by its own growing logarithm, then n squared, which is n multiplied by all of itself rather than just a logarithm — each step in this order eventually overtakes the one before it.
Prediction
Predict first
Given that both bounds above were already proven, which is the strongest true statement to report?
Answer it in your own words, now, with nothing to choose from. The options are on the next slide — and picking the right one off a list is an easier skill than producing it.
Correct: 5n squared + 2n + 1 is big-theta of n squared.
Why: Since both a matching lower bound and a matching upper bound with the same shape, n squared, were already established, the tightest and most informative true statement is the full big-theta claim, not just one weaker half of it.
Check
Earlier in this lesson, five n squared plus two n plus one was proven to be sandwiched between five n squared and eight n squared, for every n at least one.
\[ 5n^2 \ \le \ 5n^2+2n+1 \ \le \ 8n^2 \quad (n \ge 1) \]
Check your understanding
Given that both bounds above were already proven, which is the strongest true statement to report?
Answer: A
Why: Since both a matching lower bound and a matching upper bound with the same shape, n squared, were already established, the tightest and most informative true statement is the full big-theta claim, not just one weaker half of it.
Elimination
Eliminate the wrong options
For this specific input size of five items, what can you correctly conclude?
3 of these 4 are wrong. Strike them one at a time, and say what rules each one out before you strike the next. The survivor is the answer.
Survives elimination: A
Why: Big-O bounds describe what happens once n grows large enough past some threshold; they say nothing about which algorithm is faster at one small, specific input size, where hidden constants can easily flip the outcome.
Check
Algorithm A runs in a time that is big-oh of n squared. Algorithm B runs in a time that is big-oh of n log n. You need to process a list of exactly five items.
Check your understanding
For this specific input size of five items, what can you correctly conclude?
Answer: A
Why: Big-O bounds describe what happens once n grows large enough past some threshold; they say nothing about which algorithm is faster at one small, specific input size, where hidden constants can easily flip the outcome.
Connect it up
Draw it
One page, no notation unless you need it: draw how these connect — The reusable recipe for any O, Omega, or Theta proof · What n represents: the size of the input · What we track: running time as n grows · Race analogy: a head start doesn't matter over a marathon · Asymptotic means eventually, as n grows large. Put an arrow wherever one of them is what makes another possible, and label the arrow with why.
Recap
You now have the full toolkit for talking precisely about how running time grows.
| Symbol | What it claims |
|---|---|
| Big-O | upper bound: never grows faster than |
| Big-Omega | lower bound: never grows slower than |
| Big-Theta | tight bound: grows at exactly this rate |
Want this taught 1-on-1? Alexander tutors CS3000 Algorithms — $55/session, free consultation.