Lesson 9 completes Kleene's theorem by converting any finite automaton back into a regular expression. It explains why this direction cannot be done by structural recursion, then introduces generalized automata whose arrows carry expressions, the four conditions of normal form, and the state-elimination rip-out rule, including the starred self-loop term people drop. It works conversions, among them machines with dead states, shows how the elimination order controls the size of the result, and presents Kleene's original build-up recurrence and its kinship with Floyd-Warshall. It closes with the four now-equivalent definitions of a regular language.
Subject: Theory of Computation · 105 slides · symbolic lesson
Open the interactive version of this deck · Homework for this lesson
Title
Theory of Computation · Lesson 9
Rip the states out one at a time, and write on the arrows what each removed state used to allow. When two states remain, the label is the answer.
Objectives
Lesson 8 turned every expression into a machine. This lesson turns every machine back into an expression, completing Kleene's theorem. By the end you can:
Warm-up
Discussion prompt
Before we open DFA to Regular Expression: without looking back, what was the main idea of Regular Expressions, and what could you do by the end of it that you could not do before?
Hint: One sentence for the idea, one for the skill. If the second one is blank, that is the part to revisit.
Answer:
Lesson 8 introduces the notation that describes exactly the languages finite automata recognize. Covers the six-rule inductive syntax, the semantic clauses that assign a language to every expression, the star-then-concatenation-then-union precedence order, design idioms built around the starred alphabet, the algebraic laws including the identities, distribution and the star laws, and Thompson's construction converting any expression into a linear-size epsilon-NFA.
Section
Section 1
Concept
Kleene's theorem has two halves. Lesson 8 proved one of them by Thompson's construction, and the other is still outstanding.
| Direction | Status | Method |
|---|---|---|
| expression to machine | done, Lesson 8 | structural induction, one gadget per operator |
| machine to expression | this lesson | state elimination |
Until both are proved, 'regular' has two definitions that merely happen to share a name. Finishing this direction is what fuses them into one concept.
\[ \exists R : L = L(R) \quad \iff \quad \exists M : L = L(M) \]
Comparison
Comparison matrix
From What is still owed: refill the Status column from what you know. The rest of the table is as it appeared.
| Direction | Status | Method |
|---|---|---|
| expression to machine | done, Lesson 8 | structural induction, one gadget per operator |
| machine to expression | this lesson | state elimination |
Intuition
The first direction was easy because expressions are built inductively: there are leaves to start from and a root to finish at, so a recursion has somewhere to stand.
A machine has neither. A diagram with cycles has no leaves, no root, and no notion of a smaller sub-machine to recurse into. Splitting it into pieces does not obviously help, because the pieces are joined by arrows running in both directions.
So a different kind of argument is needed: not induction on the structure of the object, but induction on its size, removing one state at a time until nothing is left to remove.
\[ \text{induct on } |Q|, \text{ not on structure} \]
Counterexample
Discussion prompt
The first direction was easy because expressions are built inductively: there are leaves to start from and a root to finish at, so a recursion has somewhere to stand.
That is stated as though it always holds. Do one of two things: produce a case where it fails, or say precisely what rules such a case out. "It just does" is not on the menu.
Hint: Hunt at the extremes first — zero, one, negative, empty, equal. If every extreme survives, the reason they survive is the proof.
Concept
Deleting a state loses information — every path that used to run through it disappears. The trick is to write that information onto the surviving arrows before deleting.
If a path went from one state into the doomed state and out to another, then after deletion the arrow between those two must be relabelled to allow exactly the strings that detour used to allow.
Doing this needs arrow labels richer than single symbols — a label must be able to say 'any string of this shape'. That is precisely what a regular expression is, so the labels become expressions.
\[ \text{arrow labels: symbols} \;\longrightarrow\; \text{arrow labels: expressions} \]
Analogy
Discussion prompt
Explain The idea: absorb a state into the arrows around it by analogy to something with no Theory of Computation in it at all — a queue, a recipe, a map, a bank balance, whatever fits. Then say where your analogy breaks.
Hint: An analogy that never breaks is not an analogy, it is the same idea wearing a hat. Find the seam — that is the part that is actually new.
Answer:
Deleting a state loses information — every path that used to run through it disappears. The trick is to write that information onto the surviving arrows before deleting.
Concept
generalized finite automaton — A machine whose arrows are labelled with regular expressions rather than single symbols; taking an arrow consumes any string matching its label.
An ordinary machine is a special case: each of its arrows carries a one-symbol expression. So nothing is lost by moving to the generalized model, and a great deal of room is gained.
Acceptance is the natural extension. A string is accepted when it can be cut into consecutive pieces, each matching the label of the next arrow along some path from the start state to an accepting one.
\[ w = w_1w_2\cdots w_k, \quad w_i \in L(R_i) \text{ along a path} \]
Explain it
Discussion prompt
Explain Generalized machines to a student a year behind you. No notation, no jargon they have not met — and it still has to be true.
Hint: If your explanation needs a symbol they have never seen, you are describing the notation rather than the idea.
Answer:
An ordinary machine is a special case: each of its arrows carries a one-symbol expression. So nothing is lost by moving to the generalized model, and a great deal of room is gained.
Concept
The shift from symbols to expressions is a shift from one string per arrow to a whole set of them, and it changes what taking an arrow means.
In an ordinary machine, taking an arrow consumes exactly one symbol. In a generalized one, taking an arrow consumes any string belonging to the language its label denotes — which may be one symbol, many, or none at all.
\[ \text{take } i \xrightarrow{\,R\,} j \quad \text{consuming any } w \in L(R) \]
So a single arrow can now stand for infinitely many transitions of the original machine. That compression is exactly what makes it possible to shrink a whole graph down to one arrow.
Intuition
The two extra conditions on the start and accepting states look fussy. Both exist to guarantee the final answer is a single expression.
If the start state had an incoming arrow, the algorithm could not stop cleanly: after removing everything else there would still be a loop on the start state, and no single arrow to read the answer from.
If there were several accepting states, the machine would end with several surviving arrows and the answer would be their union — correct, but no longer a single label. Forcing exactly one of each makes the last step trivial.
\[ \text{two states left} \;\Rightarrow\; \text{one arrow} \;\Rightarrow\; \text{one expression} \]
Ranking
Put in order
Put the moves of Merge parallel arrows before eliminating into the order they have to happen.
Why: These are the moves of the worked example in the order it makes them, and each one is set up by the one before it. A machine over the alphabet of a and b often has two arrows between the same states, one per symbol, drawn separately.
Worked example
Normal form allows exactly one arrow per ordered pair, so a machine drawn with several must be tidied first.
Find a pair with more than one arrow
Why: A machine over the alphabet of a and b often has two arrows between the same states, one per symbol, drawn separately.
\[ i \xrightarrow{\,a\,} j \quad \text{and} \quad i \xrightarrow{\,b\,} j \]
Replace them with a single arrow
Why: The two arrows together permit a or b, so one arrow labelled with their union permits exactly the same transitions.
\[ i \xrightarrow{\,a \cup b\,} j \]
Repeat for every such pair
Why: Do this everywhere before eliminating anything. A machine over a k-symbol alphabet can have up to k parallel arrows per pair, all collapsing into one union.
Note why this is safe
Why: Union is exactly the semantics of 'either arrow may be taken', so no path is added and none is lost. The language is untouched.
Verify the merge on a concrete string
Why: Take a machine where the merged pair sits on the only path to the accepting state. A string using the a-arrow and a string using the b-arrow are both still accepted afterwards, since both match the union label, and a string using neither is still rejected.
\[ L(\text{merged}) = L(\text{original}) \ \checkmark \]
Concept
Before eliminating anything, the machine is put into a shape that makes the elimination rule uniform. Four conditions define it.
The third condition looks extravagant, but it is what makes the rule uniform: there is never a missing arrow to special-case. Where no transition should be possible, the arrow is labelled with the empty-set constant.
\[ \text{no transition} \;\longleftrightarrow\; \text{arrow labelled } \varnothing \]
Worked example
Take the two-state machine accepting the strings ending in 1, and convert it to normal form.
Figure (svg): Automaton with states q0, q1
Add a fresh start state
Why: The original start state has an incoming arrow from the accepting state, which normal form forbids. Add a new start with an arrow into the old one labelled with the empty-string constant.
\[ s \xrightarrow{\,\varepsilon\,} q_0 \]
Add a fresh accepting state
Why: The original accepting state has outgoing arrows, which normal form also forbids. Add a new accepting state, with an arrow into it from every old accepting state, labelled with the empty-string constant.
\[ q_1 \xrightarrow{\,\varepsilon\,} f \]
Fill in every missing arrow with the empty-set constant
Why: Between each ordered pair of the remaining states there must be exactly one arrow. Where the original had none, label it with the empty-set constant, which permits no string at all.
Merge parallel arrows with union
Why: If two arrows ran between the same pair, replace them with a single arrow labelled with the union of the two expressions. Here there were none, but the step matters on larger machines.
\[ R_1, R_2 \text{ in parallel} \;\longmapsto\; R_1 \cup R_2 \]
Verify the four conditions and that the language is unchanged
Why: One start with nothing incoming, one accepting with nothing outgoing, an arrow between every ordered pair, and the two special states distinct — all four hold. The added arrows consume the empty string, so no string's fate changed: the machine still accepts exactly the strings ending in 1.
\[ L(\text{normal form}) = L(M) \ \checkmark \]
Notation
Annotate
From Put a machine into normal form — read this one piece at a time. What is each part doing?
On: \( s \xrightarrow{\,\varepsilon\,} q_0 \)
Ranking
Put in order
These are the steps of How to reach normal form, scrambled. Put them back in order before the next slide shows you.
Why: This is the order the recipe itself gives. Recalling the sequence without the slide in front of you is the difference between recognising the method and being able to run it — most of what goes wrong in practice is a step done out of turn.
Pattern
Five mechanical steps, applicable to any machine, deterministic or not.
The result has two more states than the original, and its language is identical, because every arrow added consumes nothing.
Elimination
Eliminate the wrong options
In a generalized machine, what does an arrow labelled with the empty-set constant mean?
3 of these 4 are wrong. Strike them one at a time, and say what rules each one out before you strike the next. The survivor is the answer.
Survives elimination: A
Why: The empty-set constant denotes a language with no members, so no string at all matches the label and the arrow is unusable. Labelling absent transitions this way lets normal form claim an arrow between every pair without changing any behaviour.
Check
Think about what the empty-set label permits.
Check your understanding
In a generalized machine, what does an arrow labelled with the empty-set constant mean?
Answer: A
Why: The empty-set constant denotes a language with no members, so no string at all matches the label and the arrow is unusable. Labelling absent transitions this way lets normal form claim an arrow between every pair without changing any behaviour.
Section
Section 2
Concept
Pick any state other than the two special ones and delete it. Before it goes, every path that ran through it must be preserved on the surviving arrows.
Consider a pair of surviving states, one before and one after the doomed state. A path from the first to the second either avoided the doomed state entirely, or entered it, looped inside it any number of times, and left.
Those two possibilities are a union, and the looping is a star. That is the whole rule.
\[ R'_{ij} = R_{ij} \;\cup\; R_{i\,r}\big(R_{r\,r}\big)^{*}R_{r\,j} \]
Concept
Four labels feed into each new label, and each has a job. Naming them out loud is the fastest way to stop misapplying the rule.
| Term | Means |
|---|---|
| the direct label | paths that never touched the removed state |
| into the removed state | how to get from the source into it |
| its self-loop, starred | any number of trips around inside it |
| out of the removed state | how to get from it to the destination |
The starred self-loop is the term people drop. Omitting it silently assumes the removed state could be passed through at most once, which is wrong whenever it had a loop.
Trade off
Comparison matrix
From Reading the rule term by term: every row here is a choice with a cost. Fill the Means column, then say which row you would actually pick and what you give up for it.
| Term | Means |
|---|---|
| the direct label | paths that never touched the removed state |
| into the removed state | how to get from the source into it |
| its self-loop, starred | any number of trips around inside it |
| out of the removed state | how to get from it to the destination |
Anomaly
Predict first
A student writes this, and it looks reasonable:
Remove the middle state from a three-state chain in which the middle state has a self-loop on b.
It is wrong. Say what breaks — and say it before you turn the page.
Correct: A path enters the middle state on a and leaves on c, so the new label should just be those two joined in order.
Remove the middle state from a three-state chain in which the middle state has a self-loop on b.
Why: A path enters the middle state on a and leaves on c, so the new label should just be those two joined in order.
Trap
Remove the middle state from a three-state chain in which the middle state has a self-loop on b.
Compose the way in with the way out
Why: A path enters the middle state on a and leaves on c, so the new label should just be those two joined in order.
\[ R'_{ij} \overset{?}{=} ac \]
Check which strings the machine now accepts
Why: Only ac survives. The string abbc, which used to run into the middle state, loop twice on b, and leave, has no path at all.
\[ abbc \notin L(\text{after}) \quad \text{but} \quad abbc \in L(\text{before}) \]
Remove the middle state from a three-state chain in which the middle state has a self-loop on b.
Insert the starred self-loop between the way in and the way out
Why: Entering, looping any number of times, then leaving — the star covers zero loops as well as many, so the direct passage is still included.
\[ R'_{ij} = a\,b^{*}\,c \]
Check the same strings again
Why: The string ac matches with zero loops, and abbc matches with two. Every path the middle state used to permit is now spelled out by the label.
\[ ac,\ abc,\ abbc \in L(a\,b^{*}\,c) \ \checkmark \]
Translation
\( R'_{ij} = a\,b^{*}\,c \)
Draw it
Translate both ways. First write the expression above as a sentence with no symbols in it at all. Then cover it, and write your sentence back as notation. If the two versions disagree, the disagreement is the thing to fix.
Step zero
Discussion prompt
Rip one state out of a three-state machine — before any calculation: what is the plan? Name the moves in order, in plain English, without doing the arithmetic.
Hint: It starts with: Name the labels around the doomed state
Answer:
Worked example
Apply the rule once, on the smallest machine where it does real work.
Name the labels around the doomed state
Why: The start state reaches it on a, it loops on b, and it reaches the accepting state on c. The direct arrow from start to accepting is labelled with the empty-set constant, since no such transition existed.
\[ R_{ir} = a, \; R_{rr} = b, \; R_{rj} = c, \; R_{ij} = \varnothing \]
Substitute into the rule
Why: Take the direct label, union it with the way in, the starred loop, and the way out, in that order.
\[ R'_{ij} = \varnothing \;\cup\; a\,b^{*}\,c \]
Simplify with the identity law
Why: Unioning with the empty-set constant changes nothing, so the empty-set term drops out.
\[ R'_{ij} = a\,b^{*}\,c \]
Delete the state
Why: Two states remain: the start and the accepting one, joined by a single arrow carrying the derived expression.
Verify the derived expression against the original machine
Why: The machine accepted exactly the strings that read a, then any number of b's, then c. The expression denotes exactly those. Testing ac, abc and abbc gives accept in both, and testing ab and bc gives reject in both.
\[ ac,\ abc,\ abbc \in L \qquad ab,\ bc \notin L \ \checkmark \]
Picture it
Animation
Shows: Each line of the worked example "Rip one state out of a three-state machine", appearing one at a time.
The same working the example does, in the order a tutor would write it.
Takeaway: The machine accepted exactly the strings that read a, then any number of b's, then c. The expression denotes exactly those. Testing ac, abc and abbc gives accept in both, and testing ab and bc gives reject in both.
Concept
The rule is not a heuristic; it is exactly the set of paths, rewritten. The argument is short enough to give in full.
Fix two surviving states. Every path between them, in the machine before deletion, either visits the doomed state or does not. Those two cases are disjoint and together exhaust the possibilities, so their union is everything.
A path that does visit it enters once, spends some number of complete loops inside, and leaves once. That decomposition is unique, so the concatenation with a star counts each such path exactly once.
\[ \text{paths} = \text{avoiding} \;\cup\; \text{in} \cdot \text{loops}^{*} \cdot \text{out} \]
Since every pair's label is updated simultaneously, no path is lost and none is invented. The language is unchanged, which is what an elimination step must guarantee.
Concept
The rule must be applied to all ordered pairs of surviving states, and two of those cases are easy to skip by accident.
The self-pair is the one most often forgotten, and it matters whenever a cycle ran through the removed state and back.
\[ R'_{ii} = R_{ii} \;\cup\; R_{i\,r}\big(R_{r\,r}\big)^{*}R_{r\,i} \]
With n states remaining after the removal, that is n squared labels to rewrite. Doing them in a table rather than on the diagram is far more reliable.
Intuition
Everything about the rip-out rule falls out of one question: what do the paths between two states look like, sorted by whether they touch the doomed state?
| Kind of path | Expression term |
|---|---|
| never touches the removed state | the direct label |
| touches it exactly once | in, then out |
| touches it several times | in, loops, then out |
The last two rows collapse into one, because star covers one loop, many loops and none. Union the result with the first row and the rule is complete — there is no fourth case, because a path either touches the state or it does not.
Pattern
The full procedure, from an ordinary machine to a single expression.
Step three is the one that must not be shortened. Every ordered pair needs updating, including pairs where the label was the empty-set constant and pairs from a state to itself.
Prediction
Predict first
When ripping out a state r, which part of the new label accounts for r's self-loop?
Answer it in your own words, now, with nothing to choose from. The options are on the next slide — and picking the right one off a list is an easier skill than producing it.
Correct: The starred middle factor, between the way in and the way out
Why: A path may go round r's self-loop any number of times, including none. Star is exactly the operator meaning zero or more repetitions, and it sits between the label entering r and the label leaving it.
Check
Recall which of the four terms handles repetition.
Check your understanding
When ripping out a state r, which part of the new label accounts for r's self-loop?
Answer: A
Why: A path may go round r's self-loop any number of times, including none. Star is exactly the operator meaning zero or more repetitions, and it sits between the label entering r and the label leaving it.
Section
Section 3
Picture it
Figure (svg): Automaton with states q0, q1
Discussion prompt
Read the picture before the words. What is this showing, and what is the one thing it is built to make obvious? Commit to an answer, then read on.
Hint: Name the parts, then say what changes between them — and if nothing changes, say what is being held still.
Answer:
Run the full algorithm on the two-state machine from Section 1, whose normal form has four states.
Worked example
Run the full algorithm on the two-state machine from Section 1, whose normal form has four states.
Figure (svg): Automaton with states q0, q1
Rip out the original accepting state first
Why: Its self-loop is labelled 1. Paths from the original start state into it and back out to itself become the loop label, and paths from it to the fresh accepting state become the exit.
\[ R_{q_0\,q_1} = 1, \; R_{q_1\,q_1} = 1, \; R_{q_1\,q_0} = 0, \; R_{q_1\,f} = \varepsilon \]
Update the start-to-accepting label
Why: There was no direct arrow, so the direct term is the empty-set constant. The detour is: reach the removed state on 1, loop on 1 any number of times, then exit on the empty string.
\[ R'_{q_0\,f} = \varnothing \cup 1\,1^{*}\,\varepsilon = 1\,1^{*} \]
Update the start state's self-loop
Why: The original start state had a self-loop on 0. It could also detour through the removed state and come back on 0, so the new self-loop is the union of the two.
\[ R'_{q_0\,q_0} = 0 \;\cup\; 1\,1^{*}\,0 \]
Rip out the original start state
Why: Now only the two fresh states remain. Enter on the empty string, loop on the label just derived, then exit on the start-to-accepting label.
\[ R = \varepsilon\,\big(0 \cup 1\,1^{*}\,0\big)^{*}\,1\,1^{*} \]
Verify the simplified expression against the machine
Why: Dropping the leading empty string and noting that a nonempty run of 1s is just 1 followed by any number of 1s, the expression says: any number of blocks each ending in 0, then a final run of 1s. Testing 1, 01, 11 and 101 gives accept in both machine and expression; testing the empty string, 0 and 10 gives reject in both.
\[ 1,\ 01,\ 11,\ 101 \in L \qquad \varepsilon,\ 0,\ 10 \notin L \ \checkmark \]
Reverse engineer
Discussion prompt
Work backwards. The example finished here:
Verify the simplified expression against the machine
What was it asked to do, and what must it have been given? Reconstruct the problem from its answer.
Hint: Every quantity in the result had to enter somewhere. Account for each one.
Answer:
Run the full algorithm on the two-state machine from Section 1, whose normal form has four states.
Intuition
These derivations produce long expressions, and a sign error is easy to make and hard to see. Three cheap checks catch nearly everything.
Do these before simplifying, not after. Simplification errors and derivation errors look identical in the final expression, and checking early separates them.
Picture it
Figure (svg): Automaton with states p0, p1, p2
Discussion prompt
Read the picture before the words. What is this showing, and what is the one thing it is built to make obvious? Commit to an answer, then read on.
Hint: Name the parts, then say what changes between them — and if nothing changes, say what is being held still.
Answer:
Now a machine with a genuine branch: the ends-in-01 machine from Lesson 4.
Worked example
Now a machine with a genuine branch: the ends-in-01 machine from Lesson 4.
Figure (svg): Automaton with states p0, p1, p2
Reach normal form
Why: Add a fresh start with an empty-string arrow into the original start, and a fresh accepting state with an empty-string arrow in from the original accepting state. Five states.
Choose an elimination order deliberately
Why: Eliminate the middle state first. It has the fewest arrows passing through it, so the labels it generates stay short — an ordering choice examined properly in Section 5.
Rip out the middle state
Why: It has no self-loop, so the starred factor is the star of the empty-set constant, which is the empty string, and it disappears. Each in-out pair simply concatenates.
\[ \varnothing^{*} = \varepsilon \;\Rightarrow\; R'_{ij} = R_{ij} \cup R_{ir}R_{rj} \]
Rip out the remaining two original states
Why: Each removal follows the same rule, with self-loops now present and therefore starred. The labels grow, which is normal.
Verify the final expression on four strings
Why: The derived expression must accept 01, 101 and 0101, and reject 010 and the empty string. Checking each against both the expression and the diagram gives matching verdicts on all five, so the derivation is sound.
\[ 01,\ 101,\ 0101 \in L \qquad 010,\ \varepsilon \notin L \ \checkmark \]
Notation
Annotate
From Convert a three-state machine — read this one piece at a time. What is each part doing?
On: \( 01,\ 101,\ 0101 \in L \qquad 010,\ \varepsilon \notin L \ \checkmark \)
Step zero
Discussion prompt
Convert a machine that accepts the empty string — before any calculation: what is the plan? Name the moves in order, in plain English, without doing the arithmetic.
Hint: It starts with: Set up the machine
Answer:
Worked example
When the original start state is accepting, the derived expression must contain the empty string. Watch where it comes from.
Set up the machine
Why: Take a machine whose start state is accepting and which loops back to itself on the block ab. Its language is any number of copies of that block.
Reach normal form
Why: The fresh start gets an empty-string arrow into the old start, and the old start — being accepting — gets an empty-string arrow into the fresh accepting state.
\[ s \xrightarrow{\,\varepsilon\,} q_0 \xrightarrow{\,\varepsilon\,} f \]
Rip out the old start state
Why: It has a self-loop labelled with the block, so the starred factor is that block starred. The way in and the way out are both the empty string.
\[ R = \varepsilon\,(ab)^{*}\,\varepsilon = (ab)^{*} \]
Read off where the empty string came from
Why: The star supplies it, through the zero-repetition case. Had the old start state not been accepting, there would have been no empty-string arrow out of it and the expression would have been the block starred followed by something else.
Verify the boundary case against the machine
Why: The machine accepts the empty string, since its start state is accepting and no arrow need be taken. The expression accepts it too, by taking zero copies. Both also accept ab and abab and reject aba, so they agree on the boundary and beyond.
\[ \varepsilon,\ ab,\ abab \in L \qquad aba \notin L \ \checkmark \]
Picture it
Animation
Shows: Each line of the worked example "Convert a machine that accepts the empty string", appearing one at a time.
The same working the example does, in the order a tutor would write it.
Takeaway: The machine accepts the empty string, since its start state is accepting and no arrow need be taken. The expression accepts it too, by taking zero copies. Both also accept ab and abab and reject aba, so they agree on the boundary and beyond.
Concept
A derived expression is correct but rarely pretty. Reading it aloud with the Lesson 8 recipe turns it back into a description you can check against the machine.
If the sentence and the machine's design notes disagree, one of them is wrong — and finding out which is far faster than re-deriving the expression from scratch.
Commit first
Predict first
The derived expression was (0 ∪ 110) 11*. What does its final factor guarantee?
Commit to an answer, then rate it — certain, fairly sure, or guessing — and write the rating down before you turn the page.
Correct: The string ends with at least one 1
Why: The final factor is a single 1 concatenated with any number of further 1s, so it contributes a nonempty run of 1s at the very end. Since nothing follows it, the last symbol of any matching string is a 1 — which is exactly the machine's language.
The rating matters as much as the answer: confident-and-wrong is the combination that survives revision, because nothing about it feels like it needs revisiting.
Check
This is the expression Section 3 derived for the ends-in-1 machine.
Check your understanding
The derived expression was (0 ∪ 110) 11*. What does its final factor guarantee?
Answer: A
Why: The final factor is a single 1 concatenated with any number of further 1s, so it contributes a nonempty run of 1s at the very end. Since nothing follows it, the last symbol of any matching string is a 1 — which is exactly the machine's language.
Concept
Two simplifications fire constantly during these derivations, and knowing them by sight halves the work.
\[ \varnothing^{*} = \varepsilon, \qquad R\,\varepsilon = R, \qquad R \cup \varnothing = R, \qquad R\,\varnothing = \varnothing \]
| Situation | Simplification |
|---|---|
| removed state has no self-loop | the starred factor becomes the empty string and vanishes |
| no direct arrow between the pair | the union term vanishes |
| no way into the removed state | the whole detour term vanishes |
| arrow labelled with the empty string | it disappears from a concatenation |
Applying these as you go, rather than at the end, is the difference between a two-line label and a twenty-line one.
Estimation
Predict first
Machines built by the subset construction usually carry a dead state. Watch what the algorithm does with it.
Commit before you compute: what does Convert a machine with a dead state come out to? A rough magnitude and the right form is enough — the point is to have something concrete to be wrong about.
Correct: Verify that removing it early gives the same expression
Why: A prediction you can defend turns the computation into a check rather than a leap of faith — and an answer that contradicts it is caught on the spot. Running the algorithm with and without the dead state present produces identical labels at every step, because its contribution was the empty-set constant throughout.
Worked example
Machines built by the subset construction usually carry a dead state. Watch what the algorithm does with it.
Identify the dead state
Why: It is non-accepting and every arrow out of it returns to itself. No path from it ever reaches an accepting state.
Rip it out and watch the terms
Why: For any pair, the detour term requires a way in, a starred loop, and a way out to the destination. Every arrow from the dead state to any other state is labelled with the empty-set constant.
\[ R_{r\,j} = \varnothing \;\Rightarrow\; R_{ir}\big(R_{rr}\big)^{*}R_{rj} = \varnothing \]
Apply the annihilator law
Why: Concatenating anything with the empty-set constant gives the empty-set constant, so the entire detour term collapses and only the direct label survives.
\[ R'_{ij} = R_{ij} \cup \varnothing = R_{ij} \]
Note the practical consequence
Why: A dead state contributes nothing at all to the final expression. Deleting it before starting is safe and saves an entire elimination round.
Verify that removing it early gives the same expression
Why: Running the algorithm with and without the dead state present produces identical labels at every step, because its contribution was the empty-set constant throughout. So the shortcut is sound, not merely convenient.
\[ R_{\text{with dead}} = R_{\text{without dead}} \ \checkmark \]
Picture it
Animation
Shows: Each line of the worked example "Convert a machine with a dead state", appearing one at a time.
The same working the example does, in the order a tutor would write it.
Takeaway: Running the algorithm with and without the dead state present produces identical labels at every step, because its contribution was the empty-set constant throughout. So the shortcut is sound, not merely convenient.
Prediction
Predict first
You rip out a state that has no self-loop. What happens to the starred middle factor?
Answer it in your own words, now, with nothing to choose from. The options are on the next slide — and picking the right one off a list is an easier skill than producing it.
Correct: It becomes the empty-string constant and drops out of the concatenation
Why: No self-loop means the loop's label is the empty-set constant, and starring that gives the language holding just the empty string. Concatenating with the empty string changes nothing, so the factor disappears and the detour is simply the way in followed by the way out.
Check
Consider what the star of the empty-set constant denotes.
Check your understanding
You rip out a state that has no self-loop. What happens to the starred middle factor?
Answer: A
Why: No self-loop means the loop's label is the empty-set constant, and starring that gives the language holding just the empty string. Concatenating with the empty string changes nothing, so the factor disappears and the detour is simply the way in followed by the way out.
Section
Section 4
Concept
Any elimination order yields a correct expression, because every step preserves the language. But different orders yield wildly different expressions.
This is not a defect. Many expressions denote the same language, and the algorithm simply produces one of them — the one its choices led to.
\[ R_1 \neq R_2 \text{ as strings}, \quad L(R_1) = L(R_2) \]
So two students converting the same machine can both be right and disagree completely. Checking agreement means comparing languages, not comparing symbols.
Intuition
Each removal replaces the labels on every pair that could route through the removed state, so the growth is driven by how many such pairs there are.
A state with few incoming and few outgoing arrows creates few new terms. A hub with arrows from and to everything creates the most, so removing hubs early is expensive.
\[ \text{new terms} \approx \text{in-degree} \times \text{out-degree} \]
The practical rule: remove the states with the smallest product of in-degree and out-degree first, and leave the hubs until the graph around them has already shrunk.
Ranking
Put in order
Put the moves of Compare two elimination orders on the same machine into the order they have to happen.
Why: These are the moves of the worked example in the order it makes them, and each one is set up by the one before it. A start state, two leaf states each connected only to the hub, and the hub connected to everything including the accepting state.
Worked example
Take a four-state machine with one hub, and convert it twice.
Describe the machine
Why: A start state, two leaf states each connected only to the hub, and the hub connected to everything including the accepting state.
Order one: remove the hub first
Why: Every pair of remaining states could route through the hub, so every one of their labels gains a detour term at once. The labels triple in length in a single step.
\[ \text{pairs updated} = 3 \times 3 = 9 \]
Order two: remove the leaves first
Why: Each leaf has one arrow in and one out, so removing it updates exactly one pair's label, and the new term is short.
\[ \text{pairs updated} = 1 \times 1 = 1 \text{ per leaf} \]
Compare the results
Why: Both expressions denote the same language. The second is markedly shorter, because the hub was removed after the graph around it had already been simplified.
Verify that both expressions accept the same strings
Why: Testing the shortest member, the empty string and one near miss against both derived expressions gives identical verdicts in every case — as the correctness argument guarantees, since every elimination step preserves the language regardless of order.
\[ L(R_{\text{order 1}}) = L(R_{\text{order 2}}) \ \checkmark \]
Picture it
Animation
Shows: Each line of the worked example "Compare two elimination orders on the same machine", appearing one at a time.
The same working the example does, in the order a tutor would write it.
Takeaway: Testing the shortest member, the empty string and one near miss against both derived expressions gives identical verdicts in every case — as the correctness argument guarantees, since every elimination step preserves the language regardless of order.
Constraint
Discussion prompt
Run Keeping the expression small with this step confiscated:
Simplify each label the moment it is written, using the identity and annihilator laws.
Is it still possible? If it is, say what takes its place and what it costs you. If it is not, say exactly what that step was providing that nothing else does.
Hint: A step you can drop for free was never load-bearing. If you cannot drop it, name the thing that goes wrong the moment it is gone.
Answer:
Pattern
The algorithm gives no control over correctness — that is automatic — but plenty over size.
Even with all of these, the expression can still be much larger than the machine. The next section explains why that is unavoidable.
Edge cases
Discussion prompt
Keeping the expression small works on the cases you have just seen. Push it to the edge: what is the most degenerate input it still handles — empty, zero, one item, everything equal — and what is the first case where it stops being true? Name the case, not just "it breaks".
Hint: Try the smallest legal input, then the largest, then the one where two things collide. Methods are specified at their edges; the middle takes care of itself.
Answer:
The algorithm gives no control over correctness — that is automatic — but plenty over size.
Concept
Each elimination round can roughly triple the length of a label, and there are as many rounds as there are states to remove.
\[ |R| \;=\; 2^{\Theta(|Q|)} \quad \text{in the worst case} \]
So the conversion is exponential in the worst case, and this is not an artefact of the algorithm: there are languages whose smallest machine is small and whose smallest expression is exponentially larger.
| Conversion | Cost |
|---|---|
| expression to machine | linear, Lesson 8 |
| machine to DFA | exponential, Lesson 6 |
| machine to expression | exponential, this lesson |
Comparison
Comparison matrix
From The size of the result: refill the Cost column from what you know. The rest of the table is as it appeared.
| Conversion | Cost |
|---|---|
| expression to machine | linear, Lesson 8 |
| machine to DFA | exponential, Lesson 6 |
| machine to expression | exponential, this lesson |
Intuition
Three conversions and two exponentials — it is worth knowing which direction is cheap and why.
Going from an expression to a machine is cheap because each operator maps to a fixed gadget: the structure of the expression is the structure of the machine, so nothing has to be discovered.
Going the other way, the expression must describe every path through a graph, and a graph with cycles has infinitely many paths described by finitely many nested stars. Compressing that back into a formula is where the cost appears.
\[ \text{structure known} \;\Rightarrow\; \text{cheap} \qquad \text{structure discovered} \;\Rightarrow\; \text{costly} \]
Section
Section 5
Step zero
Discussion prompt
Derive the expression for a pure cycle — before any calculation: what is the plan? Name the moves in order, in plain English, without doing the arithmetic.
Hint: It starts with: Set up a two-state cycle
Answer:
Worked example
Cycles are where stars come from, so it is worth doing one in isolation.
Set up a two-state cycle
Why: A start state reaches a second state on a, and the second returns on b. The start state is the only accepting state.
Reach normal form and rip out the second state
Why: The second state has no self-loop, so its starred factor is the empty string. The way in is a and the way out is b.
\[ R'_{q_0\,q_0} = \varnothing \;\cup\; a\,\varnothing^{*}\,b = ab \]
Rip out the original start state
Why: Its self-loop is now the block just derived, and both the entry and the exit are empty-string arrows.
\[ R = \varepsilon\,(ab)^{*}\,\varepsilon = (ab)^{*} \]
See where the star came from
Why: The star is the trace of the cycle. Every cycle in the machine becomes a starred factor in the expression, and a machine with no cycles produces an expression with no stars at all — hence a finite language.
Verify by testing three round-trip counts
Why: Zero trips gives the empty string, one gives ab, two gives abab — all accepted by both the machine and the expression. The half-trip a is rejected by both, since the cycle must complete to return to the accepting state.
\[ \varepsilon,\ ab,\ abab \in L \qquad a \notin L \ \checkmark \]
Picture it
Animation
Shows: Each line of the worked example "Derive the expression for a pure cycle", appearing one at a time.
The same working the example does, in the order a tutor would write it.
Takeaway: Zero trips gives the empty string, one gives ab, two gives abab — all accepted by both the machine and the expression. The half-trip a is rejected by both, since the cycle must complete to return to the accepting state.
Intuition
Once the correspondence is noticed it makes derived expressions much easier to read and to check.
| In the machine | In the expression |
|---|---|
| a simple cycle | one starred factor |
| a cycle inside a cycle | a star inside a star |
| a branch point | a union |
| a path with no choices | a concatenation |
| no cycles at all | no stars — a finite language |
The last row is a useful quick check: if the machine has no cycle on any path from the start to an accepting state, the language is finite, and any derived expression containing a star must be wrong.
Concept
State elimination is the modern presentation. Kleene's own proof went the other way: instead of removing states, it grows the set of states a path is allowed to pass through.
Define an expression for the strings that take the machine from one state to another while passing only through states drawn from the first k of them. Start with k equal to zero and increase it.
\[ R^{(k)}_{ij} = \text{strings from } i \text{ to } j \text{ using intermediates among the first } k \]
When k reaches the total number of states, no restriction remains, and the answer is the union of those expressions over all accepting destinations.
Concept
Allowing one more intermediate state gives the same case split as the rip-out rule, read forwards instead of backwards.
\[ R^{(k)}_{ij} = R^{(k-1)}_{ij} \;\cup\; R^{(k-1)}_{ik}\big(R^{(k-1)}_{kk}\big)^{*}R^{(k-1)}_{kj} \]
Either the path does not use the newly allowed state, or it does — reaching it, looping inside the already-allowed region any number of times, and leaving. Exactly the same union, concatenation and star.
The base case is the machine itself: with no intermediates allowed, the only paths are single arrows, plus the empty string when the two states coincide.
\[ R^{(0)}_{ij} = \bigcup \{\, a : \delta(i,a) = j \,\} \;\cup\; (\varepsilon \text{ if } i = j) \]
Step zero
Discussion prompt
Run the build-up method on a two-state machine — before any calculation: what is the plan? Name the moves in order, in plain English, without doing the arithmetic.
Hint: It starts with: Write the base case
Answer:
Worked example
Apply the recurrence to the ends-in-1 machine, to see it produce the same kind of answer.
Write the base case
Why: With no intermediates allowed, each label is just the symbols on the direct arrow, plus the empty string on the diagonal.
\[ R^{(0)}_{00} = 0 \cup \varepsilon, \; R^{(0)}_{01} = 1, \; R^{(0)}_{10} = 0, \; R^{(0)}_{11} = 1 \cup \varepsilon \]
Allow the first state as an intermediate
Why: Apply the recurrence with the start state newly permitted. Paths may now loop through it before continuing.
\[ R^{(1)}_{01} = R^{(0)}_{01} \cup R^{(0)}_{00}\big(R^{(0)}_{00}\big)^{*}R^{(0)}_{01} \]
Simplify
Why: The starred term absorbs the leading copy, since a language unioned with the empty string, starred, is just its star. So the label reduces to a run of zeros followed by a 1.
\[ R^{(1)}_{01} = 0^{*}\,1 \]
Allow the second state too, and read off the answer
Why: The final label from the start state to the accepting state is the expression for the whole machine.
\[ R^{(2)}_{01} = \big(0 \cup 1\,1^{*}\,0\big)^{*}\,1\,1^{*} \]
Verify the result matches the elimination answer
Why: This is the same expression state elimination produced in Section 3, up to the leading empty string dropped there. Both methods accept 1, 01 and 101 and reject the empty string, 0 and 10 — as they must, since both compute the same language.
\[ L(R^{(2)}_{01}) = \{w : w \text{ ends in } 1\} \ \checkmark \]
Picture it
Animation
Shows: Each line of the worked example "Run the build-up method on a two-state machine", appearing one at a time.
The same working the example does, in the order a tutor would write it.
Takeaway: This is the same expression state elimination produced in Section 3, up to the leading empty string dropped there. Both methods accept 1, 01 and 101 and reject the empty string, 0 and 10 — as they must, since both compute the same language.
Intuition
Elimination and build-up look different on the page and are the same idea viewed from two ends.
| State elimination | Build-up | |
|---|---|---|
| direction | shrink the graph | grow the permitted set |
| invariant | labels describe all paths in the current graph | labels describe paths using the first k states |
| step | remove one state, relabel | permit one state, relabel |
| formula | the same union with a starred middle | the same union with a starred middle |
The recurrence is also the shape of the Floyd-Warshall algorithm for shortest paths, with union in place of minimum and concatenation in place of addition. That is not a coincidence: both are computing a closure over paths in a graph.
Elimination
Eliminate the wrong options
What is the base case of the build-up method, before any intermediate states are permitted?
3 of these 4 are wrong. Strike them one at a time, and say what rules each one out before you strike the next. The survivor is the answer.
Survives elimination: A
Why: With no intermediates allowed, a path from one state to another can only be a single arrow. When the two states coincide, the path of length zero is also available, which contributes the empty string.
Check
Both compute the same thing by opposite routes.
Check your understanding
What is the base case of the build-up method, before any intermediate states are permitted?
Answer: A
Why: With no intermediates allowed, a path from one state to another can only be a single arrow. When the two states coincide, the path of length zero is also available, which contributes the empty string.
Section
Section 6
Pattern
Both produce a correct expression. Pick by what you are doing.
The last line is worth taking seriously. Comparing two expressions symbol by symbol proves nothing, since many expressions denote one language.
Concept
The recurrence fills a three-dimensional table, so its running cost is easy to state exactly.
\[ |Q|^{3} \text{ table entries, each a union of four pieces} \]
That is polynomial in the number of states — but each entry is an expression, and the expressions themselves grow. Every level of the recurrence can triple a label's length, so the final expression is still exponential in the worst case.
\[ \text{entries: polynomial} \qquad \text{expression length: exponential} \]
Both methods therefore share the same worst case, which is reassuring: the blowup lives in the answer, not in either algorithm's cleverness.
Concept
With this lesson's construction in hand, the theorem is fully proved.
\[ \text{DFA} \;\equiv\; \text{NFA} \;\equiv\; \varepsilon\text{-NFA} \;\equiv\; \text{regular expression} \]
Four definitions, one class. From here on, showing a language is regular means producing whichever of the four is easiest, with no further justification required.
Explain it
Discussion prompt
Explain Both directions, at last to a student a year behind you. No notation, no jargon they have not met — and it still has to be true.
Hint: If your explanation needs a symbol they have never seen, you are describing the notation rather than the idea.
Answer:
With this lesson's construction in hand, the theorem is fully proved.
Ranking
Put in order
Put the moves of Use the theorem to prove a closure property in one line into the order they have to happen.
Why: These are the moves of the worked example in the order it makes them, and each one is set up by the one before it. By the theorem, each is denoted by some regular expression.
Worked example
The equivalence turns some closure proofs from constructions into observations. Prove the regular languages are closed under concatenation, using expressions.
Take two regular languages
Why: By the theorem, each is denoted by some regular expression. Name them.
\[ L_1 = L(R_1), \qquad L_2 = L(R_2) \]
Write the expression for the concatenation
Why: Concatenation is one of the three operators the notation is built from, so the concatenation of the two expressions is itself a regular expression.
\[ R = R_1R_2 \]
Read off its language
Why: By the semantic clause for concatenation, the language of that expression is the concatenation of the two languages.
\[ L(R) = L(R_1)L(R_2) = L_1L_2 \]
Conclude
Why: A regular expression denotes the target language, so by the theorem the target language is regular. No machine was built at all.
Verify this agrees with the Lesson 7 construction
Why: Lesson 7 proved the same result by wiring two machines with free moves and demoting the first accepting set. Both proofs establish exactly the same statement; this one is shorter because the theorem did the work of translating between formalisms.
\[ L_1L_2 \text{ regular} \ \checkmark \]
Picture it
Animation
Shows: Each line of the worked example "Use the theorem to prove a closure property in one line", appearing one at a time.
The same working the example does, in the order a tutor would write it.
Takeaway: Lesson 7 proved the same result by wiring two machines with free moves and demoting the first accepting set. Both proofs establish exactly the same statement; this one is shorter because the theorem did the work of translating between formalisms.
Concept
A small corollary worth banking, because it settles a surprising number of exercises immediately.
A finite language holds finitely many strings. Write each one as a concatenation of its symbols — a regular expression — and union them all together. The union of finitely many expressions is an expression.
\[ L = \{w_1, \dots, w_k\} \;\Rightarrow\; R = w_1 \cup w_2 \cup \cdots \cup w_k \]
So every finite language is regular, including the empty language and the language holding only the empty string. The converse fails badly: most regular languages are infinite.
The useful contrapositive: if a language is not regular, it is certainly infinite. So every non-regularity proof in Lesson 11 must exhibit unboundedly long strings, and one that argues about a finite set has gone wrong.
Analogy
Discussion prompt
Explain Every finite language is regular, and the proof is now one line by analogy to something with no Theory of Computation in it at all — a queue, a recipe, a map, a bank balance, whatever fits. Then say where your analogy breaks.
Hint: An analogy that never breaks is not an analogy, it is the same idea wearing a hat. Find the seam — that is the part that is actually new.
Answer:
A small corollary worth banking, because it settles a surprising number of exercises immediately.
Intuition
Four descriptions of one class, each strong at a different job — but all four have the same limitation.
| Definition | Good for |
|---|---|
| deterministic machine | running fast, complementing |
| nondeterministic machine | designing by guess-and-verify |
| machine with free moves | wiring machines together |
| regular expression | writing, communicating, searching |
What they share is that each proves membership by producing an object. That is why none of them can prove a language irregular: there is no object to produce, only infinitely many to rule out.
Trade off
Comparison matrix
From The four definitions, and what they share: every row here is a choice with a cost. Fill the Good for column, then say which row you would actually pick and what you give up for it.
| Definition | Good for |
|---|---|
| deterministic machine | running fast, complementing |
| nondeterministic machine | designing by guess-and-verify |
| machine with free moves | wiring machines together |
| regular expression | writing, communicating, searching |
Intuition
Having four equivalent definitions is only useful if you pick well. The choice is usually obvious once the question is classified.
| Task | Best formalism |
|---|---|
| prove closure under a regular operation | expressions |
| prove closure under complement or intersection | deterministic machines |
| design something for a described language | nondeterministic machines, or expressions |
| run it fast on many inputs | deterministic machines |
| prove a language is not regular | none of them — Lessons 10 and 11 |
The last row is the important one. Every formalism here is good at showing a language is regular, by exhibiting an object. None of them is any help at showing a language is not, because that requires ruling out all objects at once.
Comparison
Comparison matrix
From Which formalism to reach for: refill the Best formalism column from what you know. The rest of the table is as it appeared.
| Task | Best formalism |
|---|---|
| prove closure under a regular operation | expressions |
| prove closure under complement or intersection | deterministic machines |
| design something for a described language | nondeterministic machines, or expressions |
| run it fast on many inputs | deterministic machines |
| prove a language is not regular | none of them — Lessons 10 and 11 |
Concept
Four equivalent definitions describe the class from the inside. Nothing so far describes it from the outside.
Every technique in Lessons 4 to 9 proves membership by construction: build a machine, or write an expression. To prove a language is not regular, none of that helps — there is no way to exhaust all possible machines by trying them.
What is needed is a property that every regular language must have, so that exhibiting a language lacking it settles the matter. Lesson 10 supplies exactly that, and Lesson 6 already hinted at its shape when counting distinguishable prefixes.
\[ \text{every regular language has property } P \;\Rightarrow\; \lnot P(L) \text{ means } L \text{ not regular} \]
Counterexample
Discussion prompt
Four equivalent definitions describe the class from the inside. Nothing so far describes it from the outside.
That is stated as though it always holds. Do one of two things: produce a case where it fails, or say precisely what rules such a case out. "It just does" is not on the menu.
Hint: Hunt at the extremes first — zero, one, negative, empty, equal. If every extreme survives, the reason they survive is the proof.
Ranking
Put in order
These are the steps of Converting a machine to an expression, end to end, scrambled. Put them back in order before the next slide shows you.
Why: This is the order the recipe itself gives. Recalling the sequence without the slide in front of you is the difference between recognising the method and being able to run it — most of what goes wrong in practice is a step done out of turn.
Pattern
The complete routine, collected.
When two states remain, read the label off the single surviving arrow. That expression denotes the machine's language, and by Kleene's theorem the two objects are interchangeable from then on.
Real world
Discussion prompt
Outside this lesson: where does DFA to Regular Expression actually turn up? Name one concrete situation — a job, a piece of software someone ships, a decision somebody has to make — and say which part of Converting a machine to an expression, end to end is doing the work in it.
Hint: Vague is the failure mode here. "Engineering" is not a situation; "deciding whether this build is fast enough to ship" is.
Answer:
Lesson 9 completes Kleene's theorem by converting any finite automaton back into a regular expression. It explains why this direction cannot be done by structural recursion, then introduces generalized automata whose arrows carry expressions, the four conditions of normal form, and the state-elimination rip-out rule, including the starred self-loop term people drop. It works conversions, among them machines with dead states, shows how the elimination order controls the size of the result, and presents Kleene's original build-up recurrence and its kinship with Floyd-Warshall. It closes with the four now-equivalent definitions of a regular language.
Connect it up
Draw it
One page, no notation unless you need it: draw how these connect — The Remaining Direction · The Rip-Out Rule · Worked Conversions · Order Matters · An Alternative: Build Up by Allowed States · Kleene's Theorem, Completed. Put an arrow wherever one of them is what makes another possible, and label the arrow with why.
Recap
You can move in both directions between machines and expressions, which closes the loop opened in Lesson 4.
| Situation | Move |
|---|---|
| a machine to describe in words | convert to an expression and read it aloud |
| a state with a self-loop to remove | way in, starred loop, way out — never drop the star |
| a long expression appearing | check the elimination order, and remove dead states first |
| a closure proof to write | pick expressions or machines, whichever makes it one line |
| a language you suspect is not regular | nothing here helps — go to Lesson 10 |
Lesson 10 turns to the other question entirely: proving that a language has no machine at all, by finding a property every regular language is forced to have.
Want this taught 1-on-1? Alexander tutors Theory of Computation — $55/session, free consultation.