17c Operator Overloading, Type-Based Dispatch, and Polymorphism

This lesson makes operators work on programmer-defined types with special methods, handles operands of different types with dispatch and the right-side add, and closes with polymorphism — functions that work on types they were never written for.

Subject: Python · 65 slides · code lesson

Open the interactive version of this deck

What this lesson covers

The lesson, slide by slide

1. Lesson 17c Operator Overloading, Type-Based Dispatch, and Polymorphism

Title

Python · Chapter 17 — Classes and methods

§17.7-17.11, pp. 165-169

2. By the end of this lesson you can

Objectives

Five things, each one you can check yourself at an interpreter prompt.

Think Python, 2nd edition — Allen B. Downey §17.7-17.11, pp. 165-169 — the pages these objectives are drawn from

3. Before we start: why does + work on strings?

Warm-up

The same operator, on three types.

Discussion prompt

The plus sign adds integers, concatenates strings, and joins lists. Given what the last lesson said about print invoking __str__, what do you now suspect is happening?

Hint: print did not decide how to display a Time.

Answer:

Each type must define what plus means for it, and the operator asks the object rather than deciding for itself — exactly as print asked the Time for its string.

Which suggests that if you defined the right method on your own class, the plus sign would work on your objects too.

That is exactly right, and it is this lesson. For every operator in Python there is a corresponding special method.

4. The one idea behind this lesson: operators ask the object

Concept

By defining other special methods, you can specify the behaviour of operators on programmer-defined types. For example, if you define a method named __add__ for the Time class, you can use the + operator on Time objects.

operator overloading — Changing the behaviour of an operator like + so it works with a programmer-defined type.

# inside class Time:
    def __add__(self, other):
        seconds = self.time_to_int() + other.time_to_int()
        return int_to_time(seconds)

>>> print(start + duration)
11:20:00
What you writeWhat Python doesNote
start + durationinvokes __add__with duration as other
print(...)invokes __str__on the result
one linetwo special methodsbehind the scenes

When you apply the + operator to Time objects, Python invokes __add__; when you print the result, Python invokes __str__. So there is a lot happening behind the scenes.

Figure (svg): A pipeline showing one printed expression invoking two special methods

One short line, and two methods you wrote are called without being named.

Think Python, 2nd edition — Allen B. Downey §17.7-17.11, pp. 165-166

5. Operator overloading

Section

Section 1

6. Every operator has a special method

Concept

Changing the behaviour of an operator so that it works with programmer-defined types is called operator overloading. For every operator in Python there is a corresponding special method, like __add__.

# inside class Time:
    def __add__(self, other):
        seconds = self.time_to_int() + other.time_to_int()
        return int_to_time(seconds)

>>> start = Time(9, 45)
>>> duration = Time(1, 35)
>>> print(start + duration)
11:20:00
PartWhat it receivesNote
selfthe left operandstart
otherthe right operandduration
the return valuea new Timewhich is what + produces

The method takes self and other, exactly like is_after — the left operand becomes the subject and the right becomes the argument. And the body is chapter 16's add_time unchanged, which is the point: the operator is new syntax for an operation that already existed.

Think Python, 2nd edition — Allen B. Downey §17.7-17.11, pp. 165-166

7. Picture it: which operand becomes which parameter

Picture it

The operator is a method call written differently.

Figure (svg): A call diagram showing an addition expression mapped onto a method call

The plus sign is method syntax with the subject on the left of the operator instead of the dot.

Which is why the left operand matters so much, and why the next idea's problem arises when it is not a Time.

8. Worked example: what one line actually does

Worked example

The book's own observation: a lot happening behind the scenes.

>>> print(start + duration)
11:20:00

# 1. start + duration  invokes start.__add__(duration)
# 2. __add__ calls time_to_int on both
# 3. it calls int_to_time to build a Time
# 4. print invokes __str__ on that Time
# 5. print displays the returned string
StageWhat is invokedNote
the operatorinvokes __add__one special method
insidetwo conversionsordinary methods
printinvokes __str__another special method

Trace the operator.

Why: When you apply the + operator to Time objects, Python invokes __add__ with the left operand as the subject.

Trace the body.

Why: It converts both to integers, adds them, and builds a new Time — chapter 16's designed-development approach, reused unchanged.

Trace the printing.

Why: When you print the result, Python invokes __str__ — so a single expression involves two methods that are never named.

Figure (svg): The state of the program after each line of Worked example what one line actually does, drawn as a ladder with one rung per traced line

The whole run at once: each drop is one line of the program.

Five steps from one short line, two of them methods invoked by syntax. That is the mechanism working, and it is worth tracing once so it stops feeling like magic.

Verify: Check that the arithmetic is the chapter 16 version.

Why: The body is time_to_int on both, add, and int_to_time — identical to add_time from lesson 16b. So no new logic was written: the operator is new syntax for an operation that already existed, which is what makes overloading cheap to add.

9. Predict: which method does the plus sign invoke?

Prediction

Two Time objects.

start = Time(9, 45)
duration = Time(1, 35)
print(start + duration)
ExpressionWhat is invokedNote
start + durationinvokes __add__on start
durationbecomes otherthe right operand
printinvokes __str__on the result

Predict first

What does this print?

  • 11:20:00
  • <__main__.Time object at 0x...>
  • A TypeError about unsupported operand types
  • 10:80:00

Correct: 11:20:00 — the plus sign invokes __add__ and print invokes __str__.

Why: Both special methods are called by syntax that never names them, which is why the book says there is a lot happening behind the scenes. Option B is what you would get without a __str__ method, and option C without an __add__ — each missing method removes one of the two steps.

10. Worked example: overloading is not restricted to plus

Worked example

For every operator there is a corresponding special method.

# __add__      +
# __sub__      -
# __mul__      *
# __lt__       <
# __eq__       ==
# __len__      len(x)
# __getitem__  x[i]
CategoryHow many methodsNote
arithmetic operatorsone method eachadd, sub, mul
comparisonslikewiselt, eq
built-in functionsalsolen, indexing

Note the generality.

Why: For every operator in Python there is a corresponding special method — the plus sign is one instance of a general mechanism.

Note that built-ins are included.

Why: len and indexing work the same way, which is why they have always worked across the built-in types.

Note what this makes possible.

Why: A class can be given the same syntax as a built-in type, so code written for one can work on the other.

Figure (svg): Operators and built-in functions paired with the special methods they invoke

Ordinary syntax on the left, a method call on the right.

A method per operator, which is what makes the mechanism general. The next idea's __radd__ is another member of the same family.

Verify: Connect it to chapter 15's disappointment.

Why: Two Points with identical coordinates compared unequal, because Python doesn't know what should be considered equivalent for a programmer-defined type. Defining __eq__ is how you tell it — which is what the book's at least, not yet was pointing at, and it is the same mechanism as __add__.

11. Trap: making __add__ modify its operands

Trap

The trap

An __add__ method adds the other Time's seconds into self and returns self.

Do the work in place and hand it back

Why: It avoids constructing a new object.

Now a + b changes a, which nothing about the syntax suggests. Writing c = a + b silently destroys a, and a in a loop accumulates results nobody asked for.

The fix

Return a new object.

Build the result and return it

Why: Which is what int_to_time does here.

Because that is what the operator means

Why: 3 + 4 does not change the 3.

Operator overloading inherits an expectation from the built-in types: arithmetic operators produce new values and leave their operands alone. A modifying __add__ is legal and violates what every reader assumes.

12. Complete it: make plus work on Times

Faded example

The name is the operator's special method.

Fill in the blanks

def __add__(self, other):
seconds = self.time_to_int() + other.time_to_int()
return int_to_time(seconds)

Why: Defining __add__ is what makes the plus operator work on Time objects, with the left operand as self and the right as other. The body is chapter 16's add_time unchanged — the operator is new syntax for an operation that already existed rather than new logic.

13. Discriminate: which special method does this need?

Discrimination

Every operator has one.

Sort into buckets

For each thing you want to work on your class, which method must you define?

__add__ or __sub__
a + b; a - b
__str__
print(a); str(a)
__eq__ or __init__
a == b; creating an instance with arguments
add
Both are arithmetic operators, each with its own method — __add__ for plus and __sub__ for minus.
str
Both need a string representation, and both invoke the same method: print displays what str would return.
oth
One is comparison and one is construction — different jobs, and each has its own special method.

14. Explain it yourself: why is this called overloading?

Explain it to yourself

The word is doing specific work.

Discussion prompt

Defining __add__ is called operator overloading. What is being overloaded, and what does that word capture?

Hint: How many meanings does the plus sign already have?

Answer:

The operator itself. The plus sign already means several things — integer addition, string concatenation, list joining — so it carries more than one meaning, which is what overloaded describes.

Defining __add__ on your class adds one more meaning, chosen by the type of the left operand. Nothing about the symbol changes; the set of types it works on grows.

Which is why the mechanism is safe: your definition cannot affect how plus behaves on integers or strings, because those types answer for themselves. You are adding a meaning rather than replacing one.

15. Type-based dispatch

Section

Section 2

16. Different methods for different argument types

Concept

You might also want to add an integer to a Time object. Here is a version of __add__ that checks the type of other and invokes either add_time or increment.

type-based dispatch — A programming pattern that checks the type of an operand and invokes different functions for different types.

# inside class Time:
    def __add__(self, other):
        if isinstance(other, Time):
            return self.add_time(other)
        else:
            return self.increment(other)
TestThe branchNote
isinstance(other, Time)is it a Time?True or False
if soadd_timeconverts both
otherwiseincrementassumes a number

The built-in function isinstance takes a value and a class object, and returns True if the value is an instance of the class. If other is a Time object, __add__ invokes add_time; otherwise it assumes the parameter is a number and invokes increment.

Think Python, 2nd edition — Allen B. Downey §17.7-17.11, pp. 166-166

17. Picture it: one operator, two computations

Picture it

The type of the right operand chooses the branch.

Figure (svg): A flowchart showing an addition dispatching on the type of its right operand

It dispatches the computation to different methods based on the type of the arguments.

Note the else branch assumes rather than checks — anything that is not a Time is treated as a number, which is a decision worth noticing.

18. Worked example: both kinds of addition

Worked example

One operator, two argument types.

>>> start = Time(9, 45)
>>> duration = Time(1, 35)
>>> print(start + duration)
11:20:00
>>> print(start + 1337)
10:07:17
Right operandWhich branchResult
a Time on the rightadd_time11:20:00
a number on the rightincrement10:07:17
the same operatortwo computationschosen by type

Add two Times.

Why: isinstance reports True, so add_time converts both and adds the totals.

Add a number of seconds.

Why: isinstance reports False, so increment adds the seconds to the subject's total.

Note the syntax is identical.

Why: Only the type of the operand differs, and the method chooses what to do about it.

Figure (svg): The state of the program after each line of Worked example both kinds of addition, drawn as a ladder with one rung per traced line

The whole run at once: each drop is one line of the program.

Two additions with one operator. The convenience is real, and the next slide shows what it does not cover.

Verify: Try something that is neither.

Why: start + 'abc' takes the else branch, which assumes a number and fails inside increment with a TypeError about adding a string to an integer — some way from the actual mistake. The else branch assumes rather than checks, so an unexpected type produces a confusing error rather than a clear one.

19. Predict: which branch runs?

Prediction

The right operand is a number.

    def __add__(self, other):
        if isinstance(other, Time):
            return self.add_time(other)
        else:
            return self.increment(other)

print(start + 1337)
TestResultNote
isinstance(1337, Time)Falsenot a Time
the else branchincrementassumes a number
the result10:07:17seconds added

Predict first

What does this print, given start is 09:45:00?

  • 10:07:17
  • 11:20:00
  • 1337
  • A TypeError

Correct: 10:07:17 — 1337 is not a Time, so the else branch invokes increment.

Why: 1337 seconds is 22 minutes and 17 seconds, which added to 09:45:00 gives 10:07:17. The dispatch chose increment because isinstance reported False — and note that it assumed a number rather than checking, so a string would take the same branch and fail inside increment.

20. Worked example: the isinstance function

Worked example

A value and a class object.

>>> isinstance(start, Time)
True
>>> isinstance(1337, Time)
False
>>> isinstance(1337, int)
True
CallWhat it asksResult
a Time and TimeTrueit is an instance
an integer and TimeFalseit is not
an integer and intTruebuilt-in classes work too

Note the two arguments.

Why: isinstance takes a value and a class object, and returns True if the value is an instance of the class.

Note that the class is not quoted.

Why: It is the class object itself — Time, not 'Time' — unlike hasattr, whose second argument is a string.

Note it works on built-in types too.

Why: isinstance(1337, int) is True, because every object is an instance of some class.

Figure (svg): The state of the program after each line of Worked example the isinstance function, drawn as a ladder with one rung per traced line

The whole run at once: each drop is one line of the program.

A boolean about membership, using the class object. It is the tool lesson 15b introduced for debugging, put to work as a dispatch test.

Verify: Compare with type.

Why: type(start) returns the class object itself and isinstance answers a yes-or-no question about it — so isinstance is what a conditional wants. The distinction matters more once inheritance exists, since isinstance also accepts instances of subclasses.

21. Trap: an else branch that assumes

Trap

The trap

__add__ tests for Time and treats everything else as a number.

Handle the two cases you care about

Why: Which is what the book's version does.

A third type — a string, a list, a Point — takes the number branch and fails somewhere inside increment, with a message about the operation rather than about the argument. The error is real and points at the wrong place.

The fix

Test for what you accept, and reject the rest clearly.

elif isinstance(other, int): ...

Why: So the third case is identifiable.

else: raise TypeError with a message

Why: Naming the type that was not understood.

The book's version is deliberately compact for teaching. In a class other people use, the difference between I do not accept this and a crash three calls deep is worth four extra lines.

22. Sort: which branch does this take?

Sorting

The test is whether the right operand is a Time.

Sort into buckets

For each right operand, which branch of __add__ runs?

add_time
another Time; Time(0, 5); the result of int_to_time(60)
increment
the integer 1337; the float 60.5; the string 'abc'
t
Each is a Time, so isinstance reports True and add_time converts both operands and adds the totals.
i
None is a Time, so each takes the else branch — which assumes a number. Two of them are numbers and the string is not, which is why it fails inside increment rather than at the dispatch.

23. Complete it: dispatch on the type

Faded example

A value and a class object.

Fill in the blanks

def __add__(self, other):
if isinstance(other, Time):
return self.add_time(other)
else:
return self.increment(other)

Why: isinstance takes a value and a class object and returns True if the value is an instance of the class — so the class name is not quoted, unlike hasattr's string argument. Using type(other) == Time would also work here and is less flexible, since isinstance also accepts subclasses.

24. Explain it: why is it called dispatch?

Explain it

The word describes what the method does.

Discussion prompt

A classmate asks what type-based dispatch means, since the code is just an if statement. Explain the name.

Hint: What is the if statement deciding?

Answer:

It is deciding which computation to send the work to — dispatching it, in the sense of sending something off to be handled elsewhere.

What makes it type-based is the criterion: the decision is made on the type of an operand rather than on its value. Ordinary conditionals branch on what a value is; this one branches on what kind of thing it is.

And the name distinguishes it from the alternative, which is the next idea: writing code that works for several types without asking. The book is explicit that type-based dispatch is useful when it is necessary, and often you can avoid it.

25. The right-side add

Section

Section 3

26. When the object is on the wrong side

Concept

Unfortunately, this implementation of addition is not commutative. If the integer is the first operand, you get an error.

>>> print(start + 1337)
10:07:17
>>> print(1337 + start)
TypeError: unsupported operand type(s) for +: 'int' and 'instance'
ExpressionWhich object is askedResult
Time on the leftTime.__add__ is invokedworks
integer on the leftint's add is invokedand it does not know Times
the problemthe wrong object was asked

The problem is that instead of asking the Time object to add an integer, Python is asking an integer to add a Time object, and it doesn't know how. But there is a clever solution: the special method __radd__, which stands for right-side add.

Think Python, 2nd edition — Allen B. Downey §17.7-17.11, pp. 167-167

27. Picture it: which operand gets asked

Picture it

The left one, always — and that is the whole problem.

Figure (svg): Two columns showing which object the plus operator asks in each ordering

Python asks the left operand first, which is why the ordering matters.

__radd__ is the fallback: when the left operand does not know what to do, the right one is asked instead.

28. Worked example: __radd__

Worked example

Two lines, and the addition becomes commutative.

# inside class Time:
    def __radd__(self, other):
        return self.__add__(other)

>>> print(1337 + start)
10:07:17
StageWhat happensNote
the integer's addfailsit does not know Times
Python then triesstart.__radd__(1337)the right operand
__radd__delegates to __add__same computation

Note when it is invoked.

Why: This method is invoked when a Time object appears on the right side of the + operator and the left operand could not handle it.

Note what it does.

Why: It delegates to __add__, because addition of a Time and a number gives the same answer whichever order they are written.

Note the result.

Why: 1337 + start now works and gives the same answer as start + 1337.

Figure (svg): A flowchart showing Python falling back to the right operand's method

The left operand first, then the right — which is why __radd__ exists at all.

Commutative addition, from a two-line method. The delegation works here because the operation genuinely is symmetric.

Verify: Ask when delegating would be wrong.

Why: For subtraction: 5 - t and t - 5 are different questions, so __rsub__ could not simply call __sub__ — it would have to reverse the operands. The one-line delegation is correct only for genuinely commutative operations, which addition is and most operators are not.

29. Predict: does this work?

Prediction

The integer is on the left, and __radd__ is not defined.

# Time defines __add__ but not __radd__
print(1337 + start)
StepWhat happensResult
the left operandan integerasked first
int's adddoes not know Timesfails
no fallback__radd__ undefinedTypeError

Predict first

What happens?

  • A TypeError about unsupported operand types
  • 10:07:17, the same as start + 1337
  • Python swaps the operands automatically
  • Time.__add__ is invoked with 1337 as self

Correct: A TypeError about unsupported operand types — Python asks the integer, which does not know how to add a Time.

Why: Instead of asking the Time object to add an integer, Python is asking an integer to add a Time object. Nothing swaps the operands automatically, which is exactly why __radd__ exists: it gives the right-hand object a chance when the left one cannot help.

30. Worked example: reading the error message

Worked example

It names both types, which tells you what happened.

TypeError: unsupported operand type(s) for +: 'int' and 'instance'

# the operator:        +
# the left operand:    int
# the right operand:   your object
Part of the messageWhat it tells youNote
the operatornamed first+
the left typeintthe one that was asked
the right typeinstancethe one that could have helped

Read the operator.

Why: It names which operation failed, which matters when a line contains several.

Read the left type.

Why: That is the object Python asked, and its inability is the immediate cause.

Read the right type.

Why: If that is your class, the fix is to define the corresponding right-side method.

Figure (svg): The state of the program after each line of Worked example reading the error message, drawn as a ladder with one rung per traced line

The whole run at once: each drop is one line of the program.

A message that identifies the operator and both operand types. When your own class is the second one named, the reversed special method is what is missing.

Verify: Check the message with the operands the other way round.

Why: start + 'abc' gives a different failure — inside increment rather than at the operator — because Time.__add__ accepted the call and then failed on the arithmetic. So the operand-type message means neither object could handle the operation at all, which is a distinct situation from one that tried and failed.

31. Trap: assuming an overloaded operator is commutative

Trap

The trap

A class defines __add__ and its author assumes both orderings work.

Take addition to be symmetric

Why: It is for numbers, and the operation being implemented usually is too.

Python asks the left operand, so an integer on the left is asked to add your object and cannot. The failure appears only in the ordering nobody tested, which is usually the less natural one.

The fix

Define __radd__ when the reversed ordering is meaningful.

return self.__add__(other)

Why: For a genuinely commutative operation.

And test both orderings

Why: Since only one of them exercises the new method.

The general rule: overloading an operator handles the case where your object is on the left. The reversed method handles the other case, and it is a separate decision because not every operation is symmetric.

32. Watch the fallback: 1337 + start, with and without __radd__

Invariant

Two runs of the same expression.

Step through it

What is the same in both runs, and what differs?

  1. Python always asks the left operand first, whatever the types are.
  2. The integer's addition has never heard of a Time, and with no fallback available the expression fails.
  3. The first step is identical in the second run — defining __radd__ changes nothing about who is asked first.
  4. Now the fallback exists, so the Time is asked instead, and it delegates to __add__ for the same answer.

The left operand is asked first either way — that never changes. What differs is whether there is anywhere to go when it cannot help, which is the whole of what __radd__ provides.

33. Complete it: handle the reversed operands

Faded example

Addition is symmetric, so delegate.

Fill in the blanks

def __radd__(self, other):
return self.__add__(other)

Why: __radd__ is invoked when a Time appears on the right side of the plus operator and the left operand could not handle it, and delegating to __add__ works because addition gives the same answer either way. For subtraction the delegation would be wrong, since t - 5 and 5 - t are different questions.

34. Think it through: why does Python ask the left operand first?

Socratic

It could ask both and pick.

Discussion prompt

Python asks the left operand and only falls back to the right. Why not ask both?

Hint: What would happen if both knew how?

Answer:

Because both might know, and then something would have to choose between two possibly different answers — which is a decision the language cannot make sensibly.

Asking the left first gives a definite rule: the left operand's meaning wins if it has one, and the right operand's is the fallback. Nothing is ambiguous.

It also matches how the operators already behave. 3 + 4 asks the integer, and 'a' + 'b' asks the string; the left operand has always been the one consulted, and overloading follows the existing rule rather than inventing a new one.

35. Polymorphism

Section

Section 4

36. Functions that work with several types

Concept

Type-based dispatch is useful when it is necessary, but fortunately it is not always necessary. Often you can avoid it by writing functions that work correctly for arguments with different types.

polymorphic — Pertaining to a function that can work with more than one type.

def histogram(s):
    d = dict()
    for c in s:
        if c not in d:
            d[c] = 1
        else:
            d[c] = d[c]+1
    return d

>>> t = ['spam', 'egg', 'spam', 'spam', 'bacon', 'spam']
>>> histogram(t)
{'bacon': 1, 'egg': 1, 'spam': 4}
AspectWhat is trueNote
written for stringsin chapter 11counting letters
given a listworkscounting words
whyevery operation inside works for lists too

This function also works for lists, tuples, and even dictionaries, as long as the elements of s are hashable, so they can be used as keys in d.

Think Python, 2nd edition — Allen B. Downey §17.7-17.11, pp. 167-168

37. Picture it: one function, several types

Picture it

Nothing in histogram mentions strings.

Figure (svg): The histogram function applied to several different argument types

One function, written for one type, working on four.

The condition is that the elements be hashable, so they can be used as keys — which is a requirement on the elements rather than on the container.

38. Worked example: why histogram works so widely

Worked example

Look at what it actually requires.

for c in s:            # s must be iterable
    if c not in d:     # c must be hashable
        d[c] = 1       # likewise
LineWhat it requiresNote
the loopneeds something iterablestring, list, tuple, dict
the dictionary keyneeds something hashablethe elements
nothing elseno type is named

List the requirements.

Why: The argument must be something a for loop can traverse, and its elements must be usable as dictionary keys.

Note what is not required.

Why: Nothing about strings, or about any particular type — the function never asks what it has been given.

State the general rule.

Why: If all of the operations inside a function work with a given type, the function works with that type.

Figure (svg): The state of the program after each line of Worked example why histogram works so widely, drawn as a ladder with one rung per traced line

The whole run at once: each drop is one line of the program.

Two requirements, both about capabilities rather than types. That is what makes a function polymorphic, and it happened here without being planned.

Verify: Find something it does not work on.

Why: A list of lists fails, because a list is unhashable and cannot be a dictionary key — chapter 11's restriction. So the boundary is exactly the requirement, which confirms the rule: the function works with any type for which its operations work.

39. Predict: does histogram work on a list?

Prediction

It was written for strings.

t = ['spam', 'egg', 'spam', 'spam', 'bacon', 'spam']
print(histogram(t))
RequirementIs it met?Note
the loopa list is iterableone word per pass
the keysstrings are hashableusable as keys
the resultword counts{'bacon': 1, 'egg': 1, 'spam': 4}

Predict first

What does this print?

  • {'bacon': 1, 'egg': 1, 'spam': 4}
  • A TypeError, since histogram expects a string
  • A count of the characters in the whole list
  • {'s': 4, 'p': 4, ...}

Correct: {'bacon': 1, 'egg': 1, 'spam': 4} — the function works on any iterable whose elements are hashable.

Why: Nothing in histogram mentions strings. It needs something a for loop can traverse and elements usable as dictionary keys, and a list of strings satisfies both — so it counts words rather than characters, with no change to the function. That is polymorphism, and it was not planned.

40. Worked example: why sum works on Times

Worked example

A built-in function, and a class it was never written for.

>>> t1 = Time(7, 43)
>>> t2 = Time(7, 41)
>>> t3 = Time(7, 37)
>>> total = sum([t1, t2, t3])
>>> print(total)
23:01:00
PartWhat it needsNote
sumadds the elements of a sequencewith the plus operator
Time objectsprovide an add methodso plus works
the resulta Timeprinted with __str__

Note what sum requires.

Why: The built-in function sum, which adds the elements of a sequence, works as long as the elements of the sequence support addition.

Note what Time provides.

Why: Since Time objects provide an add method, they work with sum.

Note that nobody arranged this.

Why: sum was written long before your class existed, and works on it because your class supports the operation sum uses.

Figure (svg): A pipeline showing sum adding Time objects using their add method

Three special methods, and sum knows about none of them.

23:01:00, from a built-in function that has never heard of Times. Polymorphism can facilitate code reuse, and this is what that means concretely.

Verify: Check what sum starts from.

Why: sum begins with 0 and adds each element, so the first addition is 0 + t1 — which needs __radd__, not __add__. That is the previous idea earning its place: without the right-side method, sum would fail on the first element with a message about int and instance.

41. Trap: writing a type check that was not needed

Trap

The trap

A function begins by checking isinstance on its argument, to be safe.

Guard against the wrong type

Why: Which prevents a confusing failure later.

It also rejects every type the function would have worked on. histogram with an isinstance(s, str) guard would refuse the lists and tuples it handles perfectly well, for no benefit.

The fix

Let the operations decide.

If all the operations inside work with a type, the function works with that type

Why: Which is the book's own rule.

Use dispatch when the behaviour must differ

Why: Not merely when the types do.

The book's phrasing is careful: type-based dispatch is useful when it is necessary, and often you can avoid it. Necessary means the computation genuinely differs — as it does between adding a Time and adding a number.

42. Sort: will histogram work on this?

Sorting

Iterable, with hashable elements.

Sort into buckets

For each argument, does histogram work?

works
a string; a list of words; a tuple of numbers; a dictionary
fails
a list of lists; an integer
yes
Each is iterable and yields hashable elements — a dictionary yields its keys, which are hashable by definition. Both of the function's requirements are met.
no
One yields lists, which are unhashable and cannot be dictionary keys; the other is not iterable at all, so the for loop fails immediately.

43. Complete it: sum a list of Times

Faded example

It works because Times support addition.

Fill in the blanks

total = sum([t1, t2, t3])
print(total) # 23:01:00

Why: sum adds the elements of a sequence and works as long as the elements support addition — which Time objects do, because they provide an add method. Note that sum starts from 0, so the first addition is 0 + t1 and needs __radd__ as well as __add__.

44. Where something works on more than it was built for

Real world

The best kind of polymorphism is the unintentional kind.

Discussion prompt

Think of a tool or a standard that turned out to work for something its designers never considered. What made that possible?

Hint: What did it require, rather than assume?

Answer:

A fitting that works on anything the right size, a file format read by programs that did not exist when it was defined, a socket that powers devices nobody imagined.

What made it possible is that the design specified requirements rather than a list of approved things — anything meeting the requirement works, including things nobody thought of.

That is the book's closing line: the best kind of polymorphism is the unintentional kind, where you discover that a function you already wrote can be applied to a type you never planned for. Writing in terms of operations rather than types is what makes it possible.

45. Dispatch or polymorphism?

Section

Section 5

46. One is for when the computation genuinely differs

Concept

Type-based dispatch is useful when it is necessary, but fortunately it is not always necessary. Often you can avoid it by writing functions that work correctly for arguments with different types.

The best kind of polymorphism is the unintentional kind, where you discover that a function you already wrote can be applied to a type you never planned for.

Think Python, 2nd edition — Allen B. Downey §17.7-17.11, pp. 167-168

47. Picture it: when the branches are needed

Picture it

Ask whether the computations differ, not whether the types do.

Figure (svg): A decision flowchart for choosing between dispatch and polymorphism

Dispatch is useful when it is necessary, and often it is not.

The default is the right-hand branch, because a function that never asks about types works on more of them.

48. Worked example: why __add__ genuinely needs dispatch

Worked example

The two computations are not the same.

# adding a Time: convert BOTH and add
seconds = self.time_to_int() + other.time_to_int()

# adding a number: convert ONE and add
seconds = other + self.time_to_int()
ArgumentWhat must happenNote
a Time argumentneeds convertingit is not a number
a number argumentalready a numberconvert only self
the differenceone conversion or twogenuinely different

Compare the two bodies.

Why: One converts both operands and the other converts only the subject, because the argument is already in the right units.

Note that no single body covers both.

Why: Calling time_to_int on an integer would fail, and treating a Time as a number of seconds would be wrong.

Conclude.

Why: The computations differ, so the dispatch is necessary — which is exactly the condition the book gives.

Figure (svg): Two columns comparing the two computations behind a single overloaded operator

No single body covers both, which is what necessary means.

Two different computations behind one operator, which is what makes dispatch the right tool here rather than a failure of imagination.

Verify: Ask whether the difference could be removed.

Why: It could, by converting the argument to a Time first — int_to_time(other) — and then always doing the two-operand version. That would replace the dispatch with a conversion, which is chapter 16's move again: making the general case cover the special one. Whether it is better is a judgement, and noticing the option is the point.

49. Discriminate: dispatch, or write it once?

Discrimination

Ask whether the computations differ.

Sort into buckets

For each situation, which approach fits?

type-based dispatch
adding a Time or a number of seconds; formatting a date differently for two calendars; parsing input that may be a string or a file
write it once
counting the elements of any sequence; summing anything that supports addition; finding the largest element of any sequence
d
In each, the computation genuinely differs by type — different units, different formats, different sources — so no single body could cover them.
p
In each, one computation works for every type that supports the operations used. Adding a type check would only refuse types that would have worked.

50. Worked example: the same problem without dispatch

Worked example

Polymorphism removes the question rather than answering it.

# with dispatch: ask what it is
if isinstance(other, Time):
    ...

# without: convert, then proceed
def __add__(self, other):
    if not isinstance(other, Time):
        other = int_to_time(other)
    return int_to_time(self.time_to_int() + other.time_to_int())
VersionHow many computationsNote
the dispatch versiontwo bodiesone per type
the converting versionone bodyafter normalising
what remainsone type checkat the boundary

Notice where the check moved.

Why: It is still there, at the top, but it normalises rather than branching — so only one computation follows.

Notice the general pattern.

Why: Convert the unusual case into the usual one at the boundary, and the body handles a single type.

Notice it is chapter 16's move.

Why: Making the general case cover the special one, which is what the base-60 conversion did for carrying.

Figure (svg): The state of the program after each line of Worked example the same problem without dispatch, drawn as a ladder with one rung per traced line

The whole run at once: each drop is one line of the program.

One computation instead of two, with the type question confined to a single normalising line. Whether that is better depends on the class, and knowing both shapes is what lets you choose.

Verify: Ask what each version costs when a third type arrives.

Why: The dispatch version needs a third branch; the normalising version needs one more conversion at the boundary and no change to the body. That difference grows with the number of types, which is the practical argument for pushing the checking to the edges.

51. Trap: reaching for isinstance as a habit

Trap

The trap

Every function that takes an argument begins by checking its type.

Be defensive about inputs

Why: A wrong type does produce confusing failures.

It rejects every type the function would have worked on, which is usually more than the author imagined — and it is the opposite of the polymorphism the book calls the best kind.

The fix

Check types when the behaviour must differ.

Dispatch when the computations are genuinely different

Why: As in __add__, where they are.

Otherwise write it once and let the operations decide

Why: If all of the operations inside work with a type, the function works with that type.

The unintentional polymorphism is the payoff, and a type check is what prevents it. histogram would never have worked on lists if its author had guarded it.

52. Predict: what makes a function polymorphic?

Prediction

histogram never mentions a type.

def histogram(s):
    d = dict()
    for c in s:
        ...
    return d
LineWhat it requiresNote
the for loopneeds an iterablea capability
the dictionary keysneed hashable elementsanother
no type checkanywhere

Predict first

What determines which types histogram works on?

  • Whether all the operations inside it work with that type
  • Whether the author listed the type as supported
  • Whether the type is a sequence
  • Whether the type defines __str__

Correct: Whether all the operations inside it work with that type — which is the book's own general rule.

Why: The function needs something a for loop can traverse and elements usable as dictionary keys, and any type meeting both works — including a dictionary, which is not a sequence. Nothing was listed or planned, which is why the book calls unintentional polymorphism the best kind.

53. Two truths and a lie: dispatch and polymorphism

Two truths and a lie

Two are true. Keep the lie.

Eliminate the wrong options

Rule out the two true statements.

  • A. Type-based dispatch is useful when it is necessary, and often it is not necessary
  • B. If all the operations inside a function work with a type, the function works with that type
  • C. A polymorphic function must check the types of its arguments

Survives elimination: C

Why: C describes the opposite. A polymorphic function works with several types precisely because it does not ask — it uses operations, and any type supporting them works. Adding a type check would refuse types the function handles perfectly well, which is how unintentional polymorphism gets prevented.

54. Explain it: when should I check the type?

Explain it

The book gives a clear condition.

Discussion prompt

A classmate asks whether their function should check what type it was given. Give them the question that decides it.

Hint: What would the check be for?

Answer:

Ask whether the different types need different computations. If they do — as adding a Time and adding a number do — the check is necessary and the branches are the point.

If one computation would work for all of them, the check only refuses types that would have succeeded. histogram works on lists and tuples and dictionaries precisely because nobody guarded it.

The book's rule is the test: if all of the operations inside a function work with a given type, the function works with that type. So write the operations, and let them decide what the function accepts.

55. Compare: dispatch and polymorphism

Comparison

Fill the blanks. Two answers to what type is this?

Comparison matrix

QuestionType-based dispatchPolymorphism
Does the function ask about types?yes — isinstanceno — it just uses operations
How many computations?one per typeone, for all of them
Which types work?the ones you listedevery type the operations work for
When is it right?when the computations genuinely differthe rest of the time

The book's phrasing is careful: dispatch is useful when it is necessary, and often you can avoid it.

56. The procedure: overloading an operator

Pattern

Six steps, and the fifth is the one people forget until something fails.

  1. Find the special method for the operator — __add__ for plus, and one for every other.
  2. Give it self and other: the left operand becomes the subject and the right the argument.
  3. Return a new object rather than modifying either operand, since that is what arithmetic operators mean.
  4. If the argument may be of more than one type, decide whether the computations genuinely differ.
  5. If your object may appear on the right of the operator, define the reversed method too.
  6. Test both orderings, since only one of them exercises the reversed method.

Step 5 is what makes sum work: it starts from 0, so the first addition puts your object on the right, and without __radd__ it fails on the first element.

Python documentation — Classes Classes

57. Check yourself 1 of 3: which methods are invoked

Check

One short line.

print(start + duration)
SyntaxMethod invokedNote
the plus operator__add__on start
print__str__on the result
neithernamed in the code

Check your understanding

How many special methods does this line invoke?

  • A. Two — __add__ and __str__ (correct)
  • B. One — __add__
  • C. One — __str__
  • D. None; both are ordinary functions

Answer: A

Why: When you apply the + operator to Time objects, Python invokes __add__; when you print the result, Python invokes __str__. As the book says, there is a lot happening behind the scenes — and neither method is named anywhere in the line.

Why B tempts people
The printing also invokes a method: without __str__ the output would be a memory address.
Why C tempts people
The addition invokes one too: without __add__ the expression would raise a TypeError.
Why D tempts people
Both are methods on the class, invoked by syntax rather than by name.

58. Check yourself 2 of 3: the reversed operands

Check

The integer is on the left.

Check your understanding

Why does 1337 + start fail when start + 1337 works?

  • A. Python asks the left operand, and an integer does not know how to add a Time (correct)
  • B. Time.__add__ rejects integers
  • C. Addition is never commutative in Python
  • D. The integer is too large

Answer: A

Why: Instead of asking the Time object to add an integer, Python is asking an integer to add a Time object, and it doesn't know how. The fix is __radd__ — the right-side add — which is invoked when a Time appears on the right and the left operand could not help.

Why B tempts people
Time.__add__ handles integers via its dispatch; it is never consulted in this ordering.
Why C tempts people
Addition is commutative for numbers and strings. The asymmetry here is about which object is asked, not about the operation.
Why D tempts people
The value is irrelevant; the same failure occurs for 1 + start.

59. Check yourself 3 of 3: polymorphism

Check

A function written for strings.

Check your understanding

Why does histogram work on a list of words?

  • A. Because all the operations inside it work with lists (correct)
  • B. Because it checks the type of its argument
  • C. Because lists are a kind of string
  • D. Because the author added support for lists

Answer: A

Why: The function needs something a for loop can traverse and elements usable as dictionary keys, and a list of strings satisfies both. In general, if all of the operations inside a function work with a given type, the function works with that type — which the book calls the best kind of polymorphism, the unintentional kind.

Why B tempts people
It checks nothing, which is precisely why it works so widely. A check would refuse types that succeed.
Why C tempts people
Lists and strings are different types; what they share is being iterable with hashable elements.
Why D tempts people
Nobody added anything. It works on a type it was never planned for.

60. Where this shows up outside this course

Real world

Specifying a requirement rather than a list of approved things.

Discussion prompt

Think of a rule written as anything that can do X against one written as a list of allowed items. What happens to each when something new appears?

Hint: A standard against a whitelist.

Answer:

The requirement admits the new thing automatically, if it meets the condition. The list has to be amended, by someone who knows to do it.

Which is the difference between polymorphism and type-based dispatch exactly: one function works on every type supporting its operations, and the other works on the types someone enumerated.

The list is right when the cases genuinely differ and needs maintaining forever. The requirement is right the rest of the time, and it is what makes the unintentional kind possible — a function applied to a type you never planned for.

61. Confidence wager: commit before you check

Commit first

Answer, then rate your confidence.

Predict first

Your class defines __add__ and you write 1337 + t. What happens, and why?

  • TypeError — Python asks the left operand, and an integer cannot add your object
  • It works, because Python tries both operands
  • It works, because addition is commutative
  • TypeError, because __add__ only accepts Times

Correct: TypeError — Python asks the left operand, and an integer cannot add your object.

Why: The book puts it exactly: instead of asking the Time object to add an integer, Python is asking an integer to add a Time object, and it doesn't know how. Defining __add__ handles the case where your object is on the LEFT of the operator, and nothing about it applies when the object is on the right. The fix is __radd__, which stands for right-side add and is invoked when your object appears on the right and the left operand could not help — and for a commutative operation it can simply delegate: return self.__add__(other). This matters more than it sounds, because sum starts from 0 and adds each element, so the very first addition is 0 + t1 — with your object on the right. Without __radd__, sum fails on the first element with exactly this error, which is why the polymorphism section's sum example depends on the previous section's fix.

62. Explain it to someone else

Explain it

One line, and a lot happening behind it.

Discussion prompt

A classmate asks how print(start + duration) can possibly work when neither method is mentioned. Walk them through it.

Hint: Two special methods, invoked by syntax.

Answer:

The plus sign invokes __add__ on the left operand, with the right operand as its argument — so start + duration is start.__add__(duration), which builds and returns a new Time.

Then print invokes __str__ on that result and displays the string it returns, which is the previous lesson's mechanism doing its half.

So two methods they wrote are called without being named, in response to ordinary syntax. That is what makes them special methods, and it is the same mechanism that has always made len and plus work across the built-in types.

63. Exit ticket

Exit ticket

One honest answer. It decides what the next lesson opens with.

Predict first

Which of these is still least solid for you?

  • Operator overloading, and which operand becomes which parameter
  • Type-based dispatch, and when it is necessary
  • The right-side add, and why the ordering matters
  • Polymorphism, and the rule for which types a function works on

Correct: Whichever you picked is the right answer — this one is for you, not for a mark.

Why: The overloading itself is the previous lesson's mechanism applied to operators, and the thing worth holding is that the left operand becomes self. Dispatch is mechanically simple and the judgement is what matters — the book's condition is that the computations genuinely differ. The right-side add is the piece everyone omits until sum fails on the first element, since sum starts from zero. And polymorphism is the payoff of the whole chapter: writing in terms of operations rather than types is what lets a function you already wrote work on something you never planned for.

64. Synthesis: draw the map of this lesson

Connect it up

One page, from memory.

Draw it

Write one addition expression and draw arrows from each operand to the parameter it fills. Beside it, write the dispatch version of __add__ and mark the condition that makes the dispatch necessary. Underneath, draw the two orderings of a Time and an integer, marking which object Python asks in each and where __radd__ comes in. Finally state the rule for which types a function works on, and give one example of a function that works on more than its author intended.

65. What you can do now

Recap

Five pages, and chapter 17 is finished: the features earn their keep.

If you remember one thingIt is this
From overloadingFor every operator in Python there is a corresponding special method.
From the two-method lineprint(a + b) invokes __add__ and then __str__, naming neither.
From dispatchBranch when the computation differs, not merely when the type does.
From __radd__Python asks the left operand. sum starts from 0, so your object is on the right.
From polymorphismThe best kind is the unintentional kind.

The next chapter introduces inheritance — defining a class as a modified version of another — through a case study of card games: Card, Deck and Hand, with class attributes, comparison methods, and the class diagram that shows how the pieces relate.

Think Python, 2nd edition — Allen B. Downey §17.7-17.11, pp. 165-169 — everything on these slides traces back here

Sources

  1. Think Python, 2nd edition — Allen B. Downey — Allen B. Downey, Think Python: How to Think Like a Computer Scientist, 2nd edition (Green Tea Press, 2015), §17.7-17.11, pp. 165-169
  2. Python documentation — Classes
  3. Python documentation — Expressions

Want this taught 1-on-1? Alexander tutors Python — $55/session, free consultation.

Book on Wyzant · Text (657) 465-8108