19c Named Tuples and Gathering Keyword Arguments

This lesson replaces a simple class with a named tuple, weighs what that gives up, and completes the gather-and-scatter pair with the double star for keyword arguments.

Subject: Python · 65 slides · code lesson

Open the interactive version of this deck

What this lesson covers

The lesson, slide by slide

1. Lesson 19c Named Tuples and Gathering Keyword Arguments

Title

Python · Chapter 19 — The Goodies

§19.8-19.9, pp. 189-190

2. By the end of this lesson you can

Objectives

Five things, each one you can check yourself at an interpreter prompt.

Think Python, 2nd edition — Allen B. Downey §19.8-19.9, pp. 189-190 — the pages these objectives are drawn from

3. Before we start: how much code for two numbers?

Warm-up

Look back at chapter 15's Point.

Discussion prompt

A Point holds two numbers. Count the lines needed to define it with an __init__ and a __str__, and say what proportion of that code is about the two numbers.

Hint: How many times do x and y appear?

Answer:

Six lines or so, of which the informative content is the two names — everything else is the same boilerplate any two-attribute class would need.

The book's own verdict: this is a lot of code to convey a small amount of information.

Python provides a more concise way to say the same thing, and this lesson is it — along with the drawback that comes with it.

4. The one idea behind this lesson: a class that is mostly boilerplate can be one line

Concept

Many simple objects are basically collections of related values. When you define a class like this, you usually start with an init method and a str method — which is a lot of code to convey a small amount of information.

named tuple — A tuple whose elements can be accessed by name as well as by position, created by the namedtuple function.

from collections import namedtuple
Point = namedtuple('Point', ['x', 'y'])

>>> p = Point(1, 2)
>>> p
Point(x=1, y=2)
PartWhat it isNote
the first argumentthe name of the class'Point'
the secondthe attributes, as strings['x', 'y']
the return valuea class objectwhich you call to make instances

Point automatically provides methods like __init__ and __str__ so you don't have to write them. The init method assigns the arguments to attributes using the names you provided, and the str method prints a representation of the object and its attributes.

Figure (svg): Two columns comparing a hand-written class with the equivalent named tuple

A lot of code to convey a small amount of information, replaced by the information.

Think Python, 2nd edition — Allen B. Downey §19.8-19.9, pp. 189-190

5. Creating a named tuple

Section

Section 1

6. A name and a list of attribute names

Concept

The first argument is the name of the class you want to create. The second is a list of the attributes Point objects should have, as strings. The return value from namedtuple is a class object.

from collections import namedtuple
Point = namedtuple('Point', ['x', 'y'])

>>> Point
<class '__main__.Point'>
>>> p = Point(1, 2)
>>> p
Point(x=1, y=2)
ExpressionWhat it givesNote
namedtuple(...)returns a class objectnot an instance
Pointprints as a classlike any other
Point(1, 2)an instanceprinted with its attribute names

To create a Point object, you use the Point class as a function — exactly as chapter 15 did. The difference is that nobody wrote the __init__ that receives the arguments.

Think Python, 2nd edition — Allen B. Downey §19.8-19.9, pp. 190-190

7. Picture it: a function that returns a class

Picture it

Two arguments in, a class object out.

Figure (svg): A pipeline showing namedtuple producing a class object which produces instances

The return value from namedtuple is a class object, which is then used exactly as chapter 15's was.

So the factory metaphor has one more level: a function that makes the factory that makes the objects.

8. Worked example: what it provides for you

Worked example

Two methods, without being written.

# chapter 15's class needed both of these:
#   def __init__(self, x=0, y=0): ...
#   def __str__(self): ...

>>> p = Point(1, 2)
>>> p.x
1
>>> p
Point(x=1, y=2)
MethodWhere it came fromNote
__init__providedassigns by the names you gave
__str__providedshows the names and values
the printed forminformativeunlike an address

Note the init method.

Why: It assigns the arguments to attributes using the names you provided, which is exactly what chapter 17's version did by hand.

Note the str method.

Why: It prints a representation of the object and its attributes — so printing shows Point(x=1, y=2) rather than a memory address.

Note what that removes.

Why: The two methods the book said to write first in any new class, both of which are pure boilerplate for a collection of values.

Figure (svg): The state of the program after each line of Worked example what it provides for you, drawn as a ladder with one rung per traced line

The whole run at once: each drop is one line of the program.

The two methods lesson 17b recommended writing first, supplied automatically. That is most of what a simple class consists of.

Verify: Compare the printed form with chapter 15's.

Why: A hand-written Point without a __str__ printed as <__main__.Point object at 0x...>, which lesson 15a listed as one of four gaps in the minimal class style. The named tuple closes it by default, which is a genuine improvement over the starting point rather than merely over the finished class.

9. Predict: what does namedtuple return?

Prediction

A function that makes something.

Point = namedtuple('Point', ['x', 'y'])
print(Point)
PartWhat it isNote
namedtuplea functioncalled once
its return valuea class objectnot an instance
printing it<class '__main__.Point'>

Predict first

What does this print?

  • <class '__main__.Point'>
  • Point(x=1, y=2)
  • ('x', 'y')
  • <function namedtuple>

Correct: <class '__main__.Point'> — the return value from namedtuple is a class object.

Why: It is the same printed form as chapter 15's hand-written Point, because it is the same kind of thing: a class you then call to make instances. Option B is what printing an instance gives, which is one of the methods namedtuple provides for you.

10. Worked example: equality comes too

Worked example

Something chapter 15 could not manage.

>>> Point(1, 2) == Point(1, 2)
True

# chapter 15's hand-written Point:
#   two Points with the same coordinates compared unequal,
#   because for instances == means is
TypeHow == behavesNote
a named tuplecompares by valuelike a tuple
a plain classcompares by identityunless __eq__ is written
the differenceinherited from tuple

Recall chapter 15's disappointment.

Why: For instances, the default behaviour of == is the same as is: it checks object identity, not object equivalence.

Note what a named tuple does.

Why: It is a tuple, so it compares by contents — two Points with the same coordinates are equal.

Note where that comes from.

Why: Not from namedtuple's cleverness but from the tuple underneath, which already knew how to compare itself.

Figure (svg): Two columns comparing a hand-written class with a named tuple on four properties

Three of the four gaps lesson 15a listed, closed by default.

Value equality, without writing __eq__. It arrives because a named tuple really is a tuple, which is also the source of everything else in the next idea.

Verify: Check that a named tuple is usable as a dictionary key.

Why: It is, because a tuple is hashable and immutable — so a Point can index a dictionary where chapter 15's mutable class instance would have been a poor choice. That is another property inherited rather than provided.

11. Trap: passing the attribute names unquoted

Trap

The trap

A student writes namedtuple('Point', [x, y]).

Write the names as you would use them

Why: p.x uses a bare name, so the definition looks like it should too.

The second argument is a list of the attributes as strings, so unquoted names are looked up as variables — raising NameError, or worse, finding unrelated variables that happen to exist.

The fix

Quote them.

namedtuple('Point', ['x', 'y'])

Why: The names, as strings.

As with hasattr and getattr

Why: Which also take an attribute name as a string.

The rule is the same everywhere: naming an attribute takes a string, and using one takes bare syntax. The class's own name is quoted for the same reason — it does not exist until this call creates it.

12. Complete it: define a Point

Faded example

The attribute names are strings.

Fill in the blanks

from collections import namedtuple
Point = namedtuple('Point', ['x', 'y'])

Why: The second argument is a list of the attributes Point objects should have, as strings — so the names are quoted, like hasattr's second argument. Unquoted, they would be looked up as variables and almost certainly raise NameError.

13. Compare: a hand-written class and a named tuple

Comparison

Fill the blanks. One line against six.

Comparison matrix

QuestionThe classThe named tuple
Who writes __init__?youit is provided
What does printing show?an address, unless you write __str__the attribute names and values
How does == behave?by identity, unless you write __eq__by value, like a tuple
Can the attributes be changed?yes — it is mutableno — it is a tuple

The bottom row is the one that decides between them, and it is the subject of the next idea.

14. Explain it yourself: why is namedtuple a function?

Explain it to yourself

It could have been syntax.

Discussion prompt

namedtuple is an ordinary function that returns a class object. What does that mean about classes?

Hint: What can a function return?

Answer:

That a class is an object like any other, which chapter 15 said and this makes concrete: a function can build one and return it, because classes are values.

So there is nothing special about the class statement — it is a convenient way to make a class object, and namedtuple is another way, computed at run time from a name and a list of strings.

Which is why the returned object behaves exactly like a hand-written class: it is one. The printed form is identical, it is called the same way, and it can be inherited from — as the next idea's Pointier shows.

15. A named tuple is a tuple

Section

Section 2

16. Three ways to reach the same values

Concept

You can access the elements of the named tuple by name — but you can also treat a named tuple as a tuple.

>>> p = Point(1, 2)
>>> p.x, p.y
(1, 2)
>>> p[0], p[1]
(1, 2)
>>> x, y = p
>>> x, y
(1, 2)
AccessHow it looksNote
by namep.xthe named part
by positionp[0]the tuple part
by unpackingx, y = palso the tuple part

Which means everything from chapter 12 applies: comparison element by element, use as a dictionary key, unpacking in a for loop. The names are added on top of a tuple rather than replacing it.

Think Python, 2nd edition — Allen B. Downey §19.8-19.9, pp. 190-190

17. Picture it: names on top of positions

Picture it

The same two values, reachable three ways.

Figure (svg): One named tuple shown accessed by name, by index and by unpacking

The names are added to a tuple, so everything a tuple can do is still available.

Which makes it a genuine improvement on both alternatives: a tuple's operations with a class's readability.

18. Worked example: what it inherits from tuples

Worked example

Everything chapter 12 established.

>>> Point(1, 2) < Point(1, 3)
True                       # element by element
>>> d = {Point(0, 0): 'origin'}
>>> len(Point(1, 2))
2
OperationWhere it comes fromNote
comparisonelement by elementchapter 12's rule
as a dictionary keyhashablebecause immutable
lenthe number of elementstwo

Note the comparison.

Why: Tuples compare element by element from the left, so Points order by x and then by y — with no __lt__ to write.

Note the hashability.

Why: It is immutable, so it can be a dictionary key — which chapter 12 said was the reason tuples exist.

Note the sequence operations.

Why: len, indexing and unpacking all work, because it is a tuple.

Figure (svg): The state of the program after each line of Worked example what it inherits from tuples, drawn as a ladder with one rung per traced line

The whole run at once: each drop is one line of the program.

A great deal, none of it written. The named tuple is the tuple of chapter 12 with the attribute names of chapter 15 added.

Verify: Check the field order's effect on comparison.

Why: The order given to namedtuple decides the comparison priority, exactly as lesson 18a's Card tuple did — so ['x', 'y'] orders by x first. That is a design decision recorded in the definition, and it is invisible unless you know tuples compare from the left.

19. Predict: can you change it?

Prediction

It looks like a class.

p = Point(1, 2)
p.x = 5
AspectWhat is trueNote
a named tupleis a tupleimmutable
assigning an attributewould modify itnot allowed
the resultAttributeError

Predict first

What happens?

  • An AttributeError — a named tuple is immutable
  • It works, and p.x becomes 5
  • A TypeError about item assignment
  • It creates a new attribute

Correct: An AttributeError — a named tuple is a tuple, and tuples are immutable.

Why: The class-like interface hides this, which is why it catches people: chapter 15's hand-written Point could be modified freely. Immutability arrives with everything else the tuple provides — value comparison, hashability, indexing — so it is a consequence rather than a separate decision.

20. Worked example: the drawback

Worked example

The book states it immediately.

# 'Named tuples provide a quick way to define simple
#  classes. The drawback is that simple classes don't
#  always stay simple.'

# if you later want methods:
class Pointier(Point):
    # add more methods here
    ...

# or switch to a conventional class definition
StageWhat happensNote
the assumptionthe class stays simplea collection of values
when it does notyou want methodsor mutability
the two optionsinherit, or rewrite

Note what named tuples are for.

Why: They provide a quick way to define simple classes — collections of related values with no behaviour.

Note the drawback.

Why: Simple classes don't always stay simple. You might decide later that you want to add methods.

Note the two ways out.

Why: You could define a new class that inherits from the named tuple, or you could switch to a conventional class definition.

Figure (svg): A flowchart showing the two options when a named tuple stops being sufficient

Simple classes don't always stay simple, and the exit depends on what changed.

A feature whose assumption may expire. The escape routes exist and both cost something — one adds a class to explain, the other rewrites what was working.

Verify: Ask what inheriting does not fix.

Why: Immutability: a class inheriting from a named tuple is still a tuple underneath, so its attributes still cannot be assigned. If what you needed was a mutable object, inheriting does not help and the conventional class definition is the only route.

21. Trap: trying to assign to a named tuple's attribute

Trap

The trap

A program creates a Point and then writes p.x = 5 to move it.

Treat it as a class

Why: It has attributes, and chapter 15's Points could be modified.

It is a tuple, so it is immutable — the assignment raises AttributeError. The class-like interface hides a fundamental difference from the class it replaced.

The fix

Build a new one.

p = Point(5, p.y)

Why: Which is the immutable pattern: replace rather than modify.

Or use a conventional class if the object must change

Why: Which is one of the book's two escape routes.

Immutability is inherited from tuple along with everything else, so it is a consequence rather than a design choice. If a mutable object is what you need, a named tuple is the wrong tool from the start.

22. Sort: does this work on a named tuple?

Sorting

It is a tuple with names.

Sort into buckets

For each operation on a Point named tuple, does it work?

works
p.x; p[0]; x, y = p; Point(1, 2) == Point(1, 2)
fails
p.x = 5; p.append(3)
yes
Each is either a name access the named tuple adds or a tuple operation it inherits — indexing, unpacking and value comparison all come from the tuple underneath.
no
Both would modify the object, and tuples are immutable. One raises AttributeError and the other fails because tuples have no modifying methods at all.

23. Complete it: change a coordinate

Faded example

You cannot modify it, so build a new one.

Fill in the blanks

p = Point(1, 2)
p = Point(5, p.y)

Why: A named tuple is immutable, so p.x = 5 raises AttributeError — the replacement idiom is to build a new one from the parts you want. That is chapter 12's rule for tuples: you can't modify the elements, but you can replace one tuple with another.

24. Explain it: should I use named tuples for everything?

Explain it

One line instead of six is tempting.

Discussion prompt

A classmate wants to replace every simple class with a named tuple. Give them the book's own drawback.

Hint: What is the assumption?

Answer:

The assumption is that the class is and stays a collection of related values. Named tuples provide a quick way to define simple classes, and the drawback is that simple classes don't always stay simple.

When you later want methods, you can define a new class that inherits from the named tuple — or switch to a conventional class definition, which means rewriting what was working.

And inheriting does not restore mutability, since a named tuple is a tuple all the way down. So the question to ask up front is whether the object will ever need to change, because that answer cannot be revised cheaply.

25. Gathering keyword arguments

Section

Section 3

26. The double star, in a definition

Concept

The single star operator doesn't gather keyword arguments. To gather them, you can use the double star operator.

def printall(*args):
    print(args)

>>> printall(1, 2.0, third='3')
TypeError: printall() got an unexpected keyword argument 'third'

def printall(*args, **kwargs):
    print(args, kwargs)

>>> printall(1, 2.0, third='3')
(1, 2.0) {'third': '3'}
ParameterWhat it gathersNote
*argsthe positional argumentsas a tuple
**kwargsthe keyword argumentsas a dictionary
togetherany call at all

You can call the keyword gathering parameter anything you want, but kwargs is a common choice. The result is a dictionary that maps from keywords to values.

Think Python, 2nd edition — Allen B. Downey §19.8-19.9, pp. 190-191

27. Picture it: two stars, two containers

Picture it

One star gathers into a tuple and two gather into a dictionary.

Figure (svg): Two columns comparing the single and double star gather parameters

Each container matches what it holds: positions in a tuple, names in a dictionary.

Which is why the containers differ — a positional argument has only its position, and a keyword argument has a name, which is exactly a dictionary key.

28. Worked example: why the single star is not enough

Worked example

It gathers one kind of argument only.

>>> printall(1, 2.0, '3')
(1, 2.0, '3')              # all positional: fine

>>> printall(1, 2.0, third='3')
TypeError: printall() got an unexpected keyword argument 'third'
CallWhat happensNote
positional argumentsgathered by *argsany number
a keyword argumentnot gatherednowhere to put it
the errorunexpected keyword argumentnames the keyword

Recall what the single star does.

Why: You can call this function with any number of positional arguments — that is, arguments that don't have keywords.

Note what it does not.

Why: But the star operator doesn't gather keyword arguments, so a call with one has nowhere to put it.

Read the error.

Why: It names the keyword that had no home, which is precise enough to fix immediately.

Figure (svg): The state of the program after each line of Worked example why the single star is not enough, drawn as a ladder with one rung per traced line

The whole run at once: each drop is one line of the program.

A function accepting any number of positional arguments and no keyword ones. The two kinds are gathered separately because they are different kinds.

Verify: Check that the two parameters cover everything.

Why: def f(*args, **kwargs) accepts any call whatsoever — any number of positional arguments and any keywords. That combination is why it appears so often in code that forwards arguments to something else: it can receive anything and pass it on.

29. Predict: what does kwargs hold?

Prediction

One keyword argument.

def printall(*args, **kwargs):
    print(args, kwargs)

printall(1, 2.0, third='3')
ArgumentWhere it goesNote
1, 2.0positionalinto args
third='3'keywordinto kwargs
the outputa tuple and a dictionary

Predict first

What does this print?

  • (1, 2.0) {'third': '3'}
  • (1, 2.0, '3') {}
  • () {'third': '3'}
  • A TypeError about an unexpected keyword argument

Correct: (1, 2.0) {'third': '3'} — positional arguments go into the tuple and keyword arguments into the dictionary.

Why: The split is decided by how each argument was written at the call site rather than by its type. Option D is what the single-star version gives, since the star operator doesn't gather keyword arguments and there is nowhere for third to go.

30. Worked example: what arrives in each

Worked example

A tuple and a dictionary.

def printall(*args, **kwargs):
    print(args, kwargs)

>>> printall(1, 2.0, third='3')
(1, 2.0) {'third': '3'}
ParameterWhat it holdsNote
args(1, 2.0)the positional ones, in order
kwargs{'third': '3'}keyword to value
the splitby how they were writtennot by type

Note where each argument goes.

Why: Positional arguments into the tuple, keyword arguments into the dictionary — decided by how the call was written.

Note the dictionary's shape.

Why: It maps from keywords to values, so the parameter names become string keys.

Note the naming convention.

Why: You can call the keyword gathering parameter anything you want, but kwargs is a common choice — as args is for the other.

Figure (svg): A call diagram showing arguments split between a tuple and a dictionary

The split is decided by how each argument was written at the call site.

Two containers, populated by the two ways of writing an argument. Everything from lesson 12b about *args applies, with a dictionary in place of the tuple.

Verify: Check what happens with no keyword arguments.

Why: kwargs is an empty dictionary rather than missing, exactly as args is an empty tuple when no positional arguments are passed — so the body can loop over either without a special case. Both gather parameters always exist.

31. Trap: putting the gather parameters in the wrong order

Trap

The trap

A function is defined as def f(**kwargs, *args).

Order them however you like

Why: Both are gather parameters, so the order looks arbitrary.

Positional arguments have to be matched before keyword ones, so the tuple gatherer must come first. The definition is a syntax error, caught before the function is ever called.

The fix

Positional first, then keyword.

def f(*args, **kwargs)

Why: Which is the conventional and the only legal order.

With any required parameters before both

Why: The required-before-optional rule from lesson 13b.

The ordering follows how arguments are matched: by position first, then by name. A parameter list is read in the same order, which is why the two stars cannot be swapped.

32. Discriminate: which parameter receives it?

Discrimination

How was the argument written?

Sort into buckets

For a function def f(*args, **kwargs), where does each argument go?

args, the tuple
f(1); f('a', 'b'); f(1, 2, key=3) — the 1 and 2
kwargs, the dictionary
f(x=1); f(name='Terry'); f(1, 2, key=3) — the key=3
a1
Each is written without a parameter name, which makes it positional — and positional arguments are gathered into the tuple, in order.
k
Each is written as name=value, which makes it a keyword argument — and those are gathered into a dictionary mapping keywords to values.

33. Complete it: accept any call at all

Faded example

Two gather parameters, in order.

Fill in the blanks

def printall(*args, ******kwargs):
print(args, kwargs)

Why: The double star gathers keyword arguments into a dictionary, where the single star gathers positional ones into a tuple. Together they accept any call whatsoever, which is why the pair appears wherever a function forwards its arguments to something else.

34. Think it through: why a dictionary rather than a tuple?

Socratic

The single star uses a tuple.

Discussion prompt

Keyword arguments are gathered into a dictionary and positional ones into a tuple. Why the different containers?

Hint: What identifies each kind of argument?

Answer:

A positional argument is identified by where it appears, so a sequence is the natural home — position in the tuple corresponds to position in the call.

A keyword argument is identified by its name, and a container mapping names to values is a dictionary. Its position in the call is irrelevant, which is exactly what a keyword argument means.

So each container matches how its contents are identified — which is chapter 11's point about dictionaries generalising lists: the index can be anything, and here it is a parameter name.

35. Scattering a dictionary

Section

Section 4

36. The double star, at a call site

Concept

If you have a dictionary of keywords and values, you can use the scatter operator to call a function.

>>> d = dict(x=1, y=2)
>>> Point(**d)
Point(x=1, y=2)

>>> Point(d)
TypeError: __new__() missing 1 required positional argument: 'y'
CallWhat arrivesNote
Point(**d)the dictionary is spreadx=1 and y=2
Point(d)one positional argumentthe whole dictionary
the errornothing left for ywhich the message names

Without the scatter operator, the function would treat d as a single positional argument, so it would assign d to x and complain because there's nothing to assign to y.

Think Python, 2nd edition — Allen B. Downey §19.8-19.9, pp. 191-191

37. Picture it: the four star operations

Picture it

Two symbols, two directions each.

Figure (svg): The four gather and scatter operations formed by one and two stars

Lesson 12b's pair, completed: the same position rule with a second symbol.

The rule is unchanged from lesson 12b: in a definition it gathers, and at a call site it scatters. The number of stars decides which kind of argument.

38. Worked example: what happens without the stars

Worked example

The dictionary arrives as one argument.

>>> d = dict(x=1, y=2)
>>> Point(d)
TypeError: __new__() missing 1 required positional argument: 'y'
PartWhat happensNote
done objecta dictionary
as an argumentassigned to xpositionally
ynothing leftthe error

Note what is passed.

Why: Without the scatter operator, the function would treat d as a single positional argument.

Note where it lands.

Why: It would assign d to x — the whole dictionary, as the first coordinate.

Note the complaint.

Why: And complain because there's nothing to assign to y, which the message names.

Figure (svg): Two columns contrasting passing a dictionary whole with scattering it

One operator, and the difference between a container and its contents.

One argument where two were needed. The failure is the same shape as lesson 12b's divmod(t), and the fix is the same operator with one more star.

Verify: Compare with the single-star failure.

Why: divmod(t) said expected 2 arguments, got 1 and this says missing 1 required positional argument: y — both are argument-count errors from passing a container whole. The difference is only which star would have fixed it, decided by whether the container is a sequence or a mapping.

39. Predict: what does the scatter produce?

Prediction

A dictionary of two keys.

d = dict(x=1, y=2)
print(Point(**d))
PartWhat happensResult
**dspreads the dictionaryinto keyword arguments
x=1, y=2matched by nameto the parameters
the resultPoint(x=1, y=2)

Predict first

What does this print?

  • Point(x=1, y=2)
  • Point(x={'x': 1, 'y': 2}, y=None)
  • A TypeError about a missing argument
  • Point(x='x', y='y')

Correct: Point(x=1, y=2) — the scatter operator spreads the dictionary into keyword arguments matched by name.

Why: Without the operator, the function would treat d as a single positional argument, assign it to x, and complain because there's nothing to assign to y — which is option C. And option D is what one star would give, since iterating a dictionary yields its keys.

40. Worked example: why this is useful

Worked example

The book's own reason.

# a dictionary of options, built once
opts = dict(colour='red', width=3, style='dashed')

# passed to any function taking those keywords
draw(shape, **opts)
PartWhat it doesNote
the dictionarynamed optionsbuilt once
the scatterspreads themas keyword arguments
the benefitoptions can be passed aroundas data

Note when it helps.

Why: When you are working with functions that have a large number of parameters, it is often useful to create and pass around dictionaries that specify frequently used options.

Note what the dictionary buys.

Why: The options become data: they can be built up, stored, modified and passed between functions before being applied.

Note the scatter's role.

Why: It is the moment the data become arguments, which is the only place the function's parameter names are needed.

Figure (svg): The state of the program after each line of Worked example why this is useful, drawn as a ladder with one rung per traced line

The whole run at once: each drop is one line of the program.

A set of options handled as a value rather than as a call. That is the practical case for the operator, and it is why it pairs so naturally with the gather.

Verify: Ask what happens on a key the function does not accept.

Why: A TypeError naming an unexpected keyword argument — the same message as passing it directly, because the scatter really does turn each key into a written keyword argument. So the dictionary's keys must match the parameter names, which is a coupling worth being aware of.

41. Trap: using one star on a dictionary

Trap

The trap

A program writes f(*d) to pass a dictionary of options.

Use the scatter operator

Why: One star scattered a tuple, so it looks like the general form.

One star scatters a sequence, and iterating a dictionary gives its keys — so the function receives the key strings as positional arguments, which is legal and almost never what was meant.

The fix

Two stars for a dictionary.

f(**d)

Why: Which spreads it into keyword arguments.

And one star for a sequence

Why: f(*t), as in lesson 12b.

The number of stars is decided by what you have: a sequence, whose elements are positional, or a mapping, whose keys are names. Getting it wrong with a dictionary is quiet, because passing the keys is a legal call.

42. Sort: how many stars?

Sorting

A sequence or a mapping, in a definition or a call.

Sort into buckets

For each situation, which operator is needed?

one star
gathering any number of positional arguments; passing a tuple as separate positional arguments; def f(*things)
two stars
gathering keyword arguments; passing a dictionary as keyword arguments; f(**options)
one
Each involves positional arguments and a sequence — gathering them into a tuple, or spreading a sequence back out into separate arguments.
two
Each involves keyword arguments and a dictionary — gathering names and values into one, or spreading one back out into named arguments.

43. Complete it: pass a dictionary of options

Faded example

A mapping needs two stars.

Fill in the blanks

d = dict(x=1, y=2)
p = Point(******d)
print(p) # Point(x=1, y=2)

Why: The double star scatters a dictionary into keyword arguments matched by name. Passing d alone would make it one positional argument assigned to x, leaving nothing for y; passing *d would spread the keys as positional arguments, which is legal and meaningless.

44. Two truths and a lie: the star operators

Two truths and a lie

Two are true. Keep the lie.

Eliminate the wrong options

Rule out the two true statements.

  • A. In a definition the star gathers; at a call site it scatters
  • B. The double star gathers keyword arguments into a dictionary
  • C. One star and two stars are interchangeable

Survives elimination: C

Why: C ignores the distinction the second star exists for. One star handles positional arguments and sequences; two handle keyword arguments and dictionaries. Using one on a dictionary spreads its keys as positional arguments — legal, and almost never what was meant.

45. The end of the main text

Section

Section 5

46. What the last chapter was for

Concept

This is the end of the book's main text, and the chapter it ends with is deliberately optional: everything in it has a longer equivalent taught earlier.

The book's framing at the start of the chapter applies to all of them: Python provides a number of features that are not really necessary — you can write good code without them — but with them you can sometimes write code that's more concise, readable or efficient.

Think Python, 2nd edition — Allen B. Downey §19.8-19.9, pp. 183-191

47. Picture it: every feature, and what it replaced

Picture it

Nothing here was new work.

Figure (svg): Two columns pairing each feature of the chapter with what it replaces

Every item in the right-hand column is something the course built by hand first.

Which is why the chapter is last: each shortcut is only obvious once you could have written the long version, and each has a case where the long version is still right.

48. Worked example: the reservations, collected

Worked example

The book attaches one to nearly every feature.

# comprehensions: harder to debug - no print inside
# generators: exhausted after one pass
# defaultdict: reading a key creates it
# named tuples: simple classes don't always stay simple
FeatureIts reservationNote
each featurerecommendedwith a cost named
the patterna condition on its usenot a prohibition
the skillknowing which applies

Note that each has one.

Why: The book recommends every feature in this chapter and names a cost for nearly all of them.

Note the shape of each cost.

Why: None is this is bad — each is a condition under which the longer form remains the better choice.

Note what that asks of you.

Why: To choose rather than to adopt, which is what the whole book has been building towards.

Figure (svg): Each feature of the chapter with the condition attached to it

The book gives both halves for each feature.

Four features and four conditions. The recommendation and the reservation are both the book's, and taking only the first half is a misreading.

Verify: Check the strongest of them.

Why: The comprehension warning is the strongest — for beginners that means never — and it is attached to the feature the book calls concise, easy to read and usually faster. That combination is the clearest signal that the reservations are meant literally rather than as hedging.

49. Discriminate: which feature would you use?

Discrimination

Six situations from across the chapter.

Sort into buckets

For each, which of the chapter's features applies?

a named tuple
a class holding two numbers and nothing else
a set or a Counter
counting occurrences of each element; membership testing with no values to store
a star operator, or any
a function that forwards any arguments it receives; checking whether any element passes a test; passing a stored set of options to a function
nt
A collection of related values with no behaviour is exactly what a named tuple is for — and a lot of code to convey a small amount of information otherwise.
st
Both are dictionary patterns with a specialised container: counts for one, membership and uniqueness for the other.
ss
Each involves gathering or scattering arguments, or the early-exit search that any provides.

50. Worked example: what the whole book has been for

Worked example

The chapter's placement is the argument.

# 'One of my goals for this book has been to teach
#  you as little Python as possible. When there were
#  two ways to do something, I picked one and avoided
#  mentioning the other.'

# and now: 'I want to go back for some of the good
#  bits that got left behind.'
StageWhat happenedNote
the omissionsdeliberateone way at a time
the returnat the endonce the equivalents are known
the resultyou can choosewhich was the goal

Note the strategy.

Why: One way to do each thing, taught first — so that nothing had to be chosen between before either option was understood.

Note the return.

Why: The shortcuts arrive last, when each one is recognisable as a compression of something already written.

Note the goal.

Why: The same one lesson 17a named: being comfortable converting between forms so you can choose the best for whatever you are doing.

Figure (svg): The state of the program after each line of Worked example what the whole book has been for, drawn as a ladder with one rung per traced line

The whole run at once: each drop is one line of the program.

A deliberate sequence, and this chapter is its conclusion. The features are the reward for having done without them.

Verify: Check the claim against your own reaction to sets.

Why: The set section makes sense immediately because chapter 13's dictionary of Nones is fresh — the container arrives as an answer rather than as a new thing to learn. Encountering set first would have taught the type without the reason for it, which is the difference the ordering buys.

51. Trap: treating the shortcuts as the real Python

Trap

The trap

A student concludes that the loops and dictionaries they learned were a teaching fiction, and the shortcuts are how Python is really written.

Prefer the idiomatic form

Why: The shortcuts are more concise and often faster.

The book says the opposite: these features are not really necessary and you can write good code without them. The longer forms remain correct, readable, and sometimes better — a loop where a comprehension would need debugging, a dictionary where a KeyError is wanted.

The fix

Treat them as additions to what you can choose between.

Each has a condition, and each has cases it does not fit

Why: Which the book names for nearly all of them.

And the ability to convert is the actual skill

Why: Rather than a preference for either form.

Sometimes more concise, readable or efficient is the claim. Sometimes is doing real work in that sentence, and reading past it turns a set of options into a set of rules.

52. Predict: which claim does the book make?

Prediction

The chapter's framing.

# 'Python provides a number of features that are
#  not really necessary - you can write good code
#  without them - but with them you can sometimes
#  write code that's more concise, readable or
#  efficient, and sometimes all three.'
QuestionThe book's answerNote
necessary?noexplicitly
better?sometimesand in three ways
all three?sometimesnot always

Predict first

What does the book claim for the features in this chapter?

  • They are not necessary, and can sometimes make code more concise, readable or efficient
  • They are the correct way to write Python
  • They are always faster than the alternatives
  • They should replace the forms taught earlier

Correct: They are not necessary, and can sometimes make code more concise, readable or efficient.

Why: The wording is careful in both directions: not really necessary, and sometimes better in three specific ways that do not always arrive together. Reading past the sometimes turns a set of options into a set of rules, which is exactly what the chapter's individual reservations argue against.

53. Complete it: forward every argument

Faded example

Both gather parameters, in order.

Fill in the blanks

def wrapper(*args, **kwargs):
return target(*args, ******kwargs)

Why: The pair gathers any call at all and then scatters both parts back out — positional arguments with one star and keyword arguments with two. That round trip is why the two gather parameters appear together so often: a function written this way can receive anything and pass it on unchanged.

54. Where learning the long way first pays off

Real world

A shortcut you could not have appreciated earlier.

Discussion prompt

Think of something you learned the laborious way before discovering a shortcut. What did the laborious version leave you with?

Hint: What happens when the shortcut does not apply?

Answer:

An understanding of what the shortcut is doing, which is what lets you recognise the cases it does not cover — and there is always a case.

And the ability to fall back. Someone who only knows the shortcut is stuck the first time it does not fit; someone who learned the long way has somewhere to go.

Which is the book's stated strategy: teach as little Python as possible, then come back for the good bits. The shortcuts in this chapter are useful precisely because you could write every one of them out by hand.

55. Compare: one star and two

Comparison

Fill the blanks. The position rule is the same for both.

Comparison matrix

QuestionOne starTwo stars
Which arguments?positionalkeyword
Gathered into what?a tuplea dictionary
Scattered from what?a sequencea dictionary
How is the direction decided?definition gathers, call scattersdefinition gathers, call scatters

The bottom row is unchanged from lesson 12b — the second star adds a kind of argument rather than a new rule.

56. The procedure: replacing a simple class with a named tuple

Pattern

Five steps, and the first is the one that decides whether to start.

  1. Check the class is a collection of related values with no behaviour and no need to change.
  2. Call namedtuple with the class name and the attribute names, all as strings.
  3. Use the returned class object exactly as you would a hand-written one.
  4. Take the inherited tuple behaviour into account: value comparison, hashability, indexing and immutability.
  5. If methods are wanted later, inherit from the named tuple; if mutability is, switch to a conventional class definition.

Step 1's no need to change is the one that cannot be revised cheaply. Methods can be added by inheriting; immutability is inherited all the way down.

Python documentation — collections — Container datatypes collections — Container datatypes

57. Check yourself 1 of 3: named tuples

Check

The class-like interface hides something.

Point = namedtuple('Point', ['x', 'y'])
p = Point(1, 2)
p.x = 5
AspectWhat is trueNote
a named tupleis a tupleimmutable
the assignmentwould modify it
the resultAttributeError

Check your understanding

What happens on the last line?

  • A. An AttributeError — a named tuple is immutable (correct)
  • B. p.x becomes 5
  • C. A new attribute is created
  • D. A KeyError

Answer: A

Why: A named tuple is a tuple, so its elements cannot be changed — which is inherited along with value comparison, hashability and indexing. Chapter 15's hand-written Point could be modified freely, which is why the class-like interface makes this surprising.

Why B tempts people
This is what the hand-written class would do; the named tuple replaced mutability with a set of tuple properties.
Why C tempts people
Named tuples have a fixed set of fields, so no new attribute can be added either.
Why D tempts people
KeyError belongs to dictionaries. Attribute assignment on an immutable object raises AttributeError.

58. Check yourself 2 of 3: gathering

Check

One star and two.

Check your understanding

Which parameter gathers keyword arguments?

  • A. **kwargs — into a dictionary (correct)
  • B. *args — into a tuple
  • C. Either, depending on the call
  • D. Neither; keyword arguments cannot be gathered

Answer: A

Why: The star operator doesn't gather keyword arguments — a call with one raises TypeError about an unexpected keyword argument. The double star gathers them into a dictionary that maps from keywords to values, and kwargs is the conventional name for it.

Why B tempts people
One star gathers positional arguments only, which is what lesson 12b covered.
Why C tempts people
The split is by how each argument is written, and each parameter takes one kind.
Why D tempts people
That was true of the single star alone, which is exactly why the double star exists.

59. Check yourself 3 of 3: scattering

Check

A dictionary at a call site.

d = dict(x=1, y=2)
Point(d)
PartWhat happensNote
done positional argumentthe whole dictionary
xreceives d
ynothing leftTypeError

Check your understanding

Why does this raise?

  • A. The dictionary is passed as one positional argument, so there is nothing for y (correct)
  • B. Dictionaries cannot be passed to functions
  • C. The keys do not match the parameter names
  • D. Point takes no arguments

Answer: A

Why: Without the scatter operator, the function treats d as a single positional argument, assigns it to x, and complains because there's nothing to assign to y. Writing Point(**d) spreads it into keyword arguments matched by name.

Why B tempts people
They can, and often are — the issue is that this call passes one where two values were wanted.
Why C tempts people
The keys do match, which is exactly why the double star works.
Why D tempts people
It takes two, and the message names the one that went unfilled.

60. Where this shows up outside this course

Real world

A record with named fields, and a set of options as data.

Discussion prompt

Think of settings you have saved and reused rather than re-entering. What does keeping them as data allow that typing them each time does not?

Hint: What can you do with a saved set that you cannot with a typed one?

Answer:

They can be stored, copied, modified in one place, shared, and applied to several things — none of which is possible when the settings exist only as something you type each time.

Which is the book's own reason for the scatter operator: when working with functions that have a large number of parameters, it is often useful to create and pass around dictionaries that specify frequently used options.

And the scatter is the moment the data become a call. Until then the options are a value like any other, which is what makes them manipulable — the same benefit as naming a tuple's fields rather than remembering their positions.

61. Confidence wager: commit before you check

Commit first

Answer, then rate your confidence.

Predict first

You create a Point named tuple and write p.x = 5. What happens?

  • An AttributeError — a named tuple is a tuple, and tuples are immutable
  • It works, exactly as it did for chapter 15's Point class
  • A TypeError about item assignment
  • It creates a new attribute alongside the existing ones

Correct: An AttributeError — a named tuple is a tuple, and tuples are immutable.

Why: This is the thing to hold about named tuples: the interface looks like a class and the object is a tuple. Everything good about that is inherited — value comparison, so two Points with the same coordinates are equal where chapter 15's were not; hashability, so a Point can be a dictionary key; indexing and unpacking, so p[0] and x, y = p both work. And immutability comes with the rest, which is a genuine difference from the class it replaces, since chapter 15's Point could be modified freely. The book's own drawback is the related one: named tuples provide a quick way to define simple classes, and simple classes don't always stay simple. If you later want methods you can define a class that inherits from the named tuple — but that does not restore mutability, because it is a tuple all the way down. So will this ever need to change? is the question to answer before choosing, since it is the one that cannot be revised cheaply afterwards.

62. Explain it to someone else

Explain it

One line replacing a class.

Discussion prompt

A classmate asks whether named tuples make chapter 15 obsolete. Give them what a named tuple provides and what it gives up.

Hint: What can a class do that a tuple cannot?

Answer:

It provides a great deal: __init__ and __str__ without writing them, value comparison, hashability, and access by name, by index and by unpacking. For a collection of related values that is most of what a class was for.

It gives up mutability and, initially, methods — and the book's own drawback is that simple classes don't always stay simple. Methods can be added by inheriting from the named tuple; mutability cannot be added at all.

So it replaces one specific kind of class: a record with no behaviour that never changes. Everything chapter 15 taught still applies to the rest, and understanding the class version is what makes the shortcut readable.

63. Exit ticket

Exit ticket

One honest answer. It decides what the next lesson opens with.

Predict first

Which of these is still least solid for you?

  • Creating a named tuple, and what it provides for you
  • What a named tuple inherits from tuples, including immutability
  • Gathering keyword arguments with the double star
  • Scattering a dictionary at a call site

Correct: Whichever you picked is the right answer — this one is for you, not for a mark.

Why: The creation is one line with two quoted arguments, and the thing worth noticing is that a function returns a class. What it inherits is where the surprises are — value equality is a gift and immutability is a constraint, and both come from the same place. The double star completes lesson 12b's pair with one new rule: the number of stars follows the kind of argument. And the scatter is the operator that turns a dictionary of options into a call, which is the practical reason the pair exists.

64. Synthesis: draw the map of this lesson

Connect it up

One page, from memory.

Draw it

Write chapter 15's Point class and the namedtuple line that replaces it, side by side, and mark which methods the second one provides. Beside them, list three things a named tuple inherits from tuples and one thing it gives up. Underneath, draw the four star operations — one and two stars, in a definition and at a call — and write one example of each.

65. What you can do now

Recap

Two pages, and the end of the book's main text.

If you remember one thingIt is this
From namedtupleA function that returns a class, because classes are objects.
From the dual natureValue equality is a gift; immutability is a constraint. Both come from tuple.
From the drawbackMethods can be added by inheriting. Mutability cannot.
From the double starOne star for positions and sequences; two for names and dictionaries.
From the chapterNot necessary, and sometimes better — with sometimes doing real work.

The next two lessons cover appendix A, on debugging: the three kinds of error, what their messages actually tell you, and the strategies for a semantic error — the kind that produces no message at all.

Think Python, 2nd edition — Allen B. Downey §19.8-19.9, pp. 189-190 — everything on these slides traces back here

Sources

  1. Think Python, 2nd edition — Allen B. Downey — Allen B. Downey, Think Python: How to Think Like a Computer Scientist, 2nd edition (Green Tea Press, 2015), §19.8-19.9, pp. 189-190
  2. Python documentation — collections — Container datatypes
  3. Python documentation — More Control Flow Tools

Want this taught 1-on-1? Alexander tutors Python — $55/session, free consultation.

Book on Wyzant · Text (657) 465-8108