Learning Python (2013)

---

^[64^] As of this chapter’s interaction listings, I’ve started omitting some blank lines and shortening some hex addresses to 32 bits in object displays, to reduce size and clutter. I’m going to assume that by this point in the book, you’ll find such small details irrelevant.

New-Style Class Extensions

Beyond the changes described in the prior section (some of which, frankly, may seem too academic and obscure to matter to many readers of this book), new-style classes provide a handful of more advanced class tools that have more direct and practical application—slots, properties,descriptors, and more. The following sections provide an overview of each of these additional features, available for new-style class in Python 2.X and all classes in Python 3.X. Also in this extensions category are the __mro__ attribute and the super call, both covered elsewhere—the former in the previous section to explore a change, and the latter postponed until chapter end to serve as a larger case study.

Slots: Attribute Declarations

By assigning a sequence of string attribute names to a special __slots__ class attribute, we can enable a new-style class to both limit the set of legal attributes that instances of the class will have, and optimize memory usage and possibly program speed. As we’ll find, though, slots should be used only in applications that clearly warrant the added complexity. They will complicate your code, may complicate or break code you may use, and require universal deployment to be effective.

Slot basics

To use slots, assign a sequence of string names to the special __slots__ variable and attribute at the top level of a class statement: only those names in the __slots__ list can be assigned as instance attributes. However, like all names in Python, instance attribute names must still be assigned before they can be referenced, even if they’re listed in __slots__:

>>> class limiter(object):

__slots__ = ['age', 'name', 'job']

>>> x = limiter()

>>> x.age # Must assign before use

AttributeError: age

>>> x.age = 40 # Looks like instance data

>>> x.age

40

>>> x.ape = 1000 # Illegal: not in __slots__

AttributeError: 'limiter' object has no attribute 'ape'

This feature is envisioned as both a way to catch typo errors like this (assignments to illegal attribute names not in __slots__ are detected) as well as an optimization mechanism.

Allocating a namespace dictionary for every instance object can be expensive in terms of memory if many instances are created and only a few attributes are required. To save space, instead of allocating a dictionary for each instance, Python reserves just enough space in each instance to hold a value for each slot attribute, along with inherited attributes in the common class to manage slot access. This might additionally speed execution, though this benefit is less clear and might vary per program, platform, and Python.

Slots are also something of a major break with Python’s core dynamic nature, which dictates that any name may be created by assignment. In fact, they imitate C++ for efficiency at the expense of flexibility, and even have the potential to break some programs. As we’ll see, slots also come with a plethora of special-case usage rules. Per Python’s own manual, they should not be used except in clearly warranted cases—they are difficult to use correctly, and are, to quote the manual:

best reserved for rare cases where there are large numbers of instances in a memory-critical application.

In other words, this is yet another feature that should be used only if clearly warranted. Unfortunately, slots seem to be showing up in Python code much more often than they should; their obscurity seems to be a draw in itself. As usual, knowledge is your best ally in such things, so let’s take a quick look here.

NOTE

In Python 3.3, non-slots attribute space requirements have been reduced with a key-sharing dictionary model, where the __dict__ dictionaries used for objects’ attributes may share part of their internal storage, including that of their keys. This may lessen some of the value of __slots__ as an optimization tool; per benchmark reports, this change reduces memory use by 10% to 20% for object-oriented programs, gives a small improvement in speed for programs that create many similar objects, and may be optimized further in the future. On the other hand, this won’t negate the presence of __slots__ in existing code you may need to understand!

Slots and namespace dictionaries

Potential benefits aside, slots can complicate the class model—and code that relies on it—substantially. In fact, some instances with slots may not have a __dict__ attribute namespace dictionary at all, and others will have data attributes that this dictionary does not include. To be clear: this is a major incompatibility with the traditional class model—one that can complicate any code that accesses attributes generically, and may even cause some programs to fail altogether.

For instance, programs that list or access instance attributes by name string may need to use more storage-neutral interfaces than __dict__ if slots may be used. Because an instance’s data may include class-level names such as slots—either in addition to or instead of namespace dictionary storage—both attribute sources may need to be queried for completeness.

Let’s see what this means in terms of code, and explore more about slots along the way. First off, when slots are used, instances do not normally have an attribute dictionary—instead, Python uses the class descriptors feature introduced ahead to allocate and manage space reserved for slot attributes in the instance. In Python 3.X, and in 2.X for new-style classes derived from object:

>>> class C: # Requires "(object)" in 2.X only

__slots__ = ['a', 'b'] # __slots__ means no __dict__ by default

>>> X = C()

>>> X.a = 1

>>> X.a

1

>>> X.__dict__

AttributeError: 'C' object has no attribute '__dict__'

However, we can still fetch and set slot-based attributes by name string using storage-neutral tools such as getattr and setattr (which look beyond the instance __dict__ and thus include class-level names like slots) and dir (which collects all inherited names throughout a class tree):

>>> getattr(X, 'a')

1

>>> setattr(X, 'b', 2) # But getattr() and setattr() still work

>>> X.b

2

>>> 'a' in dir(X) # And dir() finds slot attributes too

True

>>> 'b' in dir(X)

True

Also keep in mind that without an attribute namespace dictionary, it’s not possible to assign new names to instances that are not names in the slots list:

>>> class D: # Use D(object) for same result in 2.X

__slots__ = ['a', 'b']

def __init__(self):

self.d = 4 # Cannot add new names if no __dict__

>>> X = D()

AttributeError: 'D' object has no attribute 'd'

We can still accommodate extra attributes, though, by including __dict__ explicitly in __slots__, in order to create an attribute namespace dictionary too:

>>> class D:

__slots__ = ['a', 'b', '__dict__'] # Name __dict__ to include one too

c = 3 # Class attrs work normally

def __init__(self):

self.d = 4 # d stored in __dict__, a is a slot

>>> X = D()

>>> X.d

4

>>> X.c

3

>>> X.a # All instance attrs undefined until assigned

AttributeError: a

>>> X.a = 1

>>> X.b = 2

In this case, both storage mechanisms are used. This renders __dict__ too limited for code that wishes to treat slots as instance data, but generic tools such as getattr still allow us to process both storage forms as a single set of attributes:

>>> X.__dict__ # Some objects have both __dict__ and slot names

{'d': 4} # getattr() can fetch either type of attr

>>> X.__slots__

['a', 'b', '__dict__']

>>> getattr(X, 'a'), getattr(X, 'c'), getattr(X, 'd') # Fetches all 3 forms

(1, 3, 4)

Because dir also returns all inherited attributes, though, it might be too broad in some contexts; it also includes class-level methods, and even all object defaults. Code that wishes to list just instance attributes may in principle still need to allow for both storage forms explicitly. We might at first naively code this as follows:

>>> for attr in list(X.__dict__) + X.__slots__: # Wrong...

print(attr, '=>', getattr(X, attr))

Since either can be omitted, we may more correctly code this as follows, using getattr to allow for defaults—a noble but nonetheless inaccurate approach, as the next section will explain:

>>> for attr in list(getattr(X, '__dict__', [])) + getattr(X, '__slots__', []):

print(attr, '=>', getattr(X, attr))

d => 4

a => 1 # Less wrong...

b => 2

__dict__ => {'d': 4}

Multiple __slot__ lists in superclasses

The preceding code works in this specific case, but in general it’s not entirely accurate. Specifically, this code addresses only slot names in the lowest __slots__ attribute inherited by an instance, but slot lists may appear more than once in a class tree. That is, a name’s absence in the lowest__slots__ list does not preclude its existence in a higher __slots__. Because slot names become class-level attributes, instances acquire the union of all slot names anywhere in the tree, by the normal inheritance rule:

>>> class E:

__slots__ = ['c', 'd'] # Superclass has slots

>>> class D(E):

__slots__ = ['a', '__dict__'] # But so does its subclass

>>> X = D()

>>> X.a = 1; X.b = 2; X.c = 3 # The instance is the union (slots: a, c)

>>> X.a, X.c

(1, 3)

Inspecting just the inherited slots list won’t pick up slots defined higher in a class tree:

>>> E.__slots__ # But slots are not concatenated

['c', 'd']

>>> D.__slots__

['a', '__dict__']

>>> X.__slots__ # Instance inherits *lowest* __slots__

['a', '__dict__']

>>> X.__dict__ # And has its own an attr dict

{'b': 2}

>>> for attr in list(getattr(X, '__dict__', [])) + getattr(X, '__slots__', []):

print(attr, '=>', getattr(X, attr))

b => 2 # Other superclass slots missed!

a => 1

__dict__ => {'b': 2}

>>> dir(X) # But dir() includes all slot names

[...many names omitted... 'a', 'b', 'c', 'd']

In other words, in terms of listing instance attributes generically, one __slots__ isn’t always enough—they are potentially subject to the full inheritance search procedure. See the earlier mapattrs-slots.py for another example of slots appearing in multiple superclasses. If multiple classes in a class tree have their own __slots__ attributes, generic programs must develop other policies for listing attributes—as the next section explains.

Handling slots and other “virtual” attributes generically

At this point, you may wish to review the discussion of slots policy options at the coverage of the lister.py display mix-in classes near the end of the preceding chapter—a prime example of why generic programs may need to care about slots. Such tools that attempt to list instance data attributes generically must account for slots, and perhaps other such “virtual” instance attributes like properties and descriptors discussed ahead—names that similarly reside in classes but may provide attribute values for instances on request. Slots are the most data-centric of these, but are representative of a larger category.

Such attributes require inclusive approaches, special handling, or general avoidance—the latter of which becomes unsatisfactory as soon as any programmer uses slots in subject code. Really, class-level instance attributes like slots probably necessitate a redefinition of the term instance data—as locally stored attributes, the union of all inherited attributes, or some subset thereof.

For example, some programs might classify slot names as attributes of classes instead of instances; these attributes do not exist in instance namespace dictionaries, after all. Alternatively, as shown earlier, programs can be more inclusive by relying on dir to fetch all inherited attribute names and getattr to fetch their corresponding values for the instance—without regard to their physical location or implementation. If you must support slots as instance data, this is likely the most robust way to proceed:

>>> class Slotful:

__slots__ = ['a', 'b', '__dict__']

def __init__(self, data):

self.c = data

>>> I = Slotful(3)

>>> I.a, I.b = 1, 2

>>> I.a, I.b, I.c # Normal attribute fetch

(1, 2, 3)

>>> I.__dict__ # Both __dict__ and slots storage

{'c': 3}

>>> [x for x in dir(I) if not x.startswith('__')]

['a', 'b', 'c']

>>> I.__dict__['c'] # __dict__ is only one attr source

3

>>> getattr(I, 'c'), getattr(I, 'a') # dir+getattr is broader than __dict__

(3, 1) # applies to slots, properties, descrip

>>> for a in (x for x in dir(I) if not x.startswith('__')):

print(a, getattr(I, a))

a 1

b 2

c 3

Under this dir/getattr model, you can still map attributes to their inheritance sources, and filter them more selectively by source or type if needed, by scanning the MRO—as we did earlier in both mapattrs.py and its application to slots in mapattrs-slots.py. As an added bonus, such tools and policies for handling slots will potentially apply automatically to properties and descriptors too, though these attributes are more explicitly computed values, and less obviously instance-related data than slots.

Also keep in mind that this is not just a tools issue. Class-based instance attributes like slots also impact the traditional coding of the __setattr__ operator overloading method we met in Chapter 30. Because slots and some other attributes are not stored in the instance __dict__, and may even imply its absence, new-style classes must instead generally run attribute assignments by routing them to the object superclass. In practice, this may make this method fundamentally different in some classic and new-style classes.

Slot usage rules

Slot declarations can appear in multiple classes in a class tree, but when they do they are subject to a number of constraints that are somewhat difficult to rationalize unless you understand the implementation of slots as class-level descriptors for each slot name that are inherited by the instances where the managed space is reserved (descriptors are an advanced tool we’ll study in detail in the last part of this book):

§ Slots in subs are pointless when absent in supers: If a subclass inherits from a superclass without a __slots__, the instance __dict__ attribute created for the superclass will always be accessible, making a __slots__ in the subclass largely pointless. The subclass still manages its slots, but doesn’t compute their values in any way, and doesn’t avoid a dictionary—the main reason to use slots.

§ Slots in supers are pointless when absent in subs: Similarly, because the meaning of a __slots__ declaration is limited to the class in which it appears, subclasses will produce an instance __dict__ if they do not define a __slots__, rendering a __slots__ in a superclass largely pointless.

§ Redefinition renders super slots pointless: If a class defines the same slot name as a superclass, its redefinition hides the slot in the superclass per normal inheritance. You can access the version of the name defined by the superclass slot only by fetching its descriptor directly from the superclass.

§ Slots prevent class-level defaults: Because slots are implemented as class-level descriptors (along with per-instance space), you cannot use class attributes of the same name to provide defaults as you can for normal instance attributes: assigning the same name in the class overwrites the slot descriptor.

§ Slots and __dict__: As shown earlier, __slots__ preclude both an instance __dict__ and assigning names not listed, unless __dict__ is listed explicitly too.

We’ve already seen the last of these in action, and the earlier mapattrs-slots.py illustrates the third. It’s easy to demonstrate how the new rules here translate to actual code—most crucially, a namespace dictionary is created when any class in a tree omits slots, thereby negating the memory optimization benefit:

>>> class C: pass # Bullet 1: slots in sub but not super

>>> class D(C): __slots__ = ['a'] # Makes instance dict for nonslots

>>> X = D() # But slot name still managed in class

>>> X.a = 1; X.b = 2

>>> X.__dict__

{'b': 2}

>>> D.__dict__.keys()

dict_keys([... 'a', '__slots__', ...])

>>> class C: __slots__ = ['a'] # Bullet 2: slots in super but not sub

>>> class D(C): pass # Makes instance dict for nonslots

>>> X = D() # But slot name still managed in class

>>> X.a = 1; X.b = 2

>>> X.__dict__

{'b': 2}

>>> C.__dict__.keys()

dict_keys([... 'a', '__slots__', ...])

>>> class C: __slots__ = ['a'] # Bullet 3: only lowest slot accessible

>>> class D(C): __slots__ = ['a']

>>> class C: __slots__ = ['a']; a = 99 # Bullet 4: no class-level defaults

ValueError: 'a' in __slots__ conflicts with class variable

In other words, besides their program-breaking potential, slots essentially require both universal and careful deployment to be effective—because slots do not compute values dynamically like properties (coming up in the next section), they are largely pointless unless each class in a tree uses them and is cautious to define only new slot names not defined by other classes. It’s an all-or-nothing feature—an unfortunate property shared by the super call discussed ahead:

>>> class C: __slots__ = ['a'] # Assumes universal use, differing names

>>> class D(C): __slots__ = ['b']

>>> X = D()

>>> X.a = 1; X.b = 2

>>> X.__dict__

AttributeError: 'D' object has no attribute '__dict__'

>>> C.__dict__.keys(), D.__dict__.keys()

(dict_keys([... 'a', '__slots__', ...]), dict_keys([... 'b', '__slots__', ...]))

Such rules—among others regarding weak references omitted here for space—are part of the reason slots are not generally recommended, except in pathological cases where their space reduction is significant. Even then, their potential to complicate or break code should be ample cause to carefully consider the tradeoffs. Not only must they be spread almost neurotically throughout a framework, they may also break tools you rely on.

Example impacts of slots: ListTree and mapattrs

As a more realistic example of slots’ effects, due to the first bullet in the prior section, Chapter 31’s ListTree class does not fail when mixed in to a class that defines __slots__, even though it scans instance namespace dictionaries. The lister class’s own lack of slots is enough to ensure that the instance will still have a __dict__, and hence not trigger an exception when fetched or indexed. For example, both of the following display without error—the second also allows names not in the slots list to be assigned as instances attributes, including any required by the superclass:

class C(ListTree): pass

X = C() # OK: no __slots__ used

print(X)

class C(ListTree): __slots__ = ['a', 'b'] # OK: superclass produces __dict__

X = C()

X.c = 3

print(X) # Displays c at X, a and b at C

The following classes display correctly as well—any nonslot class like ListTree generates an instance __dict__, and can thus safely assume its presence:

class A: __slots__ = ['a'] # Both OK by bullet 1 above

class B(A, ListTree): pass

class A: __slots__ = ['a']

class B(A, ListTree): __slots__ = ['b'] # Displays b at B, a at A

Although it renders subclass slots pointless, this is a positive side effect for tools classes like ListTree (and its Chapter 28 predecessor). In general, though, some tools might need to catch exceptions when __dict__ is absent or use a hasattr or getattr to test or provide defaults if slot usage may preclude a namespace dictionary in instance objects inspected.

For example, you should now be able to understand why the mapattrs.py program earlier in this chapter must check for the presence of a __dict__ before fetching it—instance objects created from classes with __slots__ won’t have one. In fact, if we use the highlighted alternative line in the following, the mapattrs function fails with an exception when attempting to look for an attribute name in the instance at the front of the inheritance path sequence:

def mapattrs(instance, withobject=False, bysource=False):

for attr in dir(instance):

for obj in inherits:

if attr in obj.__dict__: # May fail if __slots__ used

>>> class C: __slots__ = ['a']

>>> X = C()

>>> mapattrs(X)

AttributeError: 'C' object has no attribute '__dict__'

Either of the following works around the issue, and allows the tool to support slots—the first provides a default, and the second is more verbose but seems marginally more explicit in its intent:

if attr in getattr(obj, '__dict__', {}):

if hasattr(obj, '__dict__') and attr in obj.__dict__:

As mentioned earlier, some tools may benefit from mapping dir results to objects in the MRO this way, instead of scanning an instance __dict__ in general—without this more inclusive approach, attributes implemented by class-level tools like slots won’t be reported as instance data. Even so, this doesn’t necessarily excuse such tools from allowing for a missing __dict__ in the instance too!

What about slots speed?

Finally, while slots primarily optimize memory use, their speed impact is less clear-cut. Here’s a simple test script using the timeit techniques we studied in Chapter 21. For both the slots and nonslots (instance dictionary) storage models, it makes 1,000 instances, assigns and fetches 4 attributes on each, and repeats 1,000 times—for both models taking the best of 3 runs that each exercise a total of 8M attribute operations:

# File slots-test.py

from __future__ import print_function

import timeit

base = """

Is = []

for i in range(1000):

X = C()

X.a = 1; X.b = 2; X.c = 3; X.d = 4

t = X.a + X.b + X.c + X.d

Is.append(X)

"""

stmt = """

class C:

__slots__ = ['a', 'b', 'c', 'd']

""" + base

print('Slots =>', end=' ')

print(min(timeit.repeat(stmt, number=1000, repeat=3)))

stmt = """

class C:

pass

""" + base

print('Nonslots=>', end=' ')

print(min(timeit.repeat(stmt, number=1000, repeat=3)))

At least on this code, on my laptop, and in my installed versions (Python 3.3 and 2.7), the best times imply that slots are slightly quicker in 3.X and a wash in 2.X, though this says little about memory space, and is prone to change arbitrarily in the future:

c:\code> py −3 slots-test.py

Slots => 0.7780903942045899

Nonslots=> 0.9888108080898417

c:\code> py −2 slots-test.py

Slots => 0.80868754371

Nonslots=> 0.802224740747

For more on slots in general, see the Python standard manual set. Also watch for the Private decorator case study of Chapter 39—an example that naturally allows for attributes based on both __slots__ and __dict__ storage, by using delegation and storage-neutral accessor tools likegetattr.

Properties: Attribute Accessors

Our next new-style extension is properties—a mechanism that provides another way for new-style classes to define methods called automatically for access or assignment to instance attributes. This feature is similar to properties (a.k.a. “getters” and “setters”) in languages like Java and C#, but in Python is generally best used sparingly, as a way to add accessors to attributes after the fact as needs evolve and warrant. Where needed, though, properties allow attribute values to be computed dynamically without requiring method calls at the point of access.

Though properties cannot support generic attribute routing goals, at least for specific attributes they are an alternative to some traditional uses of the __getattr__ and __setattr__ overloading methods we first studied in Chapter 30. Properties have a similar effect to these two methods, but by contrast incur an extra method call only for accesses to names that require dynamic computation—other nonproperty names are accessed normally with no extra calls. Although __getattr__ is invoked only for undefined names, the __setattr__ method is instead called for assignment to every attribute.

Properties and slots are related too, but serve different goals. Both implement instance attributes that are not physically stored in instance namespace dictionaries—a sort of “virtual” attribute—and both are based on the notion of class-level attribute descriptors. In contrast, slots manage instance storage, while properties intercept access and compute values arbitrarily. Because their underlying descriptor implementation tool is too advanced for us to cover here, properties and descriptors both get full treatment in Chapter 38.

Property basics

As a brief introduction, though, a property is a type of object assigned to a class attribute name. You generate a property by calling the property built-in function, passing in up to three accessor methods—handlers for get, set, and delete operations—as well as an optional docstring for the property. If any argument is passed as None or omitted, that operation is not supported.

The resulting property object is typically assigned to a name at the top level of a class statement (e.g., name=property()), and a special @ syntax we’ll meet later is available to automate this step. When thus assigned, later accesses to the class property name itself as an object attribute (e.g., obj.name) are automatically routed to one of the accessor methods passed into the property call.

For example, we’ve seen how the __getattr__ operator overloading method allows classes to intercept undefined attribute references in both classic and new-style classes:

>>> class operators:

def __getattr__(self, name):

if name == 'age':

return 40

else:

raise AttributeError(name)

>>> x = operators()

>>> x.age # Runs __getattr__

40

>>> x.name # Runs __getattr__

AttributeError: name

Here is the same example, coded with properties instead; note that properties are available for all classes but require the new-style object derivation in 2.X to work properly for intercepting attribute assignments (and won’t complain if you forget this—but will silently overwrite your property with the new data!):

>>> class properties(object): # Need object in 2.X for setters

def getage(self):

return 40

age = property(getage, None, None, None) # (get, set, del, docs), or use @

>>> x = properties()

>>> x.age # Runs getage

40

>>> x.name # Normal fetch

AttributeError: 'properties' object has no attribute 'name'

For some coding tasks, properties can be less complex and quicker to run than the traditional techniques. For example, when we add attribute assignment support, properties become more attractive—there’s less code to type, and no extra method calls are incurred for assignments to attributes we don’t wish to compute dynamically:

>>> class properties(object): # Need object in 2.X for setters

def getage(self):

return 40

def setage(self, value):

print('set age: %s' % value)

self._age = value

age = property(getage, setage, None, None)

>>> x = properties()

>>> x.age # Runs getage

40

>>> x.age = 42 # Runs setage

set age: 42

>>> x._age # Normal fetch: no getage call

42

>>> x.age # Runs getage

40

>>> x.job = 'trainer' # Normal assign: no setage call

>>> x.job # Normal fetch: no getage call

'trainer'

The equivalent class based on operator overloading incurs extra method calls for assignments to attributes not being managed and needs to route attribute assignments through the attribute dictionary to avoid loops (or, for new-style classes, to the object superclass’s __setattr__ to better support “virtual” attributes such as slots and properties coded in other classes):

>>> class operators:

def __getattr__(self, name): # On undefined reference

if name == 'age':

return 40

else:

raise AttributeError(name)

def __setattr__(self, name, value): # On all assignments

print('set: %s %s' % (name, value))

if name == 'age':

self.__dict__['_age'] = value # Or object.__setattr__()

else:

self.__dict__[name] = value

>>> x = operators()

>>> x.age # Runs __getattr__

40

>>> x.age = 41 # Runs __setattr__

set: age 41

>>> x._age # Defined: no __getattr__ call

41

>>> x.age # Runs __getattr__

40

>>> x.job = 'trainer' # Runs __setattr__ again

set: job trainer

>>> x.job # Defined: no __getattr__ call

'trainer'

Properties seem like a win for this simple example. However, some applications of __getattr__ and __setattr__ still require more dynamic or generic interfaces than properties directly provide.

For example, in many cases the set of attributes to be supported cannot be determined when the class is coded, and may not even exist in any tangible form (e.g., when delegating arbitrary attribute references to a wrapped/embedded object generically). In such contexts, a generic__getattr__ or a __setattr__ attribute handler with a passed-in attribute name is usually preferable. Because such generic handlers can also support simpler cases, properties are often an optional and redundant extension—albeit one that may avoid extra calls on assignments, and one that some programmers may prefer when applicable.

For more details on both options, stay tuned for Chapter 38 in the final part of this book. As we’ll see there, it’s also possible to code properties using the @ symbol function decorator syntax—a topic introduced later in this chapter, and an equivalent and automatic alternative to manual assignment in the class scope:

class properties(object):

@property # Coding properties with decorators: ahead

def age(self):

...

@age.setter

def age(self, value):

...

To make sense of this decorator syntax, though, we must move ahead.

__getattribute__ and Descriptors: Attribute Tools

Also in the class extensions department, the __getattribute__ operator overloading method, available for new-style classes only, allows a class to intercept all attribute references, not just undefined references. This makes it more potent than its __getattr__ cousin we used in the prior section, but also trickier to use—it’s prone to loops much like __setattr__, but in different ways.

For more specialized attribute interception goals, in addition to properties and operator overloading methods, Python supports the notion of attribute descriptors—classes with __get__ and __set__ methods, assigned to class attributes and inherited by instances, that intercept read and write accesses to specific attributes. As a preview, here’s one of the simplest descriptors you’re likely to encounter:

>>> class AgeDesc(object):

def __get__(self, instance, owner): return 40

def __set__(self, instance, value): instance._age = value

>>> class descriptors(object):

age = AgeDesc()

>>> x = descriptors()

>>> x.age # Runs AgeDesc.__get__

40

>>> x.age = 42 # Runs AgeDesc.__set__

>>> x._age # Normal fetch: no AgeDesc call

42

Descriptors have access to state in instances of themselves as well as their client class, and are in a sense a more general form of properties; in fact, properties are a simplified way to define a specific type of descriptor—one that runs functions on access. Descriptors are also used to implement the slots feature we met earlier, and other Python tools.

Because __getattribute__ and descriptors are too substantial to cover well here, we’ll defer the rest of their coverage, as well as much more on properties, to Chapter 38 in the final part of this book. We’ll also employ them in examples in Chapter 39 and study how they factor into inheritance in Chapter 40.

Other Class Changes and Extensions

As mentioned, we’re also postponing coverage of the super built-in—an additional major new-style class extension that relies on its MRO—until the end of this chapter. Before we get there, though, we’re going to explore additional class-related changes and extensions that are not necessarily bound to new-style classes, but were introduced at roughly the same time: static and class methods, decorators, and more.

Many of the changes and feature additions of new-style classes integrate with the notion of subclassable types mentioned earlier in this chapter, because subclassable types and new-style classes were introduced in conjunction with a merging of the type/class dichotomy in Python 2.2 and beyond. As we’ve seen, in 3.X, this merging is complete: classes are now types, and types are classes, and Python classes today still reflect both that conceptual merging and its implementation.

Along with these changes, Python also grew a more coherent and generalized protocol for coding metaclasses—classes that subclass the type object, intercept class creation calls, and may provide behavior acquired by classes. Accordingly, they provide a well-defined hook for management and augmentation of class objects. They are also an advanced topic that is optional for most Python programmers, so we’ll postpone further details here. We’ll glimpse metaclasses again later in this chapter in conjunction with class decorators—a feature whose roles often overlap—but we’ll postpone their full coverage until Chapter 40, in the final part of this book. For our purpose here, let’s move on to a handful of additional class-related extensions.

Static and Class Methods

As of Python 2.2, it is possible to define two kinds of methods within a class that can be called without an instance: static methods work roughly like simple instance-less functions inside a class, and class methods are passed a class instead of an instance. Both are similar to tools in other languages (e.g., C++ static methods). Although this feature was added in conjunction with the new-style classes discussed in the prior sections, static and class methods work for classic classes too.

To enable these method modes, you must call special built-in functions named staticmethod and classmethod within the class, or invoke them with the special @name decoration syntax we’ll meet later in this chapter. These functions are required to enable these special method modes in Python 2.X, and are generally needed in 3.X. In Python 3.X, a staticmethod declaration is not required for instance-less methods called only through a class name, but is still required if such methods are called through instances.

Why the Special Methods?

As we’ve learned, a class’s method is normally passed an instance object in its first argument, to serve as the implied subject of the method call—that’s the “object” in “object-oriented programming.” Today, though, there are two ways to modify this model. Before I explain what they are, I should explain why this might matter to you.

Sometimes, programs need to process data associated with classes instead of instances. Consider keeping track of the number of instances created from a class, or maintaining a list of all of a class’s instances that are currently in memory. This type of information and its processing are associated with the class rather than its instances. That is, the information is usually stored on the class itself and processed apart from any instance.

For such tasks, simple functions coded outside a class can often suffice—because they can access class attributes through the class name, they have access to class data and never require access to an instance. However, to better associate such code with a class, and to allow such processing to be customized with inheritance as usual, it would be better to code these types of functions inside the class itself. To make this work, we need methods in a class that are not passed, and do not expect, a self instance argument.

Python supports such goals with the notion of static methods—simple functions with no self argument that are nested in a class and are designed to work on class attributes instead of instance attributes. Static methods never receive an automatic self argument, whether called through a class or an instance. They usually keep track of information that spans all instances, rather than providing behavior for instances.

Although less commonly used, Python also supports the notion of class methods—methods of a class that are passed a class object in their first argument instead of an instance, regardless of whether they are called through an instance or a class. Such methods can access class data through their class argument—what we’ve called self thus far—even if called through an instance. Normal methods, now known in formal circles as instance methods, still receive a subject instance when called; static and class methods do not.

Static Methods in 2.X and 3.X

The concept of static methods is the same in both Python 2.X and 3.X, but its implementation requirements have evolved somewhat in Python 3.X. Since this book covers both versions, I need to explain the differences in the two underlying models before we get to the code.

Really, we already began this story in the preceding chapter, when we explored the notion of unbound methods. Recall that both Python 2.X and 3.X always pass an instance to a method that is called through an instance. However, Python 3.X treats methods fetched directly from a class differently than 2.X—a difference in Python lines that has nothing to do with new-style classes:

§ Both Python 2.X and 3.X produce a bound method when a method is fetched through an instance.

§ In Python 2.X, fetching a method from a class produces an unbound method, which cannot be called without manually passing an instance.

§ In Python 3.X, fetching a method from a class produces a simple function, which can be called normally with no instance present.

In other words, Python 2.X class methods always require an instance to be passed in, whether they are called through an instance or a class. By contrast, in Python 3.X we are required to pass an instance to a method only if the method expects one—methods that do not include an instance argument can be called through the class without passing an instance. That is, 3.X allows simple functions in a class, as long as they do not expect and are not passed an instance argument. The net effect is that:

§ In Python 2.X, we must always declare a method as static in order to call it without an instance, whether it is called through a class or an instance.

§ In Python 3.X, we need not declare such methods as static if they will be called through a class only, but we must do so in order to call them through an instance.

To illustrate, suppose we want to use class attributes to count how many instances are generated from a class. The following file, spam.py, makes a first attempt—its class has a counter stored as a class attribute, a constructor that bumps up the counter by one each time a new instance is created, and a method that displays the counter’s value. Remember, class attributes are shared by all instances. Therefore, storing the counter in the class object itself ensures that it effectively spans all instances:

class Spam:

numInstances = 0

def __init__(self):

Spam.numInstances = Spam.numInstances + 1

def printNumInstances():

print("Number of instances created: %s" % Spam.numInstances)

The printNumInstances method is designed to process class data, not instance data—it’s about all the instances, not any one in particular. Because of that, we want to be able to call it without having to pass an instance. Indeed, we don’t want to make an instance to fetch the number of instances, because this would change the number of instances we’re trying to fetch! In other words, we want a self-less “static” method.

Whether this code’s printNumInstances works or not, though, depends on which Python you use, and which way you call the method—through the class or through an instance. In 2.X, calls to a self-less method function through both the class and instances fail (as usual, I’ve omitted some error text here for space):

C:\code> c:\python27\python

>>> from spam import Spam

>>> a = Spam() # Cannot call unbound class methods in 2.X

>>> b = Spam() # Methods expect a self object by default

>>> c = Spam()

>>> Spam.printNumInstances()

TypeError: unbound method printNumInstances() must be called with Spam instance

as first argument (got nothing instead)

>>> a.printNumInstances()

TypeError: printNumInstances() takes no arguments (1 given)

The problem here is that unbound instance methods aren’t exactly the same as simple functions in 2.X. Even though there are no arguments in the def header, the method still expects an instance to be passed in when it’s called, because the function is associated with a class. In Python 3.X, calls to self-less methods made through classes work, but calls from instances fail:

C:\code> c:\python33\python

>>> from spam import Spam

>>> a = Spam() # Can call functions in class in 3.X

>>> b = Spam() # Calls through instances still pass a self

>>> c = Spam()

>>> Spam.printNumInstances() # Differs in 3.X

Number of instances created: 3

>>> a.printNumInstances()

TypeError: printNumInstances() takes 0 positional arguments but 1 was given

That is, calls to instance-less methods like printNumInstances made through the class fail in Python 2.X but work in Python 3.X. On the other hand, calls made through an instance fail in both Pythons, because an instance is automatically passed to a method that does not have an argument to receive it:

Spam.printNumInstances() # Fails in 2.X, works in 3.X

instance.printNumInstances() # Fails in both 2.X and 3.X (unless static)

If you’re able to use 3.X and stick with calling self-less methods through classes only, you already have a static method feature. However, to allow self-less methods to be called through classes in 2.X and through instances in both 2.X and 3.X, you need to either adopt other designs or be able to somehow mark such methods as special. Let’s look at both options in turn.

Static Method Alternatives

Short of marking a self-less method as special, you can sometimes achieve similar results with different coding structures. For example, if you just want to call functions that access class members without an instance, perhaps the simplest idea is to use normal functions outside the class, not class methods. This way, an instance isn’t expected in the call. The following mutation of spam.py illustrates, and works the same in Python 3.X and 2.X:

def printNumInstances():

print("Number of instances created: %s" % Spam.numInstances)

class Spam:

numInstances = 0

def __init__(self):

Spam.numInstances = Spam.numInstances + 1

C:\code> c:\python33\python

>>> import spam

>>> a = spam.Spam()

>>> b = spam.Spam()

>>> c = spam.Spam()

>>> spam.printNumInstances() # But function may be too far removed

Number of instances created: 3 # And cannot be changed via inheritance

>>> spam.Spam.numInstances

3

Because the class name is accessible to the simple function as a global variable, this works fine. Also, note that the name of the function becomes global, but only to this single module; it will not clash with names in other files of the program.

Prior to static methods in Python, this structure was the general prescription. Because Python already provides modules as a namespace-partitioning tool, one could argue that there’s not typically any need to package functions in classes unless they implement object behavior. Simple functions within modules like the one here do much of what instance-less class methods could, and are already associated with the class because they live in the same module.

Unfortunately, this approach is still less than ideal. For one thing, it adds to this file’s scope an extra name that is used only for processing a single class. For another, the function is much less directly associated with the class by structure; in fact, its definition could be hundreds of lines away. Perhaps worse, simple functions like this cannot be customized by inheritance, since they live outside a class’s namespace: subclasses cannot directly replace or extend such a function by redefining it.

We might try to make this example work in a version-neutral way by using a normal method and always calling it through (or with) an instance, as usual:

class Spam:

numInstances = 0

def __init__(self):

Spam.numInstances = Spam.numInstances + 1

def printNumInstances(self):

print("Number of instances created: %s" % Spam.numInstances)

C:\code> c:\python33\python

>>> from spam import Spam

>>> a, b, c = Spam(), Spam(), Spam()

>>> a.printNumInstances()

Number of instances created: 3

>>> Spam.printNumInstances(a)

Number of instances created: 3

>>> Spam().printNumInstances() # But fetching counter changes counter!

Number of instances created: 4

Unfortunately, as mentioned earlier, such an approach is completely unworkable if we don’t have an instance available, and making an instance changes the class data, as illustrated in the last line here. A better solution would be to somehow mark a method inside a class as never requiring an instance. The next section shows how.

Using Static and Class Methods

Today, there is another option for coding simple functions associated with a class that may be called through either the class or its instances. As of Python 2.2, we can code classes with static and class methods, neither of which requires an instance argument to be passed in when invoked. To designate such methods, classes call the built-in functions staticmethod and classmethod, as hinted in the earlier discussion of new-style classes. Both mark a function object as special—that is, as requiring no instance if static and requiring a class argument if a class method. For example, in the file bothmethods.py (which unifies 2.X and 3.X printing with lists, though displays still vary slightly for 2.X classic classes):

# File bothmethods.py

class Methods:

def imeth(self, x): # Normal instance method: passed a self

print([self, x])

def smeth(x): # Static: no instance passed

print([x])

def cmeth(cls, x): # Class: gets class, not instance

print([cls, x])

smeth = staticmethod(smeth) # Make smeth a static method (or @: ahead)

cmeth = classmethod(cmeth) # Make cmeth a class method (or @: ahead)

Notice how the last two assignments in this code simply reassign (a.k.a. rebind) the method names smeth and cmeth. Attributes are created and changed by any assignment in a class statement, so these final assignments simply overwrite the assignments made earlier by the defs. As we’ll see in a few moments, the special @ syntax works here as an alternative to this just as it does for properties—but makes little sense unless you first understand the assignment form here that it automates.

Technically, Python now supports three kinds of class-related methods, with differing argument protocols:

§ Instance methods, passed a self instance object (the default)

§ Static methods, passed no extra object (via staticmethod)

§ Class methods, passed a class object (via classmethod, and inherent in metaclasses)

Moreover, Python 3.X extends this model by also allowing simple functions in a class to serve the role of static methods without extra protocol, when called through a class object only. Despite its name, the bothmethods.py module illustrates all three method types, so let’s expand on these in turn.

Instance methods are the normal and default case that we’ve seen in this book. An instance method must always be called with an instance object. When you call it through an instance, Python passes the instance to the first (leftmost) argument automatically; when you call it through a class, you must pass along the instance manually:

>>> from bothmethods import Methods # Normal instance methods

>>> obj = Methods() # Callable through instance or class

>>> obj.imeth(1)

[<bothmethods.Methods object at 0x0000000002A15710>, 1]

>>> Methods.imeth(obj, 2)

[<bothmethods.Methods object at 0x0000000002A15710>, 2]

Static methods, by contrast, are called without an instance argument. Unlike simple functions outside a class, their names are local to the scopes of the classes in which they are defined, and they may be looked up by inheritance. Instance-less functions can be called through a class normally in Python 3.X, but never by default in 2.X. Using the staticmethod built-in allows such methods to also be called through an instance in 3.X and through both a class and an instance in Python 2.X (that is, the first of the following works in 3.X without staticmethod, but the second does not):

>>> Methods.smeth(3) # Static method: call through class

[3] # No instance passed or expected

>>> obj.smeth(4) # Static method: call through instance

[4] # Instance not passed

Class methods are similar, but Python automatically passes the class (not an instance) in to a class method’s first (leftmost) argument, whether it is called through a class or an instance:

>>> Methods.cmeth(5) # Class method: call through class

[<class 'bothmethods.Methods'>, 5] # Becomes cmeth(Methods, 5)

>>> obj.cmeth(6) # Class method: call through instance

[<class 'bothmethods.Methods'>, 6] # Becomes cmeth(Methods, 6)

In Chapter 40, we’ll also find that metaclass methods—a unique, advanced, and technically distinct method type—behave similarly to the explicitly-declared class methods we’re exploring here.

Counting Instances with Static Methods

Now, given these built-ins, here is the static method equivalent of this section’s instance-counting example—it marks the method as special, so it will never be passed an instance automatically:

class Spam:

numInstances = 0 # Use static method for class data

def __init__(self):

Spam.numInstances += 1

def printNumInstances():

print("Number of instances: %s" % Spam.numInstances)

printNumInstances = staticmethod(printNumInstances)

Using the static method built-in, our code now allows the self-less method to be called through the class or any instance of it, in both Python 2.X and 3.X:

>>> from spam_static import Spam

>>> a = Spam()

>>> b = Spam()

>>> c = Spam()

>>> Spam.printNumInstances() # Call as simple function

Number of instances: 3

>>> a.printNumInstances() # Instance argument not passed

Number of instances: 3

Compared to simply moving printNumInstances outside the class, as prescribed earlier, this version requires an extra staticmethod call (or an @ line we’ll see ahead). However, it also localizes the function name in the class scope (so it won’t clash with other names in the module); moves the function code closer to where it is used (inside the class statement); and allows subclasses to customize the static method with inheritance—a more convenient and powerful approach than importing functions from the files in which superclasses are coded. The following subclass and new testing session illustrate (be sure to start a new session after changing files, so that your from imports load the latest version of the file):

class Sub(Spam):

def printNumInstances(): # Override a static method

print("Extra stuff...") # But call back to original

Spam.printNumInstances()

printNumInstances = staticmethod(printNumInstances)

>>> from spam_static import Spam, Sub

>>> a = Sub()

>>> b = Sub()

>>> a.printNumInstances() # Call from subclass instance

Extra stuff...

Number of instances: 2

>>> Sub.printNumInstances() # Call from subclass itself

Extra stuff...

Number of instances: 2

>>> Spam.printNumInstances() # Call original version

Number of instances: 2

Moreover, classes can inherit the static method without redefining it—it is run without an instance, regardless of where it is defined in a class tree:

>>> class Other(Spam): pass # Inherit static method verbatim

>>> c = Other()

>>> c.printNumInstances()

Number of instances: 3

Notice how this also bumps up the superclass’s instance counter, because its constructor is inherited and run—a behavior that begins to encroach on the next section’s subject.

Counting Instances with Class Methods

Interestingly, a class method can do similar work here—the following has the same behavior as the static method version listed earlier, but it uses a class method that receives the instance’s class in its first argument. Rather than hardcoding the class name, the class method uses the automatically passed class object generically:

class Spam:

numInstances = 0 # Use class method instead of static

def __init__(self):

Spam.numInstances += 1

def printNumInstances(cls):

print("Number of instances: %s" % cls.numInstances)

printNumInstances = classmethod(printNumInstances)

This class is used in the same way as the prior versions, but its printNumInstances method receives the Spam class, not the instance, when called from both the class and an instance:

>>> from spam_class import Spam

>>> a, b = Spam(), Spam()

>>> a.printNumInstances() # Passes class to first argument

Number of instances: 2

>>> Spam.printNumInstances() # Also passes class to first argument

Number of instances: 2

When using class methods, though, keep in mind that they receive the most specific (i.e., lowest) class of the call’s subject. This has some subtle implications when trying to update class data through the passed-in class. For example, if in module spam_class.py we subclass to customize as before, augment Spam.printNumInstances to also display its cls argument, and start a new testing session:

class Spam:

numInstances = 0 # Trace class passed in

def __init__(self):

Spam.numInstances += 1

def printNumInstances(cls):

print("Number of instances: %s %s" % (cls.numInstances, cls))

printNumInstances = classmethod(printNumInstances)

class Sub(Spam):

def printNumInstances(cls): # Override a class method

print("Extra stuff...", cls) # But call back to original

Spam.printNumInstances()

printNumInstances = classmethod(printNumInstances)

class Other(Spam): pass # Inherit class method verbatim

The lowest class is passed in whenever a class method is run, even for subclasses that have no class methods of their own:

>>> from spam_class import Spam, Sub, Other

>>> x = Sub()

>>> y = Spam()

>>> x.printNumInstances() # Call from subclass instance

Extra stuff... <class 'spam_class.Sub'>

Number of instances: 2 <class 'spam_class.Spam'>

>>> Sub.printNumInstances() # Call from subclass itself

Extra stuff... <class 'spam_class.Sub'>

Number of instances: 2 <class 'spam_class.Spam'>

>>> y.printNumInstances() # Call from superclass instance

Number of instances: 2 <class 'spam_class.Spam'>

In the first call here, a class method call is made through an instance of the Sub subclass, and Python passes the lowest class, Sub, to the class method. All is well in this case—since Sub’s redefinition of the method calls the Spam superclass’s version explicitly, the superclass method in Spamreceives its own class in its first argument. But watch what happens for an object that inherits the class method verbatim:

>>> z = Other() # Call from lower sub's instance

>>> z.printNumInstances()

Number of instances: 3 <class 'spam_class.Other'>

This last call here passes Other to Spam’s class method. This works in this example because fetching the counter finds it in Spam by inheritance. If this method tried to assign to the passed class’s data, though, it would update Other, not Spam! In this specific case, Spam is probably better off hardcoding its own class name to update its data if it means to count instances of all its subclasses too, rather than relying on the passed-in class argument.

Counting instances per class with class methods

In fact, because class methods always receive the lowest class in an instance’s tree:

§ Static methods and explicit class names may be a better solution for processing data local to a class.

§ Class methods may be better suited to processing data that may differ for each class in a hierarchy.

Code that needs to manage per-class instance counters, for example, might be best off leveraging class methods. In the following, the top-level superclass uses a class method to manage state information that varies for and is stored on each class in the tree—similar in spirit to the way instance methods manage state information that varies per class instance:

class Spam:

numInstances = 0

def count(cls): # Per-class instance counters

cls.numInstances += 1 # cls is lowest class above instance

def __init__(self):

self.count() # Passes self.__class__ to count

count = classmethod(count)

class Sub(Spam):

numInstances = 0

def __init__(self): # Redefines __init__

Spam.__init__(self)

class Other(Spam): # Inherits __init__

numInstances = 0

>>> from spam_class2 import Spam, Sub, Other

>>> x = Spam()

>>> y1, y2 = Sub(), Sub()

>>> z1, z2, z3 = Other(), Other(), Other()

>>> x.numInstances, y1.numInstances, z1.numInstances # Per-class data!

(1, 2, 3)

>>> Spam.numInstances, Sub.numInstances, Other.numInstances

(1, 2, 3)

Static and class methods have additional advanced roles, which we will finesse here; see other resources for more use cases. In recent Python versions, though, the static and class method designations have become even simpler with the advent of function decoration syntax—a way to apply one function to another that has roles well beyond the static method use case that was its initial motivation. This syntax also allows us to augment classes in Python 2.X and 3.X—to initialize data like the numInstances counter in the last example, for instance. The next section explainshow.

NOTE

For a postscript on Python’s method types, be sure to watch for coverage of metaclass methods in Chapter 40—because these are designed to process a class that is an instance of a metaclass, they turn out to be very similar to the class methods defined here, but require no classmethod declaration, and apply only to the shadowy metaclass realm.

Decorators and Metaclasses: Part 1

Because the staticmethod and classmethod call technique described in the prior section initially seemed obscure to some observers, a device was eventually added to make the operation simpler. Python decorators—similar to the notion and syntax of annotations in Java—both addressed this specific need and provided a general tool for adding logic that manages both functions and classes, or later calls to them.

This is called a “decoration,” but in more concrete terms is really just a way to run extra processing steps at function and class definition time with explicit syntax. It comes in two flavors:

§ Function decorators—the initial entry in this set, added in Python 2.4—augment function definitions. They specify special operation modes for both simple functions and classes’ methods by wrapping them in an extra layer of logic implemented as another function, usually called ametafunction.

§ Class decorators—a later extension, added in Python 2.6 and 3.0—augment class definitions. They do the same for classes, adding support for management of whole objects and their interfaces. Though perhaps simpler, they often overlap in roles with metaclasses.

Function decorators turn out to be very general tools: they are useful for adding many types of logic to functions besides the static and class method use cases. For instance, they may be used to augment functions with code that logs calls made to them, checks the types of passed arguments during debugging, and so on. Function decorators can be used to manage either functions themselves or later calls to them. In the latter mode, function decorators are similar to the delegation design pattern we explored in Chapter 31, but they are designed to augment a specific function or method call, not an entire object interface.

Python provides a few built-in function decorators for operations such as marking static and class methods and defining properties (as sketched earlier, the property built-in works as a decorator automatically), but programmers can also code arbitrary decorators of their own. Although they are not strictly tied to classes, user-defined function decorators often are coded as classes to save the original functions for later dispatch, along with other data as state information.

This proved such a useful hook that it was extended in Python 2.6, 2.7, and 3.X—class decorators bring augmentation to classes too, and are more directly tied to the class model. Like their function cohorts, class decorators may manage classes themselves or later instance creation calls, and often employ delegation in the latter mode. As we’ll find, their roles also often overlap with metaclasses; when they do, the newer class decorators may offer a more lightweight way to achieve the same goals.

Function Decorator Basics

Syntactically, a function decorator is a sort of runtime declaration about the function that follows. A function decorator is coded on a line by itself just before the def statement that defines a function or method. It consists of the @ symbol, followed by what we call a metafunction—a function (or other callable object) that manages another function. Static methods since Python 2.4, for example, may be coded with decorator syntax like this:

class C:

@staticmethod # Function decoration syntax

def meth():

...

Internally, this syntax has the same effect as the following—passing the function through the decorator and assigning the result back to the original name:

class C:

def meth():

...

meth = staticmethod(meth) # Name rebinding equivalent

Decoration rebinds the method name to the decorator’s result. The net effect is that calling the method function’s name later actually triggers the result of its staticmethod decorator first. Because a decorator can return any sort of object, this allows the decorator to insert a layer of logic to be run on every call. The decorator function is free to return either the original function itself, or a new proxy object that saves the original function passed to the decorator to be invoked indirectly after the extra logic layer runs.

With this addition, here’s a better way to code our static method example from the prior section in either Python 2.X or 3.X:

class Spam:

numInstances = 0

def __init__(self):

Spam.numInstances = Spam.numInstances + 1

@staticmethod

def printNumInstances():

print("Number of instances created: %s" % Spam.numInstances)

>>> from spam_static_deco import Spam

>>> a = Spam()

>>> b = Spam()

>>> c = Spam()

>>> Spam.printNumInstances() # Calls from classes and instances work

Number of instances created: 3

>>> a.printNumInstances()

Number of instances created: 3

Because they also accept and return functions, the classmethod and property built-in functions may be used as decorators in the same way—as in the following mutation of the prior bothmethods.py:

# File bothmethods_decorators.py

class Methods(object): # object needed in 2.X for property setters

def imeth(self, x): # Normal instance method: passed a self

print([self, x])

@staticmethod

def smeth(x): # Static: no instance passed

print([x])

@classmethod

def cmeth(cls, x): # Class: gets class, not instance

print([cls, x])

@property # Property: computed on fetch

def name(self):

return 'Bob ' + self.__class__.__name__

>>> from bothmethods_decorators import Methods

>>> obj = Methods()

>>> obj.imeth(1)

[<bothmethods_decorators.Methods object at 0x0000000002A256A0>, 1]

>>> obj.smeth(2)

[2]

>>> obj.cmeth(3)

[<class 'bothmethods_decorators.Methods'>, 3]

>>> obj.name

'Bob Methods'

Keep in mind that staticmethod and its kin here are still built-in functions; they may be used in decoration syntax, just because they take a function as an argument and return a callable to which the original function name can be rebound. In fact, any such function can be used in this way—even user-defined functions we code ourselves, as the next section explains.

A First Look at User-Defined Function Decorators

Although Python provides a handful of built-in functions that can be used as decorators, we can also write custom decorators of our own. Because of their wide utility, we’re going to devote an entire chapter to coding decorators in the final part of this book. As a quick example, though, let’s look at a simple user-defined decorator at work.

Recall from Chapter 30 that the __call__ operator overloading method implements a function-call interface for class instances. The following code uses this to define a call proxy class that saves the decorated function in the instance and catches calls to the original name. Because this is a class, it also has state information—a counter of calls made:

class tracer:

def __init__(self, func): # Remember original, init counter

self.calls = 0

self.func = func

def __call__(self, *args): # On later calls: add logic, run original

self.calls += 1

print('call %s to %s' % (self.calls, self.func.__name__))

return self.func(*args)

@tracer # Same as spam = tracer(spam)

def spam(a, b, c): # Wrap spam in a decorator object

return a + b + c

print(spam(1, 2, 3)) # Really calls the tracer wrapper object

print(spam('a', 'b', 'c')) # Invokes __call__ in class

Because the spam function is run through the tracer decorator, when the original spam name is called it actually triggers the __call__ method in the class. This method counts and logs the call, and then dispatches it to the original wrapped function. Note how the *name argument syntax is used to pack and unpack the passed-in arguments; because of this, this decorator can be used to wrap any function with any number of positional arguments.

The net effect, again, is to add a layer of logic to the original spam function. Here is the script’s 3.X and 2.X output—the first line comes from the tracer class, and the second gives the return value of the spam function itself:

c:\code> python tracer1.py

call 1 to spam

6

call 2 to spam

abc

Trace through this example’s code for more insight. As it is, this decorator works for any function that takes positional arguments, but it does not handle keyword arguments, and cannot decorate class-level method functions (in short, for methods its __call__ would be passed a tracerinstance only). As we’ll see in Part VIII, there are a variety of ways to code function decorators, including nested def statements; some of the alternatives are better suited to methods than the version shown here.

For example, by using nested functions with enclosing scopes for state, instead of callable class instances with attributes, function decorators often become more broadly applicable to class-level methods too. We’ll postpone the full details on this, but here’s a brief look at this closure based coding model; it uses function attributes for counter state for portability, but could leverage variables and nonlocal instead in 3.X only:

def tracer(func): # Remember original

def oncall(*args): # On later calls

oncall.calls += 1

print('call %s to %s' % (oncall.calls, func.__name__))

return func(*args)

oncall.calls = 0

return oncall

class C:

@tracer

def spam(self,a, b, c): return a + b + c

x = C()

print(x.spam(1, 2, 3))

print(x.spam('a', 'b', 'c')) # Same output as tracer1 (in tracer2.py)

A First Look at Class Decorators and Metaclasses

Function decorators turned out to be so useful that Python 2.6 and 3.0 expanded the model, allowing decorators to be applied to classes as well as functions. In short, class decorators are similar to function decorators, but they are run at the end of a class statement to rebind a class name to a callable. As such, they can be used to either manage classes just after they are created, or insert a layer of wrapper logic to manage instances when they are later created. Symbolically, the code structure:

def decorator(aClass): ...

@decorator # Class decoration syntax

class C: ...

is mapped to the following equivalent:

def decorator(aClass): ...

class C: ... # Name rebinding equivalent

C = decorator(C)

The class decorator is free to augment the class itself, or return a proxy object that intercepts later instance construction calls. For example, in the code of the section Counting instances per class with class methods, we could use this hook to automatically augment the classes with instance counters and any other data required:

def count(aClass):

aClass.numInstances = 0

return aClass # Return class itself, instead of a wrapper

@count

class Spam: ... # Same as Spam = count(Spam)

@count

class Sub(Spam): ... # numInstances = 0 not needed here

@count

class Other(Spam): ...

In fact, as coded, this decorator can be applied to class or functions—it happily returns the object being defined in either context after initializing the object’s attribute:

@count

def spam(): pass # Like spam = count(spam)

@count

class Other: pass # Like Other = count(Other)

spam.numInstances # Both are set to zero

Other.numInstances

Though this decorator manages a function or class itself, as we’ll see later in this book, class decorators can also manage an object’s entire interface by intercepting construction calls, and wrapping the new instance object in a proxy that deploys attribute accessor tools to intercept later requests—a multilevel coding technique we’ll use to implement class attribute privacy in Chapter 39. Here’s a preview of the model:

def decorator(cls): # On @ decoration

class Proxy:

def __init__(self, *args): # On instance creation: make a cls

self.wrapped = cls(*args)

def __getattr__(self, name): # On attribute fetch: extra ops here

return getattr(self.wrapped, name)

return Proxy

@decorator

class C: ... # Like C = decorator(C)

X = C() # Makes a Proxy that wraps a C, and catches later X.attr

Metaclasses, mentioned briefly earlier, are a similarly advanced class-based tool whose roles often intersect with those of class decorators. They provide an alternate model, which routes the creation of a class object to a subclass of the top-level type class, at the conclusion of a classstatement:

class Meta(type):

def __new__(meta, classname, supers, classdict):

...extra logic + class creation via type call...

class C(metaclass=Meta):

...my creation routed to Meta... # Like C = Meta('C', (), {...})

In Python 2.X, the effect is the same, but the coding differs—use a class attribute instead of a keyword argument in the class header:

class C:

__metaclass__ = Meta

... my creation routed to Meta...

In either line, Python calls a class’s metaclass to create the new class object, passing in the data defined during the class statement’s run; in 2.X, the metaclass simply defaults to the classic class creator:

classname = Meta(classname, superclasses, attributedict)

To assume control of the creation or initialization of a new class object, a metaclass generally redefines the __new__ or __init__ method of the type class that normally intercepts this call. The net effect, as with class decorators, is to define code to be run automatically at class creation time. Here, this step binds the class name to the result of a call to a user-defined metaclass. In fact, a metaclass need not be a class at all—a possibility we’ll explore later that blurs some of the distinction between this tool and decorators, and may even qualify the two as functionally equivalent in many roles.

Both schemes, class decorators and metaclasses, are free to augment a class or return an arbitrary object to replace it—a protocol with almost limitless class-based customization possibilities. As we’ll see later, metaclasses may also define methods that process their instance classes, rather than normal instances of them—a technique that’s similar to class methods, and might be emulated in spirit by methods and data in class decorator proxies, or even a class decorator that returns a metaclass instance. Such mind-binding concepts will require Chapter 40’s conceptual groundwork (and quite possibly sedation!).

For More Details

Naturally, there’s much more to the decorator and metaclass stories than I’ve shown here. Although they are a general mechanism whose usage may be required by some packages, coding new user-defined decorators and metaclasses is an advanced topic of interest primarily to tool writers, not application programmers. Because of this, we’ll defer additional coverage until the final and optional part of this book:

§ Chapter 38 shows how to code properties using function decorator syntax in more depth.

§ Chapter 39 has much more on decorators, including more comprehensive examples.

§ Chapter 40 covers metaclasses, and more on the class and instance management story.

Although these chapters cover advanced topics, they’ll also provide us with a chance to see Python at work in more substantial examples than much of the rest of the book was able to provide. For now, let’s move on to our final class-related topic.

The super Built-in Function: For Better or Worse?

So far, I’ve mentioned Python’s super built-in function only briefly in passing because it is relatively uncommon and may even be controversial to use. Given this call’s increased visibility in recent years, though, it merits some further elaboration in this edition. Besides introducing super, this section also serves as a language design case study to close out a chapter on so many tools whose presence may to some seem curious in a scripting language like Python.

Some of this section calls this proliferation of tools into question, and I encourage you to judge any subjective content here for yourself (and we’ll return to such things at the end of this book after we’ve expanded on other advanced tools such as metaclasses and descriptors). Still, Python’s rapid growth rate in recent years represents a strategic decision point for its community going forward, and super seems as good a representative example as any.

The Great super Debate

As noted in Chapter 28 and Chapter 29, Python has a super built-in function that can be used to invoke superclass methods generically, but was deferred until this point of the book. This was deliberate—because super has substantial downsides in typical code, and a sole use case that seems obscure and complex to many observers, most beginners are better served by the traditional explicit-name call scheme used so far. See the sidebar What About super? in Chapter 28 for a brief summary of the rationale for this policy.

The Python community itself seems split on this subject, with online articles about it running the gamut from “Python’s Super Considered Harmful” to “Python’s super() considered super!”^[65^] Frankly, in my live classes this call seems to be most often of interest to Java programmers starting to use Python anew, because of its conceptual similarity to a tool in that language (many a new Python feature ultimately owes its existence to programmers of other languages bringing their old habits to a new model). Python’s super is not Java’s—it translates differently to Python’s multiple inheritance, and has a use case beyond Java’s—but it has managed to generate both controversy and misunderstanding since its conception.

This book postponed the super call until now (and omitted it almost entirely in prior editions) because it has significant issues—it’s prohibitively cumbersome to use in 2.X, differs in form between 2.X and 3.X, is based upon unusual semantics in 3.X, and mixes poorly with Python’s multiple inheritance and operator overloading in typical Python code. In fact, as we’ll see, in some code super can actually mask problems, and discourage a more explicit coding style that offers better control.

In its defense, this call does have a valid use case too—cooperative same-named method dispatch in diamond multiple inheritance trees—but it seems to ask a lot of newcomers. It requires that super be used universally and consistently (if not neurotically), much like __slots__ discussed earlier; relies on the arguably obscure MRO algorithm to order calls; and addresses a use case that seems far more the exception than the norm in Python programs. In this role, super seems an advanced tool based upon esoteric principles, which may be beyond much of Python’s audience, and seems artificial to real program goals. That aside, its expectation of universal use seems unrealistic for the vast amount of existing Python code.

Because of all these factors, this introductory-level book has preferred the traditional explicit-name call scheme thus far and recommends the same for newcomers. You’re better off learning the traditional scheme first, and might be better off sticking with that in general, rather than using an extra special-case tool that may not work in some contexts, and relies on arcane magic in the valid but atypical use case it addresses. This is not just your author’s opinion; despite its advocate’s best intentions, super is not widely recognized as “best practice” in Python today, for completely valid reasons.

On the other hand, just as for other tools the increasing use of this call in Python code in recent years makes it no longer optional for many Python programmers—the first time you see it, it’s officially mandatory! For readers who may wish to experiment with super, and for other readers who may have it imposed upon them, this section provides a brief look at this tool and its rationale—beginning with alternatives to it.

Traditional Superclass Call Form: Portable, General

In general, this book’s examples prefer to call back to superclass methods when needed by naming the superclass explicitly, because this technique is traditional in Python, because it works the same in both Python 2.X and 3.X, and because it sidesteps limitations and complexities related to this call in both 2.X and 3.X. As shown earlier, the traditional superclass method call scheme to augment a superclass method works as follows:

>>> class C: # In Python 2.X and 3.X

def act(self):

print('spam')

>>> class D(C):

def act(self):

C.act(self) # Name superclass explicitly, pass self

print('eggs')

>>> X = D()

>>> X.act()

spam

eggs

This form works the same in 2.X and 3.X, follows Python’s normal method call mapping model, applies to all inheritance tree forms, and does not lead to confusing behavior when operator overloading is used. To see why these distinctions matter, let’s see how super compares.

Basic super Usage and Its Tradeoffs

In this section, we’ll both introduce super in basic, single-inheritance mode, and look at its perceived downsides in this role. As we’ll find, in this context super does work as advertised, but is not much different from traditional calls, relies on unusual semantics, and is cumbersome to deploy in 2.X. More critically, as soon as your classes grow to use multiple inheritance, this super usage mode can both mask problems in your code and route calls in ways you may not expect.

Odd semantics: A magic proxy in Python 3.X

The super built-in actually has two intended roles. The more esoteric of these—cooperative multiple inheritance dispatch protocols in diamond multiple-inheritance trees (yes, a mouthful!)—relies on the 3.X MRO, was borrowed from the Dylan language, and will be covered later in this section.

The role we’re interested in here is more commonly used, and more frequently requested by people with Java backgrounds—to allow superclasses to be named generically in inheritance trees. This is intended to promote simpler code maintenance, and to avoid having to type long superclass reference paths in calls. In Python 3.X, this call seems at least at first glance to achieve this purpose well:

>>> class C: # In Python 3.X (only: see 2.X super form ahead)

def act(self):

print('spam')

>>> class D(C):

def act(self):

super().act() # Reference superclass generically, omit self

print('eggs')

>>> X = D()

>>> X.act()

spam

eggs

This works, and minimizes code changes—you don’t need to update the call if D’s superclass changes in the future. One of the biggest downsides of this call in 3.X, though, is its reliance on deep magic: though prone to change, it operates today by inspecting the call stack in order to automatically locate the self argument and find the superclass, and pairs the two in a special proxy object that routes the later call to the superclass version of the method. If that sounds complicated and strange, it’s because it is. In fact, this call form doesn’t work at all outside the context of a class’s method:

>>> super # A "magic" proxy object that routes later calls

<class 'super'>

>>> super()

SystemError: super(): no arguments

>>> class E(C):

def method(self): # self is implicit in super...only!

proxy = super() # This form has no meaning outside a method

print(proxy) # Show the normally hidden proxy object

proxy.act() # No arguments: implicitly calls superclass method!

>>> E().method()

<super: <class 'E'>, <E object>>

spam

Really, this call’s semantics resembles nothing else in Python—it’s neither a bound nor unbound method, and somehow finds a self even though you omit one in the call. In single inheritance trees, a superclass is available from self via the path self.__class__.__bases__[0], but the heavily implicit nature of this call makes this difficult to see, and even flies in the face of Python’s explicit self policy that holds true everywhere else. That is, this call violates a fundamental Python idiom for a single use case. It also soundly contradicts Python’s longstanding EIBTI design rule (run an “import this” for more on this rule).

Pitfall: Adding multiple inheritance naively

Besides its unusual semantics, even in 3.X this super role applies most directly to single inheritance trees, and can become problematic as soon as classes employ multiple inheritance with traditionally coded classes. This seems a major limitation of scope; due to the utility of mix-in classes in Python, multiple inheritance from disjoint and independent superclasses is probably more the norm than the exception in realistic code. The super call seems a recipe for disaster in classes coded to naively use its basic mode, without allowing for its much more subtle implications in multiple inheritance trees.

The following illustrates the trap. This code begins its life happily deploying super in single-inheritance mode to invoke a method one level up from C:

>>> class A: # In Python 3.X

def act(self): print('A')

>>> class B:

def act(self): print('B')

>>> class C(A):

def act(self):

super().act() # super applied to a single-inheritance tree

>>> X = C()

>>> X.act()

A

If such classes later grow to use more than one superclass, though, super can become error-prone, and even unusable—it does not raise an exception for multiple inheritance trees, but will naively pick just the leftmost superclass having the method being run (technically, the first per the MRO), which may or may not be the one that you want:

>>> class C(A, B): # Add a B mix-in class with the same method

def act(self):

super().act() # Doesn't fail on multi-inher, but picks just one!

>>> X = C()

>>> X.act()

A

>>> class C(B, A):

def act(self):

super().act() # If B is listed first, A.act() is no longer run!

>>> X = C()

>>> X.act()

B

Perhaps worse, this silently masks the fact that you should probably be selecting superclasses explicitly in this case, as we learned earlier in both this chapter and its predecessor. In other words, super usage may obscure a common source of errors in Python—one so common that it shows up again in this part’s “Gotchas.” If you may need to use direct calls later, why not use them earlier too?

>>> class C(A, B): # Traditional form

def act(self): # You probably need to be more explicit here

A.act(self) # This form handles both single and multiple inher

B.act(self) # And works the same in both Python 3.X and 2.X

>>> X = C() # So why use the super() special case at all?

>>> X.act()

A

B

As we’ll see in a few moments, you might also be able to address such cases by deploying super calls in every class of the tree. But that’s also one of the biggest downsides of super—why code it in every class, when it’s usually not needed, and when using the preceding simpler traditional form in a single class will usually suffice? Especially in existing code—and new code that uses existing code—this super requirement seems harsh, if not unrealistic.

Much more subtly, as we’ll also see ahead, once you step up to multiple inheritance calls this way, the super calls in your code might not invoke the class you expect them to. They’ll be routed per the MRO order, which, depending on where else super might be used, may invoke a method in a class that is not the caller’s superclass at all—an implicit ordering that might make for interesting debugging sessions! Unless you completely understand what super means once multiple inheritance is introduced, you may be better off not deploying it in single-inheritance mode either.

This coding situation isn’t nearly as abstract as it may seem. Here’s a real-world example of such a case, taken from the PyMailGUI case study in Programming Python—the following very typical Python classes use multiple inheritance to mix in both application logic and window tools from independent, standalone classes, and hence must invoke both superclass constructors explicitly with direct calls by name. As coded, a super().__init__() here would run only one constructor, and adding super throughout this example’s disjoint class trees would be more work, would be no simpler, and wouldn’t make sense in tools meant for arbitrary deployment in clients that may use super or not:

class PyMailServerWindow(PyMailServer, windows.MainWindow):

"a Tk, with extra protocol and mixed-in methods"

def __init__(self):

windows.MainWindow.__init__(self, appname, srvrname)

PyMailServer.__init__(self)

class PyMailFileWindow(PyMailFile, windows.PopupWindow):

"a Toplevel, with extra protocol and mixed-in methods"

def __init__(self, filename):

windows.PopupWindow.__init__(self, appname, filename)

PyMailFile.__init__(self, filename)

The crucial point here is that using super for just the single inheritance cases where it applies most clearly is a potential source of error and confusion, and means that programmers must remember two ways to accomplish the same goal, when just one—explicit direct calls—could suffice for all cases.

In other words, unless you can be sure that you will never add a second superclass to a class in a tree over your software’s entire lifespan, you cannot use super in single-inheritance mode without understanding and allowing for its much more sophisticated role in multiple-inheritance trees. We’ll discuss the latter ahead, but it’s not optional if you deploy super at all.

From a more practical view, it’s also not clear that the trivial amount of code maintenance that this super role is envisioned to avoid fully justifies its presence. In Python practice, superclass names in headers are rarely changed; when they are, there are usually at most a very small number of superclass calls to update within the class. And consider this: if you add a new superclass in the future that doesn’t use super (as in the preceding example), you’ll have to either wrap it in an adaptor proxy or augment all the super calls in your class to use the traditional explicit-name call scheme anyhow—a maintenance task that seems just as likely, but perhaps more error-prone if you’ve grown to rely on super magic.

Limitation: Operator overloading

As briefly noted in Python’s library manual, super also doesn’t fully work in the presence of __X__ operator overloading methods. If you study the following code, you’ll see that direct named calls to overload methods in the superclass operate normally, but using the super result in an expression fails to dispatch to the superclass’s overload method:

>>> class C: # In Python 3.X

def __getitem__(self, ix): # Indexing overload method

print('C index')

>>> class D(C):

def __getitem__(self, ix): # Redefine to extend here

print('D index')

C.__getitem__(self, ix) # Traditional call form works

super().__getitem__(ix) # Direct name calls work too

super()[ix] # But operators do not! (__getattribute__)

>>> X = C()

>>> X[99]

C index

>>> X = D()

>>> X[99]

D index

C index

C index

Traceback (most recent call last):

File "", line 1, in

File "", line 6, in __getitem__

TypeError: 'super' object is not subscriptable

This behavior is due to the very same new-style (and 3.X) class change described earlier in this chapter (see Attribute Fetch for Built-ins Skips Instances)—because the proxy object returned by super uses __getattribute__ to catch and dispatch later method calls, it fails to intercept the automatic __X__ method invocations run by built-in operations including expressions, as these begin their search in the class instead of the instance. This may seem less severe than the multiple-inheritance limitation, but operators should generally work the same as the equivalent method call, especially for a built-in like this. Not supporting this adds another exception for super users to confront and remember.

Other languages’ mileage may vary, but in Python, self is explicit, multiple-inheritance mix-ins and operator overloading are common, and superclass name updates are rare. Because super adds an odd special case to the language—one with strange semantics, limited scope, rigid requirements, and questionable reward—most Python programmers may be better served by the more broadly applicable traditional call scheme. While super has some advanced applications too that we’ll study ahead, they may be too obscure to warrant making it a mandatory part of every Python programmer’s toolbox.

Use differs in Python 2.X: Verbose calls

If you are a Python 2.X user reading this dual-version book, you should also know that the super technique is not portable between Python lines. Its form differs between 2.X and 3.X—and not just between classic and new-style classes. It’s really a different tool in 2.X, which cannot run 3.X’s simpler form.

To make this call work in Python 2.X, you must first use new-style classes. Even then, you must also explicitly pass in the immediate class name and self to super, making this call so complex and verbose that in most cases it’s probably easier to avoid it completely, and simply name the superclass explicitly per the previous traditional code pattern (for brevity, I’ll leave it to readers to consider what changing a class’s own name means for code maintenance when using the 2.X super form!):

>>> class C(object): # In Python 2.X: for new-style classes only

def act(self):

print('spam')

>>> class D(C):

def act(self):

super(D, self).act() # 2.X: different call format - seems too complex

print('eggs') # "D" may be just as much to type/change as "C"!

>>> X = D()

>>> X.act()

spam

eggs

Although you can use the 2.X call form in 3.X for backward compatibility, it’s too cumbersome to deploy in 3.X-only code, and the more reasonable 3.X form is not usable in 2.X:

>>> class D(C):

def act(self):

super().act() # Simpler 3.X call format fails in 2.X

print('eggs')

>>> X = D()

>>> X.act()

TypeError: super() takes at least 1 argument (0 given)

On the other hand, the traditional call form with explicit class names works in 2.X in both classic and new-style classes, and exactly as it does in 3.X:

>>> class D(C):

def act(self):

C.act(self) # But traditional pattern works portably

print('eggs') # And may often be simpler in 2.X code

>>> X = D()

>>> X.act()

spam

eggs

So why use a technique that works in only limited contexts instead of one that works in many more? Though its basis is complex, the next sections attempt to rally support for the super cause.

The super Upsides: Tree Changes and Dispatch

Having just shown you the downsides of super, I should also confess that I’ve been tempted to use this call in code that would only ever run on 3.X, and which used a very long superclass reference path through a module package (that is, mostly for laziness, but coding brevity can matter too). To be fair, super may still be useful in some use cases, the chief among which merit a brief introduction here:

§ Changing class trees at runtime: When a superclass may be changed at runtime, it’s not possible to hardcode its name in a call expression, but it is possible to dispatch calls via super.

On the other hand, this case is extremely rare in Python programming, and other techniques can often be used in this context as well.

§ Cooperative multiple inheritance method dispatch: When multiple inheritance trees must dispatch to the same-named method in multiple classes, super can provide a protocol for orderly call routing.

On the other hand, the class tree must rely upon the ordering of classes by the MRO—a complex tool in its own right that is artificial to the problem a program is meant to address—and must be coded or augmented to use super in each version of the method in the tree to be effective. Such dispatch can also often be implemented in other ways (e.g., via instance state).

As discussed earlier, super can also be used to select a superclass generically as long as the MRO’s default makes sense, though in traditional code naming a superclass explicitly is often preferable, and may even be required. Moreover, even valid super use cases tend to be uncommon in many Python programs—to the point of seeming academic curiosity to some. The two cases just listed, however, are most often cited as super rationales, so let’s take a quick look at each.

Runtime Class Changes and super

Superclass that might be changed at runtime dynamically preclude hardcoding their names in a subclass’s methods, while super will happily look up the current superclass dynamically. Still, this case may be too rare in practice to warrant the super model by itself, and can often be implemented in other ways in the exceptional cases where it is needed. To illustrate, the following changes the superclass of C dynamically by changing the subclass’s __bases__ tuple in 3.X:

>>> class X:

def m(self): print('X.m')

>>> class Y:

def m(self): print('Y.m')

>>> class C(X): # Start out inheriting from X

def m(self): super().m() # Can't hardcode class name here

>>> i = C()

>>> i.m()

X.m

>>> C.__bases__ = (Y,) # Change superclass at runtime!

>>> i.m()

Y.m

This works (and shares behavior-morphing goals with other deep magic, such as changing an instance’s __class__), but seems rare in the extreme. Moreover, there may be other ways to achieve the same effect—perhaps most simply, calling through the current superclass tuple’s value indirectly: special code to be sure, but only for a very special case (and perhaps not any more special than implicit routing by MROs):

>>> class C(X):

def m(self): C.__bases__[0].m(self) # Special code for a special case

>>> i = C()

>>> i.m()

X.m

>>> C.__bases__ = (Y,) # Same effect, without super()

>>> i.m()

Y.m

Given the preexisting alternatives, this case alone doesn’t seem to justify super, though in more complex trees, the next rationale—based on the tree’s MRO order instead of physical superclass links—may apply here as well.

Cooperative Multiple Inheritance Method Dispatch

The second of the use cases listed earlier is the main rationale commonly given for super, and also borrows from other programming languages (most notably, Dylan), where its use case may be more common than it is in typical Python code. It generally applies to diamond pattern multiple inheritance trees, discussed earlier in this chapter, and allows for cooperative and conformant classes to route calls to a same-named method coherently among multiple class implementations. Especially for constructors, which have multiple implementations normally, this can simplify call routing protocol when used consistently.

In this mode, each super call selects the method from a next class following it in the MRO ordering of the class of the self subject of a method call. The MRO was introduced earlier; it’s the path Python follows for inheritance in new-style classes. Because the MRO’s linear ordering depends on which class self was made from, the order of method dispatch orchestrated by super can vary per class tree, and visits each class just once as long as all classes use super to dispatch.

Since every class participates in a diamond under object in 3.X (and 2.X new-style classes), the applications are broader than you might expect. In fact, some of the earlier examples that demonstrated super shortcomings in multiple inheritance trees could use this call to achieve their dispatch goals. To do so, however, super must be used universally in the class tree to ensure that method call chains are passed on—a fairly major requirement that may be difficult to enforce in much existing and new code.

The basics: Cooperative super call in action

Let’s take a look at what this role means in code. In this and the following sections, we’ll both learn how super works, and explore the tradeoffs it implies along the way. To get started, consider the following traditionally coded Python classes (condensed somewhat here as usual for space):

>>> class B:

def __init__(self): print('B.__init__') # Disjoint class tree branches

>>> class C:

def __init__(self): print('C.__init__')

>>> class D(B, C): pass

>>> x = D() # Runs leftmost only by default

B.__init__

In this case, superclass tree branches are disjoint (they don’t share a common explicit ancestor), so subclasses that combine them must call through each superclass by name—a common situation in much existing Python code that super cannot address directly without code changes:

>>> class D(B, C):

def __init__(self): # Traditional form

B.__init__(self) # Invoke supers by name

C.__init__(self)

>>> x = D()

B.__init__

C.__init__

In diamond class tree patterns, though, explicit-name calls may by default trigger the top-level class’s method more than once, though this might be subverted with additional protocols (e.g., status markers in the instance):

>>> class A:

def __init__(self): print('A.__init__')

>>> class B(A):

def __init__(self): print('B.__init__'); A.__init__(self)

>>> class C(A):

def __init__(self): print('C.__init__'); A.__init__(self)

>>> x = B()

B.__init__

A.__init__

>>> x = C() # Each super works by itself

C.__init__

A.__init__

>>> class D(B, C): pass # Still runs leftmost only

>>> x = D()

B.__init__

A.__init__

>>> class D(B, C):

def __init__(self): # Traditional form

B.__init__(self) # Invoke both supers by name

C.__init__(self)

>>> x = D() # But this now invokes A twice!

B.__init__

A.__init__

C.__init__

A.__init__

By contrast, if all classes use super, or are appropriately coerced by proxies to behave as if they do, the method calls are dispatched according to class order in the MRO, such that the top-level class’s method is run just once:

>>> class A:

def __init__(self): print('A.__init__')

>>> class B(A):

def __init__(self): print('B.__init__'); super().__init__()

>>> class C(A):

def __init__(self): print('C.__init__'); super().__init__()

>>> x = B() # Runs B.__init__, A is next super in self's B MRO

B.__init__

A.__init__

>>> x = C()

C.__init__

A.__init__

>>> class D(B, C): pass

>>> x = D() # Runs B.__init__, C is next super in self's D MRO!

B.__init__

C.__init__

A.__init__

The real magic behind this is the linear MRO list constructed for the class of self—because each class appears just once on this list, and because super dispatches to the next class on this list, it ensures an orderly invocation chain that visits each class just once. Crucially, the next class following B in the MRO differs depending on the class of self—it’s A for a B instance, but C for a D instance, accounting for the order of constructors run:

>>> B.__mro__

(<class '__main__.B'>, <class '__main__.A'>, <class 'object'>)

>>> D.__mro__

(<class '__main__.D'>, <class '__main__.B'>, <class '__main__.C'>,

<class '__main__.A'>, <class 'object'>)

The MRO and its algorithm were presented earlier in this chapter. By selecting a next class in the MRO sequence, a super call in a class’s method propagates the call through the tree, so long as all classes do the same. In this mode super does not necessarily choose a superclass at all; it picks the next in the linearized MRO, which might be a sibling—or even a lower relative—in the class tree of a given instance. See Tracing the MRO for other examples of the path super dispatch would follow, especially for nondiamonds.

The preceding works—and may even seem clever at first glance—but its scope may also appear limited to some. Most Python programs do not rely on the nuances of diamond pattern multiple inheritance trees (in fact, many Python programmers I’ve met do not know what the term means!). Moreover, super applies most directly to single inheritance and cooperative diamond cases, and may seem superfluous for disjoint nondiamond cases, where we might want to invoke superclass methods selectively or independently. Even cooperative diamonds can be managed in other ways that may afford programmers more control than an automatic MRO ordering can. To evaluate this tool objectively, though, we need to look deeper.

Constraint: Call chain anchor requirement

The super call comes with complexities that may not be apparent on first encounter, and may even seem initially like features. For example, because all classes inherit from object in 3.X automatically (and explicitly in 2.X new-style classes), the MRO ordering can be used even in cases where the diamond is only implicit—in the following, triggering constructors in independent classes automatically:

>>> class B:

def __init__(self): print('B.__init__'); super().__init__()

>>> class C:

def __init__(self): print('C.__init__'); super().__init__()

>>> x = B() # object is an implied super at the end of MRO

B.__init__

>>> x = C()

C.__init__

>>> class D(B, C): pass # Inherits B.__init__ but B's MRO differs for D

>>> x = D() # Runs B.__init__, C is next super in self's D MRO!

B.__init__

C.__init__

Technically, this dispatch model generally requires that the method being called by super must exist, and must have the same argument signature across the class tree, and every appearance of the method but the last must use super itself. This prior example works only because the impliedobject superclass at the end of the MRO of all three classes happens to have a compatible __init__ that satisfies these rules:

>>> B.__mro__

(<class '__main__.B'>, <class 'object'>)

>>> D.__mro__

(<class '__main__.D'>, <class '__main__.B'>, <class '__main__.C'>, <class 'object'>)

Here, for a D instance, the next class in the MRO after B is C, which is followed by object whose __init__ silently accepts the call from C and ends the chain. Thus, B’s method calls C’s, which ends in object’s version, even though C is not a superclass to B.

Really, though, this example is atypical—and perhaps even lucky. In most cases, no such suitable default will exist in object, and it may be less trivial to satisfy this model’s expectations. Most trees will require an explicit—and possibly extra—superclass to serve the anchoring role thatobject does here, to accept but not forward the call. Other trees may require careful design to adhere to this requirement. Moreover, unless Python optimizes it away, the call to object (or other anchor) defaults at the end of the chain may also add extra performance costs.

By contrast, in such cases direct calls incur neither extra coding requirements nor added performance cost, and make dispatch more explicit and direct:

>>> class B:

def __init__(self): print('B.__init__')

>>> class C:

def __init__(self): print('C.__init__')

>>> class D(B, C):

def __init__(self): B.__init__(self); C.__init__(self)

>>> x = D()

B.__init__

C.__init__

Scope: An all-or-nothing model

Also keep in mind that traditional classes that were not written to use super in this role cannot be directly used in such cooperative dispatch trees, as they will not forward calls along the MRO chain. It’s possible to incorporate such classes with proxies that wrap the original object and add the requisite super calls, but this imposes both additional coding requirements and performance costs on the model. Given that there are many millions of lines of existing Python code that do not use super, this seems a major detriment.

Watch what happens, for example, if any one class fails to pass along the call chain by omitting a super, ending the call chain prematurely—like __slots__, super is generally an all-or-nothing feature:

>>> class B:

def __init__(self): print('B.__init__'); super().__init__()

>>> class C:

def __init__(self): print('C.__init__'); super().__init__()

>>> class D(B, C):

def __init__(self): print('D.__init__'); super().__init__()

>>> X = D()

D.__init__

B.__init__

C.__init__

>>> D.__mro__

(<class '__main__.D'>, <class '__main__.B'>, <class '__main__.C'>, <class 'object'>)

# What if you must use a class that doesn't call super?

>>> class B:

def __init__(self): print('B.__init__')

>>> class D(B, C):

def __init__(self): print('D.__init__'); super().__init__()

>>> X = D()

D.__init__

B.__init__ # It's an all-or-nothing tool...

Satisfying this mandatory propagation requirement may be no simpler than direct by-name calls—which you might still forget, but which you won’t need to require of all the code your classes employ. As mentioned, it’s possible to adapt a class like B by inheriting from a proxy class that embeds B instances, but that seems artificial to program goals, adds an extra call to each wrapped method, is subject to the new-style class problems we met earlier regarding interface proxies and built-ins, and seems an extraordinary and even stunning added coding requirement inherent in a model intended to simplify code.

Flexibility: Call ordering assumptions

Routing with super also assumes that you really mean to pass method calls throughout all your classes per the MRO, which may or may not match your call ordering requirements. For example, imagine that—irrespective of other inheritance ordering needs—the following requires that the class C’s version of a given method be run before B’s in some contexts. If the MRO says otherwise, you’re back to traditional calls, which may conflict with super usage—in the following, invoking C’s method twice:

# What if method call ordering needs differ from the MRO?

>>> class B:

def __init__(self): print('B.__init__'); super().__init__()

>>> class C:

def __init__(self): print('C.__init__'); super().__init__()

>>> class D(B, C):

def __init__(self): print('D.__init__'); C.__init__(self); B.__init__(self)

>>> X = D()

D.__init__

C.__init__

B.__init__

C.__init__ # It's the MRO xor explicit calls...

Similarly, if you want some methods to not run at all, the super automatic path won’t apply as directly as explicit calls may, and will make it difficult to take more explicit control of the dispatch process. In realistic programs with many methods, resources, and state variables, these seem entirely plausible scenarios. While you could reorder superclasses in D for this method, that may break other expectations.

Customization: Method replacement

On a related note, the universal deployment expectations of super may make it difficult for a single class to replace (override) an inherited method altogether. Not passing the call higher with super—intentionally in this case—works fine for the class itself, but may break the call chain of trees it’s mixed into, thereby preventing methods elsewhere in the tree from running. Consider the following tree:

>>> class A:

def method(self): print('A.method'); super().method()

>>> class B(A):

def method(self): print('B.method'); super().method()

>>> class C:

def method(self): print('C.method') # No super: must anchor the chain!

>>> class D(B, C):

def method(self): print('D.method'); super().method()

>>> X = D()

>>> X.method()

D.method

B.method

A.method # Dispatch to all per the MRO automatically

C.method

Method replacement here breaks the super model, and probably leads us back to the traditional form:

# What if a class needs to replace a super's default entirely?

>>> class B(A):

def method(self): print('B.method') # Drop super to replace A's method

>>> class D(B, C):

def method(self): print('D.method'); super().method()

>>> X = D()

>>> X.method()

D.method

B.method # But replacement also breaks the call chain...

>>> class D(B, C):

def method(self): print('D.method'); B.method(self); C.method(self)

>>> D().method()

D.method

B.method

C.method # It's back to explicit calls...

Once again, the problem with assumptions is that they assume things! Although the assumption of universal routing might be reasonable for constructors, it would also seem to conflict with one of the core tenets of OOP—unrestricted subclass customization. This might suggest restrictingsuper usage to constructors, but even these might sometimes warrant replacement, and this adds an odd special-case requirement for one specific context. A tool that can be used only for certain categories of methods might be seen by some as redundant—and even spurious, given the extra complexity it implies.

Coupling: Application to mix-in classes

Subtly, when we say super selects the next class in the MRO, we really mean the next class in the MRO that implements the requested method—it technically skips ahead until it finds a class with the requested name. This matters for independent mix-in classes, which might be added to arbitrary client trees. Without this skipping-ahead behavior, such mix-ins wouldn’t work at all—they would otherwise drop the call chain of their clients’ arbitrary methods, and couldn’t rely on their own super calls to work as expected.

In the following independent branches, for example, C’s call to method is passed on, even though Mixin, the next class in the C instance’s MRO, doesn’t define that method’s name. As long as method name sets are disjoint, this just works—the call chains of each branch can exist independently:

# Mix-ins work for disjoint method sets

>>> class A:

def other(self): print('A.other')

>>> class Mixin(A):

def other(self): print('Mixin.other'); super().other()

>>> class B:

def method(self): print('B.method')

>>> class C(Mixin, B):

def method(self): print('C.method'); super().other(); super().method()

>>> C().method()

C.method

Mixin.other

A.other

B.method

>>> C.__mro__

(<class '__main__.C'>, <class '__main__.Mixin'>, <class '__main__.A'>,

<class '__main__.B'>, <class 'object'>)

Similarly, mixing the other way doesn’t break call chains of the mix-in either. For instance, in the following, even though B doesn’t define other when called in C, classes do later in the MRO. In fact, the call chains work even if one of the branches doesn’t use super at all—as long as a method is defined somewhere ahead on the MRO, its call works:

>>> class C(B, Mixin):

def method(self): print('C.method'); super().other(); super().method()

>>> C().method()

C.method

Mixin.other

A.other

B.method

>>> C.__mro__

(<class '__main__.C'>, <class '__main__.B'>, <class '__main__.Mixin'>,

<class '__main__.A'>, <class 'object'>)

This is also true in the presence of diamonds—disjoint method sets are dispatched as expected, even if not implemented by each disjoint branch, because we select the next on the MRO with the method. Really, because the MRO contains the same classes in these cases, and because a subclass always appears before its superclass in the MRO, they are equivalent contexts. For example, the call in Mixin to other in the following still finds it in A, even though the next class after Mixin on the MRO is B (the call to method in C works again for similar reasons):

# Explicit diamonds work too

>>> class A:

def other(self): print('A.other')

>>> class Mixin(A):

def other(self): print('Mixin.other'); super().other()

>>> class B(A):

def method(self): print('B.method')

>>> class C(Mixin, B):

def method(self): print('C.method'); super().other(); super().method()

>>> C().method()

C.method

Mixin.other

A.other

B.method

>>> C.__mro__

(<class '__main__.C'>, <class '__main__.Mixin'>, <class '__main__.B'>,

<class '__main__.A'>, <class 'object'>)

# Other mix-in orderings work too

>>> class C(B, Mixin):

def method(self): print('C.method'); super().other(); super().method()

>>> C().method()

C.method

Mixin.other

A.other

B.method

>>> C.__mro__

(<class '__main__.C'>, <class '__main__.B'>, <class '__main__.Mixin'>,

<class '__main__.A'>, <class 'object'>)

Still, this has an effect that is no different—but may seem wildly more implicit—than direct by-name calls, which also work the same in this case regardless of superclass ordering, and whether there is a diamond or not. In this case, the motivation for relying on MRO ordering seems on shaky ground, if the traditional form is both simpler and more explicit, and offers more control and flexibility:

# But direct calls work here too: explicit is better than implicit

>>> class C(Mixin, B):

def method(self): print('C.method'); Mixin.other(self); B.method(self)

>>> X = C()

>>> X.method()

C.method

Mixin.other

A.other

B.method

More crucially, this example so far assumes that method names are disjoint in its branches; the dispatch order for same-named methods in diamonds like this may be much less fortuitous. In a diamond like the preceding, for example, it’s not impossible that a client class could invalidate asuper call’s intent—the call to method in Mixin in the following works to run A’s version as expected, unless it’s mixed into a tree that drops the call chain:

# But for nondisjoint methods: super creates overly strong coupling

>>> class A:

def method(self): print('A.method')

>>> class Mixin(A):

def method(self): print('Mixin.method'); super().method()

>>> Mixin().method()

Mixin.method

A.method

>>> class B(A):

def method(self): print('B.method') # super here would invoke A after B

>>> class C(Mixin, B):

def method(self): print('C.method'); super().method()

>>> C().method()

C.method

Mixin.method

B.method # We miss A in this context only!

It may be that B shouldn’t redefine this method anyhow (and frankly, we may be encroaching on problems inherent in multiple inheritance in general), but this need not also break the mix-in—direct calls give you more control in such cases, and allow mix-in classes to be much more independent of usage contexts:

# And direct calls do not: they are immune to context of use

>>> class A:

def method(self): print('A.method')

>>> class Mixin(A):

def method(self): print('Mixin.method'); A.method(self) # C irrelevant

>>> class C(Mixin, B):

def method(self): print('C.method'); Mixin.method(self)

>>> C().method()

C.method

Mixin.method

A.method

More to the point, by making mix-ins more self-contained, direct calls minimize component coupling that always skews program complexity higher—a fundamental software principle that seems neglected by super’s variable and context-specific dispatch model.

Customization: Same-argument constraints

As a final note, you should also consider the consequences of using super when method arguments differ per class—because a class coder can’t be sure which version of a method super might invoke (indeed, this may vary per tree!), every version of the method must generally accept the same arguments list, or choose its inputs with analysis of generic argument lists—either of which imposes additional requirements on your code. In realistic programs, this constraint may in fact be a true showstopper for many potential super applications, precluding its use entirely.

To illustrate why this can matter, recall the pizza shop employee classes we wrote in Chapter 31. As coded there, both subclasses use direct by-name calls to invoke the superclass constructor, filling in an expected salary argument automatically—the logic being that the subclass implies the pay grade:

>>> class Employee:

def __init__(self, name, salary): # Common superclass

self.name = name

self.salary = salary

>>> class Chef1(Employee):

def __init__(self, name): # Differing arguments

Employee.__init__(self, name, 50000) # Dispatch by direct call

>>> class Server1(Employee):

def __init__(self, name):

Employee.__init__(self, name, 40000)

>>> bob = Chef1('Bob')

>>> sue = Server1('Sue')

>>> bob.salary, sue.salary

(50000, 40000)

This works, but since this is a single-inheritance tree, we might be tempted to deploy super here to route the constructor calls generically. Doing so works for either subclass in isolation, since its MRO includes just itself and its actual superclass:

>>> class Chef2(Employee):

def __init__(self, name):

super().__init__(name, 50000) # Dispatch by super()

>>> class Server2(Employee):

def __init__(self, name):

super().__init__(name, 40000)

>>> bob = Chef2('Bob')

>>> sue = Server2('Sue')

>>> bob.salary, sue.salary

(50000, 40000)

Watch what happens, though, when an employee is a member of both categories. Because the constructors in the tree have differing argument lists, we’re in trouble:

>>> class TwoJobs(Chef2, Server2): pass

>>> tom = TwoJobs('Tom')

TypeError: __init__() takes 2 positional arguments but 3 were given

The problem here is that the super call in Chef2 no longer invokes its Employee superclass, but instead invokes its sibling class and follower on the MRO, Server2. Since this sibling has a differing argument list than the true superclass—expecting just self and name—the code breaks. This is inherent in super use: because the MRO can differ per tree, it might call different versions of a method in different trees—even some you may not be able to anticipate when coding a class by itself:

>>> TwoJobs.__mro__

(<class '__main__.TwoJobs'>, <class '__main__.Chef2'>, <class '__main__.Server2'>

<class '__main__.Employee'>, <class 'object'>)

>>> Chef2.__mro__

(<class '__main__.Chef2'>, <class '__main__.Employee'>, <class 'object'>)

By contrast, the direct by-name call scheme still works when the classes are mixed, though the results are a bit dubious—the combined category gets the pay of the leftmost superclass:

>>> class TwoJobs(Chef1, Server1): pass

>>> tom = TwoJobs('Tom')

>>> tom.salary

50000

Really, we probably want to route the call to the top-level class in this event with a new salary—a model that is possible with direct calls but not with super alone. Moreover, calling Employee directly in this one class means our code uses two dispatch techniques when just one—direct calls—would suffice:

>>> class TwoJobs(Chef1, Server1):

def __init__(self, name): Employee.__init__(self, name, 70000)

>>> tom = TwoJobs('Tom')

>>> tom.salary

70000

>>> class TwoJobs(Chef2, Server2):

def __init__(self, name): super().__init__(name, 70000)

>>> tom = TwoJobs('Tom')

TypeError: __init__() takes 2 positional arguments but 3 were given

This example may warrant redesign in general—splitting off shareable parts of Chef and Server to mix-in classes without a constructor, for example. It’s also true that polymorphism in general assumes that the methods in an object’s external interface have the same argument signature, though this doesn’t quite apply to customization of superclass methods—an internal implementation technique that should by nature support variation, especially in constructors.

But the crucial point here is that because direct calls do not make code dependent on a magic ordering that can vary per tree, they more directly support argument list flexibility. More broadly, the questionable (or weak) performances super turns in on method replacement, mix-in coupling, call ordering, and argument constraints should make you evaluate its deployment carefully. Even in single-inheritance mode, its potential for later impacts as trees grow is considerable.

In sum, the three requirements of super in this role are also the source of most of its usability issues:

§ The method called by super must exist—which requires extra code if no anchor is present.

§ The method called by super must have the same argument signature across the class tree—which impairs flexibility, especially for implementation-level methods like constructors.

§ Every appearance of the method called by super but the last must use super itself—which makes it difficult to use existing code, change call ordering, override methods, and code self-contained classes.

Taken together, these seem to make for a tool with both substantial complexity and significant tradeoffs—downsides that will assert themselves the moment the code grows to incorporate multiple inheritance.

Naturally, there may be creative workarounds for the super dilemmas just posed, but additional coding steps would further dilute the call’s benefits—and we’ve run out of space here in any event. There are also alternative non-super solutions to some diamond method dispatch problems, but these will have to be left as a user exercise for space reasons too. In general, when superclass methods are called by explicit name, root classes of diamonds might check state in instances to avoid firing twice—a similarly complex coding pattern, but required rarely in most code, and which to some may seem no more difficult than using super itself.

The super Summary

So there it is—the bad and the good. As with all Python extensions, you should be the judge on this one too. I’ve tried to give both sides of the debate a fair shake here to help you decide. But because the super call:

§ Differs in form between 2.X and 3.X

§ In 3.X, relies on arguably non-Pythonic magic, and does not fully apply to operator overloading or traditionally coded multiple-inheritance trees

§ In 2.X, seems so verbose in this intended role that it may make code more complex instead of less

§ Claims code maintenance benefits that may be more hypothetical than real in Python practice

even ex–Java programmers should also consider this book’s preferred traditional technique of explicit-name superclass calls to be at least as valid a solution as Python’s super—a call that on some levels seems an unusual and limited answer to a question that was not being asked by most Python programmers, and was not deemed important for much of Python’s history.

At the same time, the super call offers one solution to the difficult problem of same-named method dispatch in multiple inheritance trees, for programs that choose to use it universally and consistently. But therein lies one of its largest obstacles: it requires universal deployment to address a problem most programmers probably do not have. Moreover, at this point in Python’s history, asking programmers to change their existing code to use this call widely enough to make it reliable seems highly unrealistic.

Perhaps the chief problem of this role, though, is the role itself—same-named method dispatch in multiple inheritance trees is relatively rare in real Python programs, and obscure enough to have generated both much controversy and much misunderstanding surrounding this role. People don’t use Python the same way they use C++, Java, or Dylan, and lessons from other such languages do not necessarily apply.

Also keep in mind that using super makes your program’s behavior dependent on the MRO algorithm—a procedure that we’ve covered only informally here due to its complexity, that is artificial to your program’s purpose, and that seems tersely documented and understood in the Python world. As we’ve seen, even if you understand the MRO, its implications on customization, coupling, and flexibility are remarkably subtle. If you don’t completely understand this algorithm—or have goals that its application does not address—you may be better served not relying on it to implicitly trigger actions in your code.

Or, to quote a Python motto from its import this creed:

If the implementation is hard to explain, it’s a bad idea.

The super call seems firmly in this category. Most programmers won’t use an arcane tool aimed at a rare use case, no matter how clever it may be. This is especially true in a scripting language that bills itself as friendly to nonspecialists. Regrettably, use by any programmer can impose such a tool on others anyhow—the real reason I’ve covered it here, and a theme we’ll revisit at the end of this book.

As usual, time and user base will tell if this call’s tradeoffs or momentum lead to broader adoption or not. At the least, it behooves you to also know about the traditional explicit-name superclass call technique, as it is still commonly used and often either simpler or required in today’s real-world Python programming. If you do choose to use this tool, my own advice to readers is to remember that using super:

§ In single-inheritance mode can mask later problems and lead to unexpected behavior as trees grow

§ In multiple-inheritance mode brings with it substantial complexity for an atypical Python use case

For other opinions on Python’s super that go into further details both good and bad, search the Web for related articles. You can find plenty of additional positions, though in the end, Python’s future relies as much on yours as any other.