Learning Python (2013)

---

^[21^] As for lists, you might not see dictionaries coded in full using literals very often—programs rarely know all their data before they are run, and more typically extract it dynamically from users, files, and so on. Lists and dictionaries are grown in different ways, though. In the next section you’ll see that you often build up dictionaries by assigning to new keys at runtime; this approach fails for lists, which are commonly grown with append or extend instead.

Dictionaries in Action

As Table 8-2 suggests, dictionaries are indexed by key, and nested dictionary entries are referenced by a series of indexes (keys in square brackets). When Python creates a dictionary, it stores its items in any left-to-right order it chooses; to fetch a value back, you supply the key with which it is associated, not its relative position. Let’s go back to the interpreter to get a feel for some of the dictionary operations in Table 8-2.

Basic Dictionary Operations

In normal operation, you create dictionaries with literals and store and access items by key with indexing:

% python

>>> D = {'spam': 2, 'ham': 1, 'eggs': 3} # Make a dictionary

>>> D['spam'] # Fetch a value by key

2

>>> D # Order is "scrambled"

{'eggs': 3, 'spam': 2, 'ham': 1}

Here, the dictionary is assigned to the variable D; the value of the key 'spam' is the integer 2, and so on. We use the same square bracket syntax to index dictionaries by key as we did to index lists by offset, but here it means access by key, not by position.

Notice the end of this example—much like sets, the left-to-right order of keys in a dictionary will almost always be different from what you originally typed. This is on purpose: to implement fast key lookup (a.k.a. hashing), keys need to be reordered in memory. That’s why operations that assume a fixed left-to-right order (e.g., slicing, concatenation) do not apply to dictionaries; you can fetch values only by key, not by position. Technically, the ordering is pseudo-random—it’s not truly random (you might be able to decipher it given Python’s source code and a lot of time to kill), but it’s arbitrary, and might vary per release and platform, and even per interactive session in Python 3.3.

The built-in len function works on dictionaries, too; it returns the number of items stored in the dictionary or, equivalently, the length of its keys list. The dictionary in membership operator allows you to test for key existence, and the keys method returns all the keys in the dictionary. The latter of these can be useful for processing dictionaries sequentially, but you shouldn’t depend on the order of the keys list. Because the keys result can be used as a normal list, however, it can always be sorted if order matters (more on sorting and dictionaries later):

>>> len(D) # Number of entries in dictionary

3

>>> 'ham' in D # Key membership test alternative

True

>>> list(D.keys()) # Create a new list of D's keys

['eggs', 'spam', 'ham']

Observe the second expression in this listing. As mentioned earlier, the in membership test used for strings and lists also works on dictionaries—it checks whether a key is stored in the dictionary. Technically, this works because dictionaries define iterators that step through their keys lists automatically. Other types provide iterators that reflect their common uses; files, for example, have iterators that read line by line. We’ll discuss iterators more formally in Chapter 14 and Chapter 20.

Also note the syntax of the last example in this listing. We have to enclose it in a list call in Python 3.X for similar reasons—keys in 3.X returns an iterable object, instead of a physical list. The list call forces it to produce all its values at once so we can print them interactively, though this call isn’t required some other contexts. In 2.X, keys builds and returns an actual list, so the list call isn’t even needed to display a result; more on this later in this chapter.

Changing Dictionaries in Place

Let’s continue with our interactive session. Dictionaries, like lists, are mutable, so you can change, expand, and shrink them in place without making new dictionaries: simply assign a value to a key to change or create an entry. The del statement works here, too; it deletes the entry associated with the key specified as an index. Notice also the nesting of a list inside a dictionary in this example (the value of the key 'ham'). All collection data types in Python can nest inside each other arbitrarily:

>>> D

{'eggs': 3, 'spam': 2, 'ham': 1}

>>> D['ham'] = ['grill', 'bake', 'fry'] # Change entry (value=list)

>>> D

{'eggs': 3, 'spam': 2, 'ham': ['grill', 'bake', 'fry']}

>>> del D['eggs'] # Delete entry

>>> D

{'spam': 2, 'ham': ['grill', 'bake', 'fry']}

>>> D['brunch'] = 'Bacon' # Add new entry

>>> D

{'brunch': 'Bacon', 'spam': 2, 'ham': ['grill', 'bake', 'fry']}

Like lists, assigning to an existing index in a dictionary changes its associated value. Unlike lists, however, whenever you assign a new dictionary key (one that hasn’t been assigned before) you create a new entry in the dictionary, as was done in the previous example for the key 'brunch'. This doesn’t work for lists because you can only assign to existing list offsets—Python considers an offset beyond the end of a list out of bounds and raises an error. To expand a list, you need to use tools such as the append method or slice assignment instead.

More Dictionary Methods

Dictionary methods provide a variety of type-specific tools. For instance, the dictionary values and items methods return all of the dictionary’s values and (key,value) pair tuples, respectively; along with keys, these are useful in loops that need to step through dictionary entries one by one (we’ll start coding examples of such loops in the next section). As for keys, these two methods also return iterable objects in 3.X, so wrap them in a list call there to collect their values all at once for display:

>>> D = {'spam': 2, 'ham': 1, 'eggs': 3}

>>> list(D.values())

[3, 2, 1]

>>> list(D.items())

[('eggs', 3), ('spam', 2), ('ham', 1)]

In realistic programs that gather data as they run, you often won’t be able to predict what will be in a dictionary before the program is launched, much less when it’s coded. Fetching a nonexistent key is normally an error, but the get method returns a default value—None, or a passed-in default—if the key doesn’t exist. It’s an easy way to fill in a default for a key that isn’t present, and avoid a missing-key error when your program can’t anticipate contents ahead of time:

>>> D.get('spam') # A key that is there

2

>>> print(D.get('toast')) # A key that is missing

None

>>> D.get('toast', 88)

88

The update method provides something similar to concatenation for dictionaries, though it has nothing to do with left-to-right ordering (again, there is no such thing in dictionaries). It merges the keys and values of one dictionary into another, blindly overwriting values of the same key if there’s a clash:

>>> D

{'eggs': 3, 'spam': 2, 'ham': 1}

>>> D2 = {'toast':4, 'muffin':5} # Lots of delicious scrambled order here

>>> D.update(D2)

>>> D

{'eggs': 3, 'muffin': 5, 'toast': 4, 'spam': 2, 'ham': 1}

Notice how mixed up the key order is in the last result; again, that’s just how dictionaries work. Finally, the dictionary pop method deletes a key from a dictionary and returns the value it had. It’s similar to the list pop method, but it takes a key instead of an optional position:

# pop a dictionary by key

>>> D

{'eggs': 3, 'muffin': 5, 'toast': 4, 'spam': 2, 'ham': 1}

>>> D.pop('muffin')

5

>>> D.pop('toast') # Delete and return from a key

4

>>> D

{'eggs': 3, 'spam': 2, 'ham': 1}

# pop a list by position

>>> L = ['aa', 'bb', 'cc', 'dd']

>>> L.pop() # Delete and return from the end

'dd'

>>> L

['aa', 'bb', 'cc']

>>> L.pop(1) # Delete from a specific position

'bb'

>>> L

['aa', 'cc']

Dictionaries also provide a copy method; we’ll revisit this in Chapter 9, as it’s a way to avoid the potential side effects of shared references to the same dictionary. In fact, dictionaries come with more methods than those listed in Table 8-2; see the Python library manual, dir and help, or other reference sources for a comprehensive list.

NOTE

Your dictionary ordering may vary: Don’t be alarmed if your dictionaries print in a different order than shown here. As mentioned, key order is arbitrary, and might vary per release, platform, and interactive session in 3.3 (and quite possibly per day of the week, and phase of the moon!).

Most of the dictionary examples in this book reflect Python 3.3’s key ordering, but it has changed both since and prior to 3.0. Your Python’s key order may vary, but you’re not supposed to care anyhow: dictionaries are processed by key, not position. Programs shouldn’t rely on the arbitrary order of keys in dictionaries, even if shown in books.

There are extension types in Python’s standard library that maintain insertion order among their keys—see OrderedDict in the collections module—but they are hybrids that incur extra space and speed overheads to achieve their extra utility, and are not true dictionaries. In short, keys are kept redundantly in a linked list to support sequence operations.

As we’ll see in Chapter 9, this module also implements a namedtuple that allows tuple items to be accessed by both attribute name and sequence position—a sort of tuple/class/dictionary hybrid that adds processing steps and is not a core object type in any event. Python’s library manual has the full story on these and other extension types.

Example: Movie Database

Let’s look at a more realistic dictionary example. In honor of Python’s namesake, the following example creates a simple in-memory Monty Python movie database, as a table that maps movie release date years (the keys) to movie titles (the values). As coded, you fetch movie names by indexing on release year strings:

>>> table = {'1975': 'Holy Grail', # Key: Value

... '1979': 'Life of Brian',

... '1983': 'The Meaning of Life'}

>>>

>>> year = '1983'

>>> movie = table[year] # dictionary[Key] => Value

>>> movie

'The Meaning of Life'

>>> for year in table: # Same as: for year in table.keys()

... print(year + '\t' + table[year])

...

1979 Life of Brian

1975 Holy Grail

1983 The Meaning of Life

The last command uses a for loop, which we previewed in Chapter 4 but haven’t covered in detail yet. If you aren’t familiar with for loops, this command simply iterates through each key in the table and prints a tab-separated list of keys and their values. We’ll learn more about for loops in Chapter 13.

Dictionaries aren’t sequences like lists and strings, but if you need to step through the items in a dictionary, it’s easy—calling the dictionary keys method returns all stored keys, which you can iterate through with a for. If needed, you can index from key to value inside the for loop as you go, as was done in this code.

In fact, Python also lets you step through a dictionary’s keys list without actually calling the keys method in most for loops. For any dictionary D, saying for key in D works the same as saying the complete for key in D.keys(). This is really just another instance of the iteratorsmentioned earlier, which allow the in membership operator to work on dictionaries as well; more on iterators later in this book.

Preview: Mapping values to keys

Notice how the prior table maps year to titles, but not vice versa. If you want to map the other way—titles to years—you can either code the dictionary differently, or use methods like items that give searchable sequences, though using them to best effect requires more background information than we yet have:

>>> table = {'Holy Grail': '1975', # Key=>Value (title=>year)

... 'Life of Brian': '1979',

... 'The Meaning of Life': '1983'}

>>>

>>> table['Holy Grail']

'1975'

>>> list(table.items()) # Value=>Key (year=>title)

[('The Meaning of Life', '1983'), ('Holy Grail', '1975'), ('Life of Brian', '1979')]

>>> [title for (title, year) in table.items() if year == '1975']

['Holy Grail']

The last command here is in part a preview for the comprehension syntax introduced in Chapter 4 and covered in full in Chapter 14. In short, it scans the dictionary’s (key, value) tuple pairs returned by the items method, selecting keys having a specified value. The net effect is to indexbackward—from value to key, instead of key to value—useful if you want to store data just once and map backward only rarely (searching through sequences like this is generally much slower than a direct key index).

In fact, although dictionaries by nature map keys to values unidirectionally, there are multiple ways to map values back to keys with a bit of extra generalizable code:

>>> K = 'Holy Grail'

>>> table[K] # Key=>Value (normal usage)

'1975'

>>> V = '1975'

>>> [key for (key, value) in table.items() if value == V] # Value=>Key

['Holy Grail']

>>> [key for key in table.keys() if table[key] == V] # Ditto

['Holy Grail']

Note that both of the last two commands return a list of titles: in dictionaries, there’s just one value per key, but there may be many keys per value. A given value may be stored under multiple keys (yielding multiple keys per value), and a value might be a collection itself (supporting multiple values per key). For more on this front, also watch for a dictionary inversion function in Chapter 32’s mapattrs.py example—code that would surely stretch this preview past its breaking point if included here. For this chapter’s purposes, let’s explore more dictionary basics.

Dictionary Usage Notes

Dictionaries are fairly straightforward tools once you get the hang of them, but here are a few additional pointers and reminders you should be aware of when using them:

§ Sequence operations don’t work. Dictionaries are mappings, not sequences; because there’s no notion of ordering among their items, things like concatenation (an ordered joining) and slicing (extracting a contiguous section) simply don’t apply. In fact, Python raises an error when your code runs if you try to do such things.

§ Assigning to new indexes adds entries. Keys can be created when you write a dictionary literal (embedded in the code of the literal itself), or when you assign values to new keys of an existing dictionary object individually. The end result is the same.

§ Keys need not always be strings. Our examples so far have used strings as keys, but any other immutable objects work just as well. For instance, you can use integers as keys, which makes the dictionary look much like a list (when indexing, at least). Tuples may be used as dictionary keys too, allowing compound key values—such as dates and IP addresses—to have associated values. User-defined class instance objects (discussed in Part VI) can also be used as keys, as long as they have the proper protocol methods; roughly, they need to tell Python that their values are “hashable” and thus won’t change, as otherwise they would be useless as fixed keys. Mutable objects such as lists, sets, and other dictionaries don’t work as keys, but are allowed as values.

Using dictionaries to simulate flexible lists: Integer keys

The last point in the prior list is important enough to demonstrate with a few examples. When you use lists, it is illegal to assign to an offset that is off the end of the list:

>>> L = []

>>> L[99] = 'spam'

Traceback (most recent call last):

File "<stdin>", line 1, in ?

IndexError: list assignment index out of range

Although you can use repetition to preallocate as big a list as you’ll need (e.g., [0]*100), you can also do something that looks similar with dictionaries that does not require such space allocations. By using integer keys, dictionaries can emulate lists that seem to grow on offset assignment:

>>> D = {}

>>> D[99] = 'spam'

>>> D[99]

'spam'

>>> D

{99: 'spam'}

Here, it looks as if D is a 100-item list, but it’s really a dictionary with a single entry; the value of the key 99 is the string 'spam'. You can access this structure with offsets much like a list, catching nonexistent keys with get or in tests if required, but you don’t have to allocate space for all the positions you might ever need to assign values to in the future. When used like this, dictionaries are like more flexible equivalents of lists.

As another example, we might also employ integer keys in our first movie database’s code earlier to avoid quoting the year, albeit at the expense of some expressiveness (keys cannot contain nondigit characters):

>>> table = {1975: 'Holy Grail',

... 1979: 'Life of Brian', # Keys are integers, not strings

... 1983: 'The Meaning of Life'}

>>> table[1975]

'Holy Grail'

>>> list(table.items())

[(1979, 'Life of Brian'), (1983, 'The Meaning of Life'), (1975, 'Holy Grail')]

Using dictionaries for sparse data structures: Tuple keys

In a similar way, dictionary keys are also commonly leveraged to implement sparse data structures—for example, multidimensional arrays where only a few positions have values stored in them:

>>> Matrix = {}

>>> Matrix[(2, 3, 4)] = 88

>>> Matrix[(7, 8, 9)] = 99

>>>

>>> X = 2; Y = 3; Z = 4 # ; separates statements: see Chapter 10

>>> Matrix[(X, Y, Z)]

88

>>> Matrix

{(2, 3, 4): 88, (7, 8, 9): 99}

Here, we’ve used a dictionary to represent a three-dimensional array that is empty except for the two positions (2,3,4) and (7,8,9). The keys are tuples that record the coordinates of nonempty slots. Rather than allocating a large and mostly empty three-dimensional matrix to hold these values, we can use a simple two-item dictionary. In this scheme, accessing an empty slot triggers a nonexistent key exception, as these slots are not physically stored:

>>> Matrix[(2,3,6)]

Traceback (most recent call last):

File "<stdin>", line 1, in ?

KeyError: (2, 3, 6)

Avoiding missing-key errors

Errors for nonexistent key fetches are common in sparse matrixes, but you probably won’t want them to shut down your program. There are at least three ways to fill in a default value instead of getting such an error message—you can test for keys ahead of time in if statements, use a trystatement to catch and recover from the exception explicitly, or simply use the dictionary get method shown earlier to provide a default for keys that do not exist. Consider the first two of these previews for statement syntax we’ll begin studying in Chapter 10:

>>> if (2, 3, 6) in Matrix: # Check for key before fetch

... print(Matrix[(2, 3, 6)]) # See Chapters 10 and 12 for if/else

... else:

... print(0)

...

0

>>> try:

... print(Matrix[(2, 3, 6)]) # Try to index

... except KeyError: # Catch and recover

... print(0) # See Chapters 10 and 34 for try/except

...

0

>>> Matrix.get((2, 3, 4), 0) # Exists: fetch and return

88

>>> Matrix.get((2, 3, 6), 0) # Doesn't exist: use default arg

0

Of these, the get method is the most concise in terms of coding requirements, but the if and try statements are much more general in scope; again, more on these starting in Chapter 10.

Nesting in dictionaries

As you can see, dictionaries can play many roles in Python. In general, they can replace search data structures (because indexing by key is a search operation) and can represent many types of structured information. For example, dictionaries are one of many ways to describe the properties of an item in your program’s domain; that is, they can serve the same role as “records” or “structs” in other languages.

The following, for example, fills out a dictionary describing a hypothetical person, by assigning to new keys over time (if you are a Bob, my apologies for picking on your name in this book—it’s easy to type!):

>>> rec = {}

>>> rec['name'] = 'Bob'

>>> rec['age'] = 40.5

>>> rec['job'] = 'developer/manager'

>>>

>>> print(rec['name'])

Bob

Especially when nested, Python’s built-in data types allow us to easily represent structured information. The following again uses a dictionary to capture object properties, but it codes it all at once (rather than assigning to each key separately) and nests a list and a dictionary to represent structured property values:

>>> rec = {'name': 'Bob',

... 'jobs': ['developer', 'manager'],

... 'web': 'www.bobs.org/˜Bob',

... 'home': {'state': 'Overworked', 'zip': 12345}}

To fetch components of nested objects, simply string together indexing operations:

>>> rec['name']

'Bob'

>>> rec['jobs']

['developer', 'manager']

>>> rec['jobs'][1]

'manager'

>>> rec['home']['zip']

12345

Although we’ll learn in Part VI that classes (which group both data and logic) can be better in this record role, dictionaries are an easy-to-use tool for simpler requirements. For more on record representation choices, see also the upcoming sidebar Why You Will Care: Dictionaries Versus Lists, as well as its extension to tuples in Chapter 9 and classes in Chapter 27.

Also notice that while we’ve focused on a single “record” with nested data here, there’s no reason we couldn’t nest the record itself in a larger, enclosing database collection coded as a list or dictionary, though an external file or formal database interface often plays the role of top-level container in realistic programs:

db = []

db.append(rec) # A list "database"

db.append(other)

db[0]['jobs']

db = {}

db['bob'] = rec # A dictionary "database"

db['sue'] = other

db['bob']['jobs']

Later in the book we’ll meet tools such as Python’s shelve, which works much the same way, but automatically maps objects to and from files to make them permanent (watch for more in this chapter’s sidebar Why You Will Care: Dictionary Interfaces).

Other Ways to Make Dictionaries

Finally, note that because dictionaries are so useful, more ways to build them have emerged over time. In Python 2.3 and later, for example, the last two calls to the dict constructor (really, type name) shown here have the same effect as the literal and key-assignment forms above them:

{'name': 'Bob', 'age': 40} # Traditional literal expression

D = {} # Assign by keys dynamically

D['name'] = 'Bob'

D['age'] = 40

dict(name='Bob', age=40) # dict keyword argument form

dict([('name', 'Bob'), ('age', 40)]) # dict key/value tuples form

All four of these forms create the same two-key dictionary, but they are useful in differing circumstances:

§ The first is handy if you can spell out the entire dictionary ahead of time.

§ The second is of use if you need to create the dictionary one field at a time on the fly.

§ The third involves less typing than the first, but it requires all keys to be strings.

§ The last is useful if you need to build up keys and values as sequences at runtime.

We met keyword arguments earlier when sorting; the third form illustrated in this code listing has become especially popular in Python code today, since it has less syntax (and hence there is less opportunity for mistakes). As suggested previously in Table 8-2, the last form in the listing is also commonly used in conjunction with the zip function, to combine separate lists of keys and values obtained dynamically at runtime (parsed out of a data file’s columns, for instance):

dict(zip(keyslist, valueslist)) # Zipped key/value tuples form (ahead)

More on zipping dictionary keys in the next section. Provided all the key’s values are the same initially, you can also create a dictionary with this special form—simply pass in a list of keys and an initial value for all of the values (the default is None):

>>> dict.fromkeys(['a', 'b'], 0)

{'a': 0, 'b': 0}

Although you could get by with just literals and key assignments at this point in your Python career, you’ll probably find uses for all of these dictionary-creation forms as you start applying them in realistic, flexible, and dynamic Python programs.

The listings in this section document the various ways to create dictionaries in both Python 2.X and 3.X. However, there is yet another way to create dictionaries, available only in Python 3.X and 2.7: the dictionary comprehension expression. To see how this last form looks, we need to move on to the next and final section of this chapter.

WHY YOU WILL CARE: DICTIONARIES VERSUS LISTS

With all the objects in Python’s core types arsenal, some readers may be puzzled over the choice between lists and dictionaries. In short, although both are flexible collections of other objects, lists assign items to positions, and dictionaries assign them to more mnemonic keys. Because of this, dictionary data often carries more meaning to human readers. For example, the nested list structure in row 3 of Table 8-1 could be used to represent a record too:

>>> L = ['Bob', 40.5, ['dev', 'mgr']] # List-based "record"

>>> L[0]

'Bob'

>>> L[1] # Positions/numbers for fields

40.5

>>> L[2][1]

'mgr'

For some types of data, the list’s access-by-position makes sense—a list of employees in a company, the files in a directory, or numeric matrixes, for example. But a more symbolic record like this may be more meaningfully coded as a dictionary along the lines of row 2 in Table 8-2, with labeled fields replacing field positions (this is similar to a record we coded in Chapter 4):

>>> D = {'name': 'Bob', 'age': 40.5, 'jobs': ['dev', 'mgr']}

>>> D['name']

'Bob'

>>> D['age'] # Dictionary-based "record"

40.5

>>> D['jobs'][1] # Names mean more than numbers

'mgr'

For variety, here is the same record recoded with keywords, which may seem even more readable to some human readers:

>>> D = dict(name='Bob', age=40.5, jobs=['dev', 'mgr'])

>>> D['name']

'Bob'

>>> D['jobs'].remove('mgr')

>>> D

{'jobs': ['dev'], 'age': 40.5, 'name': 'Bob'}

In practice, dictionaries tend to be best for data with labeled components, as well as structures that can benefit from quick, direct lookups by name, instead of slower linear searches. As we’ve seen, they also may be better for sparse collections and collections that grow at arbitrary positions.

Python programmers also have access to the sets we studied in Chapter 5, which are much like the keys of a valueless dictionary; they don’t map keys to values, but can often be used like dictionaries for fast lookups when there is no associated value, especially in search routines:

>>> D = {}

>>> D['state1'] = True # A visited-state dictionary

>>> 'state1' in D

True

>>> S = set()

>>> S.add('state1') # Same, but with sets

>>> 'state1' in S

True

Watch for a rehash of this record representation thread in the next chapter, where we’ll see how tuples and named tuples compare to dictionaries in this role, as well as in Chapter 27, where we’ll learn how user-defined classes factor into this picture, combining both data and logic to process it.

Dictionary Changes in Python 3.X and 2.7

This chapter has so far focused on dictionary basics that span releases, but the dictionary’s functionality has mutated in Python 3.X. If you are using Python 2.X code, you may come across some dictionary tools that either behave differently or are missing altogether in 3.X. Moreover, 3.X coders have access to additional dictionary tools not available in 2.X, apart from two back-ports to 2.7.

Specifically, dictionaries in Python 3.X:

§ Support a new dictionary comprehension expression, a close cousin to list and set comprehensions

§ Return set-like iterable views instead of lists for the methods D.keys, D.values, and D.items

§ Require new coding styles for scanning by sorted keys, because of the prior point

§ No longer support relative magnitude comparisons directly—compare manually instead

§ No longer have the D.has_key method—the in membership test is used instead

As later back-ports from 3.X, dictionaries in Python 2.7 (but not earlier in 2.X):

§ Support item 1 in the prior list—dictionary comprehensions—as a direct back-port from 3.X

§ Support item 2 in the prior list—set-like iterable views—but do so with special method names D.viewkeys, D.viewvalues, D.viewitems); their nonview methods return lists as before

Because of this overlap, some of the material in this section pertains both to 3.X and 2.7, but is presented here in the context of 3.X extensions because of its origin. With that in mind, let’s take a look at what’s new in dictionaries in 3.X and 2.7.

Dictionary comprehensions in 3.X and 2.7

As mentioned at the end of the prior section, dictionaries in 3.X and 2.7 can also be created with dictionary comprehensions. Like the set comprehensions we met in Chapter 5, dictionary comprehensions are available only in 3.X and 2.7 (not in 2.6 and earlier). Like the longstanding list comprehensions we met briefly in Chapter 4 and earlier in this chapter, they run an implied loop, collecting the key/value results of expressions on each iteration and using them to fill out a new dictionary. A loop variable allows the comprehension to use loop iteration values along the way.

To illustrate, a standard way to initialize a dictionary dynamically in both 2.X and 3.X is to combine its keys and values with zip, and pass the result to the dict call. The zip built-in function is the hook that allows us to construct a dictionary from key and value lists this way—if you cannot predict the set of keys and values in your code, you can always build them up as lists and zip them together. We’ll study zip in detail in Chapter 13 and Chapter 14 after exploring statements; it’s an iterable in 3.X, so we must wrap it in a list call to show its results there, but its basic usage is otherwise straightforward:

>>> list(zip(['a', 'b', 'c'], [1, 2, 3])) # Zip together keys and values

[('a', 1), ('b', 2), ('c', 3)]

>>> D = dict(zip(['a', 'b', 'c'], [1, 2, 3])) # Make a dict from zip result

>>> D

{'b': 2, 'c': 3, 'a': 1}

In Python 3.X and 2.7, though, you can achieve the same effect with a dictionary comprehension expression. The following builds a new dictionary with a key/value pair for every such pair in the zip result (it reads almost the same in Python, but with a bit more formality):

>>> D = {k: v for (k, v) in zip(['a', 'b', 'c'], [1, 2, 3])}

>>> D

{'b': 2, 'c': 3, 'a': 1}

Comprehensions actually require more code in this case, but they are also more general than this example implies—we can use them to map a single stream of values to dictionaries as well, and keys can be computed with expressions just like values:

>>> D = {x: x ** 2 for x in [1, 2, 3, 4]} # Or: range(1, 5)

>>> D

{1: 1, 2: 4, 3: 9, 4: 16}

>>> D = {c: c * 4 for c in 'SPAM'} # Loop over any iterable

>>> D

{'S': 'SSSS', 'P': 'PPPP', 'A': 'AAAA', 'M': 'MMMM'}

>>> D = {c.lower(): c + '!' for c in ['SPAM', 'EGGS', 'HAM']}

>>> D

{'eggs': 'EGGS!', 'spam': 'SPAM!', 'ham': 'HAM!'}

Dictionary comprehensions are also useful for initializing dictionaries from keys lists, in much the same way as the fromkeys method we met at the end of the preceding section:

>>> D = dict.fromkeys(['a', 'b', 'c'], 0) # Initialize dict from keys

>>> D

{'b': 0, 'c': 0, 'a': 0}

>>> D = {k:0 for k in ['a', 'b', 'c']} # Same, but with a comprehension

>>> D

{'b': 0, 'c': 0, 'a': 0}

>>> D = dict.fromkeys('spam') # Other iterables, default value

>>> D

{'s': None, 'p': None, 'a': None, 'm': None}

>>> D = {k: None for k in 'spam'}

>>> D

{'s': None, 'p': None, 'a': None, 'm': None}

Like related tools, dictionary comprehensions support additional syntax not shown here, including nested loops and if clauses. Unfortunately, to truly understand dictionary comprehensions, we need to also know more about iteration statements and concepts in Python, and we don’t yet have enough information to address that story well. We’ll learn much more about all flavors of comprehensions (list, set, dictionary, and generator) in Chapter 14 and Chapter 20, so we’ll defer further details until later. We’ll also revisit the zip built-in we used in this section in more detail inChapter 13, when we explore for loops.

Dictionary views in 3.X (and 2.7 via new methods)

In 3.X the dictionary keys, values, and items methods all return view objects, whereas in 2.X they return actual result lists. This functionality is also available in Python 2.7, but in the guise of the special, distinct method names listed at the start of this section (2.7’s normal methods still return simple lists, so as to avoid breaking existing 2.X code); because of this, I’ll refer to this as a 3.X feature in this section.

View objects are iterables, which simply means objects that generate result items one at a time, instead of producing the result list all at once in memory. Besides being iterable, dictionary views also retain the original order of dictionary components, reflect future changes to the dictionary, and may support set operations. On the other hand, because they are not lists, they do not directly support operations like indexing or the list sort method, and do not display their items as a normal list when printed (they do show their components as of Python 3.1 but not as a list, and are still a divergence from 2.X).

We’ll discuss the notion of iterables more formally in Chapter 14, but for our purposes here it’s enough to know that we have to run the results of these three methods through the list built-in if we want to apply list operations or display their values. For example, in Python 3.3 (other version’s outputs may differ slightly):

>>> D = dict(a=1, b=2, c=3)

>>> D

{'b': 2, 'c': 3, 'a': 1}

>>> K = D.keys() # Makes a view object in 3.X, not a list

>>> K

dict_keys(['b', 'c', 'a'])

>>> list(K) # Force a real list in 3.X if needed

['b', 'c', 'a']

>>> V = D.values() # Ditto for values and items views

>>> V

dict_values([2, 3, 1])

>>> list(V)

[2, 3, 1]

>>> D.items()

dict_items([('b', 2), ('c', 3), ('a', 1)])

>>> list(D.items())

[('b', 2), ('c', 3), ('a', 1)]

>>> K[0] # List operations fail unless converted

TypeError: 'dict_keys' object does not support indexing

>>> list(K)[0]

'b'

Apart from result displays at the interactive prompt, you will probably rarely even notice this change, because looping constructs in Python automatically force iterable objects to produce one result on each iteration:

>>> for k in D.keys(): print(k) # Iterators used automatically in loops

...

b

c

a

In addition, 3.X dictionaries still have iterators themselves, which return successive keys—as in 2.X, it’s still often not necessary to call keys directly:

>>> for key in D: print(key) # Still no need to call keys() to iterate

...

b

c

a

Unlike 2.X’s list results, though, dictionary views in 3.X are not carved in stone when created—they dynamically reflect future changes made to the dictionary after the view object has been created:

>>> D = {'a': 1, 'b': 2, 'c': 3}

>>> D

{'b': 2, 'c': 3, 'a': 1}

>>> K = D.keys()

>>> V = D.values()

>>> list(K) # Views maintain same order as dictionary

['b', 'c', 'a']

>>> list(V)

[2, 3, 1]

>>> del D['b'] # Change the dictionary in place

>>> D

{'c': 3, 'a': 1}

>>> list(K) # Reflected in any current view objects

['c', 'a']

>>> list(V) # Not true in 2.X! - lists detached from dict

[3, 1]

Dictionary views and sets

Also unlike 2.X’s list results, 3.X’s view objects returned by the keys method are set-like and support common set operations such as intersection and union; values views are not set-like, but items results are if their (key, value) pairs are unique and hashable (immutable). Given that sets behave much like valueless dictionaries (and may even be coded in curly braces like dictionaries in 3.X and 2.7), this is a logical symmetry. Per Chapter 5, set items are unordered, unique, and immutable, just like dictionary keys.

Here is what keys views look like when used in set operations (continuing the prior section’s session); dictionary value views are never set-like, since their items are not necessarily unique or immutable:

>>> K, V

(dict_keys(['c', 'a']), dict_values([3, 1]))

>>> K | {'x': 4} # Keys (and some items) views are set-like

{'c', 'x', 'a'}

>>> V & {'x': 4}

TypeError: unsupported operand type(s) for &: 'dict_values' and 'dict'

>>> V & {'x': 4}.values()

TypeError: unsupported operand type(s) for &: 'dict_values' and 'dict_values'

In set operations, views may be mixed with other views, sets, and dictionaries; dictionaries are treated the same as their keys views in this context:

>>> D = {'a': 1, 'b': 2, 'c': 3}

>>> D.keys() & D.keys() # Intersect keys views

{'b', 'c', 'a'}

>>> D.keys() & {'b'} # Intersect keys and set

{'b'}

>>> D.keys() & {'b': 1} # Intersect keys and dict

{'b'}

>>> D.keys() | {'b', 'c', 'd'} # Union keys and set

{'b', 'c', 'a', 'd'}

Items views are set-like too if they are hashable—that is, if they contain only immutable objects:

>>> D = {'a': 1}

>>> list(D.items()) # Items set-like if hashable

[('a', 1)]

>>> D.items() | D.keys() # Union view and view

{('a', 1), 'a'}

>>> D.items() | D # dict treated same as its keys

{('a', 1), 'a'}

>>> D.items() | {('c', 3), ('d', 4)} # Set of key/value pairs

{('d', 4), ('a', 1), ('c', 3)}

>>> dict(D.items() | {('c', 3), ('d', 4)}) # dict accepts iterable sets too

{'c': 3, 'a': 1, 'd': 4}

See Chapter 5’s coverage of sets if you need a refresher on these operations. Here, let’s wrap up with three other quick coding notes for 3.X dictionaries.

Sorting dictionary keys in 3.X

First of all, because keys does not return a list in 3.X, the traditional coding pattern for scanning a dictionary by sorted keys in 2.X won’t work in 3.X:

>>> D = {'a': 1, 'b': 2, 'c': 3}

>>> D

{'b': 2, 'c': 3, 'a': 1}

>>> Ks = D.keys() # Sorting a view object doesn't work!

>>> Ks.sort()

AttributeError: 'dict_keys' object has no attribute 'sort'

To work around this, in 3.X you must either convert to a list manually or use the sorted call (introduced in Chapter 4 and covered in this chapter) on either a keys view or the dictionary itself:

>>> Ks = list(Ks) # Force it to be a list and then sort

>>> Ks.sort()

>>> for k in Ks: print(k, D[k]) # 2.X: omit outer parens in prints

...

a 1

b 2

c 3

>>> D

{'b': 2, 'c': 3, 'a': 1}

>>> Ks = D.keys() # Or you can use sorted() on the keys

>>> for k in sorted(Ks): print(k, D[k]) # sorted() accepts any iterable

... # sorted() returns its result

a 1

b 2

c 3

Of these, using the dictionary’s keys iterator is probably preferable in 3.X, and works in 2.X as well:

>>> D

{'b': 2, 'c': 3, 'a': 1} # Better yet, sort the dict directly

>>> for k in sorted(D): print(k, D[k]) # dict iterators return keys

...

a 1

b 2

c 3

Dictionary magnitude comparisons no longer work in 3.X

Secondly, while in Python 2.X dictionaries may be compared for relative magnitude directly with <, >, and so on, in Python 3.X this no longer works. However, you can simulate it by comparing sorted keys lists manually:

sorted(D1.items()) < sorted(D2.items()) # Like 2.X D1 < D2

Dictionary equality tests (e.g., D1 == D2) still work in 3.X, though. Since we’ll revisit this near the end of the next chapter in the context of comparisons at large, we’ll postpone further details here.

The has_key method is dead in 3.X: Long live in!

Finally, the widely used dictionary has_key key presence test method is gone in 3.X. Instead, use the in membership expression, or a get with a default test (of these, in is generally preferred):

>>> D

{'b': 2, 'c': 3, 'a': 1}

>>> D.has_key('c') # 2.X only: True/False

AttributeError: 'dict' object has no attribute 'has_key'

>>> 'c' in D # Required in 3.X

True

>>> 'x' in D # Preferred in 2.X today

False

>>> if 'c' in D: print('present', D['c']) # Branch on result

...

present 3

>>> print(D.get('c')) # Fetch with default

3

>>> print(D.get('x'))

None

>>> if D.get('c') != None: print('present', D['c']) # Another option

...

present 3

To summarize, the dictionary story changes substantially in 3.X. If you work in 2.X and care about 3.X compatibility (or suspect that you might someday), here are some pointers. Of the 3.X changes we’ve met in this section:

§ The first (dictionary comprehensions) can be coded only in 3.X and 2.7.

§ The second (dictionary views) can be coded only in 3.X, and with special method names in 2.7.

However, the last three techniques—sorted, manual comparisons, and in—can be coded in 2.X today to ease 3.X migration in the future.

WHY YOU WILL CARE: DICTIONARY INTERFACES

Dictionaries aren’t just a convenient way to store information by key in your programs—some Python extensions also present interfaces that look like and work the same as dictionaries. For instance, Python’s interface to DBM access-by-key files looks much like a dictionary that must be opened. You store and fetch strings using key indexes:

import dbm # Named anydbm in Python 2.X

file = dbm.open("filename") # Link to file

file['key'] = 'data' # Store data by key

data = file['key'] # Fetch data by key

In Chapter 28, you’ll see that you can store entire Python objects this way, too, if you replace dbm in the preceding code with shelve (shelves are access-by-key databases that store persistent Python objects, not just strings). For Internet work, Python’s CGI script support also presents a dictionary-like interface. A call to cgi.FieldStorage yields a dictionary-like object with one entry per input field on the client’s web page:

import cgi

form = cgi.FieldStorage() # Parse form data

if 'name' in form:

showReply('Hello, ' + form['name'].value)

Though dictionaries are the only core mapping type, all of these others are instances of mappings, and support most of the same operations. Once you learn dictionary interfaces, you’ll find that they apply to a variety of built-in tools in Python.

For another dictionary use case, see also Chapter 9’s upcoming overview of JSON—a language-neutral data format used for databases and data transfer. Python dictionaries, lists, and nested combinations of them can almost pass for records in this format as is, and may be easily translated to and from formal JSON text strings with Python’s json standard library module.

Chapter Summary

In this chapter, we explored the list and dictionary types—probably the two most common, flexible, and powerful collection types you will see and use in Python code. We learned that the list type supports positionally ordered collections of arbitrary objects, and that it may be freely nested and grown and shrunk on demand. The dictionary type is similar, but it stores items by key instead of by position and does not maintain any reliable left-to-right order among its items. Both lists and dictionaries are mutable, and so support a variety of in-place change operations not available for strings: for example, lists can be grown by append calls, and dictionaries by assignment to new keys.

In the next chapter, we will wrap up our in-depth core object type tour by looking at tuples and files. After that, we’ll move on to statements that code the logic that processes our objects, taking us another step toward writing complete programs. Before we tackle those topics, though, here are some chapter quiz questions to review.

Test Your Knowledge: Quiz

1. Name two ways to build a list containing five integer zeros.

2. Name two ways to build a dictionary with two keys, 'a' and 'b', each having an associated value of 0.

3. Name four operations that change a list object in place.

4. Name four operations that change a dictionary object in place.

5. Why might you use a dictionary instead of a list?

Test Your Knowledge: Answers

1. A literal expression like [0, 0, 0, 0, 0] and a repetition expression like [0] * 5 will each create a list of five zeros. In practice, you might also build one up with a loop that starts with an empty list and appends 0 to it in each iteration, with L.append(0). A list comprehension ([0 for i in range(5)]) could work here, too, but this is more work than you need to do for this answer.

2. A literal expression such as {'a': 0, 'b': 0} or a series of assignments like D = {}, D['a'] = 0, and D['b'] = 0 would create the desired dictionary. You can also use the newer and simpler-to-code dict(a=0, b=0) keyword form, or the more flexible dict([('a', 0), ('b', 0)]) key/value sequences form. Or, because all the values are the same, you can use the special form dict.fromkeys('ab', 0). In 3.X and 2.7, you can also use a dictionary comprehension: {k:0 for k in 'ab'}, though again, this may be overkill here.

3. The append and extend methods grow a list in place, the sort and reverse methods order and reverse lists, the insert method inserts an item at an offset, the remove and pop methods delete from a list by value and by position, the del statement deletes an item or slice, and index and slice assignment statements replace an item or entire section. Pick any four of these for the quiz.

4. Dictionaries are primarily changed by assignment to a new or existing key, which creates or changes the key’s entry in the table. Also, the del statement deletes a key’s entry, the dictionary update method merges one dictionary into another in place, and D.pop(key) removes a key and returns the value it had. Dictionaries also have other, more exotic in-place change methods not presented in this chapter, such as setdefault; see reference sources for more details.

5. Dictionaries are generally better when the data is labeled (a record with field names, for example); lists are best suited to collections of unlabeled items (such as all the files in a directory). Dictionary lookup is also usually quicker than searching a list, though this might vary per program.



All materials on the site are licensed Creative Commons Attribution-Sharealike 3.0 Unported CC BY-SA 3.0 & GNU Free Documentation License (GFDL)

If you are the copyright holder of any material contained on our site and intend to remove it, please contact our site administrator for approval.

© 2016-2025 All site design rights belong to S.Y.A.