PEP 584 – Add Union Operators To dict | peps.python.org
PEP 584 – Add Union Operators To dict PEP 584 – Add Union Operators To dict Author : Steven D’Aprano <steve at pearwood.info>, Brandt Bucher <brandt at python.org> BDFL-Delegate : Guido van Rossum <guido at python.org> Status : Final Type : Standards Track Created : 01-Mar-2019 Python-Version : 3.9 Post-History : 01-Mar-2019, 16-Oct-2019, 02-Dec-2019, 04-Feb-2020, 17-Feb-2020 Resolution : Python-Dev thread Table of Contents Abstract Motivation dict.update {**d1, **d2} collections.ChainMap dict(d1, **d2) Rationale Specification Reference Implementation Major Objections Dict Union Is Not Commutative Response Dict Union Will Be Inefficient Response Dict Union Is Lossy Response Only One Way To Do It Response More Than One Way To Do It Response Dict Union Makes Code Harder To Understand Response What About The Full set API?
Response What About Mapping And MutableMapping ? Response Rejected Ideas Rejected Semantics Raise Add The Values (As Counter Does, with + ) Leftmost Value (First-Seen) Wins Concatenate Values In A List Rejected Alternatives Use The Addition Operator Use The Left Shift Operator Use A New Left Arrow Operator Use A Method Advantages Disadvantages Use a Function Advantages Disadvantages Examples IPython/zmq/ipkernel.py IPython/zmq/kernelapp.py matplotlib/backends/backend_svg.py matplotlib/delaunay/triangulate.py matplotlib/legend.py numpy/ma/core.py praw/internal.py pygments/lexer.py requests/sessions.py sphinx/domains/__init__.py sphinx/ext/doctest.py sphinx/ext/inheritance_diagram.py sphinx/highlighting.py sphinx/quickstart.py sympy/abc.py sympy/parsing/maxima.py sympy/printing/ccode.py and sympy/printing/fcode.py sympy/utilities/runtests.py Related Discussions Copyright Abstract This PEP proposes adding merge ( | ) and update ( |= ) operators to the built-in dict class.
Note After this PEP was accepted, the decision was made to also implement the new operators for several other standard library mappings . Motivation The current ways to merge two dicts have several disadvantages: dict.update d1.update(d2) modifies d1 in-place. e = d1.copy(); e.update(d2) is not an expression and needs a temporary variable. {**d1, **d2} Dict unpacking looks ugly and is not easily discoverable. Few people would be able to guess what it means the first time they see it, or think of it as the “obvious way” to merge two dicts.
As Guido said : I’m sorry for PEP 448, but even if you know about **d in simpler contexts, if you were to ask a typical Python user how to combine two dicts into a new one, I doubt many people would think of {**d1, **d2} . I know I myself had forgotten about it when this thread started! {**d1, **d2} ignores the types of the mappings and always returns a dict . type(d1)({**d1, **d2}) fails for dict subclasses such as defaultdict that have an incompatible __init__ method. collections.ChainMap ChainMap is unfortunately poorly-known and doesn’t qualify as “obvious”.
It also resolves duplicate keys in the opposite order to that expected (“first seen wins” instead of “last seen wins”). Like dict unpacking, it is tricky to get it to honor the desired subclass. For the same reason, type(d1)(ChainMap(d2, d1)) fails for some subclasses of dict. Further, ChainMaps wrap their underlying dicts, so writes to the ChainMap will modify the original dict: >>> d1 = { 'spam' : 1 } >>> d2 = { 'eggs' : 2 } >>> merged = ChainMap ( d2 , d1 ) >>> merged [ 'eggs' ] = 999 >>> d2 {'eggs': 999} dict(d1, **d2) This “neat trick” is not well-known, and only works when d2 is entirely string-keyed: >>> d1 = { "spam" : 1 } >>> d2 = { 3665 : 2 } >>> dict ( d1 , ** d2 ) Traceback (most recent call last): ...
TypeError : keywords must be strings Rationale The new operators will have the same relationship to the dict.update method as the list concatenate ( + ) and extend ( += ) operators have to list.extend . Note that this is somewhat different from the relationship that | / |= have with set.update ; the authors have determined that allowing the in-place operator to accept a wider range of types (as list does) is a more useful design, and that restricting the types of the binary operator’s operands (again, as list does) will help avoid silent errors caused by complicated implicit type casting on both sides.
Key conflicts will be resolved by keeping the rightmost value. This matches the existing behavior of similar dict operations, where the last seen value always wins: { 'a' : 1 , 'a' : 2 } { ** d , ** e } d . update ( e ) d [ k ] = v { k : v for x in ( d , e ) for ( k , v ) in x . items ()} All of the above follow the same rule. This PEP takes the position that this behavior is simple, obvious, usually the behavior we want, and should be the default behavior for dicts.
This means that dict union is not commutative; in general d | e != e | d . Similarly, the iteration order of the key-value pairs in the dictionary will follow the same semantics as the examples above, with each newly added key (and its value) being appended to the current sequence. Specification Dict union will return a new dict consisting of the left operand merged with the right operand, each of which must be a dict (or an instance of a dict subclass).
If a key appears in both operands, the last-seen value (i.e. that from the right-hand operand) wins: >>> d = { 'spam' : 1 , 'eggs' : 2 , 'cheese' : 3 } >>> e = { 'cheese' : 'cheddar' , 'aardvark' : 'Ethel' } >>> d | e {'spam': 1, 'eggs': 2, 'cheese': 'cheddar', 'aardvark': 'Ethel'} >>> e | d {'cheese': 3, 'aardvark': 'Ethel', 'spam': 1, 'eggs': 2} The augmented assignment version operates in-place: >>> d |= e >>> d {'spam': 1, 'eggs': 2, 'cheese': 'cheddar', 'aardvark': 'Ethel'} Augmented assignment behaves identically to the update method called with a single positional argument, so it also accepts anything implementing the Mapping protocol (more specifically, anything with the keys and __getitem__ methods) or iterables of key-value pairs.
This is analogous to list += and list.extend , which accept any iterable, not just lists. Continued from above: >>> d | [( 'spam' , 999 )] Traceback (most recent call last): ... TypeError : can only merge dict (not "list") to dict >>> d |= [( 'spam' , 999 )] >>> d {'spam': 999, 'eggs': 2, 'cheese': 'cheddar', 'aardvark': 'Ethel'} When new keys are added, their order matches their order within the right-hand mapping, if any exists for its type.
Reference Implementation One of the authors has written a C implementation . An approximate pure-Python implementation is: def __or__ ( self , other ): if not isinstance ( other , dict ): return NotImplemented new = dict ( self ) new . update ( other ) return new def __ror__ ( self , other ): if not isinstance ( other , dict ): return NotImplemented new = dict ( other ) new . update ( self ) return new def __ior__ ( self , other ): dict . update ( self , other ) return self Major Objections Dict Union Is Not Commutative Union is commutative, but dict union will not be ( d | e != e | d ).
Response There is precedent for non-commutative unions in Python: >>> { 0 } | { False } {0} >>> { False } | { 0 } {False} While the results may be equal, they are distinctly different. In general, a | b is not the same operation as b | a . Dict Union Will Be Inefficient Giving a pipe operator to mappings is an invitation to writing code that doesn’t scale well. Repeated dict union is inefficient: d | e | f | g | h creates and destroys three temporary mappings.
Response The same argument applies to sequence concatenation. Sequence concatenation grows with the total number of items in the sequences, leading to O(N**2) (quadratic) performance. Dict union is likely to involve duplicate keys, so the temporary mappings will not grow as fast. Just as it is rare for people to concatenate large numbers of lists or tuples, the authors of this PEP believe that it will be rare for people to merge large numbers of dicts. collections.Counter is a dict subclass that supports man
The rest of this page is on the live site.