13.1.5.2 Pickling and unpickling extension types

When the Pickler encounters an object of a type it knows nothing about -- such as an extension type -- it looks in two places for a hint of how to pickle it. One alternative is for the object to implement a __reduce__() method. If provided, at pickling time __reduce__() will be called with no arguments, and it must return either a string or a tuple.

If a string is returned, it names a global variable whose contents are pickled as normal. The string returned by __reduce__ should be the object's local name relative to its module; the pickle module searches the module namespace to determine the object's module.

When a tuple is returned, it must be between two and five elements long. Optional elements can either be omitted, or None can be provided as their value. The semantics of each element are:

• A callable object that will be called to create the initial version of the object. The next element of the tuple will provide arguments for this callable, and later elements provide additional state information that will subsequently be used to fully reconstruct the pickled date.

  In the unpickling environment this object must be either a class, a callable registered as a ``safe constructor'' (see below), or it must have an attribute __safe_for_unpickling__ with a true value. Otherwise, an UnpicklingError will be raised in the unpickling environment. Note that as usual, the callable itself is pickled by name.

• A tuple of arguments for the callable object. Changed in version 2.5: Formerly, this argument could also be None.

• Optionally, the object's state, which will be passed to the object's __setstate__() method as described in section 13.1.5. If the object has no __setstate__() method, then, as above, the value must be a dictionary and it will be added to the object's __dict__.

• Optionally, an iterator (and not a sequence) yielding successive list items. These list items will be pickled, and appended to the object using either obj.append(item) or obj.extend(list_of_items). This is primarily used for list subclasses, but may be used by other classes as long as they have append() and extend() methods with the appropriate signature. (Whether append() or extend() is used depends on which pickle protocol version is used as well as the number of items to append, so both must be supported.)

• Optionally, an iterator (not a sequence) yielding successive dictionary items, which should be tuples of the form (key, value). These items will be pickled and stored to the object using obj[key] = value. This is primarily used for dictionary subclasses, but may be used by other classes as long as they implement __setitem__.

It is sometimes useful to know the protocol version when implementing __reduce__. This can be done by implementing a method named __reduce_ex__ instead of __reduce__. __reduce_ex__, when it exists, is called in preference over __reduce__ (you may still provide __reduce__ for backwards compatibility). The __reduce_ex__ method will be called with a single integer argument, the protocol version.

The object class implements both __reduce__ and __reduce_ex__; however, if a subclass overrides __reduce__ but not __reduce_ex__, the __reduce_ex__ implementation detects this and calls __reduce__.

An alternative to implementing a __reduce__() method on the object to be pickled, is to register the callable with the copy_reg module. This module provides a way for programs to register ``reduction functions'' and constructors for user-defined types. Reduction functions have the same semantics and interface as the __reduce__() method described above, except that they are called with a single argument, the object to be pickled.

The registered constructor is deemed a ``safe constructor'' for purposes of unpickling as described above.

See About this document... for information on suggesting changes.