Python 2.5 adds a simple way to pass values into a generator. As introduced in Python 2.3, generators only produce output; once a generator's code was invoked to create an iterator, there was no way to pass any new information into the function when its execution is resumed. Sometimes the ability to pass in some information would be useful. Hackish solutions to this include making the generator's code look at a global variable and then changing the global variable's value, or passing in some mutable object that callers then modify.
To refresh your memory of basic generators, here's a simple example:
def counter (maximum): i = 0 while i < maximum: yield i i += 1
When you call counter(10)
, the result is an iterator that
returns the values from 0 up to 9. On encountering the
yield statement, the iterator returns the provided value and
suspends the function's execution, preserving the local variables.
Execution resumes on the following call to the iterator's
next() method, picking up after the yield statement.
In Python 2.3, yield was a statement; it didn't return any value. In 2.5, yield is now an expression, returning a value that can be assigned to a variable or otherwise operated on:
val = (yield i)
I recommend that you always put parentheses around a yield expression when you're doing something with the returned value, as in the above example. The parentheses aren't always necessary, but it's easier to always add them instead of having to remember when they're needed.
(PEP 342 explains the exact rules, which are that a
yield-expression must always be parenthesized except when it
occurs at the top-level expression on the right-hand side of an
assignment. This means you can write val = yield i
but have to
use parentheses when there's an operation, as in val = (yield i)
+ 12
.)
Values are sent into a generator by calling its send(value) method. The generator's code is then resumed and the yield expression returns the specified value. If the regular next() method is called, the yield returns None.
Here's the previous example, modified to allow changing the value of the internal counter.
def counter (maximum): i = 0 while i < maximum: val = (yield i) # If value provided, change counter if val is not None: i = val else: i += 1
And here's an example of changing the counter:
>>> it = counter(10) >>> print it.next() 0 >>> print it.next() 1 >>> print it.send(8) 8 >>> print it.next() 9 >>> print it.next() Traceback (most recent call last): File ``t.py'', line 15, in ? print it.next() StopIteration
Because yield will often be returning None, you should always check for this case. Don't just use its value in expressions unless you're sure that the send() method will be the only method used resume your generator function.
In addition to send(), there are two other new methods on generators:
If you need to run cleanup code when a GeneratorExit occurs,
I suggest using a try: ... finally:
suite instead of
catching GeneratorExit.
The cumulative effect of these changes is to turn generators from one-way producers of information into both producers and consumers.
Generators also become coroutines, a more generalized form of subroutines. Subroutines are entered at one point and exited at another point (the top of the function, and a return statement), but coroutines can be entered, exited, and resumed at many different points (the yield statements). We'll have to figure out patterns for using coroutines effectively in Python.
The addition of the close() method has one side effect that
isn't obvious. close() is called when a generator is
garbage-collected, so this means the generator's code gets one last
chance to run before the generator is destroyed. This last chance
means that try...finally
statements in generators can now be
guaranteed to work; the finally clause will now always get a
chance to run. The syntactic restriction that you couldn't mix
yield statements with a try...finally
suite has
therefore been removed. This seems like a minor bit of language
trivia, but using generators and try...finally
is actually
necessary in order to implement the with statement
described by PEP 343. I'll look at this new statement in the following
section.
Another even more esoteric effect of this change: previously, the
gi_frame attribute of a generator was always a frame object.
It's now possible for gi_frame to be None
once the generator has been exhausted.
See Also:
Earlier versions of these features were proposed in PEP 288 by Raymond Hettinger and PEP 325 by Samuele Pedroni.
See About this document... for information on suggesting changes.