3 PEP 342: New Generator Features

As introduced in Python 2.3, generators only produce output; once a generator's code was invoked to create an iterator, there's no way to pass new parameters into the function when its execution is resumed. Hackish solutions to this include making the generator's code look at a global variable and then changing the global variable's value, or passing in some mutable object that callers then modify. Python 2.5 adds the ability to pass values into a generator.

To refresh your memory of basic generators, here's a simple example:

def counter (maximum):
    i = 0
    while i < maximum:
        yield i
	i += 1

When you call counter(10), the result is an iterator that returns the values from 0 up to 9. On encountering the yield statement, the iterator returns the provided value and suspends the function's execution, preserving the local variables. Execution resumes on the following call to the iterator's next() method, picking up after the yield.

In Python 2.3, yield was a statement; it didn't return any value. In 2.5, yield is now an expression, returning a value that can be assigned to a variable or otherwise operated on:

val = (yield i)

I recommend that you always put parentheses around a yield expression when you're doing something with the returned value, as in the above example. The parentheses aren't always necessary, but it's easier to always add them instead of having to remember when they're needed. The exact rules are that a yield-expression must always be parenthesized except when it occurs at the top-level expression on the right-hand side of an assignment, meaning you can to write val = yield i but val = (yield i) + 12.

Values are sent into a generator by calling its send(value) method. The generator's code is then resumed and the yield expression produces value. If the regular next() method is called, the yield returns None.

Here's the previous example, modified to allow changing the value of the internal counter.

def counter (maximum):
    i = 0
    while i < maximum:
        val = (yield i)
	# If value provided, change counter
        if val is not None:
            i = val
	else:
  	    i += 1

And here's an example of changing the counter:

>>> it = counter(10)
>>> print it.next()
0
>>> print it.next()
1
>>> print it.send(8)
8
>>> print it.next()
9
>>> print it.next()
Traceback (most recent call last):
  File ``t.py'', line 15, in ?
    print it.next()
StopIteration

Because yield will often be returning None, you shouldn't just use its value in expressions unless you're sure that only the send() method will be used.

There are two other new methods on generators in addition to send():

throw(type, value=None, traceback=None) is used to raise an exception inside the generator; the exception is raised by the yield expression where the generator's execution is paused.
close() raises a new GeneratorExit exception inside the generator to terminate the iteration. On receiving this exception, the generator's code must either raise GeneratorExit or StopIteration; catching the exception and doing anything else is illegal and will trigger a RuntimeError. close() will also be called by Python's garbage collection when the generator is garbage-collected.
If you need to run cleanup code in case of a GeneratorExit, I suggest using a try: ... finally: suite instead of catching GeneratorExit.

The cumulative effect of these changes is to turn generators from one-way producers of information into both producers and consumers. Generators also become coroutines, a more generalized form of subroutines; subroutines are entered at one point and exited at another point (the top of the function, and a return statement), but coroutines can be entered, exited, and resumed at many different points (the yield statements).science term