@@ -5,12 +5,13 @@ Generators and Coroutines
55Generators
66---------- 
77
8- The implementation of generators in CPython consists of the builtin object type 
9- ` PyGenObject `   and bytecode instructions that operate on instances of this type.
8+ The implementation of generators in CPython consists of instances of  ` PyGenObject ` 
9+ and bytecode instructions that operate on instances of this type.
1010
11- A generator object executes in its own [ ` frame ` ] ( frames.md ) , like a function.
12- The difference is that a function returns only once, while a generator
13- "returns" to the caller every time it emits a new item with a
11+ A generator object is invoked in a [ ` frame ` ] ( frames.md ) , like a function.
12+ The difference is that a function returns to the calling frame only once,
13+ while a generator "returns" to the caller every time it emits a new item
14+ with a
1415[ ` yield `  expression] ( https://docs.python.org/dev/reference/expressions.html#yield-expressions ) .
1516This is implemented by the
1617[ ` YIELD_VALUE ` ] ( https://docs.python.org/dev/library/dis.html#opcode-YIELD_VALUE ) 
@@ -23,22 +24,65 @@ instruction pointer and stores the interpreter's exception state on the generato
2324object. When the generator is resumed, this exception state is copied back to the
2425interpreter state.
2526
26- The ` frame `  of a generator is embedded in the generator object struct (see
27- ` _PyGenObject_HEAD `  in [ ` pycore_genobject.h ` ] ( ../Include/internal/pycore_genobject.h ) ).
27+ The ` frame `  of a generator is embedded in the generator object struct as a
28+ [ ` _PyInterpreterFrame ` ] ( frames.md )  (see ` _PyGenObject_HEAD `  in
29+ [ ` pycore_genobject.h ` ] ( ../Include/internal/pycore_genobject.h ) ).
2830This means that we can get the frame from the generator or the generator
2931from the frame (see ` _PyGen_GetGeneratorFromFrame `  in the same file).
3032Other fields of the generator struct include metadata (such as the name of
3133the generator function) and runtime state information (such as whether its
3234frame is executing, suspended, cleared, etc.).
3335
36+ Generator Object Creation and Destruction
37+ ----------------------------------------- 
38+ 
39+ The bytecode of a generator function begins with a
40+ [ ` RETURN_GENERATOR ` ] ( https://docs.python.org/dev/library/dis.html#opcode-RETURN_GENERATOR ) 
41+ instruction, which creates a generator object, along with its embedded frame.
42+ The generator's frame is initialized as a copy of the frame in which
43+ ` RETURN_GENERATOR `  is executing, but its ` owner `  field is overwritten to indicate
44+ that it is owned by a generator. Finally, ` RETURN_GENERATOR `  pushes the new generator
45+ object to the stack and returns to the caller of the generator function. When the
46+ generator is next resumed by [ ` gen_send_ex2() ` ] ( ../Objects/genobject.c ) ,
47+ ` _PyEval_EvalFrame() `  is called to continue executing the generator function,
48+ in the frame that is embedded in the generator object.
49+ 
50+ When a generator object is destructed in [ ` gen_dealloc ` ] ( ../Objects/genobject.c ) ,
51+ its embedded ` _PyInterpreterFrame `  field may need to be preserved, if it is exposed
52+ to Python as part of a [ ` PyFrameObject ` ] ( frames.md#frame-objects ) . This is detected
53+ in [ ` _PyFrame_ClearExceptCode ` ] ( ../Python/frame.c )  by the fact that the interpreter
54+ frame's ` frame_obj `  field is set, and the frame object it points to has refcount
55+ greater than 1. If so, the ` take_ownership() `  function is called to create a new
56+ copy of the interpreter frame and transfer ownership of it from the generator to
57+ the frame object.
58+ 
59+ Iteration
60+ --------- 
61+ 
62+ The [ ` FOR_ITER ` ] ( https://docs.python.org/dev/library/dis.html#opcode-FOR_ITER ) 
63+ instruction calls ` __next__ `  on the iterator which is on the top of the stack,
64+ and pushes the result to the stack. It has [ ` specializations ` ] ( adaptive.md ) 
65+ for a few common iterator types, including ` FOR_ITER_GEN ` , for iterating over
66+ a generator.
67+ 
3468Chained Generators
3569------------------ 
3670
3771A ` yield from `  expression creates a generator that efficiently yields the
3872sequence created by another generator. This is implemented with the
3973[ ` SEND `  instruction] ( https://docs.python.org/dev/library/dis.html#opcode-SEND ) ,
4074which pushes the value of its arg to the stack of the generator's frame, sets
41- the exception state on this frame, and resumes execution of the chained generator. 
75+ the exception state on this frame, and resumes execution of the chained generator.
76+ On return from ` SEND ` , the value at the top of the stack is sent back up
77+ the generator chain with a ` YIELD_VALUE ` . This sequence of ` SEND `  followed by
78+ ` YIELD_VALUE `  is repeated in a loop, until a ` StopIteration `  exception is
79+ raised to indicate that the generator has no more values to emit.
80+ 
81+ The [ ` CLEANUP_THROW ` ] ( https://docs.python.org/dev/library/dis.html#opcode-CLEANUP_THROW ) 
82+ instruction is used to handle exceptions raised from the send-yield loop.
83+ Exceptions of type ` StopIteration `  is handled, their ` value `  field hold the
84+ value to be returned by the generator's ` close() `  function. Any other
85+ exception is re-raised by ` CLEANUP_THROW ` .
4286
4387Coroutines
4488---------- 
0 commit comments