Skip to content

Commit 628a3a4

Browse files
committed
added stuff as suggested by Mark
1 parent dbee274 commit 628a3a4

File tree

1 file changed

+52
-8
lines changed

1 file changed

+52
-8
lines changed

InternalDocs/generators.md

Lines changed: 52 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -5,12 +5,13 @@ Generators and Coroutines
55
Generators
66
----------
77

8-
The implementation of generators in CPython consists of the builtin object type
9-
`PyGenObject` and bytecode instructions that operate on instances of this type.
8+
The implementation of generators in CPython consists of instances of `PyGenObject`
9+
and bytecode instructions that operate on instances of this type.
1010

11-
A generator object executes in its own [`frame`](frames.md), like a function.
12-
The difference is that a function returns only once, while a generator
13-
"returns" to the caller every time it emits a new item with a
11+
A generator object is invoked in a [`frame`](frames.md), like a function.
12+
The difference is that a function returns to the calling frame only once,
13+
while a generator "returns" to the caller every time it emits a new item
14+
with a
1415
[`yield` expression](https://docs.python.org/dev/reference/expressions.html#yield-expressions).
1516
This is implemented by the
1617
[`YIELD_VALUE`](https://docs.python.org/dev/library/dis.html#opcode-YIELD_VALUE)
@@ -23,22 +24,65 @@ instruction pointer and stores the interpreter's exception state on the generato
2324
object. When the generator is resumed, this exception state is copied back to the
2425
interpreter state.
2526

26-
The `frame` of a generator is embedded in the generator object struct (see
27-
`_PyGenObject_HEAD` in [`pycore_genobject.h`](../Include/internal/pycore_genobject.h)).
27+
The `frame` of a generator is embedded in the generator object struct as a
28+
[`_PyInterpreterFrame`](frames.md) (see `_PyGenObject_HEAD` in
29+
[`pycore_genobject.h`](../Include/internal/pycore_genobject.h)).
2830
This means that we can get the frame from the generator or the generator
2931
from the frame (see `_PyGen_GetGeneratorFromFrame` in the same file).
3032
Other fields of the generator struct include metadata (such as the name of
3133
the generator function) and runtime state information (such as whether its
3234
frame is executing, suspended, cleared, etc.).
3335

36+
Generator Object Creation and Destruction
37+
-----------------------------------------
38+
39+
The bytecode of a generator function begins with a
40+
[`RETURN_GENERATOR`](https://docs.python.org/dev/library/dis.html#opcode-RETURN_GENERATOR)
41+
instruction, which creates a generator object, along with its embedded frame.
42+
The generator's frame is initialized as a copy of the frame in which
43+
`RETURN_GENERATOR` is executing, but its `owner` field is overwritten to indicate
44+
that it is owned by a generator. Finally, `RETURN_GENERATOR` pushes the new generator
45+
object to the stack and returns to the caller of the generator function. When the
46+
generator is next resumed by [`gen_send_ex2()`](../Objects/genobject.c),
47+
`_PyEval_EvalFrame()` is called to continue executing the generator function,
48+
in the frame that is embedded in the generator object.
49+
50+
When a generator object is destructed in [`gen_dealloc`](../Objects/genobject.c),
51+
its embedded `_PyInterpreterFrame` field may need to be preserved, if it is exposed
52+
to Python as part of a [`PyFrameObject`](frames.md#frame-objects). This is detected
53+
in [`_PyFrame_ClearExceptCode`](../Python/frame.c) by the fact that the interpreter
54+
frame's `frame_obj` field is set, and the frame object it points to has refcount
55+
greater than 1. If so, the `take_ownership()` function is called to create a new
56+
copy of the interpreter frame and transfer ownership of it from the generator to
57+
the frame object.
58+
59+
Iteration
60+
---------
61+
62+
The [`FOR_ITER`](https://docs.python.org/dev/library/dis.html#opcode-FOR_ITER)
63+
instruction calls `__next__` on the iterator which is on the top of the stack,
64+
and pushes the result to the stack. It has [`specializations`](adaptive.md)
65+
for a few common iterator types, including `FOR_ITER_GEN`, for iterating over
66+
a generator.
67+
3468
Chained Generators
3569
------------------
3670

3771
A `yield from` expression creates a generator that efficiently yields the
3872
sequence created by another generator. This is implemented with the
3973
[`SEND` instruction](https://docs.python.org/dev/library/dis.html#opcode-SEND),
4074
which pushes the value of its arg to the stack of the generator's frame, sets
41-
the exception state on this frame, and resumes execution of the chained generator.
75+
the exception state on this frame, and resumes execution of the chained generator.
76+
On return from `SEND`, the value at the top of the stack is sent back up
77+
the generator chain with a `YIELD_VALUE`. This sequence of `SEND` followed by
78+
`YIELD_VALUE` is repeated in a loop, until a `StopIteration` exception is
79+
raised to indicate that the generator has no more values to emit.
80+
81+
The [`CLEANUP_THROW`](https://docs.python.org/dev/library/dis.html#opcode-CLEANUP_THROW)
82+
instruction is used to handle exceptions raised from the send-yield loop.
83+
Exceptions of type `StopIteration` is handled, their `value` field hold the
84+
value to be returned by the generator's `close()` function. Any other
85+
exception is re-raised by `CLEANUP_THROW`.
4286

4387
Coroutines
4488
----------

0 commit comments

Comments
 (0)