Skip to content

Commit eb383a9

Browse files
committed
update impl details a bit w.r.t. to what the PythonObjectLibrary is
1 parent f9def19 commit eb383a9

File tree

1 file changed

+39
-11
lines changed

1 file changed

+39
-11
lines changed

docs/contributor/IMPLEMENTATION_DETAILS.md

Lines changed: 39 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1,17 +1,38 @@
11
# Implementation Details
22

3-
### Abstract Operations on Python Objects
3+
## Abstract Operations on Python Objects
44

55
Many generic operations on Python objects in CPython are defined in the header
66
files `abstract.c` and `abstract.h`. These operations are widely used and their
77
interplay and intricacies are the cause for the conversion, error message, and
88
control flow bugs when not mimicked correctly. Our current approach is to
99
provide many of these abstract operations as part of the
10-
`PythonObjectLibrary`. Usually, this means there are at least two messages for
11-
each operation - one that takes a `ThreadState` argument, and one that
12-
doesn't. The intent is to allow passing of exception state and caller
13-
information similar to how we do it with the `PFrame` argument even across
14-
library messages, which cannot take a VirtualFrame.
10+
`PythonObjectLibrary`.
11+
12+
### Common operations in the PythonObjectLibrary
13+
14+
The code has evolved over time, so not all built-in nodes are prime examples of
15+
messages that should be used from the PythonObjectLibrary. We are refactoring
16+
this as we go, but here are a few examples for things you can (or should soon be
17+
able to) use the PythonObjectLibrary for:
18+
19+
- casting and coercion to `java.lang.String`, array-sized Java `int`, Python
20+
index, fileno, `double`, filesystem path, iterator, and more
21+
- reading the class of an object
22+
- accessing the `__dict__` attribute of an object
23+
- hashing objects and testing for equality
24+
- testing for truthy-ness
25+
- getting the length
26+
- testing for abstract types such as `mapping`, `sequence`, `callable`
27+
- invoking methods or executing callables
28+
- access objects through the buffer protocol
29+
30+
### PythonObjectLibrary functions with and without state
31+
32+
Usually, there are at least two messages for each operation - one that takes a
33+
`ThreadState` argument, and one that doesn't. The intent is to allow passing of
34+
exception state and caller information similar to how we do it with the `PFrame`
35+
argument even across library messages, which cannot take a VirtualFrame.
1536

1637
All nodes that are used in message implementations must allow uncached
1738
usage. Often (e.g. in the case of the generic `CallNode`) they offer execute
@@ -38,7 +59,14 @@ long messageWithState(ThreadState state,
3859
avoids materialization of the frame state in more cases, as described on the
3960
section on Python's global thread state above.
4061

41-
### Python Global Thread State
62+
### Other libraries in the codebase
63+
64+
Accessing hashing storages (the storage for `dict`, `set`, and `frozenset`)
65+
should be done via the `HashingStorageLibrary`. We are in the process of
66+
creating a `SequenceStorageLibrary` for sequence types (`tuple`, `list`) to
67+
replace the `SequenceStorageNodes` collection of classes.
68+
69+
## Python Global Thread State
4270

4371
In CPython, each stack frame is allocated on the heap, and there's a global
4472
thread state holding on to the chain of currently handled exceptions (e.g. if
@@ -59,15 +87,15 @@ be forced to the heap.
5987
In Graal Python, the implementation is thus a bit more involved. Here's how it
6088
works.
6189

62-
#### The PFrame.Reference
90+
### The PFrame.Reference
6391

6492
A `PFrame.Reference` is created when entering a Python function. By default it
6593
only holds on to another reference, that of the Python caller. If there are
6694
non-Python frames between the newly entered frame and the last Python frame,
6795
those are ignored - our linked list only connects Python frames. The entry point
6896
into the interpreter has a `PFrame.Reference` with no caller.
6997

70-
###### ExecutionContext.CallContext and ExecutionContext.CalleeContext
98+
#### ExecutionContext.CallContext and ExecutionContext.CalleeContext
7199

72100
If we're only calling between Python, we pass our `PFrame.Reference` as implicit
73101
argument to any callees. On entry, they will create their own `PFrame.Reference`
@@ -98,7 +126,7 @@ ExecutionContext.CalleeContext classes. These also use profiling information to
98126
eagerly fill in frame information if the callees actually access the stack, for
99127
example, so that no further stack walks need to take place.
100128

101-
###### ExecutionContext.IndirectCallContext and ExecutionContext.IndirectCalleeContext
129+
#### ExecutionContext.IndirectCallContext and ExecutionContext.IndirectCalleeContext
102130

103131
If we're mixing Python frames with non-Python frames, or if we are making calls
104132
to methods and cannot pass the Truffle frame, we need to store the last
@@ -110,7 +138,7 @@ caller, it initially walks the stack to find it. But it will also tell the last
110138
Python node that made a call to a "foreign" callee that it will have to store
111139
its `PFrame.Reference` globally in the future for it to be available later.
112140

113-
#### The current PException
141+
### The current PException
114142

115143
Now that we have a mechanism to lazily make available only as much frame state
116144
as needed, we use the same mechanism to also pass the currently handled

0 commit comments

Comments
 (0)