1
1
# Implementation Details
2
2
3
- ### Abstract Operations on Python Objects
3
+ ## Abstract Operations on Python Objects
4
4
5
5
Many generic operations on Python objects in CPython are defined in the header
6
6
files ` abstract.c ` and ` abstract.h ` . These operations are widely used and their
7
7
interplay and intricacies are the cause for the conversion, error message, and
8
8
control flow bugs when not mimicked correctly. Our current approach is to
9
9
provide many of these abstract operations as part of the
10
- ` PythonObjectLibrary ` . Usually, this means there are at least two messages for
11
- each operation - one that takes a ` ThreadState ` argument, and one that
12
- doesn't. The intent is to allow passing of exception state and caller
13
- information similar to how we do it with the ` PFrame ` argument even across
14
- library messages, which cannot take a VirtualFrame.
10
+ ` PythonObjectLibrary ` .
11
+
12
+ ### Common operations in the PythonObjectLibrary
13
+
14
+ The code has evolved over time, so not all built-in nodes are prime examples of
15
+ messages that should be used from the PythonObjectLibrary. We are refactoring
16
+ this as we go, but here are a few examples for things you can (or should soon be
17
+ able to) use the PythonObjectLibrary for:
18
+
19
+ - casting and coercion to ` java.lang.String ` , array-sized Java ` int ` , Python
20
+ index, fileno, ` double ` , filesystem path, iterator, and more
21
+ - reading the class of an object
22
+ - accessing the ` __dict__ ` attribute of an object
23
+ - hashing objects and testing for equality
24
+ - testing for truthy-ness
25
+ - getting the length
26
+ - testing for abstract types such as ` mapping ` , ` sequence ` , ` callable `
27
+ - invoking methods or executing callables
28
+ - access objects through the buffer protocol
29
+
30
+ ### PythonObjectLibrary functions with and without state
31
+
32
+ Usually, there are at least two messages for each operation - one that takes a
33
+ ` ThreadState ` argument, and one that doesn't. The intent is to allow passing of
34
+ exception state and caller information similar to how we do it with the ` PFrame `
35
+ argument even across library messages, which cannot take a VirtualFrame.
15
36
16
37
All nodes that are used in message implementations must allow uncached
17
38
usage. Often (e.g. in the case of the generic ` CallNode ` ) they offer execute
@@ -38,7 +59,14 @@ long messageWithState(ThreadState state,
38
59
avoids materialization of the frame state in more cases, as described on the
39
60
section on Python's global thread state above.
40
61
41
- ### Python Global Thread State
62
+ ### Other libraries in the codebase
63
+
64
+ Accessing hashing storages (the storage for ` dict ` , ` set ` , and ` frozenset ` )
65
+ should be done via the ` HashingStorageLibrary ` . We are in the process of
66
+ creating a ` SequenceStorageLibrary ` for sequence types (` tuple ` , ` list ` ) to
67
+ replace the ` SequenceStorageNodes ` collection of classes.
68
+
69
+ ## Python Global Thread State
42
70
43
71
In CPython, each stack frame is allocated on the heap, and there's a global
44
72
thread state holding on to the chain of currently handled exceptions (e.g. if
@@ -59,15 +87,15 @@ be forced to the heap.
59
87
In Graal Python, the implementation is thus a bit more involved. Here's how it
60
88
works.
61
89
62
- #### The PFrame.Reference
90
+ ### The PFrame.Reference
63
91
64
92
A ` PFrame.Reference ` is created when entering a Python function. By default it
65
93
only holds on to another reference, that of the Python caller. If there are
66
94
non-Python frames between the newly entered frame and the last Python frame,
67
95
those are ignored - our linked list only connects Python frames. The entry point
68
96
into the interpreter has a ` PFrame.Reference ` with no caller.
69
97
70
- ###### ExecutionContext.CallContext and ExecutionContext.CalleeContext
98
+ #### ExecutionContext.CallContext and ExecutionContext.CalleeContext
71
99
72
100
If we're only calling between Python, we pass our ` PFrame.Reference ` as implicit
73
101
argument to any callees. On entry, they will create their own ` PFrame.Reference `
@@ -98,7 +126,7 @@ ExecutionContext.CalleeContext classes. These also use profiling information to
98
126
eagerly fill in frame information if the callees actually access the stack, for
99
127
example, so that no further stack walks need to take place.
100
128
101
- ###### ExecutionContext.IndirectCallContext and ExecutionContext.IndirectCalleeContext
129
+ #### ExecutionContext.IndirectCallContext and ExecutionContext.IndirectCalleeContext
102
130
103
131
If we're mixing Python frames with non-Python frames, or if we are making calls
104
132
to methods and cannot pass the Truffle frame, we need to store the last
@@ -110,7 +138,7 @@ caller, it initially walks the stack to find it. But it will also tell the last
110
138
Python node that made a call to a "foreign" callee that it will have to store
111
139
its ` PFrame.Reference ` globally in the future for it to be available later.
112
140
113
- #### The current PException
141
+ ### The current PException
114
142
115
143
Now that we have a mechanism to lazily make available only as much frame state
116
144
as needed, we use the same mechanism to also pass the currently handled
0 commit comments