@@ -96,9 +96,9 @@ files to get an idea of how rules are set for patches to be applied.
9696
9797We always run with a GIL, because C extensions in CPython expect to do so and
9898are usually not written to be reentrant. The reason to always have the GIL
99- enabled is that when using Python, at least Sulong/LLVM is always available in
100- the same context and we cannot know if someone may be using that (or another
101- polyglot language or the Java host interop) to start additional threads that
99+ enabled is that when using Python, another polyglot language or the Java host
100+ interop can be available in the same context, and we cannot know if someone
101+ may be using that to start additional threads that
102102could call back into Python. This could legitimately happen in C extensions when
103103the C extension authors use knowledge of how CPython works to do something
104104GIL-less in a C thread that is fine to do on CPython's data structures, but not
@@ -165,3 +165,127 @@ For embedders, it may be important to be able to interrupt Python threads by
165165other means. We use the TruffleSafepoint mechanism to mark our threads waiting
166166to acquire the GIL as blocked for the purpose of safepoints. The Truffle
167167safepoint action mechanism can thus be used to kill threads waiting on the GIL.
168+
169+ ## C Extensions and Memory Management
170+
171+ ### High-level
172+
173+ C extensions assume reference counting, but on the managed side we want to leverage
174+ Java tracing GC. This creates a mismatch. The approach is to do both, reference
175+ counting and tracing GC, at the same time.
176+
177+ On the native side we use reference counting. The native code is responsible for doing
178+ the counting, i.e., calling the ` Py_IncRef ` and ` Py_DecRef ` API functions. Inside those
179+ functions we add special handling for the point when first reference from the native
180+ code is created and when the last reference from the native code is destroyed.
181+
182+ On the managed side we rely on tracing GC, so managed references are not ref-counted.
183+ For the ref-counting scheme on the native side, we approximate all the managed references
184+ as a single reference, i.e., we increment the refcount when object is referenced from managed
185+ code, and using a ` PhantomReference ` and reference queue we decrement the refcount when
186+ there are no longer any managed references (but we do not clean the object as long as
187+ ` refcount > 0 ` , because that means that there are still native references to it).
188+
189+ ### Details
190+
191+ There are two kinds of Python objects in GraalPy: managed and native.
192+
193+ #### Managed Objects
194+
195+ Managed objects are allocated in the interpreter. If there is no native code involved,
196+ we do not do anything special and let the Java GC handle them. When a managed object
197+ is passed to a native extension code:
198+
199+ * We wrap it in ` PythonObjectNativeWrapper ` . This is mostly in order to provide different
200+ interop protocol: we do not want to expose ` toNative ` and ` asPointer ` on Python objects.
201+
202+ * When NFI calls ` toNative ` /` asPointer ` we:
203+ * Allocate C memory that will represent the object on the native side (including the refcount field)
204+ * Add a mapping of that memory address to the ` PythonObjectNativeWrapper ` object to a hash map ` CApiTransitions.nativeLookup ` .
205+ * We initialize the refcount field to a constant ` MANAGED_REFCNT ` (larger number, because some
206+ extensions like to special case on some small refcount values)
207+ * Create ` PythonObjectReference ` : a weak reference to the ` PythonObjectNativeWrapper ` ,
208+ when this reference is enqueued (i.e., no managed references exist), we decrement the refcount by
209+ ` MANAGED_REFCNT ` and if the recount falls back to ` 0 ` , we deallocate the native memory of the object,
210+ otherwise we need to wait for the native code to eventually call ` Py_DecRef ` and make it ` 0 ` .
211+
212+ * When extension code wants to create a new reference, it will call ` Py_IncRef ` .
213+ In the C implementation of ` Py_IncRef ` we check if a managed object with
214+ ` refcount==MANAGED_REFCNT ` wants to increment its refcount. In such case, the native code is
215+ creating a first reference to the managed object, we must make sure to keep the object alive
216+ as long as there are some native references. We set a field ` PythonObjectReference.strongReference ` ,
217+ which will keep the ` PythonObjectNativeWrapper ` alive even when all other managed references die.
218+
219+ * When extension code is done with the object, it will call ` Py_DecRef ` .
220+ In the C implementation of ` Py_DecRef ` we check if a managed object with ` refcount == MANAGED_REFCNT+1 `
221+ wants to decrement its refcount to MANAGED_REFCNT, which means that there are no native references
222+ to that object anymore. In such case we clear the ` PythonObjectReference.strongReference ` field,
223+ and the memory management is then again left solely to the Java tracing GC.
224+
225+ #### Native Objects
226+
227+ Native objects allocated using ` PyObject_GC_New ` in the native code are backed by native memory
228+ and may never be passed to managed code (as a return value of extension function or as an argument
229+ to some C API call). If a native object is not made available to managed code, it is just reference
230+ counted as usual, where ` Py_DecRef ` call that reaches ` 0 ` will deallocate the object. If a native
231+ object is passed to managed code:
232+
233+ * We increment the refcount of the native object by ` MANAGED_REFCNT `
234+ * We create:
235+ * ` PythonAbstractNativeObject ` Java object to mirror it on the managed side
236+ * ` NativeObjectReference ` , a weak reference to the ` PythonAbstractNativeObject ` .
237+ * Add mapping: native object address => ` NativeObjectReference ` into hash map ` CApiTransitions.nativeLookup `
238+ * Next time we just fetch the existing wrapper and don't do any of this
239+ * When ` NativeObjectReference ` is enqueued, we decrement the refcount by ` MANAGED_REFCNT `
240+ * If the refcount falls to ` 0 ` , it means that there are no references to the object even from
241+ native code, and we can destroy it. If it does not fall to ` 0 ` , we just wait for the native
242+ code to eventually call ` Py_DecRef ` that makes it fall to ` 0 ` .
243+
244+ #### Weak References
245+
246+ TODO
247+
248+ ### Cycle GC
249+
250+ We leverage the CPython's GC module to detect cycles for objects that participate
251+ in the reference counting scheme (native objects or managed objects that got passed
252+ to native code).
253+ See: https://devguide.python.org/internals/garbage-collector/index.html .
254+
255+ There are two issues:
256+
257+ * Objects that are referenced from the managed code have ` refcount >= MANAGED_REFCNT ` and
258+ until Java GC runs we do not know if they are garbage or not.
259+ * We cannot traverse the managed objects: since we don't do refcounting on the managed
260+ side, we cannot traverse them and decrement refcounts to see if there is a cycle.
261+
262+ The high level solution is that when we see a "dead" cycle going through a managed object
263+ (i.e., cycle not referenced by any native object from the "outside" of the collected set),
264+ we fully replicate the object graphs (and the cycle) on the managed side (refcounts of native objects
265+ in the cycle, which were not referenced from managed yet, will get new ` NativeObjectReference `
266+ created and refcount incremented by ` MANAGED_REFCNT ` ). Managed objects already refer
267+ to the ` PythonAbstractNativeObject ` wrappers of the native objects (e.g., some Python container
268+ with managed storage), but we also make the native wrappers refer to whatever their referents
269+ are on the Java side (we use ` tp_traverse ` to find their referents).
270+
271+ Then we make the managed objects in the cycle only weakly referenced on the Java side.
272+ One can think about this as pushing the baseline reference count when the
273+ object is eligible for being GC'ed and thus freed. Normally when the object has
274+ ` refcount > MANAGED_REFCNT ` we keep it alive with a strong reference assuming that
275+ there are some native references to it. In this case, we know that all the native
276+ references to that object are part of potentially dead cycle, and we do not
277+ count them into this limit. Let us call this limit * weak to strong limit* .
278+
279+ After this, if the managed objects are garbage, eventually Java GC will collect them
280+ together with the whole cycle.
281+
282+ If some of the managed objects are not garbage, and they passed back to native code,
283+ the native code can then access and resurrect the whole cycle. W.r.t. the refcounts
284+ integrity this is fine, because we did not alter the refcounts. The native references
285+ between the objects are still factored in their refcounts. What may seem like a problem
286+ is that we pushed the * weak to strong limit* for some objects. Such an object may be
287+ passed to native, get ` Py_IncRef ` 'ed making it strong reference again. Since ` Py_DecRef ` is
288+ checking the same ` MANAGED_REFCNT ` limit for all objects, the subsequent ` Py_DecRef `
289+ call for this object will not detect that the reference should be made weak again!
290+ However, this is OK, it only prolongs the collection: we will make it weak again in
291+ the next run of the cycle GC on the native side.
0 commit comments