Skip to content

Commit 269c596

Browse files
committed
Move details about C extension copying to IMPLEMENTATION_DETAILS
1 parent 11986fc commit 269c596

File tree

2 files changed

+14
-10
lines changed

2 files changed

+14
-10
lines changed

docs/contributor/IMPLEMENTATION_DETAILS.md

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -302,3 +302,16 @@ checking the same `MANAGED_REFCNT` limit for all objects, the subsequent `Py_Dec
302302
call for this object will not detect that the reference should be made weak again!
303303
However, this is OK, it only prolongs the collection: we will make it weak again in
304304
the next run of the cycle GC on the native side.
305+
306+
## C extension copying
307+
308+
On Linux, Python native extensions expect to lookup Python C API functions in the global namespace and specify no explicit dependency on any libpython.
309+
To isolate them, we copy them with a new name, change their `SONAME`, add a `DT_NEEDED` dependency on a copy of our libpython shared object, and finally load them with `RTLD_LOCAL`.
310+
311+
On Windows there is no global namespace so native extensions already have a dependency on our libpython DLL.
312+
We copy them and just change the dependency to point to the context-local copy of libpython rather than the global one.
313+
314+
On macOS, while two-level namespaces exist, Python extensions historically use `-undefined dynamic_lookup` where they (just like in Linux) expect to find C API functions in any loaded image.
315+
We have to apply a similar workaround as on Linux, copy to a new name, change the `LC_ID_DYLIB` to that name, and add a `LC_LOAD_DYLIB` section to make the linker load the symbols from our libpython.
316+
317+
Note that any code signatures are invalidated by this process.

docs/user/Native-Extensions.md

Lines changed: 1 addition & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -53,13 +53,4 @@ To support creating multiple GraalPy contexts that access native modules within
5353
The current strategy for this is to copy the libraries and modify them such that the dynamic library loader of the operating system will isolate them for us.
5454
To do this, all GraalPy contexts in the same process (not just those in the same engine!) must set the `python.IsolateNativeModules` option to `true`.
5555

56-
On Linux, Python native extensions expect to lookup Python C API functions in the global namespace and specify no explicit dependency on any libpython.
57-
To isolate them, we copy them with a new name, change their `SONAME`, add a `DT_NEEDED` dependency on a copy of our libpython shared object, and finally load them with `RTLD_LOCAL`.
58-
59-
On Windows there is no global namespace so native extensions already have a dependency on our libpython DLL.
60-
We copy them and just change the dependency to point to the context-local copy of libpython rather than the global one.
61-
62-
On macOS, while two-level namespaces exist, Python extensions historically use `-undefined dynamic_lookup` where they (just like in Linux) expect to find C API functions in any loaded image.
63-
We have to apply a similar workaround as on Linux, copy to a new name, change the `LC_ID_DYLIB` to that name, and add a `LC_LOAD_DYLIB` section to make the linker load the symbols from our libpython.
64-
65-
Note that any code signatures are invalidated by this process.
56+
For more details on this, see [our implementation details](https://github.com/oracle/graalpython/blob/master/docs/contributor/IMPLEMENTATION_DETAILS.md#c-extension-copying).

0 commit comments

Comments
 (0)