PEP 797: Shared Object Proxies for Subinterpreters #4536
Conversation
| Abstract | ||
| ======== | ||
|
|
||
| This PEP introduces a new :func:`~concurrent.interpreters.share` function to |
There was a problem hiding this comment.
| This PEP introduces a new :func:`~concurrent.interpreters.share` function to | |
| This PEP introduces a new :func:`!share` function to |
|
|
||
| This PEP introduces a new :func:`~concurrent.interpreters.share` function to | ||
| the :mod:`concurrent.interpreters` module, which allows *any* arbitrary object | ||
| to be shared for a period of time. |
There was a problem hiding this comment.
Shared between what? Processes, threads, interpreters, over the network, etc?
|
|
||
| from concurrent import interpreters | ||
|
|
||
| with open("spanish_inquisition.txt") as unshareable: |
There was a problem hiding this comment.
always use an encoding!
| with open("spanish_inquisition.txt") as unshareable: | |
| with open('spanish_inquisition.txt', encoding='utf-8') as unshareable: |
| create multiple interpreters in a single Python process. This works well for | ||
| stateless code (that is, code that doesn't need any state from a caller) and |
There was a problem hiding this comment.
| create multiple interpreters in a single Python process. This works well for | |
| stateless code (that is, code that doesn't need any state from a caller) and | |
| create multiple interpreters in a single Python process. This works well for | |
| code that doesn't need any state from a caller ('stateless code') and |
| create multiple interpreters in a single Python process. This works well for | ||
| stateless code (that is, code that doesn't need any state from a caller) and | ||
| objects that can be serialized, but it is fairly common for applications to | ||
| want to use highly-complex data structures (that cannot be serialized) with |
There was a problem hiding this comment.
In general I would use fewer paranthetical asides, though this is style. Here, I don't think the comment about serialisation adds much, it's clear from context we're talking about non-serialisable data (or even data that could be serialised but that is too expensive to do so)
| want to use highly-complex data structures (that cannot be serialized) with | |
| require highly-complex data structures with |
| In order to implement the immortality mechanism used by shared object proxies, | ||
| several assumptions had to be made about the object lifecycle in the C API. | ||
| So, some best practices in the C API (such as using the object allocator for | ||
| objects) are made harder requirements by the implementation of this PEP. |
There was a problem hiding this comment.
Be explicit about what those assumptions are. In general, it is hard to make assumptions about what code in the wild is or is not doing (Hyrum's Law etc)
| The author of this PEP believes it is unlikely that this will cause breakage, | ||
| as he has not ever seen code in the wild that violates the assumptions made | ||
| about the object lifecycle as required by the reference implementation. |
There was a problem hiding this comment.
There are millions of lines of proprietary code that you & I haven't seen ;-)
More generally just re-emphasising the point on explicitly enumerating the changes in semantics that you propose.
| which prevents any concurrent modification to their reference count. | ||
| This can cause them to take up very large amounts of memory if mismanaged. | ||
|
|
||
| The :func:`~concurrent.interpreters.share` context manager does its best |
There was a problem hiding this comment.
| The :func:`~concurrent.interpreters.share` context manager does its best | |
| The :func:`!share` context manager does its best |
| The largest issue with shared object proxies is that in order to have | ||
| thread-safe reference counting operations, they must be :term:`immortal`, | ||
| which prevents any concurrent modification to their reference count. | ||
| This can cause them to take up very large amounts of memory if mismanaged. |
There was a problem hiding this comment.
Large amounts of memory -> denial of service attack, which is the actual security concern here
| usage (due to :term:`immortality <immortal>`, which will be discussed more | ||
| later). As such, this PEP does not make other mechanisms for sharing objects |
There was a problem hiding this comment.
Which part/section are you referring to here?
Basic requirements (all PEP Types)
pep-NNNN.rst), PR title (PEP 123: <Title of PEP>) andPEPheaderAuthororSponsor, and formally confirmed their approvalAuthor,Status(Draft),TypeandCreatedheaders filled out correctlyPEP-Delegate,Topic,RequiresandReplacesheaders completed if appropriate.github/CODEOWNERSfor the PEPStandards Track requirements
Python-Versionset to valid (pre-beta) future Python version, if relevantDiscussions-ToandPost-History📚 Documentation preview 📚: https://pep-previews--4536.org.readthedocs.build/pep-0797/