Capsule API usage docs are incompatible with module reloading (etc)

[ncoghlan]

BPO	32459
Nosy	@brettcannon, @ncoghlan, @encukou, @ericsnowcurrently, @Dormouse759

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2017-12-31.05:27:01.669>
labels = ['type-bug']
title = 'Capsule API usage docs are incompatible with module reloading (etc)'
updated_at = <Date 2018-01-09.10:25:18.773>
user = 'https://github.com/ncoghlan'

bugs.python.org fields:

activity = <Date 2018-01-09.10:25:18.773>
actor = 'petr.viktorin'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = []
creation = <Date 2017-12-31.05:27:01.669>
creator = 'ncoghlan'
dependencies = []
files = []
hgrepos = []
issue_num = 32459
keywords = []
message_count = 1.0
messages = ['309266']
nosy_count = 5.0
nosy_names = ['brett.cannon', 'ncoghlan', 'petr.viktorin', 'eric.snow', 'Dormouse759']
pr_nums = []
priority = 'normal'
resolution = None
stage = 'needs patch'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue32459'
versions = []

[ncoghlan]

After commenting [1] on the fact that the current datetime module C API [2] is problematic due to its reliance on C level global variables, I discovered that this is actually the outcome of our recommended approach to using capsules to export a C API as function pointers: https://docs.python.org/3/extending/extending.html#providing-a-c-api-for-an-extension-module

So we first need to fix the documentation to propose a reloading friendly way of using capsules in other extension modules, and then we can then propose updating the public datetime C API accordingly (probably by defining new macros that accept the capsule reference as their first argument).

[1] https://bugs.python.org/issue10381#msg309214
[2] https://docs.python.org/3/c-api/datetime.html
[3] https://docs.python.org/3/extending/extending.html#providing-a-c-api-for-an-extension-module

[devdanzin]

The linked docs still suggest capsules as "a special mechanism to pass C-level information (pointers) from one extension module to another one", so the issue is still valid.

[ncoghlan]

Narrowing down the actual problem, it is this bit in the capsule consuming code:

static void **PySpam_API;

That holds a capsule object reference, and should be stored in the module state, not a C global.

The publishing code is fine, since its static array only holds C function pointers, with no Python object references.

(cc @ericsnowcurrently)