-
-
Couldn't load subscription status.
- Fork 33.2k
gh-116738: Make _csv module thread-safe #118344
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, Sorry for the late.
@colesbury Do you have any comments about this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry for the long delay in reviewing this. Two requests:
- Let's change
field_limitto aPy_ssize_t. That seems more appropriate here and avoids the need for adding new atomic bindings. I'd prefer to avoid adding additional atomic bindings because its more code to maintain and its easy to add subtle bugs in the atomic bindings. Additionally,field_limitis compared tofield_len, which is already aPy_ssize_t. - Let's use relaxed atomics for reading and writing
field_limit, e.g.,FT_ATOMIC_STORE_SSIZE_RELAXED. Relaxed ordering is fine for these sorts of global settings.
For the data type change:
%ld->%zdPyLong_AsLong->PyLong_AsSsize_t
Thanks for the review, updated! |
|
Thanks @aisk for the PR, and @kumaraditya303 for merging it 🌮🎉.. I'm working now to backport this PR to: 3.13. |
(cherry picked from commit a00221e) Co-authored-by: AN Long <[email protected]>
|
GH-125328 is a backport of this pull request to the 3.13 branch. |
This extension module has some module state values which may be read or written in different threads:
field_limit: A simple integer value to store the maximum field size for the parser;dialects: A dict value to store all the dialects by string names.field_limitThere is a public function
field_size_limitthat can read and write it,using a critical section to protect it. For other internal usages,_Py_atomic_load_longand_Py_atomic_store_longare used to make it thread safe.dialectsThere are three public functions
get_dialect,unregister_dialectandlist_dialectsthat can access this field.For the first two functions, a critical section is used to protect it.And the
list_dialectsfunction just returns the dict's keys withPyDict_Keys. I think its result is readonly andPyDict_Keysitself is thread safe, so we don't need to do anything.Other internal usages of
dialectsinvolve accessing its value withget_dialect_from_registry, which simply callsPyDict_GetItemRefand returns the result. I think we don't need to change it too.Update: I think these APIs is just call
dictAPIs which is already thread-safe, and there are no other state changes inside these APIs, and no intermediate variables need to protect in these functions, so we don't need add locks on them.