Skip to content

Conversation

@vstinner
Copy link
Member

@vstinner vstinner commented Sep 15, 2025

Replace PyBytes_FromStringAndSize(NULL, size) and _PyBytes_Resize() with the new public PyBytesWriter API.

Replace PyBytes_FromStringAndSize(NULL, size) and _PyBytes_Resize()
with the new public PyBytesWriter API.
@vstinner
Copy link
Member Author

Benchmark:

import pyperf
import os

file = open(__file__, "rb")
def func():
    file.seek(0)
    for _ in range(100):
        file.read1(1)

runner = pyperf.Runner()
runner.bench_func('bench', func)

file.close()

Result: Mean +- std dev: [ref] 87.0 us +- 1.5 us -> [change] 86.0 us +- 1.5 us: 1.01x faster.

@vstinner
Copy link
Member Author

cc @cmaloney

Copy link
Contributor

@cmaloney cmaloney left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 looks good to me; could make the PyBytesWriter a little later and save some cleanup code (but would mean making it during the buffered lock)

if (n == 0)
return PyBytes_FromStringAndSize(NULL, 0);
if (n == 0) {
return Py_GetConstant(Py_CONSTANT_EMPTY_BYTES);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comes up fairly frequently in the I/O code, thinking of making a _Py_EMPTY_BYTES internal macro for it (separate from this PR)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you mean a macro specific to the _io module?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking pycore_bytesobject.h; currently shows up 24 times across the CPython source code. BufferedIO and a couple other places also use a pattern: (PyObject *)&_Py_SINGLETON(bytes_empty)

Move PyBytesWriter_Create() call
@vstinner vstinner enabled auto-merge (squash) September 15, 2025 21:29
@vstinner vstinner merged commit 4554486 into python:main Sep 15, 2025
43 checks passed
@vstinner vstinner deleted the pybyteswriter_bufferedio branch September 15, 2025 21:50
@vstinner
Copy link
Member Author

Merged, thanks for the review and suggestions @cmaloney.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants