Skip to content

Commit 7aec662

Browse files
committed
Merge branch 'develop' into master
* develop: Update CHANGELOG.md Fix pylance complaint (#917) Add more references to README.rst (#915) Pin github-release-commenter (#914) Fix doctest (#913) Bump softprops/action-gh-release in the github-actions group (#912) Bump minimum supported version to Python 3.10 (#911) Bump the github-actions group with 2 updates (#910)
2 parents 5ae510e + f83805a commit 7aec662

File tree

8 files changed

+89
-63
lines changed

8 files changed

+89
-63
lines changed

.github/workflows/python-package.yml

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -17,17 +17,17 @@ jobs:
1717
matrix:
1818
include:
1919
# sync with linting steps below
20-
- {python-version: '3.9', os: ubuntu-24.04}
20+
- {python-version: '3.10', os: ubuntu-24.04}
2121
- {python-version: '3.14', os: ubuntu-24.04}
2222

23-
- {python-version: '3.9', os: windows-2025}
23+
- {python-version: '3.10', os: windows-2025}
2424
- {python-version: '3.14', os: windows-2025}
2525

26-
- {python-version: '3.9', os: macos-15}
26+
- {python-version: '3.10', os: macos-15}
2727
- {python-version: '3.14', os: macos-15}
2828

2929
steps:
30-
- uses: actions/checkout@v5
30+
- uses: actions/checkout@v6
3131
with:
3232
fetch-depth: 0 # fetch git tags for setuptools_scm (smart_open.__version__)
3333

@@ -48,11 +48,11 @@ jobs:
4848
run: uv pip install -e .[test]
4949

5050
- name: Run flake8 linter (source)
51-
if: matrix.python-version == '3.9' # sync with matrix above
51+
if: matrix.python-version == '3.10' # sync with matrix above
5252
run: flake8 --show-source smart_open
5353

5454
- name: "Check whether help.txt update was forgotten"
55-
if: matrix.python-version == '3.9' && github.event_name == 'pull_request' # sync with matrix above
55+
if: matrix.python-version == '3.10' && github.event_name == 'pull_request' # sync with matrix above
5656
run: |
5757
python update_helptext.py
5858
test ! "$(git diff)" && echo "no changes" || ( git diff && echo 'looks like "python update_helptext.py" was forgotten' && exit 1 )

.github/workflows/release.yml

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@ jobs:
1414
pull-requests: write # apexskier/github-release-commenter
1515

1616
steps:
17-
- uses: actions/checkout@v5
17+
- uses: actions/checkout@v6
1818
with:
1919
fetch-depth: 0 # fetch git tags for setuptools_scm (smart_open.__version__)
2020

@@ -32,16 +32,15 @@ jobs:
3232
python -m build
3333
3434
- name: Upload package distributions as release assets
35-
uses: softprops/action-gh-release@v2.4.1
35+
uses: softprops/action-gh-release@v2.5.0
3636
with:
3737
files: dist/*
3838

3939
# https://github.com/pypa/gh-action-pypi-publish#trusted-publishing
4040
- name: Publish package distributions to PyPI
4141
uses: pypa/gh-action-pypi-publish@v1.13.0
4242

43-
- uses: apexskier/github-release-commenter@v1.3.6
43+
- uses: apexskier/github-release-commenter@v1.4.1
4444
with:
45-
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
4645
comment-template: |
4746
Released {release_link}

CHANGELOG.md

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,13 @@
1+
# 7.5.1, 2026-02-23
2+
3+
- Bump the github-actions group with 2 updates (PR [#910](https://github.com/piskvorky/smart_open/pull/910), [@dependabot[bot]](https://github.com/apps/dependabot))
4+
- Bump minimum supported version to Python 3.10 (PR [#911](https://github.com/piskvorky/smart_open/pull/911), [@DeflateAwning](https://github.com/DeflateAwning))
5+
- Bump softprops/action-gh-release from 2.4.2 to 2.5.0 in the github-actions group (PR [#912](https://github.com/piskvorky/smart_open/pull/912), [@dependabot[bot]](https://github.com/apps/dependabot))
6+
- Fix doctest (PR [#913](https://github.com/piskvorky/smart_open/pull/913), [@ddelange](https://github.com/ddelange))
7+
- Pin github-release-commenter (PR [#914](https://github.com/piskvorky/smart_open/pull/914), [@ddelange](https://github.com/ddelange))
8+
- Add more references to README.rst (PR [#915](https://github.com/piskvorky/smart_open/pull/915), [@ddelange](https://github.com/ddelange))
9+
- Fix pylance complaint (PR [#917](https://github.com/piskvorky/smart_open/pull/917), [@ddelange](https://github.com/ddelange))
10+
111
# 7.5.0, 2025-11-08
212

313
- [s3] Support very large buckets in iter_bucket, add client_kwars (PR [#908](https://github.com/piskvorky/smart_open/pull/908), [@ddelange](https://github.com/ddelange))

README.rst

Lines changed: 34 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -3,16 +3,20 @@ smart_open — utils for streaming large files in Python
33
======================================================
44

55

6-
|License|_ |CI|_ |Coveralls|_ |Downloads|_
6+
|License|_ |CI|_ |Coveralls|_ |Version|_ |Python|_ |Downloads|_
77

88
.. |License| image:: https://img.shields.io/pypi/l/smart_open.svg
99
.. |CI| image:: https://github.com/piskvorky/smart_open/actions/workflows/python-package.yml/badge.svg?branch=develop&event=push
1010
.. |Coveralls| image:: https://coveralls.io/repos/github/RaRe-Technologies/smart_open/badge.svg?branch=develop
11+
.. |Version| image:: https://img.shields.io/pypi/v/smart-open.svg?logo=pypi&logoColor=white
12+
.. |Python| image:: https://img.shields.io/pypi/pyversions/smart-open.svg?logo=python&logoColor=white
1113
.. |Downloads| image:: https://pepy.tech/badge/smart-open/month
1214
.. _License: https://github.com/piskvorky/smart_open/blob/master/LICENSE
1315
.. _CI: https://github.com/piskvorky/smart_open/actions/workflows/python-package.yml
1416
.. _Coveralls: https://coveralls.io/github/RaRe-Technologies/smart_open?branch=HEAD
15-
.. _Downloads: https://pypi.org/project/smart-open/
17+
.. _Version: https://pypi.org/project/smart-open/
18+
.. _Python: https://pypi.org/project/smart-open/
19+
.. _Downloads: https://pypistats.org/packages/smart-open
1620

1721

1822
What?
@@ -22,8 +26,6 @@ What?
2226

2327
``smart_open`` is a drop-in replacement for Python's built-in ``open()``: it can do anything ``open`` can (100% compatible, falls back to native ``open`` wherever possible), plus lots of nifty extra stuff on top.
2428

25-
**Python 2.7 is no longer supported. If you need Python 2.7, please use** `smart_open 1.10.1 <https://github.com/piskvorky/smart_open/releases/tag/1.10.0>`_, **the last version to support Python 2.**
26-
2729
Why?
2830
====
2931

@@ -79,56 +81,49 @@ How?
7981
b'User'
8082
8183
>>> # stream from HTTP
82-
>>> for line in open('http://example.com/index.html'):
84+
>>> for line in open('http://example.com'):
8385
... print(repr(line[:15]))
8486
... break
8587
'<!doctype html>'
8688
8789
.. _doctools_after_examples:
8890

89-
Other examples of URIs that ``smart_open`` accepts::
91+
For more examples of URIs that ``smart_open`` accepts, see `help.txt <https://github.com/piskvorky/smart_open/blob/master/help.txt>`__ or ``help('smart_open')``.
92+
Some examples::
9093

9194
s3://bucket/key
9295
s3://access_key_id:secret_access_key@bucket/key
93-
s3://access_key_id:secret_access_key@server:port@bucket/key
9496
gs://bucket/blob
9597
azure://bucket/blob
96-
hdfs:///path/file
9798
hdfs://path/file
98-
webhdfs://host:port/path/file
99-
./local/path/file
100-
~/local/path/file
101-
local/path/file
10299
./local/path/file.gz
103-
file:///home/user/file
104100
file:///home/user/file.bz2
105-
[ssh|scp|sftp]://username@host//path/file
106-
[ssh|scp|sftp]://username@host/path/file
107101
[ssh|scp|sftp]://username:password@host/path/file
108102

109103

110104
Documentation
111105
=============
112106

113-
The API reference can be viewed at `help.txt <https://github.com/piskvorky/smart_open/blob/master/help.txt>`__
107+
The API reference can be viewed at `help.txt <https://github.com/piskvorky/smart_open/blob/master/help.txt>`__ or using ``help('smart_open')``.
114108

115109
Installation
116110
------------
117111

118112
``smart_open`` supports a wide range of storage solutions. For all options, see the API reference.
119113
Each individual solution has its own dependencies.
120-
By default, ``smart_open`` does not install any dependencies, in order to keep the installation size small.
121-
You can install one or more of these dependencies explicitly using optional dependencies:
114+
By default, ``smart_open`` does not install any dependencies in order to keep the installation size small.
115+
You can install one or more of these dependencies explicitly using optional dependencies defined in
116+
`pyproject.toml <https://github.com/piskvorky/smart_open/blob/master/pyproject.toml>`__ :
122117

123-
pip install smart_open[s3,gcs,azure,http,webhdfs,ssh,zst]
118+
.. code-block:: sh
124119
125-
Or, if you don't mind installing a large number of third party libraries, you can install all dependencies using::
120+
pip install 'smart_open[s3,gcs,azure,http,webhdfs,ssh,zst]'
126121
127-
pip install smart_open[all]
122+
Or, if you don't mind installing a large number of third party libraries, you can install all dependencies using:
128123

129-
Be warned that this option increases the installation size significantly, e.g. over 100MB.
124+
.. code-block:: sh
130125
131-
If you're upgrading from ``smart_open`` versions 2.x and below, please check out the `Migration Guide <MIGRATING_FROM_OLDER_VERSIONS.rst>`_.
126+
pip install 'smart_open[all]'
132127
133128
Built-in help
134129
-------------
@@ -144,9 +139,11 @@ or view `help.txt <https://github.com/piskvorky/smart_open/blob/master/help.txt>
144139
More examples
145140
-------------
146141

147-
For the sake of simplicity, the examples below assume you have all the dependencies installed, i.e. you have done::
142+
For the sake of simplicity, the examples below assume you have all the dependencies installed, i.e. you have done:
143+
144+
.. code-block:: sh
148145
149-
pip install smart_open[all]
146+
pip install 'smart_open[all]'
150147
151148
.. code-block:: python
152149
@@ -237,11 +234,14 @@ The supported values for this parameter are:
237234

238235
- ``infer_from_extension`` (default behavior)
239236
- ``disable``
240-
- ``.gz``
241237
- ``.bz2``
238+
- ``.gz``
239+
- ``.xz``
242240
- ``.zst``
243241

244-
By default, ``smart_open`` determines the compression algorithm to use based on the file extension.
242+
By default, ``smart_open`` automatically (de)compresses the file if the filename ends with one of these extensions.
243+
`See also <https://github.com/piskvorky/smart_open/blob/master/smart_open/compression.py>`__
244+
``smart_open.compression.get_supported_compression_types`` and ``mart_open.compression.register_compressor``.
245245

246246
.. code-block:: python
247247
@@ -292,13 +292,15 @@ This is just an example: ``lzma`` is in the standard library and is registered b
292292
Transport-specific Options
293293
--------------------------
294294

295-
``smart_open`` supports a wide range of transport options out of the box, including:
295+
``smart_open`` supports a wide range of transport options out of the box.
296+
For the full list of supported URI schemes, see `help.txt <https://github.com/piskvorky/smart_open/blob/master/help.txt>`__ or ``help('smart_open')``.
297+
Some examples::
296298

297-
- S3
299+
- AWS S3 (and any S3-Compatible)
298300
- HTTP, HTTPS (read-only)
299301
- SSH, SCP and SFTP
300-
- WebHDFS
301-
- GCS
302+
- HDFS / WebHDFS
303+
- Google Cloud Storage
302304
- Azure Blob Storage
303305

304306
Each option involves setting up its own set of parameters.
@@ -312,7 +314,7 @@ Here are some examples of using this parameter:
312314
>>> fin = open('s3://commoncrawl/robots.txt', transport_params=dict(client=boto3.client('s3')))
313315
>>> fin = open('s3://commoncrawl/robots.txt', transport_params=dict(buffer_size=1024))
314316
315-
For the full list of keyword arguments supported by each transport option, see the documentation:
317+
For the full list of keyword arguments supported by each transport option, see `help.txt <https://github.com/piskvorky/smart_open/blob/master/help.txt>`__ or ``help('smart_open')``.
316318

317319
.. code-block:: python
318320

help.txt

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -337,7 +337,7 @@ FUNCTIONS
337337
b'User'
338338

339339
>>> # stream from HTTP
340-
>>> for line in open('http://example.com/index.html'):
340+
>>> for line in open('http://example.com'):
341341
... print(repr(line[:15]))
342342
... break
343343

pyproject.toml

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ description = "Utils for streaming large files (S3, HDFS, GCS, SFTP, Azure Blob
1111
readme = "README.rst"
1212
urls = {Repository = "https://github.com/piskvorky/smart_open"}
1313
authors = [{name = "Radim Rehurek", email = "me@radimrehurek.com"}]
14-
requires-python = ">=3.9,<4.0" # sync with classifiers below
14+
requires-python = ">=3.10,<4.0" # sync with classifiers below
1515
classifiers = [
1616
"Development Status :: 5 - Production/Stable",
1717
"Environment :: Console",
@@ -20,7 +20,6 @@ classifiers = [
2020
"Operating System :: OS Independent",
2121
"Programming Language :: Python",
2222
"Programming Language :: Python :: 3",
23-
"Programming Language :: Python :: 3.9",
2423
"Programming Language :: Python :: 3.10",
2524
"Programming Language :: Python :: 3.11",
2625
"Programming Language :: Python :: 3.12",

smart_open/doctools.py

Lines changed: 30 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -16,11 +16,21 @@
1616
import io
1717
import os.path
1818
import re
19+
import sys
1920

2021
from . import compression
2122
from . import transport
2223

23-
PLACEHOLDER = ' smart_open/doctools.py magic goes here'
24+
#
25+
# Python 3.13+ automatically trims docstrings (like inspect.cleandoc),
26+
# so we need to adjust the placeholder and indentation accordingly.
27+
#
28+
if sys.version_info >= (3, 13):
29+
PLACEHOLDER = 'smart_open/doctools.py magic goes here'
30+
LPAD = ''
31+
else:
32+
PLACEHOLDER = ' smart_open/doctools.py magic goes here'
33+
LPAD = ' '
2434

2535

2636
def extract_kwargs(docstring):
@@ -143,7 +153,7 @@ def to_docstring(kwargs, lpad=''):
143153
return buf.getvalue()
144154

145155

146-
def extract_examples_from_readme_rst(indent=' '):
156+
def extract_examples_from_readme_rst(indent=None):
147157
"""Extract examples from this project's README.rst file.
148158
149159
Parameters
@@ -160,6 +170,8 @@ def extract_examples_from_readme_rst(indent=' '):
160170
-----
161171
Quite fragile, depends on named labels inside the README.rst file.
162172
"""
173+
if indent is None:
174+
indent = LPAD
163175
curr_dir = os.path.dirname(os.path.abspath(__file__))
164176
readme_path = os.path.join(curr_dir, '..', 'README.rst')
165177
try:
@@ -180,7 +192,7 @@ def tweak_open_docstring(f):
180192
root_path = os.path.dirname(os.path.dirname(__file__))
181193

182194
with contextlib.redirect_stdout(buf):
183-
print(' smart_open supports the following transport mechanisms:')
195+
print(f'{LPAD}smart_open supports the following transport mechanisms:')
184196
print()
185197
for scheme, submodule in sorted(transport._REGISTRY.items()):
186198
if scheme == transport.NO_SCHEME or submodule in seen:
@@ -194,27 +206,27 @@ def tweak_open_docstring(f):
194206

195207
relpath = os.path.relpath(submodule.__file__, start=root_path)
196208
heading = '%s (%s)' % ("/".join(schemes), relpath)
197-
print(' %s' % heading)
198-
print(' %s' % ('~' * len(heading)))
199-
print(' %s' % submodule.__doc__.split('\n')[0])
209+
print(f'{LPAD}{heading}')
210+
print(f'{LPAD}{"~" * len(heading)}')
211+
print(f'{LPAD}{submodule.__doc__.split(chr(10))[0]}')
200212
print()
201213

202214
kwargs = extract_kwargs(submodule.open.__doc__)
203215
if kwargs:
204-
print(to_docstring(kwargs, lpad=u' '))
216+
print(to_docstring(kwargs, lpad=LPAD))
205217

206-
print(' Examples')
207-
print(' --------')
218+
print(f'{LPAD}Examples')
219+
print(f'{LPAD}--------')
208220
print()
209-
print(extract_examples_from_readme_rst())
221+
print(extract_examples_from_readme_rst(indent=LPAD))
210222

211-
print(' This function also supports transparent compression and decompression ')
212-
print(' using the following codecs:')
223+
print(f'{LPAD}This function also supports transparent compression and decompression ')
224+
print(f'{LPAD}using the following codecs:')
213225
print()
214226
for extension in compression.get_supported_extensions():
215-
print(' * %s' % extension)
227+
print(f'{LPAD}* {extension}')
216228
print()
217-
print(' The function depends on the file extension to determine the appropriate codec.')
229+
print(f'{LPAD}The function depends on the file extension to determine the appropriate codec.')
218230

219231
#
220232
# The docstring can be None if -OO was passed to the interpreter.
@@ -246,15 +258,15 @@ def tweak_parse_uri_docstring(f):
246258
schemes.append(scheme)
247259

248260
with contextlib.redirect_stdout(buf):
249-
print(' Supported URI schemes are:')
261+
print(f'{LPAD}Supported URI schemes are:')
250262
print()
251263
for scheme in schemes:
252-
print(' * %s' % scheme)
264+
print(f'{LPAD}* {scheme}')
253265
print()
254-
print(' Valid URI examples::')
266+
print(f'{LPAD}Valid URI examples::')
255267
print()
256268
for example in examples:
257-
print(' * %s' % example)
269+
print(f'{LPAD}* {example}')
258270

259271
if f.__doc__:
260272
f.__doc__ = f.__doc__.replace(PLACEHOLDER, buf.getvalue())

smart_open/utils.py

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -214,6 +214,10 @@ def __init__(self, outer, inner):
214214
super().__init__(outer)
215215
self.__inner = inner
216216

217+
def __enter__(self):
218+
"""This explicit proxy method is only required for pylance ref #916."""
219+
return self.__wrapped__.__enter__()
220+
217221
def __exit__(self, *args, **kwargs):
218222
"""Exit inner after exiting outer."""
219223
try:

0 commit comments

Comments
 (0)