Skip to content

WIP: updated single_source_version with a much simpler page. #1578

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 20 commits into from
Closed

WIP: updated single_source_version with a much simpler page. #1578

Changes from 10 commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
fb3ec88
updated single_source_version with a much simpler page -- essentially
ChrisBarker-NOAA Jul 25, 2024
c7fa00c
Update source/single_source_version.rst
ChrisBarker-NOAA Jul 25, 2024
49842a8
Update source/single_source_version.rst
ChrisBarker-NOAA Jul 25, 2024
840801f
Added links to build tools
ChrisBarker-NOAA Jul 25, 2024
3724c8d
swap prefer for require
ChrisBarker-NOAA Jul 25, 2024
5368956
replace text about __version__
ChrisBarker-NOAA Jul 25, 2024
56db0d9
Update source/single_source_version.rst
ChrisBarker-NOAA Jul 26, 2024
9bace5d
Update source/single_source_version.rst
ChrisBarker-NOAA Jul 26, 2024
035c2bd
Update source/single_source_version.rst
ChrisBarker-NOAA Jul 26, 2024
dd1b70e
updated the __version__ description
ChrisBarker-NOAA Jul 26, 2024
eaf458a
Update source/single_source_version.rst
ChrisBarker-NOAA Jul 30, 2024
29aa220
Update source/single_source_version.rst
ChrisBarker-NOAA Jul 30, 2024
de722f6
Update source/single_source_version.rst
ChrisBarker-NOAA Jul 30, 2024
63061bd
Update source/single_source_version.rst
ChrisBarker-NOAA Jul 30, 2024
bfdc474
Update source/single_source_version.rst
ChrisBarker-NOAA Jul 30, 2024
c69e2c0
a few suggestions from the PR discussion
ChrisBarker-NOAA Jul 30, 2024
ddb077c
Merge branch 'simplify_single_source' of https://github.com/ChrisBark…
ChrisBarker-NOAA Jul 30, 2024
648c427
minor formatting edit
ChrisBarker-NOAA Jul 30, 2024
b9ced45
Update source/single_source_version.rst
ChrisBarker-NOAA Jul 31, 2024
0f5d2d3
a few more edits from the PR comments, and adding it back to the inde…
ChrisBarker-NOAA Jul 31, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
109 changes: 19 additions & 90 deletions source/single_source_version.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,111 +7,40 @@ Single-sourcing the Project Version
:Page Status: Complete
:Last Reviewed: 2015-09-08

One of the challenges in building packages is that the version string can be required in multiple places.

There are a few techniques to store the version in your project code without duplicating the value stored in
``setup.py``:
* It needs to be specified when building the package (e.g. in :file:``pyproject.toml``)
- That will assure that it is properly assigned in the distribution file name, and in the installed package.

#. Read the file in ``setup.py`` and parse the version with a regex. Example (
from `pip setup.py <https://github.com/pypa/pip/blob/1.5.6/setup.py#L33>`_)::
* A package may set a top level ``__version__`` attribute to provide runtime access to the version of the installed package. If this is done, the value of ``__version__`` attribute and that used by the build system to set the distribution's version should be kept in sync in :ref:`the build systems's recommended way <how_to_set_version_links>`.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like this introduced a lot of controversy in the past. The way it's written here, it reads as though the “__version__” name specifically must be used and nothing else. The reality is that the variable name can be arbitrary. And it doesn't have to be top-level to work, either. Additionally, I see that here and in the previous point, the word “package” is used, but it means different things. Here, you talk about an importable, while there it's about a distribution. Can we do anything about this?

Could you try rephrasing this to say that an in-project importable module may have a hard-coded variable that can be wired into packaging metadata by name? We can mention that many projects use __version__ for the variable name, but there is no standard mandating it to be that.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggested an edit in response to your comment.


def read(*names, **kwargs):
with io.open(
os.path.join(os.path.dirname(__file__), *names),
encoding=kwargs.get("encoding", "utf8")
) as fp:
return fp.read()
In the cases where a package does not set a top level ``__version__`` attribute, the version may still be accessible using ``importlib.metadata.version("distribution_name")``.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additionally, we should maybe mention that __version__ may be populated from importlib.metadata.version(__name__) to provide the version string from installed packaging metadata as an importable.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(ref #1276 (comment))

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As near as I can tell, the push to get rid of __version__ came from people protesting to this pattern (as a waste of runtime on import).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I'm starting to understand this better now, having read another thread: #1578 (comment)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It also comes from the fact that __version__ is just a conventional place to duplicate the information in that already has a canonical place: not very pythonic

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It also comes from the fact that version is just a conventional place to duplicate the information in that already has a canonical place: not very pythonic

I respectfully, but completely disagree with this characterization. __version__ is the (defacto) standard which predates the existence of pip by at least 7 years (in Matplotlib I have a commit from 2004 using __version__) let alone the current push to bootstrap another (Python -specific) packaging ecosystem.

From my perspective the "duplicate" here is dist-info and as I noted on discourse I am not worried about this. Package managers (apt/dpkg, yum, pacman, conda, pip, ....) all need their own copy of the version metadata (and dependencies) to do their jobs. This is OK and expected as each of these systems, and runtime, have different needs and hence (may) need their own copy of the information.

It comes across as very hostile to invent a new place to put the version information and then declare the existing location an upythonic duplicate!

As a thought experiment, if I said "We can scrap all of dist-info, we can query pacman to get all of it when we need it" I don't think it would be taken seriously. "just use the metadata for pip" reads exactly the same to me.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@flying-sheep I think that it's a bit more complicated than that. Duplicating the hardcoded values is bad. But if only one place is the actual source of the information and another just reflects the same information though a different API, that should be okay.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"just use the metadata for pip" reads exactly the same to me.

@tacaswell TBF, from the Python packaging perspective, that is not “metadata for pip” by a standardized machine-readable information. That said, if a project is meant to be installed and the installed dist metadata contains a version, it's reasonable to threat it as the source for the most accurate information (even if it originated from some importables that could be accessed separately). From that perspective, it might be the source of truth in many contexts.

But it's rather tricky to explain/document this in an accessible manner. Hence, my suggestion to attempt visualizing these relationships with my diagram #1276 (comment) (or something close to it).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if only one place is the actual source of the information and another just reflects the same information though a different API, that should be okay.

there should be one – and preferably only one – way to do it

the preferable way is the standard way

I respectfully, but completely disagree with this characterization.

__version__ is older, but it’s not standardized in a PEP, so it’s not the standard.

also putting that info into an import package doesn’t make sense, as the versioned unit is the distribution package – i.e. the thing that has the dist-info directory in it that contains the standard metadata. most distribution packages might come with one main import package (e.g. Pillow contains the import package PIL), but that’s not a rule, so the only place that will reliably match 1:1 with versioned units is the distribution package, complete with its metadata.


def find_version(*file_paths):
version_file = read(*file_paths)
version_match = re.search(r"^__version__ = ['\"]([^'\"]*)['\"]",
version_file, re.M)
if version_match:
return version_match.group(1)
raise RuntimeError("Unable to find version string.")
To ensure that version numbers do not get out of sync, it is recommended that there is a single source of truth for the version number.

setup(
...
version=find_version("package", "__init__.py")
...
)
In general, the options are:

.. note::
1) If the code is in a version control system (VCS), e.g. git, then the version can be extracted from the VCS.

This technique has the disadvantage of having to deal with complexities of regular expressions.
2) The version can be hard-coded into the `pyproject.toml` file -- and the build system can copy it into other locations it may be required.

#. Use an external build tool that either manages updating both locations, or
offers an API that both locations can use.
3) The version string can be hard-coded into the source code -- either in a special purpose file, such as ``_version.txt``, or as a attribute in the ``__init__.py``, and the build system can extract it at build time.

Few tools you could use, in no particular order, and not necessarily complete:
`bumpversion <https://pypi.python.org/pypi/bumpversion>`_,
`changes <https://pypi.python.org/pypi/changes>`_, `zest.releaser <https://pypi.python.org/pypi/zest.releaser>`_.

Consult your build system's documentation for their recommended method.

#. Set the value to a ``__version__`` global variable in a dedicated module in
your project (e.g. ``version.py``), then have ``setup.py`` read and ``exec`` the
value into a variable.
.. _how_to_set_version_links:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like other references use spaces. Perhaps, follow the same style?


Using ``execfile``:
Build System Version Handling
----------------------------

::
* `Hatch <https://hatch.pypa.io/1.9/version/>`_

execfile('...sample/version.py')
# now we have a `__version__` variable
# later on we use: __version__
* `Setuptools <https://setuptools.pypa.io/en/latest/userguide/distribution.html#specifying-your-project-s-version>`_

Using ``exec``:
- `setuptools_scm <https://setuptools-scm.readthedocs.io/en/latest/>`_

::
* `Flit <https://flit.pypa.io/en/stable/>`_

version = {}
with open("...sample/version.py") as fp:
exec(fp.read(), version)
# later on we use: version['__version__']
* `PDM <https://pdm-project.org/en/latest/reference/pep621/#__tabbed_1_2>`_

Example using this technique: `warehouse <https://github.com/pypa/warehouse/blob/master/warehouse/__about__.py>`_.

#. Place the value in a simple ``VERSION`` text file and have both ``setup.py``
and the project code read it.

::

with open(os.path.join(mypackage_root_dir, 'VERSION')) as version_file:
version = version_file.read().strip()

An advantage with this technique is that it's not specific to Python. Any
tool can read the version.

.. warning::

With this approach you must make sure that the ``VERSION`` file is included in
all your source and binary distributions.

#. Set the value in ``setup.py``, and have the project code use the
``pkg_resources`` API.

::

import pkg_resources
assert pkg_resources.get_distribution('pip').version == '1.2.0'

Be aware that the ``pkg_resources`` API only knows about what's in the
installation metadata, which is not necessarily the code that's currently
imported.


#. Set the value to ``__version__`` in ``sample/__init__.py`` and import
``sample`` in ``setup.py``.

::

import sample
setup(
...
version=sample.__version__
...
)

Although this technique is common, beware that it will fail if
``sample/__init__.py`` imports packages from ``install_requires``
dependencies, which will very likely not be installed yet when ``setup.py``
is run.