Skip to content

Commit 4f102f8

Browse files
edmorleyedmorley
committed
Re-apply Pipenv --system bug workaround
In #2000, Pipenv was updated to v2026.0.3, which contained a number of fixes for the historically very buggy `--system` mode. Since that Pipenv release was meant to fix many of the `--system` bugs, one of the workarounds for those bugs was removed, since even without it the existing tests for that bug still passed (implying the upstream fixes had worked as expected). However, it appears that the bug still exists in some form that was not being picked up by the original testcase, as reported in: #2000 (comment) As such, I've had to re-add the workaround. This also affects the logs, since in the new Pipenv version, if `PIPENV_VERBOSITY="-1"` is set (which is required to stop an obnoxious multi-line warning about a venv being present), then the `Installing dependencies from ...` line is no longer printed. It seems the bug now only reproduces with different versions of the `certifi` package, so I've updated the test fixture accordingly. (My first thought was that perhaps the difference is whether our testcase's `certifi` matches the Pipenv vendored version exactly or not, however, varying to another older version also didn't reproduce?). I've also added `packaging` as another testcase dependency, in the hope that in the future at least one of these packages will be able to reproduce the issue, should we try and remove the workaround again. Longer term, these continual Pipenv bugs are making me inclined to deprecate/strongly discourage Pipenv usage, given there are now much more reliable/better maintained/faster alternatives (such as uv) that don't suffer from these chronic bugs. GUS-W-20819376.
1 parent f2c6773 commit 4f102f8

File tree

7 files changed

+53
-20
lines changed

7 files changed

+53
-20
lines changed

‎CHANGELOG.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@
22

33
## [Unreleased]
44

5+
- Re-apply a workaround for a Pipenv bug when using `--system`, that causes packages to not be installed correctly if they are also a dependency of Pipenv (such as `certifi` or `packaging`). ([#2011](https://github.com/heroku/heroku-buildpack-python/pull/2011))
56

67
## [v329] - 2026-01-08
78

@@ -197,7 +198,7 @@
197198

198199
## [v293] - 2025-07-23
199200

200-
- Work around a Pipenv bug when using `--system`, that causes packages to not be installed correctly if they are also a dependency of Pipenv (such as `certifi` ). ([#1842](https://github.com/heroku/heroku-buildpack-python/pull/1842))
201+
- Work around a Pipenv bug when using `--system`, that causes packages to not be installed correctly if they are also a dependency of Pipenv (such as `certifi`). ([#1842](https://github.com/heroku/heroku-buildpack-python/pull/1842))
201202

202203
## [v292] - 2025-07-23
203204

‎lib/pipenv.sh‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -81,17 +81,31 @@ function pipenv::install_pipenv() {
8181
export PATH="${pipenv_bin_dir}:${PATH}"
8282
# Force Pipenv to manage the system Python site-packages instead of using venvs.
8383
export PIPENV_SYSTEM="1"
84+
# Hide Pipenv's notice about finding/using an existing virtual environment.
85+
export PIPENV_VERBOSITY="-1"
86+
# Work around a Pipenv bug when using `--system`, whereby it doesn't correctly install dependencies
87+
# that happen to also be a dependency of Pipenv (such as `certifi` and `packaging`). In general
88+
# Pipenv's support for its `--system` mode is very buggy. Longer term we should explore moving
89+
# to venvs, however, that will need to be coordinated across all package managers and will also
90+
# change paths for Python which could break other use cases. Be careful removing this even if the
91+
# `pip_basic` test that installs certifi/packaging still passes, since the repro seems to depend
92+
# on specific package version combinations / other factors and so the testcase is very fragile.
93+
export VIRTUAL_ENV="${python_home}"
8494

8595
# Set the same env vars in the environment used by later buildpacks.
8696
cat >>"${export_file}" <<-EOF
8797
export PATH="${pipenv_bin_dir}:\${PATH}"
8898
export PIPENV_SYSTEM="1"
99+
export PIPENV_VERBOSITY="-1"
100+
export VIRTUAL_ENV="${python_home}"
89101
EOF
90102

91103
# And the environment used at app run-time.
92104
cat >>"${profile_d_file}" <<-EOF
93105
export PATH="${pipenv_bin_dir}:\${PATH}"
94106
export PIPENV_SYSTEM="1"
107+
export PIPENV_VERBOSITY="-1"
108+
export VIRTUAL_ENV="${python_home}"
95109
EOF
96110
}
97111

‎spec/fixtures/pipenv_basic/Pipfile‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,8 +4,11 @@ verify_ssl = true
44
name = "pypi"
55

66
[packages]
7-
certifi = "*"
87
typing-extensions = "*"
8+
# Test that dependencies that happen to also be vendored dependencies of Pipenv are correctly installed,
9+
# since Pipenv's --system mode is buggy and requires a workaround to ensure they aren't skipped.
10+
certifi = "*"
11+
packaging = "*"
912

1013
[dev-packages]
1114

‎spec/fixtures/pipenv_basic/Pipfile.lock‎

Lines changed: 13 additions & 4 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎spec/fixtures/pipenv_basic/bin/compile‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,10 @@ pip list --disable-pip-version-check --exclude pip
2323
echo
2424

2525
python -c 'import typing_extensions; print(typing_extensions)'
26+
# Test that dependencies that happen to also be vendored dependencies of Pipenv are correctly installed,
27+
# since Pipenv's --system mode is buggy and requires a workaround to ensure they aren't skipped.
28+
python -c 'import certifi; print(certifi)'
29+
python -c 'import packaging; print(packaging)'
2630
echo
2731

2832
jq --sort-keys '.' "${CACHE_DIR}/build-data/python.json"

‎spec/hatchet/ci_spec.rb‎

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -105,8 +105,6 @@
105105
-----> Installing Python #{DEFAULT_PYTHON_FULL_VERSION}
106106
-----> Installing Pipenv #{PIPENV_VERSION}
107107
-----> Installing dependencies using 'pipenv install --deploy --dev'
108-
Installing dependencies from Pipfile.lock \\(.+\\)...
109-
Installing dependencies from Pipfile.lock \\(.+\\)...
110108
-----> Skipping Django collectstatic since the env var DISABLE_COLLECTSTATIC is set.
111109
-----> Running bin/post_compile hook
112110
BUILD_DIR=/app
@@ -123,8 +121,10 @@
123121
LIBRARY_PATH=/app/.heroku/python/lib
124122
PATH=/app/.heroku/python/pipenv/bin:/app/.heroku/python/bin:/usr/local/bin:/usr/bin:/bin:/app/.sprettur/bin/
125123
PIPENV_SYSTEM=1
124+
PIPENV_VERBOSITY=-1
126125
PKG_CONFIG_PATH=/app/.heroku/python/lib/pkg-config
127126
PYTHONUNBUFFERED=1
127+
VIRTUAL_ENV=/app/.heroku/python
128128
-----> Saving cache
129129
130130
! Note: We recently added support for the package manager uv:
@@ -143,9 +143,11 @@
143143
LIBRARY_PATH=/app/.heroku/python/lib
144144
PATH=/app/.heroku/python/bin:/app/.heroku/python/pipenv/bin:/usr/local/bin:/usr/bin:/bin:/app/.sprettur/bin/
145145
PIPENV_SYSTEM=1
146+
PIPENV_VERBOSITY=-1
146147
PYTHONHOME=/app/.heroku/python
147148
PYTHONPATH=/app
148149
PYTHONUNBUFFERED=true
150+
VIRTUAL_ENV=/app/.heroku/python
149151
-----> No test-setup command provided. Skipping.
150152
-----> Running test command `./bin/print-env-vars.sh && pytest --version`...
151153
CI=true
@@ -157,9 +159,11 @@
157159
LIBRARY_PATH=/app/.heroku/python/lib
158160
PATH=/app/.heroku/python/bin:/app/.heroku/python/pipenv/bin:/usr/local/bin:/usr/bin:/bin:/app/.sprettur/bin/:/app/.sprettur/bin/
159161
PIPENV_SYSTEM=1
162+
PIPENV_VERBOSITY=-1
160163
PYTHONHOME=/app/.heroku/python
161164
PYTHONPATH=/app
162165
PYTHONUNBUFFERED=true
166+
VIRTUAL_ENV=/app/.heroku/python
163167
WEB_CONCURRENCY=5
164168
pytest .+
165169
-----> test command `./bin/print-env-vars.sh && pytest --version` completed successfully
@@ -173,8 +177,6 @@
173177
-----> Using cached install of Python #{DEFAULT_PYTHON_FULL_VERSION}
174178
-----> Using cached Pipenv #{PIPENV_VERSION}
175179
-----> Installing dependencies using 'pipenv install --deploy --dev'
176-
Installing dependencies from Pipfile.lock \\(.+\\)...
177-
Installing dependencies from Pipfile.lock \\(.+\\)...
178180
-----> Skipping Django collectstatic since the env var DISABLE_COLLECTSTATIC is set.
179181
-----> Running bin/post_compile hook
180182
REGEX

‎spec/hatchet/pipenv_spec.rb‎

Lines changed: 10 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,6 @@
1515
remote: -----> Installing Python #{DEFAULT_PYTHON_FULL_VERSION}
1616
remote: -----> Installing Pipenv #{PIPENV_VERSION}
1717
remote: -----> Installing dependencies using 'pipenv install --deploy'
18-
remote: Installing dependencies from Pipfile.lock \\(.+\\)...
1918
remote: -----> Running bin/post_compile hook
2019
remote: BUILD_DIR=/tmp/build_.+
2120
remote: CACHE_DIR=/tmp/codon/tmp/cache
@@ -27,8 +26,10 @@
2726
remote: LIBRARY_PATH=/app/.heroku/python/lib
2827
remote: PATH=/app/.heroku/python/pipenv/bin:/app/.heroku/python/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
2928
remote: PIPENV_SYSTEM=1
29+
remote: PIPENV_VERBOSITY=-1
3030
remote: PKG_CONFIG_PATH=/app/.heroku/python/lib/pkg-config
3131
remote: PYTHONUNBUFFERED=1
32+
remote: VIRTUAL_ENV=/app/.heroku/python
3233
remote: -----> Saving cache
3334
remote:
3435
remote: ! Note: We recently added support for the package manager uv:
@@ -47,9 +48,11 @@
4748
remote: LIBRARY_PATH=/app/.heroku/python/lib
4849
remote: PATH=/app/.heroku/python/bin:/app/.heroku/python/pipenv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
4950
remote: PIPENV_SYSTEM=1
51+
remote: PIPENV_VERBOSITY=-1
5052
remote: PYTHONHOME=/app/.heroku/python
5153
remote: PYTHONPATH=/app
5254
remote: PYTHONUNBUFFERED=true
55+
remote: VIRTUAL_ENV=/app/.heroku/python
5356
remote:
5457
remote: \\['',
5558
remote: '/app',
@@ -61,10 +64,13 @@
6164
remote: pipenv, version #{PIPENV_VERSION}
6265
remote: Package Version
6366
remote: ----------------- -+
64-
remote: certifi 2025.11.12
67+
remote: certifi 2026.1.4
68+
remote: packaging 25.0
6569
remote: typing_extensions 4.15.0
6670
remote:
6771
remote: <module 'typing_extensions' from '/app/.heroku/python/lib/python3.14/site-packages/typing_extensions.py'>
72+
remote: <module 'certifi' from '/app/.heroku/python/lib/python3.14/site-packages/certifi/__init__.py'>
73+
remote: <module 'packaging' from '/app/.heroku/python/lib/python3.14/site-packages/packaging/__init__.py'>
6874
remote:
6975
remote: \\{
7076
remote: "cache_restore_duration": [0-9.]+,
@@ -99,7 +105,6 @@
99105
remote: -----> Using cached install of Python #{DEFAULT_PYTHON_FULL_VERSION}
100106
remote: -----> Using cached Pipenv #{PIPENV_VERSION}
101107
remote: -----> Installing dependencies using 'pipenv install --deploy'
102-
remote: Installing dependencies from Pipfile.lock \\(.+\\)...
103108
remote: -----> Running bin/post_compile hook
104109
remote: .+
105110
remote: -----> Saving cache
@@ -115,9 +120,11 @@
115120
LIBRARY_PATH=/app/.heroku/python/lib
116121
PATH=/app/.heroku/python/bin:/app/.heroku/python/pipenv/bin:/usr/local/bin:/usr/bin:/bin
117122
PIPENV_SYSTEM=1
123+
PIPENV_VERBOSITY=-1
118124
PYTHONHOME=/app/.heroku/python
119125
PYTHONPATH=/app
120126
PYTHONUNBUFFERED=true
127+
VIRTUAL_ENV=/app/.heroku/python
121128
WEB_CONCURRENCY=2
122129
pipenv, version #{PIPENV_VERSION}
123130
OUTPUT
@@ -143,7 +150,6 @@
143150
remote: -----> Installing Python #{DEFAULT_PYTHON_FULL_VERSION}
144151
remote: -----> Installing Pipenv #{PIPENV_VERSION}
145152
remote: -----> Installing dependencies using 'pipenv install --deploy'
146-
remote: Installing dependencies from Pipfile.lock \\(.+\\)...
147153
remote: -----> Saving cache
148154
REGEX
149155
end
@@ -198,7 +204,6 @@
198204
remote: -----> Installing Python 3.10.0
199205
remote: -----> Installing Pipenv #{PIPENV_VERSION}
200206
remote: -----> Installing dependencies using 'pipenv install --deploy'
201-
remote: Installing dependencies from Pipfile.lock \\(.+\\)...
202207
remote: -----> Saving cache
203208
REGEX
204209
end
@@ -216,7 +221,6 @@
216221
remote: -----> Installing Python #{LATEST_PYTHON_3_13}
217222
remote: -----> Installing Pipenv #{PIPENV_VERSION}
218223
remote: -----> Installing dependencies using 'pipenv install --deploy'
219-
remote: Installing dependencies from Pipfile.lock \\(.+\\)...
220224
remote: -----> Saving cache
221225
REGEX
222226
end
@@ -275,7 +279,6 @@
275279
remote: -----> Installing Python #{DEFAULT_PYTHON_FULL_VERSION}
276280
remote: -----> Installing Pipenv #{PIPENV_VERSION}
277281
remote: -----> Installing dependencies using 'pipenv install --deploy'
278-
remote: Installing dependencies from Pipfile.lock \\(.+\\)...
279282
remote: -----> Saving cache
280283
REGEX
281284
end
@@ -468,7 +471,6 @@
468471
remote: -----> Installing Python #{LATEST_PYTHON_3_14}
469472
remote: -----> Installing Pipenv #{PIPENV_VERSION}
470473
remote: -----> Installing dependencies using 'pipenv install --deploy'
471-
remote: Installing dependencies from Pipfile.lock \\(.+\\)...
472474
remote: -----> Saving cache
473475
REGEX
474476
end
@@ -483,7 +485,6 @@
483485
app.deploy do |app|
484486
expect(clean_output(app.output)).to match(Regexp.new(<<~REGEX, Regexp::MULTILINE))
485487
remote: -----> Installing dependencies using 'pipenv install --deploy'
486-
remote: Installing dependencies from Pipfile.lock \\(.+\\)...
487488
remote: -----> Running bin/post_compile hook
488489
remote: __editable___gunicorn_23_0_0_finder.py:/app/.heroku/python/src/gunicorn/gunicorn'}
489490
remote: __editable___local_package_pyproject_toml_0_0_1_finder.py:/tmp/build_.+/packages/local_package_pyproject_toml/local_package_pyproject_toml'}
@@ -526,7 +527,6 @@
526527
app.push!
527528
expect(clean_output(app.output)).to match(Regexp.new(<<~REGEX, Regexp::MULTILINE))
528529
remote: -----> Installing dependencies using 'pipenv install --deploy'
529-
remote: Installing dependencies from Pipfile.lock \\(.+\\)...
530530
remote: -----> Running bin/post_compile hook
531531
remote: __editable___gunicorn_23_0_0_finder.py:/app/.heroku/python/src/gunicorn/gunicorn'}
532532
remote: __editable___local_package_pyproject_toml_0_0_1_finder.py:/tmp/build_.+/packages/local_package_pyproject_toml/local_package_pyproject_toml'}

0 commit comments

Comments
 (0)