poetry doesn't seem to respect poetry config certificates.pypi.cert false. Getting CERTIFICATE_VERIFY_FAILED #6670
Description
-
Poetry version: 1.2.1
-
Python version: 3.9
-
OS version and name: Windows 10
-
pyproject.toml: https://gist.github.com/brno32/ea2d3629c351bdd9029380d920e7fb32
-
I am on the latest stable Poetry version, installed using a recommended method.
-
I have searched the issues of this repo and believe that this is not a duplicate.
-
I have consulted the FAQ and blog for any relevant entries or release notes.
-
If an exception occurs when executing a command, I executed it again in debug mode (
-vvvoption) and have included the output below.
Issue
I'm behind a corporate firewall which I cannot change. I won't pretend to know what's really going on here, but we're swapping out some of the SSL certificates with our own, self-signed certificate. This poses problems when trying to pip install packages on pypi. but for plain ol' pip installs, my team just runs this
pip install --upgrade --trusted-host pypi.python.org --trusted-host pypi.org --trusted-host files.pythonhosted.org pip setuptools wheel python-certifi-win32
and we're good to go, but poetry doesn't seem to respect these settings. I think it doesn't take advantage of whatever python-certifi-win32 does or pay attention to trusted hosts.
When I try to add a package via poetry, this happens:
poetry add pysmb -vvv
Loading configuration file C:\Users\me\AppData\Roaming\pypoetry\config.toml
Loading configuration file C:\Users\me\AppData\Roaming\pypoetry\auth.toml
Using virtualenv: D:\p4_root\WWHPRP-vnZExBYp_crypy_secrets_2822\venv
[keyring.backend] Loading KWallet
[keyring.backend] Loading SecretService
[keyring.backend] Loading Windows
[keyring.backend] Loading chainer
[keyring.backend] Loading libsecret
[keyring.backend] Loading macOS
Creating new session for [pypi.org](http://pypi.org/)
[urllib3.connectionpool] Starting new HTTPS connection (1): [pypi.org:443](http://pypi.org:443/)
Retrying HTTP request in 0.5 seconds.
[urllib3.connectionpool] Starting new HTTPS connection (2): [pypi.org:443](http://pypi.org:443/)
Retrying HTTP request in 1.0 seconds.
[urllib3.connectionpool] Starting new HTTPS connection (3): [pypi.org:443](http://pypi.org:443/)
Retrying HTTP request in 1.5 seconds.
[urllib3.connectionpool] Starting new HTTPS connection (4): [pypi.org:443](http://pypi.org:443/)
Retrying HTTP request in 2.0 seconds.
[urllib3.connectionpool] Starting new HTTPS connection (5): [pypi.org:443](http://pypi.org:443/)
Retrying HTTP request in 2.5 seconds.
[urllib3.connectionpool] Starting new HTTPS connection (6): [pypi.org:443](http://pypi.org:443/)
Stack trace:
8 ~\AppData\Roaming\pypoetry\venv\lib\site-packages\urllib3\connectionpool.py:703 in urlopen
701│
702│ # Make the request on the httplib connection object.
→ 703│ httplib_response = self._make_request(
704│ conn,
705│ method,
7 ~\AppData\Roaming\pypoetry\venv\lib\site-packages\urllib3\connectionpool.py:386 in _make_request
384│ # Trigger any extra validation we need to do.
385│ try:
→ 386│ self._validate_conn(conn)
387│ except (SocketTimeout, BaseSSLError) as e:
388│ # Py2 raises this as a BaseSSLError, Py3 raises it as socket timeout.
6 ~\AppData\Roaming\pypoetry\venv\lib\site-packages\urllib3\connectionpool.py:1042 in _validate_conn
1040│ # Force connect early to allow us to validate the connection.
1041│ if not getattr(conn, "sock", None): # AppEngine might not have `.sock`
→ 1042│ conn.connect()
1043│
1044│ if not conn.is_verified:
5 ~\AppData\Roaming\pypoetry\venv\lib\site-packages\urllib3\connection.py:414 in connect
412│ context.load_default_certs()
413│
→ 414│ self.sock = ssl_wrap_socket(
415│ sock=conn,
416│ keyfile=self.key_file,
4 ~\AppData\Roaming\pypoetry\venv\lib\site-packages\urllib3\util\ssl_.py:449 in ssl_wrap_socket
447│
448│ if send_sni:
→ 449│ ssl_sock = _ssl_wrap_socket_impl(
450│ sock, context, tls_in_tls, server_hostname=server_hostname
451│ )
3 ~\AppData\Roaming\pypoetry\venv\lib\site-packages\urllib3\util\ssl_.py:493 in _ssl_wrap_socket_impl
491│
492│ if server_hostname:
→ 493│ return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
494│ else:
495│ return ssl_context.wrap_socket(sock)
2 ~\AppData\Local\Programs\Python\Python39\lib\ssl.py:501 in wrap_socket
499│ # SSLSocket class handles server_hostname encoding before it calls
500│ # ctx._wrap_socket()
→ 501│ return self.sslsocket_class._create(
502│ sock=sock,
503│ server_side=server_side,
1 ~\AppData\Local\Programs\Python\Python39\lib\ssl.py:1041 in _create
1039│ # non-blocking
1040│ raise ValueError("do_handshake_on_connect should not be specified for non-blocking sockets")
→ 1041│ self.do_handshake()
1042│ except (OSError, ValueError):
1043│ self.close()
SSLCertVerificationError
[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1129)
at ~\AppData\Local\Programs\Python\Python39\lib\ssl.py:1310 in do_handshake
1306│ timeout = self.gettimeout()
1307│ try:
1308│ if timeout == 0.0 and block:
1309│ self.settimeout(None)
→ 1310│ self._sslobj.do_handshake()
1311│ finally:
1312│ self.settimeout(timeout)
1313│
1314│ def _real_connect(self, addr, connect_ex):
The following error occurred when trying to handle this error:
Stack trace:
2 ~\AppData\Roaming\pypoetry\venv\lib\site-packages\requests\adapters.py:489 in send
487│ try:
488│ if not chunked:
→ 489│ resp = conn.urlopen(
490│ method=request.method,
491│ url=url,
1 ~\AppData\Roaming\pypoetry\venv\lib\site-packages\urllib3\connectionpool.py:787 in urlopen
785│ e = ProtocolError("Connection aborted.", e)
786│
→ 787│ retries = retries.increment(
788│ method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
789│ )
MaxRetryError
HTTPSConnectionPool(host='[pypi.org](http://pypi.org/)', port=443): Max retries exceeded with url: /pypi/pysmb/json (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1129)')))
at ~\AppData\Roaming\pypoetry\venv\lib\site-packages\urllib3\util\retry.py:592 in increment
588│ history=history,
589│ )
590│
591│ if new_retry.is_exhausted():
→ 592│ raise MaxRetryError(_pool, url, error or ResponseError(cause))
593│
594│ log.debug("Incremented Retry for (url='%s'): %r", url, new_retry)
595│
596│ return new_retry
The following error occurred when trying to handle this error:
Stack trace:
25 ~\AppData\Roaming\pypoetry\venv\lib\site-packages\cleo\application.py:329 in run
327│
328│ try:
→ 329│ exit_code = self._run(io)
330│ except Exception as e:
331│ if not self._catch_exceptions:
24 ~\AppData\Roaming\pypoetry\venv\lib\site-packages\poetry\console\application.py:185 in _run
183│ self._load_plugins(io)
184│
→ 185│ exit_code: int = super()._run(io)
186│ return exit_code
187│
23 ~\AppData\Roaming\pypoetry\venv\lib\site-packages\cleo\application.py:423 in _run
421│ io.input.set_stream(stream)
422│
→ 423│ exit_code = self._run_command(command, io)
424│ self._running_command = None
425│
22 ~\AppData\Roaming\pypoetry\venv\lib\site-packages\cleo\application.py:465 in _run_command
463│
464│ if error is not None:
→ 465│ raise error
466│
467│ return event.exit_code
21 ~\AppData\Roaming\pypoetry\venv\lib\site-packages\cleo\application.py:449 in _run_command
447│
448│ if event.command_should_run():
→ 449│ exit_code = command.run(io)
450│ else:
451│ exit_code = ConsoleCommandEvent.RETURN_CODE_DISABLED
20 ~\AppData\Roaming\pypoetry\venv\lib\site-packages\cleo\commands\base_command.py:119 in run
117│ io.input.validate()
118│
→ 119│ status_code = self.execute(io)
120│
121│ if status_code is None:
19 ~\AppData\Roaming\pypoetry\venv\lib\site-packages\cleo\commands\command.py:83 in execute
81│
82│ try:
→ 83│ return self.handle()
84│ except KeyboardInterrupt:
85│ return 1
18 ~\AppData\Roaming\pypoetry\venv\lib\site-packages\poetry\console\commands\add.py:158 in handle
156│ return 0
157│
→ 158│ requirements = self._determine_requirements(
159│ packages,
160│ allow_prereleases=self.option("allow-prereleases"),
17 ~\AppData\Roaming\pypoetry\venv\lib\site-packages\poetry\console\commands\init.py:363 in _determine_requirements
361│ elif "version" not in requirement:
362│ # determine the best version automatically
→ 363│ name, version = self._find_best_version_for_package(
364│ requirement["name"],
365│ allow_prereleases=allow_prereleases,
16 ~\AppData\Roaming\pypoetry\venv\lib\site-packages\poetry\console\commands\init.py:398 in _find_best_version_for_package
396│
397│ selector = VersionSelector(self._get_pool())
→ 398│ package = selector.find_best_candidate(
399│ name, required_version, allow_prereleases=allow_prereleases, source=source
400│ )
15 ~\AppData\Roaming\pypoetry\venv\lib\site-packages\poetry\version\version_selector.py:39 in find_best_candidate
37│ },
38│ )
→ 39│ candidates = self._pool.find_packages(dependency)
40│ only_prereleases = all(c.version.is_unstable() for c in candidates)
41│
14 ~\AppData\Roaming\pypoetry\venv\lib\site-packages\poetry\repositories\pool.py:181 in find_packages
179│ packages = []
180│ for repo in self._repositories:
→ 181│ packages += repo.find_packages(dependency)
182│
183│ return packages
13 ~\AppData\Roaming\pypoetry\venv\lib\site-packages\poetry\repositories\repository.py:46 in find_packages
44│ ignored_pre_release_packages = []
45│
→ 46│ for package in self._find_packages([dependency.name](http://dependency.name/), constraint):
47│ if package.yanked and not isinstance(constraint, Version):
48│ # PEP 592: yanked files are always ignored, unless they are the only
12 ~\AppData\Roaming\pypoetry\venv\lib\site-packages\poetry\repositories\pypi_repository.py:117 in _find_packages
115│ """
116│ try:
→ 117│ info = self.get_package_info(name)
118│ except PackageNotFound:
119│ self._log(
11 ~\AppData\Roaming\pypoetry\venv\lib\site-packages\poetry\repositories\pypi_repository.py:105 in get_package_info
103│ return self._get_package_info(name)
104│
→ 105│ package_info: dict[str, Any] = self._cache.store("packages").remember_forever(
106│ name, lambda: self._get_package_info(name)
107│ )
10 ~\AppData\Roaming\pypoetry\venv\lib\site-packages\cachy\repository.py:174 in remember_forever
172│ return val
173│
→ 174│ val = value(callback)
175│
176│ self.forever(key, val)
9 ~\AppData\Roaming\pypoetry\venv\lib\site-packages\cachy\helpers.py:6 in value
4│ def value(val):
5│ if callable(val):
→ 6│ return val()
7│
8│ return val
8 ~\AppData\Roaming\pypoetry\venv\lib\site-packages\poetry\repositories\pypi_repository.py:106 in <lambda>
104│
105│ package_info: dict[str, Any] = self._cache.store("packages").remember_forever(
→ 106│ name, lambda: self._get_package_info(name)
107│ )
108│ return package_info
7 ~\AppData\Roaming\pypoetry\venv\lib\site-packages\poetry\repositories\pypi_repository.py:156 in _get_package_info
154│
155│ def _get_package_info(self, name: NormalizedName) -> dict[str, Any]:
→ 156│ data = self._get(f"pypi/{name}/json")
157│ if data is None:
158│ raise PackageNotFound(f"Package [{name}] not found.")
6 ~\AppData\Roaming\pypoetry\venv\lib\site-packages\poetry\repositories\pypi_repository.py:244 in _get
242│ def _get(self, endpoint: str) -> dict[str, Any] | None:
243│ try:
→ 244│ json_response = self.session.get(
245│ self._base_url + endpoint,
246│ raise_for_status=False,
5 ~\AppData\Roaming\pypoetry\venv\lib\site-packages\poetry\utils\authenticator.py:246 in get
244│
245│ def get(self, url: str, **kwargs: Any) -> requests.Response:
→ 246│ return self.request("get", url, **kwargs)
247│
248│ def post(self, url: str, **kwargs: Any) -> requests.Response:
4 ~\AppData\Roaming\pypoetry\venv\lib\site-packages\poetry\utils\authenticator.py:228 in request
226│ except (requests.exceptions.ConnectionError, OSError) as e:
227│ if is_last_attempt:
→ 228│ raise e
229│ else:
230│ if resp.status_code not in [502, 503, 504] or is_last_attempt:
3 ~\AppData\Roaming\pypoetry\venv\lib\site-packages\poetry\utils\authenticator.py:225 in request
223│ is_last_attempt = attempt >= 5
224│ try:
→ 225│ resp = session.send(prepared_request, **send_kwargs)
226│ except (requests.exceptions.ConnectionError, OSError) as e:
227│ if is_last_attempt:
2 ~\AppData\Roaming\pypoetry\venv\lib\site-packages\requests\sessions.py:701 in send
699│
700│ # Send the request
→ 701│ r = adapter.send(request, **kwargs)
702│
703│ # Total elapsed time of the request (approximately)
1 ~\AppData\Roaming\pypoetry\venv\lib\site-packages\cachecontrol\adapter.py:57 in send
55│ request.headers.update(self.controller.conditional_headers(request))
56│
→ 57│ resp = super(CacheControlAdapter, self).send(request, **kw)
58│
59│ return resp
SSLError
HTTPSConnectionPool(host='[pypi.org](http://pypi.org/)', port=443): Max retries exceeded with url: /pypi/pysmb/json (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1129)')))
at ~\AppData\Roaming\pypoetry\venv\lib\site-packages\requests\adapters.py:563 in send
559│ raise ProxyError(e, request=request)
560│
561│ if isinstance(e.reason, _SSLError):
562│ # This branch is for urllib3 v1.22 and later.
→ 563│ raise SSLError(e, request=request)
564│
565│ raise ConnectionError(e, request=request)
566│
567│ except ClosedPoolError as e:
So I tried running poetry config certificates.pypi.cert false where pypi is the name of the default pypi repo.
However even after setting this, I still get:
HTTPSConnectionPool(host='[pypi.org](http://pypi.org/)', port=443): Max retries exceeded with url: /pypi/pysmb/json (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1129)')))
I've tried modifying the source code of requests and hardcoding verify = False around the send method, and this works haha. How can I get poetry to ignore SSL using already built-in functionality? Yes I know I'm vulnerable to man-in-the-middle attacks by doing that, but the question is how to do it, not if it's a good idea or not :)