Network is unreachable during the `model.deploy`

[Deezzir]

I have the following test_build_and_deploy function:

@pytest.mark.abort_on_fail
async def test_build_and_deploy(ops_test: OpsTest):
    charm = await ops_test.build_charm(".")
    model = get_model(ops_test)

    await asyncio.gather(
        model.deploy(charm, application_name=APP_NAME),
        model.wait_for_idle(apps=[APP_NAME], status="blocked", timeout=TIMEOUT),
    )

Run using tox -vve integration -- --model test fails with:

Traceback (most recent call last):
  File "/home/deezzir/Desktop/Repos/Canonical/velero-operator/.tox/integration/lib/python3.12/site-packages/_pytest/runner.py", line 341, in from_call
    result: TResult | None = func()
                             ^^^^^^
  File "/home/deezzir/Desktop/Repos/Canonical/velero-operator/.tox/integration/lib/python3.12/site-packages/_pytest/runner.py", line 242, in <lambda>
    lambda: runtest_hook(item=item, **kwds), when=when, reraise=reraise
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/deezzir/Desktop/Repos/Canonical/velero-operator/.tox/integration/lib/python3.12/site-packages/pluggy/_hooks.py", line 513, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/deezzir/Desktop/Repos/Canonical/velero-operator/.tox/integration/lib/python3.12/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/deezzir/Desktop/Repos/Canonical/velero-operator/.tox/integration/lib/python3.12/site-packages/pluggy/_callers.py", line 139, in _multicall
    raise exception.with_traceback(exception.__traceback__)
  File "/home/deezzir/Desktop/Repos/Canonical/velero-operator/.tox/integration/lib/python3.12/site-packages/pluggy/_callers.py", line 122, in _multicall
    teardown.throw(exception)  # type: ignore[union-attr]
    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/deezzir/Desktop/Repos/Canonical/velero-operator/.tox/integration/lib/python3.12/site-packages/_pytest/threadexception.py", line 92, in pytest_runtest_call
    yield from thread_exception_runtest_hook()
  File "/home/deezzir/Desktop/Repos/Canonical/velero-operator/.tox/integration/lib/python3.12/site-packages/_pytest/threadexception.py", line 68, in thread_exception_runtest_hook
    yield
  File "/home/deezzir/Desktop/Repos/Canonical/velero-operator/.tox/integration/lib/python3.12/site-packages/pluggy/_callers.py", line 122, in _multicall
    teardown.throw(exception)  # type: ignore[union-attr]
    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/deezzir/Desktop/Repos/Canonical/velero-operator/.tox/integration/lib/python3.12/site-packages/_pytest/unraisableexception.py", line 95, in pytest_runtest_call
    yield from unraisable_exception_runtest_hook()
  File "/home/deezzir/Desktop/Repos/Canonical/velero-operator/.tox/integration/lib/python3.12/site-packages/_pytest/unraisableexception.py", line 70, in unraisable_exception_runtest_hook
    yield
  File "/home/deezzir/Desktop/Repos/Canonical/velero-operator/.tox/integration/lib/python3.12/site-packages/pluggy/_callers.py", line 122, in _multicall
    teardown.throw(exception)  # type: ignore[union-attr]
    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/deezzir/Desktop/Repos/Canonical/velero-operator/.tox/integration/lib/python3.12/site-packages/_pytest/logging.py", line 846, in pytest_runtest_call
    yield from self._runtest_for(item, "call")
  File "/home/deezzir/Desktop/Repos/Canonical/velero-operator/.tox/integration/lib/python3.12/site-packages/_pytest/logging.py", line 829, in _runtest_for
    yield
  File "/home/deezzir/Desktop/Repos/Canonical/velero-operator/.tox/integration/lib/python3.12/site-packages/pluggy/_callers.py", line 122, in _multicall
    teardown.throw(exception)  # type: ignore[union-attr]
    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/deezzir/Desktop/Repos/Canonical/velero-operator/.tox/integration/lib/python3.12/site-packages/_pytest/capture.py", line 898, in pytest_runtest_call
    return (yield)
            ^^^^^
  File "/home/deezzir/Desktop/Repos/Canonical/velero-operator/.tox/integration/lib/python3.12/site-packages/pluggy/_callers.py", line 122, in _multicall
    teardown.throw(exception)  # type: ignore[union-attr]
    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/deezzir/Desktop/Repos/Canonical/velero-operator/.tox/integration/lib/python3.12/site-packages/_pytest/skipping.py", line 257, in pytest_runtest_call
    return (yield)
            ^^^^^
  File "/home/deezzir/Desktop/Repos/Canonical/velero-operator/.tox/integration/lib/python3.12/site-packages/pluggy/_callers.py", line 103, in _multicall
    res = hook_impl.function(*args)
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/deezzir/Desktop/Repos/Canonical/velero-operator/.tox/integration/lib/python3.12/site-packages/_pytest/runner.py", line 174, in pytest_runtest_call
    item.runtest()
  File "/home/deezzir/Desktop/Repos/Canonical/velero-operator/.tox/integration/lib/python3.12/site-packages/_pytest/python.py", line 1627, in runtest
    self.ihook.pytest_pyfunc_call(pyfuncitem=self)
  File "/home/deezzir/Desktop/Repos/Canonical/velero-operator/.tox/integration/lib/python3.12/site-packages/pluggy/_hooks.py", line 513, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/deezzir/Desktop/Repos/Canonical/velero-operator/.tox/integration/lib/python3.12/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/deezzir/Desktop/Repos/Canonical/velero-operator/.tox/integration/lib/python3.12/site-packages/pluggy/_callers.py", line 182, in _multicall
    return outcome.get_result()
           ^^^^^^^^^^^^^^^^^^^^
  File "/home/deezzir/Desktop/Repos/Canonical/velero-operator/.tox/integration/lib/python3.12/site-packages/pluggy/_result.py", line 100, in get_result
    raise exc.with_traceback(exc.__traceback__)
  File "/home/deezzir/Desktop/Repos/Canonical/velero-operator/.tox/integration/lib/python3.12/site-packages/pluggy/_callers.py", line 103, in _multicall
    res = hook_impl.function(*args)
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/deezzir/Desktop/Repos/Canonical/velero-operator/.tox/integration/lib/python3.12/site-packages/_pytest/python.py", line 159, in pytest_pyfunc_call
    result = testfunction(**testargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/deezzir/Desktop/Repos/Canonical/velero-operator/.tox/integration/lib/python3.12/site-packages/pytest_asyncio/plugin.py", line 529, in inner
    _loop.run_until_complete(task)
  File "/usr/lib/python3.12/asyncio/base_events.py", line 687, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/home/deezzir/Desktop/Repos/Canonical/velero-operator/tests/integration/test_s3_relation.py", line 43, in test_build_and_deploy
    await asyncio.gather(
  File "/home/deezzir/Desktop/Repos/Canonical/velero-operator/.tox/integration/lib/python3.12/site-packages/juju/model/__init__.py", line 1989, in deploy
    identifier = await self.add_local_charm_dir(charm_dir, charm_series)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/deezzir/Desktop/Repos/Canonical/velero-operator/.tox/integration/lib/python3.12/site-packages/juju/model/__init__.py", line 923, in add_local_charm_dir
    charm_url = await loop.run_in_executor(None, func)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/deezzir/Desktop/Repos/Canonical/velero-operator/.tox/integration/lib/python3.12/site-packages/juju/model/__init__.py", line 953, in add_local_charm
    conn.request("POST", path, charm_file, headers)
  File "/usr/lib/python3.12/http/client.py", line 1336, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib/python3.12/http/client.py", line 1382, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.12/http/client.py", line 1331, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.12/http/client.py", line 1091, in _send_output
    self.send(msg)
  File "/usr/lib/python3.12/http/client.py", line 1035, in send
    self.connect()
  File "/usr/lib/python3.12/http/client.py", line 1470, in connect
    super().connect()
  File "/usr/lib/python3.12/http/client.py", line 1001, in connect
    self.sock = self._create_connection(
                ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/socket.py", line 865, in create_connection
    raise exceptions[0]
  File "/usr/lib/python3.12/socket.py", line 850, in create_connection
    sock.connect(sa)
OSError: [Errno 101] Network is unreachable

It only occurs during the deploy step, and other tests involving interacting with the model work fine. My current workaround is to build and deploy manually, and then run the other tests.

I am using Canonical K8s 1.32/stable and Juju 3.6.5-genericlinux-amd64

Note: Deploying a non-local charm works fine as well, so the error only occurs during the deployment of a locally built charm

[addyess]

Thank you for using pytest-operator, but it looks like the issue you're incurring shows up by using add_local_charm_dir(...) from the juju library. Can you let me know which version of the python juju package is being used:

tox -e integration -- pip freeze | grep juju

and perhaps raise the issue against https://github.com/juju/python-libjuju

[Deezzir]

Thanks for the reply.

Juju version is 3.6.1.1

Also, looking at the logs, the library uses the correct IP, Port and model UUID: 10.152.183.81 17070 /model/dbb4020d-67bd-4048-856b-5db11fd0d0ef/charms?series=, so I it seems like a Juju library issue

[Deezzir]

juju/python-libjuju#1263