No retries for constructing IterStream in Client.open()

[ZhengKeli]

I encountered a crash caused by a ConnectTimeout error in a long-running processing task, which read files from cloud storage in streaming mode. The error is likely caused by poor network connection. After checking the source code, I found that Client has a retry argument, which is applied in serval methods, but not including Client.open(). Although the opened stream is auto recovered from network errors inside iter_url(), any error raised from first-time construction is not handled.

My problem can be quickly solved via setting a longer timeout. But this work around still cannot handle other kind of network error. Maybe handling such errors in Client.open() via the existing retry mechanism is a good solution?

I'll soon launch a pull request about this. Since I am not very familiar with this library, my solution may not conform to the original design. Any comments or discussions are welcome.

The error logs is attached here:

( ... trancated, because it is too long)

  File "/root/Develop/zkl-llmpt-llama3-synergy/.venv/lib/python3.12/site-packages/pyarrow/fs.py", line 421, in open_input_file
    return PythonFile(self.fs.open(path, mode="rb"), mode="r")
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/Develop/zkl-llmpt-llama3-synergy/.venv/lib/python3.12/site-packages/fsspec/implementations/dirfs.py", line 372, in open
    return self.fs.open(
           ^^^^^^^^^^^^^
  File "/root/Develop/zkl-llmpt-llama3-synergy/.venv/lib/python3.12/site-packages/fsspec/spec.py", line 1310, in open
    f = self._open(
        ^^^^^^^^^^^
  File "/root/miniconda3/lib/python3.12/contextlib.py", line 81, in inner
    return func(*args, **kwds)
           ^^^^^^^^^^^^^^^^^^^
  File "/root/Develop/zkl-llmpt-llama3-synergy/.venv/lib/python3.12/site-packages/webdav4/fsspec.py", line 289, in _open
    return WebdavFile(
           ^^^^^^^^^^^
  File "/root/Develop/zkl-llmpt-llama3-synergy/.venv/lib/python3.12/site-packages/webdav4/fsspec.py", line 416, in __init__
    self.reader: Union[TextIO, BinaryIO] = self.fobj.__enter__()
                                           ^^^^^^^^^^^^^^^^^^^^^
  File "/root/miniconda3/lib/python3.12/contextlib.py", line 137, in __enter__
    return next(self.gen)
           ^^^^^^^^^^^^^^
  File "/root/Develop/zkl-llmpt-llama3-synergy/.venv/lib/python3.12/site-packages/webdav4/client.py", line 581, in open
    with IterStream(
  File "/root/Develop/zkl-llmpt-llama3-synergy/.venv/lib/python3.12/site-packages/webdav4/stream.py", line 147, in __enter__
    self._initial_response, self._iterator = self._cm.__enter__()
                                             ^^^^^^^^^^^^^^^^^^^^
  File "/root/miniconda3/lib/python3.12/contextlib.py", line 137, in __enter__
    return next(self.gen)
           ^^^^^^^^^^^^^^
  File "/root/Develop/zkl-llmpt-llama3-synergy/.venv/lib/python3.12/site-packages/webdav4/stream.py", line 80, in iter_url
    response = request(client.http, url, pos=pos)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/Develop/zkl-llmpt-llama3-synergy/.venv/lib/python3.12/site-packages/webdav4/stream.py", line 39, in request
    return client.send(req, stream=True, follow_redirects=True)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/Develop/zkl-llmpt-llama3-synergy/.venv/lib/python3.12/site-packages/httpx/_client.py", line 914, in send
    response = self._send_handling_auth(
               ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/Develop/zkl-llmpt-llama3-synergy/.venv/lib/python3.12/site-packages/httpx/_client.py", line 942, in _send_handling_auth
    response = self._send_handling_redirects(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/Develop/zkl-llmpt-llama3-synergy/.venv/lib/python3.12/site-packages/httpx/_client.py", line 979, in _send_handling_redirects
    response = self._send_single_request(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/Develop/zkl-llmpt-llama3-synergy/.venv/lib/python3.12/site-packages/httpx/_client.py", line 1014, in _send_single_request
    response = transport.handle_request(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/Develop/zkl-llmpt-llama3-synergy/.venv/lib/python3.12/site-packages/httpx/_transports/default.py", line 249, in handle_request
    with map_httpcore_exceptions():
  File "/root/miniconda3/lib/python3.12/contextlib.py", line 158, in __exit__
    self.gen.throw(value)
  File "/root/Develop/zkl-llmpt-llama3-synergy/.venv/lib/python3.12/site-packages/httpx/_transports/default.py", line 118, in map_httpcore_exceptions
    raise mapped_exc(message) from exc
httpx.ConnectTimeout: timed out