Commit 59d8dba
committed
ReferenceFileSystem: use fs.open instead of fs._open
There is a bug in `fsspec==2024.12.0` that causes the `ReferenceFileSystem` to
incorrectly make `fs._open` return a coroutine object instead of a file-like object.
(See a proposed PR to fix this issue: fsspec/filesystem_spec#1769.)
We have a test for the expected behavior (`test_arrow_generator_partitioned` in `tests/unit/lib/test_arrow.py`)
running in the CI environment.
But that does not fail because the latest version of `fsspec` does not get installed in the CI
due to the upper limit set by the `datasets` library.
The `datasets` library is only installed as part of the `hf` and `tests` extras,
so the default installation of `datachain` will encounter this issue.
Fixes #806.1 parent aad99e2 commit 59d8dba
1 file changed
+1
-1
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
33 | 33 | | |
34 | 34 | | |
35 | 35 | | |
36 | | - | |
| 36 | + | |
37 | 37 | | |
38 | 38 | | |
39 | 39 | | |
| |||
0 commit comments