Skip to content

Conversation

QuLogic
Copy link
Contributor

@QuLogic QuLogic commented Sep 23, 2024

Similar to test_append_parquet (#1523).

Similar to test_append_parquet.
@martindurant
Copy link
Member

OK, but I don't immediately see why the test needs kerchunk. You are right that this is kerchunk-derived functionality, though.

@martindurant martindurant merged commit 4af4e76 into fsspec:master Sep 23, 2024
11 checks passed
@QuLogic
Copy link
Contributor Author

QuLogic commented Sep 23, 2024

OK, but I don't immediately see why the test needs kerchunk.

Here is the original backtrace I should have posted:

________________________________ test_deep_parq ________________________________
m = <fsspec.implementations.memory.MemoryFileSystem object at 0x7f43626c1be0>
    def test_deep_parq(m):
        zarr = pytest.importorskip("zarr")
        lz = fsspec.implementations.reference.LazyReferenceMapper.create(
            "memory://out.parq", fs=m
        )
        g = zarr.open_group(lz, mode="w")
        g2 = g.create_group("instant")
        g2.create_dataset(name="one", data=[1, 2, 3])
>       lz.flush()
fsspec/implementations/tests/test_reference.py:772: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
fsspec/implementations/reference.py:493: in flush
    self.write(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
self = <fsspec.implementations.reference.LazyReferenceMapper object at 0x7f433a757ed0>
field = 'instant/one', record = 0, base_url = None, storage_options = None
    def write(self, field, record, base_url=None, storage_options=None):
        # extra requirements if writing
>       import kerchunk.df
E       ModuleNotFoundError: No module named 'kerchunk'
fsspec/implementations/reference.py:407: ModuleNotFoundError

@QuLogic QuLogic deleted the skip-kerchunk branch September 23, 2024 19:59
@martindurant
Copy link
Member

OK, I had overlooked that. It could be worked around.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants