"RuntimeError: asyncio.run() cannot be called from a running event loop" opening zarr dataset

[atakeigor]

open_virtual_dataset(
        url='s3://iruizatake-hazardmaps-test/HM/spill_2018-01-01.zarr',
        parser=ZarrParser(),
        registry=registry,
        loadable_variables=["time"],
    )

This code gives the error "RuntimeError: asyncio.run() cannot be called from a running event loop".

I can open the zarr without virtualizarr.

I'm guessing this is problem user error?

[rsignell]

This issue came up in our Pangeo workshop here in Bologna today as folks are working on their team projects!

[TomNicholas]

My guess is that you're running this inside a jupyter notebook? It's not possible / a massive pain to run python async code inside a jupyter notebook, and it looks like we use async inside the virtualizarr zarr parser.

The solutions are:

1. Don't use the zarr parser inside jupyter (obviously we want a better solution that that long-term),
2. Add some kind of flag to the zarr parser to allow you to choose whether or not you want to use async concurrency,
3. Add some kind of sync() code like zarr-python itself does so that we get concurrency but it still works inside a running event loop.

There might be other options that I'm not familiar with - I vaguely recall some kind of ipython "magic" command for this too.

cc @norlandrhagen - I didn't realise our use of async would cause this problem!

[rsignell]

Thanks @TomNicholas for the super speedy reply! We will explore these solutions and report back!