Hugging Face cache control? #68
Description
The latest error I run into, branching from #63, is again related to attempting to avoid using my home directory
During the curation step
CURATION
Curating recording: block0_acquisition-ElectricalSeriesRaw_recording1
Curation query: isi_violations_ratio < 0.5 and presence_ratio > 0.8 and amplitude_cutoff < 0.1
Passing default QC: 18 / 18
Applying noise-neural classifier from SpikeInterface/UnitRefine_noise_neural_classifier
Command error:
WARNING: While bind mounting '/orcd/data/dandi/001/test_aind/work:/orcd/data/dandi/001/test_aind/work': destination is already in the mount point list
Cloning into 'capsule-repo'...
Switched to a new branch 'engaging_fixes'
+ python -u run_capsule.py
Traceback (most recent call last):
File "capsule/code/run_capsule.py", line 202, in <module>
noise_neuron_labels = scur.auto_label_units(
^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.11/site-packages/spikeinterface/curation/model_based_curation.py", line 268, in auto_label_units
model, model_info = load_model(
^^^^^^^^^^^
File "/opt/conda/lib/python3.11/site-packages/spikeinterface/curation/model_based_curation.py", line 321, in load_model
model, model_info = _load_model_from_huggingface(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.11/site-packages/spikeinterface/curation/model_based_curation.py", line 347, in _load_model_from_huggingface
full_path = hf_hub_download(repo_id=repo_id, filename=filename)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn
return fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 1010, in hf_hub_download
return _hf_hub_download_to_cache_dir(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 1127, in _hf_hub_download_to_cache_dir
os.makedirs(os.path.dirname(blob_path), exist_ok=True)
File "<frozen os>", line 215, in makedirs
File "<frozen os>", line 215, in makedirs
File "<frozen os>", line 215, in makedirs
[Previous line repeated 2 more times]
File "<frozen os>", line 225, in makedirs
OSError: [Errno 30] Read-only file system: '/home/<user>'Similar to the approach with Numba, I would also like to setup a custom Hugging Face cache under the workdir but am unsure how to tell the pipeline not to use the default
I assume SpikeInterface exposes some option for this?
cc: @alejoe91 @yarikoptic