Model producing nan for loss, ema_loss and grad_norm after ~7000 iterations #21
Description
Hi
I was trying to train your model on my own scRef. Unfortunately in three runs it started showing loss:nan | ema_loss:nan | grad_norm:nan after ~7000 iterations.
For the first run I applied minimal pre-processing to my scRef (subsetting to subject of interest, excluding cells with unknown label, and keeping only ~3000 pre-selected genes). This run started showing nan after 7300 iterations and did not recover for the next ~8000 iterations until I manually interrupted it.
For the second run I then applied stricter pre-processing (additionally to steps from run 1: removing zero-sum genes, applying normalization (scanpy's normalize_total and log1p), removing label groups with less than 25 cells and subsampling big label groups to 3000). This run started showing nan after 7100 iterations and did not recover until I stopped it ~5000 iterations later.
For the third run I took the scRef with the stricter pre-processing applied and changed the default settings for sigma_begin, sigma_end and step_lr (as you did for the Heart_D2 scRef). This run also resutled in nan after 7100 iterations and did not recover until for the next 5000 iterations, so I stopped it again.
I am running the training on four Tesla P100 GPUs inside a singularity shell based on the docker image you provided.
Could you assist me in fixing this issue?
Please find my logs and additional information below.
Best
Michael
Session Info
singularity inspect spatialscope.sif
org.label-schema.build-arch: amd64
org.label-schema.build-date: Monday_18_August_2025_16:0:6_CEST
org.label-schema.schema-version: 1.0
org.label-schema.usage.singularity.deffile.bootstrap: docker
org.label-schema.usage.singularity.deffile.from: xiaojs95/spatialscope
org.label-schema.usage.singularity.version: 3.8.1-1.el8
Python:
Version: 3.9.15 | packaged by conda-forge | (main, Nov 22 2022, 08:45:29) [GCC 10.4.0]
Platform:
Linux-4.18.0-305.12.1.el8_4.x86_64-x86_64-with-glibc2.31
Torch:
Version: 1.12.1+cu113
CUDA available: True
CUDA version (pytorch): 11.3
cuDNN version: 8302
GPU:
GPU 0: Tesla P100-PCIE-16GB
GPU 1: Tesla P100-PCIE-16GB
GPU 2: Tesla P100-PCIE-16GB
GPU 3: Tesla P100-PCIE-16GB
System Level:
NVIDIA-SMI 470.57.02 Driver Version: 470.57.02 CUDA Version: 11.4
Packages:
# packages in environment at /opt/conda/envs/SpatialScope:
#
# Name Version Build Channel
_libgcc_mutex 0.1 conda_forge conda-forge
_openmp_mutex 4.5 2_gnu conda-forge
abseil-cpp 20210324.2 h9c3ff4c_0 conda-forge
absl-py 2.1.0 pyhd8ed1ab_0 conda-forge
aiohttp 3.9.1 py39hd1e30aa_0 conda-forge
aiosignal 1.3.1 pyhd8ed1ab_0 conda-forge
alsa-lib 1.2.8 h166bdaf_0 conda-forge
anndata 0.8.0 pyhd8ed1ab_1 conda-forge
annotated-types 0.6.0 pyhd8ed1ab_0 conda-forge
aom 3.5.0 h27087fc_0 conda-forge
arpack 3.7.0 hdefa2d7_2 conda-forge
arrow-cpp 9.0.0 py39h763f843_2_cpu conda-forge
asciitree 0.3.3 py_2 conda-forge
astunparse 1.6.3 pyhd8ed1ab_0 conda-forge
async-timeout 4.0.3 pyhd8ed1ab_0 conda-forge
attr 2.5.1 h166bdaf_1 conda-forge
attrs 23.2.0 pyh71513ae_0 conda-forge
aws-c-cal 0.5.11 h95a6274_0 conda-forge
aws-c-common 0.6.2 h7f98852_0 conda-forge
aws-c-event-stream 0.2.7 h3541f99_13 conda-forge
aws-c-io 0.10.5 hfb6a706_0 conda-forge
aws-checksums 0.1.11 ha31a3da_7 conda-forge
aws-sdk-cpp 1.8.186 hecaee15_4 conda-forge
blinker 1.7.0 pyhd8ed1ab_0 conda-forge
blosc 1.21.5 h0f2a231_0 conda-forge
bokeh 3.3.3 pyhd8ed1ab_0 conda-forge
brotli 1.0.9 h166bdaf_9 conda-forge
brotli-bin 1.0.9 h166bdaf_9 conda-forge
brotli-python 1.0.9 py39h5a03fae_9 conda-forge
brunsli 0.1 h9c3ff4c_0 conda-forge
bzip2 1.0.8 hd590300_5 conda-forge
c-ares 1.25.0 hd590300_0 conda-forge
c-blosc2 2.12.0 hb4ffafa_0 conda-forge
ca-certificates 2023.11.17 hbcca054_0 conda-forge
cached-property 1.5.2 hd8ed1ab_1 conda-forge
cached_property 1.5.2 pyha770c72_1 conda-forge
cachetools 5.3.2 pyhd8ed1ab_0 conda-forge
cairo 1.16.0 ha61ee94_1014 conda-forge
certifi 2023.11.17 pyhd8ed1ab_0 conda-forge
cffi 1.16.0 py39h7a31438_0 conda-forge
cfitsio 4.2.0 hd9d235c_0 conda-forge
charls 2.4.2 h59595ed_0 conda-forge
charset-normalizer 3.3.2 pyhd8ed1ab_0 conda-forge
click 8.0.4 pypi_0 pypi
cloudpickle 3.0.0 pyhd8ed1ab_0 conda-forge
colorama 0.4.6 pyhd8ed1ab_0 conda-forge
contourpy 1.2.0 py39h7633fee_0 conda-forge
cryptography 39.0.0 py39hd598818_0 conda-forge
csbdeep 0.7.4 py39hf3d152e_1 conda-forge
cuda-version 11.8 h70ddcb2_2 conda-forge
cudatoolkit 11.8.0 h4ba93d1_12 conda-forge
cudnn 8.8.0.121 hcdd5f01_4 conda-forge
cvxopt 1.3.2 py39hef4d4b4_1 conda-forge
cycler 0.12.1 pyhd8ed1ab_0 conda-forge
cython 3.0.7 py39h3d6467e_0 conda-forge
cytoolz 0.12.2 py39hd1e30aa_1 conda-forge
dask 2024.1.0 pyhd8ed1ab_0 conda-forge
dask-core 2024.1.0 pyhd8ed1ab_0 conda-forge
dask-image 2023.3.0 pyhd8ed1ab_0 conda-forge
dav1d 1.2.1 hd590300_0 conda-forge
dbus 1.13.6 h5008d03_3 conda-forge
distlib 0.3.8 pypi_0 pypi
distributed 2024.1.0 pyhd8ed1ab_0 conda-forge
docrep 0.3.2 pyh44b312d_0 conda-forge
dsdp 5.8 hd9d9efa_1203 conda-forge
ecos 2.0.11 py39h44dd56e_1 conda-forge
expat 2.5.0 hcb278e6_1 conda-forge
fasteners 0.17.3 pyhd8ed1ab_0 conda-forge
fftw 3.3.10 nompi_hc118613_108 conda-forge
filelock 3.13.1 pypi_0 pypi
font-ttf-dejavu-sans-mono 2.37 hab24e00_0 conda-forge
font-ttf-inconsolata 3.000 h77eed37_0 conda-forge
font-ttf-source-code-pro 2.038 h77eed37_0 conda-forge
font-ttf-ubuntu 0.83 h77eed37_1 conda-forge
fontconfig 2.14.2 h14ed4e7_0 conda-forge
fonts-conda-ecosystem 1 0 conda-forge
fonts-conda-forge 1 0 conda-forge
fonttools 4.47.2 py39hd1e30aa_0 conda-forge
freetype 2.12.1 h267a509_2 conda-forge
frozenlist 1.4.1 py39hd1e30aa_0 conda-forge
fsspec 2023.12.2 pyhca7485f_0 conda-forge
gast 0.5.4 pyhd8ed1ab_0 conda-forge
gettext 0.21.1 h27087fc_0 conda-forge
gflags 2.2.2 he1b5a44_1004 conda-forge
giflib 5.2.1 h0b41bf4_3 conda-forge
glib 2.78.3 hfc55251_0 conda-forge
glib-tools 2.78.3 hfc55251_0 conda-forge
glog 0.6.0 h6f12383_0 conda-forge
glpk 5.0 h445213a_0 conda-forge
gmp 6.3.0 h59595ed_0 conda-forge
google-auth 2.26.2 pyhca7485f_0 conda-forge
google-auth-oauthlib 0.4.6 pyhd8ed1ab_0 conda-forge
google-pasta 0.2.0 pyh8c360ce_0 conda-forge
graphite2 1.3.13 h58526e2_1001 conda-forge
grpc-cpp 1.45.2 h9d3bbbb_5 conda-forge
grpcio 1.43.0 pypi_0 pypi
gsl 2.7 he838d99_0 conda-forge
gst-plugins-base 1.21.3 h4243ec0_1 conda-forge
gstreamer 1.21.3 h25f0c4b_1 conda-forge
gstreamer-orc 0.4.34 hd590300_0 conda-forge
h5py 3.9.0 nompi_py39h4dfffb9_100 conda-forge
harfbuzz 6.0.0 h8e241bc_0 conda-forge
hdf5 1.14.0 nompi_h5231ba7_103 conda-forge
icu 70.1 h27087fc_0 conda-forge
idna 3.6 pyhd8ed1ab_0 conda-forge
igraph 0.10.3 hb9ddf80_0 conda-forge
imagecodecs 2023.1.23 py39hd061359_0 conda-forge
imageio 2.33.1 pyh8c1a49c_0 conda-forge
importlib-metadata 7.0.1 pyha770c72_0 conda-forge
importlib_metadata 7.0.1 hd8ed1ab_0 conda-forge
inflect 7.0.0 pyhd8ed1ab_0 conda-forge
intel-openmp 2022.1.0 h9e868ea_3769 anaconda
jack 1.9.22 h11f4161_0 conda-forge
jinja2 3.1.3 pyhd8ed1ab_0 conda-forge
joblib 1.3.2 pyhd8ed1ab_0 conda-forge
jpeg 9e h0b41bf4_3 conda-forge
jsonschema 4.21.1 pypi_0 pypi
jsonschema-specifications 2023.12.1 pypi_0 pypi
jxrlib 1.1 hd590300_3 conda-forge
keras 2.8.0 pyhd8ed1ab_0 conda-forge
keras-preprocessing 1.1.2 pyhd8ed1ab_0 conda-forge
keyutils 1.6.1 h166bdaf_0 conda-forge
kiwisolver 1.4.5 py39h7633fee_1 conda-forge
krb5 1.20.1 hf9c8cef_0 conda-forge
lame 3.100 h166bdaf_1003 conda-forge
lazy_loader 0.3 pyhd8ed1ab_0 conda-forge
lcms2 2.15 hfd0df8a_0 conda-forge
ld_impl_linux-64 2.40 h41732ed_0 conda-forge
leidenalg 0.9.1 py39h227be39_0 conda-forge
lerc 4.0.0 h27087fc_0 conda-forge
libaec 1.1.2 h59595ed_1 conda-forge
libavif 0.11.1 h8182462_2 conda-forge
libblas 3.9.0 21_linux64_openblas conda-forge
libbrotlicommon 1.0.9 h166bdaf_9 conda-forge
libbrotlidec 1.0.9 h166bdaf_9 conda-forge
libbrotlienc 1.0.9 h166bdaf_9 conda-forge
libcap 2.66 ha37c62d_0 conda-forge
libcblas 3.9.0 21_linux64_openblas conda-forge
libclang 15.0.7 default_hb11cfb5_4 conda-forge
libclang13 15.0.7 default_ha2b6cf4_4 conda-forge
libcrc32c 1.1.2 h9c3ff4c_0 conda-forge
libcups 2.3.3 h36d4200_3 conda-forge
libcurl 7.87.0 h6312ad2_0 conda-forge
libdb 6.2.32 h9c3ff4c_0 conda-forge
libdeflate 1.17 h0b41bf4_0 conda-forge
libedit 3.1.20191231 he28a2e2_2 conda-forge
libev 4.33 hd590300_2 conda-forge
libevent 2.1.10 h9b69904_4 conda-forge
libexpat 2.5.0 hcb278e6_1 conda-forge
libffi 3.4.2 h7f98852_5 conda-forge
libflac 1.4.3 h59595ed_0 conda-forge
libgcc-ng 13.2.0 h807b86a_3 conda-forge
libgcrypt 1.10.3 hd590300_0 conda-forge
libgfortran-ng 13.2.0 h69a702a_3 conda-forge
libgfortran5 13.2.0 ha4646dd_3 conda-forge
libglib 2.78.3 h783c2da_0 conda-forge
libgomp 13.2.0 h807b86a_3 conda-forge
libgoogle-cloud 1.40.2 habd0e3a_0 conda-forge
libgpg-error 1.47 h71f35ed_0 conda-forge
libhwloc 2.9.1 hd6dc26d_0 conda-forge
libiconv 1.17 hd590300_2 conda-forge
liblapack 3.9.0 21_linux64_openblas conda-forge
libllvm14 14.0.6 hcd5def8_4 conda-forge
libllvm15 15.0.7 hadd5161_1 conda-forge
libnghttp2 1.51.0 hdcd2b5c_0 conda-forge
libnsl 2.0.1 hd590300_0 conda-forge
libogg 1.3.4 h7f98852_1 conda-forge
libopenblas 0.3.26 pthreads_h413a1c8_0 conda-forge
libopus 1.3.1 h7f98852_1 conda-forge
libosqp 0.6.3 h6a678d5_0 anaconda
libpng 1.6.39 h753d276_0 conda-forge
libpq 15.1 h2baec63_3 conda-forge
libprotobuf 3.20.3 h3eb15da_0 conda-forge
libqdldl 0.1.7 hcb278e6_0 conda-forge
libsndfile 1.2.2 hc60ed4a_1 conda-forge
libsqlite 3.44.2 h2797004_0 conda-forge
libssh2 1.10.0 haa6b8db_3 conda-forge
libstdcxx-ng 13.2.0 h7e041cc_3 conda-forge
libsystemd0 252 h2a991cd_0 conda-forge
libthrift 0.16.0 h491838f_2 conda-forge
libtiff 4.5.0 h6adf6a1_2 conda-forge
libtool 2.4.7 h27087fc_0 conda-forge
libudev1 253 h0b41bf4_0 conda-forge
libutf8proc 2.8.0 h166bdaf_0 conda-forge
libuuid 2.38.1 h0b41bf4_0 conda-forge
libvorbis 1.3.7 h9c3ff4c_0 conda-forge
libwebp-base 1.3.2 hd590300_0 conda-forge
libxcb 1.13 h7f98852_1004 conda-forge
libxkbcommon 1.5.0 h79f4944_1 conda-forge
libxml2 2.10.3 hca2bb57_4 conda-forge
libzlib 1.2.13 hd590300_5 conda-forge
libzopfli 1.0.3 h9c3ff4c_0 conda-forge
llvmlite 0.41.1 py39h174d805_0 conda-forge
locket 1.0.0 pyhd8ed1ab_0 conda-forge
lz4 4.3.3 py39h79d96da_0 conda-forge
lz4-c 1.9.4 hcb278e6_0 conda-forge
magma 2.5.4 hc72dce7_4 conda-forge
markdown 3.5.2 pyhd8ed1ab_0 conda-forge
markupsafe 2.1.4 py39hd1e30aa_0 conda-forge
matplotlib 3.5.2 py39hf3d152e_1 conda-forge
matplotlib-base 3.5.2 py39h700656a_1 conda-forge
matplotlib-scalebar 0.8.1 pyhd8ed1ab_0 conda-forge
metis 5.1.0 h59595ed_1007 conda-forge
mkl 2022.1.0 hc2b9512_224 anaconda
mpfr 4.2.1 h9458935_0 conda-forge
mpg123 1.32.4 h59595ed_0 conda-forge
msgpack-python 1.0.7 py39h7633fee_0 conda-forge
multidict 6.0.4 py39hd1e30aa_1 conda-forge
munkres 1.1.4 pyh9f0ad1d_0 conda-forge
mysql-common 8.0.32 h14678bc_0 conda-forge
mysql-libs 8.0.32 h54cf53e_0 conda-forge
natsort 8.4.0 pyhd8ed1ab_0 conda-forge
nccl 2.19.4.1 h6103f9b_0 conda-forge
ncurses 6.4 h59595ed_2 conda-forge
networkx 3.2 pyhd8ed1ab_0 conda-forge
ninja 1.11.1 h924138e_0 conda-forge
nspr 4.35 h27087fc_0 conda-forge
nss 3.97 h1d7d5a4_0 conda-forge
numba 0.58.1 py39h615d6bd_0 conda-forge
numcodecs 0.12.1 py39h3d6467e_0 conda-forge
numpy 1.24.4 py39h6183b62_0 conda-forge
oauthlib 3.2.2 pyhd8ed1ab_0 conda-forge
omnipath 1.0.8 pyhd8ed1ab_0 conda-forge
openjpeg 2.5.0 hfec8fc6_2 conda-forge
openssl 1.1.1w hd590300_0 conda-forge
opt_einsum 3.3.0 pyhc1e730c_2 conda-forge
orc 1.7.6 h6c59b99_0 conda-forge
osqp 0.6.3 py39hddac248_2 conda-forge
packaging 23.2 pyhd8ed1ab_0 conda-forge
pandas 1.4.2 py39h1832856_2 conda-forge
parquet-cpp 1.5.1 2 conda-forge
partd 1.4.1 pyhd8ed1ab_0 conda-forge
patsy 0.5.6 pyhd8ed1ab_0 conda-forge
pcre2 10.42 hcad00b1_0 conda-forge
pillow 9.4.0 py39h2320bf1_1 conda-forge
pims 0.6.1 pyhd8ed1ab_1 conda-forge
pip 23.3.2 pyhd8ed1ab_0 conda-forge
pixman 0.43.0 h59595ed_0 conda-forge
platformdirs 4.1.0 pypi_0 pypi
ply 3.11 py_1 conda-forge
protobuf 3.20.3 py39h227be39_1 conda-forge
psutil 5.9.8 py39hd1e30aa_0 conda-forge
pthread-stubs 0.4 h36c2ea0_1001 conda-forge
pulseaudio 16.1 h4ab2085_1 conda-forge
pyarrow 9.0.0 py39h59baa0a_2_cpu conda-forge
pyarrow-hotfix 0.6 pyhd8ed1ab_0 conda-forge
pyasn1 0.5.1 pyhd8ed1ab_0 conda-forge
pyasn1-modules 0.3.0 pyhd8ed1ab_0 conda-forge
pycparser 2.21 pyhd8ed1ab_0 conda-forge
pydantic 1.10.14 pypi_0 pypi
pydantic-core 2.14.6 py39h9fdd4d6_1 conda-forge
pyjwt 2.8.0 pyhd8ed1ab_0 conda-forge
pynndescent 0.5.11 pyhca7485f_0 conda-forge
pyopenssl 23.2.0 pyhd8ed1ab_1 conda-forge
pyparsing 3.1.1 pyhd8ed1ab_0 conda-forge
pyqt 5.15.7 py39h5c7b992_3 conda-forge
pyqt5-sip 12.11.0 py39h227be39_3 conda-forge
pysocks 1.7.1 pyha2e5f31_6 conda-forge
python 3.9.15 h47a2c10_0_cpython conda-forge
python-dateutil 2.8.2 pyhd8ed1ab_0 conda-forge
python-flatbuffers 23.5.26 pyhd8ed1ab_0 conda-forge
python-igraph 0.10.4 py39hfac54ea_0 conda-forge
python_abi 3.9 4_cp39 conda-forge
pytorch 1.12.1 cuda112py39ha0cca9b_200 conda-forge
pytz 2023.3.post1 pyhd8ed1ab_0 conda-forge
pyu2f 0.1.5 pyhd8ed1ab_0 conda-forge
pywavelets 1.4.1 py39h44dd56e_1 conda-forge
pyyaml 6.0.1 py39hd1e30aa_1 conda-forge
qdldl-python 0.1.7.post0 py39hddac248_1 conda-forge
qpsolvers 2.4.0 pyhd8ed1ab_0 conda-forge
qt-main 5.15.6 h18908ee_6 conda-forge
quadprog 0.1.11 py39hf939315_1 conda-forge
ray 2.0.0 pypi_0 pypi
re2 2022.06.01 h27087fc_1 conda-forge
readline 8.2 h8228510_1 conda-forge
referencing 0.32.1 pypi_0 pypi
requests 2.31.0 pyhd8ed1ab_0 conda-forge
requests-oauthlib 1.3.1 pyhd8ed1ab_0 conda-forge
rpds-py 0.17.1 pypi_0 pypi
rsa 4.9 pyhd8ed1ab_0 conda-forge
s2n 1.0.10 h9b69904_0 conda-forge
scanpy 1.9.1 pyhd8ed1ab_0 conda-forge
scikit-image 0.22.0 py39hddac248_2 conda-forge
scikit-learn 1.1.1 py39h4037b75_0 conda-forge
scipy 1.8.1 py39hddc5342_3 conda-forge
seaborn 0.13.1 hd8ed1ab_0 conda-forge
seaborn-base 0.13.1 pyhd8ed1ab_0 conda-forge
session-info 1.0.0 pyhd8ed1ab_0 conda-forge
setuptools 69.0.3 pyhd8ed1ab_0 conda-forge
sip 6.7.12 py39h3d6467e_0 conda-forge
six 1.16.0 pyh6c4a22f_0 conda-forge
sleef 3.5.1 h9b69904_2 conda-forge
slicerator 1.1.0 pyhd8ed1ab_0 conda-forge
snappy 1.1.10 h9fff704_0 conda-forge
sortedcontainers 2.4.0 pyhd8ed1ab_0 conda-forge
sqlite 3.44.2 h2c6b66d_0 conda-forge
squidpy 1.2.2 pyhd8ed1ab_0 conda-forge
stardist 0.8.3 py39h4661b88_0 conda-forge
statsmodels 0.14.1 py39h44dd56e_0 conda-forge
stdlib-list 0.10.0 pyhd8ed1ab_0 conda-forge
suitesparse 5.10.1 h9e50725_1 conda-forge
tbb 2021.9.0 hf52228f_0 conda-forge
tblib 3.0.0 pyhd8ed1ab_0 conda-forge
tensorboard 2.8.0 pyhd8ed1ab_1 conda-forge
tensorboard-data-server 0.6.1 py39hd97740a_4 conda-forge
tensorboard-plugin-wit 1.8.1 pyhd8ed1ab_0 conda-forge
tensorflow 2.8.1 cuda112py39h01bd6f0_0 conda-forge
tensorflow-base 2.8.1 cuda112py39he716a45_0 conda-forge
tensorflow-estimator 2.8.1 cuda112py39hd320b7a_0 conda-forge
termcolor 2.4.0 pyhd8ed1ab_0 conda-forge
texttable 1.7.0 pyhd8ed1ab_0 conda-forge
threadpoolctl 3.2.0 pyha21a80b_0 conda-forge
tifffile 2023.8.12 pyhd8ed1ab_0 conda-forge
tk 8.6.13 noxft_h4845f30_101 conda-forge
toml 0.10.2 pyhd8ed1ab_0 conda-forge
tomli 2.0.1 pyhd8ed1ab_0 conda-forge
toolz 0.12.0 pyhd8ed1ab_0 conda-forge
tornado 6.3.3 py39hd1e30aa_1 conda-forge
tqdm 4.64.0 pyhd8ed1ab_0 conda-forge
typing-extensions 4.9.0 hd8ed1ab_0 conda-forge
typing_extensions 4.9.0 pyha770c72_0 conda-forge
tzdata 2023d h0c530f3_0 conda-forge
umap-learn 0.5.5 py39hf3d152e_0 conda-forge
unicodedata2 15.1.0 py39hd1e30aa_0 conda-forge
urllib3 2.1.0 pyhd8ed1ab_0 conda-forge
validators 0.22.0 pyhd8ed1ab_0 conda-forge
virtualenv 20.25.0 pypi_0 pypi
werkzeug 3.0.1 pyhd8ed1ab_0 conda-forge
wheel 0.42.0 pyhd8ed1ab_0 conda-forge
wrapt 1.16.0 py39hd1e30aa_0 conda-forge
xarray 2023.12.0 pyhd8ed1ab_0 conda-forge
xcb-util 0.4.0 h516909a_0 conda-forge
xcb-util-image 0.4.0 h166bdaf_0 conda-forge
xcb-util-keysyms 0.4.0 h516909a_0 conda-forge
xcb-util-renderutil 0.3.9 h166bdaf_0 conda-forge
xcb-util-wm 0.4.1 h516909a_0 conda-forge
xkeyboard-config 2.38 h0b41bf4_0 conda-forge
xorg-kbproto 1.0.7 h7f98852_1002 conda-forge
xorg-libice 1.1.1 hd590300_0 conda-forge
xorg-libsm 1.2.4 h7391055_0 conda-forge
xorg-libx11 1.8.4 h0b41bf4_0 conda-forge
xorg-libxau 1.0.11 hd590300_0 conda-forge
xorg-libxdmcp 1.1.3 h7f98852_0 conda-forge
xorg-libxext 1.3.4 h0b41bf4_2 conda-forge
xorg-libxrender 0.9.10 h7f98852_1003 conda-forge
xorg-renderproto 0.11.1 h7f98852_1002 conda-forge
xorg-xextproto 7.3.0 h0b41bf4_1003 conda-forge
xorg-xproto 7.0.31 h7f98852_1007 conda-forge
xyzservices 2023.10.1 pyhd8ed1ab_0 conda-forge
xz 5.2.6 h166bdaf_0 conda-forge
yaml 0.2.5 h7f98852_2 conda-forge
yarl 1.9.4 py39hd1e30aa_0 conda-forge
zarr 2.16.1 pyhd8ed1ab_0 conda-forge
zfp 1.0.1 h59595ed_0 conda-forge
zict 3.0.0 pyhd8ed1ab_0 conda-forge
zipp 3.17.0 pyhd8ed1ab_0 conda-forge
zlib 1.2.13 hd590300_5 conda-forge
zlib-ng 2.0.7 h0b41bf4_0 conda-forge
zstd 1.5.5 hfc55251_0 conda-forge