Skip to content

Commit bb6959b

Browse files
LucaMarconatoLucaMarconato
authored
Merge pull request #149 from scverse/fix/notebooks
Easier to execute notebooks
2 parents afcba90 + 43bfde8 commit bb6959b

File tree

15 files changed

+1156
-653
lines changed

15 files changed

+1156
-653
lines changed

‎.gitignore‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -46,3 +46,5 @@ notebooks/examples/logs/
4646

4747
# others
4848
node_modules/
49+
notebooks/examples/_latest_run_notebook.ipynb
50+
notebooks/developers_resources/storage_format/_latest_run_notebook.ipynb

‎.readthedocs.yaml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,3 +13,4 @@ python:
1313
path: .
1414
extra_requirements:
1515
- doc
16+
- pre

‎datasets/README.md‎

Lines changed: 53 additions & 48 deletions
Original file line numberDiff line numberDiff line change
@@ -1,35 +1,28 @@
11
# Spatial omics datasets
22

3-
Here you can find all datasets necessary to run the example notebooks already converted to the ZARR file format.
4-
5-
If you want to convert additional datasets check out the scripts available in the [spatialdata sandbox](https://github.com/giovp/spatialdata-sandbox).
6-
7-
:::{note}
8-
S3 URLs cannot be opened directly in a web browser. They should be treated as Zarr stores.
9-
For example, appending `.zgroup` to any of the URLs will allow you to see that file.
10-
:::
11-
12-
13-
| Technology | Sample | File Size | Filename (spatialdata-sandbox) | download data | work with data remotely (**see note below**) | license |
14-
| :---------------------------------------- | :-------------------------------------------------------- | --------: | :----------------------------- | :---------------------------------------------------------------------------------------------- | :----------------------------------------------------------------------------------------- | :---------------- |
15-
| Visium HD | Mouse intestin [^2] | 1 GB | visium_hd_3.0.0_id | [.zarr.zip](https://s3.embl.de/spatialdata/spatialdata-sandbox/visium_hd_3.0.0_io.zip) | [S3](https://s3.embl.de/spatialdata/spatialdata-sandbox/visium_hd_3.0.0_io.zarr/) | CCA |
16-
| Visium | Breast cancer [^3] | 1.5 GB | visium_associated_xenium_io | [.zarr.zip](https://s3.embl.de/spatialdata/spatialdata-sandbox/visium_associated_xenium_io.zip) | [S3](https://s3.embl.de/spatialdata/spatialdata-sandbox/visium_associated_xenium_io.zarr/) | CCA |
17-
| Xenium | Breast cancer [^3] | 2.8 GB | xenium_rep1_io | [.zarr.zip](https://s3.embl.de/spatialdata/spatialdata-sandbox/xenium_rep1_io.zip) | [S3](https://s3.embl.de/spatialdata/spatialdata-sandbox/xenium_rep1_io.zarr/) | CCA |
18-
| Xenium | Breast cancer [^3] | 3.7 GB | xenium_rep2_io | [.zarr.zip](https://s3.embl.de/spatialdata/spatialdata-sandbox/xenium_rep2_io.zip) | [S3](https://s3.embl.de/spatialdata/spatialdata-sandbox/xenium_rep2_io.zarr/) | CCA |
19-
| CyCIF (MCMICRO output) | Small lung adenocarcinoma [^4] | 250 MB | mcmicro_io | [.zarr.zip](https://s3.embl.de/spatialdata/spatialdata-sandbox/mcmicro_io.zip) | [S3](https://s3.embl.de/spatialdata/spatialdata-sandbox/mcmicro_io.zarr/) | CC BY-NC 4.0 DEED |
20-
| MERFISH | Mouse brain [^5] | 50 MB | merfish | [.zarr.zip](https://s3.embl.de/spatialdata/spatialdata-sandbox/merfish.zip) | [S3](https://s3.embl.de/spatialdata/spatialdata-sandbox/merfish.zarr/) | CC0 1.0 DEED |
21-
| MIBI-TOF | Colorectal carcinoma [^6] | 25 MB | mibitof | [.zarr.zip](https://s3.embl.de/spatialdata/spatialdata-sandbox/mibitof.zip) | [S3](https://s3.embl.de/spatialdata/spatialdata-sandbox/mibitof.zarr/) | CC BY 4.0 DEED |
22-
| Imaging Mass Cytometry (Steinbock output) | 4 different cancers (SCCHN, BCC, NSCLC, CRC) [^7][^8][^9] | 820 MB | steinbock_io | [.zarr.zip](https://s3.embl.de/spatialdata/spatialdata-sandbox/steinbock_io.zip) | [S3](https://s3.embl.de/spatialdata/spatialdata-sandbox/steinbock_io.zarr/) | CC BY 4.0 DEED |
23-
| Molecular Cartography (SPArrOW output) | Mouse Liver [^10][^11] | 70 MB | MouseLiver | [.zarr.zip](https://s3.embl.de/spatialdata/spatialdata-sandbox/mouse_liver.zip) | [S3](https://s3.embl.de/spatialdata/spatialdata-sandbox/mouse_liver.zarr) | CC BY 4.0 DEED |
24-
| SpaceM | T cells [^12] | 116 MB | spacem_scseahorse1 | [.zarr.zip](https://s3.embl.de/spatialdata/raw_data/20220121_ScSeahorse1.zip) | NA | CC BY 4.0 DEED |
25-
| SpaceM | Hepa and NIH3T3 cells [^13] | 59 MB | spacem_hepanih3t3 | [.zarr.zip](https://s3.embl.de/spatialdata/spatialdata-sandbox/spacem_helanih3t3.zip) | [.zarr.zip](https://s3.embl.de/spatialdata/spatialdata-sandbox/spacem_helanih3t3.zarr) | CC BY 4.0 DEED |
26-
27-
For the first 3 datasets, we also provide a version of them in which they are all aligned in a common coordinate system, and where we added the cell-type information, as described in our paper, to annotate the Xenium cells.
28-
| Technology | Sample | File Size | Filename (spatialdata-sandbox) | download data | work with data remotely (**see note below**) | license |
29-
| :---------------------------------------- | :-------------------------------------------------------- | --------: | :-------------------------- | :---------------------------------------------------------------------------------------------- | :----------------------------------------------------------------------------------------- | :---------------- |
30-
| Visium | Breast Cancer [^3] | 1.5 GB | visium_associated_xenium_io | [.zarr.zip](https://s3.embl.de/spatialdata/spatialdata-sandbox/visium_associated_xenium_io_aligned.zip) | [S3](https://s3.embl.de/spatialdata/spatialdata-sandbox/visium_associated_xenium_io_aligned.zarr/) | CCA |
31-
| Xenium | Breast Cancer [^3] | 2.8 GB | xenium_rep1_io | [.zarr.zip](https://s3.embl.de/spatialdata/spatialdata-sandbox/xenium_rep1_io_aligned.zip) | [S3](https://s3.embl.de/spatialdata/spatialdata-sandbox/xenium_rep1_io_aligned.zarr/) | CCA |
32-
| Xenium | Breast Cancer [^3] | 3.7 GB | xenium_rep2_io | [.zarr.zip](https://s3.embl.de/spatialdata/spatialdata-sandbox/xenium_rep2_io_aligned.zip) | [S3](https://s3.embl.de/spatialdata/spatialdata-sandbox/xenium_rep2_io_aligned.zarr/) | CCA |
3+
Here you can find all datasets necessary to run the example notebooks already converted
4+
to the SpatialData Zarr file format.
5+
6+
Scripts to convert data from several other technologies into SpatialData Zarr are
7+
available in the [spatialdata sandbox](https://github.com/giovp/spatialdata-sandbox); in
8+
particular:
9+
10+
- CyCIF (MCMICRO output)
11+
- Imaging Mass Cytometry, IMC (Steinbock output)
12+
- seqFISH
13+
14+
| Technology | Sample | File Size | Filename (spatialdata-sandbox) | download data (latest stable release) | license |
15+
|:------------------------------------------|:----------------------------------------------------------|----------:|:-------------------------------|:------------------------------------------------------------------------------------------------|:------------------|
16+
| Visium HD | Mouse intestin [^1] | ~2.4 GB | visium_hd_3.0.0_io | [.zarr.zip](https://s3.embl.de/spatialdata/spatialdata-sandbox/visium_hd_3.0.0_io.zip) | CCA |
17+
| Visium | Breast cancer [^2] | ~1.5 GB | visium_associated_xenium_io | [.zarr.zip](https://s3.embl.de/spatialdata/spatialdata-sandbox/visium_associated_xenium_io.zip) | CCA |
18+
| Xenium | Breast cancer [^2] | ~2.8 GB | xenium_rep1_io | [.zarr.zip](https://s3.embl.de/spatialdata/spatialdata-sandbox/xenium_rep1_io.zip) | CCA |
19+
| Xenium | Lung cancer [^3] | ~5.4 GB | xenium_2.0.0_io | [.zarr.zip](https://s3.embl.de/spatialdata/spatialdata-sandbox/xenium_2.0.0_io.zip) | CCA |
20+
| CyCIF (MCMICRO output) | Small lung adenocarcinoma [^4] | ~250 MB | mcmicro_io | [.zarr.zip](https://s3.embl.de/spatialdata/spatialdata-sandbox/mcmicro_io.zip) | CC BY-NC 4.0 DEED |
21+
| MERFISH | Mouse brain [^5] | ~50 MB | merfish | [.zarr.zip](https://s3.embl.de/spatialdata/spatialdata-sandbox/merfish.zip) | CC0 1.0 DEED |
22+
| MIBI-TOF | Colorectal carcinoma [^6] | ~25 MB | mibitof | [.zarr.zip](https://s3.embl.de/spatialdata/spatialdata-sandbox/mibitof.zip) | CC BY 4.0 DEED |
23+
| Imaging Mass Cytometry (Steinbock output) | 4 different cancers (SCCHN, BCC, NSCLC, CRC) [^7][^8][^9] | ~800 MB | steinbock_io | [.zarr.zip](https://s3.embl.de/spatialdata/spatialdata-sandbox/steinbock_io.zip) | CC BY 4.0 DEED |
24+
| Molecular Cartography (SPArrOW output) | Mouse Liver [^10][^11] | ~70 MB | MouseLiver | [.zarr.zip](https://s3.embl.de/spatialdata/spatialdata-sandbox/mouse_liver.zip) | CC BY 4.0 DEED |
25+
| SpaceM | Hepa and NIH3T3 cells [^12] | ~60 MB | spacem_hepanih3t3 | [.zarr.zip](https://s3.embl.de/spatialdata/spatialdata-sandbox/spacem_helanih3t3.zip) | CC BY 4.0 DEED |
3326

3427
## Licenses abbreviations
3528

@@ -40,39 +33,51 @@ For the first 3 datasets, we also provide a version of them in which they are al
4033

4134
The data retains the license of the original published data.
4235

43-
<!-- to add: raccoon, blobs, "additional resources for methods developers" -->
44-
<!-- Artificial datasets
45-
| Description | File Size| Filename | download data | work with data remotely [^1] |
46-
| :--------------------- | :------------------------- | --------:| :-------------------------- | :---------------------------------------------------------------------------------------------- | :----------------------------------------------------------------------------------------- || - | - | 11 kB| toy | [.zarr.zip](https://s3.embl.de/spatialdata/spatialdata-sandbox/toy.zip) | [S3](https://s3.embl.de/spatialdata/spatialdata-sandbox/toy.zarr/) | -->
47-
4836
# Artificial datasets
4937

50-
Also, here you can find [additional datasets and resources for methods developers](https://github.com/scverse/spatialdata-notebooks/blob/main/notebooks/developers_resources/storage_format/).
38+
Also, here you can
39+
find [additional datasets and resources for methods developers](https://github.com/scverse/spatialdata-notebooks/blob/main/notebooks/developers_resources/storage_format/).
5140

5241
# References
5342

5443
If you use the datasets please cite the original sources and double-check their license.
5544

56-
[^2]: From https://www.10xgenomics.com/datasets/visium-hd-cytassist-gene-expression-libraries-of-mouse-intestine
45+
[^1]:
46+
From https://www.10xgenomics.com/datasets/visium-hd-cytassist-gene-expression-libraries-of-mouse-intestine
5747

58-
[^3]: Janesick, A. et al. High resolution mapping of the breast cancer tumor microenvironment using integrated single cell, spatial and in situ analysis of FFPE tissue. bioRxiv 2022.10.06.510405 (2022) doi:10.1101/2022.10.06.510405.
48+
[^2]: Janesick, A. et al. High resolution mapping of the breast cancer tumor
49+
microenvironment using integrated single cell, spatial and in situ analysis of FFPE
50+
tissue. bioRxiv 2022.10.06.510405 (2022) doi:10.1101/2022.10.06.510405.
5951

60-
[^4]: Schapiro, D. et al. MCMICRO: A scalable, modular image-processing pipeline for multiplexed tissue imaging. Cold Spring Harbor Laboratory 2021.03.15.435473 (2021) doi:10.1101/2021.03.15.435473.
52+
[^3]:
53+
From https://www.10xgenomics.com/datasets/preview-data-ffpe-human-lung-cancer-with-xenium-multimodal-cell-segmentation-1-standard
6154

62-
[^5]: Moffitt, J. R. et al. Molecular, spatial, and functional single-cell profiling of the hypothalamic preoptic region. Science 362, (2018).
55+
[^4]: Schapiro, D. et al. MCMICRO: A scalable, modular image-processing pipeline for
56+
multiplexed tissue imaging. Cold Spring Harbor Laboratory 2021.03.15.435473 (2021) doi:
57+
10.1101/2021.03.15.435473.
6358

64-
[^6]: Hartmann, F. J. et al. Single-cell metabolic profiling of human cytotoxic T cells. Nat. Biotechnol. (2020) doi:10.1038/s41587-020-0651-8.
59+
[^5]: Moffitt, J. R. et al. Molecular, spatial, and functional single-cell profiling of
60+
the hypothalamic preoptic region. Science 362, (2018).
6561

66-
[^7]: Windhager, J., Bodenmiller, B. & Eling, N. An end-to-end workflow for multiplexed image processing and analysis. bioRxiv 2021.11.12.468357 (2021) doi:10.1101/2021.11.12.468357.
62+
[^6]: Hartmann, F. J. et al. Single-cell metabolic profiling of human cytotoxic T cells.
63+
Nat. Biotechnol. (2020) doi:10.1038/s41587-020-0651-8.
6764

68-
[^8]: Eling, N. & Windhager, J. Example imaging mass cytometry raw data. (2022). doi:10.5281/zenodo.5949116.
65+
[^7]: Windhager, J., Bodenmiller, B. & Eling, N. An end-to-end workflow for multiplexed
66+
image processing and analysis. bioRxiv 2021.11.12.468357 (2021) doi:
67+
10.1101/2021.11.12.468357.
6968

70-
[^9]: Eling, N. & Windhager, J. steinbock results of IMC example data. (2022). doi:10.5281/zenodo.7412972.
69+
[^8]: Eling, N. & Windhager, J. Example imaging mass cytometry raw data. (2022). doi:
70+
10.5281/zenodo.5949116.
7171

72-
[^10]: Guilliams, Martin, et al. "Spatial proteogenomics reveals distinct and evolutionarily conserved hepatic macrophage niches." Cell 185.2 (2022) doi:10.1016/j.cell2021.12.018
72+
[^9]: Eling, N. & Windhager, J. steinbock results of IMC example data. (2022). doi:
73+
10.5281/zenodo.7412972.
7374

74-
[^11]: Pollaris, Lotte, et al. "SPArrOW: a flexible, interactive and scalable pipeline for spatial transcriptomics analysis." bioRxiv (2024) doi:10.1101/2024.07.04.601829
75+
[^10]: Guilliams, Martin, et al. "Spatial proteogenomics reveals distinct and
76+
evolutionarily conserved hepatic macrophage niches." Cell 185.2 (2022) doi:
77+
10.1016/j.cell2021.12.018
7578

76-
[^12]: See https://github.com/giovp/spatialdata-sandbox/blob/main/spacem_scseahorse1/README.md
79+
[^11]: Pollaris, Lotte, et al. "SPArrOW: a flexible, interactive and scalable pipeline
80+
for spatial transcriptomics analysis." bioRxiv (2024) doi:10.1101/2024.07.04.601829
7781

78-
[^13]: See https://github.com/giovp/spatialdata-sandbox/blob/main/spacem_helanih3t3/README.md
82+
[^12]:
83+
See https://github.com/giovp/spatialdata-sandbox/blob/main/spacem_helanih3t3/README.md

‎notebooks/developers_resources/storage_format/_latest_run_notebook.ipynb‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -923,4 +923,4 @@
923923
},
924924
"nbformat": 4,
925925
"nbformat_minor": 5
926-
}
926+
}

‎notebooks/developers_resources/storage_format/multiple_elements.zarr/.zattrs‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,4 +3,4 @@
33
"spatialdata_software_version": "0.4.1.dev11+gb889b53",
44
"version": "0.1"
55
}
6-
}
6+
}

‎notebooks/developers_resources/storage_format/multiple_elements.zarr/zmetadata‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1437,4 +1437,4 @@
14371437
}
14381438
},
14391439
"zarr_consolidated_format": 1
1440-
}
1440+
}

‎notebooks/examples/alignment_using_landmarks.ipynb‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -570,7 +570,7 @@
570570
"name": "python",
571571
"nbconvert_exporter": "python",
572572
"pygments_lexer": "ipython3",
573-
"version": "3.10.0"
573+
"version": "3.11.0"
574574
},
575575
"vscode": {
576576
"interpreter": {

‎notebooks/examples/alignment_utils.py‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
from spatialdata.transformations import Affine
2+
from spatialdata import SpatialData
3+
from spatialdata.transformations import (
4+
BaseTransformation,
5+
Sequence,
6+
get_transformation,
7+
set_transformation,
8+
)
9+
10+
11+
AFFINE_VISIUM_XENIUM = Affine(
12+
[
13+
[1.61711846e-01, 2.58258090e00, -1.24575040e04],
14+
[-2.58258090e00, 1.61711846e-01, 3.98647301e04],
15+
[0.0, 0.0, 1.0],
16+
],
17+
input_axes=("x", "y"),
18+
output_axes=("x", "y"),
19+
)
20+
21+
22+
def postpone_transformation(
23+
sdata: SpatialData,
24+
transformation: BaseTransformation,
25+
source_coordinate_system: str,
26+
target_coordinate_system: str,
27+
):
28+
for element_type, element_name, element in sdata._gen_elements():
29+
old_transformations = get_transformation(element, get_all=True)
30+
if source_coordinate_system in old_transformations:
31+
old_transformation = old_transformations[source_coordinate_system]
32+
sequence = Sequence([old_transformation, transformation])
33+
set_transformation(element, sequence, target_coordinate_system)

0 commit comments

Comments
 (0)