-
-
Notifications
You must be signed in to change notification settings - Fork 138
Add imaging studies for all TCGA Pancan separated by modality #2260
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,21 +1,56 @@ | ||
| # OHIF Viewer for TCGA CT Scans | ||
| OHIF URLs for CT Scans were obtained from the Imaging Data Commons and added as | ||
| resource data to all tcga_pan_can_2018 studies, | ||
| # TCGA Imaging Data Integration with cBioPortal | ||
|
|
||
| ## Steps | ||
| ### 1. Download Data from IDC | ||
| This document describes the process of acquiring TCGA imaging studies from the NCI Imaging Data Commons (IDC) and integrating them as patient resources in cBioPortal's TCGA Pan Cancer Atlas 2018 studies. The images can be viewed using OHIF (radiology) and SLIM (pathology) viewers within the portal. | ||
|
|
||
| Go to IDC and select TCGA -> CT Scans. Download a manifest with all the links to ~/Downloads/idc_ohif.tsv. | ||
| ## Data Structure in IDC | ||
|
|
||
| ### 2. Curate coadread_tcga semi-manually | ||
| IDC organizes imaging data hierarchically: | ||
|
|
||
| Add all *resource* files for coad and read data and link patients to samples. This was done semi-manually. | ||
| ``` | ||
| Collections → Patients → Studies → Series → Instances | ||
| ``` | ||
|
|
||
| ### 3. Do the rest using a command one liner | ||
| - **Study**: One complete imaging exam for a patient, regardless of modality (e.g., "CT Chest with Contrast", "MRI Brain"). One patient could have undergone multiple imaging studies over time. | ||
| - **Series**: Individual image batches within a study (e.g., different CT scan sections or sequences) | ||
|
|
||
| Generate them for the rest: | ||
| ```bash | ||
| for f in $(cut -f2 ~/Downloads/idc_ohif.tsv | gsort | uniq | grep tcga_ | grep -v Filters | grep -v coad | grep -v read); do (head -1 coadread_tcga_pan_can_atlas_2018/data_resource_patient.txt; cut -f1,2,4 ~/Downloads/idc_ohif.tsv | tail -n +9 | grep $f | cut -f1,3 | awk -vFS='\t' -vOFS='\t' '{$1=substr($1,0,12); $3="https://viewer.imaging.datacommons.cancer.gov/viewer/"$2; $2="IDC_OHIF_V2"; print $0}' | gsort -k1,1 | uniq | rev | uniq -f2 | rev; ) > ${f/tcga_/}_tcga_pan_can_atlas_2018/*data_resource*patient*; done | ||
| ``` | ||
| **Note**: While a single study may contain multiple imaging modalities, data was extracted at the study level and then split by modality for cBioPortal integration. This modality-level organization allows users to selectively access specific imaging types (e.g., CT scans vs. H&E slides) for each patient. | ||
|
|
||
| ## Available Imaging Modalities | ||
|
|
||
| | Code | Modality | Viewer | Resource ID | Resource Tab Name | | ||
| |------|----------|--------|-------------|-------------------| | ||
| | CR | Computed Radiography | OHIF | IDC_OHIF_CR | Computed Radiography | | ||
| | CT | Computed Tomography | OHIF | IDC_OHIF_CT | CT Scan | | ||
| | DX | Digital Radiography | OHIF | IDC_OHIF_DX | Digital Radiography | | ||
| | MG | Mammography | OHIF | IDC_OHIF_MG | Mammography | | ||
| | MR | Magnetic Resonance | OHIF | IDC_OHIF_MR | Magnetic Resonance | | ||
| | NM | Nuclear Medicine | OHIF | IDC_OHIF_NM | Nuclear Medicine | | ||
| | PT | Positron Emission Tomography | OHIF | IDC_OHIF_PT | PET Scan | | ||
| | SM | Slide Microscopy (H&E) | SLIM | IDC_SLIM | H&E Slide | | ||
|
|
||
| **Note**: Annotation (ANN), Segmentation (SEG) and Structured Report (SR) are excluded as standalone resources since they are automatically loaded with their parent imaging studies in OHIF. Other (OT) modality is also excluded. | ||
|
|
||
| ## Implementation Steps | ||
|
|
||
| ### Generate Resource Files | ||
|
|
||
| Patient-level resource files are generated for each TCGA Pan Cancer study by: | ||
| 1. Downloading TCGA imaging metadata from IDC using the idc-index package | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. You should keep track of the version of IDC data you use for generating those links. You can get it with https://idc-index.readthedocs.io/en/latest/api/idc_index.html#idc_index.index.IDCClient.get_idc_version. It would also be best if you updated your links after each IDC release. Although data removal is rare, it can happen, and would lead to broken viewer links. |
||
| 2. Linking patients to their imaging studies via OHIF and SLIM viewer URLs | ||
|
|
||
| **Script**: [generate_imaging_resources.py](https://github.com/cBioPortal/datahub-study-curation-tools/tree/master/generate_imaging_resources) | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The script referenced uses custom code to generate viewer URLs. I recommend you use this function in |
||
|
|
||
| ## Output Files | ||
|
|
||
| For each TCGA Pan-Cancer Atlas 2018 study, two files are generated: | ||
|
|
||
| 1. **`data_resource_patient.txt`**: Links patients to their imaging studies | ||
| - Columns: `PATIENT_ID`, `RESOURCE_ID`, `URL` | ||
|
|
||
| 2. **`data_resource_definition.txt`**: Defines the resource types available in the study | ||
| - Columns: `RESOURCE_ID`, `DISPLAY_NAME`, `RESOURCE_TYPE`, `DESCRIPTION`, `OPEN_BY_DEFAULT`, `PRIORITY` | ||
|
|
||
| ## Viewer URLs | ||
|
|
||
| Note: there are a few patients that have multiple CT Scans. Not entirely sure what the difference is, the above command just selects the first one | ||
| - **OHIF Viewer** (for radiology imaging): `https://viewer.imaging.datacommons.cancer.gov/viewer/{StudyInstanceUID}` | ||
| - **SLIM Viewer** (for slide microscopy): `https://viewer.imaging.datacommons.cancer.gov/slim/studies/{StudyInstanceUID}` | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The URL patterns you provide at the bottom of this document are at the study level, so this paragraph is confusing.