Merge pull request #65 from DanielaSchacherer/master

Refinements for ISBI Tutorial

Author: fedorov

notebooks/labs/idc_isbi2024.ipynb

@@ -3,8 +3,8 @@
     {
       "cell_type": "markdown",
       "metadata": {
-        "colab_type": "text",
-        "id": "view-in-github"
+        "id": "view-in-github",
+        "colab_type": "text"
       },
       "source": [
         "<a href=\"https://colab.research.google.com/github/ImagingDataCommons/IDC-Tutorials/blob/master/notebooks/labs/idc_isbi2024.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
@@ -16,7 +16,7 @@
         "id": "ATBH0iwpkHXP"
       },
       "source": [
-        "# Experimenting with AI inference on slide microscopy data in IDC \n",
+        "# Experimenting with AI inference on slide microscopy data in IDC\n",
         "\n",
         "This notebook is part of a [tutorial given at ISBI 2024](https://biomedicalimaging.org/2024/tutorials-final/).\n",
         "It demonstrates how the Imaging Data Commons (IDC) can be used to work with whole slide images (WSIs) and provides an example of the application of deep learning (DL) to computational pathology analysis.\n",
@@ -188,9 +188,10 @@
       "source": [
         "For most computational pathology experiments, the first step is to select a cohort of WSIs by filtering for the desired metadata attributes.\n",
         "\n",
-        "The IDC uses the DICOM standard for data representation. Here, a WSI corresponds to a series of DICOM image objects, each representing the slide at a different resolution. Each DICOM object is stored as a separate DICOM file. Cohort selection is done easiest by executing SQL-like statements using the Python package idc-index against an index table, which lists all available DICOM files (rows) with the corresponding metadata atrributes (columns).\n",
+        "The IDC uses the DICOM standard for data representation. Here, a WSI corresponds to a series of DICOM image objects, each representing the slide at a different resolution. Each DICOM object is stored as a separate DICOM file. Cohort selection is done easiest by executing SQL-like statements using the Python package idc-index against an index table, which lists all available DICOM files (rows) with the corresponding metadata atrributes (columns). For getting started with the idc-index, we refer to this [introductory notebook](https://github.com/ImagingDataCommons/IDC-Tutorials/blob/master/notebooks/getting_started/part2_searching_basics.ipynb).\n",
         "\n",
-        "In the following we retrieve three slides from the CPTAC-LUAD and CPTAC-LSCC collections."
+        "In the following we retrieve three slides from the CPTAC-LUAD and CPTAC-LSCC collections.\n",
+        "\n"
       ]
     },
     {
@@ -220,12 +221,8 @@
         "    index.SeriesInstanceUID as digital_slide_id,\n",
         "    index.StudyInstanceUID as case_id,\n",
         "    (REPLACE (REPLACE(index.collection_id, 'cptac_luad', 'luad'), 'cptac_lscc', 'lscc')) AS cancer_subtype,\n",
-        "    -- The 'tissue_types' subquery indicates whether a slides contains normal, tumor or other tisse.\n",
-        "    CASE sm_index.primaryAnatomicStructureModifier_CodeMeaning\n",
-        "       WHEN 'Normal' THEN 'normal'\n",
-        "       WHEN 'Neoplasm, Primary' THEN 'tumor'\n",
-        "       ELSE 'other' -- meaning e.g.: 'Neoplasm, Metastatic'\n",
-        "    END AS tissue_type,\n",
+        "    -- The 'tissue_types' indicates whether a slides contains normal, tumor or other tisse.\n",
+        "    sm_index.primaryAnatomicStructureModifier_CodeMeaning as tissue_type\n",
         "FROM\n",
         "    index\n",
         "JOIN\n",
@@ -234,7 +231,6 @@
         "    (index.SeriesInstanceUID = '1.3.6.1.4.1.5962.99.1.261553051.626883586.1640939092891.2.0'\n",
         "    OR index.SeriesInstanceUID = '1.3.6.1.4.1.5962.99.1.223072275.1494200661.1640900612115.2.0'\n",
         "    OR index.SeriesInstanceUID = '1.3.6.1.4.1.5962.99.1.255269468.207807501.1640932809308.2.0')\n",
-        "    -- OR index.SeriesInstanceUID = '1.3.6.1.4.1.5962.99.1.237882392.736020130.1640915422232.2.0')\n",
         "    AND index.Modality = 'SM'\n",
         "'''\n",
         "\n",
@@ -365,7 +361,7 @@
         "id": "msi0SjqdLkNM"
       },
       "source": [
-        "**Note: we can of course do much more with wsidicom and openslide. Could be mentioned or shown.**"
+        "Note: we can of course do much more with wsidicom and openslide. Just check out the respective documentations."
       ]
     },
     {
@@ -569,9 +565,9 @@
   ],
   "metadata": {
     "colab": {
-      "include_colab_link": true,
       "provenance": [],
-      "toc_visible": true
+      "toc_visible": true,
+      "include_colab_link": true
     },
     "environment": {
       "kernel": "python3",

notebooks/labs/idc_isbi2024_utils/utils.py

@@ -21,7 +21,7 @@ def _get_reference_class_label(slide_metadata: pd.DataFrame) -> str:
         return tissue_type
     else: 
         return slide_metadata['cancer_subtype']
-
+    
 
 def create_slides_metadata(bq_results_df: pd.DataFrame, local_slides_dir: str) -> Dict[str, Any]: 
     """
@@ -45,10 +45,19 @@ def create_slides_metadata(bq_results_df: pd.DataFrame, local_slides_dir: str) -
 
         if not image_id in slides_metadata:
             slides_metadata[image_id] = slide_metadata
+            
+            # rename tissue type
+            if slides_metadata[image_id]['tissue_type'] == 'Normal':
+                slides_metadata[image_id]['tissue_type'] = 'normal'
+            elif slides_metadata[image_id]['tissue_type'] == 'Neoplasm, Primary':
+                slides_metadata[image_id]['tissue_type'] = 'tumor'
+            else: 
+                slides_metadata[image_id]['tissue_type'] = 'other'
+
             local_path = os.path.join(local_slides_dir, image_id)
             slides_metadata[image_id]['local_path'] = local_path
             slides_metadata[image_id]['reference_class_label'] = _get_reference_class_label(slide_metadata)
 
 
     return pd.DataFrame.from_records(list(slides_metadata.values()),
-                                     index=list(slides_metadata.keys()))
+                                     index=list(slides_metadata.keys()))