Patch iterator compatibility with .geojson annotation files

[mdeleeuw1]

Hi Mart,

I hope all is well. I am trying to perform whole-slide inference on a WSI. I want to use the Patch iterator class so that I can extract adjacent patches in an ordered fashion. I also want to extract the corresponding annotation mask so that I can calculate Dice scores and other metrics. However, the Patch iterator class does not seem to be compatible with .geojson annotations that I have. Is this indeed the case, and is this something that is in the pipeline for this library?

Thanks in advance!

Kind regards,
Mike

[martvanrijthoven]

Dear Mike,

The way you could do it, is via first converting geojson to a tif mask and then the patch iterator can accept that tif mask as input. Eg: https://github.com/DIAGNijmegen/pathology-whole-slide-data/blob/main/scripts/convert_annotations_to_mask.sh

However, it should not be very difficult to implement the same functionality with geojson without the need to first convert to mask. Actually you can already do it with the batchiterator but the settings are then a bit complex.

I am on vacation now, but will either let you know the settings for the batchiterator or implement new feature for the patchiterator, next week.

Best wishes
Mart

[mdeleeuw1]

Dear Mart,

Thank you for getting back to me.

I will check the link you sent later this week, looks promising though.

Kind regards,

Mike

[SuooL]

Dear Mike,

The way you could do it, is via first converting geojson to a tif mask and then the patch iterator can accept that tif mask as input. Eg: https://github.com/DIAGNijmegen/pathology-whole-slide-data/blob/main/scripts/convert_annotations_to_mask.sh

However, it should not be very difficult to implement the same functionality with geojson without the need to first convert to mask. Actually you can already do it with the batchiterator but the settings are then a bit complex.

I am on vacation now, but will either let you know the settings for the batchiterator or implement new feature for the patchiterator, next week.

Best wishes Mart

Hi Mart,
I'm also currently experiencing an issue where batchiterator seems to be incompatible with .geojson format annotations.
Can you explain how to set up the batchiterator so that it is compatible with .geojson annotations?

Looking forward to your answer to this question.

I wish you all the best.

Hu

[martvanrijthoven]

Dear Hu,

To use geojson files in the batchiterator, you will need to set the geojson parser in the batchiterator config like this (among the other settings):

wholeslidedata:
  default:
    annotation_parser:
      "*object": wholeslidedata.interoperability.qupath.parser.QuPathAnnotationParser

If you stil have problems with this, please send me the error and I will be happy to help solve the issue.

Best wishes,
Mart

[mdeleeuw1]

Hi Mart,

Hope all is well.

I have tried to implement the batch_iterator to iterate in a sliding window approach, basing my config file on https://github.com/DIAGNijmegen/pathology-whole-slide-data/blob/main/wholeslidedata/configuration/presets/slidingwindow.yml.

However, the mask does not seem to correspond with the patch extracted from the WSI. I have tried many different configurations, but none of them seem to work for me. You mentioned before that the batch_iterator settings are complicated for a sliding window approach, could you maybe share how this can be achieved?

Thank you!

Kind regards,

Mike

P.S. here is an example of a config file I used:
``wholeslidedata:
default:
seed: 123

    image_backend: openslide

    annotation_parser:
        "*object": wholeslidedata.interoperability.qupath.parser.QuPathAnnotationParser
    
    dataset:
        "*object": wholeslidedata.data.dataset.WholeSlideDataSet
        load_images: True

    _dataset_offset:
        "*object": wholeslidedata.data.dataset.fix_annotation_offsets
        dataset: ${dataset}
    
    batch_shape:
        batch_size: 1
        shape: [512, 512, 3]
        spacing: [16.0]
        y_shape: [512, 512]
    
    annotation_sampler_name: OrderedAnnotationSampler
    annotation_sampler:
        "*object": wholeslidedata.samplers.annotationsampler.${annotation_sampler_name}
    point_sampler_name: TopLeftPointSampler
    patch_sampler:
        center: False
        relative: False


    sample_callbacks:
        - "*object": wholeslidedata.samplers.callbacks.CropSampleCallback
          output_shape: [512, 512]"

[martvanrijthoven]

Dear Mike,

Sorry for my late reply.

If you want to use qupath parser and sliding window you will need to use the TiledAnnotationCallback

It would work something like this:

Basically it converts all your annotations to rectangles with a specified tile size.

you can configure it like this in the config file:

...
sample_label_names: ['my_label1','my_label2','my_label3']

tiled_annotation_callback:
  "*object": wholeslidedata.annotation.callbacks.TiledAnnotationCallback
  tile_size: 512
  label_names: ${sample_label_names} # labels in the annotation that need to be tiled
  overlap: 0
  full_coverage: False # if true, a (tile) rectangle should fall compeletly within the annotation

annotation_parser:
 "*object": wholeslidedata.interoperability.qupath.parser.QuPathAnnotationParser
 sample_label_names: ${sample_label_names}
 callbacks: [${tiled_annotation_callback}]

...

Please let me know if you have any issues. Also please feel free to send me your image and annotation file and I will make a complete config that works for your use case.

[mdeleeuw1]

Hi Mart,

No apologies needed! Thanks for getting back to me.

Does this approach mean that I will get overlapping tiles near borders of annotations that are close to each other? I want to merge all the adjacent tiles together into a large numpy array, do you think the batch iterator or patch iterator works better for that purpose?

By the way, I am also trying to perform the same but with the patch iterator, do you know if the patch iterator works when using a .mrxs WSI file and a .tif mask file? The patch iterator does not seem to be working when using .tif masks (that I converted from my .geojson annotations using wholeslidedata, I can open these .tif masks perfectly fine in ASAP).

Kind regards,

Mike

[martvanrijthoven]

Dear Mike,

Thanks!

Yes indeed tiles will overlap if they are near each other. To solve you could write another callback to merge adjacent annotations, although this callback does not exists yet.

However, a more straightforward approach is indeed to use the patch iterator. It was actually created exactly for processing a WSI and a .tif mask file. So converting a .geojson file to a mask file and using it with the patchiterator should work.

Please let me know what is going wrong and if possible share (to mart.vanrijthoven@gmail.com) your WSI and .geojson file and i will get back to you with a working solution. If it is not possible to share the data, you can provide me the full config file.

Best wishes,
Mart