Improvements to titiler-cmr compatibility report

[abarciauskas-bgse]

Feedback from Brian F and Buddy:

• make sure the texts in the charts are readable. Using a legend for the very small pie chart slice is fine.
• For the unsupported data format, provide answers to the following questions: are they mainly because there is no standardized method to interpret the data as a map, or are there other reasons? Is there any solution we can recommend?

[abarciauskas-bgse]

I have crafted the following text to answer the questions about why formats are not supported:

Why are some formats considered "unsupported"?

The simple answer to this question is the backend libraries used by titiler (rio_tiler + rasterio or rio_tiler + xarray with h5netcdf or zarr backends) don't support formats.

A reasonable follow up question would be of course Why don't these libraries support tiling more formats? which requires a more long-winded answer:

1. Library simplicity: Software libraries have to balance flexibility with stability. Trying to support all functionality for all possible geospatial data formats will introduce complexity, making the libraries (titiler and rio_tiler in this case) harder to maintain.
2. No geospatial structure: Many formats (txt, csv) don't have consistent structure for geospatial data, required for generating map tiles. In fact, even those formats that have consistent structure don't have consistency in how geospatial information is represented. Libraries (again, rio_tiler in this case) often make a "best guess" at which coordinates are the spatial ones.
3. Inefficient to read over the network: NetCDF-3 and HDF4 were originally designed for reading from a local filesystem. Metadata is scattered throughout the file. Reading these files over a network connection is thus very slow given the hundreds of requests and responses required. While map tiles could be produced in theory, these non-cloud-optimized-formats make the process over the network prohibitively slow. rio_tiler has chosed to use xarray with the h5netcdf backend for simplicity and thus does not support reading these formats. GeoTIFFs are similarly inefficient to read over the network. While rio_tiler's rasterio reader could in theory download the entire GeoTIFFs to generate map tiles, it is extremely inefficient to do so and thus not supported.

@maxrjones @vincentsarago curious if you agree with that response

[maxrjones]

Nice, I think your answer addresses "why are some datasets considered "unsupported"?" rather than "why are some formats considered "unsupported"?" since the metadata answer matches to no x_y coordinates in the categories rather than the unsupported format categories. I also would prefer to see mentions of solutions in the answers. For 1, we will prioritize implementation and maintenance of highly impactful features (e.g., hierarchical groups). For 3, we will address this by working with DAACs to add virtual indexes to the CMR collections for cloud-native access.

An aside: If you're remaking the charts anyways, it's be really helpful to see absolute numbers along side the percentages because those would help with prioritization of features.

[abarciauskas-bgse]

Thanks for reviewing @maxrjones.

the metadata answer matches to no x_y coordinates in the categories rather than the unsupported format categories.

With reason (2) above "No geospatial structure: " perhaps I should reword this to say "Lack of dependable structure": Some of these formats (like csv, txt, ascii) lack support because (I assume) there is no dependable structure from which to read data into map tiles. I see this as distinct from formats like NetCDF, HDF and GeoTIFF, which do have common ways to structure geospatial coordinates which libraries rely on (however even this is imperfect). So I still see this lack of dependable structure as a reason some formats are categorically not supported by libraries we rely on for map tiling.

[abarciauskas-bgse]

closed via #118