Feature request

When preparing to pull data from a catalog, especially a remotely located one, it would be very good to have some estimate of how much data will be transferred. While an exact estimate is not possible without actually performing the computation (because of the effect of row filtering, for example) the effects of region searches and column selection can be well-estimated, using the metadata from the selected partitions.

When a lazy representation of a Catalog is shown in a notebook, the number of partitions is reported, so the partition collection is known. Each of these partitions has metadata describing the data consumed for each column. Therefore it is possible to report an upper limit on the data that will be required for a given region+column selection. While not the same as complete preflighting, this upper bound would be valuable to users, who might choose to reduce their region or column selection further before starting computation.

Note that this information does not appear obtainable today via the Catalog attributes; it is something that will need to come from looking at the metadata in the Parquet files underlying the partitions.

From verbal discussions during LSDB Busy Week 2025 at STSci.

Before submitting
Please check the following:

• I have described the purpose of the suggested change, specifying what I need the enhancement to accomplish, i.e. what problem it solves.
• I have included any relevant links, screenshots, environment information, and data relevant to implementing the requested feature, as well as pseudocode for how I want to access the new functionality.
• If I have ideas for how the new feature could be implemented, I have provided explanations and/or pseudocode and/or task lists for the steps.