DataTable performance enhancement #5951
Description
The DataTable class is really handy for displaying tabular data, but I find it somewhat sluggish to scroll when populated with a moderate amount of data (say a few hundred rows by a few dozen columns).
A bit searching suggests this is kind of a known limitation. A popular suggestion is to use textual-fastdatatable, but that package is not perfect either. The main issues I find are that it offers a different, immutable programming interface, and requires pyarrow which seems like an overkill.
Playing with the built-in DataTable, it seems the amount of sluggishness scales mainly with the number of columns rather than with the number of rows. Since the number of columns in a table is usually bounded, this suggests that some sort of micro-optimization could help to improve the UI performance without changing any API or behavior.
I did a quick profiling using the (great) Python profiler py-spy. It seems much time is spent in tight loops whose overhead could be drastically reduced a small amount of micro-optimization. I think this profiling-guided optimization approach could be promising.
In summary, I'd like to ask if you have any prior study on the performance of DataTable, and if you'd welcome a PR to improve the performance. I intend the PR to contain localized optimizations that retain the current API and behavior. I expect the PR to reduce the UI sluggishness "sufficiently" for moderate-sized data.