How to get DataTable columns to dynamically resize beyond the first two "inserted" columns?

[rlad78]

I'm noticing some really weird behavior for columns in DataTable that I don't fully understand.

I've got a program that asynchronously inserts data into a DataTable using a callback to an "add_row" method. The program will insert data into columns in varying orders depending on the results of the async work going on in the background. This means that columns[0] might not always be the first column entered into.

What I've observed is that, the first two columns that get any data inserted into them will auto-resize based on new cell content, but other columns that get data any later will not update their widths. I've tried refresh_column and refresh_row but they did not seem to make any changes to rendering.

I've written a little demo program that shows what I'm experiencing. Is there something that I might be missing?

import asyncio
from typing import Callable
from random import random, randint, choice, shuffle, choices

from textual.app import App, ComposeResult
from textual.worker import Worker
from textual.containers import ScrollableContainer
from textual.widgets import DataTable, TextLog

RAND_WORDS = [
    "audience",
    "excitement",
    "writing",
    "student",
    "promotion",
    "enthusiasm",
    "passenger",
    "shirt",
    "law",
    "engine",
    "disease",
    "teacher",
    "location",
    "variety",
    "love",
]

COLUMN_NAMES = ["First", "Second", "Third", "Fourth", "Fifth"]


def generate_random_string() -> str:
    """Generates random strings of random length
    that will be used as entries in our DataTable"""
    # use a random separator to eliminate chance that issue is tied to a type of character
    separators = (".", ", ", "_", " ", "-", "X")

    # choose up to 3 words
    words = choices(RAND_WORDS, k=randint(1, 3))
    # choose a random separator char
    separator = choice(separators)
    # join random word(s) to separator, ex. 'love-location'
    return separator.join(words)


class FakeDataGenerator:
    def __init__(self, add_to_table_callback: Callable[[str, str, str], None]) -> None:
        """Produces semi-random data and asynchronously inserts it into DataTable"""
        self._callback = add_to_table_callback

        # create random batch of identities (these will be our row keys)
        # identities will always be unique strings
        self.identities: set[str] = set()
        batch_count = randint(3, 12)
        while len(self.identities) < batch_count:
            self.identities.add(generate_random_string())

    async def _generate_identity_data(self, identity: str) -> None:
        """Generates random data to insert into an identity's row.
        Will always insert by columns in order of 'Second' to 'Fifth'.
        The 'First' column is always reserved for the identity."""
        for column in COLUMN_NAMES[1:]:
            await asyncio.sleep(randint(0, 3) + random())
            self._callback(identity, column, generate_random_string())

    async def generate_data(self) -> None:
        """Start tasks corresponding to each identity
        for inserting data into the identity's row."""
        tasks = [
            asyncio.create_task(self._generate_identity_data(ident))
            for ident in self.identities
        ]
        await asyncio.gather(*tasks)


class RandomTableData(App):
    def compose(self) -> ComposeResult:
        with ScrollableContainer():
            yield DataTable(id="ProgressData", show_cursor=False)
        yield TextLog(wrap=True, markup=True)

    def on_mount(self) -> None:
        """Initialize table and start generation job"""
        table = self.query_one(DataTable)
        logview = self.query_one(TextLog)

        table.styles.height = "2fr"
        logview.styles.height = "1fr"

        # create columns in a random order
        # eliminates the chance that this issue is limited to column order
        # 'First' column will contain data equal to row key on creation
        columns = COLUMN_NAMES.copy()
        shuffle(columns)
        for column in columns:
            table.add_column(column, key=column)
        logview.write("[green]on_mount[/green]: Table creation complete")

        data_gen = FakeDataGenerator(self._table_add_callback)
        self.run_worker(data_gen.generate_data())

    def on_worker_state_changed(self, event: Worker.StateChanged) -> None:
        if event.worker.is_finished:
            self.query_one(TextLog).write("[green bold]Program finished")

    def _table_add_callback(self, identity: str, column: str, data: str) -> None:
        """Callback method for inserting data into table,
        where identity represents the key for each row."""
        table = self.query_one(DataTable)
        logview = self.query_one(TextLog)

        # create a new row if the identity hasn't been used before
        if identity not in table.rows:
            ordered_column_keys = [col.key for col in table.ordered_columns]
            identity_column_index = ordered_column_keys.index("First")  # type: ignore

            # we pad the 'cells' iterable with None values in order to insert at the correct position
            table.add_row(*([None] * identity_column_index + [identity]), key=identity)
            table.sort("First")

        table.update_cell(identity, column, data)
        logview.write(
            f"[blue]_table_add_callback[/blue]: At row [red]{identity}[/red],"
            + f"column [yellow]{column}[/yellow],"
            + f"inserted [green]{data}[/green]"
        )

        # ordered_column_keys = [col.key for col in table.ordered_columns]
        # ordered_row_keys = [row.key for row in table.ordered_rows]
        # table.refresh_column(ordered_column_keys.index(column))  # type: ignore
        # table.refresh_row(ordered_row_keys.index(identity))  # type: ignore


def main():
    app = RandomTableData()
    app.run()


if __name__ == "__main__":
    main()

[TomJGooding]

My best guess is that the DataTable never reaches an idle state, which seems to be when various re-sizing occurs looking at the code.. I'm not really sure though, hopefully someone with more experience of async workers/messages in Textual can offer more help here!

[TomJGooding]

After looking again at the code, I think you might need to add update_width=True to your update_cell method.

I'm likely wrong and actually there's a better way of doing this - as resizing after updating each cell doesn't seem very efficient!

[rlad78]

Shoot, didn't even realize there was an update_width parameter for update_cell(). That's already immensely helpful!

From looking at the source for update_cell(), it appears that setting the update_width flag stores the affected cell in the self._updated_cells list to be dealt with on the next _on_idle() call, as you mentioned before.

textual/src/textual/widgets/_data_table.py

Lines 1425 to 1431 in 49e1080

	if self._updated_cells:
	# Cell contents have already been updated at this point.
	# Now we only need to worry about measuring column widths.
	updated_columns = self._updated_cells.copy()
	self._updated_cells.clear()
	self._update_column_widths(updated_columns)

Luckily, and please correct me if I'm wrong here, setting update_width on each insert doesn't seem too computationally expensive given that there is technically always a chance of increasing the column width with each new cell update. The _update_column_widths() method doesn't actually call any refreshes itself, so the only thing expensive here might be the measure() function (doesn't seem like it, but I'm not very familiar with how Rich works).

Now I'm just wondering why the first two columns to get their cells updated have their column widths automatically adjusted without the update_width flag being manually set for each call. Can't seem to track that down what's happening there...