Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
65d90c2
feat: Implement column sorting for interactive table widget
shuoweil Nov 11, 2025
75174e3
Merge branch 'main' into shuowei-anywidget-sort-by-col-name
shuoweil Nov 12, 2025
a31771a
update error handling and introduce three stages for sort
shuoweil Nov 12, 2025
021d35a
Merge branch 'main' into shuowei-anywidget-sort-by-col-name
shuoweil Nov 14, 2025
f5420d2
change to hoveble dot
shuoweil Nov 14, 2025
8fac06c
make arrow visiable after sorting
shuoweil Nov 14, 2025
0e40d69
merge main
shuoweil Nov 18, 2025
b4dcce7
Merge branch 'main' into shuowei-anywidget-sort-by-col-name
shuoweil Nov 20, 2025
0680139
remove unnecessary exception catch and use dataclass
shuoweil Nov 21, 2025
688ec48
add js unit test framework
shuoweil Nov 21, 2025
b0f051c
bug fix to display table in notebook
shuoweil Nov 21, 2025
eb2f648
fix: nox system-3.9 run
shuoweil Nov 21, 2025
e4e302c
Revert "fix: nox system-3.9 run"
shuoweil Nov 21, 2025
7f747b7
add reset
shuoweil Nov 21, 2025
07634b9
Deduplication
shuoweil Nov 21, 2025
9669a39
Merge branch 'main' into shuowei-anywidget-sort-by-col-name
shuoweil Nov 22, 2025
6abc1d6
Merge branch 'main' into shuowei-anywidget-sort-by-col-name
shuoweil Nov 24, 2025
580b492
Update bigframes/display/table_widget.js
shuoweil Nov 24, 2025
96e49eb
code refactor
shuoweil Nov 24, 2025
a708c57
revert a testcase change
shuoweil Nov 24, 2025
25a6ade
Merge branch 'main' into shuowei-anywidget-sort-by-col-name
shuoweil Nov 25, 2025
a04c92b
disable sorting for integer or multiindex
shuoweil Nov 25, 2025
7d48abd
Merge branch 'main' into shuowei-anywidget-sort-by-col-name
shuoweil Nov 25, 2025
f9bac38
fix mypy
shuoweil Nov 25, 2025
7b87d2a
fix mypy
shuoweil Nov 25, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 31 additions & 1 deletion bigframes/display/anywidget.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,9 @@

from importlib import resources
import functools
import logging
import math
from typing import Any, Dict, Iterator, List, Optional, Type
from typing import Any, Dict, Iterator, List, Optional, Tuple, Type
import uuid

import pandas as pd
Expand Down Expand Up @@ -57,6 +58,8 @@ class TableWidget(WIDGET_BASE):
page_size = traitlets.Int(0).tag(sync=True)
row_count = traitlets.Int(0).tag(sync=True)
table_html = traitlets.Unicode().tag(sync=True)
sort_column = traitlets.Unicode("").tag(sync=True)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have we considered having multiple columns as a possibility? I think a single column is a good starting point, but I think it's an alternative worth considering, especially when a particular column contains lots of duplicate values, like a "date" column.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that multi-column sorting is particularly valuable when a column has many duplicate values. I would like to get the single column sorting checked in first as a PR. Then check in a second PR for multi-column sorting. This current PR is already complex enough. I prefer two separate PRs as enhancements.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, separate PR makes sense to me, thanks.

sort_ascending = traitlets.Bool(True).tag(sync=True)
_initial_load_complete = traitlets.Bool(False).tag(sync=True)
_batches: Optional[blocks.PandasBatches] = None
_error_message = traitlets.Unicode(allow_none=True, default_value=None).tag(
Expand All @@ -83,6 +86,7 @@ def __init__(self, dataframe: bigframes.dataframe.DataFrame):
self._all_data_loaded = False
self._batch_iter: Optional[Iterator[pd.DataFrame]] = None
self._cached_batches: List[pd.DataFrame] = []
self._last_sort_state: Optional[Tuple[str, bool]] = None
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can guess based on context what these mean, but I think a frozen dataclass would be much easier to understand at a glance.


# respect display options for initial page size
initial_page_size = bigframes.options.display.max_rows
Expand Down Expand Up @@ -215,6 +219,27 @@ def _set_table_html(self) -> None:
)
return

# Apply sorting if a column is selected
df_to_display = self._dataframe
if self.sort_column:
try:
df_to_display = df_to_display.sort_values(
by=self.sort_column, ascending=self.sort_ascending
)
except KeyError:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm afraid of this. KeyError is a relatively common error in Python. Just looking at this code, it's not clear to me that this would always be the case of a missing column. Please check for the column presence explicitly, instead.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should trigger this error to be honest. The user only allows the click on a sortable column name to trigger this part of code. Theoretically, we do not even need to catch this bug, because this exception should never be hit in our design. I will remove this try catch block.

logging.warning(
f"Attempted to sort by unknown column: {self.sort_column}"
)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are we catching this exception and dropping it? If we get to this state something bad has happened and the user should know about it.

Copy link
Contributor Author

@shuoweil shuoweil Nov 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here is my plan:
When a KeyError is caught, the current implementation does the following:

  1. Notifies the user: It sets self._error_message to a user-friendly message like "Column '...' not found.". The frontend will then display this error.
  2. Reverts to unsorted: It resets self.sort_column = "".
  3. Displays the unsorted table: The function then continues, but since sort_column is now empty, it fetches and displays the data in its original, unsorted order.


# Reset batches when sorting changes
if self._last_sort_state != (self.sort_column, self.sort_ascending):
self._batches = df_to_display._to_pandas_batches(page_size=self.page_size)
self._cached_batches = []
self._batch_iter = None
self._all_data_loaded = False
self._last_sort_state = (self.sort_column, self.sort_ascending)
self.page = 0 # Reset to first page

start = self.page * self.page_size
end = start + self.page_size

Expand All @@ -235,6 +260,11 @@ def _set_table_html(self) -> None:
table_id=f"table-{self._table_id}",
)

@traitlets.observe("sort_column", "sort_ascending")
def _sort_changed(self, _change: Dict[str, Any]):
"""Handler for when sorting parameters change from the frontend."""
self._set_table_html()

@traitlets.observe("page")
def _page_changed(self, _change: Dict[str, Any]) -> None:
"""Handler for when the page number is changed from the frontend."""
Expand Down
2 changes: 1 addition & 1 deletion bigframes/display/table_widget.css
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@

.bigframes-widget th {
background-color: var(--colab-primary-surface-color, var(--jp-layout-color0));
/* Uncomment once we support sorting: cursor: pointer; */
cursor: pointer;
position: sticky;
top: 0;
z-index: 1;
Expand Down
32 changes: 32 additions & 0 deletions bigframes/display/table_widget.js
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,8 @@ const ModelProperty = {
PAGE_SIZE: "page_size",
ROW_COUNT: "row_count",
TABLE_HTML: "table_html",
SORT_COLUMN: "sort_column",
SORT_ASCENDING: "sort_ascending",
};

const Event = {
Expand Down Expand Up @@ -124,6 +126,36 @@ function render({ model, el }) {
// Note: Using innerHTML is safe here because the content is generated
// by a trusted backend (DataFrame.to_html).
tableContainer.innerHTML = model.get(ModelProperty.TABLE_HTML);

// Add click handlers to column headers for sorting
const headers = tableContainer.querySelectorAll("th");
headers.forEach((header) => {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is quite a bit of logic. I would like to get JavaScript-level unit tests setup before merging this PR.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've set up a comprehensive JavaScript unit testing environment and implemented tests for the TableWidget's interactive sorting functionality.

Testing Framework & Environment:
* Jest: Chosen as the primary testing framework due to its all-in-one nature (runner, assertion library, mocking) and strong community support.
* JSDOM: Utilized within Jest to simulate a browser environment, enabling DOM manipulation and event handling within Node.js.
* Babel: Configured to transpile modern JavaScript (ES modules) used in the widget and tests, ensuring compatibility.

Test Coverage: The table_widget.test.js file now includes unit tests that verify:
* The basic DOM structure generated by the widget.
* The interaction logic for column header clicks, including:
* Initiating ascending sort on the first click.
* Reversing to descending sort on the second click.
* Clearing the sort (returning to unsorted state) on the third click.
* The correct display of sorting indicators (▲ for ascending, ▼ for descending, ● for unsorted).
* Proper interaction with the model.set and model.save_changes methods.

**How to Run**:
To execute these tests from the project root directory:
```bash
cd tests/js
npm install  # Only needed if dependencies haven't been installed or have changed
npm test
```

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Could you also add a workflow for this to our GitHub Actions folder? https://github.com/googleapis/python-bigquery-dataframes/tree/main/.github/workflows

Something like this:

name: js-tests
on: push
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2
    - name: Install modules
      working-directory: ./tests/js
      run: npm install
    - name: Run tests
      working-directory: ./tests/js
      run: npm test

const columnName = header.textContent.trim();
if (columnName) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not all data types are sortable. See the orderable property in our dtypes.

orderable: bool = False

We should not add the handler or arrow to columns that we can't sort.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here is the plan:

  1. Display a dot (●) indicator for all sortable columns by default
  2. Allow users to cycle through three states: unsorted (●) → ascending (▲) → descending (▼) → unsorted (●)
  3. Only show sorting UI for columns with orderable data types

header.style.cursor = "pointer";
header.addEventListener(Event.CLICK, () => {
const currentSortColumn = model.get(ModelProperty.SORT_COLUMN);
const currentSortAscending = model.get(ModelProperty.SORT_ASCENDING);

if (currentSortColumn === columnName) {
// Toggle sort direction
model.set(ModelProperty.SORT_ASCENDING, !currentSortAscending);
} else {
// New column, default to ascending
model.set(ModelProperty.SORT_COLUMN, columnName);
model.set(ModelProperty.SORT_ASCENDING, true);
}
model.save_changes();
});

// Add visual indicator for sorted column
if (model.get(ModelProperty.SORT_COLUMN) === columnName) {
const arrow = model.get(ModelProperty.SORT_ASCENDING) ? " ▲" : " ▼";
header.textContent = columnName + arrow;
}
}
});

updateButtonStates();
}

Expand Down
Loading