feat(LAM-1411): Add ability to filter based on custom SQL column in traces table

[laminar-coding-agent]

Summary

• Custom SQL columns now appear in the filter dropdown (both DataTableFilter and AdvancedSearch) with correct data types (string/number operators)
• Backend query builder dynamically creates filter processors for custom columns using their SQL expression and ClickHouse data type
• Added dbType to custom column meta so the filter config can use the correct ClickHouse type for parameterized queries

Changes

• frontend/lib/actions/traces/utils.ts: Added buildFilterConfigWithCustomColumns() that clones the static filter config and adds dynamic processors for each custom column. Used in buildTracesQueryWithParams instead of the static config.
• frontend/components/traces/traces-table/index.tsx: Renamed filters import to staticFilters, added allFilters memo that merges static filters with custom column filters (derived from customColumns state). Passed allFilters to both DataTableFilter and AdvancedSearch.
• frontend/components/traces/traces-table/traces-table-store.ts: Added dbType to custom column meta (Float64 for number, String for string) so the filter processor can cast parameters correctly.

Screenshots

1. Custom column "Token Ratio" added to the table

2. Custom column appears in filter dropdown

3. Filter configured: Token Ratio > 0

4. Filtered results showing only traces with tokens

Video demo

Custom column filter demo (WebM)

Test plan

• Create a custom SQL column (e.g. "Token Ratio" with if(total_tokens > 0, input_tokens * 100 / total_tokens, 0), type Number)
• Verify the custom column appears in the filter dropdown
• Apply a filter on the custom column (e.g. Token Ratio > 0)
• Verify results are correctly filtered (only traces with non-zero token ratio shown)
• Verify number operators (=, <, >, <=, >=, !=) are available for number-type custom columns
• TypeScript type-check passes (tsc --noEmit)
• Pre-commit hooks pass (prettier, eslint, type-check)

Closes LAM-1411

🤖 Generated with Claude Code

---

Note

Medium Risk
Adds dynamic, user-defined column filtering that feeds into ClickHouse query construction; incorrect SQL/type handling could cause query errors or unexpected filtering. Changes touch both traces list and stats endpoints, affecting core query paths.

Overview
Custom SQL columns are now first-class filter targets in the traces UI: the traces table merges static filter definitions with persisted custom columns and passes them to both DataTableFilter and AdvancedSearch.

The backend now accepts a customColumns payload for traces list/count and stats queries, validates it via parseCustomColumnsJson, and dynamically extends the column filter processor map to generate parameterized ClickHouse conditions (with basic type casting and safer parameter keys). Custom columns also carry a dbType (Float64/String) so filters are typed correctly end-to-end.

^{Written by Cursor Bugbot for commit 4922656. This will update automatically on new commits. Configure here.}

[greptile-apps]

Greptile Summary

This PR enables custom SQL columns to appear in the filter dropdown for the traces table, covering both the DataTableFilter and AdvancedSearch components. On the backend, a new buildFilterConfigWithCustomColumns helper dynamically adds ClickHouse filter processors for each custom column using its SQL expression and dbType, and this config is wired into buildTracesQueryWithParams.

Key changes:

• traces-table-store.ts: adds dbType (Float64 / String) to custom column meta so the filter processor can cast values correctly.
• index.tsx: derives allFilters by merging the static filters export with per-custom-column ColumnFilter entries (keyed as custom:<name>), and passes this to both filter components.
• utils.ts: introduces buildFilterConfigWithCustomColumns which clones the static processor map and appends a dynamic processor for each custom column; buildTracesQueryWithParams now uses it.

Issues found:

• buildTracesStatsWhereConditions hard-codes tracesColumnFilterConfig for processor lookup (line 395). Custom column filter entries have no registered processor there, so they are silently dropped — the stats/chart panel will show unfiltered aggregate data even when a custom column filter is active in the table.
• buildTracesCountQueryWithParams and buildTracesIdsQueryWithParams have the same gap: neither accepts customColumns nor uses buildFilterConfigWithCustomColumns, so counts and ID lists could be inflated when a custom column filter is applied.

Confidence Score: 3/5

Safe to merge for the core filter feature, but the stats chart will visibly show unfiltered data when a custom column filter is active.

The main traces table query is correctly updated. However, buildTracesStatsWhereConditions, buildTracesCountQueryWithParams, and buildTracesIdsQueryWithParams still use the static filter config, causing custom column filters to be silently ignored in those code paths. The stats chart inconsistency is immediately user-visible and was not possible before this PR, making it a clear regression introduced here.

frontend/lib/actions/traces/utils.ts — buildTracesStatsWhereConditions (line 395), buildTracesCountQueryWithParams, and buildTracesIdsQueryWithParams all need to use buildFilterConfigWithCustomColumns.

Important Files Changed

Filename	Overview
frontend/lib/actions/traces/utils.ts	Adds buildFilterConfigWithCustomColumns and integrates it into buildTracesQueryWithParams. buildTracesStatsWhereConditions, buildTracesCountQueryWithParams, and buildTracesIdsQueryWithParams still use the static config and will silently ignore custom column filters.
frontend/components/traces/traces-table/index.tsx	Cleanly merges static filters with custom column filters via a memoised allFilters array; correctly passes allFilters to both DataTableFilter and AdvancedSearch.
frontend/components/traces/traces-table/traces-table-store.ts	Adds dbType (Float64 / String) to custom column meta so the filter processor can cast parameters correctly; straightforward single-field addition.

Sequence Diagram

sequenceDiagram
    participant UI as TracesTableContent
    participant DF as DataTableFilter / AdvancedSearch
    participant API as /api/traces
    participant QB as buildTracesQueryWithParams
    participant SB as buildTracesStatsWhereConditions

    UI->>DF: allFilters (staticFilters + customColumnFilters)
    DF->>UI: filter URL param (e.g. custom:Token Ratio gt 0)
    UI->>API: GET /traces?filter=custom:Token+Ratio:gt:0&customColumns=[...]
    API->>QB: buildTracesQueryWithParams({ filters, customColumns })
    QB->>QB: buildFilterConfigWithCustomColumns(customColumns)
    QB-->>API: filtered SQL query ✅

    UI->>API: GET /traces/stats?filter=custom:Token+Ratio:gt:0
    API->>SB: buildTracesStatsWhereConditions({ filters })
    SB->>SB: tracesColumnFilterConfig.processors.get("custom:Token Ratio") → undefined
    SB-->>API: filter silently ignored ❌
    API-->>UI: unfiltered stats data

Loading

Comments Outside Diff (2)

1. frontend/lib/actions/traces/utils.ts, line 395-403 (link)

   Stats chart ignores custom column filters

   buildTracesStatsWhereConditions still uses the static tracesColumnFilterConfig directly. When a user applies a custom column filter (e.g. custom:Token Ratio > 0), tracesColumnFilterConfig.processors.get('custom:Token Ratio') returns undefined, so the filter is silently skipped. The stats/chart panel will show unfiltered aggregate data while the traces table correctly shows filtered rows — a visible inconsistency introduced by this PR.

   The fix is to accept an optional customColumns parameter here (similar to buildTracesQueryWithParams) and delegate to buildFilterConfigWithCustomColumns when looking up processors, rather than hard-coding tracesColumnFilterConfig at line 395.

2. frontend/lib/actions/traces/utils.ts, line 280-316 (link)

   Count and IDs queries don't handle custom column filters

   buildTracesCountQueryWithParams (line 280) and buildTracesIdsQueryWithParams (line 329) both still use the static tracesColumnFilterConfig. If these queries are called with a filter array that includes a custom column filter, the custom column condition will be silently ignored — producing an inflated count or a superset of expected trace IDs.

   If either function is used for pagination totals or for export/evaluation ID gathering, the results will be incorrect whenever a custom column filter is active. Consider accepting an optional customColumns?: CustomColumn[] parameter in both functions (same pattern as buildTracesQueryWithParams) and using buildFilterConfigWithCustomColumns for the columnFilterConfig field.

_{Reviews (1): Last reviewed commit: "feat(LAM-1411): add ability to filter ba..." | Re-trigger Greptile}

[laminar-coding-agent]

Both comments have been addressed:

Comment 1 (Medium - unsanitized paramKey): Valid bug. buildTracesStatsWhereConditions constructed paramKey as ${filter.column}_${index} without sanitizing special characters. For custom columns with filter.column = "custom:Token Ratio", the colon and space in the paramKey would break ClickHouse's {name:Type} syntax. Fixed by adding filter.column.replace(/[^a-zA-Z0-9_]/g, "_") to match the sanitization pattern used in the shared buildColumnFilters in query-builder.ts.

Comment 2 (Low - duplicated parsing logic): Valid concern. The identical Zod schema + try/catch parsing block was copy-pasted in getTraces, countTraces, and getTraceStats. Extracted a shared parseCustomColumnsJson() helper in utils.ts and replaced all three copies with calls to it, eliminating the duplication and risk of schema drift.

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}