Skip to content

Conversation

@sundy-li
Copy link
Member

@sundy-li sundy-li commented Jan 22, 2026

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

  • Ensure TopN pruning respects NULL ordering so DESC queries with NULLS LAST don't drop all non-null blocks
  • Add a sqllogic regression that reproduces the nullable timestamp scenario seen on population_all_columns

Changes

The pruner now compares block statistics through a null-aware helper. This guarantees that when NULLS LAST is requested, blocks containing real timestamps remain eligible even if some segments only contain NULL stats. A new sqllogic test writes multiple blocks (with compaction disabled) and verifies ORDER BY ts DESC keeps the newest non-NULL rows ahead of NULL rows.

Implementation

  1. Added compare_scalar_for_sorting and compare_block_stats to centralize NULL-first/last handling inside the TopN pruner
  2. Added a regression unit test plus helpers for constructing NULL-only block stats
  3. Extended tests/sqllogictests/suites/query/order.test with a nullable timestamp table flow matching the production query

Tests

  • Unit Test (cargo test -p databend-storages-common-pruner)
  • Logic Test (make sqllogic-test) — not run in this environment
  • Benchmark Test
  • No Test - Pair with the reviewer to explain why

Type of change

  • Bug fix (non-breaking change which fixes an issue)

This change is Reviewable

@github-actions github-actions bot added the pr-bugfix this PR patches a bug in codebase label Jan 22, 2026
@sundy-li sundy-li requested a review from b41sh January 22, 2026 09:01
@sundy-li sundy-li changed the title fix: respect null ordering in topn pruner fix(query): respect null ordering in topn pruner Jan 22, 2026
@sundy-li sundy-li requested a review from wubx January 22, 2026 09:13
@github-actions
Copy link
Contributor

github-actions bot commented Jan 23, 2026

🤖 CI Job Analysis

Workflow: 21333729791

📊 Summary

  • Total Jobs: 84
  • Failed Jobs: 1
  • Retryable: 0
  • Code Issues: 1

NO RETRY NEEDED

All failures appear to be code/test issues requiring manual fixes.

🔍 Job Details

  • linux / test_unit: Not retryable (Code/Test)

🤖 About

Automated analysis using job annotations to distinguish infrastructure issues (auto-retried) from code/test issues (manual fixes needed).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-bugfix this PR patches a bug in codebase

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants