v1.2.729-nightly

What's Changed

Build/Testing/CI Infra Changes 🔌

• ci: fix pack deb by @everpcpc in #17832

Others 📒

• chore(query): disable cluster level semaphore by @zhang2014 in #17830

Full Changelog: v1.2.728-nightly...v1.2.729-nightly

v1.2.728-nightly

What's Changed

Others 📒

• chore: add more info logging to semaphore by @drmingdrmer in #17831

Full Changelog: v1.2.727-nightly...v1.2.728-nightly

v1.2.727-nightly

What's Changed

Exciting New Features ✨

• feat: hybrid column data cache by @dantengsky in #17771

Thoughtful Bug Fix 🔧

• fix(metrics): remove high cardinality labels in metrics by @flaneur2020 in #17820

Code Refactor 🎉

• refactor(query): remove unsafe implementation of Send and Sync for AcquireQueueGuard by @drmingdrmer in #17818

Build/Testing/CI Infra Changes 🔌

• ci: fix ttc test by @everpcpc in #17807
• ci: fix check system info by @everpcpc in #17822
• ci: fix include deb by @everpcpc in #17823
• ci: fix deb package info by @everpcpc in #17826

Others 📒

• chore(query): migrate benchmark to divan framework by @sundy-li in #17816
• chore: add logging to semphore leaser by @drmingdrmer in #17825
• chore(ci): add test_stateless_standalone timeout minutes to 18 by @TCeason in #17828
• chore(query): support local semaphore by @zhang2014 in #17827

Full Changelog: v1.2.726-nightly...v1.2.727-nightly

v1.2.726-nightly

What's Changed

Exciting New Features ✨

• feat: impl Create/Drop Index for Ngram Index by @KKould in #17789
• feat: block stream write by @zhyass in #17744
• feat: Bump opendal to 0.53.1 by @Xuanwo in #17817

Thoughtful Bug Fix 🔧

• fix: Remove set nonblocking = false for mysql tcp stream by @Xuanwo in #17798

Code Refactor 🎉

• refactor: Bump opendal to 0.53 for better metrics by @Xuanwo in #17788
• refactor(fuse): runtime filter into two files: expr_bloom_filter.rs and expr_runtime_pruner.rs by @BohuTANG in #17804
• refactor: move segment generation into TableMutationAggregator for MERGE INTO by @zhyass in #17800
• refactor: upgrade min-meta-service-version to 1.2.677 by @drmingdrmer in #17783
• refactor(query): auto generate virtual columns for variant column by @b41sh in #17707

Build/Testing/CI Infra Changes 🔌

• ci: fix build feature for benchmark by @everpcpc in #17796
• ci: use larger runner for release by @everpcpc in #17799

Others 📒

• chore(ci): change the sccache log level to debug by @forsaken628 in #17790
• chore: Extracting databend_common_version to help improve compilation cache hit rate by @forsaken628 in #17801
• chore: make databend_common_expression compilation cacheable by @forsaken628 in #17806
• chore(ci): add ci test_stateless_cluster_linux timeout minute to 18 by @TCeason in #17813

Full Changelog: v1.2.725-nightly...v1.2.726-nightly

v1.2.725

What's Changed

Exciting New Features ✨

• feat: Add hdfs support in iceberg and fill iceberg statistics by @Xuanwo in #17352
• feat: support specifying columns to include in the "Attach table" statement by @dantengsky in #17442
• feat(metrics): add progress metrics via collector by @flaneur2020 in #17359
• feat(binder): support update from by @Dousir9 in #17464
• feat(query): add table stats api to admin interface by @everpcpc in #17471
• feat: Use max/min to estimate ndv by @Xuanwo in #17468
• feat: add table option copy_dedup_full_path by @youngsofun in #17473
• feat(query): add markov model function and feistel function for obfuscator by @forsaken628 in #17437
• feat(sqlsmith): fuzz test support more dbs as seed by @b41sh in #17489
• feat: SHOW CREATE TABLE add quote for ident by @KKould in #17505
• feat: object warehouse support rbac by @TCeason in #17262
• feat: (semi-)sync table/field comments with the table being attached by @dantengsky in #17478
• feat(raft-log): enhance raft-log reliability and add compatibility tests by @drmingdrmer in #17510
• feat: add log.file.max_size and rolling test by @BohuTANG in #17508
• feat: csv/tsv/ndjson support querying file meta data. by @youngsofun in #17512
• feat(query): support explain(DECORRELATED,verbose) by @forsaken628 in #17518
• feat(query): add show_roles table function by @TCeason in #17517
• feat: support query metadata from parquet. by @youngsofun in #17527
• feat: skip reading empty files when load/query location. by @youngsofun in #17522
• feat(base): enhanced query-level memory management by @zhang2014 in #17358
• feat(query): Supports folding of show grants results for roles and users by @TCeason in #17543
• feat(query): support query level spill setting by @zhang2014 in #17542
• feat: support WITHIN GROUP clause for AggregateFunction by @KKould in #17519
• feat(query): add catalog apis by @everpcpc in #17493
• feat: support load Avro Files. by @youngsofun in #17548
• feat(query): add api for users & roles by @everpcpc in #17553
• feat(query): support nondeterministic update by @forsaken628 in #17555
• feat: string_agg is compatible with multiple types as arguments instead of just String by @KKould in #17570
• feat: add table function fuse_vacuum2() by @SkyFan2002 in #16049
• feat: auto vacuum by @dantengsky in #17579
• feat(query): database dml for iceberg catalog by @sundy-li in #17578
• feat: UNPIVOT supports AS by @forsaken628 in #17595
• feat(query): bloom index support prune expr with simple casts by @forsaken628 in #17533
• feat(query): add catalog stats api by @everpcpc in #17602
• feat(query): support create/drop table under iceberg catalog by @TCeason in #17619
• feat(query): add iceberg table functions by @sundy-li in #17626
• feat: heartbeat to avoid query result timeout. by @youngsofun in #17624
• feat: Enable test and doctest for all crates by @Xuanwo in #17633
• feat: Add bendsave which can backup and restore databend data by @Xuanwo in #17503
• feat: impl function strip_null_value for Variant by @KKould in #17640
• feat(query): support left plan's from clause contains subquery by @forsaken628 in #17621
• feat: support specifying compression when unloading to parquet. by @youngsofun in #17664
• feat(meta): implement distributed semaphore based on meta-service by @drmingdrmer in #17651
• feat: add support for storing logs in table by @dqhl76 in #17598
• feat(config): warn unknown field instead of panic on start up by @flaneur2020 in #17504
• feat: support nextval as field default value. by @youngsofun in #17670
• feat: impl function: regexp_extract & regexp_extract_all by @KKould in #17658
• feat(query): support mul fro interval type with int types by @sundy-li in #17684
• feat: impl udf server add custom header by @KKould in #17691
• feat(query): support query iceberg table by version by @TCeason in #17693
• feat: hybrid cache by @dantengsky in #17704
• feat: introduce column oriented segment into by @SkyFan2002 in #17653
• feat: databend-metactl add command watch and upsert by @drmingdrmer in #17739
• feat: Support Displaying Runtime Filters in EXPLAIN Output by @SkyFan2002 in #17726
• feat(query): page index and range index support eliminate cast by @forsaken628 in #17740
• feat(optimizer): optimizer trace log by @BohuTANG in #17746
• feat(optimizer): add optimizer skip list by @BohuTANG in #17750
• feat: function extract, date_part,date_diff support more date part by @TCeason in #17759
• feat: table function that dumps snapshot information by @dantengsky in #17763
• feat(optimizer): SExpr visitor by @BohuTANG in #17768
• feat(meta-client): add cache crate for databend-meta service by @drmingdrmer in #17766
• feat(query): table_statistics(<database_name>[, table_name]) table functions by @BohuTANG in #17781
• feat(query): support cluster level concurrent limit by @zhang2014 in #17778
• feat(query): support date_between(date_part, startdate, enddate) by @TCeason in #17782
• feat: impl Create/Drop Index for Ngram Index by @KKould in #17789
• feat: block stream write by @zhyass in #17744
• feat: Bump opendal to 0.53.1 by @Xuanwo in #17817
• feat: hybrid column data cache by @dantengsky in #17771

Thoughtful Bug Fix 🔧

• fix(query): fix WindowPartitionTopNExchange painc with an empty block by @forsaken628 in #17453
• fix(parser): allow timestamp as a function name by @forsaken628 in #17455
• fix(query): fix order by derived column with limit return wrong values by @b41sh in #17457
• fix(query): fold constant subquery to build filter plan instead of join plan by @b41sh in #17448
• fix(query): fix left semi optimize to inner join by @sundy-li in #17458
• fix(query): fix bind internal column by @b41sh in #17463
• fix(query): spill config should be mask by @forsaken628 in #17467
• fix(query): fix variant get string function auto cast null to SQL NULL by @b41sh in #17466
• fix: InterpreterMetrics could subtract with overflow by @forsaken628 in #17461
• fix(query): fix stack overflow if excessive use of union all by @zhang2014 in #17475
• fix: check table mutability in copy into by @dantengsky in #17480
• fix: impl hive disable_table_info_refresh for hive to avoid error when show table status from by @BohuTANG in #17484
• fix(binder): AggregateRewrite...

v1.2.725-nightly

What's Changed

Exciting New Features ✨

• feat(query): support date_between(date_part, startdate, enddate) by @TCeason in #17782

Thoughtful Bug Fix 🔧

• fix(query): don't check file format in proto-conv by @sundy-li in #17793

Others 📒

• chore: refine error handling in persistent log table by @dqhl76 in #17792

Full Changelog: v1.2.724-nightly...v1.2.725-nightly

v1.2.724-nightly

What's Changed

Exciting New Features ✨

• feat(meta-client): add cache crate for databend-meta service by @drmingdrmer in #17766
• feat(query): table_statistics(<database_name>[, table_name]) table functions by @BohuTANG in #17781
• feat(query): support cluster level concurrent limit by @zhang2014 in #17778

Thoughtful Bug Fix 🔧

• fix: correct state type for Decimal256 AVG aggregation by @dantengsky in #17774

Code Refactor 🎉

• refactor: DecimalSumState overflow handling in SUM aggregation by @dantengsky in #17773
• refactor(optimizer): refine the tpcds tests by @BohuTANG in #17777
• refactor(query): Combining TransformSortMergeLimit, TransformSortMerge, and TransformSortSpill by @forsaken628 in #17762
• refactor: add query_profile and query_details persistent log tables by @dqhl76 in #17702

Build/Testing/CI Infra Changes 🔌

• ci: fix test with ttc conainter by @everpcpc in #17769
• ci: tmp disable test 20_0025_stackoverflow by @everpcpc in #17770
• ci: fix start test container by @everpcpc in #17772

Others 📒

• chore: upgrade outdated dependencies by @forsaken628 in #17776
• chore(optimizer): spill out statistics yaml in tpcds_test by @BohuTANG in #17784
• chore(query): add iceberg table metadata cache by @sundy-li in #17780
• chore(ci): remove hdfs from docker image by @everpcpc in #17786
• chore: bump jsonb 0.5.0 by @b41sh in #17698

Full Changelog: v1.2.723-nightly...v1.2.724-nightly

v1.2.723-nightly

What's Changed

Exciting New Features ✨

• feat: function extract, date_part,date_diff support more date part by @TCeason in #17759
• feat: table function that dumps snapshot information by @dantengsky in #17763
• feat(optimizer): SExpr visitor by @BohuTANG in #17768

Thoughtful Bug Fix 🔧

• fix: align EXPLAIN output with actual partition pruning statistics by @dqhl76 in #17491

Code Refactor 🎉

• refactor(meta-client): Improve MetaHandshakeError by @drmingdrmer in #17755
• refactor: Bump arrow-udf-runtime so that we can use new pyo3 by @Xuanwo in #17758
• refactor: add helper methods to trait LicenseManager by @dantengsky in #17761
• refactor: reduce MetaClientError by @drmingdrmer in #17764
• refactor(optimizer): s_expr.rs by @BohuTANG in #17767

Others 📒

• chore: check ee license for disk cache by @dantengsky in #17749

Full Changelog: v1.2.722-nightly...v1.2.723-nightly

v1.2.722-nightly

What's Changed

Exciting New Features ✨

• feat(optimizer): add optimizer skip list by @BohuTANG in #17750

Code Refactor 🎉

• refactor(optimizer): refine hyper_dp/dphyp.rs by @BohuTANG in #17753

Build/Testing/CI Infra Changes 🔌

• ci: fix archive benchmark result by @everpcpc in #17754

Others 📒

• chore: Bump Rust toolchain to nightly-2025-04-04 by @Xuanwo in #17752
• chore: show file path when copy decompress fail. by @youngsofun in #17732

Full Changelog: v1.2.721-nightly...v1.2.722-nightly

v1.2.721-nightly

What's Changed

Exciting New Features ✨

• feat: hybrid cache by @dantengsky in #17704
• feat: introduce column oriented segment into by @SkyFan2002 in #17653
• feat: databend-metactl add command watch and upsert by @drmingdrmer in #17739
• feat: Support Displaying Runtime Filters in EXPLAIN Output by @SkyFan2002 in #17726
• feat(query): page index and range index support eliminate cast by @forsaken628 in #17740
• feat(optimizer): optimizer trace log by @BohuTANG in #17746

Thoughtful Bug Fix 🔧

• fix(query): process_like directly unwrap is not safe by @TCeason in #17722
• fix: flaky hybrid cache test by @dantengsky in #17735
• fix(query): ClientSessionManager::refresh_state subtracts with overflow if clock drift is present by @forsaken628 in #17742

Code Refactor 🎉

• refactor(optimizer): cascade by @BohuTANG in #17714
• refactor(query): Introduces dedicated structs for expression types by @forsaken628 in #17719
• refactor: extract crate databendlabs/display-more to display various types by @drmingdrmer in #17721
• refactor(optimizer): rules by @BohuTANG in #17727
• refactor(optimizer): move rule/utils by @BohuTANG in #17731
• refactor: use Weak ref to break cyclic references by @drmingdrmer in #17743

Others 📒

• chore: show file path when copy meet invalid parquet file. by @youngsofun in #17716
• chore: impl Display for WatchResponse and related pb types by @drmingdrmer in #17725
• chore: add local build databend package by @wubx in #17737
• chore(query): delete no useful function epoch(Int64Type)->IntervalType by @TCeason in #17730
• chore: use Usage privilege optimize USEDB Plan by @TCeason in #17679
• chore(logger): upgrade logforth to support global max_files limit for log_dir by @BohuTANG in #17637
• chore: increase the concurrency of fuse_time_travel_size by @SkyFan2002 in #17745
• chore: Using an ordered BTreeSet instead of a HashSet as a ColumnSet implementation by @forsaken628 in #17747
• chore: Bump rust toolchain by @Xuanwo in #17751

Full Changelog: v1.2.720-nightly...v1.2.721-nightly