v1.2.666-nightly

What's Changed

Exciting New Features ✨

• feat(query): Support geometry relation functions by @b41sh in #16927
• feat(query): virtual column allow cast to other type by @b41sh in #16903
• feat: copy support option COLUMN_MATCH_MODE by @youngsofun in #16963
• feat(query): speed up fetching mysql data from dictionaries via batch processing. by @Dragonliu2018 in #16948

Thoughtful Bug Fix 🔧

• fix(query): make memory engine as non-local table by @sundy-li in #16955

Code Refactor 🎉

• refactor: make test_watch_expired_events() less sensitive to inaccurate time by @drmingdrmer in #16966

Others 📒

• chore(storage): do compact before recluster during compact hook by @zhyass in #16949
• chore(ci): upgrade bendsql by @everpcpc in #16965
• chore(storage): refine error message for stream read offset snapshot by @zhyass in #16964
• chore: bump ruzstd from 0.7.2 to 0.7.3 by @dqhl76 in #16969
• chore(ci): install bendsql from packages by @everpcpc in #16970
• chore(planner): improve physical join by @Dousir9 in #16962

Full Changelog: v1.2.665-nightly...v1.2.666-nightly

v1.2.665-nightly

What's Changed

Thoughtful Bug Fix 🔧

• fix: lock attach table's snapshot location by @dantengsky in #16952
• fix(ci): fix release dbg upload path by @zhang2014 in #16958

Others 📒

• chore(storage): hook compact avoid scan all segments by @zhyass in #16954

Full Changelog: v1.2.664-nightly...v1.2.665-nightly

v1.2.664-nightly

What's Changed

Exciting New Features ✨

• feat(query): add new function TO_TIMESTAMP<int, scale> by @TCeason in #16924
• feat: disable specifying copy options when create stage. by @youngsofun in #16925
• feat(doc): Introducing Databend Guru on Gurubase.io by @kursataktas in #16773

Thoughtful Bug Fix 🔧

• fix(query): join predict use cast_expr_to_non_null_boolean by @sundy-li in #16937
• fix(ci): flaky test by @zhyass in #16933
• fix(query): add_hours function may panic if the argument is too big by @TCeason in #16929

Code Refactor 🎉

• refactor: remove ValueRef and use Value instead by @andylokandy in #16891

Build/Testing/CI Infra Changes 🔌

• ci: fix flaky test by @youngsofun in #16945

Others 📒

• chore: bump rustls version 0.23.18 by @TCeason in #16936
• chore(query): spilt binary symbol by @zhang2014 in #16861
• chore: k way merge sort add boundary check to prevent process crash by @forsaken628 in #16934
• chore(ci): remove aarch64 musl in release by @zhang2014 in #16946
• chore(planner): improve cardinality estimation by @Dousir9 in #16938
• chore(planner): refine project set cardinality by @Dousir9 in #16944
• chore: support recursive cte with normal cte by @xudong963 in #16932
• chore(binder): add enable_materialized_cte settings by @Dousir9 in #16950

New Contributors

• @kursataktas made their first contribution in #16773

Full Changelog: v1.2.663-nightly...v1.2.664-nightly

v1.2.663-nightly

What's Changed

Exciting New Features ✨

• feat: databend-meta adds more metrics about raft-log by @drmingdrmer in #16884
• feat(query): speed up fetching redis data from dictionaries via mget. by @Dragonliu2018 in #16766

Thoughtful Bug Fix 🔧

• fix(query): fix distinct set-returning function by @b41sh in #16883
• fix(query): fix incorrect total_bytes_len in string view by @sundy-li in #16877
• fix: StreamVersionMismatched when using CTAS in multi statement trans… by @SkyFan2002 in #16889
• fix(ci): flaky test by @zhyass in #16898
• fix(storge): write progress in compact hook by @zhyass in #16901
• fix: ensure atomicity in create_lock_revision by @zhyass in #16907
• fix: dropped temporary table should not be shown in system.temporary_tables by @SkyFan2002 in #16911

Code Refactor 🎉

• refactor: use jiff crate replace chrono in date/timestamp func by @TCeason in #16787
• refactor: add new raft-log metrics to "metactl status" response by @drmingdrmer in #16899
• refactor: remove obsolete gRPC read API by @drmingdrmer in #16909
• refactor: deprecate: Operation::AsIs and will be removed by @drmingdrmer in #16913
• refactor(sqlsmith): refactor sqlsmith using http client by @b41sh in #16890

Others 📒

• chore(ci): adjust runner size by @everpcpc in #16879
• chore(settings): enable spill by @Dousir9 in #16888
• chore: remove unused config_id from RaftConfig by @drmingdrmer in #16902
• chore(ci): flaky test by @zhyass in #16917
• chore(ci): enable overflow-checks in ci profile by @sundy-li in #16895
• chore(ci): flaky test by @zhyass in #16919
• chore(query): add snapshot logs in read partitions by @sundy-li in #16918

Full Changelog: v1.2.662-nightly...v1.2.663-nightly

v1.2.662-nightly

What's Changed

Thoughtful Bug Fix 🔧

• fix: raft-log wont write when exceeding 256M by @drmingdrmer in #16876

Code Refactor 🎉

• refactor: make ListKV non-blocking by returning stream directly by @drmingdrmer in #16868
• refactor: optimize query system.tables when query single table by @TCeason in #16869
• refactor(query): refactor geometry functions by @b41sh in #16870

Others 📒

• chore: enable stderr logging for table_meta_inspector by @dantengsky in #16865
• chore(query): remove common/arrow crate by @sundy-li in #16846

Full Changelog: v1.2.661-nightly...v1.2.662-nightly

v1.2.661-nightly

What's Changed

Exciting New Features ✨

• feat(query): add distributed pruning settings by @zhang2014 in #16864

Others 📒

• chore(query): improve distributed runtime filter by @Dousir9 in #16862

Full Changelog: v1.2.660-nightly...v1.2.661-nightly

v1.2.660-nightly

What's Changed

Exciting New Features ✨

• feat: add aws glue as an iceberg connection type by @Rowlandev in #16824

Code Refactor 🎉

• refactor: remove EmbeddedMeta by @drmingdrmer in #16847
• refactor: remove unused sled related codes by @drmingdrmer in #16857

Others 📒

• chore(query): remove useless new line for query log by @zhang2014 in #16860
• chore: add some log by @SkyFan2002 in #16863

New Contributors

• @Rowlandev made their first contribution in #16824

Full Changelog: v1.2.659-nightly...v1.2.660-nightly

v1.2.659-nightly

What's Changed

Exciting New Features ✨

• feat(query): Support rename dictionary ddl. by @Winnie-Hong0927 in #16754
• feat(query): add last_day, previous_day, next_day function by @Freejww in #16740
• feat: implement StringColumn using StringViewArray by @andylokandy in #16610
• feat: Implement WAL-based RaftLog storage by @drmingdrmer in #16776
• feat(query): expand trim/ltrim/rtrim by @TCeason in #16802
• feat: support pushdown predicate into iceberg engine by @sundy-li in #16650
• feat(query): TopN window operator by @forsaken628 in #16726
• feat(query): support SHOW DROP DATABASES by @TCeason in #16811
• feat: uuid function from v4 to v7 by @BohuTANG in #16827
• feat: refine the rule PushDownFilterWindowTopN in case top n is equal to 0. by @forsaken628 in #16830

Thoughtful Bug Fix 🔧

• fix: token and cookie session id miss match. by @youngsofun in #16786
• fix: clustering_information panic if string domain is none by @zhyass in #16792
• fix(query): fix broken file log layout by @zhang2014 in #16803
• fix: re-ensure dir before upgrading by @drmingdrmer in #16805
• fix: purge may not work on tables after a flash back operation by @SkyFan2002 in #16812
• fix(storage): recluster endless loop by @zhyass in #16831
• fix: incorrect table data disk cache key by @dantengsky in #16837
• fix(query): fix group by with alias column can't bind the column by @b41sh in #16804
• fix(cluster): fix the possibility of connection leak when the cluster state is broken. by @zhang2014 in #16842
• fix(query): fix aggregator_groups_builder to work with string view by @sundy-li in #16843
• fix(query): check storage request method with arg list-type by @everpcpc in #16849
• fix: replace risky dma_buffer_as_vec implementations by @forsaken628 in #16829
• fix(query): fix and check total_buffer_len and total_bytes_len by @sundy-li in #16854

Code Refactor 🎉

• refactor(base): add stacktrace to replace backtrace by @zhang2014 in #16643
• refactor: when upgrading, do not import already purged garbage log entries by @drmingdrmer in #16809
• refactor: align metrics field names with current behavior by @drmingdrmer in #16826
• refactor(base): support stacktrace in cluster mode by @zhang2014 in #16821
• refactor: refactor cte binder and fix materialized cte used in subquery by @xudong963 in #16785

Build/Testing/CI Infra Changes 🔌

• ci: Enable incremental back to see how it works by @Xuanwo in #16817

Documentation 📔

• docs: add stream to readme by @BohuTANG in #16810

Others 📒

• chore: remove obsolete version by @everpcpc in #16789
• chore: add snapshot location cache for attached table by @dantengsky in #16795
• chore: update meta-service version info and compatibility doc by @drmingdrmer in #16800
• chore(query): embedded public key of license by @zhang2014 in #16813
• chore(query): refactor common/arrow removes arrow/io crates by @sundy-li in #16808
• chore: purge dropped view metadata during vacuum by @dantengsky in #16819
• chore(query): keep queries detail and profile log format by @zhang2014 in #16822
• chore(ci): test with different license by @everpcpc in #16815
• chore(base): rollback to kernel signal handler after dump backtrace by @zhang2014 in #16828
• chore: remove unused codes by @zhyass in #16834
• chore: fix typo in ColumnCacheKeyBuilder by @dantengsky in #16838
• chore(cluster): disable parallel commit of cluster tasks by @zhang2014 in #16851
• chore(query): remove SliceExt by @sundy-li in #16853
• chore(ci): separate build profile by @everpcpc in #16855

Full Changelog: v1.2.654-nightly...v1.2.659-nightly

v1.2.654-nightly

What's Changed

Exciting New Features ✨

• feat(query): function about convert_timezone #16177 by @florann in #16181
• feat(functions): add new function: map_pick by @hanxuanliang in #15573
• feat(metrics): new storage_http_requests_count by @everpcpc in #16769
• feat(functions): add map_insert function by @hanxuanliang in #15567
• feat: filter null value before join by @xudong963 in #16722

Thoughtful Bug Fix 🔧

• fix: Files::remove_file_in_batch should not swallow errors by @dantengsky in #16761
• fix(query): add tls config for udf client by @everpcpc in #16782

Code Refactor 🎉

• refactor: change blocking pruning process to non-blocking channel by @dqhl76 in #16718
• refactor: ensure last-purged log-id cursor in exported from databend-meta service by @drmingdrmer in #16759
• refactor: Consolidate MetaStorageError variants by @drmingdrmer in #16760
• refactor(storage): refactor read virtual columns by @b41sh in #16758
• refactor: remove location_prefix, which is no longer supported by @dantengsky in #16757

Others 📒

• chore: make VACUUM TEMPORARY FILES killable by @dantengsky in #16751
• chore: tweak batch deletion concurrency by @dantengsky in #16770
• chore: refine raft-types import path by @drmingdrmer in #16764

New Contributors

• @florann made their first contribution in #16181

Full Changelog: v1.2.653-nightly...v1.2.654-nightly

v1.2.653-nightly

What's Changed

Exciting New Features ✨

• feat: HTTP handler use cookie to pass session id to and from clients. by @youngsofun in #16735
• feat: impl opendal::HttpFetch for StorageHttpClient by @everpcpc in #16738

Thoughtful Bug Fix 🔧

• fix: date_add/sub need respect timezone by @TCeason in #16721
• fix(query): remove single quotes for system.columns comment by @TCeason in #16732
• fix: HTTP handle lost rows when result set is wide and large. by @youngsofun in #16748
• fix: keep column statistics of all NULL column by @dantengsky in #16753

Code Refactor 🎉

• refactor: Enforce all deps must be declared at root by @Xuanwo in #16741
• refactor: Remove not used async-std by @Xuanwo in #16745

Others 📒

• chore: tweak cargo feature "jemalloc" by @dantengsky in #16693
• chore: remove flaky test by @TCeason in #16739
• chore: refine vacuum by @dantengsky in #16743
• chore: Remove not used dep config by @Xuanwo in #16746
• chore(query): Introduce VecExt and SliceExt to check unsafe bounds in debug mode by @sundy-li in #16747

Full Changelog: v1.2.652-nightly...v1.2.653-nightly