Skip to content

Commit 363bbbe

Browse files
committed
backfill: merge vectors from a temp forward index into a vector index
Previously mutation of values in a vector index during backfill was prohibited, effectively taking the table offline while the index is built. This patch does a few things: * Re-enables mutations on vector indexes undergoing backfill. * Changes the temporary index created for vector indexes to capture changes during backfill from a VECTOR index to a FORWARD index. * Changes the schema of the temporary index so that it is keyed by the index prefix and primary key columns, with the vector as a payload column. * Refactors the mvcc merger to take a functor argument to merge individual entries, with the existing kv merger as the default. * Adds a second merger that re-encodes vector index entries read from the temporary index. Because forward indexes don't record values for tombstone entries, we don't delete vectors from the vector index that are deleted during backfill. Since reading from a vector index requires an index join on the PK, this doesn't create a correctness issue, but it could cause accuracy issues if a large number of vector index entries are deleted during backfill. Fixes: #144443 Release note (sql change): Tables with vector indexes will no longer be taken offline while the vector index builds.
1 parent cf87d27 commit 363bbbe

File tree

11 files changed

+775
-99
lines changed

11 files changed

+775
-99
lines changed

pkg/sql/backfill/BUILD.bazel

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,7 @@ go_library(
4040
"//pkg/sql/sem/idxtype",
4141
"//pkg/sql/sem/transform",
4242
"//pkg/sql/sem/tree",
43+
"//pkg/sql/span",
4344
"//pkg/sql/sqlerrors",
4445
"//pkg/sql/types",
4546
"//pkg/sql/vecindex",

0 commit comments

Comments
 (0)