Unify vector setter dispatch - #164
Draft
mlafeldt wants to merge 4 commits into
Draft
Conversation
Member
Author
|
@taniabogatsch I think this breaking change is justified. Agree? |
Route every vector write through the setter selected during vector initialization. Remove the duplicate type switch and its identity wrapper, and drop generic parameters from setters that now always receive any values. This adds ARRAY support to SetChunkValue and makes nil values set SQL NULL instead of panicking. BREAKING: SetChunkValue and SetRowValue no longer treat strings as raw JSON documents. Strings are marshaled as JSON string values, maps and structs as JSON objects, and pre-serialized documents should use json.RawMessage. This matches DataChunk.SetValue and appender behavior.
mlafeldt
force-pushed
the
unify-vector-setter-dispatch
branch
from
July 24, 2026 13:34
9a0d3a4 to
406940d
Compare
Member
|
The breaking change is fine by me. Another question, I remember the PR that added the |
mlafeldt
force-pushed
the
unify-vector-setter-dispatch
branch
from
July 24, 2026 15:33
406940d to
7d1b75b
Compare
marcboeker#276 made SetChunkValue generic to avoid allocating when table UDFs write values. Routing every write through the any-valued callback restored correctness locality but boxed each generic value. Install optional exact-type callbacks for each vector canonical Go type, with setFn as the fallback for all other values, and keep numeric conversion setters generic. The numeric optimization defaults to the vector-installed setFn, so types it does not recognize retain the single correctness dispatch in vector initialization. Add benchmarks for both public table UDF write paths and a matrix covering cross-numeric and representative canonical writes. These paths now match or improve on main allocation counts.
mlafeldt
force-pushed
the
unify-vector-setter-dispatch
branch
from
July 24, 2026 16:11
7d1b75b to
f877027
Compare
Let HUGEINT, UHUGEINT, and BIGNUM writes use the vector-installed setter instead of the numeric fast path. Their setters normalize typed-nil *big.Int values to SQL NULL, so intercepting them changed SetChunkValue semantics and violated the fallback invariant. Document that exact typed setters bypass nil normalization and that the numeric optimization only covers setters with an interface-nil guard. Clarify how SetChunkValue can avoid boxing exact canonical values, and align ENUM initialization with the other typed setters.
Move the generic SetChunkValue benchmark helper and type matrix into one file so the exact BIGINT case does not duplicate setup. Cover []byte-to-VARCHAR and string-to-BLOB fallback writes explicitly. Canonical fast paths preserve their allocation behavior, while supported cross-type fallback writes may box. The benchmark suite now exposes that boundary instead of implying every accepted conversion is allocation-free.
mlafeldt
marked this pull request as draft
July 24, 2026 16:40
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Use the setter selected during vector initialization as the single correctness dispatch for chunk, row, appender, and table UDF writes.
ARRAYsupport toSetChunkValue.*big.Int, as SQL NULL.BREAKING: JSON writes now match
SetValueand appenders. Strings become JSON strings,[]bytebecomes a base64-encoded JSON string, and maps or structs become JSON objects. Usejson.RawMessagefor pre-serialized JSON documents.mainstring("hello")hello; reading returns an API error wrappingjson.SyntaxError"hello"; reads back as Go string"hello"string("{\"a\":1}"){"a":1}; reads back as a map"{\"a\":1}"; reads back as Go string{"a":1}[]byte("{\"a\":1}"){"a":1}; reads back as a map"eyJhIjoxfQ=="; reads back as Go string"eyJhIjoxfQ=="map[string]any{"a": 1}{"a":1}; reads back as a mapjson.RawMessage("{\"a\":1}"){"a":1}; reads back as a map