Skip to content

Commit 23622ee

Browse files
committed
Added support for float16 and bfloat16. Renamed vector_convert_type to vector_as_type.
1 parent f2b8a4b commit 23622ee

File tree

10 files changed

+1918
-164
lines changed

10 files changed

+1918
-164
lines changed

API.md

Lines changed: 11 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -163,22 +163,23 @@ SELECT vector_cleanup('documents', 'embedding');
163163

164164
---
165165

166-
## `vector_convert_f32(value)`
166+
## `vector_as_f32(value)`
167167

168-
## `vector_convert_f16(value)`
168+
## `vector_as_f16(value)`
169169

170-
## `vector_convert_bf16(value)`
170+
## `vector_as_bf16(value)`
171171

172-
## `vector_convert_i8(value)`
172+
## `vector_as_i8(value)`
173173

174-
## `vector_convert_u8(value)`
174+
## `vector_as_u8(value)`
175175

176176
**Returns:** `BLOB`
177177

178178
**Description:**
179179
Encodes a vector into the required internal BLOB format to ensure correct storage and compatibility with the system’s vector representation.
180+
A real conversion is performed ONLY in case of JSON input. When input is a BLOB, it is assumed to be already properly formatted.
180181

181-
Functions in the `vector_convert_` family should be used in all `INSERT`, `UPDATE`, and `DELETE` statements to properly format vector values. However, they are *not* required when specifying input vectors for the `vector_full_scan` or `vector_quantize_scan` virtual tables.
182+
Functions in the `vector_as_` family should be used in all `INSERT`, `UPDATE`, and `DELETE` statements to properly format vector values. However, they are *not* required when specifying input vectors for the `vector_full_scan` or `vector_quantize_scan` virtual tables.
182183

183184
**Parameters:**
184185

@@ -193,10 +194,10 @@ Functions in the `vector_convert_` family should be used in all `INSERT`, `UPDAT
193194

194195
```sql
195196
-- Insert a Float32 vector using JSON
196-
INSERT INTO documents(embedding) VALUES(vector_convert_f32('[0.1, 0.2, 0.3]'));
197+
INSERT INTO documents(embedding) VALUES(vector_as_f32('[0.1, 0.2, 0.3]'));
197198

198199
-- Insert a UInt8 vector using raw BLOB (ensure correct formatting!)
199-
INSERT INTO compressed_vectors(embedding) VALUES(vector_convert_u8(X'010203'));
200+
INSERT INTO compressed_vectors(embedding) VALUES(vector_as_u8(X'010203'));
200201
```
201202

202203
---
@@ -219,7 +220,7 @@ Performs a brute-force nearest neighbor search using the given vector. Despite i
219220

220221
```sql
221222
SELECT rowid, distance
222-
FROM vector_full_scan('documents', 'embedding', vector_convert_f32('[0.1, 0.2, 0.3]'), 5);
223+
FROM vector_full_scan('documents', 'embedding', vector_as_f32('[0.1, 0.2, 0.3]'), 5);
223224
```
224225

225226
---
@@ -250,7 +251,7 @@ You **must run `vector_quantize()`** before using `vector_quantize_scan()` and w
250251

251252
```sql
252253
SELECT rowid, distance
253-
FROM vector_quantize_scan('documents', 'embedding', vector_convert_f32('[0.1, 0.2, 0.3]'), 10);
254+
FROM vector_quantize_scan('documents', 'embedding', vector_as_f32('[0.1, 0.2, 0.3]'), 10);
254255
```
255256

256257
---

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -66,7 +66,7 @@ CREATE TABLE images (
6666
INSERT INTO images (embedding, label) VALUES (?, 'cat');
6767

6868
-- Insert a JSON vector (Float32, 384 dimensions)
69-
INSERT INTO images (embedding, label) VALUES (vector_convert_f32('[0.3, 1.0, 0.9, 3.2, 1.4,...]'), 'dog');
69+
INSERT INTO images (embedding, label) VALUES (vector_as_f32('[0.3, 1.0, 0.9, 3.2, 1.4,...]'), 'dog');
7070

7171
-- Initialize the vector. By default, the distance function is L2.
7272
-- To use a different metric, specify one of the following options:

examples/semantic_search/semantic_search.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -124,7 +124,7 @@ def index_file(self, filepath: str) -> int:
124124
embedding_json = json.dumps(embedding.tolist())
125125

126126
cursor.execute(
127-
"INSERT INTO documents (filepath, content, embedding) VALUES (?, ?, vector_convert_f32(?))",
127+
"INSERT INTO documents (filepath, content, embedding) VALUES (?, ?, vector_as_f32(?))",
128128
(filepath, chunk, embedding_json)
129129
)
130130
chunk_count += 1
@@ -168,7 +168,7 @@ def search(self, query: str, limit: int = 3) -> Tuple[float, List[Tuple[str, str
168168
cursor.execute("""
169169
SELECT d.id, d.filepath, d.content, v.distance
170170
FROM documents AS d
171-
JOIN vector_quantize_scan('documents', 'embedding', vector_convert_f32(?), ?) AS v
171+
JOIN vector_quantize_scan('documents', 'embedding', vector_as_f32(?), ?) AS v
172172
ON d.id = v.rowid;
173173
""", (query_json, limit))
174174
elapsed_ms = round((time.time() - start_time) * 1000, 2)

0 commit comments

Comments
 (0)