Skip to content

Commit 15ea8a8

Browse files
committed
feature #198 [Store][PG] Allow to change vector type and index type (lyrixx)
This PR was merged into the main branch. Discussion ---------- [Store][PG] Allow to change vector type and index type | Q | A | ------------- | --- | Bug fix? | no | New feature? | yes | Docs? | no | Issues | | License | MIT Gemini gives me vector of 3072 dimms. But PG vector can' handle that much dimms. [ref](pgvector/pgvector#461) So a workarround is to change the vectory type, and so the index. Example: ``` app=# \d gemini Table "public.gemini" Column | Type | Collation | Nullable | Default -----------+---------------+-----------+----------+--------- id | uuid | | not null | metadata | jsonb | | | embedding | halfvec(3072) | | not null | Indexes: "gemini_pkey" PRIMARY KEY, btree (id) "gemini_embedding_idx" hnsw (embedding halfvec_cosine_ops) app=# select count(*) from gemini ; count ------- 384 (1 row) ``` Commits ------- 83d27e8 [Store][PG] Allow to change vector type and index type
2 parents 4d98e7b + 83d27e8 commit 15ea8a8

File tree

1 file changed

+12
-2
lines changed

1 file changed

+12
-2
lines changed

src/store/src/Bridge/Postgres/Store.php

Lines changed: 12 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -120,6 +120,13 @@ public function query(Vector $vector, array $options = [], ?float $minScore = nu
120120
return $documents;
121121
}
122122

123+
/**
124+
* @param array{vector_type?: string, vector_size?: positive-int, index_method?: string, index_opclass?: string} $options
125+
*
126+
* Good configurations $options are:
127+
* - For Mistral: ['vector_size' => 1024]
128+
* - For Gemini: ['vector_type' => 'halfvec', 'vector_size' => 3072, 'index_method' => 'hnsw', 'index_opclass' => 'halfvec_cosine_ops']
129+
*/
123130
public function initialize(array $options = []): void
124131
{
125132
$this->connection->exec('CREATE EXTENSION IF NOT EXISTS vector');
@@ -129,20 +136,23 @@ public function initialize(array $options = []): void
129136
'CREATE TABLE IF NOT EXISTS %s (
130137
id UUID PRIMARY KEY,
131138
metadata JSONB,
132-
%s vector(%d) NOT NULL
139+
%s %s(%d) NOT NULL
133140
)',
134141
$this->tableName,
135142
$this->vectorFieldName,
143+
$options['vector_type'] ?? 'vector',
136144
$options['vector_size'] ?? 1536,
137145
),
138146
);
139147
$this->connection->exec(
140148
\sprintf(
141-
'CREATE INDEX IF NOT EXISTS %s_%s_idx ON %s USING ivfflat (%s vector_cosine_ops)',
149+
'CREATE INDEX IF NOT EXISTS %s_%s_idx ON %s USING %s (%s %s)',
142150
$this->tableName,
143151
$this->vectorFieldName,
144152
$this->tableName,
153+
$options['index_method'] ?? 'ivfflat',
145154
$this->vectorFieldName,
155+
$options['index_opclass'] ?? 'vector_cosine_ops',
146156
),
147157
);
148158
}

0 commit comments

Comments
 (0)