Skip to content

Commit c3a12ea

Browse files
authored
feat(genapi): document embedding dimensions size issues
1 parent 2332039 commit c3a12ea

File tree

1 file changed

+21
-2
lines changed

1 file changed

+21
-2
lines changed

pages/generative-apis/troubleshooting/fixing-common-issues.mdx

Lines changed: 21 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -120,7 +120,26 @@ Below are common issues that you may encounter when using Generative APIs, their
120120
- When displaying the Cockpit of a specific Project, but waiting for average token consumption to display:
121121
- Counter for **Tokens Processed** or **API Requests** should display a correct value (different from 0)
122122
- Graph across time should be empty
123-
```
123+
124+
## Embeddings vectors cannot be stored in database or used with a third-party library
125+
126+
### Cause
127+
- The embedding model you are using generates vector representations of fixed dimensions number, which is too high for your database or third-party library.
128+
- For example, the embedding model `bge-multilingual-gemma2` generates vector representations with `3584` dimensions. However, when storing vectors using PostgreSQL `pgvector` extensions, indexes (in `hnsw` or `ivvflat` formats) only support up to `2000` dimensions.
129+
130+
### Solution
131+
- Use a vector store supporting higher dimensions number, such as [Qdrant](https://www.scaleway.com/en/docs/tutorials/deploying-qdrant-vectordb-kubernetes/).
132+
- Do not use indexes for vectors or disable them from your third party library. This may limit performance in vector similarity search for significant volumes.
133+
- When using [Langchain PGVector method](https://python.langchain.com/docs/integrations/vectorstores/pgvector/), this method does not create index by default, and should not raise errors.
134+
- When using [Mastra](https://mastra.ai/) library with `vectorStoreName: "pgvector"`, specify indexConfig type as `flat` to avoid creating any index on vector dimensions.
135+
```typescript
136+
await vectorStore.createIndex({
137+
indexName: 'papers',
138+
dimension: 3584,
139+
indexConfig: {"type":"flat"},
140+
});
141+
```
142+
- Use a model with a lower number of dimensions. Using [Managed Inference](https://console.scaleway.com/inference/deployments), you can deploy for instance `sentence-t5-xxl` model which represent vectors with `768` dimensions.
124143

125144
## Best practices for optimizing model performance
126145

@@ -135,4 +154,4 @@ Below are common issues that you may encounter when using Generative APIs, their
135154
### Debugging silent errors
136155
- For cases where no explicit error is returned:
137156
- Verify all fields in the API request are correctly named and formatted.
138-
- Test the request with smaller and simpler inputs to isolate potential issues.
157+
- Test the request with smaller and simpler inputs to isolate potential issues.

0 commit comments

Comments
 (0)