Skip to content

Commit 49fb933

Browse files
authored
docs(colpali): add documentation for colpali embedding functions (#836)
1 parent d77ae24 commit 49fb933

File tree

1 file changed

+50
-0
lines changed

1 file changed

+50
-0
lines changed

docs/docs/ops/functions.md

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -188,3 +188,53 @@ Input data:
188188
* `text` (*Str*, required): The text to embed.
189189

190190
Return: *Vector[Float32, N]*, where *N* is the dimension of the embedding vector determined by the model.
191+
192+
## ColPaliEmbedImage
193+
194+
`ColPaliEmbedImage` embeds images using the ColPali multimodal model.
195+
196+
ColPali (Contextual Late-interaction over Patches) uses late interaction between image patch embeddings and text token embeddings for retrieval.
197+
198+
:::note Optional Dependency Required
199+
200+
This function requires the `colpali-engine` library, which is an optional dependency. Install CocoIndex with:
201+
202+
```bash
203+
pip install 'cocoindex[colpali]'
204+
```
205+
:::
206+
207+
The spec takes the following fields:
208+
209+
* `model` (`str`): The ColPali model name to use (e.g., "vidore/colpali-v1.2")
210+
211+
Input data:
212+
213+
* `img_bytes` (*Bytes*): The image data in bytes format.
214+
215+
Return: *Vector[Vector[Float32, N]]*, where *N* is the hidden dimension determined by the model. This returns a multi-vector format with variable patches and fixed hidden dimension.
216+
217+
## ColPaliEmbedQuery
218+
219+
`ColPaliEmbedQuery` embeds text queries using the ColPali multimodal model.
220+
221+
This produces query embeddings compatible with ColPali image embeddings for late interaction scoring (MaxSim).
222+
223+
:::note Optional Dependency Required
224+
225+
This function requires the `colpali-engine` library, which is an optional dependency. Install CocoIndex with:
226+
227+
```bash
228+
pip install 'cocoindex[colpali]'
229+
```
230+
:::
231+
232+
The spec takes the following fields:
233+
234+
* `model` (`str`): The ColPali model name to use (e.g., "vidore/colpali-v1.2")
235+
236+
Input data:
237+
238+
* `query` (*Str*): The text query to embed.
239+
240+
Return: *Vector[Vector[Float32, N]]*, where *N* is the hidden dimension determined by the model. This returns a multi-vector format with variable tokens and fixed hidden dimension.

0 commit comments

Comments
 (0)