Skip to content

Commit 61d7376

Browse files
jmhsiehclaude
jmhsieh
and
claude
committed
docs: add built-in UDF provider pages for OpenAI, Gemini, Sentence Transformers
Add dedicated documentation pages for each built-in UDF provider with usage examples and GPU acceleration guidance. Clarify that fractional num_gpus is a Ray scheduling mechanism, not GPU memory partitioning. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 parent 5a45fbe commit 61d7376

File tree

6 files changed

+308
-0
lines changed

6 files changed

+308
-0
lines changed

‎docs/docs.json‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -156,6 +156,15 @@
156156
"group": "User Defined Functions (UDFs)",
157157
"pages": [
158158
"geneva/udfs/index",
159+
{
160+
"group": "Built-in Providers",
161+
"pages": [
162+
"geneva/udfs/providers/index",
163+
"geneva/udfs/providers/openai",
164+
"geneva/udfs/providers/gemini",
165+
"geneva/udfs/providers/sentence-transformers"
166+
]
167+
},
159168
"geneva/udfs/blobs",
160169
"geneva/udfs/error_handling",
161170
"geneva/udfs/advanced-configuration"
@@ -383,6 +392,10 @@
383392
{
384393
"source": "/tutorials/vector-search/:slug*",
385394
"destination": "tutorials/search/:slug*"
395+
},
396+
{
397+
"source": "/geneva/udfs/built-in",
398+
"destination": "/geneva/udfs/providers"
386399
}
387400
]
388401
}

‎docs/geneva/udfs/index.mdx‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -105,6 +105,10 @@ class OpenAIEmbedding(Callable):
105105
return pa.array(resp.data[0].embeddings)
106106
```
107107

108+
<Tip>
109+
For common providers like OpenAI and Gemini, Geneva ships [built-in UDFs](/geneva/udfs/providers) that handle API keys, retries, and batching for you — no custom class needed.
110+
</Tip>
111+
108112
> **Note**: The state is will be independently managed on each distributed Worker.
109113
110114
## UDF options

‎docs/geneva/udfs/providers/gemini.mdx‎

Lines changed: 86 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,86 @@
1+
---
2+
title: Gemini
3+
sidebarTitle: Gemini
4+
icon: google
5+
---
6+
7+
Embed text and generate completions using Google's Gemini models.
8+
See the API reference for [Gemini UDFs](https://lancedb.github.io/geneva/api/gemini/) and
9+
[Embedding UDFs](https://lancedb.github.io/geneva/api/embeddings/) for all parameters.
10+
11+
```python
12+
pip install 'geneva[udf-text-gemini]'
13+
```
14+
15+
<Warning>
16+
Gemini UDFs make API calls that incur **per-token costs**. Each row processed results in one
17+
or more API requests billed to your account. Review
18+
[Gemini pricing](https://ai.google.dev/gemini-api/docs/pricing) before running on large tables.
19+
</Warning>
20+
21+
<Note>
22+
Set the `GEMINI_API_KEY` environment variable before calling any factory function below.
23+
The key is read **at UDF creation time** and serialized with the UDF — no cluster-level
24+
`env_vars` configuration is needed.
25+
</Note>
26+
27+
## Embeddings
28+
29+
Embed text with optional task-type hints for retrieval, classification, and clustering scenarios.
30+
See the [API reference](https://lancedb.github.io/geneva/api/embeddings/#geneva.udfs.text.embeddings.gemini_embedding_udf) for all parameters.
31+
32+
**Multiple embeddings tuned for different retrieval tasks:**
33+
34+
```python
35+
from geneva.udfs import gemini_embedding_udf
36+
37+
table.add_columns({
38+
# Full-dimension embedding for document retrieval
39+
"embedding_doc": gemini_embedding_udf(
40+
column="body",
41+
model="gemini-embedding-001",
42+
task_type="RETRIEVAL_DOCUMENT",
43+
),
44+
# Compact embedding for semantic similarity
45+
"embedding_sim_256": gemini_embedding_udf(
46+
column="body",
47+
model="gemini-embedding-001",
48+
task_type="SEMANTIC_SIMILARITY",
49+
output_dimensionality=256,
50+
),
51+
})
52+
```
53+
54+
## Generation
55+
56+
Generate text from Gemini models. Supports text, image, audio, video, and document inputs.
57+
See the [API reference](https://lancedb.github.io/geneva/api/gemini/#geneva.udfs.text.gemini.gemini_udf) for all parameters.
58+
59+
**Enrich a table with sentiment, captions, and transcriptions at once:**
60+
61+
```python
62+
from geneva.udfs import gemini_udf
63+
64+
table.add_columns({
65+
# Classify review sentiment with a fast model
66+
"sentiment": gemini_udf(
67+
column="review",
68+
prompt="Classify the sentiment as positive, negative, or neutral. Return only the label.",
69+
model="gemini-2.5-flash",
70+
),
71+
# Caption product images with a more capable model
72+
"caption": gemini_udf(
73+
column="image",
74+
prompt="Describe the main subject of this image in one sentence",
75+
model="gemini-2.5-pro",
76+
mime_type="image/jpeg",
77+
),
78+
# Transcribe audio clips
79+
"transcript": gemini_udf(
80+
column="audio",
81+
prompt="Transcribe this audio clip",
82+
model="gemini-2.5-flash",
83+
mime_type="audio/mp3",
84+
),
85+
})
86+
```

‎docs/geneva/udfs/providers/index.mdx‎

Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,66 @@
1+
---
2+
title: Built-in LLM and Embedding UDFs
3+
sidebarTitle: Overview
4+
icon: sparkles
5+
---
6+
7+
Geneva ships pre-built UDFs for common LLM providers so you don't have to write custom classes
8+
for everyday embedding and generation tasks.
9+
10+
| Provider | Embeddings | Generation | Runs locally | Install extra |
11+
|----------|:----------:|:----------:|:------------:|---------------|
12+
| [OpenAI](/geneva/udfs/providers/openai) | ✓ | ✓ | — | `geneva[udf-text-openai]` |
13+
| [Gemini](/geneva/udfs/providers/gemini) | ✓ | ✓ | — | `geneva[udf-text-gemini]` |
14+
| [Sentence Transformers](/geneva/udfs/providers/sentence-transformers) | ✓ | — | ✓ | `geneva[udf-text-sentence-transformers]` |
15+
16+
OpenAI and Gemini UDFs make remote API calls that incur per-token costs.
17+
Sentence Transformers run locally on your workers with no API costs — see
18+
[GPU acceleration](/geneva/udfs/providers/sentence-transformers#gpu-acceleration)
19+
for performance tips.
20+
21+
## Comparing models and prompts
22+
23+
Because `add_columns` accepts a dictionary, you can evaluate multiple models, parameter
24+
settings, or prompts in a single pass over your data. Each entry produces its own column,
25+
so results sit side by side in the same table for easy comparison.
26+
27+
```python
28+
from geneva.udfs import openai_udf, gemini_udf, openai_embedding_udf
29+
30+
table.add_columns({
31+
# Compare two embedding models
32+
"emb_small": openai_embedding_udf(column="body", model="text-embedding-3-small"),
33+
"emb_large": openai_embedding_udf(column="body", model="text-embedding-3-large"),
34+
35+
# Compare the same task across providers
36+
"summary_openai": openai_udf(
37+
column="body",
38+
prompt="Summarize in one sentence",
39+
model="gpt-5-mini",
40+
),
41+
"summary_gemini": gemini_udf(
42+
column="body",
43+
prompt="Summarize in one sentence",
44+
model="gemini-2.5-flash",
45+
),
46+
})
47+
```
48+
49+
This works for any combination — different models from the same provider, different providers,
50+
different prompts with the same model, or different dimensionality settings. All columns are
51+
computed in parallel during the same backfill job.
52+
53+
## What's included
54+
55+
All built-in UDFs share these capabilities:
56+
57+
- **API key handling** — Keys are captured from your local environment at UDF creation time and serialized with the UDF. No cluster-level environment configuration required.
58+
- **Retry with backoff** — Transient API errors (rate limits, timeouts, server errors) are automatically retried with exponential backoff.
59+
- **Batch processing** — Embedding UDFs batch multiple rows per API call for better throughput.
60+
- **L2 normalization** — Embedding UDFs support optional L2 normalization via the `normalize` parameter (disabled by default since both providers return pre-normalized vectors).
61+
62+
## See also
63+
64+
- [Working with UDFs](/geneva/udfs/index) — Write custom scalar, batched, and stateful UDFs
65+
- [Error handling](/geneva/udfs/error_handling) — Fine-grained retry and skip policies
66+
- [Working with blobs](/geneva/udfs/blobs) — Process binary data (images, audio, video)

‎docs/geneva/udfs/providers/openai.mdx‎

Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
---
2+
title: OpenAI
3+
sidebarTitle: OpenAI
4+
icon: brain
5+
---
6+
7+
Embed text and generate completions using OpenAI models.
8+
See the [API reference](https://lancedb.github.io/geneva/api/openai/) for all parameters.
9+
10+
```python
11+
pip install 'geneva[udf-text-openai]'
12+
```
13+
14+
<Warning>
15+
OpenAI UDFs make API calls that incur **per-token costs**. Each row processed results in one
16+
or more API requests billed to your account. Review
17+
[OpenAI pricing](https://openai.com/api/pricing/) before running on large tables.
18+
</Warning>
19+
20+
<Note>
21+
Set the `OPENAI_API_KEY` environment variable before calling any factory function below.
22+
The key is read **at UDF creation time** and serialized with the UDF — no cluster-level
23+
`env_vars` configuration is needed.
24+
</Note>
25+
26+
## Embeddings
27+
28+
**Compare models by adding multiple embedding columns at once:**
29+
30+
```python
31+
from geneva.udfs import openai_embedding_udf
32+
33+
table.add_columns({
34+
# Default model — fast, 1536 dimensions
35+
"embedding_small": openai_embedding_udf(
36+
column="body",
37+
model="text-embedding-3-small",
38+
),
39+
# Higher-quality model — 3072 dimensions
40+
"embedding_large": openai_embedding_udf(
41+
column="body",
42+
model="text-embedding-3-large",
43+
),
44+
# Same large model, truncated to 256 dimensions for storage efficiency
45+
"embedding_large_256": openai_embedding_udf(
46+
column="body",
47+
model="text-embedding-3-large",
48+
output_dimensionality=256,
49+
),
50+
})
51+
```
52+
53+
## Generation
54+
55+
Generate text from OpenAI chat completion models. Supports both text and binary (image)
56+
input columns.
57+
See the [API reference](https://lancedb.github.io/geneva/api/openai/#geneva.udfs.openai.openai_udf) for all parameters.
58+
59+
**Add a summary and an image caption in one call, using different models:**
60+
61+
```python
62+
from geneva.udfs import openai_udf
63+
64+
table.add_columns({
65+
# Fast model for bulk text summarization
66+
"summary": openai_udf(
67+
column="body",
68+
prompt="Summarize this document in 3 bullet points",
69+
model="gpt-5-mini",
70+
),
71+
# More capable model for nuanced image captions
72+
"caption": openai_udf(
73+
column="image",
74+
prompt="Provide a 1 sentence description of the scene",
75+
model="gpt-5",
76+
mime_type="image/jpeg",
77+
),
78+
})
79+
```

‎docs/geneva/udfs/providers/sentence-transformers.mdx‎

Lines changed: 60 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,60 @@
1+
---
2+
title: Sentence Transformers
3+
sidebarTitle: Sentence Transformers
4+
icon: microchip
5+
---
6+
7+
Embed text using any HuggingFace Sentence Transformer model locally — no API key needed.
8+
See the [API reference](https://lancedb.github.io/geneva/api/embeddings/#geneva.udfs.text.embeddings.sentence_transformer_udf) for all parameters.
9+
10+
```python
11+
pip install 'geneva[udf-text-sentence-transformers]'
12+
```
13+
14+
<Tip>
15+
Sentence Transformer models run **locally** on your workers — there are no API calls and no
16+
per-token costs. This makes them a good fit for large-scale embedding jobs where cost is a
17+
concern.
18+
</Tip>
19+
20+
## Embeddings
21+
22+
**Compare a lightweight and a high-quality model side by side:**
23+
24+
```python
25+
from geneva.udfs import sentence_transformer_udf
26+
27+
table.add_columns({
28+
# Lightweight default model — fast, CPU-friendly
29+
"embedding_mini": sentence_transformer_udf(
30+
column="body",
31+
model="sentence-transformers/all-MiniLM-L6-v2",
32+
),
33+
# Larger model with GPU acceleration
34+
"embedding_bge": sentence_transformer_udf(
35+
column="body",
36+
model="BAAI/bge-large-en-v1.5",
37+
num_gpus=1.0,
38+
),
39+
})
40+
```
41+
42+
## GPU acceleration
43+
44+
Sentence Transformer models can run on CPU or GPU. Smaller models like `all-MiniLM-L6-v2`
45+
work well on CPU, but larger models like `bge-large-en-v1.5` benefit significantly from GPU
46+
acceleration. Use the `num_gpus` parameter to request GPU resources for a worker:
47+
48+
```python
49+
# CPU-only (default) — suitable for lightweight models
50+
sentence_transformer_udf(column="body", model="sentence-transformers/all-MiniLM-L6-v2")
51+
52+
# GPU-accelerated — recommended for larger models
53+
sentence_transformer_udf(column="body", model="BAAI/bge-large-en-v1.5", num_gpus=1.0)
54+
```
55+
56+
Setting `num_gpus` to a fractional value (e.g., `0.5`) tells the
57+
[Ray scheduler](https://docs.ray.io/en/latest/ray-core/scheduling/accelerators.html)
58+
to co-locate multiple workers on the same physical GPU. For example, two UDFs with
59+
`num_gpus=0.5` will be scheduled on a single GPU. Note that Ray does not enforce GPU memory
60+
limits — it is your responsibility to ensure the combined models fit in GPU memory.

0 commit comments

Comments
 (0)