Skip to content

Commit 2660bac

Browse files
lastzerolastzero
committed
AI: Use OLLAMA_API_KEY as API auth token if specified photoprism#5361
Signed-off-by: Michael Mayer <michael@photoprism.app>
1 parent d4aef5c commit 2660bac

File tree

14 files changed

+367
-199
lines changed

14 files changed

+367
-199
lines changed

‎internal/ai/vision/README.md‎

Lines changed: 48 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -53,41 +53,59 @@ The `vision.yml` file is usually kept in the `storage/config` directory (overrid
5353

5454
The model `Options` adjust model parameters such as temperature, top-p, and schema constraints when using [Ollama](ollama/README.md) or [OpenAI](openai/README.md):
5555

56-
| Option | Default | Description |
57-
|-------------------|-----------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------|
58-
| `Temperature` | engine default (`0.1` for Ollama) | Controls randomness with a value between `0.01` and `2.0`; not used for OpenAI's GPT-5. |
59-
| `TopK` | engine default (model-specific) | Limits sampling to the top K tokens to reduce rare or noisy outputs. |
60-
| `TopP` | engine default (`0.9` for some Ollama label defaults; unset for OpenAI) | Nucleus sampling; keeps the smallest token set whose cumulative probability ≥ `p`. |
61-
| `MinP` | engine default (unset unless provided) | Drops tokens whose probability mass is below `p`, trimming the long tail. |
62-
| `TypicalP` | engine default (unset unless provided) | Keeps tokens with typicality under the threshold; combine with TopP/MinP for flow. |
63-
| `Seed` | random per run (unless set) | Fix for reproducible outputs; unset for more variety between runs. |
64-
| `RepeatLastN` | engine default (model-specific) | Number of recent tokens considered for repetition penalties. |
65-
| `RepeatPenalty` | engine default (model-specific) | Multiplier >1 discourages repeating the same tokens or phrases. |
66-
| `NumPredict` | engine default (Ollama only) | Ollama-specific max output tokens; synonymous intent with `MaxOutputTokens`. |
67-
| `MaxOutputTokens` | engine default (OpenAI caption 512, labels 1024) | Upper bound on generated tokens; adapters raise low values to defaults. |
68-
| `ForceJson` | engine-specific (`true` for OpenAI labels; `false` for Ollama labels; captions `false`) | Forces structured output when enabled. |
69-
| `SchemaVersion` | derived from schema name | Override when coordinating schema migrations. |
70-
| `Stop` | engine default | Array of stop sequences (e.g., `["\\n\\n"]`). |
71-
| `NumThread` | runtime auto | Caps CPU threads for local engines. |
72-
| `NumCtx` | engine default | Context window length (tokens). |
56+
| Option | Default | Description |
57+
|--------------------|-----------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------|
58+
| `Temperature` | engine default (`0.1` for Ollama) | Controls randomness with a value between `0.01` and `2.0`; not used for OpenAI's GPT-5. |
59+
| `TopK` | engine default (model-specific) | Limits sampling to the top K tokens to reduce rare or noisy outputs. |
60+
| `TopP` | engine default (`0.9` for some Ollama label defaults; unset for OpenAI) | Nucleus sampling; keeps the smallest token set whose cumulative probability ≥ `p`. |
61+
| `MinP` | engine default (unset unless provided) | Drops tokens whose probability mass is below `p`, trimming the long tail. |
62+
| `TypicalP` | engine default (unset unless provided) | Keeps tokens with typicality under the threshold; combine with TopP/MinP for flow. |
63+
| `Seed` | random per run (unless set) | Fix for reproducible outputs; unset for more variety between runs. |
64+
| `RepeatLastN` | engine default (model-specific) | Number of recent tokens considered for repetition penalties. |
65+
| `RepeatPenalty` | engine default (model-specific) | Multiplier >1 discourages repeating the same tokens or phrases. |
66+
| `PenalizeNewline` | engine default | Whether to apply repetition penalties to newline tokens. |
67+
| `PresencePenalty` | engine default (OpenAI-style) | Increases the likelihood of introducing new tokens by penalizing existing ones. |
68+
| `FrequencyPenalty` | engine default (OpenAI-style) | Penalizes tokens in proportion to their frequency so far. |
69+
| `TfsZ` | engine default | Tail free sampling parameter; lower values reduce repetition. |
70+
| `NumKeep` | engine default (Ollama) | How many tokens to keep from the prompt before sampling starts. |
71+
| `NumPredict` | engine default (Ollama only) | Ollama-specific max output tokens; synonymous intent with `MaxOutputTokens`. |
72+
| `MaxOutputTokens` | engine default (OpenAI caption 512, labels 1024) | Upper bound on generated tokens; adapters raise low values to defaults. |
73+
| `ForceJson` | engine-specific (`true` for OpenAI labels; `false` for Ollama labels; captions `false`) | Forces structured output when enabled. |
74+
| `SchemaVersion` | derived from schema name | Override when coordinating schema migrations. |
75+
| `Stop` | engine default | Array of stop sequences (e.g., `["\\n\\n"]`). |
76+
| `NumThread` | runtime auto | Caps CPU threads for local engines. |
77+
| `NumCtx` | engine default | Context window length (tokens). |
78+
| `Mirostat` | engine default (Ollama) | Enables Mirostat sampling (`0` off, `1/2` modes). |
79+
| `MirostatTau` | engine default | Controls surprise target for Mirostat sampling. |
80+
| `MirostatEta` | engine default | Learning rate for Mirostat adaptation. |
81+
| `NumBatch` | engine default (Ollama) | Batch size for prompt processing. |
82+
| `NumGpu` | engine default (Ollama) | Number of GPUs to distribute work across. |
83+
| `MainGpu` | engine default (Ollama) | Primary GPU index when multiple GPUs are present. |
84+
| `LowVram` | engine default (Ollama) | Enable VRAM-saving mode; may reduce performance. |
85+
| `VocabOnly` | engine default (Ollama) | Load vocabulary only for quick metadata inspection. |
86+
| `UseMmap` | engine default (Ollama) | Memory map model weights instead of fully loading them. |
87+
| `UseMlock` | engine default (Ollama) | Lock model weights in RAM to reduce paging. |
88+
| `Numa` | engine default (Ollama) | Enable NUMA-aware allocations when available. |
89+
| `Detail` | engine default (OpenAI) | Controls OpenAI vision detail level (`low`, `high`, `auto`). |
90+
| `CombineOutputs` | engine default (OpenAI multi-output) | Controls whether multi-output models combine results automatically. |
7391

7492
#### Model Service
7593

7694
Configures the endpoint URL, method, format, and authentication for [Ollama](ollama/README.md), [OpenAI](openai/README.md), and other engines that perform remote HTTP requests:
7795

78-
| Field | Default | Notes |
79-
|------------------------------------|------------------------------------------|------------------------------------------------------|
80-
| `Uri` | required for remote | Endpoint base. Empty keeps model local (TensorFlow). |
81-
| `Method` | `POST` | Override verb if provider needs it. |
82-
| `Key` | `""` | Bearer token; prefer env expansion. |
83-
| `Username` / `Password` | `""` | Injected as basic auth when URI lacks userinfo. |
84-
| `Model` | `""` | Endpoint-specific override; wins over model/name. |
85-
| `Org` / `Project` | `""` | OpenAI headers (org/proj IDs) |
86-
| `RequestFormat` / `ResponseFormat` | set by engine alias | Explicit values win over alias defaults. |
87-
| `FileScheme` | set by engine alias (`data` or `base64`) | Controls image transport. |
88-
| `Disabled` | `false` | Disable the endpoint without removing the model. |
89-
90-
> **Authentication:** All credentials and identifiers support `${ENV_VAR}` expansion. `Service.Key` sets `Authorization: Bearer <token>`; `Username`/`Password` injects HTTP basic authentication into the service URI when it is not already present.
96+
| Field | Default | Notes |
97+
|------------------------------------|------------------------------------------|------------------------------------------------------------------------------------------|
98+
| `Uri` | required for remote | Endpoint base. Empty keeps model local (TensorFlow). |
99+
| `Method` | `POST` | Override verb if provider needs it. |
100+
| `Key` | `""` | Bearer token; prefer env expansion (OpenAI: `OPENAI_API_KEY`, Ollama: `OLLAMA_API_KEY`). |
101+
| `Username` / `Password` | `""` | Injected as basic auth when URI lacks userinfo. |
102+
| `Model` | `""` | Endpoint-specific override; wins over model/name. |
103+
| `Org` / `Project` | `""` | OpenAI headers (org/proj IDs) |
104+
| `RequestFormat` / `ResponseFormat` | set by engine alias | Explicit values win over alias defaults. |
105+
| `FileScheme` | set by engine alias (`data` or `base64`) | Controls image transport. |
106+
| `Disabled` | `false` | Disable the endpoint without removing the model. |
107+
108+
> **Authentication:** All credentials and identifiers support `${ENV_VAR}` expansion. `Service.Key` sets `Authorization: Bearer <token>`; `Username`/`Password` injects HTTP basic authentication into the service URI when it is not already present. When `Service.Key` is empty, PhotoPrism defaults to `OPENAI_API_KEY` (OpenAI engine) or `OLLAMA_API_KEY` (Ollama engine), also honoring their `_FILE` counterparts.
91109
92110
### Field Behavior & Precedence
93111

‎internal/ai/vision/engine.go‎

Lines changed: 1 addition & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,6 @@ import (
55
"strings"
66
"sync"
77

8-
"github.com/photoprism/photoprism/internal/ai/vision/openai"
98
"github.com/photoprism/photoprism/pkg/http/scheme"
109
)
1110

@@ -61,14 +60,6 @@ func init() {
6160
FileScheme: scheme.Data,
6261
DefaultResolution: DefaultResolution,
6362
})
64-
65-
RegisterEngineAlias(openai.EngineName, EngineInfo{
66-
Uri: "https://api.openai.com/v1/responses",
67-
RequestFormat: ApiFormatOpenAI,
68-
ResponseFormat: ApiFormatOpenAI,
69-
FileScheme: scheme.Data,
70-
DefaultResolution: openai.DefaultResolution,
71-
})
7263
}
7364

7465
// RegisterEngine adds/overrides an engine implementation for a specific API format.
@@ -85,6 +76,7 @@ type EngineInfo struct {
8576
ResponseFormat ApiFormat
8677
FileScheme string
8778
DefaultResolution int
79+
DefaultKey string // Optional placeholder key (e.g., ${OPENAI_API_KEY}); applied only when Service.Key is empty.
8880
}
8981

9082
// RegisterEngineAlias maps a logical engine name (e.g., "ollama") to a

‎internal/ai/vision/engine_ollama.go‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,7 @@ func init() {
3030
ResponseFormat: ApiFormatOllama,
3131
FileScheme: scheme.Base64,
3232
DefaultResolution: ollama.DefaultResolution,
33+
DefaultKey: ollama.APIKeyPlaceholder,
3334
})
3435

3536
CaptionModel.Engine = ollama.EngineName

‎internal/ai/vision/engine_openai.go‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,15 @@ func init() {
2828
Parser: openaiParser{},
2929
Defaults: openaiDefaults{},
3030
})
31+
32+
RegisterEngineAlias(openai.EngineName, EngineInfo{
33+
Uri: "https://api.openai.com/v1/responses",
34+
RequestFormat: ApiFormatOpenAI,
35+
ResponseFormat: ApiFormatOpenAI,
36+
FileScheme: scheme.Data,
37+
DefaultResolution: openai.DefaultResolution,
38+
DefaultKey: openai.APIKeyPlaceholder,
39+
})
3140
}
3241

3342
// SystemPrompt returns the default OpenAI system prompt for the specified model type.

‎internal/ai/vision/model.go‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -491,10 +491,10 @@ func (m *Model) ApplyEngineDefaults() {
491491
if info.DefaultResolution > 0 && m.Resolution <= 0 {
492492
m.Resolution = info.DefaultResolution
493493
}
494-
}
495494

496-
if engine == openai.EngineName && strings.TrimSpace(m.Service.Key) == "" {
497-
m.Service.Key = "${OPENAI_API_KEY}"
495+
if strings.TrimSpace(m.Service.Key) == "" && strings.TrimSpace(info.DefaultKey) != "" {
496+
m.Service.Key = info.DefaultKey
497+
}
498498
}
499499

500500
m.Engine = engine

0 commit comments

Comments
 (0)