Skip to content

Commit 66404e4

Browse files
committed
Restore top_k config option in VLM app
Upstream vLLM bug seems to have been fixed and vLLM no longer returns a HTTP 400 when top_k parameter is included.
1 parent 65bc202 commit 66404e4

File tree

2 files changed

+8
-3
lines changed

2 files changed

+8
-3
lines changed

charts/azimuth-image-analysis/azimuth-ui.schema.yaml

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,5 @@ sortOrder:
2828
- /azimuth-llm/ui/appSettings/llm_params/frequency_penalty
2929
- /azimuth-llm/ui/appSettings/llm_params/presence_penalty
3030
- /azimuth-llm/ui/appSettings/llm_params/top_p
31-
# vLLM responds with HTTP 400 BadRequest when top_k is
32-
# passed to a vision model (but ollama accepts it)
33-
# - /azimuth-llm/ui/appSettings/llm_params/top_k
31+
- /azimuth-llm/ui/appSettings/llm_params/top_k
3432
- /azimuth-llm/api/modelMaxContextLength

charts/azimuth-image-analysis/values.schema.json

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -89,6 +89,13 @@
8989
"exclusiveMinimum": 0,
9090
"maximum": 1
9191
},
92+
"top_k": {
93+
"type": "integer",
94+
"title": "LLM Top K",
95+
"description": "The [top k](https://docs.vllm.ai/en/stable/dev/sampling_params.html) value to use when generating LLM responses (must be an integer).",
96+
"default": -1,
97+
"minimum": -1
98+
},
9299
"presence_penalty": {
93100
"type": "number",
94101
"title": "LLM Presence Penalty",

0 commit comments

Comments
 (0)