Skip to content

Commit 4fc2556

Browse files
Update JinaAI rerank model token limit in rerankerWindowSize method
1 parent ffd491d commit 4fc2556

File tree

1 file changed

+1
-1
lines changed
  • x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/services/openshiftai

1 file changed

+1
-1
lines changed

x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/services/openshiftai/OpenShiftAiService.java

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -342,7 +342,7 @@ private OpenShiftAiModel createModelFromPersistent(
342342
@Override
343343
public int rerankerWindowSize(String modelId) {
344344
// OpenShift AI uses Cohere and JinaAI rerank protocols for reranking
345-
// JinaAI rerank model has 8000 tokens limit length https://jina.ai/models/jina-reranker-v2-base-multilingual
345+
// JinaAI rerank model has 131K tokens limit https://jina.ai/models/jina-reranker-v3/
346346
// Cohere rerank model truncates at 4096 tokens https://docs.cohere.com/reference/rerank
347347
// We choose a conservative limit based on these two models
348348
// Using 1 token = 0.75 words as a rough estimate, we get 3072 words allowing for some headroom, we set the window size below 3072

0 commit comments

Comments
 (0)