feat: make rerank timeout configurable via rerankTimeoutMs

[jlin53882]

feat: make rerank timeout configurable via rerankTimeoutMs

Summary

Add a configurable rerankTimeoutMs parameter to RetrievalConfig so that users running local/CPU-based rerank servers can adjust the timeout without patching source code. Closes #346.

Changes (4 lines across 1 file)

1. src/retriever.ts — RetrievalConfig interface

/** Rerank API timeout in milliseconds (default: 5000). Increase for local/CPU-based rerank servers. */
rerankTimeoutMs?: number;

2. src/retriever.ts — DEFAULT_RETRIEVAL_CONFIG

rerankEndpoint: "https://api.jina.ai/v1/rerank",
rerankTimeoutMs: 5000,

3. src/retriever.ts — rerankResults() method

// Before
const timeout = setTimeout(() => controller.abort(), 5000);

// After
const timeout = setTimeout(() => controller.abort(), this.config.rerankTimeoutMs ?? 5000);

4. src/retriever.ts — Warning message

// Before
console.warn("Rerank API timed out (5s), falling back to cosine");

// After
console.warn(`Rerank API timed out (${this.config.rerankTimeoutMs ?? 5000}ms), falling back to cosine`);

Usage Example

{
  "plugins": {
    "entries": {
      "memory-lancedb-pro": {
        "config": {
          "retrieval": {
            "rerankTimeoutMs": 120000
          }
        }
      }
    }
  }
}

Local Model Example (BAAI/bge-reranker-base via reranker_server.py)

{
  "plugins": {
    "entries": {
      "memory-lancedb-pro": {
        "config": {
          "retrieval": {
            "rerank": "cross-encoder",
            "rerankProvider": "siliconflow",
            "rerankEndpoint": "http://127.0.0.1:18799/v1/rerank",
            "rerankTimeoutMs": 120000
          }
        }
      }
    }
  }
}

Backward Compatibility

• Default 5000ms is identical to the current hardcoded behavior
• Users who don't specify rerankTimeoutMs see no change
• GPU users can keep short timeouts; CPU users can increase as needed

Issue

Closes #346

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

[rwmjhb]

1. 公开配置面还没补完整。现在只改了 src/retriever.ts，但 openclaw.plugin.json 里的
   retrieval schema 还没有 rerankTimeoutMs，index.ts 里的 PluginConfig.retrieval 也没声明
   这个字段。这样运行时内部虽然有了配置位，但对用户来说配置入口还是不完整，schema/UI 侧也可能不认
   这个字段。
2. 这里的 timeout 最好放到 finally 里清理。当前是 fetch() 返回后才
   clearTimeout(timeout)，如果请求很快失败直接进 catch，timer 会一直挂到超时触发。以前 5s 影
   响不大，但这个 PR 的目标就是支持更长超时，比如 120000ms，这样失败路径下会把“快速回退”变成“进程
   多挂一段时间”。

[jlin53882]

Hi @rwmjhb, thanks for the detailed review! Both issues have been addressed:

1. Schema completeness ✅

• Added
  erankTimeoutMs to openclaw.plugin.json retrieval schema (with default: 5000, description, type: number)
• Added
  erankTimeoutMs?: number to PluginConfig.retrieval in index.ts

2. Timeout cleanup in finally ✅
\\ ypescript
const timeout = setTimeout(() => controller.abort(), timeoutMs);
let response: Response;
try {
response = await fetch(endpoint, { ... });
} finally {
clearTimeout(timeout); // always cleaned up, even on fast failure
}
\\

Updated push — the branch \eat/rerank-timeout-configurable\ now contains all fixes. Thanks!

[ggzeng]

Hi, I noticed the current default for rerankTimeoutMs is 5s. For self-hosted reranker setups (e.g., self-hosted Jina/Cross-encoder endpoints), latency can be significantly higher than cloud-hosted APIs. Would you consider raising the default to 10s? This would reduce timeout failures for users running rerankers on slower hardware without impacting cloud API users. Happy to discuss further!

[AliceLJY]

Thanks for the rerankTimeoutMs addition — the core feature is clean and correct! However the diff contains much more than the timeout change: large deletions of TraceCollector, RetrievalStatsCollector, retrieveWithTrace(), and several config interface fields (omitDimensions, memoryCompaction, sessionCompression etc.) from index.ts that aren't mentioned in the PR description.

Could you clarify whether these deletions are intentional? If yes, they should be in a dedicated refactor PR so they can be reviewed independently and won't conflict with #365/#367 which appear to have the same deletions. If not, please rebase to remove the unrelated changes.

Happy to approve the timeout feature once the scope is clean!

[jlin53882]

@AliceLJY

Review feedback addressed ✅

All three items from the review have been implemented:

• clearTimeout moved into finally block — timer is always cleared even on fast failure paths
• rerankTimeoutMs added to openclaw.plugin.json retrieval schema + uiHints
• rerankTimeoutMs added to index.ts PluginConfig.retrieval interface
• Warning message now shows actual configured timeout value

The PR now contains two clean commits on top of latest master:

1. feat: make rerank timeout configurable via rerankTimeoutMs (closes #346)
2. fix: address review feedback - schema completeness + timeout cleanup in finally

[AliceLJY]

Nice work! The try/finally cleanup for clearTimeout is the right pattern — especially important now that timeouts can be 120s+, you don't want orphaned timers on fast-failure paths.

Re @ggzeng's suggestion about bumping the default to 10s: keeping 5s is fine for backward compatibility, and anyone who needs more can configure it.

Clean feature, good review response. LGTM!

[ggzeng]

Hi, I noticed this PR uses a 5s default for rerankTimeoutMs. I submitted a similar PR (#371) that used a 10s default, and the maintainer suggested I share this feedback here.

For self-hosted reranker setups where network latency is typically higher, a 10s default would be more practical and avoid unnecessary timeouts. The 5s default might work well for cloud-hosted endpoints but could be tight for on-prem deployments.

Would you consider raising the default to 10s (or making it more easily configurable)?

[jlin53882]

Thanks for the feedback @ggzeng!

The 5s default was chosen to maintain backward compatibility — it's the same as the previous hardcoded value. Cloud API users (Jina, SiliconFlow, etc.) typically respond within 1-3s, so 5s is reasonable for them.

For self-hosted/local rerankers, the PR already makes it configurable via:

{ "retrieval": { "rerankTimeoutMs": 10000 } }

10s is a reasonable default for self-hosted setups. However, changing the default from 5s to 10s would slightly increase wait time for cloud API users who hit the timeout. We could alternatively:

1. Keep 5s default (current) — cloud-friendly, self-hosted users configure manually
2. Change to 10s default — self-hosted-friendly, cloud users with slow connections might wait longer
3. Add a separate config for default-per-environment (more complex)

What's your preference? Happy to adjust the default if there's consensus.

Also, if you're interested in maintaining this feature, feel free to take over the PR since you have a similar PR #371 with the 10s default approach.