Hybrid retrieval in knowledge bases with model-based reranking. Is it possible for the number of documents fed into the reranking model to be larger than the top-k value?General Discussion

[xiaopangalf]

Self Checks

• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:)
• Please do not modify this template :) and fill in all the required fields.

Content

If the number of documents for model reranking is less than or equal to top-k, and the model used for generation can consume up to top-k documents without exceeding the context window limit, does model reranking become meaningless in this scenario?

[xiaopangalf]

@dosu

[dosubot[bot]]

Yes, in Dify, the reranking model can be applied to more documents than the final top-k value—reranking is performed on all retrieved documents, and then only the top-k are selected for output. If the number of documents retrieved is less than or equal to top-k and your generation model can process all of them within the context window, reranking is generally unnecessary since all documents can be directly used for generation without further filtering or reordering.

Keep in mind that while the reranking logic itself doesn't limit the number of documents, practical constraints like model timeout settings (especially with integrations like GPUstack) can affect how high you can set top-k in real deployments. For most use cases, reranking is most valuable when you retrieve more documents than you can pass to the generation model, so you need to select the most relevant subset before generation.
Source 1 | Source 2

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}

   

[xiaopangalf]

你提到的代码中，模型重排时似乎是一个top_n 的参数，但是我们在前台全局只有一个top k的配置。假设我配置的top k是10，且向量库中可以检索到大量的数据，那进入模型重排时，处理的文档数量是多少？@dosu

[xiaopangalf]

@dosu

[dosubot[bot]]

在 Dify 里，前台/全局的 top-k 配置（比如 10）会限制最终返回给生成模型的文档数量，但进入 rerank（重排）模型的文档数量可以远大于 top-k，尤其在混合检索（hybrid retrieval）或外部知识库场景下。实际流程是：先从向量库/全文库检索出一批候选文档（数量可远超 top-k），然后全部送入 rerank 模型，rerank 按相关性排序后只保留 top-k 返回给下游生成模型。也就是说，重排模型处理的文档数量取决于检索阶段返回的候选集大小，而不是 top-k 本身，top-k 只决定最终输出数量。你可以通过调整知识库设置里的 top-k 或检索参数来控制候选集规模，但如果检索返回很多文档，rerank 依然会全量处理再截断 top-k 输出。

代码和官方讨论都明确了这一点，详见：rerank runner 处理逻辑、官方讨论1、官方讨论2。如果你配置 top-k=10，但检索阶段返回 100 条，rerank 会处理这 100 条，最后只输出 10 条。

注意：大批量候选文档会带来性能和超时风险，尤其在混合检索场景，建议根据实际硬件和业务需求合理设置 top-k 和检索参数，避免系统过载或响应超时。

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}