-
Notifications
You must be signed in to change notification settings - Fork 2.5k
[Feature Request] Introduce ScoringModel abstraction for Document Reranking #5776
Description
Expected Behavior
I would love to see a native ScoringModel abstraction introduced in the spring-ai-model module to officially support Document Reranking. I believe the best approach is to follow the exact same structural paradigm as the existing Model<TReq, TRes> interfaces.
Specifically, I propose introducing the following core components:
ScoringRequest(encapsulating the query and documents, implementingModelRequest<List<Document>>)ScoringResponse(returning the scored results, implementingModelResponse<ScoringResult>)ScoringModel(the core interface implementingModel<ScoringRequest, ScoringResponse>)
On the RAG module side (spring-ai-rag), we can introduce aScoringDocumentPostProcessorthat wraps thisScoringModel. This would allow developers to seamlessly plug the reranker into existing RAG pipelines.
Ideally, using it in the code would look as simple as this:
// 1. Initialize a specific scoring model (e.g., Jina, Cohere)
JinaScoringModel scoringModel = new JinaScoringModel(jinaApi, defaultOptions);
// 2. Use it seamlessly as a DocumentPostProcessor in a RAG pipeline
DocumentPostProcessor reranker = new ScoringDocumentPostProcessor(scoringModel);
// 3. Process the documents retrieved from a VectorStore
List<Document> rescoredAndRankedDocs = reranker.process(query, retrievedDocuments);Current Behavior
Currently, Spring AI lacks an official, provider-agnostic abstraction for external Reranker models. Right now, if we want to do reranking, we have to either rely on heavily-coupled options inside specific Vector Stores (like Bedrock), or manually write hardcoded HTTP client calls inside a custom DocumentPostProcessor. Because there is no standardized ScoringModel contract, we can't easily swap out different reranking providers (like Cohere, BGE, or Jina) in an idiomatic Spring AI way like other frameworks (e.g., LangChain4j) do.
Context
- Pain point: I am trying to build a standard 2-stage retrieval RAG pipeline (lightweight Vector Search followed by a heavy Cross-Encoder Re-ranking). Since the framework doesn't provide an interface for this, I am forced to manually wire up HTTP calls and bypass Spring AI's clean architecture.
- Goal: I want to help mature Spring AI's RAG ecosystem by providing a unified interface for external reranking providers, making it easier for the community to integrate tools like Jina or Cohere rerankers.
- Alternatives: I considered just building some utility classes for my personal project, but I strongly feel this architecture belongs in spring-ai-core so the entire community can benefit from it.
Implementation Details
I have already done a deep dive into the spring-ai-model and spring-ai-rag internals, and I would love to submit the initial Pull Request for this feature. To prove the abstraction works, I plan to include the core interfaces and a spring-ai-jina (Jina AI Reranker) integration module.
Here is a sneak peek of the core code structure I plan to implement:
- ScoringRequest.java
package org.springframework.ai.scoring;
public class ScoringRequest implements ModelRequest<List<Document>> {
private final String query;
private final List<Document> documents;
private final ScoringOptions options;
public ScoringRequest(String query, List<Document> documents, ScoringOptions options) {
this.query = query;
this.documents = documents;
this.options = options;
}
public String getQuery() { return query; }
@Override
public List<Document> getInstructions() { return documents; }
@Override
public ScoringOptions getOptions() { return options; }
}- ScoringResult.java & ScoringResponse.java
package org.springframework.ai.scoring;
public class ScoringResult implements ModelResult<Document> {
private final Document document;
public ScoringResult(Document document) { this.document = document; }
@Override
public Document getOutput() { return document; }
@Override
public ScoringResultMetadata getMetadata() { return new ScoringResultMetadata(); }
}
public class ScoringResponse implements ModelResponse<ScoringResult> {
private final List<ScoringResult> results;
public ScoringResponse(List<ScoringResult> results) { this.results = results; }
@Override
public ScoringResult getResult() { return results.isEmpty() ? null : results.get(0); }
@Override
public List<ScoringResult> getResults() { return results; }
@Override
public ScoringResponseMetadata getMetadata() { return new ScoringResponseMetadata(); }
}- ScoringModel.java (The Core Interface)
package org.springframework.ai.scoring;
public interface ScoringModel extends Model<ScoringRequest, ScoringResponse> {
@Override
ScoringResponse call(ScoringRequest request);
}- ScoringDocumentPostProcessor.java (RAG Integration)
package org.springframework.ai.rag.postretrieval.document;
import java.util.List;
import org.springframework.ai.document.Document;
import org.springframework.ai.rag.Query;
import org.springframework.ai.scoring.ScoringModel;
import org.springframework.ai.scoring.ScoringRequest;
import org.springframework.ai.scoring.ScoringResponse;
import org.springframework.ai.scoring.ScoringResult;
public class ScoringDocumentPostProcessor implements DocumentPostProcessor {
private final ScoringModel scoringModel;
public ScoringDocumentPostProcessor(ScoringModel scoringModel) {
this.scoringModel = scoringModel;
}
@Override
public List<Document> process(Query query, List<Document> documents) {
ScoringRequest request = new ScoringRequest(query.text(), documents, null);
ScoringResponse response = this.scoringModel.call(request);
return response.getResults().stream()
.map(ScoringResult::getOutput)
.toList();
}
}Could the maintainers let me know their thoughts on the naming (e.g., ScoringModel vs RerankingModel) and if a PR following this structure would be welcomed?