Introduce DocumentPostProcessor API for Modular RAG

The DocumentPostProcessor API has been introduced to implement post-retrieval components in a Modular RAG architecture, superseding the DocumentCompressor, DocumentRanker, DocumentSelector APIs that are now deprecated.

Signed-off-by: Thomas Vitale <[email protected]>

Author: ThomasVitale

spring-ai-docs/src/main/antora/modules/ROOT/pages/api/retrieval-augmented-generation.adoc

@@ -361,31 +361,11 @@ List<Document> documents = documentJoiner.join(documentsForQuery);
 
 Post-Retrieval modules are responsible for processing the retrieved documents to achieve the best possible generation results.
 
-==== Document Ranking
+==== Document Post-Processing
 
-A component for ordering and ranking documents based on their relevance to a query to bring the most relevant documents
-to the top of the list, addressing challenges such as _lost-in-the-middle_.
+A component for post-processing retrieved documents based on a query, addressing challenges such as _lost-in-the-middle_, context length restrictions from the model, and the need to reduce noise and redundancy in the retrieved information.
 
-Unlike `DocumentSelector`, this component does not remove entire documents from the list, but rather changes
-the order/score of the documents in the list. Unlike `DocumentCompressor`, this component does not alter the content
-of the documents.
-
-==== Document Selection
-
-A component for removing irrelevant or redundant documents from a list of retrieved documents, addressing challenges
-such as _lost-in-the-middle_ and context length restrictions from the model.
-
-Unlike `DocumentRanker`, this component does not change the order/score of the documents in the list, but rather
-removes irrelevant or redundant documents. Unlike `DocumentCompressor`, this component does not alter the content
-of the documents, but rather removes entire documents.
-
-==== Document Compression
-
-A component for compressing the content of each document to reduce noise and redundancy in the retrieved information,
-addressing challenges such as _lost-in-the-middle_ and context length restrictions from the model.
-
-Unlike `DocumentSelector`, this component does not remove entire documents from the list, but rather alters the content
-of the documents. Unlike `DocumentRanker`, this component does not change the order/score of the documents in the list.
+For example, it could rank documents based on their relevance to the query, remove irrelevant or redundant documents, or compress the content of each document to reduce noise and redundancy.
 
 === Generation
 

spring-ai-docs/src/main/antora/modules/ROOT/pages/upgrade-notes.adoc

@@ -59,6 +59,10 @@ This approach can save time and reduce the chance of errors when upgrading multi
 
 * The `PromptTemplate` API has been redesigned to support a more flexible and extensible way of templating prompts, relying on a new `TemplateRenderer` API. As part of this change, the `getInputVariables()` and `validate()` methods have been deprecated and will throw an `UnsupportedOperationException` if called. Any logic specific to a template engine should be available through the `TemplateRenderer` API.
 
+=== Retrieval Augmented Generation
+
+* The `DocumentPostProcessor` API has been introduced to implement post-retrieval components in a Modular RAG architecture, superseding the `DocumentCompressor`, `DocumentRanker`, `DocumentSelector` APIs that are now deprecated.
+
 [[upgrading-to-1-0-0-m7]]
 == Upgrading to 1.0.0-M7
 

spring-ai-rag/src/main/java/org/springframework/ai/rag/postretrieval/compression/DocumentCompressor.java

@@ -21,6 +21,7 @@
 
 import org.springframework.ai.document.Document;
 import org.springframework.ai.rag.Query;
+import org.springframework.ai.rag.postretrieval.document.DocumentPostProcessor;
 import org.springframework.ai.rag.postretrieval.ranking.DocumentRanker;
 import org.springframework.ai.rag.postretrieval.selection.DocumentSelector;
 
@@ -33,7 +34,9 @@
  * the list, but rather alters the content of the documents. Unlike
  * {@link DocumentRanker}, this component does not change the order/score of the documents
  * in the list.
+ * @deprecated in favour of {@link DocumentPostProcessor}.
  */
+@Deprecated
 public interface DocumentCompressor extends BiFunction<Query, List<Document>, List<Document>> {
 
 	/**

spring-ai-rag/src/main/java/org/springframework/ai/rag/postretrieval/document/DocumentPostProcessor.java

@@ -0,0 +1,45 @@
+/*
+ * Copyright 2023-2025 the original author or authors.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *      https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.springframework.ai.rag.postretrieval.document;
+
+import org.springframework.ai.document.Document;
+import org.springframework.ai.rag.Query;
+
+import java.util.List;
+import java.util.function.BiFunction;
+
+/**
+ * A component for post-processing retrieved documents based on a query, addressing
+ * challenges such as "lost-in-the-middle", context length restrictions from the model,
+ * and the need to reduce noise and redundancy in the retrieved information.
+ * <p>
+ * For example, it could rank documents based on their relevance to the query, remove irrelevant or
+ * redundant documents, or compress the content of each document to reduce noise and
+ * redundancy.
+ *
+ * @author Thomas Vitale
+ * @since 1.0.0
+ */
+public interface DocumentPostProcessor extends BiFunction<Query, List<Document>, List<Document>> {
+
+	List<Document> process(Query query, List<Document> documents);
+
+	default List<Document> apply(Query query, List<Document> documents) {
+		return process(query, documents);
+	}
+
+}

spring-ai-rag/src/main/java/org/springframework/ai/rag/postretrieval/document/package-info.java

@@ -0,0 +1,22 @@
+/*
+ * Copyright 2023-2025 the original author or authors.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *      https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+@NonNullApi
+@NonNullFields
+package org.springframework.ai.rag.postretrieval.document;
+
+import org.springframework.lang.NonNullApi;
+import org.springframework.lang.NonNullFields;

spring-ai-rag/src/main/java/org/springframework/ai/rag/postretrieval/ranking/DocumentRanker.java

@@ -22,6 +22,7 @@
 import org.springframework.ai.document.Document;
 import org.springframework.ai.rag.Query;
 import org.springframework.ai.rag.postretrieval.compression.DocumentCompressor;
+import org.springframework.ai.rag.postretrieval.document.DocumentPostProcessor;
 import org.springframework.ai.rag.postretrieval.selection.DocumentSelector;
 
 /**
@@ -32,7 +33,9 @@
  * Unlike {@link DocumentSelector}, this component does not remove entire documents from
  * the list, but rather changes the order/score of the documents in the list. Unlike
  * {@link DocumentCompressor}, this component does not alter the content of the documents.
+ * @deprecated in favour of {@link DocumentPostProcessor}.
  */
+@Deprecated
 public interface DocumentRanker extends BiFunction<Query, List<Document>, List<Document>> {
 
 	/**

spring-ai-rag/src/main/java/org/springframework/ai/rag/postretrieval/selection/DocumentSelector.java

@@ -22,6 +22,7 @@
 import org.springframework.ai.document.Document;
 import org.springframework.ai.rag.Query;
 import org.springframework.ai.rag.postretrieval.compression.DocumentCompressor;
+import org.springframework.ai.rag.postretrieval.document.DocumentPostProcessor;
 import org.springframework.ai.rag.postretrieval.ranking.DocumentRanker;
 
 /**
@@ -33,7 +34,9 @@
  * documents in the list, but rather removes irrelevant or redundant documents. Unlike
  * {@link DocumentCompressor}, this component does not alter the content of the documents,
  * but rather removes entire documents.
+ * @deprecated in favour of {@link DocumentPostProcessor}.
  */
+@Deprecated
 public interface DocumentSelector extends BiFunction<Query, List<Document>, List<Document>> {
 
 	/**