Skip to content

Commit c7e5e1d

Browse files
committed
Add Advisors documentation
1 parent 5d9a47f commit c7e5e1d

File tree

5 files changed

+150
-0
lines changed

5 files changed

+150
-0
lines changed
470 KB
Loading
356 KB
Loading
425 KB
Loading

spring-ai-docs/src/main/antora/modules/ROOT/nav.adoc

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -90,6 +90,7 @@
9090
*** xref:api/vectordbs/weaviate.adoc[]
9191
9292
93+
** xref:api/advisors.adoc[Advisors]
9394
** xref:api/functions.adoc[Function Calling]
9495
** xref:api/multimodality.adoc[Multimodality]
9596
** xref:api/prompt.adoc[]
Lines changed: 149 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,149 @@
1+
[[Advisors]]
2+
3+
= Spring AI Advisors API
4+
5+
== Overview
6+
7+
The Spring AI Advisors API introduces a powerful component called `AroundAdvisor`.
8+
This component dynamically intercepts chat-completion requests and responses, allowing for transformation or utilization of information within these interactions.
9+
10+
The Advisors are important for a number of reasons:
11+
12+
- Encapsulation of Recurring Tasks: AroundAdvisors enable the packaging of common GenAI patterns into reusable units.
13+
- Response Transformation: They facilitate the augmentation of data sent to Language Models (LLMs) and formatting of responses sent back to clients.
14+
- Portability: As part of the Spring AI core specification, developers can create reusable transformation components that work across various models and use cases.
15+
16+
The xref:api/chatclient.adoc#_advisor_configuration_in_chatclient[ChatClient API] provides a simple way to configure and use advisors. For example you can enable the built-in chat-memory and question-answer advisors like this:
17+
18+
[source,java]
19+
----
20+
ChatClient.builder(chatModel)
21+
.build()
22+
.prompt()
23+
.advisors(
24+
new ChatMemoryAdvisor(chatMemory),
25+
new QuestionAnswerAdvisor(vectorStore, SearchRequest.defaults())
26+
)
27+
.user(userText)
28+
.call();
29+
----
30+
31+
One cane use some of the Spring AI built-in advisors or implement custom advisors to meet your specific requirements.
32+
33+
We'll outline the Advisors API, and describe how to implement customized advisors.
34+
Spring AI support adviser for streaming and non-streaming workloads.
35+
36+
== Core Components
37+
38+
The Advisor API is defined in the `org.springframework.ai.chat.client.advisor.api` package by the pair of `CallAroundAdvisor` and `CallAroundAdvisorChain` interfaces and the corresponding `StreamAroundAdvisor` and `StreamAroundAdvisorChain` pair for the streaming responses.
39+
40+
image::advisors-api-classes.jpg[Advisors API Classes, width=400, align="center"]
41+
42+
=== AdvisedRequest and AdvisedResponse
43+
44+
* `AdvisedRequest`: Represents the unsealed Prompt request.
45+
* `AdvisedResponse`: Represents the Chat Completion response.
46+
* Both contain an `AdviserContext` object for sharing state across the advisor chain.
47+
48+
=== Advisor Interfaces
49+
50+
. For non-streaming workloads:
51+
** `CallAroundAdvisor`
52+
** `CallAroundAdvisorChain`
53+
54+
. For streaming workloads:
55+
** `StreamAroundAdvisor`
56+
** `StreamAroundAdvisorChain`
57+
58+
==== Key Methods
59+
60+
. `nextAroundCall()` / `nextAroundStream()`: The core of the advisor logic, typically performing these actions:
61+
** Examining the unsealed Prompt data
62+
** Customizing and augmenting the Prompt data
63+
** Invoking the next entity in the advisor chain
64+
** Optionally blocking the request
65+
** Examining the chat completion response
66+
** Throwing exceptions to indicate processing errors
67+
68+
. `getOrder()`: Determines the advisor's priority in the chain.
69+
. `getName()`: Provides a unique identifier for the advisor.
70+
71+
=== Advisor Chain
72+
73+
The advisor chain, provided by the Spring AI client, allows for the sequential invocation of multiple advisors. This chain is represented by `CallAroundAdvisorChain` for non-streaming and `StreamAroundAdvisorChain` for streaming scenarios.
74+
75+
=== Usage Flow
76+
77+
image::advisors-flow.jpg[Advisors API Flow, width=400, align="left"]
78+
79+
. The Spring AI framework creates an `AdvisedRequest` from user's `Prompt` along with an empty `AdvisorContext` object.
80+
. Each advisor in the chain processes the request, potentially modifying it. Alternatively, it can choose to block the request by not making the call to invoke the next entity. In the latter case, the advisor is responsible for filling out the response.
81+
. The final advisor, provided by the framework, sends the request to the `Chat Model`.
82+
. The Chat Model's response is then passed back through the advisor chain and converted into `AdvisedResponse`. Later includes the shared `AdvisorContext` instance.
83+
. Each advisor can process or modify the response.
84+
. The final `AdvisedResponse` is returned to the client by extracting the `ChatCompletion`.
85+
86+
== Implementing an Advisor
87+
88+
To create an advisor, implement either `CallAroundAdvisor` or `StreamAroundAdvisor` (or both). The key method to implement is `nextAroundCall()` for non-streaming or `nextAroundStream()` for streaming advisors.
89+
90+
=== Logging Advisor Example
91+
92+
[source,java]
93+
----
94+
public class SimpleLoggerAdvisor implements CallAroundAdvisor, StreamAroundAdvisor {
95+
96+
private static final Logger logger = LoggerFactory.getLogger(SimpleLoggerAdvisor.class);
97+
98+
@Override
99+
public String getName() {
100+
return this.getClass().getSimpleName();
101+
}
102+
103+
@Override
104+
public int getOrder() {
105+
return 0;
106+
}
107+
108+
@Override
109+
public AdvisedResponse aroundCall(AdvisedRequest advisedRequest, CallAroundAdvisorChain chain) {
110+
111+
logger.debug("BEFORE: {}", advisedRequest);
112+
113+
AdvisedResponse advisedResponse = chain.nextAroundCall(advisedRequest);
114+
115+
logger.debug("AFTER: {}", advisedResponse);
116+
117+
return advisedResponse;
118+
}
119+
120+
@Override
121+
public Flux<AdvisedResponse> aroundStream(AdvisedRequest advisedRequest, StreamAroundAdvisorChain chain) {
122+
123+
logger.debug("BEFORE: {}", advisedRequest);
124+
125+
Flux<AdvisedResponse> advisedResponses = chain.nextAroundStream(advisedRequest);
126+
127+
return new MessageAggregator().aggregateAdvisedResponse(advisedResponses,
128+
advisedResponse -> logger.debug("AFTER: {}", advisedResponse));
129+
}
130+
}
131+
----
132+
133+
== Streaming vs Non-Streaming
134+
135+
image::advisors-non-stream-vs-stream.jpg[Advisors Streaming vs Non-Streaming Flow, width=800, align="left"]
136+
137+
* Non-streaming advisors work with complete requests and responses.
138+
* Streaming advisors handle requests and responses as continuous streams, using reactive programming concepts (e.g., Flux for responses).
139+
140+
== Best Practices
141+
142+
. Keep advisors focused on specific tasks for better modularity.
143+
. Use the `adviseContext` to share state between advisors when necessary.
144+
. Implement both streaming and non-streaming versions of your advisor for maximum flexibility.
145+
. Carefully consider the order of advisors in your chain to ensure proper data flow.
146+
147+
== Conclusion
148+
149+
The Spring AI Advisors API provides a flexible and powerful way to intercept, modify, and enhance AI-driven interactions in your Spring applications. By leveraging this API, developers can create more sophisticated, reusable, and maintainable AI components.

0 commit comments

Comments
 (0)