| 
 | 1 | +[[Advisors]]  | 
 | 2 | + | 
 | 3 | += Spring AI Advisors API  | 
 | 4 | + | 
 | 5 | +== Overview  | 
 | 6 | + | 
 | 7 | +The Spring AI Advisors API introduces a powerful component called `AroundAdvisor`.   | 
 | 8 | +This component dynamically intercepts chat-completion requests and responses, allowing for transformation or utilization of information within these interactions.  | 
 | 9 | + | 
 | 10 | +The Advisors are important for a number of reasons:  | 
 | 11 | + | 
 | 12 | + - Encapsulation of Recurring Tasks: AroundAdvisors enable the packaging of common GenAI patterns into reusable units.  | 
 | 13 | + - Response Transformation: They facilitate the augmentation of data sent to Language Models (LLMs) and formatting of responses sent back to clients.  | 
 | 14 | + - Portability: As part of the Spring AI core specification, developers can create reusable transformation components that work across various models and use cases.  | 
 | 15 | + | 
 | 16 | +The xref:api/chatclient.adoc#_advisor_configuration_in_chatclient[ChatClient API] provides a simple way to configure and use advisors. For example you can enable the built-in chat-memory and question-answer advisors like this:  | 
 | 17 | + | 
 | 18 | +[source,java]  | 
 | 19 | +----  | 
 | 20 | +ChatClient.builder(chatModel)  | 
 | 21 | +    .build()  | 
 | 22 | +    .prompt()  | 
 | 23 | +    .advisors(  | 
 | 24 | +        new ChatMemoryAdvisor(chatMemory),  | 
 | 25 | +        new QuestionAnswerAdvisor(vectorStore, SearchRequest.defaults())  | 
 | 26 | +    )  | 
 | 27 | +    .user(userText)  | 
 | 28 | +    .call();  | 
 | 29 | +----  | 
 | 30 | + | 
 | 31 | +One cane use some of the Spring AI built-in advisors or implement custom advisors to meet your specific requirements.  | 
 | 32 | + | 
 | 33 | +We'll outline the Advisors API, and describe how to implement customized advisors.  | 
 | 34 | +Spring AI support adviser for streaming and non-streaming workloads.  | 
 | 35 | + | 
 | 36 | +== Core Components  | 
 | 37 | + | 
 | 38 | +The Advisor API is defined in the `org.springframework.ai.chat.client.advisor.api` package by the pair of `CallAroundAdvisor` and `CallAroundAdvisorChain` interfaces and the corresponding `StreamAroundAdvisor` and `StreamAroundAdvisorChain` pair for the streaming responses.  | 
 | 39 | + | 
 | 40 | +image::advisors-api-classes.jpg[Advisors API Classes, width=400, align="center"]  | 
 | 41 | + | 
 | 42 | +=== AdvisedRequest and AdvisedResponse  | 
 | 43 | + | 
 | 44 | +* `AdvisedRequest`: Represents the unsealed Prompt request.  | 
 | 45 | +* `AdvisedResponse`: Represents the Chat Completion response.  | 
 | 46 | +* Both contain an `AdviserContext` object for sharing state across the advisor chain.  | 
 | 47 | + | 
 | 48 | +=== Advisor Interfaces  | 
 | 49 | + | 
 | 50 | +. For non-streaming workloads:  | 
 | 51 | +** `CallAroundAdvisor`  | 
 | 52 | +** `CallAroundAdvisorChain`  | 
 | 53 | + | 
 | 54 | +. For streaming workloads:  | 
 | 55 | +** `StreamAroundAdvisor`  | 
 | 56 | +** `StreamAroundAdvisorChain`  | 
 | 57 | + | 
 | 58 | +==== Key Methods  | 
 | 59 | + | 
 | 60 | +. `nextAroundCall()` / `nextAroundStream()`: The core of the advisor logic, typically performing these actions:  | 
 | 61 | +** Examining the unsealed Prompt data  | 
 | 62 | +** Customizing and augmenting the Prompt data  | 
 | 63 | +** Invoking the next entity in the advisor chain  | 
 | 64 | +** Optionally blocking the request  | 
 | 65 | +** Examining the chat completion response  | 
 | 66 | +** Throwing exceptions to indicate processing errors  | 
 | 67 | + | 
 | 68 | +. `getOrder()`: Determines the advisor's priority in the chain.  | 
 | 69 | +. `getName()`: Provides a unique identifier for the advisor.  | 
 | 70 | + | 
 | 71 | +=== Advisor Chain  | 
 | 72 | + | 
 | 73 | +The advisor chain, provided by the Spring AI client, allows for the sequential invocation of multiple advisors. This chain is represented by `CallAroundAdvisorChain` for non-streaming and `StreamAroundAdvisorChain` for streaming scenarios.  | 
 | 74 | + | 
 | 75 | +=== Usage Flow  | 
 | 76 | + | 
 | 77 | +image::advisors-flow.jpg[Advisors API Flow, width=400, align="left"]  | 
 | 78 | + | 
 | 79 | +. The Spring AI framework creates an `AdvisedRequest` from user's `Prompt` along with an empty `AdvisorContext` object.  | 
 | 80 | +. Each advisor in the chain processes the request, potentially modifying it. Alternatively, it can choose to block the request by not making the call to invoke the next entity. In the latter case, the advisor is responsible for filling out the response.  | 
 | 81 | +. The final advisor, provided by the framework, sends the request to the `Chat Model`.  | 
 | 82 | +. The Chat Model's response is then passed back through the advisor chain and converted into `AdvisedResponse`. Later includes the shared `AdvisorContext` instance.  | 
 | 83 | +. Each advisor can process or modify the response.  | 
 | 84 | +. The final `AdvisedResponse` is returned to the client by extracting the `ChatCompletion`.  | 
 | 85 | + | 
 | 86 | +== Implementing an Advisor  | 
 | 87 | + | 
 | 88 | +To create an advisor, implement either `CallAroundAdvisor` or `StreamAroundAdvisor` (or both). The key method to implement is `nextAroundCall()` for non-streaming or `nextAroundStream()` for streaming advisors.  | 
 | 89 | + | 
 | 90 | +=== Logging Advisor Example  | 
 | 91 | + | 
 | 92 | +[source,java]  | 
 | 93 | +----  | 
 | 94 | +public class SimpleLoggerAdvisor implements CallAroundAdvisor, StreamAroundAdvisor {  | 
 | 95 | +
  | 
 | 96 | +	private static final Logger logger = LoggerFactory.getLogger(SimpleLoggerAdvisor.class);  | 
 | 97 | +
  | 
 | 98 | +	@Override  | 
 | 99 | +	public String getName() {  | 
 | 100 | +		return this.getClass().getSimpleName();  | 
 | 101 | +	}  | 
 | 102 | +
  | 
 | 103 | +	@Override  | 
 | 104 | +	public int getOrder() {  | 
 | 105 | +		return 0;  | 
 | 106 | +	}  | 
 | 107 | +
  | 
 | 108 | +	@Override  | 
 | 109 | +	public AdvisedResponse aroundCall(AdvisedRequest advisedRequest, CallAroundAdvisorChain chain) {  | 
 | 110 | +
  | 
 | 111 | +		logger.debug("BEFORE: {}", advisedRequest);  | 
 | 112 | +
  | 
 | 113 | +		AdvisedResponse advisedResponse = chain.nextAroundCall(advisedRequest);  | 
 | 114 | +
  | 
 | 115 | +		logger.debug("AFTER: {}", advisedResponse);  | 
 | 116 | +
  | 
 | 117 | +		return advisedResponse;  | 
 | 118 | +	}  | 
 | 119 | +
  | 
 | 120 | +	@Override  | 
 | 121 | +	public Flux<AdvisedResponse> aroundStream(AdvisedRequest advisedRequest, StreamAroundAdvisorChain chain) {  | 
 | 122 | +
  | 
 | 123 | +		logger.debug("BEFORE: {}", advisedRequest);  | 
 | 124 | +
  | 
 | 125 | +		Flux<AdvisedResponse> advisedResponses = chain.nextAroundStream(advisedRequest);  | 
 | 126 | +		  | 
 | 127 | +        return new MessageAggregator().aggregateAdvisedResponse(advisedResponses,   | 
 | 128 | +                    advisedResponse -> logger.debug("AFTER: {}", advisedResponse));  | 
 | 129 | +	}  | 
 | 130 | +}  | 
 | 131 | +----  | 
 | 132 | + | 
 | 133 | +== Streaming vs Non-Streaming  | 
 | 134 | + | 
 | 135 | +image::advisors-non-stream-vs-stream.jpg[Advisors Streaming vs Non-Streaming Flow, width=800, align="left"]  | 
 | 136 | + | 
 | 137 | +* Non-streaming advisors work with complete requests and responses.  | 
 | 138 | +* Streaming advisors handle requests and responses as continuous streams, using reactive programming concepts (e.g., Flux for responses).  | 
 | 139 | + | 
 | 140 | +== Best Practices  | 
 | 141 | + | 
 | 142 | +. Keep advisors focused on specific tasks for better modularity.  | 
 | 143 | +. Use the `adviseContext` to share state between advisors when necessary.  | 
 | 144 | +. Implement both streaming and non-streaming versions of your advisor for maximum flexibility.  | 
 | 145 | +. Carefully consider the order of advisors in your chain to ensure proper data flow.  | 
 | 146 | + | 
 | 147 | +== Conclusion  | 
 | 148 | + | 
 | 149 | +The Spring AI Advisors API provides a flexible and powerful way to intercept, modify, and enhance AI-driven interactions in your Spring applications. By leveraging this API, developers can create more sophisticated, reusable, and maintainable AI components.  | 
0 commit comments