-
Notifications
You must be signed in to change notification settings - Fork 2k
Description
Bug description
The nova models (tested with amazon.nova-lite-v1:0
) prefix the response with <thinking>...</thinking>
. This output is NOT ignored in BeanOutputConverter
.
Environment
Spring AI Version: 1.0.3
Model: amazon.nova-lite-v1:0
JVM: zulu-25
Steps to reproduce
Configure structured output for example:
responseSpec.entity(ActorsFilms.class);
Note
I haven't seen this problem using the ActorsFilms
example, but it happens ~75% of the time when I'm also using a tool.
Expected behavior
Entity is parsed/returned with no additional configuration needed.
Partial Solution
Sometimes the response text ONLY contains JSON, most of the time I've seen the <thinking>
prefix.
My workaround was to strip the thinking block:
private static class NovaBeanOutputConverter<T> extends BeanOutputConverter<T> {
public NovaBeanOutputConverter(Class<T> clazz) {
super(clazz);
}
@Override
public T convert(@NonNull String text) {
text = text.trim().replaceFirst("<thinking>.*</thinking>\\s", "");
return super.convert(text);
}
}
I'm just hacking my way through this, so if there is a better way to strip the <thinking>
prefix, let me know; otherwise, I can try to create a standalone example if needed.