Replies: 2 comments
-
|
I get timeouts as well when calling the spring ai applications on api (to ask questions). |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
This "workaround" works but I was hoping for something cleaner (ideally, just properties): @Bean
public JettyServletWebServerFactory jettyServletWebServerFactory() {
JettyServletWebServerFactory factory = new JettyServletWebServerFactory();
factory.addServerCustomizers(server -> {
// Set idle timeout for all connectors
for (Connector connector : server.getConnectors()) {
if (connector instanceof ServerConnector) {
ServerConnector serverConnector = (ServerConnector) connector;
// Set idle timeout to 10 minutes (600000 ms)
serverConnector.setIdleTimeout(600000);
log.info("Set Jetty server connector idle timeout to {} ms", 600000);
}
}
});
return factory;
}
@Bean
@Primary
public JettyClientHttpRequestFactory jettyClientHttpRequestFactory() {
try {
HttpClient httpClient = new HttpClient();
// Configure the underlying HttpClient with longer timeouts
httpClient.setConnectTimeout(Duration.ofMinutes(10).toMillis());
httpClient.setIdleTimeout(Duration.ofMinutes(10).toMillis());
httpClient.start();
JettyClientHttpRequestFactory factory = new JettyClientHttpRequestFactory(httpClient);
factory.setConnectTimeout(Duration.ofMinutes(10));
factory.setReadTimeout(Duration.ofMinutes(10));
log.info("Configured JettyClientHttpRequestFactory with connect timeout: {} ms, read timeout: {} ms",
Duration.ofMinutes(10).toMillis(), Duration.ofMinutes(10).toMillis());
return factory;
}
catch (Exception e) {
log.error("Failed to configure JettyClientHttpRequestFactory", e);
throw new RuntimeException(e);
}
}
// Ensure the custom HTTP request factory is used by all RestClients
@Bean
public RestClientCustomizer restClientCustomizer(JettyClientHttpRequestFactory requestFactory) {
return restClientBuilder -> {
restClientBuilder.requestFactory(requestFactory);
log.info("Applied custom JettyClientHttpRequestFactory to RestClient");
};
}
// Alternative: Create a specific RestClient bean for Spring AI
@Bean("ollamaRestClient")
public RestClient ollamaRestClient(JettyClientHttpRequestFactory requestFactory) {
return RestClient.builder().requestFactory(requestFactory).build();
} |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
This is related to and #512
I've tried so many different ways of setting the timeout in application.yml but I can't seem to get it to work.
Does anyone know what configuration I have to do to increase the timeout?
My current workaround is to chunk the documents and write 50 documents at a time to the vector store.
se.lfv.sysdev.app.lfvbot.backend.ingestion.pipeline.exceptions.DocumentWriteException: Failed to write documents to vector store at se.lfv.sysdev.app.lfvbot.backend.ingestion.pipeline.storage.VectorStoreWriter.write(VectorStoreWriter.java:34) ~[classes/:?] at se.lfv.sysdev.app.lfvbot.backend.ingestion.IngestionOrchestrator.startIngestion(IngestionOrchestrator.java:74) ~[classes/:?] at se.lfv.sysdev.app.lfvbot.backend.ingestion.IngestionConfig.lambda$0(IngestionConfig.java:49) ~[classes/:?] at org.springframework.boot.SpringApplication.lambda$callRunner$4(SpringApplication.java:784) ~[spring-boot-3.5.4.jar:3.5.4] at org.springframework.util.function.ThrowingConsumer$1.acceptWithException(ThrowingConsumer.java:82) ~[spring-core-6.2.9.jar:6.2.9] at org.springframework.util.function.ThrowingConsumer.accept(ThrowingConsumer.java:60) ~[spring-core-6.2.9.jar:6.2.9] at org.springframework.util.function.ThrowingConsumer$1.accept(ThrowingConsumer.java:86) ~[spring-core-6.2.9.jar:6.2.9] at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:796) ~[spring-boot-3.5.4.jar:3.5.4] at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:784) ~[spring-boot-3.5.4.jar:3.5.4] at org.springframework.boot.SpringApplication.lambda$callRunners$3(SpringApplication.java:772) ~[spring-boot-3.5.4.jar:3.5.4] at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184) [?:?] at java.base/java.util.stream.SortedOps$SizedRefSortingSink.end(SortedOps.java:357) [?:?] at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:510) [?:?] at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499) [?:?] at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151) [?:?] at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174) [?:?] at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) [?:?] at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596) [?:?] at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:772) [spring-boot-3.5.4.jar:3.5.4] at org.springframework.boot.SpringApplication.run(SpringApplication.java:325) [spring-boot-3.5.4.jar:3.5.4] at org.springframework.boot.SpringApplication.run(SpringApplication.java:1361) [spring-boot-3.5.4.jar:3.5.4] at org.springframework.boot.SpringApplication.run(SpringApplication.java:1350) [spring-boot-3.5.4.jar:3.5.4] at se.lfv.sysdev.app.lfvbot.SpringBootApp.main(SpringBootApp.java:10) [classes/:?] at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) ~[?:?] at java.base/java.lang.reflect.Method.invoke(Method.java:580) ~[?:?] at org.springframework.boot.devtools.restart.RestartLauncher.run(RestartLauncher.java:50) [spring-boot-devtools-3.5.4.jar:3.5.4] Caused by: org.springframework.web.client.ResourceAccessException: I/O error on POST request for "http://localhost:11434/api/embed": Idle timeout expired: 30000/30000 ms at org.springframework.web.client.DefaultRestClient$DefaultRequestBodyUriSpec.createResourceAccessException(DefaultRestClient.java:704) ~[spring-web-6.2.9.jar:6.2.9] at org.springframework.web.client.DefaultRestClient$DefaultRequestBodyUriSpec.exchangeInternal(DefaultRestClient.java:589) ~[spring-web-6.2.9.jar:6.2.9] at org.springframework.web.client.DefaultRestClient$DefaultRequestBodyUriSpec.exchange(DefaultRestClient.java:540) ~[spring-web-6.2.9.jar:6.2.9] at org.springframework.web.client.RestClient$RequestHeadersSpec.exchange(RestClient.java:680) ~[spring-web-6.2.9.jar:6.2.9] at org.springframework.web.client.DefaultRestClient$DefaultResponseSpec.executeAndExtract(DefaultRestClient.java:821) ~[spring-web-6.2.9.jar:6.2.9] at org.springframework.web.client.DefaultRestClient$DefaultResponseSpec.body(DefaultRestClient.java:757) ~[spring-web-6.2.9.jar:6.2.9] at org.springframework.ai.ollama.api.OllamaApi.embed(OllamaApi.java:188) ~[spring-ai-ollama-1.0.1.jar:1.0.1] at org.springframework.ai.ollama.OllamaEmbeddingModel.lambda$call$2(OllamaEmbeddingModel.java:123) ~[spring-ai-ollama-1.0.1.jar:1.0.1] at io.micrometer.observation.Observation.observe(Observation.java:564) ~[micrometer-observation-1.15.2.jar:1.15.2] at org.springframework.ai.ollama.OllamaEmbeddingModel.call(OllamaEmbeddingModel.java:122) ~[spring-ai-ollama-1.0.1.jar:1.0.1] at org.springframework.ai.embedding.EmbeddingModel.embed(EmbeddingModel.java:91) ~[spring-ai-model-1.0.1.jar:1.0.1] at org.springframework.ai.vectorstore.pgvector.PgVectorStore.doAdd(PgVectorStore.java:259) ~[spring-ai-pgvector-store-1.0.1.jar:1.0.1] at org.springframework.ai.vectorstore.observation.AbstractObservationVectorStore.lambda$add$1(AbstractObservationVectorStore.java:85) ~[spring-ai-vector-store-1.0.1.jar:1.0.1] at io.micrometer.observation.Observation.observe(Observation.java:498) ~[micrometer-observation-1.15.2.jar:1.15.2] at org.springframework.ai.vectorstore.observation.AbstractObservationVectorStore.add(AbstractObservationVectorStore.java:85) ~[spring-ai-vector-store-1.0.1.jar:1.0.1] at org.springframework.ai.vectorstore.VectorStore.accept(VectorStore.java:55) ~[spring-ai-vector-store-1.0.1.jar:1.0.1] at se.lfv.sysdev.app.lfvbot.backend.ingestion.pipeline.storage.VectorStoreWriter.write(VectorStoreWriter.java:29) ~[classes/:?] ... 25 more Caused by: java.io.IOException: Idle timeout expired: 30000/30000 ms at org.springframework.http.client.JettyClientHttpRequest.executeInternal(JettyClientHttpRequest.java:121) ~[spring-web-6.2.9.jar:6.2.9] at org.springframework.http.client.AbstractStreamingClientHttpRequest.executeInternal(AbstractStreamingClientHttpRequest.java:71) ~[spring-web-6.2.9.jar:6.2.9] at org.springframework.http.client.AbstractClientHttpRequest.execute(AbstractClientHttpRequest.java:81) ~[spring-web-6.2.9.jar:6.2.9] at org.springframework.web.client.DefaultRestClient$DefaultRequestBodyUriSpec.exchangeInternal(DefaultRestClient.java:583) ~[spring-web-6.2.9.jar:6.2.9] at org.springframework.web.client.DefaultRestClient$DefaultRequestBodyUriSpec.exchange(DefaultRestClient.java:540) ~[spring-web-6.2.9.jar:6.2.9] at org.springframework.web.client.RestClient$RequestHeadersSpec.exchange(RestClient.java:680) ~[spring-web-6.2.9.jar:6.2.9] at org.springframework.web.client.DefaultRestClient$DefaultResponseSpec.executeAndExtract(DefaultRestClient.java:821) ~[spring-web-6.2.9.jar:6.2.9] at org.springframework.web.client.DefaultRestClient$DefaultResponseSpec.body(DefaultRestClient.java:757) ~[spring-web-6.2.9.jar:6.2.9] at org.springframework.ai.ollama.api.OllamaApi.embed(OllamaApi.java:188) ~[spring-ai-ollama-1.0.1.jar:1.0.1] at org.springframework.ai.ollama.OllamaEmbeddingModel.lambda$call$2(OllamaEmbeddingModel.java:123) ~[spring-ai-ollama-1.0.1.jar:1.0.1] at io.micrometer.observation.Observation.observe(Observation.java:564) ~[micrometer-observation-1.15.2.jar:1.15.2] at org.springframework.ai.ollama.OllamaEmbeddingModel.call(OllamaEmbeddingModel.java:122) ~[spring-ai-ollama-1.0.1.jar:1.0.1] at org.springframework.ai.embedding.EmbeddingModel.embed(EmbeddingModel.java:91) ~[spring-ai-model-1.0.1.jar:1.0.1] at org.springframework.ai.vectorstore.pgvector.PgVectorStore.doAdd(PgVectorStore.java:259) ~[spring-ai-pgvector-store-1.0.1.jar:1.0.1] at org.springframework.ai.vectorstore.observation.AbstractObservationVectorStore.lambda$add$1(AbstractObservationVectorStore.java:85) ~[spring-ai-vector-store-1.0.1.jar:1.0.1] at io.micrometer.observation.Observation.observe(Observation.java:498) ~[micrometer-observation-1.15.2.jar:1.15.2] at org.springframework.ai.vectorstore.observation.AbstractObservationVectorStore.add(AbstractObservationVectorStore.java:85) ~[spring-ai-vector-store-1.0.1.jar:1.0.1] at org.springframework.ai.vectorstore.VectorStore.accept(VectorStore.java:55) ~[spring-ai-vector-store-1.0.1.jar:1.0.1] at se.lfv.sysdev.app.lfvbot.backend.ingestion.pipeline.storage.VectorStoreWriter.write(VectorStoreWriter.java:29) ~[classes/:?] ... 25 more Caused by: java.util.concurrent.TimeoutException: Idle timeout expired: 30000/30000 ms at org.eclipse.jetty.io.IdleTimeout.checkIdleTimeout(IdleTimeout.java:167) ~[jetty-io-12.0.23.jar:12.0.23] at org.eclipse.jetty.io.IdleTimeout.idleCheck(IdleTimeout.java:113) ~[jetty-io-12.0.23.jar:12.0.23] at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572) ~[?:?] at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317) ~[?:?] at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) ~[?:?] at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[?:?] at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?] at java.base/java.lang.Thread.run(Thread.java:1583) ~[?:?]Beta Was this translation helpful? Give feedback.
All reactions