Skip to content

Sync call to getEmbeddings via Vector store Document Writer in Reactive workflow #5137

@KaustubhKhati

Description

@KaustubhKhati

Bug description

When using spring webflux it is expected all the downstream calls are async. When using Azure Open AI Embeddings and Vector store to write documents, the default implementation uses getEmbeddingsWithResponse which uses getEmbeddingsSync. This leads java.lang.IllegalStateException: block()/blockFirst()/blockLast() are blocking, which is not supported in thread parallel-7

Stack trace

com.azure.ai.openai.implementation.OpenAIClientImpl.getEmbeddingsWithResponse(OpenAIClientImpl.java:2361)
\tat com.azure.ai.openai.OpenAIClient.getEmbeddingsWithResponse(OpenAIClient.java:168)
\tat com.azure.ai.openai.OpenAIClient.getEmbeddings(OpenAIClient.java:620)
\tat org.springframework.ai.azure.openai.AzureOpenAiEmbeddingModel.lambda$call$1(AzureOpenAiEmbeddingModel.java:143)
\tat io.micrometer.observation.Observation.observe(Observation.java:564)
\tat org.springframework.ai.azure.openai.AzureOpenAiEmbeddingModel.call(AzureOpenAiEmbeddingModel.java:142)
\tat org.springframework.ai.embedding.EmbeddingModel.embed(EmbeddingModel.java:91)
\tat org.springframework.ai.vectorstore.qdrant.QdrantVectorStore.doAdd(QdrantVectorStore.java:179)
\tat org.springframework.ai.vectorstore.observation.AbstractObservationVectorStore.lambda$add$1(AbstractObservationVectorStore.java:85)
\tat io.micrometer.observation.Observation.observe(Observation.java:498)
\tat org.springframework.ai.vectorstore.observation.AbstractObservationVectorStore.add(AbstractObservationVectorStore.java:85)
\tat org.springframework.ai.vectorstore.VectorStore.accept(VectorStore.java:55)
\tat org.springframework.ai.vectorstore.VectorStore.accept(VectorStore.java:40)
\tat org.springframework.ai.document.DocumentWriter.write(DocumentWriter.java:30)

Environment
Spring AI version: 1.1.2,
Java version - 17

Steps to reproduce
Use spring webflux, Azure Open AI Embedding, Qdrant Vector Store.
Write any document using vectorStore.write() method.

Expected behavior
Embedding is fetched using the provided OpenAIAsyncClient rather than OpenAIClient.

Minimal Complete Reproducible example

spring:
  main:
    web-application-type: reactive
  ai:
    vectorstore:
      qdrant:
        host: localhost
        port: 6334
        collection-name: "collection"
        use-tls: false
        initialize-schema: true
    azure:
      openai:
        endpoint: ${azure-openai-endpoint}
        api-key: ${azure-openai-api-key}
        embeddings:
          options:
            deployment-name: "text-embedding-ada-002"
            api-version: "2023-05-15"
vectorStore.write(Document.builder()
                    .text(userMessage.getText())
                    .metadata(metadata)
                    .build();) 

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions