Pass data dynamically to LangServer server file

[DameStoilovski99]

Hello guys, I am trying to pass data dynamically from a function in a .py file to a LangServe server file, does anyone have been through this process and maybe have an idea how it can be done?

Below is my code of the LangServe server where I want to accept the collection_name dynamically, for example, instead of collection_name="openAi" to be collection_name=collection_name

app = FastAPI(
    title="LangChain Server",
    version="1.0",
    description="A simple api server using Langchain's Runnable interfaces",
)


def create_runnable():
    embeddings = OpenAIEmbeddings()

    db = PGVector(
        collection_name="openAi",
        connection_string=f"postgresql+psycopg2://{POSTGRES_USER}:{POSTGRES_PASSWORD}@127.0.0.1:{POSTGRES_PORT}/{POSTGRES_DB}",
        embedding_function=embeddings,
    )

    retriever = db.as_retriever(
        search_type="similarity",
    )

    client = OpenAI(
        api_key=OPENAI_API_KEY
    )

    message_text = [
        (
            "some text"
        ),
    ]
    prompt = ChatPromptTemplate.from_messages(message_text)

    llm = ChatOpenAI(client=client)

    runnable = ({"context": retriever, "question": RunnablePassthrough()} | prompt | llm)

    return runnable


add_routes(
    app,
    create_runnable(),
    path="/openai",
)

if __name__ == "__main__":
    import uvicorn

    uvicorn.run(app, host="localhost", port=8000)

[dosubot[bot]]

🤖

Hello @DameStoilovski99! I'm Dosu, a bot that helps with bug-solving, answering queries, and guiding contributors in the LangChain community. While waiting for a human maintainer, I'm here to assist you. Let's get started on your issue!

To pass data dynamically from a function in a .py file to a LangServe server file, you can modify the create_runnable function to accept the collection_name as a parameter. Here's how you can do it:

def create_runnable(collection_name):
    embeddings = OpenAIEmbeddings()

    db = PGVector(
        collection_name=collection_name,
        connection_string=f"postgresql+psycopg2://{POSTGRES_USER}:{POSTGRES_PASSWORD}@127.0.0.1:{POSTGRES_PORT}/{POSTGRES_DB}",
        embedding_function=embeddings,
    )

    # rest of your code

Now, when you call create_runnable, you can pass the dynamic collection name:

add_routes(
    app,
    create_runnable("my_dynamic_collection_name"),
    path="/openai",
)

In this example, "my_dynamic_collection_name" is the dynamic collection name that you want to use. You can replace it with any string variable that contains the collection name you want to use.

This approach is based on the definition of the PGVector class in the LangChain codebase, which accepts collection_name as a parameter in its constructor. You can find more details about the PGVector class in the LangChain repository.

I hope this helps! If you have any other questions, feel free to ask.

Best regards,
Dosu

Sources

• libs/langchain/langchain/vectorstores/pgvector.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[DameStoilovski99]

This is how I instantiate and call the the .ainvoke method, as shown below, is it possible to pass the collection_name argument while calling the .ainvoke function?

    openai = RemoteRunnable("http://host.docker.internal:8000/openai/")
    return await openai.ainvoke(data.question, collection_name=collection_name)

[rhyswynn]

The automated suggestion does not answer the question, the OP wants to make properties of the route dynamic, instead of static when instantiated. The alternative is to define additional routes for every option which becomes a problem at any kind of scale.

[rhyswynn]

I have the same question

[TheExecutivesNetwork]

Nope, you can't create a route dynamically based on the API call parameters. Routes and endpoints have to be created when the server starts or reloads, so your create_runnable() call must create the runnable first, then add it as a route and endpoint to the fastapi server. Then you can call it.

I had the same issue, had to create multiple routes. I suggest you get a list of all your collections from the db, then in a for loop, add a bunch of routes dynamically, named differently, one for each collection.

Hope this helps.

[rhyswynn]

Looks like a configurable chain is the way to go.

https://github.com/langchain-ai/langserve/blob/main/examples/configurable_chain/client.ipynb