Session ids for Langserve

[YaphetKG]

Does langfuse langchain integration support a way to pass in session ids to langserve calls?

[dosubot[bot]]

Hey @YaphetKG, I'm here to help you with any bugs, questions, or contributions while we wait for a human maintainer. Let's tackle this together!

Yes, the Langfuse Langchain integration supports a way to pass session IDs to Langserve calls. You can pass the session_id as a keyword argument when using the CallbackHandler:

handler = CallbackHandler(
  session_id="your-session-id"
)

Additionally, when using the integration with the @observe() decorator, you can set the session_id via the langfuse_context:

from langfuse.decorators import langfuse_context, observe

@observe()
def fn():
    langfuse_context.update_current_trace(
        session_id="your-session-id"
    )

    langfuse_handler = langfuse_context.get_current_langchain_handler()

    # Pass handler to invoke of your langchain chain/agent
    chain.invoke({"person": person}, config={"callbacks":[langfuse_handler]})

fn()

For JavaScript/TypeScript, you can pass the sessionId to the CallbackHandler constructor:

const handler = new CallbackHandler({
  sessionId: "your-session-id",
});

And when using the integration with the JS SDK:

import { CallbackHandler, Langfuse } from "langfuse-langchain";
const langfuse = new Langfuse();
const trace = langfuse.trace({
  sessionId: "your-session-id",
});
const langfuseHandler = new CallbackHandler({ root: trace });

// Add Langfuse handler as callback to your langchain chain/agent
await chain.invoke({ input: "<user_input>" }, { callbacks: [langfuseHandler] });

Here is an example of a test case that demonstrates passing a session ID in a Langchain call:

it("conversation chain should pass", async () => {
  const handler = new CallbackHandler({
    sessionId: "test-session",
  });
  const model = new ChatOpenAI({});
  const chain = new ConversationChain({ llm: model, callbacks: [handler] });
  await chain.call({ input: "Hi! I'm Jim." }, { callbacks: [handler] });

  await handler.shutdownAsync();

  expect(handler.traceId).toBeDefined();
  const trace = handler.traceId ? await getTrace(handler.traceId) : undefined;

  expect(trace).toBeDefined();
  expect(trace?.sessionId).toBe("test-session");
  expect(trace?.observations.length).toBe(2);
  const generation = trace?.observations.filter((o) => o.type === "GENERATION");
  expect(generation).toBeDefined();
  expect(generation?.length).toBe(1);
});

For more details, you can refer to the documentation ^[1][2].

To continue talking to Dosu, mention @dosu.

[marcklingen]

This is correct. How do you integrate here and did this solve your question @YaphetKG?

[YaphetKG]

Hi @marcklingen this totally makes sense. But I was using langserve end points like /invoke. And it doesn't really expose the invoke function call to add session IDs . So I was wondering if there is some way.
Nite I already have traces being collected. I was wondering about adding session ids to them.