linking prompts with tracing in langchain integration

[SatheeshJM]

Describe the feature or potential improvement

The documentation under "Prompt Management" (https://langfuse.com/docs/prompts/get-started#link-with-langfuse-tracing-optional) says This is currently unavailable when using the LangChain or LlamaIndex integration.

Any plans on supporting this in the near future?

The prompt manager feature is pretty cool, but seems incomplete without a langchain integration

Additional information

No response

[marcklingen]

Currently exploring options how to make this work as prompts are then used deeply nested within a Llamaindex/Langchain trace and are not easy to detect. Do you have any ideas?

Fyi, you can use prompt management without linking traces to prompts even though being able to link is really useful.

[ajram23]

Is this why the generation count is zero when using prompts in Langchain?

[marcklingen]

yes!

[aiakubovich]

I was looking at some similar question about outputting final prompt with Langchain. There is some discussion about it: langchain-ai/langchain#912. Not sure if it helps or not with Langchain integration.

[marcklingen]

thanks for sharing!

[ajram23]

@marcklingen Curious, Will this be prioritized anytime soon?

[marcklingen]

Generally yes, but not timeline as currently a lot of effort goes into v3 infra (scaling), manual annotation workflow features, and improvements to evals. Will keep you in the loop here.

[hassiebp]

Linking Langfuse prompts to Langchain executions has been one of the most requested features recently, and we have just released support for it 🫡 🥳

For more details, please check out:

• Docs on how to integrate, see tabs for Langchain in examples
• Python SDK release
• JS SDK release

[ajram23]

@hassiebp no such luck. See trace ID 093ba2c5-f446-4768-b53e-cb0fe1629aeb

[hassiebp]

🤔 I've created a chain that is similar to the example you have emailed me. When using with_config, one has to be mindful that the Runnable is wrapped in another Runnable, so the langfuse_prompt must be passed under the metadata key as well in the with_config.

Here is a full example where the generation is nested, and here is a link to the resulting trace:

from langfuse import Langfuse
from langchain_openai import ChatOpenAI
from langchain_core.pydantic_v1 import BaseModel, Field
from langchain_core.prompts import PromptTemplate

class Joke(BaseModel):
    setup: str = Field(description="The setup of the joke")
    punchline: str = Field(description="The punchline to the joke")

langfuse = Langfuse()
trace_name = "test_complex_chain"

# Create prompts
joke_prompt_name = "joke_prompt_" + create_uuid()[:8]
joke_prompt_string = "Tell me a joke involving the animal {{animal}}"

langfuse.create_prompt(
    name=joke_prompt_name,
    prompt=joke_prompt_string,
    labels=["production"],
)

# Get prompts
langfuse_joke_prompt = langfuse.get_prompt(joke_prompt_name)
langchain_joke_prompt = PromptTemplate.from_template(
    langfuse_joke_prompt.get_langchain_prompt(),
).with_config(
    {
        "run_name": "joke_prompt",
        "metadata": {"langfuse_prompt": langfuse_joke_prompt},
    }
)

# Create chain
llm = (
    ChatOpenAI(model="gpt-3.5-turbo-0125", temperature=0)
    .with_structured_output(Joke)
    .with_config({"run_name": "llm"})
)

chain = langchain_joke_prompt | llm

# Run chain
langfuse_handler = CallbackHandler(debug=True)
chain.invoke(
    {"animal": "dog"},
    config={
        "callbacks": [langfuse_handler],
        "run_name": trace_name,
    },
)

langfuse_handler.flush()

[ajram23]

@hassiebp We might be going around in circles here (and mistake could well be mine). I suggest jumping on a zoom call if needed. Here are something that don't add up from end. 1. If I am sending you data in the trace, shouldn't you be able to look in your servers to see if you are getting the meta data or not? 2. Even in your example, I don't see a link to a prompt. May be I am mistaken?
. In my case when it works I see this

[hassiebp]

Yes, let's hop on a call, will email you an invite 👍🏾

If I am sending you data in the trace, shouldn't you be able to look in your servers to see if you are getting the meta data or not

The issue here is on the SDK that do not append the prompt information to our server calls. So to debug, we need visibility on the specific chain running.

Even in your example, I don't see a link to a prompt. May be I am mistaken?

Apologies, the public view cuts links to other features on the platform. For logged in users, here's the view:

[ajram23]

@hassiebp thanks for a great call! Much appreciated and glad we found a solution! Will wait for the helper for compile/langchain and then ship this! @marcklingen thank you as well!