Gradio integration

[tkmamidi]

Thank you so much for the amazing tool!

I've been using callback handler in python with gradio but struggling to get the user feedback recorded with the trace.

Here's the example with langchain agent and gradio - https://www.gradio.app/main/guides/agents-and-tool-usage#a-real-example-using-langchain-agents

Here's the like/dislike event documentation with gradio - https://www.gradio.app/main/guides/chatbot-specific-events#the-like-event

I was able to record the whole conversation as a trace but I'd like to record every call as a trace (as does the callback handler) along with user feedback using gradio.like function.

Any help is appreciated!

Thanks!

[dosubot[bot]]

To record user feedback using the gradio.like function and associate it with specific traces in Langfuse, you can follow these steps:

1. Set Up the Callback Handler: Ensure that your Langfuse callback handler is correctly set up to record traces for each call in your Langchain application. This involves adding the handler as a callback to your Langchain methods, such as invoke, run, or predict.

2. Capture User Feedback: Use Gradio's like event to capture user feedback. You can attach an event handler to the like event that triggers a custom function whenever a user interacts with the like/dislike buttons.

3. Associate Feedback with Traces: In your custom function, use the Langfuse SDK to append the user feedback to the corresponding trace. You can do this by using the score method to add a score or comment to the trace using the trace_id.

Here's a basic example of how you might implement this in Python:

from langfuse import Langfuse
from langfuse.callback import CallbackHandler
import gradio as gr

# Initialize Langfuse and CallbackHandler
langfuse = Langfuse()
langfuse_handler = CallbackHandler(
    secret_key="sk-lf-...",
    public_key="pk-lf-...",
    host="https://cloud.langfuse.com"
)

# Your Langchain code with the callback handler
# chain.invoke({"input": "<user_input>"}, config={"callbacks": [langfuse_handler]})

# Function to handle like event
def handle_like(trace_id, feedback_value):
    langfuse.score(
        trace_id=trace_id,
        name="user-feedback",
        value=feedback_value,
        comment="User feedback recorded"
    )

# Gradio interface setup
def gradio_interface(input_text):
    # Your processing logic
    return "Processed output"

iface = gr.Interface(
    fn=gradio_interface,
    inputs="text",
    outputs="text"
)

# Attach the like event handler
iface.like(fn=lambda trace_id: handle_like(trace_id, 1))

iface.launch()

In this setup, replace "Processed output" and other placeholders with your actual processing logic and trace management. Ensure that the trace_id used in handle_like corresponds to the trace you want to associate the feedback with. This approach allows you to capture and associate user feedback with specific traces in your Langfuse setup ^[1].

To continue talking to Dosu, mention @dosu.

[tkmamidi]

This is the function with langchain code and callback handler:

async def interact_with_langchain_agent(prompt, messages):
    messages.append(ChatMessage(role="user", content=prompt))
    yield messages
    async for chunk in agent_with_chat_history.astream(
        {"input": prompt},
        config={
            "configurable": {"session_id": "<foo>"},
            "callbacks": [langfuse_handler],
        },
    ):
        if "steps" in chunk:
            for step in chunk["steps"]:
                messages.append(
                    ChatMessage(
                        role="assistant",
                        content=step.action.log,
                        metadata={"title": f"🛠️ Used tool {step.action.tool}"},
                    )
                )
                yield messages
        if "output" in chunk:
            messages.append(ChatMessage(role="assistant", content=chunk["output"]))
            yield messages

However, the iface.like probably doesn't know where the trace_id is coming from?

[dosubot[bot]]

To ensure that the iface.like function in Gradio knows the correct trace_id to associate user feedback with specific traces in your Langchain agent code, you can follow these steps:

1. Generate a Predefined run_id: Before invoking the Langchain agent, generate a unique run_id that will serve as the trace_id. This run_id should be passed to the Langchain invocation and stored for later use when handling the like event.

2. Store the run_id: Keep track of the run_id in a way that it can be accessed when the like event is triggered. This might involve storing it in a session or a global variable, depending on your application's architecture.

3. Use the run_id in the like Event Handler: When the like event is triggered, use the stored run_id to associate the feedback with the correct trace.

Here's an example of how you might implement this:

import uuid
from langfuse import Langfuse
from langfuse.callback import CallbackHandler
import gradio as gr

# Initialize Langfuse and CallbackHandler
langfuse = Langfuse()
langfuse_handler = CallbackHandler()

# Store the run_id globally or in a session
current_run_id = None

async def interact_with_langchain_agent(prompt, messages):
    global current_run_id
    current_run_id = str(uuid.uuid4())  # Generate a unique run_id
    messages.append(ChatMessage(role="user", content=prompt))
    yield messages
    async for chunk in agent_with_chat_history.astream(
        {"input": prompt},
        config={
            "run_id": current_run_id,  # Pass the run_id to the config
            "callbacks": [langfuse_handler],
        },
    ):
        if "steps" in chunk:
            for step in chunk["steps"]:
                messages.append(
                    ChatMessage(
                        role="assistant",
                        content=step.action.log,
                        metadata={"title": f"🛠️ Used tool {step.action.tool}"},
                    )
                )
                yield messages
        if "output" in chunk:
            messages.append(ChatMessage(role="assistant", content=chunk["output"]))
            yield messages

# Function to handle like event
def handle_like(feedback_value):
    global current_run_id
    if current_run_id:
        langfuse.score(
            trace_id=current_run_id,
            name="user-feedback",
            value=feedback_value,
            comment="User feedback recorded"
        )

# Gradio interface setup
iface = gr.Interface(
    fn=interact_with_langchain_agent,
    inputs="text",
    outputs="text"
)

# Attach the like event handler
iface.like(fn=lambda: handle_like(1))

iface.launch()

In this setup, the current_run_id is generated and stored when the Langchain agent is invoked. The handle_like function uses this run_id to associate the feedback with the correct trace ^[1].

To continue talking to Dosu, mention @dosu.

[tkmamidi]

Amazing that the bot could answer my specific question!

I love this tool even more!!

[marcklingen]

super happy to see that this worked so well! let us know if you have any other questions, maintainers are monitoring these threads to make sure answers are good

[marcklingen]

New reference notebook for gradio here: https://langfuse.com/docs/integrations/other/gradio

[marcklingen]

thanks @tkmamidi for your contribution