LlamaIndex traces grouping

[ivanviragine]

Hi there!

When using the llamaindex integration as in the docs (langfuse_callback_handler / CallbackManager), I get the spans and generations correctly logged in LangFuse, but each step (ReAct agent) is being logged as a trace.
That's not ideal, because in the end, the end-user has no idea of the intermediate steps, so he/she will send feedbacks for the answer only, not the steps in between, so I guess it would be better to have all the agents' steps as one big trace with a lot of spans and generations.
I don't know if this is happening only because I'm using the step-wise execution (https://docs.llamaindex.ai/en/stable/examples/agent/agent_runner/agent_runner/) or not.

[dosubot[bot]]

Hello @ivanviragine! I'm here to help you with any bugs, questions, or contributions you have.

I found a similar discussion regarding customizing trace grouping in Langfuse. You can configure the LlamaIndex integration to log all ReAct agent steps as a single trace by setting a root trace for your LlamaIndexCallbackHandler. This will nest all subsequent traces under the root trace, effectively grouping them together.

Here's how you can do it:

from llama_index.core import Settings
from llama_index.core.callbacks import CallbackManager
from langfuse import Langfuse
from langfuse.llama_index import LlamaIndexCallbackHandler

# Instantiate a Langfuse Client
langfuse = Langfuse()

# Instantiate a new LlamaIndexCallbackHandler and register it in the LlamaIndex Settings
langfuse_callback_handler = LlamaIndexCallbackHandler(
    public_key="pk-lf-...",
    secret_key="sk-lf-...",
    host="https://cloud.langfuse.com"
)
Settings.callback_manager = CallbackManager([langfuse_callback_handler])

def my_func():
  # Create a new trace on your main execution path
  root_trace = langfuse.trace(name="trace-name")

  # Set the root trace, subsequent LlamaIndex observations will be nested under root
  langfuse_callback_handler.set_root(root_trace)

  # Your LlamaIndex code here

  # Reset root, subsequent LlamaIndex observations will now use the default grouping and trace creation
  langfuse_callback_handler.set_root(None)

This setup ensures that all steps are logged under a single trace. Make sure to configure the LlamaIndexCallbackHandler with your Langfuse API keys and host URL ^[1][2].

To continue talking to Dosu, mention @dosu.

[ivanviragine]

That actually works!

[marcklingen]

Can you try the (beta) integration via the instrumentation module?
https://langfuse.com/docs/integrations/llama-index/get-started#beta-observability-based-on-llamaindex-instrumentation-module

@hassiebp is currently rewriting the integration as llamaindex will eventually deprecate the callback based integration.

[ivanviragine]

Thank you @marcklingen ! I wasn't using the instrumentation module because I'm production already, but I'll try it when possible.
Anyway, the dosu's answer was correct.

[marcklingen]

Ok perfect, let us know in case you run into any issues