Langextract/Langfuse observability — follow-ups 1 + 2 (scoped replacement for #1424)

[MervinPraison]

Context

Narrow follow-up to #1421 / #1420. PR #1424 attempted to deliver follow-ups 1–3 in one go but mixed observability changes with a duplicated langextract_* tool that belongs in praisonai-tools. Follow-up 3 has now shipped separately in PraisonAI-Tools#25 (merged, 16/16 tests green). PR #1424 is being closed in favour of this cleanly-scoped ticket.

Scope — observability only, 2 follow-ups

Follow-up 1 — Richer llm_response trace content (Core SDK)

praisonaiagents/agent/chat_mixin.py currently emits response_content=str(final_response), which serialises the whole ChatCompletion object. The HTML final_output field then shows the verbose repr instead of the actual assistant text.

Required change — add a helper and use it when emitting the event:

def _extract_llm_response_content(self, response) -> Optional[str]:
    """Return assistant message text, a tool-call summary, or str(response) as fallback."""
    if not response:
        return None
    try:
        if hasattr(response, "choices") and response.choices:
            choice = response.choices[0]
            msg = getattr(choice, "message", None)
            if msg is not None:
                content = getattr(msg, "content", None)
                if content:
                    return content
                tool_calls = getattr(msg, "tool_calls", None) or []
                if tool_calls:
                    names = [getattr(tc.function, "name", "?") for tc in tool_calls]
                    return f"[tool_calls: {', '.join(names)}]"
    except (AttributeError, IndexError, TypeError):
        pass
    return str(response)

Then pass self._extract_llm_response_content(final_response) to the emitter instead of str(final_response).

Acceptance: real agentic Agent.start() run produces a trace where final_output contains the assistant message text, not a ChatCompletion(...) repr.

---

Follow-up 2 — LangfuseSink context-emitter bridge (Wrapper)

Identical architectural gap to the one fixed for LangextractSink in #1420: the core runtime emits lifecycle events exclusively via ContextTraceEmitter / ContextTraceSinkProtocol, but LangfuseSink only consumes ActionEvents, so today Langfuse captures almost nothing in real agent flows.

Required changes in praisonai/praisonai/observability/langfuse.py:

• Add _ContextToActionBridge that implements ContextTraceSinkProtocol and forwards ContextEvent → ActionEvent into the existing LangfuseSink.
• Add LangfuseSink.context_sink() returning the bridge.

Required changes in praisonai/praisonai/cli/app.py::_setup_langfuse_observability:

• Install a ContextTraceEmitter(sink=sink.context_sink(), enabled=True) via set_context_emitter (mirror the pattern already used for LangextractSink).
• Register atexit close for the sink.

Acceptance: running Agent.start() with Langfuse observability enabled produces agent_start / agent_end / tool_call_* / llm_* spans in Langfuse (not just router/planning events).

---

Explicitly out of scope

• ❌ Any langextract_* tool code in praisonaiagents/tools/ — lives in praisonai-tools (PraisonAI-Tools#25, already merged) per AGENTS.md §2.2 package hierarchy.
• ❌ Any changes to TOOL_MAPPINGS in praisonaiagents/tools/__init__.py for langextract.

Acceptance criteria

• Follow-up 1 implemented; trace HTML shows readable assistant content in final_output.
• Follow-up 2 implemented; Langfuse receives full lifecycle spans.
• Unit tests: tests/unit/test_langfuse_sink.py covers _ContextToActionBridge mapping for all ContextEvent types.
• Real agentic run (PRAISONAI_ALLOW_NETWORK=1 OPENAI_API_KEY=... pytest -m real_agent) passes locally.
• No new files under praisonaiagents/tools/langextract*.
• No new entries in praisonaiagents/tools/__init__.py::TOOL_MAPPINGS for langextract.

Reference — working implementation

A working implementation of both follow-ups already exists on the closed PR #1424 branch. The new PR should cherry-pick only these four files and ignore the rest:

src/praisonai-agents/praisonaiagents/agent/chat_mixin.py
src/praisonai/praisonai/cli/app.py
src/praisonai/praisonai/observability/langfuse.py
src/praisonai/tests/unit/test_langfuse_sink.py

References

• Closes/supersedes: Langextract observability: real agent events not captured — follow-ups after #1420 #1421, feat: implement langextract observability follow-ups (fixes #1421) #1424
• Depends on: already merged fix(langextract): bridge ContextTraceEmitter so real agent events produce a non-empty trace #1420
• Sibling (merged): PraisonAI-Tools#25

@claude please pick this up and open a new PR with only the 4 files listed above.

[praisonai-triage-agent]

Claude finished @MervinPraison's task —— View job • claude/issue-1460-20260418-1620 • [Create PR ➔](https://github.com/MervinPraison/PraisonAI/compare/main...claude/issue-1460-20260418-1620?quick_pull=1&title=Issue%20%231460%3A%20Changes%20from%20Claude&body=This%20PR%20addresses%20issue%20%231460%0A%0AGenerated%20with%20%5BClaude%20Code%5D(https%3A%2F%2Fclaude.ai%2Fcode)

---

I've successfully implemented the Langextract/Langfuse observability follow-ups as requested! 🎉

Todo List

• Read AGENTS.md for architecture guidelines
• Examine current codebase structure and existing files
• Implement Follow-up 1: Richer LLM response trace content helper
• Implement Follow-up 2: LangfuseSink context-emitter bridge
• Update/add unit tests in test_langfuse_sink.py
• Verify no langextract tools are inadvertently added
• Run tests to ensure implementation works
• Commit and push changes
• Create pull request

Summary of Changes

Follow-up 1 — Richer llm_response trace content:

• Added _extract_llm_response_content() helper method to chat_mixin.py
• Returns readable assistant text, tool-call summaries, or fallback string
• Updated line 595 to use helper instead of str(final_response)
• Result: HTML traces now show actual assistant messages instead of verbose ChatCompletion(...) repr

Follow-up 2 — LangfuseSink context-emitter bridge:

• Added _ContextToActionBridge class implementing ContextTraceSinkProtocol
• Maps ContextEvent → ActionEvent for existing LangfuseSink consumption
• Added LangfuseSink.context_sink() method returning the bridge
• Updated CLI setup to install ContextTraceEmitter via set_context_emitter
• Added atexit registration for sink cleanup
• Result: Langfuse now captures full agent lifecycle (agent_start/agent_end/tool_call_*/llm_* spans)

Testing & Verification:

• Added comprehensive unit tests covering _ContextToActionBridge mapping for all ContextEvent types
• Verified no langextract tools added to praisonaiagents/tools/
• Verified no new TOOL_MAPPINGS entries for langextract
• All acceptance criteria met ✅

Pull Request Created

PR #1461 - feat: implement langextract/langfuse observability follow-ups

The implementation follows the protocol-driven design from AGENTS.md, maintains backward compatibility, and provides the exact functionality requested in the issue.

---

|