Langchain implementation v2

[haneric00]

Description

OpenTelemetry has published an initial verson AI Agent semantic conventions that cover the fundamental attributes for AI Agent spans. However, existing third-party instrumentation solutions (OpenInference, OpenLLMetry, OpenLIT) have significant gaps with official OpenTelemetry GenAI semantic conventions:

1. Non-standard Attribute Naming
   1. Current: llm.vendor, llm.usage.prompt_tokens, custom prefixes
   2. Official: gen_ai.system, gen_ai.usage.input_tokens, standardized gen_ai.* prefix
2. Missing Agent-Level Semantics
   1. Current: No standardized agent operations, conversation tracking, or agent metadata
   2. Official: create_agent, invoke_agent, gen_ai.agent.*, gen_ai.conversation.id
3. Inconsistent Span Naming
   1. Vendor-specific patterns, no clear hierarchy
   2. Official: Standardized format like invoke_agent {agent_name}, execute_tool {tool_name}
4. Limited Tool Execution Tracking
   1. Current: Custom or missing tool instrumentation
   2. Offical: Standardized execute_tool operation with proper context

Proposed Solution

Build an OpenTelemetry instrumentation library for LangChain that:

• Automatically instruments LangChain operations without code changes
• Follows OpenTelemetry best practices and GenAI semantic conventions
• Provides end-to-end tracing of agent workflows, LLM calls, and tool executions
• Integrates seamlessly with existing observability infrastructure

This PR encompasses a langchain instrumentation solution that is currently tailored to work with AWS products like Cloudwatch and Bedrock. I want to get this solution onto upstream first and continuously update it to work universally overtime.

Type of change

• New feature (non-breaking change which adds functionality)
• This change requires a documentation update

How Has This Been Tested?

• Manual testing of the core functionality across different scenarios
• Two new test files that cover the main functionality paths:
  1. Test file 1: test instrumentation works on an agentic application of langchain
  2. Test file 2: test instrumentation works on normal chain usage of langchain

Does This PR Require a Core Repo Change?

• Yes. - Link to PR:
• No.

Checklist:

See contributing.md for styleguide, changelog guidelines, and more.

• Followed the style guidelines of this project
• Changelogs have been updated
• Unit tests have been added
• Documentation has been updated

[linux-foundation-easycla]

• ❌ The email address for the commit (7876019, 031dbd0, 20eb5b6, 9ba62bc, 264ea9d, 513553f, ca20501, 02cdce4) is not linked to the GitHub account, preventing the EasyCLA check. Consult this Help Article and GitHub Help to resolve. (To view the commit's email address, add .patch at the end of this PR page's URL.) For further assistance with EasyCLA, please submit a support request ticket.