diff --git a/components-mdx/integration-learn-more-js.mdx b/components-mdx/integration-learn-more-js.mdx index 80cdb0469..e18adb789 100644 --- a/components-mdx/integration-learn-more-js.mdx +++ b/components-mdx/integration-learn-more-js.mdx @@ -10,12 +10,12 @@ import { ## Interoperability with the JS/TS SDK -You can use this integration together with the Langfuse [JS/TS SDK](/docs/observability/sdk/typescript/overview) to add additional attributes or group observations into a single trace. +You can use this integration together with the Langfuse [SDKs](/docs/observability/sdk/overview) to add additional attributes or group observations into a single trace. -The [Context Manager](/docs/observability/sdk/typescript/instrumentation#context-management-with-callbacks) allows you to wrap your instrumented code using context managers (with `with` statements), which allows you to add additional attributes to the trace. Any observation created inside the callback will automatically be nested under the active observation, and the observation will be ended when the callback finishes. +The [Context Manager](/docs/observability/sdk/instrumentation#context-management-with-callbacks) allows you to wrap your instrumented code using context managers (with `with` statements), which allows you to add additional attributes to the trace. Any observation created inside the callback will automatically be nested under the active observation, and the observation will be ended when the callback finishes. ```typescript import { startActiveObservation, propagateAttributes } from "npm:@langfuse/tracing"; @@ -51,12 +51,12 @@ await startActiveObservation("context-manager", async (span) => { }); ``` -Learn more about using the Context Manager in the [JS/TS SDK](https://langfuse.com/docs/observability/sdk/typescript/instrumentation#context-management-with-callbacks) docs. +Learn more about using the Context Manager in the [Langfuse SDK instrumentation docs](https://langfuse.com/docs/observability/sdk/instrumentation#context-management-with-callbacks). -The [`observe` wrapper](/docs/observability/sdk/typescript/instrumentation#observe-wrapper) is a powerful tool for tracing existing functions without modifying their internal logic. It acts as a decorator that automatically creates a span or generation around the function call. You can use the `propagateAttributes` function to add attributes to the observation from within the wrapped function. +The [`observe` wrapper](/docs/observability/sdk/instrumentation#observe-wrapper) is a powerful tool for tracing existing functions without modifying their internal logic. It acts as a decorator that automatically creates a span or generation around the function call. You can use the `propagateAttributes` function to add attributes to the observation from within the wrapped function. ```typescript import { observe, propagateAttributes } from "@langfuse/tracing"; @@ -98,7 +98,7 @@ const processUserRequest = observe( const result = await processUserRequest("some query"); ``` -Learn more about using the Decorator in the [JS/TS SDK docs](/docs/observability/sdk/typescript/instrumentation#observe-wrapper) docs. +Learn more about using the Decorator in the [Langfuse SDK instrumentation docs](/docs/observability/sdk/instrumentation#observe-wrapper). diff --git a/components-mdx/integration-learn-more.mdx b/components-mdx/integration-learn-more.mdx index 4b00145f7..14276b115 100644 --- a/components-mdx/integration-learn-more.mdx +++ b/components-mdx/integration-learn-more.mdx @@ -10,12 +10,12 @@ import { ## Interoperability with the Python SDK -You can use this integration together with the Langfuse [Python SDK](/docs/observability/sdk/python/overview) to add additional attributes to the trace. +You can use this integration together with the Langfuse [SDKs](/docs/observability/sdk/overview) to add additional attributes to the trace. -The [`@observe()` decorator](/docs/observability/sdk/python/instrumentation#custom-instrumentation) provides a convenient way to automatically wrap your instrumented code and add additional attributes to the trace. +The [`@observe()` decorator](/docs/observability/sdk/instrumentation#custom-instrumentation) provides a convenient way to automatically wrap your instrumented code and add additional attributes to the trace. ```python from langfuse import observe, propagate_attributes, get_client @@ -45,12 +45,12 @@ def my_llm_pipeline(input): return result ``` -Learn more about using the Decorator in the [Python SDK](/docs/observability/sdk/python/instrumentation#custom-instrumentation) docs. +Learn more about using the Decorator in the [Langfuse SDK instrumentation docs](/docs/observability/sdk/instrumentation#custom-instrumentation). -The [Context Manager](/docs/observability/sdk/python/instrumentation#custom-instrumentation) allows you to wrap your instrumented code using context managers (with `with` statements), which allows you to add additional attributes to the trace. +The [Context Manager](/docs/observability/sdk/instrumentation#custom-instrumentation) allows you to wrap your instrumented code using context managers (with `with` statements), which allows you to add additional attributes to the trace. ```python from langfuse import get_client, propagate_attributes @@ -79,7 +79,7 @@ with langfuse.start_as_current_observation(as_type="span", name="my-trace") as s langfuse.flush() ``` -Learn more about using the Context Manager in the [Python SDK](/docs/observability/sdk/python/instrumentation#custom-instrumentation) docs. +Learn more about using the Context Manager in the [Langfuse SDK instrumentation docs](/docs/observability/sdk/instrumentation#custom-instrumentation). diff --git a/components-mdx/prompt-linking.mdx b/components-mdx/prompt-linking.mdx index 9e6bda5e1..27e0b065a 100644 --- a/components-mdx/prompt-linking.mdx +++ b/components-mdx/prompt-linking.mdx @@ -124,7 +124,7 @@ openai.chat.completions.create( -Please make sure you have [OpenTelemetry already set up](/docs/observability/sdk/typescript/setup) for tracing. +Please make sure you have [OpenTelemetry already set up](/docs/observability/sdk/overview#initialize-tracing) for tracing. ```ts /langfusePrompt,/ import { observeOpenAI } from "@langfuse/openai"; @@ -217,7 +217,7 @@ chat_chain.invoke({"movie": "Dune 2", "criticlevel": "expert"}, config={"callbac -Please make sure you have [OpenTelemetry already set up](/docs/observability/sdk/typescript/setup) for tracing. +Please make sure you have [OpenTelemetry already set up](/docs/observability/sdk/overview#initialize-tracing) for tracing. ```ts import { LangfuseClient } from "@langfuse/client"; diff --git a/components/PropagationRestrictionsCallout.tsx b/components/PropagationRestrictionsCallout.tsx index 22a9728f7..e9d54deeb 100644 --- a/components/PropagationRestrictionsCallout.tsx +++ b/components/PropagationRestrictionsCallout.tsx @@ -8,9 +8,9 @@ export function PropagationRestrictionsCallout({ attributes = ["userId", "sessionId", "metadata", "version", "tags"], }: PropagationRestrictionsCalloutProps) { const pythonLink = - "/docs/observability/sdk/python/instrumentation#propagate-attributes"; + "/docs/observability/sdk/instrumentation#propagate-attributes"; const tsLink = - "/docs/observability/sdk/typescript/instrumentation#propagate-attributes"; + "/docs/observability/sdk/instrumentation#propagate-attributes"; const formatAttributes = (attrs: string[]) => { if (attrs.length === 1) return `\`${attrs[0]}\``; diff --git a/lib/redirects.js b/lib/redirects.js index 1f6542caa..b03486d4e 100644 --- a/lib/redirects.js +++ b/lib/redirects.js @@ -72,7 +72,7 @@ const nonPermanentRedirects = [ ["/docs/reference", "https://api.reference.langfuse.com/"], ["/docs/integrations/api", "/docs/api"], ["/docs/integrations/sdk/typescript", "/docs/sdk/typescript"], - ["/docs/integrations/sdk/python", "/docs/observability/sdk/python/overview"], + ["/docs/integrations/sdk/python", "/docs/observability/sdk/overview"], ["/docs/langchain", "/docs/integrations/langchain/tracing"], ["/docs/langchain/python", "/docs/integrations/langchain/tracing"], ["/docs/langchain/typescript", "/docs/integrations/langchain/tracing"], @@ -143,7 +143,7 @@ const nonPermanentRedirects = [ ["/docs/cloud", "/docs/deployment/cloud"], ["/docs/guides/sdk-integration", "/docs/sdk/overview"], ["/docs/sdk", "/docs/sdk/overview"], - ["/docs/sdk/python", "/docs/observability/sdk/python/overview"], + ["/docs/sdk/python", "/docs/observability/sdk/overview"], ["/cookbook", "/guides"], ["/cookbook/:path*", "/guides/cookbook/:path*"], ["/docs/sdk/typescript", "/docs/sdk/typescript/guide"], @@ -208,7 +208,7 @@ const nonPermanentRedirects = [ ], [ "/docs/sdk/python/decorators", - "/docs/observability/sdk/python/instrumentation#custom-instrumentation", + "/docs/observability/sdk/instrumentation#custom-instrumentation", ], ]; @@ -717,17 +717,17 @@ const mainDocsReorder202507 = [ "/docs/evaluation/dataset-runs/datasets", ], ["/docs/sdk/overview", "/docs/observability/sdk/overview"], - ["/docs/sdk/python/example", "/docs/observability/sdk/python/overview"], - ["/docs/sdk/python/low-level-sdk", "/docs/observability/sdk/python/overview"], - ["/docs/sdk/python/sdk-v3", "/docs/observability/sdk/python/overview"], + ["/docs/sdk/python/example", "/docs/observability/sdk/overview"], + ["/docs/sdk/python/low-level-sdk", "/docs/observability/sdk/overview"], + ["/docs/sdk/python/sdk-v3", "/docs/observability/sdk/overview"], [ "/docs/sdk/typescript/example-notebook", - "/docs/observability/sdk/typescript/example-notebook", + "/guides/cookbook/js_langfuse_sdk", ], - ["/docs/sdk/typescript/guide", "/docs/observability/sdk/typescript/guide"], + ["/docs/sdk/typescript/guide", "/docs/observability/sdk/overview"], [ "/docs/sdk/typescript/guide-web", - "/docs/observability/sdk/typescript/guide-web", + "/docs/observability/sdk/advanced-features#custom-scores-from-browser", ], [ "/docs/security/example-python", @@ -860,27 +860,27 @@ const oldDatasetRunSection202508 = [ const pythonv3sdkSection202508 = [ [ "/docs/observability/sdk/python/sdk-v3", - "/docs/observability/sdk/python/overview", + "/docs/observability/sdk/overview", ], [ "/docs/observability/sdk/python/decorators", - "/docs/observability/sdk/python/overview", + "/docs/observability/sdk/instrumentation#custom-instrumentation", ], [ "/docs/observability/sdk/python/example", - "/docs/observability/sdk/python/overview", + "/docs/observability/sdk/overview", ], [ "/docs/observability/sdk/python/low-level-sdk", - "/docs/observability/sdk/python/overview", + "/docs/observability/sdk/overview", ], [ "/guides/cookbook/python_decorators#interoperability-with-other-integrations", - "/docs/observability/sdk/python/instrumentation#native-instrumentations", + "/docs/observability/sdk/instrumentation#native-instrumentation", ], [ "/guides/cookbook/python_decorators#customize-inputoutput", - "/docs/observability/sdk/python/instrumentation#trace-inputoutput-behavior", + "/docs/observability/sdk/instrumentation#trace-inputoutput-behavior", ], ]; @@ -888,11 +888,11 @@ const pythonv3sdkSection202508 = [ const typescriptsdkSection202508 = [ [ "/docs/observability/sdk/typescript/guide", - "/docs/observability/sdk/typescript/overview", + "/docs/observability/sdk/overview", ], [ "/docs/observability/sdk/typescript/guide-web", - "/docs/observability/sdk/typescript/advanced-usage#custom-scores-from-browser", + "/docs/observability/sdk/advanced-features#custom-scores-from-browser", ], [ "/docs/observability/sdk/typescript/example-notebook", @@ -904,16 +904,16 @@ const typescriptsdkSection202508 = [ ], [ "/docs/observability/sdk/typescript/evaluation", - "/docs/evaluation/evaluation-methods/custom-scores", + "/docs/observability/sdk/advanced-features#create-scores", ], [ "/guides/cookbook/integration_haystack", "/integrations/frameworks/haystack", ], - ["/sdk-v3", "/docs/observability/sdk/python/overview"], + ["/sdk-v3", "/docs/observability/sdk/overview"], [ "/docs/observability/sdk/typescript/sdk-v4", - "/docs/observability/sdk/typescript/overview", + "/docs/observability/sdk/overview", ], ]; @@ -976,7 +976,7 @@ const handbookRestructure202510 = [ ["/handbook/working-at-langfuse/principles", "/handbook/how-we-work/principles"], ["/handbook/working-at-langfuse/spending-money", "/handbook/tools-and-processes/spending-money"], ["/handbook/working-at-langfuse/time-off", "/handbook/tools-and-processes/time-off"], - ["/docs/sdk/python/sdk-v3#multi-project-setup-experimental", "/docs/observability/sdk/python/advanced-usage#multi-project-setup-experimental"], + ["/docs/sdk/python/sdk-v3#multi-project-setup-experimental", "/docs/observability/sdk/advanced-features#multi-project-setup-experimental"], ]; // Launch Week 4 - Nov 2025 diff --git a/pages/blog/2025-09-30-langfuse-september-update.mdx b/pages/blog/2025-09-30-langfuse-september-update.mdx index a7972d3f3..a7c2e0803 100644 --- a/pages/blog/2025-09-30-langfuse-september-update.mdx +++ b/pages/blog/2025-09-30-langfuse-september-update.mdx @@ -57,9 +57,9 @@ Ensure your LLM responses conform to a specific JSON schema in your Prompt Exper We’ve taken JS/TypeScript tracing up a notch: The Langfuse JS/TypeScript SDK v4 is out of beta as of September 2025! We’ve rebuilt this SDK on top of OpenTelemetry to improve the developer experience, make context management more robust, and allow for easy integrations with the JS/TS ecosystem (Provider SDKs, Vercel AI SDK, LangChain JS, Mastra, ...). -→ **[See the documentation](/docs/observability/sdk/typescript/overview)** +→ **[See the documentation](/docs/observability/sdk/overview)** -→ **[Upgrade now from v3 to v4](/docs/observability/sdk/typescript/upgrade-path)** +→ **[Upgrade now from v3 to v4](/docs/observability/sdk/upgrade-path#typescript-sdk---upgrade-path-v3-to-v4)** If you are on Python, all of this is already available in the current version of the Python SDK. diff --git a/pages/blog/2025-11-25-vibe-coding-custom-annotation-ui.mdx b/pages/blog/2025-11-25-vibe-coding-custom-annotation-ui.mdx index 9a575cd79..f99522506 100644 --- a/pages/blog/2025-11-25-vibe-coding-custom-annotation-ui.mdx +++ b/pages/blog/2025-11-25-vibe-coding-custom-annotation-ui.mdx @@ -43,7 +43,7 @@ Conversely, if you're the only reviewer or your traces render fine in a generic ## The Setup That Makes This Possible -This project took an afternoon because the foundation already existed. Langfuse exposes [annotation queues](/docs/evaluation/evaluation-methods/annotation), [score configs](/faq/all/manage-score-configs#create-a-score-config), and [trace data](/docs/observability/overview) through a [clean API](/docs/api-and-data-platform/features/public-api). The [TypeScript SDK](/docs/observability/sdk/typescript/overview) provides type safety. All the infrastructure for managing human evaluation is already running. +This project took an afternoon because the foundation already existed. Langfuse exposes [annotation queues](/docs/evaluation/evaluation-methods/annotation), [score configs](/faq/all/manage-score-configs#create-a-score-config), and [trace data](/docs/observability/overview) through a [clean API](/docs/api-and-data-platform/features/public-api). The [Langfuse SDKs](/docs/observability/sdk/overview) provide type safety. All the infrastructure for managing human evaluation is already running. What remained was straightforward: build a UI that fetches queue items, renders them in a domain-specific way, and submits scores back. No auth system to build. No database schema to design. No score aggregation logic to write. diff --git a/pages/changelog/2023-10-25-support-pydantic-v1-and-v2.mdx b/pages/changelog/2023-10-25-support-pydantic-v1-and-v2.mdx index 36a95a4b0..61d52646b 100644 --- a/pages/changelog/2023-10-25-support-pydantic-v1-and-v2.mdx +++ b/pages/changelog/2023-10-25-support-pydantic-v1-and-v2.mdx @@ -16,5 +16,5 @@ From `1.1.3` onwards, the Python SDK supports both Pydantic v1 and v2. import { Code } from "lucide-react"; - } /> + } /> diff --git a/pages/changelog/2025-01-30-sampling-js-ts-sdk.mdx b/pages/changelog/2025-01-30-sampling-js-ts-sdk.mdx index 48da8a1e5..54ea81da6 100644 --- a/pages/changelog/2025-01-30-sampling-js-ts-sdk.mdx +++ b/pages/changelog/2025-01-30-sampling-js-ts-sdk.mdx @@ -51,7 +51,7 @@ import { Shuffle } from "lucide-react"; /> } /> diff --git a/pages/changelog/2025-08-28-typescript-sdk-v4-ga.mdx b/pages/changelog/2025-08-28-typescript-sdk-v4-ga.mdx index b07a00398..8d0ed171f 100644 --- a/pages/changelog/2025-08-28-typescript-sdk-v4-ga.mdx +++ b/pages/changelog/2025-08-28-typescript-sdk-v4-ga.mdx @@ -47,7 +47,7 @@ await startActiveObservation("user-request", async (span) => { }); ``` -[Learn more →](/docs/observability/sdk/typescript/instrumentation#custom-instrumentation) +[Learn more →](/docs/observability/sdk/instrumentation#custom-instrumentation) @@ -66,7 +66,7 @@ const tracedFunction = observe(async (source: string) => { const result = await tracedFunction("API"); ``` -[Learn more →](/docs/observability/sdk/typescript/instrumentation#observe-wrapper) +[Learn more →](/docs/observability/sdk/instrumentation#observe-wrapper) @@ -86,7 +86,7 @@ span.update({ output: "Successfully answered." }); span.end(); // Required: manually end the observation ``` -[Learn more →](/docs/observability/sdk/typescript/instrumentation#manual-observations) +[Learn more →](/docs/observability/sdk/instrumentation#manual-observations) @@ -98,17 +98,17 @@ span.end(); // Required: manually end the observation } title="Overview" - href="/docs/observability/sdk/typescript/overview" + href="/docs/observability/sdk/overview" arrow /> } title="Instrumentation" - href="/docs/observability/sdk/typescript/instrumentation" + href="/docs/observability/sdk/instrumentation" arrow /> -**Upgrading from v3?** Follow the [upgrade guide](/docs/observability/sdk/typescript/upgrade-path) for step-by-step migration instructions. +**Upgrading from v3?** Follow the [upgrade guide](/docs/observability/sdk/upgrade-path#typescript-sdk---upgrade-path-v3-to-v4) for step-by-step migration instructions. Have thoughts or issues? **Share feedback in the discussion:** [GitHub Discussions #8403](https://github.com/orgs/langfuse/discussions/8403). diff --git a/pages/docs/evaluation/evaluation-methods/custom-scores.mdx b/pages/docs/evaluation/evaluation-methods/custom-scores.mdx index 1fe125b58..175b16bd5 100644 --- a/pages/docs/evaluation/evaluation-methods/custom-scores.mdx +++ b/pages/docs/evaluation/evaluation-methods/custom-scores.mdx @@ -275,7 +275,7 @@ await langfuse.flush(); -→ More details in [Python SDK docs](/docs/observability/sdk/python/evaluation#create-scores) and [JS/TS SDK docs](/docs/sdk/typescript/guide#score). See [API reference](/docs/api) for more details on POST/GET score configs endpoints. +→ More details in the [Langfuse SDK advanced features guide](/docs/observability/sdk/advanced-features#create-scores). See [API reference](/docs/api) for more details on POST/GET score configs endpoints. ### Preventing Duplicate Scores @@ -478,7 +478,7 @@ await langfuse.flush(); -→ More details in [Python SDK docs](/docs/observability/sdk/python/evaluation#create-scores) and [JS/TS SDK docs](/docs/sdk/typescript/guide#score). See [API reference](/docs/api) for more details on POST/GET score configs endpoints. +→ More details in the [Langfuse SDK advanced features guide](/docs/observability/sdk/advanced-features#create-scores). See [API reference](/docs/api) for more details on POST/GET score configs endpoints. diff --git a/pages/docs/evaluation/experiments/experiments-via-sdk.mdx b/pages/docs/evaluation/experiments/experiments-via-sdk.mdx index b718aece6..a58ee76c6 100644 --- a/pages/docs/evaluation/experiments/experiments-via-sdk.mdx +++ b/pages/docs/evaluation/experiments/experiments-via-sdk.mdx @@ -75,7 +75,7 @@ print(result.format()) {/* JS/TS SDK */} -Make sure that OpenTelemetry is properly set up for traces to be delivered to Langfuse. See the [tracing setup documentation](/docs/observability/sdk/typescript/setup#tracing-setup) for configuration details. Always flush the span processor at the end of execution to ensure all traces are sent. +Make sure that OpenTelemetry is properly set up for traces to be delivered to Langfuse. See the [tracing setup documentation](/docs/observability/sdk/overview#initialize-tracing) for configuration details. Always flush the span processor at the end of execution to ensure all traces are sent. ```typescript import { OpenAI } from "openai"; @@ -134,7 +134,7 @@ console.log(await result.format()); await otelSdk.shutdown(); ``` -**Note for JS/TS SDK**: OpenTelemetry must be properly set up for traces to be delivered to Langfuse. See the [tracing setup documentation](/docs/observability/sdk/typescript/setup#tracing-setup) for configuration details. Always flush the span processor at the end of execution to ensure all traces are sent. +**Note for JS/TS SDK**: OpenTelemetry must be properly set up for traces to be delivered to Langfuse. See the [tracing setup documentation](/docs/observability/sdk/overview#initialize-tracing) for configuration details. Always flush the span processor at the end of execution to ensure all traces are sent. @@ -853,7 +853,7 @@ _See [Python SDK](/docs/sdk/python/sdk-v3) docs for more details._ {/* JS/TS SDK */} -Please make sure you have [the JS/TS SDK](/docs/observability/sdk/typescript/setup) set up for tracing of your application. If you use Langfuse for [observability](/docs/observability/overview), this is the same setup. +Please make sure you have [the Langfuse SDK](/docs/observability/sdk/overview#initialize-tracing) set up for tracing of your application. If you use Langfuse for [observability](/docs/observability/overview), this is the same setup. Example: diff --git a/pages/docs/observability/features/trace-ids-and-distributed-tracing.mdx b/pages/docs/observability/features/trace-ids-and-distributed-tracing.mdx index 6392c5c91..18843595f 100644 --- a/pages/docs/observability/features/trace-ids-and-distributed-tracing.mdx +++ b/pages/docs/observability/features/trace-ids-and-distributed-tracing.mdx @@ -185,7 +185,7 @@ const scoringTraceId = await createTraceId(externalId); Setting a parentSpanContext will detach the created span from the active span context as it no longer inherits from the current active span in the context. -Learn more in the [JS/TS SDK](/docs/observability/sdk/typescript/advanced-usage#managing-trace-and-observation-ids) docs. +Learn more in the [Langfuse SDK instrumentation docs](/docs/observability/sdk/instrumentation#managing-trace-and-observation-ids). diff --git a/pages/docs/observability/sdk/_meta.tsx b/pages/docs/observability/sdk/_meta.tsx index b31463039..4608539b4 100644 --- a/pages/docs/observability/sdk/_meta.tsx +++ b/pages/docs/observability/sdk/_meta.tsx @@ -1,5 +1,7 @@ export default { overview: "Overview", - python: "Python", - typescript: "JS/TS", + instrumentation: "Instrumentation", + "advanced-features": "Advanced Features", + "troubleshooting-and-faq": "Troubleshooting & FAQ", + "upgrade-path": "Upgrade Path", }; diff --git a/pages/docs/observability/sdk/advanced-features.mdx b/pages/docs/observability/sdk/advanced-features.mdx new file mode 100644 index 000000000..d0a372329 --- /dev/null +++ b/pages/docs/observability/sdk/advanced-features.mdx @@ -0,0 +1,535 @@ +--- +title: Advanced features of the Langfuse SDKs +description: Configure masking, logging, sampling, multi-project routing, evaluations, and environment-specific behaviors for Python and JS/TS. +category: SDKs +--- + +# Advanced features + +Use these patterns to harden your Langfuse instrumentation, protect sensitive data, and adapt the SDKs to complex environments. Each section shows the equivalent Python and JS/TS setup side-by-side. + +## Mask sensitive data + +Route spans to different Langfuse projects (multi-tenant, staging vs. prod, etc.) by registering separate exporters. Note the limitations below. + +Both SDKs expose helpers to attach scores to observations or entire traces, plus dataset utilities for repeatable experiments. + + + +Provide a `mask` function when instantiating the client to scrub inputs, outputs, and metadata before they leave your infrastructure. + +```python +from langfuse import Langfuse +import re + +def pii_masker(data: any, **kwargs) -> any: + if isinstance(data, str): + return re.sub(r"[a-zA-Z0-9_.+-]+@[a-zA-Z0-9-]+\.[a-zA-Z0-9-.]+", "[EMAIL_REDACTED]", data) + elif isinstance(data, dict): + return {k: pii_masker(data=v) for k, v in data.items()} + elif isinstance(data, list): + return [pii_masker(data=item) for item in data] + return data + +langfuse = Langfuse(mask=pii_masker) +``` + + +Pass a `mask` callback to the `LangfuseSpanProcessor`. The callback receives stringified JSON of the attribute and should return the masked payload. + +```ts filename="instrumentation.ts" /mask: / +import { NodeSDK } from "@opentelemetry/sdk-node"; +import { LangfuseSpanProcessor } from "@langfuse/otel"; + +const spanProcessor = new LangfuseSpanProcessor({ + mask: ({ data }) => + data.replace(/\b\d{4}[- ]?\d{4}[- ]?\d{4}[- ]?\d{4}\b/g, "***MASKED_CREDIT_CARD***"), +}); + +const sdk = new NodeSDK({ spanProcessors: [spanProcessor] }); + +sdk.start(); +``` + + + +## Logging & debugging + + + +The Langfuse SDK uses Python's standard `logging` module. The main logger is named `"langfuse"`. Enable detailed debug logging by either passing `debug=True` when instantiating `Langfuse`, setting the `LANGFUSE_DEBUG="True"` environment variable, or configuring the logger manually: + +```python +import logging + +langfuse_logger = logging.getLogger("langfuse") +langfuse_logger.setLevel(logging.DEBUG) +``` + +The default log level for `"langfuse"` is `logging.WARNING`. + + +Configure the global SDK logger (or set `LANGFUSE_LOG_LEVEL`) to control verbosity. Available log levels are `DEBUG`, `INFO`, `WARN`, and `ERROR`. + +```ts /configureGlobalLogger/ +import { configureGlobalLogger, LogLevel } from "@langfuse/core"; + +configureGlobalLogger({ level: LogLevel.DEBUG }); +``` + +```bash +export LANGFUSE_LOG_LEVEL="DEBUG" +``` + + + +## Sampling + + + +Sample traces directly on the client via the `sample_rate` constructor argument or the `LANGFUSE_SAMPLE_RATE` environment variable. The value must be between `0.0` (drop everything) and `1.0` (send all traces). If a trace is not sampled, none of its observations or scores are exported. + +```python +from langfuse import Langfuse + +langfuse = Langfuse(sample_rate=0.2) +``` + + +Langfuse respects OpenTelemetry's sampling decisions. Configure a sampler on your OTEL `NodeSDK` to control which traces reach Langfuse and reduce noise/costs in high-volume workloads. + +```ts filename="instrumentation.ts" /TraceIdRatioBasedSampler/ +import { NodeSDK } from "@opentelemetry/sdk-node"; +import { LangfuseSpanProcessor } from "@langfuse/otel"; +import { TraceIdRatioBasedSampler } from "@opentelemetry/sdk-trace-base"; + +const sdk = new NodeSDK({ + sampler: new TraceIdRatioBasedSampler(0.2), + spanProcessors: [new LangfuseSpanProcessor()], +}); + +sdk.start(); +``` + + + +## Filter exported spans [#filtering-spans] + + + +Exclude spans from specific instrumentation scopes. + +```python +from langfuse import Langfuse + +langfuse = Langfuse(blocked_instrumentation_scopes=["sqlalchemy", "psycopg"]) +``` + +**How it works:** every OpenTelemetry span contains an instrumentation scope (visible in Langfuse under `metadata.scope.name`). Langfuse evaluates the scope against `blocked_instrumentation_scopes` when deciding whether to export. Be careful when filtering parents—child spans may become orphaned. + + +Filtering parent spans may result in orphaned children in Langfuse. + + + +Provide a `shouldExportSpan` predicate. + +```ts filename="instrumentation.ts" /shouldExportSpan/ +import { NodeSDK } from "@opentelemetry/sdk-node"; +import { LangfuseSpanProcessor, ShouldExportSpan } from "@langfuse/otel"; + +const shouldExportSpan: ShouldExportSpan = ({ otelSpan }) => + otelSpan.instrumentationScope.name !== "express"; + +const sdk = new NodeSDK({ + spanProcessors: [new LangfuseSpanProcessor({ shouldExportSpan })], +}); + +sdk.start(); +``` + +Use allowlists when you only want to forward specific instrumentation scopes (e.g., Langfuse SDK + AI SDK). + +```ts +import { ShouldExportSpan } from "@langfuse/otel"; + +const shouldExportSpan: ShouldExportSpan = ({ otelSpan }) => + ["langfuse-sdk", "ai"].includes(otelSpan.instrumentationScope.name); +``` + + + +## Tracer provider isolation + + + +Create a dedicated OTEL `TracerProvider` for Langfuse spans when you need isolation from other observability backends (Datadog, Jaeger, Zipkin, etc.). + +**Benefits** + +- Prevent Langfuse spans from reaching other exporters. +- Prevent third-party spans from appearing inside Langfuse. +- Configure sampling/export policies independently. + +```python +from opentelemetry.sdk.trace import TracerProvider +from langfuse import Langfuse + +langfuse_tracer_provider = TracerProvider() +langfuse = Langfuse(tracer_provider=langfuse_tracer_provider) +langfuse.start_span(name="isolated").end() +``` + + +TracerProviders still share the same context, so mixing providers may create orphaned spans. + + + +Isolate Langfuse spans with a custom provider and avoid sending them to other exporters. + +```ts /setLangfuseTracerProvider/ +import { NodeTracerProvider } from "@opentelemetry/sdk-trace-node"; +import { setLangfuseTracerProvider } from "@langfuse/tracing"; +import { LangfuseSpanProcessor } from "@langfuse/otel"; + +const langfuseTracerProvider = new NodeTracerProvider({ + spanProcessors: [new LangfuseSpanProcessor()], +}); + +setLangfuseTracerProvider(langfuseTracerProvider); +``` + + +As with Python, isolating tracer providers can break parent/child relationships when contexts overlap. + + + + +## Multi-project setups [#multi-project-setup-experimental] + + + +Instantiate dedicated clients per project and pass the `langfuse_public_key` context where needed. + +```python +from langfuse import Langfuse, observe + +project_a = Langfuse(public_key="pk-lf-project-a-...", secret_key="sk-lf-project-a-...") +project_b = Langfuse(public_key="pk-lf-project-b-...", secret_key="sk-lf-project-b-...") + +@observe +def process_data_for_project_a(data, langfuse_public_key="pk-lf-project-a-..."): + return {"processed": data} + +@observe +def process_data_for_project_b(data, langfuse_public_key="pk-lf-project-b-..."): + return {"processed": data} +``` + +You can also route OpenAI or LangChain integrations by passing `langfuse_public_key` on each call. + + +Multi-project setups are experimental. Third-party OTEL spans (HTTP clients, databases, etc.) lack the Langfuse public key attribute and will therefore be exported to **all** projects. + + +**How it works** + +1. Langfuse spans carry a public-key attribute. +2. You register one exporter per project. +3. Each exporter filters spans by that attribute before sending them. + +**Important considerations** + +- Always provide the correct `langfuse_public_key` to the top-most observed function or integration call. +- Missing the key routes traces to the default project or drops them. +- Third-party spans without the attribute go to every project. + + +Register multiple span processors—one per Langfuse project—and optionally set filters per processor. + +```ts filename="instrumentation.ts" +import { NodeSDK } from "@opentelemetry/sdk-node"; +import { LangfuseSpanProcessor } from "@langfuse/otel"; + +const sdk = new NodeSDK({ + spanProcessors: [ + new LangfuseSpanProcessor({ + publicKey: "pk-lf-public-key-project-1", + secretKey: "sk-lf-secret-key-project-1", + }), + new LangfuseSpanProcessor({ + publicKey: "pk-lf-public-key-project-2", + secretKey: "sk-lf-secret-key-project-2", + }), + ], +}); + +sdk.start(); +``` + + + +## Environment-specific considerations + + + +### Thread pools and multiprocessing + +Use the OpenTelemetry threading instrumentor so context flows across worker threads. + +```python +from opentelemetry.instrumentation.threading import ThreadingInstrumentor + +ThreadingInstrumentor().instrument() +``` + +For multiprocessing, follow the [OpenTelemetry guidance](https://github.com/open-telemetry/opentelemetry-python/issues/2765#issuecomment-1158402076). If you use Pydantic Logfire, enable `distributed_tracing=True`. + +### Distributed tracing + +Prefer native OTEL propagation when linking services. The `trace_context` argument should be a last resort because it forces root-span semantics server-side. + +### Time to first token (TTFT) + +```python +from langfuse import get_client +import datetime, time + +langfuse = get_client() + +with langfuse.start_as_current_observation(as_type="generation", name="TTFT-Generation") as generation: + time.sleep(3) + generation.update( + completion_start_time=datetime.datetime.now(), + output="some response", + ) + +langfuse.flush() +``` + +### Self-signed TLS certificates + +```bash filename=".env" +OTEL_EXPORTER_OTLP_TRACES_CERTIFICATE="/path/to/my-selfsigned-cert.crt" +``` + +```python +import os, httpx +from langfuse import Langfuse + +httpx_client = httpx.Client(verify=os.environ["OTEL_EXPORTER_OTLP_TRACES_CERTIFICATE"]) +langfuse = Langfuse(httpx_client=httpx_client) +``` + + +Understand the security implications before trusting self-signed certificates. + + + +### Serverless environments + +```ts filename="instrumentation.ts" /langfuseSpanProcessor/ +import { NodeSDK } from "@opentelemetry/sdk-node"; +import { LangfuseSpanProcessor } from "@langfuse/otel"; + +export const langfuseSpanProcessor = new LangfuseSpanProcessor({ + exportMode: "immediate", +}); + +const sdk = new NodeSDK({ spanProcessors: [langfuseSpanProcessor] }); + +sdk.start(); +``` + +```ts filename="handler.ts" /forceFlush/ +import { langfuseSpanProcessor } from "./instrumentation"; + +export async function handler(event, context) { + // ... logic ... + await langfuseSpanProcessor.forceFlush(); +} +``` + +Use `after` in Vercel Cloud Functions to schedule the flush after the response has been sent. + +### Custom scores from the browser [#custom-scores-from-browser] + +```ts +import { LangfuseWeb } from "langfuse"; + +export function UserFeedbackComponent(props: { traceId: string }) { + const langfuseWeb = new LangfuseWeb({ + publicKey: env.NEXT_PUBLIC_LANGFUSE_PUBLIC_KEY, + baseUrl: "https://cloud.langfuse.com", + }); + + const handleUserFeedback = async (value: number) => + await langfuseWeb.score({ traceId: props.traceId, name: "user_feedback", value }); + + return ( +
+ + +
+ ); +} +``` + + +Never expose your Langfuse secret key in browser code. Use only the public key with `LangfuseWeb`. + + + +When sending client-side scores, make sure the frontend has access to the relevant `traceId` (and optional `observationId`). Expose those IDs in your backend response or use deterministic IDs that exist in both environments. + + +Learn more in the [custom scores documentation](/docs/evaluation/evaluation-methods/custom-scores). + +### Langfuse + Sentry + +```ts filename="instrumentation.ts" +import * as Sentry from "@sentry/node"; +import { LangfuseSpanProcessor } from "@langfuse/otel"; +import { NodeTracerProvider } from "@opentelemetry/sdk-trace-node"; +import { + SentryPropagator, + SentrySampler, + SentrySpanProcessor, +} from "@sentry/opentelemetry"; + +const sentryClient = Sentry.init({ + dsn: "", + skipOpenTelemetrySetup: true, + tracesSampleRate: 1, +}); + +const provider = new NodeTracerProvider({ + sampler: sentryClient ? new SentrySampler(sentryClient) : undefined, + spanProcessors: [new LangfuseSpanProcessor(), new SentrySpanProcessor()], +}); + +provider.register({ + propagator: new SentryPropagator(), + contextManager: new Sentry.SentryContextManager(), +}); +``` + + +If you only use Sentry for error monitoring, omit `tracesSampleRate` and the `SentrySpanProcessor` so its sampling rules don't affect Langfuse traces. + +
+
+ +## Evaluation & scoring [#create-scores] + + + +Use observation methods, context-aware helpers, or low-level APIs to submit scores. + +```python +from langfuse import get_client + +langfuse = get_client() + +with langfuse.start_as_current_observation(as_type="generation", name="summary_generation") as gen: + gen.update(output="summary text...") + gen.score(name="conciseness", value=0.8, data_type="NUMERIC") + gen.score_trace(name="user_feedback_rating", value="positive", data_type="CATEGORICAL") + +with langfuse.start_as_current_observation(as_type="span", name="complex_task"): + langfuse.score_current_span(name="task_component_quality", value=True, data_type="BOOLEAN") +``` + +```python +langfuse.create_score( + name="fact_check_accuracy", + value=0.95, + trace_id="abcdef1234567890abcdef1234567890", + observation_id="1234567890abcdef", + data_type="NUMERIC", + comment="Source verified for 95% of claims.", +) +``` + +### Dataset runs + +```python +from langfuse import get_client + +langfuse = get_client() + +dataset = langfuse.get_dataset(name="my-eval-dataset") +for item in dataset.items: + print(item.input, item.expected_output) + +langfuse.create_dataset(name="new-summarization-tasks") +langfuse.create_dataset_item( + dataset_name="new-summarization-tasks", + input={"text": "Long article..."}, + expected_output={"summary": "Short summary."} +) +``` + + +Server-side scoring uses the `LangfuseClient`. + +```ts +import { LangfuseClient } from "@langfuse/client"; + +const langfuse = new LangfuseClient(); + +await langfuse.score.create({ + traceId: "trace_id_here", + name: "accuracy", + value: 0.9, +}); +``` + +Browser feedback can be captured via `LangfuseWeb` (see the environment section above) and attached to traces or observations. + +For datasets, use the `@langfuse/client` helpers described in the [dataset runs guide](/docs/evaluation/dataset-runs/remote-run#setup--run-via-sdk). + + + +## Observation types + + + +Specify observation types via decorators or context managers. + +```python /as_type="tool"/ +from langfuse import observe + +@observe(as_type="tool") +def retrieve_context(query): + return vector_store.get(query) +``` + +```python /as_type="embedding"/ +from langfuse import get_client + +langfuse = get_client() + +with langfuse.start_as_current_observation(as_type="chain", name="retrieval-pipeline") as chain: + with langfuse.start_as_current_observation(as_type="retriever", name="vector-search") as retriever: + retriever.update(output={"results": perform_vector_search("user question")}) +``` + + +The `asType` option is available on `startObservation`, `startActiveObservation`, and the `observe` wrapper. + +```ts /{ asType: "tool" }/ +import { startObservation } from "@langfuse/tracing"; + +const span = startObservation("user-request"); + +const toolCall = span.startObservation( + "fetch-weather", + { input: { city: "Paris" } }, + { asType: "tool" } +); + +toolCall.end(); +``` + + diff --git a/pages/docs/observability/sdk/instrumentation.mdx b/pages/docs/observability/sdk/instrumentation.mdx new file mode 100644 index 000000000..374ffd6c1 --- /dev/null +++ b/pages/docs/observability/sdk/instrumentation.mdx @@ -0,0 +1,790 @@ +--- +title: Instrument your application with the Langfuse SDKs +description: Use native integrations or custom instrumentation patterns in Python and JavaScript/TypeScript to capture rich traces. +category: SDKs +--- + +import GetStartedJsLangchain from "@/components-mdx/get-started/js-langchain.mdx"; +import { PropagationRestrictionsCallout } from "@/components/PropagationRestrictionsCallout"; + +# Instrumentation + +Langfuse SDKs build on OpenTelemetry so you can mix native integrations, wrappers, and fully custom spans. Use the tabs below to see the equivalent Python and JS/TS patterns side-by-side. + +## Native integrations [#native-integrations] + +Langfuse supports native integrations for popular LLM and agent libraries. They capture prompts, responses, usage, and errors automatically while preserving full OpenTelemetry context. Explore the full ecosystem in the [Integrations gallery](/integrations). + +### OpenAI + + + +Langfuse provides a drop-in replacement for the OpenAI Python SDK to automatically trace all API calls. + +```diff +- import openai ++ from langfuse.openai import openai + +# Your existing OpenAI code continues to work as is +# For example: +# client = openai.OpenAI() +# completion = client.chat.completions.create(...) +``` + +**What's captured automatically:** prompts (incl. streaming), responses, timings, errors, token usage, estimated costs, and multimedia payloads. If the OpenAI call runs inside an active Langfuse span it is nested correctly. + +You can also pass Langfuse-specific metadata to enrich the trace: + +```python /metadata={"langfuse_session_id":/ +from langfuse.openai import openai + +client = openai.OpenAI() + +response = client.chat.completions.create( + model="gpt-4o", + messages=[{"role": "user", "content": "What is OpenTelemetry?"}], + metadata={ + "langfuse_session_id": "session_123", + "langfuse_user_id": "user_456", + "langfuse_tags": ["production", "chat-bot"], + "custom_field": "additional metadata", + }, +) +``` + +Integrations are interoperable with manual tracing. Wrap calls with Langfuse spans to propagate tags or session data: + +```python +from langfuse import get_client, propagate_attributes +from langfuse.openai import openai + +langfuse = get_client() +client = openai.OpenAI() + +with langfuse.start_as_current_observation(as_type="span", name="qna-bot-openai"): + with propagate_attributes(tags=["qna-bot-openai"]): + response = client.chat.completions.create( + name="qna-bot-openai", + model="gpt-4o", + messages=[{"role": "user", "content": "What is OpenTelemetry?"}], + ) +``` + + +Wrap any OpenAI client with `observeOpenAI`. All subsequent calls emit generations that inherit the current OpenTelemetry context. + +```ts /observeOpenAI/ +import { OpenAI } from "openai"; +import { observeOpenAI } from "@langfuse/openai"; + +const openai = new OpenAI(); + +const tracedOpenAI = observeOpenAI(openai, { + traceName: "my-openai-trace", + sessionId: "user-session-123", + userId: "user-abc", + tags: ["openai-integration"], +}); + +const completion = await tracedOpenAI.chat.completions.create({ + model: "gpt-4", + messages: [{ role: "user", content: "What is OpenTelemetry?" }], +}); +``` + +Mix native integrations with custom spans (e.g., `startActiveObservation`) whenever you need additional metadata or nesting control. + + + +### LangChain + + + +Use the Langfuse LangChain callback handler to automatically trace chains, tools, retrievers, and LLM calls. + +```python +from langfuse import get_client, propagate_attributes +from langfuse.langchain import CallbackHandler +from langchain_openai import ChatOpenAI +from langchain_core.prompts import ChatPromptTemplate + +langfuse = get_client() +langfuse_handler = CallbackHandler() +llm = ChatOpenAI(model_name="gpt-4o") +prompt = ChatPromptTemplate.from_template("Tell me a joke about {topic}") +chain = prompt | llm + +with langfuse.start_as_current_observation(as_type="span", name="joke-chain"): + with propagate_attributes(tags=["joke-chain"]): + response = chain.invoke({"topic": "cats"}, config={"callbacks": [langfuse_handler]}) +``` + +Trace attributes (`session_id`, `user_id`, `tags`) can be set via `metadata` on the LangChain call so every observation inherits them. + + + + + + +### Framework & third-party telemetry + + + +Any OTEL-instrumented library can forward spans to Langfuse. Example: tracing Anthropic via the community instrumentation package. + +```python +from anthropic import Anthropic +from opentelemetry.instrumentation.anthropic import AnthropicInstrumentor + +from langfuse import get_client + +AnthropicInstrumentor().instrument() + +langfuse = get_client() +client = Anthropic() + +with langfuse.start_as_current_observation(as_type="span", name="myspan"): + message = client.messages.create( + model="claude-3-7-sonnet-20250219", + max_tokens=1024, + messages=[{"role": "user", "content": "Hello, Claude"}], + ) + +langfuse.flush() +``` + +See the [Langfuse + LlamaIndex integration](/integrations/frameworks/llamaindex) for another OTEL-based example. + + +The JS SDK integrates with frameworks like the Vercel AI SDK. Register the `LangfuseSpanProcessor` in your OTEL setup and flush spans after streaming responses. + +```ts filename="instrumentation.ts" +import { LangfuseSpanProcessor, ShouldExportSpan } from "@langfuse/otel"; +import { NodeTracerProvider } from "@opentelemetry/sdk-trace-node"; + +const shouldExportSpan: ShouldExportSpan = (span) => + span.otelSpan.instrumentationScope.name !== "next.js"; + +export const langfuseSpanProcessor = new LangfuseSpanProcessor({ shouldExportSpan }); + +const tracerProvider = new NodeTracerProvider({ + spanProcessors: [langfuseSpanProcessor], +}); + +tracerProvider.register(); +``` + +```ts filename="route.ts" /observe/ /forceFlush/ +import { streamText } from "ai"; +import { after } from "next/server"; + +import { openai } from "@ai-sdk/openai"; +import { observe, updateActiveObservation, updateActiveTrace } from "@langfuse/tracing"; +import { trace } from "@opentelemetry/api"; + +import { langfuseSpanProcessor } from "@/src/instrumentation"; + +const handler = async (req: Request) => { + const { messages, chatId, userId } = await req.json(); + + updateActiveObservation({ input: messages[messages.length - 1] }); + updateActiveTrace({ name: "my-ai-sdk-trace", sessionId: chatId, userId }); + + const result = streamText({ + experimental_telemetry: { isEnabled: true }, + onFinish: async (result) => { + updateActiveObservation({ output: result.content }); + updateActiveTrace({ output: result.content }); + trace.getActiveSpan()?.end(); + }, + }); + + after(async () => await langfuseSpanProcessor.forceFlush()); + + return result.toUIMessageStreamResponse(); +}; + +export const POST = observe(handler, { + name: "handle-chat-message", + endOnExit: false, +}); +``` + +Any OTEL-instrumented library (databases, HTTP clients, agents) automatically exports spans to Langfuse once the `LangfuseSpanProcessor` is registered. + + + +## Custom instrumentation patterns [#custom-instrumentation] + +You can also create custom instrumentation patterns using the Langfuse SDK. + +All custom patterns are interoperable—you can nest a decorator-created observation inside a context manager or mix manual spans with native integrations. + +### Observe decorator [#observe-wrapper] + + + +Use the `@observe()` decorator to automatically capture inputs, outputs, timings, and errors of a wrapped function. + +```python +from langfuse import observe + +@observe() +def my_data_processing_function(data, parameter): + return {"processed_data": data, "status": "ok"} + +@observe(name="llm-call", as_type="generation") +async def my_async_llm_call(prompt_text): + return "LLM response" +``` + +**Parameters**: `name`, `as_type`, `capture_input`, `capture_output`, `transform_to_string`. Special kwargs such as `langfuse_trace_id` or `langfuse_parent_observation_id` let you stitch into existing traces. + +The decorator automatically propagates the OTEL trace context. Pass `langfuse_trace_id` when you need to force a specific trace ID (e.g., to align with an external system) and `langfuse_parent_observation_id` to attach to an existing parent span. + + +Capturing large inputs/outputs may add overhead. Disable IO capture per decorator (`capture_input=False`, `capture_output=False`) or via the `LANGFUSE_OBSERVE_DECORATOR_IO_CAPTURE_ENABLED` env var. + + + +Wrap existing functions with `observe` to trace calls without modifying their internals. + +```ts /observe/ /updateActiveObservation/ +import { observe, updateActiveObservation } from "@langfuse/tracing"; + +async function fetchData(source: string) { + updateActiveObservation({ metadata: { source: "API" } }); + return { data: `some data from ${source}` }; +} + +const tracedFetchData = observe(fetchData, {}); + +const result = await tracedFetchData("API"); +``` + +| Option | Description | Default | +| ------ | ----------- | ------- | +| `name` | Observation name | Function name | +| `asType` | Observation type (`span`, `generation`, etc.) | `"span"` | +| `captureInput` | Capture arguments as `input` | `true` | +| `captureOutput` | Capture return values/errors as `output` | `true` | + + + +### Context managers & callbacks [#context-management-with-callbacks] + +Context helpers keep spans correctly parented and automatically close them for you. + + + +`langfuse.start_as_current_observation()` is the primary way to create spans or generations while ensuring the active OpenTelemetry context is updated. Any child spans created inside the `with` block inherit the parent automatically. + +```python +from langfuse import get_client, propagate_attributes + +langfuse = get_client() + +with langfuse.start_as_current_observation( + as_type="span", + name="user-request-pipeline", + input={"user_query": "Tell me a joke"}, +) as root_span: + with propagate_attributes(user_id="user_123", session_id="session_abc"): + with langfuse.start_as_current_observation( + as_type="generation", + name="joke-generation", + model="gpt-4o", + ) as generation: + generation.update(output="Why did the span cross the road?") + + root_span.update(output={"final_joke": "..."}) +``` + + +`startActiveObservation` accepts a callback, makes the new span active for the callback scope, and ends it automatically—even across async boundaries. + +```ts /startActiveObservation/ +import { startActiveObservation, startObservation } from "@langfuse/tracing"; + +await startActiveObservation("user-request", async (span) => { + span.update({ input: { query: "Capital of France?" } }); + + const generation = startObservation( + "llm-call", + { model: "gpt-4", input: [{ role: "user", content: "Capital of France?" }] }, + { asType: "generation" } + ); + generation.update({ output: { content: "Paris." } }).end(); + + span.update({ output: "Answered." }); +}); +``` + + + +### Manual observations [#manual-observations] + +Manual APIs are useful when you need to create spans without altering the currently active context (e.g., background work or parallel tasks). + + + +Use `start_span()` / `start_generation()` when you need manual control without changing the active context. + +```python +from langfuse import get_client + +langfuse = get_client() + +span = langfuse.start_span(name="manual-span") +span.update(input="Data for side task") +child = span.start_span(name="child-span") +child.end() +span.end() +``` + + +Spans created via `start_span()` / `start_generation()` must be ended explicitly via `.end()`. + + + +`startObservation` gives you full control over lifecycle and nesting. + +```ts /startObservation/ +import { startObservation } from "@langfuse/tracing"; + +const span = startObservation("user-request", { + input: { query: "Capital of France?" }, +}); + +const toolCall = span.startObservation("fetch-weather", { input: { city: "Paris" } }); +await new Promise((resolve) => setTimeout(resolve, 100)); +toolCall.update({ output: { temperature: "15°C" } }).end(); + +const generation = span.startObservation( + "llm-call", + { model: "gpt-4", input: [{ role: "user", content: "Capital of France?" }] }, + { asType: "generation" } +); +generation.update({ output: { content: "Paris." } }).end(); + +span.update({ output: "Done." }).end(); +``` + + + +### Nesting observations + + + +The function call hierarchy is automatically captured by the `@observe` decorator and reflected in the trace. + +```python +from langfuse import observe + +@observe +def my_data_processing_function(data, parameter): + return {"processed_data": data, "status": "ok"} + + +@observe +def main_function(data, parameter): + return my_data_processing_function(data, parameter) +``` + + +Nesting happens automatically via OpenTelemetry context propagation. When you create a new observation with `startActiveObservation`, it becomes a child of whatever was active at the time. + +```ts +import { startActiveObservation } from "@langfuse/tracing"; + +await startActiveObservation("outer-process", async () => { + await startActiveObservation("llm-step-1", async (span) => { + span.update({ output: "LLM 1 output" }); + }); + + await startActiveObservation("intermediate-step", async (span) => { + await startActiveObservation("llm-step-2", async (child) => { + child.update({ output: "LLM 2 output" }); + }); + + span.update({ output: "Intermediate processing done" }); + }); +}); +``` + + + +### Update observations + + + +Update observation objects directly or use context-aware helpers. + +```python +from langfuse import get_client + +langfuse = get_client() + +with langfuse.start_as_current_observation(as_type="generation", name="llm-call") as gen: + gen.update( + input={"prompt": "Why is the sky blue?"}, + output="Rayleigh scattering", + usage_details={"input_tokens": 5, "output_tokens": 50}, + ) + +with langfuse.start_as_current_observation(as_type="span", name="data-processing"): + langfuse.update_current_span(metadata={"step1_complete": True}) +``` + + +Update the active observation or trace without holding a reference. + +```ts /updateActiveTrace/ +import { startActiveObservation, updateActiveTrace } from "@langfuse/tracing"; + +await startActiveObservation("user-request", async (span) => { + span.update({ input: { path: "/api/process" } }); + + await new Promise((resolve) => setTimeout(resolve, 50)); + const user = { id: "user-5678", name: "Jane Doe" }; + updateActiveTrace({ + userId: user.id, + metadata: { userName: user.name }, + }); + + span.update({ output: { status: "success" } }).end(); +}); +``` + + + +### Add attributes to observations +Propagate attributes such as `userId`, `sessionId`, `metadata`, `version`, and `tags` to keep downstream analytics consistent. These helpers mirror the Python `propagate_attributes` context manager and the TypeScript `propagateAttributes` callback wrapper from the standalone SDK docs. Use propagation for attributes that should appear on **every** observation and `updateTrace()`/`update_current_trace()` for single-trace fields like `name`, `input`, `output`, or `public`. + +**Propagatable attributes** + +- `userId` / `user_id` +- `sessionId` / `session_id` +- `metadata` +- `version` +- `tags` + +**Trace-only attributes** (use `updateTrace` / `update_current_trace`) + +- `name` +- `input` +- `output` +- `public` + + + +```python /propagate_attributes/ +from langfuse import get_client, propagate_attributes + +langfuse = get_client() + +with langfuse.start_as_current_observation(as_type="span", name="user-workflow"): + with propagate_attributes( + user_id="user_123", + session_id="session_abc", + metadata={"experiment": "variant_a"}, + version="1.0", + ): + with langfuse.start_as_current_observation(as_type="generation", name="llm-call"): + pass +``` + + +```ts /propagateAttributes/ +import { startActiveObservation, propagateAttributes, startObservation } from "@langfuse/tracing"; + +await startActiveObservation("user-workflow", async () => { + await propagateAttributes( + { + userId: "user_123", + sessionId: "session_abc", + metadata: { experiment: "variant_a", env: "prod" }, + version: "1.0", + }, + async () => { + const generation = startObservation("llm-call", { model: "gpt-4" }, { asType: "generation" }); + generation.end(); + } + ); +}); +``` + + + + + +### Cross-service propagation + +Use baggage propagation only when you need to forward attributes across HTTP boundaries. It pushes the values into every outbound request header, so prefer non-sensitive identifiers (session IDs, experiment versions, etc.). + + + + + +```python /as_baggage=True/ +from langfuse import get_client, propagate_attributes +import requests + +langfuse = get_client() + +with langfuse.start_as_current_observation(as_type="span", name="api-request"): + with propagate_attributes( + user_id="user_123", + session_id="session_abc", + as_baggage=True, + ): + requests.get("https://service-b.example.com/api") +``` + + +```ts /asBaggage/ +import { propagateAttributes, startActiveObservation } from "@langfuse/tracing"; + +await startActiveObservation("api-request", async () => { + await propagateAttributes( + { + userId: "user_123", + sessionId: "session_abc", + asBaggage: true, + }, + async () => { + await fetch("https://service-b.example.com/api"); + } + ); +}); +``` + + + + +When baggage propagation is enabled, attributes are added to **all** outbound HTTP headers. Only use it for non-sensitive values needed for distributed tracing. + + +### Trace-level metadata & inputs/outputs [#trace-inputoutput-behavior] + +By default, trace input/output mirror whatever you set on the **root observation**. Override them explicitly whenever evaluations, AB-tests, or judge models need a different payload than the root span captured. + +The snippets below illustrate both the default behavior and how to call `update_current_trace` / `updateActiveTrace()` to set trace-level payloads later in the workflow. + + +LLM-as-a-judge and evaluation workflows typically rely on trace-level inputs/outputs. Make sure to set them deliberately rather than relying on the root span if your evaluation payload differs. + + + + +Trace input/output default to the root observation. Override them explicitly when needed (e.g., for evaluations). + +```python +from langfuse import get_client + +langfuse = get_client() + +with langfuse.start_as_current_observation(as_type="span", name="complex-pipeline") as root_span: + root_span.update(input="Step 1 data", output="Step 1 result") + root_span.update_trace( + input={"original_query": "User question"}, + output={"final_answer": "Complete response", "confidence": 0.95}, + ) +``` + +```python /update_current_trace/ +from langfuse import observe, get_client + +langfuse = get_client() + +@observe() +def process_user_query(user_question: str): + answer = call_llm(user_question) + langfuse.update_current_trace( + input={"question": user_question}, + output={"answer": answer}, + ) + return answer +``` + + +Update trace-level fields via the observation object itself. + +```ts /updateTrace/ +import { startObservation } from "@langfuse/tracing"; + +const rootSpan = startObservation("data-processing"); + +// ... some initial steps ... + +const userId = "user-123"; +const sessionId = "session-abc"; +rootSpan.updateTrace({ + userId: userId, + sessionId: sessionId, + tags: ["authenticated-user"], + metadata: { plan: "premium" }, +}); + +const generation = rootSpan.startObservation( + "llm-call", + {}, + { asType: "generation" } +); + +generation.end(); + +rootSpan.end(); +``` + + + +### Trace and observation IDs [#managing-trace-and-observation-ids] + +Langfuse follows the W3C Trace Context standard: trace IDs are 32-character lowercase hex strings (16 bytes) and observation IDs are 16-character lowercase hex strings (8 bytes). You cannot set arbitrary observation IDs, but you can generate deterministic trace IDs to correlate with external systems. + + + +Langfuse uses W3C Trace Context IDs. Access current IDs or create deterministic ones. + +```python +from langfuse import get_client, Langfuse + +langfuse = get_client() + +with langfuse.start_as_current_observation(as_type="span", name="my-op") as current_op: + trace_id = langfuse.get_current_trace_id() + observation_id = langfuse.get_current_observation_id() + print(trace_id, observation_id) + +external_request_id = "req_12345" +deterministic_trace_id = Langfuse.create_trace_id(seed=external_request_id) +``` + + +Generate deterministic IDs or read the active trace ID. + +```ts +import { createTraceId, startObservation } from "@langfuse/tracing"; + +const externalId = "support-ticket-54321"; + +const langfuseTraceId = await createTraceId(externalId); + +const rootSpan = startObservation( + "process-ticket", + {}, + { + parentSpanContext: { + traceId: langfuseTraceId, + spanId: "0123456789abcdef", + traceFlags: 1, + }, + } +); +``` + +```ts +import { startObservation, getActiveTraceId } from "@langfuse/tracing"; + +await startObservation("run", async (span) => { + const traceId = getActiveTraceId(); + console.log(`Current trace ID: ${traceId}`); +}); +``` + + + +### Link to existing traces + +When integrating with upstream services that already have trace IDs, supply the W3C trace context so Langfuse spans join the existing tree rather than creating a new one. + + + +```python +from langfuse import get_client + +langfuse = get_client() + +existing_trace_id = "abcdef1234567890abcdef1234567890" +existing_parent_span_id = "fedcba0987654321" + +with langfuse.start_as_current_observation( + as_type="span", + name="process-downstream-task", + trace_context={ + "trace_id": existing_trace_id, + "parent_span_id": existing_parent_span_id, + }, +): + pass +``` + + +```ts +import { startObservation } from "@langfuse/tracing"; + +const span = startObservation( + "downstream-task", + {}, + { + parentSpanContext: { + traceId: "abcdef1234567890abcdef1234567890", + spanId: "fedcba0987654321", + traceFlags: 1, + }, + } +); + +span.end(); +``` + + + +### Client lifecycle & flushing + +Both SDKs buffer spans in the background. Always flush or shut down the exporter in short-lived processes (scripts, serverless functions, workers) to avoid losing data. + + + + + +Flush or shut down the client to ensure all buffered data is delivered—especially in short-lived jobs. + +```python +from langfuse import get_client + +langfuse = get_client() +# ... create traces ... +langfuse.flush() +langfuse.shutdown() +``` + + +Export the span processor from your OTEL setup and flush before the process exits (or use `after` in Vercel Functions). + +```ts filename="instrumentation.ts" /langfuseSpanProcessor/ +import { NodeSDK } from "@opentelemetry/sdk-node"; +import { LangfuseSpanProcessor } from "@langfuse/otel"; + +export const langfuseSpanProcessor = new LangfuseSpanProcessor({ + exportMode: "immediate", +}); + +const sdk = new NodeSDK({ + spanProcessors: [langfuseSpanProcessor], +}); + +sdk.start(); +``` + +```ts filename="handler.ts" /forceFlush/ +import { langfuseSpanProcessor } from "./instrumentation"; + +export async function handler(event, context) { + // ... logic ... + await langfuseSpanProcessor.forceFlush(); +} +``` + + diff --git a/pages/docs/observability/sdk/overview.mdx b/pages/docs/observability/sdk/overview.mdx index add6826d5..3f292e26a 100644 --- a/pages/docs/observability/sdk/overview.mdx +++ b/pages/docs/observability/sdk/overview.mdx @@ -1,50 +1,411 @@ --- -description: Fully typed and async SDKs to integrate with Langfuse for Python and Typescript. +title: Langfuse SDKs +description: Fully typed SDKs for Python and JavaScript/TypeScript with unified setup, instrumentation, and advanced guidance. +category: SDKs --- +import GetStartedPythonSdk from "@/components-mdx/get-started/python-sdk.mdx"; +import EnvJS from "@/components-mdx/env-js.mdx"; +import JSSDKPackages from "@/components-mdx/js-sdk-packages.mdx"; +import { Rocket, Plug, Settings, LifeBuoy, BookOpen } from "lucide-react"; + # Langfuse SDKs -The Langfuse SDKs are the recommended way to integrate with Langfuse for custom instrumentation. +Langfuse offers two SDKs for **Python** and **JS/TS**. The Langfuse SDKs are the recommended way to integrate custom instrumentation, evaluations, and prompt tooling with Langfuse. Both SDKs are OpenTelemetry-based, async by default, and interoperate with Langfuse integrations. -Properties: +
+Legacy documentation -- Based on OpenTelemetry, so you can use any OTEL-based instrumentation library for your LLM. -- Fully async requests, using Langfuse adds almost no latency -- Accurate latency tracking using synchronous timestamps -- IDs available for downstream use -- Great DX when nesting observations -- Cannot break your application, all errors are caught and logged -- Interoperable with Langfuse [integrations](/integrations) + + +This documentation is for the Python SDK v3. Documentation for the legacy Python SDK v2 can be found [here](https://python-sdk-v2.docs-snapshot.langfuse.com/docs/observability/sdk/python/decorators). + + +This documentation is for the TypeScript SDK v4. Documentation for the legacy TypeScript SDK v3 can be found [here](https://js-sdk-v3.docs-snapshot.langfuse.com/docs/observability/sdk/typescript/guide/). + + -## Python +
- +
+Requirements for self-hosted Langfuse -Please see our [Python SDK documentation](/docs/observability/sdk/python/overview) on how to get started. + + + +If you are self-hosting Langfuse, the Python SDK v3 requires [**Langfuse platform version >= 3.63.0**](https://github.com/langfuse/langfuse/releases/tag/v3.63.0) for traces to be correctly processed. + + + + -## JS/TS +If you are self-hosting Langfuse, the TypeScript SDK v4 requires **Langfuse platform version ≥ 3.95.0** for all features to work correctly. + + + -import JSSDKPackages from "@/components-mdx/js-sdk-packages.mdx"; +
+ + + + +**Key benefits** + +- Based on OpenTelemetry, so you can use any OTEL-based instrumentation library for your LLM stack. +- Fully async requests, meaning Langfuse adds almost no latency. +- Accurate latency tracking via synchronous timestamps. +- IDs available for downstream use. +- Great DX when nesting observations. +- Cannot break your application—SDK errors are caught and logged. +- Interoperable with Langfuse [integrations](/integrations). + +## Quickstart + +Follow the path for your runtime to get the first trace into Langfuse. + + + + + + + +**Install packages** + +Install the packages that power tracing with OpenTelemetry: + +```bash +npm install @langfuse/tracing @langfuse/otel @opentelemetry/sdk-node +``` + +Learn more about the packages [here](/docs/observability/sdk/overview#packages). + +**Set environment variables** + +Add your Langfuse credentials to your environment. Make sure that you have a `.env` file in your project root and a package like `dotenv` to load the variables. + + + +**Initialize OpenTelemetry** + +Create an `instrumentation.ts` to register the Langfuse span processor so traces reach Langfuse. + +```ts filename="instrumentation.ts" /LangfuseSpanProcessor/ +import { NodeSDK } from "@opentelemetry/sdk-node"; +import { LangfuseSpanProcessor } from "@langfuse/otel"; + +const sdk = new NodeSDK({ + spanProcessors: [new LangfuseSpanProcessor()], +}); + +sdk.start(); +``` + +Import this file at the top of your app's entry point (e.g., `index.ts`). + +**Instrument your app** + +Use a Langfuse [integration](/integrations) or manual instrumentation. The `startActiveObservation` helper manages the OpenTelemetry context for you. + +```ts filename="index.ts" /startActiveObservation/ +import "./instrumentation"; +import { startActiveObservation } from "@langfuse/tracing"; + +async function main() { + await startActiveObservation("my-first-trace", async (span) => { + span.update({ + input: "Hello, Langfuse!", + output: "This is my first trace!", + }); + }); +} + +main(); +``` + +**Run your application** + +Execute your application and the trace will appear in Langfuse. + +```bash +npx tsx index.ts +``` + + + + + +## Install the SDK + + + +```bash +pip install langfuse +``` + + +```bash +npm install @langfuse/tracing @langfuse/otel @opentelemetry/sdk-node +``` -Please see our [TypeScript SDK documentation](/docs/observability/sdk/typescript/overview) on how to get started. + + + +## Configure credentials + + + +Set credentials via environment variables (recommended) or constructor arguments. + +```bash filename=".env" +LANGFUSE_PUBLIC_KEY="pk-lf-..." +LANGFUSE_SECRET_KEY="sk-lf-..." +LANGFUSE_BASE_URL="https://cloud.langfuse.com" # US: https://us.cloud.langfuse.com +``` + + +If you create multiple `Langfuse` instances with the same `public_key`, the singleton instance is reused and new arguments are ignored. + + +**Key configuration options** + +| Constructor Argument | Environment Variable | Description | Default value | +| --------------------------- | ------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------ | +| `public_key` | `LANGFUSE_PUBLIC_KEY` | Your Langfuse project's public API key. **Required.** | | +| `secret_key` | `LANGFUSE_SECRET_KEY` | Your Langfuse project's secret API key. **Required.** | | +| `base_url` | `LANGFUSE_BASE_URL` | API host for your Langfuse instance. | `"https://cloud.langfuse.com"` | +| `timeout` | `LANGFUSE_TIMEOUT` | Timeout in seconds for API requests. | `5` | +| `httpx_client` | - | Custom `httpx.Client` for non-tracing HTTP requests. | | +| `debug` | `LANGFUSE_DEBUG` | Enables verbose logging. | `False` | +| `tracing_enabled` | `LANGFUSE_TRACING_ENABLED` | Toggles Langfuse instrumentation; if `False`, tracing calls become no-ops. | `True` | +| `flush_at` | `LANGFUSE_FLUSH_AT` | Number of spans to batch before sending. | `512` | +| `flush_interval` | `LANGFUSE_FLUSH_INTERVAL` | Seconds between batch flushes. | `5` | +| `environment` | `LANGFUSE_TRACING_ENVIRONMENT` | Environment name (lowercase alphanumeric, hyphen/underscore). | `"default"` | +| `release` | `LANGFUSE_RELEASE` | Release identifier for grouping analytics. | | +| `media_upload_thread_count` | `LANGFUSE_MEDIA_UPLOAD_THREAD_COUNT` | Background threads for media uploads. | `1` | +| `sample_rate` | `LANGFUSE_SAMPLE_RATE` | [Sampling](/docs/observability/features/sampling) rate between `0.0` and `1.0`. | `1.0` | +| `mask` | - | [Mask](/docs/observability/sdk/advanced-features#mask-sensitive-data) sensitive data before export. | | +| | `LANGFUSE_MEDIA_UPLOAD_ENABLED` | Whether to upload media files to Langfuse storage (useful to disable when self-hosting). | `True` | + + +Add credentials via environment variables so both tracing packages and `@langfuse/client` pick them up. + + + + + +## Initialize tracing + + + +```python filename="Initialize client" +from langfuse import get_client + +langfuse = get_client() + +# Verify connection +if langfuse.auth_check(): + print("Langfuse client is authenticated and ready!") +else: + print("Authentication failed. Please check your credentials and host.") +``` + +
+Key configuration options + +All key configuration options are listed in the [Python SDK reference](https://python.reference.langfuse.com/langfuse#Langfuse). + +| Constructor Argument | Environment Variable | Description | Default value | +| --------------------------- | ------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------ | +| `public_key` | `LANGFUSE_PUBLIC_KEY` | Your Langfuse project's public API key. **Required.** | | +| `secret_key` | `LANGFUSE_SECRET_KEY` | Your Langfuse project's secret API key. **Required.** | | +| `base_url` | `LANGFUSE_BASE_URL` | API host for your Langfuse instance. | `"https://cloud.langfuse.com"` | +| `timeout` | `LANGFUSE_TIMEOUT` | Timeout in seconds for API requests. | `5` | +| `httpx_client` | - | Custom `httpx.Client` for non-tracing HTTP requests. | | +| `debug` | `LANGFUSE_DEBUG` | Enables verbose logging. | `False` | +| `tracing_enabled` | `LANGFUSE_TRACING_ENABLED` | Toggles Langfuse instrumentation; if `False`, tracing calls become no-ops. | `True` | +| `flush_at` | `LANGFUSE_FLUSH_AT` | Number of spans to batch before sending. | `512` | +| `flush_interval` | `LANGFUSE_FLUSH_INTERVAL` | Seconds between batch flushes. | `5` | +| `environment` | `LANGFUSE_TRACING_ENVIRONMENT` | Environment name (lowercase alphanumeric, hyphen/underscore). | `"default"` | +| `release` | `LANGFUSE_RELEASE` | Release identifier for grouping analytics. | | +| `media_upload_thread_count` | `LANGFUSE_MEDIA_UPLOAD_THREAD_COUNT` | Background threads for media uploads. | `1` | +| `sample_rate` | `LANGFUSE_SAMPLE_RATE` | [Sampling](/docs/observability/features/sampling) rate between `0.0` and `1.0`. | `1.0` | +| `mask` | - | [Mask](/docs/observability/sdk/advanced-features#mask-sensitive-data) sensitive data before export. | | +| | `LANGFUSE_MEDIA_UPLOAD_ENABLED` | Whether to upload media files to Langfuse storage (useful to disable when self-hosting). | `True` | + +
+ +
+ +```ts filename="instrumentation.ts" /LangfuseSpanProcessor/ +import { NodeSDK } from "@opentelemetry/sdk-node"; +import { LangfuseSpanProcessor } from "@langfuse/otel"; + +const sdk = new NodeSDK({ + spanProcessors: [new LangfuseSpanProcessor()], +}); + +sdk.start(); +``` + +Configure tracing via environment variables (see above) and by passing options to `LangfuseSpanProcessor`, e.g. `mask`, `shouldExportSpan`, or `exportMode`. Advanced options such as sampling, isolated tracer providers, serverless flushing, and multi-project exports are covered in [Advanced features](/docs/observability/sdk/advanced-features). + +All key configuration options are listed in the [JS/TS SDK reference](https://js.reference.langfuse.com/classes/_langfuse_otel.LangfuseSpanProcessor.html). + + +**Next.js users:** use the `NodeSDK` setup above instead of `registerOTel` from `@vercel/otel`, which does not yet support the OpenTelemetry JS SDK v2. + + +
+ +## Access the client globally + + + +The Langfuse client is a singleton. It can be accessed anywhere in your application using the `get_client()` function. + +Optionally, you can initialize the client via `Langfuse()` to manually pass in the Langfuse credentials. Otherwise, it is created automatically when you call `get_client()` based on environment variables. + +```python +from langfuse import get_client + +# Get the default client +client = get_client() +``` +
+Alternative: configure via constructor + +```python filename="Initialize client" +from langfuse import Langfuse + +langfuse = Langfuse( + public_key="pk-lf-...", + secret_key="sk-lf-...", + base_url="https://cloud.langfuse.com" +) +``` +
+ + +
+ +Use `LangfuseClient` for non-tracing APIs (datasets, prompts, scores). Environment variables are picked up automatically. + +```ts filename="client.ts" +import { LangfuseClient } from "@langfuse/client"; + +const langfuse = new LangfuseClient(); +``` + +
+Alternative: configure via constructor + +```ts filename="client.ts" +import { LangfuseClient } from "@langfuse/client"; + +const langfuse = new LangfuseClient({ + publicKey: "your-public-key", + secretKey: "your-secret-key", + baseUrl: "https://cloud.langfuse.com", +}); +``` + +
+ + +
+
+ +## OpenTelemetry foundation + +Building on OpenTelemetry provides: + +- **Standardization** with the wider observability ecosystem and tooling. +- **Robust context propagation** so nested spans stay connected, even across async workloads. +- **Attribute propagation** to keep `userId`, `sessionId`, `metadata`, `version`, and `tags` aligned across observations. +- **Ecosystem interoperability** meaning third-party instrumentations automatically appear inside Langfuse traces. + +The following diagram shows how Langfuse maps to native OpenTelemetry concepts: + +```mermaid +graph TD + subgraph OTEL_Core_Concepts ["OpenTelemetry"] + direction LR + OTEL_Trace["OTel Trace"] + Root_OTEL_Span["Root OTel Span"] + Child_OTEL_Span["Child OTel Span"] + + OTEL_Trace -- is defined by --> Root_OTEL_Span + Root_OTEL_Span -- Hierarchy via
Context Propagation --> Child_OTEL_Span + end + + subgraph Langfuse_Mapping ["Langfuse"] + direction LR + LF_Trace["Langfuse Trace"] + LF_Observation["Langfuse Observation
(typed as either Span, Generation or Event)"] + + LF_Trace -- Collects one or more --> LF_Observation + end + + OTEL_Trace -.->|shares ID with | LF_Trace + + Root_OTEL_Span -.->|Mapped to| LF_Observation + Child_OTEL_Span -.->|Mapped to| LF_Observation + + Root_OTEL_Span -.->|sets default input and output | LF_Trace + Root_OTEL_Span -.->|can hold trace attributes| LF_Trace + Child_OTEL_Span -.->|can hold trace attributes| LF_Trace + + classDef otel fill:#D6EAF8,stroke:#3498DB,stroke-width:2px,color:#000; + classDef langfuse fill:#D5F5E3,stroke:#2ECC71,stroke-width:2px,color:#000; + class OTEL_Trace,Root_OTEL_Span,Child_OTEL_Span otel; + class LF_Trace,LF_Observation langfuse; +``` + +## Learn more -## Other Languages + + } + title="Instrument your app" + href="/docs/observability/sdk/instrumentation" + arrow + /> + } + title="Advanced features" + href="/docs/observability/sdk/advanced-features" + arrow + /> + } + title="Upgrade path" + href="/docs/observability/sdk/upgrade-path" + arrow + /> + } + title="Troubleshooting & FAQ" + href="/docs/observability/sdk/troubleshooting-and-faq" + arrow + /> + } + title="Python API reference" + href="https://python.reference.langfuse.com" + newWindow + arrow + /> + } + title="JS/TS API reference" + href="https://js.reference.langfuse.com/" + newWindow + arrow + /> + -Via the public API, you can integrate with Langfuse from any language. +## Other languages -For tracing, use the OpenTelemetry SDK of your choice and send traces to the [Langfuse OTel Endpoint](/integrations/native/opentelemetry). +Via the [public API](/docs/api-and-data-platform/features/public-api) you can integrate Langfuse from any runtime. For tracing specifically, send OpenTelemetry spans from your preferred instrumentation (Java, Go, etc.) to the [Langfuse OTel endpoint](/integrations/native/opentelemetry). diff --git a/pages/docs/observability/sdk/python/_meta.tsx b/pages/docs/observability/sdk/python/_meta.tsx deleted file mode 100644 index 4308f276f..000000000 --- a/pages/docs/observability/sdk/python/_meta.tsx +++ /dev/null @@ -1,14 +0,0 @@ -export default { - overview: "Overview", - setup: "Setup", - instrumentation: "Instrumentation", - evaluation: "Evaluation", - "advanced-usage": "Advanced usage", - "upgrade-path": "Upgrade Path", - "troubleshooting-and-faq": "Troubleshooting", - reference: { - title: "Reference ↗", - href: "https://python.reference.langfuse.com", - newWindow: true, - }, -}; \ No newline at end of file diff --git a/pages/docs/observability/sdk/python/advanced-usage.mdx b/pages/docs/observability/sdk/python/advanced-usage.mdx deleted file mode 100644 index 86a1fd4cf..000000000 --- a/pages/docs/observability/sdk/python/advanced-usage.mdx +++ /dev/null @@ -1,423 +0,0 @@ ---- -title: Advanced usage of the Langfuse Python SDK -description: Advanced usage of the Langfuse Python SDK for data masking, logging, sampling, filtering, and more. -category: SDKs ---- - -# Advanced Usage - -The Python SDK provides advanced usage options for your application. This includes data masking, logging, sampling, filtering, and more. - -## Masking Sensitive Data - -If your trace data (inputs, outputs, metadata) might contain sensitive information (PII, secrets), you can provide a `mask` function during client initialization. This function will be applied to all relevant data before it's sent to Langfuse. - -The `mask` function should accept `data` as a keyword argument and return the masked data. The returned data must be JSON-serializable. - -```python -from langfuse import Langfuse -import re - -def pii_masker(data: any, **kwargs) -> any: - # Example: Simple email masking. Implement your more robust logic here. - if isinstance(data, str): - return re.sub(r"[a-zA-Z0-9_.+-]+@[a-zA-Z0-9-]+\.[a-zA-Z0-9-.]+", "[EMAIL_REDACTED]", data) - elif isinstance(data, dict): - return {k: pii_masker(data=v) for k, v in data.items()} - elif isinstance(data, list): - return [pii_masker(data=item) for item in data] - return data - -langfuse = Langfuse(mask=pii_masker) - -# Now, any input/output/metadata will be passed through pii_masker -with langfuse.start_as_current_observation(as_type="span", name="user-query", input={"email": "test@example.com", "query": "..."}) as span: - # The 'email' field in the input will be masked. - pass -``` - -## Logging - -The Langfuse SDK uses Python's standard `logging` module. The main logger is named `"langfuse"`. -To enable detailed debug logging, you can either: - -1. Set the `debug=True` parameter when initializing the `Langfuse` client. -2. Set the `LANGFUSE_DEBUG="True"` environment variable. -3. Configure the `"langfuse"` logger manually: - -```python -import logging - -langfuse_logger = logging.getLogger("langfuse") -langfuse_logger.setLevel(logging.DEBUG) -``` - -The default log level for the `langfuse` logger is `logging.WARNING`. - -## Sampling - -You can configure the SDK to sample traces by setting the `sample_rate` parameter during client initialization (or via the `LANGFUSE_SAMPLE_RATE` environment variable). This value should be a float between `0.0` (sample 0% of traces) and `1.0` (sample 100% of traces). - -If a trace is not sampled, none of its observations (spans, generations) or associated scores will be sent to Langfuse. - -```python -from langfuse import Langfuse - -# Sample approximately 20% of traces -langfuse_sampled = Langfuse(sample_rate=0.2) -``` - -## Filtering by Instrumentation Scope - -You can configure the SDK to filter out spans from specific instrumentation libraries by using the `blocked_instrumentation_scopes` parameter. This is useful when you want to exclude infrastructure spans while keeping your LLM and application spans. - -```python -from langfuse import Langfuse - -# Filter out database spans -langfuse = Langfuse( - blocked_instrumentation_scopes=["sqlalchemy", "psycopg"] -) -``` - -**How it works:** - -When third-party libraries create OpenTelemetry spans (through their instrumentation packages), each span has an associated "instrumentation scope" that identifies which library created it. The Langfuse SDK filters spans at the export level based on these scope names. - -You can see the instrumentation scope name for any span in the Langfuse UI under the span's metadata (`metadata.scope.name`). Use this to identify which scopes you want to filter. - - -**Cross-Library Span Relationships** - -When filtering instrumentation scopes, be aware that blocking certain libraries may break trace tree relationships if spans from blocked and non-blocked libraries are nested together. - -For example, if you block parent spans but keep child spans from a separate library, you may see "orphaned" LLM spans whose parent spans were filtered out. This can make traces harder to interpret. - -Consider the impact on trace structure when choosing which scopes to filter. - - -## Isolated TracerProvider - -You can configure a separate OpenTelemetry TracerProvider for use with Langfuse. This creates isolation between Langfuse tracing and your other observability systems. - -**Benefits of isolation:** -- Langfuse spans won't be sent to your other observability backends (e.g., Datadog, Jaeger, Zipkin) -- Third-party library spans won't be sent to Langfuse -- Independent configuration and sampling rates - - -While TracerProviders are isolated, they share the same OpenTelemetry context for tracking active spans. This can cause span relationship issues where: -- A parent span from one TracerProvider might have children from another TracerProvider -- Some spans may appear "orphaned" if their parent spans belong to a different TracerProvider -- Trace hierarchies may be incomplete or confusing - -Plan your instrumentation carefully to avoid confusing trace structures. - - -```python -from opentelemetry.sdk.trace import TracerProvider - -from langfuse import Langfuse - -langfuse_tracer_provider = TracerProvider() # do not set to global tracer provider to keep isolation - -langfuse = Langfuse(tracer_provider=langfuse_tracer_provider) - -langfuse.start_span(name="myspan").end() # Span will be isolated from remaining OTEL instrumentation -``` - -## Using `ThreadPoolExecutors` - -Please use the [OpenTelemetry ThreadingInstrumentor](https://opentelemetry-python-contrib.readthedocs.io/en/latest/instrumentation/threading/threading.html) to ensure that the OpenTelemetry context is correctly propagated to all threads. - -```python filename="main.py" -from opentelemetry.instrumentation.threading import ThreadingInstrumentor - -ThreadingInstrumentor().instrument() -``` - -## Distributed tracing - -To maintain the trace context across service / process boundaries, please rely on the OpenTelemetry native [context propagation](https://opentelemetry.io/docs/concepts/context-propagation/) across service / process boundaries as much as possible. - -Using the `trace_context` argument to 'force' the parent child relationship may lead to unexpected trace updates as the resulting span will be treated as a root span server side. - -- If you are using multiprocessing, [see here for details on how to propagate the OpenTelemetry context](https://github.com/open-telemetry/opentelemetry-python/issues/2765#issuecomment-1158402076). -- If you are using Pydantic Logfire, please set `distributed_tracing` to `True`. - - -## Multi-Project Setup (Experimental) - - -Multi-project setups are **experimental** and have important limitations regarding third-party OpenTelemetry integrations. - - -The Langfuse Python SDK supports routing traces to different projects within the same application by using multiple public keys. This works because the Langfuse SDK adds a specific span attribute containing the public key to all spans it generates. - -**How it works:** - -1. **Span Attributes**: The Langfuse SDK adds a specific span attribute containing the public key to spans it creates -2. **Multiple Processors**: Multiple span processors are registered onto the global tracer provider, each with their respective exporters bound to a specific public key -3. **Filtering**: Within each span processor, spans are filtered based on the presence and value of the public key attribute - -**Important Limitation with Third-Party Libraries:** - -Third-party libraries that emit OpenTelemetry spans automatically (e.g., HTTP clients, databases, other instrumentation libraries) do **not** have the Langfuse public key span attribute. As a result: - -- These spans cannot be routed to a specific project -- They are processed by all span processors and sent to all projects -- All projects will receive these third-party spans - -**Why is this experimental?** -This approach requires that the `public_key` parameter be passed to all Langfuse SDK executions across all integrations to ensure proper routing, and third-party spans will appear in all projects. - -### Initialization - -To set up multiple projects, initialize separate Langfuse clients for each project: - -```python -from langfuse import Langfuse - -# Initialize clients for different projects -project_a_client = Langfuse( - public_key="pk-lf-project-a-...", - secret_key="sk-lf-project-a-...", - base_url="https://cloud.langfuse.com" -) - -project_b_client = Langfuse( - public_key="pk-lf-project-b-...", - secret_key="sk-lf-project-b-...", - base_url="https://cloud.langfuse.com" -) -``` - -### Integration Usage - -For all integrations in multi-project setups, you must specify the `public_key` parameter to ensure traces are routed to the correct project. - -**Observe Decorator:** - -Pass `langfuse_public_key` as a keyword argument to the *top-most* observed function (not the decorator). From Python SDK >= 3.2.2, nested decorated functions will automatically pick up the public key from the execution context they are currently into. Also, calls to `get_client` will be also aware of the current `langfuse_public_key` in the decorated function execution context, so passing the `langfuse_public_key` here again is not necessary. - -```python -from langfuse import observe - -@observe -def nested(): - # get_client call is context aware - # if it runs inside another decorated function that has - # langfuse_public_key passed, it does not need passing here again - - -@observe -def process_data_for_project_a(data): - # passing `langfuse_public_key` here again is not necessarily - # as it is stored in execution context - nested() - - return {"processed": data} - -@observe -def process_data_for_project_b(data): - # passing `langfuse_public_key` here again is not necessarily - # as it is stored in execution context - nested() - - return {"enhanced": data} - -# Route to Project A -# Top-most decorated function needs `langfuse_public_key` kwarg -result_a = process_data_for_project_a( - data="input data", - langfuse_public_key="pk-lf-project-a-..." -) - -# Route to Project B -# Top-most decorated function needs `langfuse_public_key` kwarg -result_b = process_data_for_project_b( - data="input data", - langfuse_public_key="pk-lf-project-b-..." -) -``` - -**OpenAI Integration:** - -Add `langfuse_public_key` as a keyword argument to the OpenAI execution: - -```python -from langfuse.openai import openai - -client = openai.OpenAI() - -# Route to Project A -response_a = client.chat.completions.create( - model="gpt-4o", - messages=[{"role": "user", "content": "Hello from Project A"}], - langfuse_public_key="pk-lf-project-a-..." -) - -# Route to Project B -response_b = client.chat.completions.create( - model="gpt-4o", - messages=[{"role": "user", "content": "Hello from Project B"}], - langfuse_public_key="pk-lf-project-b-..." -) -``` - -**Langchain Integration:** - -Add `public_key` to the CallbackHandler constructor: - -```python -from langfuse.langchain import CallbackHandler -from langchain_openai import ChatOpenAI -from langchain_core.prompts import ChatPromptTemplate - -# Create handlers for different projects -handler_a = CallbackHandler(public_key="pk-lf-project-a-...") -handler_b = CallbackHandler(public_key="pk-lf-project-b-...") - -llm = ChatOpenAI(model_name="gpt-4o") -prompt = ChatPromptTemplate.from_template("Tell me about {topic}") -chain = prompt | llm - -# Route to Project A -response_a = chain.invoke( - {"topic": "machine learning"}, - config={"callbacks": [handler_a]} -) - -# Route to Project B -response_b = chain.invoke( - {"topic": "data science"}, - config={"callbacks": [handler_b]} -) -``` - -**Important Considerations:** - -- Every Langfuse SDK execution across all integrations must include the appropriate public key parameter -- Missing public key parameters may result in traces being routed to the default project or lost -- Third-party OpenTelemetry spans (from HTTP clients, databases, etc.) will appear in all projects since they lack the Langfuse public key attribute - -## Passing `completion_start_time` for TTFT tracking - -If you are using the Python SDK to manually create generations, you can pass the `completion_start_time` parameter. This allows langfuse to calculate the time to first token (TTFT) for you. - -```python -from langfuse import get_client -import datetime -import time - -langfuse = get_client() - -# Start observation with specific type -with langfuse.start_as_current_observation( - as_type="generation", - name="TTFT-Generation" -) as generation: - - # simulate LLM time to first token - time.sleep(3) - - # Update the generation with the time the model started to generate - generation.update( - completion_start_time=datetime.datetime.now(), - output="some response", - ) - -# Flush events in short-lived applications -langfuse.flush() -``` - -## Self-signed SSL certificates (self-hosted Langfuse) - -If you are [self-hosting](/docs/deployment/self-host) Langfuse and you'd like to use self-signed SSL certificates, you will need to configure the SDK to trust the self-signed certificate: - - -Changing SSL settings has major security implications depending on your environment. Be sure you understand these implications before you proceed. - - -**1. Set OpenTelemetry span exporter to trust self-signed certificate** - -```bash filename=".env" -OTEL_EXPORTER_OTLP_TRACES_CERTIFICATE="/path/to/my-selfsigned-cert.crt" -``` - -**2. Set HTTPX to trust certificate for all other API requests to Langfuse instance** - -```python filename="main.py" -import os - -import httpx - -from langfuse import Langfuse - -httpx_client = httpx.Client(verify=os.environ["OTEL_EXPORTER_OTLP_TRACES_CERTIFICATE"]) - -langfuse = Langfuse(httpx_client=httpx_client) -``` - -## Observation Types - -Langfuse supports multiple observation types to provide context for different components of LLM applications. -The full list of the observation types is document here: [Observation types](/docs/observability/features/observation-types). - -### Setting observation types with the `@observe` decorator - -By setting the `as_type` parameter in the `@observe` decorator, you can specify the observation type for a method: - -```python /as_type="tool"/ -from langfuse import observe - -# Tool calls to external services -@observe(as_type="tool") -def retrieve_context(query): - results = vector_store.get(query) - return results -``` - -### Setting observation types with client methods and context manager - -With the Langfuse client, you can directly create an observation with a defined type: - -```python /as_type="embedding"/ -from langfuse import get_client() -langfuse = get_client() - -def process_with_manual_tracing(): - trace = langfuse.trace(name="document-processing") - - # Create different observation types - embedding_obs = trace.start_observation( - as_type="embedding", - name="document-embedding", - input={"document": "text content"} - ) - - embeddings = generate_embeddings("text content") - embedding_obs.update(output={"embeddings": embeddings}) - embedding_obs.end() -``` - -The context manager approach provides automatic resource cleanup: - -```python /as_type="chain"/ -from langfuse import get_client -langfuse = get_client() - -def process_with_context_managers(): - with langfuse.start_as_current_observation( - as_type="chain", - name="retrieval-pipeline", - ) as chain: - - # Retrieval step - with langfuse.start_as_current_observation( - as_type="retriever", - name="vector-search", - ) as retriever: - search_results = perform_vector_search("user question") - retriever.update(output={"results": search_results}) -``` diff --git a/pages/docs/observability/sdk/python/evaluation.mdx b/pages/docs/observability/sdk/python/evaluation.mdx deleted file mode 100644 index ef72e286a..000000000 --- a/pages/docs/observability/sdk/python/evaluation.mdx +++ /dev/null @@ -1,130 +0,0 @@ ---- -title: Evaluations with the Langfuse Python SDK -description: Evaluate your application with the Langfuse Python SDK. -category: SDKs ---- - -# Evaluations - -The Python SDK provides ways to evaluate your application. You can add custom scores to your traces and observations, or use the SDK to execute Dataset Runs. - - - -This page shows the evaluation methods that are supported by the Python SDK. Please refer to the [Evaluation documentation](/docs/evaluation/overview) for more information on how to evaluate your application in Langfuse. - - - -## Create Scores - - - - - - `span_or_generation_obj.score()`: Scores the specific observation object. - - `span_or_generation_obj.score_trace()`: Scores the entire trace to which the object belongs. - - ```python - from langfuse import get_client - - langfuse = get_client() - - with langfuse.start_as_current_observation(as_type="generation", name="summary_generation") as gen: - # ... LLM call ... - gen.update(output="summary text...") - # Score this specific generation - gen.score(name="conciseness", value=0.8, data_type="NUMERIC") - # Score the overall trace - gen.score_trace(name="user_feedback_rating", value="positive", data_type="CATEGORICAL") - ``` - - - - - - `langfuse.score_current_span()`: Scores the currently active observation in the context. - - `langfuse.score_current_trace()`: Scores the trace of the currently active observation. - - ```python - from langfuse import get_client - - langfuse = get_client() - - with langfuse.start_as_current_observation(as_type="span", name="complex_task") as task_span: - # ... perform task ... - langfuse.score_current_span(name="task_component_quality", value=True, data_type="BOOLEAN") - # ... - if task_is_fully_successful: - langfuse.score_current_trace(name="overall_success", value=1.0, data_type="NUMERIC") - ``` - - - - - - Creates a score for a specified `trace_id` and optionally `observation_id`. - - Useful when IDs are known, or for scoring after the trace/observation has completed. - - ```python - from langfuse import get_client - - langfuse = get_client() - - langfuse.create_score( - name="fact_check_accuracy", - value=0.95, # Can be float for NUMERIC/BOOLEAN, string for CATEGORICAL - trace_id="abcdef1234567890abcdef1234567890", - observation_id="1234567890abcdef", # Optional: if scoring a specific observation - session_id="session_123", # Optional: if scoring a specific session - data_type="NUMERIC", # "NUMERIC", "BOOLEAN", "CATEGORICAL" - comment="Source verified for 95% of claims." - ) - ``` - - - - - -**Score Parameters:** - -| Parameter | Type | Description | -| :--------------- | :------------------------ | :--------------------------------------------------------------------------------------------------------------------- | -| `name` | `str` | Name of the score (e.g., "relevance", "accuracy"). **Required.** | -| `value` | `Union[float, str]` | Score value. Float for `NUMERIC`/`BOOLEAN`, string for `CATEGORICAL`. **Required.** | -| `trace_id` | `str` | ID of the trace to associate with (for `create_score`). **Required.** | -| `observation_id` | `Optional[str]` | ID of the specific observation to score (for `create_score`). | -| `session_id` | `Optional[str]` | ID of the specific session to score (for `create_score`). | -| `score_id` | `Optional[str]` | Custom ID for the score (auto-generated if None). | -| `data_type` | `Optional[ScoreDataType]` | `"NUMERIC"`, `"BOOLEAN"`, or `"CATEGORICAL"`. Inferred if not provided based on value type and score config on server. | -| `comment` | `Optional[str]` | Optional comment or explanation for the score. | -| `config_id` | `Optional[str]` | Optional ID of a pre-defined score configuration in Langfuse. | - -See [Scoring](/docs/scores/overview) for more details. - -## Dataset Runs - -[Langfuse Datasets](/docs/datasets/overview) are essential for evaluating and testing your LLM applications by allowing you to manage collections of inputs and their expected outputs. - -### Create a Dataset - -- **Creating**: You can programmatically create new datasets with `langfuse.create_dataset(...)` and add items to them using `langfuse.create_dataset_item(...)`. -- **Fetching**: Retrieve a dataset and its items using `langfuse.get_dataset(name: str)`. This returns a `DatasetClient` instance, which contains a list of `DatasetItemClient` objects (accessible via `dataset.items`). Each `DatasetItemClient` holds the `input`, `expected_output`, and `metadata` for an individual data point. - -```python -from langfuse import get_client - -langfuse = get_client() - -# Fetch an existing dataset -dataset = langfuse.get_dataset(name="my-eval-dataset") -for item in dataset.items: - print(f"Input: {item.input}, Expected: {item.expected_output}") - -# Briefly: Creating a dataset and an item -new_dataset = langfuse.create_dataset(name="new-summarization-tasks") -langfuse.create_dataset_item( - dataset_name="new-summarization-tasks", - input={"text": "Long article..."}, - expected_output={"summary": "Short summary."} -) -``` - -### Run experiment on dataset - -After fetching your dataset, you can execute a run against it. This will create a new trace for each item in the dataset. Please refer to the [Experiments via SDK documentation](/docs/evaluation/experiments/experiments-via-sdk) for more details. diff --git a/pages/docs/observability/sdk/python/instrumentation.mdx b/pages/docs/observability/sdk/python/instrumentation.mdx deleted file mode 100644 index 02dc68035..000000000 --- a/pages/docs/observability/sdk/python/instrumentation.mdx +++ /dev/null @@ -1,964 +0,0 @@ ---- -title: Instrument your application with the Langfuse Python SDK -description: Instrument your application with the Langfuse Python SDK to trace your application and ingest data into Langfuse. -category: SDKs ---- - -# Instrumentation - -To instrument your application to send traces to Langfuse, you can use either native library instrumentations that work out of the box, or use custom instrumentation methods for fine-grained control. - -## Custom Instrumentation - -There are three main ways to create spans with the Langfuse Python SDK. All of them are fully interoperable with each other. - - - - - -The `@observe()` decorator provides a convenient way to automatically trace function executions, including capturing their inputs, outputs, execution time, and any errors. It supports both synchronous and asynchronous functions. - -```python -from langfuse import observe - -@observe() -def my_data_processing_function(data, parameter): - # ... processing logic ... - return {"processed_data": data, "status": "ok"} - -@observe(name="llm-call", as_type="generation") -async def my_async_llm_call(prompt_text): - # ... async LLM call ... - return "LLM response" -``` - -**Parameters:** - -- `name: Optional[str]`: Custom name for the created span/generation. Defaults to the function name. -- `as_type: Optional[Literal["generation"]]`: If set to `"generation"`, a Langfuse generation object is created, suitable for LLM calls. Otherwise, a regular span is created. -- `capture_input: bool`: Whether to capture function arguments as input. Defaults to env var `LANGFUSE_OBSERVE_DECORATOR_IO_CAPTURE_ENABLED` or `True` if not set. -- `capture_output: bool`: Whether to capture function return value as output. Defaults to env var `LANGFUSE_OBSERVE_DECORATOR_IO_CAPTURE_ENABLED` or `True` if not set. -- `transform_to_string: Optional[Callable[[Iterable], str]]`: For functions that return generators (sync or async), this callable can be provided to transform the collected chunks into a single string for the `output` field. If not provided, and all chunks are strings, they will be concatenated. Otherwise, the list of chunks is stored. - -**Trace Context and Special Keyword Arguments:** - -The `@observe` decorator automatically propagates the OTEL trace context. If a decorated function is called from within an active Langfuse span (or another OTEL span), the new observation will be nested correctly. - -You can also pass special keyword arguments to a decorated function to control its tracing behavior: - -- `langfuse_trace_id: str`: Explicitly set the trace ID for this function call. Must be a valid W3C Trace Context trace ID (32-char hex). If you have a trace ID from an external system, you can use `Langfuse.create_trace_id(seed=external_trace_id)` to generate a valid deterministic ID. -- `langfuse_parent_observation_id: str`: Explicitly set the parent observation ID. Must be a valid W3C Trace Context span ID (16-char hex). - -```python -@observe() -def my_function(a, b): - return a + b - -# Call with a specific trace context -my_function(1, 2, langfuse_trace_id="1234567890abcdef1234567890abcdef") -``` - - - -The observe decorator is capturing the args, kwargs and return value of decorated functions by default. This may lead to performance issues in your application if you have large or deeply nested objects there. To avoid this, explicitly disable function IO capture on the decorated function by passing `capture_input / capture_output` with value `False` or globally by setting the environment variable `LANGFUSE_OBSERVE_DECORATOR_IO_CAPTURE_ENABLED=False`. - - - - - - -You can create spans or generations anywhere in your application. If you need more control than the `@observe` decorator, the primary way to do this is using context managers (with `with` statements), which ensure that observations are properly started and ended. - -- `langfuse.start_as_current_observation(as_type="span")`: Creates a new span and sets it as the currently active observation in the OTel context for its duration. Any new observations created within this block will be its children. -- `langfuse.start_as_current_observation(as_type="generation")`: Similar to the above, but creates a specialized "generation" observation type for LLM calls. -- You can see an overview of the different observation types [here](/docs/observability/features/observation-types). - -```python -from langfuse import get_client, propagate_attributes - -langfuse = get_client() - -with langfuse.start_as_current_observation( - as_type="span", - name="user-request-pipeline", - input={"user_query": "Tell me a joke about OpenTelemetry"}, -) as root_span: - # This span is now active in the context. - - # Propagate trace attributes to all child observations - with propagate_attributes( - user_id="user_123", - session_id="session_abc", - tags=["experimental", "comedy"] - ): - - # Create a nested generation - with langfuse.start_as_current_observation( - as_type="generation", - name="joke-generation", - model="gpt-4o", - input=[{"role": "user", "content": "Tell me a joke about OpenTelemetry"}], - model_parameters={"temperature": 0.7} - ) as generation: - # Simulate an LLM call - joke_response = "Why did the OpenTelemetry collector break up with the span? Because it needed more space... for its attributes!" - token_usage = {"input_tokens": 10, "output_tokens": 25} - - generation.update( - output=joke_response, - usage_details=token_usage - ) - # Generation ends automatically here - - root_span.update(output={"final_joke": joke_response}) - # Root span ends automatically here -``` - - - - -For scenarios where you need to create an observation (a span or generation) without altering the currently active OpenTelemetry context, you can use `langfuse.start_span()` or `langfuse.start_generation()`. - -```python -from langfuse import get_client - -langfuse = get_client() - -span = langfuse.start_span(name="my-span") - -span.end() # Important: Manually end the span -``` - - - If you use `langfuse.start_span()` or `langfuse.start_generation()`, you are - responsible for calling `.end()` on the returned observation object. Failure - to do so will result in incomplete or missing observations in Langfuse. Their - `start_as_current_...` counterparts used with a `with` statement handle this - automatically. - - -**Key Characteristics:** - -- **No Context Shift**: Unlike their `start_as_current_...` counterparts, these methods **do not** set the new observation as the active one in the OpenTelemetry context. The previously active span (if any) remains the current context for subsequent operations in the main execution flow. -- **Parenting**: The observation created by `start_span()` or `start_generation()` will still be a child of the span that was active in the context at the moment of its creation. -- **Manual Lifecycle**: These observations are not managed by a `with` block and therefore **must be explicitly ended** by calling their `.end()` method. -- **Nesting Children**: - - Subsequent observations created using the global `langfuse.start_as_current_observation()` (or similar global methods) will _not_ be children of these "manual" observations. Instead, they will be parented by the original active span. - - To create children directly under a "manual" observation, you would use methods _on that specific observation object_ (e.g., `manual_span.start_as_current_observation(...)`). - -**When to Use:** - -This approach is useful when you need to: - -- Record work that is self-contained or happens in parallel to the main execution flow but should still be part of the same overall trace (e.g., a background task initiated by a request). -- Manage the observation's lifecycle explicitly, perhaps because its start and end are determined by non-contiguous events. -- Obtain an observation object reference before it's tied to a specific context block. - -**Example with more complex nesting:** - -```python -from langfuse import get_client - -langfuse = get_client() - -# This outer span establishes an active context. -with langfuse.start_as_current_observation(as_type="span", name="main-operation") as main_operation_span: - # 'main_operation_span' is the current active context. - - # 1. Create a "manual" span using langfuse.start_span(). - # - It becomes a child of 'main_operation_span'. - # - Crucially, 'main_operation_span' REMAINS the active context. - # - 'manual_side_task' does NOT become the active context. - manual_side_task = langfuse.start_span(name="manual-side-task") - manual_side_task.update(input="Data for side task") - - # 2. Start another operation that DOES become the active context. - # This will be a child of 'main_operation_span', NOT 'manual_side_task', - # because 'manual_side_task' did not alter the active context. - with langfuse.start_as_current_observation(as_type="span", name="core-step-within-main") as core_step_span: - # 'core_step_span' is now the active context. - # 'manual_side_task' is still open but not active in the global context. - core_step_span.update(input="Data for core step") - # ... perform core step logic ... - core_step_span.update(output="Core step finished") - # 'core_step_span' ends. 'main_operation_span' is the active context again. - - # 3. Complete and end the manual side task. - # This could happen at any point after its creation, even after 'core_step_span'. - manual_side_task.update(output="Side task completed") - manual_side_task.end() # Manual end is crucial for 'manual_side_task' - - main_operation_span.update(output="Main operation finished") -# 'main_operation_span' ends automatically here. - -# Expected trace structure in Langfuse: -# - main-operation -# |- manual-side-task -# |- core-step-within-main -# (Note: 'core-step-within-main' is a sibling to 'manual-side-task', both children of 'main-operation') -``` - - - - -### Nesting Observations - - - - -The function call hierarchy is automatically captured by the `@observe` decorator reflected in the trace. - -```python -from langfuse import observe - -@observe -def my_data_processing_function(data, parameter): - # ... processing logic ... - return {"processed_data": data, "status": "ok"} - - -@observe -def main_function(data, parameter): - return my_data_processing_function(data, parameter) -``` - - - - -Nesting is handled automatically by OpenTelemetry's context propagation. When you create a new observation using `start_as_current_observation`, it becomes a child of the observation that was active in the context when it was created. - -```python -from langfuse import get_client - -langfuse = get_client() - -with langfuse.start_as_current_observation(as_type="span", name="outer-process") as outer_span: - # outer_span is active - - with langfuse.start_as_current_observation(as_type="generation", name="llm-step-1") as gen1: - # gen1 is active, child of outer_span - gen1.update(output="LLM 1 output") - - with outer_span.start_as_current_span(name="intermediate-step") as mid_span: - # mid_span is active, also a child of outer_span - # This demonstrates using the yielded span object to create children - - with mid_span.start_as_current_observation(as_type="generation", name="llm-step-2") as gen2: - # gen2 is active, child of mid_span - gen2.update(output="LLM 2 output") - - mid_span.update(output="Intermediate processing done") - - outer_span.update(output="Outer process finished") -``` - - - - -If you are creating observations manually (not `_as_current_`), you can use the methods on the parent `LangfuseSpan` or `LangfuseGeneration` object to create children. These children will _not_ become the current context unless their `_as_current_` variants are used. - -```python -from langfuse import get_client - -langfuse = get_client() - -parent = langfuse.start_span(name="manual-parent") - -child_span = parent.start_span(name="manual-child-span") -# ... work ... -child_span.end() - -child_gen = parent.start_generation(name="manual-child-generation") -# ... work ... -child_gen.end() - -parent.end() -``` - - - - -### Updating Observations - -You can update observations with new information as your code executes. - -- For spans/generations created via context managers or assigned to variables: use the `.update()` method on the object. -- To update the _currently active_ observation in the context (without needing a direct reference to it): use `langfuse.update_current_span()` or `langfuse.update_current_generation()`. - -**`LangfuseSpan.update()` / `LangfuseGeneration.update()` parameters:** - -| Parameter | Type | Description | Applies To | -| :---------------------- | :------------------------------ | :-------------------------------------------------------------------- | :--------- | -| `input` | `Optional[Any]` | Input data for the operation. | Both | -| `output` | `Optional[Any]` | Output data from the operation. | Both | -| `metadata` | `Optional[Any]` | Additional metadata (JSON-serializable). | Both | -| `version` | `Optional[str]` | Version identifier for the code/component. | Both | -| `level` | `Optional[SpanLevel]` | Severity: `"DEBUG"`, `"DEFAULT"`, `"WARNING"`, `"ERROR"`. | Both | -| `status_message` | `Optional[str]` | A message describing the status, especially for errors. | Both | -| `completion_start_time` | `Optional[datetime]` | Timestamp when the LLM started generating the completion (streaming). | Generation | -| `model` | `Optional[str]` | Name/identifier of the AI model used. | Generation | -| `model_parameters` | `Optional[Dict[str, MapValue]]` | Parameters used for the model call (e.g., temperature). | Generation | -| `usage_details` | `Optional[Dict[str, int]]` | Token usage (e.g., `{"input_tokens": 10, "output_tokens": 20}`). | Generation | -| `cost_details` | `Optional[Dict[str, float]]` | Cost information (e.g., `{"total_cost": 0.0023}`). | Generation | -| `prompt` | `Optional[PromptClient]` | Associated `PromptClient` object from Langfuse prompt management. | Generation | - -```python -from langfuse import get_client - -langfuse = get_client() - -with langfuse.start_as_current_observation(as_type="generation", name="llm-call", model="gpt-5-mini") as gen: - gen.update(input={"prompt": "Why is the sky blue?"}) - # ... make LLM call ... - response_text = "Rayleigh scattering..." - gen.update( - output=response_text, - usage_details={"input_tokens": 5, "output_tokens": 50}, - metadata={"confidence": 0.9} - ) - -# Alternatively, update the current observation in context: -with langfuse.start_as_current_observation(as_type="span", name="data-processing"): - # ... some processing ... - langfuse.update_current_span(metadata={"step1_complete": True}) - # ... more processing ... - langfuse.update_current_span(output={"result": "final_data"}) -``` - -### Setting Trace Attributes - -Trace-level attributes apply to the entire trace, not just a single observation. You can set or update these using: - -- the `propagate_attributes` context manager that sets attributes on all observations inside its context and on the trace -- The `.update_trace()` method on any `LangfuseSpan` or `LangfuseGeneration` object within that trace. -- `langfuse.update_current_trace()` to update the trace associated with the currently active observation. - -**Trace attribute parameters:** - -| Parameter | Type | Description | Recommended Method | -| :----------- | :-------------------- | :--------------------------------------------------------------- | :----------------- | -| `name` | `Optional[str]` | Name for the trace. | `update_trace()` | -| `user_id` | `Optional[str]` | ID of the user associated with this trace. | `propagate_attributes()` | -| `session_id` | `Optional[str]` | Session identifier for grouping related traces. | `propagate_attributes()` | -| `version` | `Optional[str]` | Version of your application/service for this trace. | `propagate_attributes()` | -| `input` | `Optional[Any]` | Overall input for the entire trace. | `update_trace()` | -| `output` | `Optional[Any]` | Overall output for the entire trace. | `update_trace()` | -| `metadata` | `Optional[Any]` | Additional metadata for the trace. | `propagate_attributes()` | -| `tags` | `Optional[List[str]]` | List of tags to categorize the trace. | `propagate_attributes()` | -| `public` | `Optional[bool]` | Whether the trace should be publicly accessible (if configured). | `update_trace()` | - - - **Note:** For `user_id`, `session_id`, `metadata`, `version`, and `tags`, consider - using `propagate_attributes()` (see below) to ensure these attributes are - applied to **all spans**, not just the trace object. - - In the near-term future filtering and aggregating observations by these attributes requires them to be present on all observations, and `propagate_attributes` is the future-proof solution. - - -#### Propagating Attributes [#propagate-attributes] - -Certain attributes (`user_id`, `session_id`, `metadata`, `version`, `tags`) should be applied to **all spans** created within some execution scope. This is important because Langfuse aggregation queries (e.g., filtering by user_id, calculating costs by session_id) will soon operate across individual observations rather than the trace level. - -Use the `propagate_attributes()` context manager to automatically propagate these attributes to all child observations: - - - - -```python /propagate_attributes/ -from langfuse import get_client, propagate_attributes - -langfuse = get_client() - -with langfuse.start_as_current_observation(as_type="span", name="user-workflow") as span: - # Propagate attributes to all child observations - with propagate_attributes( - user_id="user_123", - session_id="session_abc", - metadata={"experiment": "variant_a", "env": "prod"}, - version="1.0" - ): - # All spans created here inherit these attributes - with langfuse.start_as_current_observation( - as_type="generation", - name="llm-call", - model="gpt-4o" - ) as gen: - # This generation automatically has user_id, session_id, metadata, version - pass -``` - - - - -```python /propagate_attributes/ -from langfuse import observe, propagate_attributes - -@observe() -def my_llm_pipeline(user_id: str, session_id: str): - # Propagate early in the trace - with propagate_attributes( - user_id=user_id, - session_id=session_id, - metadata={"pipeline": "main"} - ): - # All nested @observe functions inherit these attributes - result = call_llm() - return result - -@observe() -def call_llm(): - # This automatically has user_id, session_id, metadata from parent - pass -``` - - - - -import { PropagationRestrictionsCallout } from "@/components/PropagationRestrictionsCallout"; - - - - -##### Cross-Service Propagation - -For distributed tracing across multiple services, use the `as_baggage` parameter (see [OpenTelemetry documentation](https://opentelemetry.io/docs/concepts/signals/baggage/) for more details) to propagate attributes via HTTP headers: - -```python /propagate_attributes/ -from langfuse import get_client, propagate_attributes -import requests - -langfuse = get_client() - -# Service A - originating service -with langfuse.start_as_current_observation(as_type="span", name="api-request"): - with propagate_attributes( - user_id="user_123", - session_id="session_abc", - as_baggage=True # Propagate via HTTP headers - ): - # HTTP request to Service B - response = requests.get("https://service-b.example.com/api") - # user_id and session_id are now in HTTP headers - -# Service B will automatically extract and apply these attributes -``` - - - **Security Warning:** When `as_baggage=True`, attribute values are added to - HTTP headers on ALL outbound requests. Only enable for non-sensitive values - and when you need cross-service tracing. - - -### Trace Input/Output Behavior - -In v3, trace input and output are automatically set from the **root observation** (first span/generation) by default. This differs from v2 where integrations could set trace-level inputs/outputs directly. - -#### Default Behavior - -```python -from langfuse import get_client - -langfuse = get_client() - -with langfuse.start_as_current_observation( - as_type="span", - name="user-request", - input={"query": "What is the capital of France?"} # This becomes the trace input -) as root_span: - - with langfuse.start_as_current_observation( - as_type="generation", - name="llm-call", - model="gpt-4o", - input={"messages": [{"role": "user", "content": "What is the capital of France?"}]} - ) as gen: - response = "Paris is the capital of France." - gen.update(output=response) - # LLM generation input/output are separate from trace input/output - - root_span.update(output={"answer": "Paris"}) # This becomes the trace output -``` - -#### Override Default Behavior - -If you need different trace inputs/outputs than the root observation, explicitly set them: - -```python -from langfuse import get_client - -langfuse = get_client() - -with langfuse.start_as_current_observation(as_type="span", name="complex-pipeline") as root_span: - # Root span has its own input/output - root_span.update(input="Step 1 data", output="Step 1 result") - - # But trace should have different input/output (e.g., for LLM-as-a-judge) - root_span.update_trace( - input={"original_query": "User's actual question"}, - output={"final_answer": "Complete response", "confidence": 0.95} - ) - - # Now trace input/output are independent of root span input/output -``` - -#### Critical for LLM-as-a-Judge Features - -LLM-as-a-judge and evaluation features typically rely on trace-level inputs and outputs. Make sure to set these appropriately: - -```python -from langfuse import observe, get_client - -langfuse = get_client() - -@observe() -def process_user_query(user_question: str): - # LLM processing... - answer = call_llm(user_question) - - # Explicitly set trace input/output for evaluation features - langfuse.update_current_trace( - input={"question": user_question}, - output={"answer": answer} - ) - - return answer -``` - -### Trace and Observation IDs - -Langfuse uses W3C Trace Context compliant IDs: - -- **Trace IDs**: 32-character lowercase hexadecimal string (16 bytes). -- **Observation IDs (Span IDs)**: 16-character lowercase hexadecimal string (8 bytes). - -You can retrieve these IDs: - -- `langfuse.get_current_trace_id()`: Gets the trace ID of the currently active observation. -- `langfuse.get_current_observation_id()`: Gets the ID of the currently active observation. -- `span_obj.trace_id` and `span_obj.id`: Access IDs directly from a `LangfuseSpan` or `LangfuseGeneration` object. - -For scenarios where you need to generate IDs outside of an active trace (e.g., to link scores to traces/observations that will be created later, or to correlate with external systems), use: - -- `Langfuse.create_trace_id(seed: Optional[str] = None)`(static method): Generates a new trace ID. If a `seed` is provided, the ID is deterministic. Use the same seed to get the same ID. This is useful for correlating external IDs with Langfuse traces. - -```python -from langfuse import get_client, Langfuse - -langfuse = get_client() - -# Get current IDs -with langfuse.start_as_current_observation(as_type="span", name="my-op") as current_op: - trace_id = langfuse.get_current_trace_id() - observation_id = langfuse.get_current_observation_id() - print(f"Current Trace ID: {trace_id}, Current Observation ID: {observation_id}") - print(f"From object: Trace ID: {current_op.trace_id}, Observation ID: {current_op.id}") - -# Generate IDs deterministically -external_request_id = "req_12345" -deterministic_trace_id = Langfuse.create_trace_id(seed=external_request_id) -print(f"Deterministic Trace ID for {external_request_id}: {deterministic_trace_id}") -``` - -**Linking to Existing Traces (Trace Context)** - -If you have a `trace_id` (and optionally a `parent_span_id`) from an external source (e.g., another service, a batch job), you can link new observations to it using the `trace_context` parameter. Note that OpenTelemetry offers native cross-service context propagation, so this is not necessarily required for calls between services that are instrumented with OTEL. - -```python -from langfuse import get_client - -langfuse = get_client() - -existing_trace_id = "abcdef1234567890abcdef1234567890" # From an upstream service -existing_parent_span_id = "fedcba0987654321" # Optional parent span in that trace - -with langfuse.start_as_current_observation( - as_type="span", - name="process-downstream-task", - trace_context={ - "trace_id": existing_trace_id, - "parent_span_id": existing_parent_span_id # If None, this becomes a root span in the existing trace - } -) as span: - # This span is now part of the trace `existing_trace_id` - # and a child of `existing_parent_span_id` if provided. - print(f"This span's trace_id: {span.trace_id}") # Will be existing_trace_id - pass -``` - -### Client Management - -#### `flush()` - -Manually triggers the sending of all buffered observations (spans, generations, scores, media metadata) to the Langfuse API. This is useful in short-lived scripts or before exiting an application to ensure all data is persisted. - -```python -from langfuse import get_client - -langfuse = get_client() -# ... create traces and observations ... -langfuse.flush() # Ensures all pending data is sent -``` - -The `flush()` method blocks until the queued data is processed by the respective background threads. - -#### `shutdown()` - -Gracefully shuts down the Langfuse client. This includes: - -1. Flushing all buffered data (similar to `flush()`). -2. Waiting for background threads (for data ingestion and media uploads) to finish their current tasks and terminate. - -It's crucial to call `shutdown()` before your application exits to prevent data loss and ensure clean resource release. The SDK automatically registers an `atexit` hook to call `shutdown()` on normal program termination, but manual invocation is recommended in scenarios like: - -- Long-running daemons or services when they receive a shutdown signal. -- Applications where `atexit` might not reliably trigger (e.g., certain serverless environments or forceful terminations). - -```python -from langfuse import get_client - -langfuse = get_client() -# ... application logic ... - -# Before exiting: -langfuse.shutdown() -``` - -## Native Instrumentations - -The Langfuse Python SDK has native integrations for the OpenAI and LangChain SDK. You can also use any other OTel-based instrumentation library to automatically trace your calls in Langfuse. - -### OpenAI Integration - -Langfuse offers a drop-in replacement for the OpenAI Python SDK to automatically trace all your OpenAI API calls. Simply change your import statement: - -```diff -- import openai -+ from langfuse.openai import openai - -# Your existing OpenAI code continues to work as is -# For example: -# client = openai.OpenAI() -# completion = client.chat.completions.create(...) -``` - -**What's automatically captured:** - -- **Requests & Responses**: All prompts/completions, including support for streaming, async operations, and function/tool calls. -- **Timings**: Latencies for API calls. -- **Errors**: API errors are captured with their details. -- **Model Usage**: Token counts (input, output, total). -- **Cost**: Estimated cost in USD (based on model and token usage). -- **Media**: Input audio and output audio from speech-to-text and text-to-speech endpoints. - -The integration is fully interoperable with `@observe` and manual tracing methods (`start_as_current_span`, etc.). If an OpenAI call is made within an active Langfuse span, the OpenAI generation will be correctly nested under it. - -**Passing Langfuse arguments to OpenAI calls:** - -You can pass Langfuse-specific arguments directly to OpenAI client methods. These will be used to enrich the trace data. - -```python -from langfuse import get_client, propagate_attributes -from langfuse.openai import openai - -langfuse = get_client() - -client = openai.OpenAI() - -with langfuse.start_as_current_observation(as_type="span", name="qna-bot-openai") as span: - - with propagate_attributes( - tags=["qna-bot-openai"] - ): - # This will be traced as a Langfuse generation - response = client.chat.completions.create( - name="qna-bot-openai", # Custom name for this generation in Langfuse - metadata={"user_tier": "premium", "request_source": "web_api"}, # will be added to the Langfuse generation - model="gpt-4o", - messages=[{"role": "user", "content": "What is OpenTelemetry?"}], - ) -``` - -**Setting trace attributes via metadata:** - -You can set trace attributes (`session_id`, `user_id`, `tags`) directly on OpenAI calls using special fields in the `metadata` parameter: - -```python -from langfuse.openai import openai - -client = openai.OpenAI() - -response = client.chat.completions.create( - model="gpt-4o", - messages=[{"role": "user", "content": "Hello"}], - metadata={ - "langfuse_session_id": "session_123", - "langfuse_user_id": "user_456", - "langfuse_tags": ["production", "chat-bot"], - "custom_field": "additional metadata" # Regular metadata fields work too - } -) -``` - -The special metadata fields are: - -- `langfuse_session_id`: Sets the session ID for the trace -- `langfuse_user_id`: Sets the user ID for the trace -- `langfuse_tags`: Sets tags for the trace (should be a list of strings) - -Supported Langfuse arguments: `name`, `metadata`, `langfuse_prompt` - -Learn more in the [OpenAI integration](/integrations/model-providers/openai-py) documentation. - -### Langchain Integration - -Langfuse provides a callback handler for Langchain to trace its operations. - -**Setup:** - -Initialize the `CallbackHandler` and add it to your Langchain calls, either globally or per-call. - -```python -from langfuse import get_client, propagate_attributes -from langfuse.langchain import CallbackHandler -from langchain_openai import ChatOpenAI # Example LLM -from langchain_core.prompts import ChatPromptTemplate - -langfuse = get_client() - -# Initialize the Langfuse handler -langfuse_handler = CallbackHandler() - -# Example: Using it with an LLM call -llm = ChatOpenAI(model_name="gpt-4o") -prompt = ChatPromptTemplate.from_template("Tell me a joke about {topic}") -chain = prompt | llm - -with langfuse.start_as_current_observation(as_type="span", name="joke-chain") as span: - - with propagate_attributes( - tags=["joke-chain"] - ): - response = chain.invoke({"topic": "cats"}, config={"callbacks": [langfuse_handler]}) - print(response) -``` - -**Setting trace attributes via metadata:** - -You can set trace attributes (`session_id`, `user_id`, `tags`) directly during chain invocation using special fields in the `metadata` configuration: - -```python -from langfuse.langchain import CallbackHandler -from langchain_openai import ChatOpenAI -from langchain_core.prompts import ChatPromptTemplate - -# Initialize the Langfuse handler -langfuse_handler = CallbackHandler() - -# Create your LangChain components -llm = ChatOpenAI(model_name="gpt-4o") -prompt = ChatPromptTemplate.from_template("Tell me a joke about {topic}") -chain = prompt | llm - -# Set trace attributes via metadata in chain invocation -response = chain.invoke( - {"topic": "cats"}, - config={ - "callbacks": [langfuse_handler], - "metadata": { - "langfuse_session_id": "session_123", - "langfuse_user_id": "user_456", - "langfuse_tags": ["production", "humor-bot"], - "custom_field": "additional metadata" # Regular metadata fields work too - } - } -) -``` - -The special metadata fields are: - -- `langfuse_session_id`: Sets the session ID for the trace -- `langfuse_user_id`: Sets the user ID for the trace -- `langfuse_tags`: Sets tags for the trace (should be a list of strings) - -You can also pass `update_trace=True` to the CallbackHandler init to force a trace update with the chains input, output and metadata. - -**What's captured:** - -The callback handler maps various Langchain events to Langfuse observations: - -- **Chains (`on_chain_start`, `on_chain_end`, `on_chain_error`):** Traced as spans. -- **LLMs (`on_llm_start`, `on_llm_end`, `on_llm_error`, `on_chat_model_start`):** Traced as generations, capturing model name, prompts, responses, and usage if available from the LLM provider. -- **Tools (`on_tool_start`, `on_tool_end`, `on_tool_error`):** Traced as spans, capturing tool input and output. -- **Retrievers (`on_retriever_start`, `on_retriever_end`, `on_retriever_error`):** Traced as spans, capturing the query and retrieved documents. -- **Agents (`on_agent_action`, `on_agent_finish`):** Agent actions and final finishes are captured within their parent chain/agent span. - -Langfuse attempts to parse model names, usage, and other relevant details from the information provided by Langchain. The `metadata` argument in Langchain calls can be used to pass additional information to Langfuse, including `langfuse_prompt` to link with managed prompts. - -Learn more in the [Langchain integration](/integrations/frameworks/langchain) documentation. - -### Third-party integrations - -The Langfuse SDK seamlessly integrates with any third-party library that uses OpenTelemetry instrumentation. When these libraries emit spans, they are automatically captured and properly nested within your trace hierarchy. This enables unified tracing across your entire application stack without requiring any additional configuration. - - - - - - } - title="Vercel AI SDK" - href="/integrations/frameworks/vercel-ai-sdk" - arrow - /> - - - - } - title="Llamaindex" - href="/integrations/frameworks/llamaindex" - arrow - /> - - - - } - title="CrewAI" - href="/integrations/frameworks/crewai" - arrow - /> - - - - } - title="Ollama" - href="/integrations/model-providers/ollama" - arrow - /> - - - - } - title="LiteLLM" - href="/integrations/gateways/litellm" - arrow - /> - - - - } - title="AutoGen" - href="/integrations/frameworks/autogen" - arrow - /> - - - - } - title="Google ADK" - href="/integrations/frameworks/google-adk" - arrow - /> - - - -For example, if you're using OpenTelemetry-instrumented databases, HTTP clients, or other services alongside your LLM operations, all these spans will be correctly organized within your traces in Langfuse. - - - - -You can use any third-party, OTEL-based instrumentation library for Anthropic to automatically trace all your Anthropic API calls in Langfuse. - -In this example, we are using the [`opentelemetry-instrumentation-anthropic` library](https://pypi.org/project/opentelemetry-instrumentation-anthropic/). - -```python -from anthropic import Anthropic -from opentelemetry.instrumentation.anthropic import AnthropicInstrumentor - -from langfuse import get_client - -# This will automatically emit OTEL-spans for all Anthropic API calls -AnthropicInstrumentor().instrument() - -langfuse = get_client() -anthropic_client = Anthropic() - -with langfuse.start_as_current_observation(as_type="span", name="myspan"): - # This will be traced as a Langfuse generation nested under the current span - message = anthropic_client.messages.create( - model="claude-3-7-sonnet-20250219", - max_tokens=1024, - messages=[{"role": "user", "content": "Hello, Claude"}], - ) - - print(message.content) - -# Flush events to Langfuse in short-lived applications -langfuse.flush() -``` - -Learn more in the [Anthropic integration](/integrations/model-providers/anthropic) documentation. - - - - -You can use the third-party, OTEL-based instrumentation library for LlamaIndex to automatically trace your LlamaIndex calls in Langfuse. - -In this example, we are using the [`openinference-instrumentation-llama-index` library](https://pypi.org/project/openinference-instrumentation-llama-index/). - -```python -from llama_index.core.llms.openai import OpenAI -from openinference.instrumentation.llama_index import LlamaIndexInstrumentor - -from langfuse import get_client - -LlamaIndexInstrumentor().instrument() - -langfuse = get_client() -llm = OpenAI(model="gpt-4o") - -with langfuse.start_as_current_observation(as_type="span", name="myspan"): - response = llm.complete("Hello, world!") - print(response) - -langfuse.flush() -``` - -Learn more in the [Llamaindex integration](/integrations/frameworks/llamaindex) documentation. - - - - diff --git a/pages/docs/observability/sdk/python/overview.mdx b/pages/docs/observability/sdk/python/overview.mdx deleted file mode 100644 index 9d174ac67..000000000 --- a/pages/docs/observability/sdk/python/overview.mdx +++ /dev/null @@ -1,127 +0,0 @@ ---- -title: Overview of the Python SDK -description: OTel-based Python SDK (v3) with better developer experience and third-party integrations. -category: SDKs ---- - -# Overview - - - -If you are self-hosting Langfuse, the Python SDK v3 requires [**Langfuse platform version >= 3.63.0**](https://github.com/langfuse/langfuse/releases/tag/v3.63.0) for traces to be correctly processed. - -Documentation for the legacy Python SDK v2 can be found [here](https://python-sdk-v2.docs-snapshot.langfuse.com/docs/observability/sdk/python/decorators). - - - -Our [OpenTelemetry-based](https://opentelemetry.io/) Python SDK (v3) is the latest generation of the SDK designed for a improved developer experience and enhanced ease of use. Built on the OpenTelemetry Python SDK, it offers a more intuitive API for comprehensive tracing of your LLM application. - -**Key benefits:** - -- **Improved Developer Experience**: A more intuitive API means less code to write for tracing your application, simplifying the integration process. -- **Unified Context Sharing**: Seamlessly hook into the tracing context of the current span to update it or create child spans. This is particularly beneficial for integrating with other instrumented libraries. -- **Broad Third-Party Integrations**: Any library instrumented with OpenTelemetry will work out-of-the-box with the Langfuse SDK. Spans from these libraries are automatically captured and correctly nested within your Langfuse traces. - -## Quickstart - -There are three main ways of instrumenting your application with the new Langfuse SDK. All of them are fully interoperable with each other. - -import GetStartedPythonSdk from "@/components-mdx/get-started/python-sdk.mdx"; - - - -## Learn more - -import { Rocket, Plug, Beaker, Settings, LifeBuoy, BookOpen } from "lucide-react"; - - - } - title="Setup" - href="/docs/observability/sdk/python/setup" - arrow - /> - } - title="Instrumentation" - href="/docs/observability/sdk/python/instrumentation" - arrow - /> - } - title="Evaluation" - href="/docs/observability/sdk/python/evaluation" - arrow - /> - } - title="Advanced usage" - href="/docs/observability/sdk/python/advanced-usage" - arrow - /> - } - title="Troubleshooting & FAQ" - href="/docs/observability/sdk/python/troubleshooting-and-faq" - arrow - /> - } - title="Reference" - href="https://python.reference.langfuse.com" - newWindow - arrow - /> - - -## OTel and Langfuse - -The Python SDK v3 is built upon [OpenTelemetry (OTel)](https://opentelemetry.io/), a standard for observability. Understanding the relation between OTel and Langfuse is not required to use the SDK, but it is helpful to have a basic understanding of the concepts. OTel related concepts are abstracted away and you can use the SDK without being deeply familiar with them. - -```mermaid -graph TD - subgraph OTEL_Core_Concepts ["OpenTelemetry"] - direction LR - OTEL_Trace["OTel Trace"] - Root_OTEL_Span["Root OTel Span"] - Child_OTEL_Span["Child OTel Span"] - - OTEL_Trace -- is defined by --> Root_OTEL_Span - Root_OTEL_Span -- Hierarchy via
Context Propagation --> Child_OTEL_Span - end - - subgraph Langfuse_Mapping ["Langfuse"] - direction LR - LF_Trace["Langfuse Trace"] - LF_Observation["Langfuse Observation
(typed as either Span, Generation or Event)"] - - LF_Trace -- Collects one or more --> LF_Observation - end - - OTEL_Trace -.->|shares ID with | LF_Trace - - Root_OTEL_Span -.->|Mapped to| LF_Observation - Child_OTEL_Span -.->|Mapped to| LF_Observation - - Root_OTEL_Span -.->|sets default input and output | LF_Trace - Root_OTEL_Span -.->|can hold trace attributes| LF_Trace - Child_OTEL_Span -.->|can hold trace attributes| LF_Trace - - classDef otel fill:#D6EAF8,stroke:#3498DB,stroke-width:2px,color:#000; - classDef langfuse fill:#D5F5E3,stroke:#2ECC71,stroke-width:2px,color:#000; - class OTEL_Trace,Root_OTEL_Span,Child_OTEL_Span otel; - class LF_Trace,LF_Observation langfuse; -``` - -- **OTel Trace**: An OTel-trace represents the entire lifecycle of a request or transaction as it moves through your application and its services. A trace is typically a sequence of operations, like an LLM generating a response followed by a parsing step. The root (first) span created in a sequence defines the OTel trace. OTel traces do not have a start and end time, they are defined by the root span. -- **OTel Span**: A span represents a single unit of work or operation within a trace. Spans have a start and end time, a name, and can have attributes (key-value pairs of metadata). Spans can be nested to create a hierarchy, showing parent-child relationships between operations. -- **Langfuse Trace**: A Langfuse trace collects observations and holds trace attributes such as `session_id`, `user_id` as well as overall input and outputs. It shares the same ID as the OTel trace and its attributes are set via specific OTel span attributes that are automatically propagated to the Langfuse trace. -- **Langfuse Observation**: In Langfuse terminology, an "observation" is a Langfuse-specific representation of an OTel span. It can be a generic span (Langfuse-span), a specialized "generation" (Langfuse-generation), a point-in-time event (Langfuse-event), or [other observation types](/docs/observability/features/observation-types). - - **Langfuse Span**: A Langfuse-span is a generic OTel span in Langfuse, designed for non-LLM operations. - - **Langfuse Generation**: A Langfuse-generation is a specialized type of OTel span in Langfuse, designed specifically for Large Language Model (LLM) calls. It includes additional fields like `model`, `model_parameters`, `usage_details` (tokens), and `cost_details`. - - **Langfuse Event**: A Langfuse-event tracks a point in time action. -- **Context Propagation**: OpenTelemetry automatically handles the propagation of the current trace and span context. This means when you call another function (whether it's also traced by Langfuse, an OTel-instrumented library, or a manually created span), the new span will automatically become a child of the currently active span, forming a correct trace hierarchy. -- **Attribute Propagation**: Certain trace attributes (`user_id`, `session_id`, `metadata`, `version`, `tags`) can be automatically propagated to all child observations using `propagate_attributes()`. This ensures consistent attribute coverage across all observations in a trace. See the [instrumentation docs](/docs/observability/sdk/python/instrumentation#propagating-trace-attributes) for details. - -The Langfuse SDK provides wrappers around OTel spans (`LangfuseSpan`, `LangfuseGeneration`) that offer convenient methods for interacting with Langfuse-specific features like scoring and media handling, while still being native OTel spans under the hood. You can also use these wrapper objects to add Langfuse trace attributes via `update_trace()` or use `propagate_attributes()` for automatic propagation to all child observationss. - diff --git a/pages/docs/observability/sdk/python/setup.mdx b/pages/docs/observability/sdk/python/setup.mdx deleted file mode 100644 index 5963002c4..000000000 --- a/pages/docs/observability/sdk/python/setup.mdx +++ /dev/null @@ -1,117 +0,0 @@ ---- -title: Setup of the Langfuse Python SDK -description: Setup the Langfuse Python SDK for tracing your application and ingesting data into Langfuse. -category: SDKs ---- - -# Setup - -To get started with the Langfuse Python SDK, you need to install the SDK and initialize the client. - - -## Installation - -To install the Langfuse Python SDK, run: - -```bash -pip install langfuse -``` - -## Initialize Client - -Begin by initializing the `Langfuse` client. You must provide your Langfuse public and secret keys. These can be passed as constructor arguments or set as environment variables (recommended). - -If you are self-hosting Langfuse or using a data region other than the default (EU, `https://cloud.langfuse.com`), ensure you configure the `host` argument or the `LANGFUSE_BASE_URL` environment variable (recommended). - - - -```bash filename=".env" -LANGFUSE_PUBLIC_KEY="pk-lf-..." -LANGFUSE_SECRET_KEY="sk-lf-..." -LANGFUSE_BASE_URL="https://cloud.langfuse.com" # US region: https://us.cloud.langfuse.com -``` - - - - - -```python filename="Initialize client" -from langfuse import Langfuse - -# Initialize with constructor arguments - -langfuse = Langfuse( - public_key="YOUR_PUBLIC_KEY", - secret_key="YOUR_SECRET_KEY", - base_url="https://cloud.langfuse.com" # US region: https://us.cloud.langfuse.com -) -``` - - - -If you are reinstantiating Langfuse client with different constructor arguments but the same `public_key`, the client will reuse the same instance and ignore the new arguments. - - - - - - - -
- -Verify connection with `langfuse.auth_check()` - - -You can also verify your connection to the Langfuse server using `langfuse.auth_check()`. We do not recommend using this in production as this adds latency to your application. - -```python -from langfuse import get_client - -langfuse = get_client() - -# Verify connection, do not use in production as this is a synchronous call -if langfuse.auth_check(): - print("Langfuse client is authenticated and ready!") -else: - print("Authentication failed. Please check your credentials and host.") -``` - -
- -Key configuration options: - -| Constructor Argument | Environment Variable | Description | Default value | -| --------------------------- | ------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------ | -| `public_key` | `LANGFUSE_PUBLIC_KEY` | Your Langfuse project's public API key. **Required.** | | -| `secret_key` | `LANGFUSE_SECRET_KEY` | Your Langfuse project's secret API key. **Required.** | | -| `base_url` | `LANGFUSE_BASE_URL` | The API host for your Langfuse instance. | `"https://cloud.langfuse.com"` | -| `timeout` | `LANGFUSE_TIMEOUT` | Timeout in seconds for API requests. | `5` | -| `httpx_client` | - | Custom `httpx.Client` for making non-tracing HTTP requests. | | -| `debug` | `LANGFUSE_DEBUG` | Enables debug mode for more verbose logging. Set to `True` or `"True"`. | `False` | -| `tracing_enabled` | `LANGFUSE_TRACING_ENABLED` | Enables or disables the Langfuse client. If `False`, all observability calls become no-ops. | `True` | -| `flush_at` | `LANGFUSE_FLUSH_AT` | Number of spans to batch before sending to the API. | `512` | -| `flush_interval` | `LANGFUSE_FLUSH_INTERVAL` | Time in seconds between batch flushes. | `5` | -| `environment` | `LANGFUSE_TRACING_ENVIRONMENT` | Environment name for tracing (e.g., "development", "staging", "production"). Must be lowercase alphanumeric with hyphens/underscores. | `"default"` | -| `release` | `LANGFUSE_RELEASE` | [Release](/docs/tracing-features/releases-and-versioning) version/hash of your application. Used for grouping analytics. | | -| `media_upload_thread_count` | `LANGFUSE_MEDIA_UPLOAD_THREAD_COUNT` | Number of background threads for handling media uploads. | `1` | -| `sample_rate` | `LANGFUSE_SAMPLE_RATE` | [Sampling](/docs/tracing-features/sampling) rate for traces (float between 0.0 and 1.0). `1.0` means 100% of traces are sampled. | `1.0` | -| `mask` | - | A function `(data: Any) -> Any` to [mask](/docs/tracing-features/masking) sensitive data in traces before sending to the API. | | -| | `LANGFUSE_MEDIA_UPLOAD_ENABLED` | Whether to upload media files to Langfuse S3. In self-hosted environments this might be useful to disable. | `True` | - -
- -## Accessing the Client Globally - -The Langfuse client is a singleton. It can be accessed anywhere in your application using the `get_client` function. - -Optionally, you can initialize the client via `Langfuse()` to pass in configuration options (see above). Otherwise, it is created automatically when you call `get_client()` based on environment variables. - -```python -from langfuse import get_client - -# Optionally, initialize the client with configuration options -# langfuse = Langfuse(public_key="pk-lf-...", secret_key="sk-lf-...") - -# Get the default client -client = get_client() -``` \ No newline at end of file diff --git a/pages/docs/observability/sdk/python/troubleshooting-and-faq.mdx b/pages/docs/observability/sdk/python/troubleshooting-and-faq.mdx deleted file mode 100644 index 6d53a9474..000000000 --- a/pages/docs/observability/sdk/python/troubleshooting-and-faq.mdx +++ /dev/null @@ -1,37 +0,0 @@ ---- -title: Troubleshooting and FAQ of the Langfuse Python SDK -description: Troubleshooting and FAQ of the Langfuse Python SDK to resolve common issues. -category: SDKs ---- - -# Troubleshooting and FAQ - -This page addresses frequently asked questions and common troubleshooting topics for the Langfuse Python SDK. - -If you don't find a solution to your issue here, try using [Ask AI](/docs/ask-ai) for instant answers. For bug reports, please open a ticket on [GitHub Issues](/issues). For general questions or support, visit our [support page](/support). - -- **Authentication Issues**: - - Ensure `LANGFUSE_PUBLIC_KEY`, `LANGFUSE_SECRET_KEY`, and `LANGFUSE_BASE_URL` (if not using default cloud) are correctly set either as environment variables or in the `Langfuse()` constructor. - - Use `langfuse.auth_check()` after initialization to verify credentials. Do not use this in production as this method waits for a response from the server. -- **No Traces Appearing**: - - Check if `tracing_enabled` is `True` (default). - - Verify `sample_rate` is not `0.0`. - - Ensure `langfuse.shutdown()` is called or the program exits cleanly to allow `atexit` hooks to flush data. Manually call `langfuse.flush()` to force data sending. - - Enable debug logging (`debug=True` or `LANGFUSE_DEBUG="True"`) to see SDK activity and potential errors during exporting. -- **Incorrect Nesting or Missing Spans**: - - If you are self-hosting Langfuse, the Python SDK v3 requires [**Langfuse platform version >= 3.63.0**](https://github.com/langfuse/langfuse/releases/tag/v3.63.0) for traces to be correctly processed. You can find a snapshot of the legacy v2 SDK documentation [here](https://python-sdk-v2.docs-snapshot.langfuse.com/docs/observability/sdk/python/decorators). - - Ensure you are using context managers (`with langfuse.start_as_current_observation(...)`) for proper context propagation. - - If manually creating spans (`langfuse.start_span()`), ensure they are correctly ended with `.end()`. - - In async code, ensure context is not lost across `await` boundaries if not using Langfuse's async-compatible methods. -- **Langchain/OpenAI Integration Not Working**: - - Confirm the respective integration (e.g., `from langfuse.openai import openai` or `LangfuseCallbackHandler`) is correctly set up _before_ the calls to the LLM libraries are made. - - Check for version compatibility issues between Langfuse, Langchain, and OpenAI SDKs. -- **Media Not Appearing**: - - Ensure `LangfuseMedia` objects are correctly initialized and passed in `input`, `output`, or `metadata`. - - Check debug logs for any media upload errors. Media uploads happen in background threads. - -If you encounter persistent issues, please: - -1. Enable debug logging to gather more information. -2. Check the Langfuse status page (if applicable for cloud users). -3. Raise an issue on our [GitHub repository](/issues) with details about your setup, SDK version, code snippets, and debug logs. diff --git a/pages/docs/observability/sdk/troubleshooting-and-faq.mdx b/pages/docs/observability/sdk/troubleshooting-and-faq.mdx new file mode 100644 index 000000000..3e2508868 --- /dev/null +++ b/pages/docs/observability/sdk/troubleshooting-and-faq.mdx @@ -0,0 +1,49 @@ +--- +title: Langfuse SDK troubleshooting & FAQ +description: Resolve common Python and JS/TS SDK issues with side-by-side guidance. +category: SDKs +--- + +# Troubleshooting & FAQ + +If you cannot find your issue below, try [Ask AI](/docs/ask-ai), open a [GitHub issue](/issues), or contact [support](/support). + + + +## Authentication issues + +- Ensure `LANGFUSE_PUBLIC_KEY`, `LANGFUSE_SECRET_KEY`, and (if self-hosting) `LANGFUSE_BASE_URL` are set as environment variables or passed to `Langfuse()`. +- Use `langfuse.auth_check()` during setup (not in production) to confirm connectivity. + +## No traces appearing + +- Confirm `tracing_enabled` is `True` and `sample_rate` is not `0.0`. +- Call `langfuse.shutdown()` (or `langfuse.flush()` in short-lived jobs) so queued data is exported. +- Enable debug logging (`debug=True` or `LANGFUSE_DEBUG="True"`) to inspect exporter output. + +## Incorrect nesting or missing spans + +- Self-hosted users need Langfuse platform **>= 3.63.0** for the v3 SDK. +- Prefer context managers (`with langfuse.start_as_current_observation(...)`) to maintain OTEL context. +- If using manual spans (`langfuse.start_span()`), always call `.end()`. +- In async code, rely on Langfuse helpers to avoid losing context across `await` boundaries. + +## LangChain/OpenAI integration issues + +- Ensure Langfuse wrappers (`from langfuse.openai import openai` or `LangfuseCallbackHandler`) are instantiated before API calls. +- Check version compatibility between Langfuse, LangChain, and the model SDKs. + +## Media not appearing + +- Use `LangfuseMedia` objects for audio/image payloads and inspect debug logs to surface upload errors (uploads run on background threads). + + +## Missing traces in serverless environments + +Use the serverless export pattern from the [advanced features guide](/docs/observability/sdk/advanced-features#environment-specific-considerations): export the `LangfuseSpanProcessor`, call `forceFlush()` before the runtime freezes, or schedule a flush with `after` on Vercel Functions. + +## Missing traces with `@vercel/otel` + +Use the manual OpenTelemetry setup via `NodeSDK` and register the `LangfuseSpanProcessor`. The `@vercel/otel` helper does not yet support the OpenTelemetry JS SDK v2 that Langfuse depends on. See the [TypeScript instrumentation docs](/docs/observability/sdk/instrumentation#framework-third-party-telemetry) for a full example. + + diff --git a/pages/docs/observability/sdk/typescript/_meta.tsx b/pages/docs/observability/sdk/typescript/_meta.tsx deleted file mode 100644 index c011ea22d..000000000 --- a/pages/docs/observability/sdk/typescript/_meta.tsx +++ /dev/null @@ -1,13 +0,0 @@ -export default { - overview: "Overview", - setup: "Setup", - instrumentation: "Instrumentation", - "advanced-usage": "Advanced usage", - "upgrade-path": "Upgrade Path (v3 to v4)", - "troubleshooting-and-faq": "Troubleshooting & FAQ", - reference: { - title: "Reference ↗", - href: "https://langfuse-js-git-main-langfuse.vercel.app/", - newWindow: true, - }, -}; diff --git a/pages/docs/observability/sdk/typescript/advanced-usage.mdx b/pages/docs/observability/sdk/typescript/advanced-usage.mdx deleted file mode 100644 index 6473a0a3e..000000000 --- a/pages/docs/observability/sdk/typescript/advanced-usage.mdx +++ /dev/null @@ -1,501 +0,0 @@ ---- -title: TypeScript SDK - Advanced Configuration -description: Advanced configuration options for the TypeScript SDK. -label: "Version: JS SDK v4" ---- - -# TypeScript SDK - Advanced Configuration - -## Masking - -To prevent sensitive data from being sent to Langfuse, you can provide a `mask` function to the `LangfuseSpanProcessor`. This function will be applied to the `input`, `output`, and `metadata` of every observation. - -The function receives an object `{ data }`, where `data` is the stringified JSON of the attribute's value. It should return the masked data. - -```ts filename="instrumentation.ts" /mask:/ -import { NodeSDK } from "@opentelemetry/sdk-node"; -import { LangfuseSpanProcessor } from "@langfuse/otel"; - -const spanProcessor = new LangfuseSpanProcessor({ - mask: ({ data }) => { - // A simple regex to mask credit card numbers - const maskedData = data.replace( - /\b\d{4}[- ]?\d{4}[- ]?\d{4}[- ]?\d{4}\b/g, - "***MASKED_CREDIT_CARD***" - ); - return maskedData; - }, -}); - -const sdk = new NodeSDK({ - spanProcessors: [spanProcessor], -}); - -sdk.start(); -``` - -## Filtering Spans - -You can provide a predicate function `shouldExportSpan` to the `LangfuseSpanProcessor` to decide on a per-span basis whether it should be exported to Langfuse. - - - Filtering spans may break the parent-child relationships in your traces. For - example, if you filter out a parent span but keep its children, you may see - "orphaned" observations in the Langfuse UI. Consider the impact on trace - structure when configuring `shouldExportSpan`. - - -```ts filename="instrumentation.ts" /shouldExportSpan/ -import { NodeSDK } from "@opentelemetry/sdk-node"; -import { LangfuseSpanProcessor, ShouldExportSpan } from "@langfuse/otel"; - -// Example: Filter out all spans from the 'express' instrumentation -const shouldExportSpan: ShouldExportSpan = ({ otelSpan }) => - otelSpan.instrumentationScope.name !== "express"; - -const sdk = new NodeSDK({ - spanProcessors: [new LangfuseSpanProcessor({ shouldExportSpan })], -}); - -sdk.start(); -``` - -If you want to include only LLM observability related spans, you can configure an allowlist like so: - -```ts filename="instrumentation.ts" -import { ShouldExportSpan } from "@langfuse/otel"; - -const shouldExportSpan: ShouldExportSpan = ({ otelSpan }) => - ["langfuse-sdk", "ai"].includes(otelSpan.instrumentationScope.name); -``` - - - If you would like to exclude Langfuse spans from being sent to third-party - observability backends configured in your OpenTelemetry setup, see [the - documentation on isolating the Langfuse tracer - provider](#isolated-tracer-provider). - - -## Sampling - -Langfuse respects OpenTelemetry's sampling decisions. You can configure a sampler in your OTEL SDK to control which traces are sent to Langfuse. This is useful for managing costs and reducing noise in high-volume applications. - -Here is an example of how to configure a `TraceIdRatioBasedSampler` to send only 20% of traces: - -```ts filename="instrumentation.ts" /new TraceIdRatioBasedSampler(0.2)/ -import { NodeSDK } from "@opentelemetry/sdk-node"; -import { LangfuseSpanProcessor } from "@langfuse/otel"; -import { TraceIdRatioBasedSampler } from "@opentelemetry/sdk-trace-base"; - -const sdk = new NodeSDK({ - // Sample 20% of all traces - sampler: new TraceIdRatioBasedSampler(0.2), - spanProcessors: [new LangfuseSpanProcessor()], -}); - -sdk.start(); -``` - -For more advanced sampling strategies, refer to the [OpenTelemetry JS Sampling Documentation](https://opentelemetry.io/docs/languages/js/sampling/). - -## Managing trace and observation IDs - -In Langfuse, every trace and observation has a unique identifier. Understanding their format and how to set them is useful for integrating with other systems. - -- **Trace IDs** are 32-character lowercase hexadecimal strings, representing 16 bytes of data -- **Observation IDs** (also known as Span IDs in OpenTelemetry) are 16-character lowercase hexadecimal strings, representing 8 bytes - -While the SDK handles ID generation automatically, you may manually set them to align with external systems or create specific trace structures. This is done using the `parentSpanContext` option in tracing methods. - -When starting a new trace by setting a `traceId`, you must also provide an arbitrary parent-`spanId` for the parent observation. The parent span ID value is irrelevant as long as it is a valid 16-hexchar string as the span does not actually exist but is only used for trace ID inheritance of the created observation. - -You can create valid, deterministic trace IDs from a seed string using `createTraceId`. This is useful for correlating Langfuse traces with IDs from external systems, like a support ticket ID. - -```typescript -import { createTraceId, startObservation } from "@langfuse/tracing"; - -const externalId = "support-ticket-54321"; - -// Generate a valid, deterministic traceId from the external ID -const langfuseTraceId = await createTraceId(externalId); - -// You can now start a new trace with this ID -const rootSpan = startObservation( - "process-ticket", - {}, - { - parentSpanContext: { - traceId: langfuseTraceId, - spanId: "0123456789abcdef", // A valid 16 hexchar string; value is irrelevant as parent span does not exist but only used for inheritance - traceFlags: 1, // mark trace as sampled - }, - } -); - -// Later, you can regenerate the same traceId to score or retrieve the trace -const scoringTraceId = await createTraceId(externalId); -// scoringTraceId will be the same as langfuseTraceId -``` - -You may also access the current active trace ID via the `getActiveTraceId` function: - -```typescript -import { startObservation, getActiveTraceId } from "@langfuse/tracing"; - -await startObservation("run", async (span) => { - const traceId = getActiveTraceId(); - console.log(`Current trace ID: ${traceId}`); -}); -``` - -## Logging & Debugging - -You can configure the global SDK logger to control the verbosity of log output. This is useful for debugging. - -**In code:** - -```typescript /configureGlobalLogger/ -import { configureGlobalLogger, LogLevel } from "@langfuse/core"; - -// Set the log level to DEBUG to see all log messages -configureGlobalLogger({ level: LogLevel.DEBUG }); -``` - -Available log levels are `DEBUG`, `INFO`, `WARN`, and `ERROR`. - -**Via environment variable:** - -You can also set the log level using the `LANGFUSE_LOG_LEVEL` environment variable to enable the debug mode. - -```bash -export LANGFUSE_LOG_LEVEL="DEBUG" -``` - -## Serverless environments - -In short-lived environments such as serverless functions (e.g., Vercel Functions, AWS Lambda), you must explicitly flush the traces before the process exits or the runtime environment is frozen. - - - -{/* Generic serverless */} -Export the processor from your OTEL SDK setup file in order to flush it later. - -```ts filename="instrumentation.ts" /langfuseSpanProcessor/ /forceFlush/ /exportMode: "immediate"/ -import { NodeSDK } from "@opentelemetry/sdk-node"; -import { LangfuseSpanProcessor } from "@langfuse/otel"; - -// Export the processor to be able to flush it -export const langfuseSpanProcessor = new LangfuseSpanProcessor({ - exportMode: "immediate" // optional: configure immediate span export in serverless environments -}); - -const sdk = new NodeSDK({ - spanProcessors: [langfuseSpanProcessor], -}); - -sdk.start(); -``` - -In your serverless function handler, call `forceFlush()` on the span processor before the function exits. - -```ts filename="handler.ts" /forceFlush/ -import { langfuseSpanProcessor } from "./instrumentation"; - -export async function handler(event, context) { - // ... your application logic ... - - // Flush before exiting - await langfuseSpanProcessor.forceFlush(); -} -``` - - - - -{/* Vercel Cloud Functions */} - -Export the processor from your `instrumentation.ts` file in order to flush it later. - -```ts filename="instrumentation.ts" /langfuseSpanProcessor/ /forceFlush/ /exportMode: "immediate"/ -import { NodeSDK } from "@opentelemetry/sdk-node"; -import { LangfuseSpanProcessor } from "@langfuse/otel"; - -// Export the processor to be able to flush it -export const langfuseSpanProcessor = new LangfuseSpanProcessor(); - -const sdk = new NodeSDK({ - spanProcessors: [langfuseSpanProcessor], -}); - -sdk.start(); -``` - -In Vercel Cloud Functions, please use the `after` utility to schedule a flush after the request has completed. - -```ts filename="route.ts" /after/ /forceFlush/ -import { after } from "next/server"; - -import { langfuseSpanProcessor } from "./instrumentation.ts"; - -export async function POST() { - // ... existing request logic ... - - // Schedule flush after request has completed - after(async () => { - await langfuseSpanProcessor.forceFlush(); - }); - - // ... send response ... -} -``` - - - - -## Isolated tracer provider - -The Langfuse JS SDK uses the global OpenTelemetry TracerProvider to attach its span processor and create tracers that emit spans. This means that if you have an existing OpenTelemetry setup with another destination configured for your spans (e.g., Datadog), you will see Langfuse spans in those third-party observability backends as well. - -If you'd like to avoid sending Langfuse spans to third-party observability backends in your existing OpenTelemetry setup, you will need to use an isolated OpenTelemetry TracerProvider that is separate from the global one. - - - If you would like to simply limit the spans that are sent to Langfuse and you - have no third-party observability backend where you'd like to exclude Langfuse - spans from, [see filtering spans instead](#filtering-spans). - - - - Using an isolated TracerProvider may break the parent-child relationships in - your traces, as all TracerProviders still share the same active span context. - For example, if you have an active parent span from the global TracerProvider - but children from an isolated TracerProvider, you may see "orphaned" - observations in the Langfuse UI. Consider the impact on trace structure when - configuring an isolated tracer provider. - - -```typescript /setLangfuseTracerProvider/ /LangfuseSpanProcessor/ /langfuseTracerProvider/ -import { NodeTracerProvider } from "@opentelemetry/sdk-trace-node"; -import { setLangfuseTracerProvider } from "@langfuse/tracing"; - -// Create a new TracerProvider and register the LangfuseSpanProcessor -// do not set this TracerProvider as the global TracerProvider -const langfuseTracerProvider = new NodeTracerProvider( - spanProcessors: [new LangfuseSpanProcessor()], -) - -// Register the isolated TracerProvider -setLangfuseTracerProvider(langfuseTracerProvider) -``` - -## Multi-project Setup - -You can configure the SDK to send traces to multiple Langfuse projects. This is useful for multi-tenant applications or for sending traces to different environments. Simply register multiple `LangfuseSpanProcessor` instances, each with its own credentials. - -```ts filename="instrumentation.ts" -import { NodeSDK } from "@opentelemetry/sdk-node"; -import { LangfuseSpanProcessor } from "@langfuse/otel"; - -const sdk = new NodeSDK({ - spanProcessors: [ - new LangfuseSpanProcessor({ - publicKey: "pk-lf-public-key-project-1", - secretKey: "sk-lf-secret-key-project-1", - }), - new LangfuseSpanProcessor({ - publicKey: "pk-lf-public-key-project-2", - secretKey: "sk-lf-secret-key-project-2", - }), - ], -}); - -sdk.start(); -``` - -This configuration will send every trace to both projects. You can also configure a custom `shouldExportSpan` filter for each processor to control which traces go to which project. - -## Custom scores from browser - - - Sending custom scores directly from the browser is not yet supported in the TypeScript SDK v4. The docs below describe the still valid approach with the SDK v3. - - -The TypeScript SDK can be used to report custom scores client-side directly from the browser. It is commonly used to ingest scores into Langfuse which are based on implicit user interactions and feedback. - -**Example** - - - - -```ts -import { LangfuseWeb } from "langfuse"; - -export function UserFeedbackComponent(props: { traceId: string }) { - const langfuseWeb = new LangfuseWeb({ - publicKey: env.NEXT_PUBLIC_LANGFUSE_PUBLIC_KEY, - baseUrl: "https://cloud.langfuse.com", // 🇪🇺 EU region - // baseUrl: "https://us.cloud.langfuse.com", // 🇺🇸 US region - }); - - const handleUserFeedback = async (value: number) => - await langfuseWeb.score({ - traceId: props.traceId, - name: "user_feedback", - value, - }); - - return ( -
- - -
- ); -} -``` - -
- - -```html - - - -``` - - -
- -We integrated the Web SDK into the Vercel AI Chatbot project to collect user feedback on individual messages. Read the [blog post](/blog/showcase-llm-chatbot) for more details and code examples. - -**Installation** - -```sh npm2yarn -npm i langfuse # this is still the v3 installation as v4 does not yet support scores from browser env -``` - -In your application, set the **public api key** to create a client. - -```ts -import { LangfuseWeb } from "langfuse"; - -const langfuseWeb = new LangfuseWeb({ - publicKey: "pk-lf-...", - baseUrl: "https://cloud.langfuse.com", // 🇪🇺 EU region - // baseUrl: "https://us.cloud.langfuse.com", // 🇺🇸 US region -}); -``` - -Hint for Next.js users: you need to prefix the public key with `NEXT_PUBLIC_` to expose it in the frontend. - - - - Never set your Langfuse **secret key** in public browser code. The `LangfuseWeb` requires only the public key. - - - -**Create custom scores** - -Scores are used to evaluate executions/traces. They are attached to a single trace. If the score relates to a specific step of the trace, the score can optionally also be attached to the observation to enable evaluating it specifically. - - - While integrating Langfuse, it is important to either include the Langfuse Ids - in the response to the frontend or to use an own id as the trace id which is - available in both backend and frontend. - - -```ts -// pass traceId and observationId to front end -await langfuseWeb.score({ - traceId: message.traceId, - observationId: message.observationId, - name: "user-feedback", - value: 1, - comment: "I like how personalized the response is", -}); -``` - -Learn more about [custom scores here](/docs/evaluation/evaluation-methods/custom-scores). - -## Setup with Sentry - -If you're using both Sentry and Langfuse in your application, you'll need to configure a custom OpenTelemetry setup since both tools use OpenTelemetry for tracing. This guide shows how to send error monitoring data to Sentry while simultaneously capturing LLM observability traces in Langfuse. - -To use Langfuse alongside Sentry, configure `skipOpenTelemetrySetup: true` in your Sentry initialization and manually set up the OpenTelemetry components: - - - -**Important:** If you're using Sentry's tracing feature, be aware that Sentry's `tracesSampleRate` will also apply to your Langfuse traces, since both services share the same underlying OpenTelemetry setup. - -If you only want to use Sentry for error monitoring (without tracing), omit the `tracesSampleRate` in `Sentry.init()` and remove the `SentrySpanProcessor` from the span processors array. - - - -```ts filename="instrumentation.ts" -import * as Sentry from "@sentry/node"; - -import { LangfuseSpanProcessor } from "@langfuse/otel"; -import { - SentryPropagator, - SentrySampler, - SentrySpanProcessor, -} from "@sentry/opentelemetry"; -import { NodeTracerProvider } from "@opentelemetry/sdk-trace-node"; - -const sentryClient = Sentry.init({ - dsn: "", - skipOpenTelemetrySetup: true, - tracesSampleRate: 1, // <-- must be dropped if Sentry tracing is not used - // ... -}); - -const provider = new NodeTracerProvider({ - sampler: sentryClient ? new SentrySampler(sentryClient) : undefined, - spanProcessors: [ - new LangfuseSpanProcessor(), - new SentrySpanProcessor(), // <-- must be dropped if Sentry tracing is not used - ], -}); - -provider.register({ - propagator: new SentryPropagator(), - contextManager: new Sentry.SentryContextManager(), -}); -``` - -For more details on custom OpenTelemetry setup with Sentry, see the [Sentry documentation](https://docs.sentry.io/platforms/javascript/guides/express/opentelemetry/custom-setup/). diff --git a/pages/docs/observability/sdk/typescript/instrumentation.mdx b/pages/docs/observability/sdk/typescript/instrumentation.mdx deleted file mode 100644 index 399b9ef4a..000000000 --- a/pages/docs/observability/sdk/typescript/instrumentation.mdx +++ /dev/null @@ -1,519 +0,0 @@ ---- -title: TypeScript SDK - Instrumentation -description: Instrumentation methods for the TypeScript SDK. -label: "Version: JS SDK v4" ---- - -import GetStartedJsLangchain from "@/components-mdx/get-started/js-langchain.mdx"; -import { BookOpen, Code } from "lucide-react"; - -# TypeScript SDK - Instrumentation - -To instrument your application to send traces to Langfuse, you can use - -1. [**Native instrumentation**](#native-instrumentation) of llm/agent libraries for out-of-the-box tracing -2. [**Custom instrumentation**](#custom-instrumentation) methods for fine-grained control - - Context manager: `startActiveObservation` - - Wrapper: `observe` - - Manual: `startObservation` - -These components are interoperable. Please refer to this [API route handler](https://github.com/langfuse/langfuse-docs/blob/main/components/qaChatbot/apiHandler.ts), which powers [langfuse.com/demo](/demo), as an example of how to combine the auto-instrumentation of the AI SDK V5 with custom instrumentation. This approach captures more details and groups multiple LLM calls into a single trace. - -## Native instrumentation [#native-instrumentation] - -Langfuse integrates with many llm/agent libraries to automatically trace your application. For a full list, see the [Langfuse Integrations](/integrations) page. - -These are the most popular ones: - - - - - -The `@langfuse/openai` package provides a wrapper to automatically trace calls to the OpenAI SDK. - -For an end-to-end example, see the [Langfuse + OpenAI JS/TS Cookbook](https://langfuse.com/guides/cookbook/js_integration_openai). - -**Installation:** - -```bash -npm install @langfuse/openai -``` - -**Usage:** - -The `observeOpenAI` function wraps your OpenAI client instance. All subsequent API calls made with the wrapped client will be traced as generations and nested automatically in the current trace tree. If there's no active trace in context, a new one will be created automatically. - -```typescript /observeOpenAI/ -import { OpenAI } from "openai"; -import { observeOpenAI } from "@langfuse/openai"; - -// Instantiate the OpenAI client as usual -const openai = new OpenAI(); - -// Wrap it with Langfuse -const tracedOpenAI = observeOpenAI(openai, { - // Pass trace-level attributes that will be applied to all calls - traceName: "my-openai-trace", - sessionId: "user-session-123", - userId: "user-abc", - tags: ["openai-integration"], -}); - -// Use the wrapped client just like the original -const completion = await tracedOpenAI.chat.completions.create({ - model: "gpt-4", - messages: [{ role: "user", content: "What is OpenTelemetry?" }], -}); -``` - - - - - -The Vercel AI SDK offers native instrumentation with OpenTelemetry. You can enable the Vercel AI SDK telemetry by passing `{ experimental_telemetry: { isEnabled: true }}` to your AI SDK function calls. - -The LangfuseSpanProcessor automatically detects and handles multimodal data in your traces. - -Here is a full example on how to set up tracing with the - -- AI SDK v5 -- Next JS -- deployed on Vercel - -```ts filename="instrumentation.ts" -import { LangfuseSpanProcessor, ShouldExportSpan } from "@langfuse/otel"; -import { NodeTracerProvider } from "@opentelemetry/sdk-trace-node"; - -// Optional: filter our NextJS infra spans -const shouldExportSpan: ShouldExportSpan = (span) => { - return span.otelSpan.instrumentationScope.name !== "next.js"; -}; - -const langfuseSpanProcessor = new LangfuseSpanProcessor({ - shouldExportSpan, -}); - -const tracerProvider = new NodeTracerProvider({ - spanProcessors: [langfuseSpanProcessor], -}); - -tracerProvider.register(); -``` - - - If you are using Next.js, please use a manual OpenTelemetry setup via the - `NodeTracerProvider` rather than via `registerOTel` from `@vercel/otel`. This - is because [the `@vercel/otel` package does not yet support the OpenTelemetry - JS SDK v2](https://github.com/vercel/otel/issues/154) on which the - `@langfuse/tracing` and `@langfuse/otel` packages are based. - - -```ts filename="route.ts" -import { streamText } from "ai"; -import { after } from "next/server"; - -import { openai } from "@ai-sdk/openai"; -import { - observe, - updateActiveObservation, - updateActiveTrace, -} from "@langfuse/tracing"; -import { trace } from "@opentelemetry/api"; - -import { langfuseSpanProcessor } from "@/src/instrumentation"; - -const handler = async (req: Request) => { - const { - messages, - chatId, - userId, - }: { messages: UIMessage[]; chatId: string; userId: string } = - await req.json(); - - // Set session id and user id on active trace - const inputText = messages[messages.length - 1].parts.find( - (part) => part.type === "text" - )?.text; - - updateActiveObservation({ - input: inputText, - }); - - updateActiveTrace({ - name: "my-ai-sdk-trace", - sessionId: chatId, - userId, - input: inputText, - }); - - const result = streamText({ - // ... other streamText options ... - experimental_telemetry: { - isEnabled: true, - }, - onFinish: async (result) => { - updateActiveObservation({ - output: result.content, - }); - updateActiveTrace({ - output: result.content, - }); - - // End span manually after stream has finished - trace.getActiveSpan().end(); - }, - }); - - // Important in serverless environments: schedule flush after request is finished - after(async () => await langfuseSpanProcessor.forceFlush()); - - return result.toUIMessageStreamResponse(); -}; - -export const POST = observe(handler, { - name: "handle-chat-message", - endOnExit: false, // end observation _after_ stream has finished -}); -``` - -Learn more about the AI SDK Telemetry in the [Vercel AI SDK documentation on Telemetry](https://ai-sdk.dev/docs/ai-sdk-core/telemetry). - - - - - -The `@langfuse/langchain` package offers a `CallbackHandler` to integrate Langfuse tracing into your LangChain applications. - - - - - } - title="Documentation" - href="/integrations/frameworks/langchain" - arrow - /> - } - title="Notebook" - href="/guides/cookbook/js_integration_langchain" - arrow - /> - - - - - - -Many LLM and data libraries are built with OpenTelemetry support. If a library you use supports OTEL, you just need to ensure the `LangfuseSpanProcessor` is registered in your OTEL setup. All traces generated by that library will automatically be sent to Langfuse. - - - - - -## Custom instrumentation [#custom-instrumentation] - -You can add custom instrumentations to your application via - -- the `observe` wrapper -- `startActiveObservation` context managers -- manually managing the observation lifecycle and its nesting with the `startObservation` function - - - -For an end-to-end example, see the [JS Instrumentation Cookbook](/guides/cookbook/js_langfuse_sdk). - - - -### Context management with callbacks - -To simplify nesting and context management, you can use `startActiveObservation`. These functions take a callback and automatically manage the observation's lifecycle and the OpenTelemetry context. Any observation created inside the callback will automatically be nested under the active observation, and the observation will be ended when the callback finishes. - -This is the recommended approach for most use cases as it prevents context leakage and ensures observations are properly ended. - -```typescript /startActiveObservation/ /span.update/ -import { startActiveObservation, startObservation } from "@langfuse/tracing"; - -await startActiveObservation( - // name - "user-request", - // callback - async (span) => { - span.update({ - input: { query: "What is the capital of France?" }, - }); - - // Example child, could also use startActiveObservation - // This manually created generation (see docs below) will automatically be a child of "user-request" - const generation = startObservation( - "llm-call", - { - model: "gpt-4", - input: [{ role: "user", content: "What is the capital of France?" }], - }, - { asType: "generation" } - ); - generation.update({ - usageDetails: { input: 10, output: 5 }, - output: { content: "The capital of France is Paris." }, - }); - generation.end(); - - span.update({ output: "Successfully answered." }); - } -); -``` - -### `observe` wrapper - -The `observe` wrapper is a powerful tool for tracing existing functions without modifying their internal logic. It acts as a decorator that automatically creates a **span** or **generation** around the function call. You can use the `updateActiveObservation` function to add attributes to the observation from within the wrapped function. - -```typescript /observe/ /updateActiveObservation/ -import { observe, updateActiveObservation } from "@langfuse/tracing"; - -// An existing function -async function fetchData(source: string) { - updateActiveObservation({ metadata: { source: "API" } }); - // ... logic to fetch data - return { data: `some data from ${source}` }; -} - -// Wrap the function to trace it -const tracedFetchData = observe( - // method - fetchData, - // options, optional, see below - {} -); - -// Now, every time you call tracedFetchData, a span is created. -// Its input and output are automatically populated with the -// function's arguments and return value. -const result = await tracedFetchData("API"); -``` - -You can configure the `observe` wrapper by passing an options object as the second argument: - -| Option | Description | Default | -| --------------- | ---------------------------------------------------------------------------------------------------------------- | ----------------------------- | -| `name` | The name of the observation. | The original function's name. | -| `asType` | The [type of observation](/docs/observability/features/observation-types) to create (e.g. `span`, `generation`). | `"span"` | -| `captureInput` | Whether to capture the function's arguments as the `input` of the observation. | `true` | -| `captureOutput` | Whether to capture the function's return value or thrown error as the `output` of the observation. | `true` | - -### Manual observations - -The core tracing function (`startObservation`) gives you full control over creating observations. You can pass the `asType` option to specify the [type of observation](/docs/observability/features/observation-types) to create. - -When you call one of these functions, the new observation is automatically linked as a child of the currently active operation in the OpenTelemetry context. However, it does **not** make this new observation the active one. This means any further operations you trace will still be linked to the _original_ parent, not the one you just created. - -To create nested observations manually, use the methods on the returned object (e.g., `parentSpan.startObservation(...)`). - -```typescript /startObservation/ /end/ /asType/ -import { startObservation } from "@langfuse/tracing"; - -// Start a root span for a user request -const span = startObservation( - // name - "user-request", - // params - { - input: { query: "What is the capital of France?" }, - } -); - -// Create a nested span for, e.g., a tool call -const toolCall = span.startObservation( - // name - "fetch-weather", - // params - { - input: { city: "Paris" }, - }, - // Specify observation type in asType - // This will type the attributes argument accordingly - // Default is 'span' - { asType: "tool" } -); - -// Simulate work and end the tool call span -await new Promise((resolve) => setTimeout(resolve, 100)); -toolCall.update({ output: { temperature: "15°C" } }).end(); - -// Create a nested generation for the LLM call -const generation = span.startObservation( - "llm-call", - { - model: "gpt-4", - input: [{ role: "user", content: "What is the capital of France?" }], - }, - { asType: "generation" } -); - -generation.update({ - usageDetails: { input: 10, output: 5 }, - output: { content: "The capital of France is Paris." }, -}); - -generation.end(); - -// End the root span -span.update({ output: "Successfully answered user request." }).end(); -``` - - - If you use `startObservation()`, you are responsible for calling `.end()` on - the returned observation object. Failure to do so will result in incomplete or - missing observations in Langfuse. - - -### Updating Traces - -Often, you might not have all the information about a trace (like a `userId` or `sessionId`) when you start it. The SDK lets you add or update trace-level attributes either via attribute propagation across all observations or via the `updateTrace` method. - -**Propagatable Attributes:** - -- `userId` - User identifier -- `sessionId` - Session identifier -- `metadata` - Key-value metadata (both keys and values must be strings) -- `version` - Version identifier -- `tags` - Array of tag strings - -**Non-Propagatable Attributes** (use `updateTrace()` instead): - -- `name` - Trace name -- `input` / `output` - Trace input/output -- `public` - Public visibility flag - -#### Propagating Attributes [#propagate-attributes] - -Certain attributes (`userId`, `sessionId`, `metadata`, `version`, `tags`) should be applied to **all spans** created within some execution scope. This is important because Langfuse aggregation queries (e.g., filtering by userId, calculating costs by sessionId) will soon operate across individual observations rather than the trace level. - -Use the `propagateAttributes()` function to automatically propagate these attributes to all child observations: - -```typescript /propagateAttributes/ -import { startActiveObservation, propagateAttributes } from "@langfuse/tracing"; - -await startActiveObservation("user-workflow", async (span) => { - await propagateAttributes( - { - userId: "user_123", - sessionId: "session_abc", - metadata: { experiment: "variant_a", env: "prod" }, - version: "1.0", - }, - async () => { - // All spans created here inherit these attributes - const generation = startObservation( - "llm-call", - { model: "gpt-4" }, - { asType: "generation" } - ); - // This generation automatically has userId, sessionId, metadata, version - generation.end(); - } - ); -}); -``` - -import { PropagationRestrictionsCallout } from "@/components/PropagationRestrictionsCallout"; - - - -##### Cross-Service Propagation - -For distributed tracing across multiple services, use the `asBaggage` parameter (see [OpenTelemetry documentation](https://opentelemetry.io/docs/concepts/signals/baggage/) for more details) to propagate attributes via HTTP headers: - -```typescript /propagateAttributes/ -import { propagateAttributes, startActiveObservation } from "@langfuse/tracing"; - -await startActiveObservation("api-request", async () => { - await propagateAttributes( - { - userId: "user_123", - sessionId: "session_abc", - asBaggage: true, // Propagate via HTTP headers - }, - async () => { - // HTTP request to Service B - const response = await fetch("https://service-b.example.com/api"); - // userId and sessionId are now in HTTP headers - } - ); -}); -``` - - - **Security Warning:** When `asBaggage: true`, attribute values are added to - HTTP headers on ALL outbound requests. Only enable for non-sensitive values - and when you need cross-service tracing. - - -#### `.updateTrace()` on an observation - -When you create an observation manually with `startObservation`, the returned object has an `.updateTrace()` method. You can call this at any time before the root span ends to apply attributes to the entire trace. - -```typescript /updateTrace/ -import { startObservation } from "@langfuse/tracing"; - -// Start a trace without knowing the user yet -const rootSpan = startObservation("data-processing"); - -// ... some initial steps ... - -// Later, once the user is authenticated, update the trace -const userId = "user-123"; -const sessionId = "session-abc"; -rootSpan.updateTrace({ - userId: userId, - sessionId: sessionId, - tags: ["authenticated-user"], - metadata: { plan: "premium" }, -}); - -// ... continue with the rest of the trace ... -const generation = rootSpan.startObservation( - "llm-call", - {}, - { asType: "generation" } -); - -generation.end(); - -rootSpan.end(); -``` - -#### `updateActiveTrace()` - - - **Note:** For `userId`, `sessionId`, `metadata`, and `version`, use - `propagateAttributes()` (see below) to ensure these attributes are applied to - **all observations**, not just the trace object. - -In the near-term future filtering and aggregating observations by these attributes requires them to be present on all observations, and `propagateAttributes` is the future-proof solution. - - - -When you're inside a callback from `startActiveObservation`, or a function wrapped with `observe`, you might not have a direct reference to an observation object. In these cases, use the `updateActiveTrace()` function. It automatically finds the currently active trace in the context and applies the new attributes. - -```typescript /updateActiveTrace/ -import { startActiveObservation, updateActiveTrace } from "@langfuse/tracing"; - -await startActiveObservation("user-request", async (span) => { - // Initial part of the request - span.update({ input: { path: "/api/process" } }); - - // Simulate fetching user data - await new Promise((resolve) => setTimeout(resolve, 50)); - const user = { id: "user-5678", name: "Jane Doe" }; - - // Update the active trace with the user's information - updateActiveTrace({ - userId: user.id, - metadata: { userName: user.name }, - }); - - // ... continue logic ... - span.update({ output: { status: "success" } }).end(); -}); -``` diff --git a/pages/docs/observability/sdk/typescript/overview.mdx b/pages/docs/observability/sdk/typescript/overview.mdx deleted file mode 100644 index d28dc6a36..000000000 --- a/pages/docs/observability/sdk/typescript/overview.mdx +++ /dev/null @@ -1,162 +0,0 @@ ---- -title: TypeScript SDK - Overview -description: Improved OTEL-based TypeScript SDK (v4) with better developer experience and third-party integrations. -category: SDKs -label: "Version: JS SDK v4" ---- - -# TypeScript SDK - Overview - -The modular Langfuse TypeScript SDK (v4) is built on **[OpenTelemetry](https://opentelemetry.io/)** for robust observability, better context management, and easy integration with third-party libraries. - - - This documentation is for the TypeScript SDK v4. If you are looking for the - documentation for the TypeScript SDK v3, please [click - here](https://js-sdk-v3.docs-snapshot.langfuse.com/docs/observability/sdk/typescript/guide/). - - - - If you are self-hosting Langfuse, the TypeScript SDK v4 requires **Langfuse - platform version ≥ 3.95.0** for all features to work correctly. - - -## Quickstart - -Get your first trace into Langfuse in just a few minutes. - - - -### Install packages - -Install the relevant packages to get started with tracing: - -```bash -npm install @langfuse/tracing @langfuse/otel @opentelemetry/sdk-node -``` - -Learn more about the packages [here](/docs/observability/sdk/typescript/overview#packages). - -### Set up environment variables - -import EnvJS from "@/components-mdx/env-js.mdx"; - -Add your Langfuse credentials to your environment variables. Make sure that you have a `.env` file in your project root and a package like `dotenv` to load the variables. - - - -### Set up OpenTelemetry - -Create a file named `instrumentation.ts` to initialize the OpenTelemetry SDK. The **`LangfuseSpanProcessor`** is the key component that sends traces to Langfuse. - -```ts filename="instrumentation.ts" /LangfuseSpanProcessor/ -import { NodeSDK } from "@opentelemetry/sdk-node"; -import { LangfuseSpanProcessor } from "@langfuse/otel"; - -const sdk = new NodeSDK({ - spanProcessors: [new LangfuseSpanProcessor()], -}); - -sdk.start(); -``` - -Import the `instrumentation.ts` file at the very top of your application's entry point (e.g., `index.ts`). - -```ts filename="index.ts" -import "./instrumentation"; // Must be the first import -``` - -Learn more about setting up OpenTelemetry [here](/docs/observability/sdk/typescript/setup). - -### Instrument your application - -Use one of the native Langfuse framework [integrations](/integrations) to automatically trace your application. - -Alternatively, manually instrument your application, e.g. by using the `startActiveObservation`. This function takes a callback and automatically manages the observation's lifecycle and the OpenTelemetry context. Any observation created inside the callback will automatically be nested under the active observation, and the observation will be ended when the callback finishes. - -This is just an example, check out the [instrumentation](/docs/observability/sdk/typescript/instrumentation) page for more details. - -```ts filename="index.ts" /startActiveObservation/ -import "./instrumentation"; -import { startActiveObservation } from "@langfuse/tracing"; - -async function main() { - await startActiveObservation("my-first-trace", async (span) => { - span.update({ - input: "Hello, Langfuse!", - output: "This is my first trace!", - }); - }); -} - -main(); -``` - -### Run your application - -Execute your application. You should see your trace appear in the Langfuse UI. - -```bash -npx tsx index.ts -``` - - - -## Packages - -import JSSDKPackages from "@/components-mdx/js-sdk-packages.mdx"; - - - -## OpenTelemetry foundation - -Building on OpenTelemetry is a core design choice for this SDK. It offers several key advantages: - -- **Standardization**: It aligns with the industry standard for observability, making it easier to integrate with existing monitoring and APM tools. -- **Robust Context Management**: OpenTelemetry provides reliable [context propagation](https://opentelemetry.io/docs/concepts/context-propagation/), ensuring that traces are correctly linked even in complex, asynchronous applications. -- **Attribute Propagation**: Certain trace attributes (`userId`, `sessionId`, `metadata`, `version`, `tags`) can be automatically propagated to all child observations using `propagateAttributes()`. This ensures consistent attribute coverage across all observations in a trace. See the [instrumentation docs](/docs/observability/sdk/typescript/instrumentation#propagating-trace-attributes) for details. -- **Ecosystem & Interoperability**: You can leverage a vast ecosystem of third-party instrumentations. If a library you use supports OpenTelemetry, its traces can be sent to Langfuse automatically. - -## Learn more - -import { - Rocket, - Plug, - Beaker, - Settings, - LifeBuoy, - BookOpen, -} from "lucide-react"; - - - } - title="Setup" - href="/docs/observability/sdk/typescript/setup" - arrow - /> - } - title="Instrumentation" - href="/docs/observability/sdk/typescript/instrumentation" - arrow - /> - } - title="Advanced usage" - href="/docs/observability/sdk/typescript/advanced-usage" - arrow - /> - } - title="Troubleshooting & FAQ" - href="/docs/observability/sdk/typescript/troubleshooting-and-faq" - arrow - /> - } - title="Reference" - href="https://js.reference.langfuse.com/" - newWindow - arrow - /> - diff --git a/pages/docs/observability/sdk/typescript/setup.mdx b/pages/docs/observability/sdk/typescript/setup.mdx deleted file mode 100644 index 5fa712ca7..000000000 --- a/pages/docs/observability/sdk/typescript/setup.mdx +++ /dev/null @@ -1,141 +0,0 @@ ---- -title: TypeScript SDK - Setup -description: Tracing setup for the TypeScript SDK. -label: "Version: JS SDK v4" ---- - -# TypeScript SDK - Setup - -import EnvJS from "@/components-mdx/env-js.mdx"; - -The Langfuse TypeScript SDK offers two setup approaches: - -1. [**Tracing**](#tracing-setup) for [Langfuse Observability](/docs/observability/overview) using OpenTelemetry -2. [**Client**](#client-setup) for other Langfuse features like prompt management, evaluation, or accessing the Langfuse API - -## Tracing Setup [#tracing-setup] - - - -### Installation - -Install the relevant packages for a full tracing setup: - -```bash -npm install @langfuse/tracing @langfuse/otel @opentelemetry/sdk-node -``` - -- **`@langfuse/tracing`**: Core tracing functions (`startObservation`, `startActiveObservation`, etc.) -- **`@langfuse/otel`**: The `LangfuseSpanProcessor` to export traces to Langfuse. -- **`@opentelemetry/sdk-node`**: The OpenTelemetry SDK for Node.js. - -Learn more about the packages [here](/docs/observability/sdk/typescript/overview#packages). - -### Register your credentials - -Add your Langfuse credentials to your environment variables. Make sure that you have a `.env` file in your project root and a package like `dotenv` to load the variables. - - - -### Initialize OpenTelemetry - -The Langfuse TypeScript SDK's tracing is built on top of OpenTelemetry, so you need to set up the OpenTelemetry SDK. The `LangfuseSpanProcessor` is the key component that sends traces to Langfuse. - -```ts filename="instrumentation.ts" /LangfuseSpanProcessor/ -import { NodeSDK } from "@opentelemetry/sdk-node"; -import { LangfuseSpanProcessor } from "@langfuse/otel"; - -const sdk = new NodeSDK({ - spanProcessors: [new LangfuseSpanProcessor()], -}); - -sdk.start(); -``` - -The `LangfuseSpanProcessor` is the key component that sends traces to Langfuse. - -For more options to configure the LangfuseSpanProcessor such as masking, filtering, and more, see [the advanced usage](/docs/observability/sdk/typescript/advanced-usage). - -You can learn more about setting up OpenTelemetry in your JS environment [here](https://opentelemetry.io/docs/languages/js/getting-started/nodejs/). - - -**Next.js users:** - -If you are using Next.js, please use the OpenTelemetry setup via the `NodeSDK` described above rather than via `registerOTel` from `@vercel/otel`. This is because [the `@vercel/otel` package does not yet support the OpenTelemetry JS SDK v2](https://github.com/vercel/otel/issues/154) on which the `@langfuse/tracing` and `@langfuse/otel` packages are based. - -[See here for a full example for the Vercel AI SDK with NextJS on Vercel](/docs/observability/sdk/typescript/instrumentation#native-instrumentation). - - - - - -## Client Setup [#client-setup] - - - -### Installation - -```bash -npm install @langfuse/client -``` - -### Register your credentials - -Add your Langfuse credentials to your environment variables. Make sure that you have a `.env` file in your project root and a package like `dotenv` to load the variables. - - - -### Initialize the client - -Initialize the `LangfuseClient` to interact with Langfuse. The client will automatically use the environment variables you set above. - -```ts filename="client.ts" -import { LangfuseClient } from "@langfuse/client"; - -const langfuse = new LangfuseClient(); -``` - -
-Alternative: Configure via constructor - -You can also pass configuration options directly to the constructor: - -```ts filename="client.ts" -import { LangfuseClient } from "@langfuse/client"; - -const langfuse = new LangfuseClient({ - publicKey: "your-public-key", - secretKey: "your-secret-key", - baseUrl: "https://cloud.langfuse.com", // or your self-hosted instance -}); -``` - -
- -
- -## Learn more - -import { - Rocket, - Plug, - Beaker, - Settings, - LifeBuoy, - BookOpen, -} from "lucide-react"; - - - } - title="Cookbook: Instrumentation" - href="/guides/cookbook/js_langfuse_sdk" - arrow - /> - } - title="Advanced Usage" - href="/docs/observability/sdk/typescript/advanced-usage" - arrow - /> - diff --git a/pages/docs/observability/sdk/typescript/troubleshooting-and-faq.mdx b/pages/docs/observability/sdk/typescript/troubleshooting-and-faq.mdx deleted file mode 100644 index 4fc3c6ead..000000000 --- a/pages/docs/observability/sdk/typescript/troubleshooting-and-faq.mdx +++ /dev/null @@ -1,19 +0,0 @@ ---- -title: TypeScript SDK - Troubleshooting and FAQ -description: Troubleshooting and FAQ for the TypeScript SDK. -label: "Version: JS SDK v4" ---- - -# TypeScript SDK - Troubleshooting and FAQ - -If your issue is not covered here, please see our [support page](/support). - -## Missing traces in serverless environments - -Please see the documentation on [serverless environments](/docs/observability/sdk/typescript/advanced-usage#serverless-environments) - -## Missing traces with `@vercel/otel` - -If you are using Next.js, please use a manual OpenTelemetry setup via the `NodeTracerProvider` rather than via `registerOTel` from `@vercel/otel`. This is because [the `@vercel/otel` package does not yet support the OpenTelemetry JS SDK v2](https://github.com/vercel/otel/issues/154) on which the `@langfuse/tracing` and `@langfuse/otel` packages are based. - -If you are still missing traces, please see the documentation on [serverless environments](/docs/observability/sdk/typescript/advanced-usage#serverless-environments) diff --git a/pages/docs/observability/sdk/typescript/upgrade-path.mdx b/pages/docs/observability/sdk/typescript/upgrade-path.mdx deleted file mode 100644 index 64a7586ef..000000000 --- a/pages/docs/observability/sdk/typescript/upgrade-path.mdx +++ /dev/null @@ -1,131 +0,0 @@ ---- -title: TypeScript SDK - Upgrade Path -description: Upgrade path from v3 to v4 for the TypeScript SDK. ---- - -# TypeScript SDK - Upgrade Path v3 to v4 - -Please follow each section below to upgrade your application from v3 to v4. - -If you encounter any questions or issues while upgrading, please raise an [issue](/issues) on GitHub. - - - -## Initialization - -The Langfuse base URL environment variable is now `LANGFUSE_BASE_URL` and no longer `LANGFUSE_BASEURL`. For backward compatibility however, the latter will still work in v4 but not in future versions. - -## Tracing - -The v4 SDK tracing is a major rewrite based on OpenTelemetry and introduces several breaking changes. - -1. **OTEL-based Architecture**: The SDK is now built on top of OpenTelemetry. An OpenTelemetry Setup is required now and done by registering the `LangfuseSpanProcessor` with an OpenTelemetry `NodeSDK`. -2. **New Tracing Functions**: The `langfuse.trace()`, `langfuse.span()`, and `langfuse.generation()` methods have been replaced by `startObservation`, `startActiveObservation`, etc., from the `@langfuse/tracing` package. -3. **Separation of Concerns**: - - The **`@langfuse/tracing`** and **`@langfuse/otel`** packages are for tracing. - - The **`@langfuse/client`** package and the `LangfuseClient` class are now only for non-tracing features like scoring, prompt management, and datasets. - -See the [SDK v4 docs](/docs/observability/sdk/typescript/overview) for details on each. - -## Prompt Management - -- **Import**: The import of the Langfuse client is now: - - ```typescript - import { LangfuseClient } from "@langfuse/client"; - ``` - -- **Usage**: The usage of the Langfuse client is now: - - ```typescript - const langfuse = new LangfuseClient(); - - const prompt = await langfuse.prompt.get("my-prompt"); - - const compiledPrompt = prompt.compile({ topic: "developers" }); - - const response = await openai.chat.completions.create({ - model: "gpt-4o", - messages: [{ role: "user", content: compiledPrompt }], - }); - ``` - -- `version` is now an optional property of the options object of `langfuse.prompt.get()` instead of a positional argument. - - ```typescript - const prompt = await langfuse.prompt.get("my-prompt", { version: "1.0" }); - ``` - -## OpenAI integration - -- **Import**: The import of the OpenAI integration is now: - - ```typescript - import { observeOpenAI } from "@langfuse/openai"; - ``` - -- You can set the `environment` and `release` now via the `LANGFUSE_TRACING_ENVIRONMENT` and `LANGFUSE_TRACING_RELEASE` environment variables. - -## Vercel AI SDK - -Works very similarly to v3, but replaces `LangfuseExporter` from `langfuse-vercel` with the regular `LangfuseSpanProcessor` from `@langfuse/otel`. - -Please see [full example on usage with the AI SDK](/docs/observability/sdk/typescript/instrumentation#vercel-ai-sdk) for more details. - - - -Please note that provided tool definitions to the LLM are now mapped to `metadata.tools` and no longer in `input.tools`. This is relevant in case you are running evaluations on your generations. - - - -## Langchain integration - -- **Import**: The import of the Langchain integration is now: - - ```typescript - import { CallbackHandler } from "@langfuse/langchain"; - ``` - -- You can set the `environment` and `release` now via the `LANGFUSE_TRACING_ENVIRONMENT` and `LANGFUSE_TRACING_RELEASE` environment variables. - -## `langfuseClient.getTraceUrl` - -- method is now asynchronous and returns a promise - - ```typescript - const traceUrl = await langfuseClient.getTraceUrl(traceId); - ``` - -## Scoring - -- **Import**: The import of the Langfuse client is now: - - ```typescript - import { LangfuseClient } from "@langfuse/client"; - ``` - -- **Usage**: The usage of the Langfuse client is now: - - ```typescript - const langfuse = new LangfuseClient(); - - await langfuse.score.create({ - traceId: "trace_id_here", - name: "accuracy", - value: 0.9, - }); - ``` - -See [custom scores documentation](/docs/evaluation/evaluation-methods/custom-scores) for new scoring methods. - -## Datasets - -See [datasets documentation](/docs/evaluation/dataset-runs/remote-run#setup--run-via-sdk) for new dataset methods. diff --git a/pages/docs/observability/sdk/python/upgrade-path.mdx b/pages/docs/observability/sdk/upgrade-path.mdx similarity index 67% rename from pages/docs/observability/sdk/python/upgrade-path.mdx rename to pages/docs/observability/sdk/upgrade-path.mdx index 5bfbd9d30..d9937bd54 100644 --- a/pages/docs/observability/sdk/python/upgrade-path.mdx +++ b/pages/docs/observability/sdk/upgrade-path.mdx @@ -1,10 +1,16 @@ --- -title: Upgrade path from v2 to v3 of the Langfuse Python SDK -description: Upgrade path from v2 to v3 of the Langfuse Python SDK. +title: Langfuse SDK upgrade paths +description: Migrate from Python SDK v2 → v3 and TypeScript SDK v3 → v4 with side-by-side guides. category: SDKs --- -# Upgrade from the Python SDK v2 +# Upgrade paths + +Pick your SDK to follow the relevant migration steps. + + + +## Upgrade from the Python SDK v2 The Python SDK v3 introduces significant improvements and changes compared to the legacy v2 SDK. It is **not fully backward compatible**. This comprehensive guide will help you migrate based on your current integration. @@ -19,9 +25,9 @@ The Python SDK v3 introduces significant improvements and changes compared to th - **Trace Attributes** (`user_id`, `session_id`, etc.) Can be set via enclosing spans OR directly on integrations using metadata fields (OpenAI call, Langchain invocation) - **Context Management**: Automatic OTEL [context propagation](https://opentelemetry.io/docs/concepts/context-propagation/) -## Migration Path by Integration Type +### Migration Path by Integration Type -### `@observe` Decorator Users +#### `@observe` Decorator Users **v2 Pattern:** @@ -50,7 +56,7 @@ def my_function(): return "result" ``` -### OpenAI Integration +#### OpenAI Integration **v2 Pattern:** @@ -120,7 +126,7 @@ with langfuse.start_as_current_observation(as_type="span", name="chat-request") ) ``` -### LangChain Integration +#### LangChain Integration **v2 Pattern:** @@ -186,7 +192,7 @@ with langfuse.start_as_current_observation(as_type="span", name="langchain-reque ) ``` -### LlamaIndex Integration Users +#### LlamaIndex Integration Users **v2 Pattern:** @@ -224,7 +230,7 @@ with langfuse.start_as_current_observation(as_type="span", name="llamaindex-quer ) ``` -### Low-Level SDK Users +#### Low-Level SDK Users **v2 Pattern:** @@ -249,8 +255,7 @@ generation.end(output="Response") **v3 Migration:** - In v3, all spans / generations must be ended by calling `.end()` on the - returned object. + In v3, all spans / generations must be ended by calling `.end()` on the returned object. ```python @@ -284,7 +289,7 @@ with langfuse.start_as_current_observation( ) ``` -## Key Migration Checklist +### Key Migration Checklist 1. **Update Imports**: @@ -343,7 +348,7 @@ The `link` method on the dataset item objects has been replaced by a context man See the [datasets documentation](/docs/evaluation/dataset-runs/remote-run) for more details. -## Detailed Change Summary +### Detailed Change Summary 1. **Core Change: OpenTelemetry Foundation** @@ -373,12 +378,140 @@ See the [datasets documentation](/docs/evaluation/dataset-runs/remote-run) for m - **Migration**: Use `langfuse.get_current_trace_id()` instead of `get_trace_id()` 6. **Event Size Limitations**: + - **v2**: Events were limited to 1MB in size - **v3**: No size limits enforced on the SDK-side for events -## Future support for v2 +### Future support for v2 We will continue to support the v2 SDK for the foreseeable future with critical bug fixes and security patches. We will not be adding any new features to the v2 SDK. You can find a snapshot of the v2 SDK documentation [here](https://python-sdk-v2.docs-snapshot.langfuse.com/docs/observability/sdk/python/decorators). + + +## TypeScript SDK - Upgrade Path v3 to v4 + +Please follow each section below to upgrade your application from v3 to v4. + +If you encounter any questions or issues while upgrading, please raise an [issue](/issues) on GitHub. + + + +### Initialization + +The Langfuse base URL environment variable is now `LANGFUSE_BASE_URL` and no longer `LANGFUSE_BASEURL`. For backward compatibility however, the latter will still work in v4 but not in future versions. + +### Tracing + +The v4 SDK tracing is a major rewrite based on OpenTelemetry and introduces several breaking changes. + +1. **OTEL-based Architecture**: The SDK is now built on top of OpenTelemetry. An OpenTelemetry Setup is required now and done by registering the `LangfuseSpanProcessor` with an OpenTelemetry `NodeSDK`. +2. **New Tracing Functions**: The `langfuse.trace()`, `langfuse.span()`, and `langfuse.generation()` methods have been replaced by `startObservation`, `startActiveObservation`, etc., from the `@langfuse/tracing` package. +3. **Separation of Concerns**: + - The **`@langfuse/tracing`** and **`@langfuse/otel`** packages are for tracing. + - The **`@langfuse/client`** package and the `LangfuseClient` class are now only for non-tracing features like scoring, prompt management, and datasets. + +See the [SDK v4 docs](/docs/observability/sdk/overview) for details on each. + +### Prompt Management + +- **Import**: The import of the Langfuse client is now: + + ```typescript + import { LangfuseClient } from "@langfuse/client"; + ``` + +- **Usage**: The usage of the Langfuse client is now: + + ```typescript + const langfuse = new LangfuseClient(); + + const prompt = await langfuse.prompt.get("my-prompt"); + + const compiledPrompt = prompt.compile({ topic: "developers" }); + + const response = await openai.chat.completions.create({ + model: "gpt-4o", + messages: [{ role: "user", content: compiledPrompt }], + }); + ``` + +- `version` is now an optional property of the options object of `langfuse.prompt.get()` instead of a positional argument. + + ```typescript + const prompt = await langfuse.prompt.get("my-prompt", { version: "1.0" }); + ``` + +### OpenAI integration + +- **Import**: The import of the OpenAI integration is now: + + ```typescript + import { observeOpenAI } from "@langfuse/openai"; + ``` + +- You can set the `environment` and `release` now via the `LANGFUSE_TRACING_ENVIRONMENT` and `LANGFUSE_TRACING_RELEASE` environment variables. + +### Vercel AI SDK + +Works very similarly to v3, but replaces `LangfuseExporter` from `langfuse-vercel` with the regular `LangfuseSpanProcessor` from `@langfuse/otel`. + +Please see [full example on usage with the AI SDK](/docs/observability/sdk/instrumentation#framework-third-party-telemetry) for more details. + + + +Please note that provided tool definitions to the LLM are now mapped to `metadata.tools` and no longer in `input.tools`. This is relevant in case you are running evaluations on your generations. + + + +### Langchain integration + +- **Import**: The import of the Langchain integration is now: + + ```typescript + import { CallbackHandler } from "@langfuse/langchain"; + ``` + +- You can set the `environment` and `release` now via the `LANGFUSE_TRACING_ENVIRONMENT` and `LANGFUSE_TRACING_RELEASE` environment variables. + +### `langfuseClient.getTraceUrl` + +- method is now asynchronous and returns a promise + + ```typescript + const traceUrl = await langfuseClient.getTraceUrl(traceId); + ``` + +### Scoring + +- **Import**: The import of the Langfuse client is now: + + ```typescript + import { LangfuseClient } from "@langfuse/client"; + ``` + +- **Usage**: The usage of the Langfuse client is now: + + ```typescript + const langfuse = new LangfuseClient(); + + await langfuse.score.create({ + traceId: "trace_id_here", + name: "accuracy", + value: 0.9, + }); + ``` +See [custom scores documentation](/docs/evaluation/evaluation-methods/custom-scores) for new scoring methods. +### Datasets +See [datasets documentation](/docs/evaluation/dataset-runs/remote-run#setup--run-via-sdk) for new dataset methods. + + diff --git a/pages/faq/all/cutting-costs.mdx b/pages/faq/all/cutting-costs.mdx index d8472e584..83be9e239 100644 --- a/pages/faq/all/cutting-costs.mdx +++ b/pages/faq/all/cutting-costs.mdx @@ -29,7 +29,7 @@ Every observation within a trace counts toward your unit total. Some observation 2. Update your integration/instrumentation to exclude these observations. - For most integrations, you define which observations are created. Thus, you can remove them by updating your instrumentation. - If you use the Python SDK (v3, OpenTelemetry-based), all OpenTelemetry spans are exported to Langfuse by default. If some observations are not relevant, you can [filter out observations by instrumentation scope](/docs/sdk/python/sdk-v3#filtering-by-instrumentation-scope). - - If you use the JS SDK (v4, OpenTelemetry-based), you can [filter out irrelevant spans and observations](/docs/observability/sdk/typescript/advanced-usage#filtering-spans). + - If you use the JS SDK (v4, OpenTelemetry-based), you can [filter out irrelevant spans and observations](/docs/observability/sdk/advanced-features#filter-exported-spans). ## Option 2: Sample fewer traces diff --git a/pages/faq/all/empty-trace-input-and-output.mdx b/pages/faq/all/empty-trace-input-and-output.mdx index 2188c0185..5dc13cf06 100644 --- a/pages/faq/all/empty-trace-input-and-output.mdx +++ b/pages/faq/all/empty-trace-input-and-output.mdx @@ -23,7 +23,7 @@ with langfuse.start_as_current_observation(as_type="span", name="complex-pipelin output={"final_answer": "Complete response", "confidence": 0.95} ) ``` -[(1)](https://langfuse.com/docs/observability/sdk/python/sdk-v3) +[(1)](https://langfuse.com/docs/observability/sdk/overview) If you want to set trace input/output for evaluation features, you can also use: @@ -45,6 +45,6 @@ def process_user_query(user_question: str): return answer ``` -[(1)](https://langfuse.com/docs/observability/sdk/python/sdk-v3) +[(1)](https://langfuse.com/docs/observability/sdk/overview) -If you use the decorator-based integration, input/output are captured automatically, but you can override or disable this behavior using parameters like capture_input, capture_output, or by calling update_current_trace as shown above[(2)](/docs/observability/sdk/python/instrumentation#trace-inputoutput-behavior)[(1)](https://langfuse.com/docs/observability/sdk/python/sdk-v3). \ No newline at end of file +If you use the decorator-based integration, input/output are captured automatically, but you can override or disable this behavior using parameters like capture_input, capture_output, or by calling update_current_trace as shown above[(2)](/docs/observability/sdk/instrumentation#trace-inputoutput-behavior)[(1)](https://langfuse.com/docs/observability/sdk/overview). \ No newline at end of file diff --git a/pages/guides/cookbook/js_langfuse_sdk.mdx b/pages/guides/cookbook/js_langfuse_sdk.mdx index bb5e07ad3..da6eee2c1 100644 --- a/pages/guides/cookbook/js_langfuse_sdk.mdx +++ b/pages/guides/cookbook/js_langfuse_sdk.mdx @@ -8,7 +8,7 @@ category: Examples # Cookbook: Langfuse JS/TS SDK -JS/TS applications can either be traced via the [Langfuse JS/TS SDK](https://langfuse.com/docs/observability/sdk/typescript/overview), or by using one of the native integrations such as [OpenAI](https://langfuse.com/integrations/model-providers/openai-js), [LangChain](https://langfuse.com/integrations/frameworks/langchain) or [Vercel AI SDK](https://langfuse.com/integrations/frameworks/vercel-ai-sdk). +JS/TS applications can either be traced via the [Langfuse SDKs](https://langfuse.com/docs/observability/sdk/overview), or by using one of the native integrations such as [OpenAI](https://langfuse.com/integrations/model-providers/openai-js), [LangChain](https://langfuse.com/integrations/frameworks/langchain) or [Vercel AI SDK](https://langfuse.com/integrations/frameworks/vercel-ai-sdk). In this notebook, we will walk you through a **simple end-to-end example** that: @@ -382,5 +382,5 @@ After ingesting your spans, you can view them in your Langfuse dashboard. ## Learn More -- [Langfuse JS/TS SDK Documentation](https://langfuse.com/docs/observability/sdk/typescript/overview) +- [Langfuse SDK Documentation](https://langfuse.com/docs/observability/sdk/overview) - [Langfuse Integrations](https://langfuse.com/integrations) diff --git a/pages/integrations/frameworks/langchain.mdx b/pages/integrations/frameworks/langchain.mdx index 281f2c811..0b30d14ae 100644 --- a/pages/integrations/frameworks/langchain.mdx +++ b/pages/integrations/frameworks/langchain.mdx @@ -50,7 +50,7 @@ LANGFUSE_BASE_URL = "https://cloud.langfuse.com" # 🇪🇺 EU region OPENAI_API_KEY = "sk-proj-..." ``` -With the environment variables set, we can now initialize the Langfuse Client and the CallbackHandler. You can also use [constructor arguments](/docs/observability/sdk/python/setup#initialize-client) to initialize the Langfuse client. +With the environment variables set, we can now initialize the Langfuse Client and the CallbackHandler. You can also use [constructor arguments](/docs/observability/sdk/overview#initialize-tracing) to initialize the Langfuse client. ```python from langfuse import get_client diff --git a/pages/integrations/other/exa.mdx b/pages/integrations/other/exa.mdx index d123e347e..0901d14f6 100644 --- a/pages/integrations/other/exa.mdx +++ b/pages/integrations/other/exa.mdx @@ -59,7 +59,7 @@ langfuse = get_client() ## Example 1: Trace Exa `search_and_contents` -To monitor your Exa search operations, we use the [Langfuse `@observe()` decorator](https://langfuse.com/docs/sdk/python/decorators). In this example, the `@observe()` decorator captures the inputs, outputs, and execution time of the `search_with_exa()` function. For more control over the data you are sending to Langfuse, you can use the [Context Manager or create manual observations](https://langfuse.com/docs/observability/sdk/python/instrumentation#custom-instrumentation) using the Python SDK. +To monitor your Exa search operations, we use the [Langfuse `@observe()` decorator](https://langfuse.com/docs/observability/sdk/instrumentation#custom-instrumentation). In this example, the `@observe()` decorator captures the inputs, outputs, and execution time of the `search_with_exa()` function. For more control over the data you are sending to Langfuse, you can use the [Context Manager or create manual observations](https://langfuse.com/docs/observability/sdk/instrumentation#custom-instrumentation) using the Langfuse SDK. ```python diff --git a/pages/integrations/other/parallel-ai.mdx b/pages/integrations/other/parallel-ai.mdx index 684f64883..41f393112 100644 --- a/pages/integrations/other/parallel-ai.mdx +++ b/pages/integrations/other/parallel-ai.mdx @@ -46,7 +46,7 @@ os.environ["OPENAI_API_KEY"] = "sk-proj-..." ## Example 1: Tracing the Parallel Task API -To monitor the Task API requests, we use the [Langfuse `@observe()` decorator](https://langfuse.com/docs/sdk/python/decorators). In this example, the `@observe()` decorator captures the inputs, outputs, and execution time of the `parallel_task()` function. For more control over the data you are sending to Langfuse, you can use the [Context Manager or create manual observations](https://langfuse.com/docs/observability/sdk/python/instrumentation#custom-instrumentation) using the Python SDK. +To monitor the Task API requests, we use the [Langfuse `@observe()` decorator](https://langfuse.com/docs/observability/sdk/instrumentation#custom-instrumentation). In this example, the `@observe()` decorator captures the inputs, outputs, and execution time of the `parallel_task()` function. For more control over the data you are sending to Langfuse, you can use the [Context Manager or create manual observations](https://langfuse.com/docs/observability/sdk/instrumentation#custom-instrumentation) using the Langfuse SDK. ```python