langfuse · rishiraj-tf · Jul 25, 2025 · Jul 25, 2025 · Jul 25, 2025 · Aug 1, 2025
diff --git a/cookbook/_routes.json b/cookbook/_routes.json
@@ -302,5 +302,9 @@
   {
     "notebook": "integration_anthropic.ipynb",
     "docsPath": "integrations/model-providers/anthropic"
+  },
+  {
+    "notebook": "integration_truefoundry.ipynb",
+    "docsPath": "integrations/gateways/truefoundry"
   }
 ]
diff --git a/cookbook/integration_truefoundry.ipynb b/cookbook/integration_truefoundry.ipynb
@@ -0,0 +1,276 @@
+{
+  "cells": [
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "jpK4ctfAsVYs"
+      },
+      "source": [
+        "# What is Truefoundry?"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "IVKtUonV3xSl"
+      },
+      "source": [
+        "> [TrueFoundry](https://www.truefoundry.com/ai-gateway) is an enterprise-grade AI Gateway and control plane that lets you deploy, govern, and monitor any LLM or Gen-AI workload behind a single OpenAI-compatible API—bringing rate-limiting, cost controls, observability, and on-prem support to production AI applications."
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "PT7e2ers3xSl"
+      },
+      "source": [
+        "## How Truefoundry Integrates with Langfuse\n",
+        "\n",
+        "Truefoundry’s AI Gateway and Langfuse combine to give you enterprise-grade observability, governance, and cost control over every LLM request—set up in minutes.\n",
+        "\n",
+        "<details>\n",
+        "<summary><strong>Unified OpenAI-Compatible Endpoint</strong></summary>\n",
+        "\n",
+        "Point the Langfuse OpenAI client at Truefoundry’s gateway URL. Truefoundry routes to any supported model (OpenAI, Anthropic, self-hosted, etc.), while Langfuse transparently captures each call—no code changes required.\n",
+        "</details>\n",
+        "\n",
+        "<details>\n",
+        "<summary><strong>End-to-End Tracing & Metrics</strong></summary>\n",
+        "\n",
+        "Langfuse delivers:\n",
+        "- **Full request/response logs** (including system messages)  \n",
+        "- **Token usage** (prompt, completion, total)  \n",
+        "- **Latency breakdowns** per call  \n",
+        "- **Cost analytics** by model and environment  \n",
+        "Drill into any trace in seconds to optimize performance or debug regressions.\n",
+        "</details>\n",
+        "\n",
+        "<details>\n",
+        "<summary><strong>Production-Ready Controls</strong></summary>\n",
+        "\n",
+        "Truefoundry augments your LLM stack with:\n",
+        "- **Rate limiting & quotas** per team or user  \n",
+        "- **Budget alerts & spend caps** to prevent overruns  \n",
+        "- **Scoped API keys** with RBAC for dev, staging, prod  \n",
+        "- **On-prem/VPC deployment** for full data sovereignty  \n",
+        "</details>"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "K352a8ShtBJv"
+      },
+      "source": [
+        "# Prerequisites"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "kqu2TXK5s-DC"
+      },
+      "source": [
+        "Before integrating Langfuse with TrueFoundry, ensure you have:\n",
+        "1. TrueFoundry Account: Create a [Truefoundry account](https://www.truefoundry.com/register) with atleast one model provider and generate a Personal Access Token by following the instructions in [quick start](https://docs.truefoundry.com/gateway/quick-start) and [generating tokens](https://docs.truefoundry.com/gateway/authentication)\n",
+        "2. Langfuse Account: Sign up for a free [Langfuse Cloud account](https://cloud.langfuse.com/) or self-host Langfuse"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "B3Jz0Xnd3xSl"
+      },
+      "source": [
+        "<!-- STEPS_START -->\n",
+        "## Step 1: Install Dependencies"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "pGOLCw303xSl"
+      },
+      "outputs": [],
+      "source": [
+        "%pip install openai langfuse"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "EHm1jDq23xSm"
+      },
+      "source": [
+        "## Step 2: Set Up Environment Variables\n",
+        "\n",
+        "Next, set up your Langfuse API keys. You can get these keys by signing up for a free [Langfuse Cloud](https://cloud.langfuse.com/) account or by [self-hosting Langfuse](https://langfuse.com/self-hosting). These environment variables are essential for the Langfuse client to authenticate and send data to your Langfuse project."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "w5z-wci23xSm"
+      },
+      "outputs": [],
+      "source": [
+        "import os\n",
+        "\n",
+        "# Langfuse Configuration\n",
+        "os.environ[\"LANGFUSE_PUBLIC_KEY\"] = \"pk-lf-...\"\n",
+        "os.environ[\"LANGFUSE_SECRET_KEY\"] = \"sk-lf-...\"\n",
+        "os.environ[\"LANGFUSE_HOST\"] = \"https://cloud.langfuse.com\"  # 🇪🇺 EU region\n",
+        "# os.environ[\"LANGFUSE_HOST\"] = \"https://us.cloud.langfuse.com\"  # 🇺🇸 US region\n",
+        "\n",
+        "# TrueFoundry Configuration\n",
+        "os.environ[\"TRUEFOUNDRY_API_KEY\"] = \"your-truefoundry-token\"\n",
+        "os.environ[\"TRUEFOUNDRY_BASE_URL\"] = \"https://your-control-plane.truefoundry.cloud/api/llm\""
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "NJuNI3KB3xSm",
+        "outputId": "c1b24c54-1ba3-46fb-f5e5-0658e016ecbb"
+      },
+      "outputs": [
+        {
+          "data": {
+            "text/plain": [
+              "True"
+            ]
+          },
+          "execution_count": 2,
+          "metadata": {},
+          "output_type": "execute_result"
+        }
+      ],
+      "source": [
+        "from langfuse import get_client\n",
+        "\n",
+        "# Test Langfuse authentication\n",
+        "get_client().auth_check()"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "VT023zjp3xSn"
+      },
+      "source": [
+        "## Step 3: Use Langfuse OpenAI Drop-in Replacement\n",
+        "\n",
+        "Use Langfuse's OpenAI-compatible client to capture and trace every request routed through the TrueFoundry AI Gateway. Detailed steps for configuring the gateway and generating virtual LLM keys are available in the [TrueFoundry documentation](https://docs.truefoundry.com/gateway/langfuse)."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "3mje3Ste3xSn"
+      },
+      "outputs": [],
+      "source": [
+        "from langfuse.openai import OpenAI\n",
+        "import os\n",
+        "\n",
+        "# Initialize OpenAI client with TrueFoundry Gateway\n",
+        "client = OpenAI(\n",
+        "    api_key=os.environ[\"TRUEFOUNDRY_API_KEY\"],\n",
+        "    base_url=os.environ[\"TRUEFOUNDRY_BASE_URL\"]  # Base URL from unified code snippet\n",
+        ")"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "snT4qt-e3xSn"
+      },
+      "source": [
+        "## Step 4: Run an Example"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "YwTS3Z3_3xSn"
+      },
+      "outputs": [],
+      "source": [
+        "# Make a request through TrueFoundry Gateway with Langfuse tracing\n",
+        "response = client.chat.completions.create(\n",
+        "    model=\"openai-main/gpt-4o\",  # Paste the model ID you copied from TrueFoundry Gateway\n",
+        "    messages=[\n",
+        "        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant specialized in explaining AI concepts.\"},\n",
+        "        {\"role\": \"user\", \"content\": \"Why does an AI gateway help enterprises?\"},\n",
+        "    ],\n",
+        "    max_tokens=500,\n",
+        "    temperature=0.7\n",
+        ")\n",
+        "\n",
+        "print(response.choices[0].message.content)\n",
+        "\n",
+        "# Ensure all traces are sent to Langfuse\n",
+        "langfuse = get_client()\n",
+        "langfuse.flush()"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "zwVNFN_u3xSn"
+      },
+      "source": [
+        "## Step 5: See Traces in Langfuse\n",
+        "\n",
+        "After running the example, log in to Langfuse to view the detailed traces, including:\n",
+        "\n",
+        "- Request parameters\n",
+        "- Response content\n",
+        "- Token usage and latency metrics\n",
+        "- LLM model information through Truefoundry gateway\n",
+        "\n",
+        "\n",
+        "> **Note**: All other features of Langfuse will work as expected, including prompt management, evaluations, custom dashboards, and advanced observability features. The TrueFoundry integration seamlessly supports the full Langfuse feature set.\n",
+        "<!-- STEPS_END -->"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "l6yXrarr3xSn"
+      },
+      "source": [
+        "<!-- MARKDOWN_COMPONENT name: \"LearnMore\" path: \"@/components-mdx/integration-learn-more.mdx\" -->"
+      ]
+    }
+  ],
+  "metadata": {
+    "colab": {
+      "provenance": []
+    },
+    "kernelspec": {
+      "display_name": "Python (venv)",
+      "language": "python",
+      "name": "venv"
+    },
+    "language_info": {
+      "codemirror_mode": {
+        "name": "ipython",
+        "version": 3
+      },
+      "file_extension": ".py",
+      "mimetype": "text/x-python",
+      "name": "python",
+      "nbconvert_exporter": "python",
+      "pygments_lexer": "ipython3",
+      "version": "3.12.7"
+    }
+  },
+  "nbformat": 4,
+  "nbformat_minor": 0
+}
diff --git a/next.config.mjs b/next.config.mjs
@@ -346,6 +346,7 @@ const nonPermanentRedirects = [
   ["/docs/integrations/litellm/tracing", "/integrations/gateways/litellm"],
   ["/docs/integrations/portkey", "/integrations/gateways/portkey"],
   ["/docs/integrations/other/openrouter", "/integrations/gateways/openrouter"],
+  ["/docs/integrations/other/truefoundry", "/integrations/gateways/truefoundry"],
 
   // No-code integrations - redirect to integration pages where they exist
   ["/docs/integrations/dify", "/integrations/no-code/dify"],