Skip to content
Merged
Show file tree
Hide file tree
Changes from 23 commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
7496fef
docs: langfuse on spcaes guide and gradio example
jannikmaierhoefer Dec 16, 2024
0fffbdb
edit toctree
jannikmaierhoefer Dec 17, 2024
fcc7485
text edit
jannikmaierhoefer Dec 17, 2024
82e3972
edit troubleshooting part
jannikmaierhoefer Dec 19, 2024
f024ddc
edit text
jannikmaierhoefer Dec 20, 2024
41dfe8a
update numbers
jannikmaierhoefer Dec 20, 2024
ac70405
fix spelling
jannikmaierhoefer Dec 20, 2024
8462e6f
Update docs/hub/spaces-sdks-docker-langfuse.md
jannikmaierhoefer Dec 20, 2024
ad18cdd
Update docs/hub/spaces-sdks-docker-langfuse.md
jannikmaierhoefer Dec 20, 2024
a0dfd6e
Update docs/hub/spaces-sdks-docker-langfuse.md
jannikmaierhoefer Dec 20, 2024
112e7e9
Update docs/hub/spaces-sdks-docker-langfuse.md
jannikmaierhoefer Dec 20, 2024
35f25b3
Update docs/hub/spaces-sdks-docker-langfuse.md
jannikmaierhoefer Dec 20, 2024
19dc6c0
Update docs/hub/spaces-sdks-docker-langfuse.md
jannikmaierhoefer Dec 20, 2024
737301c
move troubleshoot section to gradio template readme as this is only g…
jannikmaierhoefer Dec 20, 2024
4c74941
Update docs/hub/spaces-sdks-docker-langfuse.md
jannikmaierhoefer Dec 20, 2024
9849195
edit gradio link name
jannikmaierhoefer Dec 20, 2024
9336d5d
Apply suggestions from code review
andrewrreed Dec 20, 2024
192fb20
fix setup steps numbered list formatting
andrewrreed Dec 20, 2024
b1a5a3d
Add simple tracing example with HF Serverless API
andrewrreed Dec 20, 2024
0ca2049
remove <tip> for link formatting
andrewrreed Dec 20, 2024
d08059e
point "Deploy on HF" to preselected template
andrewrreed Dec 20, 2024
27485c5
Update docs/hub/spaces-sdks-docker-langfuse.md
andrewrreed Jan 2, 2025
fc5070c
include note about HF OAuth
andrewrreed Jan 2, 2025
3b7d39a
add note about AUTH_DISABLE_SIGNUP
andrewrreed Jan 6, 2025
5e976ec
fix tip syntax
andrewrreed Jan 6, 2025
cb366e6
alt tip syntax
andrewrreed Jan 6, 2025
42262e0
update note
andrewrreed Jan 6, 2025
345cd96
back to [!TIP]
andrewrreed Jan 6, 2025
5b73543
clarify user access
andrewrreed Jan 6, 2025
1e27b68
minor cleanup
andrewrreed Jan 7, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docs/hub/_toctree.yml
Original file line number Diff line number Diff line change
Expand Up @@ -285,6 +285,8 @@
title: Evidence on Spaces
- local: spaces-sdks-docker-marimo
title: marimo on Spaces
- local: spaces-sdks-docker-langfuse
title: Langfuse on Spaces
- local: spaces-embed
title: Embed your Space
- local: spaces-run-with-docker
Expand Down
104 changes: 104 additions & 0 deletions docs/hub/spaces-sdks-docker-langfuse.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
# Langfuse on Spaces

This guide shows you how to deploy Langfuse on Hugging Face Spaces and start instrumenting your LLM application. This integration helps you to experiment on Hugging Face models, manage your prompts in one place and evaluate model outputs.

## What is Langfuse?

[Langfuse](https://langfuse.com) is an open-source LLM engineering platform that helps teams collaboratively debug, evaluate, and iterate on their LLM applications.

Key features of Langfuse include LLM tracing to capture the full context of your application's execution flow, prompt management for centralized and collaborative prompt iteration, evaluation metrics to assess output quality, dataset creation for testing and benchmarking, and a playground to experiment with prompts and model configurations.

_This video is a 10 min walkthrough of the Langfuse features:_
<iframe width="700" height="394" src="https://www.youtube.com/embed/2E8iTvGo9Hs?si=i_mPeArwkWc5_4EO" title="10 min Walkthrough of Langfuse – Open Source LLM Observability, Evaluation, and Prompt Management" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>

## Why LLM Observability?

- As language models become more prevalent, understanding their behavior and performance is important.
- **LLM observability** involves monitoring and understanding the internal states of an LLM application through its outputs.
- It is essential for addressing challenges such as:
- **Complex control flows** with repeated or chained calls, making debugging challenging.
- **Non-deterministic outputs**, adding complexity to consistent quality assessment.
- **Varied user intents**, requiring deep understanding to improve user experience.
- Building LLM applications involves intricate workflows, and observability helps in managing these complexities.

## Step 1: Set up Langfuse on Spaces

The Langfuse Hugging Face Space allows you to get up and running with a deployed version of Langfuse with just a few clicks.

<a href="https://huggingface.co/new-space?template=langfuse/langfuse-template-space">
<img src="https://huggingface.co/datasets/huggingface/badges/resolve/main/deploy-to-spaces-lg.svg" />
</a>

To get started, click the button above or follow these steps:

1. Create a [**new Hugging Face Space**](https://huggingface.co/new-space)
2. Select **Docker** as the Space SDK
3. Select **Langfuse** as the Space template
4. Enable **persistent storage** to ensure your Langfuse data is persisted across restarts
5. [Optional but recommended] For a secure deployment, replace the default values of the **environment variables**:
- `NEXTAUTH_SECRET`: Used to validate login session cookies, generate secret with at least 256 entropy using `openssl rand -base64 32`.
- `SALT`: Used to salt hashed API keys, generate secret with at least 256 entropy using `openssl rand -base64 32`.
- `ENCRYPTION_KEY`: Used to encrypt sensitive data. Must be 256 bits, 64 string characters in hex format, generate via: `openssl rand -hex 32`.

![Clone the Langfuse Space](https://langfuse.com/images/cookbook/huggingface/huggingface-space-setup.png)

## Step 2: Use Langfuse

Now that you have Langfuse running, you can start instrumenting your LLM application to capture traces and manage your prompts. Your Langfuse Space is pre-configured to use Hugging Face OAuth for secure authentication, so you'll need to authorize `read` access to your Hugging Face account upon first login.

### Monitor Any Application

Langfuse is model agnostic and can be used to trace any application. Follow the [get-started guide](https://langfuse.com/docs) in Langfuse documentation to see how you can instrument your code.

Langfuse maintains native integrations with many popular LLM frameworks, including [Langchain](https://langfuse.com/docs/integrations/langchain/tracing), [LlamaIndex](https://langfuse.com/docs/integrations/llama-index/get-started) and [OpenAI](https://langfuse.com/docs/integrations/openai/python/get-started) and offers Python and JS/TS SDKs to instrument your code. Langfuse also offers various API endpoints to ingest data and has been integrated by other open source projects such as [Langflow](https://langfuse.com/docs/integrations/langflow), [Dify](https://langfuse.com/docs/integrations/dify) and [Haystack](https://langfuse.com/docs/integrations/haystack/get-started).

### Example 1: Trace Calls to HF Serverless API

As a simple example, here's how to trace LLM calls to the HF Serverless API using the Langfuse Python SDK.

Be sure to first configure your `LANGFUSE_HOST`, `LANGFUSE_PUBLIC_KEY` and `LANGFUSE_SECRET_KEY` environment variables, and make sure you've [authenticated with your Hugging Face account](https://huggingface.co/docs/huggingface_hub/en/quick-start#authentication).

```python
from langfuse.openai import openai
from huggingface_hub import get_token

client = openai.OpenAI(
base_url="https://api-inference.huggingface.co/v1/",
api_key=get_token(),
)

messages = [{"role": "user", "content": "What is observability for LLMs?"}]

response = client.chat.completions.create(
model="meta-llama/Llama-3.3-70B-Instruct",
messages=messages,
max_tokens=100,
)
```

### Example 2: Monitor a Gradio Application

We created a Gradio template space that shows how to create a simple chat application using a Hugging Face model and trace model calls and user feedback in Langfuse - without leaving Hugging Face.

<a href="https://huggingface.co/spaces/langfuse/langfuse-gradio-example-template?duplicate=true">
<img src="https://huggingface.co/datasets/huggingface/badges/resolve/main/deploy-to-spaces-lg.svg" />
</a>

To get started, [duplicate this Gradio template space](https://huggingface.co/spaces/langfuse/langfuse-gradio-example-template?duplicate=true) and follow the instructions in the [README](https://huggingface.co/spaces/langfuse/langfuse-gradio-example-template/blob/main/README.md).

## Step 3: View Traces in Langfuse

Once you have instrumented your application, and ingested traces or user feedback into Langfuse, you can view your traces in Langfuse.

![Example trace with Gradio](https://langfuse.com/images/cookbook/huggingface/huggingface-gradio-example-trace.png)

_[Example trace in the Langfuse UI](https://langfuse-langfuse-template-space.hf.space/project/cm4r1ajtn000a4co550swodxv/traces/9cdc12fb-71bf-4074-ab0b-0b8d212d839f?timestamp=2024-12-20T12%3A12%3A50.089Z&view=preview)_

## Additional Resources and Support

- [Langfuse documentation](https://langfuse.com/docs)
- [Langfuse GitHub repository](https://github.com/langfuse/langfuse)
- [Langfuse Discord](https://langfuse.com/discord)
- [Langfuse template Space](https://huggingface.co/spaces/langfuse/langfuse-template-space)

For more help, open a support thread on [GitHub discussions](https://langfuse.com/discussions) or [open an issue](https://github.com/langfuse/langfuse/issues).