Skip to content

Commit bdd58a8

Browse files
GabrielBianconiGabrielBianconi
authored
Merge docs into main (tensorzero#3195)
* Add docs starter codebase * Configure the theme * Change logo * Add more docs * Add more docs * Clean up down to tutorial * Add model provider guides * Clean up warning callouts * Cleanup * Cleanup * Cleanup * Cleanup * Cleanup --------- Co-authored-by: Gabriel Bianconi <[email protected]>
1 parent 8a1c9b3 commit bdd58a8

File tree

87 files changed

+18906
-0
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

87 files changed

+18906
-0
lines changed

‎docs/README.md‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
# Mintlify Starter Kit
2+
3+
Use the starter kit to get your docs deployed and ready to customize.
4+
5+
Click the green **Use this template** button at the top of this repo to copy the Mintlify starter kit. The starter kit contains examples with
6+
7+
- Guide pages
8+
- Navigation
9+
- Customizations
10+
- API reference pages
11+
- Use of popular components
12+
13+
**[Follow the full quickstart guide](https://starter.mintlify.com/quickstart)**
14+
15+
## Development
16+
17+
Install the [Mintlify CLI](https://www.npmjs.com/package/mint) to preview your documentation changes locally. To install, use the following command:
18+
19+
```
20+
npm i -g mint
21+
```
22+
23+
Run the following command at the root of your documentation, where your `docs.json` is located:
24+
25+
```
26+
mint dev
27+
```
28+
29+
View your local preview at `http://localhost:3000`.
30+
31+
## Publishing changes
32+
33+
Install our GitHub app from your [dashboard](https://dashboard.mintlify.com/settings/organization/github-app) to propagate changes from your repo to your deployment. Changes are deployed to production automatically after pushing to the default branch.
34+
35+
## Need help?
36+
37+
### Troubleshooting
38+
39+
- If your dev environment isn't running: Run `mint update` to ensure you have the most recent version of the CLI.
40+
- If a page loads as a 404: Make sure you are running in a folder with a valid `docs.json`.
41+
42+
### Resources
43+
- [Mintlify documentation](https://mintlify.com/docs)
44+
- [Mintlify community](https://mintlify.com/community)

‎docs/comparison/dspy.mdx‎

Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
---
2+
title: "Comparison: TensorZero vs. DSPy"
3+
sidebarTitle: "DSPy"
4+
description: "TensorZero is an open-source alternative to DSPy featuring an LLM gateway, observability, optimization, evaluations, and experimentation."
5+
---
6+
7+
TensorZero and DSPy serve **different but complementary** purposes in the LLM ecosystem.
8+
TensorZero is a full-stack LLM engineering platform focused on production applications and optimization, while DSPy is a framework for programming with language models through modular prompting.
9+
**You can get the best of both worlds by using DSPy and TensorZero together!**
10+
11+
## Similarities
12+
13+
- **LLM Optimization.**
14+
Both TensorZero and DSPy focus on LLM optimization, but in different ways.
15+
DSPy focuses on automated prompt engineering, while TensorZero provides a complete set of tools for optimizing LLM systems (including prompts, models, and inference strategies).
16+
17+
- **LLM Programming Abstractions.**
18+
Both TensorZero and DSPy provide abstractions for working with LLMs in a structured way, moving beyond raw prompting to more maintainable approaches.<br />
19+
[→ Prompt Templates & Schemas with TensorZero](/docs/gateway/guides/prompt-templates-schemas/)
20+
21+
- **Automated Prompt Engineering.**
22+
TensorZero implements MIPROv2, the automated prompt engineering algorithm recommended by DSPy.
23+
MIPROv2 jointly optimizes instructions and in-context examples in prompts.<br />
24+
[→ Recipe: Automated Prompt Engineering with MIPRO](https://github.com/tensorzero/tensorzero/tree/main/recipes/mipro)
25+
26+
## Key Differences
27+
28+
### TensorZero
29+
30+
- **Production Infrastructure.**
31+
TensorZero provides complete production infrastructure including **observability, optimization, evaluations, and experimentation** capabilities.
32+
DSPy focuses on the development phase and prompt programming patterns.
33+
34+
- **Model Optimization.**
35+
TensorZero provides tools for optimizing models, including fine-tuning and RLHF.
36+
DSPy primarily focuses on automated prompt engineering.<br />
37+
[→ Optimization Recipes with TensorZero](/docs/recipes/)
38+
39+
- **Inference-Time Optimization.**
40+
TensorZero provides inference-time optimizations like dynamic in-context learning.
41+
DSPy focuses on offline optimization strategies (e.g. static in-context learning).<br />
42+
[→ Inference-Time Optimizations with TensorZero](/docs/gateway/guides/inference-time-optimizations/)
43+
44+
### DSPy
45+
46+
- **Advanced Automated Prompt Engineering.**
47+
DSPy provides sophisticated automated prompt engineering tools for LLMs like teleprompters, recursive reasoning, and self-improvement loops.
48+
TensorZero has some built-in prompt optimization features (more on the way) and integrates with DSPy for additional capabilities.<br />
49+
[→ Improving Math Reasoning &mdash; Combining TensorZero and DSPy](https://github.com/tensorzero/tensorzero/tree/main/examples/gsm8k-custom-recipe-dspy)
50+
51+
- **Lightweight Design.**
52+
DSPy is a lightweight framework focused solely on LLM programming patterns, particularly during the R&D stage.
53+
TensorZero is a more comprehensive platform with additional infrastructure components covering end-to-end LLM engineering workflows.
54+
55+
<Tip title="Feedback">
56+
57+
Is TensorZero missing any features that are really important to you? Let us know on [GitHub Discussions](https://github.com/tensorzero/tensorzero/discussions), [Slack](https://www.tensorzero.com/slack), or [Discord](https://www.tensorzero.com/discord).
58+
59+
</Tip>
60+
61+
## Combining TensorZero and DSPy
62+
63+
You can get the best of both worlds by using DSPy and TensorZero together!
64+
65+
TensorZero provides a number of pre-built optimization recipes covering common LLM engineering workflows like supervised fine-tuning and RLHF.
66+
But you can also easily create your own recipes and workflows.
67+
This example shows how to optimize a TensorZero function using a tool like DSPy.
68+
69+
[→ Improving Math Reasoning &mdash; Combining TensorZero and DSPy](https://github.com/tensorzero/tensorzero/tree/main/examples/gsm8k-custom-recipe-dspy)

‎docs/comparison/langchain.mdx‎

Lines changed: 115 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,115 @@
1+
---
2+
title: "Comparison: TensorZero vs. LangChain"
3+
sidebarTitle: "LangChain"
4+
description: "TensorZero is an open-source alternative to LangChain featuring an LLM gateway, observability, optimization, evaluations, and experimentation."
5+
---
6+
7+
TensorZero and LangChain both provide tools for LLM orchestration, but they serve different purposes in the ecosystem.
8+
While LangChain focuses on rapid prototyping with a large ecosystem of integrations, TensorZero is designed for production-grade deployments with built-in observability, optimization, evaluations, and experimentation capabilities.
9+
10+
<Tip title="Interested in LangGraph?">
11+
12+
We provide a minimal example [integrating TensorZero with LangGraph](https://github.com/tensorzero/tensorzero/tree/main/examples/integrations/langgraph).
13+
14+
</Tip>
15+
16+
## Similarities
17+
18+
- **LLM Orchestration.**
19+
Both TensorZero and LangChain are developer tools that streamline LLM engineering workflows.
20+
TensorZero focuses on production-grade deployments and end-to-end LLM engineering workflows (inference, observability, optimization, evaluations, experimentation).
21+
LangChain focuses on rapid prototyping and offers complementary commercial products for features like observability.
22+
23+
- **Open Source.**
24+
Both TensorZero (Apache 2.0) and LangChain (MIT) are open-source.
25+
TensorZero is fully open-source (including TensorZero UI for observability), whereas LangChain requires a commercial offering for certain features (e.g. LangSmith for observability).
26+
27+
- **Unified Interface.**
28+
Both TensorZero and LangChain offer a unified interface that allows you to access LLMs from most major model providers with a single integration, with support for structured outputs, tool use, streaming, and more.<br />
29+
[→ TensorZero Gateway Quickstart](/docs/quickstart/)
30+
31+
- **Inference-Time Optimizations.**
32+
Both TensorZero and LangChain offer inference-time optimizations like dynamic in-context learning.<br />
33+
[→ Inference-Time Optimizations with TensorZero](/docs/gateway/guides/inference-time-optimizations/)
34+
35+
- **Inference Caching.**
36+
Both TensorZero and LangChain allow you to cache requests to improve latency and reduce costs.<br />
37+
[→ Inference Caching with TensorZero](/docs/gateway/guides/inference-caching/)
38+
39+
## Key Differences
40+
41+
### TensorZero
42+
43+
- **Separation of Concerns: Application Engineering vs. LLM Optimization.**
44+
TensorZero enables a clear separation between application logic and LLM implementation details.
45+
By treating LLM functions as interfaces with structured inputs and outputs, TensorZero allows you to swap implementations without changing application code.
46+
This approach makes it easier to manage complex LLM applications, enables GitOps for prompt and configuration management, and streamlines optimization and experimentation workflows.
47+
LangChain blends application logic with LLM implementation details, streamlining rapid prototyping but making it harder to maintain and optimize complex applications.<br />
48+
[→ Prompt Templates & Schemas with TensorZero](/docs/gateway/guides/prompt-templates-schemas/)<br />
49+
[→ Advanced: Think of LLM Applications as POMDPs — Not Agents](/blog/think-of-llm-applications-as-pomdps-not-agents/)
50+
51+
- **Open-Source Observability.**
52+
TensorZero offers built-in observability features (including UI), collecting inference and feedback data in your own database.
53+
LangChain requires a separate commercial service (LangSmith) for observability.
54+
55+
- **Built-in Optimization.**
56+
TensorZero offers built-in optimization features, including supervised fine-tuning, RLHF, and automated prompt engineering recipes.
57+
With the TensorZero UI, you can fine-tune models using your inference and feedback data in just a few clicks.
58+
LangChain doesn't offer any built-in optimization features.<br />
59+
[→ Optimization Recipes with TensorZero](/docs/recipes/)
60+
61+
- **Built-in Evaluations.**
62+
TensorZero offers built-in evaluation functionality, including heuristics and LLM judges.
63+
LangChain requires a separate commercial service (LangSmith) for evaluations.<br />
64+
[→ TensorZero Evaluations Overview](/docs/evaluations/)
65+
66+
- **Built-in Experimentation (A/B Testing).**
67+
TensorZero offers built-in experimentation features, allowing you to run experiments on your prompts, models, and inference strategies.
68+
LangChain doesn't offer any experimentation features.<br />
69+
[→ Experimentation (A/B Testing) with TensorZero](/docs/gateway/guides/experimentation/)
70+
71+
- **Performance & Scalability.**
72+
TensorZero is built from the ground up for high performance, with a focus on low latency and high throughput.
73+
LangChain introduces substantial latency and memory overhead to your application.<br />
74+
[→ TensorZero Gateway Benchmarks](/docs/gateway/benchmarks/)
75+
76+
- **Language and Platform Agnostic.**
77+
TensorZero is language and platform agnostic; in addition to its Python client, it supports any language that can make HTTP requests.
78+
LangChain only supports applications built in Python and JavaScript.<br />
79+
[→ TensorZero Gateway API Reference](/docs/gateway/api-reference/inference/)
80+
81+
- **Batch Inference.**
82+
TensorZero supports batch inference with certain model providers, which significantly reduces inference costs.
83+
LangChain doesn't support batch inference.<br />
84+
[→ Batch Inference with TensorZero](/docs/gateway/guides/batch-inference/)
85+
86+
- **Credential Management.**
87+
TensorZero streamlines credential management for your model providers, allowing you to manage your API keys in a single place and set up advanced workflows like load balancing between API keys.
88+
LangChain only offers basic credential management features.<br />
89+
[→ Credential Management with TensorZero](/docs/gateway/guides/credential-management/)
90+
91+
- **Automatic Fallbacks for Higher Reliability.**
92+
TensorZero allows you to very easily set up retries, fallbacks, load balancing, and routing to increase reliability.
93+
LangChain only offers basic, cumbersome fallback functionality.<br />
94+
[→ Retries & Fallbacks with TensorZero](/docs/gateway/guides/retries-fallbacks/)
95+
96+
### LangChain
97+
98+
- **Focus on Rapid Prototyping.**
99+
LangChain is designed for rapid prototyping, with a focus on ease of use and rapid iteration.
100+
TensorZero is designed for production-grade deployments, so it requires more setup and configuration (e.g. a database to store your observability data) &mdash; but you can still get started in minutes.<br />
101+
[→ TensorZero Quickstart — From 0 to Observability & Fine-Tuning](/docs/quickstart/)
102+
103+
- **Ecosystem of Integrations.**
104+
LangChain has a large ecosystem of integrations with other libraries and tools, including model providers, vector databases, observability tools, and more.
105+
TensorZero provides many integrations with model providers, but delegates other integrations to the user.
106+
107+
- **Managed Service.**
108+
LangChain offers paid managed (hosted) services for features like observability (LangSmith).
109+
TensorZero is fully open-source and self-hosted.
110+
111+
<Tip title="Feedback">
112+
113+
Is TensorZero missing any features that are really important to you? Let us know on [GitHub Discussions](https://github.com/tensorzero/tensorzero/discussions), [Slack](https://www.tensorzero.com/slack), or [Discord](https://www.tensorzero.com/discord).
114+
115+
</Tip>

‎docs/comparison/langfuse.mdx‎

Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,82 @@
1+
---
2+
title: "Comparison: TensorZero vs. Langfuse"
3+
sidebarTitle: "Langfuse"
4+
description: "TensorZero is an open-source alternative to Langfuse featuring an LLM gateway, observability, optimization, evaluations, and experimentation."
5+
---
6+
7+
TensorZero and Langfuse both provide open-source tools that streamline LLM engineering workflows.
8+
TensorZero focuses on inference and optimization, while Langfuse specializes in powerful interfaces for observability and evals.
9+
That said, **you can get the best of both worlds by using TensorZero alongside Langfuse**.
10+
11+
## Similarities
12+
13+
- **Open Source & Self-Hosted.**
14+
Both TensorZero and Langfuse are open source and self-hosted.
15+
Your data never leaves your infrastructure, and you don't risk downtime by relying on external APIs.
16+
TensorZero is fully open-source, whereas Langfuse gates some of its features behind a paid license.
17+
18+
- **Built-in Observability.**
19+
Both TensorZero and Langfuse offer built-in observability features, collecting inference in your own database.
20+
Langfuse offers a broader set of advanced observability features, including application-level tracing.
21+
TensorZero focuses more on structured data collection for optimization, including downstream metrics and feedback.
22+
23+
- **Built-in Evaluations.**
24+
Both TensorZero and Langfuse offer built-in evaluations features, enabling you to sanity check and benchmark the performance of your prompts, models, and more &mdash; using heuristics and LLM judges.
25+
TensorZero LLM judges are also TensorZero functions, which means you can optimize them using TensorZero's optimization recipes.
26+
Langfuse offers a broader set of built-in heuristics and UI features for evaluations.<br />
27+
[→ TensorZero Evaluations Overview](/docs/evaluations/)
28+
29+
## Key Differences
30+
31+
### TensorZero
32+
33+
- **Unified Inference API.**
34+
TensorZero offers a unified inference API that allows you to access LLMs from most major model providers with a single integration, with support for structured outputs, tool use, streaming, and more.
35+
Langfuse doesn't provide a built-in LLM gateway.<br />
36+
[→ TensorZero Gateway Quickstart](/docs/quickstart/)
37+
38+
- **Built-in Inference-Time Optimizations.**
39+
TensorZero offers built-in inference-time optimizations (e.g. dynamic in-context learning), allowing you to optimize your inference performance.
40+
Langfuse doesn't offer any inference-time optimizations.<br />
41+
[→ Inference-Time Optimizations with TensorZero](/docs/gateway/guides/inference-time-optimizations/)
42+
43+
- **Optimization Recipes.**
44+
TensorZero offers optimization recipes (e.g. supervised fine-tuning, RLHF, MIPRO) that leverage your own data to improve your LLM's performance.
45+
Langfuse doesn't offer built-in features like this.<br />
46+
[→ Optimization Recipes with TensorZero](/docs/recipes/)
47+
48+
- **Automatic Fallbacks for Higher Reliability.**
49+
TensorZero offers automatic fallbacks to increase reliability.
50+
Langfuse doesn't offer any such features.<br />
51+
[→ Retries & Fallbacks with TensorZero](/docs/gateway/guides/retries-fallbacks/)
52+
53+
- **Built-in Experimentation (A/B Testing).**
54+
TensorZero offers built-in experimentation features, allowing you to run experiments on your prompts, models, and inference strategies.
55+
Langfuse doesn't offer any experimentation features.<br />
56+
[→ Experimentation (A/B Testing) with TensorZero](/docs/gateway/guides/experimentation/)
57+
58+
### Langfuse
59+
60+
- **Advanced Observability & Evaluations.**
61+
While both TensorZero and Langfuse offer observability and evaluations features, Langfuse takes it further with advanced observability features.
62+
Additionally, Langfuse offers a prompt playground, which TensorZero doesn't offer (coming soon!).
63+
64+
- **Access Controls.**
65+
Langfuse offers access controls, which TensorZero doesn't offer.
66+
That said, some of Langfuse's access control features (e.g. SSO) are only available in their paid plans.
67+
68+
- **Managed Service.**
69+
Langfuse offers a paid managed (hosted) service in addition to the open-source version.
70+
TensorZero is fully open-source and self-hosted.
71+
72+
<Tip title="Feedback">
73+
74+
Is TensorZero missing any features that are really important to you? Let us know on [GitHub Discussions](https://github.com/tensorzero/tensorzero/discussions), [Slack](https://www.tensorzero.com/slack), or [Discord](https://www.tensorzero.com/discord).
75+
76+
</Tip>
77+
78+
## Combining TensorZero and Langfuse
79+
80+
You can combine TensorZero and Langfuse to get the best of both worlds.
81+
82+
A leading voice agent startup uses TensorZero for inference and optimization, alongside Langfuse for more advanced observability and evals.

0 commit comments

Comments
 (0)