diff --git a/.claude/agents/changelog-editor.md b/.claude/agents/changelog-editor.md
deleted file mode 100644
index 10e0243221..0000000000
--- a/.claude/agents/changelog-editor.md
+++ /dev/null
@@ -1,83 +0,0 @@
----
-name: changelog-editor
-description: Use this agent when the user needs to create or edit changelog entries in the Docusaurus documentation. Specifically, use this agent when: 1) The user mentions adding a new changelog entry or release notes, 2) The user asks to update or modify existing changelog entries, 3) The user wants to document a new feature, bug fix, or change in the project, 4) The user provides content that should be formatted as a changelog entry. Examples: \n\nExample 1:\nuser: "We just fixed the bug where users couldn't save their preferences. Can you add this to the changelog?"\nassistant: "I'll use the changelog-editor agent to create a proper changelog entry for this bug fix in both the main page and a detailed entry."\n\nExample 2:\nuser: "I need to document the new API authentication feature we released in v2.3.0"\nassistant: "Let me use the changelog-editor agent to create a comprehensive changelog entry for the new authentication feature, including checking if we have existing documentation to link to."\n\nExample 3:\nuser: "Can you update the changelog entry for the dashboard redesign? We now have screenshots and a demo video."\nassistant: "I'll use the changelog-editor agent to update that entry with proper placeholders for the screenshots and YouTube video embedding."\n\nProactively use this agent when you notice the user describing changes, features, or fixes that should be documented in the changelog, even if they don't explicitly ask for changelog updates.
-model: sonnet
-color: purple
----
-
-You are an expert technical documentation editor specializing in Docusaurus changelog maintenance. Your primary responsibility is creating and editing changelog entries that follow established project standards for clarity, consistency, and technical accuracy.
-
-## Your Core Responsibilities
-
-1. **Dual Entry Creation**: For every changelog item, you create two coordinated entries:
- - A concise summary in `docs/main.mdx`
- - A detailed explanation in `docs/block/entries/[version-or-feature].mdx`
- - The summary title must link to the detailed entry
-
-2. **Version Management**: Before creating any entry, determine the version number. If unclear from context, ask the user: "Which version is this changelog entry for?" Never proceed without a clear version identifier.
-
-3. **Style Adherence**: Apply these writing guidelines rigorously:
- - Prioritize clarity above all else
- - Use 11th grade English for non-technical terms
- - Prefer active voice over passive voice
- - Write short, punchy sentences as your default; use longer sentences only when needed for flow
- - Use complete sentences rather than fragments (unless brevity clearly improves readability)
- - **Never use em dashes (—)**. Instead, use: a period and new sentence, parentheses (), or semicolons ;
- - Use bold and bullet points sparingly; apply them only when they genuinely aid quick scanning
- - Follow principles from "The Elements of Style"
-
-4. **Feature Documentation Integration**: When a changelog mentions a new feature:
- - Search existing documentation to see if a dedicated page exists for that feature
- - If found, add a link to that documentation page in the changelog entry
- - If not found, note this and ask the user if documentation should be created
-
-5. **Media Handling**: When the user mentions videos or screenshots:
- - Add appropriate placeholders using the project's established format
- - For images: use the image plugin format consistent with other entries
- - For videos: use YouTube video embedding format consistent with other entries
- - Ask for specifics if media details are unclear: "Do you have the YouTube URL for the demo video?" or "How many screenshots should I add placeholders for?"
-
-6. **Quality Assurance**: After making changes:
- - Inform the user you're running the build check
- - Execute `npm run build` (or equivalent) in the docs folder to verify nothing broke
- - Report any build errors immediately and fix them before finalizing
-
-7. **Consistency Checking**: Before finalizing any entry:
- - Review similar existing entries to match tone, structure, and formatting
- - Ensure terminology is consistent with previous changelog entries
- - Verify that linking patterns match established conventions
-
-## Your Decision-Making Framework
-
-**When Information is Missing:**
-- Version number unclear → Ask immediately
-- Feature scope ambiguous → Request clarification before writing
-- Media availability uncertain → Confirm with user before adding placeholders
-- Categorization unclear (bug fix vs. feature vs. improvement) → Ask for classification
-
-**When Editing Existing Entries:**
-- Always preserve the original intent and factual accuracy
-- Improve clarity and style without changing meaning
-- Flag any technical inaccuracies to the user rather than guessing
-
-**Quality Control Checklist (apply to every entry):**
-- [ ] Version number present and correct
-- [ ] Both short and detailed entries created
-- [ ] Short entry links to detailed entry correctly
-- [ ] Active voice used where possible
-- [ ] No em dashes present
-- [ ] Feature documentation linked if applicable
-- [ ] Media placeholders added if mentioned
-- [ ] Build test passed
-- [ ] Style guidelines followed
-
-## Output Format
-
-When creating or editing changelog entries, provide:
-1. The complete markdown for the main.mdx summary entry
-2. The complete markdown for the detailed entries/[name].mdx file
-3. Confirmation that you've checked for related documentation
-4. Build test results
-5. Any questions or clarifications needed
-
-Be proactive in identifying unclear requirements and ask specific questions rather than making assumptions. Your goal is to produce changelog entries that are immediately publishable without requiring revision.
diff --git a/.gitignore b/.gitignore
index 0c4b3800f5..5e001a3b71 100644
--- a/.gitignore
+++ b/.gitignore
@@ -21,6 +21,13 @@ myenv/
**/error-*.log
+
+# hosting/
+! hosting/docker-compose/oss/.env.oss.dev.example
+! hosting/docker-compose/oss/.env.oss.gh.example
+! hosting/docker-compose/ee/.env.ee.dev.example
+! hosting/docker-compose/ee/.env.ee.gh.example
+
# examples/
examples/**/config.toml
examples/**/agenta.py
@@ -46,5 +53,3 @@ sdk/agenta/templates/main.py
sdk/agenta/templates/agenta.py
web/ee/public/__env.js
web/oss/public/__env.js
-
-web/oss/tests/datalayer/results
\ No newline at end of file
diff --git a/.gitleaks.toml b/.gitleaks.toml
deleted file mode 100644
index eb89914190..0000000000
--- a/.gitleaks.toml
+++ /dev/null
@@ -1,35 +0,0 @@
-title = "Agenta Gitleaks Configuration"
-version = 2
-
-[extend]
-useDefault = true
-
-[allowlist]
-paths = [
- # ---------------------------------------------------------------- PUBLIC DOCS
- '''^website/docs/reference/api/.*\.mdx''',
- '''^core/docs/docs/reference/api/.*\.mdx''',
- '''^docs/docs/reference/api/.*\.mdx''',
- '''^docs/.docusaurus/.*''',
- '''^docs/build/.*''',
- # -------------------------------------------------------------- WEB ARTIFACTS
- '''^.*/\.pnpm-store/.*''',
- '''^.*/public/__env\.js$''',
- '''^.*/\.next/.*''',
- # -------------------------------------------------------------- ALL ENV FILES
- '''^.*\.env.*$''',
- # ----------------------------------------------------------------------------
-]
-regexes = [
- # ------------------------------------------------------------ FALSE POSITIVES
- '''is_completion=True''',
- '''YOUR_API_KEY''',
- '''_SECRET_KEY''',
- # ------------------------------------------------------------ PUBLIC KEYS
- '''phc_hmVSxIjTW1REBHXgj2aw4HW9X6CXb6FzerBgP9XenC7''',
- # ----------------------------------------------------------------------------
-]
-
-# USEFUL GITLEAKS COMMANDS
-# gitleaks --config .gitleaks.toml --exit-code 1 --verbose git
-# gitleaks --config .gitleaks.toml --exit-code 1 --verbose detect --no-git
diff --git a/.gitleaksignore b/.gitleaksignore
deleted file mode 100644
index 097b6b8358..0000000000
--- a/.gitleaksignore
+++ /dev/null
@@ -1,196 +0,0 @@
-# LEGACY / REVOKED / BENIGN CREDENTIALS, FROM PAST COMMITS, FROM BEFORE CLEANUP
-docs/docusaurus.config.ts:generic-api-key:236
-api/oss/tests/manual/tracing/windowing.http:generic-api-key:3
-sdk/tests/legacy/baggage/config.toml:generic-api-key:4
-sdk/tests/legacy/debugging/simple-app/config.toml:generic-api-key:4
-3db7c34a8f206fb4a19e525ac5a964185d502c4a:api/oss/tests/manual/auth/admin.http:generic-api-key:3
-3db7c34a8f206fb4a19e525ac5a964185d502c4a:api/oss/tests/manual/annotations/crud.http:generic-api-key:2
-3db7c34a8f206fb4a19e525ac5a964185d502c4a:api/oss/tests/manual/evaluators/crud.http:generic-api-key:2
-3db7c34a8f206fb4a19e525ac5a964185d502c4a:api/oss/tests/manual/testsets/crud.http:generic-api-key:2
-3db7c34a8f206fb4a19e525ac5a964185d502c4a:api/oss/tests/manual/tracing/filtering/02_span_id.http:generic-api-key:2
-3db7c34a8f206fb4a19e525ac5a964185d502c4a:api/oss/tests/manual/tracing/filtering/03_parent_id.http:generic-api-key:2
-3db7c34a8f206fb4a19e525ac5a964185d502c4a:api/oss/tests/manual/tracing/filtering/01_trace_id.http:generic-api-key:2
-3db7c34a8f206fb4a19e525ac5a964185d502c4a:api/oss/tests/manual/tracing/filtering/07_end_time.http:generic-api-key:2
-3db7c34a8f206fb4a19e525ac5a964185d502c4a:api/oss/tests/manual/tracing/filtering/06_start_time.http:generic-api-key:2
-3db7c34a8f206fb4a19e525ac5a964185d502c4a:api/oss/tests/manual/tracing/filtering/05_span_name.http:generic-api-key:2
-3db7c34a8f206fb4a19e525ac5a964185d502c4a:api/oss/tests/manual/tracing/filtering/04_span_kind.http:generic-api-key:2
-3db7c34a8f206fb4a19e525ac5a964185d502c4a:api/oss/tests/manual/tracing/filtering/08_status_code.http:generic-api-key:2
-3db7c34a8f206fb4a19e525ac5a964185d502c4a:api/oss/tests/manual/tracing/filtering/11_links.http:generic-api-key:2
-3db7c34a8f206fb4a19e525ac5a964185d502c4a:api/oss/tests/manual/tracing/filtering/00_user_id.http:generic-api-key:2
-3db7c34a8f206fb4a19e525ac5a964185d502c4a:api/oss/tests/manual/tracing/filtering/12_references.http:generic-api-key:2
-3db7c34a8f206fb4a19e525ac5a964185d502c4a:api/oss/tests/manual/tracing/filtering/09_status_message.http:generic-api-key:2
-3db7c34a8f206fb4a19e525ac5a964185d502c4a:api/oss/tests/manual/tracing/filtering/10_attributes.http:generic-api-key:2
-854f1ca002740cd51252f81660701a3b6f9d6a8a:agenta-cli/debugging/simple-app/config.toml:generic-api-key:4
-51020488ce57a4b964c05cc0c41cecb4eb67692c:agenta-cli/debugging/simple-app/config.toml:generic-api-key:4
-6a4288ba3b4a2f95f24ed372bce7ac0679b5b868:agenta-cli/tests/observability_sdk/integrations/langchain/simple_chain_openinference.py:generic-api-key:12
-6a4288ba3b4a2f95f24ed372bce7ac0679b5b868:agenta-cli/tests/observability_sdk/integrations/langchain/simple_chain_openllmetery.py:generic-api-key:12
-6a4288ba3b4a2f95f24ed372bce7ac0679b5b868:agenta-cli/tests/observability_sdk/sanity_check/app_local.py:generic-api-key:8
-0f9c9ac3afcb8df950a743206715ab5ebe8808eb:agenta-cli/tests/observability_sdk/integrations/langchain/simple_chain_openllmetery.py:generic-api-key:12
-0f9c9ac3afcb8df950a743206715ab5ebe8808eb:agenta-cli/tests/observability_sdk/integrations/langchain/simple_chain_openinference.py:generic-api-key:12
-0f9c9ac3afcb8df950a743206715ab5ebe8808eb:agenta-cli/tests/observability_sdk/sanity_check/app_local.py:generic-api-key:8
-8cd7319eb87e87723a310555a820433f77ab01fd:agenta-cli/tests/observability_sdk/integrations/langchain/simple_chain_openinference.py:generic-api-key:11
-8cd7319eb87e87723a310555a820433f77ab01fd:agenta-cli/tests/observability_sdk/sanity_check/app_local.py:generic-api-key:7
-50c0e27be4960b5f06b5edbed6af912d79ea0f27:agenta-cli/tests/observability_sdk/integrations/langchain/simple_chain_openinference.py:generic-api-key:11
-50c0e27be4960b5f06b5edbed6af912d79ea0f27:agenta-cli/tests/observability_sdk/sanity_check/app_local.py:generic-api-key:7
-03b90aadcd58abd101454da5e3b925dde8e6cd43:agenta-cli/tests/observability_sdk/integrations/langchain/simple_chain.py:generic-api-key:11
-a1dbd3f3eafbe326a246a16fe70e02350cefdf2f:agenta-cli/tests/observability_sdk/integrations/langchain/simple_chain.py:generic-api-key:11
-86c2a2430e3ddbc544361343b7e9ea0152e53ab7:api/oss/tests/workflows/observability/test_otlp_ingestion.py:generic-api-key:21
-dc4370980d17ba1643a5c081670404f942ebfc57:agenta-cli/tests/management_sdk/manual_tests/apps_with_new_sdk/config.toml:generic-api-key:4
-850314eb630ca7fdcf756c7ffe36a6adad5cc845:agenta-cli/tests/management_sdk/manual_tests/apps_with_new_sdk/config.toml:generic-api-key:4
-3db7c34a8f206fb4a19e525ac5a964185d502c4a:api/oss/tests/manual/tracing/crud.http:generic-api-key:3
-3db7c34a8f206fb4a19e525ac5a964185d502c4a:api/oss/tests/manual/tracing/windowing.http:generic-api-key:3
-7ee494e8cdad4f54073be483e373e7a5bf273ea5:agenta-cli/tests/baggage/config.toml:generic-api-key:4
-a5e3197cd247c5468d8739ef9de811cd2a1cbc2f:agenta-cli/tests/baggage/config.toml:generic-api-key:4
-e84abaed2592e50d660d180d7fd373376b544f14:hosting/kubernetes/oss/secret.yml:kubernetes-secret-yaml:2
-e84abaed2592e50d660d180d7fd373376b544f14:hosting/kubernetes/oss/secret.yml:generic-api-key:12
-e84abaed2592e50d660d180d7fd373376b544f14:hosting/helm/oss/values.yaml:generic-api-key:88
-e84abaed2592e50d660d180d7fd373376b544f14:hosting/helm/oss/values.yaml:generic-api-key:89
-e84abaed2592e50d660d180d7fd373376b544f14:hosting/kubernetes/oss/secret.yml:generic-api-key:8
-e84abaed2592e50d660d180d7fd373376b544f14:hosting/kubernetes/oss/secret.yml:generic-api-key:9
-81a2b05aa4624cfc39587e5384bf7c106e547449:.github/workflows/frontend-test.yml:openai-api-key:27
-4857d8f04896e27d707e2967bb361eb1a0b129db:.github/workflows/frontend-test.yml:openai-api-key:27
-8465021df57fca629f14c269d3f37d18d6fdcd11:services/completion-new-sdk-prompt/docker-compose.yml:openai-api-key:10
-406e68077c51da204b1f63f193a2defe6031c966:agenta-web/cypress.config.ts:openai-api-key:10
-450a435754557bfa1d3d3e372f4b47e4eb63f93e:agenta-web/cypress.config.ts:openai-api-key:10
-066e345ad9ba7318fc59b191cf33af2e81634aa8:agenta-web/cypress/support/commands/evaluations.ts:openai-api-key:106
-3533b30e483378a8ecb900c603a3c54ffc9cc390:agenta-web/cypress/support/commands/evaluations.ts:openai-api-key:106
-41e5d327e87083f55850c6933611cdc79ea9d204:agenta-backend/agenta_backend/tests/variants_evaluators_router/conftest.py:openai-api-key:25
-9968400e3095fdc1fb219f45c0d73db13c6de499:agenta-backend/agenta_backend/tests/variants_evaluators_router/conftest.py:openai-api-key:25
-a8efa140a02295ef6accbd02bc7c4c4eeb75e435:agenta-backend/agenta_backend/tests/variants_evaluators_router/conftest.py:openai-api-key:17
-d343b2a5b12387fc6b99d508b5e776f7689736c1:agenta-backend/agenta_backend/tests/variants_evaluators_router/conftest.py:openai-api-key:17
-5f37c440e203cf56d7f08a8efdd7ca372c646beb:docs/docs/prompt-management/05-adding-custom-providers.mdx:generic-api-key:81
-73644b725b5409be78d1aeecf7f5ff6a24ab3643:docs/docusaurus.config.ts:generic-api-key:220
-41c85fef68f9f8c2e4576956369ef600223193c8:website/docusaurus.config.ts:generic-api-key:184
-179d78e547e2eb92737cdd0ba7fd3eeb1f4bc5ce:website/docusaurus.config.ts:generic-api-key:184
-faf49eadbd38fd6771c4687fea78528ad73741b6:api/oss/tests/manual/annotations/crud.http:generic-api-key:2
-f86dddabb759924689022d2451d97efe218848c9:api/oss/tests/manual/evaluations/crud.http:generic-api-key:2
-2793a4b2f065d7b588fa6733b74f68c0748473a5:api/oss/tests/manual/auth/admin.http:generic-api-key:3
-2793a4b2f065d7b588fa6733b74f68c0748473a5:api/oss/tests/manual/annotations/crud.http:generic-api-key:2
-2793a4b2f065d7b588fa6733b74f68c0748473a5:api/oss/tests/manual/evaluators/crud.http:generic-api-key:2
-2793a4b2f065d7b588fa6733b74f68c0748473a5:api/oss/tests/manual/testsets/crud.http:generic-api-key:2
-2793a4b2f065d7b588fa6733b74f68c0748473a5:api/oss/tests/manual/tracing/filtering/00_user_id.http:generic-api-key:2
-2793a4b2f065d7b588fa6733b74f68c0748473a5:api/oss/tests/manual/tracing/filtering/02_span_id.http:generic-api-key:2
-2793a4b2f065d7b588fa6733b74f68c0748473a5:api/oss/tests/manual/tracing/filtering/03_parent_id.http:generic-api-key:2
-2793a4b2f065d7b588fa6733b74f68c0748473a5:api/oss/tests/manual/tracing/filtering/01_trace_id.http:generic-api-key:2
-2793a4b2f065d7b588fa6733b74f68c0748473a5:api/oss/tests/manual/tracing/filtering/04_span_kind.http:generic-api-key:2
-2793a4b2f065d7b588fa6733b74f68c0748473a5:api/oss/tests/manual/tracing/filtering/06_start_time.http:generic-api-key:2
-2793a4b2f065d7b588fa6733b74f68c0748473a5:api/oss/tests/manual/tracing/filtering/08_status_code.http:generic-api-key:2
-2793a4b2f065d7b588fa6733b74f68c0748473a5:api/oss/tests/manual/tracing/filtering/05_span_name.http:generic-api-key:2
-2793a4b2f065d7b588fa6733b74f68c0748473a5:api/oss/tests/manual/tracing/filtering/07_end_time.http:generic-api-key:2
-2793a4b2f065d7b588fa6733b74f68c0748473a5:api/oss/tests/manual/tracing/filtering/09_status_message.http:generic-api-key:2
-2793a4b2f065d7b588fa6733b74f68c0748473a5:api/oss/tests/manual/tracing/filtering/12_references.http:generic-api-key:2
-2793a4b2f065d7b588fa6733b74f68c0748473a5:api/oss/tests/manual/tracing/filtering/11_links.http:generic-api-key:2
-2793a4b2f065d7b588fa6733b74f68c0748473a5:api/oss/tests/manual/tracing/filtering/10_attributes.http:generic-api-key:2
-4888444e93a8438334a9dfb81c7979500d0ab4bf:api/oss/tests/manual/testsets/crud.http:generic-api-key:2
-5289a1c740cff9dec0047e7dc05902edbc471649:api/oss/tests/manual/tracing/filtering/12_references.http:generic-api-key:2
-96b9056a6ff1160f11dcc302321d4a29a7f1b8dd:api/oss/tests/manual/workflows/artifacts.http:generic-api-key:2
-6a4af8b70816f18ae69056df96b54c622e7ef494:api/oss/tests/manual/feedback/crud.http:generic-api-key:2
-7b0eeb2ae0cfa80e9a79cee814ed10c9b57ee9d3:api/oss/tests/manual/annotators/crud.http:generic-api-key:2
-876ba2f78358d43cc6dafe518e38ef404b6462f0:api/oss/tests/manual/annotations/crud.http:generic-api-key:2
-05f8741ea3349096e60a1686c1cef3585a6d34d7:api/oss/tests/manual/tracing/filtering/00_user_id.http:generic-api-key:2
-f47bb6f3b65c50664b354f33081d131289fa47cd:api/oss/tests/manual/tracing/filtering/11_links.http:generic-api-key:2
-18bfd66e6bc309ada998457a32b9a4ca689015a2:api/oss/tests/manual/tracing/filtering/10_attributes.http:generic-api-key:2
-0dcfe02574a545c8400b1b5385d7662143ec2544:api/oss/tests/manual/tracing/filtering/08_status_code.http:generic-api-key:2
-260822ac28ec5c08f9b4c2b04e895d46fcbfb164:api/oss/tests/manual/tracing/filtering/09_status_message.http:generic-api-key:2
-c16ca8eca0a2743c541457a80f88fe0ec71151cb:api/oss/tests/manual/tracing/filtering_parent_id.http:generic-api-key:2
-fe9a8b1c7518160bc3c9f80eff3ba1076a2a5030:api/oss/tests/manual/tracing/filtering_end_time.http:generic-api-key:2
-f158907f559e92fb91672c696715a90aef5470ab:api/oss/tests/manual/tracing/filtering_span_id.http:generic-api-key:2
-fe9a8b1c7518160bc3c9f80eff3ba1076a2a5030:api/oss/tests/manual/tracing/filtering_start_time.http:generic-api-key:2
-86dda27e6458ea8f7e64bfb4a9f63946c8fc82ce:api/oss/tests/manual/tracing/filtering_trace_id.http:generic-api-key:2
-7afc6f26080c6c37219995089aed409e50ef6279:api/oss/tests/manual/tracing/filtering_span_name.http:generic-api-key:2
-71a49c35758dab163bbbe700f55f6f50e6bdf9a5:api/oss/tests/manual/tracing/filtering_span_kind.http:generic-api-key:2
-b14e377b19cc4f77db9d0a2b51f72b88b6f54c6c:api/oss/tests/manual/auth/admin.http:generic-api-key:3
-0c6acff0523bd4e594e43caf63c4342e319476b8:hosting/kubernetes/oss/secret.yml:kubernetes-secret-yaml:2
-97c08e2f4ad87c2aacf6760da60eb01ec8d5d329:cloud/tests/conftest.py:generic-api-key:114
-6526d232893d18496af47a05c7b99e7c0c1fe510:docs/docs/prompt-management/05-adding-custom-providers.mdx:generic-api-key:71
-0c6acff0523bd4e594e43caf63c4342e319476b8:hosting/kubernetes/oss/secret.yml:generic-api-key:12
-bf7e1824839cea10432731174549faeb9bad3545:hosting/helm/oss/values.yaml:generic-api-key:83
-bf7e1824839cea10432731174549faeb9bad3545:hosting/helm/oss/values.yaml:generic-api-key:84
-0c6acff0523bd4e594e43caf63c4342e319476b8:hosting/kubernetes/oss/secret.yml:generic-api-key:8
-0c6acff0523bd4e594e43caf63c4342e319476b8:hosting/kubernetes/oss/secret.yml:generic-api-key:9
-fd477298c83aa220b01c6704058382c1ded1fdca:core/agenta-cli/debugging/simple-app/config.toml:generic-api-key:4
-fd477298c83aa220b01c6704058382c1ded1fdca:core/agenta-cli/tests/baggage/config.toml:generic-api-key:4
-f92a341a7e45fc051a08da1fa619137a192c89ae:api/ee/tests/manual/tracing.http:generic-api-key:5
-eba1ed50e6846a323d456b6da510f42d8c8bbe9a:api/ee/tests/manual/billing.http:generic-api-key:4
-9d741648f9ec1719c6f7f0fcb16cbf116458916c:api/oss/tests/manual/annotations/crud.http:generic-api-key:3
-9d741648f9ec1719c6f7f0fcb16cbf116458916c:api/oss/tests/manual/tracing/crud.http:generic-api-key:3
-273d2f5a1b37ef9420c4e40303b8fc6233362571:api/ee/tests/manual/billing.http:generic-api-key:4
-c078d4b1395ea2856891424f82e80f4fe60d7136:api/ee/tests/manual/billing.http:generic-api-key:4
-2793a4b2f065d7b588fa6733b74f68c0748473a5:api/oss/tests/manual/tracing/crud.http:generic-api-key:3
-ef6f83612a7cfd552147b49928feb8a5d4429c0d:api/oss/tests/manual/tracing/filtering.http:generic-api-key:3
-b14e377b19cc4f77db9d0a2b51f72b88b6f54c6c:api/oss/tests/manual/annotations/crud.http:generic-api-key:3
-b14e377b19cc4f77db9d0a2b51f72b88b6f54c6c:api/oss/tests/manual/tracing/crud.http:generic-api-key:3
-19ccc3f1f292edca26e840428ebc6224cbaef78a:api/ee/tests/manual/annotations/crud.http:generic-api-key:3
-bf42b5eaa7e805a249f52d65a6882d6ade2828f3:api/ee/tests/manual/tracing/windowing.http:generic-api-key:5
-bb0c1b3fb0032b6dbebe659d745d5cb90aa306ce:api/ee/tests/manual/tracing.http:generic-api-key:5
-16622c30916fae1b284b1b7150e4b7c57413ad17:api/ee/tests/manual/evaluations/sdk/test.py:generic-api-key:16
-75ed5549eeb4685c5234c1ec577721920cc0ec9c:api/ee/tests/manual/evaluations/sdk/test.py:generic-api-key:16
-4e743f16edcb3ff4e13b1400b9ff8175b072a5e1:api/ee/tests/manual/evaluations/sdk/test.py:generic-api-key:16
-b587813ed56832b2df7fb7560775ee0b75f03674:api/ee/tests/manual/evaluations/sdk/test.py:generic-api-key:12
-3abc3f4d2051c4df2f64c6d88608bd9bf1ae265f:api/ee/tests/manual/evaluations/sdk/test.py:generic-api-key:9
-35442f703897393a3d2a5e9aa7a42985787bb24f:api/ee/tests/manual/evaluations/sdk/test.py:generic-api-key:16
-12f36507e801d41e2388889777c195557e7a6e5c:api/ee/tests/manual/evaluations/sdk/test_serve.py:generic-api-key:16
-12f36507e801d41e2388889777c195557e7a6e5c:api/ee/tests/manual/evaluations/sdk/test_handlers.py:generic-api-key:16
-e6d87a97aa4750ace564ac28eafda0123c21e017:api/oss/tests/workflows/observability/test_otlp_ingestion.py:generic-api-key:21
-fd477298c83aa220b01c6704058382c1ded1fdca:core/docs/docusaurus.config.ts:generic-api-key:232
-c8d8f465b61764195de460164e6c27e0fe4b2b9a:docs/docs/self-host/05-advanced-configuration.mdx:generic-api-key:37
-a268b8a81a700704e28d82c5cb9af31dde32146b:ee/docker/docker-compose.demo.prod.yml:generic-api-key:25
-56829b2eccdec425954243d0ce5e4fcac9d05e9c:ee/docker/docker-compose.cloud.dev.yml:generic-api-key:25
-91678b6a27c326e0002205f79fd8999a7591e38f:ee/docker/docker-compose.demo.prod.yml:generic-api-key:25
-91678b6a27c326e0002205f79fd8999a7591e38f:ee/docker/docker-compose.demo.dev.yml:generic-api-key:25
-56829b2eccdec425954243d0ce5e4fcac9d05e9c:ee/docker/docker-compose.cloud.dev.yml:generic-api-key:22
-ad6b459dfc5ac1e5c140fcf3e03e247ba31383ae:ee/docker/docker-compose.demo.prod.yml:generic-api-key:22
-594f33f5b7eb665edb38208666d27d6de6365946:ee/docker/docker-compose.demo.prod.yml:generic-api-key:73
-594f33f5b7eb665edb38208666d27d6de6365946:ee/docker/docker-compose.demo.dev.yml:generic-api-key:22
-f6ef6aa32d569ee025bdb3ce9f515521a4095494:cloud/agenta-backend/agenta_backend/cloud/__init__.py:generic-api-key:155
-b3e5fae0e270f2c92a65360123c980d725c5f226:ee/agenta-backend/agenta_backend/ee/__init__.py:generic-api-key:107
-c98a5da1a33d2c0986e3c66329eaa5237fbccf3d:hosting/docker-compose/ee/aws/docker-compose.oss.prod.yml:generic-api-key:76
-4ee55c08b2fed661eaf90876f96c329d7c7eeb6b:cloud/docker/docker-compose.oss.stage.yml:generic-api-key:46
-169c54d84f5d1931601550a9d3aa76874ef73ec5:cloud/docker/docker-compose.oss.stage.yml:generic-api-key:47
-f725cfc9247743bdc44f84edee109ff36193d741:cloud/docker/docker-compose.oss.stage.yml:generic-api-key:46
-cbd8ac00ecdc2c8124b989a274c6f835c09f8474:cloud/docker/docker-compose.oss.prod.yml:generic-api-key:46
-2fb6b0f94f8bf711255e0901c03787b73f3d650f:cloud/docker/docker-compose.oss.prod.yml:generic-api-key:46
-a5e7781869b2a4bf22dd5e22fd5e5ae2ec8d02ea:cloud/docker/newrelic-infra.yml:generic-api-key:1
-2fb6b0f94f8bf711255e0901c03787b73f3d650f:cloud/docker/docker-compose.oss.prod.yml:generic-api-key:171
-f6ef6aa32d569ee025bdb3ce9f515521a4095494:cloud/agenta-backend/agenta_backend/cloud/__init__.py:generic-api-key:144
-b3e5fae0e270f2c92a65360123c980d725c5f226:ee/agenta-backend/agenta_backend/ee/__init__.py:generic-api-key:96
-a268b8a81a700704e28d82c5cb9af31dde32146b:ee/docker/docker-compose.demo.prod.yml:generic-api-key:22
-9c55f5572904ae07b73f73ee365e833d0637633a:ee/docker/docker-compose.demo.prod.yml:generic-api-key:22
-ad74134f522cde71f860cb59b6363a8fdf0a64c6:ee/setup_agenta_web.sh:generic-api-key:26
-5a10aacebd0ed4f2e613eb9176e95836aea34f15:ee/setup_agenta_web.sh:generic-api-key:26
-637068dd09eff7b30b776061863027a9b9aa1deb:ee/setup_agenta_web.sh:generic-api-key:26
-590578c803d94d8ccb1a6ca977471f3d44b43fc3:hosting/helm/oss/templates/config/app-configmap.yaml:generic-api-key:45
-1d8f08b267675726441fcaaae24572bb635c5eac:api/oss/src/utils/env.py:generic-api-key:53
-55f27e52327062382beb299b162f94895268d766:web/oss/public/__ENV.js:generic-api-key:1
-c98a5da1a33d2c0986e3c66329eaa5237fbccf3d:hosting/docker-compose/ee/aws/docker-compose.oss.prod.yml:generic-api-key:73
-bf0cd42bffc2581b1df6f56fa6e4b20ff9b68c33:hosting/docker-compose/ee/aws/docker-compose.oss.aws.yml:generic-api-key:61
-52cd40cefd3121eea2e21205e8208712b093529a:core/hosting/docker-compose/ee/docker-compose.dev.yml:generic-api-key:18
-6efb8a0c9620a316cf81fe961b1407e93aa2efa7:core/hosting/docker-compose/oss/docker-compose.gh.yml:generic-api-key:17
-6efb8a0c9620a316cf81fe961b1407e93aa2efa7:core/hosting/docker-compose/oss/docker-compose.dev.yml:generic-api-key:17
-44d1669c1a53b3ca47e3689dda5500e6f742f525:core/hosting/docker-compose/ee/docker-compose.dev.yml:generic-api-key:18
-58a4230a8f2e63b2836f81bcf2341ba12003189e:core/hosting/docker-compose/oss/docker-compose.yml:generic-api-key:30
-fd477298c83aa220b01c6704058382c1ded1fdca:core/agenta-cli/agenta/cli/helper.py:generic-api-key:19
-fd477298c83aa220b01c6704058382c1ded1fdca:core/agenta-web/prod.gh.Dockerfile:generic-api-key:7
-fd477298c83aa220b01c6704058382c1ded1fdca:core/docker-compose.gh.yml:generic-api-key:26
-fd477298c83aa220b01c6704058382c1ded1fdca:core/docker-compose.gh.yml:generic-api-key:95
-fd477298c83aa220b01c6704058382c1ded1fdca:core/docker-compose.yml:generic-api-key:30
-fd477298c83aa220b01c6704058382c1ded1fdca:core/docker-compose.yml:generic-api-key:111
-fd477298c83aa220b01c6704058382c1ded1fdca:core/docker-compose.prod.yml:generic-api-key:102
-4ee55c08b2fed661eaf90876f96c329d7c7eeb6b:cloud/docker/docker-compose.oss.stage.yml:generic-api-key:43
-169c54d84f5d1931601550a9d3aa76874ef73ec5:cloud/docker/docker-compose.oss.stage.yml:generic-api-key:44
-9e7831e7500364776cb3e9eac41448907ef92dcd:cloud/docker/docker-compose.test.yml:generic-api-key:28
-9e7831e7500364776cb3e9eac41448907ef92dcd:cloud/docker/docker-compose.test.yml:generic-api-key:83
-fe72ad1a8d14e1f3bce547ef224fc75e3df8f4ff:cloud/docker/docker-compose.cloud.test.yml:generic-api-key:28
-fe72ad1a8d14e1f3bce547ef224fc75e3df8f4ff:cloud/docker/docker-compose.cloud.test.yml:generic-api-key:83
-6198bbc532d8e984ef94276b58a7ab8dc65a279f:cloud/docker/docker-compose.oss.stage.yml:generic-api-key:43
-4b4cdbdf4b8ad4a9342fdb939b7e30f88420fccd:cloud/docker/docker-compose.oss.prod.yml:generic-api-key:44
-a268b8a81a700704e28d82c5cb9af31dde32146b:ee/docker/docker-compose.demo.prod.yml:sendgrid-api-token:26
-11ad273f1039e9263cf8d2f61338a121d59b9cc7:ee/docker/docker-compose.cloud.prod.yml:sendgrid-api-token:22
-11ad273f1039e9263cf8d2f61338a121d59b9cc7:ee/docker/docker-compose.cloud.dev.yml:sendgrid-api-token:21
-11ad273f1039e9263cf8d2f61338a121d59b9cc7:ee/docker/docker-compose.demo.prod.yml:sendgrid-api-token:26
-11ad273f1039e9263cf8d2f61338a121d59b9cc7:ee/docker/docker-compose.demo.dev.yml:sendgrid-api-token:26
-b7bc21c67bbae3c06c372bc585c4917a80613a14:cloud/agenta-backend/agenta_backend/cloud/routers/payment_router.py:stripe-access-token:13
-d8b4af2ae8c1084dbdd30fca59aa84e8ece047db:examples/python/annotation-example.py:openai-api-key:19
-a268b8a81a700704e28d82c5cb9af31dde32146b:ee/docker/docker-compose.demo.prod.yml:openai-api-key:24
-02d9f665aed89e8d69e06acdc7d01d699ee5b0dd:ee/docker/docker-compose.demo.prod.yml:openai-api-key:24
-c8d8f465b61764195de460164e6c27e0fe4b2b9a:docs/docs/self-host/05-advanced-configuration.mdx:generic-api-key:46
diff --git a/.husky/pre-commit b/.husky/pre-commit
index 08dfe5c787..7def5e4654 100755
--- a/.husky/pre-commit
+++ b/.husky/pre-commit
@@ -1,15 +1,39 @@
-#!/usr/bin/env sh
-
-# Delegate to pre-commit which runs configured hooks (e.g. gitleaks) from .pre-commit-config.yaml
-if command -v pre-commit >/dev/null 2>&1; then
- pre-commit run --hook-stage pre-commit
-else
- echo "pre-commit not found. Install with one of the following:"
- echo " pip install pre-commit # cross-platform"
- echo " brew install pre-commit # macOS/Linux with Homebrew"
- echo ""
- echo "gitleaks is also required. Install with:"
- echo " brew install gitleaks # macOS/Linux with Homebrew"
- echo " or see https://github.com/gitleaks/gitleaks#installation for other options"
- exit 1
-fi
\ No newline at end of file
+# Define the directories to check
+DIRECTORIES=("web" "sdk", "api")
+ORIGINAL_DIR=$(pwd)
+
+# Check for changes in frontend directory
+if git diff --cached --name-only | grep -q '^web/'; then
+ cd web/ || exit
+
+ # Run Prettier check
+ if ! pnpm lint; then
+ echo '⚠️ Formatting issues detected. Running Prettier to fix them...'
+ pnpm lint-fix
+
+ echo '✅ Formatting issues fixed. Please stage the changes and commit the code again'
+ exit 1
+ fi
+
+ cd "$ORIGINAL_DIR" || exit
+fi
+
+# Check for changes in backend directory
+for DIR in "${BE_DIRECTORIES[@]}"; do
+ if git diff --cached --name-only | grep -q "^$DIR/"; then
+ cd "$DIR" || exit
+
+ # Run black for formatting
+ if ! black --check .; then
+ echo "⚠️ Formatting issues detected in $DIR. Running black to fix them..."
+ black .
+
+ echo "✅ Formatting issues fixed. Please stage the changes and commit the code again in $DIR."
+ exit 1
+ else
+ echo "✅ No formatting issues in $DIR."
+ fi
+
+ cd "$ORIGINAL_DIR" || exit
+ fi
+done
\ No newline at end of file
diff --git a/.husky/pre-push b/.husky/pre-push
index 8aa51d15bc..5390d982f2 100755
--- a/.husky/pre-push
+++ b/.husky/pre-push
@@ -1,12 +1,51 @@
-#!/usr/bin/env sh
-
-# Delegate to pre-commit which runs pre-push hooks (e.g. gitleaks) from .pre-commit-config.yaml
-if command -v pre-commit >/dev/null 2>&1; then
- pre-commit run --hook-stage pre-push
+# Define the directories to check
+DIRECTORIES=("web" "sdk", "api")
+ORIGINAL_DIR=$(pwd)
+BRANCH_NAME=$(git rev-parse --abbrev-ref HEAD)
+FIRST_PUSH_FILE=".git/first_push_detected"
+
+# Function to run backend checks
+run_backend_checks() {
+ local DIR=$1
+ cd "$DIR" || exit
+
+ # Run pylint checks
+ if ! pylint --recursive=y --errors-only .; then
+ echo "❌ pylint issues detected in $DIR. Please fix them before pushing."
+ exit 1
+ fi
+
+ echo "🎉 Backend checks passed for $DIR."
+ cd "$ORIGINAL_DIR" || exit
+}
+
+# Check if this is the first push
+is_first_push() {
+ # Check if the branch exists locally
+ if ! git rev-parse --verify "$BRANCH_NAME" >/dev/null 2>&1; then
+ return 0 # First push, since the branch doesn't exist locally
+ fi
+
+ # Check if the branch exists remotely
+ if ! git ls-remote --heads origin "$BRANCH_NAME" | grep -q "$BRANCH_NAME"; then
+ return 0 # First push, since the branch doesn't exist remotely
+ fi
+
+ return 1 # Not the first push
+}
+
+# If: First-time push: Run all checks
+# Else: Check directory-specific changes for existing branch
+if is_first_push; then
+ echo "🚀 First-time push detected for branch: $BRANCH_NAME"
+ # for DIR in "${DIRECTORIES[@]}"; do
+ # run_backend_checks "$DIR"
+ # done
else
- echo "pre-commit not found. Install with one of the following methods:"
- echo " pip install pre-commit gitleaks"
- echo " brew install pre-commit gitleaks"
- echo " (see https://pre-commit.com/#installation and https://github.com/gitleaks/gitleaks#installation for more options)"
- exit 1
+ echo "🔍 Checking directory changes for existing branch: $BRANCH_NAME"
+ # for DIR in "${DIRECTORIES[@]}"; do
+ # if git diff --name-only --cached origin/"$BRANCH_NAME" | grep -q "^$DIR/"; then
+ # run_backend_checks "$DIR"
+ # fi
+ # done
fi
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
deleted file mode 100644
index f498cd623a..0000000000
--- a/.pre-commit-config.yaml
+++ /dev/null
@@ -1,14 +0,0 @@
-repos:
- - repo: local
- hooks:
- - id: gitleaks-pre-commit
- name: gitleaks git (staged only)
- entry: bash -c 'gitleaks --config .gitleaks.toml --exit-code 1 --verbose git --staged'
- language: system
- pass_filenames: false
- - id: gitleaks-pre-push
- name: gitleaks git (pre-push, scan diff)
- entry: bash -c 'gitleaks --config .gitleaks.toml --exit-code 1 --verbose git --log-opts "$(git merge-base HEAD "origin/$(git rev-parse --abbrev-ref HEAD)" 2>/dev/null || git merge-base HEAD origin/main)..HEAD"'
- language: system
- stages: [pre-push]
- pass_filenames: false
diff --git a/AGENTS.md b/AGENTS.md
index c32182aba6..5707cd532d 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -3,7 +3,6 @@
## Dev Environment Tips
- If you make changes to the frontend, make sure to run `pnpm format-fix` within the web folder
- If you make changes to the backend or sdk, make sure to run black within the sdk or api folder
-- If you update Ant Design tokens, run `pnpm generate:tailwind-tokens` in the web folder and commit the generated file
## Testing Instructions
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index 5810643c7c..5cfd9357c1 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -40,9 +40,4 @@ We had many zombie issues and PRs (assigned but inactive) in the past. We want t
- An issue may only be assigned to one person for up to one week (three days for very simple issues). If the issue remains unsolved after a week, it will be unassigned and made available to others.
- Any pull request (PR) left inactive by the author for over a week will be closed. The author can reopen it if they wish to continue.
-We look forward to seeing your contributions to Agenta!
-
-## Contributor License Agreement
-If you want to contribute, we need you to sign a Contributor License Agreement. We need this to avoid potential intellectual property problems in the future. You can sign the agreement by clicking a button. Here is how it works:
-
-After you open a PR, a bot will automatically comment asking you to sign the agreement. Click on the link in the comment, login with your Github account, and sign the agreement.
\ No newline at end of file
+We look forward to seeing your contributions to Agenta!
\ No newline at end of file
diff --git a/LICENSE b/LICENSE
index 1fff9c4444..79b3725428 100644
--- a/LICENSE
+++ b/LICENSE
@@ -1,17 +1,7 @@
-Copyright (c) 2023–2025
-Agentatech UG (haftungsbeschränkt), doing business as “Agenta”
-
-Portions of this software are licensed as follows:
-
-- All content that resides under any "ee/" directory of this repository, if
-such directories exist, are licensed under the license defined in "ee/LICENSE".
-- All third party components incorporated into the Agenta Software are licensed
-under the original license provided by the owner of the applicable component.
-- Content outside of the above mentioned directories or restrictions above is
-available under the "MIT Expat" license as defined below.
-
The MIT License
+Copyright (c) Agentatech UG (haftungsbeschränkt)
+
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
diff --git a/README.md b/README.md
index d07ac59e52..082ea25f29 100644
--- a/README.md
+++ b/README.md
@@ -2,12 +2,11 @@
-
-We've made lots of improvements to the playground. Here are some of the highlights:
-
-## JSON Editor Improvements
-
-**Enhanced Error Display and Editing**
-
-The JSON editor now provides clearer error messages and improved editing functionality. We've fixed issues with error display that previously made it difficult to debug JSON configuration problems.
-
-**Undo Support with Ctrl+Z**
-
-You can now use Ctrl+Z (or Cmd+Z on Mac) to undo changes in the JSON editor, making it much easier to iterate on complex JSON configurations without fear of losing your work.
-
-**Bug Fix: JSON Field Order Preservation**
-
-The structured output JSON field order is now preserved throughout the system. This is crucial when working with LLMs that are sensitive to the ordering of JSON fields in their responses.
-
-Previously, JSON objects might have their field order changed during processing, which could affect LLM behavior and evaluation consistency. Now, the exact order you define is maintained across all operations.
-
-## Playground Improvements
-
-**Dynamic variables**
-
-We've improved the editor behavior with dynamic variables in the prompt.
-
-**Markdown and Text View Toggle**
-
-You can now switch between markdown and text view for messages.
-
-**Collapsible Interface Elements**
-
-We've added the ability to collapse various sections of the playground interface, helping you focus on what matters most for your current task.
-
-**Collapsible Test Cases for Large Sets**
-
-When loading large test sets, you can now collapse individual test cases to better manage the interface.
-
-**Visual diff when committing changes**
-
-The playground now shows a visual diff when you're committing changes, making it easy to review exactly what modifications you're about to save.
-
-
-
-
----
diff --git a/docs/blog/entries/more-reliable-evaluations.mdx b/docs/blog/entries/more-reliable-evaluations.mdx
index 7c1287fb95..c5bb504702 100644
--- a/docs/blog/entries/more-reliable-evaluations.mdx
+++ b/docs/blog/entries/more-reliable-evaluations.mdx
@@ -32,6 +32,8 @@ We have worked extensively on improving the **reliability of evaluations**. Spec
- We fixed small UI issues with large output in human evaluations.
- We have added a new export button in the evaluation view to export the results as a CSV file.
+Additionally, we have added a new [Cookbook for run evaluation using the SDK](/tutorials/sdk/evaluate-with-SDK).
+
In **observability**:
- We have added a **new integration with [Litellm](https://litellm.ai/)** to automatically trace all LLM calls done through it.
diff --git a/docs/blog/entries/multiple-metrics-in-human-evaluation.mdx b/docs/blog/entries/multiple-metrics-in-human-evaluation.mdx
deleted file mode 100644
index 203f3c8db9..0000000000
--- a/docs/blog/entries/multiple-metrics-in-human-evaluation.mdx
+++ /dev/null
@@ -1,27 +0,0 @@
----
-title: "Multiple Metrics in Human Evaluation"
-slug: multiple-metrics-in-human-evaluation
-date: 2025-09-09
-tags: [v0.51.0]
----
-
-We spent the past months rethinking how evaluation should work. Today we're announcing one of the first big improvements.
-
-The fastest teams building LLM apps were using human evaluation to check their outputs before going live. Agenta was helping them do this in minutes.
-
-But we also saw that they were limited. You could only score the outputs with one metric.
-
-That's why we rebuilt the human evaluation workflow.
-
-Now you can set multiple evaluators and metrics and use them to score the outputs. This lets you evaluate the same output on different metrics like **relevance** or **completeness**. You can also create binary, numerical scores, or even use strings for **comments** or **expected answer**.
-
-This unlocks a whole new set of use cases:
-
-- Compare your prompts on multiple metrics and understand where you can improve.
-- Turn your annotations into test sets and use them in prompt engineering. For instance, you can add comments that help you later in improve your prompts.
-- Use human evaluation to bootstrap automatic evaluation. You can annotate your outputs with the expected answer or a rubic, then use it to set up an automatic evaluation.
-
-
-Watch the video below and read the [post](/changelog/multiple-metrics-in-human-evaluation) for more details. Or check out the [docs](/evaluation/human-evaluation/quick-start) to learn how to use the new human evaluation workflow.
-
----
\ No newline at end of file
diff --git a/docs/blog/entries/new-evaluation-results-dashboard.mdx b/docs/blog/entries/new-evaluation-results-dashboard.mdx
deleted file mode 100644
index 8eb0b740ec..0000000000
--- a/docs/blog/entries/new-evaluation-results-dashboard.mdx
+++ /dev/null
@@ -1,41 +0,0 @@
----
-title: "New Evaluation Results Dashboard"
-slug: new-evaluation-results-dashboard
-date: 2025-09-26
-tags: [v0.54.0]
----
-
-We rebuilt the evaluation results dashboard. Now you can check your results faster and see how well your AI performs.
-
-## What's New
-
-### **Charts and Graphs**
-We added charts that show your AI's performance. You can quickly spot problems and see patterns in your data.
-
-### **Compare Results Side by Side**
-Compare multiple tests at once. See which prompts or models work better. View charts and detailed results together.
-
-### **Better Results Table**
-Results now show in a clean table format. It works great for small tests (10 cases) and big tests (10,000+ cases). The page loads fast no matter how much data you have.
-
-### **Detailed View**
-Click on any result to see more details. Find out why a test passed or failed. Get the full picture of what happened.
-
-### **See Your Settings**
-Check exactly which settings you used for each test. This helps you repeat successful tests and understand your results better.
-
-### **Name Your Tests**
-Give your tests names and descriptions. Stay organized and help your team understand what each test does.
-
-
-
-
-
diff --git a/docs/blog/entries/new-onboarding-flow.mdx b/docs/blog/entries/new-onboarding-flow.mdx
index 189ffb75e9..e2cdc0ac28 100644
--- a/docs/blog/entries/new-onboarding-flow.mdx
+++ b/docs/blog/entries/new-onboarding-flow.mdx
@@ -38,4 +38,6 @@ We've redesigned our platform's onboarding to make getting started simpler and m
- Enhanced frontend performance
- Fixed scroll behavior in trace view
+You can check out the tutorial for the RAG demo project [here](/tutorials/cookbooks/RAG-QA-docs).
+
---
diff --git a/docs/blog/entries/observability-and-prompt-management.mdx b/docs/blog/entries/observability-and-prompt-management.mdx
index e36a5960d2..d6cbb6d1a2 100644
--- a/docs/blog/entries/observability-and-prompt-management.mdx
+++ b/docs/blog/entries/observability-and-prompt-management.mdx
@@ -40,11 +40,11 @@ We’ll publish a full blog post soon, but here’s a quick look at what the new
**Next: Prompt Management**
-We’ve completely rewritten the [prompt management SDK](/prompt-engineering/managing-prompts-programatically/setup), giving you full CRUD capabilities for prompts and configurations. This includes creating, updating, reading history, deploying new versions, and deleting old ones. You can find a first tutorial for this [here](/tutorials/sdk/manage-prompts-with-SDK).
+We’ve completely rewritten the [prompt management SDK](/prompt-engineering/overview), giving you full CRUD capabilities for prompts and configurations. This includes creating, updating, reading history, deploying new versions, and deleting old ones. You can find a first tutorial for this [here](/tutorials/sdk/manage-prompts-with-SDK).
**And finally: LLM-as-a-Judge Overhaul**
-We've made significant upgrades to the [LLM-as-a-Judge evaluator](/evaluation/configure-evaluators/llm-as-a-judge). It now supports prompts with multiple messages and has access to all variables in a test case. You can also switch models (currently supporting OpenAI and Anthropic). These changes make the evaluator much more flexible, and we're seeing better results with it.
+We’ve made significant upgrades to the [LLM-as-a-Judge evaluator](/evaluation/evaluators/llm-as-a-judge). It now supports prompts with multiple messages and has access to all variables in a test case. You can also switch models (currently supporting OpenAI and Anthropic). These changes make the evaluator much more flexible, and we’re seeing better results with it.
-
-### [Documentation Overhaul](/changelog/documentation-architecture-overhaul)
-
-_3 November 2025_
-
-**v0.59.10**
-
-
-
-We've completely rewritten and restructured our documentation with a new architecture. This is one of the largest updates we've made, involving a near-complete rewrite of existing content.
-
-Key improvements include:
-- **[Diataxis Framework](https://diataxis.fr/)**: Organized content into Tutorials, How-to Guides, Reference, and Explanation sections for better discoverability
-- **[Expanded Observability Docs](/observability/overview)**: Added missing documentation for tracing, annotations, and observability features
-- **[JavaScript/TypeScript Support](/observability/quick-start-opentelemetry)**: Added code examples and documentation for JavaScript developers alongside Python
-- **Ask AI Feature**: Ask questions directly to the documentation for instant answers
-
----
-
-### [Vertex AI Provider Support](/changelog/vertex-ai-provider-support)
-
-_24 October 2025_
-
-**v0.59.6**
-
-We've added support for Google Cloud's Vertex AI platform. You can now use Gemini models and other Vertex AI partner models in the playground, configure them in the Model Hub, and access them through the Gateway using InVoke endpoints.
-
-Check out the documentation for [configuring Vertex AI models](/prompt-engineering/playground/custom-providers#configuring-vertex-ai).
-
----
-
-### [Filtering Traces by Annotation](/changelog/filtering-traces-by-annotation)
-
-_14 October 2025_
-
-**v0.58.0**
-
-
-
-You can now filter and search traces based on their annotations. This helps you find traces with low scores or bad feedback quickly.
-
-We rebuilt the filtering system in observability with a simpler dropdown and more options. You can now filter by span status, input keys, app or environment references, and any key within your span.
-
-The new annotation filtering lets you find:
-- Spans evaluated by a specific evaluator
-- Spans with user feedback like `success=True`
-
-
-This enables powerful workflows: [capture user feedback](/tutorials/cookbooks/capture-user-feedback) from your app, filter to find traces with bad feedback, add them to test sets, and improve your prompts based on real user data.
-
----
-
-### [New Evaluation Results Dashboard](/changelog/new-evaluation-results-dashboard)
-
-_26 September 2025_
-
-**v0.54.0**
-
-We've completely redesigned the evaluation results dashboard. You can analyse your evaluation results more easily and understand performance across different metrics.
-
-Here's what's new:
-
-- **Metrics plots**: We've added plots for all the evaluator metrics. You can not see the distribution of the results and easily spot outliers.
-- **Side-by-side comparison**: You can now compare multiple evaluations simultaneously. You can compare the plots but also the single outputs.
-- **Improved test cases view**: The results are now displayed in a tabular format works both for small and large datasets.
-- **Focused detail view**: A new focused drawer lets you examine individual data points in more details. It's very helpful if your data is large.
-- **Configuration view**: See exactly which configurations were used in each evaluation
-- **Evaluation Run naming and descriptions**: Add names and descriptions to your evaluation runs to organize things better.
-
-
-
-
-
----
-
-### [Deep URL Support for Sharable Links](/changelog/deep-url-support-for-sharable-links)
-
-_24 September 2025_
-
-**v0.53.0**
-
-URLs across Agenta now include workspace context, making them fully shareable between team members. Previously, URLs would always point to the default workspace, causing issues when refreshing pages or sharing links.
-
-Now you can deep link to almost anything in the platform - prompts, evaluations, and more - in any workspace. Share links directly with team members and they'll see exactly what you intended, regardless of their default workspace settings.
-
----
-
-### [Major Speed Improvements and Bug Fixes](/changelog/speed-improvements-in-the-playground)
-
-_19 September 2025_
-
-**v0.52.5**
-
-We rewrote most of Agenta's frontend. You'll see much faster speeds when you create prompts or use the playground.
-
-We also made many improvements and fixed bugs:
-
-**Improvements:**
-
-- [LLM-as-a-judge](/evaluation/configure-evaluators/llm-as-a-judge) now uses double curly braces `{{}}` instead of single curly braces `{` and `}`. This matches how normal prompts work. Old LLM-as-a-judge prompts with single curly braces still work. We updated the LLM-as-a-judge playground to make editing prompts easier.
-
-**Self-hosting:**
-
-- You can now use [an external Redis instance](/self-host/configuration#redis-caching) for caching by setting it as an environment variable
-
-**Bug fixes:**
-
-- Fixed the [custom workflow quick start tutorial](/custom-workflows/quick-start) and examples
-- Fixed SDK compatibility issues with Python 3.9
-- Fixed default filtering in observability dashboard
-- Fixed error handling in the evaluator playground
-- Fixed the Tracing SDK to allow instrumenting streaming responses and overriding OTEL environment variables
----
-
-### [Multiple Metrics in Human Evaluation](/changelog/multiple-metrics-in-human-evaluation)
-
-_9 September 2025_
-
-**v0.51.0**
-
-We rebuilt the human evaluation workflow from scratch. Now you can set multiple evaluators and metrics and use them to score the outputs.
-
-This lets you evaluate the same output on different metrics like **relevance** or **completeness**. You can also create binary, numerical scores, or even use strings for **comments** or **expected answer**.
-
-Watch the video below and read the [post](/changelog/multiple-metrics-in-human-evaluation) for more details. Or check out the [docs](/evaluation/human-evaluation/quick-start) to learn how to use the new human evaluation workflow.
-
-
-
-
-
----
-
-### [DSPy Integration](/observability/integrations/dspy)
-
-_29 August 2025_
-
-We've added DSPy integration to Agenta. You can now trace and debug your DSPy applications with Agenta.
-
-
-
-[**View the full DSPy integration →**](/observability/integrations/dspy)
-
----
-
-### [Open-sourcing our Product Roadmap](/roadmap)
-
-_12 August 2025_
-
-We've made our product roadmap completely transparent and community-driven.
-
-You can now see exactly what we're building, what's shipped, and what's coming next. Plus vote on features that matter most to you.
-
-**Why we're doing this:** We believe open-source startups succeed when they create the most value possible, and the best way to do that is by building with our community, not in isolation. Up until now, we've been secretive with our roadmap, but we're losing something important: your feedback and the ability to let you shape our direction. Today we're open-sourcing our roadmap because we want to build a community of owners, not just passive users.
-
-[**View the full roadmap →**](/roadmap)
-
----
-
-### [Major Playground Improvements and Enhancements](/changelog/major-playground-improvements-and-enhancements)
-_7 August 2025_
-**v0.50.5**
-
-We've made significant improvements to the playground. Key features include:
-- Improving the error handling in JSON editor for structured output
-- Preventing the JSON field order from being changed
-- Visual diff when committing changes
-- Markdown and text view toggle
-- Collapsible interface elements
-- Collapsible test cases for large sets
-
-
-
-
-
-
----
-
### [Support for Images in the Playground](/changelog/support-for-images-in-playground)
_29 July 2025_
**v0.50.0**
@@ -270,7 +61,7 @@ This is useful to:
- Run custom evaluation workflows
- Measure application performance in real-time
-Check out the how to [annotate traces from API](/observability/trace-with-python-sdk/annotate-traces) for more details. Or try our new tutorial (available as [jupyter notebook](https://github.com/Agenta-AI/agenta/blob/main/examples/jupyter/capture_user_feedback.ipynb)) [here](/tutorials/cookbooks/capture-user-feedback).
+Check out the how to [annotate traces from API](/evaluation/annotate-api) for more details. Or try our new tutorial (available as [jupyter notebook](https://github.com/Agenta-AI/agenta/blob/main/examples/jupyter/capture_user_feedback.ipynb)) [here](/tutorials/cookbooks/capture-user-feedback).
+
+
+
+
+
+
+
+
+
+
+### Run evaluations
+
+
+
+
+
+
+
+
+
+
+
+
+### Available evaluators
+
+| **Evaluator Name** | **Use Case** | **Type** | **Description** |
+| ------------------------------------------------------------------------------------------------- | -------------------------------- | ------------------ | -------------------------------------------------------------------------------- |
+| [Exact Match](/evaluation/evaluators/classification-entiry-extraction#exact-match) | Classification/Entity Extraction | Pattern Matching | Checks if the output exactly matches the expected result. |
+| [Contains JSON](/evaluation/evaluators/classification-entiry-extraction#contains-json) | Classification/Entity Extraction | Pattern Matching | Ensures the output contains valid JSON. |
+| [Regex Test](/evaluation/evaluators/pattern-matching#regular-expression) | Classification/Entity Extraction | Pattern Matching | Checks if the output matches a given regex pattern. |
+| [JSON Field Match](/evaluation/evaluators/classification-entiry-extraction#json-field-match) | Classification/Entity Extraction | Pattern Matching | Compares specific fields within JSON data. |
+| [JSON Diff Match](/evaluation/evaluators/classification-entiry-extraction#json-diff-match) | Classification/Entity Extraction | Similarity Metrics | Compares generated JSON with a ground truth JSON based on schema or values. |
+| [Similarity Match](/evaluation/evaluators/semantic-similarity#similarity-match) | Text Generation / Chatbot | Similarity Metrics | Compares generated output with expected using Jaccard similarity. |
+| [Semantic Similarity Match](/evaluation/evaluators/semantic-similarity#semantic-similarity-match) | Text Generation / Chatbot | Semantic Analysis | Compares the meaning of the generated output with the expected result. |
+| [Starts With](/evaluation/evaluators/pattern-matching#starts-with) | Text Generation / Chatbot | Pattern Matching | Checks if the output starts with a specified prefix. |
+| [Ends With](/evaluation/evaluators/pattern-matching#ends-with) | Text Generation / Chatbot | Pattern Matching | Checks if the output ends with a specified suffix. |
+| [Contains](/evaluation/evaluators/pattern-matching#contains) | Text Generation / Chatbot | Pattern Matching | Checks if the output contains a specific substring. |
+| [Contains Any](/evaluation/evaluators/pattern-matching#contains-any) | Text Generation / Chatbot | Pattern Matching | Checks if the output contains any of a list of substrings. |
+| [Contains All](/evaluation/evaluators/pattern-matching#contains-all) | Text Generation / Chatbot | Pattern Matching | Checks if the output contains all of a list of substrings. |
+| [Levenshtein Distance](/evaluation/evaluators/semantic-similarity#levenshtein-distance) | Text Generation / Chatbot | Similarity Metrics | Calculates the Levenshtein distance between output and expected result. |
+| [LLM-as-a-judge](/evaluation/evaluators/llm-as-a-judge) | Text Generation / Chatbot | LLM-based | Sends outputs to an LLM model for critique and evaluation. |
+| [RAG Faithfulness](/evaluation/evaluators/rag-evaluators) | RAG / Text Generation / Chatbot | LLM-based | Evaluates if the output is faithful to the retrieved documents in RAG workflows. |
+| [RAG Context Relevancy](/evaluation/evaluators/rag-evaluators) | RAG / Text Generation / Chatbot | LLM-based | Measures the relevancy of retrieved documents to the given question in RAG. |
+| [Custom Code Evaluation](/evaluation/evaluators/custom-evaluator) | Custom Logic | Custom | Allows users to define their own evaluator in Python. |
+| [Webhook Evaluator](/evaluation/evaluators/webhook-evaluator) | Custom Logic | Custom | Sends output to a webhook for external evaluation. |
diff --git a/docs/docs/evaluation/01-quick-start-ui.mdx b/docs/docs/evaluation/01-quick-start-ui.mdx
deleted file mode 100644
index cbcdacc035..0000000000
--- a/docs/docs/evaluation/01-quick-start-ui.mdx
+++ /dev/null
@@ -1,10 +0,0 @@
----
-title: "Quick Start: Evaluation from UI"
-sidebar_label: "Quick Start (UI)"
-description: "Get started with evaluating your LLM applications using the Agenta web interface"
-sidebar_position: 1
----
-
-import { Redirect } from '@docusaurus/router';
-
-
diff --git a/docs/docs/evaluation/02-create-test-sets.mdx b/docs/docs/evaluation/02-create-test-sets.mdx
new file mode 100644
index 0000000000..dd5af47f99
--- /dev/null
+++ b/docs/docs/evaluation/02-create-test-sets.mdx
@@ -0,0 +1,183 @@
+---
+title: "Create Test Sets"
+---
+
+```mdx-code-block
+import { Stream } from '@cloudflare/stream-react';
+import Image from "@theme/IdealImage";
+```
+
+This guide outlines the various methods for creating test sets in Agenta and provides specifications for the test set schema.
+
+Test sets are one of the most critical components for building reliable LLM-powered applications. They allow you to evaluate your application, find edge cases, prevent regressions, and systematically improve performance over time.
+
+## What is a Test Set?
+
+A test set is a collection of test cases, each containing:
+
+- **Inputs**: The data your LLM application expects (required)
+- **Ground Truth**: The expected answer from your application (optional, often stored as "correct_answer")
+- **Annotations**: Additional metadata or rules about the test case (optional)
+
+You can create a test set in Agenta using the following methods:
+
+- [By uploading a CSV or JSON file](#creating-a-test-set-from-a-csv-or-json)
+- [Using the API](#creating-a-test-set-using-the-api)
+- [Using the UI](#creatingediting-a-test-set-from-the-ui)
+- [From the playground](#creating-a-test-set-from-the-playground)
+- [From traces in observability](#adding-data-from-traces)
+
+## Creating a Test Set from a CSV or JSON
+
+To create a test set from a CSV or JSON file:
+
+1. Go to `Test sets`
+2. Click `Upload test sets`
+3. Select either `CSV` or `JSON`
+
+
+
+### CSV Format
+
+We use CSV with commas (,) as separators and double quotes (") as quote characters. The first row should contain the header with column names. Each input should have its own column. The column containing the reference answer can have any name, but we use "correct_answer" by default.
+
+:::info
+If you choose a different column name for the reference answer, you'll need to configure the evaluator later with that specific name.
+:::
+
+Here's an example of a valid CSV:
+
+```csv
+text,instruction,correct_answer
+Hello,How are you?,I'm good.
+"Tell me a joke.",Sure, here's one:...
+```
+
+### JSON Format
+
+The test set should be in JSON format with the following structure:
+
+1. A JSON file containing an array of objects.
+2. Each object in the array represents a row, with keys as column headers and values as row data. Here's an example of a valid JSON file:
+
+```json
+[
+ { "recipe_name": "Chicken Parmesan", "correct_answer": "Chicken" },
+ { "recipe_name": "a, special, recipe", "correct_answer": "Beef" }
+]
+```
+
+### Schema for Chat Applications
+
+For chat applications created using the chat template in Agenta, the input should be saved in the column called `chat`, which would contain the input list of messages:
+
+```json
+[
+ { "content": "message.", "role": "user" },
+ { "content": "message.", "role": "assistant" }
+ // Add more messages if necessary
+]
+```
+
+The reference answer column (by default `correct_answer`) should follow the same format:
+
+```json
+{ "content": "message.", "role": "assistant" }
+```
+
+## Creating a Test Set Using the API
+
+You can upload a test set using our API. Find the [API endpoint reference here](/reference/api/upload-file).
+
+Here's an example of such a call:
+
+**HTTP Request:**
+
+```
+POST /testsets
+
+```
+
+**Request Body:**
+
+```json
+{
+ "name": "testsetname",
+ "csvdata": [
+ { "column1": "row1col1", "column2": "row1col2" },
+ { "column1": "row2col1", "column2": "row2col2" }
+ ]
+}
+```
+
+## Creating/Editing a Test Set from the UI
+
+To create or edit a test set from the UI:
+
+1. Go to `Test sets`
+2. Choose `Create a test set with UI` or select the test set
+3. Name your test set and specify the columns for input types.
+4. Add the dataset.
+
+Remember to click `Save test set`
+
+
+
+## Creating a Test Set from the Playground
+
+
+
+
+The playground offers a convenient way to create and add data to a test set. This workflow is useful when you discover interesting cases or edge cases while experimenting with your LLM application.
+
+To add a data point to a test set from the playground:
+
+1. Work with your application in the playground
+2. When you find an interesting case, click the `Add to test set` button located near the `Run` button
+3. A drawer will display showing the inputs and outputs from the playground
+4. You can modify inputs and correct answers if needed
+5. Select an existing test set to add to, or choose `+Add new` to create a new one
+6. Once you're satisfied, click `Add` to finalize
+
+## Adding Data From Traces
+
+One of the most valuable sources of test cases is your production data. Traces captured in the Observability view represent real user interactions with your LLM application.
+
+
+
+
+### Adding a Single Trace
+
+To add a single trace to a test set:
+
+1. Navigate to the **Observability** view in Agenta
+2. Find a trace you want to add to a test set
+3. Click the **Add to test set** button at the top of the trace
+4. Choose to create a new test set or select an existing one
+5. Review the mapping between trace data and test set columns
+ - Agenta will automatically map the inputs and outputs to appropriate columns
+ - You can edit the expected answer if you don't agree with the output
+6. Click **Save** to add the trace to your test set
+
+### Adding Multiple Traces at Once
+
+To efficiently add multiple traces:
+
+1. In the Observability view, use the search function to filter traces
+ - For example, search for specific response patterns like "I don't have enough information"
+2. Select all relevant traces by checking the boxes next to them
+3. Click **Add to test set**
+4. Choose an existing test set or create a new one
+5. Review the mapping for the traces
+6. Click **Save** to add all selected traces to your test set
+
+## Using Your Test Sets
+
+Once you have created test sets, you can use them for:
+
+1. **Playground Iteration**: Load test sets in the playground to test and refine your prompts
+2. **Automated Evaluation**: Run systematic evaluations comparing outputs against expected answers
+3. **Human Evaluation**: Collect human feedback on your application's performance
+4. **Regression Testing**: Ensure new changes don't break existing functionality
+
+To learn more about using your test sets for evaluation, see our [Evaluation documentation](/evaluation/overview).
diff --git a/docs/docs/evaluation/03-concepts.mdx b/docs/docs/evaluation/03-concepts.mdx
deleted file mode 100644
index e781a28de1..0000000000
--- a/docs/docs/evaluation/03-concepts.mdx
+++ /dev/null
@@ -1,56 +0,0 @@
----
-title: "Concepts"
-description: "Understand the key concepts of LLM evaluation in Agenta"
-sidebar_position: 3
----
-
-import Image from "@theme/IdealImage";
-
-## What is evaluation?
-
-The key to building production-ready LLM applications is to have a tight feedback loop of prompt engineering and evaluation. Whether you are optimizing a chatbot, working on Retrieval-Augmented Generation (RAG), or fine-tuning a text generation task, evaluation is a critical step to ensure consistent performance across different inputs, models, and parameters.
-
-## Key concepts
-
-### Evaluators
-
-Evaluators are functions that assess the output of an LLM application.
-
-Evaluators typically take as input:
-
-- The output of the LLM application
-- (Optional) The reference answer (i.e., expected output or ground truth)
-- (Optional) The inputs to the LLM application
-- Any other relevant data, such as context
-
-Evaluators return different types of results based on the evaluator type. Simple evaluators return single values like boolean (true/false) or numeric scores. Evaluators with schemas (such as LLM-as-a-Judge or custom evaluators) can return structured results with multiple fields, allowing you to capture various aspects of the evaluation in a single result.
-
-
-### Test sets
-
-Test sets are collections of test cases used to evaluate your LLM application. Each test case contains:
-
-- **Inputs**: The data your LLM application expects (required)
-- **Ground Truth**: The expected answer from your application (optional, often stored as "correct_answer")
-- **Annotations**: Additional metadata or rules about the test case (optional)
-
-Test sets are critical for:
-- Evaluating your application systematically
-- Finding edge cases
-- Preventing regressions
-- Measuring improvements over time
-
-### Evaluation workflows
-
-Agenta supports multiple evaluation workflows:
-
-1. **Automated Evaluation (UI)**: Run evaluations from the web interface with configurable evaluators
-2. **Automated Evaluation (SDK)**: Run evaluations programmatically for integration into CI/CD pipelines
-3. **Online Evaluation**: Run evaluations on new traces as they are generated by your LLM application
-4. **Human Evaluation**: Collect expert feedback and annotations for qualitative assessment
-
-## Next steps
-
-- [Configure evaluators](/evaluation/configure-evaluators/overview) for your use case
-- [Create test sets](/evaluation/managing-test-sets/upload-csv) from various sources
-- [Run your first evaluation](/evaluation/evaluation-from-ui/running-evaluations) from the UI
diff --git a/docs/docs/evaluation/03-configure-evaluators.mdx b/docs/docs/evaluation/03-configure-evaluators.mdx
new file mode 100644
index 0000000000..04973265ec
--- /dev/null
+++ b/docs/docs/evaluation/03-configure-evaluators.mdx
@@ -0,0 +1,87 @@
+---
+title: "Configure Evaluators"
+description: "Set up evaluators for your use case"
+---
+
+import Image from "@theme/IdealImage";
+
+In this guide will show you how to configure evaluators for your LLM application.
+
+### What are evaluators?
+
+Evaluators are functions that assess the output of an LLM application.
+
+Evaluators typically take as input:
+
+- The output of the LLM application
+- (Optional) The reference answer (i.e., expected output or ground truth)
+- (Optional) The inputs to the LLM application
+- Any other relevant data, such as context
+
+Evaluators return either a float or a boolean value.
+
+
+
+### Configuring evaluators
+
+To create a new evaluator, click on the `Configure Evaluators` button in the `Evaluations` view.
+
+
+
+### Selecting evaluators
+
+Agenta offers a growing list of pre-built evaluators suitable for most use cases. We also provide options for [creating custom evaluators](/evaluation/evaluators/custom-evaluator) (by writing your own Python function) or [using webhooks](/evaluation/evaluators/webhook-evaluator) for evaluation.
+
+
+Available Evaluators
+
+| **Evaluator Name** | **Use Case** | **Type** | **Description** |
+| ------------------------------------------------------------------------------------------------- | -------------------------------- | ------------------ | -------------------------------------------------------------------------------- |
+| [Exact Match](/evaluation/evaluators/classification-entiry-extraction#exact-match) | Classification/Entity Extraction | Pattern Matching | Checks if the output exactly matches the expected result. |
+| [Contains JSON](/evaluation/evaluators/classification-entiry-extraction#contains-json) | Classification/Entity Extraction | Pattern Matching | Ensures the output contains valid JSON. |
+| [Regex Test](/evaluation/evaluators/pattern-matching#regular-expression) | Classification/Entity Extraction | Pattern Matching | Checks if the output matches a given regex pattern. |
+| [JSON Field Match](/evaluation/evaluators/classification-entiry-extraction#json-field-match) | Classification/Entity Extraction | Pattern Matching | Compares specific fields within JSON data. |
+| [JSON Diff Match](/evaluation/evaluators/classification-entiry-extraction#json-diff-match) | Classification/Entity Extraction | Similarity Metrics | Compares generated JSON with a ground truth JSON based on schema or values. |
+| [Similarity Match](/evaluation/evaluators/semantic-similarity#similarity-match) | Text Generation / Chatbot | Similarity Metrics | Compares generated output with expected using Jaccard similarity. |
+| [Semantic Similarity Match](/evaluation/evaluators/semantic-similarity#semantic-similarity-match) | Text Generation / Chatbot | Semantic Analysis | Compares the meaning of the generated output with the expected result. |
+| [Starts With](/evaluation/evaluators/pattern-matching#starts-with) | Text Generation / Chatbot | Pattern Matching | Checks if the output starts with a specified prefix. |
+| [Ends With](/evaluation/evaluators/pattern-matching#ends-with) | Text Generation / Chatbot | Pattern Matching | Checks if the output ends with a specified suffix. |
+| [Contains](/evaluation/evaluators/pattern-matching#contains) | Text Generation / Chatbot | Pattern Matching | Checks if the output contains a specific substring. |
+| [Contains Any](/evaluation/evaluators/pattern-matching#contains-any) | Text Generation / Chatbot | Pattern Matching | Checks if the output contains any of a list of substrings. |
+| [Contains All](/evaluation/evaluators/pattern-matching#contains-all) | Text Generation / Chatbot | Pattern Matching | Checks if the output contains all of a list of substrings. |
+| [Levenshtein Distance](/evaluation/evaluators/semantic-similarity#levenshtein-distance) | Text Generation / Chatbot | Similarity Metrics | Calculates the Levenshtein distance between output and expected result. |
+| [LLM-as-a-judge](/evaluation/evaluators/llm-as-a-judge) | Text Generation / Chatbot | LLM-based | Sends outputs to an LLM model for critique and evaluation. |
+| [RAG Faithfulness](/evaluation/evaluators/rag-evaluators) | RAG / Text Generation / Chatbot | LLM-based | Evaluates if the output is faithful to the retrieved documents in RAG workflows. |
+| [RAG Context Relevancy](/evaluation/evaluators/rag-evaluators) | RAG / Text Generation / Chatbot | LLM-based | Measures the relevancy of retrieved documents to the given question in RAG. |
+| [Custom Code Evaluation](/evaluation/evaluators/custom-evaluator) | Custom Logic | Custom | Allows users to define their own evaluator in Python. |
+| [Webhook Evaluator](/evaluation/evaluators/webhook-evaluator) | Custom Logic | Custom | Sends output to a webhook for external evaluation. |
+
+
+
+
+
+## Evaluators' settings
+
+Each evaluator comes with it's unique settings. For instance in the screen below, the JSON field match evaluator requires you to specify which field in the output JSON you need to consider for evaluation. You'll find detailed information about these parameters on each evaluator's documentation page.
+
+
+
+## Mappings evaluator's inputs to the LLM data
+
+Evaluators need to know which parts of the data contain the output and the reference answer. Most evaluators allow you to configure this mapping, typically by specifying the name of the column in the test set that contains the `reference answer`.
+
+For more sophisticated evaluators, such as `RAG evaluators` (_available only in cloud and enterprise versions_), you need to define more complex mappings (see figure below).
+
+
+
+Configuring the evaluator is done by mapping the evaluator inputs to the generation data:
+
+
diff --git a/docs/docs/evaluation/04-no-code-evaluation.mdx b/docs/docs/evaluation/04-no-code-evaluation.mdx
new file mode 100644
index 0000000000..0323af74bf
--- /dev/null
+++ b/docs/docs/evaluation/04-no-code-evaluation.mdx
@@ -0,0 +1,68 @@
+---
+title: "No-code Evaluation"
+description: Run evaluation from the UI.
+---
+
+import Image from "@theme/IdealImage";
+
+This guide will show you how to run evaluations from the UI.
+
+Before you get started, make sure that you have [created a test set](/evaluation/create-test-sets) and [configured evaluators](/evaluation/configure-evaluators) appropriate for your task.
+
+## Running Evaluations
+
+To start an evaluation, navigate to the Evaluations page and click the `Start new evaluation` button. A modal will appear, allowing you to setup the evaluation.
+
+
+
+### Setting Up Evaluation Parameters
+
+In the modal, specify the following:
+
+- Testset: Choose the testset(s) for your evaluation.
+- Variants: Choose one or more variants to evaluate.
+- Evaluators: Pick one or more evaluators for assessment.
+
+
+
+#### Advanced Configuration
+
+Additional settings allow you to adjust batching and retry parameters for LLM calls. This help mitigating rate limit errors from your LLM provider.
+
+Advanced configuration options include:
+
+- **Batch Size:** Number of test cases to run concurrently in each batch (default: 10).
+- **Retry Delay:** Time to wait before retrying a failed call (default: 3s).
+- **Max Retries:** Maximum number of retry attempts for a failed call (default: 3).
+- **Delay Between Batches:** Pause duration between batch runs (default: 5s).
+
+## Analyzing Evaluation Results
+
+The main view offers an aggregated summary of results. Each column displays the average score per evaluator for each variant/test set combination. You'll also see average latency, total cost, creation date, and evaluation status.
+
+For a detailed view of an evaluation, click on a completed evaluation row.
+
+
+
+The evaluation table columns show inputs, reference answers used by evaluators, LLM application output, evaluator results, cost, and latency.
+
+## Comparing Evaluations
+
+Once evaluations are marked "completed," you can compare two or more evaluations from the same test set. Click the `Compare` button to access the Evaluation comparison view, where you can analyze outputs from multiple evaluations side by side.
+
+
diff --git a/docs/docs/evaluation/05-sdk-evaluation.mdx b/docs/docs/evaluation/05-sdk-evaluation.mdx
new file mode 100644
index 0000000000..5f50de5483
--- /dev/null
+++ b/docs/docs/evaluation/05-sdk-evaluation.mdx
@@ -0,0 +1,144 @@
+---
+title: "Evaluate from SDK"
+description: "Run evaluation programmatically from the SDK."
+---
+
+import Image from "@theme/IdealImage";
+
+This guide explains how to run evaluations programmaticaly from the SDK in agenta. We will do the following:
+
+- Create a test set
+- Create and configure an evaluator
+- Run an evaluation
+- Retrieve the results of evaluations
+
+## How agenta evaluation works
+
+In **agenta**, evaluation is a **fully managed service** . It takes place entirely on our backend, and take care of:
+
+- **Queuing** and managing evaluation jobs
+- **Batching** LLM app calls to optimize performance and avoid exceeding rate limits.
+- **Handle retries and errors** automatically to ensure robust evaluation runs.
+
+Our evaluation service takes a set of test sets, evaluators, and app variants and runs asynchronous jobs for evaluation.
+
+
+
+:::info
+You can open this guide in a [jupyter notebook](https://github.com/Agenta-AI/agenta/blob/main/cookbook/evaluations_with_sdk.ipynb)
+:::
+
+## 1. Setup
+
+1. Create an LLM app and a couple of variants in agenta and install our sdk using `pip install -U agenta`.
+
+2. Retrieve the application id either from the url.
+
+## 2. Setup the SDK client
+
+```python
+
+app_id = "667d8cfad1812781f7e375d9"
+
+# You can create the API key under the settings page. If you are using the OSS version, you should keep this as an empty string
+api_key = "EUqJGOUu.xxxx"
+
+# Host.
+host = "https://cloud.agenta.ai"
+
+# Initialize the client
+
+client = AgentaApi(base_url=host + "/api", api_key=api_key)
+```
+
+## 3. Create a test set
+
+```python
+from agenta.client.types.new_testset import NewTestset
+
+csvdata = [
+ {"country": "france", "capital": "Paris"},
+ {"country": "Germany", "capital": "Berlin"}
+ ]
+
+response = client.testsets.create_testset(request=NewTestset(name="test set", csvdata=csvdata))
+test_set_id = response.id
+
+```
+
+## 4. Create evaluators
+
+Let's create a custom code evaluator that return 1.0 if the first letter of the app output is uppercase
+
+```python
+code_snippet = """
+from typing import Dict
+
+def evaluate(
+ app_params: Dict[str, str],
+ inputs: Dict[str, str],
+ output: str, # output of the llm app
+ datapoint: Dict[str, str] # contains the testset row
+) -> float:
+ if output and output[0].isupper():
+ return 1.0
+ else:
+ return 0.0
+"""
+
+response = client.evaluators.create_new_evaluator_config(app_id=app_id, name="capital_letter_evaluator", evaluator_key="auto_custom_code_run", settings_values={"code": code_snippet})
+letter_match_eval_id = response.id
+```
+
+## 5. Run an evaluation
+
+First let's grab the first variants in the app
+
+```python
+response = client.apps.list_app_variants(app_id=app_id)
+print(response)
+myvariant_id = response[0].variant_id
+```
+
+Then , let's start the evaluation jobs
+
+```python
+from agenta.client.types.llm_run_rate_limit import LlmRunRateLimit
+
+rate_limit_config = LlmRunRateLimit(
+ batch_size=10, # number of rows to call in parallel
+ max_retries=3, # max number of time to retry a failed llm call
+ retry_delay=2, # delay before retrying a failed llm call
+ delay_between_batches=5, # delay between batches
+ )
+response = client.evaluations.create_evaluation(app_id=app_id,
+ variant_ids=[myvariant_id],
+ testset_id=test_set_id,
+ evaluators_configs=[letter_match_eval_id],
+ rate_limit=rate_limit_config)
+print(response)
+```
+
+Now we can check for the status of the job
+
+```python
+client.evaluations.fetch_evaluation_status('667d98fbd1812781f7e3761a')
+```
+
+As soon as it is done, we can fetch the overall results
+
+```python
+response = client.evaluations.fetch_evaluation_results('667d98fbd1812781f7e3761a')
+
+results = [(evaluator["evaluator_config"]["name"], evaluator["result"]) for evaluator in response["results"]]
+```
+
+and the detailed results
+
+```python
+client.evaluations.fetch_evaluation_scenarios(evaluations_ids='667d98fbd1812781f7e3761a')
+```
diff --git a/docs/docs/evaluation/06-human_evaluation.mdx b/docs/docs/evaluation/06-human_evaluation.mdx
new file mode 100644
index 0000000000..c7a534658d
--- /dev/null
+++ b/docs/docs/evaluation/06-human_evaluation.mdx
@@ -0,0 +1,60 @@
+---
+title: 'Human Evaluation'
+---
+
+import Image from "@theme/IdealImage";
+
+Sometimes, you may need to evaluate the performance of your models using human judgment. This is where the Human Evaluation feature comes in. It allows you to conduct A/B tests and single model tests to evaluate the performance of your models using human judgment.
+
+## Single Model Evaluation
+Single model test allows you to score the performance of a single LLM app manually.
+
+To start a new evaluation with the single model test, you need to:
+- Select a variant would you like to evaluate
+- Select a testset you want to use for the evaluation
+
+To start a new evaluation with the single model test, follow these steps:
+
+1. Select the variant you would like to evaluate.
+2. Choose the testset you want to use for the evaluation.
+
+Click on the "Start a new evaluation" button to begin the evaluation process. Once the evaluation is initiated, you will be directed to the Single Model Test Evaluation view. Here, you can perform the following actions:
+
+- Score: Enter a numerical score to evaluate the performance of the chosen variant.
+- Additional Notes: Add any relevant notes or comments to provide context or details about the evaluation.
+- Export Results: Use the "Export Results" functionality to save and export the evaluation results.
+
+
+
+
+## A/B Test
+A/B tests allow you to compare the performance of two different variants manually. For each data point you can select which variant is better, or if they are equally good or bad.
+
+To start a new evaluation with an A/B Test, follow these steps:
+
+1. Select two variants that you would like to evaluate.
+2. Choose the testset you want to use for the evaluation.
+
+### Invite Collaborators
+
+In an A/B Test, you can invite members of your workspace to collaborate on the evaluation by sharing a link to the evaluation. For information on how to add members to your workspace, please refer to this guide.
+
+Click on the "Start a new evaluation" button to begin the evaluation process. Once the evaluation is initiated, you will be directed to the A/B Test Evaluation view. Here, you can perform the following actions:
+
+- Scoring between variants: Evaluate and score the performance of each variant for the expected output.
+- Additional Notes: Add any relevant notes or comments to provide context or details about the evaluation.
+- Export Results: Use the "Export Results" functionality to save and export the evaluation results.
+
+
+
+
+## Toggle Evaluation View (Card/Table mode)
+You can also switch to the table view for a different perspective in your evaluation.
+
+### A/B Test View
+
+
+
+### Single Model Test View
+
+
diff --git a/docs/docs/evaluation/07-annotate-api.mdx b/docs/docs/evaluation/07-annotate-api.mdx
new file mode 100644
index 0000000000..289c6340e1
--- /dev/null
+++ b/docs/docs/evaluation/07-annotate-api.mdx
@@ -0,0 +1,520 @@
+---
+title: Annotate Traces from API
+sidebar_label: "Annotate Traces from API"
+description: "Learn how to add annotations to traces using the Agenta API"
+---
+
+```mdx-code-block
+import Image from "@theme/IdealImage";
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+```
+
+:::info
+Annotations are currently in preview. The interface is subject to change.
+:::
+
+Annotations in Agenta let you enrich the traces created by your LLM applications. You can add scores, comments, expected answers and other metrics to help evaluate your application's performance.
+
+
+
+## What You Can Do With Annotations
+
+- Collect user feedback on LLM responses
+- Run custom evaluation workflows
+- Measure application performance in real-time
+
+## Understanding How Annotations Work
+
+Annotations **link** to specific points (spans) in your application traces. You can **link** them to:
+- Parent spans (like an agent invocation)
+- Child spans (like a single LLM call within a larger process)
+
+Each annotation can include:
+- Numbers (scores, ratings)
+- Categories (labels, classifications)
+- Text (comments, reasoning)
+
+### Annotation Structure
+
+Here's what an annotation looks like:
+
+```json
+{
+ "data": {
+ "outputs": {
+ "score": 80,
+ "normalized_score": 0.8,
+ "reasoning": "The answer is mainly correct"
+ }
+ },
+ "references": {
+ "evaluator": {
+ "slug": "my_evaluator"
+ }
+ },
+ "links": {
+ "invocation": {
+ "trace_id": "47cb4a13b42a47bab2198509543ff14d",
+ "span_id": "1a9130f6ad398e55"
+ }
+ },
+ "metadata": {
+ "someattr": "somevalue"
+ }
+}
+```
+
+An annotation has four main parts:
+
+1. **Data**: The actual evaluation content (scores, comments)
+2. **References**: Which evaluator to use (will be created automatically if it doesn't exist)
+3. **Links**: Which trace and span you're annotating
+4. **Metadata** (optional): Any extra information you want to include
+
+### What You Can Include in Annotations
+
+Annotations are flexible - you can include whatever makes sense for your evaluation:
+
+```json
+// Just a simple score
+{"score": 3}
+
+// A score with explanation
+{"score": 3, "comment": "The response is not grounded"}
+
+// Multiple metrics with reference information
+{"score": 3, "normalized_score": 0.5, "comment": "The response is not grounded", "expected_answer": "The capital of France is Paris"}
+```
+
+You can use numbers, text, booleans, or categorical values.
+
+### Connecting Annotations to Traces
+
+Each annotation links to a specific span in your trace using a `trace_id` and `span_id`. These IDs follow the OpenTelemetry format.
+
+:::warning
+Agenta doesn't check if the trace and span IDs exist. Make sure they're valid before sending.
+:::
+
+### Working with Evaluators
+
+Every annotation must reference an `Evaluator`, which:
+- Defines what your annotation should contain
+- Provides context for interpreting the annotation
+
+If you reference an evaluator that doesn't exist yet, Agenta will automatically create one based on your first annotation.
+
+You can add multiple annotations from the same evaluator to a single span, which is useful for collecting feedback from different sources.
+
+### Using Metadata
+
+The `metadata` field lets you add extra information to your annotation. Use it to store things like user IDs, session information, or other context.
+
+## Creating Annotations
+
+
+
+
+```python
+import requests
+
+base_url = "https://cloud.agenta.ai"
+headers = {
+ "Content-Type": "application/json",
+ "Authorization": "ApiKey YOUR_API_KEY"
+}
+
+# Annotation data
+annotation_data = {
+ "annotation": {
+ "data": {
+ "outputs": {
+ "score": 40,
+ "reasoning": "Response contains some inaccuracies",
+ "labels": ["Partially Correct", "Needs Improvement"]
+ }
+ },
+ "references": {
+ "evaluator": {
+ "slug": "content_quality"
+ }
+ },
+ "links": {
+ "invocation": {
+ "trace_id": "47cb4a13b42a47bab2198509543ff14d",
+ "span_id": "1a9130f6ad398e55"
+ }
+ }
+ }
+}
+
+# Make the API request
+response = requests.post(
+ f"{base_url}/api/preview/annotations",
+ headers=headers,
+ json=annotation_data
+)
+
+# Process the response
+if response.status_code == 200:
+ print("Annotation created successfully")
+ print(response.json())
+else:
+ print(f"Error: {response.status_code}")
+ print(response.text)
+```
+
+
+
+
+```javascript
+async function createAnnotation() {
+ const baseUrl = 'https://cloud.agenta.ai';
+
+ const headers = {
+ 'Content-Type': 'application/json',
+ 'Authorization': 'ApiKey YOUR_API_KEY'
+ };
+
+ const annotationData = {
+ annotation: {
+ data: {
+ outputs: {
+ score: 40,
+ reasoning: "Response contains some inaccuracies",
+ labels: ["Partially Correct", "Needs Improvement"]
+ }
+ },
+ references: {
+ evaluator: {
+ slug: "content_quality"
+ }
+ },
+ links: {
+ invocation: {
+ trace_id: "47cb4a13b42a47bab2198509543ff14d",
+ span_id: "1a9130f6ad398e55"
+ }
+ }
+ }
+ };
+
+ try {
+ const response = await fetch(`${baseUrl}/api/preview/annotations`, {
+ method: 'POST',
+ headers: headers,
+ body: JSON.stringify(annotationData)
+ });
+
+ if (response.ok) {
+ const data = await response.json();
+ console.log("Annotation created successfully");
+ console.log(data);
+ } else {
+ console.error(`Error: ${response.status}`);
+ console.error(await response.text());
+ }
+ } catch (error) {
+ console.error("Request failed:", error);
+ }
+}
+
+createAnnotation();
+```
+
+
+
+
+:::info
+If the evaluator doesn't exist yet, Agenta will create it automatically based on your first annotation.
+:::
+
+## Understanding the Response
+
+When you create an annotation, you'll receive a response that includes the annotation data and its ID:
+
+```json
+{
+ "annotation": {
+ "created_at": "2025-05-13T16:54:52.245664Z",
+ "created_by_id": "019315dc-a332-7ba5-a426-d079c43ab776",
+ "span_id": "a1713ea8f59291f6",
+ "trace_id": "bb39070937f74f169b60dfc420729ad3",
+ "kind": "custom",
+ "source": "api",
+ "data": {
+ "outputs": {
+ "score": 0,
+ "reasoning": "The answer is totally false.",
+ "expected_answer": "The capital of France is Paris",
+ "normalized_score": 0
+ }
+ },
+ "metadata": {
+ "user_id": "miamia"
+ },
+ "references": {
+ "evaluator": {
+ "id": "0196ca64-9cbf-7ee3-b507-2b559dd13090",
+ "slug": "main-scorer"
+ }
+ },
+ "links": {
+ "invocation": {
+ "span_id": "09383e1124b264d0",
+ "trace_id": "5d52a529b48135fd58790f8aa3e33fb8"
+ }
+ }
+ }
+}
+```
+
+:::info
+Annotations themselves are stored as traces with their own `trace_id` and `span_id`, which is different from the invocation they're linked to.
+:::
+
+## Viewing Annotations in the UI
+
+You can see all annotations for a trace in the Agenta UI. Open any trace and check the Annotations tab to see detailed information. The right sidebar shows average metrics for each evaluator.
+
+
+
+## Querying Annotations
+
+
+
+
+You can query annotations in several ways:
+
+**1. Query by annotation ID:**
+
+```python
+import requests
+
+base_url = "https://cloud.agenta.ai"
+headers = {
+ "Content-Type": "application/json",
+ "Authorization": "ApiKey YOUR_API_KEY"
+}
+
+# Query by annotation ID
+query_data = {
+ "annotation": {
+ "trace_id": "bb39070937f74f169b60dfc420729ad3",
+ "span_id": "a1713ea8f59291f6"
+ }
+}
+
+response = requests.post(
+ f"{base_url}/api/preview/annotations/query",
+ headers=headers,
+ json=query_data
+)
+
+if response.status_code == 200:
+ print("Query successful")
+ print(response.json())
+else:
+ print(f"Error: {response.status_code}")
+ print(response.text)
+```
+
+**2. Query all annotations for an invocation:**
+
+```python
+# Query all annotations for an invocation
+query_data = {
+ "annotation": {
+ "links": {
+ "invocation": {
+ "trace_id": "5d52a529b48135fd58790f8aa3e33fb8",
+ "span_id": "09383e1124b264d0"
+ }
+ }
+ }
+}
+
+response = requests.post(
+ f"{base_url}/api/preview/annotations/query",
+ headers=headers,
+ json=query_data
+)
+```
+
+
+
+
+You can query annotations in several ways:
+
+**1. Query by annotation ID:**
+
+```javascript
+async function queryAnnotation() {
+ const baseUrl = 'https://cloud.agenta.ai';
+
+ const headers = {
+ 'Content-Type': 'application/json',
+ 'Authorization': 'ApiKey YOUR_API_KEY'
+ };
+
+ // Query by annotation ID
+ const queryData = {
+ annotation: {
+ trace_id: "bb39070937f74f169b60dfc420729ad3",
+ span_id: "a1713ea8f59291f6"
+ }
+ };
+
+ try {
+ const response = await fetch(`${baseUrl}/api/preview/annotations/query`, {
+ method: 'POST',
+ headers: headers,
+ body: JSON.stringify(queryData)
+ });
+
+ if (response.ok) {
+ const data = await response.json();
+ console.log("Query successful");
+ console.log(data);
+ } else {
+ console.error(`Error: ${response.status}`);
+ console.error(await response.text());
+ }
+ } catch (error) {
+ console.error("Request failed:", error);
+ }
+}
+
+queryAnnotation();
+```
+
+**2. Query all annotations for an invocation:**
+
+```javascript
+async function queryAnnotationsForInvocation() {
+ const baseUrl = 'https://cloud.agenta.ai';
+
+ const headers = {
+ 'Content-Type': 'application/json',
+ 'Authorization': 'ApiKey YOUR_API_KEY'
+ };
+
+ // Query all annotations for an invocation
+ const queryData = {
+ annotation: {
+ links: {
+ invocation: {
+ trace_id: "5d52a529b48135fd58790f8aa3e33fb8",
+ span_id: "09383e1124b264d0"
+ }
+ }
+ }
+ };
+
+ try {
+ const response = await fetch(`${baseUrl}/api/preview/annotations/query`, {
+ method: 'POST',
+ headers: headers,
+ body: JSON.stringify(queryData)
+ });
+
+ if (response.ok) {
+ const data = await response.json();
+ console.log("Query successful");
+ console.log(data);
+ } else {
+ console.error(`Error: ${response.status}`);
+ console.error(await response.text());
+ }
+ } catch (error) {
+ console.error("Request failed:", error);
+ }
+}
+
+queryAnnotationsForInvocation();
+```
+
+
+
+
+## Removing Annotations
+
+
+
+
+```python
+import requests
+
+base_url = "https://cloud.agenta.ai"
+headers = {
+ "Content-Type": "application/json",
+ "Authorization": "ApiKey YOUR_API_KEY"
+}
+
+# Define the trace_id and span_id of the annotation to delete
+trace_id = "bb39070937f74f169b60dfc420729ad3"
+span_id = "a1713ea8f59291f6"
+
+# Make the delete request
+response = requests.delete(
+ f"{base_url}/api/preview/annotations/{trace_id}/{span_id}",
+ headers=headers
+)
+
+# Process the response
+if response.status_code == 200:
+ print("Annotation deleted successfully")
+else:
+ print(f"Error: {response.status_code}")
+ print(response.text)
+```
+
+
+
+
+```javascript
+async function deleteAnnotation() {
+ const baseUrl = 'https://cloud.agenta.ai';
+
+ const headers = {
+ 'Content-Type': 'application/json',
+ 'Authorization': 'ApiKey YOUR_API_KEY'
+ };
+
+ // Define the trace_id and span_id of the annotation to delete
+ const traceId = "bb39070937f74f169b60dfc420729ad3";
+ const spanId = "a1713ea8f59291f6";
+
+ try {
+ const response = await fetch(`${baseUrl}/api/preview/annotations/${traceId}/${spanId}`, {
+ method: 'DELETE',
+ headers: headers
+ });
+
+ if (response.ok) {
+ console.log("Annotation deleted successfully");
+ } else {
+ console.error(`Error: ${response.status}`);
+ console.error(await response.text());
+ }
+ } catch (error) {
+ console.error("Request failed:", error);
+ }
+}
+
+deleteAnnotation();
+```
+
+
+
diff --git a/docs/docs/evaluation/_02-quick-start-sdk.mdx b/docs/docs/evaluation/_02-quick-start-sdk.mdx
deleted file mode 100644
index 854b62b6bc..0000000000
--- a/docs/docs/evaluation/_02-quick-start-sdk.mdx
+++ /dev/null
@@ -1,10 +0,0 @@
----
-title: "Quick Start: Evaluation from SDK"
-sidebar_label: "Quick Start (SDK)"
-description: "Get started with evaluating your LLM applications programmatically using the Agenta SDK"
-sidebar_position: 2
----
-
-import { Redirect } from '@docusaurus/router';
-
-
diff --git a/docs/docs/evaluation/_evaluation-from-sdk/01-quick-start.mdx b/docs/docs/evaluation/_evaluation-from-sdk/01-quick-start.mdx
deleted file mode 100644
index c7d594aeff..0000000000
--- a/docs/docs/evaluation/_evaluation-from-sdk/01-quick-start.mdx
+++ /dev/null
@@ -1,74 +0,0 @@
----
-title: "Quick Start"
-sidebar_label: "Quick Start"
-description: "Quick start guide for running evaluations programmatically with the Agenta SDK"
-sidebar_position: 1
----
-
-This quick start guide will help you run your first evaluation using the Agenta Python SDK.
-
-## Prerequisites
-
-- Python 3.8 or higher
-- Agenta account with API key
-- An LLM application deployed in Agenta
-
-## Installation
-
-```bash
-pip install -U agenta
-```
-
-## What you'll learn
-
-- How to initialize the SDK
-- How to create test sets programmatically
-- How to configure and run evaluations
-- How to retrieve evaluation results
-
-## Quick example
-
-```python
-import agenta as ag
-from agenta.client.api import AgentaApi
-
-# Initialize the SDK
-client = AgentaApi(
- base_url="https://cloud.agenta.ai/api",
- api_key="your-api-key"
-)
-
-# Create a test set
-test_set = client.testsets.create_testset(
- request={
- "name": "my_test_set",
- "csvdata": [
- {"input": "Hello", "expected": "Hi there!"},
- {"input": "How are you?", "expected": "I'm doing well!"}
- ]
- }
-)
-
-# Run evaluation
-evaluation = client.evaluations.create_evaluation(
- app_id="your-app-id",
- variant_ids=["variant-id"],
- testset_id=test_set.id,
- evaluators_configs=["evaluator-config-id"]
-)
-
-# Check status
-status = client.evaluations.fetch_evaluation_status(evaluation.id)
-print(f"Evaluation status: {status}")
-
-# Get results when complete
-results = client.evaluations.fetch_evaluation_results(evaluation.id)
-print(results)
-```
-
-## Next steps
-
-- Learn about [setup and configuration](/evaluation/evaluation-from-sdk/setup-configuration)
-- Explore [managing test sets with the SDK](/evaluation/evaluation-from-sdk/managing-test-sets)
-- Understand [configuring evaluators](/evaluation/evaluation-from-sdk/configuring-evaluators)
-- See how to [run evaluations](/evaluation/evaluation-from-sdk/running-evaluations) in detail
diff --git a/docs/docs/evaluation/_evaluation-from-sdk/02-setup-configuration.mdx b/docs/docs/evaluation/_evaluation-from-sdk/02-setup-configuration.mdx
deleted file mode 100644
index 28b858b81a..0000000000
--- a/docs/docs/evaluation/_evaluation-from-sdk/02-setup-configuration.mdx
+++ /dev/null
@@ -1,60 +0,0 @@
----
-title: "Setup and Configuration"
-sidebar_label: "Setup & Configuration"
-description: "Learn how to set up and configure the Agenta SDK for evaluation"
-sidebar_position: 2
----
-
-
-
-## Setup
-
-First, install the Agenta SDK:
-
-```bash
-pip install -U agenta
-```
-
-## Initialize the SDK client
-
-```python
-from agenta.client.api import AgentaApi
-
-app_id = "667d8cfad1812781f7e375d9"
-
-# You can create the API key under the settings page.
-# If you are using the OSS version, you should keep this as an empty string
-api_key = "EUqJGOUu.xxxx"
-
-# Host
-host = "https://cloud.agenta.ai"
-
-# Initialize the client
-client = AgentaApi(base_url=host + "/api", api_key=api_key)
-```
-
-## Configuration options
-
-
-
-## Environment variables
-
-You can also configure the SDK using environment variables:
-
-```bash
-export AGENTA_API_KEY="your-api-key"
-export AGENTA_HOST="https://cloud.agenta.ai"
-```
-
-```python
-import agenta as ag
-
-# Initialize will read from environment variables
-ag.init()
-```
-
-## Next steps
-
-- Learn about [managing test sets](/evaluation/evaluation-from-sdk/managing-test-sets)
-- Explore [configuring evaluators](/evaluation/evaluation-from-sdk/configuring-evaluators)
-- Start [running evaluations](/evaluation/evaluation-from-sdk/running-evaluations)
diff --git a/docs/docs/evaluation/_evaluation-from-sdk/03-managing-test-sets.mdx b/docs/docs/evaluation/_evaluation-from-sdk/03-managing-test-sets.mdx
deleted file mode 100644
index d866710948..0000000000
--- a/docs/docs/evaluation/_evaluation-from-sdk/03-managing-test-sets.mdx
+++ /dev/null
@@ -1,42 +0,0 @@
----
-title: "Managing Test Sets"
-sidebar_label: "Managing Test Sets"
-description: "Learn how to create, load, and manage test sets using the SDK"
-sidebar_position: 3
----
-
-
-
-## Creating test sets
-
-```python
-from agenta.client.types.new_testset import NewTestset
-
-csvdata = [
- {"country": "france", "capital": "Paris"},
- {"country": "Germany", "capital": "Berlin"}
-]
-
-response = client.testsets.create_testset(
- request=NewTestset(name="test set", csvdata=csvdata)
-)
-test_set_id = response.id
-```
-
-## Loading existing test sets
-
-
-
-## Updating test sets
-
-
-
-## Deleting test sets
-
-
-
-## Next steps
-
-- Learn about [configuring evaluators](/evaluation/evaluation-from-sdk/configuring-evaluators)
-- Start [running evaluations](/evaluation/evaluation-from-sdk/running-evaluations)
-- Explore [viewing results](/evaluation/evaluation-from-sdk/viewing-results)
diff --git a/docs/docs/evaluation/_evaluation-from-sdk/04-configuring-evaluators.mdx b/docs/docs/evaluation/_evaluation-from-sdk/04-configuring-evaluators.mdx
deleted file mode 100644
index 0de67a068f..0000000000
--- a/docs/docs/evaluation/_evaluation-from-sdk/04-configuring-evaluators.mdx
+++ /dev/null
@@ -1,53 +0,0 @@
----
-title: "Configuring Evaluators"
-sidebar_label: "Configuring Evaluators"
-description: "Learn how to configure built-in and custom evaluators using the SDK"
-sidebar_position: 4
----
-
-
-
-## Creating evaluators
-
-### Custom code evaluator
-
-Let's create a custom code evaluator that returns 1.0 if the first letter of the app output is uppercase:
-
-```python
-code_snippet = """
-from typing import Dict
-
-def evaluate(
- app_params: Dict[str, str],
- inputs: Dict[str, str],
- output: str, # output of the llm app
- datapoint: Dict[str, str] # contains the testset row
-) -> float:
- if output and output[0].isupper():
- return 1.0
- else:
- return 0.0
-"""
-
-response = client.evaluators.create_new_evaluator_config(
- app_id=app_id,
- name="capital_letter_evaluator",
- evaluator_key="auto_custom_code_run",
- settings_values={"code": code_snippet}
-)
-letter_match_eval_id = response.id
-```
-
-## Using built-in evaluators
-
-
-
-## Configuring evaluator settings
-
-
-
-## Next steps
-
-- Learn about [running evaluations](/evaluation/evaluation-from-sdk/running-evaluations)
-- Explore [viewing results](/evaluation/evaluation-from-sdk/viewing-results)
-- See all [evaluator types](/evaluation/configure-evaluators/overview)
diff --git a/docs/docs/evaluation/_evaluation-from-sdk/05-running-evaluations.mdx b/docs/docs/evaluation/_evaluation-from-sdk/05-running-evaluations.mdx
deleted file mode 100644
index aa9a523a02..0000000000
--- a/docs/docs/evaluation/_evaluation-from-sdk/05-running-evaluations.mdx
+++ /dev/null
@@ -1,61 +0,0 @@
----
-title: "Running Evaluations"
-sidebar_label: "Running Evaluations"
-description: "Learn how to run evaluations programmatically using the SDK"
-sidebar_position: 5
----
-
-
-
-## Running an evaluation
-
-First, let's grab the first variant in the app:
-
-```python
-response = client.apps.list_app_variants(app_id=app_id)
-print(response)
-myvariant_id = response[0].variant_id
-```
-
-Then, let's start the evaluation jobs:
-
-```python
-from agenta.client.types.llm_run_rate_limit import LlmRunRateLimit
-
-rate_limit_config = LlmRunRateLimit(
- batch_size=10, # number of rows to call in parallel
- max_retries=3, # max number of time to retry a failed llm call
- retry_delay=2, # delay before retrying a failed llm call
- delay_between_batches=5, # delay between batches
-)
-
-response = client.evaluations.create_evaluation(
- app_id=app_id,
- variant_ids=[myvariant_id],
- testset_id=test_set_id,
- evaluators_configs=[letter_match_eval_id],
- rate_limit=rate_limit_config
-)
-print(response)
-```
-
-## Checking evaluation status
-
-Now we can check for the status of the job:
-
-```python
-client.evaluations.fetch_evaluation_status('667d98fbd1812781f7e3761a')
-```
-
-## Configuring rate limits
-
-
-
-## Handling errors
-
-
-
-## Next steps
-
-- Learn about [viewing results](/evaluation/evaluation-from-sdk/viewing-results)
-- Explore [advanced evaluation patterns](/evaluation/evaluation-from-sdk/setup-configuration)
diff --git a/docs/docs/evaluation/_evaluation-from-sdk/06-viewing-results.mdx b/docs/docs/evaluation/_evaluation-from-sdk/06-viewing-results.mdx
deleted file mode 100644
index 7e8609e71f..0000000000
--- a/docs/docs/evaluation/_evaluation-from-sdk/06-viewing-results.mdx
+++ /dev/null
@@ -1,47 +0,0 @@
----
-title: "Viewing Results"
-sidebar_label: "Viewing Results"
-description: "Learn how to retrieve and analyze evaluation results using the SDK"
-sidebar_position: 6
----
-
-
-
-## Fetching overall results
-
-As soon as the evaluation is done, we can fetch the overall results:
-
-```python
-response = client.evaluations.fetch_evaluation_results('667d98fbd1812781f7e3761a')
-
-results = [
- (evaluator["evaluator_config"]["name"], evaluator["result"])
- for evaluator in response["results"]
-]
-print(results)
-```
-
-## Fetching detailed results
-
-Get detailed results for each test case:
-
-```python
-detailed_results = client.evaluations.fetch_evaluation_scenarios(
- evaluations_ids='667d98fbd1812781f7e3761a'
-)
-print(detailed_results)
-```
-
-## Analyzing results
-
-
-
-## Exporting results
-
-
-
-## Next steps
-
-- Learn about [human evaluation](/evaluation/human-evaluation/quick-start)
-- Explore [comparing evaluations in the UI](/evaluation/evaluation-from-ui/comparing-runs)
-- See all [evaluator types](/evaluation/configure-evaluators/overview)
diff --git a/docs/docs/evaluation/_evaluation-from-sdk/_category_.json b/docs/docs/evaluation/_evaluation-from-sdk/_category_.json
deleted file mode 100644
index f2be6bfd52..0000000000
--- a/docs/docs/evaluation/_evaluation-from-sdk/_category_.json
+++ /dev/null
@@ -1,6 +0,0 @@
-{
- "label": "Evaluation from SDK",
- "position": 7,
- "collapsible": true,
- "collapsed": true
-}
diff --git a/docs/docs/evaluation/configure-evaluators/01-overview.mdx b/docs/docs/evaluation/configure-evaluators/01-overview.mdx
deleted file mode 100644
index 7cb1292849..0000000000
--- a/docs/docs/evaluation/configure-evaluators/01-overview.mdx
+++ /dev/null
@@ -1,85 +0,0 @@
----
-title: "Configure Evaluators"
-sidebar_label: "Overview"
-description: "Set up evaluators for your use case"
-sidebar_position: 1
----
-
-import Image from "@theme/IdealImage";
-
-This guide shows you how to configure evaluators for your LLM application.
-
-## Configuring evaluators
-
-To create a new evaluator, click the `Create New` button in the `Evaluators` page.
-
-## Selecting evaluators
-
-Agenta offers a growing list of pre-built evaluators suitable for most use cases. You can also [create custom evaluators](/evaluation/configure-evaluators/custom-evaluator) by writing your own Python function or [use webhooks](/evaluation/configure-evaluators/webhook-evaluator) for evaluation.
-
-
-Available Evaluators
-
-| **Evaluator Name** | **Use Case** | **Type** | **Description** |
-| ------------------------------------------------------------------------------------------------------------- | -------------------------------- | ------------------ | -------------------------------------------------------------------------------- |
-| [Exact Match](/evaluation/configure-evaluators/classification-entity-extraction#exact-match) | Classification/Entity Extraction | Pattern Matching | Checks if the output exactly matches the expected result. |
-| [Contains JSON](/evaluation/configure-evaluators/classification-entity-extraction#contains-json) | Classification/Entity Extraction | Pattern Matching | Ensures the output contains valid JSON. |
-| [Regex Test](/evaluation/configure-evaluators/regex-evaluator) | Classification/Entity Extraction | Pattern Matching | Checks if the output matches a given regex pattern. |
-| [JSON Field Match](/evaluation/configure-evaluators/classification-entity-extraction#json-field-match) | Classification/Entity Extraction | Pattern Matching | Compares specific fields within JSON data. |
-| [JSON Diff Match](/evaluation/configure-evaluators/classification-entity-extraction#json-diff-match) | Classification/Entity Extraction | Similarity Metrics | Compares generated JSON with a ground truth JSON based on schema or values. |
-| [Similarity Match](/evaluation/configure-evaluators/semantic-similarity#similarity-match) | Text Generation / Chatbot | Similarity Metrics | Compares generated output with expected using Jaccard similarity. |
-| [Semantic Similarity Match](/evaluation/configure-evaluators/semantic-similarity#semantic-similarity-match) | Text Generation / Chatbot | Semantic Analysis | Compares the meaning of the generated output with the expected result. |
-| [Starts With](/evaluation/configure-evaluators/regex-evaluator) | Text Generation / Chatbot | Pattern Matching | Checks if the output starts with a specified prefix. |
-| [Ends With](/evaluation/configure-evaluators/regex-evaluator) | Text Generation / Chatbot | Pattern Matching | Checks if the output ends with a specified suffix. |
-| [Contains](/evaluation/configure-evaluators/regex-evaluator) | Text Generation / Chatbot | Pattern Matching | Checks if the output contains a specific substring. |
-| [Contains Any](/evaluation/configure-evaluators/regex-evaluator) | Text Generation / Chatbot | Pattern Matching | Checks if the output contains any of a list of substrings. |
-| [Contains All](/evaluation/configure-evaluators/regex-evaluator) | Text Generation / Chatbot | Pattern Matching | Checks if the output contains all of a list of substrings. |
-| [Levenshtein Distance](/evaluation/configure-evaluators/semantic-similarity#levenshtein-distance) | Text Generation / Chatbot | Similarity Metrics | Calculates the Levenshtein distance between output and expected result. |
-| [LLM-as-a-judge](/evaluation/configure-evaluators/llm-as-a-judge) | Text Generation / Chatbot | LLM-based | Sends outputs to an LLM model for critique and evaluation. |
-| [RAG Faithfulness](/evaluation/configure-evaluators/rag-evaluators) | RAG / Text Generation / Chatbot | LLM-based | Evaluates if the output is faithful to the retrieved documents in RAG workflows. |
-| [RAG Context Relevancy](/evaluation/configure-evaluators/rag-evaluators) | RAG / Text Generation / Chatbot | LLM-based | Measures the relevancy of retrieved documents to the given question in RAG. |
-| [Custom Code Evaluation](/evaluation/configure-evaluators/custom-evaluator) | Custom Logic | Custom | Allows users to define their own evaluator in Python. |
-| [Webhook Evaluator](/evaluation/configure-evaluators/webhook-evaluator) | Custom Logic | Custom | Sends output to a webhook for external evaluation. |
-
-
-
-
-
-
-
-## Evaluators' playground
-
-Each evaluator comes with its unique playground. For instance, in the screen below, the LLM-as-a-judge evaluator requires you to specify the prompt to use for the evaluation. You'll find detailed information about these parameters on each evaluator's documentation page.
-
-
-
-
-
-The evaluator playground lets you test your evaluator with sample input to make sure it's configured correctly.
-
-To use it, follow these steps:
-1. Load a test case from a test set
-2. Select a prompt and run it
-3. Run the evaluator to see the result
-
-You can adjust the configuration until you are happy with the result. When finished, commit your changes.
-
-
-## Next steps
-
-Explore the different evaluator types:
-
-- [Classification and Entity Extraction](/evaluation/configure-evaluators/classification-entity-extraction)
-- [Pattern Matching](/evaluation/configure-evaluators/regex-evaluator)
-- [Semantic Similarity](/evaluation/configure-evaluators/semantic-similarity)
-- [LLM as a Judge](/evaluation/configure-evaluators/llm-as-a-judge)
-- [RAG Evaluators](/evaluation/configure-evaluators/rag-evaluators)
-- [Custom Evaluators](/evaluation/configure-evaluators/custom-evaluator)
-- [Webhook Evaluators](/evaluation/configure-evaluators/webhook-evaluator)
diff --git a/docs/docs/evaluation/configure-evaluators/05-llm-as-a-judge.mdx b/docs/docs/evaluation/configure-evaluators/05-llm-as-a-judge.mdx
deleted file mode 100644
index 399dfde99e..0000000000
--- a/docs/docs/evaluation/configure-evaluators/05-llm-as-a-judge.mdx
+++ /dev/null
@@ -1,59 +0,0 @@
----
-title: "LLM-as-a-Judge"
----
-
-LLM-as-a-Judge is an evaluator that uses an LLM to assess LLM outputs. It's particularly useful for evaluating text generation tasks or chatbots where there's no single correct answer.
-
-
-
-The evaluator has the following parameters:
-
-#### The Prompt
-
-You can configure the prompt used for evaluation. The prompt can contain multiple messages in OpenAI format (role/content). All messages in the prompt have access to the inputs, outputs, and reference answers (any columns in the testset). To reference these in your prompts, use the following variables (inside double curly braces):
-
-- `{{inputs}}`: all the inputs to the llm application formatted as key-value pairs
-- `{{outputs}}`: the output of the llm application
-- `{{reference}}`: the column with the reference answer in the testset (optional). You can configure the name of this column under `Advanced Setting` in the configuration modal.
-- `{{correct_answer}}`: alias for `{{reference}}` (for backward compatibility)
-- `{{prediction}}`: alias for `{{outputs}}` (for backward compatibility)
-- `{{$input_column_name}}`: the value of any input column for the given row of your testset (e.g. `{{country}}`)
-
-If no `correct_answer` column is present in your testset, the variable will be left blank in the prompt.
-
-Here's the default prompt:
-
-**System prompt:**
-```
-You are an expert evaluator grading model outputs. Your task is to grade the responses based on the criteria and requirements provided below.
-
-Given the model output and inputs (and any other data you might get) assign a grade to the output.
-
-## Grading considerations
-- Evaluate the overall value provided in the model output
-- Verify all claims in the output meticulously
-- Differentiate between minor errors and major errors
-- Evaluate the outputs based on the inputs and whether they follow the instruction in the inputs if any
-- Give the highst and lowest score for cases where you have complete certainty about correctness and value
-
-## Scoring Criteria
-- The score should be between 0 and 10
-- A score of 10 means that the answer is perfect. This is the highest (best) score
-- A score of 0 means that the answer does not meet any of the criteria. This is the lowest possible score you can give.
-
-## output format
-ANSWER ONLY THE SCORE. DO NOT USE MARKDOWN. DO NOT PROVIDE ANYTHING OTHER THAN THE NUMBER
-```
-
-**User prompt:**
-
-```
-## Model inputs
-{{inputs}}
-## Model outputs
-{{outputs}}
-```
-
-### The Model
-
-The model can be configured to select one of the supported options (`gpt-3.5-turbo`, `gpt-4o`, `gpt-5`, `gpt-5-mini`, `gpt-5-nano`, `claude-3-5-sonnet`, `claude-3-5-haiku`, `claude-3-5-opus`). To use LLM-as-a-Judge, you'll need to set your OpenAI or Anthropic API key in the settings. The key is saved locally and only sent to our servers for evaluation—it's not stored there.
diff --git a/docs/docs/evaluation/configure-evaluators/_category_.json b/docs/docs/evaluation/configure-evaluators/_category_.json
deleted file mode 100644
index 838261f53c..0000000000
--- a/docs/docs/evaluation/configure-evaluators/_category_.json
+++ /dev/null
@@ -1,6 +0,0 @@
-{
- "label": "Configure Evaluators",
- "position": 8,
- "collapsible": true,
- "collapsed": true
-}
diff --git a/docs/docs/evaluation/configure-evaluators/regex-evaluator.mdx b/docs/docs/evaluation/configure-evaluators/regex-evaluator.mdx
deleted file mode 100644
index 23e27ce4bc..0000000000
--- a/docs/docs/evaluation/configure-evaluators/regex-evaluator.mdx
+++ /dev/null
@@ -1,16 +0,0 @@
----
-title: "Regex Evaluator"
----
-
-Regular Expressions (Regex) are sequences of characters that define search patterns, often used for pattern matching
-within strings. The `Regex Test` evaluator checks if the generated answer matches a regular expression pattern.
-
-The evaluator takes the regex pattern, and whether to match or not match for that pattern.
-
-Here are some examples:
-
-| Output | Regex | Match/Mismatch | Evaluator Output |
-| -------------------------------------- | ------------- | -------------- | ---------------- |
-| The iPhone 6 has a 1024px screen | `.*iphone.*` | `match` | True |
-| The Samsung galaxy has a 1024px screen | `.*Samsung.*` | `mismatch` | False |
-
diff --git a/docs/docs/evaluation/evaluation-from-ui/01-quick-start.mdx b/docs/docs/evaluation/evaluation-from-ui/01-quick-start.mdx
deleted file mode 100644
index a2354b42a0..0000000000
--- a/docs/docs/evaluation/evaluation-from-ui/01-quick-start.mdx
+++ /dev/null
@@ -1,111 +0,0 @@
----
-title: "Quick Start Evaluation from UI"
-sidebar_label: "Quick Start"
-description: "Quick start guide for running LLM and prompt evaluations from the Agenta UI"
-sidebar_position: 1
----
-import Image from "@theme/IdealImage";
-
-This quick start guide will help you run your first evaluation using the Agenta UI.
-
-You will create a prompt that classifies tweets based on their sentiment (positive, negative, neutral). Then you will run an evaluation to measure its performance.
-
-## Prerequisites
-
-Before you get started, create the prompt and the test set:
-
-### Create the prompt
-
-Create a `completion` prompt that classifies tweets based on their sentiment.
-
-Use the following prompt:
-
-**_System prompt_**
-```
-Classify this tweet based on its sentiment. Return only the sentiment. The sentiment should be one of the following: positive, negative, neutral.
-```
-**_User prompt_**
-```
-{{tweet}}
-```
-
-Commit the prompt to the default variant.
-
-
-
-
-### Create the test set
-
-Download a test set with 50 tweets based on the [Sentiment140 kaggle dataset](https://www.kaggle.com/datasets/kazanova/sentiment140).
-Download the test set [here](/examples/sentiment140_first50.csv). Then upload it to Agenta (see how to [upload a test set](/evaluation/managing-test-sets/upload-csv)).
-
-## Running the evaluation
-
-Navigate to the **Evaluations** page. Click on **Start new evaluation**.
-
-1. Choose the test set `sentiment140_first50`.
-2. Choose the latest revision from the default variant.
-3. Choose the evaluator `Exact Match`.
-4. Click on **Start evaluation**.
-
-
-
-:::tip
-
-Agenta offers a variety of built-in evaluators for most use cases. You can also create custom evaluators in Python. One of the most commonly used evaluators is `LLM as a Judge`. See more about [evaluators](/evaluation/configure-evaluators/overview).
-:::
-
-## Viewing the results
-
-Once the evaluation completes, click on the evaluation run to see the results.
-
-The overview tab shows the aggregated results for all evaluators.
-
-
-
-The test set tab shows the results for each test case.
-
-
-
-Click the expand button in the top right corner of the output cell to open a drawer with the full output. Click the tree icon to open a drawer with the trace.
-
-
-
-
-## Making a change and comparing the results
-
-Make a small change to the prompt and compare the results. Modify the prompt to use gpt-4o-mini instead of gpt-4o. Commit the change. Then rerun the evaluation with the new prompt revision.
-
-Now compare the results between the two evaluations. Click the `+ Compare` button on the top right corner of the evaluation results page. Select the evaluation to compare to.
-
-
-
-You might find that `gpt-4o-mini` performs differently than `gpt-4o` on this task. It may cost less but could have lower accuracy or different latency.
-
-## Next steps
-
-You have run a simple evaluation for a classification task in Agenta. Next, learn more about the different evaluators and how to configure them. See more about [evaluators](/evaluation/configure-evaluators/overview).
diff --git a/docs/docs/evaluation/evaluation-from-ui/02-running-evaluations.mdx b/docs/docs/evaluation/evaluation-from-ui/02-running-evaluations.mdx
deleted file mode 100644
index 018e795934..0000000000
--- a/docs/docs/evaluation/evaluation-from-ui/02-running-evaluations.mdx
+++ /dev/null
@@ -1,62 +0,0 @@
----
-title: "Running Evaluations"
-sidebar_label: "Running Evaluations"
-description: "Learn how to run evaluations from the Agenta web interface"
-sidebar_position: 2
----
-
-import Image from "@theme/IdealImage";
-
-This guide will show you how to run evaluations from the UI.
-
-## Prerequisites
-
-Before you get started, make sure that you have [created a test set](/evaluation/managing-test-sets/upload-csv) and [configured evaluators](/evaluation/configure-evaluators/overview) appropriate for your task.
-
-## Starting an evaluation
-
-To start an evaluation, navigate to the Evaluations page and click the `Start new evaluation` button. A modal will appear, allowing you to setup the evaluation.
-
-
-
-## Setting up evaluation parameters
-
-In the modal, specify the following:
-
-- **Testset**: Choose the testset(s) for your evaluation
-- **Variants**: Choose one or more variants to evaluate
-- **Evaluators**: Pick one or more evaluators for assessment
-
-
-
-## Advanced configuration
-
-Additional settings allow you to adjust batching and retry parameters for LLM calls. This helps mitigate rate limit errors from your LLM provider.
-
-Advanced configuration options include:
-
-- **Batch Size**: Number of test cases to run concurrently in each batch (default: 10)
-- **Retry Delay**: Time to wait before retrying a failed call (default: 3s)
-- **Max Retries**: Maximum number of retry attempts for a failed call (default: 3)
-- **Delay Between Batches**: Pause duration between batch runs (default: 5s)
-
-## Monitoring evaluation progress
-
-Once you start an evaluation:
-
-1. The evaluation will appear in the evaluations list
-2. You'll see the status (Running, Completed, Failed)
-3. Progress indicators show how many test cases have been processed
-4. You can view partial results while the evaluation is running
-
-## Next steps
-
-- Learn how to [view evaluation results](/evaluation/evaluation-from-ui/viewing-results)
-- Understand how to [compare evaluations](/evaluation/evaluation-from-ui/comparing-runs)
-- Try [human evaluation](/evaluation/human-evaluation/quick-start) for expert feedback
diff --git a/docs/docs/evaluation/evaluation-from-ui/03-viewing-results.mdx b/docs/docs/evaluation/evaluation-from-ui/03-viewing-results.mdx
deleted file mode 100644
index 52b3b7eb55..0000000000
--- a/docs/docs/evaluation/evaluation-from-ui/03-viewing-results.mdx
+++ /dev/null
@@ -1,78 +0,0 @@
----
-title: "Viewing Evaluation Results"
-sidebar_label: "Viewing Results"
-description: "Learn how to view and analyze evaluation results in Agenta"
-sidebar_position: 3
----
-
-import Image from "@theme/IdealImage";
-
-## Overview
-
-Once your evaluation completes, Agenta provides comprehensive views to analyze the results and understand your LLM application's performance.
-
-## Overview evaluation tab
-
-The main view offers an aggregated summary of results.
-
-
-
-- Average score per evaluator for each variant/test set combination
-- Average latency
-- Total cost
-- Creation date
-
-## Test cases evaluation tab
-
-The test cases evaluation tab provides a detailed view of each test case.
-
-
-
-The evaluation table columns show:
-
-- **Inputs**: The input data from your test set
-- **Reference Answers**: The expected/correct answers used by evaluators
-- **LLM Output**: The actual output from your application
-- **Evaluator Results**: Scores or boolean values from each evaluator
-- **Cost**: The cost of running this test case
-- **Latency**: How long the test case took to execute
-
-If you click on a test case, you will see a drawer with the full output and the evaluator results.
-
-
-
-## Prompt configuration tab
-
-The prompt configuration tab shows the prompt configuration used for this evaluation.
-
-
-
-## Exporting results
-
-Export your evaluation results for further analysis:
-
-1. Click the **Export** button on the evaluation detail page
-2. Choose CSV format
-3. Open in your preferred analysis tool (Excel, Python, R, etc.)
-
-## Next steps
-
-- Learn how to [compare multiple evaluations](/evaluation/evaluation-from-ui/comparing-runs)
-- Try [human evaluation](/evaluation/human-evaluation/quick-start) for qualitative assessment
-- Explore [evaluation concepts](/evaluation/concepts) to understand evaluation approaches
diff --git a/docs/docs/evaluation/evaluation-from-ui/04-comparing-runs.mdx b/docs/docs/evaluation/evaluation-from-ui/04-comparing-runs.mdx
deleted file mode 100644
index 5546045a93..0000000000
--- a/docs/docs/evaluation/evaluation-from-ui/04-comparing-runs.mdx
+++ /dev/null
@@ -1,72 +0,0 @@
----
-title: "Comparing Evaluation Runs"
-sidebar_label: "Comparing Runs"
-description: "Learn how to compare multiple evaluation runs to find the best performing variant"
-sidebar_position: 4
----
-
-import Image from "@theme/IdealImage";
-
-## Overview
-
-Compare evaluations to understand which variant performs better. This helps you make data-driven decisions about your LLM application.
-
-## Prerequisites
-
-To compare evaluations, you need:
-
-- Two or more completed evaluations
-- All evaluations must use the same test set
-
-## Starting a comparison
-
-After your evaluations complete, you can compare two or more of them:
-
-1. Go to the Evaluations page
-2. Click the compare button in the top right corner of the evaluation results page
-3. Select the evaluations you want to compare
-
-
-## Overview comparison tab
-
-The overview comparison tab shows aggregated results for all evaluators. The figures let you compare results between evaluations.
-
-
-
-## Test set comparison tab
-
-The test set comparison tab shows results for each test case. The figures let you compare results between evaluations.
-
-
-
-Click on a row to see a drawer with the full output and evaluator results side by side.
-
-
-
-## Prompt configuration comparison tab
-
-The prompt configuration comparison tab shows the prompt configuration used for each evaluation. The figures let you compare prompt configurations between evaluations.
-
-
-
-## Next steps
-
-- Learn about [human evaluation](/evaluation/human-evaluation/quick-start) for qualitative feedback
-- Explore [evaluation concepts](/evaluation/concepts) to understand evaluation approaches
-- Understand [evaluator types](/evaluation/configure-evaluators/overview) to choose the right metrics
diff --git a/docs/docs/evaluation/evaluation-from-ui/_category_.json b/docs/docs/evaluation/evaluation-from-ui/_category_.json
deleted file mode 100644
index 190fc0bdb3..0000000000
--- a/docs/docs/evaluation/evaluation-from-ui/_category_.json
+++ /dev/null
@@ -1,6 +0,0 @@
-{
- "label": "Evaluation from UI",
- "position": 6,
- "collapsible": true,
- "collapsed": true
-}
diff --git a/docs/docs/evaluation/configure-evaluators/02-classification-entity-extraction.mdx b/docs/docs/evaluation/evaluators/01-classification-entiry-extraction.mdx
similarity index 100%
rename from docs/docs/evaluation/configure-evaluators/02-classification-entity-extraction.mdx
rename to docs/docs/evaluation/evaluators/01-classification-entiry-extraction.mdx
diff --git a/docs/docs/evaluation/evaluators/02-pattern-matching.mdx b/docs/docs/evaluation/evaluators/02-pattern-matching.mdx
new file mode 100644
index 0000000000..283c9e8165
--- /dev/null
+++ b/docs/docs/evaluation/evaluators/02-pattern-matching.mdx
@@ -0,0 +1,47 @@
+---
+title: "Pattern Matching Evaluators"
+---
+
+Pattern Matching Evaluation is a method used to assess the performance of LLMs by identifying specific patterns within the output generated by the model.
+In Agenta, to perform pattern matching evaluation you can make use of the following evaluators:
+
+- Regex Test
+- Starts with
+- Ends with
+- Contains
+- Contains Any
+- Contains All
+
+## Regular Expression
+
+Regular Expressions (Regex) are sequences of characters that define search patterns, often used for pattern matching
+within strings. The `Regex Test` evaluator checks if the generated answer matches a regular expression pattern.
+
+The evaluator takes the regex pattern, and whether to match or not match for that pattern.
+
+Here are some examples:
+
+| Output | Regex | Match/Mismatch | Evaluator Output |
+| -------------------------------------- | ------------- | -------------- | ---------------- |
+| The iPhone 6 has a 1024px screen | `.*iphone.*` | `match` | True |
+| The Samsung galaxy has a 1024px screen | `.*Samsung.*` | `mismatch` | False |
+
+## Starts With
+
+**Starts With evaluator** checks if the output starts with a specified prefix, considering case sensitivity based on the settings.
+
+## Ends With
+
+**Ends With** evaluator checks if the output ends with a specified suffix, considering case sensitivity based on the settings.
+
+## Contains
+
+**Contains evaluator** checks if the output contains a specified substring, considering case sensitivity based on the settings.
+
+## Contains Any
+
+**Contains Any evaluator** checks if the output contains any of the specified substrings from a comma-separated list, considering case sensitivity based on the settings.
+
+## Contains All
+
+**Contains All evaluator** checks if the output contains all of the specified substrings from a comma-separated list, considering case sensitivity based on the settings.
diff --git a/docs/docs/evaluation/configure-evaluators/04-semantic-similarity.mdx b/docs/docs/evaluation/evaluators/03-semantic-similarity.mdx
similarity index 100%
rename from docs/docs/evaluation/configure-evaluators/04-semantic-similarity.mdx
rename to docs/docs/evaluation/evaluators/03-semantic-similarity.mdx
diff --git a/docs/docs/evaluation/evaluators/04-llm-as-a-judge.mdx b/docs/docs/evaluation/evaluators/04-llm-as-a-judge.mdx
new file mode 100644
index 0000000000..c9e8c55544
--- /dev/null
+++ b/docs/docs/evaluation/evaluators/04-llm-as-a-judge.mdx
@@ -0,0 +1,52 @@
+---
+title: "LLM-as-a-Judge"
+---
+
+LLM-as-a-Judge is an evaluator that uses an LLM to assess LLM outputs. It's particularly useful for evaluating text generation tasks or chatbots where there's no single correct answer.
+
+
+
+The evaluator has the following parameters:
+
+#### The Prompt
+
+You can configure the prompt used for evaluation. The prompt can contain multiple messages in OpenAI format (role/content). All messages in the prompt have access to the inputs, outputs, and reference answers (any columns in the test set). To reference these in your prompts, use the following variables:
+
+- `correct_answer`: the column with the reference answer in the test set (optional). You can configure the name of this column under `Advanced Setting` in the configuration modal.
+- `prediction`: the output of the llm application
+- `$input_column_name`: the value of any input column for the given row of your test set
+
+If no `correct_answer` column is present in your test set, the variable will be left blank in the prompt.
+
+Here's the default prompt used for the country expert demo application (note that it uses the `country` input column from our test set):
+
+**System prompt:**
+
+```
+You are an evaluator grading an LLM App.
+ You will be given INPUTS, the LLM APP OUTPUT, the CORRECT ANSWER, the PROMPT used in the LLM APP.
+ Here is the grade criteria to follow:
+:- Ensure that the LLM APP OUTPUT has the same meaning as the CORRECT ANSWER
+
+SCORE:
+-The score should be between 0 and 10
+-A score of 10 means that the answer is perfect. This is the highest (best) score.
+A score of 0 means that the answer does not any of of the criteria. This is the lowest possible score you can give.
+
+ANSWER ONLY THE SCORE. DO NOT USE MARKDOWN. DO NOT PROVIDE ANYTHING OTHER THAN THE NUMBER
+```
+
+**User prompt:**
+
+```
+
+INPUTS:
+country: {country}
+CORRECT ANSWER:{correct_answer}
+LLM APP OUTPUT: {prediction}.
+
+```
+
+### The Model
+
+The model can be configured to select one of the supported options (`gpt-3.5-turbo`, `gpt-4`, `claude-3-5-sonnet`, `claude-3-5-haiku`, `claude-3-5-opus`). To use LLM-as-a-Judge, you'll need to set your OpenAI or Anthropic API key in the settings. The key is saved locally and only sent to our servers for evaluation—it's not stored there.
diff --git a/docs/docs/evaluation/configure-evaluators/06-rag-evaluators.mdx b/docs/docs/evaluation/evaluators/05-rag-evaluators.mdx
similarity index 100%
rename from docs/docs/evaluation/configure-evaluators/06-rag-evaluators.mdx
rename to docs/docs/evaluation/evaluators/05-rag-evaluators.mdx
diff --git a/docs/docs/evaluation/configure-evaluators/07-custom-evaluator.mdx b/docs/docs/evaluation/evaluators/06-custom-evaluator.mdx
similarity index 100%
rename from docs/docs/evaluation/configure-evaluators/07-custom-evaluator.mdx
rename to docs/docs/evaluation/evaluators/06-custom-evaluator.mdx
diff --git a/docs/docs/evaluation/configure-evaluators/08-webhook-evaluator.mdx b/docs/docs/evaluation/evaluators/07-webhook-evaluator.mdx
similarity index 100%
rename from docs/docs/evaluation/configure-evaluators/08-webhook-evaluator.mdx
rename to docs/docs/evaluation/evaluators/07-webhook-evaluator.mdx
diff --git a/docs/docs/evaluation/evaluators/_category_.json b/docs/docs/evaluation/evaluators/_category_.json
new file mode 100644
index 0000000000..43e5ebe305
--- /dev/null
+++ b/docs/docs/evaluation/evaluators/_category_.json
@@ -0,0 +1,4 @@
+{
+ "position": 8,
+ "label": "Evaluators"
+}
diff --git a/docs/docs/evaluation/human-evaluation/01-quick-start.mdx b/docs/docs/evaluation/human-evaluation/01-quick-start.mdx
deleted file mode 100644
index ae78c98295..0000000000
--- a/docs/docs/evaluation/human-evaluation/01-quick-start.mdx
+++ /dev/null
@@ -1,62 +0,0 @@
----
-title: "Quick Start"
-sidebar_label: "Quick Start"
-description: "Get started with human evaluation in Agenta"
-sidebar_position: 1
----
-
-import Image from "@theme/IdealImage";
-
-## Overview
-
-Human evaluation lets you evaluate your LLM application's performance using human judgment instead of automated metrics.
-
-
- ⏯️ Watch a short demo of the human evaluation feature.
-
-
-
-
-## Why use human evaluation?
-
-Automated metrics can't capture everything. Sometimes you need human experts to evaluate results and identify why errors occur.
-
-Human evaluation helps you:
-- Get expert feedback to compare different versions of your application
-- Collect human feedback and insights to improve your prompts and configuration
-- Collect annotations to bootstrap automated evaluation
-
-## How human evaluation works
-
-Human evaluation follows the same process as automatic evaluation:
-1. Choose a test set
-2. Select the versions you want to evaluate
-3. Pick your evaluators
-4. Start the evaluation
-
-The only difference is that humans provide the evaluation scores instead of automated systems.
-
-## Quick workflow
-
-1. **Start evaluation**: Go to Evaluations → Human annotation → Start new evaluation
-2. **Select test set**: Choose the data you want to evaluate against
-3. **Select variant**: Pick the version of your application to test
-4. **Configure evaluators**: Create or select evaluators (boolean, integer, multi-choice, etc.)
-5. **Run**: Click "Start evaluation" and generate outputs
-6. **Annotate**: Review each response and provide feedback
-7. **Review results**: Analyze aggregated scores and export data
-
-## Next steps
-
-- Learn about [configuring evaluators](/evaluation/human-evaluation/configuring-evaluators)
-- Understand [how to run evaluations](/evaluation/human-evaluation/running-evaluations)
-- Explore [viewing results](/evaluation/human-evaluation/viewing-results)
-- Try [A/B testing](/evaluation/human-evaluation/ab-testing)
diff --git a/docs/docs/evaluation/human-evaluation/02-configuring-evaluators.mdx b/docs/docs/evaluation/human-evaluation/02-configuring-evaluators.mdx
deleted file mode 100644
index 209114c22b..0000000000
--- a/docs/docs/evaluation/human-evaluation/02-configuring-evaluators.mdx
+++ /dev/null
@@ -1,71 +0,0 @@
----
-title: "Configuring Evaluators"
-sidebar_label: "Configuring Evaluators"
-description: "Learn how to configure evaluators for human evaluation"
-sidebar_position: 2
----
-
-import Image from "@theme/IdealImage";
-
-## Creating evaluators
-
-If you don't have evaluators yet, click **Create new** in the **Evaluator** section.
-
-Each evaluator has:
-- **Name** - What you're measuring (e.g., "correctness")
-- **Description** - What the evaluator does
-- **Feedback types** - How evaluators will score responses
-
-For example, a "correctness" evaluator might have:
-- `is_correct` - A yes/no question about accuracy
-- `error_type` - A multiple-choice field for categorizing mistakes
-
-
-
-## Available feedback types
-
-- **Boolean** - Yes/no questions
-- **Integer** - Whole number ratings
-- **Decimal** - Precise numerical scores
-- **Single-choice** - Pick one option
-- **Multi-choice** - Pick multiple options
-- **String** - Free-text comments or notes
-
-## Grouping related feedback types
-
-:::tip
-Evaluators can include multiple related feedback types. For example:
-
-**Correctness evaluator:**
-- `is_correct` - Yes/no question about accuracy
-- `error_type` - Multiple-choice field to categorize mistakes (only if incorrect)
-
-**Style adherence evaluator:**
-- `is_adherent` - Yes/no question about style compliance
-- `comment` - Text field explaining why the style doesn't match (if needed)
-
-This grouping helps you evaluate different aspects of your LLM's performance in an organized way.
-:::
-
-## Selecting evaluators
-
-After creating evaluators:
-
-1. Select the evaluators you want to use
-2. You can use multiple evaluators in a single evaluation
-3. Each evaluator will appear in the annotation interface
-
-## Next steps
-
-- Learn about [running evaluations](/evaluation/human-evaluation/running-evaluations)
-- Understand [how to view results](/evaluation/human-evaluation/viewing-results)
-- Try [A/B testing](/evaluation/human-evaluation/ab-testing)
diff --git a/docs/docs/evaluation/human-evaluation/03-running-evaluations.mdx b/docs/docs/evaluation/human-evaluation/03-running-evaluations.mdx
deleted file mode 100644
index dbc5544269..0000000000
--- a/docs/docs/evaluation/human-evaluation/03-running-evaluations.mdx
+++ /dev/null
@@ -1,87 +0,0 @@
----
-title: "Running Evaluations"
-sidebar_label: "Running Evaluations"
-description: "Learn how to run human evaluation sessions in Agenta"
-sidebar_position: 3
----
-
-import Image from "@theme/IdealImage";
-
-## Starting a new evaluation
-
-1. Go to the **Evaluations** page
-2. Select the **Human annotation** tab
-3. Click **Start new evaluation**
-
-
-
-## Configuring your evaluation
-
-1. **Select your test set** - Choose the data you want to evaluate against
-2. **Select your revision** - Pick the version of your application to test
-
-:::warning
-Your test set columns must match the input variables in your revision. If they don't match, you'll see an error message.
-:::
-
-3. **Choose evaluators** - Select how you want to measure performance
-
-
-
-## Running the evaluation
-
-After configuring:
-
-1. Click **Start evaluation**
-2. You'll be redirected to the annotation interface
-3. Click **Run all** to generate outputs and begin evaluation
-
-
-
-## Annotating responses
-
-For each test case:
-1. Review the input and output
-2. Use the evaluation form on the right to score the response
-3. Click **Annotate** to save your assessment
-4. Click **Next** to move to the next test case
-
-:::tip
-Select the **Unannotated** tab to see only the test cases you haven't reviewed yet.
-:::
-
-## Collaboration
-
-You can invite team members to help with evaluation by sharing the evaluation link. Team members must be added to your workspace first.
-
-## Next steps
-
-- Learn about [viewing results](/evaluation/human-evaluation/viewing-results)
-- Try [A/B testing](/evaluation/human-evaluation/ab-testing) to compare variants
diff --git a/docs/docs/evaluation/human-evaluation/04-viewing-results.mdx b/docs/docs/evaluation/human-evaluation/04-viewing-results.mdx
deleted file mode 100644
index e80c933b6f..0000000000
--- a/docs/docs/evaluation/human-evaluation/04-viewing-results.mdx
+++ /dev/null
@@ -1,61 +0,0 @@
----
-title: "Viewing Results"
-sidebar_label: "Viewing Results"
-description: "Learn how to view and export human evaluation results"
-sidebar_position: 4
----
-
-## Overview
-
-After completing annotations, you can review and export the results of your human evaluation.
-
-## Viewing results
-
-The **Results** section shows:
-
-- Aggregated scores across all test cases
-- Individual annotations for each test case
-- Evaluator performance metrics
-- Comments and feedback provided by annotators
-
-## Comparing with other experiments
-
-You can compare human evaluation results with:
-
-- Automated evaluation runs
-- Other human evaluation sessions
-- Different variants or versions
-
-This helps you understand how human judgment aligns with automated metrics and identify areas for improvement.
-
-## Exporting results
-
-### Export to CSV
-
-Click **Export results** to download your evaluation data in CSV format. The exported file includes:
-
-- Test case inputs
-- LLM outputs
-- All annotation scores and feedback
-- Timestamp and annotator information
-
-### Saving as test set
-
-Click **Save test set** to create a new test set from annotated data. This is useful for:
-
-- Bootstrapping automated evaluation with human-validated examples
-- Creating regression test suites
-- Building training data for custom evaluators
-
-## Use cases for exported data
-
-- **Analysis**: Perform statistical analysis on evaluation results
-- **Reporting**: Create reports for stakeholders
-- **Training**: Use annotations to train or fine-tune models
-- **Quality Assurance**: Track quality metrics over time
-
-## Next steps
-
-- Learn about [A/B testing](/evaluation/human-evaluation/ab-testing)
-- Explore [automated evaluation](/evaluation/evaluation-from-ui/quick-start)
-- Understand [configure evaluators](/evaluation/configure-evaluators/overview)
diff --git a/docs/docs/evaluation/human-evaluation/05-ab-testing.mdx b/docs/docs/evaluation/human-evaluation/05-ab-testing.mdx
deleted file mode 100644
index c2da594532..0000000000
--- a/docs/docs/evaluation/human-evaluation/05-ab-testing.mdx
+++ /dev/null
@@ -1,69 +0,0 @@
----
-title: "A/B Testing"
-sidebar_label: "A/B Testing"
-description: "Learn how to compare two variants using human evaluation"
-sidebar_position: 5
----
-
-import Image from "@theme/IdealImage";
-
-## Overview
-
-A/B testing lets you compare two versions of your application side-by-side. For each test case, you choose which version performs better.
-
-## Setting up A/B testing
-
-1. Select **two versions** you want to compare
-2. Choose your **test set**
-3. For each test case, decide which version is better (or if they're equal)
-
-
-
-## A/B testing features
-
-During A/B evaluation, you can:
-- **Compare variants** - Score which version performs better for each test case
-- **Add notes** - Include context or detailed feedback
-- **Export results** - Download your evaluation data for further analysis
-
-## Collaborating on A/B tests
-
-You can invite team members to help with A/B testing by sharing the evaluation link. Team members must be added to your workspace first.
-
-This is particularly useful for:
-- Getting diverse perspectives on performance
-- Reducing individual bias
-- Speeding up evaluation with multiple annotators
-
-## Interpreting A/B test results
-
-After completing the A/B test, you'll see:
-
-- Win/loss/tie counts for each variant
-- Percentage of cases where each variant performed better
-- Specific test cases where variants differed significantly
-- Notes and comments from annotators
-
-## Use cases
-
-A/B testing is ideal for:
-
-- **Prompt optimization**: Compare different prompt wordings
-- **Model selection**: Evaluate different LLM models (GPT-4 vs Claude vs others)
-- **Parameter tuning**: Test different temperature or max_tokens settings
-- **Feature comparison**: Compare variants with different features enabled
-
-## Next steps
-
-- Learn about [exporting results](/evaluation/human-evaluation/viewing-results)
-- Explore [automated evaluation](/evaluation/evaluation-from-ui/comparing-runs) for larger-scale comparisons
-- Understand [evaluation concepts](/evaluation/concepts)
diff --git a/docs/docs/evaluation/human-evaluation/_category_.json b/docs/docs/evaluation/human-evaluation/_category_.json
deleted file mode 100644
index 44fdaf5ba8..0000000000
--- a/docs/docs/evaluation/human-evaluation/_category_.json
+++ /dev/null
@@ -1,6 +0,0 @@
-{
- "label": "Human Evaluation",
- "position": 9,
- "collapsible": true,
- "collapsed": true
-}
diff --git a/docs/docs/evaluation/managing-test-sets/01-upload-csv.mdx b/docs/docs/evaluation/managing-test-sets/01-upload-csv.mdx
deleted file mode 100644
index 13591eddb7..0000000000
--- a/docs/docs/evaluation/managing-test-sets/01-upload-csv.mdx
+++ /dev/null
@@ -1,87 +0,0 @@
----
-title: "Upload Test Sets as CSVs"
-sidebar_label: "Upload CSV/JSON"
-description: "Learn how to upload test sets from CSV or JSON files"
-sidebar_position: 1
----
-
-```mdx-code-block
-import Image from "@theme/IdealImage";
-```
-
-## Overview
-
-You can quickly create test sets by uploading CSV or JSON files. This is the fastest way to import existing test data into Agenta.
-
-## Uploading a file
-
-To create a test set from a CSV or JSON file:
-
-1. Go to `Test sets`
-2. Click `Upload test sets`
-3. Select either `CSV` or `JSON`
-
-
-
-## CSV Format
-
-We use CSV with commas (,) as separators and double quotes (") as quote characters. The first row should contain the header with column names. Each input should have its own column. The column containing the reference answer can have any name, but we use "correct_answer" by default.
-
-:::info
-If you choose a different column name for the reference answer, you'll need to configure the evaluator later with that specific name.
-:::
-
-Here's an example of a valid CSV:
-
-```csv
-text,instruction,correct_answer
-Hello,How are you?,I'm good.
-"Tell me a joke.",Sure, here's one:...
-```
-
-## JSON Format
-
-The test set should be in JSON format with the following structure:
-
-1. A JSON file containing an array of objects.
-2. Each object in the array represents a row, with keys as column headers and values as row data. Here's an example of a valid JSON file:
-
-```json
-[
- { "recipe_name": "Chicken Parmesan", "correct_answer": "Chicken" },
- { "recipe_name": "a, special, recipe", "correct_answer": "Beef" }
-]
-```
-
-## Test set schema for Chat Applications
-
-For chat applications created using the chat template in Agenta, the input should be saved in the column called `messages`, which would contain the input list of messages:
-
-```json
-[
- { "content": "message.", "role": "user" },
- { "content": "message.", "role": "assistant" }
- // Add more messages if necessary
-]
-```
-
-In case the prompt includes other variables (e.g. `context`), make sure to have a column with the same name and the value of the variable.
-
-The reference answer column (by default `correct_answer`) should follow the same format:
-
-```json
-{ "content": "message.", "role": "assistant" }
-```
-
-Here is an example of a valid CSV for testing the default chat prompt template:
-
-```csv chat_test_set.csv
-context,messages,correct_answer
-test,"[{""role"":""user"",""content"":""hi""}]","{""content"":""Hello! How can I assist you today?"",""role"":""assistant"",""annotations"":[]}"
-```
-
-## Next steps
-
-- [Create test sets programmatically](/evaluation/managing-test-sets/create-programatically) using the SDK or API
-- [Create test sets from traces](/evaluation/managing-test-sets/create-from-traces) to capture real production data
-- [Create test sets from playground](/evaluation/managing-test-sets/create-from-playground) during experimentation
diff --git a/docs/docs/evaluation/managing-test-sets/02-create-programatically.mdx b/docs/docs/evaluation/managing-test-sets/02-create-programatically.mdx
deleted file mode 100644
index 79c271af22..0000000000
--- a/docs/docs/evaluation/managing-test-sets/02-create-programatically.mdx
+++ /dev/null
@@ -1,87 +0,0 @@
----
-title: "Create Test Sets Programmatically"
-sidebar_label: "Create Programmatically"
-description: "Learn how to create test sets using the API or SDK"
-sidebar_position: 2
----
-
-## Overview
-
-Creating test sets programmatically allows you to automate test set generation, integrate with your CI/CD pipeline, or dynamically generate test cases from existing data sources.
-
-## Creating via API
-
-You can upload a test set using our API. Find the [API endpoint reference here](/reference/api/upload-file).
-
-Here's an example of such a call:
-
-**HTTP Request:**
-
-```
-POST /testsets
-
-```
-
-**Request Body:**
-
-```json
-{
- "name": "testsetname",
- "csvdata": [
- { "column1": "row1col1", "column2": "row1col2" },
- { "column1": "row2col1", "column2": "row2col2" }
- ]
-}
-```
-
-### Example with curl
-
-```bash
-curl -X POST "https://cloud.agenta.ai/api/testsets" \
- -H "Content-Type: application/json" \
- -H "Authorization: ApiKey YOUR_API_KEY" \
- -d '{
- "name": "my_test_set",
- "csvdata": [
- {"input": "Hello", "expected": "Hi there!"},
- {"input": "How are you?", "expected": "I am doing well!"}
- ]
- }'
-```
-
-## Creating via SDK
-
-```python
-from agenta.client.api import AgentaApi
-from agenta.client.types.new_testset import NewTestset
-
-# Initialize the client
-client = AgentaApi(
- base_url="https://cloud.agenta.ai/api",
- api_key="your-api-key"
-)
-
-# Create test set data
-csvdata = [
- {"country": "France", "capital": "Paris"},
- {"country": "Germany", "capital": "Berlin"},
- {"country": "Spain", "capital": "Madrid"}
-]
-
-# Create the test set
-response = client.testsets.create_testset(
- request=NewTestset(
- name="countries_capitals",
- csvdata=csvdata
- )
-)
-
-test_set_id = response.id
-print(f"Created test set with ID: {test_set_id}")
-```
-
-## Next steps
-
-- [Create test sets from UI](/evaluation/managing-test-sets/create-from-ui) for manual creation
-- [Create test sets from traces](/evaluation/managing-test-sets/create-from-traces) to capture production data
-- [Create test sets from playground](/evaluation/managing-test-sets/create-from-playground) during experimentation
diff --git a/docs/docs/evaluation/managing-test-sets/03-create-from-ui.mdx b/docs/docs/evaluation/managing-test-sets/03-create-from-ui.mdx
deleted file mode 100644
index d6146d5a67..0000000000
--- a/docs/docs/evaluation/managing-test-sets/03-create-from-ui.mdx
+++ /dev/null
@@ -1,48 +0,0 @@
----
-title: "Create Test Sets from UI"
-sidebar_label: "Create from UI"
-description: "Learn how to create and edit test sets using the Agenta web interface"
-sidebar_position: 3
----
-
-```mdx-code-block
-import Image from "@theme/IdealImage";
-```
-
-## Overview
-
-The Agenta UI provides an intuitive interface for creating and editing test sets manually. This is ideal when you want to quickly create or edit small test sets.
-
-## Creating a test set from the UI
-
-To create a new test set from the UI:
-
-1. Go to `Test sets` in the navigation
-2. Click `Create a test set with UI`
-3. Name your test set
-4. Specify the columns for input types
-5. Add your test cases row by row
-6. Click `Save test set` when done
-
-
-
-## Editing an existing test set
-
-To edit an existing test set:
-
-1. Go to `Test sets`
-2. Select the test set you want to edit
-3. Make your changes (add, edit, or delete rows)
-4. Click `Save test set` to persist your changes
-
-:::tip
-Remember to click `Save test set` before navigating away, or your changes will be lost!
-:::
-
-
-## Next steps
-
-- [Upload test sets as CSV](/evaluation/managing-test-sets/upload-csv) for bulk imports
-- [Create test sets programmatically](/evaluation/managing-test-sets/create-programatically) using API/SDK
-- [Create test sets from traces](/evaluation/managing-test-sets/create-from-traces) to capture production data
-- Learn about [running evaluations](/evaluation/evaluation-from-ui/running-evaluations) with your test sets
diff --git a/docs/docs/evaluation/managing-test-sets/04-create-from-traces.mdx b/docs/docs/evaluation/managing-test-sets/04-create-from-traces.mdx
deleted file mode 100644
index 32f4ba8da5..0000000000
--- a/docs/docs/evaluation/managing-test-sets/04-create-from-traces.mdx
+++ /dev/null
@@ -1,56 +0,0 @@
----
-title: "Create Test Sets from Traces"
-sidebar_label: "Create from Traces"
-description: "Learn how to create test sets from production traces in observability"
-sidebar_position: 4
----
-
-```mdx-code-block
-import { Stream } from '@cloudflare/stream-react';
-```
-
-## Overview
-
-One of the most valuable sources of test cases is your production data. Traces captured in the Observability view represent real user interactions with your LLM application.
-
-
-
-
-## Adding a Single Trace
-
-To add a single trace to a test set:
-
-1. Navigate to the **Observability** view in Agenta
-2. Find a trace you want to add to a test set
-3. Click the **Add to test set** button at the top of the trace
-4. Choose to create a new test set or select an existing one
-5. Review the mapping between trace data and test set columns
- - Agenta will automatically map the inputs and outputs to appropriate columns
- - You can edit the expected answer if you don't agree with the output
-6. Click **Save** to add the trace to your test set
-
-## Adding Multiple Traces at Once
-
-To efficiently add multiple traces:
-
-1. In the Observability view, use the search function to filter traces
- - For example, search for specific response patterns like "I don't have enough information"
-2. Select all relevant traces by checking the boxes next to them
-3. Click **Add to test set**
-4. Choose an existing test set or create a new one
-5. Review the mapping for the traces
-6. Click **Save** to add all selected traces to your test set
-
-## Use cases
-
-Creating test sets from traces is particularly useful for:
-
-- **Edge Cases**: Capture unusual or problematic user interactions
-- **Regression Testing**: Save examples of correct behavior to prevent future regressions
-- **Error Analysis**: Collect failed cases to understand and fix issues
-
-## Next steps
-
-- [Create test sets from playground](/evaluation/managing-test-sets/create-from-playground) during experimentation
-- [Upload test sets as CSVs](/evaluation/managing-test-sets/upload-csv) for bulk imports
-- Learn about [running evaluations](/evaluation/evaluation-from-ui/running-evaluations) with your test sets
diff --git a/docs/docs/evaluation/managing-test-sets/05-create-from-playground.mdx b/docs/docs/evaluation/managing-test-sets/05-create-from-playground.mdx
deleted file mode 100644
index a9c8d5d603..0000000000
--- a/docs/docs/evaluation/managing-test-sets/05-create-from-playground.mdx
+++ /dev/null
@@ -1,34 +0,0 @@
----
-title: "Create Test Sets from Playground"
-sidebar_label: "Create from Playground"
-description: "Learn how to create test sets directly from the playground while experimenting"
-sidebar_position: 5
----
-
-```mdx-code-block
-import { Stream } from '@cloudflare/stream-react';
-```
-
-## Overview
-
-The playground offers a convenient way to create and add data to a test set. This workflow is useful when you discover interesting cases or edge cases while experimenting with your LLM application.
-
-
-
-
-## Adding data points from the playground
-
-To add a data point to a test set from the playground:
-
-1. Work with your application in the playground
-2. When you find an interesting case, click the `Add to test set` button located near the `Run` button
-3. A drawer will display showing the inputs and outputs from the playground
-4. You can modify inputs and correct answers if needed
-5. Select an existing test set to add to, or choose `+Add new` to create a new one
-6. Once you're satisfied, click `Add` to finalize
-
-## Next steps
-
-- [Upload test sets as CSVs](/evaluation/managing-test-sets/upload-csv) for bulk imports
-- [Create test sets from traces](/evaluation/managing-test-sets/create-from-traces) to capture production data
-- Learn about [running evaluations](/evaluation/evaluation-from-ui/running-evaluations) with your test sets
diff --git a/docs/docs/evaluation/managing-test-sets/_category_.json b/docs/docs/evaluation/managing-test-sets/_category_.json
deleted file mode 100644
index 45dbdaf1b2..0000000000
--- a/docs/docs/evaluation/managing-test-sets/_category_.json
+++ /dev/null
@@ -1,6 +0,0 @@
-{
- "label": "Manage Test Sets",
- "position": 5,
- "collapsible": true,
- "collapsed": true
-}
\ No newline at end of file
diff --git a/docs/docs/getting-started/01-introduction.mdx b/docs/docs/getting-started/01-introduction.mdx
index d7e9e591c8..b27a8dcdec 100644
--- a/docs/docs/getting-started/01-introduction.mdx
+++ b/docs/docs/getting-started/01-introduction.mdx
@@ -14,52 +14,31 @@ import Image from "@theme/IdealImage";
alt="Screenshots of Agenta LLMOPS platform"
loading="lazy"
/>
-Agenta is an **open-source LLMOps platform** that helps **developers** and **product teams** build reliable LLM applications.
+Agenta is an open-source platform that helps **developers** and **product teams**
+build robust AI applications powered by LLMs. It offers all the tools for **prompt
+management and evaluation**.
+### With Agenta, you can:
-Agenta covers the entire LLM development lifecycle: **prompt management**, **evaluation**, and **observability**.
+1. Rapidly [**experiment** and **compare** prompts](/prompt-engineering/overview) on [any LLM workflow](/custom-workflows/overview) (chain-of-prompts, Retrieval Augmented Generation (RAG), LLM agents...)
+2. Rapidly [**create test sets**](/evaluation/create-test-sets) and **golden datasets** for evaluation
+3. **Evaluate** your application with pre-existing or **custom evaluators**
+4. **Annotate** and **A/B test** your applications with **human feedback**
+5. [**Collaborate with product teams**](/misc/team_management) for prompt engineering and evaluation
+6. [**Deploy your application**](/concepts/concepts#environments) in one-click in the UI, through CLI, or through github workflows.
-## Features
+Agenta focuses on increasing the speed of the development cycle of LLM applications by increasing the speed of experimentation.
-### Prompt Engineering and Management
+## How is Agenta different?
-Teams often struggle with prompt collaboration. They keep prompts in code where subject matter experts cannot edit them. Or they use spreadsheets in an unreliable process.
-
-Agenta organizes prompts for your team. Subject matter experts can collaborate with developers without touching the codebase. Developers can version prompts and deploy them to production.
-
-The playground lets teams experiment with prompts. You can load traces and test sets. You can test prompts side by side.
-
-
-### Evaluation
-
-Most teams lack a systematic evaluation process. They make random prompt changes based on vibes. Some changes improve quality but break other cases because LLMs are stochastic.
-
-Agenta provides one place to evaluate systematically. Teams can run three types of evaluation:
-
-- Automatic evaluation with LLMs at scale before production
-- Human annotation where subject matter experts review results and provide feedback to AI engineers
-- Online evaluation for applications already in production
-
-Both subject matter experts and engineers can run evaluations from the UI.
-
-### Observability
-
-Agenta helps you understand what happens in production. You can capture user feedback through an API (thumbs up or implicit signals). You can debug agents and applications with tracing to see what happens inside them.
-
-Track costs over time. Find edge cases where things fail. Add those cases to your test sets. Have subject matter experts annotate the results.
+### Works with any LLM app workflow
+Agenta enables prompt engineering and evaluation on any LLM app architecture, such as **Chain of Prompts**, **RAG**, or **LLM agents**. It is compatible with any framework like **Langchain** or **LlamaIndex**, and works with any model provider, such as **OpenAI**, **Cohere**, or **local models**.
-## Why Agenta?
+[Jump here](/custom-workflows/overview) to see how to use your own custom application with Agenta.
### Enable collaboration between developers and product teams
Agenta empowers **non-developers** to iterate on the configuration of any custom LLM application, evaluate it, annotate it, A/B test it, and deploy it, all within the user interface.
-### Open-source and MIT licensed
-
-Agenta is open-source and MIT licensed, so you can self-host it, modify it, and use it in commercial projects without restrictions.
-
-### Works with any LLM app workflow
-
-Agenta enables prompt engineering and evaluation on any LLM app architecture, such as **Chain of Prompts**, **RAG**, or **LLM agents**. It is compatible with any framework like **Langchain** or **LlamaIndex**, and works with any model provider, such as **OpenAI**, **Cohere**, or **local models**.
-
+By **adding a few lines to your application code**, you can create a prompt playground that allows non-developers to experiment with prompts for your application and use all the tools within Agenta.
diff --git a/docs/docs/getting-started/02-quick-start.mdx b/docs/docs/getting-started/02-quick-start.mdx
index 0535b21280..a4b33d93ce 100644
--- a/docs/docs/getting-started/02-quick-start.mdx
+++ b/docs/docs/getting-started/02-quick-start.mdx
@@ -12,13 +12,13 @@ import TabItem from '@theme/TabItem';
This guide walks you through creating, evaluating, and deploying your first LLM application using Agenta's user interface. In just a few minutes, you'll have a working prompt that you can use in production.
- ⏯️ Watch a short demo of the quick start guide.
+ Prefer learning by watching? We have a **3-minute video** that covers the same steps.
:::info
-Agenta comes with a [set of built-in evaluators](/evaluation/concepts) that you can use to evaluate your prompt. You can also create your own custom evaluator [using code](/evaluation/configure-evaluators/custom-evaluator) or [webhooks](/evaluation/configure-evaluators/webhook-evaluator).
+Agenta comes with a [set of built-in evaluators](/evaluation/overview) that you can use to evaluate your prompt. You can also create your own custom evaluator [using code](/evaluation/evaluators/custom-evaluator) or [webhooks](/evaluation/evaluators/webhook-evaluator).
:::
### Analyzing the Results
@@ -199,7 +199,7 @@ Agenta provides two main ways to integrate with your application:
- **Use Agenta for Prompt Management**: Fetch the prompt configuration and use it in your own code
- **Use Agenta as a Proxy**: Use Agenta as a middleware that forwards requests to the LLM
-In this guide, we'll use the first approach. The second approach is covered in the [Integration Guide](/prompt-engineering/integrating-prompts/integrating-with-agenta). Its advantages are simpler integration and getting observability out of the box. The disadvantages are that it does not support streaming responses and adds a slight latency to the response (approximately 0.3 seconds).
+In this guide, we'll use the first approach. The second approach is covered in the [Integration Guide](/prompt-engineering/prompt-management/how-to-integrate-with-agenta). Its advantages are simpler integration and getting observability out of the box. The disadvantages are that it does not support streaming responses and adds a slight latency to the response (approximately 0.3 seconds).
:::
We are going to fetch the prompt configuration and use it in our own code:
@@ -279,7 +279,7 @@ response = client.chat.completions.create(
-
+
```javascript
const fetchConfigs = async () => {
@@ -444,6 +444,6 @@ Congratulations! You've successfully created, tested, deployed, and set up obser
Here are some next steps to explore:
-- [Set up your evaluation workflow](/evaluation/concepts)
+- [Set up your evaluation workflow](/evaluation/overview)
- [Set up observability](/observability/overview) and explore the different integrations
-- [Set up custom workflows](/custom-workflows/overview) to enable product teams to run evaluations on complex applications from the UI
+- [Set up custom workflows](/custom-workflows/overview) to enable product teams to run evaluations on complex applications from the UI
\ No newline at end of file
diff --git a/docs/docs/misc/02-faq.mdx b/docs/docs/misc/02-faq.mdx
index 7755a8f51e..d98b4dc8e6 100644
--- a/docs/docs/misc/02-faq.mdx
+++ b/docs/docs/misc/02-faq.mdx
@@ -24,7 +24,7 @@ Yes, Agenta can be used with TypeScript, though with varying levels of native su
Agenta is fully OpenTelemetry compliant. You can auto-instrument your TypeScript application using Opentelemetry compatible integrations such as [OpenLLMetry](https://github.com/traceloop/openllmetry-js) which works well with TypeScript projects.
- We have documentation on [setting up OpenTelemetry](/observability/trace-with-opentelemetry/distributed-tracing#opentelemetry-tracing-without-agenta-sdk) with your API key.
+ We have documentation on [setting up OpenTelemetry](/observability/opentelemetry#opentelemetry-tracing-without-agenta-sdk) with your API key.
3. **Evaluation**
@@ -60,7 +60,8 @@ In addition it works natively with
- [Azure OpenAI](https://azure.microsoft.com/en-us/products/ai-services/openai-service/)
- [Vertex AI](https://cloud.google.com/vertex-ai)
-You can add any OpenAI compatible endpoint, including self-hosted models and custom models (for instance using [Ollama](https://docs.agenta.ai/prompt-engineering/playground/custom-providers#configuring-openai-compatible-endpoints-eg-ollama)).
-You can also dynamically add new models to any provider already listed in the playground, such as [OpenRouter](/prompt-engineering/playground/custom-providers#adding-models-to-a-provider-eg-openrouter), Anthropic, Gemini, Cohere, and others.
+You can add any OpenAI compatible endpoint, including self-hosted models and custom models (for instance using [Ollama](https://docs.agenta.ai/prompt-engineering/playground/adding-custom-providers#configuring-openai-compatible-endpoints-eg-ollama)).
+You can also dynamically add new models to any provider already listed in the playground, such as [OpenRouter](/prompt-engineering/playground/adding-custom-providers#adding-models-to-a-provider-eg-openrouter), Anthropic, Gemini, Cohere, and others.
+
+You can learn more about setting up different models in the [documentation](/prompt-engineering/playground/adding-custom-providers).
-You can learn more about setting up different models in the [documentation](/prompt-engineering/playground/custom-providers).
diff --git a/docs/docs/observability/01-overview.mdx b/docs/docs/observability/01-overview.mdx
deleted file mode 100644
index c8f5b34955..0000000000
--- a/docs/docs/observability/01-overview.mdx
+++ /dev/null
@@ -1,201 +0,0 @@
----
-title: "Overview"
-sidebar_label: "Overview"
-description: "Complete guide to LLM observability with Agenta - monitor costs, performance, and trace every request with automatic instrumentation for your AI applications"
-sidebar_position: 2
----
-
-```mdx-code-block
-import CustomDocCard from '@site/src/components/CustomDocCard';
-import clsx from 'clsx';
-import Image from "@theme/IdealImage";
-```
-
-Observability is the practice of monitoring and understanding the behavior of your LLM application. With Agenta, you can add a few lines of code to start tracking all inputs, outputs, and metadata of your application.
-
-This allows you to:
-
-- **Debug Effectively**: View exact prompts sent and contexts retrieved. For complex workflows like agents, you can trace the data flow and quickly identify root causes.
-- **Bootstrap Test Sets**: Track real-world inputs and outputs and use them to bootstrap test sets in which you can continuously iterate.
-- **Find Edge Cases**: Identify latency spikes and cost increases. Understand performance bottlenecks to optimize your app's speed and cost-effectiveness.
-- **Track Costs and Latency Over Time**: Monitor how your app's expenses and response times change.
-- **Compare App Versions**: Compare the behavior in production of different versions of your application to see which performs better.
-
-
-
-## Observability in Agenta
-
-Agenta's observability features are built on **OpenTelemetry (OTel)**, an open-source standard for application observability. This provides several advantages:
-
-- **Wide Library Support**: Use many supported libraries right out of the box.
-- **Vendor Neutrality**: Send your traces to platforms like New Relic or Datadog without code changes. Switch vendors at will.
-- **Proven Reliability**: Use a mature and actively maintained SDK that's trusted in the industry.
-- **Ease of Integration**: If you're familiar with OTel, you already know how to instrument your app with Agenta. No new concepts or syntax to learn—Agenta uses familiar OTel concepts like traces and spans.
-
-## Next Steps
-
-
-
-
-
-
-
-
-
-
-
-
-### Integrations
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
diff --git a/docs/docs/observability/01-quickstart.mdx b/docs/docs/observability/01-quickstart.mdx
new file mode 100644
index 0000000000..1e00398f51
--- /dev/null
+++ b/docs/docs/observability/01-quickstart.mdx
@@ -0,0 +1,107 @@
+---
+title: Quick Start
+---
+
+```mdx-code-block
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+import Image from "@theme/IdealImage";
+
+```
+
+Agenta enables you to capture all inputs, outputs, and metadata from your LLM applications, **whether they're hosted within Agenta or running in your own environment**.
+
+This guide will walk you through setting up observability for an OpenAI application running locally.
+
+:::note
+If you create an application through the Agenta UI, tracing is enabled by default. No additional setup is required—simply go to the observability view to see all your requests.
+:::
+
+## Step-by-Step Guide
+
+### 1. Install Required Packages
+
+First, install the Agenta SDK, OpenAI, and the OpenTelemetry instrumentor for OpenAI:
+
+```bash
+pip install -U agenta openai opentelemetry-instrumentation-openai
+```
+
+### 2. Configure Environment Variables
+
+
+
+If you're using Agenta Cloud or Enterprise Edition, you'll need an API key:
+
+1. Visit the [Agenta API Keys page](https://cloud.agenta.ai/settings?tab=apiKeys).
+2. Click on **Create New API Key** and follow the prompts.
+
+```python
+import os
+
+os.environ["AGENTA_API_KEY"] = "YOUR_AGENTA_API_KEY"
+os.environ["AGENTA_HOST"] = "https://cloud.agenta.ai"
+```
+
+
+
+
+```python
+import os
+
+os.environ["AGENTA_HOST"] = "http://localhost"
+```
+
+
+
+
+### 3. Instrument Your Application
+
+Below is a sample script to instrument an OpenAI application:
+
+```python
+# highlight-start
+import agenta as ag
+from opentelemetry.instrumentation.openai import OpenAIInstrumentor
+import openai
+# highlight-end
+
+# highlight-start
+ag.init()
+# highlight-end
+
+# highlight-start
+OpenAIInstrumentor().instrument()
+# highlight-end
+
+response = openai.ChatCompletion.create(
+ model="gpt-3.5-turbo",
+ messages=[
+ {"role": "system", "content": "You are a helpful assistant."},
+ {"role": "user", "content": "Write a short story about AI Engineering."},
+ ],
+)
+
+print(response.choices[0].message.content)
+```
+
+**Explanation**:
+
+- **Import Libraries**: Import Agenta, OpenAI, and the OpenTelemetry instrumentor.
+- **Initialize Agenta**: Call `ag.init()` to initialize the Agenta SDK.
+- **Instrument OpenAI**: Use `OpenAIInstrumentor().instrument()` to enable tracing for OpenAI calls.
+
+### 4. View Traces in the Agenta UI
+
+After running your application, you can view the captured traces in Agenta:
+
+1. Log in to your Agenta dashboard.
+2. Navigate to the **Observability** section.
+3. You'll see a list of traces corresponding to your application's requests.
+
+
diff --git a/docs/docs/observability/02-overview.mdx b/docs/docs/observability/02-overview.mdx
new file mode 100644
index 0000000000..4fe0a2a962
--- /dev/null
+++ b/docs/docs/observability/02-overview.mdx
@@ -0,0 +1,124 @@
+---
+title: Overview
+description: Learn how to instrument your application with Agenta for enhanced observability. This guide covers the benefits of observability, how Agenta helps, and how to get started.
+---
+
+```mdx-code-block
+import DocCard from '@theme/DocCard';
+import clsx from 'clsx';
+import Image from "@theme/IdealImage";
+
+```
+
+## Why Observability?
+
+Observability is the practice of monitoring and understanding the behavior of your LLM application. With Agenta, you can add a few lines of code to start tracking all inputs, outputs, and metadata of your application.
+This allows you to:
+
+- **Debug Effectively**: View exact prompts sent and contexts retrieved. For complex workflows like agents, you can trace the data flow and quickly identify root causes.
+- **Bootstrap Test Sets**: Track real-world inputs and outputs and use them to bootstrap test sets in which you can continuously iterate.
+- **Find Edge Cases**: Identify latency spikes and cost increases. Understand performance bottlenecks to optimize your app's speed and cost-effectiveness.
+- **Track Costs and Latency Over Time**: Monitor how your app's expenses and response times change.
+- **Compare App Versions**: Compare the behavior in productions of different versions of your application to see which performs better.
+
+
+
+## Observability in Agenta
+
+Agenta's observability features are built on **OpenTelemetry (OTel)**, an open-source standard for application observability. This provides several advantages:
+
+- **Wide Library Support**: Use many supported libraries right out of the box.
+- **Vendor Neutrality**: Send your traces to platforms like New Relic or Datadog without code changes. Switch vendors at will.
+- **Proven Reliability**: Use a mature and actively maintained SDK that's trusted in the industry.
+- **Ease of Integration**: If you're familiar with OTel, you already know how to instrument your app with Agenta. No new concepts or syntax to learn—Agenta uses familiar OTel concepts like traces and spans.
+
+## Key Concepts
+
+**Traces**: A trace represents the complete journey of a request through your application. In our context, a trace corresponds to a single request to your LLM application.
+
+**Spans**: A span is a unit of work within a trace. Spans can be nested, forming a tree-like structure. The root span represents the overall operation, while child spans represent sub-operations. Agenta enriches each span with cost information and metadata when you make LLM calls.
+
+## Next Steps
+
+
+
+
+
+
+
+
+
+
+
+
+### Integrations
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/docs/docs/observability/02-quickstart-python.mdx b/docs/docs/observability/02-quickstart-python.mdx
deleted file mode 100644
index e947ed683a..0000000000
--- a/docs/docs/observability/02-quickstart-python.mdx
+++ /dev/null
@@ -1,97 +0,0 @@
----
-title: Quick Start Guide - Python LLM Observability
-sidebar_label: Quick Start (Python)
-description: Implement LLM observability in Python with Agenta SDK. Learn to set up distributed tracing, monitor requests, and debug your applications step-by-step.
-sidebar_position: 2
----
-
-```mdx-code-block
-import Tabs from '@theme/Tabs';
-import TabItem from '@theme/TabItem';
-import Image from "@theme/IdealImage";
-import GoogleColabButton from "@site/src/components/GoogleColabButton";
-
-```
-
-Agenta captures all inputs, outputs, and metadata from your LLM applications. You can use it whether your applications run inside Agenta or in your own environment.
-
-This guide shows you how to set up observability for an OpenAI application running locally.
-
-
-
- Open in Google Colaboratory
-
-
-## Step-by-Step Guide
-
-### 1. Install Required Packages
-
-Install the Agenta SDK, OpenAI, and the OpenTelemetry instrumentor for OpenAI:
-
-```bash
-pip install -U agenta openai opentelemetry-instrumentation-openai
-```
-
-### 2. Configure Environment Variables
-
-You need an API key to start tracing your application. Visit the Agenta API Keys page under settings. Click on Create New API Key and follow the prompts.
-
-```python
-import os
-
-os.environ["AGENTA_API_KEY"] = "YOUR_AGENTA_API_KEY"
-os.environ["AGENTA_HOST"] = "https://cloud.agenta.ai" # Change for self-hosted
-```
-
-### 3. Instrument Your Application
-
-Here is a sample script that instruments an OpenAI application:
-
-```python
-# highlight-start
-import agenta as ag
-from opentelemetry.instrumentation.openai import OpenAIInstrumentor
-import openai
-# highlight-end
-
-# highlight-start
-ag.init()
-# highlight-end
-
-# highlight-start
-OpenAIInstrumentor().instrument()
-# highlight-end
-
-# highlight-start
-@ag.instrument()
-# highlight-end
-def generate():
- response = openai.chat.completions.create(
- model="gpt-3.5-turbo",
- messages=[
- {"role": "system", "content": "You are a helpful assistant."},
- {"role": "user", "content": "Write a short story about AI Engineering."},
- ],
- )
- return response.choices[0].message.content
-
-if __name__ == "__main__":
- print(generate())
-```
-
-The script uses two mechanisms to trace your application.
-
-First, `OpenAIInstrumentor().instrument()` automatically traces all OpenAI calls. It monkey patches the OpenAI library to capture each request.
-
-Second, the `@ag.instrument()` decorator traces the decorated function. It creates a span for the function and records its inputs and outputs.
-
-### 4. View Traces in the Agenta UI
-
-After you run your application, you can view the captured traces in Agenta. Log in to your Agenta dashboard and navigate to the Observability section. You will see a list of traces that correspond to your application's requests.
-
-
diff --git a/docs/docs/observability/03-observability-sdk.mdx b/docs/docs/observability/03-observability-sdk.mdx
new file mode 100644
index 0000000000..175c99d7be
--- /dev/null
+++ b/docs/docs/observability/03-observability-sdk.mdx
@@ -0,0 +1,287 @@
+---
+title: Observability SDK
+---
+
+```mdx-code-block
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+import Image from "@theme/IdealImage";
+
+```
+
+This guide shows you how to use the Agenta observability SDK to instrument workflows in your application.
+
+If you're exclusively using a framework like [LangChain](/observability/integrations/langchain), you can use the auto-instrumentation packages to automatically instrument your application.
+
+However, if you need more flexibility, you can use the SDK to:
+
+- Instrument custom functions in your workflow
+- Add additional **metadata** to the spans
+- Link traces to **applications**, **variants**, and **environments** in Agenta
+
+## Setup
+
+**1. Install the Agenta SDK**
+
+```bash
+pip install -U agenta
+```
+
+**2. Set environment variables**
+
+
+
+
+1. Visit the [Agenta API Keys page](https://cloud.agenta.ai/settings?tab=apiKeys).
+2. Click on **Create New API Key** and follow the prompts.
+
+```python
+import os
+
+os.environ["AGENTA_API_KEY"] = "YOUR_AGENTA_API_KEY"
+os.environ["AGENTA_HOST"] = "https://cloud.agenta.ai"
+```
+
+
+
+
+```python
+import os
+
+os.environ["AGENTA_HOST"] = "http://localhost"
+```
+
+
+
+
+## Instrumenting functions with the decorator
+
+To instrument a function, add the `@ag.instrument()` decorator. This automatically captures all input and output data.
+
+The decorator has a `spankind` argument to categorize each span in the UI. Available types are:
+
+`agent`, `chain`, `workflow`, `tool`, `embedding`, `query`, `completion`, `chat`, `rerank`
+
+:::caution
+The instrument decorator should be the top-most decorator on a function (i.e. the last decorator before the function call).
+:::
+
+```python
+@ag.instrument(spankind="task")
+def myllmcall(country:str):
+ prompt = f"What is the capital of {country}"
+ response = client.chat.completions.create(
+ model='gpt-4',
+ messages=[
+ {'role': 'user', 'content': prompt},
+ ],
+ )
+ return response.choices[0].text
+
+@ag.instrument(spankind="workflow")
+def generate(country:str):
+ return myllmcall(country)
+
+```
+
+Agenta automatically determines the parent span based on the function call and nests the spans accordingly.
+
+## Modify a span's metadata
+
+You can add additional information to a span's metadata using `ag.tracing.store_meta()`. This function accesses the active span from the context and adds the key-value pairs to the metadata.
+
+```python
+@ag.instrument(spankind="task")
+def compile_prompt(country:str):
+ prompt = f"What is the capital of {country}"
+
+ # highlight-next-line
+ ag.tracing.store_meta({"prompt_template": prompt})
+
+ formatted_prompt = prompt.format(country=country)
+ return formatted_prompt
+
+```
+
+Metadata is displayed in the span's raw data view.
+
+## Linking spans to applications, variants, and environments
+
+You can link a span to an application, variant, and environment by calling `ag.tracing.store_refs()`.
+
+Applications, variants, and environments can be referenced by their slugs, versions, and commit IDs (for specific versions).
+You can link a span to an application and variant like this:
+
+```python
+
+@ag.instrument(spankind="workflow")
+def generate(country:str):
+ prompt = f"What is the capital of {country}"
+
+
+ formatted_prompt = prompt.format(country=country)
+
+ completion = client.chat.completions.create(
+ model='gpt-4',
+ messages=[
+ {'role': 'user', 'content': formatted_prompt},
+ ],
+ )
+
+ # highlight-start
+ ag.tracing.store_refs(
+ {
+ "application.slug": "capital-app",
+ "environment.slug": "production",
+ }
+ )
+ # highlight-end
+ return completion.choices[0].message.content
+
+```
+
+`ag.tracing.store_refs()` takes a dict with keys from `application.slug`, `application.id`, `application.version`, `variant.slug`, `variant.id`, `variant.version`, `environment.slug`, `environment.id` and `environment.commit_id`, with the values being the slug, id, version or commit id of the application, variant, and environment respectively.
+
+## Storing Internals
+
+Internals are additional data stored in the span. Compared to metadata, internals have the following differences:
+
+- Internals are saved within the span data and are searchable with plain text queries.
+- Internals are shown by default in the span view in a collapsible section, while metadata is only shown as part of the JSON file with the raw data (i.e. better visibility with internals).
+- **Internals can be used for evaluation**. For instance, you can save the retrieved context in the internals and then use it to evaluate the factuality of the response.
+
+As a rule of thumb, use metadata for additional information that is not used for evaluation and not elementary to understand the span, otherwise use internals.
+
+Internals can be stored similarly to metadata:
+
+```python
+@ag.instrument(spankind="workflow")
+def rag_workflow(query:str):
+
+ context = retrieve_context(query)
+
+ # highlight-start
+ ag.tracing.store_internals({"context": context})
+ # highlight-end
+
+ prompt = f"Answer the following question {query} based on the context: {context}"
+
+ completion = client.chat.completions.create(
+ model='gpt-4',
+ messages=[
+ {'role': 'user', 'content': formatted_prompt},
+ ],
+ )
+ return completion.choices[0].message.content
+
+```
+
+## Redacting sensitive data: how to exclude data from capture
+
+In some cases, you may want to exclude parts of the inputs or outputs due to privacy concerns or because the data is too large to be stored in the span.
+
+You can do this by setting the `ignore_inputs` and/or `ignore_outputs` arguments to `True` in the instrument decorator.
+
+```python
+@ag.instrument(
+ spankind="workflow",
+ ignore_inputs=True,
+ ignore_outputs=True
+)
+def rag_workflow(query:str):
+ ...
+```
+
+If you want more control, you can specify which parts of the inputs and outputs to exclude:
+
+```python
+@ag.instrument(
+ spankind="workflow",
+ ignore_inputs=["user_id"],
+ ignore_outputs=["pii"],
+)
+def rag_workflow(query:str, user_id:str):
+ ...
+ return {
+ "result": ...,
+ "pii": ...
+ }
+```
+
+For even finer control, you can use a custom `redact()` callback, along with instructions in the case of errors.
+
+```python
+def my_redact(name, field, data):
+ if name == "rag_workflow":
+ if field == "inputs":
+ del data["user_id"]
+ if field == "outputs":
+ del data["pii"]
+
+ return data
+
+
+@ag.instrument(
+ spankind="workflow",
+ redact=my_redact,
+ redact_on_error=False,
+)
+def rag_workflow(query:str, user_id:str):
+ ...
+ return {
+ "result": ...,
+ "pii": ...
+ }
+```
+
+Finally, if you want to set up global rules for redaction, you can provide a global `redact()` callback that applies everywhere.
+
+```python
+def global_redact(
+ name:str,
+ field:str,
+ data: Dict[str, Any]
+):
+ if "pii" in data:
+ del data["pii"]
+
+ return data
+
+
+ag.init(
+ redact=global_redact,
+ redact_on_error=True,
+)
+
+def local_redact(
+ name:str,
+ field:str,
+ data: Dict[str, Any]
+):
+ if name == "rag_workflow":
+ if field == "inputs":
+ del data["user_id"]
+
+ return data
+
+
+@ag.instrument(
+ spankind="workflow",
+ redact=local_redact,
+ redact_on_error=False,
+)
+def rag_workflow(query:str, user_id:str):
+ ...
+ return {
+ "result": ...,
+ "pii": ...
+ }
+```
+
+## Troubleshooting
+
+### Payload Too Large
+
+If your collector receives a **413** response when posting to `/otlp/v1/traces`, the batch size is too large. Agenta accepts batches up to **5 MB** by default.
+
+Reduce the batch size or enable compression in your collector configuration to keep requests under this limit.
diff --git a/docs/docs/observability/03-quick-start-opentelemetry.mdx b/docs/docs/observability/03-quick-start-opentelemetry.mdx
deleted file mode 100644
index 3b91f1f7a1..0000000000
--- a/docs/docs/observability/03-quick-start-opentelemetry.mdx
+++ /dev/null
@@ -1,245 +0,0 @@
----
-title: "Quick Start: OpenTelemetry for JavaScript/TypeScript (Node.js)"
-sidebar_label: "Quick Start (OpenTelemetry JS/TS)"
-description: "Set up LLM observability with OpenTelemetry in JavaScript, TypeScript, and Node.js. Learn how to instrument LLM apps, enable tracing and distributed tracing, and send telemetry to Agenta."
-sidebar_position: 3
----
-
-```mdx-code-block
-import Image from "@theme/IdealImage";
-import GitHubExampleButton from "@site/src/components/GitHubExampleButton";
-```
-
-Agenta captures all inputs, outputs, and metadata from your LLM applications using OpenTelemetry. This guide shows you how to instrument a Node.js application with OpenTelemetry and send traces to Agenta.
-
-
- View Complete Example on GitHub
-
-
-## Step-by-Step Guide
-
-### 1. Install Required Packages
-
-Install OpenTelemetry packages, OpenAI, and the OpenInference instrumentation for OpenAI:
-
-```bash
-npm install @opentelemetry/api \
- @opentelemetry/sdk-trace-node \
- @opentelemetry/exporter-trace-otlp-proto \
- @opentelemetry/instrumentation \
- @opentelemetry/resources \
- @opentelemetry/semantic-conventions \
- @arizeai/openinference-instrumentation-openai \
- @arizeai/openinference-semantic-conventions \
- openai
-```
-
-
-### 2. Configure Environment Variables
-
-You need an API key to start tracing your application. Visit the Agenta API Keys page under settings and create a new API key.
-
-```bash
-export AGENTA_API_KEY="YOUR_AGENTA_API_KEY"
-export AGENTA_HOST="https://cloud.agenta.ai" # Change for self-hosted
-export OPENAI_API_KEY="YOUR_OPENAI_API_KEY"
-```
-
-### 3. Set Up Instrumentation
-
-Create an `instrumentation.js` file to configure OpenTelemetry:
-
-```javascript
-// instrumentation.js
-// highlight-start
-import { registerInstrumentations } from "@opentelemetry/instrumentation";
-import { OpenAIInstrumentation } from "@arizeai/openinference-instrumentation-openai";
-import { OTLPTraceExporter } from "@opentelemetry/exporter-trace-otlp-proto";
-import { NodeTracerProvider } from "@opentelemetry/sdk-trace-node";
-import { Resource } from "@opentelemetry/resources";
-import { SimpleSpanProcessor } from "@opentelemetry/sdk-trace-base";
-import { ATTR_SERVICE_NAME } from "@opentelemetry/semantic-conventions";
-import OpenAI from "openai";
-// highlight-end
-
-// highlight-start
-// Configure the OTLP exporter to send traces to Agenta
-const otlpExporter = new OTLPTraceExporter({
- url: `${process.env.AGENTA_HOST}/api/otlp/v1/traces`,
- headers: {
- Authorization: `ApiKey ${process.env.AGENTA_API_KEY}`,
- },
-});
-// highlight-end
-
-// highlight-start
-// Create and configure the tracer provider
-const tracerProvider = new NodeTracerProvider({
- resource: new Resource({
- [ATTR_SERVICE_NAME]: "openai-quickstart",
- }),
-});
-
-// Use SimpleSpanProcessor for immediate export (better for short scripts)
-// For long-running services, use BatchSpanProcessor for better performance
-tracerProvider.addSpanProcessor(new SimpleSpanProcessor(otlpExporter));
-tracerProvider.register();
-// highlight-end
-
-// highlight-start
-// Register OpenAI instrumentation
-const instrumentation = new OpenAIInstrumentation();
-instrumentation.manuallyInstrument(OpenAI);
-
-registerInstrumentations({
- instrumentations: [instrumentation],
-});
-// highlight-end
-
-console.log("👀 OpenTelemetry instrumentation initialized");
-```
-
-### 4. Instrument Your Application
-
-Create your application file `app.js`:
-
-```javascript
-// app.js
-// highlight-start
-import OpenAI from "openai";
-import { trace } from "@opentelemetry/api";
-// highlight-end
-
-const openai = new OpenAI({
- apiKey: process.env.OPENAI_API_KEY,
-});
-
-// highlight-start
-const tracer = trace.getTracer("my-app", "1.0.0");
-// highlight-end
-
-async function generate() {
- // highlight-start
- // Create a span using Agenta's semantic conventions
- return tracer.startActiveSpan("generate", async (span) => {
- try {
- // Set span type
- span.setAttribute("ag.type.node", "workflow");
-
- const messages = [
- { role: "system", content: "You are a helpful assistant." },
- { role: "user", content: "Write a short story about AI Engineering." },
- ];
-
- // Set inputs
- span.setAttribute("ag.data.inputs", JSON.stringify({
- messages: messages,
- model: "gpt-3.5-turbo"
- }));
- // highlight-end
-
- const response = await openai.chat.completions.create({
- model: "gpt-3.5-turbo",
- messages: messages,
- });
-
- const content = response.choices[0].message.content;
-
- // highlight-start
- // Set outputs
- span.setAttribute("ag.data.outputs", JSON.stringify({
- content: content
- }));
-
- return content;
- } finally {
- span.end();
- }
- });
- // highlight-end
-}
-
-async function main() {
- const result = await generate();
- console.log(result);
-
- // Flush traces before exit
- await trace.getTracerProvider().forceFlush();
-}
-
-main();
-```
-
-### 5. Run Your Application
-
-Run your application with the instrumentation loaded first:
-
-```bash
-node --import ./instrumentation.js app.js
-```
-
-Or add it to your `package.json`:
-
-```json
-{
- "type": "module",
- "scripts": {
- "start": "node --import ./instrumentation.js app.js"
- }
-}
-```
-
-Then run:
-
-```bash
-npm start
-```
-
-## How It Works
-
-The instrumentation uses two mechanisms to trace your application:
-
-1. **Auto-instrumentation**: `OpenAIInstrumentation` automatically captures all OpenAI API calls, including prompts, completions, tokens, and costs.
-
-2. **Manual spans**: You can create custom spans using `tracer.startActiveSpan()` to track your own functions and add metadata using [Agenta's semantic conventions](/observability/trace-with-opentelemetry/semantic-conventions).
-
-:::tip Span Processors
-This guide uses `SimpleSpanProcessor` which sends spans immediately. This is ideal for:
-- Short-lived scripts and CLI tools
-- Development and debugging
-- Ensuring traces are captured before process exit
-
-For long-running services (web servers, background workers), use `BatchSpanProcessor` for better performance by batching multiple spans before sending.
-:::
-
-### Agenta Semantic Conventions
-
-The example uses Agenta's semantic conventions for proper trace display:
-
-- **`ag.type.node`** - Defines the operation type (workflow, task, tool, etc.)
-- **`ag.data.inputs`** - Stores input parameters as JSON
-- **`ag.data.outputs`** - Stores output results as JSON
-- **`ag.data.internals`** - Stores intermediate values and metadata (optional)
-
-## View Traces in the Agenta UI
-
-After running your application, log in to your Agenta dashboard and navigate to the Observability section. You will see traces showing:
-
-- Complete execution timeline
-- Input messages and parameters
-- Output content
-- Token usage and costs
-- Latency metrics
-
-
-
-## Next Steps
-
-- Learn about [semantic conventions](/observability/trace-with-opentelemetry/semantic-conventions) for better trace formatting
-- Explore [distributed tracing](/observability/trace-with-opentelemetry/distributed-tracing) across services
-- See [integration examples](/observability/integrations/openai) for other frameworks
diff --git a/docs/docs/observability/04-concepts.mdx b/docs/docs/observability/04-concepts.mdx
deleted file mode 100644
index 8a226e0935..0000000000
--- a/docs/docs/observability/04-concepts.mdx
+++ /dev/null
@@ -1,163 +0,0 @@
----
-title: "Concepts of LLM Observability in Agenta"
-sidebar_label: "Concepts"
-description: "Understand key observability concepts in Agenta including traces, spans, and how OpenTelemetry powers our observability features"
-sidebar_position: 3
----
-
-```mdx-code-block
-import Image from "@theme/IdealImage";
-```
-
-## Tracing in Agenta
-
-Agenta uses OpenTelemetry to track what happens in your LLM applications. OpenTelemetry is a free, open-source tool that makes monitoring applications easy. You write the monitoring code once, and it works with any observability platform. Read more about OpenTelemetry in [this guide we wrote for AI engineers](https://agenta.ai/blog/the-ai-engineer-s-guide-to-llm-observability-with-opentelemetry).
-
-
- ⏯️ Watch a video about OpenTelemetry and tracing in Agenta.
-
-
-
-
-## Getting Started: Basic Concepts
-
-### Traces
-
-A trace represents the complete journey of a request through your application. In our context, a trace corresponds to a single request to your LLM application.
-
-For example, when a user asks your chatbot a question, that entire interaction is captured as one trace. The trace includes receiving the query, processing it, and returning the response.
-
-### Spans
-
-A span is a unit of work within a trace. Spans can be nested, forming a tree-like structure.
-
-The **root span** represents the overall operation (like "handle user query"). **Child spans** represent sub-operations (like "retrieve context", "call LLM", or "format response").
-
-Agenta enriches each span with cost information for LLM calls, latency measurements, input/output data, and custom metadata you add.
-
-### Span Kinds
-
-Agenta categorizes spans using span kinds. These help you understand different types of operations in your LLM workflow.
-
-Available span kinds:
-- `agent` for autonomous agent operations
-- `chain` for sequential operations
-- `workflow` for complex multi-step processes
-- `tool` for tool or function calls
-- `embedding` for vector embedding generation
-- `query` for database or search queries
-- `completion` for LLM completions
-- `chat` for chat-based LLM interactions
-- `rerank` for re-ranking operations
-
-### Events
-
-Spans can contain events. These are timestamped records of things that happen during span execution. Agenta automatically logs exceptions as events, which helps you debug errors in your traces.
-
-## Working with OpenTelemetry
-
-### Direct OpenTelemetry Integration
-
-You can instrument your application using standard OpenTelemetry SDKs. Agenta accepts any OpenTelemetry span that follows the specification. For Agenta-specific features (like cost tracking and formatted messages), use attributes in our semantic conventions. See the [semantic conventions guide](/observability/trace-with-opentelemetry/semantic-conventions) for details.
-
-### Auto-Instrumentation Compatibility
-
-Agenta works with auto-instrumentation from popular libraries, even if they are not listed in our integrations. We support semantic conventions from [OpenInference](https://github.com/Arize-ai/openinference), [OpenLLMetry](https://github.com/traceloop/openllmetry), and [PydanticAI](/observability/integrations/pydanticai).
-
-When these libraries send spans to Agenta, we automatically translate their conventions to our format. No extra configuration is needed. This means that if you have any package that auto-instruments using these conventions, it will work with Agenta.
-
-## Understanding Span Types
-
-Agenta distinguishes between two types of spans. This separation helps you analyze application behavior independently from evaluation results.
-
-### Invocation Spans
-
-Invocation spans capture your application's actual work. They record what your LLM application does when it executes.
-
-Examples include LLM calls and completions, retrieval operations, tool executions, and agent reasoning steps.
-
-### Annotation Spans
-
-Annotation spans capture evaluations and feedback about invocations. They include automatic evaluations (like LLM-as-a-judge or custom metrics), human feedback and ratings, and evaluation results from test runs.
-
-When you evaluate a span or add feedback, Agenta creates an annotation span. The annotation span links to the original invocation span (explained in the Links section below). This keeps application traces clean while still capturing evaluation data.
-
-## Organizing and Filtering Traces
-
-### Attributes: Adding Metadata
-
-Attributes add information to spans. They are key-value pairs attached to each span. Agenta treats certain attributes specially for better UI experience.
-
-**Special attributes** use the `ag.` namespace. Cost and tokens get displayed prominently with user-friendly filtering. Model and system information appears in span details. Data attributes (inputs and outputs) are formatted based on span kind.
-
-**Custom attributes** can be any key-value pair you add. They are searchable and filterable, but they do not get special UI treatment.
-
-See all available attributes in our [semantic conventions guide](/observability/trace-with-opentelemetry/semantic-conventions).
-
-### References: Linking to Agenta Entities
-
-References connect spans to entities you have created in Agenta. They use a structured format and enable powerful organization.
-
-You can reference applications and their variants, environments (production, staging, development), test sets and test cases, and evaluators.
-
-Common use cases include filtering traces by application (like "show all traces from my chatbot-v2 variant"), comparing performance, and tracking prompt versions.
-
-Each reference can point to a specific variant and version. This gives you precise control over trace organization. References are especially useful for teams managing multiple applications and configurations.
-
-Learn more about using references in the [reference prompt versions guide](/observability/trace-with-python-sdk/reference-prompt-versions).
-
-### Links: Connecting Related Spans
-
-Links connect spans across different traces. Agenta uses them to connect annotations to invocations.
-
-When you evaluate a span, we cannot modify it because spans are immutable in OpenTelemetry. Instead, we create a new annotation span and link it to the original invocation span. This preserves the original trace while connecting evaluation results to the spans they evaluate.
-
-Links enable several features. You can view all evaluations for a specific application run. You can see feedback attached to the relevant invocation. You can filter traces by evaluation results.
-
-Links happen automatically when you use Agenta's evaluation features.
-
-## Applications, Variants, and Environments
-
-Agenta organizes your observability data around three key concepts:
-
-**Applications** are top-level containers for your LLM applications. An application could be a chatbot, a summarization tool, or any other LLM-powered feature.
-
-**Variants** are different versions or configurations of your application. You might have a "gpt-4-turbo" variant and a "claude-opus" variant. Or you might have variants for different prompts or parameters.
-
-**Environments** are deployment stages. Common environments include development, staging, and production.
-
-This organization helps you compare performance across different configurations and track behavior in different environments.
-
-## How Agenta Enhances OpenTelemetry
-
-Agenta uses standard OpenTelemetry for tracing. We add LLM-specific enhancements on top of it.
-
-### Automatic Cost Tracking and Token Counting
-
-We calculate costs for LLM calls based on model pricing. We track token usage (prompt tokens, completion tokens, and total) for each interaction. These metrics appear prominently in the UI and support user-friendly filtering.
-
-### Prompt Versioning Integration
-
-You can link traces to specific prompt versions in your registry. This helps you understand which prompt configuration generated each trace.
-
-### Test Set Integration
-
-You can convert production traces into test cases with one click. This makes it easy to build test sets from real user interactions.
-
-### LLM-Aware UI
-
-The Agenta UI understands LLM-specific data. Chat messages are formatted nicely. You can filter by cost, tokens, model, and other LLM-specific attributes. The UI shows parent-child relationships in your agent workflows clearly.
-
-## Next Steps
-
-- [Get started with Python SDK](/observability/quickstart-python)
-- [Learn about tracing with OpenTelemetry](/observability/trace-with-opentelemetry/getting-started)
-- [Explore integrations](/observability/integrations/openai) for popular LLM frameworks
diff --git a/docs/docs/observability/04-opentelemetry.mdx b/docs/docs/observability/04-opentelemetry.mdx
new file mode 100644
index 0000000000..8fc4f076d1
--- /dev/null
+++ b/docs/docs/observability/04-opentelemetry.mdx
@@ -0,0 +1,192 @@
+---
+title: Distributed Tracing with OpenTelemetry
+sidebar_label: Distributed Tracing in Otel
+description: "Learn how to use OpenTelemetry to instrument your LLM application with Agenta for enhanced observability."
+---
+
+Agenta provides built-in OpenTelemetry instrumentation to simplify distributed tracing in your applications. This guide explains how to implement and manage distributed tracing using the Agenta SDK, and how to integrate external tracing setups with Agenta.
+
+## Using OpenTelemetry with Agenta
+Agenta supports distributed tracing out of the box when using the provided SDK functions:
+
+### 1. Sending Requests (Propagation)
+
+When making requests to other services or sub-systems, use `agenta.tracing.inject()` to inject necessary headers:
+
+```python
+method = "POST"
+url = "https://example-service/api"
+params = {}
+headers = agenta.tracing.inject() # automatically injects 'Authorization', 'Traceparent', 'Baggage'
+body = {"key": "value"}
+
+response = requests.request(
+ method=method,
+ url=url,
+ params=params,
+ headers=headers,
+ json=body,
+)
+```
+The `agenta.tracing.inject()` function returns headers containing:
+
+- `Authorization`: Authentication information
+- `Traceparent`: Identifies the current trace and span
+- `Baggage`: Contains application-specific context
+
+These headers can be modified before sending them as part of the request if needed.
+
+### 2. Receiving Requests (Extraction)
+
+Agenta simplifies receiving and handling incoming trace contexts:
+
+- If you're using `ag.route()` and `ag.instrument()`, extraction is automatic.
+- For manual extraction, use `agenta.tracing.extract()`:
+
+```python
+traceparent, baggage = agenta.tracing.extract() # includes 'Traceparent', 'Baggage'
+
+# Use traceparent and baggage to set up your OpenTelemetry context
+# (Implementation depends on your specific use case)
+```
+
+:::note
+`extract()` does not provide `Authorization` because there are many authentication methods (apikey, bearer, secret, access tokens), each requiring different handling. The middlewares and decorators in the Agenta SDK handle this automatically when you use `ag.route()` and `ag.instrument()`.
+:::
+
+## OpenTelemetry Tracing without Agenta SDK
+If you're working with systems that don't use the Agenta SDK, you can still integrate with Agenta's tracing infrastructure using standard OpenTelemetry.
+
+### 1. Setup Requirements
+Install dependencies:
+```bash
+pip install opentelemetry-api opentelemetry-sdk opentelemetry-exporter-otlp
+```
+
+### 2. Configure Environment Variables
+
+```bash
+# OTEL_PROPAGATORS = unset or "tracecontext,baggage"
+# OTEL_EXPORTER_OTLP_COMPRESSION = unset or "gzip"
+# OTEL_EXPORTER_OTLP_ENDPOINT = "https://cloud.agenta.ai/api/otlp"
+# OTEL_EXPORTER_OTLP_HEADERS = "authorization=ApiKey xxx"
+# OTEL_EXPORTER_OTLP_TRACES_ENDPOINT = "https://cloud.agenta.ai/api/otlp/v1/traces"
+# OTEL_EXPORTER_OTLP_TRACES_HEADERS = "authorization=ApiKey xxx"
+```
+
+### 3. Setup in Code
+```python
+from opentelemetry.trace.propagation.tracecontext import TraceContextTextMapPropagator
+from opentelemetry.baggage.propagation import W3CBaggagePropagator
+from opentelemetry.sdk.trace import TracerProvider, Span
+from opentelemetry.sdk.trace.export import BatchSpanProcessor
+from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter, Compression
+
+# Configuration
+endpoint = "https://cloud.agenta.ai/api/otlp/v1/traces"
+compression = Compression.Gzip
+headers = {
+ "traceparent": "00-xxx-xxx-01",
+ "baggage": "ag.refs.application.id=xxx",
+ "authorization": "ApiKey xxx",
+}
+
+# Set up provider, processor, and tracer
+provider = TracerProvider()
+
+processor = BatchSpanProcessor(
+ OTLPSpanExporter(
+ endpoint=endpoint,
+ headers={"authorization": headers["authorization"]},
+ compression=compression,
+ )
+)
+
+provider.add_span_processor(processor)
+
+tracer = provider.get_tracer("agenta.tracer")
+
+# Extract incoming trace context
+carrier = {"traceparent": headers["traceparent"]}
+context = TraceContextTextMapPropagator().extract(carrier=carrier, context=None)
+
+carrier = {"baggage": headers["baggage"]}
+context = W3CBaggagePropagator().extract(carrier=carrier, context=context)
+
+# Create and use spans
+with tracer.start_as_current_span(name="agenta", context=context) as span:
+ span: Span
+
+ print(hex(span.get_span_context().trace_id))
+ print(hex(span.get_span_context().span_id))
+ print(span.name)
+```
+
+## Using an OTEL Collector
+
+If you're using an OpenTelemetry collector, you can configure it to export traces to Agenta.
+Here's a sample configuration (`otel-collector-config.yml`):
+
+```yaml
+receivers:
+ otlp:
+ protocols:
+ http:
+ endpoint: 0.0.0.0:4318
+
+processors:
+ batch: {}
+
+exporters:
+ otlphttp/agenta:
+ endpoint: "https://cloud.agenta.ai/api/otlp"
+
+service:
+ pipelines:
+ traces:
+ receivers: [otlp]
+ processors: [batch]
+ exporters: [otlphttp/agenta]
+```
+
+With this configuration:
+
+- The collector receives traces via OTLP/HTTP on port 4318
+- It batches the spans for efficiency
+- It exports them to Agenta's OTLP endpoint
+
+## Span Attributes
+
+When using OpenTelemetry without the Agenta SDK, you need to manually set the appropriate attributes on your spans to integrate properly with Agenta's ecosystem.
+
+### Namespace Convention
+
+Agenta uses the `ag.*` namespace for its attributes. Here are the key namespaces:
+
+- `ag.refs.*`: References to Agenta entities (applications, etc.)
+- `ag.data.*`: Input, internal, and output data
+- `ag.metrics.*`: Performance metrics and costs
+
+### Examples
+
+```python
+# Reference to Agenta application
+span.set_attribute("ag.refs.application.id", AGENTA_APPLICATION_ID)
+
+# Data attributes
+span.set_attribute("ag.data.inputs.key", "Hello,")
+span.set_attribute("ag.data.internals.key", "(Leo)")
+span.set_attribute("ag.data.outputs.key", "World!")
+
+# Metrics - unit values
+span.set_attribute("ag.metrics.unit.some_key", 3)
+span.set_attribute("ag.metrics.acc.some_key", 15)
+
+# Cost and token metrics
+span.set_attribute("ag.metrics.unit.costs.total", 1)
+span.set_attribute("ag.metrics.unit.tokens.total", 100)
+```
+
+:::info
+Apart from these custom attributes, standard OpenTelemetry events, links, status, and exceptions work as usual.
+:::
\ No newline at end of file
diff --git a/docs/docs/observability/05-otel-semconv.mdx b/docs/docs/observability/05-otel-semconv.mdx
new file mode 100644
index 0000000000..440657179c
--- /dev/null
+++ b/docs/docs/observability/05-otel-semconv.mdx
@@ -0,0 +1,251 @@
+---
+title: Agenta OpenTelemetry Semantic Conventions
+sidebar_label: Otel Semantic Conventions
+description: "Learn about the OpenTelemetry semantic conventions used by Agenta's LLM observability system."
+---
+
+This document describes how Agenta applies domain-specific OpenTelemetry conventions to capture and analyze traces from LLM applications.
+
+## How It Works
+Agenta accepts any span that follows the OpenTelemetry specification. To unlock LLM-specific features—such as nicely formatted chat messages, per-request cost and latency, and links to prompt configurations or evaluators—we add attributes under the `ag.` namespace.
+
+:::info
+The OTLP endpoint accepts batches up to **5 MB** by default (after decompression). Larger requests return an HTTP `413` status code.
+:::
+
+We support two primary instrumentation approaches:
+
+- **Agenta SDK:** When using our SDK with decorators like `@ag.instrument`, we automatically handle the semantic conventions
+- **Auto-instrumentation libraries:** We provide adapters that transform data from standard auto-instrumentation libraries (which follow their own conventions) into Agenta's format
+
+## Agenta Namespace
+
+All Agenta-specific attributes are organized under the `ag` namespace to avoid conflicts with other OpenTelemetry conventions.
+
+## Core Semantic Conventions
+
+### ag.data
+
+The `ag.data` namespace contains the core execution data for each span:
+
+- **`ag.data.inputs`**: Input parameters for the span
+- **`ag.data.outputs`**: Output results from the span
+- **`ag.data.internals`**: Internal variables and intermediate values
+
+#### Data Format
+
+**Inputs and Outputs**: Stored as JSON format. For chat applications using LLM spans, these are formatted as a list of messages:
+
+```json
+{
+ "data": {
+ "inputs": {
+ "prompt": [
+ { "role": "system", "content": "System instruction" },
+ { "role": "user", "content": "User query" }
+ ],
+ "functions": [...],
+ "tools": [...]
+ },
+ "outputs": {
+ "completion": [
+ { "role": "assistant", "content": "Assistant response" }
+ ]
+ }
+ }
+}
+```
+
+
+**Internals**: User-provided internal information such as context variables, intermediate calculations, or evaluation data that aren't part of the primary inputs/outputs. These are set by the user [in the SDK using](/observability/observability-sdk#storing-internals) `ag.store_internals()`.
+
+#### SDK Integration
+
+When using the `@ag.instrument` decorator in Python:
+
+```python
+@ag.instrument
+def my_function(input_param):
+ # Function inputs and outputs are automatically captured
+ # unless explicitly masked
+ return result
+```
+
+The decorator automatically captures function inputs and outputs in `ag.data.inputs` and `ag.data.outputs` unless you choose to [mask sensitive data](/observability/observability-sdk).
+
+### ag.meta
+
+The `ag.meta` namespace stores metadata about the span execution:
+
+```json
+{
+ "meta": {
+ "system": "openai",
+ "request": {
+ "base_url": "https://api.openai.com/v1",
+ "endpoint": "/chat/completions",
+ "headers": {...},
+ "streaming": false,
+ "model": "gpt-4",
+ "max_tokens": 1000,
+ "temperature": 0.7,
+ "top_p": 0.9,
+ "top_k": 50
+ },
+ "response": {
+ "model": "gpt-4-0613"
+ },
+ "configuration": {
+ "prompt": {
+ "input_keys": ["country"],
+ "messages": [...],
+ "template_format": "curly",
+ "llm_config": {
+ "model": "mistral/mistral-tiny"
+ }
+ }
+ }
+ }
+}
+```
+
+#### Standard Metadata
+
+:::info
+Auto-instrumentation maps common semantic-convention keys—e.g. gen_ai.system, gen_ai.request.*—to the structure above.
+:::
+
+- `ag.meta.system`: Identifies the LLM provider or system being used (e.g., "openai", "anthropic")
+- `ag.meta.request.base_url`: The base URL for the API request
+- `ag.meta.request.endpoint`: The specific API endpoint called
+- `ag.meta.request.headers`: HTTP headers sent with the request
+- `ag.meta.request.streaming`: Boolean indicating if streaming mode was used
+- `ag.meta.request.model`: The model name requested
+- `ag.meta.request.max_tokens`: Maximum token limit for the response
+- `ag.meta.request.temperature`: Sampling temperature parameter
+- `ag.meta.request.top_p`: Nucleus sampling parameter
+- `ag.meta.request.top_k`: Top-k sampling parameter
+
+Metadata is displayed in the observability overview page as contextual information to help navigate and understand span execution.
+
+### ag.refs
+
+The `ag.refs` namespace contains references to entities within the Agenta system:
+
+```json
+{
+ "refs": {
+ "application": {
+ "id": "uuid"
+ },
+ "variant": {
+ "id": "uuid",
+ "version": "1"
+ },
+ "environment": {
+ "slug": "production",
+ "id": "uuid"
+ }
+ }
+}
+```
+
+#### Reference Types
+
+- **Application**: Links to the Agenta application
+- **Variant**: References specific prompt variants and versions
+- **Environment**: Links to deployment environments (production, staging, etc.)
+- **Test Set**: References to test datasets
+- **Test Case**: Links to individual test cases
+- **Evaluation Run**: References to evaluation executions
+- **Evaluator**: Links to evaluation functions
+
+These references enable navigation within the Agenta UI and allow filtering spans by specific entities.
+
+### ag.metrics
+
+The `ag.metrics` namespace tracks performance and cost metrics:
+
+```json
+{
+ "metrics": {
+ "acc": {
+ "costs": {
+ "total": 0.00003925
+ },
+ "tokens": {
+ "total": 157,
+ "prompt": 26,
+ "completion": 131
+ },
+ "duration": {
+ "total": 1251.157
+ }
+ },
+ "unit": {
+ "costs": {
+ "total": 0.00003925
+ },
+ "tokens": {
+ "total": 157,
+ "prompt": 26,
+ "completion": 131
+ }
+ }
+ }
+}
+```
+
+#### Metric Types
+
+- **Accumulated (`acc`)**: Metrics rolled up across child spans
+- **Unit (`unit`)**: Metrics specific to the individual span
+
+#### Available Metrics
+
+- **Costs**: Total cost in USD for LLM API calls
+- **Tokens**: Token usage breakdown (total, prompt, completion)
+- **Duration**: Execution time in milliseconds
+
+## Additional Agenta Attributes
+
+### ag.type
+
+The `ag.type` namespace contains type information about the span:
+
+`ag.type.node` can be `workflow`, `task`, `tool`, `embedding`, `query`, `completion`, `chat`, `rerank`.
+
+
+### ag.tags
+
+The `ag.tags` namespace allows you to add custom tags to spans for categorization and filtering:
+
+```python
+# Add custom tags using the SDK
+ag.add_tags({"key":"value"})
+```
+
+### ag.version
+
+Tracks version information for the instrumented application or component.
+
+### ag.flags
+
+Contains system-specific flags and metadata used internally by Agenta for span processing and display.
+
+:::note
+When using auto-instrumentation libraries, most attributes are saved twice - once in their original format and once processed under the `ag` namespace
+:::
+
+## Standard OpenTelemetry Attributes
+
+In addition to Agenta-specific conventions, traces include standard OpenTelemetry attributes:
+
+- **Links**: Relationships between spans
+- **Events**: Timestamped events within spans
+- **Version**: OpenTelemetry version information
+- **Status Code**: Span completion status
+- **Start Time**: Span initiation timestamp
+- **Span Name**: Human-readable span identifier
+- **Span Kind**: Type of span (server, client, internal, etc.)
+
diff --git a/docs/docs/observability/09-troubleshooting.mdx b/docs/docs/observability/09-troubleshooting.mdx
deleted file mode 100644
index 1894bf298d..0000000000
--- a/docs/docs/observability/09-troubleshooting.mdx
+++ /dev/null
@@ -1,100 +0,0 @@
----
-title: "Troubleshooting"
-sidebar_label: "Troubleshooting"
-description: "Common issues and solutions for Agenta observability with Python SDK and OpenTelemetry"
-sidebar_position: 9
----
-
-## Common Issues
-
-This page covers common issues you might encounter when using Agenta observability with either the Python SDK or OpenTelemetry.
-
-## Invalid Content Format
-
-You may receive a 500 error with the message "Failed to parse OTLP stream." This happens when you send trace data in JSON format instead of Protobuf.
-
-Agenta's OTLP endpoints accept only the Protobuf format (binary encoding). The server cannot parse JSON payloads. When you configure your OpenTelemetry exporter to use JSON encoding, the request will fail.
-
-### Solution
-
-Configure your OpenTelemetry exporter to use Protobuf encoding. Most OpenTelemetry exporters use Protobuf by default, so you typically don't need to specify it explicitly.
-
-For the Python SDK, import `OTLPSpanExporter` from the proto package (not the json package). The exporter class name should include "proto" in its import path.
-
-For the OpenTelemetry Collector, verify that the encoding field is set to `proto` or omit it entirely (since proto is the default).
-
-For JavaScript and Node.js, use `@opentelemetry/exporter-trace-otlp-proto` instead of the JSON variant. Avoid any exporter package with "json" in its name.
-
-Do not set `encoding: json` in your configuration files. Agenta does not support JSON-encoded OTLP payloads.
-
-## Payload Too Large
-
-Your collector may receive a 413 response when posting to `/otlp/v1/traces`. This means the batch size exceeds the limit. Agenta accepts batches up to 5 MB by default.
-
-Reduce the batch size in your collector configuration. You can also enable compression (such as gzip) to keep requests under the limit.
-
-## Missing Traces in Serverless Functions
-
-Some traces may not appear in the Agenta dashboard when you run observability in serverless environments. This includes AWS Lambda, Vercel Functions, Cloudflare Workers, and Google Cloud Functions.
-
-OpenTelemetry batches spans in background processes before exporting them. This improves efficiency. However, serverless functions terminate abruptly. They often stop before the background processes finish sending trace data to Agenta. The spans get buffered but never exported.
-
-### Solution
-
-Call the `force_flush()` method before your function terminates. This ensures all spans export before the function exits. Import `get_tracer_provider` from `opentelemetry.trace` and call `force_flush()` on it. Place this call in a finally block so it runs even if errors occur.
-
-## Traces Not Appearing in UI
-
-First, verify that your `AGENTA_API_KEY` is set correctly and has the necessary permissions.
-
-Next, check your endpoint configuration. Point to the correct Agenta host. For cloud deployments, use `https://cloud.agenta.ai`. For self-hosted instances, use your instance URL (such as `http://localhost`).
-
-Finally, confirm that you call `ag.init()` before any instrumented functions execute.
-
-## Authentication Errors
-
-You may receive 401 Unauthorized errors when sending traces. This happens for three main reasons.
-
-First, verify that your API key is correct. Check for typos or missing characters.
-
-Second, confirm that the key has not expired. Some API keys have expiration dates.
-
-Third, ensure you use the correct format. The authorization header should follow this pattern: `ApiKey YOUR_KEY_HERE`.
-
-## Performance Issues
-
-### High memory usage
-
-You can reduce memory usage in three ways. Enable gzip compression for OTLP exports to reduce the size of data in memory. Lower the number of spans sent per batch. Implement sampling to avoid sending 100% of traces in high-volume scenarios.
-
-### High latency
-
-Instrumentation should not add significant latency. If it does, check three things. Ensure spans export in the background using async export. Tune your batch size and export intervals to find the right balance. Review custom instrumentation to verify that custom spans are not performing expensive operations.
-
-## OpenTelemetry-Specific Issues
-
-### Context propagation not working
-
-Distributed tracing may fail to work across services. Check three things to fix this. Verify that propagators are configured correctly (set `OTEL_PROPAGATORS=tracecontext,baggage`). Confirm that headers pass between services. Ensure all services use compatible OpenTelemetry versions.
-
-### Spans not nesting correctly
-
-Spans may appear flat instead of nested in the trace view. This indicates a context problem. Verify that context passes correctly between functions. Check that you use `start_as_current_span` with proper context. Make parent-child relationships explicit in your code.
-
-## Python SDK-Specific Issues
-
-### Decorator not capturing data
-
-The `@ag.instrument()` decorator may fail to capture inputs and outputs. This happens for three reasons. The decorator must be the top-most decorator on your function. You must call `ag.init()` before the function runs. The function must return a value (printing alone is not enough).
-
-### Metadata not appearing
-
-Data from `ag.tracing.store_meta()` may not show in the UI. Call this method only within an instrumented function. Check that the span context is active when you call it. Verify that your data format is JSON-serializable.
-
-## Need More Help?
-
-Check the [Agenta documentation](/observability/concepts) for more details about observability concepts. Review our [integration guides](/observability/integrations/openai) for framework-specific help. Visit our [GitHub issues](https://github.com/anthropics/agenta/issues) to report bugs. Join our community for support.
-
-## Next steps
-
-Review the [setup instructions](/observability/trace-with-python-sdk/setup-tracing) to ensure correct configuration. Explore [distributed tracing](/observability/trace-with-opentelemetry/distributed-tracing) to understand how traces work across services. Check the [integrations](/observability/integrations/openai) page for your specific framework.
diff --git a/docs/docs/observability/_using-the-ui/01-filtering-traces.mdx b/docs/docs/observability/_using-the-ui/01-filtering-traces.mdx
deleted file mode 100644
index 8a7e64686b..0000000000
--- a/docs/docs/observability/_using-the-ui/01-filtering-traces.mdx
+++ /dev/null
@@ -1,17 +0,0 @@
----
-title: "Filtering Traces"
-sidebar_label: "Filtering Traces"
-description: "Learn how to filter and search traces in the Agenta observability UI"
-sidebar_position: 1
----
-
-
-
-## Overview
-
-Learn how to filter and search traces in the Agenta observability UI
-
-## Next steps
-
-- Explore [query data options](/observability/query-data/query-api)
-- Learn about [Python SDK tracing](/observability/trace-with-python-sdk/setup-tracing)
diff --git a/docs/docs/observability/_using-the-ui/02-adding-comments.mdx b/docs/docs/observability/_using-the-ui/02-adding-comments.mdx
deleted file mode 100644
index 98e180d2a2..0000000000
--- a/docs/docs/observability/_using-the-ui/02-adding-comments.mdx
+++ /dev/null
@@ -1,17 +0,0 @@
----
-title: "Adding Comments"
-sidebar_label: "Adding Comments"
-description: "Learn how to add comments and notes to traces for team collaboration"
-sidebar_position: 2
----
-
-
-
-## Overview
-
-Learn how to add comments and notes to traces for team collaboration
-
-## Next steps
-
-- Explore [query data options](/observability/query-data/query-api)
-- Learn about [Python SDK tracing](/observability/trace-with-python-sdk/setup-tracing)
diff --git a/docs/docs/observability/_using-the-ui/03-adding-annotations.mdx b/docs/docs/observability/_using-the-ui/03-adding-annotations.mdx
deleted file mode 100644
index 99fe14cdcd..0000000000
--- a/docs/docs/observability/_using-the-ui/03-adding-annotations.mdx
+++ /dev/null
@@ -1,17 +0,0 @@
----
-title: "Adding Annotations"
-sidebar_label: "Adding Annotations"
-description: "Learn how to annotate traces with custom evaluations and feedback"
-sidebar_position: 3
----
-
-
-
-## Overview
-
-Learn how to annotate traces with custom evaluations and feedback
-
-## Next steps
-
-- Explore [query data options](/observability/query-data/query-api)
-- Learn about [Python SDK tracing](/observability/trace-with-python-sdk/setup-tracing)
diff --git a/docs/docs/observability/_using-the-ui/04-exporting-data.mdx b/docs/docs/observability/_using-the-ui/04-exporting-data.mdx
deleted file mode 100644
index 47275e0d0c..0000000000
--- a/docs/docs/observability/_using-the-ui/04-exporting-data.mdx
+++ /dev/null
@@ -1,17 +0,0 @@
----
-title: "Exporting Data"
-sidebar_label: "Exporting Data"
-description: "Learn how to export trace data from the Agenta UI for analysis"
-sidebar_position: 4
----
-
-
-
-## Overview
-
-Learn how to export trace data from the Agenta UI for analysis
-
-## Next steps
-
-- Explore [query data options](/observability/query-data/query-api)
-- Learn about [Python SDK tracing](/observability/trace-with-python-sdk/setup-tracing)
diff --git a/docs/docs/observability/_using-the-ui/_category_.json b/docs/docs/observability/_using-the-ui/_category_.json
deleted file mode 100644
index 91b2c97fee..0000000000
--- a/docs/docs/observability/_using-the-ui/_category_.json
+++ /dev/null
@@ -1,6 +0,0 @@
-{
- "label": "Using the UI",
- "position": 7,
- "collapsible": true,
- "collapsed": true
-}
diff --git a/docs/docs/observability/integrations/01-openai.mdx b/docs/docs/observability/integrations/01-openai.mdx
index 24f5f3fb39..84000d407b 100644
--- a/docs/docs/observability/integrations/01-openai.mdx
+++ b/docs/docs/observability/integrations/01-openai.mdx
@@ -1,7 +1,5 @@
---
-title: Tracing and Observability for OpenAI with Agenta
-sidebar_label: OpenAI
-description: Learn how to instrument OpenAI traces with Agenta for enhanced LLM observability. This guide covers setup, configuration, and best practices for monitoring OpenAI API calls and workflows.
+title: OpenAI
---
```mdx-code-block
diff --git a/docs/docs/observability/integrations/02-langchain.mdx b/docs/docs/observability/integrations/02-langchain.mdx
index 8d365ce859..4f94b2ede3 100644
--- a/docs/docs/observability/integrations/02-langchain.mdx
+++ b/docs/docs/observability/integrations/02-langchain.mdx
@@ -1,6 +1,5 @@
---
-title: Tracing and Observability for LangChain with Agenta
-sidebar_label: LangChain
+title: LangChain
description: Learn how to instrument LangChain traces with Agenta for enhanced LLM observability. This guide covers setup, configuration, and best practices for monitoring LLM applications using LangChain and OpenAI models.
---
diff --git a/docs/docs/observability/integrations/04-instructor.mdx b/docs/docs/observability/integrations/04-instructor.mdx
index 0b07fc0b19..20fe4f129f 100644
--- a/docs/docs/observability/integrations/04-instructor.mdx
+++ b/docs/docs/observability/integrations/04-instructor.mdx
@@ -1,6 +1,5 @@
---
-title: Tracing and Observability for Instructor with Agenta
-sidebar_label: Instructor
+title: Instructor
description: Learn how to implement and monitor Instructor traces with Agenta for enhanced LLM observability. This comprehensive guide covers setup, configuration, and best practices for tracking structured data extraction using Instructor and OpenAI models.
---
diff --git a/docs/docs/observability/integrations/05-litellm.mdx b/docs/docs/observability/integrations/05-litellm.mdx
index 85f7edd07a..019f9b7892 100644
--- a/docs/docs/observability/integrations/05-litellm.mdx
+++ b/docs/docs/observability/integrations/05-litellm.mdx
@@ -1,6 +1,5 @@
---
-title: Tracing and Observability for LiteLLM with Agenta
-sidebar_label: LiteLLM
+title: LiteLLM
description: Learn how to instrument LiteLLM traces with Agenta for enhanced LLM observability. This guide covers setup, configuration, and best practices for monitoring API calls and performance using LiteLLM and OpenAI models.
---
diff --git a/docs/docs/observability/integrations/06-llamaindex.mdx b/docs/docs/observability/integrations/06-llamaindex.mdx
index 5253e193e3..d1074cc474 100644
--- a/docs/docs/observability/integrations/06-llamaindex.mdx
+++ b/docs/docs/observability/integrations/06-llamaindex.mdx
@@ -1,24 +1,26 @@
---
-title: Tracing and Observability for LlamaIndex with Agenta
+title: Observability for LlamaIndex with Agenta
sidebar_label: LlamaIndex
description: Monitor and debug your LlamaIndex applications with Agenta's observability platform. Get complete visibility into your RAG pipelines and document search workflows.
---
```mdx-code-block
import Image from "@theme/IdealImage";
-import GoogleColabButton from "@site/src/components/GoogleColabButton";
```
+
+# Observability for LlamaIndex with Agenta
+
+:::info
+This guide is available as Jupyter Notebook [here](https://github.com/agenta-ai/agenta/blob/main/examples/jupyter/integrations/observability-openinference-llamaindex.ipynb).
+:::
+
Learn how to monitor your **[LlamaIndex](https://www.llamaindex.ai/)** applications using **Agenta's** observability platform. Get complete visibility into your document search, RAG pipelines, and LLM interactions.
> **What is Agenta?** [Agenta](https://agenta.ai) is an open-source LLMOps platform designed to streamline the deployment, management, and scaling of large language models. It offers comprehensive observability, testing, and deployment capabilities for AI applications.
> **What is LlamaIndex?** [LlamaIndex](https://www.llamaindex.ai/) ([GitHub](https://github.com/run-llama/llama_index)) is a powerful data framework that connects LLMs with your private data sources. It simplifies working with various data formats (PDFs, APIs, databases, documents) and creates searchable indices for context-aware AI applications.
-
- Open in Google Colaboratory
-
-
### 1. Install Required Packages
Install the necessary dependencies for the integration:
@@ -238,4 +240,4 @@ def rag_pipeline(query: str):
## Next Steps
-For more advanced observability features and configuration options, see our [complete observability documentation](/observability/trace-with-python-sdk/setup-tracing).
+For more advanced observability features and configuration options, see our [complete observability documentation](/observability/observability-sdk).
diff --git a/docs/docs/observability/integrations/07-langgraph.mdx b/docs/docs/observability/integrations/07-langgraph.mdx
index 5305b0450d..e7316e8133 100644
--- a/docs/docs/observability/integrations/07-langgraph.mdx
+++ b/docs/docs/observability/integrations/07-langgraph.mdx
@@ -1,14 +1,18 @@
---
-title: Tracing and Observability for LangGraph with Agenta
+title: Observability for LangGraph with Agenta
sidebar_label: LangGraph
description: Monitor and debug your LangGraph applications with Agenta's observability platform. Get complete visibility into your graph-based LLM application performance, debugging capabilities, and execution observability.
---
```mdx-code-block
import Image from "@theme/IdealImage";
-import GoogleColabButton from "@site/src/components/GoogleColabButton";
```
+# Observability for LangGraph with Agenta
+
+:::info
+This guide is available as Jupyter Notebook [here](https://github.com/agenta-ai/agenta/blob/main/examples/jupyter/integrations/langgraph-integration.ipynb).
+:::
Learn how to monitor your **[LangGraph](https://langchain-ai.github.io/langgraph/)** applications using **Agenta's** observability platform. Get complete visibility into your graph-based LLM application performance, debugging capabilities, and execution observability.
@@ -16,9 +20,6 @@ Learn how to monitor your **[LangGraph](https://langchain-ai.github.io/langgraph
> **What is LangGraph?** [LangGraph](https://langchain-ai.github.io/langgraph/) is a library for building stateful, multi-actor applications with LLMs. It extends LangChain's capabilities by enabling the creation of complex workflows as directed graphs where nodes represent different processing steps and edges define the flow between them.
-
- Open in Google Colaboratory
-
### 1. Install Required Packages
@@ -407,4 +408,4 @@ def content_moderator(user_content: str):
## Next Steps
-For more advanced observability features and configuration options, see our [complete observability documentation](/observability/trace-with-python-sdk/setup-tracing).
+For more advanced observability features and configuration options, see our [complete observability documentation](/observability/observability-sdk).
diff --git a/docs/docs/observability/integrations/08-openai-agents.mdx b/docs/docs/observability/integrations/08-openai-agents.mdx
index 9a4491915e..31cebcb6c9 100644
--- a/docs/docs/observability/integrations/08-openai-agents.mdx
+++ b/docs/docs/observability/integrations/08-openai-agents.mdx
@@ -1,12 +1,11 @@
---
-title: Tracing and Observability for OpenAI Agents SDK with Agenta
+title: Observability for OpenAI Agents SDK with Agenta
sidebar_label: OpenAI Agents SDK
description: Learn how to set up observability and tracing for OpenAI Agents SDK using Agenta open-source LLM observability platform. This comprehensive guide covers setup, configuration, and best practices for tracing multi-agent workflows using OpenAI Agents SDK and OpenAI models.
---
```mdx-code-block
import Image from "@theme/IdealImage";
-import GoogleColabButton from "@site/src/components/GoogleColabButton";
```
Learn how to set up tracing for your [OpenAI Agents SDK](https://openai.github.io/openai-agents-python/) applications with **Agenta** for complete observability into your multi-agent application performance, debugging capabilities, and execution observability.
@@ -15,9 +14,9 @@ Learn how to set up tracing for your [OpenAI Agents SDK](https://openai.github.i
> **What is OpenAI Agents?** The [OpenAI Agents](https://github.com/openai/openai-agents-python) SDK is a framework for building AI agents that can interact with tools, run asynchronous operations, and orchestrate complex multi-agent workflows. It provides a structured approach to creating intelligent agents with tool calling capabilities.
-
- Open in Google Colaboratory
-
+:::info
+This tutorial is also available as a [Jupyter notebook](https://github.com/Agenta-AI/agenta/tree/main/examples/jupyter/integrations/openai-agents-integration.ipynb) for interactive learning and experimentation.
+:::
## Setup Guide
@@ -175,7 +174,7 @@ The trace provides visibility into your application's execution, helping you:
- Analyze orchestration effectiveness and workflow optimization, and identify bottlenecks
:::info
-After setting up observability for your OpenAI Agents SDK application, you can use Agenta's [evaluation](/evaluation/concepts) features to evaluate the performance of your agents.
+After setting up observability for your OpenAI Agents SDK application, you can use Agenta's [evaluation](/evaluation/overview) features to evaluate the performance of your agents.
:::
## Real-world Example
@@ -276,4 +275,4 @@ async def content_creation_system(content_brief: str):
## Next Steps
-For more detailed information about Agenta's observability features and advanced configuration options, visit the [Agenta Observability SDK Documentation](/observability/trace-with-python-sdk/setup-tracing).
\ No newline at end of file
+For more detailed information about Agenta's observability features and advanced configuration options, visit the [Agenta Observability SDK Documentation](/observability/observability-sdk).
\ No newline at end of file
diff --git a/docs/docs/observability/integrations/10-pydanticai.mdx b/docs/docs/observability/integrations/10-pydanticai.mdx
index eec3ddd731..b882c5bb69 100644
--- a/docs/docs/observability/integrations/10-pydanticai.mdx
+++ b/docs/docs/observability/integrations/10-pydanticai.mdx
@@ -1,5 +1,5 @@
---
-title: Tracing and Observability for PydanticAI with Agenta
+title: Observability for PydanticAI with Agenta
sidebar_label: PydanticAI
description: Instrument and debug your PydanticAI applications with Agenta's comprehensive observability platform. Set up tracing to debug, monitor and evaluate your PydanticAI applications.
category: Integrations
@@ -7,7 +7,6 @@ category: Integrations
```mdx-code-block
import Image from "@theme/IdealImage";
-import GoogleColabButton from "@site/src/components/GoogleColabButton";
```
Learn how to connect **Agenta** with **PydanticAI** for complete visibility into your AI agent performance, debugging capabilities, and execution observability.
@@ -16,9 +15,9 @@ Learn how to connect **Agenta** with **PydanticAI** for complete visibility into
> **What is PydanticAI?** [PydanticAI](https://ai.pydantic.dev/) is a Python agent framework designed to make it less painful to build production-grade applications with generative AI. It provides type safety, structured outputs, and dependency injection for building robust AI agents.
-
- Open in Google Colaboratory
-
+:::info
+This tutorial is also available as a [Jupyter notebook](https://github.com/Agenta-AI/agenta/tree/main/examples/jupyter/integrations/pydanticai-integration.ipynb) for interactive learning and experimentation.
+:::
## Setup Guide
@@ -249,4 +248,4 @@ The trace provides comprehensive visibility into your application's execution, h
## Next Steps
-For more detailed information about Agenta's observability features and advanced configuration options, visit the [Agenta Observability SDK Documentation](/observability/trace-with-python-sdk/setup-tracing).
+For more detailed information about Agenta's observability features and advanced configuration options, visit the [Agenta Observability SDK Documentation](/observability/observability-sdk).
diff --git a/docs/docs/observability/integrations/11-dspy.mdx b/docs/docs/observability/integrations/11-dspy.mdx
deleted file mode 100644
index f32e59dfdd..0000000000
--- a/docs/docs/observability/integrations/11-dspy.mdx
+++ /dev/null
@@ -1,279 +0,0 @@
----
-title: Tracing and Observability for DSPy with Agenta
-sidebar_label: DSPy
-description: Instrument and debug your DSPy applications with Agenta's observability platform. Set up quickly tracing for an improved LLMOps workflows.
-category: Integrations
----
-
-```mdx-code-block
-import Image from "@theme/IdealImage";
-import GoogleColabButton from "@site/src/components/GoogleColabButton";
-```
-
-Learn how to connect **Agenta** with **DSPy** for complete visibility into your LLM application performance, debugging capabilities, and execution observability.
-
-> **What is Agenta?** [Agenta](https://agenta.ai) is an open-source LLMOps platform designed to streamline the deployment, management, and scaling of large language models. It offers comprehensive observability, testing, and deployment capabilities for AI applications.
-
-> **What is DSPy?** [DSPy](https://dspy.ai/) ([GitHub](https://github.com/stanfordnlp/dspy)) is a framework for algorithmically optimizing LM prompts and weights. It provides composable and declarative modules for instructing language models in a more systematic way than traditional prompting.
-
-
-
- Open in Google Colaboratory
-
-
-## Setup Guide
-
-Follow these steps to integrate DSPy with Agenta's observability platform for real-time tracing and performance analysis.
-
-### Install Required Packages
-
-First, install the required dependencies. These packages are essential for the integration:
-
-```bash
-pip install agenta dspy openinference-instrumentation-dspy
-```
-
-**Package Overview:**
-- `agenta`: The core Agenta SDK for prompt engineering and observability
-- `dspy`: The DSPy framework for building systematic LLM applications
-- `openinference-instrumentation-dspy`: Automatic instrumentation library for DSPy operations
-
-### Setup and Configuration
-
-Initialize your environment and configure the Agenta SDK:
-
-```python
-import os
-import agenta as ag
-import dspy
-from openinference.instrumentation.dspy import DSPyInstrumentor
-
-# Load configuration from environment
-os.environ["AGENTA_API_KEY"] = "your_agenta_api_key"
-os.environ["AGENTA_HOST"] = "https://cloud.agenta.ai" # Optional, defaults to the Agenta cloud API
-os.environ["OPENAI_API_KEY"] = "your_openai_api_key" # Required for OpenAI models
-
-# Start Agenta SDK
-ag.init()
-```
-
-> **What does `ag.init()` do?**
-> The `ag.init()` function initializes the Agenta SDK and sets up the necessary configuration for observability. It establishes connection to the Agenta platform, configures tracing and logging settings, and prepares the instrumentation context for your application.
-
-### Enable DSPy Monitoring
-
-Initialize the OpenInference DSPy instrumentation to automatically capture DSPy operations:
-
-```python
-# Enable DSPy instrumentation
-DSPyInstrumentor().instrument()
-```
-
-### Configure DSPy Language Model
-
-Set up your DSPy language model configuration:
-
-```python
-# Configure DSPy with your preferred language model
-lm = dspy.LM("openai/gpt-4o")
-dspy.configure(lm=lm)
-```
-
-### Build Your Instrumented Application
-
-Here's a complete example showcasing three different DSPy use cases with Agenta instrumentation:
-
-```python
-import os
-import agenta as ag
-import dspy
-from openinference.instrumentation.dspy import DSPyInstrumentor
-
-# Configuration setup
-os.environ["AGENTA_API_KEY"] = "your_agenta_api_key"
-os.environ["AGENTA_HOST"] = "https://cloud.agenta.ai" # Optional, defaults to the Agenta cloud API
-os.environ["OPENAI_API_KEY"] = "your_openai_api_key" # Required for OpenAI models
-
-# Start Agenta observability
-ag.init()
-
-# Enable DSPy instrumentation
-DSPyInstrumentor().instrument()
-
-# Configure DSPy
-lm = dspy.LM("openai/gpt-4o")
-dspy.configure(lm=lm)
-
-
-# Use case 1: Math reasoning with Chain of Thought (CoT)
-@ag.instrument()
-def math_reasoning(question: str):
- cot = dspy.ChainOfThought("question -> answer: float")
- response = cot(question=question)
- return response
-
-
-# Use case 2: Retrieval-Augmented Generation (RAG) with Wikipedia search
-@ag.instrument(spankind="query")
-def search_wikipedia(query: str) -> list[str]:
- results = dspy.ColBERTv2(url="http://20.102.90.50:2017/wiki17_abstracts")(
- query, k=3
- )
- return [x["text"] for x in results]
-
-
-@ag.instrument()
-def rag(question: str):
- cot = dspy.ChainOfThought("context, question -> response")
- response = cot(context=search_wikipedia(question), question=question)
- return response
-
-
-# Use case 3: Drafting an article with an outline and sections
-class Outline(dspy.Signature):
- """Outline a thorough overview of a topic."""
-
- topic: str = dspy.InputField()
- title: str = dspy.OutputField()
- sections: list[str] = dspy.OutputField()
- section_subheadings: dict[str, list[str]] = dspy.OutputField(
- desc="mapping from section headings to subheadings"
- )
-
-
-class DraftSection(dspy.Signature):
- """Draft a top-level section of an article."""
-
- topic: str = dspy.InputField()
- section_heading: str = dspy.InputField()
- section_subheadings: list[str] = dspy.InputField()
- content: str = dspy.OutputField(desc="markdown-formatted section")
-
-
-class DraftArticle(dspy.Module):
- def __init__(self):
- self.build_outline = dspy.ChainOfThought(Outline)
- self.draft_section = dspy.ChainOfThought(DraftSection)
-
- def forward(self, topic: str):
- outline = self.build_outline(topic=topic)
- sections = []
- for heading, subheadings in outline.section_subheadings.items():
- section, subheadings = f"## {heading}", [
- f"### {subheading}" for subheading in subheadings
- ]
- section = self.draft_section(
- topic=outline.title,
- section_heading=section,
- section_subheadings=subheadings,
- )
- sections.append(section.content)
- return dspy.Prediction(title=outline.title, sections=sections)
-
-
-@ag.instrument()
-def journalist(topic: str):
- draft_article = DraftArticle()
- article = draft_article(topic=topic)
- return article
-
-
-# Example usage
-if __name__ == "__main__":
- # Use case 1: Math reasoning
- response = math_reasoning("What is 2 + 2?")
- print("Math reasoning response:", response)
-
- # Use case 2: RAG with Wikipedia
- rag_response = rag("What's the name of the castle that David Gregory inherited?")
- print("RAG response:", rag_response)
-
- # Use case 3: Article generation
- article = journalist("The impact of AI on society")
- print("Article generation response:", article)
-```
-
-### Understanding the @ag.instrument() Decorator
-
-The `@ag.instrument()` decorator automatically captures all input and output data from your function, enabling comprehensive observability without manual instrumentation.
-
-**Span Classification:**
-Use the `spankind` parameter to categorize different operations in Agenta WebUI. Available classifications:
-
-- `agent` - Autonomous agent operations
-- `chain` - Sequential processing workflows
-- `workflow` - End-to-end application flows (default behavior)
-- `tool` - Utility function executions
-- `embedding` - Vector embedding operations
-- `query` - Search and retrieval operations
-- `completion` - Text generation tasks
-- `chat` - Conversational interactions
-- `rerank` - Result reordering operations
-
-**Standard Behavior:**
-By default, when `spankind` is not specified, the operation becomes a root-level span, categorized as a `workflow` in Agenta.
-
-Example with specific span kind:
-
-```python
-@ag.instrument(spankind="query")
-def search_knowledge_base(search_term: str):
- # Knowledge base search implementation
- pass
-```
-
-### View Traces in Agenta
-
-Once your application runs, access detailed execution traces through Agenta's dashboard. The observability data includes:
-
-- End-to-end workflow execution timeline
-- DSPy module initialization and configuration steps
-- Chain of Thought reasoning processes
-- Retrieval operations and context augmentation
-- Language model calls and response generation
-- Performance metrics and execution duration
-
-
-
-The trace provides comprehensive visibility into your application's execution, helping you:
-- Debug complex reasoning chains and prompt optimization
-- Monitor retrieval effectiveness and context quality
-- Analyze language model performance and token usage
-- Track application behavior trends and optimization opportunities
-
-## Advanced Usage
-
-### Custom Span Configuration
-
-Configure different instrumentation levels for various application components:
-
-```python
-@ag.instrument(spankind="workflow")
-def complex_reasoning_pipeline(query: str):
- return math_reasoning(query)
-
-
-@ag.instrument(spankind="tool")
-def custom_retrieval_function(query: str):
- # Custom retrieval logic
- pass
-
-
-@ag.instrument(spankind="chain")
-def multi_step_reasoning(question: str):
- # Multi-step reasoning implementation
- return question
-```
-
-## Next Steps
-
-For more detailed information about Agenta's observability features and advanced configuration options, visit the [Agenta Observability SDK Documentation](/observability/trace-with-python-sdk/setup-tracing).
\ No newline at end of file
diff --git a/docs/docs/observability/integrations/12-agno.mdx b/docs/docs/observability/integrations/12-agno.mdx
deleted file mode 100644
index 28c816c8fd..0000000000
--- a/docs/docs/observability/integrations/12-agno.mdx
+++ /dev/null
@@ -1,251 +0,0 @@
----
-title: Tracing and Observability for Agno with Agenta
-sidebar_label: Agno
-description: Trace and debug your Agno applications with Agenta's observability platform. Set up instrumentation and observability for Agno Agents for a better LLMOps workflows.
-category: Integrations
----
-
-```mdx-code-block
-import Image from "@theme/IdealImage";
-import GoogleColabButton from "@site/src/components/GoogleColabButton";
-```
-
-Learn how to connect **Agenta** with **Agno** for complete visibility into your AI agent performance, debugging capabilities, and execution observability.
-
-> **What is Agenta?** [Agenta](https://agenta.ai) is an open-source LLMOps platform designed to streamline the deployment, management, and scaling of large language models. It offers comprehensive observability, testing, and deployment capabilities for AI applications.
-
-> **What is Agno?** [Agno](https://github.com/agno-agi/agno) is a Python framework for building AI agents with tools, structured outputs, and efficient workflows. It provides a simple yet powerful interface for creating intelligent agents that can interact with external systems and perform complex tasks.
-
-
-
- Open in Google Colaboratory
-
-
-
-Here are the steps to setup tracing and observability for Agno applications with Agenta.
-
-## Install Required Packages
-
-First, install the required dependencies:
-
-```bash
-pip install agenta openinference-instrumentation-agno agno
-```
-
-**Package Overview:**
-- `agenta`: The core Agenta SDK for prompt engineering and observability
-- `agno`: The Agno framework for building AI agents with tools
-- `openinference-instrumentation-agno`: Automatic instrumentation library for Agno operations
-
-## Setup and Configuration
-
-Initialize your environment and configure the Agenta SDK:
-
-```python
-import os
-import re
-from itertools import permutations
-import agenta as ag
-from agno.agent import Agent
-from agno.models.openai import OpenAIChat
-from openinference.instrumentation.agno import AgnoInstrumentor
-
-
-# Set up the environment
-os.environ["AGENTA_API_KEY"] = "your_agenta_api_key"
-os.environ["AGENTA_HOST"] = "https://cloud.agenta.ai" # Optional, defaults to the Agenta cloud API
-
-# Start the Agenta SDK
-ag.init()
-```
-
-> **What does `ag.init()` do?**
-> The `ag.init()` function initializes the Agenta SDK and sets up the necessary configuration for observability. It establishes connection to the Agenta platform, configures tracing and logging settings, and prepares the instrumentation context for your application.
-
-## Enable Agno Monitoring
-
-Initialize the OpenInference Agno instrumentation to automatically capture agent operations:
-
-```python
-AgnoInstrumentor().instrument()
-```
-
-## Complete Example of an Agno Application
-
-Here's a complete example showcasing a logistics dispatch agent with Agenta instrumentation:
-
-```python
-import os
-import re
-from itertools import permutations
-import agenta as ag
-from agno.agent import Agent
-from agno.models.openai import OpenAIChat
-from openinference.instrumentation.agno import AgnoInstrumentor
-
-
-# Set up the environment
-os.environ["AGENTA_API_KEY"] = "your_agenta_api_key"
-os.environ["AGENTA_HOST"] = "https://cloud.agenta.ai" # Optional, defaults to the Agenta cloud API
-
-# Start the Agenta SDK
-ag.init()
-AgnoInstrumentor().instrument()
-
-
-# Simulated logistics data
-tracking_data = {
- "TRK10001": "In transit at Berlin Friedrichshain Distribution Center",
- "TRK10002": "Delivered on 2025-06-14 at 18:32 in Charlottenburg",
- "TRK10003": "Out for delivery — last scanned near Tempelhofer Feld",
- "TRK10004": "Held at customs near Berlin Brandenburg Airport (BER)",
- "TRK10005": "Awaiting pickup at Berlin Hauptbahnhof Parcel Station",
-}
-
-distance_matrix = {
- "Warehouse": {"A": 10, "B": 15, "C": 20},
- "A": {"Warehouse": 10, "B": 12, "C": 5},
- "B": {"Warehouse": 15, "A": 12, "C": 8},
- "C": {"Warehouse": 20, "A": 5, "B": 8},
-}
-
-driver_load = {"Alice": 2, "Bob": 3, "Charlie": 1}
-
-
-# Tool: TrackingTool
-class TrackingTool:
- def __init__(self):
- self.name = "TrackingTool"
- self.description = "Provides shipment status updates given a tracking ID."
-
- def run(self, query: str) -> str:
- match = re.search(r"\bTRK\d+\b", query.upper())
- if not match:
- return "Please provide a valid tracking ID."
-
- tid = match.group(0)
- status = tracking_data.get(tid)
- return f"Status for {tid}: {status}" if status else f"No information for {tid}."
-
-
-# Tool: RouteTool
-class RouteTool:
- def __init__(self):
- self.name = "RouteTool"
- self.description = "Computes the best delivery route given a start and destinations."
-
- def run(self, query: str) -> str:
- m = re.search(r"from\s+([\w\s]+)\s+to\s+(.+)", query, re.IGNORECASE)
- if not m:
- return "Specify route as 'from to , , ...'."
-
- origin = m.group(1).strip()
- dests = [d.strip() for d in re.split(r",| and ", m.group(2)) if d.strip()]
-
- if origin not in distance_matrix:
- return f"Unknown origin: {origin}."
- for loc in dests:
- if loc not in distance_matrix:
- return f"Unknown destination: {loc}."
-
- best_distance = float("inf")
- best_order = None
- for perm in permutations(dests):
- total = 0
- cur = origin
- for nxt in perm:
- total += distance_matrix[cur][nxt]
- cur = nxt
- if total < best_distance:
- best_distance = total
- best_order = perm
-
- route_plan = " → ".join([origin] + list(best_order)) if best_order else origin
- return f"Optimal route: {route_plan} (Total distance: {best_distance} km)"
-
-
-# Tool: WorkloadBalancerTool
-class WorkloadBalancerTool:
- def __init__(self):
- self.name = "WorkloadBalancerTool"
- self.description = "Assigns delivery locations to the least busy driver."
- self.drivers = driver_load.copy()
-
- def run(self, query: str) -> str:
- m = re.search(r"deliver(?:y|ies)? to (.+)", query, re.IGNORECASE)
- if not m:
- return "Please specify delivery locations like 'deliver to A, B, C'."
-
- locations = [
- loc.strip() for loc in re.split(r",| and ", m.group(1)) if loc.strip()
- ]
- assignments = []
- for loc in locations:
- least_loaded = min(self.drivers, key=lambda d: self.drivers[d])
- assignments.append(f"{loc} → {least_loaded}")
- self.drivers[least_loaded] += 1
- return "Delivery assignments:\n" + "\n".join(assignments)
-
-
-# Create the dispatch agent
-agent = Agent(
- model=OpenAIChat(id="gpt-4o"),
- description="You are a smart dispatch assistant for a logistics team.",
- instructions=[
- "Use TrackingTool for shipment queries.",
- "Use RouteTool for route optimization.",
- "Use WorkloadBalancerTool to assign deliveries to drivers.",
- "Always return concise, formatted answers with relevant detail.",
- ],
- tools=[TrackingTool(), RouteTool(), WorkloadBalancerTool()],
- show_tool_calls=False,
-)
-
-
-@ag.instrument()
-def handle_dispatch_request(query: str):
- result = agent.run(query)
- return result.content
-
-
-# Example usage
-if __name__ == "__main__":
- response = handle_dispatch_request(
- "Where is shipment TRK10001? Also, find the best route from Warehouse to A, B and C, "
- "and assign deliveries to the least busy drivers."
- )
- print("Response:", response)
-```
-
-
-## View Traces in Agenta
-
-Once your application runs, access detailed execution traces through Agenta's dashboard. The observability data includes:
-
-- End-to-end agent workflow execution timeline
-- Agno agent initialization and tool configuration
-- Tool execution sequences and decision-making processes
-- LLM interactions and response generation
-- Tool call chains and data flow between tools
-- Performance metrics and execution duration
-
-
-
-The trace provides comprehensive visibility into your application's execution, helping you:
-- Debug complex agent interactions and tool usage patterns
-- Monitor tool execution performance and decision logic
-- Analyze LLM reasoning and response quality
-- Track multi-tool workflows and coordination
-
-## Next Steps
-
-For more detailed information about Agenta's observability features and advanced configuration options, visit the [Agenta Observability SDK Documentation](/observability/trace-with-python-sdk/setup-tracing).
\ No newline at end of file
diff --git a/docs/docs/observability/integrations/_category_.json b/docs/docs/observability/integrations/_category_.json
index 72d05a3297..52b3e501bb 100644
--- a/docs/docs/observability/integrations/_category_.json
+++ b/docs/docs/observability/integrations/_category_.json
@@ -1,4 +1,4 @@
{
- "position": 9,
+ "position": 6,
"label": "Integrations"
-}
\ No newline at end of file
+}
diff --git a/docs/docs/observability/query-data/01-query-api.mdx b/docs/docs/observability/query-data/01-query-api.mdx
deleted file mode 100644
index a5021300f8..0000000000
--- a/docs/docs/observability/query-data/01-query-api.mdx
+++ /dev/null
@@ -1,716 +0,0 @@
----
-title: "Query Trace Data with the Agenta API"
-sidebar_label: "Query via API"
-description: "Learn how to programmatically query and filter LLM traces and spans using the Agenta Query Data API with Python and JavaScript examples"
-sidebar_position: 1
----
-
-```mdx-code-block
-import GoogleColabButton from "@site/src/components/GoogleColabButton";
-```
-
-
- Open in Google Colaboratory
-
-
-The Agenta API lets you query trace and span data programmatically. You can filter by attributes, time ranges, and status codes. The API returns data in two formats: flat spans or hierarchical trace trees.
-
-## API Endpoint
-
-```
-POST /api/preview/tracing/spans/query
-```
-
-Send queries as JSON in the request body. This approach works best for complex filters with nested conditions.
-
-## Authentication
-
-Include your API key in the request header:
-
-```http
-Authorization: ApiKey YOUR_API_KEY
-```
-
-You can create API keys from the Settings page in your Agenta workspace.
-
-## Response Format Options
-
-### Focus Parameter
-
-Choose how the API returns your data:
-
-**`span`**: Returns individual spans in a flat list. Use this when you need span-level details.
-
-```json
-{
- "focus": "span"
-}
-```
-
-Example response:
-
-```json
-{
- "count": 2,
- "spans": [
- {
- "trace_id": "abc123",
- "span_id": "span001",
- "parent_id": "root",
- "span_name": "openai.chat",
- "status_code": "STATUS_CODE_OK",
- "attributes": {
- "ag": {
- "metrics": {
- "costs": { "cumulative": { "total": 0.0023 } },
- "tokens": { "cumulative": { "total": 450 } }
- }
- }
- }
- },
- {
- "trace_id": "def456",
- "span_id": "span002",
- "parent_id": "root",
- "span_name": "generate_response",
- "status_code": "STATUS_CODE_OK",
- "attributes": { "..." }
- }
- ]
-}
-```
-
-**`trace`** (default): Returns complete traces as hierarchical trees. Each trace groups its spans by `trace_id`. Use this when you need to see the full request flow.
-
-```json
-{
- "focus": "trace"
-}
-```
-
-Example response:
-
-```json
-{
- "count": 1,
- "traces": {
- "abc123": {
- "spans": {
- "generate_response": {
- "span_id": "root",
- "span_name": "generate_response",
- "status_code": "STATUS_CODE_OK",
- "spans": {
- "openai.chat": {
- "span_id": "span001",
- "span_name": "openai.chat",
- "parent_id": "root",
- "status_code": "STATUS_CODE_OK",
- "attributes": {
- "ag": {
- "metrics": {
- "costs": { "cumulative": { "total": 0.0023 } },
- "tokens": { "cumulative": { "total": 450 } }
- }
- }
- }
- }
- }
- }
- }
- }
- }
-}
-```
-
-## Time Windows and Limits
-
-Control which traces you retrieve using time ranges and result limits.
-
-### Time Range
-
-Specify start and end timestamps using ISO 8601 format or Unix timestamps:
-
-```json
-{
- "oldest": "2024-01-01T00:00:00Z",
- "newest": "2024-01-31T23:59:59Z"
-}
-```
-
-The `oldest` timestamp is included in results. The `newest` timestamp is excluded.
-
-You can also use Unix timestamps (in seconds):
-
-```json
-{
- "oldest": 1704067200,
- "newest": 1706745599
-}
-```
-
-### Result Limit
-
-Limit the number of results:
-
-```json
-{
- "limit": 100
-}
-```
-
-Combine time ranges with limits for precise control:
-
-```json
-{
- "oldest": "2024-01-01T00:00:00Z",
- "newest": "2024-01-31T23:59:59Z",
- "limit": 50
-}
-```
-
-## Filtering Data
-
-Filter traces and spans using conditions with various operators.
-
-### Basic Filter Structure
-
-Each filter needs an operator and conditions:
-
-```json
-{
- "filter": {
- "operator": "and",
- "conditions": [
- {
- "field": "span_name",
- "operator": "is",
- "value": "my_span"
- }
- ]
- }
-}
-```
-
-### Logical Operators
-
-Combine conditions using these operators:
-
-**`and`** (default): All conditions must match.
-
-**`or`**: At least one condition must match.
-
-**`not`**: Negates the condition.
-
-**`nand`**: At least one condition does not match.
-
-**`nor`**: No conditions match.
-
-Example using OR logic:
-
-```json
-{
- "filter": {
- "operator": "or",
- "conditions": [
- { "field": "span_name", "value": "chat" },
- { "field": "span_name", "value": "completion" }
- ]
- }
-}
-```
-
-### Fields You Can Filter
-
-Filter by these standard fields:
-
-**`trace_id`**: Trace identifier
-
-**`span_id`**: Span identifier
-
-**`parent_id`**: Parent span identifier
-
-**`span_name`**: Span name
-
-**`span_kind`**: Type of span (`SPAN_KIND_UNSPECIFIED`, `SPAN_KIND_INTERNAL`, `SPAN_KIND_SERVER`, `SPAN_KIND_CLIENT`, `SPAN_KIND_PRODUCER`, `SPAN_KIND_CONSUMER`)
-
-**`start_time`**: Start timestamp
-
-**`end_time`**: End timestamp
-
-**`status_code`**: Status code (`STATUS_CODE_UNSET`, `STATUS_CODE_OK`, `STATUS_CODE_ERROR`)
-
-**`status_message`**: Status message text
-
-**`attributes`**: Span attributes (requires `key` parameter)
-
-**`links`**: Linked spans
-
-**`references`**: References to applications, variants, or revisions
-
-### Filtering by Attributes
-
-Access span attributes using the `key` parameter:
-
-```json
-{
- "field": "attributes",
- "key": "ag.type.span",
- "operator": "is",
- "value": "llm"
-}
-```
-
-For nested attributes, use dot notation:
-
-```json
-{
- "field": "attributes",
- "key": "ag.metrics.unit.cost",
- "operator": "gt",
- "value": 0.01
-}
-```
-
-### Comparison Operators
-
-#### Equality
-
-**`is`** (default): Exact match
-
-**`is_not`**: Not equal
-
-```json
-{
- "field": "span_name",
- "operator": "is",
- "value": "openai.chat"
-}
-```
-
-#### Numeric Comparisons
-
-**`eq`**: Equal to
-
-**`neq`**: Not equal to
-
-**`gt`**: Greater than
-
-**`gte`**: Greater than or equal to
-
-**`lt`**: Less than
-
-**`lte`**: Less than or equal to
-
-**`btwn`**: Between (inclusive, provide array `[min, max]`)
-
-Example filtering by duration:
-
-```json
-{
- "field": "attributes",
- "key": "ag.metrics.unit.duration",
- "operator": "gt",
- "value": 1000
-}
-```
-
-Example using between:
-
-```json
-{
- "field": "attributes",
- "key": "ag.metrics.unit.duration",
- "operator": "btwn",
- "value": [500, 2000]
-}
-```
-
-#### String Matching
-
-**`startswith`**: Starts with prefix
-
-**`endswith`**: Ends with suffix
-
-**`contains`**: Contains substring
-
-**`matches`**: Regular expression match
-
-**`like`**: SQL-like pattern (supports `%` wildcard)
-
-Example searching span names:
-
-```json
-{
- "field": "span_name",
- "operator": "contains",
- "value": "api"
-}
-```
-
-#### List Operations
-
-**`in`**: Value exists in the list
-
-**`not_in`**: Value does not exist in the list
-
-Example filtering multiple traces:
-
-```json
-{
- "field": "trace_id",
- "operator": "in",
- "value": [
- "trace_id_1",
- "trace_id_2",
- "trace_id_3"
- ]
-}
-```
-
-#### Existence Checks
-
-**`exists`**: Field or attribute exists (any value, including null)
-
-**`not_exists`**: Field or attribute does not exist
-
-Example checking for cost tracking:
-
-```json
-{
- "field": "attributes",
- "key": "ag.metrics.unit.cost",
- "operator": "exists"
-}
-```
-
-Note: When using `exists` or `not_exists`, omit the `value` field.
-
-## Advanced Filtering Examples
-
-### Multiple Conditions
-
-Filter production traces with slow response times:
-
-```json
-{
- "filter": {
- "operator": "and",
- "conditions": [
- {
- "field": "attributes",
- "key": "environment",
- "value": "production"
- },
- {
- "field": "attributes",
- "key": "ag.metrics.unit.duration",
- "operator": "gt",
- "value": 1000
- },
- {
- "field": "status_code",
- "operator": "is_not",
- "value": "STATUS_CODE_ERROR"
- }
- ]
- }
-}
-```
-
-### Nested Logical Operators
-
-Find API calls that either errored or took too long:
-
-```json
-{
- "filter": {
- "operator": "and",
- "conditions": [
- {
- "field": "span_name",
- "operator": "startswith",
- "value": "api_"
- },
- {
- "operator": "or",
- "conditions": [
- {
- "field": "status_code",
- "value": "STATUS_CODE_ERROR"
- },
- {
- "field": "attributes",
- "key": "ag.metrics.unit.duration",
- "operator": "gt",
- "value": 5000
- }
- ]
- }
- ]
- }
-}
-```
-
-### Filter by Application References
-
-Find traces for specific applications:
-
-```json
-{
- "filter": {
- "conditions": [
- {
- "field": "references",
- "operator": "in",
- "value": [
- { "id": "application_id_1" },
- { "id": "application_id_2" }
- ]
- }
- ]
- }
-}
-```
-
-### Filter by Linked Spans
-
-Find spans linked to specific traces:
-
-```json
-{
- "filter": {
- "conditions": [
- {
- "field": "links",
- "operator": "in",
- "value": [
- {
- "trace_id": "trace_id_value",
- "span_id": "span_id_value"
- }
- ]
- }
- ]
- }
-}
-```
-
-## Complete Query Example
-
-Here's a full query that finds successful LLM calls from the last month:
-
-```json
-{
- "focus": "trace",
- "oldest": "2024-01-01T00:00:00Z",
- "newest": "2024-01-31T23:59:59Z",
- "limit": 50,
- "filter": {
- "operator": "and",
- "conditions": [
- {
- "field": "attributes",
- "key": "ag.type.span",
- "value": "llm"
- },
- {
- "field": "status_code",
- "operator": "is_not",
- "value": "STATUS_CODE_ERROR"
- }
- ]
- }
-}
-```
-
-## Response Structure
-
-The API returns JSON with these fields:
-
-```json
-{
- "count": 42,
- "spans": [...], // Present when focus=span
- "traces": {...} // Present when focus=trace
-}
-```
-
-**`count`**: Number of results returned
-
-**`spans`**: Array of spans (when `focus=span`)
-
-**`traces`**: Dictionary of trace trees indexed by trace_id (when `focus=trace`)
-
-### Span Format
-
-Each span contains:
-
-```json
-{
- "trace_id": "...",
- "span_id": "...",
- "parent_id": "...",
- "span_name": "openai.chat",
- "span_kind": "SPAN_KIND_CLIENT",
- "start_time": "2024-01-15T10:30:00Z",
- "end_time": "2024-01-15T10:30:03Z",
- "status_code": "STATUS_CODE_OK",
- "attributes": {
- "ag": {
- "data": {
- "inputs": {...},
- "outputs": {...}
- },
- "metrics": {
- "costs": {...},
- "tokens": {...},
- "duration": {...}
- },
- "type": {
- "span": "llm"
- }
- }
- }
-}
-```
-
-### Trace Format
-
-Each trace organizes spans hierarchically by name:
-
-```json
-{
- "trace_id_1": {
- "spans": {
- "root_span": {
- "span_id": "...",
- "spans": {
- "child_span": {
- "span_id": "...",
- "..."
- }
- }
- }
- }
- }
-}
-```
-
-## Error Responses
-
-The API returns standard HTTP status codes:
-
-**200 OK**: Query succeeded
-
-**400 Bad Request**: Invalid filter syntax or parameters
-
-**401 Unauthorized**: Missing or invalid API key
-
-**422 Unprocessable Entity**: Validation errors in the request
-
-## Python Example
-
-Here's a complete example using Python:
-
-```python
-import requests
-from datetime import datetime, timedelta, timezone
-
-AGENTA_HOST = "https://cloud.agenta.ai"
-API_KEY = "your_api_key_here"
-
-# Query spans from the last 7 days
-now = datetime.now(timezone.utc)
-week_ago = now - timedelta(days=7)
-
-query = {
- "focus": "span",
- "oldest": week_ago.isoformat(),
- "newest": now.isoformat(),
- "limit": 100,
- "filter": {
- "operator": "and",
- "conditions": [
- {
- "field": "attributes",
- "key": "ag.type.span",
- "value": "llm"
- },
- {
- "field": "status_code",
- "value": "STATUS_CODE_OK"
- }
- ]
- }
-}
-
-response = requests.post(
- f"{AGENTA_HOST}/api/preview/tracing/spans/query",
- headers={
- "Authorization": f"ApiKey {API_KEY}",
- "Content-Type": "application/json"
- },
- json=query
-)
-
-data = response.json()
-print(f"Found {data['count']} spans")
-```
-
-## JavaScript Example
-
-Here's the same query in JavaScript:
-
-```javascript
-const AGENTA_HOST = "https://cloud.agenta.ai";
-const API_KEY = "your_api_key_here";
-
-// Query spans from the last 7 days
-const now = new Date();
-const weekAgo = new Date(now.getTime() - 7 * 24 * 60 * 60 * 1000);
-
-const query = {
- focus: "span",
- oldest: weekAgo.toISOString(),
- newest: now.toISOString(),
- limit: 100,
- filter: {
- operator: "and",
- conditions: [
- {
- field: "attributes",
- key: "ag.type.span",
- value: "llm"
- },
- {
- field: "status_code",
- value: "STATUS_CODE_OK"
- }
- ]
- }
-};
-
-const response = await fetch(
- `${AGENTA_HOST}/api/preview/tracing/spans/query`,
- {
- method: "POST",
- headers: {
- "Authorization": `ApiKey ${API_KEY}`,
- "Content-Type": "application/json"
- },
- body: JSON.stringify(query)
- }
-);
-
-const data = await response.json();
-console.log(`Found ${data.count} spans`);
-```
-
-## Next Steps
-
-Learn about the [Analytics Data API](/observability/query-data/analytics-data) for aggregated metrics and time-series data.
-
-Explore the [API Reference](/reference/api/category) for complete endpoint documentation.
-
-Check out [Filtering in the UI](/observability/concepts) to learn about the visual query builder.
diff --git a/docs/docs/observability/query-data/02-analytics-data.mdx b/docs/docs/observability/query-data/02-analytics-data.mdx
deleted file mode 100644
index 3de7455510..0000000000
--- a/docs/docs/observability/query-data/02-analytics-data.mdx
+++ /dev/null
@@ -1,473 +0,0 @@
----
-title: "Analyze Observability Metrics with the Agenta Analytics API"
-sidebar_label: "Analytics via API"
-description: "Learn how to retrieve and analyze aggregated LLM performance metrics including costs, latency, token usage, and error rates using the Agenta Analytics API with Python and JavaScript examples"
-sidebar_position: 2
----
-
-import GoogleColabButton from "@site/src/components/GoogleColabButton";
-
-
- Open in Google Colaboratory
-
-
-## Overview
-
-The Agenta Analytics API retrieves aggregated metrics from your LLM traces. The API groups your data into time buckets and calculates metrics like cost, latency, token usage, and error rates.
-
-Use analytics to:
-- Track LLM costs over time
-- Monitor performance trends
-- Identify error patterns
-- Analyze token consumption
-
-**Endpoint**: `POST /api/preview/tracing/spans/analytics`
-
-**Authentication**: You can create API keys from the Settings page in your Agenta workspace.
-
-## Quick Start
-
-### Python
-
-```python
-import requests
-from datetime import datetime, timedelta, timezone
-
-# Setup
-AGENTA_HOST = "https://cloud.agenta.ai"
-API_KEY = "your_api_key_here"
-BASE_URL = f"{AGENTA_HOST}/api/preview/tracing/spans/analytics"
-
-headers = {
- "Authorization": f"ApiKey {API_KEY}",
- "Content-Type": "application/json"
-}
-
-# Get analytics for last 7 days with daily buckets
-newest = datetime.now(timezone.utc)
-oldest = newest - timedelta(days=7)
-
-payload = {
- "focus": "trace",
- "interval": 1440, # 1440 minutes = daily buckets
- "windowing": {
- "oldest": oldest.isoformat(),
- "newest": newest.isoformat()
- }
-}
-
-response = requests.post(BASE_URL, headers=headers, json=payload)
-data = response.json()
-
-print(f"Found {data['count']} daily buckets")
-for bucket in data['buckets']:
- if bucket['total']['count'] > 0:
- print(f"\nDate: {bucket['timestamp'][:10]}")
- print(f" Traces: {bucket['total']['count']}")
- print(f" Cost: ${bucket['total']['costs']:.4f}")
- print(f" Avg Duration: {bucket['total']['duration'] / bucket['total']['count']:.0f}ms")
- print(f" Errors: {bucket['errors']['count']}")
-```
-
-### JavaScript
-
-```javascript
-const AGENTA_HOST = "https://cloud.agenta.ai";
-const API_KEY = "your_api_key_here";
-const BASE_URL = `${AGENTA_HOST}/api/preview/tracing/spans/analytics`;
-
-// Get analytics for last 7 days with daily buckets
-const newest = new Date();
-const oldest = new Date(newest.getTime() - 7 * 24 * 60 * 60 * 1000);
-
-const payload = {
- focus: "trace",
- interval: 1440, // 1440 minutes = daily buckets
- windowing: {
- oldest: oldest.toISOString(),
- newest: newest.toISOString()
- }
-};
-
-const response = await fetch(BASE_URL, {
- method: "POST",
- headers: {
- "Authorization": `ApiKey ${API_KEY}`,
- "Content-Type": "application/json"
- },
- body: JSON.stringify(payload)
-});
-
-const data = await response.json();
-
-console.log(`Found ${data.count} daily buckets`);
-data.buckets.forEach(bucket => {
- if (bucket.total.count > 0) {
- const date = bucket.timestamp.substring(0, 10);
- const avgDuration = bucket.total.duration / bucket.total.count;
- console.log(`\nDate: ${date}`);
- console.log(` Traces: ${bucket.total.count}`);
- console.log(` Cost: $${bucket.total.costs.toFixed(4)}`);
- console.log(` Avg Duration: ${avgDuration.toFixed(0)}ms`);
- console.log(` Errors: ${bucket.errors.count}`);
- }
-});
-```
-
-## Request Parameters
-
-All parameters are sent in the JSON request body:
-
-### Focus (Required)
-
-Controls the aggregation level:
-
-- **`trace`**: Aggregate by complete traces (most common)
-- **`span`**: Aggregate individual spans
-
-Most analytics queries use `trace` to analyze complete LLM requests.
-
-```python
-payload = {
- "focus": "trace",
- "interval": 1440
-}
-```
-
-### Interval (Required)
-
-Bucket size in minutes. Common values:
-
-- **`60`** = Hourly buckets
-- **`1440`** = Daily buckets (24 hours)
-- **`10080`** = Weekly buckets (7 days)
-
-```python
-payload = {
- "focus": "trace",
- "interval": 1440 # Daily buckets
-}
-```
-
-### Windowing (Optional)
-
-Specify the time range for your analytics. If not provided, defaults to the last 30 days.
-
-```python
-from datetime import datetime, timedelta, timezone
-
-newest = datetime.now(timezone.utc)
-oldest = newest - timedelta(days=30)
-
-payload = {
- "focus": "trace",
- "interval": 1440,
- "windowing": {
- "oldest": oldest.isoformat(),
- "newest": newest.isoformat()
- }
-}
-```
-
-### Filter (Optional)
-
-Filter which traces to include in analytics. Uses the same filter syntax as the Query API.
-
-**Filter by status code:**
-```python
-payload = {
- "focus": "trace",
- "interval": 1440,
- "filter": {
- "conditions": [
- {
- "field": "status.code",
- "operator": "eq",
- "value": "STATUS_CODE_OK"
- }
- ]
- }
-}
-```
-
-**Filter by span name:**
-```python
-payload = {
- "focus": "trace",
- "interval": 1440,
- "filter": {
- "conditions": [
- {
- "field": "name",
- "operator": "contains",
- "value": "openai"
- }
- ]
- }
-}
-```
-
-**Multiple conditions:**
-```python
-payload = {
- "focus": "trace",
- "interval": 1440,
- "filter": {
- "operator": "and",
- "conditions": [
- {
- "field": "status.code",
- "operator": "eq",
- "value": "STATUS_CODE_OK"
- },
- {
- "field": "attributes.ag.metrics.costs.cumulative.total",
- "operator": "gt",
- "value": 0.01
- }
- ]
- }
-}
-```
-
-## Response Format
-
-The API returns aggregated metrics grouped into time buckets:
-
-```json
-{
- "count": 7,
- "buckets": [
- {
- "timestamp": "2025-10-24T00:00:00Z",
- "interval": 1440,
- "total": {
- "count": 150,
- "duration": 45000.5,
- "costs": 0.0234,
- "tokens": 1200.0
- },
- "errors": {
- "count": 5,
- "duration": 2300.0,
- "costs": 0.0,
- "tokens": 0.0
- }
- },
- {
- "timestamp": "2025-10-25T00:00:00Z",
- "interval": 1440,
- "total": {
- "count": 200,
- "duration": 60000.0,
- "costs": 0.0312,
- "tokens": 1600.0
- },
- "errors": {
- "count": 3,
- "duration": 1500.0,
- "costs": 0.0,
- "tokens": 0.0
- }
- }
- ]
-}
-```
-
-### Response Fields
-
-- **`count`**: Number of time buckets returned
-- **`buckets`**: Array of time-based aggregated metrics
-
-### Bucket Fields
-
-- **`timestamp`**: Start time of the bucket (ISO 8601)
-- **`interval`**: Bucket size in minutes
-- **`total`**: Aggregated metrics for all traces in this bucket
- - **`count`**: Number of traces
- - **`duration`**: Total duration in milliseconds
- - **`costs`**: Total cost in USD
- - **`tokens`**: Total tokens used
-- **`errors`**: Aggregated metrics for failed traces only
- - **`count`**: Number of failed traces
- - **`duration`**: Total duration of failed traces in milliseconds
- - **`costs`**: Total cost of failed traces (usually 0)
- - **`tokens`**: Total tokens in failed traces (usually 0)
-
-## Common Use Cases
-
-### Monitor Daily Costs
-
-Track LLM spending over time:
-
-```python
-import requests
-from datetime import datetime, timedelta, timezone
-
-# Get daily costs for last 30 days
-newest = datetime.now(timezone.utc)
-oldest = newest - timedelta(days=30)
-
-payload = {
- "focus": "trace",
- "interval": 1440, # Daily buckets
- "windowing": {
- "oldest": oldest.isoformat(),
- "newest": newest.isoformat()
- }
-}
-
-response = requests.post(BASE_URL, headers=headers, json=payload)
-data = response.json()
-
-# Calculate totals
-total_cost = sum(b['total']['costs'] for b in data['buckets'])
-total_traces = sum(b['total']['count'] for b in data['buckets'])
-
-print(f"Total Cost (30 days): ${total_cost:.2f}")
-print(f"Total Traces: {total_traces:,}")
-print(f"Average Cost per Trace: ${total_cost/total_traces:.4f}")
-```
-
-### Analyze Error Trends
-
-Monitor error rates to identify reliability issues:
-
-```python
-# Get hourly metrics for last 7 days
-newest = datetime.now(timezone.utc)
-oldest = newest - timedelta(days=7)
-
-payload = {
- "focus": "trace",
- "interval": 60, # Hourly buckets
- "windowing": {
- "oldest": oldest.isoformat(),
- "newest": newest.isoformat()
- }
-}
-
-response = requests.post(BASE_URL, headers=headers, json=payload)
-data = response.json()
-
-# Find high error rate periods
-print("Hours with high error rates (>5%):")
-for bucket in data['buckets']:
- if bucket['total']['count'] > 0:
- error_rate = (bucket['errors']['count'] / bucket['total']['count']) * 100
- if error_rate > 5:
- print(f" {bucket['timestamp']}: {error_rate:.1f}%")
-```
-
-### Track Token Usage
-
-Monitor token consumption patterns:
-
-```python
-# Get daily token usage for last 7 days
-newest = datetime.now(timezone.utc)
-oldest = newest - timedelta(days=7)
-
-payload = {
- "focus": "trace",
- "interval": 1440, # Daily buckets
- "windowing": {
- "oldest": oldest.isoformat(),
- "newest": newest.isoformat()
- }
-}
-
-response = requests.post(BASE_URL, headers=headers, json=payload)
-data = response.json()
-
-print("Daily Token Usage:")
-for bucket in data['buckets']:
- if bucket['total']['count'] > 0:
- date = bucket['timestamp'][:10]
- avg_tokens = bucket['total']['tokens'] / bucket['total']['count']
- print(f" {date}: {bucket['total']['tokens']:,.0f} total ({avg_tokens:.0f} avg)")
-```
-
-### Compare Performance
-
-Analyze latency trends over time:
-
-```python
-# Get hourly performance for last 24 hours
-newest = datetime.now(timezone.utc)
-oldest = newest - timedelta(days=1)
-
-payload = {
- "focus": "trace",
- "interval": 60, # Hourly buckets
- "windowing": {
- "oldest": oldest.isoformat(),
- "newest": newest.isoformat()
- }
-}
-
-response = requests.post(BASE_URL, headers=headers, json=payload)
-data = response.json()
-
-print("Hourly Average Latency:")
-latencies = []
-for bucket in data['buckets']:
- if bucket['total']['count'] > 0:
- avg_duration = bucket['total']['duration'] / bucket['total']['count']
- latencies.append(avg_duration)
- hour = bucket['timestamp'][11:16]
- print(f" {hour}: {avg_duration:.0f}ms")
-
-if latencies:
- print(f"\nStatistics:")
- print(f" Min: {min(latencies):.0f}ms")
- print(f" Max: {max(latencies):.0f}ms")
- print(f" Avg: {sum(latencies)/len(latencies):.0f}ms")
-```
-
-### Filter by Successful Traces Only
-
-Analyze only successful requests:
-
-```python
-# Get metrics for successful traces only
-newest = datetime.now(timezone.utc)
-oldest = newest - timedelta(days=7)
-
-payload = {
- "focus": "trace",
- "interval": 1440,
- "windowing": {
- "oldest": oldest.isoformat(),
- "newest": newest.isoformat()
- },
- "filter": {
- "conditions": [
- {
- "field": "status.code",
- "operator": "eq",
- "value": "STATUS_CODE_OK"
- }
- ]
- }
-}
-
-response = requests.post(BASE_URL, headers=headers, json=payload)
-data = response.json()
-
-# Calculate success metrics
-total_count = sum(b['total']['count'] for b in data['buckets'])
-total_cost = sum(b['total']['costs'] for b in data['buckets'])
-total_duration = sum(b['total']['duration'] for b in data['buckets'])
-
-print("Successful Traces (Last 7 Days):")
-print(f" Count: {total_count:,}")
-print(f" Total Cost: ${total_cost:.4f}")
-print(f" Avg Duration: {total_duration/total_count:.0f}ms")
-```
-
-## Next Steps
-
-- Learn about [Query API](/observability/query-data/query-api) for detailed trace data
-- Explore [Using the UI](/observability/concepts) for visual analytics
-- Read about [Semantic Conventions](/observability/trace-with-opentelemetry/semantic-conventions) for available metrics
diff --git a/docs/docs/observability/query-data/_category_.json b/docs/docs/observability/query-data/_category_.json
deleted file mode 100644
index 93a5859395..0000000000
--- a/docs/docs/observability/query-data/_category_.json
+++ /dev/null
@@ -1,6 +0,0 @@
-{
- "label": "Query Data",
- "position": 8,
- "collapsible": true,
- "collapsed": true
-}
diff --git a/docs/docs/observability/trace-with-opentelemetry/01-getting-started.mdx b/docs/docs/observability/trace-with-opentelemetry/01-getting-started.mdx
deleted file mode 100644
index 2048c5c03f..0000000000
--- a/docs/docs/observability/trace-with-opentelemetry/01-getting-started.mdx
+++ /dev/null
@@ -1,103 +0,0 @@
----
-title: "Getting Started with OpenTelemetry"
-sidebar_label: "Getting Started"
-description: "Learn how to configure OpenTelemetry to send traces to Agenta's observability platform"
-sidebar_position: 1
----
-
-Agenta accepts traces via the OpenTelemetry Protocol (OTLP) endpoint. You can use any OpenTelemetry-compatible instrumentation library to send traces to Agenta.
-
-## OTLP Endpoint
-
-Agenta accepts traces via the **OTLP/HTTP protocol** using **protobuf** encoding:
-
-**Endpoint:** `https://cloud.agenta.ai/api/otlp/v1/traces`
-
-For self-hosted installations, replace `https://cloud.agenta.ai` with your instance URL.
-
-:::warning
-Agenta does **not** support `gRPC` for the OpenTelemetry endpoint. Please use **HTTP/protobuf** instead.
-:::
-
-## Authentication
-
-Agenta uses ApiKey-based authentication for the OTLP endpoint:
-
-```javascript
-headers: {
- Authorization: `ApiKey ${AGENTA_API_KEY}`
-}
-```
-
-### Getting Your API Key
-
-1. Visit the [Agenta API Keys page](https://cloud.agenta.ai/settings?tab=apiKeys)
-2. Click on **Create New API Key** and follow the prompts
-3. Copy the API key and set it as an environment variable:
-
-```bash
-export AGENTA_API_KEY="YOUR_AGENTA_API_KEY"
-export AGENTA_HOST="https://cloud.agenta.ai" # Change for self-hosted
-```
-
-## Configuration
-
-When using OpenTelemetry SDKs directly (without the Agenta SDK), configure the OTLP exporter to point to Agenta:
-
-```bash
-OTEL_EXPORTER_OTLP_ENDPOINT="https://cloud.agenta.ai/api/otlp"
-OTEL_EXPORTER_OTLP_TRACES_ENDPOINT="https://cloud.agenta.ai/api/otlp/v1/traces"
-OTEL_EXPORTER_OTLP_HEADERS="Authorization=ApiKey ${AGENTA_API_KEY}"
-```
-
-:::info
-If your collector requires signal-specific environment variables, use the trace-specific endpoint:
-
-```bash
-OTEL_EXPORTER_OTLP_TRACES_ENDPOINT="https://cloud.agenta.ai/api/otlp/v1/traces"
-OTEL_EXPORTER_OTLP_TRACES_HEADERS="Authorization=ApiKey ${AGENTA_API_KEY}"
-```
-:::
-
-## Supported Languages
-
-OpenTelemetry SDKs are available for many languages:
-
-- **Python**: Use the [Agenta Python SDK](/observability/quickstart-python) or OpenTelemetry SDK directly
-- **Node.js / TypeScript**: See the [OpenTelemetry Quick Start](/observability/quick-start-opentelemetry)
-- **Java**: Use the OpenTelemetry Java SDK
-- **Go**: Use the OpenTelemetry Go SDK
-- **.NET**: Use the OpenTelemetry .NET SDK
-- **Ruby, PHP, Rust**: OpenTelemetry SDKs available for all
-
-All can send traces to Agenta using the OTLP endpoint above.
-
-## Using OpenTelemetry Instrumentation Libraries
-
-Agenta is compatible with many OpenTelemetry instrumentation libraries that extend language and framework support. These libraries work seamlessly with Agenta's OTLP endpoint:
-
-### Popular Libraries
-
-- **[OpenLLMetry](https://github.com/Arize-ai/openllmetry)**: Supports multiple LLMs (OpenAI, Anthropic, Azure, etc.) and frameworks (LangChain, LlamaIndex)
-- **[OpenLIT](https://github.com/openlit/openlit)**: Comprehensive instrumentation for LLMs, vector DBs, and frameworks
-- **[OpenInference](https://arize-ai.github.io/openinference/)**: Arize's OpenTelemetry instrumentation for LLM applications
-
-### Framework Integrations
-
-Many frameworks have OpenTelemetry support built-in or via plugins:
-
-- **LangChain**: OpenTelemetry instrumentation available
-- **LlamaIndex**: OpenTelemetry support via plugins
-- **AutoGen**: OpenTelemetry compatible
-- **Semantic Kernel**: OpenTelemetry integration available
-- **Spring AI**: Java framework with OpenTelemetry support
-
-See the [semantic conventions](/observability/trace-with-opentelemetry/semantic-conventions) page for details on how Agenta maps OpenTelemetry attributes.
-
-## Next Steps
-
-- Follow the [OpenTelemetry Quick Start](/observability/quick-start-opentelemetry) for a complete example
-- Learn about [semantic conventions](/observability/trace-with-opentelemetry/semantic-conventions) for better trace display
-- Explore [distributed tracing](/observability/trace-with-opentelemetry/distributed-tracing) across services
-- Configure the [OpenTelemetry Collector](/observability/trace-with-opentelemetry/otel-collector-configuration) to forward traces
-
diff --git a/docs/docs/observability/trace-with-opentelemetry/03-semantic-conventions.mdx b/docs/docs/observability/trace-with-opentelemetry/03-semantic-conventions.mdx
deleted file mode 100644
index 6fbe80c3de..0000000000
--- a/docs/docs/observability/trace-with-opentelemetry/03-semantic-conventions.mdx
+++ /dev/null
@@ -1,298 +0,0 @@
----
-title: "Semantic Conventions"
-sidebar_label: "Semantic Conventions"
-description: "Learn about the OpenTelemetry semantic conventions used by Agenta's LLM observability system"
-sidebar_position: 3
----
-
-This document describes how Agenta applies domain-specific OpenTelemetry conventions to capture and analyze traces from LLM applications.
-
-## How It Works
-
-Agenta accepts any span that follows the OpenTelemetry specification. To unlock LLM-specific features—such as nicely formatted chat messages, per-request cost and latency, and links to prompt configurations or evaluators—we add attributes under the `ag.` namespace.
-
-:::info
-The OTLP endpoint accepts batches up to **5 MB** by default (after decompression). Larger requests return an HTTP `413` status code.
-:::
-
-We support two primary instrumentation approaches:
-
-- **Agenta SDK:** When using our SDK with decorators like `@ag.instrument`, we automatically handle the semantic conventions
-- **Auto-instrumentation libraries:** We provide adapters that transform data from standard auto-instrumentation libraries (which follow their own conventions) into Agenta's format
-
-## Agenta Namespace
-
-All Agenta-specific attributes are organized under the `ag` namespace to avoid conflicts with other OpenTelemetry conventions.
-
-## Core Semantic Conventions
-
-### ag.data
-
-The `ag.data` namespace contains the core execution data for each span:
-
-- **`ag.data.inputs`**: Input parameters for the span
-- **`ag.data.outputs`**: Output results from the span
-- **`ag.data.internals`**: Internal variables and intermediate values
-
-#### Data Format
-
-**Inputs and Outputs**: Stored as JSON format. For chat applications using LLM spans, these are formatted as a list of messages:
-
-```json
-{
- "data": {
- "inputs": {
- "prompt": [
- { "role": "system", "content": "System instruction" },
- { "role": "user", "content": "User query" }
- ],
- "functions": [...],
- "tools": [...]
- },
- "outputs": {
- "completion": [
- { "role": "assistant", "content": "Assistant response" }
- ]
- }
- }
-}
-```
-
-**Internals**: User-provided internal information such as context variables, intermediate calculations, or evaluation data that aren't part of the primary inputs/outputs. These are set by the user [in the SDK using](/observability/trace-with-python-sdk/adding-metadata) `ag.tracing.store_internals()`.
-
-#### SDK Integration
-
-When using the `@ag.instrument` decorator in Python:
-
-```python
-@ag.instrument
-def my_function(input_param):
- # Function inputs and outputs are automatically captured
- # unless explicitly masked
- return result
-```
-
-The decorator automatically captures function inputs and outputs in `ag.data.inputs` and `ag.data.outputs` unless you choose to [mask sensitive data](/observability/trace-with-python-sdk/redact-sensitive-data).
-
-### ag.meta
-
-The `ag.meta` namespace stores metadata about the span execution:
-
-```json
-{
- "meta": {
- "system": "openai",
- "request": {
- "base_url": "https://api.openai.com/v1",
- "endpoint": "/chat/completions",
- "headers": {...},
- "streaming": false,
- "model": "gpt-4",
- "max_tokens": 1000,
- "temperature": 0.7,
- "top_p": 0.9,
- "top_k": 50
- },
- "response": {
- "model": "gpt-4-0613"
- },
- "configuration": {
- "prompt": {
- "input_keys": ["country"],
- "messages": [...],
- "template_format": "curly",
- "llm_config": {
- "model": "mistral/mistral-tiny"
- }
- }
- }
- }
-}
-```
-
-#### Standard Metadata
-
-:::info
-Auto-instrumentation maps common semantic-convention keys—e.g. gen_ai.system, gen_ai.request.*—to the structure above.
-:::
-
-- `ag.meta.system`: Identifies the LLM provider or system being used (e.g., "openai", "anthropic")
-- `ag.meta.request.base_url`: The base URL for the API request
-- `ag.meta.request.endpoint`: The specific API endpoint called
-- `ag.meta.request.headers`: HTTP headers sent with the request
-- `ag.meta.request.streaming`: Boolean indicating if streaming mode was used
-- `ag.meta.request.model`: The model name requested
-- `ag.meta.request.max_tokens`: Maximum token limit for the response
-- `ag.meta.request.temperature`: Sampling temperature parameter
-- `ag.meta.request.top_p`: Nucleus sampling parameter
-- `ag.meta.request.top_k`: Top-k sampling parameter
-
-Metadata is displayed in the observability overview page as contextual information to help navigate and understand span execution.
-
-### References
-
-Use the top level `references` array to link spans to Agenta entities. Every entry represents one relationship and includes:
-
-- `attributes.key`: the reference category (for example `application`, `evaluator_variant`)
-- `id`, `slug`, or `version`: supply whichever identifiers you have; you can include more than one field if available
-
-Example payload:
-
-```json
-{
- "references": [
- {"id": "019a0159-82d3-7760-9868-4f8c7da8e9c0", "attributes": {"key": "application"}},
- {"slug": "production", "attributes": {"key": "environment"}},
- {"id": "019a0159-82d3-7760-9868-4f8c7da8e9c1", "version": "4", "attributes": {"key": "application_variant"}}
- ]
-}
-```
-
-Supported categories:
-
-- application, application_variant, application_revision
-- environment, environment_variant, environment_revision
-- evaluator, evaluator_variant, evaluator_revision
-- testset, testset_variant, testset_revision, testcase
-- query, query_variant, query_revision
-- workflow, workflow_variant, workflow_revision
-
-Consumers (UI, analytics, filtering) read from this array. Instrumentation libraries that cannot emit the array may still set the attribute form (`ag.references..`); the ingestion service converts that dictionary into the same array before storage.
-
-:::warning
-The legacy `ag.refs.*` namespace is deprecated and will be removed after existing SDKs migrate. Do not rely on it.
-:::
-
-### ag.metrics
-
-The `ag.metrics` namespace tracks performance, cost, and error metrics:
-
-```json
-{
- "metrics": {
- "costs": {
- "cumulative": {
- "total": 0.0070902,
- "prompt": 0.00355,
- "completion": 0.00354
- },
- "incremental": {
- "total": 0.0070902
- }
- },
- "tokens": {
- "cumulative": {
- "total": 992,
- "prompt": 175,
- "completion": 817
- },
- "incremental": {
- "total": 992,
- "prompt": 175,
- "completion": 817
- }
- },
- "duration": {
- "cumulative": 19889.343
- },
- "errors": {}
- }
-}
-```
-
-#### Aggregation Types
-
-Metrics are tracked at two levels:
-
-- **`incremental`**: Metrics for this span only (excluding child spans)
-- **`cumulative`**: Metrics for this span plus all child spans aggregated together
-
-This dual tracking allows you to see both the cost of individual operations and the total cost of complex workflows.
-
-#### Metric Categories
-
-##### Costs
-
-Tracks LLM API costs in USD with the following breakdown:
-
-**Cumulative (this span + children):**
-- **`ag.metrics.costs.cumulative.total`**: Total cost across all LLM calls in this span and its children
-- **`ag.metrics.costs.cumulative.prompt`**: Cost attributed to input tokens
-- **`ag.metrics.costs.cumulative.completion`**: Cost attributed to output/completion tokens
-
-**Incremental (this span only):**
-- **`ag.metrics.costs.incremental.total`**: Cost for this span's operations only
-- **`ag.metrics.costs.incremental.prompt`**: Prompt cost for this span only
-- **`ag.metrics.costs.incremental.completion`**: Completion cost for this span only
-
-:::info
-Cost calculation uses the latest pricing for each model provider. Costs are automatically calculated when using standard LLM integrations. Cumulative metrics are automatically calculated by the backend by aggregating incremental values.
-:::
-
-##### Tokens
-
-Tracks token usage at both aggregation levels:
-
-**Cumulative:**
-- **`ag.metrics.tokens.cumulative.total`**: Total tokens across all operations
-- **`ag.metrics.tokens.cumulative.prompt`**: Input tokens across all operations
-- **`ag.metrics.tokens.cumulative.completion`**: Output tokens across all operations
-
-**Incremental:**
-- **`ag.metrics.tokens.incremental.total`**: Tokens for this span only
-- **`ag.metrics.tokens.incremental.prompt`**: Input tokens for this span only
-- **`ag.metrics.tokens.incremental.completion`**: Output tokens for this span only
-
-##### Duration
-
-Tracks execution time in milliseconds:
-
-- **`ag.metrics.duration.cumulative`**: Total execution time including all child spans
-
-
-## Additional Agenta Attributes
-
-### ag.type
-
-The `ag.type` namespace contains type information about the span:
-
-`ag.type.node` can be `workflow`, `task`, `tool`, `embedding`, `query`, `completion`, `chat`, `rerank`.
-
-### ag.tags
-
-The `ag.tags` namespace allows you to add custom tags to spans for categorization and filtering:
-
-```python
-# Add custom tags using the SDK
-ag.add_tags({"key":"value"})
-```
-
-### ag.version
-
-Tracks version information for the instrumented application or component.
-
-### ag.flags
-
-Contains system-specific flags and metadata used internally by Agenta for span processing and display.
-
-:::note
-When using auto-instrumentation libraries, most attributes are saved twice - once in their original format and once processed under the `ag` namespace
-:::
-
-## Standard OpenTelemetry Attributes
-
-In addition to Agenta-specific conventions, traces include standard OpenTelemetry attributes:
-
-- **Links**: Relationships between spans
-- **Events**: Timestamped events within spans
-- **Version**: OpenTelemetry version information
-- **Status Code**: Span completion status
-- **Start Time**: Span initiation timestamp
-- **Span Name**: Human-readable span identifier
-- **Span Kind**: Type of span (server, client, internal, etc.)
-
-## Next steps
-
-- Learn about [distributed tracing](/observability/trace-with-opentelemetry/distributed-tracing)
-- Explore [Python SDK tracing](/observability/trace-with-python-sdk/setup-tracing) for easier instrumentation
-- See [integration guides](/observability/integrations/openai) for specific frameworks
diff --git a/docs/docs/observability/trace-with-opentelemetry/04-distributed-tracing.mdx b/docs/docs/observability/trace-with-opentelemetry/04-distributed-tracing.mdx
deleted file mode 100644
index 57a3a99021..0000000000
--- a/docs/docs/observability/trace-with-opentelemetry/04-distributed-tracing.mdx
+++ /dev/null
@@ -1,85 +0,0 @@
----
-title: "Distributed Tracing"
-sidebar_label: "Distributed Tracing"
-description: "Learn how to implement distributed tracing across services with OpenTelemetry and Agenta"
-sidebar_position: 4
----
-
-## OpenTelemetry Tracing without Agenta SDK
-
-
-If you're working with systems that don't use the Agenta SDK, you can still integrate with Agenta's tracing infrastructure using standard OpenTelemetry.
-
-### 1. Setup Requirements
-
-Install dependencies:
-
-```bash
-pip install opentelemetry-api opentelemetry-sdk opentelemetry-exporter-otlp
-```
-
-### 2. Configure Environment Variables
-
-```bash
-# OTEL_PROPAGATORS = unset or "tracecontext,baggage"
-# OTEL_EXPORTER_OTLP_COMPRESSION = unset or "gzip"
-# OTEL_EXPORTER_OTLP_ENDPOINT = "https://cloud.agenta.ai/api/otlp"
-# OTEL_EXPORTER_OTLP_HEADERS = "authorization=ApiKey xxx"
-# OTEL_EXPORTER_OTLP_TRACES_ENDPOINT = "https://cloud.agenta.ai/api/otlp/v1/traces"
-# OTEL_EXPORTER_OTLP_TRACES_HEADERS = "authorization=ApiKey xxx"
-```
-
-### 3. Setup in Code
-
-```python
-from opentelemetry.trace.propagation.tracecontext import TraceContextTextMapPropagator
-from opentelemetry.baggage.propagation import W3CBaggagePropagator
-from opentelemetry.sdk.trace import TracerProvider, Span
-from opentelemetry.sdk.trace.export import BatchSpanProcessor
-from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter, Compression
-
-# Configuration
-endpoint = "https://cloud.agenta.ai/api/otlp/v1/traces"
-compression = Compression.Gzip
-headers = {
- "traceparent": "00-xxx-xxx-01",
- "baggage": "ag.refs.application.id=xxx",
- "authorization": "ApiKey xxx",
-}
-
-# Set up provider, processor, and tracer
-provider = TracerProvider()
-
-processor = BatchSpanProcessor(
- OTLPSpanExporter(
- endpoint=endpoint,
- headers={"authorization": headers["authorization"]},
- compression=compression,
- )
-)
-
-provider.add_span_processor(processor)
-
-tracer = provider.get_tracer("agenta.tracer")
-
-# Extract incoming trace context
-carrier = {"traceparent": headers["traceparent"]}
-context = TraceContextTextMapPropagator().extract(carrier=carrier, context=None)
-
-carrier = {"baggage": headers["baggage"]}
-context = W3CBaggagePropagator().extract(carrier=carrier, context=context)
-
-# Create and use spans
-with tracer.start_as_current_span(name="agenta", context=context) as span:
- span: Span
-
- print(hex(span.get_span_context().trace_id))
- print(hex(span.get_span_context().span_id))
- print(span.name)
-```
-
-## Next steps
-
-- Learn about [semantic conventions](/observability/trace-with-opentelemetry/semantic-conventions)
-- Explore [collector configuration](/observability/trace-with-opentelemetry/otel-collector-configuration)
-- See [Python SDK distributed tracing](/observability/trace-with-python-sdk/distributed-tracing) for Agenta SDK approach
diff --git a/docs/docs/observability/trace-with-opentelemetry/05-otel-collector-configuration.mdx b/docs/docs/observability/trace-with-opentelemetry/05-otel-collector-configuration.mdx
deleted file mode 100644
index 45c6571dfc..0000000000
--- a/docs/docs/observability/trace-with-opentelemetry/05-otel-collector-configuration.mdx
+++ /dev/null
@@ -1,69 +0,0 @@
----
-title: "OpenTelemetry Collector Configuration"
-sidebar_label: "OTEL Collector Configuration"
-description: "Configure the OpenTelemetry Collector to forward traces to Agenta"
-sidebar_position: 5
----
-
-The [OpenTelemetry Collector](https://opentelemetry.io/docs/collector) is a vendor-agnostic service that receives, processes, and exports telemetry data. You can use it to collect traces from multiple sources and forward them to Agenta.
-
-## Configuration
-
-Here's a configuration file (`otel-collector-config.yml`) that receives traces via OTLP/HTTP and forwards them to Agenta:
-
-```yaml
-receivers:
- otlp:
- protocols:
- http:
- endpoint: 0.0.0.0:4318
-
-processors:
- batch:
- timeout: 5s
- send_batch_size: 512
-
-exporters:
- otlphttp/agenta:
- endpoint: "https://cloud.agenta.ai/api/otlp/v1/traces"
- headers:
- Authorization: "ApiKey ${AGENTA_API_KEY}"
-
-service:
- pipelines:
- traces:
- receivers: [otlp]
- processors: [batch]
- exporters: [otlphttp/agenta]
-```
-
-### Configuration Details
-
-**Receivers**: The collector receives traces via OTLP/HTTP on port `4318`.
-
-**Processors**: The `batch` processor collects spans and sends them in batches:
-- `timeout`: Maximum time to wait before sending a batch (default: 5s)
-- `send_batch_size`: Number of spans to collect before sending (default: 512)
-
-**Exporters**: The `otlphttp/agenta` exporter forwards traces to Agenta using HTTP/protobuf:
-- Endpoint: `https://cloud.agenta.ai/api/otlp/v1/traces`
-- Authentication: Uses `ApiKey` authentication header
-
-:::warning
-Agenta only supports **HTTP/protobuf** for the OpenTelemetry endpoint. gRPC is not supported.
-:::
-
-For self-hosted Agenta deployments, replace the endpoint in the exporter configuration:
-
-```yaml
-exporters:
- otlphttp/agenta:
- endpoint: "http://your-agenta-instance:port/api/otlp/v1/traces"
- headers:
- Authorization: "ApiKey ${AGENTA_API_KEY}"
-```
-
-## Next Steps
-
-- Learn about [semantic conventions](/observability/trace-with-opentelemetry/semantic-conventions) for proper trace formatting
-- Explore [distributed tracing](/observability/trace-with-opentelemetry/distributed-tracing) without the Agenta SDK
diff --git a/docs/docs/observability/trace-with-opentelemetry/_category_.json b/docs/docs/observability/trace-with-opentelemetry/_category_.json
deleted file mode 100644
index aef5391a04..0000000000
--- a/docs/docs/observability/trace-with-opentelemetry/_category_.json
+++ /dev/null
@@ -1,6 +0,0 @@
-{
- "label": "Trace with OpenTelemetry",
- "position": 6,
- "collapsible": true,
- "collapsed": true
-}
diff --git a/docs/docs/observability/trace-with-python-sdk/01-setup-tracing.mdx b/docs/docs/observability/trace-with-python-sdk/01-setup-tracing.mdx
deleted file mode 100644
index bd063ec70b..0000000000
--- a/docs/docs/observability/trace-with-python-sdk/01-setup-tracing.mdx
+++ /dev/null
@@ -1,63 +0,0 @@
----
-title: "Setup Tracing"
-sidebar_label: "Setup Tracing"
-description: "Learn how to set up tracing with the Agenta Python SDK for LLM observability"
-sidebar_position: 1
----
-
-```mdx-code-block
-import GoogleColabButton from "@site/src/components/GoogleColabButton";
-```
-
-
- Open in Google Colaboratory
-
-
-The Agenta Observability SDK integrates with the OpenTelemetry SDK. It wraps OpenTelemetry and provides a user-friendly way to instrument your LLM applications.
-
-The SDK provides:
-
-- Automatic OpenTelemetry setup
-- Easy function instrumentation using [decorators](/observability/trace-with-python-sdk/instrument-functions)
-- [Reference prompt versions](/observability/trace-with-python-sdk/reference-prompt-versions) to link traces to applications, variants, and environments
-- [Add attributes](/observability/trace-with-python-sdk/adding-metadata) to spans for additional metadata
-
-The SDK works with auto instrumentation. You should use auto instrumentation together with this SDK.
-
-## Installation
-
-**1. Install the Agenta SDK**
-
-```bash
-pip install -U agenta
-```
-
-## Configuration
-
-**2. Set environment variables**
-
-1. Visit the [Agenta API Keys page](https://cloud.agenta.ai/settings?tab=apiKeys).
-2. Click on **Create New API Key** and follow the prompts.
-
-```python
-import os
-
-os.environ["AGENTA_API_KEY"] = "YOUR_AGENTA_API_KEY"
-os.environ["AGENTA_HOST"] = "https://cloud.agenta.ai"
-```
-
-**3. Initialize the SDK**
-
-```python
-import agenta as ag
-
-ag.init()
-```
-
-That's it! You're now ready to instrument your functions and start capturing traces.
-
-## Next steps
-
-- Learn how to [instrument your functions](/observability/trace-with-python-sdk/instrument-functions)
-- Link traces to [prompt versions](/observability/trace-with-python-sdk/reference-prompt-versions)
-- Understand how to [redact sensitive data](/observability/trace-with-python-sdk/redact-sensitive-data)
diff --git a/docs/docs/observability/trace-with-python-sdk/02-instrument-functions.mdx b/docs/docs/observability/trace-with-python-sdk/02-instrument-functions.mdx
deleted file mode 100644
index 86249b2b4b..0000000000
--- a/docs/docs/observability/trace-with-python-sdk/02-instrument-functions.mdx
+++ /dev/null
@@ -1,55 +0,0 @@
----
-title: "Instrument Your Functions"
-sidebar_label: "Instrument Functions"
-description: "Learn how to instrument functions for LLM observability and tracing using the Agenta Python SDK decorator"
-sidebar_position: 2
----
-
-```mdx-code-block
-import GoogleColabButton from "@site/src/components/GoogleColabButton";
-```
-
-
- Open in Google Colaboratory
-
-
-To instrument a function, add the `@ag.instrument()` decorator. This automatically captures all input and output data.
-
-The decorator has a `spankind` argument to categorize each span in the UI. Available types are:
-
-`agent`, `chain`, `workflow`, `tool`, `embedding`, `query`, `completion`, `chat`, `rerank`
-
-:::info
-The default span kind is `workflow`.
-:::
-
-:::caution
-The instrument decorator should be the top-most decorator on a function (i.e. the last decorator before the function call).
-:::
-
-```python
-import agenta as ag
-
-@ag.instrument(spankind="task")
-def my_llm_call(country: str):
- prompt = f"What is the capital of {country}"
- response = client.chat.completions.create(
- model='gpt-4',
- messages=[
- {'role': 'user', 'content': prompt},
- ],
- )
- return response.choices[0].text
-
-@ag.instrument(spankind="workflow")
-def generate(country: str):
- return my_llm_call(country)
-```
-
-Agenta automatically determines the parent span based on the function call and nests the spans accordingly.
-
-## Next steps
-
-- Learn how to [add metadata and internals](/observability/trace-with-python-sdk/adding-metadata)
-- Link traces to [prompt versions](/observability/trace-with-python-sdk/reference-prompt-versions)
-- Understand how to [redact sensitive data](/observability/trace-with-python-sdk/redact-sensitive-data)
diff --git a/docs/docs/observability/trace-with-python-sdk/03-adding-metadata.mdx b/docs/docs/observability/trace-with-python-sdk/03-adding-metadata.mdx
deleted file mode 100644
index f0cbe08edc..0000000000
--- a/docs/docs/observability/trace-with-python-sdk/03-adding-metadata.mdx
+++ /dev/null
@@ -1,75 +0,0 @@
----
-title: "Adding Metadata"
-sidebar_label: "Adding Metadata"
-description: "Learn how to add metadata and internals to spans for LLM observability and tracing"
-sidebar_position: 3
----
-
-```mdx-code-block
-import GoogleColabButton from "@site/src/components/GoogleColabButton";
-```
-
-
- Open in Google Colaboratory
-
-
-You can add additional information to spans using metadata and internals. Both use semantic conventions under the `ag` namespace. Metadata is saved under `ag.meta`. Internals are saved under `ag.data.internals`.
-
-See the [semantic conventions guide](/observability/trace-with-opentelemetry/semantic-conventions) for more details on how attributes are organized.
-
-## Adding metadata
-
-Use `ag.tracing.store_meta()` to add metadata to a span. This function accesses the active span from the context and adds the key-value pairs to the metadata.
-
-```python
-@ag.instrument(spankind="task")
-def compile_prompt(country: str):
- prompt = f"What is the capital of {country}"
-
- # highlight-next-line
- ag.tracing.store_meta({"prompt_template": prompt})
-
- formatted_prompt = prompt.format(country=country)
- return formatted_prompt
-```
-
-## Storing internals
-
-Use `ag.tracing.store_internals()` to store internals in a span:
-
-```python
-@ag.instrument(spankind="workflow")
-def rag_workflow(query: str):
-
- context = retrieve_context(query)
-
- # highlight-start
- ag.tracing.store_internals({"context": context})
- # highlight-end
-
- prompt = f"Answer the following question {query} based on the context: {context}"
-
- completion = client.chat.completions.create(
- model='gpt-4',
- messages=[
- {'role': 'user', 'content': prompt},
- ],
- )
- return completion.choices[0].message.content
-```
-
-## Differences between metadata and internals
-
-Both metadata and internals can be used for evaluation and filtering. The main differences are:
-
-1. Internals are searchable using plain text queries because they are saved under `ag.data`.
-2. Internals are shown in the overview tab of the observability drawer together with inputs and outputs, making them easy to see.
-
-As a rule of thumb, if your context is short, put important information that helps understand the span into internals.
-
-## Next steps
-
-- Link traces to [prompt versions](/observability/trace-with-python-sdk/reference-prompt-versions)
-- Understand how to [redact sensitive data](/observability/trace-with-python-sdk/redact-sensitive-data)
-- Explore [distributed tracing](/observability/trace-with-python-sdk/distributed-tracing) across services
-
diff --git a/docs/docs/observability/trace-with-python-sdk/04-reference-prompt-versions.mdx b/docs/docs/observability/trace-with-python-sdk/04-reference-prompt-versions.mdx
deleted file mode 100644
index 19bbbb9d1a..0000000000
--- a/docs/docs/observability/trace-with-python-sdk/04-reference-prompt-versions.mdx
+++ /dev/null
@@ -1,79 +0,0 @@
----
-title: "Reference Prompt Versions"
-sidebar_label: "Reference Prompt Versions"
-description: "Learn how to link traces to specific applications, variants, and environments in Agenta"
-sidebar_position: 4
----
-
-```mdx-code-block
-import GoogleColabButton from "@site/src/components/GoogleColabButton";
-```
-
-
- Open in Google Colaboratory
-
-
-You can link a span to an application, variant, and environment by calling `ag.tracing.store_refs()`.
-
-Applications, variants, and environments can be referenced by their slugs, versions, and commit IDs (for specific versions).
-
-## Basic usage
-
-You can link a span to an application and variant like this:
-
-```python
-import agenta as ag
-
-@ag.instrument(spankind="workflow")
-def generate(country: str):
- prompt = f"What is the capital of {country}"
-
- formatted_prompt = prompt.format(country=country)
-
- completion = client.chat.completions.create(
- model='gpt-4',
- messages=[
- {'role': 'user', 'content': formatted_prompt},
- ],
- )
-
- # highlight-start
- ag.tracing.store_refs(
- {
- "application.slug": "capital-app",
- "environment.slug": "production",
- }
- )
- # highlight-end
- return completion.choices[0].message.content
-```
-
-## Available reference keys
-
-`ag.tracing.store_refs()` takes a dict with keys from:
-
-- `application.slug`
-- `application.id`
-- `variant.slug`
-- `variant.id`
-- `variant.version`
-- `environment.slug`
-- `environment.id`
-- `environment.version`
-
-The values should be the slug, id, or version of the application, variant, and environment respectively.
-
-## Why link traces?
-
-Linking traces to applications and variants allows you to:
-
-- **Filter traces** by application, variant, or environment in the UI
-- **Compare performance** across different variants
-- **Track production behavior** by environment
-- **Create test sets** from production traces with proper context
-
-## Next steps
-
-- Learn how to [redact sensitive data](/observability/trace-with-python-sdk/redact-sensitive-data)
-- Understand how to [track costs](/observability/trace-with-python-sdk/track-costs)
-- Explore [distributed tracing](/observability/trace-with-python-sdk/distributed-tracing) across services
diff --git a/docs/docs/observability/trace-with-python-sdk/05-redact-sensitive-data.mdx b/docs/docs/observability/trace-with-python-sdk/05-redact-sensitive-data.mdx
deleted file mode 100644
index 7b71f46203..0000000000
--- a/docs/docs/observability/trace-with-python-sdk/05-redact-sensitive-data.mdx
+++ /dev/null
@@ -1,139 +0,0 @@
----
-title: "Redact Sensitive Data"
-sidebar_label: "Redact Sensitive Data"
-description: "Learn how to exclude sensitive data from traces using the Agenta Python SDK"
-sidebar_position: 5
----
-
-```mdx-code-block
-import GoogleColabButton from "@site/src/components/GoogleColabButton";
-```
-
-
- Open in Google Colaboratory
-
-
-In some cases, you may want to exclude parts of the inputs or outputs due to privacy concerns or because the data is too large to be stored in the span.
-
-## Simple redaction
-
-You can do this by setting the `ignore_inputs` and/or `ignore_outputs` arguments to `True` in the instrument decorator.
-
-```python
-import agenta as ag
-
-@ag.instrument(
- spankind="workflow",
- ignore_inputs=True,
- ignore_outputs=True
-)
-def rag_workflow(query: str):
- ...
-```
-
-## Selective redaction
-
-If you want more control, you can specify which parts of the inputs and outputs to exclude:
-
-```python
-@ag.instrument(
- spankind="workflow",
- ignore_inputs=["user_id"],
- ignore_outputs=["pii"],
-)
-def rag_workflow(query: str, user_id: str):
- ...
- return {
- "result": ...,
- "pii": ...
- }
-```
-
-## Custom redaction callback
-
-For even finer control, you can use a custom `redact()` callback, along with instructions in the case of errors.
-
-```python
-def my_redact(name, field, data):
- if name == "rag_workflow":
- if field == "inputs":
- del data["user_id"]
- if field == "outputs":
- del data["pii"]
-
- return data
-
-
-@ag.instrument(
- spankind="workflow",
- redact=my_redact,
- redact_on_error=False,
-)
-def rag_workflow(query: str, user_id: str):
- ...
- return {
- "result": ...,
- "pii": ...
- }
-```
-
-## Global redaction rules
-
-Finally, if you want to set up global rules for redaction, you can provide a global `redact()` callback that applies everywhere.
-
-```python
-from typing import Dict, Any
-
-def global_redact(
- name: str,
- field: str,
- data: Dict[str, Any]
-):
- if "pii" in data:
- del data["pii"]
-
- return data
-
-
-ag.init(
- redact=global_redact,
- redact_on_error=True,
-)
-
-def local_redact(
- name: str,
- field: str,
- data: Dict[str, Any]
-):
- if name == "rag_workflow":
- if field == "inputs":
- del data["user_id"]
-
- return data
-
-
-@ag.instrument(
- spankind="workflow",
- redact=local_redact,
- redact_on_error=False,
-)
-def rag_workflow(query: str, user_id: str):
- ...
- return {
- "result": ...,
- "pii": ...
- }
-```
-
-## Best practices
-
-- **Use selective redaction** rather than blocking all inputs/outputs when possible
-- **Test your redaction rules** to ensure they work as expected
-- **Consider global rules** for organization-wide PII policies
-- **Document redacted fields** so team members know what data is missing
-
-## Next steps
-
-- Understand how to [track costs](/observability/trace-with-python-sdk/track-costs)
-- Explore [distributed tracing](/observability/trace-with-python-sdk/distributed-tracing) across services
-- Learn how to [annotate traces](/observability/trace-with-python-sdk/annotate-traces) programmatically
diff --git a/docs/docs/observability/trace-with-python-sdk/06-track-costs.mdx b/docs/docs/observability/trace-with-python-sdk/06-track-costs.mdx
deleted file mode 100644
index 9ca6bc1b01..0000000000
--- a/docs/docs/observability/trace-with-python-sdk/06-track-costs.mdx
+++ /dev/null
@@ -1,167 +0,0 @@
----
-title: "Track Costs"
-sidebar_label: "Track Costs"
-description: "Learn how Agenta automatically tracks and aggregates LLM costs, token usage, and performance metrics across your traces"
-sidebar_position: 6
----
-
-Agenta automatically tracks costs, token usage, and performance metrics for your LLM applications. This data is captured in the `ag.metrics` namespace of each span.
-
-## Overview
-
-When you instrument your application with Agenta, we automatically collect cost and performance metrics for spans of type chat.
-
-Costs are calculated using the latest pricing for each model provider. Token usage is tracked separately for input (prompt) and output (completion) tokens. Execution time is measured in milliseconds for each operation.
-
-## Metrics Structure
-
-### Cost Metrics
-
-Costs are tracked in USD with the following breakdown:
-
-```json
-{
- "metrics": {
- "costs": {
- "cumulative": {
- "total": 0.0070902,
- "prompt": 0.00355,
- "completion": 0.00354
- }
- }
- }
-}
-```
-
-The `total` field shows the total cost across all LLM calls in this span and its children. The `prompt` field shows the cost attributed to input tokens. The `completion` field shows the cost for output tokens.
-
-### Token Usage
-
-Token consumption is tracked with separate counts for input and output:
-
-```json
-{
- "metrics": {
- "tokens": {
- "cumulative": {
- "total": 992,
- "prompt": 175,
- "completion": 817
- }
- }
- }
-}
-```
-
-The `total` field shows all tokens used (prompt plus completion). The `prompt` field shows input tokens consumed. The `completion` field shows output tokens generated.
-
-### Duration
-
-Execution time is measured in milliseconds:
-
-```json
-{
- "metrics": {
- "duration": {
- "cumulative": 19889.343
- }
- }
-}
-```
-
-:::info
-Agenta tracks metrics at two levels. **Incremental metrics** represent costs for a single span only. **Cumulative metrics** aggregate values from the current span plus all child spans.
-:::
-
-## How to Track Costs
-
-### With Auto-Instrumentation
-
-When you use auto-instrumentation from [compatible libraries](/observability/concepts#auto-instrumentation-compatibility), prompts and tokens are automatically extracted and formatted. Costs are calculated when possible.
-
-```python
-import agenta as ag
-from openinference.instrumentation.openai import OpenAIInstrumentor
-
-ag.init()
-OpenAIInstrumentor().instrument()
-
-@ag.instrument()
-def generate_response(prompt: str):
- response = client.chat.completions.create(
- model="gpt-4",
- messages=[{"role": "user", "content": prompt}]
- )
- return response.choices[0].message.content
-```
-
-### With Manual Instrumentation
-
-You can manually add cost metrics to spans using incremental metrics:
-
-```python
-import agenta as ag
-
-@ag.instrument()
-def custom_llm_call(prompt: str):
- # Your custom LLM call logic
- response = my_custom_llm.generate(prompt)
-
- # Manually track incremental metrics (for this span only)
- ag.tracing.store_metrics({
- "costs.incremental.total": 0.0025,
- "costs.incremental.prompt": 0.0015,
- "costs.incremental.completion": 0.001,
- "tokens.incremental.total": 150,
- "tokens.incremental.prompt": 100,
- "tokens.incremental.completion": 50
- })
-
- # Cumulative metrics are automatically calculated by the backend
-
- return response
-```
-
-## Automatic Cost Calculation
-
-Agenta calculates costs automatically for major LLM providers using the LiteLLM library. When the cost is not provided in the span and the span type is chat, we try to infer the cost from the number of tokens.
-
-### Custom Pricing
-
-For custom models or providers, you can manually set costs using incremental metrics:
-
-```python
-import agenta as ag
-
-@ag.instrument()
-def custom_model_call(prompt: str):
- response = my_model.generate(prompt)
-
- # Calculate custom cost
- prompt_tokens = len(prompt.split())
- completion_tokens = len(response.split())
-
- # Custom pricing
- cost_per_prompt_token = 0.00001
- cost_per_completion_token = 0.00002
-
- prompt_cost = prompt_tokens * cost_per_prompt_token
- completion_cost = completion_tokens * cost_per_completion_token
- total_cost = prompt_cost + completion_cost
-
- # Set incremental metrics
- ag.tracing.store_metrics({
- "costs.incremental.total": total_cost,
- "costs.incremental.prompt": prompt_cost,
- "costs.incremental.completion": completion_cost,
- "tokens.incremental.total": prompt_tokens + completion_tokens,
- "tokens.incremental.prompt": prompt_tokens,
- "tokens.incremental.completion": completion_tokens
- })
-
- return response
-```
-
-## Next steps
-
-Learn about [adding metadata](/observability/trace-with-python-sdk/adding-metadata) to enrich your traces.
diff --git a/docs/docs/observability/trace-with-python-sdk/07-distributed-tracing.mdx b/docs/docs/observability/trace-with-python-sdk/07-distributed-tracing.mdx
deleted file mode 100644
index 8423b80b66..0000000000
--- a/docs/docs/observability/trace-with-python-sdk/07-distributed-tracing.mdx
+++ /dev/null
@@ -1,67 +0,0 @@
----
-title: "Distributed Tracing"
-sidebar_label: "Distributed Tracing"
-description: "Learn how to implement distributed tracing across services with the Agenta SDK"
-sidebar_position: 7
----
-
-When using the Agenta SDK, distributed tracing is handled automatically with the provided SDK functions. This guide shows you how to propagate trace context across services and extract it when receiving requests.
-
-## Using OpenTelemetry with Agenta SDK
-
-Agenta supports distributed tracing out of the box when using the provided SDK functions:
-
-### 1. Sending Requests (Propagation)
-
-When making requests to other services or sub-systems, use `agenta.tracing.inject()` to inject necessary headers:
-
-```python
-import agenta as ag
-
-method = "POST"
-url = "https://example-service/api"
-params = {}
-headers = agenta.tracing.inject() # automatically injects 'Authorization', 'Traceparent', 'Baggage'
-body = {"key": "value"}
-
-response = requests.request(
- method=method,
- url=url,
- params=params,
- headers=headers,
- json=body,
-)
-```
-
-The `agenta.tracing.inject()` function returns headers containing:
-
-- `Authorization`: Authentication information
-- `Traceparent`: Identifies the current trace and span
-- `Baggage`: Contains application-specific context
-
-These headers can be modified before sending them as part of the request if needed.
-
-### 2. Receiving Requests (Extraction)
-
-Agenta simplifies receiving and handling incoming trace contexts:
-
-- If you're using `ag.route()` and `ag.instrument()`, extraction is automatic.
-- For manual extraction, use `agenta.tracing.extract()`:
-
-```python
-traceparent, baggage = agenta.tracing.extract() # includes 'Traceparent', 'Baggage'
-
-# Use traceparent and baggage to set up your OpenTelemetry context
-# (Implementation depends on your specific use case)
-```
-
-:::note
-`extract()` does not provide `Authorization` because there are many authentication methods (apikey, bearer, secret, access tokens), each requiring different handling. The middlewares and decorators in the Agenta SDK handle this automatically when you use `ag.route()` and `ag.instrument()`.
-:::
-
-## Next Steps
-
-- Learn about [tracing without the Agenta SDK](/observability/trace-with-opentelemetry/distributed-tracing) for raw OpenTelemetry setup
-- Explore [semantic conventions](/observability/trace-with-opentelemetry/semantic-conventions) for better trace formatting
-- See [instrumenting functions](/observability/trace-with-python-sdk/instrument-functions) for automatic instrumentation
-
diff --git a/docs/docs/observability/trace-with-python-sdk/08-annotate-traces.mdx b/docs/docs/observability/trace-with-python-sdk/08-annotate-traces.mdx
deleted file mode 100644
index 4240c70168..0000000000
--- a/docs/docs/observability/trace-with-python-sdk/08-annotate-traces.mdx
+++ /dev/null
@@ -1,522 +0,0 @@
----
-title: "Annotate Traces"
-sidebar_label: "Annotate Traces"
-description: "Learn how to programmatically add annotations to traces using the Agenta API - collect feedback, scores, and custom metrics"
-sidebar_position: 8
----
-
-```mdx-code-block
-import Image from "@theme/IdealImage";
-import Tabs from '@theme/Tabs';
-import TabItem from '@theme/TabItem';
-import GoogleColabButton from "@site/src/components/GoogleColabButton";
-```
-
-Annotations in Agenta let you enrich the traces created by your LLM applications. You can add scores, comments, expected answers and other metrics to help evaluate your application's performance.
-
-
- Open in Google Colaboratory
-
-
-
-
-## What You Can Do With Annotations
-
-- Collect user feedback on LLM responses
-- Run custom evaluation workflows
-- Measure application performance in real-time
-
-## Understanding How Annotations Work
-
-Annotations **link** to specific points (spans) in your application traces. You can **link** them to:
-- Parent spans (like an agent invocation)
-- Child spans (like a single LLM call within a larger process)
-
-Each annotation can include:
-- Numbers (scores, ratings)
-- Categories (labels, classifications)
-- Text (comments, reasoning)
-
-### Annotation Structure
-
-Here's what an annotation looks like:
-
-```json
-{
- "data": {
- "outputs": {
- "score": 80,
- "normalized_score": 0.8,
- "reasoning": "The answer is mainly correct"
- }
- },
- "references": {
- "evaluator": {
- "slug": "my_evaluator"
- }
- },
- "links": {
- "invocation": {
- "trace_id": "47cb4a13b42a47bab2198509543ff14d",
- "span_id": "1a9130f6ad398e55"
- }
- },
- "metadata": {
- "someattr": "somevalue"
- }
-}
-```
-
-An annotation has four main parts:
-
-1. **Data**: The actual evaluation content (scores, comments)
-2. **References**: Which evaluator to use (will be created automatically if it doesn't exist)
-3. **Links**: Which trace and span you're annotating
-4. **Metadata** (optional): Any extra information you want to include
-
-### What You Can Include in Annotations
-
-Annotations are flexible - you can include whatever makes sense for your evaluation:
-
-```json
-// Just a simple score
-{"score": 3}
-
-// A score with explanation
-{"score": 3, "comment": "The response is not grounded"}
-
-// Multiple metrics with reference information
-{"score": 3, "normalized_score": 0.5, "comment": "The response is not grounded", "expected_answer": "The capital of France is Paris"}
-```
-
-You can use numbers, text, booleans, or categorical values.
-
-### Connecting Annotations to Traces
-
-Each annotation links to a specific span in your trace using a `trace_id` and `span_id`. These IDs follow the OpenTelemetry format.
-
-:::warning
-Agenta doesn't check if the trace and span IDs exist. Make sure they're valid before sending.
-:::
-
-### Working with Evaluators
-
-Every annotation must reference an `Evaluator`, which:
-- Defines what your annotation should contain
-- Provides context for interpreting the annotation
-
-If you reference an evaluator that doesn't exist yet, Agenta will automatically create one based on your first annotation.
-
-You can add multiple annotations from the same evaluator to a single span, which is useful for collecting feedback from different sources.
-
-### Using Metadata
-
-The `metadata` field lets you add extra information to your annotation. Use it to store things like user IDs, session information, or other context.
-
-## Creating Annotations
-
-
-
-
-```python
-import requests
-
-base_url = "https://cloud.agenta.ai"
-headers = {
- "Content-Type": "application/json",
- "Authorization": "ApiKey YOUR_API_KEY"
-}
-
-# Annotation data
-annotation_data = {
- "annotation": {
- "data": {
- "outputs": {
- "score": 40,
- "reasoning": "Response contains some inaccuracies",
- "labels": ["Partially Correct", "Needs Improvement"]
- }
- },
- "references": {
- "evaluator": {
- "slug": "content_quality"
- }
- },
- "links": {
- "invocation": {
- "trace_id": "47cb4a13b42a47bab2198509543ff14d",
- "span_id": "1a9130f6ad398e55"
- }
- }
- }
-}
-
-# Make the API request
-response = requests.post(
- f"{base_url}/api/preview/annotations/",
- headers=headers,
- json=annotation_data
-)
-
-# Process the response
-if response.status_code == 200:
- print("Annotation created successfully")
- print(response.json())
-else:
- print(f"Error: {response.status_code}")
- print(response.text)
-```
-
-
-
-
-```javascript
-async function createAnnotation() {
- const baseUrl = 'https://cloud.agenta.ai';
-
- const headers = {
- 'Content-Type': 'application/json',
- 'Authorization': 'ApiKey YOUR_API_KEY'
- };
-
- const annotationData = {
- annotation: {
- data: {
- outputs: {
- score: 40,
- reasoning: "Response contains some inaccuracies",
- labels: ["Partially Correct", "Needs Improvement"]
- }
- },
- references: {
- evaluator: {
- slug: "content_quality"
- }
- },
- links: {
- invocation: {
- trace_id: "47cb4a13b42a47bab2198509543ff14d",
- span_id: "1a9130f6ad398e55"
- }
- }
- }
- };
-
- try {
- const response = await fetch(`${baseUrl}/api/preview/annotations/`, {
- method: 'POST',
- headers: headers,
- body: JSON.stringify(annotationData)
- });
-
- if (response.ok) {
- const data = await response.json();
- console.log("Annotation created successfully");
- console.log(data);
- } else {
- console.error(`Error: ${response.status}`);
- console.error(await response.text());
- }
- } catch (error) {
- console.error("Request failed:", error);
- }
-}
-
-createAnnotation();
-```
-
-
-
-
-:::info
-If the evaluator doesn't exist yet, Agenta will create it automatically based on your first annotation.
-:::
-
-## Understanding the Response
-
-When you create an annotation, you'll receive a response that includes the annotation data and its ID:
-
-```json
-{
- "annotation": {
- "created_at": "2025-05-13T16:54:52.245664Z",
- "created_by_id": "019315dc-a332-7ba5-a426-d079c43ab776",
- "span_id": "a1713ea8f59291f6",
- "trace_id": "bb39070937f74f169b60dfc420729ad3",
- "kind": "custom",
- "source": "api",
- "data": {
- "outputs": {
- "score": 0,
- "reasoning": "The answer is totally false.",
- "expected_answer": "The capital of France is Paris",
- "normalized_score": 0
- }
- },
- "metadata": {
- "user_id": "miamia"
- },
- "references": {
- "evaluator": {
- "id": "0196ca64-9cbf-7ee3-b507-2b559dd13090",
- "slug": "main-scorer"
- }
- },
- "links": {
- "invocation": {
- "span_id": "09383e1124b264d0",
- "trace_id": "5d52a529b48135fd58790f8aa3e33fb8"
- }
- }
- }
-}
-```
-
-:::info
-Annotations themselves are stored as traces with their own `trace_id` and `span_id`, which is different from the invocation they're linked to.
-:::
-
-## Viewing Annotations in the UI
-
-You can see all annotations for a trace in the Agenta UI. Open any trace and check the Annotations tab to see detailed information. The right sidebar shows average metrics for each evaluator.
-
-
-
-## Querying Annotations
-
-
-
-
-You can query annotations in several ways:
-
-**1. Query by annotation ID:**
-
-```python
-import requests
-
-base_url = "https://cloud.agenta.ai"
-headers = {
- "Content-Type": "application/json",
- "Authorization": "ApiKey YOUR_API_KEY"
-}
-
-# Query by annotation ID
-query_data = {
- "annotation": {
- "trace_id": "bb39070937f74f169b60dfc420729ad3",
- "span_id": "a1713ea8f59291f6"
- }
-}
-
-response = requests.post(
- f"{base_url}/api/preview/annotations/query",
- headers=headers,
- json=query_data
-)
-
-if response.status_code == 200:
- print("Query successful")
- print(response.json())
-else:
- print(f"Error: {response.status_code}")
- print(response.text)
-```
-
-**2. Query all annotations for an invocation:**
-
-```python
-# Query all annotations for an invocation
-query_data = {
- "annotation": {
- "links": {
- "invocation": {
- "trace_id": "5d52a529b48135fd58790f8aa3e33fb8",
- "span_id": "09383e1124b264d0"
- }
- }
- }
-}
-
-response = requests.post(
- f"{base_url}/api/preview/annotations/query",
- headers=headers,
- json=query_data
-)
-```
-
-
-
-
-You can query annotations in several ways:
-
-**1. Query by annotation ID:**
-
-```javascript
-async function queryAnnotation() {
- const baseUrl = 'https://cloud.agenta.ai';
-
- const headers = {
- 'Content-Type': 'application/json',
- 'Authorization': 'ApiKey YOUR_API_KEY'
- };
-
- // Query by annotation ID
- const queryData = {
- annotation: {
- trace_id: "bb39070937f74f169b60dfc420729ad3",
- span_id: "a1713ea8f59291f6"
- }
- };
-
- try {
- const response = await fetch(`${baseUrl}/api/preview/annotations/query`, {
- method: 'POST',
- headers: headers,
- body: JSON.stringify(queryData)
- });
-
- if (response.ok) {
- const data = await response.json();
- console.log("Query successful");
- console.log(data);
- } else {
- console.error(`Error: ${response.status}`);
- console.error(await response.text());
- }
- } catch (error) {
- console.error("Request failed:", error);
- }
-}
-
-queryAnnotation();
-```
-
-**2. Query all annotations for an invocation:**
-
-```javascript
-async function queryAnnotationsForInvocation() {
- const baseUrl = 'https://cloud.agenta.ai';
-
- const headers = {
- 'Content-Type': 'application/json',
- 'Authorization': 'ApiKey YOUR_API_KEY'
- };
-
- // Query all annotations for an invocation
- const queryData = {
- annotation: {
- links: {
- invocation: {
- trace_id: "5d52a529b48135fd58790f8aa3e33fb8",
- span_id: "09383e1124b264d0"
- }
- }
- }
- };
-
- try {
- const response = await fetch(`${baseUrl}/api/preview/annotations/query`, {
- method: 'POST',
- headers: headers,
- body: JSON.stringify(queryData)
- });
-
- if (response.ok) {
- const data = await response.json();
- console.log("Query successful");
- console.log(data);
- } else {
- console.error(`Error: ${response.status}`);
- console.error(await response.text());
- }
- } catch (error) {
- console.error("Request failed:", error);
- }
-}
-
-queryAnnotationsForInvocation();
-```
-
-
-
-
-## Removing Annotations
-
-
-
-
-```python
-import requests
-
-base_url = "https://cloud.agenta.ai"
-headers = {
- "Content-Type": "application/json",
- "Authorization": "ApiKey YOUR_API_KEY"
-}
-
-# Define the trace_id and span_id of the annotation to delete
-trace_id = "bb39070937f74f169b60dfc420729ad3"
-span_id = "a1713ea8f59291f6"
-
-# Make the delete request
-response = requests.delete(
- f"{base_url}/api/preview/annotations/{trace_id}/{span_id}",
- headers=headers
-)
-
-# Process the response
-if response.status_code == 200:
- print("Annotation deleted successfully")
-else:
- print(f"Error: {response.status_code}")
- print(response.text)
-```
-
-
-
-
-```javascript
-async function deleteAnnotation() {
- const baseUrl = 'https://cloud.agenta.ai';
-
- const headers = {
- 'Content-Type': 'application/json',
- 'Authorization': 'ApiKey YOUR_API_KEY'
- };
-
- // Define the trace_id and span_id of the annotation to delete
- const traceId = "bb39070937f74f169b60dfc420729ad3";
- const spanId = "a1713ea8f59291f6";
-
- try {
- const response = await fetch(`${baseUrl}/api/preview/annotations/${traceId}/${spanId}`, {
- method: 'DELETE',
- headers: headers
- });
-
- if (response.ok) {
- console.log("Annotation deleted successfully");
- } else {
- console.error(`Error: ${response.status}`);
- console.error(await response.text());
- }
- } catch (error) {
- console.error("Request failed:", error);
- }
-}
-
-deleteAnnotation();
-```
-
-
-
diff --git a/docs/docs/observability/trace-with-python-sdk/_06-sample-traces.mdx b/docs/docs/observability/trace-with-python-sdk/_06-sample-traces.mdx
deleted file mode 100644
index 45982d3e81..0000000000
--- a/docs/docs/observability/trace-with-python-sdk/_06-sample-traces.mdx
+++ /dev/null
@@ -1,23 +0,0 @@
----
-title: "Sample Traces"
-sidebar_label: "Sample Traces"
-description: "Learn how to implement sampling strategies to reduce trace volume in Agenta"
-sidebar_position: 7
----
-
-
-
-## Overview
-
-Sampling allows you to control which traces are sent to Agenta, helping reduce costs and storage when dealing with high-volume applications.
-
-## Why sample?
-
-- Reduce storage costs
-- Minimize performance impact
-- Focus on interesting traces (errors, slow requests)
-
-## Next steps
-
-- Learn about [batching traces](/observability/trace-with-python-sdk/batch-traces)
-- Explore [cost tracking](/observability/trace-with-python-sdk/track-costs)
diff --git a/docs/docs/observability/trace-with-python-sdk/_07-batch-traces.mdx b/docs/docs/observability/trace-with-python-sdk/_07-batch-traces.mdx
deleted file mode 100644
index 395f1835ef..0000000000
--- a/docs/docs/observability/trace-with-python-sdk/_07-batch-traces.mdx
+++ /dev/null
@@ -1,17 +0,0 @@
----
-title: "Batch Traces"
-sidebar_label: "Batch Traces"
-description: "Learn how to batch traces for efficient transmission to Agenta"
-sidebar_position: 8
----
-
-
-
-## Overview
-
-Learn how to batch traces for efficient transmission to Agenta
-
-## Next steps
-
-- Explore other [Python SDK features](/observability/trace-with-python-sdk/setup-tracing)
-- Learn about [using the UI](/observability/using-the-ui/filtering-traces)
diff --git a/docs/docs/observability/trace-with-python-sdk/_09-track-chat-sessions.mdx b/docs/docs/observability/trace-with-python-sdk/_09-track-chat-sessions.mdx
deleted file mode 100644
index 025e2e5df6..0000000000
--- a/docs/docs/observability/trace-with-python-sdk/_09-track-chat-sessions.mdx
+++ /dev/null
@@ -1,17 +0,0 @@
----
-title: "Track Chat Sessions"
-sidebar_label: "Track Chat Sessions"
-description: "Learn how to track multi-turn conversations and chat sessions"
-sidebar_position: 10
----
-
-
-
-## Overview
-
-Learn how to track multi-turn conversations and chat sessions
-
-## Next steps
-
-- Explore other [Python SDK features](/observability/trace-with-python-sdk/setup-tracing)
-- Learn about [using the UI](/observability/using-the-ui/filtering-traces)
diff --git a/docs/docs/observability/trace-with-python-sdk/_10-track-users.mdx b/docs/docs/observability/trace-with-python-sdk/_10-track-users.mdx
deleted file mode 100644
index 6b0d4c476a..0000000000
--- a/docs/docs/observability/trace-with-python-sdk/_10-track-users.mdx
+++ /dev/null
@@ -1,17 +0,0 @@
----
-title: "Track Users"
-sidebar_label: "Track Users"
-description: "Learn how to associate traces with specific users for better analytics"
-sidebar_position: 11
----
-
-
-
-## Overview
-
-Learn how to associate traces with specific users for better analytics
-
-## Next steps
-
-- Explore other [Python SDK features](/observability/trace-with-python-sdk/setup-tracing)
-- Learn about [using the UI](/observability/using-the-ui/filtering-traces)
diff --git a/docs/docs/observability/trace-with-python-sdk/_category_.json b/docs/docs/observability/trace-with-python-sdk/_category_.json
deleted file mode 100644
index 0f24e70204..0000000000
--- a/docs/docs/observability/trace-with-python-sdk/_category_.json
+++ /dev/null
@@ -1,6 +0,0 @@
-{
- "label": "Trace with Python SDK",
- "position": 5,
- "collapsible": true,
- "collapsed": true
-}
diff --git a/docs/docs/prompt-engineering/01-quick-start.mdx b/docs/docs/prompt-engineering/01-quick-start.mdx
index dc1824c076..a5b41a504b 100644
--- a/docs/docs/prompt-engineering/01-quick-start.mdx
+++ b/docs/docs/prompt-engineering/01-quick-start.mdx
@@ -7,7 +7,7 @@ sidebar_position: 1
import Image from "@theme/IdealImage";
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
-import CustomDocCard from '@site/src/components/CustomDocCard';
+import DocCard from '@theme/DocCard';
```
In this tutorial, we'll walk through three simple steps to get started with Agenta:
@@ -15,20 +15,16 @@ In this tutorial, we'll walk through three simple steps to get started with Agen
2. Deploy it to an environment
3. Integrate it with your codebase using the Agenta SDK
-
- ⏯️ Watch a video of the quick start guide.
-
-
+ style={{ display: "block", margin: "0 auto" }}
+>
## 1. Create a prompt
@@ -166,7 +162,7 @@ response = client.chat.completions.create(
)
```
-
+
```javascript
const fetchConfigs = async () => {
@@ -276,7 +272,7 @@ Model names follow LiteLLM naming conventions: `provider/model` (e.g., `cohere/c
:::
:::info
-For simpler observability and cost tracking, Agenta also offers an endpoint to directly call LLMs with your prompt configuration. Learn more in the [proxy LLM calls](/prompt-engineering/integrating-prompts/proxy-calls) section.
+For simpler observability and cost tracking, Agenta also offers an endpoint to directly call LLMs with your prompt configuration. Learn more in the [proxy LLM calls](/prompt-engineering/prompt-management/proxy-calls) section.
:::
## Next Steps
@@ -287,26 +283,24 @@ To continue your journey with Agenta:
-
-
diff --git a/docs/docs/prompt-engineering/02-concepts.mdx b/docs/docs/prompt-engineering/02-concepts.mdx
deleted file mode 100644
index 33f69b2080..0000000000
--- a/docs/docs/prompt-engineering/02-concepts.mdx
+++ /dev/null
@@ -1,75 +0,0 @@
----
-title: "Prompt Management Concepts"
-sidebar_label: "Concepts"
-description: "Learn how to effectively manage, version, and deploy LLM prompts and configurations. Discover how prompt management helps teams collaborate, track changes, and maintain consistency across development and production environments"
-sidebar_position: 2
----
-
-
-
-```mdx-code-block
-import CustomDocCard from '@site/src/components/CustomDocCard';
-import clsx from 'clsx';
-import Image from "@theme/IdealImage";
-
-```
-
-
-## Why do I need a prompt management system?
-
-A prompt management system lets everyone on your team collaborate on prompts. This includes product owners, developers, and subject matter experts. The system organizes your prompts so that:
-- Product teams can change prompts without going through developers each time
-- You can version prompts and roll back to a previous version if needed
-- You can link LLM application spans to prompt versions
-- You can run evaluations on prompts and compare them to each other
-
-You can read more about the benefits of prompt management in our [blog post](https://agenta.ai/blog/the-definitive-guide-to-prompt-management-systems).
-
-
-:::info
-Agenta allows you to version not only prompts, but **any configuration**. For instance, for a RAG pipeline, you can version a configuration with the parameters `chunk_size` or `embedding_model`. You can read more about this in [custom workflows](/custom-workflows/overview).
-:::
-
-## Versioning in Agenta
-
-
-
-Agenta uses a Git-like structure for versioning. Instead of having one commit history for each prompt, you can create multiple branches called **variants**. Each variant has its own version history. You can then deploy specific versions to **environments** (development, staging, production).
-
-## Entities in Agenta
-
-### Applications
-
-Prompts are applications in Agenta. Each application has a different type that determines how it runs. You can create chat applications and completion applications from the UI using templates. You can also create custom applications from your own code using [custom workflows](/custom-workflows/overview).
-
-### Variants
-Variants are similar to branches in **git**. Each variant is an independent branch with its own commit history. When you create a prompt from the UI, we create a `default` variant for you. Unlike git branches, you cannot merge variants together. Use variants to experiment with different approaches or configurations.
-
-### Versions
-
-Versions are similar to commits in **git**. Each version is an immutable snapshot of a variant. When you make changes to a variant, you create a new version. Each version has a **commit id** that uniquely identifies it. Each version contains a **configuration** with prompts and other parameters.
-
-### Environments
-
-Environments are deployment targets for your prompts. Agenta provides three environments: **development**, **staging**, and **production**. Each environment points to a specific version from a variant. When you deploy a new version to an environment, the environment updates to point to that version. Environments track their deployment history, so you can roll back to any previously deployed version.
-
-
-## Best practices for Organizing Prompts
-
-1. Create a new variant for each experiment or approach you explore. Some teams create variants per model. Others create variants per user or per approach. When you try a new approach, create a new variant.
-2. Add commit notes to each version. Explain the changes you made.
-3. Deploy variant versions to staging when ready. Use at least a staging environment and a production environment.
-4. Have your internal team test the staging environment before deploying to production. At minimum, do a vibe check on the prompt.
-
diff --git a/docs/docs/prompt-engineering/02-overview.mdx b/docs/docs/prompt-engineering/02-overview.mdx
new file mode 100644
index 0000000000..6864593dd9
--- /dev/null
+++ b/docs/docs/prompt-engineering/02-overview.mdx
@@ -0,0 +1,90 @@
+---
+title: "Overview"
+description: "Learn how to effectively manage, version, and deploy LLM prompts and configurations. Discover how prompt management helps teams collaborate, track changes, and maintain consistency across development and production environments"
+---
+
+```mdx-code-block
+import DocCard from '@theme/DocCard';
+import clsx from 'clsx';
+import Image from "@theme/IdealImage";
+
+```
+
+
+Building LLM-powered applications is an iterative process. In each iteration, you aim to improve the application's performance by refining prompts, adjusting configurations, and evaluating outputs.
+
+
+
+### Why do I need a prompt management system?
+
+A prompt management system enables everyone on the team—from **product owners** to **subject matter experts**—to collaborate in creating prompts. Additionally it helps you answer the following questions:
+
+- Which prompts have we tried?
+- What were the outputs of these prompts?
+- How do the evaluation results of these prompts compare?
+- Which prompt version was used for a specific generation in production?
+- What was the effect of publishing the new version of this prompt in production?
+- Who on the team made changes to a particular prompt version in production?
+
+### Features in agenta
+
+Agenta provides you with the following capabilities:
+
+- A playground where developers and subject matter experts can collaboratively create and test prompts and compare models
+- A prompt management system where, you can:
+ - **Versioning Prompts**: Keeping track of different prompts you've tested and a history of changes in production.
+ - **Linking Prompts to Experiments**: Connecting each prompt version to its evaluation metrics to understand the effect of changes and determine the best variant.
+ - **Linking Prompts to Traces**: Monitoring how changes in prompt versions affect the traces and production metrics.
+
+:::info
+Agenta goes beyond prompt management to encompass the entire configuration of your LLM applications. If your LLM workflow is more complex than a single prompt (e.g., Retrieval-Augmented Generation (RAG) or a chain of prompts), you can version the **whole configuration** together.
+
+In contrast to a **prompt**, a **configuration** of an LLM application can include additional parameters beyond prompt templates and models (with their parameters). For instance:
+
+- An LLM application using a **chain of two prompts** would have a configuration that includes the two prompts and their respective model parameters.
+- An application that includes a **RAG pipeline** would have a configuration that includes parameters such as `top_k` and `embedding`.
+
+```json title="Example RAG configuration"
+{
+ "top_k": 3,
+ "embedding": "text-embedding-3-large",
+ "prompt-query": "We have provided context information below. {context_str}. Given this information, please answer the question: {query_str}\n",
+ "model-query": "openai/gpt-o1",
+ "temperature-query": "1.0"
+}
+```
+:::
+
+
+### Get started
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/docs/docs/prompt-engineering/_managing-prompts-ui/01-create-and-commit.mdx b/docs/docs/prompt-engineering/_managing-prompts-ui/01-create-and-commit.mdx
deleted file mode 100644
index 58b906273b..0000000000
--- a/docs/docs/prompt-engineering/_managing-prompts-ui/01-create-and-commit.mdx
+++ /dev/null
@@ -1,19 +0,0 @@
----
-title: "Create and Commit Prompts"
-sidebar_position: 1
-sidebar_label: "Create and Commit Prompts"
-description: "Learn how to create variants and commit changes using the Agenta UI."
-draft: true
-
----
-
-
-
-
-This section is under construction. For now, please refer to the [Quick Start guide](/prompt-engineering/quick-start) which covers creating and committing prompts from the UI.
diff --git a/docs/docs/prompt-engineering/_managing-prompts-ui/01-creating-prompts.mdx b/docs/docs/prompt-engineering/_managing-prompts-ui/01-creating-prompts.mdx
deleted file mode 100644
index 6506c050b0..0000000000
--- a/docs/docs/prompt-engineering/_managing-prompts-ui/01-creating-prompts.mdx
+++ /dev/null
@@ -1,17 +0,0 @@
----
-title: "Creating Prompts"
-sidebar_position: 1
-sidebar_label: "Creating Prompts"
-description: "Learn how to create prompts from the Agenta UI."
-draft: true
-
----
-
-import Image from "@theme/IdealImage";
-
-This guide shows you how to create prompts from the Agenta UI.
-## Creating a new application
-
-### Creating a new application from the UI
-
-Go to the App Management page and click on the **Create New Prompt** button, then choose the type of application you want to create (chat for multi-turn chat applications or completion for single-turn applications).
diff --git a/docs/docs/prompt-engineering/_managing-prompts-ui/02-deploy.mdx b/docs/docs/prompt-engineering/_managing-prompts-ui/02-deploy.mdx
deleted file mode 100644
index eda7f3df09..0000000000
--- a/docs/docs/prompt-engineering/_managing-prompts-ui/02-deploy.mdx
+++ /dev/null
@@ -1,20 +0,0 @@
----
-title: "Deploy to Environments"
-sidebar_position: 2
-sidebar_label: "Deploy to Environments"
-description: "Learn how to deploy variants to different environments using the Agenta UI."
-draft: true
-
----
-
-
-
-
-This section is under construction. For now, please refer to the [Quick Start guide](/prompt-engineering/quick-start) which covers deploying variants from the UI.
diff --git a/docs/docs/prompt-engineering/_managing-prompts-ui/03-versions-rollback.mdx b/docs/docs/prompt-engineering/_managing-prompts-ui/03-versions-rollback.mdx
deleted file mode 100644
index 7bb5c5763b..0000000000
--- a/docs/docs/prompt-engineering/_managing-prompts-ui/03-versions-rollback.mdx
+++ /dev/null
@@ -1,19 +0,0 @@
----
-title: "Manage Versions and Rollback"
-sidebar_position: 3
-sidebar_label: "Manage Versions and Rollback"
-description: "Learn how to view version history and rollback to previous versions using the Agenta UI."
-draft: true
----
-
-
-
-
-This section is under construction. For now, please refer to the [Quick Start guide](/prompt-engineering/quick-start) and the [Concepts page](/prompt-engineering/concepts) for information on versioning.
diff --git a/docs/docs/prompt-engineering/_managing-prompts-ui/_category_.json b/docs/docs/prompt-engineering/_managing-prompts-ui/_category_.json
deleted file mode 100644
index c209af07e8..0000000000
--- a/docs/docs/prompt-engineering/_managing-prompts-ui/_category_.json
+++ /dev/null
@@ -1,5 +0,0 @@
-{
- "position": 4,
- "label": "Managing Prompts from UI",
- "collapsed": true
-}
\ No newline at end of file
diff --git a/docs/docs/prompt-engineering/integrating-prompts/01-integrating-with-agenta.mdx b/docs/docs/prompt-engineering/integrating-prompts/01-integrating-with-agenta.mdx
deleted file mode 100644
index 8a5291e6ff..0000000000
--- a/docs/docs/prompt-engineering/integrating-prompts/01-integrating-with-agenta.mdx
+++ /dev/null
@@ -1,55 +0,0 @@
----
-title: "How to Integrate with Agenta"
-description: "Integrate applications and prompts created in Agenta into your projects."
----
-
-import Image from "@theme/IdealImage";
-
-Agenta integrates with your workflow. You can use the latest version of your deployed prompt in your application. With Agenta, **you can update prompts directly from the web interface without modifying your code** each time.
-
-Here are two ways to use prompts from Agenta in your code:
-
-### [1. As a prompt management system](/prompt-engineering/integrating-prompts/fetch-prompt-programatically)
-
-Prompts are managed and stored in the Agenta backend. You use the Agenta SDK to fetch the latest deployed version of your prompt. Then you use it in your application.
-
-**Advantages**:
-
-- Agenta operates outside your application's critical path.
-- You can fetch and cache the latest prompt version for zero latency usage.
-
-**Considerations**:
-
-- You need to set up [observability integration](/observability/quickstart-python) yourself. This is required if you want to trace your calls for debugging and cost tracking.
-
-
-
-
-
-### [2. As a middleware/gateway (invoking prompt)](/prompt-engineering/integrating-prompts/proxy-calls)
-
-You invoke your prompts directly through Agenta. Agenta provides you with an endpoint that forwards requests to the LLM on your behalf.
-
-**Advantages**:
-
-- Simplified deployment.
-- Automatic tracing without any code changes.
-
-**Considerations**:
-
-- Adds slight latency to the response (approximately 0.3 seconds).
-- Streaming is not supported for these endpoints.
-
-This approach works best for applications where latency is not critical.
-
-
diff --git a/docs/docs/prompt-engineering/integrating-prompts/02-fetch-prompt-programatically.mdx b/docs/docs/prompt-engineering/integrating-prompts/02-fetch-prompt-programatically.mdx
deleted file mode 100644
index 22ea6668c6..0000000000
--- a/docs/docs/prompt-engineering/integrating-prompts/02-fetch-prompt-programatically.mdx
+++ /dev/null
@@ -1,330 +0,0 @@
----
-title: "Fetch Prompts via SDK/API"
-sidebar_position: 2
-sidebar_label: "Fetch Prompts via SDK/API"
-description: "Learn how to fetch prompt configurations using the Agenta SDK or API."
----
-
-import GoogleColabButton from "@site/src/components/GoogleColabButton";
-import Tabs from "@theme/Tabs";
-import TabItem from "@theme/TabItem";
-
-This guide shows you how to fetch prompt configurations from variants or environments using the Agenta SDK.
-
-
- Open in Google Colaboratory
-
-
-## Fetching a Prompt Configuration
-
-You can fetch the configurations from a variant reference (`app_slug`, `variant_slug`, `variant_version`) or an environment reference (`app_slug`, `environment_slug`). The default behavior when fetching is to fetch the latest configuration from the `production` environment. If you don't provide a `variant_version` parameter but only a `variant_slug` or an `environment_slug`, the SDK will fetch the latest version of the variant from the specified **environment/variant**.
-
-:::tip
-Check the [reference](/reference/sdk/configuration-management#prompt-configuration-schema) section for more details on the data format used for prompts.
-:::
-
-### Default Behavior when fetching
-
-If you don't provide either `variant` or `environment` identifiers, the SDK fetches the latest configuration deployed to the `production` environment.
-
-
-
-```python
-config = ag.ConfigManager.get_from_registry(
- app_slug="my-app-slug",
- variant_slug="my-variant-slug",
- variant_version=2 # Optional: fetches latest if not provided
-)
-
-print("Fetched configuration from production:")
-print(config)
-```
-Example Output:
-
-```python
-{
- "prompt": {
- "messages": [
- {
- "role": "system",
- "content": "You are an assistant that provides concise answers"
- },
- {
- "role": "user",
- "content": "Explain {{topic}} in simple terms"
- }
- ],
- "llm_config": {
- "model": "gpt-3.5-turbo",
- "top_p": 1.0,
- "max_tokens": 150,
- "temperature": 0.7,
- "presence_penalty": 0.0,
- "frequency_penalty": 0.0
- },
- "template_format": "curly"
- }
-}
-```
-
-
-
-```javascript
-// Fetch configuration from production environment (default behavior)
-const fetchResponse = await fetch('https://cloud.agenta.ai/api/variants/configs/fetch', {
- method: 'POST',
- headers: {
- 'Content-Type': 'application/json',
- 'Authorization': 'Bearer YOUR_API_KEY'
- },
- body: JSON.stringify({
- application_ref: {
- slug: 'my-app-slug',
- version: null,
- id: null
- }
- })
-});
-
-const config = await fetchResponse.json();
-console.log('Fetched configuration from production:');
-console.log(config);
-```
-
-
-
-```bash
-# Fetch configuration from production environment (default behavior)
-curl -X POST "https://cloud.agenta.ai/api/variants/configs/fetch" \
- -H "Content-Type: application/json" \
- -H "Authorization: Bearer YOUR_API_KEY" \
- -d '{
- "application_ref": {
- "slug": "my-app-slug",
- "version": null,
- "id": null
- }
- }'
-```
-
-
-
-:::tip
-Agenta provides a helper class `PromptTemplate` to format the configuration and then use it to generate the prompt.
-```python
-from openai import OpenAI
-from agenta.sdk.types import PromptTemplate
-
-# Fetch configuration
-config = ag.ConfigManager.get_from_registry(
- app_slug="my-app-slug"
-)
-
-# Format the prompt with variables
-prompt = PromptTemplate(**config['prompt']).format(topic="AI")
-
-# Use with OpenAI
-client = OpenAI()
-response = client.chat.completions.create(
- **prompt.to_openai_kwargs()
-)
-
-print(response.choices[0].message.content)
-```
-:::
-
-### Fetching by Variant Reference
-
-
-
-
-```python
-# Fetch configuration by variant
-config = ag.ConfigManager.get_from_registry(
- app_slug="my-app-slug",
- variant_slug="my-variant-slug",
- variant_version=2 # Optional: If not provided, fetches the latest version
-)
-
-print("Fetched configuration:")
-print(config)
-```
-
-
-
-```javascript
-// Fetch configuration by variant
-const fetchResponse = await fetch('https://cloud.agenta.ai/api/variants/configs/fetch', {
- method: 'POST',
- headers: {
- 'Content-Type': 'application/json',
- 'Authorization': 'Bearer YOUR_API_KEY'
- },
- body: JSON.stringify({
- variant_ref: {
- slug: 'my-variant-slug',
- version: 2,
- id: null
- },
- application_ref: {
- slug: 'my-app-slug',
- version: null,
- id: null
- }
- })
-});
-
-const config = await fetchResponse.json();
-console.log('Fetched configuration:');
-console.log(config);
-```
-
-
-
-```bash
-# Fetch configuration by variant
-curl -X POST "https://cloud.agenta.ai/api/variants/configs/fetch" \
- -H "Content-Type: application/json" \
- -H "Authorization: Bearer YOUR_API_KEY" \
- -d '{
- "variant_ref": {
- "slug": "my-variant-slug",
- "version": 2,
- "id": null
- },
- "application_ref": {
- "slug": "my-app-slug",
- "version": null,
- "id": null
- }
- }'
-```
-
-
-
-### Fetching by Environment Reference
-
-
-
-
-```python
-# Fetch the latest configuration from the staging environment
-config = ag.ConfigManager.get_from_registry(
- app_slug="my-app",
- environment_slug="staging",
- environment_version=1 # Optional: If not provided, fetches the latest version
-)
-
-print("Fetched configuration from staging:")
-print(config)
-```
-
-
-
-```javascript
-// Fetch the latest configuration from the staging environment
-const fetchResponse = await fetch('https://cloud.agenta.ai/api/variants/configs/fetch', {
- method: 'POST',
- headers: {
- 'Content-Type': 'application/json',
- 'Authorization': 'Bearer YOUR_API_KEY'
- },
- body: JSON.stringify({
- environment_ref: {
- slug: 'staging',
- version: 1, // Optional: omit to fetch latest
- id: null
- },
- application_ref: {
- slug: 'my-app',
- version: null,
- id: null
- }
- })
-});
-
-const config = await fetchResponse.json();
-console.log('Fetched configuration from staging:');
-console.log(config);
-```
-
-
-
-```bash
-# Fetch the latest configuration from the staging environment
-curl -X POST "https://cloud.agenta.ai/api/variants/configs/fetch" \
- -H "Content-Type: application/json" \
- -H "Authorization: Bearer YOUR_API_KEY" \
- -d '{
- "environment_ref": {
- "slug": "staging",
- "version": 1,
- "id": null
- },
- "application_ref": {
- "slug": "my-app",
- "version": null,
- "id": null
- }
- }'
-```
-
-
-
-## Response Format
-
-The API response contains your prompt configuration under `params`:
-
-```json
-{
- "params": {
- "prompt": {
- "messages": [
- {
- "role": "system",
- "content": "You are an assistant that provides concise answers"
- },
- {
- "role": "user",
- "content": "Explain {{topic}} in simple terms"
- }
- ],
- "llm_config": {
- "model": "gpt-3.5-turbo",
- "max_tokens": 150,
- "temperature": 0.7,
- "top_p": 1.0,
- "frequency_penalty": 0.0,
- "presence_penalty": 0.0
- },
- "template_format": "curly"
- }
- },
- "url": "https://cloud.agenta.ai/services/completion",
- "application_ref": {
- "slug": "my-app-slug",
- "version": null,
- "id": "..."
- },
- "variant_ref": {
- "slug": "my-variant-slug",
- "version": 2,
- "id": "..."
- },
- "environment_ref": {
- "slug": "production",
- "version": 1,
- "id": "..."
- }
-}
-```
-
-:::tip Asynchronous Operations in Python SDK
-All SDK methods have async counterparts with an `a` prefix:
-
-```python
-async def async_operations():
- # Fetch configuration asynchronously
- config = await ag.ConfigManager.aget_from_registry(...)
-```
-:::
diff --git a/docs/docs/prompt-engineering/integrating-prompts/_category_.json b/docs/docs/prompt-engineering/integrating-prompts/_category_.json
deleted file mode 100644
index 78650487dd..0000000000
--- a/docs/docs/prompt-engineering/integrating-prompts/_category_.json
+++ /dev/null
@@ -1,5 +0,0 @@
-{
- "position": 6,
- "label": "Integrate with Agenta",
- "collapsed": true
-}
\ No newline at end of file
diff --git a/docs/docs/prompt-engineering/managing-prompts-programatically/01-setup.mdx b/docs/docs/prompt-engineering/managing-prompts-programatically/01-setup.mdx
deleted file mode 100644
index c8e9108dd8..0000000000
--- a/docs/docs/prompt-engineering/managing-prompts-programatically/01-setup.mdx
+++ /dev/null
@@ -1,62 +0,0 @@
----
-title: "Setup and Installation"
-sidebar_position: 1
-sidebar_label: "Setup and Overview"
-description: "Learn how to install and initialize the Agenta SDK for prompt management."
----
-
-import Image from "@theme/IdealImage";
-import GoogleColabButton from "@site/src/components/GoogleColabButton";
-
-This guide shows you how to set up the Agenta SDK for managing prompts programmatically.
-
-
- Open in Google Colaboratory
-
-
-## Prerequisites
-
-Before starting, familiarize yourself with how versioning works in Agenta. Details are available on the [concepts page](/concepts/concepts).
-
-:::info Versioning in Agenta
-
-{" "}
-
-
-
-Agenta uses a Git-like structure for prompt versioning:
-- Create multiple branches called **variants**
-- Each variant is versioned.
-- Deploy specific versions to **environments** (development, staging, production)
-
-**Typical workflow:**
-1. Create a variant (branch)
-2. Commit changes to the variant creating a new version
-3. Deploy the version to an environment
-
-:::
-
-## Setup the SDK
-
-Initialize the SDK before using any operations:
-
-```python
-import os
-import agenta as ag
-
-# Set your API credentials
-os.environ["AGENTA_API_KEY"] = "your-api-key-here"
-os.environ["AGENTA_HOST"] = "https://cloud.agenta.ai" # only needed if self-hosting
-
-# Initialize the SDK
-ag.init()
-```
-
-:::info
-You can skip this step if you are using the API directly.
-:::
diff --git a/docs/docs/prompt-engineering/managing-prompts-programatically/02-creating-prompts.mdx b/docs/docs/prompt-engineering/managing-prompts-programatically/02-creating-prompts.mdx
deleted file mode 100644
index 0b2c40d39f..0000000000
--- a/docs/docs/prompt-engineering/managing-prompts-programatically/02-creating-prompts.mdx
+++ /dev/null
@@ -1,82 +0,0 @@
----
-title: "Create Prompts"
-sidebar_position: 1
-sidebar_label: "Create Prompts"
-description: "Learn how to create prompts from the Agenta SDK."
----
-
-import Image from "@theme/IdealImage";
-import GoogleColabButton from "@site/src/components/GoogleColabButton";
-import Tabs from "@theme/Tabs";
-import TabItem from "@theme/TabItem";
-
-This guide shows you how to set up the Agenta SDK for managing prompts programmatically.
-
-
- Open in Google Colaboratory
-
-
-
-## Creating a new application
-
-Prompts are applications in Agenta. To create a new prompt, you need to specify the slug, and the type of the application.
-
-The type of the application can be `SERVICE:completion`, `SERVICE:chat`, or `CUSTOM`:
-
-- `SERVICE:completion` is for single-turn prompts.
-- `SERVICE:chat` is for multi-turn prompts.
-- `CUSTOM` is for custom prompts.
-
-
-
-
-```python
-# Creates an empty application
-app = ag.AppManager.create(
- app_slug="my-app-slug",
- template_key="SERVICE:completion", # we define here the app type
- # template_key="SERVICE:chat" # chat prompts
- # template_key="CUSTOM" # custom configuration (schema-less, however unless you provide a URI, you can only use the registry but not the playground)
-)
-```
-
-
-
-
-```javascript
-const response = await fetch('https://cloud.agenta.ai/api/apps', {
- method: 'POST',
- headers: {
- 'Content-Type': 'application/json',
- 'Authorization': 'Bearer YOUR_API_KEY'
- },
- body: JSON.stringify({
- app_name: 'my-app-slug',
- template_key: 'SERVICE:completion'
- })
-});
-
-const result = await response.json();
-console.log(result);
-```
-
-
-
-
-```bash
-curl -X POST "https://cloud.agenta.ai/api/apps" \
- -H "Content-Type: application/json" \
- -H "Authorization: Bearer YOUR_API_KEY" \
- -d '{
- "app_name": "my-app-slug",
- "template_key": "SERVICE:completion"
- }'
-```
-
-
-
-
-
-:::warning
-The app created until now is empty. You cannot use it from the UI yet. You need to create a variant and commit changes to it to be able to use it (next section).
-:::
\ No newline at end of file
diff --git a/docs/docs/prompt-engineering/managing-prompts-programatically/03-create-and-commit.mdx b/docs/docs/prompt-engineering/managing-prompts-programatically/03-create-and-commit.mdx
deleted file mode 100644
index 9678568066..0000000000
--- a/docs/docs/prompt-engineering/managing-prompts-programatically/03-create-and-commit.mdx
+++ /dev/null
@@ -1,978 +0,0 @@
----
-title: "Version Prompts"
-sidebar_position: 2
-sidebar_label: "Version Prompts"
-description: "Learn how to create variants and commit changes using the Agenta SDK."
----
-
-import GoogleColabButton from "@site/src/components/GoogleColabButton";
-import Tabs from "@theme/Tabs";
-import TabItem from "@theme/TabItem";
-
-This guide covers how to create variants and commit changes to them using the Agenta SDK.
-
-
- Open in Google Colaboratory
-
-
-## Creating and Managing Variants
-
-### Create a New Variant
-
-Use `VariantManager.create` to create a new variant with initial configuration:
-
-
-
-```python
-from agenta.sdk.types import PromptTemplate, Message, ModelConfig
-from pydantic import BaseModel
-
-# Define your configuration model
-class Config(BaseModel):
- prompt: PromptTemplate
-
-# Create the initial configuration
-config = Config(
- prompt=PromptTemplate(
- messages=[
- Message(role="system", content="You are an assistant that provides concise answers"),
- Message(role="user", content="Explain {{topic}} in simple terms"),
- ],
- llm_config=ModelConfig(
- model="gpt-3.5-turbo",
- max_tokens=150,
- temperature=0.7,
- top_p=1.0,
- frequency_penalty=0.0,
- presence_penalty=0.0,
- ),
- )
-)
-
-# Create the variant
-variant = ag.VariantManager.create(
- parameters=config.model_dump(),
- app_slug="my-app-slug",
- variant_slug="my-variant-slug",
-)
-```
-
-
-
-
-```javascript
-// Step 1: Add variant configuration (creates empty variant)
-const addConfigResponse = await fetch('https://cloud.agenta.ai/api/variants/configs/add', {
- method: 'POST',
- headers: {
- 'Content-Type': 'application/json',
- 'Authorization': 'Bearer YOUR_API_KEY'
- },
- body: JSON.stringify({
- variant_ref: {
- slug: 'my-variant-slug',
- version: null,
- id: null
- },
- application_ref: {
- slug: 'my-app-slug',
- version: null,
- id: null
- }
- })
-});
-
-const config = await addConfigResponse.json();
-
-// Step 2: Commit parameters to the variant
-const commitResponse = await fetch('https://cloud.agenta.ai/api/variants/configs/commit', {
- method: 'POST',
- headers: {
- 'Content-Type': 'application/json',
- 'Authorization': 'Bearer YOUR_API_KEY'
- },
- body: JSON.stringify({
- config: {
- params: {
- prompt: {
- messages: [
- {
- role: 'system',
- content: 'You are an assistant that provides concise answers'
- },
- {
- role: 'user',
- content: 'Explain {{topic}} in simple terms'
- }
- ],
- llm_config: {
- model: 'gpt-3.5-turbo',
- max_tokens: 150,
- temperature: 0.7,
- top_p: 1.0,
- frequency_penalty: 0.0,
- presence_penalty: 0.0
- }
- }
- },
- variant_ref: {
- slug: 'my-variant-slug',
- version: null,
- id: null
- },
- application_ref: {
- slug: 'my-app-slug',
- version: null,
- id: null
- }
- }
- })
-});
-
-const result = await commitResponse.json();
-console.log(result);
-```
-
-
-
-
-```bash
-# Step 1: Add variant configuration (creates empty variant)
-curl -X POST "https://cloud.agenta.ai/api/variants/configs/add" \
- -H "Content-Type: application/json" \
- -H "Authorization: Bearer YOUR_API_KEY" \
- -d '{
- "variant_ref": {
- "slug": "my-variant-slug",
- "version": null,
- "id": null
- },
- "application_ref": {
- "slug": "my-app-slug",
- "version": null,
- "id": null
- }
- }'
-
-# Step 2: Commit parameters to the variant
-curl -X POST "https://cloud.agenta.ai/api/variants/configs/commit" \
- -H "Content-Type: application/json" \
- -H "Authorization: Bearer YOUR_API_KEY" \
- -d '{
- "config": {
- "params": {
- "prompt": {
- "messages": [
- {
- "role": "system",
- "content": "You are an assistant that provides concise answers"
- },
- {
- "role": "user",
- "content": "Explain {{topic}} in simple terms"
- }
- ],
- "llm_config": {
- "model": "gpt-3.5-turbo",
- "max_tokens": 150,
- "temperature": 0.7,
- "top_p": 1.0,
- "frequency_penalty": 0.0,
- "presence_penalty": 0.0
- }
- }
- },
- "variant_ref": {
- "slug": "my-variant-slug",
- "version": null,
- "id": null
- },
- "application_ref": {
- "slug": "my-app-slug",
- "version": null,
- "id": null
- }
- }
- }'
-```
-
-
-
-
-:::tip
-Use `VariantManager.acreate` for async variant creation.
-:::
-
-This command will create a new variant and initialize it with the first commit containing the parameters provided
-
-- **Parameters:**
- - `app_slug`: The slug of your application.
- - `variant_slug`: The slug of the new variant.
- - `parameters`: A dictionary containing the initial configuration parameters.
-
-**Note:** If a variant with the same `slug` and `version` already exists, the SDK will raise an exception.
-
-**Sample Output:**
-
-```python
-{
- "messages": [
- {
- "role": "system",
- "content": "You are an assistant that provides concise answers"
- },
- {
- "role": "user",
- "content": "Explain {{topic}} in simple terms"
- }
- ],
- "llm_config": {
- "model": "gpt-3.5-turbo",
- "max_tokens": 150,
- "temperature": 0.7,
- "top_p": 1.0,
- "frequency_penalty": 0.0,
- "presence_penalty": 0.0
- },
- "template_format": "curly"
-}
-```
-
-### Committing changes to a variant (Creating a new version in a variant)
-
-To save changes to a variant (creating a new version), use the `VariantManager.commit` method with explicit parameters.
-
-
-
-```python
-
-config2=Config(
- prompt=PromptTemplate(
- messages=[
- Message(role="system", content="You are an assistant that provides VERY concise answers"),
- Message(role="user", content="Explain {{topic}} in simple terms"),
- ],
- llm_config=ModelConfig(
- model="anthropic/claude-3-5-sonnet-20240620",
- max_tokens=150,
- temperature=0.7,
- top_p=1.0,
- frequency_penalty=0.0,
- presence_penalty=0.0,
- ),
- )
-)
-
-
-# Commit the new version
-variant = ag.VariantManager.commit(
- parameters=config2.model_dump(),
- app_slug="my-app-slug",
- variant_slug="my-variant-slug",
-)
-
-print("Committed new version of variant:")
-print(variant)
-
-# Commit the new version asynchronously (optional)
-# async def commit_variant():
-# variant = await ag.VariantManager.acommit(
-# parameters=my_prompt2.model_dump(),
-# app_slug="my-app-slug",
-# variant_slug="my-variant-slug",
-# )
-
-# print("Committed new version of variant (async):")
-# print(variant)
-```
-
-
-
-```javascript
-// Commit new version with updated parameters
-const commitResponse = await fetch('https://cloud.agenta.ai/api/variants/configs/commit', {
- method: 'POST',
- headers: {
- 'Content-Type': 'application/json',
- 'Authorization': 'Bearer YOUR_API_KEY'
- },
- body: JSON.stringify({
- config: {
- params: {
- prompt: {
- messages: [
- {
- role: 'system',
- content: 'You are an assistant that provides VERY concise answers'
- },
- {
- role: 'user',
- content: 'Explain {{topic}} in simple terms'
- }
- ],
- llm_config: {
- model: 'anthropic/claude-3-5-sonnet-20240620',
- max_tokens: 150,
- temperature: 0.7,
- top_p: 1.0,
- frequency_penalty: 0.0,
- presence_penalty: 0.0
- }
- }
- },
- variant_ref: {
- slug: 'my-variant-slug',
- version: null,
- id: null
- },
- application_ref: {
- slug: 'my-app-slug',
- version: null,
- id: null
- }
- }
- })
-});
-
-const result = await commitResponse.json();
-console.log('Committed new version of variant:');
-console.log(result);
-```
-
-
-
-```bash
-# Commit new version with updated parameters
-curl -X POST "https://cloud.agenta.ai/api/variants/configs/commit" \
- -H "Content-Type: application/json" \
- -H "Authorization: Bearer YOUR_API_KEY" \
- -d '{
- "config": {
- "params": {
- "prompt": {
- "messages": [
- {
- "role": "system",
- "content": "You are an assistant that provides VERY concise answers"
- },
- {
- "role": "user",
- "content": "Explain {{topic}} in simple terms"
- }
- ],
- "llm_config": {
- "model": "anthropic/claude-3-5-sonnet-20240620",
- "max_tokens": 150,
- "temperature": 0.7,
- "top_p": 1.0,
- "frequency_penalty": 0.0,
- "presence_penalty": 0.0
- }
- }
- },
- "variant_ref": {
- "slug": "my-variant-slug",
- "version": null,
- "id": null
- },
- "application_ref": {
- "slug": "my-app-slug",
- "version": null,
- "id": null
- }
- }
- }'
-```
-
-
-
-
-:::tip
-Use `VariantManager.acommit` for async version commit.
-:::
-
-:::info Immutability
-Each commit creates a new version of the variant. Versions are immutable once created.
-:::
-
-**Sample Output:**
-
-```python
-{
- "app_id": "01963413-3d39-7650-80ce-3ad5d688da6c",
- "app_slug": "my-app-slug",
- "variant_id": "01968c11-6f7c-7773-b273-922c5807be7b",
- "variant_slug": "my-variant-slug",
- "variant_version": 5,
- "environment_id": null,
- "environment_slug": null,
- "environment_version": null,
- "committed_at": "2025-05-01T07:26:08.935406+00:00",
- "committed_by": "user@agenta.ai",
- "committed_by_id": "0196247a-ec9d-7051-8880-d58279570aa1",
- "deployed_at": null,
- "deployed_by": null,
- "deployed_by_id": null,
- "params": {
- "messages": [
- {
- "name": null,
- "role": "system",
- "content": "You are an assistant that provides concise answers",
- "tool_calls": null,
- "tool_call_id": null
- },
- {
- "name": null,
- "role": "user",
- "content": "Explain {{topic}} in simple terms",
- "tool_calls": null,
- "tool_call_id": null
- }
- ],
- "input_keys": null,
- "llm_config": {
- "model": "gpt-3.5-turbo",
- "tools": null,
- "top_p": 1.0,
- "stream": null,
- "max_tokens": 150,
- "temperature": 0.7,
- "tool_choice": null,
- "response_format": null,
- "presence_penalty": 0.0,
- "frequency_penalty": 0.0
- },
- "user_prompt": null,
- "system_prompt": null,
- "template_format": "curly"
- }
-}
-```
-
-:::warning
-The `user_prompt` and `system_prompt` are helper fields to populate the `messages` field in the `PromptTemplate`. Rely only on the `messages` field in your code.
-:::
-
-## Deploying to Environments
-
-To deploy a variant to an environment, use the `DeploymentManager.deploy` method with the variant reference and `environment_slug`: The slug of the environment (`development`, `staging`, or `production`).
-
-```python
-deployment = ag.DeploymentManager.deploy(
- app_slug="my-app-slug",
- variant_slug="my-variant-slug",
- variant_version=None, # Deploys latest version if not specified
- environment_slug="staging" # Options: development, staging, production
-)
-
-print(f"Deployed to {deployment['environment_slug']}")
-```
-
-:::warning
-- Deploying a variant without specifying a `variant_version` deploys the latest version.
-- Only predefined environments with slugs `development`, `staging`, and `production` are currently supported.
-:::
-
-**Sample Output:**
-
-```python
-Deployed variant to environment:
-{
- "app_id": "01963413-3d39-7650-80ce-3ad5d688da6c",
- "app_slug": "completion",
- "variant_id": "01968c11-6f7c-7773-b273-922c5807be7b",
- "variant_slug": "my-variant-slug4",
- "variant_version": 5,
- "environment_id": "01968c14-c35d-7440-bcc8-9def594f017f",
- "environment_slug": "staging",
- "environment_version": 2,
- "committed_at": "2025-05-01T07:26:08.935406+00:00",
- "committed_by": "user@agenta.ai",
- "committed_by_id": "0196247a-ec9d-7051-8880-d58279570aa1",
- "deployed_at": "2025-05-01T13:41:33.149595+00:00",
- "deployed_by": "user@agenta.ai",
- "deployed_by_id": "0196247a-ec9d-7051-8880-d58279570aa1"
-}
-```
-
-## Fetching a Prompt Configuration
-
-You can fetch the configurations from a variant reference (`app_slug`, `variant_slug`, `variant_version`) or an environment reference (`app_slug`, `environment_slug`). The default behavior when fetching is to fetch the latest configuration from the `production` environment. If you don't provide a `_version` parameter but only a `variant_slug` or an `environment_slug`, the SDK will fetch the latest version of the variant from the specified **environment/variant**.
-
-### Understanding the PromptTemplate
-
-When you fetch a prompt configuration, you receive a `PromptTemplate` object. This object contains all the information needed to format and send your prompt to an LLM provider.
-
-The `PromptTemplate` includes:
-
-- **messages**: An ordered list of chat messages with roles and content
-- **llm_config**: Model parameters like temperature, max tokens, and the model name
-- **template_format**: The syntax used for variable placeholders in your prompt
-- **input_keys**: An optional list of required variables for input validation
-
-Here's what a fetched configuration looks like:
-
-```python
-config = ag.ConfigManager.get_from_registry(
- app_slug="my-app-slug"
-)
-
-print(config)
-# Output:
-# {
-# "prompt": {
-# "messages": [
-# {"role": "system", "content": "You are a helpful assistant"},
-# {"role": "user", "content": "Explain {{topic}} in simple terms"}
-# ],
-# "llm_config": {
-# "model": "gpt-3.5-turbo",
-# "temperature": 0.7,
-# "max_tokens": 150
-# },
-# "template_format": "curly"
-# }
-# }
-```
-
-#### Template Format
-
-The `template_format` field determines how Agenta replaces variable placeholders in your prompts. Agenta supports three formats:
-
-**Curly format (default)**: Uses double curly braces like `{{variable}}`. This format is simple and safe. It only replaces variables that were defined in your original template, which helps catch missing inputs early.
-
-```python
-"Explain {{topic}} in {{language}}"
-```
-
-**Jinja2 format**: Uses Jinja2 syntax like `{{ variable }}` and supports advanced features like loops and filters. Choose this format when you need conditional logic or want to transform variables.
-
-```python
-"{% if variable %}...{% endif %}"
-```
-
-:::tip
-Check the [reference](/reference/sdk/configuration-management#prompt-configuration-schema) section for more details on the data format used for prompts.
-:::
-
-### Default Behavior when fetching
-
-If you don't provide either `variant` or `environment` identifiers, the SDK fetches the latest configuration deployed to the `production` environment.
-
-```python
-config = ag.ConfigManager.get_from_registry(
- app_slug="my-app-slug",
- variant_slug="my-variant-slug",
- variant_version=2 # Optional: fetches latest if not provided
-)
-
-print("Fetched configuration from production:")
-print(config)
-```
-Example Output:
-
-```python
-{
- "prompt": {
- "messages": [
- {
- "role": "system",
- "content": "You are an assistant that provides concise answers"
- },
- {
- "role": "user",
- "content": "Explain {{topic}} in simple terms"
- }
- ],
- "llm_config": {
- "model": "gpt-3.5-turbo",
- "top_p": 1.0,
- "max_tokens": 150,
- "temperature": 0.7,
- "presence_penalty": 0.0,
- "frequency_penalty": 0.0
- },
- "template_format": "curly"
- }
-}
-```
-
-:::tip
-Agenta provides a helper class `PromptTemplate` to format the configuration and then use it to generate the prompt.
-```python
-from openai import OpenAI
-from agenta.sdk.types import PromptTemplate
-
-# Fetch configuration
-config = ag.ConfigManager.get_from_registry(
- app_slug="my-app-slug"
-)
-
-# Format the prompt with variables
-prompt = PromptTemplate(**config['prompt']).format(topic="AI")
-
-# Use with OpenAI
-client = OpenAI()
-response = client.chat.completions.create(
- **prompt.to_openai_kwargs()
-)
-
-print(response.choices[0].message.content)
-```
-:::
-
-### Fetching by Variant Reference
-
-```python
-# Fetch configuration by variant
-config = ag.ConfigManager.get_from_registry(
- app_slug="my-app-slug",
- variant_slug="my-variant-slug",
- variant_version=2 # Optional: If not provided, fetches the latest version
-)
-
-print("Fetched configuration:")
-print(config)
-```
-
-### Fetching by Environment Reference
-
-```python
-# Fetch the latest configuration from the staging environment
-config = ag.ConfigManager.get_from_registry(
- app_slug="my-app",
- environment_slug="staging",
- environment_version=1 # Optional: If not provided, fetches the latest version
-)
-
-print("Fetched configuration from staging:")
-print(config)
-```
-
-## Deleting a Variant
-
-To delete a variant, use the `VariantManager.delete` method.
-
-
-
-
-```python
-# Delete a variant
-ag.VariantManager.delete(
- app_slug="my-app",
- # app_id="my-app-id", # you can also use `app_id`
- variant_slug="obsolete-variant"
-)
-
-print("Variant deleted successfully.")
-```
-
-
-
-
-```javascript
-// Delete a variant
-const deleteResponse = await fetch('https://cloud.agenta.ai/api/variants/configs/delete', {
- method: 'POST',
- headers: {
- 'Content-Type': 'application/json',
- 'Authorization': 'Bearer YOUR_API_KEY'
- },
- body: JSON.stringify({
- variant_ref: {
- slug: 'obsolete-variant',
- version: null,
- id: null
- },
- application_ref: {
- slug: 'my-app',
- version: null,
- id: null
- }
- })
-});
-
-if (deleteResponse.status === 204) {
- console.log('Variant deleted successfully.');
-} else {
- console.error('Failed to delete variant:', deleteResponse.status);
-}
-```
-
-
-
-
-```bash
-# Delete a variant
-curl -X POST "https://cloud.agenta.ai/api/variants/configs/delete" \
- -H "Content-Type: application/json" \
- -H "Authorization: Bearer YOUR_API_KEY" \
- -d '{
- "variant_ref": {
- "slug": "obsolete-variant",
- "version": null,
- "id": null
- },
- "application_ref": {
- "slug": "my-app",
- "version": null,
- "id": null
- }
- }'
-```
-
-
-
-
-:::warning
-
-- Deleting a variant removes all versions of the variant. This action is irreversible.
-- Attempting to delete a variant that is deployed to an environment will fail.
-:::
-
-## Listing All Variants
-
-To list all variants of an application, use the `VariantManager.list` method.
-
-
-
-
-```python
-# List all variants (syncrhonously)
-variants = ag.VariantManager.list(
- app_slug="my-app"
- # app_id="my-app-id", # you can also use `app_id`
-)
-
-print(variants)
-```
-
-
-
-
-```javascript
-// List all variants for an application
-const listResponse = await fetch('https://cloud.agenta.ai/api/variants/configs/list', {
- method: 'POST',
- headers: {
- 'Content-Type': 'application/json',
- 'Authorization': 'Bearer YOUR_API_KEY'
- },
- body: JSON.stringify({
- application_ref: {
- slug: 'my-app',
- version: null,
- id: null
- }
- })
-});
-
-const variants = await listResponse.json();
-console.log(variants);
-```
-
-
-
-
-```bash
-# List all variants for an application
-curl -X POST "https://cloud.agenta.ai/api/variants/configs/list" \
- -H "Content-Type: application/json" \
- -H "Authorization: Bearer YOUR_API_KEY" \
- -d '{
- "application_ref": {
- "slug": "my-app",
- "version": null,
- "id": null
- }
- }'
-```
-
-
-
-
-**Sample Output:**
-
-```python
-
-[{
- "app_id": "01963413-3d39-7650-80ce-3ad5d688da6c",
- "app_slug": "completion",
- "variant_id": "01968aba-9702-7263-a7fd-8b8565753230",
- "variant_slug": "my-variant-slug3",
- "variant_version": 2,
- "environment_id": null,
- "environment_slug": null,
- "environment_version": null,
- "committed_at": "2025-05-01T07:23:26.338901+00:00",
- "committed_by": "user@agenta.ai",
- "committed_by_id": "0196247a-ec9d-7051-8880-d58279570aa1",
- "deployed_at": null,
- "deployed_by": null,
- "deployed_by_id": null,
- "params": {
- "prompt": {
- "messages": [
- {
- "name": null,
- "role": "system",
- "content": "You are an assistant that provides concise answers",
- "tool_calls": null,
- "tool_call_id": null
- },
- {
- "name": null,
- "role": "user",
- "content": "Explain {{topic}} in simple terms",
- "tool_calls": null,
- "tool_call_id": null
- }
- ],
- "input_keys": null,
- "llm_config": {
- "model": "gpt-3.5-turbo",
- "tools": null,
- "top_p": 1.0,
- "stream": null,
- "max_tokens": 150,
- "temperature": 0.7,
- "tool_choice": null,
- "response_format": null,
- "presence_penalty": 0.0,
- "frequency_penalty": 0.0
- },
- "user_prompt": null,
- "system_prompt": null,
- "template_format": "curly"
- }
- }
-}, {
- "app_id": "01963413-3d39-7650-80ce-3ad5d688da6c",
- "app_slug": "completion",
- "variant_id": "01963f1f-28ea-78e1-b390-39d270fe93fe",
- "variant_slug": "key",
- "variant_version": 4,
- "environment_id": null,
- "environment_slug": null,
- "environment_version": null,
- "committed_at": "2025-04-16T15:02:08.874979+00:00",
- "committed_by": "user@agenta.ai",
- "committed_by_id": "0196247a-ec9d-7051-8880-d58279570aa1",
- "deployed_at": null,
- "deployed_by": null,
- "deployed_by_id": null,
- "params": {
- "ag_config": {
- "prompt": {
- "messages": [
- {
- "role": "system",
- "content": "You are an expert in geography"
- },
- {
- "role": "user",
- "content": "What is the capital of {{country}}? tell me its name in arabic"
- }
- ],
- "input_keys": [
- "country"
- ],
- "llm_config": {
- "model": "gpt-3.5-turbo"
- },
- "template_format": "curly"
- }
- }
- }
-}]
-```
-
-## Fetching a Variant's History
-
-To list all versions for a variant of an application, use the `VariantManager.history` method.
-
-
-
-
-```python
-# List all variant versions/history (synchronously)
-versions = ag.VariantManager.history(
- variant_slug="variant-slug",
- app_slug="my-app"
- # app_id="my-app-id", # you can also use `app_id`
-)
-
-print(versions)
-```
-
-
-
-
-```javascript
-// List all versions/history for a specific variant
-const historyResponse = await fetch('https://cloud.agenta.ai/api/variants/configs/history', {
- method: 'POST',
- headers: {
- 'Content-Type': 'application/json',
- 'Authorization': 'Bearer YOUR_API_KEY'
- },
- body: JSON.stringify({
- variant_ref: {
- slug: 'variant-slug',
- version: null,
- id: null
- },
- application_ref: {
- slug: 'my-app',
- version: null,
- id: null
- }
- })
-});
-
-const versions = await historyResponse.json();
-console.log(versions);
-```
-
-
-
-
-```bash
-# List all versions/history for a specific variant
-curl -X POST "https://cloud.agenta.ai/api/variants/configs/history" \
- -H "Content-Type: application/json" \
- -H "Authorization: Bearer YOUR_API_KEY" \
- -d '{
- "variant_ref": {
- "slug": "variant-slug",
- "version": null,
- "id": null
- },
- "application_ref": {
- "slug": "my-app",
- "version": null,
- "id": null
- }
- }'
-```
-
-
-
-
-**Sample Output:**
-
-Same as `VariantManager.list` but limited to the history of a specific variant.
-
-:::tip Asynchronous Operations in Python SDK
-All SDK methods have async counterparts with an `a` prefix:
-
-```python
-async def async_operations():
- # Create variant asynchronously
- variant = await ag.VariantManager.acreate(...)
-
- # Commit changes asynchronously
- updated_variant = await ag.VariantManager.acommit(...)
-```
-:::
diff --git a/docs/docs/prompt-engineering/managing-prompts-programatically/04-deploy.mdx b/docs/docs/prompt-engineering/managing-prompts-programatically/04-deploy.mdx
deleted file mode 100644
index 72b528f597..0000000000
--- a/docs/docs/prompt-engineering/managing-prompts-programatically/04-deploy.mdx
+++ /dev/null
@@ -1,211 +0,0 @@
----
-title: "Deploy to Environments"
-sidebar_position: 4
-sidebar_label: "Deploy to Environments"
-description: "Learn how to deploy variants to different environments using the Agenta SDK."
----
-
-import GoogleColabButton from "@site/src/components/GoogleColabButton";
-import Tabs from "@theme/Tabs";
-import TabItem from "@theme/TabItem";
-
-This guide shows you how to deploy variants to environments (development, staging, production) using the Agenta SDK.
-
-
- Open in Google Colaboratory
-
-
-## Deploying to Environments
-
-To deploy a variant to an environment, use the `DeploymentManager.deploy` method with the variant reference and `environment_slug`: The slug of the environment (`development`, `staging`, or `production`).
-
-
-
-
-```python
-deployment = ag.DeploymentManager.deploy(
- app_slug="my-app-slug",
- variant_slug="my-variant-slug",
- variant_version=None, # Deploys latest version if not specified
- environment_slug="staging" # Options: development, staging, production
-)
-
-print(f"Deployed to {deployment['environment_slug']}")
-```
-
-
-
-
-```javascript
-// Deploy a variant to an environment
-const deployResponse = await fetch('https://cloud.agenta.ai/api/variants/configs/deploy', {
- method: 'POST',
- headers: {
- 'Content-Type': 'application/json',
- 'Authorization': 'Bearer YOUR_API_KEY'
- },
- body: JSON.stringify({
- variant_ref: {
- slug: 'my-variant-slug',
- version: null,
- id: null
- },
- environment_ref: {
- slug: 'staging',
- version: null,
- id: null
- },
- application_ref: {
- slug: 'my-app-slug',
- version: null,
- id: null
- }
- })
-});
-
-const deployment = await deployResponse.json();
-console.log(`Deployed to ${deployment.environment_slug}`);
-```
-
-
-
-```bash
-# Deploy a variant to an environment
-curl -X POST "https://cloud.agenta.ai/api/variants/configs/deploy" \
- -H "Content-Type: application/json" \
- -H "Authorization: Bearer YOUR_API_KEY" \
- -d '{
- "variant_ref": {
- "slug": "my-variant-slug",
- "version": null,
- "id": null
- },
- "environment_ref": {
- "slug": "staging",
- "version": null,
- "id": null
- },
- "application_ref": {
- "slug": "my-app-slug",
- "version": null,
- "id": null
- }
- }'
-```
-
-
-
-:::warning
-- Deploying a variant without specifying a `variant_version` deploys the latest version.
-- Only predefined environments with slugs `development`, `staging`, and `production` are currently supported.
-:::
-
-**Sample Output:**
-
-```python
-Deployed variant to environment:
-{
- "app_id": "01963413-3d39-7650-80ce-3ad5d688da6c",
- "app_slug": "completion",
- "variant_id": "01968c11-6f7c-7773-b273-922c5807be7b",
- "variant_slug": "my-variant-slug4",
- "variant_version": 5,
- "environment_id": "01968c14-c35d-7440-bcc8-9def594f017f",
- "environment_slug": "staging",
- "environment_version": 2,
- "committed_at": "2025-05-01T07:26:08.935406+00:00",
- "committed_by": "user@agenta.ai",
- "committed_by_id": "0196247a-ec9d-7051-8880-d58279570aa1",
- "deployed_at": "2025-05-01T13:41:33.149595+00:00",
- "deployed_by": "user@agenta.ai",
- "deployed_by_id": "0196247a-ec9d-7051-8880-d58279570aa1"
-}
-```
-
-## Rolling Back to a Previous Version
-
-To rollback to a previous version, you can deploy a specific version of a variant to an environment:
-
-
-
-
-```python
-# Deploy a specific version (rollback)
-deployment = ag.DeploymentManager.deploy(
- app_slug="my-app-slug",
- variant_slug="my-variant-slug",
- variant_version=3, # Specify the version you want to rollback to
- environment_slug="production"
-)
-
-print(f"Rolled back to version {deployment['variant_version']}")
-```
-
-
-
-
-```javascript
-// Deploy a specific version (rollback)
-const deployResponse = await fetch('https://cloud.agenta.ai/api/variants/configs/deploy', {
- method: 'POST',
- headers: {
- 'Content-Type': 'application/json',
- 'Authorization': 'Bearer YOUR_API_KEY'
- },
- body: JSON.stringify({
- variant_ref: {
- slug: 'my-variant-slug',
- version: 3,
- id: null
- },
- environment_ref: {
- slug: 'production',
- version: null,
- id: null
- },
- application_ref: {
- slug: 'my-app-slug',
- version: null,
- id: null
- }
- })
-});
-
-const deployment = await deployResponse.json();
-console.log(`Rolled back to version ${deployment.variant_version}`);
-```
-
-
-
-```bash
-# Deploy a specific version (rollback)
-curl -X POST "https://cloud.agenta.ai/api/variants/configs/deploy" \
- -H "Content-Type: application/json" \
- -H "Authorization: Bearer YOUR_API_KEY" \
- -d '{
- "variant_ref": {
- "slug": "my-variant-slug",
- "version": 3,
- "id": null
- },
- "environment_ref": {
- "slug": "production",
- "version": null,
- "id": null
- },
- "application_ref": {
- "slug": "my-app-slug",
- "version": null,
- "id": null
- }
- }'
-```
-
-
-
-This effectively rolls back the environment to the specified version of the variant.
-
-:::tip
-You can view the full history of deployments and versions in the Agenta UI under the Registry and Deployments pages.
-:::
-
diff --git a/docs/docs/prompt-engineering/managing-prompts-programatically/05-fetch-prompts.mdx b/docs/docs/prompt-engineering/managing-prompts-programatically/05-fetch-prompts.mdx
deleted file mode 100644
index 0e41a8ffef..0000000000
--- a/docs/docs/prompt-engineering/managing-prompts-programatically/05-fetch-prompts.mdx
+++ /dev/null
@@ -1,10 +0,0 @@
----
-title: "Fetch Prompts from Environments"
-sidebar_position: 5
-sidebar_label: "Fetch Prompts"
-description: "Learn how to fetch prompt configurations from environments using the Agenta SDK and API."
----
-
-import { Redirect } from '@docusaurus/router';
-
-
diff --git a/docs/docs/prompt-engineering/managing-prompts-programatically/_category_.json b/docs/docs/prompt-engineering/managing-prompts-programatically/_category_.json
deleted file mode 100644
index 3ec07828bb..0000000000
--- a/docs/docs/prompt-engineering/managing-prompts-programatically/_category_.json
+++ /dev/null
@@ -1,5 +0,0 @@
-{
- "position": 3,
- "label": "Manage Prompts with SDK/API",
- "collapsed": true
-}
\ No newline at end of file
diff --git a/docs/docs/prompt-engineering/playground/01-using-playground.mdx b/docs/docs/prompt-engineering/playground/01-using-playground.mdx
deleted file mode 100644
index 5f7a030a81..0000000000
--- a/docs/docs/prompt-engineering/playground/01-using-playground.mdx
+++ /dev/null
@@ -1,156 +0,0 @@
----
-title: "Prompt Playground"
-sidebar_label: "Using the Prompt Playground"
-description: "Learn how to use Agenta's LLM playground for prompt engineering, model comparison, and deployment. A powerful alternative to OpenAI playground that supports multiple models and frameworks."
----
-
-```mdx-code-block
-import { Stream } from '@cloudflare/stream-react';
-import Image from "@theme/IdealImage";
-```
-
-The Agenta playground is a comprehensive prompt engineering IDE that helps you develop, test, and deploy LLM applications. Unlike traditional LLM playgrounds, Agenta integrates prompt management, evaluation, and deployment into a single platform, supporting multiple models and frameworks.
-
-
-
-
-
-## Key Features
-
-- **Multi-Model Support**: Test prompts across different providers including OpenAI GPT models, Anthropic Claude, Google Gemini, Mistral, DeepSeek, and OpenRouter models
-- **Framework Integration**: Works seamlessly with popular frameworks like LangChain, LlamaIndex, and CrewAI
-- **Version Control**: Track and manage prompt versions with built-in prompt management
-- **Model Comparison**: Benchmark different LLM models side by side
-- **Parameter Control**: Fine-tune model parameters including temperature, top-k, and presence penalty
-- **Environment Management**: Deploy configurations to development, staging, and production environments
-
-## Creating and Managing Prompts
-
-### Prompt Templates
-
-Prompt templates form the foundation of reusable LLM interactions. They contain variable placeholders that adapt to different inputs, making your prompts dynamic and versatile.
-
-Example of a prompt template:
-```
-Write a blog post about {{subject}} that focuses on {{key_aspect}}
-```
-
-Agenta uses curly brackets `{{variable}}` for template variables, automatically creating corresponding input fields in the playground interface.
-
-### Switching Template Formats
-
-Agenta lets you control how it processes variables in prompts through different template formats. The playground supports two main formats:
-
-The default Curly format uses simple `{{variable}}` placeholders. The Jinja2 format provides more control with conditional statements like `{% if variable %}...{% endif %}` and filters.
-
-:::warning
-The playground does not support loops in Jinja2 prompts because prompt variables must be strings.
-:::
-
-:::tip
-To use a variable only for filtering in Jinja2 without including it in the prompt text, wrap it in a false condition: `{% if False %} {{variable}} {% endif %}`. This makes the variable available to the playground while keeping it out of the final prompt.
-:::
-
-The format selector in the prompt configuration panel makes it easy to switch between formats. Click the current format name and select your preferred option from the dropdown menu.
-
-Agenta stores your chosen format with the prompt variant's messages and model settings. When you use the SDK to retrieve and format the prompt, it automatically applies the correct format.
-
-### Dynamic JSON Schema Variables
-
-When using the **JSON Schema** response format, you can reference variables directly inside the schema definition. Placeholders using `{{variable}}` work in both keys and values, and any variables you add will appear as inputs in the playground. These dynamic schemas are stored in the registry when you commit a variant. The editor does not yet highlight variables inside the schema, but they will still be replaced when executing your prompt.
-
-### Adjusting Model Parameters
-
-To modify model-specific parameters:
-
-1. Click on the model name in the configuration panel
-2. A settings modal will appear with available parameters
-3. Adjust the following settings as needed:
-
-- **Temperature** (0-2): Controls randomness in the output
- - Lower values (0.1-0.4): More focused, deterministic responses
- - Higher values (0.7-1.0): More creative, varied outputs
-
-- **Top-k** (1-100): Limits the pool of tokens the model considers
- - Lower values: More focused word choice
- - Higher values: Broader vocabulary usage
-
-- **Presence Penalty** (-2.0 to 2.0): Influences topic diversity
- - Positive values: Encourages exploring new topics
- - Negative values: Allows topic repetition
-
-- **Frequency Penalty** (-2.0 to 2.0): Affects word choice
- - Positive values: Discourages word repetition
- - Negative values: Allows word reuse
-
-## Comparing Prompts and Models
-
-Agenta's playground allows you to compare different prompts, model configurations, and LLM providers side by side. This helps you identify the most effective combination for your use case.
-
-### Creating Multiple Variants
-
-To compare different configurations:
-
-1. Click the "+Compare" button in the top right corner
-2. Select "New Variant" from the dropdown menu
-3. Choose a source variant to duplicate (this copies all current parameters)
-4. Give your new variant a descriptive name
-5. Modify the configuration as needed
-
-Each variant appears as a separate column in the playground, allowing direct comparison of outputs.
-
-### Side-by-Side Testing
-
-With multiple variants set up, you can:
-- Enter the same input text across all variants
-- Compare the generated outputs side by side
-- Review response times for each configuration
-- Monitor the cost per request for different models
-
-This parallel testing helps you understand how different prompts and models handle the same input.
-
-### Version Control and Deployment
-
-The playground integrates with Agenta's [prompt management system](/prompt-engineering/concepts) to track and deploy changes:
-
-1. Click "Commit" to save a new version of your variant
-2. Changes remain in development until explicitly deployed
-3. Use "Deploy" to push configurations to specific environments (development, staging, production)
-
-## Working with Test Sets
-
-### Loading Existing Test Sets
-
-You can import test sets that you've created through:
-
-- CSV uploads
-- Programmatic creation
-- Web UI input
-
-The test set must contain columns that match your prompt template variables.
-
-### Saving Test Cases
-
-To save an interesting test case:
-
-1. Click the three-dot menu on the right side of the test case card
-2. Select "Add to Test Set"
-3. Choose an existing test set or create a new one
-4. Review the input/output mapping (use default settings unless you have specific needs)
-5. Preview the changes to your test set
-6. Confirm to save
-
-The Agenta playground serves as a comprehensive prompt engineering environment, offering advantages over traditional playgrounds like OpenAI's by providing integrated version control, deployment capabilities, and support for multiple models and frameworks. Whether you're developing new prompts, comparing model performance, or deploying to production, the playground provides the tools needed for effective LLM application development.
-
-## FAQ
-
-
- Can I load multiple test cases in the playground at once?
-
-This depends on your prompt type:
-
-* **Completion prompts**: You can load and run multiple test cases at once in the playground.
-* **Chat prompts**: You can only load one conversation at a time in the playground.
-
-Note: You can always run evaluations against your entire test set at once, regardless of prompt type. This limitation only applies to the playground.
-
diff --git a/docs/docs/prompt-engineering/playground/01-using-the-playground.mdx b/docs/docs/prompt-engineering/playground/01-using-the-playground.mdx
new file mode 100644
index 0000000000..909641dc2a
--- /dev/null
+++ b/docs/docs/prompt-engineering/playground/01-using-the-playground.mdx
@@ -0,0 +1,125 @@
+---
+title: "Prompt Playground"
+sidebar_label: "Using the Prompt Playground"
+description: "Learn how to use Agenta's LLM playground for prompt engineering, model comparison, and deployment. A powerful alternative to OpenAI playground that supports multiple models and frameworks."
+---
+
+```mdx-code-block
+import { Stream } from '@cloudflare/stream-react';
+import Image from "@theme/IdealImage";
+```
+
+The Agenta playground is a comprehensive prompt engineering IDE that helps you develop, test, and deploy LLM applications. Unlike traditional LLM playgrounds, Agenta integrates prompt management, evaluation, and deployment into a single platform, supporting multiple models and frameworks.
+
+
+
+
+
+## Key Features
+
+- **Multi-Model Support**: Test prompts across different providers including OpenAI GPT models, Anthropic Claude, Google Gemini, Mistral, DeepSeek, and OpenRouter models
+- **Framework Integration**: Works seamlessly with popular frameworks like LangChain, LlamaIndex, and CrewAI
+- **Version Control**: Track and manage prompt versions with built-in prompt management
+- **Model Comparison**: Benchmark different LLM models side by side
+- **Parameter Control**: Fine-tune model parameters including temperature, top-k, and presence penalty
+- **Environment Management**: Deploy configurations to development, staging, and production environments
+
+## Creating and Managing Prompts
+
+### Prompt Templates
+
+Prompt templates form the foundation of reusable LLM interactions. They contain variable placeholders that adapt to different inputs, making your prompts dynamic and versatile.
+
+Example of a prompt template:
+```
+Write a blog post about {{subject}} that focuses on {{key_aspect}}
+```
+
+Agenta uses curly brackets `{{variable}}` for template variables, automatically creating corresponding input fields in the playground interface.
+
+### Dynamic JSON Schema Variables
+
+When using the **JSON Schema** response format, you can reference variables directly inside the schema definition. Placeholders using `{{variable}}` work in both keys and values, and any variables you add will appear as inputs in the playground. These dynamic schemas are stored in the registry when you commit a variant. The editor does not yet highlight variables inside the schema, but they will still be replaced when executing your prompt.
+
+### Adjusting Model Parameters
+
+To modify model-specific parameters:
+
+1. Click on the model name in the configuration panel
+2. A settings modal will appear with available parameters
+3. Adjust the following settings as needed:
+
+- **Temperature** (0-2): Controls randomness in the output
+ - Lower values (0.1-0.4): More focused, deterministic responses
+ - Higher values (0.7-1.0): More creative, varied outputs
+
+- **Top-k** (1-100): Limits the pool of tokens the model considers
+ - Lower values: More focused word choice
+ - Higher values: Broader vocabulary usage
+
+- **Presence Penalty** (-2.0 to 2.0): Influences topic diversity
+ - Positive values: Encourages exploring new topics
+ - Negative values: Allows topic repetition
+
+- **Frequency Penalty** (-2.0 to 2.0): Affects word choice
+ - Positive values: Discourages word repetition
+ - Negative values: Allows word reuse
+
+## Comparing Prompts and Models
+
+Agenta's playground allows you to compare different prompts, model configurations, and LLM providers side by side. This helps you identify the most effective combination for your use case.
+
+### Creating Multiple Variants
+
+To compare different configurations:
+
+1. Click the "+Compare" button in the top right corner
+2. Select "New Variant" from the dropdown menu
+3. Choose a source variant to duplicate (this copies all current parameters)
+4. Give your new variant a descriptive name
+5. Modify the configuration as needed
+
+Each variant appears as a separate column in the playground, allowing direct comparison of outputs.
+
+### Side-by-Side Testing
+
+With multiple variants set up, you can:
+- Enter the same input text across all variants
+- Compare the generated outputs side by side
+- Review response times for each configuration
+- Monitor the cost per request for different models
+
+This parallel testing helps you understand how different prompts and models handle the same input.
+
+### Version Control and Deployment
+
+The playground integrates with Agenta's [prompt management system](/prompt-engineering/overview) to track and deploy changes:
+
+1. Click "Commit" to save a new version of your variant
+2. Changes remain in development until explicitly deployed
+3. Use "Deploy" to push configurations to specific environments (development, staging, production)
+
+## Working with Test Sets
+
+### Loading Existing Test Sets
+
+You can import test sets that you've created through:
+
+- CSV uploads
+- Programmatic creation
+- Web UI input
+
+The test set must contain columns that match your prompt template variables.
+
+### Saving Test Cases
+
+To save an interesting test case:
+
+1. Click the three-dot menu on the right side of the test case card
+2. Select "Add to Test Set"
+3. Choose an existing test set or create a new one
+4. Review the input/output mapping (use default settings unless you have specific needs)
+5. Preview the changes to your test set
+6. Confirm to save
+
+The Agenta playground serves as a comprehensive prompt engineering environment, offering advantages over traditional playgrounds like OpenAI's by providing integrated version control, deployment capabilities, and support for multiple models and frameworks. Whether you're developing new prompts, comparing model performance, or deploying to production, the playground provides the tools needed for effective LLM application development.
\ No newline at end of file
diff --git a/docs/docs/prompt-engineering/playground/02-adding-custom-providers.mdx b/docs/docs/prompt-engineering/playground/02-adding-custom-providers.mdx
new file mode 100644
index 0000000000..c2b3121621
--- /dev/null
+++ b/docs/docs/prompt-engineering/playground/02-adding-custom-providers.mdx
@@ -0,0 +1,148 @@
+---
+title: "Adding Custom LLM Providers"
+description: "Learn how to add and configure custom LLM providers like Azure OpenAI, AWS Bedrock, Ollama, and others to use your own models in Agenta's playground and applications."
+sidebar_position: 3
+---
+
+
+```mdx-code-block
+import { Stream } from '@cloudflare/stream-react';
+import Image from "@theme/IdealImage";
+```
+
+Agenta supports a wide range of LLM providers beyond the default options. This guide shows you how to configure custom providers and models for use in your Agenta applications.
+
+You can integrate self-hosted models into Agenta by adding custom providers such as:
+
+- [Azure OpenAI](#configuring-azure-openai)
+- [AWS Bedrock](#configuring-aws-bedrock)
+- [OpenAI-compatible endpoints](#configuring-openai-compatible-endpoints-eg-ollama) (including self-hosted models, OpenAI finetuned models, [Ollama](#configuring-openai-compatible-endpoints-eg-ollama), etc.)
+
+
+:::info
+You can add custom models to any of the providers already listed in the playground, such as [OpenRouter](#adding-models-to-a-provider-eg-openrouter), Anthropic, Gemini, Cohere, and others. This is especially useful for accessing specialized or new models that aren't included in Agenta's default configurations.
+:::
+
+---
+
+## How to Add Custom LLM Providers
+
+
+
+
+### From Settings
+
+1. Navigate to **Settings** → **Model Hub**.
+2. Click **Create Custom Provider**.
+3. Select the provider type and enter the required credentials.
+4. Click **Save**.
+
+
+### From the Playground
+
+1. Open any app in the Playground.
+2. Click the model dropdown menu.
+3. Select **Add Custom Provider**.
+
+:::note
+You can add multiple models to the same provider. This allows you to organize related models under a single provider configuration.
+:::
+
+
+## Configuring Azure OpenAI
+
+To add Azure OpenAI models, you'll need the following information:
+
+- **API Key**: Your Azure OpenAI API key
+- **API Version**: The API version (e.g., `2023-05-15`)
+- **Endpoint url**: The endpoint of your Azure resource
+- **Deployment Name**: The name of your model deployment
+
+### How to Retrieve Azure Credentials
+
+1. **Access the Azure Portal:**
+ Log in to the [Azure Portal](https://portal.azure.com).
+
+2. **Locate Azure OpenAI Service:**
+ - Search for **Azure AI Services** in the portal.
+ - Click on your resource to view its details.
+
+3. **Retrieve API Keys and Endpoints:**
+ - Navigate to the **Keys and Endpoint** section.
+ - Copy the API key and endpoint URL.
+
+4. **Find Deployment Names:**
+ - Go to **Model Deployments**.
+ - Select the desired deployment and note its name.
+
+### Configuration Example
+
+```plaintext
+API Key: c98d7a8s7d6a5s4d3a2s1d...
+API Version: 2023-05-15
+API base url: Use here your endpoint URL (e.g., https://accountnameinstance.openai.azure.com
+Deployment Name: Use here the deployment name in Azure (e.g., gpt-4-turbo)
+```
+
+## Configuring AWS Bedrock
+
+To add AWS Bedrock models, you'll need:
+
+- **Access Key ID:** Your AWS access key
+- **Secret Access Key:** Your AWS secret key
+- **Region:** The region where your Bedrock models are deployed
+
+### How to Retrieve AWS Credentials
+
+Refer to these tutorials for detailed instructions:
+ - [AWS Bedrock Getting Started Guide](https://docs.aws.amazon.com/bedrock/latest/userguide/getting-started.html)
+ - [DataCamp AWS Bedrock Tutorial](https://www.datacamp.com/tutorial/aws-bedrock)
+
+
+### Configuration Example
+
+```plaintext
+Access Key ID: xxxxxxxxxx
+Secret Access Key: xxxxxxxxxxxxxxxxxxxxxxx
+Region: (e.g eu-central-1)
+Model name: (e.g anthropic.claude-3-sonnet-20240229-v1:0)
+```
+
+
+## Configuring OpenAI-Compatible Endpoints (e.g., Ollama)
+
+For any OpenAI-compatible API, including self-hosted models:
+
+- **API Key:** The API authentication key
+- **Base URL:** The API endpoint URL
+
+This configuration works for:
+- Self-hosted models using vLLM
+- LocalAI
+- 3rd party providers with OpenAI-compatible endpoints
+- Fine-tuned OpenAI models
+
+### Configuration Example
+
+```
+API Key: your-api-key
+Base URL: https://your-api-endpoint.com/v1
+```
+
+:::warning
+Make sure to include `/v1` at the end of the Base URL.
+:::
+
+:::warning
+If you're running both Ollama and self-hosting Agenta on your local machine, set the Base URL to http://host.docker.internal:11434/v1 to allow Agenta's completion service (running behind Docker) to access Ollama.
+:::
+
+## Adding Models to a Provider (e.g. OpenRouter)
+
+For some providers like OpenRouter, Agenta does not include all available models due to their high number. To add models to an existing provider:
+
+1. Navigate to **Settings** → **Model Hub**.
+2. Select the provider you want to add models to.
+3. Enter the API key and an identifier
+4. Add the models you want to use
+5. Click **Save**
diff --git a/docs/docs/prompt-engineering/playground/02-custom-providers.mdx b/docs/docs/prompt-engineering/playground/02-custom-providers.mdx
deleted file mode 100644
index bbb65252e8..0000000000
--- a/docs/docs/prompt-engineering/playground/02-custom-providers.mdx
+++ /dev/null
@@ -1,206 +0,0 @@
----
-title: "Adding Custom LLM Providers"
-description: "Learn how to add and configure custom LLM providers like Azure OpenAI, AWS Bedrock, Ollama, and others to use your own models in Agenta's playground and applications."
-sidebar_position: 3
----
-
-
-```mdx-code-block
-import { Stream } from '@cloudflare/stream-react';
-import Image from "@theme/IdealImage";
-```
-
-Agenta supports a wide range of LLM providers beyond the default options. This guide shows you how to configure custom providers and models for use in your Agenta applications.
-
-You can integrate self-hosted models into Agenta by adding custom providers such as:
-
-- [Azure OpenAI](#configuring-azure-openai)
-- [AWS Bedrock](#configuring-aws-bedrock)
-- [Vertex AI](#configuring-vertex-ai)
-- [OpenAI-compatible endpoints](#configuring-openai-compatible-endpoints-eg-ollama) (including self-hosted models, OpenAI finetuned models, [Ollama](#configuring-openai-compatible-endpoints-eg-ollama), etc.)
-
-
-:::info
-You can add custom models to any of the providers already listed in the playground, such as [OpenRouter](#adding-models-to-a-provider-eg-openrouter), Anthropic, Gemini, Cohere, and others. This is especially useful for accessing specialized or new models that aren't included in Agenta's default configurations.
-:::
-
----
-
-## How to Add Custom LLM Providers
-
-
-
-
-### From Settings
-
-1. Navigate to **Settings** → **Model Hub**.
-2. Click **Create Custom Provider**.
-3. Select the provider type and enter the required credentials.
-4. Click **Save**.
-
-
-### From the Playground
-
-1. Open any app in the Playground.
-2. Click the model dropdown menu.
-3. Select **Add Custom Provider**.
-
-:::note
-You can add multiple models to the same provider. This allows you to organize related models under a single provider configuration.
-:::
-
-## Configuring Vertex AI
-
-To add Vertex AI models, please provide:
-
-- **Vertex Project**: Your Google Cloud **project ID** (e.g., `my-project-123`)
-- **Vertex Location**: The **region** (e.g., `europe-west4`, `us-central1`)
-- **Vertex Credentials**: A **Service Account** key in **JSON** format
-
-### How to Retrieve Vertex Credentials
-
-Follow Google’s guides to authenticate and create a service account key:
-
-- Vertex AI authentication: [https://cloud.google.com/vertex-ai/docs/authentication](https://cloud.google.com/vertex-ai/docs/authentication)
-- Create / manage service account keys: [https://cloud.google.com/iam/docs/keys-create-delete#iam-service-account-keys-create-console](https://cloud.google.com/iam/docs/keys-create-delete#iam-service-account-keys-create-console)
-
-**Steps to Retrieve Vertex Credentials in the Console**:
-
-1. In the Google Cloud console, go to **IAM & Admin → Service accounts**.
-2. Select a **Project**.
-3. Click the email address of the service account that you want to create a key for.
-4. Click the **Keys** tab.
-5. Click the **Add key** drop-down menu, then select **Create new key**.
-6. Select **JSON** as the **Key type** and click **Create**.
-
-> Clicking **Create** downloads a service account key file. After you download the key file, you cannot download it again.
-
-### Configuration Example
-
-```plaintext
-Vertex project: my-project-123
-Vertex location: europe-west4
-Vertex credentials:
-{
- "type": "service_account",
- "project_id": "PROJECT_ID",
- "private_key_id": "KEY_ID",
- "private_key": "-----BEGIN PRIVATE KEY-----\nPRIVATE_KEY\n-----END PRIVATE KEY-----\n",
- "client_email": "SERVICE_ACCOUNT_EMAIL",
- "client_id": "CLIENT_ID",
- "auth_uri": "https://accounts.google.com/o/oauth2/auth",
- "token_uri": "https://accounts.google.com/o/oauth2/token",
- "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
- "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/SERVICE_ACCOUNT_EMAIL"
-}
-```
-
-### Available models
-
-See the LiteLLM docs for an up-to-date list of models:
-
-- Vertex Gemini models: [https://docs.litellm.ai/docs/providers/vertex](https://docs.litellm.ai/docs/providers/vertex)
-- Vertex partner models: [https://docs.litellm.ai/docs/providers/vertex_partner](https://docs.litellm.ai/docs/providers/vertex_partner)
-- Vertex self-deployed models: [https://docs.litellm.ai/docs/providers/vertex_self_deployed](https://docs.litellm.ai/docs/providers/vertex_self_deployed)
-
-:::warning
-**Do not include** the `vertex_ai/` prefix in the **model name**. Use the bare model string (e.g., `gemini-2.5-pro`, `claude-3-5-sonnet@20240620`).
-:::
-
-## Configuring Azure OpenAI
-
-To add Azure OpenAI models, you'll need the following information:
-
-- **API Key**: Your Azure OpenAI API key
-- **API Version**: The API version (e.g., `2023-05-15`)
-- **Endpoint url**: The endpoint of your Azure resource
-- **Deployment Name**: The name of your model deployment
-
-### How to Retrieve Azure Credentials
-
-1. **Access the Azure Portal:**
- Log in to the [Azure Portal](https://portal.azure.com).
-
-2. **Locate Azure OpenAI Service:**
- - Search for **Azure AI Services** in the portal.
- - Click on your resource to view its details.
-
-3. **Retrieve API Keys and Endpoints:**
- - Navigate to the **Keys and Endpoint** section.
- - Copy the API key and endpoint URL.
-
-4. **Find Deployment Names:**
- - Go to **Model Deployments**.
- - Select the desired deployment and note its name.
-
-### Configuration Example
-
-```plaintext
-API Key: xxxxxxxxxx
-API Version: 2023-05-15
-API base url: Use here your endpoint URL (e.g., https://accountnameinstance.openai.azure.com
-Deployment Name: Use here the deployment name in Azure (e.g., gpt-4-turbo)
-```
-
-## Configuring AWS Bedrock
-
-To add AWS Bedrock models, you'll need:
-
-- **Access Key ID:** Your AWS access key
-- **Secret Access Key:** Your AWS secret key
-- **Region:** The region where your Bedrock models are deployed
-
-### How to Retrieve AWS Credentials
-
-Refer to these tutorials for detailed instructions:
- - [AWS Bedrock Getting Started Guide](https://docs.aws.amazon.com/bedrock/latest/userguide/getting-started.html)
- - [DataCamp AWS Bedrock Tutorial](https://www.datacamp.com/tutorial/aws-bedrock)
-
-
-### Configuration Example
-
-```plaintext
-Access Key ID: xxxxxxxxxx
-Secret Access Key: xxxxxxxxxx
-Region: (e.g eu-central-1)
-Model name: (e.g anthropic.claude-3-sonnet-20240229-v1:0)
-```
-
-
-## Configuring OpenAI-Compatible Endpoints (e.g., Ollama)
-
-For any OpenAI-compatible API, including self-hosted models:
-
-- **API Key:** The API authentication key
-- **Base URL:** The API endpoint URL
-
-This configuration works for:
-- Self-hosted models using vLLM
-- LocalAI
-- 3rd party providers with OpenAI-compatible endpoints
-- Fine-tuned OpenAI models
-
-### Configuration Example
-
-```
-API Key: your-api-key
-Base URL: https://your-api-endpoint.com/v1
-```
-
-:::warning
-Make sure to include `/v1` at the end of the Base URL.
-:::
-
-:::warning
-If you're running both Ollama and self-hosting Agenta on your local machine, set the Base URL to http://host.docker.internal:11434/v1 to allow Agenta's completion service (running behind Docker) to access Ollama.
-:::
-
-## Adding Models to a Provider (e.g. OpenRouter)
-
-For some providers like OpenRouter, Agenta does not include all available models due to their high number. To add models to an existing provider:
-
-1. Navigate to **Settings** → **Model Hub**.
-2. Select the provider you want to add models to.
-3. Enter the API key and an identifier
-4. Add the models you want to use
-5. Click **Save**
diff --git a/docs/docs/prompt-engineering/playground/_03-tools-function-calling.mdx b/docs/docs/prompt-engineering/playground/_03-tools-function-calling.mdx
deleted file mode 100644
index 23f571bc15..0000000000
--- a/docs/docs/prompt-engineering/playground/_03-tools-function-calling.mdx
+++ /dev/null
@@ -1,17 +0,0 @@
----
-title: "Use Tools and Function Calling"
-sidebar_position: 3
-sidebar_label: "Tools and Function Calling"
-description: "Learn how to use tools and function calling in the Agenta playground."
----
-
-
-
-
-This section is under construction.
diff --git a/docs/docs/prompt-engineering/playground/_04-structured-output.mdx b/docs/docs/prompt-engineering/playground/_04-structured-output.mdx
deleted file mode 100644
index 59a64ee1e1..0000000000
--- a/docs/docs/prompt-engineering/playground/_04-structured-output.mdx
+++ /dev/null
@@ -1,18 +0,0 @@
----
-title: "Work with Structured Output"
-sidebar_position: 4
-sidebar_label: "Structured Output"
-description: "Learn how to use structured output and JSON schema in the Agenta playground."
----
-
-
-
-
-This section is under construction. For basic information on dynamic JSON schema variables, see the [Using the Playground guide](/prompt-engineering/playground/using-playground#dynamic-json-schema-variables).
diff --git a/docs/docs/prompt-engineering/playground/_05-test-sets.mdx b/docs/docs/prompt-engineering/playground/_05-test-sets.mdx
deleted file mode 100644
index c772b6aa8a..0000000000
--- a/docs/docs/prompt-engineering/playground/_05-test-sets.mdx
+++ /dev/null
@@ -1,18 +0,0 @@
----
-title: "Load and Save Test Sets"
-sidebar_position: 5
-sidebar_label: "Load and Save Test Sets"
-description: "Learn how to load and save test sets in the Agenta playground."
----
-
-
-
-
-This section is under construction. For basic information on test sets, see the [Working with Test Sets section](/prompt-engineering/playground/using-playground#working-with-test-sets) in the playground guide.
diff --git a/docs/docs/prompt-engineering/playground/_06-images.mdx b/docs/docs/prompt-engineering/playground/_06-images.mdx
deleted file mode 100644
index 231f5b22ab..0000000000
--- a/docs/docs/prompt-engineering/playground/_06-images.mdx
+++ /dev/null
@@ -1,18 +0,0 @@
----
-title: "Use Images in Prompts"
-sidebar_position: 6
-sidebar_label: "Images"
-description: "Learn how to use images in prompts with vision-capable models in the Agenta playground."
----
-
-
-
-
-This section is under construction.
diff --git a/docs/docs/prompt-engineering/playground/_category_.json b/docs/docs/prompt-engineering/playground/_category_.json
index 5d11187b61..9b8516fb9d 100644
--- a/docs/docs/prompt-engineering/playground/_category_.json
+++ b/docs/docs/prompt-engineering/playground/_category_.json
@@ -1,5 +1,5 @@
{
- "position": 5,
- "label": "Use the Playground",
- "collapsed": true
+ "position": 4,
+ "label": "Playground",
+ "collapsed": false
}
\ No newline at end of file
diff --git a/docs/docs/prompt-engineering/prompt-management/01-how-to-integrate-with-agenta.mdx b/docs/docs/prompt-engineering/prompt-management/01-how-to-integrate-with-agenta.mdx
new file mode 100644
index 0000000000..e76248986f
--- /dev/null
+++ b/docs/docs/prompt-engineering/prompt-management/01-how-to-integrate-with-agenta.mdx
@@ -0,0 +1,56 @@
+---
+title: "Integrating with agenta"
+description: "Integrate applications and prompts created in Agenta into your projects."
+---
+
+import Image from "@theme/IdealImage";
+
+Agenta easily integrates with your workflow, allowing you to use the latest version of the deployed prompt in your application. With Agenta, **you can update prompts directly from the web interface without modifying your code** each time.
+
+Here are the two ways you can use the prompts from Agenta in your code:
+
+### [1. As a prompt management system](/prompt-engineering/prompt-management/prompt-management-sdk):
+
+In this approach, prompts are managed and stored in the Agenta backend. You use the Agenta SDK to fetch the latest deployed version of your prompt and use it in your application.
+
+**Advantages**:
+
+- Agenta operates outside your application's critical path.
+- Allows you to fetch and cache the latest prompt version for zero latency usage.
+
+**Considerations**:
+
+- You need to set up the [integration with observability](/observability/quickstart) yourself if you want to trace your calls for debugging and cost tracking.
+
+
+
+
+
+### **[2. As a middleware / model proxy](/prompt-engineering/prompt-management/proxy-calls)**:
+
+In this setup, Agenta provides you with an endpoint that forwards requests to the LLM on your behalf.
+
+**Advantages**:
+
+- Simplified deployment.
+- Automatic tracing without any changes to your code.
+
+**Considerations**:
+
+- Adds a slight latency to the response (approximately 0.3 seconds).
+- Currently, we don't support streaming for these endpoints.
+
+Overall, this approach is best suited for applications where latency isn't critical.
+
+
+
diff --git a/docs/docs/prompt-engineering/prompt-management/02-prompt-management-sdk.mdx b/docs/docs/prompt-engineering/prompt-management/02-prompt-management-sdk.mdx
new file mode 100644
index 0000000000..b9f570deb8
--- /dev/null
+++ b/docs/docs/prompt-engineering/prompt-management/02-prompt-management-sdk.mdx
@@ -0,0 +1,718 @@
+---
+title: "How to Manage Prompts with the SDK"
+sidebar_position: 2
+sidebar_label: "Manage Prompts with the SDK"
+description: "Learn how to create variants, commit changes, deploy to environments, and fetch configurations using the Agenta SDK."
+---
+
+import Image from "@theme/IdealImage";
+
+
+This guide covers all prompt management operations using the Agenta SDK: creating variants, committing changes, deploying to environments, and fetching configurations.
+
+:::tip Looking for a simpler introduction?
+For a step-by-step tutorial, see the [prompt management tutorial](/tutorials/sdk/manage-prompts-with-SDK).
+:::
+
+## Prerequisites
+
+Before starting, familiarize yourself with how versioning works in Agenta. Details are available on the [concepts page](/concepts/concepts).
+
+:::info Versioning in Agenta
+
+{" "}
+
+
+
+Agenta uses a Git-like structure for prompt versioning:
+- Create multiple branches called **variants**
+- Each variant is versioned.
+
+- Deploy specific versions to **environments** (development, staging, production)
+
+**Typical workflow:**
+1. Create a variant (branch)
+2. Commit changes to the variant creating a new version
+3. Deploy the version to an environment
+
+:::
+
+## Setup
+
+Initialize the SDK before using any operations:
+
+```python
+import os
+import agenta as ag
+
+# Set your API credentials
+os.environ["AGENTA_API_KEY"] = "your-api-key-here"
+os.environ["AGENTA_HOST"] = "https://cloud.agenta.ai" # only needed if self-hosting
+
+# Initialize the SDK
+ag.init()
+```
+
+## Creating and Managing Variants
+
+### Create a New Variant
+
+Use `VariantManager.create` to create a new variant with initial configuration:
+
+```python
+from agenta.sdk.types import PromptTemplate, Message, ModelConfig
+from pydantic import BaseModel
+
+# Define your configuration model
+class Config(BaseModel):
+ prompt: PromptTemplate
+
+# Create the initial configuration
+config = Config(
+ prompt=PromptTemplate(
+ messages=[
+ Message(role="system", content="You are an assistant that provides concise answers"),
+ Message(role="user", content="Explain {{topic}} in simple terms"),
+ ],
+ llm_config=ModelConfig(
+ model="gpt-3.5-turbo",
+ max_tokens=150,
+ temperature=0.7,
+ top_p=1.0,
+ frequency_penalty=0.0,
+ presence_penalty=0.0,
+ ),
+ )
+)
+
+# Create the variant
+variant = ag.VariantManager.create(
+ parameters=config.model_dump(),
+ app_slug="my-app-slug",
+ variant_slug="my-variant-slug",
+)
+```
+
+
+:::tip
+Use `VariantManager.acreate` for async variant creation.
+:::
+
+This command will create a new variant and initialize it with the first commit containing the parameters provided
+
+- **Parameters:**
+ - `app_slug`: The slug of your application.
+ - `variant_slug`: The slug of the new variant.
+ - `parameters`: A dictionary containing the initial configuration parameters.
+
+**Note:** If a variant with the same `slug` and `version` already exists, the SDK will raise an exception.
+
+**Sample Output:**
+
+```python
+{
+ "messages": [
+ {
+ "role": "system",
+ "content": "You are an assistant that provides concise answers"
+ },
+ {
+ "role": "user",
+ "content": "Explain {{topic}} in simple terms"
+ }
+ ],
+ "llm_config": {
+ "model": "gpt-3.5-turbo",
+ "max_tokens": 150,
+ "temperature": 0.7,
+ "top_p": 1.0,
+ "frequency_penalty": 0.0,
+ "presence_penalty": 0.0
+ },
+ "template_format": "curly"
+}
+```
+
+### Committing changes to a variant (Creating a new version in a variant)
+
+To save changes to a variant (creating a new version), use the `VariantManager.commit` method with explicit parameters.
+
+```python
+
+config2=Config(
+ prompt=PromptTemplate(
+ messages=[
+ Message(role="system", content="You are an assistant that provides VERY concise answers"),
+ Message(role="user", content="Explain {{topic}} in simple terms"),
+ ],
+ llm_config=ModelConfig(
+ model="anthropic/claude-3-5-sonnet-20240620",
+ max_tokens=150,
+ temperature=0.7,
+ top_p=1.0,
+ frequency_penalty=0.0,
+ presence_penalty=0.0,
+ ),
+ )
+)
+
+
+# Commit the new version
+variant = ag.VariantManager.commit(
+ parameters=config2.model_dump(),
+ app_slug="my-app-slug",
+ variant_slug="my-variant-slug",
+)
+
+print("Committed new version of variant:")
+print(variant)
+
+# Commit the new version asynchronously (optional)
+# async def commit_variant():
+# variant = await ag.VariantManager.acommit(
+# parameters=my_prompt2.model_dump(),
+# app_slug="my-app-slug",
+# variant_slug="my-variant-slug",
+# )
+
+# print("Committed new version of variant (async):")
+# print(variant)
+```
+
+:::tip
+Use `VariantManager.acommit` for async version commit.
+:::
+
+:::info Immutability
+Each commit creates a new version of the variant. Versions are immutable once created.
+:::
+
+**Sample Output:**
+
+```python
+{
+ "app_id": "01963413-3d39-7650-80ce-3ad5d688da6c",
+ "app_slug": "my-app-slug",
+ "variant_id": "01968c11-6f7c-7773-b273-922c5807be7b",
+ "variant_slug": "my-variant-slug",
+ "variant_version": 5,
+ "environment_id": null,
+ "environment_slug": null,
+ "environment_version": null,
+ "committed_at": "2025-05-01T07:26:08.935406+00:00",
+ "committed_by": "user@agenta.ai",
+ "committed_by_id": "0196247a-ec9d-7051-8880-d58279570aa1",
+ "deployed_at": null,
+ "deployed_by": null,
+ "deployed_by_id": null,
+ "params": {
+ "messages": [
+ {
+ "name": null,
+ "role": "system",
+ "content": "You are an assistant that provides concise answers",
+ "tool_calls": null,
+ "tool_call_id": null
+ },
+ {
+ "name": null,
+ "role": "user",
+ "content": "Explain {{topic}} in simple terms",
+ "tool_calls": null,
+ "tool_call_id": null
+ }
+ ],
+ "input_keys": null,
+ "llm_config": {
+ "model": "gpt-3.5-turbo",
+ "tools": null,
+ "top_p": 1.0,
+ "stream": null,
+ "max_tokens": 150,
+ "temperature": 0.7,
+ "tool_choice": null,
+ "response_format": null,
+ "presence_penalty": 0.0,
+ "frequency_penalty": 0.0
+ },
+ "user_prompt": null,
+ "system_prompt": null,
+ "template_format": "curly"
+ }
+}
+```
+
+:::warning
+The `user_prompt` and `system_prompt` are helper fields to populate the `messages` field in the `PromptTemplate`. Rely only on the `messages` field in your code.
+:::
+
+## Deploying to Environments
+
+To deploy a variant to an environment, use the `DeploymentManager.deploy` method with the variant reference and `environment_slug`: The slug of the environment (`development`, `staging`, or `production`).
+
+```python
+deployment = ag.DeploymentManager.deploy(
+ app_slug="my-app-slug",
+ variant_slug="my-variant-slug",
+ variant_version=None, # Deploys latest version if not specified
+ environment_slug="staging" # Options: development, staging, production
+)
+
+print(f"Deployed to {deployment['environment_slug']}")
+```
+
+:::warning
+- Deploying a variant without specifying a `variant_version` deploys the latest version.
+- Only predefined environments with slugs `development`, `staging`, and `production` are currently supported.
+:::
+
+**Sample Output:**
+
+```python
+Deployed variant to environment:
+{
+ "app_id": "01963413-3d39-7650-80ce-3ad5d688da6c",
+ "app_slug": "completion",
+ "variant_id": "01968c11-6f7c-7773-b273-922c5807be7b",
+ "variant_slug": "my-variant-slug4",
+ "variant_version": 5,
+ "environment_id": "01968c14-c35d-7440-bcc8-9def594f017f",
+ "environment_slug": "staging",
+ "environment_version": 2,
+ "committed_at": "2025-05-01T07:26:08.935406+00:00",
+ "committed_by": "user@agenta.ai",
+ "committed_by_id": "0196247a-ec9d-7051-8880-d58279570aa1",
+ "deployed_at": "2025-05-01T13:41:33.149595+00:00",
+ "deployed_by": "user@agenta.ai",
+ "deployed_by_id": "0196247a-ec9d-7051-8880-d58279570aa1"
+}
+```
+
+## Fetching a Prompt Configuration
+
+You can fetch the configurations from a variant reference (`app_slug`, `variant_slug`, `variant_version`) or an environment reference (`app_slug`, `environment_slug`). The default behavior when fetching is to fetch the latest configuration from the `production` environment. If you don't provide a `_version` parameter but only a `variant_slug` or an `environment_slug`, the SDK will fetch the latest version of the variant from the specified **environment/variant**.
+
+:::tip
+Check the [reference](/reference/sdk/configuration-management#prompt-configuration-schema) section for more details on the data format used for prompts.
+:::
+
+### Default Behavior when fetching
+
+If you don't provide either `variant` or `environment` identifiers, the SDK fetches the latest configuration deployed to the `production` environment.
+
+```python
+config = ag.ConfigManager.get_from_registry(
+ app_slug="my-app-slug",
+ variant_slug="my-variant-slug",
+ variant_version=2 # Optional: fetches latest if not provided
+)
+
+print("Fetched configuration from production:")
+print(config)
+```
+Example Output:
+
+```python
+{
+ "prompt": {
+ "messages": [
+ {
+ "role": "system",
+ "content": "You are an assistant that provides concise answers"
+ },
+ {
+ "role": "user",
+ "content": "Explain {{topic}} in simple terms"
+ }
+ ],
+ "llm_config": {
+ "model": "gpt-3.5-turbo",
+ "top_p": 1.0,
+ "max_tokens": 150,
+ "temperature": 0.7,
+ "presence_penalty": 0.0,
+ "frequency_penalty": 0.0
+ },
+ "template_format": "curly"
+ }
+}
+```
+
+:::tip
+Agenta provides a helper class `PromptTemplate` to format the configuration and then use it to generate the prompt.
+```python
+from openai import OpenAI
+from agenta.sdk.types import PromptTemplate
+
+# Fetch configuration
+config = ag.ConfigManager.get_from_registry(
+ app_slug="my-app-slug"
+)
+
+# Format the prompt with variables
+prompt = PromptTemplate(**config['prompt']).format(topic="AI")
+
+# Use with OpenAI
+client = OpenAI()
+response = client.chat.completions.create(
+ **prompt.to_openai_kwargs()
+)
+
+print(response.choices[0].message.content)
+```
+:::
+
+### Fetching by Variant Reference
+
+```python
+# Fetch configuration by variant
+config = ag.ConfigManager.get_from_registry(
+ app_slug="my-app-slug",
+ variant_slug="my-variant-slug",
+ variant_version=2 # Optional: If not provided, fetches the latest version
+)
+
+print("Fetched configuration:")
+print(config)
+```
+
+### Fetching by Environment Reference
+
+```python
+# Fetch the latest configuration from the staging environment
+config = ag.ConfigManager.get_from_registry(
+ app_slug="my-app",
+ environment_slug="staging",
+ environment_version=1 # Optional: If not provided, fetches the latest version
+)
+
+print("Fetched configuration from staging:")
+print(config)
+```
+
+## Deleting a Variant
+
+To delete a variant, use the `VariantManager.delete` method.
+
+```python
+# Delete a variant
+ag.VariantManager.delete(
+ app_slug="my-app",
+ # app_id="my-app-id", # you can also use `app_id`
+ variant_slug="obsolete-variant"
+)
+
+print("Variant deleted successfully.")
+```
+
+:::warning
+
+- Deleting a variant removes all versions of the variant. This action is irreversible.
+- Attempting to delete a variant that is deployed to an environment will fail.
+:::
+
+## Listing All Variants
+
+To list all variants of an application, use the `VariantManager.list` method.
+
+```python
+# List all variants (syncrhonously)
+variants = ag.VariantManager.list(
+ app_slug="my-app"
+ # app_id="my-app-id", # you can also use `app_id`
+)
+
+print(variants)
+```
+
+**Sample Output:**
+
+```python
+
+[{
+ "app_id": "01963413-3d39-7650-80ce-3ad5d688da6c",
+ "app_slug": "completion",
+ "variant_id": "01968aba-9702-7263-a7fd-8b8565753230",
+ "variant_slug": "my-variant-slug3",
+ "variant_version": 2,
+ "environment_id": null,
+ "environment_slug": null,
+ "environment_version": null,
+ "committed_at": "2025-05-01T07:23:26.338901+00:00",
+ "committed_by": "user@agenta.ai",
+ "committed_by_id": "0196247a-ec9d-7051-8880-d58279570aa1",
+ "deployed_at": null,
+ "deployed_by": null,
+ "deployed_by_id": null,
+ "params": {
+ "prompt": {
+ "messages": [
+ {
+ "name": null,
+ "role": "system",
+ "content": "You are an assistant that provides concise answers",
+ "tool_calls": null,
+ "tool_call_id": null
+ },
+ {
+ "name": null,
+ "role": "user",
+ "content": "Explain {{topic}} in simple terms",
+ "tool_calls": null,
+ "tool_call_id": null
+ }
+ ],
+ "input_keys": null,
+ "llm_config": {
+ "model": "gpt-3.5-turbo",
+ "tools": null,
+ "top_p": 1.0,
+ "stream": null,
+ "max_tokens": 150,
+ "temperature": 0.7,
+ "tool_choice": null,
+ "response_format": null,
+ "presence_penalty": 0.0,
+ "frequency_penalty": 0.0
+ },
+ "user_prompt": null,
+ "system_prompt": null,
+ "template_format": "curly"
+ }
+ }
+}, {
+ "app_id": "01963413-3d39-7650-80ce-3ad5d688da6c",
+ "app_slug": "completion",
+ "variant_id": "01963f1f-28ea-78e1-b390-39d270fe93fe",
+ "variant_slug": "key",
+ "variant_version": 4,
+ "environment_id": null,
+ "environment_slug": null,
+ "environment_version": null,
+ "committed_at": "2025-04-16T15:02:08.874979+00:00",
+ "committed_by": "user@agenta.ai",
+ "committed_by_id": "0196247a-ec9d-7051-8880-d58279570aa1",
+ "deployed_at": null,
+ "deployed_by": null,
+ "deployed_by_id": null,
+ "params": {
+ "ag_config": {
+ "prompt": {
+ "messages": [
+ {
+ "role": "system",
+ "content": "You are an expert in geography"
+ },
+ {
+ "role": "user",
+ "content": "What is the capital of {{country}}? tell me its name in arabic"
+ }
+ ],
+ "input_keys": [
+ "country"
+ ],
+ "llm_config": {
+ "model": "gpt-3.5-turbo"
+ },
+ "template_format": "curly"
+ }
+ }
+ }
+}]
+```
+
+## Fetching a Variant's history
+
+To list all versions for a variant of an application, use the `VariantManager.list` method.
+
+```python
+# List all variant versions/history (synchronously)
+versions = ag.VariantManager.history(
+ variant_slug="variant-slug",
+ app_slug="my-app"
+ # app_id="my-app-id", # you can also use `app_id`
+)
+
+print(versions)
+```
+
+**Sample Output:**
+
+Same as `VariantManager.list` but limited to the history of a specific variant.
+
+## Asynchronous Operations
+
+All SDK methods have async counterparts with an `a` prefix:
+
+```python
+async def async_operations():
+ # Create variant asynchronously
+ variant = await ag.VariantManager.acreate(...)
+
+ # Commit changes asynchronously
+ updated_variant = await ag.VariantManager.acommit(...)
+
+ # Fetch configuration asynchronously
+ config = await ag.ConfigManager.aget_from_registry(...)
+```
+
+## Caching
+
+The Agenta SDK includes intelligent caching for prompt configurations to improve performance and reduce API calls. Caching is **enabled by default** and works transparently without requiring code changes.
+
+### Default Caching Behavior
+
+All `get_from_registry()` calls are automatically cached with a **60-second TTL**:
+
+```python
+# First call - fetches from API and caches result
+config = ag.ConfigManager.get_from_registry(app_slug="my-app")
+
+# Second call within 60 seconds - returns cached result instantly
+config = ag.ConfigManager.get_from_registry(app_slug="my-app")
+```
+
+The cache uses intelligent key generation based on all configuration parameters, so different parameter combinations are cached separately:
+
+```python
+# These are cached as separate entries
+config1 = ag.ConfigManager.get_from_registry(app_slug="my-app")
+config2 = ag.ConfigManager.get_from_registry(app_slug="my-app", variant_slug="v2")
+config3 = ag.ConfigManager.get_from_registry(app_slug="my-app", environment_slug="staging")
+```
+
+### Custom Cache TTL
+
+You can control the cache duration using the `cache_ttl_seconds` parameter:
+
+```python
+# Cache for 5 minutes
+config = ag.ConfigManager.get_from_registry(
+ app_slug="my-app",
+ cache_ttl_seconds=300
+)
+
+# Cache for 1 hour
+config = ag.ConfigManager.get_from_registry(
+ app_slug="my-app",
+ variant_slug="production",
+ cache_ttl_seconds=3600
+)
+```
+
+### Disable Caching
+
+For development or testing scenarios where you always want fresh data, disable caching by setting `cache_ttl_seconds=0`:
+
+```python
+# Always fetch fresh data from API
+config = ag.ConfigManager.get_from_registry(
+ app_slug="my-app",
+ cache_ttl_seconds=0
+)
+
+# Common pattern for development
+config = ag.ConfigManager.get_from_registry(
+ app_slug="my-app",
+ variant_slug="latest",
+ cache_ttl_seconds=0 # Always get latest changes
+)
+```
+
+### Fallback Configurations
+
+Ensure 100% availability by providing fallback configurations when the API is unreachable:
+
+```python
+# Define fallback configuration
+fallback_config = {
+ "params": {
+ "prompt": {
+ "messages": [
+ {"role": "system", "content": "You are a helpful assistant"},
+ {"role": "user", "content": "Hello {{name}}"}
+ ],
+ "llm_config": {
+ "model": "gpt-3.5-turbo",
+ "temperature": 0.7,
+ "max_tokens": 150
+ }
+ }
+ }
+}
+
+# This will use fallback if API fails
+config = ag.ConfigManager.get_from_registry(
+ app_slug="my-app",
+ fallback_config=fallback_config
+)
+```
+
+### Cache Management
+
+Clear the cache when needed for testing or when you know configurations have changed:
+
+```python
+# Clear all cached configurations
+ag.ConfigManager.clear_cache()
+
+# Clear only configurations for specific apps (pattern-based)
+ag.ConfigManager.clear_cache("my-app") # Clears entries containing "my-app"
+ag.ConfigManager.clear_cache("staging") # Clears entries containing "staging"
+```
+
+### Async Caching
+
+All caching features work identically with async methods:
+
+```python
+# Async with caching (60s default TTL)
+config = await ag.ConfigManager.aget_from_registry(app_slug="my-app")
+
+# Async with custom TTL
+config = await ag.ConfigManager.aget_from_registry(
+ app_slug="my-app",
+ cache_ttl_seconds=300
+)
+
+# Async with fallback
+config = await ag.ConfigManager.aget_from_registry(
+ app_slug="my-app",
+ fallback_config=fallback_config
+)
+```
+
+### Performance Benefits
+
+Caching provides significant performance improvements:
+
+- **First call**: Standard API latency + minimal cache overhead
+- **Cached calls**: Near-zero latency (typically < 1ms)
+- **Reduced API load**: Fewer requests to Agenta servers
+- **Improved reliability**: Fallback configurations ensure availability
+
+### Configuration
+
+The cache behavior can be customized using environment variables:
+
+```bash
+# Cache capacity (default: 512 entries)
+export AGENTA_SDK_CACHE_CAPACITY=1024
+
+# Default TTL in seconds (default: 60)
+export AGENTA_SDK_CACHE_TTL=120
+```
+
+:::tip
+The cache is thread-safe and works seamlessly in multi-threaded applications. Each unique combination of parameters gets its own cache entry.
+:::
+
+:::note
+Caching is applied at the configuration level. Changes to prompts or configurations in the Agenta UI will be reflected after the cache TTL expires, or you can clear the cache manually.
+:::
+
diff --git a/docs/docs/prompt-engineering/integrating-prompts/03-proxy-calls.mdx b/docs/docs/prompt-engineering/prompt-management/03-proxy-calls.mdx
similarity index 97%
rename from docs/docs/prompt-engineering/integrating-prompts/03-proxy-calls.mdx
rename to docs/docs/prompt-engineering/prompt-management/03-proxy-calls.mdx
index 2dcdface09..d0012fd612 100644
--- a/docs/docs/prompt-engineering/integrating-prompts/03-proxy-calls.mdx
+++ b/docs/docs/prompt-engineering/prompt-management/03-proxy-calls.mdx
@@ -1,8 +1,7 @@
---
-title: "Invoke Prompt (Proxy LLM Calls)"
+title: "Proxy LLM Calls"
description: "How to invoke your deployed prompts through Agenta's proxy service with automatic tracing and logging."
sidebar_position: 4
-sidebar_label: "Invoke Prompts"
---
import Image from "@theme/IdealImage";
diff --git a/docs/docs/prompt-engineering/_category_.json b/docs/docs/prompt-engineering/prompt-management/_category_.json
similarity index 100%
rename from docs/docs/prompt-engineering/_category_.json
rename to docs/docs/prompt-engineering/prompt-management/_category_.json
diff --git a/docs/docs/reference/api/accept-invitation.api.mdx b/docs/docs/reference/api/accept-invitation.api.mdx
index 88367d84c6..bce95f2bf9 100644
--- a/docs/docs/reference/api/accept-invitation.api.mdx
+++ b/docs/docs/reference/api/accept-invitation.api.mdx
@@ -5,75 +5,370 @@ description: "Accept an invitation to an organization."
sidebar_label: "Accept Organization Invitation"
hide_title: true
hide_table_of_contents: true
-api: eJztVt9v4zYM/lcEPm2AV2fF7cV7WbYVuO62NUizDUMbBIzNxrrakk+S2+YM/+8DJTt2nLb7gduelrdY1EfyI/lRDTjcWUhu4MrsUMmP6KRWsI4gI5saWfm/CczTlConUAmpHqTzVsJp/qBHF89u1a1aorRkk1slhBBvV6vFxRNfllolYjOC3XwtrENX202qM0rEV7PZs3cuB49KO3Gna5UJbUSOVtBTJQ1lE6Q3LyD9rmuBhSHM9mJLhVY7zsHl0h5l8RzarVqSq43q8/rh+urnJdlKK0uJCPRQNmHnUZt7W2FKE8Tz2Qwi0BUZb3qZQQLoITYDAERQocGSHBmuUAMKS4IExqFuZAYRSC5RhS6HCAx9qJkSSJypKQKb5lQiJA24fcXXrTNS7SACJ13BH8aVF5cZtG108HXI4FM4+q0Hm3qpjH5PqRv5+FCT2f8jJ4uAFVysAwJZ963O9nxtCphq5Ug5PsKqKmTqaYjfW277ZuSvMlwvJ8l67/qe1GthrLxBGwGVKIvXLC+8QdtGvYXecgJHyd90Dnu49XDdTwd17lqGMV1b+kC51ZJmMs3XdZqStXd1Ifoehr9ORfDy5vz8FPhXLGQWGunCGG3+BuqE4Ixcx5t0VNpTg0KnR6eo9ld3fkyOieY2675I5WhHBtr1QDYag/tRNX7UIUAuXWl3rxXuJ7IWdwQHsFdqzGSIFZ/+WaE5r+C6sxvVeqA3sPtyGt8H+p5z1puwNp4AhtqW5HLNolRp67wOuRwSiMfKY+NmIkRtfFALGzdj5WhjL2sUB5GDCCyZh17WalNAArlzlU3iOC10nZ3hjpTDM5QxVhJ4ji2ltZFu76/MF5fvaP+WMCMDyc16bHDNHRV65NjsUB6s5Dtitjr5mdcu16bffZ0C5eEW08G9uhxk5OIJy6qgkQwMBe+mfWg+kOpOe9OO9rnPTMwXlxABcxAmZ3b25RkvhcnaHRvzJGHqJ6kP3B9DNKHwQN4oIEdYfjMccFpc3BLVGC9s+eOFMF5IR9E1w2z//z74F94HXbc6enJxVaD0muQr3XQTeXP0FrAQQXL6OBiGks9PFjrPJUTd84PFJueZT26gabZo6RdTtC1/DhuZhy+TFrcFq9UdFpZe6YrPlp2wfS5eyuee9tMHwAMWNRv6sX9AI9nbp/fci4Haj332EU155DXyH7o/KpPfV3kvdk1n8l3w9oXfKgPEyZLlyMONea++L9uuR+K/uLpeQQTb7uVU6ozvGHzkjYWPIVxdhdbjpxV/a6BAtat5LyYQMPn3B8FxSMc=
+api: eJztVt9v4zYM/lcEPm2AF2fF7cV7We9W4LrbliDNdji0QcHarK2rLfkkuW3O8P8+ULJjJ2m7H7jtaXmLTH4UP5If1YLD3EJyCQuTo5Kf0UmtYBNBRjY1svZ/EzhNU6qdQCWkupfOWwmn+UBPHGdX6kqtUFqyyZUSQoi36/Xy7JGdpVaJuJ7AXn8vrEPX2OtUZ5SI7+bzJ33Ox4hKO3GrG5UJbUSBVtBjLQ1lB0ivnkH6oBuBpSHMtuKGSq1yzsEV0u5l8RTalVqRa4wa8vrpYvHrimytlaVEBHooO2DnQZs7W2NKB4gn8zlEoGsy3vQ8gwTQQ1yPABBBjQYrcmS4Qi0orAgS0Ca/lhlEILkyNboCIjD0qWEmIHGmoQhsWlCFkLTgtjV7WWekyiECJ13JBwuTi/MMui7aIe/u+yXw3w9gh1Fqoz9S6iYxPjVktv8oyDJghRCbgEDWvdbZlt0OAVOtHCnHn7CuS5l6puOPlpu8ncSrDVfHSbI+ur4j9dI11t6gi4AqlOVLlmfeoOuiwULfcAJ7yV/2AQe4zejuZ4H6cB3DmL4J/UW5sZL2YHYvmjQla2+bUgwdC3+dihDl1cnJMfDvWMosNPuZMdr8DdQDgjNyPW/SUWWPDUqd7n1FtV3c+qHYJ5rbrD+RylFOBrrNSDYag9tJNX7W4YJcusrmLxXuF7IWc4Id2As1ZjLEmr/+WaE5rxC6t5vUeqQ3sPt8Gj8G+p4KNpiwEh4BhtpW5ArNElRr67zquAISiKeSaOM2yE4X70TCxu1UMLrYaxfFQckgAkvmftCuxpSQQOFcbZM4TkvdZDPMSTmcoYyxlnC0cvxX8YZNgWfbUtoY6bYe73R5/o62bwkzMpBcbqYGF9xloW/2zXYlw1q+I2awl6TTxhXaDNuvV6UieDFF3L+rUVrOHrGqS5pIw9gEvQKMDQlS3Wpv2peiT+w1pnekWAOZpJDzfPbtbP4cE72DOF2e95OGqZ+0IQlvBtEB1zuWJ5dzhNUP4wdOkYtfoZrihZ0/fRhMdvHhLdtx9v9/LfwLr4W+cx09urguUXrN8pVu+4m9hL2JhQiS3VNhHFo+PtrzPLcQ9W8Q1qCCpSC5hLa9QUu/mbLr+Dgsap6/TFq8KVnEbrG09EIzfLXq9e5r8Vwad7Q9fBfcY9mwoZ/8ezSSo335yIMeqO005nCjnj5eKv9h1L3q+O1VDDLX9iZvQrRv/I4ZIY5WLt88eJwOovy87WayCpaLizVEcNO/oyqdsY/BB95f+BCuq+vQaPzQ4rMWSlR5w1sygYDJvz8AGfBEVQ==
sidebar_class_name: "post api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
+>
-
-
-
-
+ path={"/organizations/{org_id}/workspaces/{workspace_id}/invite/accept"}
+>
Accept an invitation to an organization.
Raises:
- HTTPException: _description_; status_code: 500
- HTTPException: Invitation not found or has expired; status_code: 400
- HTTPException: You already belong to this organization; status_code: 400
+HTTPException: _description_; status_code: 500
+HTTPException: Invitation not found or has expired; status_code: 400
+HTTPException: You already belong to this organization; status_code: 400
Returns:
- JSONResponse: Accepted invitation to workspace; status_code: 200
+JSONResponse: Accepted invitation to workspace; status_code: 200
-
-
-
-
-
+>
-
-
-
-
-
+
diff --git a/docs/docs/reference/api/add-variant-from-base-and-config.api.mdx b/docs/docs/reference/api/add-variant-from-base-and-config.api.mdx
index b79ae2556a..d0fa2cf904 100644
--- a/docs/docs/reference/api/add-variant-from-base-and-config.api.mdx
+++ b/docs/docs/reference/api/add-variant-from-base-and-config.api.mdx
@@ -5,77 +5,668 @@ description: "Add a new variant based on an existing one."
sidebar_label: "Add Variant From Base And Config"
hide_title: true
hide_table_of_contents: true
-api: eJztV01v2zgQ/SvEnHYB1c4Ge9JpnTaLBt02Rj66B8cwJuLYZiuRKkkl8Rr+74shJUuykzhNgD1tLpHF4RvOzHvD0Ro8LhykE/iKVqH2DqYJSHKZVaVXRkMKIykFCk334i7aiFt0JIXRArWgB+W80gthNA1u9CUWJNCJ8fnllRhmRs/V4kbf6JFduPRGCyFEiavcoBS/jKSsvf5pTXGCjsZx6ddU1E8iM9qj0uyAvW6PcPYh6R1JY0GJQC1FiRYL8mTdgB1foHLUuP54dTU+fcgohJaKsCaFmgu/bKEzU+VSaOPFLQmUkkO1ArOMnCMZQclXVjeo11oZPRmVZR3NBbnSaEeJGOnVNBVfOueU5FHljhGpOcgAEjAlWeQfZxJSQCln9Y7Z3JpixrHPUMtZzCgkYOlHRc6fGLmCdA2cJ9KeH7Esc5UFsOE3xyVcg8uWVCA/lZZdeUWOfwVcJfnRr0qCFJy3SrMDr3zOL7gu4kzCJgFN99tjccKf28ZB1/kQX9i23h8DeNH298F0u7stbIhSSsUhYj7uRORtRUkDam6/UeY7oOMWYZNAZopC+VlBzuEiHAb16nwO6WT3WJtk+0ZXeQ6baYv5PqCIzzXKZrPvnkulLEmWWZPwR5K5n59ezB2fTwoHNuy/9cfpCC8iIUPijo+O+F9f45dVoPe8ykXDXkheTqo2cX16YVkeYNeoLGtyse0hVrB1Q4cmd8/jNxSMPl5K3l3iltZwLQ/4Gker2leo9CFHQVyNl58T4wuFtCOiyqrXMv3aKghkulNORQbUpkp7WpDteL1ojPigltCTnKF/tcYighj5EEIp34h3HRFqvMJINVckZ7erOvuvwfxco4iTVV0gYxeo1T9BM28APu/A1Mj3xn53JWb0Bti/G4yAeaBr1TruiLSnvx1h9fTSFULSaX/9Prcl1bQn9p0LdcYRdYNoFgTPKI1ouSeKIJSRlvUlwo1xk8Dvx8f7ve8r5krG9J5aa+zPNL5+u4uXOz8pT4XbN8hN1lt9QeEabW2mbY3QWlx11PaXiQcMbHaL59rB9p5qwJ42DckQV7x6iB8cV3Rd2027vbRJb8zu02F8iOl7zFljwgPcHmCsbUF+aXh4Ko3z4er0S0hhWFPTDXmQescEhAQc2bswSEzWUNkcUlh6X7p0OMxyU8kBLkh7HKAaYqmCbhxllVV+FbaMxmefaPWRUJKFdDLtGlwyOWK5+2bbTGOpPhEHHns3jCq/NLaWOCSgmJXLuIsjY9pdtOPe6QMWZU698a2t3v6M1l/rXRvtUm+4emQ4apkJSs9NiKbRaciVGI3PuBGQjZcDHA1+GxzB3sdE1zgM91mQWZOKsAzJTlG25YAEqAgaA09Y/NEucKK48gXqLt6hvrBzvs4g/f93z3/x3VNLwtODH5Y5Kh2HlNBEo4InzeXiIIFWw9MEliz0dALrNb+5tvlmw69/VGRZptP6Wrpllk64fS4bwa7hO63icMTFfheaHJvnVRToTs/nfhx3jDI+/LO2004vYjrwtVd/pRVG8h6L99xA8R5SCN9+vDt0jPBuDTnqRRV1FzH571+oCGap
+api: eJztVk1v2zgQ/SvEnHYB1c4Ge9JplTRFg+62RpL24hjGRBzbbCVSJakkXkP/vRhSsiQ7SbOXnjaX0JzvN/OG2oHHtYN0Dl/QKtTewSIBSS63qvLKaEghk1Kg0PQg7qOOuENHUhgtUAt6VM4rvRZG0+RWX2NJAp2Yfbq+EdPc6JVa3+pbndm1S2+1EEJUuC0MSvFbJmUb9Z015Rk6mkXR76loTyI32qPSHICj7lO4fJuMUtJYUiJQS1GhxZI8WTfhwFeoHHWh39/czC4ecwqlpSLIpFAr4Te969zUhRTaeHFHAqXkUq3APCfnSEan5GurO6+ftTJ6nlVVW80VucpoR4nI9HaRio+DPCV5VIVjj9QlMoEETEUW+celhBRQymVrsVxZUy659iVquYyIQgKWvtfk/JmRW0h3wDiR9nzEqipUHpxNvzpu4Q5cvqES+VRZDuUVOf4V/CrJR7+tCFJw3irNAbzyBV9wX8SlhCYBTQ/7tBjwl8y46BYP8ZF1W/tYwKvMz4Pq3rpvbKhSSsUlYjEbVORtTUnn1Nx9pdwPnM56D00CuSlL5ZclOYfrkAzq7acVpPPDtJpkf6ProoBm0fs8D17EP62XpjkOz61SliTTrAP8CTCP8RnVPIj5LHGg4fh9PIYjXMSBDMCdnpzwvzHHr+sw3qu6EN30QvL6oToC7snasapi6XxoC+wQCPcHcFTWsJcoC8C1gh7EMVqW7pXjgoZYHdFyyf0ctrATCN503cwysiLMfqZlO4oMb5PAn6enxwh+wULJgI+4sNbY/wLfmJNxRfBJeSrdsUJh8pH0FWOrtKc1WWgW/XyitbgdsONvExNkbpRu/RI799PeOXteNYAhblj6M25wXTF0qzfoUQ9vRPf5Mt5G+J4K1qnwM3DkMPa2JL8xvIIr43wgoN9ACtN2NN2U1/EbHkBIwJG9D+tovoPaFpDCxvvKpdNpXphaTnBN2uME1RQrBUfvapCKc1YNK8VRXlvlt8FfNrv8QNv3hJIspPPFUOGaJyfOwlht3was1AdiVOKehaz2G2PVv7HBCShOYBOtuGyeyav+Rbl4xLIqaPRC9K09fgbGstGK70Wj/f3E/u3HFpRemVBNR+II1Rnm30iHTUHWRRRPJn9MTp7DtjUQ2eyy5SPmgY8dLEENkoPu7fsGCVAZyAiesPyrFzBoPCIl6qG/ny2QgzwH7/b/n1m/4jOrpYenRz+tClRh2YXm71qqz7tXyEECPdkXCWx4I6Rz2O345rMtmoavv9dkmbKL9v2644md857ddOTdwTfahm+F0Ow3YRuyelFHsh48Dry4o0WWc/Iv6i4GS4vHgd/H9qOwNJJtLD7wpsUHSCF8arJ12B7hbgcF6nUdORh98t8P8fAyug==
sidebar_class_name: "post api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
-
-
-
-
-
+>
+
Add a new variant based on an existing one.
Same as POST /config
Args:
- payload (AddVariantFromBasePayload): Payload containing base variant ID, new variant name, and parameters.
+payload (AddVariantFromBasePayload): Payload containing base variant ID, new variant name, and parameters.
Raises:
- HTTPException: Raised if the variant could not be added or accessed.
+HTTPException: Raised if the variant could not be added or accessed.
Returns:
- Union[AppVariantResponse, Any]: New variant details or exception.
+Union[AppVariantResponse, Any]: New variant details or exception.
-
-
-
-
-
-
-
-
-
-
-
-
-
-
+>
-
\ No newline at end of file
+
+
+
+
+
diff --git a/docs/docs/reference/api/archive-workflow-revision-rpc.api.mdx b/docs/docs/reference/api/archive-workflow-revision-rpc.api.mdx
index 0c356843c1..6ee704bb22 100644
--- a/docs/docs/reference/api/archive-workflow-revision-rpc.api.mdx
+++ b/docs/docs/reference/api/archive-workflow-revision-rpc.api.mdx
@@ -5,67 +5,1999 @@ description: "Archive Workflow Revision"
sidebar_label: "Archive Workflow Revision"
hide_title: true
hide_table_of_contents: true
-api: eJzNWdtuGkkQ/RVUT4k0MV5rn+ZpvXYie5ONLd/2ASGr6Cmgk55L+mIvQfPvq74BwwALBEs8YZiqU12nqrt6jqegcaQg7cE/pfw+FOWrgn4CZUUSNS+L6wxSQMnG/IWeX4PJs6QXrnhZPMuKQQIVSsxJk7RAUygwJ0ihbc0zSIAXkEKFegwJSPphuKQMUi0NJaDYmHKEdAp6UlkMpSUvRpDAsJQ5akjBGIeiuRbWIK66cxeCdK4zqOu+xVZVWShSFu7s9NR+ZKSY5JVNDFK4N4yRUkMjOnfBGBJgZaGp0NYcq0pw5njoflPWZ7qwxEpaljT3EVhpvFNYOS80jUguLPXCWSSQ0RCN0JCe1kmbJBe3mNwMHZXNGDPjF5QcC20JbZhvZq1OZhaFEQIsSy0enzy0ozGBwwdq4s8yeoNMfASUmg+RHTKF8wAZIxg9LuXBwD1anUCGmrZEtaYfNM9pI/SlBawTyEkpHG3E3gTzd3CvExgKd3is71iunplRusxXBhuUpSAsNka7Vp0Lj1AnFo5eUBjUawjfEvHjDMSDjk2Oxa8AXjmAuk6iUTn4RkyvOKg+OcracK4uGpuLwCzj9vRBcdugdYuyxfOnESkfNH9ZzG51LGBcMiNQvvtkhPhLlcWHG6Mro9/Dcq4WhGvKt/RCKXGykdclX2ixu7lLNTqPVoceE6lfyy9c6f+ldV2SLffdOHrwrRgG9n6nwVd0p05ztO4HdbkAUSfAJKGm7Bn1QU/BCw/bOXdkmSp7iyCPHjYEyUjQGwS59LAhSKRrMDngqItk/TkJwy7yddAoka1ZlEjYQaNEumZRXki2b1zbt+tTcK8TUMKM9oW5t752DB0s0cAhLk+T5mg+XPZG8n1RHiX3CGJ/BGERxoSZe//Y7aR/K0qOryE2XE3uaEiSCtY4beJC+ztNlKtQhjq+y21bkSOayUd50dmhBveBeHujxKrixbYXoGWKd7ltxUCu8HaI79393rtuqgrHen87yl5Z73w7p9QWiuQLZ7QNu16e2aEHA3Tt9JQhHxkvJ71FrItGgG3ewaJWdInuHaH1KrYDxC+6z3SnurZuv5+dtWWqJxQ8c8l1PkpZyv01qow0cjdpQ0suG4iSNZ7usqUWBoXv4nnWX0qG8Vafq9EqfW+FuOBN1ps6MjoP9ukqyufaYs/l5UMHu4YgFen17K5P49LTt6m+Vw8Pty1AX9tmUc+9pNpp6ZfgdIBxaXXXqlTayat6DCl0K6sR0ms3imaqG0VD1Z2uUlvrblBuwW/2KNC66xaMta5U2u0yUZrsBEdUaDxB3sWKu/2liBnJ9cS5nN9ef6aJH++Q9vqLBm7g+AZqms1qhxX/TJbKIA17kYv/9F0RNGF/hXP08mJYOvcoirnFdc5vr2Hh2g6nJ7+dnEKL2kVju1OQuZ0SY7vHkCyxMMsfEqDcbRPQhPkf8wd2ZbYkQSuKeBsqufRSHNjQ9K/uVgJ5sXD39UXuQSgyzLVR5Zo5FBoSSNcI67HW/QTGtnHSHkynA1T0KEVd259/GJK2nP0g7A4sub0pZFzZvzNIhygUtdY9O27g3V3YVe8786HVzCeWvLD1tjqb/QYJfKfJun8KuLNjHHtrGkzPGaNKL4C0jjrbhLPNcntz/wB1/R+GFYkO
+api: eJydVcFu2zAM/RWDpw0Q4qzYyadl3YAF3dCgzbZDYBSqzMRqbUuV5HSZ4X8fKMuOUzcbsFOU6JF6fHxkGnB8ZyHZwE9lHreFeraQMlAaDXdSVcsMEuBG5HKPd88BcmdwL61U1Z3RAhhobniJDg0laqDiJUICA0hmwEBWkIDmLgcGBp9qaTCDxJkaGViRY8khacAdNIVaZ2S1AwZbZUruIIG69lmcdAUBbkLuaJlB26aU0mpVWbSU5WI+p48MrTBSUxmQwG0tBFq7rYvoJoCBgVCVw8oRnGtdSOGrjh8sxTQjZtqQJk52LwhVd0GBsKwc7tCMGF56RMsGGfwT1eF66zUKcer+AYUbhfVd6AuElg3gqi4KaNO2ZZPwo6KbwC09n3Mov20p1/uLi6laP3ghM69F9NkYZf5fqgwdlwWdpMPSTgGFEie3E5GCG0ZK9HK36VELbgw/jJT8qjqC1IPS7l5zVw/9htbyHcKQ7DzUixGt6fZffaC6uqcDbtSSo7yduufL+NTJ99pjPeTLer2aJOx6e9rURTfHUW+IaHAZgxJdrmjYtbLOz7TLIYFYk3/xOb7nFuN+Adi4d7WNm9Gct3FYFcDAotn3G6E2BSSQO6dtEseiUHU24zusHJ9xGXMtYULV30aXBAUacIuiNtIdfL7FanmFhy/IMzSQbNIx4JZ82DnrFDY0lWt5haRxWFSL2uXKyN+dXcKqyrsoElFWW+XDg96B20cuHrGirUR1drTns3ez+bliQkC0WC3DOHHhx6nn4WHAXsg1CAUMsPSzBA55+eF4QSypbyWvxvn+0u4TgoMyDn+5WBdc+qnxNJrghA0EJwAD8gIwGNwAxy1H5+R08/eWSBnk5K1kA01DKb6bom3p56caDTU2ZbDnRvJ7knnTQCYtnTNItrywOGE9bCR4cxMG720E7PVq+uZX1Pk9L2r6Bgwe8fDiz8pvlbw3VxMQCyFQu1HsZAmSC4cxWl3frqFt/wCO9opx
sidebar_class_name: "post api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
-
-
-
-
-
-
+
+
Archive Workflow Revision
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
+
-
\ No newline at end of file
+
+
+
Path Parameters
+
+
+
+
+
+
+
+
+
+
+
+
Successful Response
+
+
+
+
+
+
+
+ Schema
+
+
+
+
+
+
+
+ revision
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ author
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ date
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ message
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ name
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ description
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ version
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ slug
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ flags
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ is_custom
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_evaluator
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_human
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ metadata
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+ data
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ service
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+ configuration
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+
+
+
+
+
+ artifact_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ artifact
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ name
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ description
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ slug
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ flags
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+ metadata
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+
+
+
+
+
+ variant_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ variant
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ name
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ description
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ slug
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ flags
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ is_custom
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_evaluator
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_human
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ metadata
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+ artifact_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ artifact
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ name
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ description
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_at
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_at
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_at
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_by_id
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_by_id
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_by_id
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ slug
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ id
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ flags
+
+ object
+
+
+
+
+ anyOf
+
+
+
+
+
+ is_custom
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_evaluator
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_human
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ metadata
+
+ object
+
+
+
+
+ anyOf
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Validation Error
+
+
+
+
+
+
+
+ Schema
+
+
+
+
+
+
+
+ detail
+ object[]
+
+
+
+
+
+ Array [
+
+
+
+
+
+
+ loc
+ object[]
+
+ required
+
+
+
+
+
+ Array [
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+ integer
+
+
+
+
+
+
+ ]
+
+
+
+
+
+
+
+
+
+ ]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/docs/docs/reference/api/archive-workflow-variant.api.mdx b/docs/docs/reference/api/archive-workflow-variant.api.mdx
index 03cbc2b65e..5be93e8fde 100644
--- a/docs/docs/reference/api/archive-workflow-variant.api.mdx
+++ b/docs/docs/reference/api/archive-workflow-variant.api.mdx
@@ -5,67 +5,1008 @@ description: "Archive Workflow Variant"
sidebar_label: "Archive Workflow Variant"
hide_title: true
hide_table_of_contents: true
-api: eJzNV9tu20gM/RWDTy0wjdJgn/S03qRFvO1ugsbtPhhGQEu0Ne1Io84lWa+hf1/MRbZl2W6aukCfchF5DnnI4XBWYHChIZ3AP1J9mQv5qGHKQNak0HBZjXJIAVVW8Ae6f4wm9w+oOFYGGNSosCRDymGsoMKSIIVdw3ueAwNeQQo1mgIYKPpquaIcUqMsMdBZQSVCugKzrB2ENopXC2Awl6pEAylY61EMN8IZtPEOPgWOwSiHppk6aF3LSpN2aBfn5+5HTjpTvHYZQQp3NstI67kVgw/RGBhksjJUGWeOdS145gVIPmvns9qKsFZOHsMDQyZtcIqB88rQgtRWpJfegkFOc7TCQHresJ5EnrZa3sy9jl2KtS3Pu2bHtWrY2qKyQoATp6eeU40BKsPnmJkTMgwjZGSYC99nh5Pk+j6z2shybwAzKQVhdZRxpAeXAaFhDo4eUFg0Uv0I4ps1SAAtbInVjwBee4CmYa2RnH2mzOzp7Ldesj5cw6Akg90gMM+561cUtx1ZD5VyC7bt2A5TOev+Zzu7/VyQcZVZgerFWyvEn1pWr26sqa15Cbu5OhBuqHyiFyqFy6O67vhCT91jzn85MZ1Hr0N/JVH/lu+5Nt+U9VCSPffv02gcWjEO+Ccp0A/B+TasO4yfB3W1BdEwyBShofwezROnV46GXhnu4znMchlgB0Mvlq3zn0HyMcBGkpwE/QSSqwAbSVq5ZssTzvtWrD+WceK3ep2UpVVrzdIKdlKWVq41ixZ28dxevXO+7vI4WXh+0/n2/REXo303yNO91wtS0ziv3y4u+vvUJxQ899vS4I1SUj1/mcrJIBd+HQjXw66BkFnn6/dM4mmzc6Nskn4vQ4CuTqVe7NtDN7eF1rigzfg8bOrFGIzd132Kb3bgic8rUEe7rXJv5A3qHk7jKsh3rLzX4/FtDzDUtlvUYVj6B7t7Nvjto5DuYVBLHR4BpoAUklrRA6fHpN1XdRKXW52s9jwJmiQ+LICBJvXQPiKsEpBCYUyt0yTJhLT5GS6oMniGPMGa+8OgKbOKm6V3Gd6O3tHymjAnBelkum1w53otdE/XbF04rPk7cjrG58vQmkIq/l9oifhwKYKXE4pXc+nd21XXBzcY3o6AgUsjKHh+9vrsHHq6bhu7Y4KZPyYtt/8MbEeFdf7AgEp/RsAQlr9vPrjIXEHiftriHS7jzj0cxTD0r0lqgdwfBh/FKlZ4ArHCsHm/aJdyrDIwSPc//dpCTxkUrmfSCaxWM9T0UYmmcf/+akm5Wk4j3MwpO1lBzrX7PYd0jkJTL+r1oIEXH+J5ejnYbEbdbNp6V0sftbDuL2DwhZYHXq1+aBRtX62i5TDLqDZbGL0Z5xpwfUxub+7G0DT/A3FdSl8=
+api: eJydVcFu2zAM/RWDpw0Q4qzYyadl3YAG3dCgzbpDYBSqzMRqbcmV5HSZ4X8fKMuOMzcYsFOU6JF6fHxkGnB8ZyHZwE9tnreFfrWQMtAVGu6kVssMEuBG5HKPD68B8rDnRnLlgEHFDS/RoaEcDSheIiQQ7h9kBgykggQq7nJgYPCllgYzSJypkYEVOZYckgbcoaJI64xUO2Cw1abkDhKoa5/FSVcQ4L5LHS0zaNuUMtpKK4uWklzM5/SRoRVGVsQfErirhUBrt3UR3QYwMBBaOVSO4LyqCil8ufGTpZhmRKwyJIaT3QtC111Q4CuVwx2aEcFLj2hZL4J/QR1utl6gEKYfn1C4UVSvfigPWjZgVV0U0KZtyybRRzk3gVl6NuVQe9tSqo8XF1Op7nkhMy9E9NUYbf5fpwwdlwWdpMPSTgGFFie3E4mCE0ZC9Fq36VEKbgw/jHT8pjuC1IDS7t5yVg/9jtbyHcKQ7DzUixGt6fZfbaC6uqcDLh17t5e3U/d8GV86+d56rIdcrderScKut6dNXXTTG/V+iO6H6S3R5ZomvNK2m2aXQwJxZXAv8TV+5BbjfuptHBxt4+Y44G0ctgMwsGj2/SaoTQEJ5M5VNoljUeg6m/EdKsdnXMa8kjDh6W+jS4ICjbZFURvpDj7fYrW8xsMV8gwNJJt0DLgjE3a2OoUNHeWVvEYSOCyoRe1ybeTvzithR+VdFCko1Vb78CB24PaZi2dUtI6ozo72fPZhNj9XTAiIFqtlmCUu/Cz1PDwM2F9yDUIBAyz9IIFDXn46XhBL6lrJ1Tjf+V6f8BuEcfjLxVXBpZ8Yz6IJNthAsAEwICMAg8EKMKw3OiYn+773Q8ogJ1slG2gaSvDDFG1LP7/UaKiracjySBpvGsikpXMGyZYXFiech10E727DyL2PgL1dS995dfBki5q+AYNnPJz+Q/l1kvfGagJgIQRWbhQ62X7kwGGAVjd3a2jbP8qKgps=
sidebar_class_name: "post api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
-
-
-
-
-
-
+
+
Archive Workflow Variant
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
+
-
\ No newline at end of file
+
+
+
Path Parameters
+
+
+
+
+
+
+
+
+
+
+
+
Successful Response
+
+
+
+
+
+
+
+ Schema
+
+
+
+
+
+
+
+ variant
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ name
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ description
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ slug
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ flags
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ is_custom
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_evaluator
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_human
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ metadata
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+ artifact_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ artifact
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ name
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ description
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ slug
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ flags
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ is_custom
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_evaluator
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_human
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ metadata
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Validation Error
+
+
+
+
+
+
+
+ Schema
+
+
+
+
+
+
+
+ detail
+ object[]
+
+
+
+
+
+ Array [
+
+
+
+
+
+
+ loc
+ object[]
+
+ required
+
+
+
+
+
+ Array [
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+ integer
+
+
+
+
+
+
+ ]
+
+
+
+
+
+
+
+
+
+ ]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/docs/docs/reference/api/archive-workflow.api.mdx b/docs/docs/reference/api/archive-workflow.api.mdx
index 9b7e347ed3..88c5714e4a 100644
--- a/docs/docs/reference/api/archive-workflow.api.mdx
+++ b/docs/docs/reference/api/archive-workflow.api.mdx
@@ -5,67 +5,598 @@ description: "Archive Workflow"
sidebar_label: "Archive Workflow"
hide_title: true
hide_table_of_contents: true
-api: eJzNV01v2zgQ/SvGnFqAjd1gTzqtN2kRb7sbo3G3B8MIaGlssaVElRw66xX03xekvi07ddMU6Mm2NPNm5s2Q85wD8a2BYAmflP6ykerBwIqBylBzEiqdRRAA12Esdnj/UJkAg4xrniChdr45pDxBCKA2uBcRMBApBJBxioGBxq9WaIwgIG2RgQljTDgEOdA+c66GtEi3wGCjdMIJArDWo5Ag6Qzq/EazCIpi5SBNplKDxqFcTibuI0ITapG5zCGAOxuGaMzGytGHyhgYhColTMmZ8yyTIvSFjj8b55N3Msu0o4FEGSFUtnSqEhYp4RZ1J8Mrb8Egwg23kiCYFKyhxIdL97cbz1cfeiN9D04bCHMfWkMq6RtVmayVkshTKFjzKLVSgiOpzm1mRlclQsEcHO64tJyU/hHENw1ICRrbhKc/AnjjAYqC1UZq/RlDOjIFbz1lQ7iCQYLE+0nwKBKux1zOe7QO0qyGsANbd7kXKVn3n3SrOx4LQqFDK7l+8dZK+adR6atbS5mll3BYqwMRhMmZXlxrvn+U1wNfGLD7mPNfjkznMZjQX4nUv9V7YeibtJ4qcuD+fRwtylGsLsGzGBim4HwL1r/AngZ13YEoGIQaOWF0z+kxwM69G3HCVyR8PqejXJWwo6kny2bRzwjysYStgkQo8ScEuS5hqyA1Xeu9W2LnxfGb6hyy/tj77dXy9axRaraaKDVhzxqlpquJYqTdPnVW75yvWx7Plp5XB9/eH8dWxxlujYwoCmf+2+XlUHX8w6WIvKYYvdFa6adLjgiJC+kFQLkQDg2kCntvv+fuXRUHO6St9r0qE3SdScz2mEpr94MxfIvthXna1JMxWri3x6huFeLS11WGruw6DW7pLdk9XcZ1Sd9jfb1ZLOYDwLK3/aZOSwk8+tRK4AQpVk4eZ8qQl8QUQwDjTONO4MO4Vn5mnHd0cTGu1DQwMKh3tYK2WkIAMVFmgvE4lMpGF3yLKfELLsY8E37KDYZWC9p7l+l89g73N8gj1BAsV12DOzdS5ZD0zZr+8Ey8Q0dXpd2nlmKlxX9l5yv1Hpdejg+RbpR3r6ib+uRG0/kMGLgySqImF68vJjCgr2vsTgMP/WmoY/vXwA5YaOoHBpj4owCEPPm9feEyc/xXwrPGG3brYLFWJBD+S+NMcuFn3UfPq0YuoWoktCLeAIOg/yen7uaKQezmIFhCnq+5wY9aFoV7/NWidg1bMdhxLfja0bfMIRLGfY8g2HBpcJBic2nAiw/V2Xg5anVNP/W6qanrqJPl7hcw+IL7g/9l/uDH9dDklcU0DDGjju/gnnLT1Yz8/PZuAUXxP5y63go=
+api: eJydVcFu2zAM/RWDpw0Q4qzYyadl3YAG3dCg7dZDEBSqzMRqZcuV5HSZ4X8fKMmOM7cYsJMN65F6fOSjW3B8ZyFbw502T1ulXyxsGOgaDXdSV8scMuBGFHKP9y8RAgxqbniJDg3FtlDxEj3QyS0X7l7mwEBWkMFzg+YADAw+N9JgDpkzDTKwosCSQ9aCO9QUa52R1Q4YbLUpuYMMmsancdIpAixi8mSZQ9dtKKWtdWXRUpaz+ZweOVphZE3UIYObRgi0dtuo5DqCgYHQlcPKEZzXtZLCV5o+WoppR8xqQzo4GW4QuglBkbCsHO7QjBiee0THYBCKrqgOV1svUozTD48o3CisV74vEDo2gKtGKeg2Xccm4UdF15HbZppzKLvrKMfHs7OpSj+5krnXIPlqjDb/L1GOjktFb9JhaacApcXJ6UScOAUjBXqZu81RA24MP4wU/KYDQdK+tLvXpqqHfkdr+Q5hSPY21IuR3NLpv/SnusLVETdqxVHeoO7bZXwJ8r12WQ+5uL1dTRKG3p42dRE8m9wdPVuiKzT5udbWeQ+7AjJIa4N7iS/pA7eY9qNr07Z/vZd5l8YdAAwsmn3v+8YoyKBwrrZZmgqlm3zGd1g5PuMy5bWECS9/mpwTFMjFFkVjpDv4fIvV8hIPF8hzNJCtN2PADQ1dGKNT2NBBXstLJEHjOlo0rtBG/g6zERdSEaJIMVlttQ/vN0zg9pmLJ6xo9VCdgfZ89mE2f6uYGJAsVsvoHXJxNqzFAAP2l1yDUMAAS28ccMjLT8cDYkndKnk1zjft7QmvQRCHv1xaKy69M/ztbWz7GmLbgQE1Ho5bywKDbNR8YP0fgMa6oOHJ1tC2FPbDqK6jz2HPUw9zafmDIl9subI44TbsFnh3HS30PgH2OucnPEz+K3uuGkL68dlzI+k6Py2s7y7xCKELIbB2o6jJPqMsgzVWVze30HV/AHk2bqA=
sidebar_class_name: "post api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
-
-
-
-
-
-
+
+
Archive Workflow
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
+
-
\ No newline at end of file
+
+
+
Query Parameters
+
+
+
+
+
+
+
+
+
+
+
+
Successful Response
+
+
+
+
+
+
+
+ Schema
+
+
+
+
+
+
+
+ workflow
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ name
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ description
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ slug
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ flags
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ is_custom
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_evaluator
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_human
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ metadata
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Validation Error
+
+
+
+
+
+
+
+ Schema
+
+
+
+
+
+
+
+ detail
+ object[]
+
+
+
+
+
+ Array [
+
+
+
+
+
+
+ loc
+ object[]
+
+ required
+
+
+
+
+
+ Array [
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+ integer
+
+
+
+
+
+
+ ]
+
+
+
+
+
+
+
+
+
+ ]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/docs/docs/reference/api/assign-role-to-user.api.mdx b/docs/docs/reference/api/assign-role-to-user.api.mdx
index 51414d68f8..5f0160cd87 100644
--- a/docs/docs/reference/api/assign-role-to-user.api.mdx
+++ b/docs/docs/reference/api/assign-role-to-user.api.mdx
@@ -5,79 +5,359 @@ description: "Assigns a role to a user in a workspace."
sidebar_label: "Assign Role To User"
hide_title: true
hide_table_of_contents: true
-api: eJztVt9v2zYQ/leIe0oBzs6CPelp3pqiRjckcJzuwTGCi3S22EqkSlJJPEH/+3CkZMl263bPW15ikffzu7vv2EBGLrWq8spoSGDmnNpqJ1BYU5DwRqCoHVmhtEDxYuxnV2FKkwf9oGd265IHLYQQFe4Kg5m4uHdkF6agN4lY5rQ/T432qLTSW+FzEsZuUau/kZ0KlcnogkpUhRSos8F5CGcSney9P6pMXDhvOyfzt8Jsgt1xfKxh6UtNzouLRfzRKbxD52e38/21efpEqQ85LcjXVvdpPRlTJGJpaxIqegiBvaATrk5Tcm5TF8WuC5MyKd5h4UgYn5N9US7itEDlqDf5frm8vX5NKQCeiHk0G/LPDDmhjRc5PpOoyJbKOUbIG/7aGFsKnysnMGXlyRmDlgTLaUHWGtvF16Pfg9s7noAEU5EN5ZhnkECUf2TBR28eWQgkVGixJE/WQbJqQGNJkMC4KiBBcRdV6HOQwPgqSxkk3tYkwaU5lQhJA35Xsa7zVuktSPDKF3zwV29MzDNo23W0Qc7/ZrIdKx6b5MYi7fkKq6pQaUhi+slxNzcjj5XlFL0ix1+h1c4Fch0EWgnjVuUMz+jcjLua45fACIbY9O5mE1A7VG7l/kTXRQGccW+OxwjaVvYCsUsPYF11iZyGObLTjyS0LVuz5CqjXcTh6vKS/x1ywN2+tcWiE4YfRzp6+eXq6tTwRyxUFuG55r78F1aP6peR7wqoPJXuVKAw6cHtDxRAaU9bstCuB8zRWtyNSvyHiQFybUu3PdcNf5JzuCXYGzvTbGFIl3z7vXpzXtF1Jzcq9ABvRPfbabyN8H3NWS/CtHJiMNa2JJ8bponKOB94weeQwHTPBG7ajFmhnfIYOJDgyD739FHbAhLIva9cMp2mhamzCW5Je5ygmmKlwjA4Smur/C6ozG7nH2j3njAjC8lqPRa4406JtT8U28OOlfpAjEJHXLPa58Z2M9MzVx61OE3uwcXAPtevWFZxmjv2GAp5QhLDVWSAod1A6Y0JUXVAz0LOYnY7BwmMTpyVy8nPk0uQx+t5LByWahpmp08pXIM8AncPK8h97J6w/HW44IS5nCXqsb2wBwSzh1gacR/3wEFIzTDC/z8e/qOPh27APL36aVWgCvQYWrDpyGE1PBOYB5KjR0Pkh7WEnBklWUHTPKGje1u0LR9/qckyBawlPKNV+MSTs2ogU45/Z5BsGLozrRmKyCT6Rnwr4J4mNHPEMxY1f4GEz7Q7fuaEFZH3PNR0Ir9Hbz8FIh9MnOw13jlRY5Yy9Gdl1yO+vb25W4KEp+4tVJqMdSy+MIT4EsM1IftAhOGsgQL1tuZVlEC0yX//ALNiIbI=
+api: eJztVk1v4zYQ/SvEnLKAaqdBTzrVu5vFGtsihuNsD44RjKWxxY1EakkqjivovxdDSpZsN+723PpikZzPNzOPrCElmxhZOqkVxDCxVm6VFSiMzkk4LVBUloyQSqDYafNsS0xo9Kge1cRsbfyohBCixH2uMRVXD5bMXOf0LhaLjA77iVYOpZJqK1xGQpstKvknslMh0yi4oAJlHglUae/chzMKTg7en2QqrqwzrZPpR6E33u4wPtYw9L0i68TVPHy0Cp/QuslsejjW62+UOJ/TnFxlVJfWWus8FgtTkZDBgw9sh1bYKknI2k2V5/s2TEoj8QlzS0K7jMxO2oDTHKWlzuTnxWJ2+5qQBzwW02DW559qskJpJzJ8IVGSKaS1jJDTvNpoUwiXSSswYeXRBYOGBMspQcZo08bXod+B2zkeQQS6JOPLMU0hhiD/xIJPTj+xEERQosGCHBkL8bIGhQVBDMOqQASSu6hEl0EEjK80lELsTEUR2CSjAiGuwe1L1rXOSLWFCJx0OW/80RkT0xSaZhVskHXvdbpnxVOT3FikHB9hWeYy8UmMv1nu5nrgsTScopNkeeVb7VIgt16giWDYqpzhBZ27YVdz/BEwgj42tb/beNSOlZvosKOqPAfOuDPHYwRNE3UCoUuPYF22iZyHObDTjSQ0DVszZEutbMDh5vqa/4454P7Q2mLeCsOPIx28/HJzc274K+YyDfDccl/+C6sn9UvJtQWUjgp7LpDr5Oj0BwoglaMtGWhWPeZoDO4HJf5NhwC5toXdXuqG38la3BIcjF1oNj+kCz79p3pzXsF1KzcodA9vQPftND4G+P7OWSfCtHJmMNS2IJdppolSW+d5wWUQw/jABHZcD1mhGfMYWIjAknnp6KMyOcSQOVfaeDxOcl2lI9yScjhCOcZSQnR6N/lT8YFF/aRYSioj3d7bm8ymX2j/mTAlA/FyNRS45zYKjXEsdqgJlvILMUQtq00ql2nTDlRHa1nQYgy4Qec9Nd2+YlGGUW+ppa/yGYP0R4Ee+l4EqTbaR9VWoU35PSbPpJheGb6AxvXo59H1Wxi1CmIym7ZDhokfsi49LwbRSRUO+EN0yMMRFr/2B5w8171ANbTnLwzBNCMWWjyEC+MotLqf9f9fGf/RV0Y7bI5e3bjMUXoe9S1Ytyyy7N8TTBjxyesiEMkqgoypJ15CXa/R0oPJm4a3v1dkmA5WEbygkbjmKVrWkErL3ynEG4buQmv6IjLbvhNvBdxRhmK+eMG84hVE8Ez70/eQv0uyjpPqVuRD8PaTZ/zexNkFyJdT0JgkDP1F2dWAmGd39wuIYN0+mgqdso7BHUOIuxCu9tl7UvR7NeSothXfWTEEm/z7C72UMEE=
sidebar_class_name: "post api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
+>
-
-
-
-
+>
Assigns a role to a user in a workspace.
Args:
- payload (UserRole): The payload containing the organization id, user email, and role to assign.
- workspace_id (str): The ID of the workspace.
- request (Request): The FastAPI request object.
+payload (UserRole): The payload containing the organization id, user email, and role to assign.
+workspace_id (str): The ID of the workspace.
+request (Request): The FastAPI request object.
Returns:
- bool: True if the role was successfully assigned, False otherwise.
+bool: True if the role was successfully assigned, False otherwise.
Raises:
- HTTPException: If the user does not have permission to perform this action.
- HTTPException: If there is an error assigning the role to the user.
+HTTPException: If the user does not have permission to perform this action.
+HTTPException: If there is an error assigning the role to the user.
-
+>
-
-
-
-
-
-
-
-
-
-
-
-
-
-
\ No newline at end of file
+
+
diff --git a/docs/docs/reference/api/configs-fetch.api.mdx b/docs/docs/reference/api/configs-fetch.api.mdx
index fcf353689e..d6eede6c4b 100644
--- a/docs/docs/reference/api/configs-fetch.api.mdx
+++ b/docs/docs/reference/api/configs-fetch.api.mdx
@@ -5,36 +5,36 @@ description: "Fetch configuration for a variant or environment."
sidebar_label: "Configs Fetch"
hide_title: true
hide_table_of_contents: true
-api: eJztWVFv2zgM/isCX7oBbtN1W3dn4IDrtg4rtqFB2u2lCQpFomPtbMmT5La5wP/9QMlJnTTLhg43tED6UkciKZIfRVLSDDyfOEgv4Au3imvvYJSARCesqrwyGlJ4h17kTBidqUltOY2yzFjG2VXkYcYy1FfKGl2i9ntDPdTHyudo5xSXFjN2OuhShaGydp6NkVXWXCmJkj1RGdMYeZVbjCdDzZnEjNeFvyPkWvmcuaKe/DWEyhpZC1JxCOxaFQVJrx3Kp0Grd6QpFzmzmKFFLZCZ8VcUnj3paJqsLpEwXlWFEsF2GniaDvUu60ftWKvvDumwQ87YUXKHZQoLSWQ7V2idMnqHLDLBq7xgXEsmuCb9HHrmDdN1URD9SRYFdO1nyi+Zw6SyKHwxJcYsAORzZBadqa1AEnNKSl0rh8lcsyUBvDB6En23Yhw5aoC+ttqlQ80YY28C9gN0ldEOPxmJRcrOc1wTFFGLbzU6j3JTgAy4cjhf4P35ef/4RmBwTkoe8HekK8e08SwztZZ7kICpMM6cSEgh0rrL4AtIoNXhtZFTSGc07VF7+uxY2/vqKMJn4ESOJaevypJcr9DRr05QBFY9Pc0gvVglI/cuz/tphZCC81bpCTTJYoRAhmaUgFe+oIEz4m0SaINkrRilPU7QbpTzpeVvEhCmLJW/LNE5PsH7KvYmSGGfWilNAkpukpVAZmzJPaRQ10pulH0ioWmS+XTcgnA7PZjvzkFEMUTcGoFNAis7dYvSg0RpJcNsUXqAKG2QQVn0cjnDNg0x2LYkBOQO9vfp33LzcFYLgc5ldcHm9QOS+6bjiltehi8upYqFtN+h8LbG71vRj9xNArUt7ovkZ0vO2wb07w9oqujKoqRmVUkYrQnxt+en6/OPQ3ulBG6hevhQbVuuRwPVtvd6VHB1a1ahMhRTUeAm0IRF7lFecn9vD0UJ7MiHslvJX5T3OUpYkTeeXm52/M+IfD1l5POu1F8XubEz+4gTLqYf50j8sHhtMXs8mM2r2Bazx4NZt5xtcXssuP2gSLZHxqWqfecysz3Nvjg4uHuA/cILJeMN5LG1xt7/9CrRcxUOnspjPMYuExRGLM3+hG8Xfc/o1g/cWj7t+tFEBQm80oU2bLUvmZN2WplI8n3S4Ax2TrM/woDsiku3dN2mbOHe6N3vm/E2um9TxNA18h2BEdsSfW7oorgyjlgq7nNIodcmatdrrzd68wtkqrpoXXB/uC+A3PvKpb2eKEwt9/gEted7XPV4pUJ4OxS1VX4aWI76Jx9w+h65RAvpxahLcEYBEiFfJlt4m1fqA5Lxmpf0+6j2ubHq34hjAooiM49cZB2F3uD2xvv4hpdVzF0rx6nYh9+iueip9+82xbdUlD3gecb/eJkdvth9+erZq90XLw8PdsfPM7F7IP48fJ4dHvKMH8L6U8FvWHXNhcz/vio1/DozAbc2Ao9CVLCj/kl3Qdjfe7a3D6sPa0vElFS4CEllDnqYhmQl/BaBBwlgGTIKeOTl37cTpBnFecl1R15MfI69ayN8SZnOE8n2wW/74PcQH/za3OjxxveqgivducqN6fxinu5c3FAU7nTqDgE/SiCn3J9ewGw25g4/26JpaPhbjZay9qjt28e0lS+ooubz/D2Df3Aa9xBtkt1Q94i8qGO+XmkDqERHjiNBVm6kHXXKU//07BwSGLfvlqWRxGP5NdVUfg0phKdP4g4FJIzNoOB6UsdMFmXS33+QQ/Ta
+api: eJztV01v2zgQ/SvEXNwCSpzmq7sCFti0TdCgLWI4bi+2EdDkyGIrkSpJJfEa/u+LIeVEdpw0bdPFHppLZGpmOHzz5g01B8+nDtIhfOJWce0djBOQ6IRVlVdGQwon6EXOhNGZmtaW0yrLjGWcXUYfZixDfams0SVqvz3SI32sfI52aXFhMWNn/bZVWCpr59kEWWXNpZIo2TOVMY3RV7mb9WSkOZOY8brwd4JcKZ8zV9TTv0ZQWSNrQSmOgF2poqDotUP5PGR1QplykTOLGVrUApmZfEbh2bNWpsn6FgnjVVUoEc5OC8/Tkd5ivZgda/LtUA4dAqOjZIdlCgtJZp1LtE4Z3aETmYAqLxjXkgmuKT+HnnnDdF0UZH+axQDt8zPlV47DpLIofDEjxywUyOfILDpTW4EU5oySulIOk2VmKwF4YfQ0Yrd2OAKqj7622qUjzRhjr0Pt++gqox1+MBKLlA1y3ECKmMXXGp1H+RBB+lw5XG7wdjDoHV8LDOCkhIC/E105po1nmam13IYETIXxzamEFKKtuwhYQAJNDq+MnEE6p9cetafH1mm7nx0xfA5O5FhyeqosxfUKHf1qkSK46tlZBulwDn5WIaQQyQMJeOULWugvedWP+wesYJHceFCRYTFeJLDGsSePv1bVJ46/SO6NQZhfrNZjsSAH2xAoQLu7s0P/VqXmvBYCncvqgi3ZBsmPFq/ilpfhiUupYtv1Whbe1nj/KXrRe5FAbYuN4DlvlZ5uAOc2yEdL4H1fKYi5yqIkUVYSxhuK82ZwtrnmDu2lEvhrN3l0U/zMJt/VHT+zUbs2hcpQzAR5PqZZ3i/Nv1mPJw+8rMGTB27j/sTBN0hGu3BNu7aKt2HsNEqyv7t7Vzw+8ULJOCuOrTX2x5VDoucqNL3yGCVk1aAwYuXtI7RBaY9TtBHlZo1by2dtBE1MkDSjdFOKvBrv1vQDOsenCDfB7jcNYLABvf1WDehccevGrlWNW3gjuvcf402E76EZQQP/TsBY2xJ9bmikV8aRS8V9Dil0G8q7bjNaustRT02G1gX4g1ZD7n3l0m5XFKaW23yK2vNtrrq8UrB+uz0Kb9lrMg3a7VDUVvlZiHfUO32Hs7fIJVpIh+O2wTmxJ/Jh1eymFLxS75CQ0byk30e1z41V/8QiJ6AogTx60dGJl/3bi8vxNS+r2H1rmkv3uXapmxsmpDvE+bJU/qJsCNKyUoTqXsb/OMgO97cOXr54ubV/cLi7NdnLxNau+PNwLzs85Bk/hM0C/B/sumFS/vJdF1SIzIS6NfRsSPGKiy+oZXtT2Nl+sb1zH4saB3bUO23Uh4ugPksCBDNI1nh6w1BIAMsgPeCRl3/fvqAsqSFKrlvxokI6dtK0wkpSrVvv72+4399w/8dvuEYnPV77blVwpVv37aj7w6X0udhQRHdIIGr/OIGchkQ6hPl8wh1+tMViQctfa7Sk4OPmqjShth7S6M2XWj6HLziLPURNshUGJJkXddTutfsCzfLocSTolA/ajltzrHd2PoAEJs2naGkk+Vh+RcOXX0EK4WuWvMMwCWtzKLie1lHVYkz6+xfaISQU
sidebar_class_name: "post api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
+>
-
-
-
-
+>
Fetch configuration for a variant or environment.
@@ -42,45 +42,2082 @@ Either variant_ref OR environment_ref must be provided (if neither is provided,
a default environment_ref with slug="production" will be used).
For each reference object (variant_ref, environment_ref, application_ref):
+
- Provide either 'slug' or 'id' field
- 'version' is optional and can be set to null
- If 'id' is provided, it will be used directly to fetch the resource
- Otherwise, 'slug' will be used along with application_ref
Returns:
- ConfigResponseModel: The configuration for the requested variant or environment.
+ConfigResponseModel: The configuration for the requested variant or environment.
Raises:
- HTTPException: If the configuration is not found.
+HTTPException: If the configuration is not found.
-
-
-
-
-
-
-
-
-
-
-
-
-
-
+>
-
\ No newline at end of file
+
+
+
+
+
diff --git a/docs/docs/reference/api/configs.tag.mdx b/docs/docs/reference/api/configs.tag.mdx
index 079f90d7a7..5debe215a4 100644
--- a/docs/docs/reference/api/configs.tag.mdx
+++ b/docs/docs/reference/api/configs.tag.mdx
@@ -5,16 +5,9 @@ description: "Configs"
custom_edit_url: null
---
-
-
-
-
-
-
```mdx-code-block
import DocCardList from '@theme/DocCardList';
import {useCurrentSidebarCategory} from '@docusaurus/theme-common';
```
-
\ No newline at end of file
diff --git a/docs/docs/reference/api/container-templates.api.mdx b/docs/docs/reference/api/container-templates.api.mdx
index 2c68b3ec1f..40bc78e698 100644
--- a/docs/docs/reference/api/container-templates.api.mdx
+++ b/docs/docs/reference/api/container-templates.api.mdx
@@ -5,59 +5,69 @@ description: "Returns a list of templates available for creating new containers.
sidebar_label: "Container Templates"
hide_title: true
hide_table_of_contents: true
-api: eJytUsuO2zAM/BWBZyFOe9SpQVG0wV4Wu9tTYBRchYnVypIq0WlTw/9eUMk6j/PqYsgcUjPDGYFxX8Bs4HMMjC5QLtBq2FKx2SV2MYCBJ+Ihh6JQeVdYxZ1i6pNHpqLwgM7jqye1i1nZTMgu7FWgP8rOIxegISbKKAPXWzAw137Mo0BDppJiKFTAjPBxuZTPLZXnwVoqZTd49XQGg67TKLDAMSXvbH2o+VmkZ4RiO+oRzDjJ0dATd1FY7IlBQ0LuwEBz4dtckyqUD2KL2YwwZA8GOuZUTNNYH4ftAvcUGBfoGkwOplY67JAdH2vL6nH9QMdvhFvKYDbtNeBZiJ3U3sJG4GMiMIDJPdARNATs5b4auIvZ/asCQYMTU7pTl0hzYRdru2Nf8ZWcWj2uQYPIONm4XHxYLOF+zzfg6oatnr69Xcug71yY9YMG6tFJjQn7T5eCMEuxcI/hat4cOfVyZfcNpfGy2vdJ4dlWpr/cJI8uwHTWM55zsLlkU+hcktBq6GJhQYzjKxb6nv00ye/fA2VZdqvhgNkJj7pp/bYZycEvOoqH1lKS0B3QD6cF3+VV8jEH9OuXF5im/3FuQv8=
+api: eJytkkFv2zAMhf+KwLMQZzvqtKwYtqCXou1OQTCwChNrlSVNorNlhv/7QMe1kwC71ZfAIfn88fF1wHgoYDZwFwOjC5QLbDXsqNjsErsYwMAjcZtDUai8K6ziXjE1ySNTUXhE5/HFk9rHrGwmZBcOKtBvZSfJBWiIiTKK4HoHBqbaj0kKNGQqKYZCBUwHH5dL+blGeWqtpVL2rVePYzPoQY0CSzum5J0dPlT9LDLTQbE1NQim6+XR0BDXUSgOxKAhIddgoJp5q0uoQvkotphNB232YKBmTsVUlfWx3S3wQIFxga7C5ODWu9VQVXfSCv1W5GybHZ8GvdXD+p5O3wh3lMFstpcNT0J9tuK6rQM+JQIDmNw9nUBDwEbeVy3XMbu/w/agwQlAfZ6SvV3Yx2HcsaeZ7TPaVwo70CB7nrGXiw+L5f+WGQfU6mE9mo92MP+NY2gDfWPXZBRooAad1Jiw+TQXhDLFwg2GC70pm+r54i5XaN2cgfeJ62gx0x+ukkcXoB/36cbAbOYQC84cma2GOhaWjq57wULfs+97+ftXS1kOv9VwxOyEY7i6fruSZOKVTuKhtZQknUf07fnYN8GWrExJ/vrlGfr+H3cdUY4=
sidebar_class_name: "get api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
-
-
-
-
-
+>
+
Returns a list of templates available for creating new containers.
-
-
-
-
-
-
-
-
-
-
-
-
-
-
\ No newline at end of file
+
+
+
+
+
Successful Response
+
+
+
+
+
+
+
+ Schema
+
+
+
any
+
+
+
+
+
+
+
+
+
+
diff --git a/docs/docs/reference/api/containers.tag.mdx b/docs/docs/reference/api/containers.tag.mdx
index 0e073f7378..cd25ac1df8 100644
--- a/docs/docs/reference/api/containers.tag.mdx
+++ b/docs/docs/reference/api/containers.tag.mdx
@@ -5,16 +5,9 @@ description: "Containers"
custom_edit_url: null
---
-
-
-
-
-
-
```mdx-code-block
import DocCardList from '@theme/DocCardList';
import {useCurrentSidebarCategory} from '@docusaurus/theme-common';
```
-
\ No newline at end of file
diff --git a/docs/docs/reference/api/create-account.api.mdx b/docs/docs/reference/api/create-account.api.mdx
index d007134d19..5ecd23392c 100644
--- a/docs/docs/reference/api/create-account.api.mdx
+++ b/docs/docs/reference/api/create-account.api.mdx
@@ -5,36 +5,33 @@ description: "Create Account"
sidebar_label: "Create Account"
hide_title: true
hide_table_of_contents: true
-api: eJy1V01v2zgQ/SvGnLlxGvSkU902QIN2N0aS7h4MI6CpicyWIlmSauo19N8XQ0qyFMdar5P1JY448/jmzYfGWwi88JAtYJaXUsOSgbHoeJBGX+WQgXDIA95zIUylAzBw+KNCH96bfAPZFoTRAXWgr9xaJUV0nX7zRtMzL9ZY8niqN9cPkC22YB1dESR6el55dGPnmpc4PA8bi5CBD07qAmrWPdGVUlAvGQQZFD34g3xrBlhyqU4FuYzOdc1aC7P6hoKkaC2+YMHF5qtHd5PEeQavZuCFsfj/hvqvJG+JwzjLauWFk5bSOEbWKq5PJTsn3yPI9qiMcB7BmaWyPey8Zwp1TYAOvTXap1Avzs/pT449YeC2EgK9f6jU5KYxBnZ8O/y3JpD5mNQMHowreYAMqkrmo2Fe5UeXchPU4Vr2Q1IyYOn3YzOu4Fr+HYW4f71ArnuwE4qKDW96STMNsNsh8mjcd2+5wFcM4q8Ws4lgd8dL6O9QW+7WGUr0KzKfJ8SGd4v/EtYtYstZ+vscS/Ms3MoYhTRDxirdTz6Sf81ie907o07mRg0xuSGAmtErMUcdJFf+VLwPPYgjh7bvNWRrz53jm9GLkuvoHd2MbOHjAHx7cbE/8/7kSuapKy6dM+70gZdjaF7JB6aGMmJweoTKUgcs0KUBNVCoJ6hJBEnE0heE/LQHWtPf0Xte9OQ+bBrFmNzR6XNK08okHea0ZlFc6erGrpesnbxJ3cNhfEzyjaX1093dfA8w5XaY1A9xw5vMug2vxLA2tPtZ4+l/y8MaMphy2hCnu03Qo/uJzsekVE5BBusQrM+mU6FMlZ/xAnXgZ1xOuZWxKj2KysmwiS6z+dVn3HxCntO7b7HsG9xS2aRCGJp1OeBWfkaSJI0cmFVhbVwzs4GBpNDWyYtipoK82a2ul794adM8aN+9DVCX3GZp3BXaboEbmj6zNKXlqDOgaaYf4ijrui6KM5nNr4AByZiScX725uwcnqZoYEwdx0Xo8UjHwJ5kodO/F0xAXr7bHRAzynKZdrkGb68iBmy6FAT8FaZWcRm7Kd69bYplAbFYgEFbLksGayqnbAHb7Yp7/OpUXdPjHxU6qoglg5/cSb4ifRbUw+u2NrbwHTdELA2b32KnkbmqUi08GTw0FJLHTAi0YdR22av4+fXtHTBYNT9uSpOTj+OP1MX8ETIA+oFE3rE447MtKK6LimZFBgmTPv8A4AWS3A==
+api: eJydVV1P2zAU/SvVfc7aDu0pTysMCcQmKlr2UlXTrXNpDIltbAfIovz36dppm5LB0PrSxD6+H+ee4zTgcesgXcEsK6WCdQLakEUvtbrMIAVhCT39QiF0pTwkYOmxIudPdVZD2oDQypPy/IjGFFKEo5N7pxWvOZFTiWFX1dd3kK4a8LUhSEFv7klwRC99wQuzmOMmJoA22UNVVRTQrgdQaNu25ZKc0cqR4zwn0yn/ZeSElYaLgRQWlRDk3F1VjG46MCQfr91YJsXLmKFyZD/W0XfaoqhvHdl91mFXbQJOaBNjD2KitVi/y8UiHmYi/snsrohA25eTkyFTP7GQWeBhdG6ttv9PU0YeZcFP0lPphoBCi6PdQe/OW6m2/eal8rQlG1k7YqhHuo4FQptA6bYc+TjeAfqDnMMtD6WDvA0NZIyWvPs3ptkV0lLGTuK+YuoO1xvWgd7I7tttfIv0vTfWi+VyPggYZ3s81LNg4tFsb+KSfK7Z3kY7fjfoc0hhgnwJTA5md2SfyLowlMoWkELuvXHpZCIKXWVj3JLyOEY5QSPhddpZ2B2dMTRI1pGorPR1iDebX15RfUGYsZ1W6z5gwZqKKjmGHXxh5BUxXwrLIPHK59rK33H0CUguII+nmBBW683h6jp/wdIU1LdzF2g/eSqDfA8qjDYdQnmn2vQab8AUqHqAluu506H8nSUjN6coHkhlkADzHGmbjj+Pp2+R2R0YzeaXnTVR+F5NEQbJq3HtB9VrzBOWXw8bXCXLoUTVizeQzlFV+3F4evETU6AMtgu5m05VKwiqggR2ulonkLPu0hU0zQYd3dqibXn5sSLL6lgn8IRW4oa5WrHZ851OGnigmguLt9KnYEmGF1XUxasbim+PeGImBBn/Lnbds8b8erGEBDbdh67UGZ+x+Mx2x2dIAfhjyaeDUMNaAwWqbcWXSgoxJv/+AJCgiB0=
sidebar_class_name: "post api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
-
-
-
-
-
+>
+
Create Account
@@ -43,29 +40,881 @@ Create Account
as={"h2"}
className={"openapi-tabs__heading"}
children={"Request"}
->
-
-
-
-
-
-
-
-
-
-
-
-
-
-
+>
-
\ No newline at end of file
+
+
+
+
+
diff --git a/docs/docs/reference/api/create-api-key.api.mdx b/docs/docs/reference/api/create-api-key.api.mdx
index ce71059e48..70d2a2e15a 100644
--- a/docs/docs/reference/api/create-api-key.api.mdx
+++ b/docs/docs/reference/api/create-api-key.api.mdx
@@ -5,65 +5,81 @@ description: "Creates an API key for a user."
sidebar_label: "Create Api Key"
hide_title: true
hide_table_of_contents: true
-api: eJzVU02P2jAQ/SvWnFrJIrRHnxq1hyIORbA9AVrNmoF4CY7XnqDSKP+9GiewC/+gucQfM+M3773pgPGQwKyhDE7N6ZJgq2FHyUYX2DUeDHyPhExJoVflYqaOdFH7JipUbaI42fiNL+MhmY1XSqlIby0lVp+Ww+KzUU8V3Y6bl1eyrGzjGZ13/qC4olxIzX4o5/P2GpwYmfIDS+I2+usbieNQ1WZkuyusCWhoAkUU4LMdGBgCnjG45yNdQEOkFBqfKIHp4Ot0Kr/7blettZTSvq3VcgwGDQKYPEs4hlA7m98oXpPkdJBsRSeUFV8CgYHE0fkDaGDHtRxca6mBTTXSDb18Gk7EVSOIQ5MYNATkCgwUR1FEQ6J4pig6ddDGGgxUzCGZorB10+4meCDPOEFXYHDQbyXDttHxJaeUi9mcLj8JdxTBrLcfA1YCfeDjPuzWCwY3z+R5PMm+bLlqovubKQANTmirhizpxfl9k9PH1ssMTkQCDdLGQPR08mUyhUez3QVnm9jM+vXtfA36gYVb/6CBTujkjglP394vBJmQe0L/od6DGg9ounfd/68pGJVj+sNFqNF56EfKutFaaziOw16J48wauu4FE/2Odd/L8VtLUdyz1XDG6PBFtFxve32VWowlQ2WgtJaCuPaMdTs45mFExHA3iy9+rZ6g7/8BgAqFZg==
+api: eJzVVMGO2jAQ/RVrTq1kAe3Rp6a7hyIORbA9AVoNzkC8JI7XnqDSKP9eTRLYXaR+QHNJbM+M37z3Ji0wHhOYDWTBqQVdEuw05JRsdIFd7cHAQyRkSgq9ypZzdaKLOtRRoWoSxcnWb30Wj8lsvVJKRXptKLH6tBo+Phv1VNBtu96/kGVla8/ovPNHxQX1hdT8UTnfL6/BiZGpv2BF3ER/vSNxHKraHll+hTUBDXWgiAJ8noOBIeAZg3s+0QU0REqh9okSmBa+zmby+tjturGWUjo0pVqNwaBBAJNnCccQSmf7O6YvSXJaSLagCuWLL4HAQOLo/BE0sONSNq611MCmGumGTh4NFXFRC+JQJwYNAbkAA9OTKKIhUTxTFJ1aaGIJBgrmkMx0asu6ySd4JM84QTfF4OBewKw/VQ8SCt1OytkmOr709bLlfEGXH4Q5RTCb3fuAtfQ1kPUx7NYoBrfomfVYyTpruKij+9PzAxqcACiGLGnU+UPdp4+8jNi+oz2Rz0GD9DnAnk2+TGb/amZMEOlHedD28lxx9GGg7+i6EQUaqEInZ0xYfXs7EJSiQoX+Xb072e5QtW8G+b/GZVSR6TdPQ4nOQzdS1o4e3MBp/CsUYk2zgbbdY6Jfsew62X5tKIqTdhrOGB3uRdfNrtNX2cVkMn0GMmspiL3PWDaDe+5mScx3m4Xlz/UTdN1fyDiT9Q==
sidebar_class_name: "post api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
-
-
-
-
-
+>
+
Creates an API key for a user.
Args:
- request (Request): The request object containing the user ID in the request state.
+request (Request): The request object containing the user ID in the request state.
Returns:
- str: The created API key.
-
-
-
-
-
-
-
-
-
-
-
-
-
+str: The created API key.
-
\ No newline at end of file
+
+
+
+
+
Successful Response
+
+
+
+
+
+
+
+ Schema
+
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/docs/docs/reference/api/create-app.api.mdx b/docs/docs/reference/api/create-app.api.mdx
index a2ba74d105..4170815040 100644
--- a/docs/docs/reference/api/create-app.api.mdx
+++ b/docs/docs/reference/api/create-app.api.mdx
@@ -5,76 +5,405 @@ description: "Create a new app for a user or organization."
sidebar_label: "Create App"
hide_title: true
hide_table_of_contents: true
-api: eJztVlFv2zYQ/ivEvawFNDsL9qSneW2AGt0WI/G2B8cILtJZYkuRLEklcQ3/9+FI2ZKcNesaYE8LAoQh777jfXf3iTsIWHnIVzCz1sM6g5J84aQN0mjI4Y0jDCRQaHoQaK3YGCdQtJ6cMPxboZafka0nN/pGz1zl8xsthBAWt8pgKV4ljJm1r3OxrOl4UBgdUGqpKxFqiugaGxKoyxGwmL8Vr0y8EarXMcwVhdbpQ6RjgMs22DakKCauT4NoelBbUUSHkkN+5xmeQ3LsBI7S0wH73XK5uHgsKIbPxXzDMI6E9AK1IOeMS3DDNIyLy8hSacgLbYKo8Z6EJddI7zmrYAQWBXl/cJtABsaSi0nPS8gh3fMWrYUMHH1qyYefTbmFfBf/lY5KyINrKQNOlHTgI7RWySLCTD94ruMOfFFTg7yyjoMESb6zveXMeR22liAHH5zUFWQQZFC8MbNW/MY2+wwCNVbxpT5SvAbq7eUG8tWp9z477uhWKdive7xlhyHe05YxrTMfqAi3svxWxEVCEPOS8R6M++gtFvQCxD8PGB3msCFfAHs56usS9kxpsjV3nEJX6FTZVV+eAcax3W9hv2cAR94a7VNBz8/O+M94jK/b2GmbVomrzhhe1DKJgecaJtH279rrK8iQJWTPs5JEoKPmx/Pzp2z8gUqWqQQXPMDfTkVJAaXilQzU+KcGyhSj06/oGakDVeRgv+7pQOdwO6DsF5MuyBQ3vnqO3V/Je6zS7EaTL5tGMsSST/+pFJxXCt3ZDSrR05vY/XIabxN9fxfsYML6+wQw1bahUBuWSWs8u1gMNeQwRf6SZeDJ3ZPzkezWKcihDsH6fDotlGnLCVakA05QTtHKOKSeitbJsI0us8X8PW3fEZbkIF+thwbX3A6pwGOzI7doJYtbBqn5YdaG2rhu8CEDyX1YJy/OhRvtqhf4i0dsrKKxQA8KNhLhfn8opP3uWA77/SeS1jcjSL0xMZ3DjEayxGwxhwyY1jRJZ5MfJmdw+mgYGccPcBEn68BFPIbspCrHekAG1MSxgkDY/NQfMFNc7Ab1AK97o8ziV3J0k10/1/+/ZP7jl0w3CIEew9QqlFGrYsV33aRGQY9vzpoHOF/BbneHnn53ar/n7U8tOR7GdQb36CTecSuuWBbrw1juII3Am1Tn76N4sblq0xieaDnrbPKYFUzFs7brgcYsLq+XkMFd9/pqTMk+Dh9YGPEBcoivN/aOuhD3dqBQVy3Lbw4Jk3/+Ajnw8lM=
+api: eJztVt9v2zYQ/leIe1kLaHYa7ElPU9MANdotRuJtD44RXKSzxEYiWZKK4xn+34cjZUt2lvRHgD41CBCGvPuO993dJ27AY+kgnUNmjINFAgW53ErjpVaQwpkl9CRQKFoJNEYstRUoWkdWaP4tUcl/ka1H1+paZbZ06bUSQgiD61pjIV5FjMyY16mYVbQ/yLXyKJVUpfAVBXSFDQlUxQGwmLwTr3S4EdavQ5hL8q1Vu0j7ABetN62PUXRYHwdRtKrXIg8OBYf8xTE8h+TYERylox32+9lsev6QUwifismSYSwJ6QQqQdZqG+GGaWgbloGlQpMTSntR4T0JQ7aRznFWXgvMc3Ju5zaCBLQhG5KeFJBCvOcNGgMJWPrckvNvdbGGdBP+lZYKSL1tKQFOlJTnIzSmlnmAGX9yXMcNuLyiBnllLAfxklxne8OZ89qvDUEKzlupSkjAS1/zRmaM+JNttgl4akzNl7qjcA1U64slpPNj722y31FtXcN20ePNOgzxgdaMaaz+RLm/kcX3Ik4jgpgUjLfS9s4ZzOkFiP/sMDrMYUO+APbioK8L2DKl0VbfcgpdoWNl5315Bhj7dr+B7ZYBLDmjlYsFPT054T+HY3zVhk5btrW47IzhRS0TGXiuYSJt39ZeX0GGLCB5npUoAh01v52ePmbjb6xlEUtwzgP8/VQU5FHWvJKeGvfYoNb5welX9IxUnkqysF30dKC1uB5Q9lHHCzLFjSufY/cPcg7LOLvB5GnTQIaY8emXSsF5xdCd3aASPb2R3afTeBfp+79gOxPW30eAsbYN+UqzTBrt2MWgryCFMfKXLAFH9p6sC2S3toYUKu+NS8fjvNZtMcKSlMcRyjEaCccfviycijM2DRPsKG+t9OuAl00nH2j9nrAgC+l8MTS44l6J1T802xOPRrLyJRAnA7LWV9p2qgAJSL5AFb04Ue7Cy179zx+wMTUdqvegmgcK3e8PVbbfPdTKfv+R3vWdClItdUhnN8CRq7eY35Hi+WTeI40nozejk6fI7RxENp10I4h5GMEdL8EMkqPy7QsHCVAT5g88YfN7f8CscVc0qAZ43WMmC5/TgxttegH4+eT5wU+ebig8PfixqVEGUQsV33QjHZQ/PE4rnvR0DpvNLTr6y9bbLW9/bsnyYC4SuEcr8Zbbcs76We1GdANxHM5inX8NKsfmdRtH8kj0WZCjR5YzFc/aLgZiNL24mkECt90zrdEF+1hcsYLiClIIzzz2DhoR9jZQoypb1ukUIib//AfmrwDx
sidebar_class_name: "post api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
-
-
-
-
-
+>
+
Create a new app for a user or organization.
Args:
- payload (CreateApp): The payload containing the app name and organization ID (optional).
+payload (CreateApp): The payload containing the app name and organization ID (optional).
Returns:
- CreateAppOutput: The output containing the newly created app's ID and name.
+CreateAppOutput: The output containing the newly created app's ID and name.
Raises:
- HTTPException: If there is an error creating the app or the user does not have permission to access the app.
+HTTPException: If there is an error creating the app or the user does not have permission to access the app.
-
-
-
-
-
-
-
-
-
-
-
-
-
-
+>
-
\ No newline at end of file
+
+
+
+
+
diff --git a/docs/docs/reference/api/delete-api-key.api.mdx b/docs/docs/reference/api/delete-api-key.api.mdx
index bafd11e1d9..92d621ddfe 100644
--- a/docs/docs/reference/api/delete-api-key.api.mdx
+++ b/docs/docs/reference/api/delete-api-key.api.mdx
@@ -5,77 +5,322 @@ description: "Delete an API key with the given key prefix for the authenticated
sidebar_label: "Delete Api Key"
hide_title: true
hide_table_of_contents: true
-api: eJztVU1v4zYQ/SvEnHYBws4GPelUAzGwQXZRI+v2khgBLY0lbiSSS47SqIL+ezGkZNlxs0BPvfRkeTgfj2/eDHsgVQbIHmDltLjDLsBOQoEh99qRtgYyuMEaCYUyYrW5Fc/YiT81VYIqFKV+QRNNzuNBv4qD9fFAtVShIZ0rwkK0Af3i0TyalS9D9miEEBz0NAZ9COQ/ZmJb4ZTGHmKWqR5ZsUdRRBzFIsV7/NFiIPHhPn2MCbTJbaNNeTy3+++YU6x+j9R6MwEodE6ZWMVfbY3yncitIaUNhysR2jzHEESDIagSReusmYyHtk5wtDUptdIBp8yft9vN+jXHSGAmbs8vo4MwlsTBtqYQ1ovCYrLssbam5Muye+QMJFiHXnGi2wIySBw8KaefnrEDCU551SCh5yb2YFSDkMFMLkjQ3ESnqAIJzIr2WEBGvkUJIa+wUZD1QJ3jyEBemxIkkKaaDXfYiU1KNQw7zhCcNQEDB11fXfHPuV6+zRzdj84ggblFQ+yunKtZGdqa5ffAMf0lkNQ3kKCKQscG1RvPZJDm2gn+BHKqI0atjmKGYRgGCb9cX1+i/EPVuogYxNp76/8NRHcChPOS0jV/acImXDrUNj87Vab77RDbdU76II8WbQhL9DDsBjnZlPeqO+nMF5sAwiChCeXPmvg1aRiOyd53jWSILZ8Oc+1jN2YBPcR7pdKj325OM9Ob2H3/GjeJvn8qNrnwQF0kTL1tkCo7T0acCKogg+UzdmHZz5MwgISA/mUaldbXkEFF5EK2XOa1bYuFKtGQWii9VE4Dyz1g3npNXQxZbW7vsPuMqkAP2cPu1OEbqyP1+9ztSLVy+i7O7Dikq5Yq6/VfqYnjnFYpiq+mzcHG8JGFVQTHewQk8DWSkK8WnxZX8HZpnznHvZZHYU+14zHINywc7w8SsImqBkLV/DofMDJnAzXKnOR7M3dv0PTzaP3/mvynr8koRcJXWrpa6bg8ogb6cXAe+PEIICE7eUR2EiobiE/7fq8C/u7rYWDzjxY9D8dOwovyWu1Zqg89FDrwdwHZQdUBfyKISDmvlI/iPYDTABlW1ouqW/4HkpGeP3ZxXVbTfPajwypn9k5CL7Y7D/Jxk9ysv6y3axiGvwHPjSEW
+api: eJztVU1v4zYQ/SvEnHYBwk6DnnSq2xjYIFvUyLq9JEYwlsYSNxKpJUdpVEH/vRhKsuy4WaCnXnqyPJyPN2/mkR0w5gGSB1jVRt1RG2CnIaOQelOzcRYSuKGSmBRatdrcqmdq1Z+GC8UFqdy8kI2m2tPBvKqD8/EAGy7IskmRKVNNIL94tI925fOQPFqllAQ9jUEfAvuPidoWNKVxh5hlqsdO7UllEUe2GOI9fWsosPpwP3yMCYxNXWVsfjx3+6+Ucqx+T9x4OwHITMqJWsVf4yz6VqXOMhor4ahCk6YUgqooBMxJNbWzk/HQlAMc4+yQGk2gKfOn7Xazfk0pEpio2/NmTFDWsTq4xmbKeZU5Gix7Kp3NpVlxj5yBBleTR0l0m0ECAwdPWJunZ2pBQ40eK2LyMsQOLFYECczkggYjQ6yRC9AgrBhPGSTsG9IQ0oIqhKQDbmuJDOyNzUEDGy7FcEet2gyp+n4nGULtbKAgQddXV/Jzvi9fZo7uR2fQINySZXHHui5lM4yzy69BYrpLIMPcQANmmYkDKjdeyGAjtQf4E8ipjhp3dVxm6Pu+1/Dj9fUlyj+wNFnEoNbeO/9vINYnQCQvoynlyzBV4dKhdOnZKdr2t0Mc1znpvT5ajGXKyUO/6/VkQ++xPZnMZzcAhF5DFfLvDfHXYYfhmOx910iG2sppP9c+TmNeoIfY11B69NvNaWZ6B3bfb+NmoO+fik0uIqiLhMNsK+LCzcqIiuACElg+UxuW3ayEHjQE8i+TVBpfQgIFcx2S5TItXZMtMCfLuECzxNrA24twFU/VL+IKooVAaeMNtzHfanN7R+0nwow8JA+7U4cvsjrDMpy7HeeAtbmLgh4VvGq4cN78NUx4FHExREnfxh5cDB8pGrH9jOkz2Qw0SJ8D7KvFD4ur95oZA+R2GhWAaVTAhCO6gX5D15Eo0EBVXH9gwuqn+UBQ1i5whfYk3xuBvkHVzRr8/9n5T5+dcS2ZXnlZl2jiLRN3oBsV9iCvTAANyclrs9NQuMBy2nV7DPS7L/tezN8a8iKUnYYX9Ab3srYPHWQmyHcGyQHLQN9ZiEi53D0f1XsAJzFZ2awXLBv5B1qQnr+K8V4tJq12o8MqFfZOQi+eARH18cq5WX9eb9fQ938DcL4vpQ==
sidebar_class_name: "delete api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
-
-
-
-
-
+>
+
Delete an API key with the given key prefix for the authenticated user.
Args:
- key_prefix (str): The prefix of the API key to be deleted.
- request (Request): The incoming request object.
+key_prefix (str): The prefix of the API key to be deleted.
+request (Request): The incoming request object.
Returns:
- dict: A dictionary containing a success message upon successful deletion.
+dict: A dictionary containing a success message upon successful deletion.
Raises:
- HTTPException: If the API key is not found or does not belong to the user.
+HTTPException: If the API key is not found or does not belong to the user.
-
+>
-
-
-
-
-
-
-
-
-
-
-
-
-
-
\ No newline at end of file
+
+
diff --git a/docs/docs/reference/api/delete-evaluations.api.mdx b/docs/docs/reference/api/delete-evaluations.api.mdx
new file mode 100644
index 0000000000..3c6ad18953
--- /dev/null
+++ b/docs/docs/reference/api/delete-evaluations.api.mdx
@@ -0,0 +1,344 @@
+---
+id: delete-evaluations
+title: "Delete Evaluations"
+description: "Delete specific comparison tables based on their unique IDs."
+sidebar_label: "Delete Evaluations"
+hide_title: true
+hide_table_of_contents: true
+api: eJzdVU1vEzEQ/SvWXABpSUrFaU+kNBIRRVRt4JJGyPVOsgav7dqzLSHKf0djb7r5aEFcySVr+3k+3rwZr4HkMkI5g/G9NK0k7WyEeQEVRhW05zWUcI4GCUX0qPRCK6Fc42XQ0VlB8tZgFLcyYiV4XaMOorX6rkUxOY+DG3tjR2EZyxtbJTPfsHclXl7oSLNIYf6qFNMatzd1hZb0QmOIwi3Y6hNOyYlsMjm5QmqDZT8jYXSk7b0MqY7vv0jxQQHOY0jxTCoo4ThKKCDgXYuRzly1gnKdljpgBSWFFgtQzhJa4iPpvdEqXRx+j0zfGqKqsZH85QM7I42RVzs+vukqbWnCJn3QyiOUEClou4RNsd2QIcgVFECaDK93CicmVYRND3W331FRF30Od3bkc96bymXuDcJmw9YCRu9szCGfnpzw374+rlulMMZFa8RVB4Z/IOUgsz9Q0AW6dSI6Ye6KN8f89vT0OMyv0ugqwcQ4BBf+Jcb9wlVIUpu9eu0DjFN7p9KuPi+gnB0n9bijLeESA2zmzxf7wqmuMgU0cXlMUg/9hDHKJfbKeR6ayBBTPv2bejiv7LrD7cinpzez+3wa55m+p5xtIR+m08sjg7m2DVLt+k6FArykGkoY7vdsxHCPISbW22CghJrIx3I4VMa11UAu0ZIcSD2UXsPhzBulU/GeobCZsznVBk2rZG90OfmIqw8oKwxQzua7gGsWTZbBPqxXutcfkQmxsuH1qKXaBf0r17YAzQHU+RZnzHK86ifQ+KdsvMEnJ8hsW15WkbYLl9x2pHY5nUn1A20FBTA/Od2TwZvByXMkdBfE6HLS9YxUqWe28ScYFAc0PxIMBWCTGgYIZfOuP+DsvIvUSLtj74m2Pohs3Xfu//46dZoh/ElDb6ROzZ+IXnfC3xvq3JG1i8Tb6zVn/iWYzYa371oMLN95AfcyaHaTtFtstcbK/oErKOF9Zvd1GgoMN20W7sGM5PmVb4yUQk9/xM53evd8fDGejqGA2+5JbVzFt4J84JEjH6CE9DTnpPjN5b01GGmXLQ+2ErJV/v0GKyEQqA==
+sidebar_class_name: "delete api-method"
+info_path: reference/api/agenta-backend
+custom_edit_url: null
+---
+
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
+import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
+import OperationTabs from "@theme/OperationTabs";
+import TabItem from "@theme/TabItem";
+
+
+
+
+
+Delete specific comparison tables based on their unique IDs.
+
+Args:
+delete_evaluations (List[str]): The unique identifiers of the comparison tables to delete.
+
+Returns:
+A list of the deleted comparison tables' IDs.
+
+
+
+
+
+
+
+
diff --git a/docs/docs/reference/api/delete-evaluator-config-evaluators-configs-evaluator-config-id-delete.api.mdx b/docs/docs/reference/api/delete-evaluator-config-evaluators-configs-evaluator-config-id-delete.api.mdx
index a4053fb136..3240633f9b 100644
--- a/docs/docs/reference/api/delete-evaluator-config-evaluators-configs-evaluator-config-id-delete.api.mdx
+++ b/docs/docs/reference/api/delete-evaluator-config-evaluators-configs-evaluator-config-id-delete.api.mdx
@@ -5,73 +5,304 @@ description: "Endpoint to delete a specific evaluator configuration."
sidebar_label: "Delete Evaluator Config"
hide_title: true
hide_table_of_contents: true
-api: eJzdVUFv2zwM/SsCTxsgJF2xk08L1gwLtmFFl32XNAhUm4612ZIq0d3yGf7vAy07Tuq2wK47xZHIJ/LxPakBUvsAyQaWD6qsFVkfYCshw5B67UhbAwksTeasNiTIigxLJBRKBIepznUqcMgUqTW53tdecd7s1tyahd+H5NYIIcawXQzb6Uy8CuRfJ2JdoKiNvq9R6AwN6VyjFzYXVOCL8DdItTfDCXfWlolYe4bJY6HaGvFLBRHqNMUQ8rqU4oMqAwpLBfpfOuAMJFiHEXWVQQKxxd2k4ONC6FfCbhqks90uAoAEp7yqkNAzxQ0YVSEk8EQOSNDMtFNUgASP97X2mEFCvkYJIS2wUpA0QAfHEIG8NnuQQJpKXjiOT7zvMMUqg7bdMlRw1gQMnH15ccE/59P9duRG3PTBICG1htAQhyvnSp12/Mx/BM5pphUx+ajMSUkDmLiKkpmUOEquXwliGrTKRA8Abdu2Et5eXk57+E+VOusqFEvvrf+bBpzn8ZOOFGVISpf8pQmrMA0obXq2q8zha96N93w2rTyuaEO4Rw/ttpXDmvJeHU7Y+mxjgdBKqML+pVl/wRDUHuEI9oIsmAyx5t12PNve/cCUznS26fqKR/dx2xFmpDey+3wbV5G+pw4bQj6u19cTwDjbCqmwowU7B1EBCcxH7817782bJ3zUgoSA/mFwXO1LSKAgciGZz9PS1tlM7dGQmik9V04DeyRgWntNhy5lcb36hIePqDL0kGy2pwHfWDRRBudhxwkopz8hE9J7fVFTYb3+P862d3kRs7hjbXLbpffkLLrixOJ6BRK4jajvi9mb2QU8vpfPglnvKu30PpzdbYN8xMKxf5CAVSd2IFTVu3GDK3M2UKXMCd4zPn5cVjNa799/OfqxE/6muSuV7vzb8d302t2M932IY2L1goTkqXdgK6GwgTitae5UwO++bFtevq/Rs0K3Eh6U1+qO9bJpINOBvzNIci7vhWG8uunt/lo8V/mgYsMS5vL4H0j4iYdnHq7uTisGtzR95CJN0dEJxuQKZlsd7X61/LxcL6Ft/wBSbf2+
+api: eJzdVcFu2zgQ/RViTi1A2NmgJ53qbbyo0S42SN29OIbBUCOLrUQq5CitK+jfFyNKlh3FAfbak2Vy5vHxzbxhA6T2AZINLJ9UUStyPsBWQopBe1ORcRYSWNq0csaSICdSLJBQKBEq1CYzWuCQKbSzmdnXXnHe7N7e24Xfh+TeCiHGsF0M25lUvAnk3yZinaOorXmsUZgULZnMoBcuE5Tjq/B3SLW3wwkPzhWJWHuGySJR46z4oYIItdYYQlYXUvylioDCUY7+hwk4Awmuwoi6SiGBeMXdhPBxIfQrYTcNMuluFwFAQqW8KpHQs8QNWFUiJPBCDkgwrHSlKAcJHh9r4zGFhHyNEoLOsVSQNECHiiECeWP3IIEMFbxwLJ/40GGKVQptu2WoUDkbMHD29dUV/5xX98tRG3HXB4ME7SyhJQ5XVVUY3ekz/xY4p5kyYvFR2RNKA5i4iS0zoTi2XL8SxDRolYoeANq2bSW8u76e3uFfVZi0YyiW3jv/fy5QeS4/mShRiqRMwV+GsAzTgMLps11lD/9kXXnPa9PK44qxhHv00G5bOawp79XhRK3PLhKEVkIZ9q/V+m8MQe0RjmCvtAWLIda8245nu4dvqOmszzbdveLRfdx2hBnljepevsZNlO+lw4aQj+v17QQw1rZEyt1owc5BlEMC89F789578+YFH7UgIaB/GhxX+wISyImqkMznunB1OlN7tKRmysxVZeD5rFt0u+IDhwIbKKCuvaFDh7e4XX3Cw0dUKXpINtvTgC/cUbFHzsOO5VGV+YSsVj8IFjXlzptfsfD9CMhjFsthbOa69F65ntufSn9HyzOD7xlpX83+mF1dukyfIBa3q94YSnfGGHh0YSCfyXUUCiRg2bkCCFX5ftxglpULVCp7gnfB8M/pNaNHf/8npm8Bwp80rwplOqN3ejd9k2/GhyHEMnGbg4TkpQdjKyF3gTitaR5UwK++aFtefqzRc7duJTwpb9QD986mgdQE/k4hyZjeK8V4c9fPhbfiEvOhoy23M9PjfyDhOx4uvHDd8MsH5zR95EJrrOgEYzKr2WLHuXCz/LxcL6Ft/wNmjQxc
sidebar_class_name: "delete api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
+>
-
-
-
-
+>
Endpoint to delete a specific evaluator configuration.
Args:
- evaluator_config_id (str): The unique identifier of the evaluator configuration.
+evaluator_config_id (str): The unique identifier of the evaluator configuration.
Returns:
- bool: True if deletion was successful, False otherwise.
+bool: True if deletion was successful, False otherwise.
-
+>
-
-
-
-
-
-
-
-
-
-
-
-
-
-
\ No newline at end of file
+
+
diff --git a/docs/docs/reference/api/deploy-to-environment.api.mdx b/docs/docs/reference/api/deploy-to-environment.api.mdx
index 21936144cd..eaa77049ca 100644
--- a/docs/docs/reference/api/deploy-to-environment.api.mdx
+++ b/docs/docs/reference/api/deploy-to-environment.api.mdx
@@ -5,74 +5,326 @@ description: "Deploys a given variant to an environment"
sidebar_label: "Deploy To Environment"
hide_title: true
hide_table_of_contents: true
-api: eJzdVcFu2zgQ/RVizqqdBnvSad3GQI3ubo3E24tjGBNpLLFLkSpJpVEF/XsxpGzJTpu21/XFNjnzhnzz5rEDj4WDdAtL/Sit0RVp72CXQE4us7L20mhI4YZqZVonUBTykbR4RCtRe+GNQC1ozL3X93phC5feayHEdGevsaJU/IMVCXMQvqTpLiPloYjwZhaThyJ7maengjIfI2dc7Balo2O5d5vNevmUUTh2KlaxTIwOVQ4olZtBAqYmixy1yiGFGLH3Zj85EiRg6XNDzr8xeQtpB5nRnjfSDrCulcwCwvyTY446cFlJFfKv2jK+l+T43yUJvObbmiAF563UBSTgpVe8MGlDoAr6BEYeXsr8ODC0yjknM1Ul/b4i57AIFVG3Hw6Qbi8R+uS0ohuloN+NmG8Divh7QOn75BhqHj5RdqRIWspZQ88uenb2CW6U08ZMbrvGVhnMoeciI6i3DYUFVxvtIp/XV1f8da7QuybLyLlDo8TtEAzJL3esD2X/uL5+DvwRlcxDmlhaa+xvoF7oICePUvEv6alyzwOUyc52f6FlUnsqyEK/G3uD1mI7UcZfJh6QdVG54iURnTp9BHtBqUyG2PDuz3TB94qlh7jdVLZHeiO7P77GTaTve8WOITz/zwBjbyvypeFZr43jlBp9CSnMJ5p18+gDkIAj+0jWBe4bqyCF0vvapfN5pkyTz7Ag7XGGco61DCPjKGus9G1IWaxX76l9R5iThXS7mwbcsTpiv8/DTlRjLd8TnyKaBSwaXxorv8YmJiBZlmXM4qux7m5Hp1o+YVUr+r7zjH2cusq4eukbo+RA6oMJpxzIXgQOxGK9YjSyLs7L1ez17AouH5CzYJ4fzML8HK8YtiG5IPtEMyRAVRge8ITVn+MGE8AtrVBP8KLBiI0RyzNDPzvUxND/fw/coCVPT35eK5Rh+gO73aD9M792gZyg/l0CJc9IuoWue0BH/1rV97z8uSHLAt8N6nlgHWzZecqj1Dv4j9rwcgRmXwV/4HDVRGlf2CVbWcxYZHytF2N3kzFef7jbQAIPw9NcmZxzLH5h78EvkEJ45Tk7zFpY60ChLpqo7IjJn2+llSM5
+api: eJzdVsGS0zgQ/RVVn73JMLUnnwiQKlIskJoJXDKpVI/dsQWyZCR5GK/L/77VkhM7GZiFK7nEkbpfS69fP6cDj4WDdAtL/SCt0RVp72CXQE4us7L20mhI4Q3VyrROoCjkA2nxgFai9sIbgVrQmHun7/TCFi6900KI6c5eY0Wp+IAVCXMQvqTpLiPloYjwZhaThyJ7maengjIfI2dc7Aalo2O5t5vNevmYUTh2KlaxTIwOVQ4olZtBAqYmixy1yiGFGLH3Zj85EiRg6VtDzr8yeQtpB5nRnjfSDrCulcwCwvyLY446cFlJFfJTbRnfS3L865IEXvNtTZCC81bqAhLw0itemLQhUAV9AiMPz2V+Hhha5ZyTmaqSfl+Rc1iEiqjbjwdIt5cIfXJa0Y1S0O9GzNcBRbwfUPo+OYaa+y+UHSmSlnLW0JOLnp19ghvltDGT266xVQZz6LnICOptQ2HB1Ua7yOf11RV/nSv0tskycu7QKHEzBEPyyx3rQ9m/r6+fAn9GJfOQJpbWGvsbqBc6yMmjVPwkPVXuaYAy2dnuL7RMak8FWeh3Y2/QWmwnyvjHxAOyLipXPCeiU6ePYM8olckQG979P13wvWLpIW43le2R3sjuz6/xJtL3o2LHEJ7/J4CxtxX50vCs18ZxSo2+hBTmE826efQBSMCRfSDrAveNVZBC6X3t0vk8U6bJZ1iQ9jhDOcdawqVbLsKueM2hYZ4cZY2Vvg14i/XqHbVvCXOykG5304Bblk4Uw3nYqQ9Yy3fER4xOAovGl8bKf2OHE5B8gDJm8b1ZlDejjS0fsaoV/diWxiZPLWdcvTSVUY8g9cGEUw6dGCh4hdlX0jkjknWRnavZi9nVzzgbEsRivRoGDbMwaMfrhjBILrpy6gckQFWYMvCE1ctxg8ng3leoJ3jRicTGiOWZ858dbuL8f96bcNCVp0c/rxXKYBOB3W4YkjNjd4GcMCa7BEoepnQLXXePjj5Z1fe8/K0hy2LfDUq6Z01s2aLKo+w7+EpteMUEZv8KRsLhqokyv/BV9ryYscj4Ws/G7ibzvv54u4EE7od3eGVyzrH4nU0Kv0MK4e8AZ4e5C2sdKNRFE1UeMfnzH9QgMcg=
sidebar_class_name: "post api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
-
-
-
-
-
+>
+
Deploys a given variant to an environment
Args:
- environment_name: Name of the environment to deploy to.
- variant_id: variant id to deploy.
+environment_name: Name of the environment to deploy to.
+variant_id: variant id to deploy.
Raises:
- HTTPException: If the deployment fails.
+HTTPException: If the deployment fails.
-
-
-
-
-
-
-
-
-
-
-
-
-
-
+>
-
\ No newline at end of file
+
+
+
+
+
diff --git a/docs/docs/reference/api/fetch-evaluation-ids-evaluations-by-resource-get.api.mdx b/docs/docs/reference/api/fetch-evaluation-ids-evaluations-by-resource-get.api.mdx
index 371cea6b02..a1a39d9667 100644
--- a/docs/docs/reference/api/fetch-evaluation-ids-evaluations-by-resource-get.api.mdx
+++ b/docs/docs/reference/api/fetch-evaluation-ids-evaluations-by-resource-get.api.mdx
@@ -5,78 +5,354 @@ description: "Fetches evaluation ids for a given resource type and id."
sidebar_label: "Fetch Evaluation Ids"
hide_title: true
hide_table_of_contents: true
-api: eJztVU2P2zYQ/SvEnBJAsLeLnnSqi2wTIymycNxevIbBlcYSU4lUOKNtXEP/vRjK+vJHgUVPBXqySb2ZeTN8jzwC64wg3sDDiy5qzcZZgm0EKVLiTSVriOEX5CRHUtiDlElJ7Z1XWmXmBa3ySK72CSo+VKi0TZVJZ0/2yS58VpdomeInq5RSuqp2JlVviP3bWK1zVMt3yu0V5yjfQtI/c5Pkip3aS+FRWZq1Sbpqu1BtlCus3X6g87p00tUnQ7z5/PwVE16m244ivS7rk11pQ9g1/WG9fnz4nmAYaKyWbbvTLgwpY190YVIlc00SJJLNFK3BdpYr5NrbLmkgSuy3sVqowhALxekRzSACV6EPG8sUYghkdwNKWh4tafd82PW8MmSIoNJel8joRShHsLpEiGFCHiIwIpRvNfoDRODxW208phCzrzECSnIsNcRHCOgYiL2xGUTAhgvZWHWjXQugaaLLQial23X2uqBrhbT3WpCGsaRLAs0VBsuUoGm2kp0qZwlD3P3dnfxMjfGlDse0rwu1OoEhgsRZRssC11VVmCSMdv6VJOb4LzmGIio4Ug2mFdKjJamfD6pv6D0yNE3TRPDj/f1lE7+L5tosD947/5oOKi/yYtPOKEXWppB/fStTQOGSyVdtD5/3QVXnLfc7xjJm6KHZyhymA+vG8sm1BGVUJWX/pLNfkUhnCH2y29AwjF6PHcyFm2GivU3oqy19wm2HNMN42+nebuNdO75rxTqIXCQXCduzLZFzJxbvXMs5xDAfeXs+8jZEQOhfOlPXvoAYcuaK4vk8KVydznSGlvVMm7muDIghCJPaGz6EkMXj8iMePqBO0UO82Y4BX0Qg7ZFPYYPmK/MRpfmTyxc1586bv9pzPNk8b6OkO2P3LoSfBrEI5NTicQkRSButlu9mP8zu4Pz9moBF2zoJ2u5qh88QnU2h7x8iwDIIGxh1+dPwQZhVjrjUdpTvmjnPOR0Hj/3/vP6Xn9eTnhm/87wqtAmXUBDS8eTBDYzIQwRjF24jyB2xgI7HZ034my+aRrbbJ06MlhrSz8XokbsppDer0530Vt1i9gcerjzewk+w4eK9qHcW077DQ8hWFt5ITLgHos63Qr6NXSRyOKOoi1dFsvQ32PuHNTTN3zPfriY=
+api: eJztVU2P2zYQ/SvEnBJAsLeLnnSqk2wTIymycJxcvIbBlcYSE4lUOKNtHEP/PRjqw/JXgUVPBXqySb2ZefM4j9wD64wgXsHdky5qzcZZgnUEKVLiTSVriOFP5CRHUjiAlElJbZ1XWmXmCa3ySK72CSreVai0TZVJJw/2wc58VpdomeIHq5RSuqo2JlUviP3LWC1zVPM3ym0V5yjfQtK/c5Pkip3aSuFRWZq0Sfpqm1BtlCus3fZA53nppKsPhnj18fErJjxP1z1Fel7WB7vQhrBv+t1yeX/3I8EgaKzmbbvHXRhSxj7pwqRKdE0SJJLNFK3BVssFcu1tnzQQJfbrWM1UYYiF4vERTSACV6EPG/MUYghkNweUtDxa0uZxtxl4ZcgQQaW9LpHRy6DsweoSIYYj8hCBkUH5XqPfQQQev9fGYwox+xojoCTHUkO8h4COgdgbm0EEbLiQjUUv7VIATROdFzIpXa+z1QVdKqS914I0jCWdE2guMJinBE2zluxUOUsY4m5vbuTn2Bif6nBM27pQiw4MESTOMloWuK6qwiRB2ulXkpj9v+QYiqjgSHUwrZAeLUm92qmhobfI0DRNE8Hvt7fnTXyRmWuz3Hnv/HM6qLyMF5tWoxRZm0L+Da0cAwqXHH3VdvdxG6bqtOVhx1jGDD00a9HhWLBelg+uJShSlZT905z9hUQ6QxiSXYcGMYZ57GEu3AxHs7cKfbWlO9z6kOYgb6vu9TbetPJdKtZD5CI5S9iebYmcO7F471rOIYbpyNvTkbchAkL/1Ju69gXEkDNXFE+nSeHqdKIztKwn2kx1ZeD0TZiFr+q1QEHcQpjU3vAu5Jvdz9/j7h3qFD3Eq/UY8Emmp52HY9jBEJV5j6JMdwXMas6dNz/bQ+7ugLyNktaN3boQ3qnUcXulk29oU4hA+mxp30x+m9xca6YLULP7eWcCnQQT9DwCDKITuQahIAIsgwOAUZd/HD4Iy8oRl9qO8l1y8Sm3/cGM/7/D/+V3uJttxh88rQptwm0VBmnfmXUFI/IQwdiu6whyRyyg/f5RE372RdPIdvsWiulSQ/qxGL2GVwfpxaK7vF6qa8y+4e7CKy/8BBtu6LN6JzHtg30IWcvCG4kJd0LUe1jIt7GzRA5nFHX2/EiW4ap7e7eEpvkFCsK8tQ==
sidebar_class_name: "get api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
+>
-
-
-
-
+>
Fetches evaluation ids for a given resource type and id.
Arguments:
- app_id (str): The ID of the app for which to fetch evaluations.
- resource_type (str): The type of resource for which to fetch evaluations.
- resource_ids List[ObjectId]: The IDs of resource for which to fetch evaluations.
+app_id (str): The ID of the app for which to fetch evaluations.
+resource_type (str): The type of resource for which to fetch evaluations.
+resource_ids List[ObjectId]: The IDs of resource for which to fetch evaluations.
Raises:
- HTTPException: If the resource_type is invalid or access is denied.
+HTTPException: If the resource_type is invalid or access is denied.
Returns:
- List[str]: A list of evaluation ids.
+List[str]: A list of evaluation ids.
-
+>
-
-
-
-
-
-
-
-
-
-
-
-
-
-
\ No newline at end of file
+
+
diff --git a/docs/docs/reference/api/fetch-evaluation-scenarios-evaluations-evaluation-scenarios-comparison-results-get.api.mdx b/docs/docs/reference/api/fetch-evaluation-scenarios-evaluations-evaluation-scenarios-comparison-results-get.api.mdx
new file mode 100644
index 0000000000..865d61070a
--- /dev/null
+++ b/docs/docs/reference/api/fetch-evaluation-scenarios-evaluations-evaluation-scenarios-comparison-results-get.api.mdx
@@ -0,0 +1,305 @@
+---
+id: fetch-evaluation-scenarios-evaluations-evaluation-scenarios-comparison-results-get
+title: "Fetch Evaluation Scenarios"
+description: "Fetches evaluation scenarios for a given evaluation ID."
+sidebar_label: "Fetch Evaluation Scenarios"
+hide_title: true
+hide_table_of_contents: true
+api: eJztVcFu20YQ/ZXFnBKAlVyjJ56qxm4iJEUNW+1FFoz1ckRuQu4yO0M3qsB/L2ZJipQlG0jOPVkevnn7dmbe7B5Y5wTpGq6fdNlott4RbBLIkEywtfwPKfyObAokhQeQIoNOB+tJbX1QWuX2Cd0UsLya3bt7twh5U6FjSu+dUmqCeLCZekMc3qZqVaBaXim/VVzglES4/ymsKRR7tRUV48GR/lZbwoH7w2p1c/3NYJSdquUJnSXlPKutb1ymRLYxSCThDJ3FrKNEboIbOD9Z4vVYnLv+9E2qFqq0xCL6XFlmkICvMcTwMoMUovyHyf0P2EmQzgOMr2odLHn3EJCakukhR4YEah10hYxBmrgHpyuEFKZ8NiNIwEobvzYYdpBAwK+NDZhByqHBBMgUWGlI98C7WvKJg3U5JMCWSwlMpkMtM4K23QgN1d4RkmReXlzIn+O5uWtigbdNqW57MCRgvGN0LHBd16U1kXj+mSRnP1XTnz7kqjiHahSj7g5TOFV4FvDuUELRIiVU75Ghbds2gV8uL0/V/61Lm3U01yH48D3S6yDNZ9sVJ0PWtpRflrGiU0DpzdFX7XZ/bmNHjzvSJoeIdYw5Bmg3bTLEdAh6N2nbJ98JhDaBivLXOvwHEukc4UD2yjBIMdRKvrbj2f7xMxo+mq51vFd3dI/bjDRjebvqvnyNq6585w4bIGL9E8KutxVy4cWAg2O4gBTmE4/Mz3luPnrup95zkABheBrM1oQSUiiYa0rnc1P6JpvpHB3rmbZzXVt4vkcX8at6J1AQCxGaJljeRb7FzfIj7j6gzjBAut5MAXcyWd2sHMMObdK1/YhStX4HLBoufLD/dgPQb4Ciy5KyWLf1U5P12n7T5gu6DBKQe3ayL2Y/zy5eukyfoBY3y94g2kSDDDoiDJJn5ToUChLAKroDGHX16/hBVNaeuNJuwvfyDniucD/a9f8X7MdesH62GL/xvC61jZskNnLfG2k9fWykl2esFOfixEybBApPLBT7/aMm/CuUbSvh7p0SS2SW9GMpu2SrS8JXGvzmtl87b9VLur/g7uzbKAFBR0c+6WDlyGjAZDCMaOnSF0a6M8k6eQeE5bBz3l+voG3/A+IHTYY=
+sidebar_class_name: "get api-method"
+info_path: reference/api/agenta-backend
+custom_edit_url: null
+---
+
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
+import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
+import OperationTabs from "@theme/OperationTabs";
+import TabItem from "@theme/TabItem";
+
+
+
+
+
+Fetches evaluation scenarios for a given evaluation ID.
+
+Arguments:
+evaluation_id (str): The ID of the evaluation for which to fetch scenarios.
+
+Raises:
+HTTPException: If the evaluation is not found or access is denied.
+
+Returns:
+List[EvaluationScenario]: A list of evaluation scenarios.
+
+
+
+
+
+
diff --git a/docs/docs/reference/api/fetch-evaluation-scenarios.api.mdx b/docs/docs/reference/api/fetch-evaluation-scenarios.api.mdx
new file mode 100644
index 0000000000..5c7f02757f
--- /dev/null
+++ b/docs/docs/reference/api/fetch-evaluation-scenarios.api.mdx
@@ -0,0 +1,771 @@
+---
+id: fetch-evaluation-scenarios
+title: "Fetch Evaluation Scenarios"
+description: "Fetches evaluation scenarios for a given evaluation ID."
+sidebar_label: "Fetch Evaluation Scenarios"
+hide_title: true
+hide_table_of_contents: true
+api: eJztV0tz2zgM/iscnNoZTpzN7Emn9SbZrafZJpNke3E8GUaCLbYSqZJQWq9H/30HFC3J8SNuzj3FIYEPwIcHoRWQWnhIpnD5rIpakbbGw0xChj51uuL/IYG/kNIcvcBOSPgUjXLaejG3Tiix0M9ohgKTi5MH82DGblGXaMgnD0YIMZB41Jl458m9T8R9jmJyIexcUI5DEMb+nus0F2TFnL3oDQf4W6U9rrE/3N/fXP5IMbidiMkWnPbCWBJzW5tMsNtpit7zcYZGY9ZCItXOrDGvtKdpT85dtD5LxFgU2hM7vYuWE5BgK3TheJJBAsH9x0H8nSxIqJRTJRI6TsYKjCoREtggCyRoTkalKAcJDr/V2mEGCbkaJfg0x1JBsgJaVqzsyWmzAAmkqeCDy0FyMmiaGYP4yhqPnvXOTk/5z2bq7+rA0bwuxG0UBgmpNYSGWFxVVaHTADv64llnte2Lck4tOQDCMtiqHJNDurWss4BkltfzEP5mBI3sTkxdFMCOr2PiQOQLno6ngBmtamp92O1am4n9iJ/4vpHxfr/cPZ83EtiBVjBefA4HTQcB9ukLprSR4WnrRhRZg8x2xbUu0AkH1jvWZaAjrg28kWBreoUDh74uaPv8p2PuMnwopZEQCeicdTsLYydFJXqvFpu0BIRtY6+xHe4GOLctAQ0XvqedLpm6fMJdtnqUc9ZtJBSK0KTLt8JcRfXXgohZO1gl1yH3B8rkOhZHCN05TOlRGf89zKkd7rfqh0kIKGIcUbgH/WOljcHdQ+DJ2gKVOTwFvLhpERoJxhK+dZx8Yt1GRu4O9UQcOdY9ptbM9eKIqWOdOG9lOwu/euo1tS2e5VGF3XfsnsK+jRk+0v7gCQ790I/NvlgOOhRc6W2Hh1SEvUoMHqW7biVoGvbs97Oz7Sf5syp01sq3yfiJ93iz2DIkpYsDVV7YdOP2iJ7ShnDBI2y2n/wr2zrIpJT+YO/8EyvwiCc2kCHaBnklqxxXa1puFWZP77rU94Vx0dK3y9hahFfSLcA2tyVSbnkxXCCFDZBySGDUF5wfrTaqrxntWR49uuf15li7AhLIiSqfjEZpYevsRC3QkDpReqQqDS+X+3G4FecsGoaGx7R2mpYBb3wz+YjLD6gydJBMZ0OBOy6rtlA2xfr3oNIfkSmLC+24ptw6/V+b/bjQ5q1WE9axuR0uR9G3P1X6FQ23H8fZun168tvJ6b5gooIY30xid6iU+nUuioF8QVdHFEjAMrQGEKryj/6Cvaysp1KZAd6BRn7h4arv1V+fVW/7rIq1RfiDRlWhdBgjIZGr2EXDsc0ZSF5O8Z2NNJOQh/VuCqvVk/L4ryuaho+/1ei4HWb87jqtnrg4pyvItOffGSRzVXg8kOp3t3H6vBf7Ili3jFl2G34CIOErLrc+BcNwzddNuYoy45STNNDeegu4e7u58/flPTTN/1ICiI8=
+sidebar_class_name: "get api-method"
+info_path: reference/api/agenta-backend
+custom_edit_url: null
+---
+
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
+import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
+import OperationTabs from "@theme/OperationTabs";
+import TabItem from "@theme/TabItem";
+
+
+
+
+
+Fetches evaluation scenarios for a given evaluation ID.
+
+Arguments:
+evaluation_id (str): The ID of the evaluation for which to fetch scenarios.
+
+Raises:
+HTTPException: If the evaluation is not found or access is denied.
+
+Returns:
+List[EvaluationScenario]: A list of evaluation scenarios.
+
+
+
+
+
+
diff --git a/docs/docs/reference/api/fetch-evaluation-status.api.mdx b/docs/docs/reference/api/fetch-evaluation-status.api.mdx
index 479ba9c3af..62df16b309 100644
--- a/docs/docs/reference/api/fetch-evaluation-status.api.mdx
+++ b/docs/docs/reference/api/fetch-evaluation-status.api.mdx
@@ -5,74 +5,299 @@ description: "Fetches the status of the evaluation."
sidebar_label: "Fetch Evaluation Status"
hide_title: true
hide_table_of_contents: true
-api: eJzVVU1v2zAM/SsGTysgxFmxk08LsG4NumFFm+2SBoVqM7Y6W3Ylulhm6L8PlL+TtsCOO0WmSOrx8ZFpgGRqIdrCxbPMa0mq1BZ2AhK0sVEVf0MEn5HiDG1AGQaWJNU2KPf+C4ewxZ2+0yuT2uhOB0EwublXSfDOkjmLjkIClbS+Bp9qtBS8u2kPnWdvLh8eMSbOf4NUG90/8XLSFiAIKCs03rROIII913A/QTX4VdLIAgkNE9GAlgVCBDP8IEAxEZWkDAQwMGUwgYhMjQJsnGEhIWqADhUHWzJKpyCAFOVsGOkN1gk4t+Mktiq1Rctx58sl/8xpv63jGK3d13lw0zmDgLjUhJrYXVZVrmKfNny0HNOMWJxzTsCH8/PTxD9lrpIWzYUxpfmHrFAZ5pVUiztBkirnkyIs7KlDXsazW6kP3/ee5zlVTgwWpQlTNOB2TvQ2aYw8TPj8WrYAwQkobPoW9d/QWpkiDMne6BKTEWz41o1vt/KbtX3r62qf7vx2Y5qR3pbd18v41NL30mO9y+Vmc32SsO1tgZSVrO0UyeuYMoggHJVrw2YmYxcOordonnvF1yaHCDKiykZhGOdlnSxkiprkQqpQVgpYrxbj2ig6+JDV9foKD5coEzQQbXdTh1vWStv9udtAvKzUFTIP3aytaspKo/60Le1mLWujuFCl96UP7zhZeXDB6noNAriMVtbLxfvFEo6X18yZZS5jL/P+bX8N4oiFoX4QgIXXOBDK4uN4wciq0lIh9SSf35XBZN5ve8pnsJpx4v7T9dr1kvA3hVUulZ9FT2LTSXE7WaIcER3v1C7XTkBWWuKApnmQFn+Y3Dk2P9VoWHA7Ac/SKPnA7d82kCjL5wSivcwtvsGtr5mH9ix4DXMvSs2KZIT8BQJ+4eHkf8DvpKyXfdP5rOIYK5pEn6xQno9hXL9cbMC5v6G8lJU=
+api: eJzVVU1v2zAM/SsGTy0gJFmxk0/Ltm4NumFFm+2SBoVqM7ZaW3Ylulhm6L8PlL+TtsCOO0WWHqmnx0emBpKJhXAD588yqySpQlvYCojRRkaV/A0hfEGKUrQBpRhYklTZoNj5L+zDZrf6Vi9NYsNbHQTB6OROxcGJJXMaHoQEKm6wBp8qtBScXDeLFtltF/cPGBHnv0aqjO6ueDlpQxAEFCUav7WKIYQdv+FuxKrHldLIHAkNC1GDljlCCBP+IECxEKWkFAQwMWUwhpBMhQJslGIuIayB9iUHWzJKJyCAFGW8McgbrGJwbstJbFloi5bjzhYL/pnKflNFEVq7q7LgugWDgKjQhJoYLssyU5FPO3+wHFMPXJxzTsD7s7PjxL9kpuKGzbkxhfmHrFAa1pVUwztGkirjlSLM7TEgK6LJqdT7Hzuv81QqJ/odpQkTNOC2TnR70hi5H+n5rWgIghOQ2+Qt6b+jtTJB6JO9USUWI1jzqRvubuw3KfvGv6u5usVthzSDvI26rz/jcyPfS5d1kIv1+uooYVPbHCkt2NsJkvcxpRDCfHCundcTG7t5b3qL5rlzfGUyCCElKm04n0dZUcUzmaAmOZNqLksFhwNh6U+DTwwFNrPFqDKK9j7f8mp1ifsLlDEaCDfbMeCGjdRYYwrrqyJLdYksUtuIy4rSwqg/Tb3bRkybKFZB6V3hw1vBWm4fZfSImjuX39nQXszezRavPaYNCJZXq7YfZOT7oePhYSAO5OqFAgGY+2YAQpl/GA6YZVlYyqUe5fNDNRgNhpuuNhN69dCa/+kcbutK+JvmZSaVb1ovYt16djOathwRHg7fNtdWQFpY4oC6vpcWf5rMOd5+qtCw+bYCnqVR8p6tsKkhVpbXMYQ7mVl8Q1v/Zu7u0+A1zp1BNbuTGfIXCHjE/dEfhh9eadcCdYtZRhGWNIo+mrXcK31ffz1fg3N/AT8ooyQ=
sidebar_class_name: "get api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
+>
-
-
-
-
+>
Fetches the status of the evaluation.
Args:
- evaluation_id (str): the evaluation id
- request (Request): the request object
+evaluation_id (str): the evaluation id
+request (Request): the request object
Returns:
- (str): the evaluation status
+(str): the evaluation status
-
+>
-
-
-
-
-
-
-
-
-
-
-
-
-
-
\ No newline at end of file
+
+
diff --git a/docs/docs/reference/api/fetch-evaluation.api.mdx b/docs/docs/reference/api/fetch-evaluation.api.mdx
new file mode 100644
index 0000000000..26f247a023
--- /dev/null
+++ b/docs/docs/reference/api/fetch-evaluation.api.mdx
@@ -0,0 +1,957 @@
+---
+id: fetch-evaluation
+title: "Fetch Evaluation"
+description: "Fetches a single evaluation based on its ID."
+sidebar_label: "Fetch Evaluation"
+hide_title: true
+hide_table_of_contents: true
+api: eJztV0tv4zYQ/ivEnHYB1U6DnnSqm6RdY/sIst5cHMNgpLHEXZnUkqOgrqH/XgwpyZJfyW6whwI92SJnPs7jm+FwCyQzB/Ecbp5kUUlSRjtYRJCiS6wq+Rti+BUpydEJKZzSWYECO2nxKB2mwmihyInp9ehBP+iJzVz8oIUQPcmlSsUbR/ZtLGY5ium1MCtB+QCMjFjxWR7lDqmyugXaGRj0vRymPe0RRGBKtP5jmkIMXma5k4AISmnlGgkte70FLdcIMQyshAgUe11KyiECi18qZTGFmGyFEbgkx7WEeAu0KVnZkVU6gwhIUcELO1PFNIW6XjCIK4126Fjv8uKCf4Yx/lAlCTq3qgpx1whDBInRhJpYXJZloRIPO/7kWGfbs6W07DqpcIJKz9nHRkWMtzwvNylLEWRLaz5hQs/I3wapRudJWiU16wSTCNfuULuO2gVprdz00O6Dvpimrg/HGXsd4J8eoQdp8Um5kPvXId81QK3NLfC3od512iyDjhy2GZB689fKE3gfrFvRVVEAU6+FmwWEJjstXiiA1yFyQBnTkaTKHbIx6J/mzYzXfT6KamjMudPvvXQdAVpr7FEfzCPzcVDDc1ijczJD6EHdeITDw+pdco5C+b1FP2OuKsgXV5ZZzCRhurR+cUiBYXya7mPsMjF6pbKX++KblU/hoEajHuRnZE4lFr0xkhGqMm0/Fgddy9irYASHQ6ap4oYji9uexaEPDu2qjyGJDgpCFP7nxnNqB1zoQtfDmnTs2jHuRBPZiYq7hohMzye0MsNlYhy9PDon6X4QmwjIkCy+I37rQSEJdbL5Pof0quYIRVfGrnkHuJp+IOWrsIW9Cqpi4pPTq7ivBfoYVD3QM9zxld9c63v9oH8Z79+lJy7C/vXV9fajne1r2gsPO3XNnvx0eXk4Bd3LQqVhdAqF980jUIokVXGm6xYmGey+4AZUmjBD2zD8eMH9bpLGywjWLjvX2/5ouk0Hdmak5GCI0AyfYQH7FY6ODsi+C2/b1k65cR3Cd+ywVuTdbHZ7ABhyu0bKDU/gGZIfuimHGMa7MduNt4OZu2aKoX1q5/LKFhBDTlS6eDxOClOlI5mhJjmSaixLBftvlInfFVcs6u8Ch0llFW083uR2+h4371CmaCGeL/oCH5hBgRNDsS4dslTv/TXaPBcmFeXGqn/aR4V/LuRBi91XemW8eteEvW2/yOQzal+OaF0w+2L04+jilDONgpjcTptCkIkvhNYOL8YlNwhXFygeBNa+CoBQrn/ebbCVpXG0lrqH5995vZfWvl3bXTH+h96ETRIJ/6ZxWUjlS9NHbNswc957AHI7i4fvwUUEub/I5rDdsm8fbVHXvPylQssEWzQ99JHTPd9Cqhz/TyFeycLhmTC+uWtK9604ZWpLQr2Bbg4CiIAnu/2nq+9MeUvzbSMzSRIsqad90Ei5Hrqi/e1mBnX9LyZMh+U=
+sidebar_class_name: "get api-method"
+info_path: reference/api/agenta-backend
+custom_edit_url: null
+---
+
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
+import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
+import OperationTabs from "@theme/OperationTabs";
+import TabItem from "@theme/TabItem";
+
+
+
+
+
+Fetches a single evaluation based on its ID.
+
+Args:
+evaluation_id (str): The ID of the evaluation to fetch.
+
+Returns:
+Evaluation: The fetched evaluation.
+
+
+
+
+
+
diff --git a/docs/docs/reference/api/fetch-evaluator-revision.api.mdx b/docs/docs/reference/api/fetch-evaluator-revision.api.mdx
deleted file mode 100644
index 94819468c9..0000000000
--- a/docs/docs/reference/api/fetch-evaluator-revision.api.mdx
+++ /dev/null
@@ -1,71 +0,0 @@
----
-id: fetch-evaluator-revision
-title: "Fetch Evaluator Revision"
-description: "Fetch Evaluator Revision"
-sidebar_label: "Fetch Evaluator Revision"
-hide_title: true
-hide_table_of_contents: true
-api: eJzNWdtu20YQ/RVhnhKAsVyjT3yq60vtJk0M3/pgCMaIHEqbLLnMXuyqAv+92AspUZRUSZYAP1kWZ87snJndHR5NQeNIQfwEFy/IDWohFQwiECVJ1EwU1ynEkJFOxs9UWzxLemGKiQIiKFFiTpqkBZlCgTlBDF3TZ5ZCBKyAGErUY4hA0k/DJKUQa2koApWMKUeIp6AnpQVRWrJiBBFkQuaoIQZjHIpmmluDZsm92xCld51CVQ0suCpFoUhZvJPjY/snJZVIVtqsIIY7kySkVGZ47zYYQwSJKDQV2ppjWXKWOBL635X1mc6tsZSWIs18hEQY7xSWzgpNI5Jzaz1zFhGklKHhGuLjKlpCkwtcTL5ljs12kJn1C0qGhbactuzXE1dFjUVhOAfLU5fKR4/tmIzgVcgfGRevBwj5d4BeiDjL8hDZLWR1gGx8hP3T1WYJpWYZJvsMcBog6whGj4XcG7hHqyJIUdOGqNb0k2Y5rYU+t4BVBDkphaO12Otg/gruVQQZd2fi6p3I1HNilBb50mBDIThhsTbateqdeYQqsnBN178FsWnzADo2ORZvAbxyAFUV1UZi+J0SvewIvnScdfFcYTS2V4Fpyuy5ivymxesGdatP1lakfNj+Zj695bEgYTIxHOWHS8P5n0oUn74ZXRr9ERaTtSBMU76hF0qJk7XELvhCh971barReXRa9D2R+lV8YUr/L62rkuy4b8fRvW/FMIzsdhx8RXfstIeG3aDO5yCqCBJJqCl9Rr3XY/DMw/ZOHVmmTA8R5MHDhiApcTpAkHMPG4LUdA0ne7zrarJ+n4TbruZrr1FqtpooNWF7jVLT1UR5IdkdJTdv18fgXkWguBntCnNnfe09tLdEA4e4eJu07+b9ZW8k2xXlQTKPwHdH4BZhTJi6d6vtTvpDUfL+GmLNbHJLGUkqktZpUy90sNWNchXKUNWvqZtW5B3dye9y0NmiBneBeDtRYlmyYtMBaJHibaatOpArvL3Ed+5+7121FZP3Or+9y15Z7Xwzo9QWiuQLS2gTdr3ytEUPBujKKUUZGxmvkh0i1lkrwEYvYbUMdo7uJaHzLrYNxlv9G1GtqqzfrycnXQ3uETlLXX69CymF3F2AS0kjc5dt6MpFAy6S1tNtdtXcXeEbeZb2F5FgPdjnarRMvVwiMHiT1aaOjN69fbqM85ly+uTy8qGDXUsyqun17K5O49zTt67AV/f3Nx1AX9t2US+tWNzrarPglICxsILyiLTTjvUYYuiXVv2k134jgqh+rYeq/nSpllyB3+e17uwmLRhrXaq430+4MOkRjqjQeISsjyVzW0tRYiTTE+dyenP9mSb+Zof4aTBv4O4a3zhts6ZmWLLPZCkMircXuNi/vhuC0u2nN0crKzLh3GtBzC2ud3pzDXMTOxwf/XJ0DIuUtoztDsHE7ZA6tnsM0QILTf4QAeVue4AmzH+bPbArK4XSQScKeGsquPA6HMjQ9I/ulxxZMTf1+uo+QaguzEm7yjVxqDBEEC//vWAQwVgobUGm0yEqepC8quzXPw1JW8VBUFqHltOnKaRM2c8pxBlyRZ31NqcLfLgNm+hjb3ZNtfOoK13YMtsF2v8ggh80WfkThzsrxnVPTYPtaZJQqedQOkebbb5md/xxcQ9V9R942M1s
-sidebar_class_name: "get api-method"
-info_path: reference/api/agenta-api
-custom_edit_url: null
----
-
-import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
-import OperationTabs from "@theme/OperationTabs";
-import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
-
-
-
-
-
-
-
-
-
-Fetch Evaluator Revision
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
\ No newline at end of file
diff --git a/docs/docs/reference/api/fetch-evaluator-variant.api.mdx b/docs/docs/reference/api/fetch-evaluator-variant.api.mdx
deleted file mode 100644
index 68c33fa925..0000000000
--- a/docs/docs/reference/api/fetch-evaluator-variant.api.mdx
+++ /dev/null
@@ -1,71 +0,0 @@
----
-id: fetch-evaluator-variant
-title: "Fetch Evaluator Variant"
-description: "Fetch Evaluator Variant"
-sidebar_label: "Fetch Evaluator Variant"
-hide_title: true
-hide_table_of_contents: true
-api: eJzNV01z2zYQ/SuaPSUziOV6euKpqj9qN2njiZ304NF4VuRKRAISDLBwqmr43zsAQUoUJdd2lJme/MHd93bfYoHdFTAuLCR3cP6AyiFrY2EqQFdkkKUurzJIYE6c5vfUWtw/oJFYMgio0GBBTMZjrKDEgiCBgeW9zECALCGBCjkHAYa+Omkog4SNIwE2zalASFbAy8pjWDayXICAuTYFMiTgXEBhycobdAGPPjUko6sM6nrqsW2lS0vWw50cH/sfGdnUyMqnBAncuDQla+dOjT5EYxCQ6pKpZG+OVaVkGhQYf7beZ7URYmW8PiwbhlS7xilGLkumBZmNUE+DhYCM5ugUQ3Jci6FIgbdcvp8HKfsca2OZ9e0el6sWnUXplAIvz1BAL5yAb9p8mSv97YAUf0XIyICG5RxTPiDDJEJGhrkKx3m/kNLep86yLnYGMNNaEZaPMl7Z0WmDUAsP15XmexC7WkTQ3BVYfg/gZQCoa9Ea6dlnSnlX/1wEzYZ4tYCCGPtRYJZJ3xWornu67qvlBmzbFz2mYtb/z2Z6u7kglSZ1Cs2rC6fU71aXb947rhy/hu1kPYhkKp7ohcbg8lFht3xhIO9jzn94Mb3H4Ij+n0T9U7+Tlv9T1n1JDtyfp9FtcxTjQ/IkBYYheN9a9K/8l0GdbUDUAlJDyJTdIz/x+sqQ6Q3LEM9+ltMGdjQJYrkq+xEkHxvYSJKRoh9ActbARpJWrtnygBd+K9avy3jlt3odlKVVq2NpBTsoSytXx2KVW7z0rN54X/96HCy8ME894QGJ89euJ+QZ7t0gVtfe7eeTk+Hc9gmVzMJUNjo3RpuXD20ZMUoVJoLmgdg2UDrtfX3OXTytt96UddbvdBOgr1RhF7sG3vV7YS0uaH2B7jcNYoxu/dddkq+H7buQV0Md7TYKvpa3UXd/GmeNfI/V9/L29noA2NS2X9QLv12MBuM8hPEj134BWVCzbHAOCYwrQw+Svo27ycuO4whtx6tdu0cNAiyZh3ZNcUZBAjlzZZPxOFXaZUe4oJLxCOUYKxnawFLqjORlcJlcX72l5SVhRgaSu+mmwY0/Y82p6Zt1BcNKviWvX1yQJo5zbeQ/zVGIm1HeeHmBZDnXwb2dckNwo8n1FQjwaTTKHR/9dHQM23r2jH17YBrao+UOn0FsqdDlDwKoCL0BTFj8sv7gI6u05TiaRrz95dt6gKMWTH/zuFIoQw+EIFaxtHcQSwsb65H1KcfygoBk53I5FZBryx5itZqhpY9G1bX/91dHxpdwGlFmXtC7FWTS+t8zSOaoLA2i7e4VePUhts/r0XoU6mfRlrlchmCV83+BgC+03LcOh0sib8/TKppO0pQq3gAZ3Gn+4HV98dv5LdT1v/I/Z7g=
-sidebar_class_name: "get api-method"
-info_path: reference/api/agenta-api
-custom_edit_url: null
----
-
-import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
-import OperationTabs from "@theme/OperationTabs";
-import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
-
-
-
-
-
-
-
-
-
-Fetch Evaluator Variant
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
\ No newline at end of file
diff --git a/docs/docs/reference/api/fetch-evaluator.api.mdx b/docs/docs/reference/api/fetch-evaluator.api.mdx
index cc07cf7275..6b5a20de3a 100644
--- a/docs/docs/reference/api/fetch-evaluator.api.mdx
+++ b/docs/docs/reference/api/fetch-evaluator.api.mdx
@@ -5,67 +5,641 @@ description: "Fetch Evaluator"
sidebar_label: "Fetch Evaluator"
hide_title: true
hide_table_of_contents: true
-api: eJzNV01z20YM/SsanJKZjaV6euKpqj9qN2njiZVeNBrPioTETZdcZhfrVtHwv3ew/BApSq7sOjM9SSKBB+ABu3jaAsm1g2gOV49Se0nGOlgIMAVaScrktwlEsEKK0wdsLEBAIa3MkNCy7xZymSFE0Fo8qAQEqBwiKCSlIMDiV68sJhCR9SjAxSlmEqIt0KZgX0dW5WsQsDI2kwQReB9QSJFmgzbB0W0CZblgTFeY3KFjmPPJhD8SdLFVBacOEdz7OEbnVl6PPtXGICA2OWFObC6LQqs4VDr+4thn20mtsMwDqSpCbHzlVGescsI12k6KF8FCQIIr6TVBNCnFjpQQL998XAXK+tgrHdpw3EC5h9g7MlnfqE5laYxGmUMp2ke51xqYpSa5Wze6qBBKwXBH8noeYtuUGjT1mcz/C+BNAChL0RiZ5ReM6dAcXAfOhnilgAxJ9rOQSaK4y1Lf9Xgd5FnPYQe26XMvUrbsP+mWdzgWxMrGXkv75tpr/asz+buPngpPb2G/WAZRhNmJXtJauXmS2D1fGND7lPNvTCZ7DEb0/0Tq7+aDcvSvtB4rcuD+PI5m1SjWF+FJDAxTYN9S9K+wl0FddiBKAbFFSZg8SHoKsHP1JpLwHamQz/EoFxXsaBrI8kXyPYJ8rmDrIAlq/A5BLivYOkhD13LDe+y0OGFZnULWz5uwv3Z8vWqUhq02SkPYq0Zp6GqjOO3XL53Ve/bl7fFq6QV9cMICObQ7TvFrpURZsv2P5+dD5fGH1CoJumJ0ZW3QTC+UHQmSVDpogGol7BtoE/fePuf2XZR7W2RX7gdTJci9ydz6kFTbbQjn5Bp3V+Zx00DGaMZvD3G9k4nzUFcVurbrtHhHb8Xu8TIuK/qeauzNbHY3AKx622/qNevgjuYJQiM1LJHXSEEWUwoRjAuLjwr/Grcay423XXFcggCH9rHRz95qiCAlKlw0Hsfa+ORMrjEneSbVWBYqzLfD2FtFm+Ayvbt9j5sblAlaiOaLrsE9j1I1HH2zti+yUO+RaaqV+9RTaqz6VnW8lu5p5cU8qHxlgntN2TQkN5re3YIALqMiaHL2w9kE9mnrGfMpkHE4BU3s8BrEHgtt/SAAs3AEgFBmP+1ecGaFcVRrzhpv2KW9jVpzQPg3jQstVRjxEHxbd3AOdQeho98dCIh6f3EWAlLjiO2326V0+NnqsuTHXz1a7tNCwKO0Si6ZtfkWEuX4ewLRSmqHg9TaOwLefKqPwtvRTsj0U256mXMjOS/+BQL+xM3+n7Fw0NNmWLa1yTSOsaCO8+Be4qlqZ/yXqxmU5T/tE9lb
+api: eJydVU1v2zAM/SsGTxsgJFmxk0/LumwNumFFm+0SBIUqM7Fa21IlOltm+L8PlD/ifGHATnGsR+rxPZKugOTGQ7yE2VZmHlYCjEUnSZtinkAMaySVPuJWZqUk40CAlU7mSOg4rIJC5ggx9IhHnYAAXUAMVlIKAhy+ltphAjG5EgV4lWIuIa5AFrvv65CFdpazeHK62ICAtXG5JIihLHUCtThG1CsBpCnjV7Pu6mieQM0nDr01hUfPl1xNJvyToFdOWy4MYngolULv12UW3bdgEKBMQVhQoGZtplXQYfzsOaYaELeOVSLd3KBM2QS1HHVBuEHWqqN4HRC12Mt0tnzz9IyK4ExpQwmKMsugXtW1OInbS71sWZ3Tqa+4rjnJ+6urU4F+ykwnofxo5lww/j/VSZCkzvhJE+b+FJAZdXB6qSsGEnQK16u9CNI5uRto99U0BFn23G8G/vRd1kG/ofdyg9AnuwwNYkQLPv2XAVxXc3WLG3ixl7dR93IZnxr5zl3WQW4Wi7uThI23h6Z+5mGOZoNhzpFSw3O+QQqzTSnEMLYOtxp/jft29eNqOOE1CPDott0SKF0GMaRE1sfjscpMmYzkBguSI6nH0mo4pjINp9E1Q8M0e1Sl07QL+aZ381vc3aBM0EG8XA0BD9xnTeccwnrTpNW3yBq2u2laUmqc/tO0Q7uc0iaKRdLF2oTwVs+W20epXrDgbcZ1NrQno3ejyaVi2oBoejdvx0WqMC4djwADcSRXLxQIwDzMChDK/MP+gFla4ymXxSDfqZ0HtHo9CH/T2GZSh1kIl1et1UtorYbBbvIgID5Y6CsBqfHE+Kp6kh5/uKyu+fVriY49WwnYSqflEyu4rCDRnp8TiNcy83hCrV8m8Oa+nZm3EYjzlDtfCzaVefE/EPCCu+NPT9gIadc4VQuZKoWWBsEnC4w7rB+GL7MF1PVfgqRuyw==
sidebar_class_name: "get api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
-
-
-
-
-
-
+
+
Fetch Evaluator
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
+
-
\ No newline at end of file
+
+
+
Path Parameters
+
+
+
+
+
+
+
+
+
+
+
+
Successful Response
+
+
+
+
+
+
+
+ Schema
+
+
+
+
+
+
+
+ evaluator
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ name
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ description
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ slug
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ flags
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ is_custom
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_evaluator
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_human
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ metadata
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+ data
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ service
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+ configuration
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Validation Error
+
+
+
+
+
+
+
+ Schema
+
+
+
+
+
+
+
+ detail
+ object[]
+
+
+
+
+
+ Array [
+
+
+
+
+
+
+ loc
+ object[]
+
+ required
+
+
+
+
+
+ Array [
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+ integer
+
+
+
+
+
+
+ ]
+
+
+
+
+
+
+
+
+
+ ]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/docs/docs/reference/api/fetch-human-evaluation-human-evaluations-evaluation-id-get.api.mdx b/docs/docs/reference/api/fetch-human-evaluation-human-evaluations-evaluation-id-get.api.mdx
index 41250d4702..be473f2825 100644
--- a/docs/docs/reference/api/fetch-human-evaluation-human-evaluations-evaluation-id-get.api.mdx
+++ b/docs/docs/reference/api/fetch-human-evaluation-human-evaluations-evaluation-id-get.api.mdx
@@ -5,73 +5,430 @@ description: "Fetches a single evaluation based on its ID."
sidebar_label: "Fetch Human Evaluation"
hide_title: true
hide_table_of_contents: true
-api: eJzVVk1v2zgQ/SvEnFpAtbPBnnRaY5Ntje5H4Li9OIbASGOLXYliyVGwXkH/vRhSsiS7cYr21JMt6c3jfLyZYQMk9w7iDbyrS6nf3D7JopakKu1gG0GGLrXK8DPE8AdSmqMTUjil9wUKPKLFo3SYiUoLRU4sb2YP+kEv7N7FD1oIMUImKhOvHNnXsVjnKJY3otoJyidkVIkdn+VZVki11T2Rd3PwMpB4MGYjihlEUBm0/mGZQQwek+Rsnoy8OX3hkmTia5LskSACI60skdByshrQskSIYQKFCBSnyUjKIQKLn2tlMYOYbI0RuDTHUkLcAB0MGzuySu8hAlJU8IshLLHMoG23TOJMpR06tru+uuKfaVHu6zRF53Z1IVYdGCJIK02oieHSmEKlnnb+ybFNM/LFWE4TqXCCyi75x05FzJdcxi2MEQFrbPUJU3oBfxdQnc0op8HgmxK2ZkgbwZO0Smo+MQREWLpzijbqX0hr5WFE+THYi2XmxnRc7x8j/NszDJQusfikXJDOD1E7seqYeq975u+jXR2tGYOOHL5UwXVAdRXsbUKTvGzFqWE7R5Lqr7g6WNwHRBtBalESZomkS/jfA0osiG1qk32DzYeA8jZDkqpHluikrTfge77rh4nYz1U8VeapsJ5TxbiUk1qcJPmYu0liJhFvhwhPZii0LQf66/X1+XT5KAuVhQ67tbay3z9aMiSpiokgp4CiSidfpT78s/PD9lS4xzdKE+7RQrt9Xs1/VmkXZQSl21+q/F/onNzj0BoXJg8noxs6L4iE4wpHd7jtuIP79IbsPh/GTUjf1w47lnW9vjsjDLUtkfKKt2C/zCiHGOZ++b0ZLb95M9loLQsL7VO/9WpbQAw5kXHxfJ4WVZ3N5B41yZlUc2kU8M5ymNZW0cGbLO6W7/HwDmWGFuLNdgy4Z6mE4k9hx7xLo94jp6Hbt4ua8sqq/0NFu32bByuOU+ld5c37TeSdE4u7JbcYWhdUfTX7ZXYFp9ebCZhVLlOv8v5s/5mbapKFY/zc9KWXOBDK8rfhA3tmKkel1CM+f5sK9xkxasYTr5qh3362+1dXQ8L/aG4KqXwL+uQ1nQI3cKZAiCCe3qq2EeSVI0Y3DYf5wRZty68/12hZZdtufD5yzTcNZMrx/wzinSwcXsjoq1XXqK/Fcw73StQHP6WLmp8ggn/xcHYB9HMo77XedJhFmqKhkfXZ2OSmOLbo29s1tO0X4FkAhA==
+api: eJzVVk2P2zYQ/SvEnBJAsTeLnnSKm90mRtpm4XVy8RoCVxpbTCSKIUeLuoL+ezGkZEt27A2SU0+2pDePM2++2ADJrYN4Be/rUupXt0+yqCWpSjtYR5ChS60y/Awx/IGU5uiEFE7pbYEC92jxKB1motJCkRPzm8mDftAzu3XxgxZCDJCJysQLR/ZlLJY5ivmNqDaC8hEZVWLDZ3mWBVJtdU/k3Tx4GUg8GLMBxQQiqAxa/zDPIAaPSXI2TwbeHL9wSTLyNUm2SBCBkVaWSGhZrAa0LBFiGEEhAsUyGUk5RGDxW60sZhCTrTECl+ZYSogboJ1hY0dW6S1EQIoKfnEIS8wzaNs1kzhTaYeO7a6vrvhnnJT7Ok3RuU1diEUHhgjSShNqYrg0plCpp51+cWzTDHwxlmUiFU5Q2SX/2KmI+ZLLuJkxImCNrb5gSs/g7wKqsxloGgx+SLAlQ9oInqRVUvOJISDC0p1StFH/QlordwPKz8FezDM3pON8/xrh357hQOkSi0/KhdL5JWonFh1T73XP/HO0i701Y9CRw+cyuAyoLoO9TWiS561YGrZzJKn+jqsHi/uAaCNILUrCLJF0Cf82oMSM2KY22Q/YfAoob3MQqXrkEh219Qp8z3f9MCr20yoeV+ZxYZ2rimEqR7k4Enmv3UiYUcTrQ4RHMxTalgP97fr6dLp8loXKQofdWlvZnx8tGZJUxaggx4CiSkdfpd593Phhe1y4+zdKE27RQrs+X81/VmkXZQSl217K/F/onNzioTUuTB4Woxs6zxQJxxWO7nDrYQf38gZ1z4dxE+T73mH7tC6XdyeEIbclUl7xFuyXGeUQw9Qvv1eD5TdtRhut5cJC+9RvvdoWEENOZFw8naZFVWcTuUVNciLVVBoFx1eGmf8q3jIUeKE5TGuraOf5ZnfzD7h7jzJDC/FqPQTccx2FyhjD9kmRRn1A1qhbxrOa8sqqf0O6u2WcBysWQelN5c37NRV8+12mX1FzS3Gcwe2ryevJ1blgOgMxu5t37SBT3w69Hx7G3TeSay8UT4fS9wIQyvLN4QN7aSpHpdQDPn/tChcfMejaI++aQ2P+3y5qXT4J/6GpKaTyverFa7pSXcFJqUIE8fj6tY4grxwxumk4zE+2aFt+/a1GyxW37ubsI+d/1UCmHP/PIN7IwuEFRV8suo5+Kc453Fel3vlxXtT8BBF8xd3JTdEPrLyv+6bDzNIUDQ2sT+YrN8i+l9/dLqFt/wPfwg8T
sidebar_class_name: "get api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
+>
-
-
-
-
+>
Fetches a single evaluation based on its ID.
Args:
- evaluation_id (str): The ID of the evaluation to fetch.
+evaluation_id (str): The ID of the evaluation to fetch.
Returns:
- HumanEvaluation: The fetched evaluation.
+HumanEvaluation: The fetched evaluation.
-
+>
-
-
-
-
-
-
-
-
-
-
-
-
-
-
\ No newline at end of file
+
+
diff --git a/docs/docs/reference/api/fetch-human-evaluation-scenarios.api.mdx b/docs/docs/reference/api/fetch-human-evaluation-scenarios.api.mdx
index c8e103e27d..da17e7a15f 100644
--- a/docs/docs/reference/api/fetch-human-evaluation-scenarios.api.mdx
+++ b/docs/docs/reference/api/fetch-human-evaluation-scenarios.api.mdx
@@ -5,76 +5,668 @@ description: "Fetches evaluation scenarios for a given evaluation ID."
sidebar_label: "Fetch Human Evaluation Scenarios"
hide_title: true
hide_table_of_contents: true
-api: eJztVsFy20YM/ZUdnJIZRnI9PfFUTa3WmqSxxlJ8kTWaNQmJm5K7zC7oROXw3ztYkhIpWZLjXnuyvAQegAc87JZAcuMgXMBtkUn9Yfws00KSMtrBMoAYXWRVzv9DCH8gRQk6gTsj4SLU0irjxNpYIcVGPaPuGkxuBo/6UY/spshQkwsftRCiY7FSsXjnyL4PxTxBMbkRZi0owS4IY39PVJQIMmLNWewDe/h7qRy22Lfz+XT8I0KfdigmR3DKCW1IrE2hY8FpRxE6x8cxaoVxDYlUWN1iflKOFntyZk30ZShGIlWOOOmXaBlAACZH648nMYTg018lzPaqw8LOAwLIpZUZElpuTAlaZggh9CiDABS3JJeUQAAWvxXKYgwh2QIDcFGCmYSwBNrm7OzIKr2BAEhRygfjTotiqKolg7jcaIeO/a6vrvhPfwBmhWdqXaTivjGGACKjCTWxuczzVEUedvjVsU95nIu0Vm65AMLMx8otU0Sqjqxij6S3d2tffr+CKtid6CJNgRNva+JCggOeXk8BM5oXVOdwIjU2WNX9OI07YSvxma1a0BWHeoXTgzerqqC1M09fMaJehxfdNPr4HS68nI8H1oeBPX7bi14ajg1MQRfYeJZWSU0XWH6orRqKW58a/TV+d7XlJU46yRxFuUxLG+QkL3cNG1yCIXzrhD6wb8X6NPa1IEoTbtCexZ15vIq1aC1GtJLafUf71jR/r1HEqEbhKXarXGmNL2vzyZgUpT4vTiemNUIVgP4PHH72HF4Yh+Nl6du3H+vLM+GnoTFpt53wV6DwHqKzP2a77V1VnNqv19fH2/NBpiqu7cfWGvszq7OvvBhJqvSMNFMT9b7+zJgtT6vgk6kTZGoytzkn37/QObnBvaTO7GEmQ8z566W2cl116Mauq60dvTW7p8u4qel7KdhuJubz6RFg3dsMKTF8k2+Q/GVNCYQw9Df6h/3cuWHZG8JqeOK2d2if26u+sCmEkBDlLhwOo9QU8UBuUJMcSDWUufJKcBgVVtHWu4ymk4+4vUUZs94Xy67BjOennoi+2f4uztVHZG6aR8aooMRY9U/d5uaRkdRelb/N1sa7NzyNfHJiNJ3w2kXr6lG/GvwyuILD92PPmEdfRn7029j+MwQHLOzqhwAw83MPhDL7bf+BM8uNo0zqDt5FrR7kV+7l+P9T921P3WaqCH/QME+l8pvCt7NshLKAI6FAAOHhvn5RK8sAEuOIQcrySTr8YtOq4uNvBVqWw7K5+594OBclxMrx7xjCtUwdnmn4u/tmzbwXp+poJaO3/onhH3QAAfyN26Pnud+iSSvKsrEZRdyqjvfR0mf17hbMn+M5VNW/asW9Lw==
+api: eJztVsFy2zYQ/RXMnpIZRnI9PfFUNVZqTdJYY6u+yBoNTK5EJCTAAEsnKof/3lmQlEjJkhzn2pNl8O3D4u2+BUoguXYQzuG6yKR+N36SaSFJGe1gEUCMLrIq5/8hhA9IUYJO4BYkXIRaWmWcWBkrpFirJ9RdwORq8KAf9Miuiww1ufBBCyE6iKWKxRtH9m0oZgmKyZUwK0EJdkmY+3uiokSQESvOYrexp7+VymHLfT2bTcc/IvRph2JyQKec0IbEyhQ6Fpx2FKFzvByjVhjXlEiF1S3nJ+VovhPnrtl9EYqRSJUjTvo5WQYQgMnR+uVJDCH49JcJq73sqLCNgAByaWWGhJYLU4KWGUIIPckgAMUlySUlEIDFb4WyGENItsAAXJRgJiEsgTY5BzuySq8hAFKU8sK4U6IYqmrBJC432qHjuMuLC/7Tb4C7wiu1KlJx24AhgMhoQk0Ml3meqsjTDr84jikPc5HWyg0fgDDze+WWJSJV76xiz6Q3Nyt//P4JqmC7oos0BU68PRMfJNjT6eUSsKJ5QXUOR1JjwLKux3HeCaPEZ0a1pEve6gVB9x5WVUGLM49fMKJehefdNPr8HS28nQ8b1m8DO/62Fr00HANMQWfUeJJWSU1nVL6vUY3EbUzN/pK4mxp5TpNOMge7nJel3eSoLjeNGnwEQ/jaDr3n2Ir9aexLSZQmXKM9yXvn+Sr2orUY0VJq9x3ta9N8X7OIUc3CXeyWudIan/fmozEpSn3anE5Ma4YqAP0LGn72Gp5ph8Nh6cu3a+vzPeG7oYG00074K1D4CNGZH3fb6V1VnNrvl5eH0/Nepiqu8WNrjf2Z0dl3XowkVXrCmqmJel9/ps0Wx13wydQJsjSZW5+y79/onFzjzlIn5jCLIWb89VxZ+Vz11g2u662tvLW6x49xVcv33GbbnpjNpgeEdW0zpMTwTb5G8pc1JRDC0N/o73Z954Zlrwmr4ZHb3qF9aq/6wqYQQkKUu3A4jFJTxAO5Rk1yINVQ5gr232Qj/1W8Z6i3icOosIo2nm80nXzEzTXKmIfBfNEF3HFz1e3Sh+0u6lx9RBaueYGMCkqMVf/WPdC8QJI6qvJX3cr48EbEJrc/ZfQVtZ/LaF2d9sXgt8HFscM0AWI0nTQekZH3SJuHh0GwJ9dWKAgAM28QIJTZH7sPnGVuHGVSd/jOmnovz3Ln2//fxK97EzcdRviDhnkqlR8pvpxl46g5HDgKAgj3B/uzploEkBhHTFKWj9LhPzatKl7+VqBlayyaR8IjN+q8hFg5/h1DuJKpwxMFf3PbzKO34tg5WvvojX+L+JcfQABfcXPwjvfjNmkNWjaYUcSl6kQf3A7s5O0k+ms8g6r6Dzy/y74=
sidebar_class_name: "get api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
+>
-
-
-
-
+>
Fetches evaluation scenarios for a given evaluation ID.
Arguments:
- evaluation_id (str): The ID of the evaluation for which to fetch scenarios.
+evaluation_id (str): The ID of the evaluation for which to fetch scenarios.
Raises:
- HTTPException: If the evaluation is not found or access is denied.
+HTTPException: If the evaluation is not found or access is denied.
Returns:
- List[EvaluationScenario]: A list of evaluation scenarios.
+List[EvaluationScenario]: A list of evaluation scenarios.
-
+>
-
-
-
-
-
-
-
-
-
-
-
-
-
-
\ No newline at end of file
+
+
diff --git a/docs/docs/reference/api/fetch-list-human-evaluations-human-evaluations-get.api.mdx b/docs/docs/reference/api/fetch-list-human-evaluations-human-evaluations-get.api.mdx
index c79ae06c90..3167a00bdb 100644
--- a/docs/docs/reference/api/fetch-list-human-evaluations-human-evaluations-get.api.mdx
+++ b/docs/docs/reference/api/fetch-list-human-evaluations-human-evaluations-get.api.mdx
@@ -5,73 +5,450 @@ description: "Fetches a list of evaluations, optionally filtered by an app ID."
sidebar_label: "Fetch List Human Evaluations"
hide_title: true
hide_table_of_contents: true
-api: eJzdVkuP2zYQ/ivEnBJAsTeLnnSK0WwTI2my8Dq5eAWDlsYWE4lSyNGirqH/Xgypp1+bNreeDFPfN5zHNzM8AMmdhXAF76tc6ld3TzKrJKlCW4gCSNDGRpX8H0L4AylO0QopMmVJFFuBPTwQhcPJLNuLrcoIDSZisxdSC1mWYv528qgf9czsbPiohRB8ulaJePG54a0smehlKGa6M9UwBRWNSUEpDm91NhdIldGt2Y/K0spF0wcThWJ2zukJBFCUaNy/eQIhbDnGNUPXKdtYD9BnTnZIEEApjcyR0HAmD6BljhCCjw8CUJy8HxWaPQRg8EelDCYQkqkwABunmEsID0D7kmmWjNI7CIAUZXww4xwkUNcRs21ZaIuWCbc3N/wzLtJDFcdo7bbKxKIBQwBxoQk1MVyWZaZi5//0m2XO4dQJaYxkbxVh7u4qDeeJlL9ZJdccZmeDNvyfCCxg698wpmfw9x7VcPoyrD3hMrEXglgypA7gSRolNd/oA2rjHJuog+OEtCa/er6YJ3Zojkv/awY/OQu9Sbs2+KQsh/mLvlqxaCy1XreW/5vZRcdmDFqy+FwFlx7VVLDl+H55nsWpYZ4lSdUZV3vGg0fUAcQGJWGylnQN/7tHiRkxpyqTn+B88SjH6ZNUbFiioz5fgZsC3TgYiP1UxWNlHgvrkiqGpRzV4ijJXe5GiRlFHPURHg1RV7Ku9n60CLcT3MwVDi4GG+TMyTvkbHG+fru9PR1eX2WmEt+od8YU5t9MrvGESpCkyka6HgOyIh59lXr/eevG97H+uxOlCXdooI4uN8XHIu6SldvdNQH9idbKHfYddmWAcTKa2fWM1jguf3WDi4aDoE2vz+7lMN769J27rFPHcnl/YtDXNkdKC96m7XqkFEKYuvX5arA+WY9ontq9WZkMQkiJShtOp3FWVMlE7lCTnEg1laUCXoEW48oo2jvK7H7+AffvUSZoIFxFQ8ADS8MXewzrt1ypPiCH3WzsWUVpYdTfvoLN4k49i+NSels4ervAnHNidj/nzkRjvYpvJq8nN3D8ehqBWdUydqpu73afuRdHWeji51mRO0kDoczf9B/Ys7KwlEs9sHetMY99O/Rd9j995DUFJ/yLpmUmletPl+lDI88VnMozCiAtLPHHw2EjLX4xWV3zsX/NsQITZeUm497byszilcy+WDRt+lJc8ug77ocPR3aFQU73bvJvWHcrnj9pq/lDw5rFMZY0YJ2MS7bStea7uyXU9T8xjSRr
+api: eJzdVkuP2zYQ/ivEnBJAsbeLnnSqmmwTI2mz8Dq5eA2DlsYWE4lSyNGirqH/Hgypp1+bNreeDFPfN5zHNzM8AMmdhXAJ76pc6ld3TzKrJKlCW1gFkKCNjSr5P4TwB1KcohVSZMqSKLYCe3ggCoeTWbYXW5URGkzEZi+kFrIsxezN5FE/6sjsbPiohRB8ulaJePGx4S0tmdXLUES6M9UwBRWNSUEpDm91NudIldGt2Q/K0tJF0wezCkV0zukJBFCUaNy/WQIhbDnGNUPXKdtYD9BnTnZIEEApjcyR0HAmD6BljhCCjw8CUJy8bxWaPQRg8FulDCYQkqkwABunmEsID0D7kmmWjNI7CIAUZXwQcQ4SqOsVs21ZaIuWCbc3N/wzLtJDFcdo7bbKxLwBQwBxoQk1MVyWZaZi5//0i2XO4dQJaYxkbxVh7u4qDeeJlL9ZJdccZmeDNvwfCCxg618wpmfw9x7VcPoyrD3hMrEXglgwpA7gSRolNd/oA2rjHJuog+OEtCY/e76YJXZojkv/cwb/chZ6k3Zt8ElZDvMnfbVi3lhqvW4t/zez847NGLRk8bkKLjyqqWDL8f3yPItTwzxLkqozrvaMB4+oA4gNSsJkLeka/rVHiYiYU5XJD3A+eZTj9EkqNizRUZ8vwU2BbhwMxH6q4rEyj4V1SRXDUo5qcZTkLnejxIwiXvURHg1RV7Ku9n60CLcT3MwVDi4GG+TMyVvkbHG+fr29PR1en2WmEt+od8YU5t9MrvGESpCkyka6HgOyIh59lXr/cevG97H+uxOlCXdooF5dbooPRdwlK7e7awL6E62VO+w77MoA42Q0s+sZrXFc/uoGtxoOgja9PruXw3jj03fusk4di8X9iUFf2xwpLXibtuuRUghh6tbnq8H6ZD2ieWr3ZmUyCCElKm04ncZZUSUTuUNNciLVVJYKjl8kkfsqXjMUeD9ajCujaO/sRfez97h/hzJBA+FyNQQ8sG68EsawfgWW6j1yTpp1HlWUFkb948vbbPXUszhopbeFo7fbzfv2u4y/ouZO5Di92zeTXyY3l4JpCCK6nzXyl7GTf+uHg3HTjtLVJYqHSu60D4Qy/63/wF6WhaVc6oG9ax187OOhb8f/6WuwKT7h3zQtM6lcI7tMHxodL+FUx6sA0sISfzwcNtLiJ5PVNR/7Zx+rMVFWbjJu0q3MLF7J7It5088vxSWPvuJ++MJkVxjkesCtiA1rcMmDKm31f2hYURxjSQPWyVxlK10Pv71bQF1/B8jEMvo=
sidebar_class_name: "get api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
-
-
-
-
-
+>
+
Fetches a list of evaluations, optionally filtered by an app ID.
Args:
- app_id (Optional[str]): An optional app ID to filter the evaluations.
+app_id (Optional[str]): An optional app ID to filter the evaluations.
Returns:
- List[HumanEvaluation]: A list of evaluations.
+List[HumanEvaluation]: A list of evaluations.
-
+>
-
-
-
-
-
-
-
-
-
-
-
-
-
-
\ No newline at end of file
+
+
diff --git a/docs/docs/reference/api/fetch-usage.api.mdx b/docs/docs/reference/api/fetch-usage.api.mdx
index b08828674c..ca33031b4a 100644
--- a/docs/docs/reference/api/fetch-usage.api.mdx
+++ b/docs/docs/reference/api/fetch-usage.api.mdx
@@ -5,59 +5,69 @@ description: "Fetch Usage User Route"
sidebar_label: "Fetch Usage User Route"
hide_title: true
hide_table_of_contents: true
-api: eJx1Uk2PmzAQ/Svona2Q9sipqdSPaC+r3eYUoWriTMAtYNceolLEf68GknQTqVwQzHvj9+ERQlVCscdH1zSuq1Aa+MCRxPlue0SBE4utv/eJKoZB5BR8lzihGPF+vdbXkZONLigDBV57azmlU99kLxcwDKzvhDtROIXQODsfkP9IyhmRbM0toRgnfczDys8qIduphGyXOGYvvhfd2rLUXkVWLDAIJDUK5IfFS34VnTieOarNEX1sUKAWCanIc9v4/riiijuhFbmcgsNUKsP20ckwUzbP2ycevjIdOaLYl28Bryp8SeMeNkKGwChAwT3xAIOOWv3e9FL76P7MAcDAqcN6Yal11538THfSzPhZXLZ53sJAbSyZrFfvVms8JnUH1szJzplfz57HMA8p3PzDgFtyOhOm9sO/gSoLPklL3Zt9/y3mTtUtCuHfkoeGXIfpomG8lLbHpTQVN9dWGtQ+iY7G8UCJd7GZJv39q+eozZQGZ4qODprTvpzMNUYt7ScPathaDno1ztT0SxsPl0/LvF2jL5++YZr+Au0pCyo=
+api: eJx1kkFvm0AQhf8KeueVcXvkVKdqUyuXKKlPFqrG6zFsA7vb3cUqRfz3agA7iaVwQTBvZr95bwckqiKKPe5M0xhboVRwngMl4+z2iAInTrr+1UWqGAqBo3c2ckQx4PN6La8jRx2Mlw4UeO605hhPXZM9LWIoaGcT2yRy8r4xejog/x2lZ0DUNbeEYhjlUTcjvwtCthOEbBc5ZE+uSzK15VQ7gaw4QcFTqlEgP8y75BfoyOHMQdYc0IUGBeqUfCzyXDeuO66oYptoRSYnb3B7+maqZl9FirGUcboLJvXTvM3j9oH7H0xHDij25VvBs2w1W/VeNiD1nlGAvHngHgqWWvnedKl2wfyb3IGCEYB67hJfjD25qd2khl/Z7ki/sD1CQfacsderT6v1R8ssDdnmcbuEQ3oK58IxyaBu7LoaBQVuyUgtMbVfXgtC6V1MLdk38z5M8B3d1ZbEf1PuGzIW48IwLOnusaQrcFO+pULtYpLSMBwo8i404yi//3QcJKVS4UzB0EE825ejulgqAb5wLwtrzV7u0Jmabk7m5pZKsNf7dv/tJ8bxP0g6Gbk=
sidebar_class_name: "get api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
-
-
-
-
-
+>
+
Fetch Usage User Route
-
-
-
-
-
-
-
-
-
-
-
-
-
-
\ No newline at end of file
+
+
+
+
+
Successful Response
+
+
+
+
+
+
+
+ Schema
+
+
+
any
+
+
+
+
+
+
+
+
+
+
diff --git a/docs/docs/reference/api/fetch-user-profile.api.mdx b/docs/docs/reference/api/fetch-user-profile.api.mdx
index 8092308b24..84dcae3422 100644
--- a/docs/docs/reference/api/fetch-user-profile.api.mdx
+++ b/docs/docs/reference/api/fetch-user-profile.api.mdx
@@ -5,59 +5,69 @@ description: "User Profile"
sidebar_label: "User Profile"
hide_title: true
hide_table_of_contents: true
-api: eJxdUk1v2zAM/SvBOwtxuqNOzWFYg16CtT0FRsEqTK3NtlSJDuoZ/u8DnY829kWw+Ci+Dw4IkROJD+1mD4sDi6teu8zpNaZw8DXDIHGOoc2cYQf8WK302HN2yUdthMVT5xznfOjqxe8zGAYutMKtKJxirL2b5hR/svYMyK7ihmCHUT8ze/Ilc1psrxwalioow3cWGESSChbFF8nM6cgpw+4GdKmGRSUSsy0KV4duv6R3boWW5AuKHmOpHa5LXvqpZb3dPHL/wLTnBLsrvwOelOhJ/S1sgPSRYUHRP3IPg5Ya/V93UoXk/02CYeBVUXXqUqm+PYSp3Us94Sdyi/V2AwOVcfJgtbxbrjB35gasHpObPL7MnsowMxeu+mHADXmtCVNz/1VQZjFkaaj99t4siBsuVwOEP6WINfkW43nycA5ph0tIpUEVsujVMLxR5pdUj6Nef3ScNIfS4EjJ05u6sitHczFNI/rLvcpzjqOuwJHq7uT9bLU0uuu6/Pr5jHH8D6yG/Jw=
+api: eJx1UsFu2zAM/ZXgnYU421GnZcOwBb0E7XoKjIKVmVqrLamSHMwz/O8DHSdpDUwXQeIj+fgeB/jAkbL1bldB48jZ1E9d4vgUoj/ahqEQOQXvEifoAZ83G7kqTibaIInQeOiM4ZSOXbO6n8FQMN5ldlngFEJjzdSn+J0kZ0AyNbcEPYxy1KLkY+K42l85tJxrLwxfOEMhUK6hUdxIJo4njgn6MKCLDTTqnEPSRWEa31VremGXaU22oGCx7LadoqtvAsVYSjnTRZv7qd52v7vj/idTxRH6UL4HPMgUZ2k+wgbkPjA0KNg77qHgqJX3tsu1j/bvpAYUrBCoz1mig3VHP6Xb3PCN21cyr+wqKMicZ9qb9af15n/DzAmr7X43m0FmMuPCY4JBLeS6CgUFbslKLDO1X24BYRl8yi25d/UWjn3gdBUj859chIaswzh3HmY3D7i4WSrUPmX5GoZnSvwYm3GU77eOo3hSKpwoWnoWhQ7lqC4Cil2v3Mt4xnCQXTlR0519WOyg2Hjdqx/ff2Ec/wGdpws6
sidebar_class_name: "get api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
-
-
-
-
-
+>
+
User Profile
-
-
-
-
-
-
-
-
-
-
-
-
-
-
\ No newline at end of file
+
+
+
+
+
Successful Response
+
+
+
+
+
+
+
+ Schema
+
+
+
any
+
+
+
+
+
+
+
+
+
+
diff --git a/docs/docs/reference/api/fetch-workflow-revision.api.mdx b/docs/docs/reference/api/fetch-workflow-revision.api.mdx
index fbb8267e0a..2989530282 100644
--- a/docs/docs/reference/api/fetch-workflow-revision.api.mdx
+++ b/docs/docs/reference/api/fetch-workflow-revision.api.mdx
@@ -5,67 +5,2193 @@ description: "Fetch Workflow Revision"
sidebar_label: "Fetch Workflow Revision"
hide_title: true
hide_table_of_contents: true
-api: eJzNWNtu20YQ/RVhnhKAsVyjT3yq60vtJk0M3/pgCMaIHEqbLLnMXpyqAv+92JskipIqKTLgJ184c2bnzOzO7pmCxpGC9An+FvJbwcUPBYMERE0SNRPVdQ4pFKSz8fOPYPAs6YUpJipIoEaJJWmSFmIKFZYEKXQsn1kOCbAKUqhRjyEBSd8Nk5RDqqWhBFQ2phIhnYKe1BZDacmqESRQCFmihhSMcSiaaW4N4np7tyFI7zqHphlYbFWLSpGycCfHx/ZHTiqTrLYpQQp3JstIqcLw3m0whgQyUWmqtDXHuuYscwz0vyrrM11YYi0tP5r5CJkw3imsnFWaRiQXlnrmLBLIqUDDNaTHTdIlycWtJl8KR2U7xsz4BSXDSltCW+abWWuSmUVlOAfLUofHRw/taEzg8IHa+LOMXiETHwGlZgVmh0zhNEDGCEaPhTwYuEdrEshR05ao1vSDZiVthD63gE0CJSmFo43Ym2D+Cu5NAgV3x8b6jmXqOTNKi3JlsKEQnLDaGO1a9c48QpNYOHpBblCvIXxLxIsZiAcdmxKrnwG8cgBNk0QjMfxKmV5xUF06yrpwri4a24vAPGf29EF+06J1i7LF86cVqRy2/7OY3epYkDGZGY7y3aXh/E8lqg9fjK6Nfg/LuVoQpqnc0gulxMlGXpd8ocPu5i7V6Dw6HfqWSP0sPjGl/5fWdUl23Hfj6N63YhjY+50Gn9GdOu3Ruh/U+QJEk0AmCTXlz6gPegqeedjeqSPL1PlrBHnwsCFITpxeIci5hw1BIl3DyQFHXSTr90kYdpGvg0aJbM2iRMIOGiXSNYvyQrJ749q+XR+De5OA4ma0L8yd9bVj6GCJBg5xeZq0R/PhsjeS7YvyIJlH4PsjcIswJszd+2O3k/61KHl7DbHhanJLBUmqstZpExc62GmiXIUyNPEtt21F3tBMfpMXnR1qcBeItzdKrGtWbXsBWqZ4l9tWDOQKb4f43t3vvZu2qvBW729vslfWO9/MKbWFIvnCMtqGXS/P7NCDAbpxekrBRsYLSa8R66wVYJs3WNSKztG9ETpPsR0gftJ9pjs1jXX79eSkK1M9Ime5S653IaWQ+2tUOWlkbtKGllw24CJrfd1lSy0MCt/F86w/iQzjrb5Uo1X63gpxwZusN3Vk9O7t11WUz7XFJ5eXDx3sWoJUpNezuz6Nc0/fpvpe3d/fdAB9bdtFvbRiaq+jXoJTAcbC6q0j0k5b1WNIoV9bgZB+9KNipvpRMVT96SqptQG/w6Mq6+5YMNa6Vmm/n3Fh8iMcUaXxCFkfa+Y2laLMSKYnzuX05vojTfxMh/RpsGjgpozvmrbZrGBYs49k+Qt6sFe22L++FYIQ7O9tjlNWFcK5RyXMLa53enMNC3d1OD765egYlvlsGdvtgZnbHjG2+wzJEguz/CEBKt3eAE1Y/jb/YFdWC6WDQBTw1pdv6R0cuND0j+7XHFm1cN31pX2CUFqYy6HK9W8oLySQrtTSBwmMhdIWYjodoqIHyZvG/vu7IWlLOAgK7tAS+jSFnCn7ew5pgVxRZ7WzcwXe3Ybt8743n07tLGKZK1tjK6jZvyCBbzRZp/67Q2Ic+2kaTE+zjGq9ANI502zjzfbFHxf30DT/AXsQfZM=
+api: eJy1VlFv0zAQ/ivVPafL6LYCeaKDARMgpjLgoaom17kkZo4dbGdbifLf0dlpmm4rIAR9aB3nu8vdd5+/pgHHcgvJAr5qc51JfWthGYGu0DAntDpPIYEMHS+ubjvAlcEbYYVWEIHB7zVad6rTNSQNcK0cKkdLVlVScJ8j/ma1oj3LCywZrSpDT3ACLV3dMCOYclcGMx+q1h8zSBYNuHWFkIBefUPuIAInnKSNOWZoUHGENupRqpYS2mVLVYUC/1HCNtobR41f7aOnbVtfjK20sqHTyeEh/aRouREVkQMJfKo5R2uzWo7mHRiiv+WS6zoEdSUL5TBHM6j5pUcMaPozijb6mPft/RFTpBBhMCWJhdqW+3P27QfqjieTh2x9YVKknovRmTHa/D1VKTomJK2Ew9I+BEjNd+4+IMk6I1Q+ZGJDt9dht8eMYesBk+91KJBmUNp8MKwu3xb6Aa1lOamyg+yHejJGl3T3d3OgvsKjO9xgJFt6A7v723gV6PvV8Xh7eXnxIGGY7e5QX9MRGm3kMJpvHaZEV2gyoRwpc8VcAQnEFYkXb+MVsxhvjp6NN5K2cdObgEhbiMCiuUFj/fRqIyGBwrnKJnHMpa7TA5ajcuyAiZhVAu7XN/N3Ry8JCu2S0vHaCLf2+WYX5+9w/RZZigaSxXII+ETiC3LahfWTZJV4h0SsYiVdz2pXaCN+BI1EIKiAIkQRcyTr+dZ3z+5YWUl8xEcF0XaUsWcn2fR4fPL0ydPx8cl0Ml4dZXw84c+nR9l0yjI2JXpknQ9VRVyF1vst5pwRq9r5ZtpHbPb/Pq8lJjLtievk1U3llPFrVOlOlsODJweH+8bYBYxmF+edezDu3WMzAQ+D6J5QeolABFh66wCHrHyxvUFVVtq6kqlBvv3q3imvV4TDOxdXkglvEb6IplP+AjrlQwSkfYigVz9sR0LrZHAC6IQX2jpK0DQU+NnItqXt7zUakvEyCvpZEbULsq9iI+gGrnHt/zm8z469yRBc1kHA9zyX/DBEzDjHyv0Suxyc8Tdnl9RX9zZR6pRCDKNm6TsBoFcT5xtMmrDXgGQqr8klEwgp6fMThXcJJQ==
sidebar_class_name: "get api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
-
-
-
-
-
-
+
+
Fetch Workflow Revision
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
+
-
\ No newline at end of file
+
+
+
+
+
Body
+
+
+
+
+
+
+ variant_ref
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ slug
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ version
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ attributes
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+
+
+
+
+
+ revision_ref
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ slug
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ version
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ attributes
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Successful Response
+
+
+
+
+
+
+
+ Schema
+
+
+
+
+
+
+
+ revision
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ author
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ date
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ message
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ name
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ description
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ version
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ slug
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ flags
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ is_custom
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_evaluator
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_human
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ metadata
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+ data
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ service
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+ configuration
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+
+
+
+
+
+ artifact_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ artifact
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ name
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ description
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ slug
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ flags
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+ metadata
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+
+
+
+
+
+ variant_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ variant
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ name
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ description
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ slug
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ flags
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ is_custom
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_evaluator
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_human
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ metadata
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+ artifact_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ artifact
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ name
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ description
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_at
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_at
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_at
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_by_id
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_by_id
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_by_id
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ slug
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ id
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ flags
+
+ object
+
+
+
+
+ anyOf
+
+
+
+
+
+ is_custom
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_evaluator
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_human
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ metadata
+
+ object
+
+
+
+
+ anyOf
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Validation Error
+
+
+
+
+
+
+
+ Schema
+
+
+
+
+
+
+
+ detail
+ object[]
+
+
+
+
+
+ Array [
+
+
+
+
+
+
+ loc
+ object[]
+
+ required
+
+
+
+
+
+ Array [
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+ integer
+
+
+
+
+
+
+ ]
+
+
+
+
+
+
+
+
+
+ ]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/docs/docs/reference/api/fetch-workflow-variant.api.mdx b/docs/docs/reference/api/fetch-workflow-variant.api.mdx
index 31c2b0ce2d..0cf3dd4cab 100644
--- a/docs/docs/reference/api/fetch-workflow-variant.api.mdx
+++ b/docs/docs/reference/api/fetch-workflow-variant.api.mdx
@@ -5,67 +5,1008 @@ description: "Fetch Workflow Variant"
sidebar_label: "Fetch Workflow Variant"
hide_title: true
hide_table_of_contents: true
-api: eJzNV01z2zgM/SsenNoZNnYze9JpvfnYZNvdZhq3e8h4MrAEW2wpUSXBZF2P/vsOKcq2LDtN03Smp3wIeA94AEFwBYwLC8kN/KvN57nS9xamAnRFBlnq8jKDBObEaX57Hw1u79BILBkEVGiwICbjEVZQYkGQwK7hrcxAgCwhgQo5BwGGvjhpKIOEjSMBNs2pQEhWwMvKQ1g2slyAgLk2BTIk4FxAYcnKG7TRDj42HIPLDOp66qFtpUtL1qMdj0b+R0Y2NbLy+UAC1y5Nydq5U4P30RgEpLpkKtmbY1UpmYb0h5+s91ltRVgZLw7LhiHVrnGKgcuSaUFmK9KTYCEgozk6xZCMatGTKNCWy3fzoGOXYm0rs67Zw1rVYm1ROqXAi9NTz6smAA3LOab8jAzjCBkZ5ip02eEkpb1NnWVd7A1gprUiLB9kvLSDkwahFh6O7lA5ZG1+BPFsDdKA5q7A8kcALwJAXYvWSM8+Ucp7Ovs8SNaHqwUUxNgNArNM+n5FddWR9VApt2Dbju0wFbPuf7az288FqTSpU2henDul/rK6fPXOceX4Jezm6kEkU/FILzQGlw/quuMLPXUfcv7bi+k9eh36K4n6j34rLX9T1kNJ9ty/T6NJ04pxwD9KgX4I3rcW3WH8NKjTLYhaQGoImbJb5EdOrwyZXrEM8RxmOWlgB+Mglquyn0HyoYGNJBkp+gkkpw1sJGnlmi2fcd63Yv2xjBO/1etZWVq11iytYM/K0sq1ZrHKLZ7aq9fe118ezxZe2HS+fX/ExWjfDfJ47/WCVNfe67fj4/4+9RGVzMK2NDgzRpunL1MZMUoV1oHmetg1UDrtfP2eSTytd26UTdJvdROgr1NhF/v20M1tYS0uaDM+D5sGMQYT/3Wf4psd+Cbk1VBHu61yb+Rt1D2cxmkj30PlvZhMrnqATW27RT33K/9gd8uGsHvk2j8KFtS8ADiHBIaVoTtJ98N2WbXDuNna4WrPe6AGAZbMXftycEZBAjlzZZPhMFXaZUe4oJLxCOUQKxlOgKXUGcnL4DK+unxDywvCjAwkN9Ntg2vfYE3LdM3W1cJKviEvXnyzjB3n2sivTR/E10reeHl1ZDnXwb3db0Nwg/HVJQjwaTSyjY5eH41gV8yOsT8bmIaz0XKHzyB2VFjnDwKoCAcDmLD4ffPBR1Zpy3EpjXgHa7dz9UYpmP7jYaVQhv4PMaxiXW8g1hU2TxbrE461BQHJvtfeVECuLXuA1WqGlj4YVdf+318cGV+/aQSZeTVvVpBJ63/PIJmjstSLdT1R4MX7eHBeDjYrUDeHtsblMsSqnP8LBHym5YHnaZgOedtLq2g5TlOqeAujN8x8061PxJ9nE6jr/wGy1UCI
+api: eJydVcFu2zAM/RWDpw0Q4qzYyadlXbcG3bCi67pDYBSqzMRqbUmV5HaZ4X8fKMuO0zQYsFOU6JF6fORjWvB84yBbwS9tH9aVfnaQM9AGLfdSq2UBGazRi/L2OQJun7iVXHlgYLjlNXq0lKEFxWuEDOL9rSyAgVSQgeG+BAYWHxtpsYDM2wYZOFFizSFrwW8NRTpvpdoAg7W2NfeQQdOELF76igA3fepkWUDX5ZTRGa0cOkpyMp/TR4FOWGmIPWTwoxECnVs3VXIVwcBAaOVReYJzYyopQrHpvaOYdkLMWJLCy/4FoZs+KPKVyuMG7YTgaUB0bBAhvKC239dBoBim7+5R+EnUoH0sDzo2YlVTVdDlXccOondyriKz/GjKsfauo1TvT04OpbrhlSyCEMmZtdr+v04Fei4rOkmPtTsEVFrs3R5IFCdhIsSgdZfvpODW8u1Ex6+6J0gNqN3mtckaoN/QOb5BGJMdhwYxkmu6/VcbqK7+6YjLp7M7yNure7yMT718rz02QM6vry8PEva93W/qZ/JuMkxDcjN6t0ZfanL3Bnsr+xIySI3FJ4nP6R13mA6Wd2kcZ5e2O3d3wMChfRrs39gKMii9Ny5LU1HpppjxDSrPZ1ym3Eh4SW4RbpNTggL52aForPTbkG9xubzA7TnyAi1kq3wK+EGT18/SPmxsIzfyAknVuJUWjS+1lX/6AYmLqeyjSDap1jqER4Ujt49cPKCiHUR19rTns3ez+bFiYkCyuFxGA3ERDDTwCDBgL+QahQIGWAf3gEdef9hdEEujna+5muQ72uA9dqMsHn/71FRcBpMEDm1s/gpi84EBtR8YjAMA40ajYzZZ8TmDUjtP4W1LYT9t1XX082ODljqZx9g70nXVQiEdnQvI1rxyeMB0XDrw5ip6620C7PUKhm6rbaBYNfQNGDzgdv+vKOyNchimNgIWQqDxk9CDNUdTN3rly9k1dN1fcHR4xA==
sidebar_class_name: "get api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
-
-
-
-
-
-
+
+
Fetch Workflow Variant
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
+
-
\ No newline at end of file
+
+
+
Path Parameters
+
+
+
+
+
+
+
+
+
+
+
+
Successful Response
+
+
+
+
+
+
+
+ Schema
+
+
+
+
+
+
+
+ variant
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ name
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ description
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ slug
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ flags
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ is_custom
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_evaluator
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_human
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ metadata
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+ artifact_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ artifact
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ name
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ description
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ slug
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ flags
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ is_custom
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_evaluator
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_human
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ metadata
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Validation Error
+
+
+
+
+
+
+
+ Schema
+
+
+
+
+
+
+
+ detail
+ object[]
+
+
+
+
+
+ Array [
+
+
+
+
+
+
+ loc
+ object[]
+
+ required
+
+
+
+
+
+ Array [
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+ integer
+
+
+
+
+
+
+ ]
+
+
+
+
+
+
+
+
+
+ ]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/docs/docs/reference/api/fetch-workflow.api.mdx b/docs/docs/reference/api/fetch-workflow.api.mdx
index 68dfcd4d87..dae5d8732d 100644
--- a/docs/docs/reference/api/fetch-workflow.api.mdx
+++ b/docs/docs/reference/api/fetch-workflow.api.mdx
@@ -5,67 +5,598 @@ description: "Fetch Workflow"
sidebar_label: "Fetch Workflow"
hide_title: true
hide_table_of_contents: true
-api: eJzNV01v20gM/SsGTy0wjd1gTzqtNx8bb7vboHG3B8MIxhJtTTvSqDOcZF1D/33BkWRLlp2maQr0ZFtDPpKP1PB5AyRXDqIZfDT281KbewdzAaZAK0mZfJJABEukOL29rw1AQCGtzJDQsucGcpkhRNAY3KoEBKgcIigkpSDA4hevLCYQkfUowMUpZhKiDdC6YFdHVuUrELA0NpMEEXgfUEiRZoMmu8EkgbKcM6QrTO7QMcrpaMQfCbrYqoLzhghufByjc0uvB+9rYxAQm5wwJzaXRaFVHMocfnLss2llVlgmgVQVITa+cqoTVjnhCm0rw7NgISDBpfSaIBqVYktJCJev3y0DX13opQ4dOG6g3G3sHZmsa1RnsjBGo8yhFNtHudcamKQmt4kbnFUIpWA4vJPaSzL2RxAvtiAVaOozmf8I4FUAKEvRGJnFJ4zpwBRcBsr6cKWADEl2k5BJorjHUl93aO2lWQ9hC7bpcidStug+aVd3OBbEysZeS/vi0mv9lzP5q3eeCk8vYb9WBlGE2SO9pLVy/SCve77QY/ch57+ZTPboTeivROo/5q1y9E1ajxXZc/8+jqbVKNaX4KMY6KfAvqXoXmBPgzpvQZQCYouSMLmV9BBg695NJOErUiGf41HOKtjBOJDli+RnBPlQwdZBEtT4E4KcV7B1kIauxZqX2OPihE31GLL+WIfttePrWaM0bG2jNIQ9a5SGrm0Up/3qqbN6w768PJ4tvaAOvr0/Dq2OR7htZURZsvlvp6d91fGv1CoJmmJwYa2xT5ccCZJUOgiAaiHsG2gTd06/5+6dl3s7ZFftW1MlyJ3J3OqQStvtB+fkCncX5nHTQMZgyqeHqN4pxFmoqwpd27UavKO3Yvd4GecVfQ/19Wo6ve4BVr3tNvWSBfDg404AZ0ipYWm8Qgp6mFKIYFhYvFN4P2xknxtuWqK4BAEO7V0jm73VEEFKVLhoOIy18cmJXGFO8kSqoSxUGG2HsbeK1sFlfD15g+srlAlaiGbztsENz1E1GV2zbVNkod4gc1QL9rGn1Fj1tWp3LdnTyotJUPnSBPear3FIbjC+noAALqNiZ3Ty+mQE+5x1jPkVkHF4BZrY4RjEHgvb+kEAZmH+gVBmv+8OOLPCOKrVZo3Xa9HeLq0pIPyPhoWWKox3iL2p2zeDun2w0+0OBETt/zVzAalxxNabzUI6/GB1WfLjLx4tN2ku4E5aJRdM2WwDiXL8PYFoKbXDXmLb2wFevK9fgpeDnYDpJtw0Mucusv7mXyDgM673/oCFNzxtBmVTW4zjGAtq+fYuJJ6o7XT/eTGFsvwfCqXUMw==
+api: eJydVU1v2zAM/SsGTxsgxFmxk0/Lum4NumFFm62HIChUmY7VypYr0ekyw/+9oL/i1C0G7JQgeiQfH/mYCkhuPURruLHuITH2ycNGgC3QSdI2X8YQQYKk0tunDgACCulkhoSOIyvIZYYQgXSkE6noVscgQOcQwWOJbg8CHD6W2mEMEbkSBXiVYiYhqoD2Bcd6cjrfgoDEukwSRFCWTRrSZBiw6JIHyxjqesMpfWFzj56znMzn/BGjV04XTBwiuC6VQu+T0gRXHRgEKJsT5sRwWRRGq6bP8N5zTDViVjhWgXRbQdmyDeoI65xwi27E8LRB1AIGobhEvv+ZNCJ1cfbuHhWNwnrd+wahFgM4L42BelPXYhJ+UHTdcdtMcw5t1zXn+HhyMlXptzQ6bjQIzpyz7v8lipGkNvxNE2Z+CjBWHb1OxOm2YKRAL3O9OWggnZP7kYLfbUuQtc/89rWt6qE/0Hu5RRiSvQ1txAhW/Pov/bmvtnSHG43iIG+r7tttfGnle61YDzlfrS4nCdvZHg/1Kzs2uDk4NkNKLXt5i9QYmFKIICwc7jQ+hXfSY9jvrQ+r/uutjmsQ4NHterOXzkAEKVHhozBUxpbxTG4xJzmTOpSFhpdkFs1rcMpQYOt6VKXTtG/yLS6XF7g/Rxmjg2i9GQOuedPa3TmGDWOThb5AVrG7QYuSUuv033YhuiuUtlEsk84T24T3Z6Xl9lmqB8z53nCfLe357MNs/lYzXUCwuFx2hmHrRsMtbGEgXsg1CAUCMGvcAoQy+3R4YJaF9ZTJfJRvMtAjVoMchH8oLIzUjRma2lU37DV0wwYBPG44HCoPAqLRyHl/U+uJg6qKwb+cqWv+uT3oPLdYe3ln2ACJNB4njIYjAu+uOq+8D0C8zvQB95M/kJ00JSObldlJp7lcsyGinyjzaEMXSmFBo6jJ4eIsgwu+na2grp8BUd9kyQ==
sidebar_class_name: "get api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
-
-
-
-
-
-
+
+
Fetch Workflow
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
+
-
\ No newline at end of file
+
+
+
Query Parameters
+
+
+
+
+
+
+
+
+
+
+
+
Successful Response
+
+
+
+
+
+
+
+ Schema
+
+
+
+
+
+
+
+ workflow
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ name
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ description
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ slug
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ flags
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ is_custom
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_evaluator
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_human
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ metadata
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Validation Error
+
+
+
+
+
+
+
+ Schema
+
+
+
+
+
+
+
+ detail
+ object[]
+
+
+
+
+
+ Array [
+
+
+
+
+
+
+ loc
+ object[]
+
+ required
+
+
+
+
+
+ Array [
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+ integer
+
+
+
+
+
+
+ ]
+
+
+
+
+
+
+
+
+
+ ]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/docs/docs/reference/api/fork-evaluator-variant.api.mdx b/docs/docs/reference/api/fork-evaluator-variant.api.mdx
deleted file mode 100644
index a719f0cc59..0000000000
--- a/docs/docs/reference/api/fork-evaluator-variant.api.mdx
+++ /dev/null
@@ -1,71 +0,0 @@
----
-id: fork-evaluator-variant
-title: "Fork Evaluator Variant"
-description: "Fork Evaluator Variant"
-sidebar_label: "Fork Evaluator Variant"
-hide_title: true
-hide_table_of_contents: true
-api: eJztXFlv2zgQ/ivGPLWAHKdpm+76adOkRbO9giTtPgSBQUuUzYY6SlJpvYH++4KHDupwZNdeOKn6kEISZ4b8hhp+nBF9BwLNOIyv4M0togkSEeNw7UAUY4YEicJTD8bgR+xmgrMGk1vECAoFOBAjhgIsMJMq7iBEAYYx1FpOiAcOkBDG8D3BbAEOMPw9IQx7MBYswQ5wd44DBOM7EItYKuGCkXAGjjQeIAFjSBKlRhBBZYO8w4Ov2srg1IM0vda6MRevI28hFVZNuVEocCjkIxTHlLhqoKNvPArlvaInMZMwCIK5vMpHteTRhOFbwolWhMLFZ1/hYrf2qYK8vQHhEzfhIgrsRgaYaRRRjEJInfxWmFAKcuQZNqd8cKw1pI5UZ/V9XY054EbpPAlQ+CsK3ykFaepkjaLpN+yKJh+/VZjV9aUOBFgguxfI84h0KaJnFq61fpo5VlJLQoFnmNmWgql9pzy8ZlvgEuYmFLEnbxNK/+ZRODwN40Q8hepYpQ4icNBNCDGGFkthtUWhhu0y2Y8SSSlRm5+7hOin6APh4j5M28ZYlV4NoUs9C02g6zT+eg+kbOqAh7nLSCxq0aK7qpOSitQBTpPZuqoupKzsFaq+TXZsusWsHt+6m/lqxFMHEkbW1fKFEa2Brq+BSg1zjDy1eK022bcFySYcSLxlGqrL6dII7S2NzefYxwyHLi4ryTp6vdJr9c64Ic14QFeP7FBY2r1Av4IDLgzqcjlFcUzCrgtAFd9VVpvMkPK6DGNrT30tndp8dFfXr92bKO2yZwWe0kuY3RIXd4FWM+0VJqBRnSp67pNZorcf27B1bBnoRD/PDa0/QYohNbJQztFsbVLw0Yiv1Ju3Ebtp7s2a25DOr0kDwV8v8me8vmfxPYvvWfwWWfwu+f4hvU0m4v8f8f3+sF7PM002R/eLfFrWEZVQc+BHxG58Gv3YhtF/jO6qzXqysU+o9Qm1hxY8+qX4d1+Ku7yKpnzRHvbXC4E9me8jSB9BHnMEuTdwNBZiN04Xy+XXElvcvMmcLNoWi1FuY3SVUW1hNC0su2e8G2G8Gcw94e2Xq3656ivIfQX5Pgf2FeQ+0PcV5B1ev3ZvorTL/uYV5Ix97kQBudqZ++vHG9zutBUV+vx+v9t5oLGt3+387rudDm9i1+z+BkOtnR5DTBAfuZs0cGRUGgsejsW8UXfjZK45RUp3K1oqEBtaFkc8rkrnNJryeVLFuT4fAmmaqrWOx1HI9Qt6sL8v/7PmGlwkros59xM6ODeNYe1jJG6UaKEKRGV2kajDNR72UUIFjPdXLYg/jozo9iduTyw6E4tH9OHA50SsQS1qUitwCy37yMnFMlg7sIt1MNpteuEyjAT2JmhpWqIUvjwk8FCQAC+1cqzVDo4UWEnsbcPIF63WGPEwxVswcqLVGiMZXNPFBgN+BtbrhQn5GV4btZKhlVvJANuolQyu3MqDSvZWP3dpWkJWEM+JmCZxLw4O6rztK6LEU6xs8IaxiK1P2jwsEFGlArNAVBvQyLWerhKLS5luvaYUo/4QuSgLJwGf1fOlDQmgTFl7UwXG4FI+vY9My3Fp06adtc3I4NXotg/jRMO3zL/vLi/Pagq1b22nSgI/qH18AIp9zCN5Wj2OuD6bLjclMIplMgv/GOXMi48MheYjebAddFoyO8OuSkIwFyLm49HIpVHi7aEZDgXaQ2SEYqImPcduwohYKJGjs9P3eKFLEDC+ui43UHlxPUfsZrl7UEzeY4mWOT1/lIh5xMi/2vHm2LwuMykE5Vw9L064v/mJgphiqJ5Qbz6TnjPeErPV2U+bm+b3DLWU1wXDK2hJvvgWs8x2V35bB6tSM1MmzGtfxSNV1ytf0vJlUXGzij124aFUESgkrdy+lY1uSBqn5ZxsqTzRhOZWkGmx3/INODz30R8v/cMXw5evnr0avnh5eDCcPvfd4YH75+Fz//AQ+egQ2j7m7irduAvdlTllfym5Dd+0flK1MvrrCNu7+pVN/tIs2SU/P/zY8UtzZpdcUUlfdh2SldnpKmSyjPsSXBL6kVpCs7yOWqAHR2en4JRmxf7es7392lisxpIQIleBmq2/6nE+dzImkHMA+SYGig2CwCj4q3ggeybJh0nGGH2thKWy4zR0QOCfYhRTRMLSByqay1yB4TLlYMAh9wHXfP1GkrS5JEHjK7i7myKOvzCapvK2/vkdSVo8wtGUSpbnI8pxrTs5U4Yn54YQPh0Um3u7mzd40f67P/KuFFHEST2ZSqddXeefz6gOaR3H2uxQsdNCtkbWJZHWEkeui2OxtO11iR2efb64BAem5seBgsiTMgxJUOXfMYD8+SMprd4wde8OKApniX6ftU757z+l9CiM
-sidebar_class_name: "post api-method"
-info_path: reference/api/agenta-api
-custom_edit_url: null
----
-
-import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
-import OperationTabs from "@theme/OperationTabs";
-import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
-
-
-
-
-
-
-
-
-
-Fork Evaluator Variant
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
\ No newline at end of file
diff --git a/docs/docs/reference/api/fork-workflow-variant-by-revision-id.api.mdx b/docs/docs/reference/api/fork-workflow-variant-by-revision-id.api.mdx
new file mode 100644
index 0000000000..b8be4a8f03
--- /dev/null
+++ b/docs/docs/reference/api/fork-workflow-variant-by-revision-id.api.mdx
@@ -0,0 +1,2491 @@
+---
+id: fork-workflow-variant-by-revision-id
+title: "Fork Workflow Variant"
+description: "Fork Workflow Variant"
+sidebar_label: "Fork Workflow Variant"
+hide_title: true
+hide_table_of_contents: true
+api: eJztWVtv2zYU/ivGeWoBOXac2N30NDdp0azrGiRpBywwjGOJstlQokpSST1D/33gRVdf6qTusIfkIYmpc/3OIfkdeQUK5xL8W/iLi7uI8QcJEw94SgQqypOLEHyIuLibPrjn03sUFBM1nS2ngtxTSXkypSF4kKLAmCgitL0VJBgT8KEpQxPwIUW1AA8E+ZpRQULwlciIBzJYkBjBXwEmy4+RMaKWqTYilaDJHDwdSowKfMgyGkLulRJJxhjkEw8UVUwvXDm/nYsQci3o4inCr8L5mhGxbMQTIZMHD+izdWzjmVh3RKrXPFxqF200Ap4okijjPU0ZDUw9el8kT/RaFVsqdLUUJdKasWmvP8FMLbg4VDZjay33IERF9rSqRbuKxmSn6XNtMPcgJlLifKftXWY+OPXcc6V/mpk/0YQLIZGBoKly2D7F1HnNRO5BIAgqEk5RHRS+M2u2M1baSZaGP8PJJ2vWOQkJIz/Bybk165wUcM2Weu8eqI0LsF4vzb6s8DqolwKt0ksB2EG9FHCVXu6JkD/Qrp+deu6BZNn8qWautW7uweEStdlFzNxcG0zy2RcSKKgUirvtrVFZN20OG4UhqtZpj2FI9XZFdtk8S7+PA00UmRPR9BbPmiszzhnBxCxt9gUBFUHGULx4mzH2u+RJ9yJJM/VSp9dIV9ugisT7KaEQuNyJclNVI77ub/vZ69C098PmO3Rrmc6t5oYqoVA0wkAdcNeMnUm3ZwoP+0VcKG+OtkY1fgKHKO0/DlxnYlPEayVe1y5YFWwSrijMbcVDJtuNXFkGBHmurQkiU55Iu8EG/b7+07h04ToLAiJllLHOlROGJxOlgGdWqbVfa3eDkfhvcK5DZyPbgJszWeZucTsdDNah+oyMhgaIzhshuHg6TiFRSJn+zx0ubQHGg8bTx5yNk7x1IFVJ/8FtgIYJynmtUuW+2cD0rMh2UQNG50Y//V4ZdF7WtZNr7MMCXovu9jTOLXy79ta7m5vLNYO2ts2ivuXirlM0Q6doMHN3Lbge1FIulRnE1AJ86KV6E5KH3gwl6RXzm+wVW1P2VrXhLO/pMQ88kETcFzNcJhj4sFAqlX6vFzCehUc4J4nCI6Q9TCm0Yxybp50zLWqOLkmCTFC1NPbGlxfvyfIdwZAI8G8ndYFr3YC2pZpiZTUxpe+JBteNcnYGof/YPnHT3MJqafR0a19VU9abbxinjLSnpGIugpMIfxlGo9Pu8NXxq+7pcDTozk6ioDsIfh2dRKMRRjiCYt6BQX8w7PaH3ePhzfGJ3z/2h6dH/WH/b6gNLlX/uYjLz03QyuX6SLDLQ53V75KrE/Ndci1uvS8ULbK8N4JN9ruvWklnK7wsL60+P8ZcyR+pnAaZVDwuRm8qp+QeWYZK90W5tshiTOznJmGskRy9eWhA7GLAk4jOM/smRS+1Gcy+odZJyXMn/XgnHaJtWi3Qont793RFLZ4L+/8o7EHOg+dt/twN5dFgqBxNIm64TDE6WqL0GoM7koSN661/dHzU38asnEJnfHnhSH2zY6yYxrrB3UrWBh6Q2DB6UATj36oHOkrNIE0epb1tpLP1PtRRNEW+qV7KkBrebkJYOT56C46PggeakYIHJSc11NuxUvDAb35pYIjpxIOFprf+LaxWWv+TYHmul+37e00vQypxxmpv8O/Isv3KX5dOR2rIqXky0/XYpN1KsRyg4MWVmxNedqo3Ks3UC8qaLOs+i4jq+ZkhaFFQ4pWTOLPOumZUqSysTW56qrIa4yAgqdopO6nNCpcfr290JdzXDzEPzbc1qMujf5touUnefrGg11bAMJlnltxam/rnXyR5jVQ=
+sidebar_class_name: "post api-method"
+info_path: reference/api/agenta-backend
+custom_edit_url: null
+---
+
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
+import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
+import OperationTabs from "@theme/OperationTabs";
+import TabItem from "@theme/TabItem";
+
+
+
+
+
+Fork Workflow Variant
+
+
+
+
+
+
Path Parameters
+
+
+
+
+
+
+
+
+
+
Query Parameters
+
+
+
+
+
+
+
+
+
+
+
+
Body
+ required
+
+
+
+
+
+
+
+ revision
+ object
+
+ required
+
+
+
+
+
+
+ author
+ object
+
+
+
+ anyOf
+
+
+
string
+
+
+
+
+
+
+
+
+ date
+ object
+
+
+
+ anyOf
+
+
+
string
+
+
+
+
+
+
+
+
+ message
+ object
+
+
+
+ anyOf
+
+
+
string
+
+
+
+
+
+
+
+
+ name
+ object
+
+
+
+ anyOf
+
+
+
string
+
+
+
+
+
+
+
+
+ description
+ object
+
+
+
+ anyOf
+
+
+
string
+
+
+
+
+
+
+
+
+ created_at
+ object
+
+
+
+ anyOf
+
+
+
string
+
+
+
+
+
+
+
+
+ updated_at
+ object
+
+
+
+ anyOf
+
+
+
string
+
+
+
+
+
+
+
+
+ deleted_at
+ object
+
+
+
+ anyOf
+
+
+
string
+
+
+
+
+
+
+
+
+ created_by_id
+ object
+
+
+
+ anyOf
+
+
+
string
+
+
+
+
+
+
+
+
+ updated_by_id
+ object
+
+
+
+ anyOf
+
+
+
string
+
+
+
+
+
+
+
+
+ deleted_by_id
+ object
+
+
+
+ anyOf
+
+
+
string
+
+
+
+
+
+
+
+
+ version
+ object
+
+
+
+ anyOf
+
+
+
string
+
+
+
+
+
+
+
+
+ slug
+ object
+
+
+
+ anyOf
+
+
+
string
+
+
+
+
+
+
+
+
+ id
+ object
+
+
+
+ anyOf
+
+
+
string
+
+
+
+
+
+
+
+
+ flags
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ is_custom
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_evaluator
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_human
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ metadata
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+ data
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ service
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+ configuration
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+
+
+
+
+
+ artifact_id
+ object
+
+
+
+ anyOf
+
+
+
string
+
+
+
+
+
+
+
+
+ artifact
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ name
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ description
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ slug
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ flags
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+ metadata
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+
+
+
+
+
+ variant_id
+ object
+
+
+
+ anyOf
+
+
+
string
+
+
+
+
+
+
+
+
+ variant
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ name
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ description
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ slug
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ flags
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ is_custom
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_evaluator
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_human
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ metadata
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+ artifact_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ artifact
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ name
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ description
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ slug
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ flags
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ is_custom
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_evaluator
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_human
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ metadata
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Successful Response
+
+
+
+
+
+
+
+ Schema
+
+
+
+
+
+
+
+ variant
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ name
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ description
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ slug
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ flags
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ is_custom
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_evaluator
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_human
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ metadata
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+ artifact_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ artifact
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ name
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ description
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ slug
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ flags
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ is_custom
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_evaluator
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_human
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ metadata
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Validation Error
+
+
+
+
+
+
+
+ Schema
+
+
+
+
+
+
+
+ detail
+ object[]
+
+
+
+
+
+ Array [
+
+
+
+
+
+
+ loc
+ object[]
+
+ required
+
+
+
+
+
+ Array [
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+ integer
+
+
+
+
+
+
+ ]
+
+
+
+
+
+
+
+
+
+ ]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/docs/docs/reference/api/fork-workflow-variant.api.mdx b/docs/docs/reference/api/fork-workflow-variant.api.mdx
index 5d01d9eb97..f92f4cb184 100644
--- a/docs/docs/reference/api/fork-workflow-variant.api.mdx
+++ b/docs/docs/reference/api/fork-workflow-variant.api.mdx
@@ -5,67 +5,2487 @@ description: "Fork Workflow Variant"
sidebar_label: "Fork Workflow Variant"
hide_title: true
hide_table_of_contents: true
-api: eJztW21T2zgQ/isZfbqbcQilQO/y6WgpA9c3BmjvQybDKLacqMiWqxfaHOP/frOS7PgtwXGTG9qGD5DY2l3pWWn3kbQ8IIWnEg1H6B8u7kLGv0o09hBPiMCK8vgiQEMUcnF3+9W9v73HguJYIQ8J8kUTqV7yYI6GD8jnsSKxgo84SRj1jYbBZ8ljeCb9GYkwfEoE6FeUSPiWKV7+5laQeyqpVYPj+YcQDUfVxiEzA1negMpbX0vFo3IjNU8IGqIJ54zgGKVe/ijWjKF07CFFFYMHF7L3ympIPVBH7jHTWHHxPRpf50qs0pmOcPw9Cs+NgjT1skZ88pn44K+sUebqMwNZXV3qoYgoXO4EDgIK/sTssgRrrZtSCRpPi2pprMiUiLKlaFJ+Uhxdsy3kU+FrhsVvZ5qxvyWP+xdxotXvqDpU0EEVidoJYSHwfCWqZVFUg3aV7DtAEiRq0/MpIfqev6VSPYbpsjFWpddD6MbOQhTjiLQcf70HIJt6KCDSFzRRtWDRXtVpQUXqIcn0tKuqa5CFXuHqaiqHpnsi6uGtvZlPTjz1kBa0q5aPgloNrLsGBhpmBAdErD3ZtwXJJhxIg1UaPMiQEVZoiLSmweoAHawMzVckJILEPikqyTo6XmtZnTs3AAQm9bb1yBMKS08v0K/hgGuHOqRTnCQ0bpsAqviuk20yQ8brEMY6T30rnXoowQJHRHVY0r/yRFkue7nAE7xExD31SRtoldBkrQnoVKcecPOQTrUl9duw9apkoA37vHKk/hQbgtRIQqXE086c4J0TX6czZ1zcNXem4x6k9SJpYPfd4n7G6nccfsfhdxx+ixz+Kfn+R1pNLuD/H+H98bBeO2O63RzXz5JLL+tGD9j/IpVs0FTFQu2wbndwtjs4+8HCxC7p/upJt8VK/GTj2/L43i0A7lj7LoDsAsjPHEAeDRxVCrUNXug64Ujb5g2V9ecj2sJIrAUsFA2xv8khnDiVzkJAEjVr1N24MGsTDKRbMTwzLxoawp03FSSAK/P85roBEVBwZe/HUZqmRUE4WjMPZMJjaSPPwf4+/CktInStfZ9IGWrWu3KNkdf1mt3n2gpV8Coe4WlzrR+QEGum0HB/zZ3EzzDBdruhX3A39EGrDmymJrUGnbGyPzmfWQVrC0LTBaOnzWh8QbAiwS1eeStXiF4BVqSvaERWWnll1fZODFg6CbZh5KNV64wEhJEtGDm1ap2RDK7JfIPxPgPr5dxF/AyvjVrJ0MqtZIBt1EoGV27lh6p1qOzhmzJIe+mcIKWGbR0eHNT51CfMaGDYUu+1EFx0J1MBUZiaOhmXHqoNGPdLb9eJxIWDA5tRFoN+y32cBZNITuvFAg3H45my5U0NGL0bePsY5YVxWdOuXWm3kcFr0V0+jFML3yr3nt/cXNYUWt+WnQpEu1fdUSFDPWYcqlcTLuF7gmHjgAYJnL6Tr4OMrMqBY7ZyAHWuyN7Im2KH0YOthkIzpRI5HAx8xnWwh6ckVngP0wFOqJnwkvhaUDU3IieXF2/I3FbfoOFoXGxgSkLsDCk3y52DE/qGAFY2g6ETrWZc0H+t2z1EYcy2wsrgBzP1alGH+/objhJGUKWutrGSNqe6BUprr/3LpDR/5jil2784brcgJHnaXcywsqvyxzZMFZq5u7W86GvxyhS0Fb+y4tdFqVmpyqlccVMohVlIlopaSmUYDdUSafG2qlCX04TmVpBZYr/58go9D/EfR+HxYf/oxbMX/cOj44P+5Hno9w/8P4+fh8fHOMTHqHIN1VaoaT/4VCZS+bh3Gw5Zdi7UFr0uMqVddVuh0ka5rZA7XNmH6U7jkJuwlG2TTdDrnVxewDDydbq/92xvv4ZeqTGkWOwbl2QxzbzOV3MWXfO4ijxEIpNfkSI4+mvxAnoG8dztbZ2+ZSmgQuBdhFXkmxokDNO4UO5qs8MIuexQgF0u3CYt+7mDpDeDrDIcoYeHCZbko2BpCo+/aCIgDYyd0ATAG43zqliTIe7I3FZMAfXom7wLzZm2CaBCQ4AiWIkT3yeJWtl2XEh8lx+ub5CHJu7fMyIegIzAMDr4PUQI/tMDpM3qNc8eEMPxVNtIY3XCz3+wbTUT
+api: eJztWW1T2zgQ/iuZ/dTOOCQEQq/+dAHKlOvdlQHKzRyTYRRbTlRky5VkaC7j/36zkvyalwYabu4DfAAi7z67erSSnnUWoMlUgX8Lfwl5H3HxqGDsgUipJJqJ5DwEHyIh7+8e3fO7ByIZSTR4kBJJYqqpRIAFJCSm4IN7fsdC8IAl4ENK9Aw8kPRbxiQNwdcyox6oYEZjAv4CSDL/HBkMPU8RQ2nJkil4GDomGnzIMhZC7pUWScY55GMPNNMcB25s2M55CDnauWwkfWCKiaSWzreMynkjn4hwtfOELl1km9HYxqNKH4twjjHadAQi0TTRJnyachaYBeh9VSLBsSq5VOLyaEaVhbFhlp+QTM+E3NV0RhYt9yAkmm6JiqZdzWK6EfoUAXMPYqoUmW7E3gTzh3PPPbf4z4P5k5h0IaQqkCzVjtvnQJ3WIHIPAkmJpuEd0Tul78TCdkYag2Rp+BJBvlhYFySknL5AkFML64IUdE3muHl3VMYFWcdzsy8rvnYapWCrjFIQttMoBV1llAcq1U+U641zzz1QPJs+F+YKfXMPdjdRO7uIm6tqBaSYfKUB3kiFQ3GZnRmXZWhz2GgSEt067kkYMtyuhF80z9If88ASTadUNqPFk+bIRAhOSWKGVseCgMkg40S+Ocs4/02JpHuepJl+i9NrTBcxmKbxdk5ESjLfyHLTFRlfjrf+7HVs2vth9SW6dplOreeKVSJSs4gEeoe7ZuQg3Z4pImyXceG8Otua9HkBVVPiP41cB7Eq46UlXvYuVAysMq4kzG2lQ8brQS6tAoI8RzRJVSoSZTfYoN/HP41LF66yIKBKRRnvXDpjeLZQCkRmnVr7tXY3GIv/huc6dTazFbw5yHLulrfDwWCZqhvCWWiI6HyQUsjn8xRSTRjH/9zh0jbgImg8fcrZOM5bB1I16d+FTdAoQTWtrVS5b1YoPWuy3tSQ0bnGpz9aBpyXDe3sGvuwoNeyu34ap5a+TXvr4/X1xRKgXdvmop4Jed8piqFzUzZeMdUzgZ1ZKpRtxPQMfOiluAnpY29CFO0VDZvquXJWvUV1QOU97OrAA0XlQ9HBZZKDDzOtU+X3egEXWbhHpjTRZI+wHkkZtDMcmaedEzQ1B5eiQSaZnhu80cX5Jzr/SElIJfi347rBFZafLaimWbmWJGWfKFLrWjnbgbB/bJW4Zm5mvZA7LOzLqsf68J3EKaftHqnoiuAgIr8Mo6PD7vDd/rvu4fBo0J0cREF3ELw/OoiOjkhEjqDodmDQHwy7/WF3f3i9f+D39/3h4V7/8P3fUGtbqupzGZefm6SVw/WGYFOEuqbfZFeX5ZvsWsp6WypaUnlrBpvad1u3UsxWfFlVWn1+ClypHpm6CzKlRVw03kzd0QfCM6KxLsqxWRaTxH5uysWaxMHNwwJqBwORRGya2RcnONTWL9umWpckr5X085W0i7JplUBL7G1d05WweF3Y/8fC7uQ8eN3mr9VQHg1GyLEkEkbLFI2jFUrHJLinSdi43vp7+3v9dcrKOXRGF+dO0jcrxpoh1w3tVqo28IDGRs+DpiT+tXqAWaJ+NPMo8dZJztbbUCfRNP2ueyknzKh2k8LCqdFbcGoUPEA9Ch6UirQ6CfFfv/GNgZGlYw9mKG39W1gs0PuL5HmOw/blPYrLkCky4bXX9/d0vvTCH1cOEzXa1MSZ4HKscm/NsOye4M2laxLedqrXKc2ZF4o1mddjFinV5mcaoFkhiBfO4MTG6po2pQJY6tqwo7IeoyCgqd5oO671CRefr65xHdxXD7EIzXcjBBcHf5tkhZm7/VIBxxbASTLNrLS1mPjzL6brhaA=
sidebar_class_name: "post api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
-
-
-
-
-
-
+
+
Fork Workflow Variant
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
+
-
\ No newline at end of file
+
+
+
Path Parameters
+
+
+
+
+
+
+
+
+
+
Query Parameters
+
+
+
+
+
+
+
+
+
+
+
+
Body
+ required
+
+
+
+
+
+
+
+ revision
+ object
+
+ required
+
+
+
+
+
+
+ author
+ object
+
+
+
+ anyOf
+
+
+
string
+
+
+
+
+
+
+
+
+ date
+ object
+
+
+
+ anyOf
+
+
+
string
+
+
+
+
+
+
+
+
+ message
+ object
+
+
+
+ anyOf
+
+
+
string
+
+
+
+
+
+
+
+
+ name
+ object
+
+
+
+ anyOf
+
+
+
string
+
+
+
+
+
+
+
+
+ description
+ object
+
+
+
+ anyOf
+
+
+
string
+
+
+
+
+
+
+
+
+ created_at
+ object
+
+
+
+ anyOf
+
+
+
string
+
+
+
+
+
+
+
+
+ updated_at
+ object
+
+
+
+ anyOf
+
+
+
string
+
+
+
+
+
+
+
+
+ deleted_at
+ object
+
+
+
+ anyOf
+
+
+
string
+
+
+
+
+
+
+
+
+ created_by_id
+ object
+
+
+
+ anyOf
+
+
+
string
+
+
+
+
+
+
+
+
+ updated_by_id
+ object
+
+
+
+ anyOf
+
+
+
string
+
+
+
+
+
+
+
+
+ deleted_by_id
+ object
+
+
+
+ anyOf
+
+
+
string
+
+
+
+
+
+
+
+
+ version
+ object
+
+
+
+ anyOf
+
+
+
string
+
+
+
+
+
+
+
+
+ slug
+ object
+
+
+
+ anyOf
+
+
+
string
+
+
+
+
+
+
+
+
+ id
+ object
+
+
+
+ anyOf
+
+
+
string
+
+
+
+
+
+
+
+
+ flags
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ is_custom
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_evaluator
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_human
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ metadata
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+ data
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ service
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+ configuration
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+
+
+
+
+
+ artifact_id
+ object
+
+
+
+ anyOf
+
+
+
string
+
+
+
+
+
+
+
+
+ artifact
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ name
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ description
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ slug
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ flags
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+ metadata
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+
+
+
+
+
+ variant_id
+ object
+
+
+
+ anyOf
+
+
+
string
+
+
+
+
+
+
+
+
+ variant
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ name
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ description
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ slug
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ flags
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ is_custom
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_evaluator
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_human
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ metadata
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+ artifact_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ artifact
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ name
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ description
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ slug
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ flags
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ is_custom
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_evaluator
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_human
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ metadata
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Successful Response
+
+
+
+
+
+
+
+ Schema
+
+
+
+
+
+
+
+ variant
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ name
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ description
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ slug
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ flags
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ is_custom
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_evaluator
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_human
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ metadata
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+ artifact_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ artifact
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ name
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ description
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ slug
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ flags
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ is_custom
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_evaluator
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_human
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ metadata
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Validation Error
+
+
+
+
+
+
+
+ Schema
+
+
+
+
+
+
+
+ detail
+ object[]
+
+
+
+
+
+ Array [
+
+
+
+
+
+
+ loc
+ object[]
+
+ required
+
+
+
+
+
+ Array [
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+ integer
+
+
+
+
+
+
+ ]
+
+
+
+
+
+
+
+
+
+ ]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/docs/docs/reference/api/get-all-workspace-permissions.api.mdx b/docs/docs/reference/api/get-all-workspace-permissions.api.mdx
index ab12ab47b7..5dcecd6447 100644
--- a/docs/docs/reference/api/get-all-workspace-permissions.api.mdx
+++ b/docs/docs/reference/api/get-all-workspace-permissions.api.mdx
@@ -5,67 +5,144 @@ description: "Get all workspace permissions."
sidebar_label: "Get All Workspace Permissions"
hide_title: true
hide_table_of_contents: true
-api: eJztVkuP2zYQ/ivEnAV726NO9SFIFunBSLbowWsItDS2mVCkMjOy4xr+78XIeq+T5gd0L2t+882LnIeuUCDn5CpxMUAK71GM9d6cI33lyuZoKqTSMbsYePEaXsMnlJoCG2u8YzFx3/DtyTpvdx7/WzN9DcYY86dj2ax7yjY1q97iAJu4+4K5sCGsCBmDuHAwcsRf82gdY+fww8vL+t33HJtUU/O8VzOExrGxwSBRJEMo5PDU+XhsGRKIFZJVO88FpHBAyaz3WU/PRnRIgJCrGBgZ0iv8/vSk/6bX/rnOc2Te1958asmQQB6DYBCl26ryLm9cLr+w6lyB8yOWVn85wbIxLpcKIQUWcuEACWCoS0g3QGiLjC8sWEICJ4fnbGRRY8TCyRhT94RWUMHsZMnZIJBAgR7fgGUs3P7SAVkew94dauqND0qd+ZEy1SFjpJPLcXDKmBNKF2t/qqtiIm0NT9kaG4aToxhKDJIVWPl40Z+jPH/MGKX9Q07jR5CFsTc6HFsLA9CGOQCNPp6sr7vL1lsYgP5FJpzW7gRrTc8wDXX2mo3LvkI7+2OgNT+GWutjqH3rmpEyih41VFsUWcDzHZT41k2kgw3un1nQM3RuZSbW9pesssznSEWX0c5539a6OhqOfb57H888zrcD9MrH5/GjRJo/wR1pON9qJIc9YTiOy4JnddHLbQhRZn03gRqWC6c4684pNCuhjOpHVdPBczbnGCy5+EhlLHvjBbn28tBRL5nrlDpU80c6g2Su863GGh+ptIJtAuLE67AblgXckm4CWiJ7gYHUTVWjC27lvfm7H+3r0ay+6V8CJcoxtpMdEqisHCGFZV/WvJwOeB1fSAzp5go1eUjhKFJxulzmPtbFwh4wiF1Yt7SVg9tWNfKanFwaldX6+SNePqAtkCDdbMeEzzrk75tjSuuHva3cR9Rcgy31vKrlGGnoG6cL5njX0uRc2MdGvb2aVROcWa2f9R2Q+L6Snha/LZ6aXh0vqglZ95PNm/3U+W7EOqgnt9Dnrw9aWqcyQVv+MQg0siqylDaM7P38tWbBXYeF+f93zK99x7QlJPhdlpW3rmmh5u2ubdVvYKh6bYXR9W8TOEYW5VyvO8v4F/nbTWGdiFra2wSaPb/TQttsb0lXh1r1X/GiFZNrJlp51tf3cp596Wg39A35/t0L3G7/AnMr6lI=
+api: eJztVU2P2zYQ/SvEnAV726NOcYMgWSQHI9miB68hjKWxzSxFqpyRHcfwfy9G1neybX5A97LW4+Ob7+EVCuI82kps8JDCexKDzplziC9cYU6molhaZhs8L579s/9MUkfPBo2zLCbsGz6e0DrcOfrvm+mzN8aYT5Zls+4p29SsesUBNmH3lXJhE6mKxOTF+oORI/2aRbRMncEPT0/rd99yakJNzeNeZSIZywa9oRhDNJEkWjp1Nn6uDAmEiiKqzmMBKRxIMnQu6+nZiA4JROIqeCaG9Aq/Pzzov2nav9R5Tsz72pnPLRkSyIMX8qJ0rCpn88bk8ivrnStwfqQS9ZcVKhtxuVQEKbBE6w+QAPm6hHQDkbDI+MJCJSRwsnTORopKLKzMoDwSCs3Aghz9AA7M7ITRopcJcwSWobD7SwdkefB7e6jvqeSfyo8ux9pnTPFkcxqMMuWRpAuq/6qrYnLaCk/Z6hv5k43Bl+QlK6hy4aI/Rxl5nTEK+1VOY0eIhakXHT5bhQFo3RyA5j6d0NVdsjULA8Cd6ITT6k6wVnqGqauzajYm+1bu9MdAKz+GWvUx1Na6ZopZDI7UVSyKzNP5Dkr40UyIB/T2+8zpGTpXmR3rnpCsQuZziEUX0c461w6FGho++3j3Lpx5HG8HjIsQ4jzld2RcaJ5Vuj9H74NM6zaFuv4fgdsExIrTsR7WItySbtYxRrzAQOr2h9FVvnLO/NUvsfVoK930L4GS5BjaHQYJVChHSGHZ14WX01Wm80eRId1coY4OUjiKVJwul7kLdbHAA3nBBdolVrYJaLzmVs2peatUuG1VLq+jlUujt1o/fqTLB8KCIqSb7ZjwRXfdfYFOaf3Ow8p+JE2Ex1K/V7UcQxy6wqoDx/stjdz6fWiut3lrffsD8xfyTddQ5LvbD4vfFg+vBdNeMKv1Y7uyMW9WdudHQ9OVNElXnyhthBKtnglh+WY4UC+rwFKiH+n9e1lnTl6HN+T/p/3Xnva2nYS+ybJyaJtZa2p3bcdjA8N46MyM0r9N4BhYlHO97pDpz+huN4X/rilqm28TaF60nTbdZntLup7UCXihi3ZMrpFoB6Kr7609e/x1MvrJff/uCW63fwCoIY1k
sidebar_class_name: "get api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
+>
-
-
-
-
+>
Get all workspace permissions.
Returns a list of all available workspace permissions.
Returns:
- List[Permission]: A list of Permission objects representing the available workspace permissions.
+List[Permission]: A list of Permission objects representing the available workspace permissions.
Raises:
- HTTPException: If there is an error retrieving the workspace permissions.
-
-
-
-
-
-
-
-
-
-
-
-
-
+HTTPException: If there is an error retrieving the workspace permissions.
-
\ No newline at end of file
+
diff --git a/docs/docs/reference/api/get-all-workspace-roles.api.mdx b/docs/docs/reference/api/get-all-workspace-roles.api.mdx
index c6801a440d..50cfaf38ca 100644
--- a/docs/docs/reference/api/get-all-workspace-roles.api.mdx
+++ b/docs/docs/reference/api/get-all-workspace-roles.api.mdx
@@ -5,67 +5,119 @@ description: "Get all workspace roles."
sidebar_label: "Get All Workspace Roles"
hide_title: true
hide_table_of_contents: true
-api: eJztVMtu2zAQ/BViz4Tl9qhTfQgSIz0YTooeHCPY0GuLCUWq5MqJK/jfi6Ut+QG0+YHqYomcfczsrDtg3CQoF/AzxLfUoCFYalhRMtE2bIOHEm6JFTqn3nuIisFRGj35Jz8nbqNPCpWziVVYZyRu0Tp8cfSvmPLJK6XUd5t4MVSfB0dzSk3wiZalmgxpLxAqvLyS4aQiNZESebZ+o7iizyqjTdQXvnt8nN18GMo0SzVdK/SKYgxRBWPamNR7ZR2pSBwtbfsK13lBQ2goomSZrqCEDfEzOvc8AJ8zEDTEI7EEZQdfx2P5uZT6oTWGUlq3TvUqgAYTPJNngWPTOGtyseI1SUwHyVRUo7xZpjonx9XKCgbdLEp3bA9FedcQlJA4Wr+Bve4PDnqeHWCMuAMNbNnJd9+NEjNMnDvNQ80zu708GmriKhxVAA0NcgUlFIMWqejFSBS3FMV7HbTRQQkVc5PKojAutKsRbsgzjtAW2FjYLyXCtNHyLodMZtN72t0RrihCuVieAx5EkAPhS9ggADb2noSfx1q+Jy1XIdrfWVjQYGUY1SFKaFm/Djn8KMckN6cmsyloEBqH8Y1HX0ZjuN6fC7DMEk2eZV87X4O+UmHgDxqoRit3TFh/O11IZ01IXKM/y/e3CV211Z1s9X/DP9vwo22YPrhoHFovq5Ln1R09voCTx2XVs+RLDVVILLdd94KJfkS338vxr5aiGHmpYYvRCp/sYt27Tjz+RjvxhxEG4jN07cG8V/8B4v1h8W5vHmG//wP6PA52
+api: eJztVE1v2zAM/SsCz0Kc7ejTsqFog+4QpB12SIOCVZhYrSx5Ep02M/LfByp20gZo+wfmi23p8evxkR0wbhKUC/gd4lNq0BAsNawomWgbtsFDCZfECp1TzwNExeAoje78nZ8Tt9EnhcrZxCqsMxK3aB0+OPrIprzzSin10yZeHKPPg6M5pSb4RMtSTY5u3yBUeHgkw0lFaiIl8mz9RnFFn0VGm2gIfHV7O7t4MZTLLNV0rdArijFEFYxpY1LPlXWkInG0tB0inPsFDaGhiOJluoISNsT36Nz9EXifgaAh9oUlKDv4Oh7L6y3VN60xlNK6dWpgATSY4Jk8CxybxlmTgxWPSWw6SKaiGuXLMtXZOa5WVjDoZlGyY3sIyruGoITE0foN7PVwcODz1QHGiDvQwJad/A/ZKBHDxLlTP9Q8V7eXR0NNXIWeBdDQIFdQQnHkIhUDGYnilqJor4M2OiihYm5SWRTGhXY1wg15xhHaAhsL55qc5Fv1Q6CwX4o700bLu+xvMpte0+6KcEURysXyNeBG2Dqw8RZ2ZAcbe01SvMda/ictVyHav5l10GAlgepgJTVbvw7ZvOeqz+07mifyK9AgdR7SHo++jMbvFdMbqMls2jcdTW76kEeGgT6j60gUaKAardwxYf3tdCFZNiFxjf6Vv/daeZZed9Lf/1Xw2SroJcT0wkXj0HqZqdyvrh+GBZyGQXZCpnypoQqJ5bbrHjDRr+j2ezn+01IUUS81bDFaqScrWg8KFL0/0U70YaQC0Ru69iDks2Uhc3Cc0MuLW9jv/wFK9h0F
sidebar_class_name: "get api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
-
-
-
-
-
+>
+
Get all workspace roles.
Returns a list of all available workspace roles.
Returns:
- List[WorkspaceRoleResponse]: A list of WorkspaceRole objects representing the available workspace roles.
+List[WorkspaceRoleResponse]: A list of WorkspaceRole objects representing the available workspace roles.
Raises:
- HTTPException: If an error occurs while retrieving the workspace roles.
-
-
-
-
-
-
-
-
-
-
-
-
-
+HTTPException: If an error occurs while retrieving the workspace roles.
-
\ No newline at end of file
+
+
+
+
+
Successful Response
+
+
+
+
+
+
+
+ Schema
+
+
+
+
+
+ Array [
+
+
+
+
+
+ ]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/docs/docs/reference/api/get-config-deployment-revision.api.mdx b/docs/docs/reference/api/get-config-deployment-revision.api.mdx
index d2cdd35c39..b8b63992a5 100644
--- a/docs/docs/reference/api/get-config-deployment-revision.api.mdx
+++ b/docs/docs/reference/api/get-config-deployment-revision.api.mdx
@@ -5,36 +5,36 @@ description: "Get Config Deployment Revision"
sidebar_label: "Get Config Deployment Revision"
hide_title: true
hide_table_of_contents: true
-api: eJyVVNFu2jAU/ZXoPG2SBazaU56G1qpF3bSqZXtBCLnJJXGX2K7toLHI/z7ZCQEKbNoTwb73+N5z7rktHC8s0gU+K7kWhcWSQWky3AklZzlSFORWWbxc5aQrta1JupWhjbBCSTBobnhNjkzAaSF5TUhxJnYlcjAIiRSauxIMhl4bYShH6kxDDDYrqeZIW7itDijWGSELMDjhqnBwPcAmjz1sMsvh/TKgWa2kJRsAriaT8JOTzYzQoRukeGqyjKxdN1Xy2AeDIVPSkXQhnGtdiSw2P36xIafdF+W99wwfr65OgX/wSuQxLbkxRpn/QIU2gXAnurpzclxU4Us4qu1pQKWyo1sut9/WkfljzjwbToR0VJCBX3q2O+PG8O0BsV9UVyA8Q22Lv2nwlazlBWEAuxwayUjm4dbv31bPL5S5I/0Xsa/u6T5uuYfZ09uxe7mN646+c4/tQu7m84cTwE7bY1FvySWdLZIzYxeKJVeq3iPRCK5EinHnFjveW2DcnreDB4Mls9l5pzEVUpTOaZuOx1mlmnzEC5KOj7gYcy0Q5txS1hjhtjFl+jC7p+0d8ZwM0sXyMOApzFg3Ncdhg2Bci3sK/PWunTauVEb87kahN2vZZQWChFyrmN5zOY3FJdOHGRhCGx1zk9GH0QRv+TwKDvbgWbTH7u14DfaGhaF/MFAdvQFHvP60vwiVaWVdzeUB3j/lO6puoMTRLzfWFRfRCrGWtpd2gV7amLyDBEN6YdstGUplXchs22du6bupvA/Hrw2ZIOCSYcON4M+BzkWLXNjwnSNd88rSSZHDVsG7x9487xOw88XvRJZB4Q2vmvAPDD9pe3lDxyVR7uap7YOnWUbaHcCc7LQweIMhbm/m8P4POBkyRQ==
+api: eJyVVNFu2jAU/ZXoPm2SRVi1pzyNtVWLumlVy/aCEHKdS+I2sV37Bo1F/vfJTghQYNWeCPa5x8fn3uMWiBcOsjlcarWShYMFA23QcpJaTXPIoEBairi5zNFUelOjoqXFtXRSK2BguOU1EtrA04LiNUIGJ7BLmQMDqSADw6kEBhZfG2kxh4xsgwycKLHmkLVAGxNYHFmpCmBAkqqwcDXQJg89bTLNwftFYHNGK4cuEFyMx+EnRyesNOE2kMFjIwQ6t2qq5KEHAwOhFaGiAOfGVFLEy6fPLtS0O1Hee8/g88XFMfEvXsk8liXX1mr7H6xgbDCcZKc7R+KyCl+SsHbHgEqLg12uNj9W0flDzzwbVqQiLNCCX3i2XePW8s2esd90JxA8g9oV/+rBd3SOFwgD2XloNCOZhV2/O1s/PaOgg/7P4726o3vcYkezs7dz9/w1rjr7Th22hdzOZvdHhF1vD5t6g5R0sUhOjF0Qi1TqPiMxCFRCBmmXFpfuIpC2p+PggYFDu95mp7EVZFASGZelqah0k494gYr4iMuUGwlvNU7ibnIZoBBC4FA0VtIm8k3up3e4uUWeo4VsvtgHPIYB7EbqEDZ0kxt5h8HcPtKThkpt5Z9uTvokl11VcE+qlY7lvdG9tq9cvKAK0Q/37GSPR59G43OX6QuSyf20zxEXMUdbHREG7I1dg1HAAOsYIiDk9ZfdRlBptKOaqz2+d/t8oHKwh/A3pabiMmYmamn7GZhDPwOxeEsJDLIzz+KCQakdhcq2feIOf9rK+7D82qANzVwwWHMr+VOwdt5CLl34ziFb8crhkcjh+YEPD33KPibATovfNlyFbq951YR/wOAFN+ef8vialNvZanvwRAg0tEdz9PiFIRySc3M9A+//AtfHQNQ=
sidebar_class_name: "get api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
+>
-
-
-
-
+>
Get Config Deployment Revision
@@ -43,29 +43,254 @@ Get Config Deployment Revision
as={"h2"}
className={"openapi-tabs__heading"}
children={"Request"}
->
-
+>
-
-
-
-
-
-
-
-
-
-
-
-
-
-
\ No newline at end of file
+
+
diff --git a/docs/docs/reference/api/get-evaluation-scenario-score-router-human-evaluations-evaluation-scenario-evaluation-scenario-id-score-get.api.mdx b/docs/docs/reference/api/get-evaluation-scenario-score-router-human-evaluations-evaluation-scenario-evaluation-scenario-id-score-get.api.mdx
index 134b72247f..58b295fd04 100644
--- a/docs/docs/reference/api/get-evaluation-scenario-score-router-human-evaluations-evaluation-scenario-evaluation-scenario-id-score-get.api.mdx
+++ b/docs/docs/reference/api/get-evaluation-scenario-score-router-human-evaluations-evaluation-scenario-evaluation-scenario-id-score-get.api.mdx
@@ -5,73 +5,317 @@ description: "Fetch the score of a specific evaluation scenario."
sidebar_label: "Get Evaluation Scenario Score Router"
hide_title: true
hide_table_of_contents: true
-api: eJzdVU1v40YM/SsDnlpAa2WDnnSqgaSJsS0aJG4vjmEwI1qarTSjnaGCuoL+e8GR/BUrQXvdk+UZ8pF8fOR0wFgEyFZw39ZoP92+YtUiG2cDrBPIKWhvGvkPGfxCrEvFJamgnSfltgpVaEibrdGKDq4qaLLojZs922c790XInq1S6sRkszfZmDxTy5LU4kbwBHwCSLFTW4keER+JW2/3oDdGiy36ndLOMhprbDFmOXovbhTaXBkOQ+YzSMA15GOQRQ4ZFMSbqeyi+ca7lslvSqHoxCxMukwemnwzYhXEkECDHmti8sJ9BxZrggymPSEBI/Q3yCUk4OlbazzlkLFvKYGgS6oRsg541whKYG9sAQmw4UoOjk1VTwdKcuj7taCFxtlAQQCur67k57zrT63WFMK2rdTjaAwJCNVkWcyxaSqjI376NYhPd5mUe/lKWirHPDexX9WDlx6wGWKfJ98fs98HVXfEaqqUpyjGx9gjFWV8YhYmXSYPF7kase6Ioe/7PoGfrq8vGfkTK5MPzrfeO/9/6GjOis6J0VTyZZjqcGlQOX12i3b3+zZK5i1fhxNjmQry0K+FxOEMvcfdiSJ+dUOCwnMdio/E8xuFgAXBAewDnQkZaim3/TH2ofNH4a5iXUPo0W59hDnSO7D7fhk3A31TwfYm98vlwwXg0NuauHTj8MeR5BIySOOUfzqZ8nRiLNNuelb7NE45JBDIv+7Hu/UVZFAyNyFLU125Np9hQZZxhibFxoCMYiDdesO76DJ/WHyh3T1hTh6y1frU4EnUNOjj3OzQGmzMFxKmxsUyb7l03vwzNH3cJ+XgJVQYu3XRfWRtHpNT84cFJCBlDMK/mn2eXcHbZ+HMOK5gHQdhHzteQ/KGhUP9kADVcQqACeufjxeSWeMC12hP8P7LGnibY3cc0O/rFRv7zfQ3p02FJk50JLobBb2CC0EL45c1QALZuw/QoOp1AqULLKBd94KB/vBV38vxt5a8CHedwCt6gy8io1UHuQnynUO2xSrQB2354XFcDz+q9+rai9uKsiVT+QcJ/EW79x/PuAbL/Rx1o/Fca2r4BOZia8vAHTbE3e0S+v5fAjE5AA==
+api: eJzdVU1v40YM/SsDnrqA1k6DnnSq26SJsS0aJG4vjmEwI1qaXWlGO0MFdQX99wVH8lcsB9trT5ZHj2/Ix0eqBcY8QLqE+6ZC+/H2FcsG2TgbYJVARkF7U8t/SOE3Yl0oLkgF7Twpt1GoQk3abIxWtA9VQZNFb9zk2T7bmc9D+myVUkeQ9Q6yNlmqFgWp+Y3wCfkIkWKnNnJ7ZHwkbrzdkd4YLVj0W6WdZTTW2HzIcoie3yi0mTIc+swnkICrycdL5hmkkBOvx7KL8LV3DZNfFyLRESyMhowemmw9cOXEkECNHiti8qJ9CxYrghTGIyEBI/LXyAUk4OlrYzxlkLJvKIGgC6oQ0hZ4WwtLYG9sDgmw4VIODk1VT3tJMui6lbCF2tlAQQiur67k57TrT43WFMKmKdXjAIYERGqyLHCs69LoyD/9HCSmPU/KvXwmLZVjlpnYr/LBSw/Y9HefJt8dst9dqu6I1VgpT9GMj7FHKtr4CBZGQ0YP55kauO6Ioeu6LoGfrq/PFfkbS5P1wbfeO/9f5KhPis6I0ZTyZJiqcA4onT55i3b75yZa5q1e+xNjmXLy0K1ExP4MvcftkSN+d32ConMV8vfM8weFgDnBnuwdn4kYaiFvu8Pd+84fjLuMdfVXD7jVgeYgb6/u5TJuevnGLttB7heLhzPCvrcVceGG4Y8jyQWkMI1T/vFoyqcjYzltx2e1m8YphwQC+dfdeDe+hBQK5jqk06kuXZNNMCfLOEEzxdrA21U7i2/VrwIFmdNAuvGGt5Fv9jD/RNt7wow8pMvVMeBJrNab5xS27xvW5hOJjMPWmTVcOG/+7R0xLJuijxKdjN24GD5IOuT2C+ovZGU7SZ192leTHydXl4oZAtTsYT5MDOo4Mbs8IgySN3LthYIEqIrjAkxY/Xx4IVnWLnCF9ojve/bF21zbwyT/vz53Q++Z/uFpXaKJox+FbgfnL+HM+aL4eQ2QQHrxS9Xbf5VA4QILadu+YKC/fNl1cvy1IS8mXiXwit7gi1hq2UJmgjxnkG6wDPROW354HPbIB3Wprp3RrbhcMpV/kMAX2l7+ysZ9Wexmqh3AM62p5iOas/Uuw7dfJXe3C+i6b5V/R48=
sidebar_class_name: "get api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
+>
-
-
-
-
+>
Fetch the score of a specific evaluation scenario.
Args:
- evaluation_scenario_id: The ID of the evaluation scenario to fetch.
+evaluation_scenario_id: The ID of the evaluation scenario to fetch.
Returns:
- Dictionary containing the scenario ID and its score.
+Dictionary containing the scenario ID and its score.
-
+>
-
-
-
-
-
-
-
-
-
-
-
-
-
-
\ No newline at end of file
+
+
diff --git a/docs/docs/reference/api/get-evaluator-config-evaluators-configs-evaluator-config-id-get.api.mdx b/docs/docs/reference/api/get-evaluator-config-evaluators-configs-evaluator-config-id-get.api.mdx
index 4afc7ecf0e..34391cf2e0 100644
--- a/docs/docs/reference/api/get-evaluator-config-evaluators-configs-evaluator-config-id-get.api.mdx
+++ b/docs/docs/reference/api/get-evaluator-config-evaluators-configs-evaluator-config-id-get.api.mdx
@@ -5,70 +5,299 @@ description: "Endpoint to fetch evaluator configurations for a specific app."
sidebar_label: "Get Evaluator Config"
hide_title: true
hide_table_of_contents: true
-api: eJzNVk2P2zYQ/SvEnFpAsLeLnHSqmyySxSapsbvJxWsYXGkkMZFIhhwt6gr678WQsi3ZXjfoqScb5JuvN2+G6oBk6SFdwc2LrFtJxnlYJ5Cjz5yypIyGFG50bo3SJMiIAimrBO7QIjO6UGXrJGO9KIwTUniLmSpUJqS1syf9pO+RWqd9+qSFEOKj8rTaB3wbPLz7Y52KhaiVJ2GK1wII8/wNM/IzSMBYjIe3OaRQIm32RptodDjww4nfnIJUvtmUSJCAlU42SOiYkQ60bBBSOGMACSgmxkqqIAGHP1rlMIeUXIsJ+KzCRkLaAW0tu/DklC4hAVJU88G+eBGrF7c59P2aXXlrtEfP1tdXV/wzbcZDm2XofdHW4n4AQwKZ0YSaGC6trVUWmJl/82zTjTKyjnkjFSOo/FKWnFQy0PA66jPf9wl75t5sLvtcRpSIvg/cfsftzxF2h1u29EikdOk3fBGLkXr7ZxE6J/Nccf2yXo7Kjc0ZIkQhQZ/sY+q2roF7sIv5MIQQX2OIPoHMoSTMN5IuJfs2osSC/UNr85+w+RJRwaY/znIssRUE/YWuTEg/ZnOS7SSN9Rleow6h7zn6m+vrU+F9lbXK4xTeOGfcf1ddjiRVHfRH2PhTQG2yye2hs1P6Rt1TmrBEB/36wJ50Tm5HHH80MUHuSuPLS+34hN7LMug6Qi5Ik8kQj3z7b53jumLoATfqxIHeyO7rZbyL9J0LtoN8eHxcnjiMvW2QKjNszLDzqIIU5odVOR9W5bw7s/l64NFzL7sd2boaUqiIrE/n86w2bT6TJWqSM6nm0qowUR6z1inaBpPF8vYOtx9Q5uggXa3HgAdWTNTAFLanX1p1F8Q9bOdFS5Vx6u/Y2GEvV9GKy1W6MMF8YGYRkhOL5S0kwGVEcV/NfptdwfHDNwGz2GUWxL6LHa55tiYs7OvnkWyC0oFQNr8fLjgzazw1Uo/8vUcSxy/DcU7dYej+p+/y0CjCv2hua6nCuAWGukFtq8Om8pFY1hskkJ57a9cJVMYTm3Xds/T4xdV9z8c/WnSsqXUCL9Ip+cwdXnWQK8//c0gLWXu8wOAv98N0/ipey3ynO82iC48NpAAJhAfr7MdBWEHVTt/dgFxkGVoa+TjZmDwI++l8f/MIff8PsRtW8A==
+api: eJzNVk1v2zgQ/SvEnHYBwc4GPelUNw3aIO2ukaS9OIbBUGOJrUyy5CioV9B/Xwwp2/LnFnvakw3yzdebN0O1QLIMkM/g9lXWjSTrA8wzKDAorx1payCHW1M4qw0JsmKJpCqBG7RQ1ix12XjJ2CCW1gspgkOll1oJ6dzo2TybB6TGm5A/GyGE+KQDzbYBb6KH9+/muZiIWgcSdnkugLAv31BRGEEG1mE6vCsghxJpsTVaJKPdQehPwuIYpIvFokSCDJz0coWEnhlpwcgVQg4nDCADzcQ4SRVk4PFHoz0WkJNvMIOgKlxJyFugtWMXgbw2JWRAmmo+2BYvUvXiroCum7Or4KwJGNj6+uqKf/ab8dgohSEsm1o89GDIQFlDaIjh0rlaq8jM+Ftgm3aQkfPMG+kUQReXsuSksp6G86g/+b7L2DP3ZnHZ5zShRPK94/Y7rn+NsHtcs2VAIm3KsOCLVIw067+WsXOyKDTXL+vpoNzUnD5CEhJ02TamaeoauAebmI99CPE1hegyUB4lYbGQdCnZm4QSE/YPjSt+weZLQkWb7jDLocRmEPUXu7JH+iGbe9nupTE/wWvSIXQdR39zfX0svK+y1kWawlvvrf/vqiuQpK6j/ghX4RhQW7V3u+vsPn2D7mlDWKKHbr5jT3ov1wOOP9mUIHdlFcpL7fiMIcgy6jpBLkiTyRBPfPtvneO6UugeN+jEjt7E7vky3if6TgXbQD4+PU2PHKberpAq22/MuPOoghzGu1U57lfluD2x+Trg0fOvmx3Z+BpyqIhcyMdjVdumGMkSDcmR1GPpNBw+JpN4K24YGsctoGq8pnX0N5ne3eP6I8oCPeSz+RDwyHJKAtmHbXsjnb6Pyu9X96Shynr9d+p6v7SrZMVcaLO00bynrc/tnVTf0fBEcZ0p7avRH6Orc8X0BmIyveunQqo4FZs8IoyHcI+uLVE8u6s4EkAoV293F5yls4FW0gz8fUASh0/IYW7tbjr/pw943zTCnzR2tdRxLiNDbS/L2W6lhUQsCxMyyE89yvMMKhuIzdr2RQb84uuu4+MfDXrW1zyDV+m1fOFuz1oodOD/BeRLWQe8wOBvD/0Y/y7OZb7RoGEBxlcJcoAM4st28isi7qpqo/W2R06UQkcDH0erlYdiO8Yfbp+g6/4BGvRlfw==
sidebar_class_name: "get api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
-
-
-
-
-
-
+
+
Endpoint to fetch evaluator configurations for a specific app.
Returns:
- List[EvaluatorConfigDB]: A list of evaluator configuration objects.
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
+List[EvaluatorConfigDB]: A list of evaluator configuration objects.
+
-
\ No newline at end of file
+
+
+
Path Parameters
+
+
+
+
+
+
+
+
+
+
+
+
Successful Response
+
+
+
+
+
+
+
+ Schema
+
+
+
+
+
+
+
+
+
+
+ settings_values
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Validation Error
+
+
+
+
+
+
+
+ Schema
+
+
+
+
+
+
+
+ detail
+ object[]
+
+
+
+
+
+ Array [
+
+
+
+
+
+
+ loc
+ object[]
+
+ required
+
+
+
+
+
+ Array [
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+ integer
+
+
+
+
+
+
+ ]
+
+
+
+
+
+
+
+
+
+ ]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/docs/docs/reference/api/get-evaluator-configs-evaluators-configs-get.api.mdx b/docs/docs/reference/api/get-evaluator-configs-evaluators-configs-get.api.mdx
index 5f3234bb06..1980ea48f0 100644
--- a/docs/docs/reference/api/get-evaluator-configs-evaluators-configs-get.api.mdx
+++ b/docs/docs/reference/api/get-evaluator-configs-evaluators-configs-get.api.mdx
@@ -5,73 +5,325 @@ description: "Endpoint to fetch evaluator configurations for a specific app."
sidebar_label: "Get Evaluator Configs"
hide_title: true
hide_table_of_contents: true
-api: eJzdVk1v2zgQ/SvEnLaAYKdBTzqttwlSI91do3F7cQyDkUYSW5lkyVFQ19B/XwwpWVI+vN3rngwM5+PNmzcjH4Fk6SHdwPWjrBtJxnnYJpCjz5yypIyGFK51bo3SJMiIAimrBPbeIjO6UGXjJPt6URgnpPAWM1WoTEhrZ/f6Xi9c6dN7LYRg007l4jdP7k0q1hWK5ZUwhaAKT+6fkBqn+4iPytPmhO99KHj1xzYVC1ErTxz8Ch5hHr5iRn4GCRiL0bjMIYUSaXcK2sUgP1j8yVQiQQJWOrlHQsdcHUHLPUIKsRVIQDFL3xt0B0jA4fdGOcwhLWTtMQGfVbiXkB5B6sPfRchAB8sZPDmlS2iTk0U3dQ3tNgFSVLNhYa1Y5tCyzaG3Rnv0nOzy4oJ/pqO6a7IMvS+aWnzqnCGBzGhCTQGCtbXKAhHzr55jjiOAHQjpnORWFOE+1LKO6SMVK6t85Nu1MABmsElH0etef/F7m3BmHtHufM5V9BIx9zC4b3g4F3YSjbjFA0d6JFK69Dt+iM0MM5F5rpgXWa9G7ZJrMOkrRD2dndddV0J8iSXaBDKHkjDfSToH9n30EgvOD43NfyHmc/QKMe1TlGMpbiAINUxlQvpTNidoJzC2L/Aal5Hx9k+96sQNkhj4j45+sPiT6QYZPMN/d3n5XNFfZK3yuM3Xzhn3X+Q8lW2OJFUdBPyyrmuTTV5/YV2VJizRQbsd6O+3p6fko4kAmaa9L8/N80/0XpZhMaLLGW0zGWLNr/82eu4rlu78RqMc6I3svt7GVaTvpWK9y4f1evUsYZztHqky3eUNB5UqSGE+XNx5d3GBt9Q99qe2cTWkUBFZn87nWW2afCZL1CRnUs2lVWH5PGaNU3QIIYvV8hYPH1Dm6CDdbMcOd6yNOO2p23D7rLoNe9Ad+UVDlXHqZxxhd+urGMWNKV2YEN7f6wBOLFZLSIDbiDK+mL2dXcDTL+vEmWUtsyDrvnZ45jWcsHDqn7d3HzQNhHL/+/DAyKzxtJd6lO/FnXwK6jjs1//jy98NlvAHzW0tVVjEwOix0+FmOII+DiIQs02gMp74/Xh8kB4/u7pt2Ry/9Sy2XHn5UI++9uGTNPw1CB8aSCHI9FE6xd5BlUmvIk4ToxZZhpZGUc/OG2c5rdLN9Rra9h/3llyW
+api: eJzdVsFu2zgQ/RViTi0g2NlgTzqtmwSpke6ukbi9OIbBUGOJrUyy5ChYr6F/L4aULDuJve11TwbIN5w3b96MvAOSZYB8ATfPsm4kWR9gmUGBQXntSFsDOdyYwlltSJAVayRVCezRQlmz1mXjJWODWFsvpAgOlV5rJaRzo0fzaCa+DPmjEULw0UoX4l0g/z4X8wrF9FrYtaAK9/B7pMabPuKTDrTY87uKCa8/LHMxEbUOxMEn+Aj79BUVhRFkYB2mw2kBOZRIq33QKgWF4STsj0okyMBJLzdI6FmrHRi5QcghlQIZaFbpe4N+Cxl4/N5ojwXk5BvMIKgKNxLyHdDWcVggr00JGZCmmg8mzolpAW275OjgrAkYOODy4oJ/jrvx0CiFIaybWtx3YMhAWUNoiOHSuVqrWOv4a+CY3WsS0nvJbDXhJuZynhUinTLr4hxhJpt1KpxG/cX3bcYvcxdW59+cJZRIbw+9+Ybbc2F7X4g73HJkQCJtyrDii1SMNNu/17Fxsig06yLr2UG5qU9dhmQZaLN9TtPUNXBv+pwPXQrxJaVoM1AeJWGxknSO7FVCiQm/D40rfiLmc0LFmPYly0O3LSB6MXblSPSXah6xPaKxfEPXNG/Mt7/qXSdukcSgfwKG4STsj26RyTP93y8vXzv6i6x1kQb2xnvrf8XOx7YtkKSuo4Hf9nVt1dHtYI1j/Q/arw1hiR7a5SB/Pz29JJ9sIsgybUJ5rp9/YgiyjIORIGe8zWKIOd/+V+u5rpS6wx20cpA3qXu6jOsk31vJesjH+Xz26sHU2w1SZbvlGncmVZDDeFiq426pAk+pf+63aeNryKEiciEfj1Vtm2IkSzQkR1KPpdPw8oM0ibfiiqFxMgOqxmvaxvcms+kdbj+iLNBDvlgeAh7YOMkKx7BhMTp9F4ekW/KThirr9b+pv92ur1IUV63N2sbwfpknbh+k+oaGh4/rTLQvRr+NLk4V0wWIyWza+V+q6P+eR4TxvB7JtReKx3wTzQ+EcvPHcMEsnQ20kebgvTeH9yW53TCI/49/AV2TCf+hsauljhMbFd11hl0M2zKkRkRhlhlUNhDf73ZPMuBnX7ctH6fvPhuv0EE+1TyQa1kHPKPlu/tudt+LU6Tid2/4ixG/ZpBDtPuz9JozRXdnvRuZQoqaKIWODqJe7VB+ZT+vtzdzaNsfLvt1uQ==
sidebar_class_name: "get api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
-
-
-
-
-
-
+
+
Endpoint to fetch evaluator configurations for a specific app.
Args:
- app_id (str): The ID of the app.
+app_id (str): The ID of the app.
Returns:
- List[EvaluatorConfigDB]: A list of evaluator configuration objects.
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
+List[EvaluatorConfigDB]: A list of evaluator configuration objects.
+
-
\ No newline at end of file
+
+
+
Query Parameters
+
+
+
+
+
+
+
+
+
+
+
+
Successful Response
+
+
+
+
+
+
+
+ Schema
+
+
+
+
+
+ Array [
+
+
+
+
+
+
+
+
+
+ settings_values
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+
+ ]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Validation Error
+
+
+
+
+
+
+
+ Schema
+
+
+
+
+
+
+
+ detail
+ object[]
+
+
+
+
+
+ Array [
+
+
+
+
+
+
+ loc
+ object[]
+
+ required
+
+
+
+
+
+ Array [
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+ integer
+
+
+
+
+
+
+ ]
+
+
+
+
+
+
+
+
+
+ ]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/docs/docs/reference/api/get-evaluators-endpoint-evaluators-get.api.mdx b/docs/docs/reference/api/get-evaluators-endpoint-evaluators-get.api.mdx
index bff073893a..8214bc6bdc 100644
--- a/docs/docs/reference/api/get-evaluators-endpoint-evaluators-get.api.mdx
+++ b/docs/docs/reference/api/get-evaluators-endpoint-evaluators-get.api.mdx
@@ -5,62 +5,271 @@ description: "Endpoint to fetch a list of evaluators."
sidebar_label: "Get Evaluators Endpoint"
hide_title: true
hide_table_of_contents: true
-api: eJy9VU1v2zAM/SsCz0KS7ejTAqzoihRr0Y9TGgSMzcRqZUmV6GJe4P8+SHYcJ+162GG+2KDIp8ePR++BcRcgW8LFG+oa2foAKwkFhdwrx8oayODCFM4qw4Kt2BLnpUChVWBht4KGuMmTeTJ3xLU3IXsyQghxrQIvB+RVJubv44TdPFPOYQISrCOP8dKrAjLYEa+P8GvqWYxtO2KQ4Ck4awIFyPbwdTaLr9MM7us8pxC2tRZ3vTNIyK1hMhzd0Tmt8nT39DnEmD2EvKQK45diqhK485Eiq+4qgxXFNzeOIIPAXpkdSGDFOhp+xvNWwgs1n7ktqIlehfKU87oOY8yNtZrQjLy/JzfxGBJ0IGZldmHNVDmNnGKxKFTMBPXtiC/7muQBtyv6CPa+BxIPB6D2bAz2gKa52UK2PE+llYPF1FpDuxrRHUG0EmwIHyId8vwM6iYEiJy2WGuGbIs6UBu7/1orT2GtdbVGp9Yv1Pz7JXc9mrjWlZg7JRYR7YNrO+GMZuO8JgcDeo/NqNIPMbBt37eiT6SIckyj1U3OyWR81PGeyyiJa9ph3gzC+4TMQQ3iklgcd4AYJD+yXRJDGx8JFXFpe42CBIdcQgbTozITUf9GPqQG1F5DBiWzC9l0mmtbFxPckWGcoJqiU6kHgfLaK25SyPz2akHND8KCPGTL1djhPiqz0+Cp29AFdGqRatdpFOY1l9ar30nhIEHFrVB2UTEhZbY2hfd1mSdyYn57BRJiGt0emU2+TGZwvh9PnONSwZyP+6E/BnlWhSF/kEAVqnjGhNW340Fk5mzgCs0I7y+tOqe1P+63/7fB+/Iz/eKp06iS6FPe+35KlkAnv5rSBo7W/X6DgR69bttofq3Jx0FYSXhDr3AT27JctfLQtTgjabHCPM/JxfQjbtf8s2UeZ2cY2cuLB2jbPxsiaS0=
+api: eJy9VU1v2zAM/SsCz0LS7ejTsq3oihRr0XanNAgYm4nVypIq0cUyw/99kOw4Tvpx2GG+2KDIp8ePRzfAuA2QLeD8BXWNbH2ApYSCQu6VY2UNZHBuCmeVYcFWbIjzUqDQKrCwG0FD3OTBPJhb4tqbkD0YIYS4UoEXA/IyE7PXccKuHynnMAEJ1pHHeOllARlsiVcH+BX1LMa2LTFI8BScNYECZA18PjuLr+MM7uo8pxA2tRa3vTNIyK1hMhzd0Tmt8nT39DHEmAZCXlKF8UsxVQnc+UiRVXeVwYrim3eOIIPAXpktSGDFOhp+xvNWwhPtPnKb0y56FcpTzqs6jDHX1mpCM/L+ntzEr5CgAzErsw0rpspp5BSLRaFiJqhvRnzZ1yT3uF3RR7B3PZC43wO1J2PQAJrd9QayxWkqrRwsptYa2uWI7giilWBDeBNpn+dHUNchQOS0wVozZBvUgdrY/edaeQorrasVOrV6ot2/X3Lbo4krXYmZU2Ie0d64thPOaDZOa7I3oPe4G1X6Pga27etW9IkUUY5ptLrJOZqMtzrecxklMUjuAxp7HYgLYnFQvxjEPrJdEEMbHwkVcWl7dYIEh1xCBtODJhNF/0I+pNLXXkMGJbML2XSaa1sXE9ySYZygmqJTcLptZulUfIuuqTWB8tor3iW82c3lnHY/CAvykC2WY4e7KNhOmsduQ3PQqXkqaSddmNVcWq/+JOGDBBUJlF1UzFaZjU3hfdF6bl8xfyJTgISYZ0f7bPJpcvZeMn2AmN1c9nsHcz6skN4N5Em5hkKBBKpQxTMmrL4cDiJLZwNXaEZ47/T0lF5zWIH/b8n3rWD6zVOnUaW9kPJu+nFaAB39jUobOFqbZo2BfnndttH8XJOPQ7GU8IJe4Tq2aLFs5b6DcV7S7oVZnpOL6UfcbhBO9n2co2G2L87voW3/AqNKdWc=
sidebar_class_name: "get api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
-
-
-
-
-
+>
+
Endpoint to fetch a list of evaluators.
Returns:
- List[Evaluator]: A list of evaluator objects.
-
-
-
-
-
-
-
-
-
-
-
-
-
+List[Evaluator]: A list of evaluator objects.
-
\ No newline at end of file
+
diff --git a/docs/docs/reference/api/get-own-org.api.mdx b/docs/docs/reference/api/get-own-org.api.mdx
index a609a250b7..c1c74e3a8c 100644
--- a/docs/docs/reference/api/get-own-org.api.mdx
+++ b/docs/docs/reference/api/get-own-org.api.mdx
@@ -5,59 +5,97 @@ description: "Get User Organization"
sidebar_label: "Get User Organization"
hide_title: true
hide_table_of_contents: true
-api: eJx9U8tu2zAQ/BVjzoTl9qhTfShSI0ATNM3JEAKa2lhMJZIhl05dQf9erOQ30OogCdoZ7s7MqocPFDVb71Y1SmyJX/yHe/FxC4VIKXiXKKHs8XmxkEdNyUQbhIEST9kYSuk1t7MfBzAUjHdMjgWuQ2itGRsUb0k4PZJpqNPyFqK0Zzt1sLXceR8IJRJH62QIttzKh1WNQcHpjv6H+i71YVBHgN+8keFRy3u2kWqUa2l0OKk6Mx/iVjv7Zxz1IXPIjEEudSP5jnj2nCjOLglQ6Igbf/AQCkFzgxKFv0Clwn8INFHcUUwo1z1ybFGiYQ6pLArT+lzP9ZYc67m2hQ4WQyUMk6Pl/UhZPq7uaf+NdE0R5bq6BDyJt5Od17CTZTrYe9ofDSixzNz4eNZhRWQzsUS9da9+pB98Wo7DzZaPKyiIjMmWxfzTfIFbs67AshbajGtx7D2WoW5cOOmHAnXaSo1Jd1/OBZks+MSddhfn/Subq6FOTjD95iK02jrZrHGE/pDbGle5QUGSqxQan1jKfb/RiZ5jOwzy+T1TlHAqhZ2OVm/EqnU1qKOTktsv2otmYyjIhux0m6dAbn4RyfO0TXdff2IY/gLwUT55
+api: eJx9U01v2zAM/SsBz0Kc7ajTsmHoggJrsa6nwCgYmY3V2pIqUekyw/99oO18Ap0PtiE+ko/vUR34QBHZereqQMOW+Mm/uycft6AgUgreJUqgO/i8WMinomSiDZIBGh6yMZTSc25mvyYwKDDeMTkWOIbQWDM0KF6S5HSQTE0tyl+I0p7t2MFW8uZ9INCQOFonJNhyIwerCnoFDlv6H+qnxPteHQB+80KGh1neso1UgV5Lo6lSecq8i1t09u9A9S5zyAy9POpq5Bvi2WOiODtPAAUtce0nDUFBQK5BQ+HPUKnw7wJNFHcUE+h1Bzk2oKFmDkkXhWl8rua4Jcc4R1tgsHBNYDlEZ98ECn0p5UyOlvdDveX96pb2PwgriqDX5TngQYQftb6EHfXEYG9pf1BHwzJz7eNpSCsE6jFLpLHu2Q/pk4gTt69oXsmJyjLnSHsx/zRffDTMlDBb3q+m/UEz7M+BxwADdSXXUShQQC1aiTFh++UUEJbBJ27RndX7yMQLckdVmP5wERq0TlZwoNBNBq/hwmBQIBaXCmqfWMJdt8FEj7Hpezl+yxTFqFLBDqPFjci2Lnt1UFU8fKW9zGwMBVmlHTZ5NOfqLom3x7W7+f4b+v4fYDlNCA==
sidebar_class_name: "get api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
-
-
-
-
-
+>
+
Get User Organization
-
-
-
-
-
-
-
-
-
-
-
-
-
-
\ No newline at end of file
+
+
+
+
+
Successful Response
+
+
+
+
+
+
+
+ Schema
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/docs/docs/reference/api/get-projects.api.mdx b/docs/docs/reference/api/get-projects.api.mdx
index 65bc6d3b68..aaf6ac0850 100644
--- a/docs/docs/reference/api/get-projects.api.mdx
+++ b/docs/docs/reference/api/get-projects.api.mdx
@@ -5,59 +5,304 @@ description: "Get Projects"
sidebar_label: "Get Projects"
hide_title: true
hide_table_of_contents: true
-api: eJytVE1v2zAM/SsGz0KS7ejTAmzoggJr0a7YIQgCRmZitbak6qNbZvi/D5Q/knhFMWzLxYFEPr73SKqBgAcP+RrupbHkYSPAWHIYlNGrAnI4UNhaZx5JBg8CHHlrtCcPeQPvFwv+FOSlU5ZTIIf7KCV5v49VdtcHgwBpdCAdOBytrZRMFeaPnnMa8LKkGvmfClQncOuYSFBdKeMOqNXPlLVVRcLRx5s95OsGwtES5OCDU/oAAvbG1RgghxhVAa0YI3SsKmg3AoIKFR/cnMFmK469rKSxprdq/TH2FwZqBXw37slblPQfRXwbMHsFpxr/Qv+EOnDvx6Bn/jbfE85tl9VzGzAGZlOUadZQO3pyW2eqv5bz4MlldwzQClB+W1BtXsXaGVMR6jfBVj77yPltK4YYs2O+aUOeo3JU8FKdOTaRvvlNqh+35QSKzuHxzJUhJLuikA150PJPTNbwIkJATaE0/T4zFwwl5DA/22xP7oWcT1ZEV0EOZQjW5/O5rEwsZnggHXCGao5WJTc8yehUOKaU5e3qmo6fCQtykK835wH3vN3dHl+GjQ6jVdfESru5gGUMpXH98oAAxZLKLou1Kr1P3RuMWSZy2fJ2BQJYRmfCYvZutoCpNRfB/DChTA/TUDtdg5i4MOoHAVSj4rtAWH84XTAza3yoUZ/hTTpxwWU0INCPMLcVKp2mnSs3fZfGMUqvc2l84LOm2aGnB1e1LR8/R3LciI2AF3QKd2zLetOKwTXu0RMdWZ+UZHkIXrCKnfmTB5l7Nw7M1aev0La/AIODGsY=
+api: eJytVE2P2jAQ/SvRnC2gPeZU+qEtWqmLlq56QAgNzkC8JLbXdralUf57Nc4HkLarqi2XIPvNmzdvZlxDwIOHdA0raSx52AgwlhwGZfQigxQOFLbWmUeSwYMAR94a7clDWsPr2Yw/GXnplOUQSGFVSUne76siue/AIEAaHUgHhqO1hZIxw/TRc0wNXuZUIv9TgcpIbh0LCapNZdwBtfoeo7Yqizz6dLeHdF1DOFmCFHxwSh9AwN64EgOkUFUqg0YMCF0VBTQbAUGFgg/uLmiTBWOvM2ks6aVcf8z9iYkaAV+NO3qLkv5jEV96zq6Cc45/kX9m7bV3Y9Apf1nvmWfZRnXaeo5e2ZhlHNXnrjy5rTPFX5fz4Mkl90zQCFB+m1Fpfsm1M6Yg1C+SLXzynuObRvQYs2O9cUOeKuUo46W6cGxU+uanUv2wLWdSdA5PF670kOSGQtLHQcM/MVrDK4SAkkJuun1mLRhySGF6sdme3DM5H62oXAEp5CFYn06nsjBVNsED6YATVFO0Csbp5vE2ecfQaJUnWTkVTpFvvlzc0ukjYUYO0vXmErDi1W+X/Bo22I9W3RLb0A4NzKuQG9dtFghQLCBvo9gIpfextb1rnba3KI+kuRNcZyt7Nnk1mf2umC4gmS8X3QuGMr5gvY4IAzGyazAKBFCJiu8CYfnmfMEqrfGhRH3BN2rZlabBjEDfwtQWqHRcC85cd+0c5i0+47nxgc/qeoeeHlzRNHz8VJHjpmwEPKNTuGOL1ptG9A5yv4504vqkJMvT8oxF1TZi9HJzH4fJuvnwGZrmB0NmKVU=
sidebar_class_name: "get api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
-
-
-
-
-
+>
+
Get Projects
-
-
-
-
-
-
-
-
-
-
-
-
-
-
\ No newline at end of file
+
+
+
+
+
Successful Response
+
+
+
+
+
+
+
+ Schema
+
+
+
+
+
+ Array [
+
+
+
+
+
+ organization_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ organization_name
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ workspace_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ workspace_name
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+
+
+ user_role
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ is_demo
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+ ]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/docs/docs/reference/api/get-single-testset.api.mdx b/docs/docs/reference/api/get-single-testset.api.mdx
index 0f30ce6838..f4f7ac966b 100644
--- a/docs/docs/reference/api/get-single-testset.api.mdx
+++ b/docs/docs/reference/api/get-single-testset.api.mdx
@@ -5,73 +5,295 @@ description: "Fetch a specific testset in Postgres."
sidebar_label: "Get Single Testset"
hide_title: true
hide_table_of_contents: true
-api: eJzVVMFu2zAM/RWBpw4Qkq7YyacFWLcV27CizXZJg0K1aVubLbkiXTQz/O8DZTtO1rXAjjvFkR6fyMdHdsCmIEg2sEZiQibYasiQ0mAbtt5BAu+R01IZRQ2mNrep4gGqrFOXnrgISIsbd+NWoaDkximlJsitzdQJcXiVqHWJymbK54pL3FOwV7nQx/gr5Da4iUICAt63SIzZ/Gauct+6TCusCJVx6uN6fXn+mGJMdwEafIPByJ+LDBIokG/JuqLC25EDNDQmmBoZg5TegTM1QgJz0qDBSumN4RI0SBo2YAYJhxY1UFpibSDpgHeNRBIH6wrQwJYrORjVVBcZ9P1WGKjxjpAk6Oz0VH6OVb5u0xSJ8rZSVyMYNKTeMToWuGmayqaxsOUPkphuTqTv+17Dm7Ozp8TfTWWzGKbOQ/DhH1ihCSIm2yHvDNnYSr4sY01PAZVPj26N233No8LHOvV6f2IdY4EB+m2vpzMTgtkdiPnZDwlCr6Gm4iXdvyCRKRD2ZM9DoxhqLbf9/La/+4EpH/V8E+sanh5x25lmlndQ9/ky3g3y/e2xCSJefkI49LZGLv1o6OhgLiGB5ehZWnaze3vQQBgeJnu3oYIESuaGkuUyrXybLUyBjs3C2KVpLIhFCdM2WN7FkNXlxSfcfUSTYYBksz0EXIs9hoYfw/Zam8Z+Qil9HKxVy6UP9tfQxXG2yiFKarMu9zF8lGEVk1OrywvQIGUMTj5dvF6cwp/r6QgszjZpdPb0drwG/YcK+/pBA9bR1sBo6rfzhWTWeOLauAO+D8jqOu4Ttd7vk6OMunm+/rvdOTaQ8ZGXTWVsnLmoXDdabjOtSQINycHK3GooPbEguu7OEH4LVd/L8X2LQWy11fBggjV30uRNB5kl+c4gyU1F+IKMJ1fjNL5SzyU5Wc+J7x5M1co/0PATd8erPW6acnJ2NwJWqahwEPpkMcoI7Ifww/ka+v43SC97gA==
+api: eJzVVcFu3DYQ/RViTglA7LpGTzplm7qJkRY17G0u64VBUyOJiUQqnJGRjaB/D4aSVrtxHaDHniSRbx5nHt+MemBTEmQ72CIxIRPsNeRINrqWXfCQwR/ItlJGUYvWFc4qHqHKeXUTiMuItLr3934TS8ruvVJqhjy4XL0ijq8zta1QuVyFQnGFRwoOqhD6FH+L3EU/U0hAxC8dEmO+nFmoInQ+1wprQmW8er/d3lx9tZjSXYGG0GI08nGdQwYl8gM5X9b4MHGAhtZE0yBjlNJ78KZByGBJGjQ4Kb01XIEGScNFzCHj2KEGshU2BrIe+NBKJHF0vgQN7LiWhUlNdZ3DMOyFgdrgCUmCLi8u5HGu8l1nLRIVXa1uJzBosMEzeha4adva2VTY+hNJTL8kMgzDoOHXy8vnxB9N7fIUpq5iDPE/sEIbRUx2Y945snG1vDnGhp4D6mDPdo0//F0khc91GvRxxXnGEiMM+0HPayZGczgR888wJgiDhobKn+n+FxKZEuFI9jI0iaG2sjssZ4fHT2j57M53qa7x6Am3X2gWeUd1Xy7j91G+fztshoiXnxGOd9sgV2EydHIwV5DBevIsrfvFvQNoIIxPs727WEMGFXNL2Xpt69DlK1OiZ7Mybm1aBz+2/CbtqrcCBfEvoe2i40Pi29xcf8DDezQ5Rsh2+1PAnXhndMM57HgRpnUfUHSZum7TcRWi+zZe8dR41RglhTtfhBQ+aTTl9puxn9FLp0qdY9oXq19WFy8VMwWozc311ALGphaY80gw0D/IdRQKNGCT/A+MpnmzbEiWbSBujD/he4es7tLgUdvj4DnLrF8a8X83ZKfLZPzK67Y2LjVnUq6fvLmb5ymBhuxktu41VIFYEH3/aAj/ifUwyPKXDqNYbK/hyURnHuXCdz3kjuQ9h6wwNeFPZHx1O7Xta/VSkrMNvXjwydSdfIGGz3g4/wekkVTNLu8nwMaKCiehzyaotMOxW99dbWEYvgMSJooP
sidebar_class_name: "get api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
-
-
-
-
-
+>
+
Fetch a specific testset in Postgres.
Args:
- testset_id (str): The id of the testset to fetch.
+testset_id (str): The id of the testset to fetch.
Returns:
- The requested testset if found, else an HTTPException.
+The requested testset if found, else an HTTPException.
-
+>
-
-
-
-
-
-
-
-
-
-
-
-
-
-
\ No newline at end of file
+
+
diff --git a/docs/docs/reference/api/get-testsets.api.mdx b/docs/docs/reference/api/get-testsets.api.mdx
index 6790f5ef5a..9121891458 100644
--- a/docs/docs/reference/api/get-testsets.api.mdx
+++ b/docs/docs/reference/api/get-testsets.api.mdx
@@ -5,65 +5,144 @@ description: "Get all testsets."
sidebar_label: "Get Testsets"
hide_title: true
hide_table_of_contents: true
-api: eJzNVE1PGzEQ/SvWnN0kVJz21KhCECG1CMIpRDDsTrKmG9vYY0q62v9ezX4kWQ6ox+7FK897no/37BoYtxGyFSwpciSOsNZQUMyD8WychQwuiRVWleIeMXmwD/aWOAUbswf7Rc1VZSIrtxkgyj2/UD4g0UTqgE9Xy+XNxXtO7dFP6rfhUkVGTlHlriB1PjtXZqOsOyRTGEhtXLLFBDQ4TwGFuygggy3x44ADDYGidzZShKyGr7OZLONO7lKeU4ybVKnbHgwacmeZLAscva9M3maYvkTh1BDzknYof4Zp1x7ugxTCpkv1aApZeO8JMogcjN2CBjZcycaigEaDxR19hvoh8UZDHgiZikfkz9DfO5Sas3CSL/6Bc9+hWk6jB1inVDu912QCFeIF6agveVTRKNX6eLR45474Z2Kf+DDZYxIMAfcnpQwQJdY6GK+RT8OOuHS9vKDBI5eQwfRE6EjhjYK4toYUKsigZPYxm07zyqVigluyjBM0U/QGmrUw8hQM71vK/GZxTfsrwoICZKv1KeBOxO5kHcMOc0Vvrmk/jCeDeeLSBfOnNQ1oMGK0smNJO8ZuXEvve5+3xan5zQI0SBudNWeTs8kMPl69EVh8inmr8ZC7DYssoykc+gcNtEMjMSbcfTsGpDLvIu/Qnpw3kuNDLfXxnvxPL0KvCtM7T32FxrYXQsZR99ZZAZ88baWLLHt1/YyR7kPVNLL9miiIO9Ya3jAYfBatVutGD1KKcX7RXoaeS7UiHlapc8SHR0MMdXDx5cUSmuYvFW3aMA==
+api: eJzNVE1PGzEQ/SvWnLdJqDjtqSlCECG1CMIpRDB4J1nDxjb2mJKu9r9Xsx/5QCrqsXvxyn4z82bes2tgXEfIFzCnyJE4wjKDgqIOxrNxFnK4IFZYVYp7xOje3tsb4hRszO/tFzVVlYms3GqAKPf0THpAoonUAR8v5/Pr83dNbepH9ctwqSIjp6i0K0idTk6VWSnrdsUUBlIrl2wxggycp4ASOysghzXxw4CDDAJF72ykCHkNXycTWY47uU1aU4yrVKmbHgwZaGeZLAscva+MbiuMn6PE1BB1SRuUP8O0aZP7IETYdKUeTCELbz1BDpGDsWvIgA1XsjEroMnA4oY+Q/2Q8yYDHQiZigfkz9BnHUpNWWKSL/4h5q5DtTFNNsA6pdrpvSYTqBAvSEc95SNGR6WW+9TinVvin4l94t1k90UwBNweUBkgSqy1M14jXwYb4tL18kIGHrmEHMYHQkcKbxTEtTWkUEEOJbOP+XisK5eKEa7JMo7QjNEb+GjnaXuqzgQKzVLS6RQMb9t80+vZFW0vCQsKkC+Wh4BbcUKn+TFsN3T05oq2w+xymCYuXTC/W0dBBkYIlF2U9GrsyrXh/WB6bt9Rv5AVDaTPjvZkdDKa/K2ZPkBNr2e9oVG3Zhh4tDDR72hcu0FBBrRBI2dMuPm2PxCW3kXeoD3Id6TbB071/kL9T09HrxDTO499hca2N0fGUfceWwAfvIGliyx7df2Eke5C1TSy/ZooiFOWGbxhMPgkui2WTTbIKiZ6oa0MXQtbERGr1Lnjw+si5trZ/eJ8Dk3zB3r16L8=
sidebar_class_name: "get api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
-
-
-
-
-
+>
+
Get all testsets.
Returns:
+
- A list of testset objects.
Raises:
-- `HTTPException` with status code 404 if no testsets are found.
-
-
-
-
-
-
-
-
-
-
-
-
+- `HTTPException` with status code 404 if no testsets are found.
-
\ No newline at end of file
+
+
+
+
+
Successful Response
+
+
+
+
+
+
+
+ Schema
+
+
+
+
+
+ Array [
+
+
+
+
+
+
+
+
+ ]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/docs/docs/reference/api/get-variant-by-env.api.mdx b/docs/docs/reference/api/get-variant-by-env.api.mdx
index 9ff0b6e622..54ba4f898b 100644
--- a/docs/docs/reference/api/get-variant-by-env.api.mdx
+++ b/docs/docs/reference/api/get-variant-by-env.api.mdx
@@ -5,77 +5,575 @@ description: "Retrieve the app variant based on the provided app_id and environm
sidebar_label: "Get Variant By Env"
hide_title: true
hide_table_of_contents: true
-api: eJztVk1vGzcQ/SvEnGJgIalGelmgQOXESIQ0jeHY7sEWBHp3pGWyIhmSq2Yj7H8vhlxKXH/IgY1eip4sL9+8NzOcD27B8ZWF/BqmWluYZ1CiLYzQTigJOZyjMwI3yFyFjGvNNtwILh275RZLpqQ/0EZtRIklIRaiZFyWDOVGGCXXKN3oRt7IqVnZ/EYyxiLqlXXmKGcXFbLZW6aWOw2nmEl1o+ZSmVFgSMhTmvRzwhftE17v0jkXFqNT7y8uzk6/F+gjz9nsvrmwTCryopHeee4auyhUib/9OpkcZUwZJpaMsyteN3hqDP1vmSGRO/jXKV62TLkKDcOo/pgZyQTH0TVGRs+nWl8FF8/RaiUthmzEWMs0ihFkoDQaTkKzEnJYoVv0h4vbdoFyAxlobvgaHRqqjS1IvkbIIVwcZCCoNr41aFrIwOC3RhgsIXemwQxsUeGaQ74F12oys84IuYIMnHA1fZhqzWYldF22o06u7uX8pwlZ182JIiTGktXxZEJ/hoX+uSkKtHbZ1CxmETIolHREkm8p9loUPm3jL5Zstokn2lBSnQgKfZ5+IgOZx4YcHEb/SZgug3hVh/n7gug1os1TOtEqammjvmDxlNZZQPVaNBieFDrhFncq3uKwhMcH/kLJpVg9qfDGw3YajRH+YmT7aekremhHhdh/kU1dA9VMZLo0ghgMboQV4d57qJAOV2gS1fMIIkcNcoflgrvnKr8JDGzqfAi6fCHfZWDo+daqFEuBJTV9yP5zOD/2LOyk7S9ImRWX4ofvlBcQf0poeua/lflqNS/wBbR/RY5+AEWguqUaHkyb6/282zXpoP/uNNagX9JG2Jf4sHyTopoPmv3OOF9A15Grr4+P7w+uK16LMmTJ75znT60SHRc1/RIO1/Y+oFbF4PQn8h9bpJvvU82N4W3SNH+o4KAvSrs61NUf0Vq+8h0dIAd2gF/AF3T61DVTXEG6x83TkRjTG7L7eBhvQ/oeEosQemPcIwx3u0ZXqX4d+/3rKshhzLW24wc3tEWzieu5MTXkUDmnbT4eF7VqyhFfoXR8xMWYa+FbwGLRGOFabzI9m33A9j3yEg3k1/MU8JkKJFz5ELbLNtfiA1Lw/faeNq5Spu/WuL+rYEXRCblU3jzWuHeOTc9m1ERowmCFyeiX0QTuvkEHYKptXvjajtr+GLI7WdjFDxng2hc2OOTr3/cH5JlW1q25TPjeoWNxEZ607NRne+DRdt9h/z+Q/4sP5L7KHX53Y11zIcMTws/G0Jl+N1jIHno9zzOolHUE2m6pDC5N3XX0Obxoqf1KYfltTeNnyWuLBwrs1Xk/qY7YY559xTZ9nG8ovZCDn8P/gtDwqb5Xm/f78JZa/JoGfhXHy7Y3nRZ0g4nVvf1ELLtZ+O70ArruHwciA0I=
+api: eJztVk1vGzcQ/SvEnGJgIalGelmgQGXHSIQ0jeHI7sEWBHp3JDFekQzJVbMR9r8Xww9p1x9yYKOXoifLy5n3hjPzZrgFx5cW8msYa21hlkGJtjBCO6Ek5HCBzgjcIHMrZFxrtuFGcOnYLbdYMiX9gTZqI0osyWIuSsZlyVBuhFFyjdINbuSNHJulzW8kYyxZvbHOHOVsukI2ecfUYsfhFDNd3sS5UGYQEDrgXZju5w5e8u/g+pAuuLCYgvownZ6ffS/Q3zxnk4fuwjKpKIpa+uC5q+28UCX+9utodJQxZZhYMM6ueFXjmTH0v2WGSO7Zv+3ay4Ypt0LDMLE/5UY0IXB0tZEp8rHWVyHEC7RaSYshG+muZfcWA8hAaTSciCYl5LBEN4+H89tmjnIDGWhu+BodGuqNLUi+RsghFA4yENQb32o0DWRg8FstDJaQO1NjBrZY4ZpDvgXXaHKzzgi5hAyccBV9GGvNJiW0bbaD7pTu9fhnHbC2nRFESIwlr+PRiP70G/1LXRRo7aKuWMoiZFAo6Qgk39LdK1H4tA2/WvLZdiLRhpLqRGCIefqJDGTeNuTgsPWfZNNmkEp1GD82RORIPs/xJK/EpY36isVzXOfBKnLRYHiW6IRb3LF4j8MU3j7gF0ouxPJZhlNvtuOojfCFkc3nhe/ovh81Yvwi66oC6pmEdGkEIRjcCCtC3aOpkA6XaDqsF8mIAjXIHZZz7l7KfBoQ2Nj5K+jylXiXASHirVUpFgJLEn3I/kswP0UUdtLEAimz5FL88Ep5BfDnDkxE/luZO6t5ga+A/SthxAGUDNUt9XBv2lzv591OpD393RNWTy9dIexbvN++naaa9cR+b5zPoW0p1LfHxw8H1xWvRBmy5HfOy6dWiY6Lin4Jh2v70KBSRe/0J/KfJNLO9qnmxvCmI5o/VAjQN6VdHlL1J7SWL72ig8mBHeAX8JROnysz3StQR7tZdySm9IbsPn2NdyF9j5ElE3pjPAAMtV2jW6m4jv3+dSvIYci1tsNHN7RFs0nruTYV5LByTtt8OCwqVZcDvkTp+ICLIdcC7r/rxv6UnZKp14fFojbCNR5vfD75iM0H5CUayK9nXYMv1D2hH/pmu1JwLT4iZSau9nHtVspEKaflvgpedHUhF8q7JwGE2E54cYfSqwxNmLwwGvwyGD11mejAxueTKAJeeBGkOLwZZPfStUsUZIBrrwBwyNe/7w8oSq2sW3PZwXuPjqWNedKwM1+WXmTbvRT/f0n/F1/SseMdfndDXXEhw1vDD9EgYb9ELGSPPbNnGayUdWS03VIbXJqqbelzePqSFEth+W1Fc2rBK4sHGuzNRRxpR+ypyO6w6b7iN5ReyMEP7H+BqP+m37PN4uK8Jblf02ZYpVGzja7jgirY8XqwyAhlNzTfn02hbf8BFTwR0Q==
sidebar_class_name: "get api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
+>
-
-
-
-
+>
Retrieve the app variant based on the provided app_id and environment.
Args:
- app_id (str): The ID of the app to retrieve the variant for.
- environment (str): The environment of the app variant to retrieve.
+app_id (str): The ID of the app to retrieve the variant for.
+environment (str): The environment of the app variant to retrieve.
Raises:
- HTTPException: If the app variant is not found (status_code=500), or if a ValueError is raised (status_code=400), or if any other exception is raised (status_code=500).
+HTTPException: If the app variant is not found (status_code=500), or if a ValueError is raised (status_code=400), or if any other exception is raised (status_code=500).
Returns:
- AppVariantResponse: The retrieved app variant.
+AppVariantResponse: The retrieved app variant.
-
+>
-
-
-
-
-
-
-
-
-
-
-
-
-
-
\ No newline at end of file
+
+
diff --git a/docs/docs/reference/api/get-workspace.api.mdx b/docs/docs/reference/api/get-workspace.api.mdx
index 5a9852e900..7fb010eb1c 100644
--- a/docs/docs/reference/api/get-workspace.api.mdx
+++ b/docs/docs/reference/api/get-workspace.api.mdx
@@ -5,67 +5,218 @@ description: "Get workspace details."
sidebar_label: "Get Workspace"
hide_title: true
hide_table_of_contents: true
-api: eJzllE1v2kAQhv/Kai69rID26FORGiUoUhslVD0QVA32gDexd52dcRKK/N+rWcAYesu1XIxn5p2Pnce7A8ENQ7aAXyE+c4M5wdJCQZxH14gLHjK4JjFvR7cpSNBVPHr0j/6epI2ejzaDq9CKkZIG8cgccodChXlzUiZvyxQ/sWFidsEPU2WP3hhj+m4yMy/7miasz5PvleiYjsKb+fzu6j2n1HtmZuu+nikCsfFBTImvZBqKtUvljQR9W4dYGykdG8xVPAILoaGI+jIrIIMNye++NFiIxE3wTAzZDr5MJvo4P7mHNs+Jed1W5v4QDBby4IW8aDg2TeXyVGL8xKrZAecl1aj/nFCdkjdROxG3L+WKJPXbH2vIFjuQbUOQAUt0fgOd7S2+rSrolhbESaWGWQGdBY81aYZz3Snqu/q7Cwo+VvDbIEVnD2EfSzVXe9cngbB6olzSGl5aF6lQitNoh5CB9gT3SY8x4nYw9XFBRnEfCPRnoSYpwwECsNCglJDBuMeBwQJTfKXIaa42VpBBKdJwNh7nVWiLEW7IC47QjbFxaTamvI1OtkkyvZvd0vaGsKAI2WI5DHhQJvbbPw/rzwsbd0s6z367MG2lDNH9SWyBBac8lnuVDuT8OiT5Yfxpas5M72ZgQcfYEzwZfR5N4PJGOAtWnDFPOB9rJzfYi1Po5wcLVKNTnxDWX08O7awJLDX6Qb7zjfwDZv89/cc31YECoXcZNxW69Lml498dYF3AANalhTKwqHW3WyHTz1h1nZpfWorK49LCK0aHK6VjsezsER5F9Zm2uuZcu1dcsGr3DF7cZopw/+VcX82h6/4CIHk7pg==
+api: eJzllE1v2zAMhv+KwMsuQpLt6NOyrWiDAlvRZtghDQpGZmK1tuRKdNss8H8fqDiOk2GXXpeLY5Ivv/TIO2DcRMgW8MuHp1ijIVhqyCmaYGu23kEGl8Tq9eBWOTHaMo7u3b27JW6CiwebwpVvWHFBg3iM0RuLTLl6tVwkbxMpfIgqUozWu2Gq7N4ppVTfTabmRV9T+fVp8r0SbaSD8Go+v7l4M5R6z9Rs3ddTuaeonGdV4AupmkJlU3nFXt7WPlSKCxsVGhGPQIOvKaC8zHLIYEP80JcGDYFi7V2kCNkOPk0m8jjd3F1jDMW4bkp12wWDBuMdk2MJx7ourUklxo9RNDuIpqAK5Z9lqlLyOkgnbPelbJ6kbvtjDdliB7ytCTKIHKzbQKt7i2vKEtqlBrZcimGWQ6vBYUWS4VR3jPou/vaMgvcV/DZI0eou7H2p5mJv+yTgV49kOB3Dc2MD5UJxGq0LGWiPcB/1GAJuB1MfDkgJ7gOB/DRUxIXvIAANNXIBGYx7HCJoiBReKMQ0VxNKyKBgrmM2HpvSN/kIN+QYR2jHWFs4v2XT5FVfJTQNHsk0wfI25ZvezK5pe0WYU4BssRwG3AkwezROw/plYm2vSYbdHz1MGy58sL8TeKDBSgPFXiXTWrf2Sd7tpuvtC5oncjlokDn3bU9GH0eTfw3TCdT0ZtZxjyZxf+gjhYE+W1e/KNBAFVrxMWH1+eiQLmsfuUI3yHd6dH8R3F+8//iT1hHB9MbjukSb7mVa/66jegEDqpcaCh9ZrLvdCiP9DGXbivm5oSBsLjW8YLC4ElIWy1YfQBJsn2grx2yke8EGy2bP49lnT3Dur9jlxRza9g/UIEo1
sidebar_class_name: "get api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
-
-
-
-
-
+>
+
Get workspace details.
Returns details about the workspace associated with the user's session.
Returns:
- Workspace: The details of the workspace.
+Workspace: The details of the workspace.
Raises:
- HTTPException: If the user does not have permission to perform this action.
-
-
-
-
-
-
-
-
-
-
-
-
-
+HTTPException: If the user does not have permission to perform this action.
-
\ No newline at end of file
+
+
+
+
+
Successful Response
+
+
+
+
+
+
+
+ Schema
+
+
+
+
+
+ Array [
+
+
+
+
+
+ id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+
+ description
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ type
+ object
+
+ required
+
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+ ]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/docs/docs/reference/api/handle-events.api.mdx b/docs/docs/reference/api/handle-events.api.mdx
index 7239e1f558..156ce9c472 100644
--- a/docs/docs/reference/api/handle-events.api.mdx
+++ b/docs/docs/reference/api/handle-events.api.mdx
@@ -5,59 +5,72 @@ description: "Handle Events"
sidebar_label: "Handle Events"
hide_title: true
hide_table_of_contents: true
-api: eJxdUk2PmzAQ/SvRO1sh7dGnplKljfbQaNOeIlRNzGzwrjGubaJSxH+vBpJ0AxcE82bmfcyATOcEfcRX65z1Z5QKbeBI2bZ+V0GjJl85/sUX9jlBIXIKrU+coAd83mzkVXEy0QbpgcahM4ZTeu3c6uUKhoJpfWafBU4hOGumFcVbkp4BydTcEPQwyqMWI58mEqtvNxIN57oVdqFNGQqBcg2N4jSrKFKONnAxky6gkDheOIrSAV10oivnkHRRGNd21ZrO7DOtyRYULMZSOkwXbe6nlu1+98z9E1PFEfpYfgQchPlsxyNsQO4DQ4OCfeYeCp4a+d52uW6j/Ts5AAUrEuu5S7Rb/9pO7Ta7CT+RW233OyiIjNmUzfrTeoOlVQ9gMZ3MZPpt91SGWrhw1w8FbshKLTM1X/4XhJnY3ZD/MG+ZzAOZuwOZ/+QiOLIe43X1cA3tiGtoktIUm1C4jZN7rCVjfcQwnCjxz+jGUX7/7jhKOqXChaKlk3h1LEd1s1KCe+deRBvDQe7kQq6bE1lcoAR6P6r998MPjOM/FmYOTA==
+api: eJx1Uk2PmzAQ/SvRO1sh7ZFTs1WljfbQqGlPCFUTMxvcNca1TVSK+O/VAMnuIpULgnkz8z5mQKJLRF7gwVhr3AWlQus5UDKtO1TIUZOrLP/kK7sUoRA4+tZFjsgHfNzt5FVx1MF46UGOU6c1x/jc2c23BQwF3brELgmcvLdGTyuyX1F6BkRdc0PIh1EetRr5OJHYfLmRaDjVrbDzbUxQ8JRq5MjOs4ospmA8ZzPpDAqRw5WDKB3QBSu6UvIxzzJt267a0oVdoi2ZjLzBev1+qm4+CxRjKeN0F0zqp3n74+GJ+0emigPyonwLOIms2av3sAGp94wc5M0T91Bw1Mj3vkt1G8zfyR4oGCFQz11ijHHP7dRukuVXbg+kX9hVUBCdM+3d9sN29z8xS8Nmfzws6ZCe0rnxmGBQK7vuRkGBGzJSS0zNp9eCsJRcGnJv5q0jfEfq7kbiPynzlozDuKwelnQLLOlKnFO+QuE2Tg63lmPICwzDmSL/CHYc5ffvjoMkVSpcKRg6i29FOaqbrRLiC/ciWmv2clBXst2czupUJdz79R2/nr5jHP8BBLsc2w==
sidebar_class_name: "post api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
+>
-
-
-
-
+>
Handle Events
-
-
-
-
-
-
-
-
-
-
-
-
-
-
\ No newline at end of file
+
+
+
+
+
Successful Response
+
+
+
+
+
+
+
+ Schema
+
+
+
any
+
+
+
+
+
+
+
+
+
+
diff --git a/docs/docs/reference/api/health-check.api.mdx b/docs/docs/reference/api/health-check.api.mdx
index 34aa4dc3e8..239ad076d7 100644
--- a/docs/docs/reference/api/health-check.api.mdx
+++ b/docs/docs/reference/api/health-check.api.mdx
@@ -5,59 +5,69 @@ description: "Health Check"
sidebar_label: "Health Check"
hide_title: true
hide_table_of_contents: true
-api: eJxdUkFu2zAQ/IoxZ8Jye9SpRhEkRi5B05wModjQm4iNRLLkyogq6O/FSoobSxdC3BnuzOwOCJETiQv+cEKJmqmR+pet2b7BIHGOwWfOKAd83e30OHG2yUWloMRjZy3n/NI1mx8LGAY2eGEvCqcYG2enDsXvrJwB2dbcEsph1M+snrybNGy+LxpaljqotlcWGESSGiWKWSkMMqczp4zyOKBLjZoQibksCtuE7rSlV/ZCW3IFRYexUobtkpN+ouwfDvfc3zGdOKE8Vp8Bj6pzNn8NGyB9ZJSg6O65h4GnVv/3ndQhub+TXxg4P4eqLHXq/EuY6E6aCT+J2+wfDjBQG3MEu+2X7Q7rYK7AGjHZKeKP3lMZZpXCxT8MuCWnNWFqv/0vqLIYsrTkP723msOVlksAwu9SxIacx7h0HpYZHZdtQmVQhyx6MwzPlPkpNeOo1386TjqGyuBMydGzhnKsRvORmU7ojXt1Zy1HXYAzNd0c/WqxdHKXZbm9+Ylx/AcGdfiJ
+api: eJx1UsFu2zAM/ZXgnYU429GnZcWwBr0U63YKjIGV2VqLLWkSHcwz/O8DHTdtDUwXQeIj+fgeR4TIicQFf6hRomFqpflpG7YnGCTOMfjMGeWIj7udXjVnm1zUFJR46K3lnJ/6dvNtAcPABi/sReEUY+vs3KH4lTVnRLYNd4RynPSYVcnbmcPmZuHQsTRBuT2zwCCSNChRXJjCIHM6c8oojyP61OoQIjGXRWHb0NdbemYvtCVXUHRYN9vP0c2NQjFVWs72yckw19vfH+54uGWqOaE8Vm8BDzrERZn3sBEyREYJiu6OBxh46vS976UJyf2dxYCB8xfFNUtlcP4pzOlOWn7l9pnsiX0NA53zQnu3/bDd/W+YJWGzvz8sXpCdvXjhMcNgVnJdhYIBd+Q0Jkzdp9eAsowhS0f+Tb2VYe84XcUQ/iNFbMl5TEvncTHzuKwdKoMmZNGfcXykzD9SO036/bvnpJZUBmdKjh5VoGM1mRf91K0TDzqdtRx1U87U9hcbVhuoLl636uuX75imf188Byc=
sidebar_class_name: "get api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
-
-
-
-
-
+>
+
Health Check
-
-
-
-
-
-
-
-
-
-
-
-
-
-
\ No newline at end of file
+
+
+
+
+
Successful Response
+
+
+
+
+
+
+
+ Schema
+
+
+
any
+
+
+
+
+
+
+
+
+
+
diff --git a/docs/docs/reference/api/human-evaluations.tag.mdx b/docs/docs/reference/api/human-evaluations.tag.mdx
index 3e4ee7417a..592e00ba89 100644
--- a/docs/docs/reference/api/human-evaluations.tag.mdx
+++ b/docs/docs/reference/api/human-evaluations.tag.mdx
@@ -5,16 +5,9 @@ description: "Human-Evaluations"
custom_edit_url: null
---
-
-
-
-
-
-
```mdx-code-block
import DocCardList from '@theme/DocCardList';
import {useCurrentSidebarCategory} from '@docusaurus/theme-common';
```
-
\ No newline at end of file
diff --git a/docs/docs/reference/api/import-testset.api.mdx b/docs/docs/reference/api/import-testset.api.mdx
index c1fb8108bd..a2ea57d6aa 100644
--- a/docs/docs/reference/api/import-testset.api.mdx
+++ b/docs/docs/reference/api/import-testset.api.mdx
@@ -5,74 +5,373 @@ description: "Import JSON testset data from an endpoint and save it to Postgres.
sidebar_label: "Import Testset"
hide_title: true
hide_table_of_contents: true
-api: eJzlVU1vGzcQ/SvEnBqAllyjpz1VTQNETRoLltKLLBj0ciQx3SUZctaOKux/L4bclVZ2LAe99rQfnDcfbx5n9kBqE6FYwgIjRaQIKwkaYxmMJ+MsFDCtvQsk/phffxKUrYRWpMQ6uFooK9Bq74wloawWUT2gMCTIiZmLtAkYR7f21k7CJha3VghxtP8pUnhTiMnAxeebjww1OeYhzCgju/B3VtXYo2mLIn27dXrvUzRr4YN7MBp1SuAGqQm2z0Gbkgqx2KIIGJuKenQX2AdXYowjkOA8BsVUTDUUkM/vuiAgwaugaiQMzOIeOBMoQDW0dcH8k4AgwTCRXxsMO5AQ8GtjAmoo1qqKKCGWW6wVFHtQdne9To5o59lRpGDsBlp5+GObqoJ2JYEMVfxjchKr5SOOgJF+c3rHXktnCS2lAN5Xpkym428Xj4+PF2sX6osmVGhLpzmp/SAfH7h8Mhj5q+8Sv5/md8zmXW/TShi26xym0574xHYtA7Ohu/+CJQ0MuaC7Jy1oW0YEjN7ZmPO8urzkx6mM503JLV03lbjpjEG+RM2XyJiXmTD6XD1TzdW/VnWuVkIZUBHqO3WW17fZSkzouwwdNbXk5LroJ85Xp4TPkeam9hUe2MhM/nJ19Zy8v1RldKJGvAvBhf/OnEZSpkocEtbxuUHlypPTH7gSxhJuMEC7OjKjQlC7AX8fXdldEAl13Jyj+k+MUW1Sb7LJGbUzGWLBp691hevKoTu7QT+O9GZ2Xy7j90zfuUvyfrGYPXOYe1sjbR2PMe9iHl60hQLG3V2K48MNlxAxPPRTrQkVFLAl8rEYj8vKNXqkNmhJjZQZK2/SQIpYNsHQLkEms+kH3L1HpTFAsVwNDeasjdztU7MD0cqbD7jrZfx0xnXzdJtRXJixa5fg/UxMyYnJbAoSuIws48vRz6NLeLrgToxZ1qpMsu5jp2OQT1g41A8SsE6aBkJV/3o84MyY6VrZgb9uny4OK+Qkm8G0/r9v3k4LhN9o7Ctl0t1NTdh3yl32KyZyE3rtriRsWd/FEvb7exXxc6jaln/nDczy1Caq+2qwg//G3Xf29oOqGs4h6ftBBcOgJGfZy4+9ZfDb3LiLNA6O2NdXLo+y7GJSlujpBXAarZzI4RrPrucLkHDf7fnaacYMPMvhB6fdtv8CFsduaw==
+api: eJzlVlFv2zgM/isCn3aAmnTFPfnpst6A5bZbgya7lzQoVItJtNmSJ9HpssD//UDJTpx2SYe97im2xY/8+JEiswNSqwDZHGYYKCAFWEjQGHJvKjLOQgbjsnKexD/Tm4+CkpXQipRYelcKZQVaXTljSSirRVAbFIYEOTFxgVYew+DO3tmRX4XszgohDvavAvk/MjHqufh0+4GhJsXchxkkZBv+3qoSOzStUcR3t4zPHUWzFJV3G6NRRwK3SLW3HQdtcsrEbI3CY6gL6tBt4Mq7HEMYgARXoVcsxVhDBun8vg0CEirlVYmEnlXcATOBDFRNa+fN9wgECYaF/Fqj34IEj19r41FDtlRFQAkhX2OpINuBstubZXRE24odBfLGrqCR+y+2LgpoFhLIUMEfRkexGj7iCBjojdNb9po7S2gpBqiqwuTRdPjt4vHx8WLpfHlR+wJt7jST2vX4VJ7TJ4OB37oq8fMxvwObt51NI6FfrnOYtvfER7ZrGJgM3cNnzKlnyAndPylB0zDCY6icDYnn1eUl/xy38bTOuaTLuhC3rTHIU9J8Dow5rYTR5/IZa87+paxTthJyj4pQ36uzul4nKzGiHyp06Kk5k2ujHzlfHAs+RZqasipwr0ZS8s+rq+fi/acKo6M04q33zv+6chpJmSJqSFiG5waFy49Of+JKGEu4Qg/N4qCM8l5te/p9cHl7QSSUYXVO6n8xBLWKtUkmZ7qdxRAzPn2pKpxXCt3a9epxkDepezqNv5N85y7Ju9ls8sxhqm2JtHY8xioX0vCiNWQwbO9SGO5vuISAftNNtdoXkMGaqArZcJgXrtYDtUJLaqDMUFUGni6NUTwV12wap1XAvPaGttHfaDJ+j9t3qDR6yOaLvsGUGye1wrHZvgqqMu9x2/X40wHYDtt1QnHWxi5dhHcDM3F7o/IvaPmucJ6J9uXg9eDyVDItQIwm47b/VR77v+MRzUA+kWsvFEjAMjY/EKryr8MBs+SSlMr2/LWLd7bfNUesemP9d1/RbV8QfqNhVSgTL3kswq5t8Xm3iwIXoWvyhYQ1X4RsDrvdgwr4yRdNw5/TquZW1Saoh6K3rL/g9gcLfqOKmjnEXt8obxgUW1t2rcjeEvg6Fe4izo0D9uXdzDMvuRjlOVZ0AhxnMBPZ3/fJzXQGEh7aPwSl04zpeZb9F6bdNP8Dnod8+g==
sidebar_class_name: "post api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
-
-
-
-
-
+>
+
Import JSON testset data from an endpoint and save it to Postgres.
Args:
- endpoint (str): An endpoint URL to import data from.
- testset_name (str): the name of the testset if provided.
+endpoint (str): An endpoint URL to import data from.
+testset_name (str): the name of the testset if provided.
Returns:
- dict: The result of the import process.
+dict: The result of the import process.
-
+>
-
-
-
-
-
-
-
-
-
-
-
-
-
-
\ No newline at end of file
+
+
diff --git a/docs/docs/reference/api/inspect-workflow.api.mdx b/docs/docs/reference/api/inspect-workflow.api.mdx
deleted file mode 100644
index eaf5f4c8e4..0000000000
--- a/docs/docs/reference/api/inspect-workflow.api.mdx
+++ /dev/null
@@ -1,71 +0,0 @@
----
-id: inspect-workflow
-title: "Inspect Workflow"
-description: "Inspect Workflow"
-sidebar_label: "Inspect Workflow"
-hide_title: true
-hide_table_of_contents: true
-api: eJztV01v2zgQ/SvGnLYAG6dBTzqtd7eLuu0iRpy2B8NY0NJYZkuRLD+ceg3992JI2ZJix00M9LTxxSI5MyTfm3kkt+B56SCbwWdtvy6lvnMwZ6ANWu6FVuMCMhDKGcz9v3eNCTCw+C2g83/oYgPZFnKtPCpPn9wYKfLoPPzitKI+l6+w4vRlLIX2Ah211midSCZcba6XkM224DcGIQPnrVAl1Gzfo4KUUM8ZeOEldXxq3GsGwYpzo3y0IkWQ50eQFGGFvEDr+lF4UQjCgstJb+utxa+CxMlQnhtmSr41A1GcisBgqW3FPWQQgihORhwXUNdsN6wXXzD30A7f4BItqhy7QXYLnR84nprpbUMDQRDz7rGM/JSpB/ETymOJtr+satHvWWgtkavYdXwuyIXNg+T2t7+DlO+cVi/HygT/Ao7tX3isHufEreWbk5j1XeEI3k8gYNqgXjOouDFClY9k4D6+T5jzn91Ep5JsJ3FTtGuR41h5tEtOOUdeJGnCYgGZtwFjhzNaubS2q8tL+ivQ5VYYH0sUpiHP0bllkIObxhjYsxg+i+GzGP4KMbwO/gw1PPB6ghwm3/+jHtYMXl9dHUreJy5FEQVt8MZabc/XuwI9F1FmGu7uG0id90afknudKkl0t7v+oNMCIx+uPAS6gyI6x0tsoX/YNIIxuKXRY5C3h8ss7itN3dh1JWsPb0L34W38leA7xe/b29vJQcDEbZ/UcbrgDz63F/wK/UrT5d9oRzEN9yvIYGgsrgXeDXdvATdsXgfAwKFdR82fbdMJAivvjcuGw1zqUFzwEpXnF1wMuRExXR3mwQq/iS6jyfg9bpJiQTabdw1iDaW06JvtGeFGvEcCSPGK2qPgV9qK/xLXDARtNJ1KETRKz5v2FfPmO6+MxN6p07IcT9RuU3ab7VnXk9l+vdOcQi11XHJD0CgCMhhNxsA6015evLq4hPsk9Yyp5ngea2633zi8X9wO+T3mwACrWHDgkVe/twO0MmK54qoT70hO9NazB97jdz80kgvVuTmkdJlBky7AYJ8wkYuUMnMGK0qvbAbb7YI7/GhlXVP3t4CWsmLOYM2t4AvCazbfXyxiwnzFDWTwZ5Kfl7H2yFyGlA/3pIhkInmM8hyNP2k771TA5Hp6CwwWzVu30gX5WB7fwfwOMgB6MZN3TNDYtwXJVRlIPTJIMen3A+hBT+M=
-sidebar_class_name: "post api-method"
-info_path: reference/api/agenta-api
-custom_edit_url: null
----
-
-import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
-import OperationTabs from "@theme/OperationTabs";
-import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
-
-
-
-
-
-
-
-
-
-Inspect Workflow
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
\ No newline at end of file
diff --git a/docs/docs/reference/api/invite-user-to-workspace.api.mdx b/docs/docs/reference/api/invite-user-to-workspace.api.mdx
index 0acb514d02..50beb5d89d 100644
--- a/docs/docs/reference/api/invite-user-to-workspace.api.mdx
+++ b/docs/docs/reference/api/invite-user-to-workspace.api.mdx
@@ -5,79 +5,407 @@ description: "Assigns a role to a user in an organization."
sidebar_label: "Invite User To Organization"
hide_title: true
hide_table_of_contents: true
-api: eJztVktv4zYQ/ivEnLKAaqdBTzrV7aZYY1skcLztwTGCiTS2uCuRCkklcQX992JIPe0k3UWBnjYXR5z3NzMfWYPDvYV4A1dmj0r+jU5qBdsIUrKJkaX/jGFhrdwrK1AYnZNwWqCoLBkhlUAl9Mh4dqtu1cLsbXyrhBAT2Z1MxZl15l0s1hmJ5Xuhd8JldOyA7Uo85BpTcbZUj9LRih4qsq617ISJVg6lkmp/4kbINAo5UoEyjwSqdMje19NGetLmiy0xoVfT6zV8cStylVFdffda57FYm4qEDNo+yBNaYaskIWt3VZ4f2pCURuI3zC0J7TIyT9K2PlFa6lx+WK+vL58T8ujHYhnc+lpSTVYo7USGjyRKMoW0lqt1mr922hTCZdIKTEZYvujQkGA9JcgYbdr8OiQ7oLrAM4hAl2Q8tMsUYpC+LXcsvHP6rocIIijRYEGODE9WDQoLghiOBgEikDxaJboMIjD0UElDKcTOVBSBTTIqEOIa3KFkc+uMVHuIwEmX88F4YsUyhaaJ+ljjlv7nQH91zkKUbfBB1v2i0wMbHrvkqSTlWIRlmcvE5zj/bHmX6tOIaAweOE1HheXz0jDWTpL/8vP7VoKXXqGJgLvmTVAdrnYe/N7n1LiJpsEZu/ZAVXkOXGXnfuW9NoOJvv9MiZtguWmzHJlN9tYHbAXXYXmhadinIVtqZUPeF+fn/DMln5t+jcSqVYavBzlE+eni4tTxn5jLNMzPJe/AN3g9alFKru3RKz3MdTKRDg067kt/IpWjPRlotsfNGqD8XYcEGd7C7t8akj/IWtzT0Pk35skTwpql/9Z1riuEbvVGAzDAG9B9vYz3Ab6XgnUqTGEnDkNvC3KZZkoqtXWeflwGMczHhGPn9RH/NPOeJOy8HhNGMw/cBhFYMo8djVUmhxgy50obz+dJrqt0hntSDmco51hKvzSWkspId/Ami+vlRzp8IEzJQLzZjhVueJTCcEzVBloo5UdimFpSW1Qu06a7o1tWy4IV48BDuhqY6fIZi5Kx2/QUMrS6ZYpNd+JTk2qnffwW84WvTiyulxAB4xDW5nz24+wcTp4IY2V/Lyd+jbrkvRiiIxh7ACHqk3SExc+DgEvjzhaoRv4Cu4hPfCmutZg8X45Sq4et/v6Q+f6Q6TnG0bOblzlKT59+LuuWPDaT14qFCOLT58vAHyw/eXJ4CtlGkDErxRuo63u09MnkTcPHDxUZZoltBI9oJN63q5pKy/+nEO8Y3jdm+WzVEvE78VpRHZMoppFHzCv+ggi+0OGFFxlfPv9j+Ali/pbLOqasW5VfQ7Qf/F00uDi5mjnzYLFIeDre1N2Orozrq5s1RHDfvuQKnbKNwSfmSHwK6eoyTAE/9fishhzVvuLbNIbgk//+AaHXmm8=
+api: eJztVk1v4zYQ/SvEnLKAaqdBTzrVu5tijW2RwPG2B8cIJtJY4kYiFZJK4gr678WQlC3HG7dFgV7aXByR8/lm5g07cFhYSFdwZQpU8nd0UitYJ5CTzYxs/GcKM2tloaxAYXRFwmmBorVkhFQCldAj5cmtulUzU9j0Vgkh+O5O5uLMOvMuFcuSxPyj0BvhSnqtx+INbiuNuTibqyfpaEGPLVkXNYfLTCuHUklVHJkRMk9CaFSjrBKBKt8H7dOInp61ebANZvRmeDsJn9OCXGvUkNa91lUqlqYlIYO0d/KMVtg2y8jaTVtV2+iS8kT8hJUloV1J5lnaaBOlpcHkp+Xy+vIlIw96KubBrM8l12SF0k6U+ESiIVNLazlbp/lro00tXCmtwGyE5TcNGhIspwQZo02Mb0ByAGpwPIEEdEPGQzvPIQXpy3LHl3dO3+0gggQaNFiTI8MN1YHCmiCFUH9IQHIjNehKSMDQYysN5ZA601ICNiupRkg7cNuGtawzUhWQgJOu4oMrU4h5Dn2f7CyPC/iP7f82GAte1sEGWfde51tWfG2Se5CU4ytsmkpmHqPpV8sD0x17RGNwy2E6qi2fN4aRdZL8l+/WUwFeeoE+Aa6RV0G1vdp4qA9cMELxQLVVBZzLYGThdfs+GST0/VfK3AFiqxjLSO1gFjmG4eI6DCT0Pds0ZButbIju4vycfw555GY3GmIRheGvQxm8/HBxcWz4V6xkHgjgkvv6b1h9VYicXKzEG5WqdHZwe1SGWLpRHaRyVJCBfr1HfuiHAcqfdQiQ4a1tcaoVfiFrsSDYGTvRNX7Il3z7Z1XnvILrKDdqgD28Ad230/gY4PuWs0GEaenIYKhtTa7UTDONts5TiishhemY4e20C5zST3cMYKfdmA36aaApSMCSeRoYqTUVpFA619h0Os0q3eYTLEg5nKCcYiPhaO/5W/GBRf0gWcpaI93W25tdzz/T9hNhTgbS1XoscMPtFRrmUGw/rY38TAxdpLNZ60pthhUc+awMWowNN+5iz0mXL1g3jOdqRx778keOWA0nPjSpNtr7j3WIyb3H7IEUMygDFfI+n3w/OX8LjaggZtfzOGaY+TEbEvFikLzCe4c0JLuAHWH94/6C0+TK16hG9gL7iC+8CJdaHLxUXoXY7af+/zfLf/jNEkfM0YubNhVKz6q+HbvIKSs44BRIIN29VPa0wsdHzwzPLOsESuaodAVdd4+Wvpiq7/n4sSXD/LBO4AmNxPs4pLm0/H8O6YZRPdG5Z4tIy+/EW7kMHKKYQJ6wavkLEnig7f7NxRvoX/R6AJRfdeVAjV0U+RC8fecX0t7E0X7myIPGLONeOCm7Hu2N66ubJSRwHx9ttc5Zx+AzkyI+h3B1E2rOrzo+66BCVbS8UlMINvnvDxCnhYQ=
sidebar_class_name: "post api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
+>
-
-
-
-
+ path={"/organizations/{org_id}/workspaces/{workspace_id}/invite"}
+>
Assigns a role to a user in an organization.
Args:
- organization_id (str): The ID of the organization.
- payload (InviteRequest): The payload containing the organization id, user email, and role to assign.
- workspace_id (str): The ID of the workspace.
+org_id (str): The ID of the organization.
+payload (InviteRequest): The payload containing the organization id, user email, and role to assign.
+workspace_id (str): The ID of the workspace.
Returns:
- bool: True if the role was successfully assigned, False otherwise.
+bool: True if the role was successfully assigned, False otherwise.
Raises:
- HTTPException: If the user does not have permission to perform this action.
- HTTPException: If there is an error assigning the role to the user.
+HTTPException: If the user does not have permission to perform this action.
+HTTPException: If there is an error assigning the role to the user.
-
+>
-
-
-
-
-
-
-
-
-
-
-
-
-
-
\ No newline at end of file
+
+
diff --git a/docs/docs/reference/api/invoke-workflow.api.mdx b/docs/docs/reference/api/invoke-workflow.api.mdx
deleted file mode 100644
index 7649c17bed..0000000000
--- a/docs/docs/reference/api/invoke-workflow.api.mdx
+++ /dev/null
@@ -1,71 +0,0 @@
----
-id: invoke-workflow
-title: "Invoke Workflow"
-description: "Invoke Workflow"
-sidebar_label: "Invoke Workflow"
-hide_title: true
-hide_table_of_contents: true
-api: eJztnVtv27gSgP+KwaezgHJp2qS7fjrZxIvmNE0C2+0CJwgCRqITbWRJS1FpfQL99wPeJOpqSbbTODt9aGBZvA2Hw5lPFucZMXwfoeE1+jOgjzMv+B6hGwsFIaGYuYF/5qAhcv2n4JHcfld3IAuFmOI5YYTyos/Ix3OChkjfcOs6yEKuj4bo75jQBbIQJX/HLiUOGs6wFxELRfYDmWM0fEbYX1zORDVsEfJqIkZd/x5ZaBbQOWZoiOLYdVBipXf4seeh5MZCzGUev6A7PzhzUMJvLPYo8uL7NfSpVR8mvK3KXjxh6mKf/UT5fJM9qJWT7uFLykv3qV5ulDy5kRv4P1FwY9WFWsmlfXxJ0aW9aiG7J0L53xfv2jfVbsJv462RiP0eOAveQtY4ozGxkB34jPhMNB6GnmsLI7T3VxT4/FrWtZByE8VcEvFP2TIn9Mm1ya1qpnznzBPmzhwbdhyXt4K9q9ytpdHfBYFHsG8OXwvkJrH0peDuL2KzRhn9ITqRWGhOGO7ZmfJUuD4j94TmW57f5a+YY6huC9kutWMP03/9EXvef6LA3znzw5j9gqqG6DIyb1cIU4oXzWLJFUWdRPqFS5KX6D+9LyDRi+DcjdgymdaNsVi6m4SmSue0Gei5vtPVbCEHF7U3v9ZMJ+G1zsg26fhVJs+EG/EwZiDZtUj2TMoysVAQMxBrT7G2sB6XSry8Goptcgs2Yr2aPOVSHeQthZQ02It1SzmzGlLCYDs2ajuk0IsWpMkBiULsd56OlavIRHc5Jd4kxP4vqEqwxrcFkXaLJiaii4mFbEowI84tZi2jXwczssPcOWms/0RWOzgWMxiHziYa+SqrVY04xCMbaORUVqsa0eK6W3CssCZeoIX1+0KAgkxea21FSyttRQtsra1ocaWtqI1ENFCstmAaxf188Sy5nSuvujvElEg21jM0uRIV5Poqb20SB/HjOV+0nHFK2oAshH0/YPpD7D/6wXcfFU3RlNdU7o8ed/um8T1HHhayH7DLGzQ4K8PRI/8TBB4vML8jjiMLa3zjeXNeNJiHHlEdth8wE2CHYv+xegBc7Ev6/+j6SzRJ939ydXxx+/ns4vT268XkanRy9sfZ6BRZxvWzi+lofHF8nrs4GY2/jca5SyfnZ6OLae7S1fjy9OtJ8b7Li8nXL6OxOSRtTj/zftcPS+KxfgomlPUCCyuDIoYpuxU2ZwUbVbnHF5vlLQ2msiQivvMirY58J20zYpjF0a0dOC1VejI9nn6d3J5cno64UozEnBrXLj8XLozG48vydIpmT3ir1RMquzUnUYTv+8+qqGXwRdWSWAgzRt27mBVdA/Dl+rnLx5k8E26XZoQS3y7KTvW26IGtj1cJMN9XSQRft9D6dle5TYGqbU7VSgXz1mWs9dBoYXnnxpn2JhbyXP+xnRp384W6elqgRi+qRtljq+t0Yo1JK2xj567/2EnHzoVWJRZ6wNFDSzMJhmmrNcrUl084euikL5+kmnDn8IdNQlbaLmHS+k3aKBUnl+0T8Yt0r2Yx6iCjznTrGIL75RHD83AtzrwZoKYVw1L+mZuDUARzngtbw4jrVKe1PpJauKzhdC/KKExFrGyKFcgnkE8gn0A+gXwC+QTyCeQTXEUgnwTI51aqGpBPUCMgn2CYtkGjgHy+xkkD8gla8cbJ5zrQ4xJ7Ju6aUlITxTH+TcOPT40AoG49GU6+HzhqQ6xemptlWJ47I/bC9hoH1I7TrhvJtqevrZFlJ0SZ1bp6lY0qd07usb041zNxOr2s1jsaBBWvRlZPdd57yvtKS1ZifuWNg4DJHqV6v2r7uvkNUE5SCzk7DZpXpAbtK4S0+qBb77GdOazBXzWSXc5hGwhs7uVFZwWRqkEX6lOSlfi8ye5sWLUlfq9dbppZFsJizjYbu5X5PAUiqmBo58Ijf/nIZLdk/QW3SolbssfygDQjrVMyTUMFAS1RzxzxRCuCzZRoLhms6HKpE2qcNRFFYVM1/dhOk5FzVPVSrd2X5cpZm1zELNqPFW/QdKLHuobOrrZ8238jjmc2I+q+qqimdq2W32lezwBOsXw/fU4Yde2NyOiLqrrDuQI9msAKZG9kCGNebye4WLdwVt32+3kjlJA1PoXm3Rm02IzEd1ZhT+IkTbtbnelbwIjXZPWWPSFtakQ9pXxBg7F5gFFrtvvDiA1ayFaheVfd0dF5l8UrTnv5URGKbDJUXRkfnMheKwG9mlnVwqxA8D3twBK4MfrBKK7dSFMa0f/dWwAWACwAWACwAGABwAKABQALABYALABYALAAYAHAAoAFAIufDiyy3+PwyTqdXla/7pZ9WRhRt5+cXIgO9lAHFT+rMFYFdsrLT73jwm9btJjgtySAZgDNAJoBNANoBtAMoBlAM4BmAM0AmgE0A2gG0AygGUAz5Oegme3o7zEPBrERX+V63bVDWQYzqRGlhtTbUMTJYuU2AaKuROcum8gsYsrPqoIeNQc51GnlP+dchwbpVp060M6Paz52oJ3QDbPe+wCc8gkFfarKNpsm/JZ/c76FnFKHy6bEIT5zsdf2ndYqMppWwdWM2JS0S2zSdVOaqKo7LMuxyu2XWBVJDpflJFvf2oup2xvBUlfW4PWvweM1PBDs9EidBOaoYI5KHe22nX5S05Do5JltZwReQ294Db2LFVFS57gCh6Hrt00CWZRvF36hGxKzTt2w93OjiSydWJAxcc3nFeTzoKnksJvZxmTViUhhO3PvY5k+exNtneQaaLNx6lS8dQ7tEre7Nruu0Sme0Pe2mCc8SRLhLEdh4EdSYw/29/kfh8glI0SEJrFtkyiaxd5grG5GvXMBr287W/fBQm8qX6rMw9Z1+ZdKdVj/suwqBmBrUiBuh3C3MWvqq5RsC7O03YlTX6XUawLzrc6duk2C3t70qa9Syq3JE2RQrTnMC/IIQB4ByCMAeQQgjwDkEYA8Al1mdbvzCLxKd27Jr5AglYAB1kDbNqptkE0ANGljmgQJBUCpNmqe/iE5BbZj3iCtACjGqrvE688sADlVgYUCCwUWCiwUWCiwUGCh4C0CCwUW+ia0DVgoaBKwUDBP26NUwEJf6bwBCwXFWHWXeP0sdB0wcolVgyyrcDImnIypQgk4GRNOxoSTMeFkTDgZs+yr6qVauy8v4GRMOBkTTsaEkzHhZEw4GfMlQnM4GRNOxmyGG5BlFYAFAAsAFgAsAFgAsABgAcACgAUACwAWACwAWACwAGDx9oHFm0/lAb8lATQDaAbQDKAZQDOAZgDNAJoBNANoBtAMoBlAM4BmAM0AmvlpaGY7+vtGs6xCrs+l234W2TlkhmOPZWHd4f6+EX2gY38Q++RHSGyOpQilAR0Eth1TSpzdwZVHcEQGjC4G+B67/sDDjNBdEatXhowlYbQ5BOZE6nHa1cP9/RbQo1G+642yjL71F9h647QGvZGxb7fUqSq7mszM9uHgAJWSsX3DnusIl3sw4oPun4nNIQy7wiut8Sa8wM592+Xd4Jt6M3eu4weuGNF9k49hhNfLAnohjIFUuCUWlI9LNl0OaDPxSunWD+NUiq9pej9Np1elCuXc5if1TGTmG/yZ0cI5YQ+Bg4YoDCJeZYjZAxqivZCn0iXf9zRYjPZkVj8k8yeKfFPXzzJ3LXpgLIyGe3u2F8TOrmCSeBe7ezh0hUJHxI6pyxaiyPHV2WeykLlS0fD6xrxBJPCUSpG/LZ0PHLqfyUJjxSE6jtlDQN3/6eDQ5cOU+XCFyLhyqkzBPDfh6AfmzJNXWJvOcPiMZh4WqUOziJxLP72WviqcqYfGD7l8XGauuVx6tKQyfVdSyjOVlJMiJUa6Hn2OZGKg+Yq+KY/u2nTPs2/TXTK7lHskZD76yW4xH+CUr6rHMFVfmMs5fZ4hbn4/w78ezo4+7Bx+fPdx58Ph0cHO3fuZvXNg/3b0fnZ0hGf4yHwI0b6QlZmNJ8MoaLDfpSKldplZUGp5r16wz5B2tzEp0qzYMjrYP/iws/9x5+C36bvD4eG74cGvu/sf3/1X8d2GG3LegNymU4CbuQHZFORgqREbN3WhaBtL9eZ3v+xqLuZKMmqX42u5RaeAVeYGmq6GQYg6KYOmPS2n5ybjKpKf1A0o4xXXz2VNMWRbJ5Abc5hGkN1q3RZj2iQXziXSG88dkmY418WE8UY7adb3pDHVeoXhEbnRzY+e+THLWp5LmJ1P3mxkVc5KFq1mltG3IvGu2AZdfxZIBTcjmMHx1Rkyzfn+7rvdfVTcNXM381nBNsuIk/o6HZ/eDNNtUDx9Ex4QYgTP/519wXvG99059o36ypt0rjup/nPl2As9/rQvyyIv9+9rpPZv49FgJHZHsYffWOiBb/fDa/T8fIcj8pV6ScIvy0eDXPscN8J3HndkZtiLiIUeCbfd6fyL+PQJezHvinDNlpUQx+R1LPOEqYvl+a09S/ZpNdXt7s2mRVdqN4u9s/I3/AN1eQXCbdILSMyWrOhEOug7U8mVddmSs877Iksc29z6N957YziJV5cTvo/cBY74RcRcbi8Ucz3j/w8RslAgFFXsP+LaM/Kwfx/LDULWyf/9H53l6Bw=
-sidebar_class_name: "post api-method"
-info_path: reference/api/agenta-api
-custom_edit_url: null
----
-
-import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
-import OperationTabs from "@theme/OperationTabs";
-import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
-
-
-
-
-
-
-
-
-
-Invoke Workflow
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
\ No newline at end of file
diff --git a/docs/docs/reference/api/list-api-keys.api.mdx b/docs/docs/reference/api/list-api-keys.api.mdx
index 4322c1a472..80f3651c79 100644
--- a/docs/docs/reference/api/list-api-keys.api.mdx
+++ b/docs/docs/reference/api/list-api-keys.api.mdx
@@ -5,65 +5,186 @@ description: "List all API keys associated with the authenticated user."
sidebar_label: "List Api Keys"
hide_title: true
hide_table_of_contents: true
-api: eJzdVMtu2zAQ/BViTy1A2GmPOlVogzZIgAZ5nBzD2NBri4lEMuQqjWvo34ulZPlRtIce64PNx+54ZjjkFhjXCYoZlMGqS9okmGtYUjLRBrbeQQFXNrHCulbl9YV6pk1SmJI3FpmW6oflSnFFCluuyLE1eblNFCcP7sGVcZ2KB6eUUpFeWkqs3t30g/eFuqtIWWd8Y9163PePT2Q4d98Qt9HtAITITL7K6wuhekMpeJdoXqhS1cLSrzLJyz+RzLRAgw8UUeRdLKEAaV1gsAsRBxrigJug2MLHszP5ObbktjWGUlq1tdqRAA3GOybHUo4h1GKF9W76lKRnC8lU1KCMLFOTwUMUJmxpmNHKvsmIN4GggMTRujVoYMu1LFz3FZ0GE0mkLZD/Vv+5r1IlS0+NiRdtGrvQbb6voJid9nd6XHFtXUM33yNeYWJ1n0ZMegu2t3KxRKZ/hT0fYdQXgek6vavt45CP5aW1kZaS1sGqIx8OWf4eEtgjYoy4OXBpV5IDpsaL0MlHQ0NceYnJmoRFQK6ggOmQlUTxlWLKcttYQwEVc0jFdGpq3y4nuCbHOEE7xWCz5ESmjZY3uaUn+Y1wSRGK2fyw4Fby0ifjuGy0EYO9JFHisJF52XLlo/2ZjQQNVrJa9V0ixbqVz+2D8DKTkysDGkRGn+6zyYfJGZw+A0fFEnU0OUW7/87boE9cGPWDBmrQyh4TNp/2G8Is+MQNugO847M4IbPd37X/8XkajpfpjaehRuskvNnX7ZC/GTwPb3XlE8t8u33ERPex7jpZfmkpSsTmGl4xWnyUA5/NO73Lg6TvmTZycsZQkGi/Yt32sTp5vCSV4zX4en4HXfcLULMi0A==
+api: eJzdVE1v2zAM/SsCTxsgJNmOPs3biq1ogRX9OKVBwMpMrMaWVYnumgX+7wNlx00ydIcdl4OjD/Lh8fFRO2BcR8jmkHurLmgbYaGhoGiC9WwbBxlc2sgKq0rlV+dqQ9uoMMbGWGQq1E/LpeKSFLZckmNr0nEbKUzu3b3Lwzpm904ppQI9tRRZvbvuF+8zdVuSss40tXXr8b55eCTDKfuauA1uDyBE5vLJr86F6jVF37hIi0zlqhKWzSqRvHiLZKIFGhpPAaW88wIykNQleruU4kBDGHAjZDv4OJvJ37EkN60xFOOqrdSeBGgwjWNyLOHofSVS2MZNH6Pk7CCakmqUlWWqE7gPwoQtDTta2RdZ8dYTZBA5WLcGDWy5koOrPqLTYAJJaUvkv8V/6aNUzpJTYeRlG8csdNsfK8jmp/mdHk9cW1XQLV4RLzGyuosjJr1420u5LJDpX2HPRhj1VWC6Tu9jezuktjy1NlAhbh2kOtLhkOWfJoFXRAwBtwcq7UOSwdQ4CJ38NNTEZSM2WZOw8MglZDAdvBIpPFOIqdw2VJBByexjNp2aqmmLCa7JMU7QTtFbOB2tPN2qLxKa9Ihk2mB5m/D6Cr4TFhQgmy8OA27ETL1tjsNGjdHbC5IyHdayz1sum2B/JZVBgxUCZZ8ldVq3alL6oMrA7TOaDbkCNEidPe3Z5MNk9lYxQ4IM4jATaJLd9jxSGOgTuUahQAPVaOWOCetPrxfC0jeRa3QHeMdNOyG1ex3K//EdG1rN9MJTX6F14vKk624w6hw2w6NeNpFlv9s9YKS7UHWdHD+1FMRuCw3PGCw+SPPni07vvSFO3NBWOmcMeZmBZ6za3mInr5w4dJyXb2e30HW/AYK5MV8=
sidebar_class_name: "get api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
-
-
-
-
-
+>
+
List all API keys associated with the authenticated user.
Args:
- request (Request): The incoming request object.
+request (Request): The incoming request object.
Returns:
- List[ListAPIKeysResponse]: A list of API Keys associated with the user.
-
-
-
-
-
-
-
-
-
-
-
-
-
+List[ListAPIKeysResponse]: A list of API Keys associated with the user.
-
\ No newline at end of file
+
+
+
+
+
Successful Response
+
+
+
+
+
+
+
+ Schema
+
+
+
+
+
+ Array [
+
+
+
+
+
+
+
+ last_used_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ expiration_date
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+ ]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/docs/docs/reference/api/list-app-variants.api.mdx b/docs/docs/reference/api/list-app-variants.api.mdx
index aa96e137fa..e5b1038d05 100644
--- a/docs/docs/reference/api/list-app-variants.api.mdx
+++ b/docs/docs/reference/api/list-app-variants.api.mdx
@@ -5,73 +5,585 @@ description: "Retrieve a list of app variants for a given app ID."
sidebar_label: "List App Variants"
hide_title: true
hide_table_of_contents: true
-api: eJzdVk1v2zgQ/SvEnFpAsNNgTzqt2xSt0XYbuE724BgGI40ltjLJkpR3vYL+ezGkaEv2Ou42tz3FIWfefL15VAOOFxbSBUy0trBMIEebGaGdUBJSmKEzArfIOKuEdUytGdeabbkRXDrL1sowzgqxRekvpjejB/kgJ6aw6YNkjNHpSuTshXXmZcrmJbLpDcG4Er2HU8zEIH1YjzNDVxsZoT4K6xYTre+D2QytVtLiMmWT89lRnEF+kIDSaDhVOM0hBXJdUZrRDxLQ3PANOjTUmwYk3yCkEGqBBAT1RnNXQgIGv9fCYA6pMzUmYLMSNxzSBtxOk5d1RsgCEnDCVXQwoURyaNsleYciLDlcX13Rn+EIvtRZhtau64rFiiGBTEmH0pE517oSma9n/NWST3OaBDeG7yhzhxsfSxvqghMhclfZTySdeNvQkKet/yCbNoGurRfwu6F2MaLPpTjRK8bSRn3F7FKs22DVxXrkFi8Ges0t7qN4j6dDePuAnym5FsXFCG+82T5GbYQfjNx9XnsODv3aZH8i66oC4lJEujOCEAxuhRWBD52pkA4LNL2os2hEiRrkDvMVd78a+U1AYBPnS9D5M/HuAkKHt1G5WAvMV4+7rvu/gvmpQ2Gvd92AlCm4FP/4DXoG8OceTIf8lzLfrOYZPgP2z4gRNCOJhuqRODwQoMVBofZLOti/o8Ua7Et/EQ4UH9K3R6rlYNmPJHlFxR8oFg69fjNShvsotG1LBf12fX0qe/e8Enno5VtjlPkvmjfUthwdFxX9OiN+lcoGtz8xpbhI7fIwkCixse6PKiToqWuLp3b/E1rLC7/3weS8qW8Gm9PtJTJQXSF0Z7fsC2dsb+ju+TJuQvv+LVg0eT+f354Ahtlu0JWKXtkCnX9XXQkpjLnWdtwEtrbj3sNr0Wzjq1ubClIondM2HY+zStX5iBcoHR9xMeZa+D2xmNVGuJ13mdxOP+DuPfIcDaSLZd/gC/EjTHxodngktfiAVHv33k9qVyrTrXR89svgRcUJuVbePS6CT45Nbqe0aWiC+sLV6NXoCo4/rQbGRG2eeWrH2P4akqMu7OuHBHDjeQ0O+eb3wwVlppV1Gy57eKfLd5RQc9iv/+lnXzdlh3+7sa64kOGd9dIQiOkFlDqT7oV0z81lAqWyjoyahuTxzlRtS8ffazREv2Vn/UhkWDSQC0u/c0jXvLL4RL9fzLq1fcnO5RkpKnc+qaqm/yCBb7g7fJl6OSoj+5vucpJlqF3P7UQ9aU32m/ru7Rza9gdYKx2V
+api: eJzdVk1z2zYQ/SuYPSUzHMn19MRTaTvTaJI0HsV2D7JGA5MrEjEFIACoVuXwv2cWICRRqqw0vvUkCnj79gP7FmjB8dJCOoNMawvzBAq0uRHaCSUhhSk6I3CNjLNaWMfUknGt2ZobwaWzbKkM46wUa5R+Y3IzepSPMjOlTR8lY4xWF6Jgb6wzb1N2VyGb3BCNq9BbOMVMdLJP63mm6BojI9VHYd0s0/ohwKZotZIW5ynLTkdHfgbxQQJKo+GU4aSAFMh0QWFGO0hAc8NX6NBQbVqQfIWQQsgFEhBUG81dBQkY/NYIgwWkzjSYgM0rXHFIW3AbTVbWGSFLSMAJV9NCRoEU0HVzsg5JWDK4vLign+ERfGnyHK1dNjWLGUMCuZIOpSM417oWuc9n/NWSTXscBDeGbyhyhyvvSxuqghPBc5/ZDwSdeGwoyMvoPwjTJdCX9Qx/f6i9j2hzzk+0ir60UV8xP+frNqB6X0/c4llHV9zi1ou3eNmFxwf+XMmlKM96uPawrY/GCH8wcvN56XtwaNcl2xXZ1DVQL0WmeyOIweBaWBH6oYcK6bBEs+d1GkEUqEHusFhw97OerwMDy5xPQRev5LsPDD3fShViKbBYPG366v8M56eehV1t+gNSpuRS/OMV9Ariz3s0PfNfyjxbzXN8Be2fkSPMjCQC1RP18GAAzXYTaivSgf4OhDXQy74Qdi0+bN+9ppoPxH4wkheU/K7FwqKf34wmw0MctF1HCf16eXk89h54LYpQy3fGKPNfZt5wthXouKjp68Twq1U+2P2BU4pC6ua7A4kjNub9UYUAfeva8iXtf0Jreel1HyCnob4Y7I52zzUD5RVc97j5/uCM5Q3VPZ3GTSjfvzmLkPd3d7dHhOFsV+gqRbdsic7fq66CFMZcaztuQ7d2472L16JZx1u3MTWkUDmnbToe57VqihEvUTo+4mLMtYDD50rmd9k1Qb2ILOaNEW7j+bLbyQfcvEdeoIF0Nt8HfKHmCe0whO1uUC0+IBWmfwxkjauU6fUe3wRVsKLMhVwqbx5VEmK74vkzSi9FNGE8w8Xol9HFqWR6A5bdTnoN8NxrIMbhYZAclGtbKEgAV14A4JCvftttUJRaWbfico/vWKUHgbU7If5P34f9iTv82411zYUMF7KfIaGD/aSlyqTbibtt4nkClbKOQG1Lc/Te1F1Hy98aNNSK8x79RI0xa6EQlr4LSJe8tvhCvd9Me32/ZafijO0qNz6ouqF/kMAzbnZPWD+3qqiEtt/M8hy12zM7GrMkma2kf393B133HXgSLCQ=
sidebar_class_name: "get api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
+>
-
-
-
-
+>
Retrieve a list of app variants for a given app ID.
Args:
- app_id (str): The ID of the app to retrieve variants for.
+app_id (str): The ID of the app to retrieve variants for.
Returns:
- List[AppVariantResponse]: A list of app variants for the given app ID.
+List[AppVariantResponse]: A list of app variants for the given app ID.
-
+>
-
-
-
-
-
-
-
-
-
-
-
-
-
-
\ No newline at end of file
+
+
diff --git a/docs/docs/reference/api/list-apps.api.mdx b/docs/docs/reference/api/list-apps.api.mdx
index 33be1d177c..3b86056d73 100644
--- a/docs/docs/reference/api/list-apps.api.mdx
+++ b/docs/docs/reference/api/list-apps.api.mdx
@@ -5,76 +5,390 @@ description: "Retrieve a list of apps filtered by app_name."
sidebar_label: "List Apps"
hide_title: true
hide_table_of_contents: true
-api: eJztVk2P2zYQ/SvEnFpAsLeLnnSqgC4aI2mz2Di9eI3FWBpLTCmSIUfbuIb+ezGUtJay3STtuSfbnK838x6HPgNjHSHfQeF9hH0GFcUyaM/aWcjhjjhoeiSFyujIyh0Veh/VURumQJU6nOTgwWJLq3t7b4tQx/zeKqWeztV3b1M6NLvIYf99rrYNqWRxR8UNiadiNyZVh1PKdEfcBTsle6Mj7wrv97kqvhHKHepIU/yr7fb25lNJCUmuNqlwIPUnRoVWUQguqDB0q22dYM3LrCAD5ymgxG8qyEGsD2KCDDwGbIkpyCjPIBAghwkNZKBlmB87CifIINDHTgeqID+iiZRBLBtqEfIzoD29PaYcfPKSI3LQtoY+ezqxnTHQ7zNgzUYOCu/Vb1Kml9NA0TsbKUq666sr+Vhy+q4rS4rx2Bl1NzpDBqWzTJYTCO+NLlOn6w9RYs4ziCMMDAGlGc3Uplo+yHxYD5Wld13N/MdGlrA3FfTZZU5f8R6aHPwHv/8+sK3Y+gw6XyFT9YD8pervBy9VMPR9Nrm5wwcqeUHobmo8m7M/K7KEIQimnxMXSeoqXce+l2o/Xl8/J/F3NLpKFKkb0e6/YXDJVEWM2si3F6g0rlxYv2Hk2jLVFKDfX6Y1CWbq940bAMoM2lh/afy/UoxYJ74m3l9yTcMY2f0KU9LXUHr0m3FzGe8w3Zfb+HkY3z8Vm1xk9zxLOHDbEjdOtklNnPYIN5DDelwrkcLjtFO6YCCHhtnHfL0ujeuqFdZkGVeo1+h1kniksguaTymkuN28ptMrwooC5Lv93OGdqGHgd+l2ueBevybpdNxmRceNC/qvgbRxpTVDlLSi7dGl8EneCZwqbjeQgbQxCPdq9cPqCj5/ZxbOImQsk5Cn2sksN2kxhaf+IQNqk4qBCdufLgZB5l3kFu0s3+WKfQbkfLlF/79987dvFAXTJ157gzpd28TGeVRtWn3pL0TjIsvv8/mAkd4H0/dyPDx/IsxKRzyY2QP4B52W7+Ujmk7qJVE/YtDinzScTZqTRENcUUpvs6hn60+yPF21X2620Pd/A86yDpM=
+api: eJztVk2P2zYQ/SvEnFpAsLeLnHSqmi4aI0mz2Di5eI3FWBpLTCiKIUfbuIb+ezCktJaz3STtuSfb5Hy8eW9m6CMw1gHyDRTOBdhmUFEovXasOws53BB7TfekUBkdWHV7hc4FtdeGyVOldgc5uLPY0uLW3trC1yG/tUqph3P105sYDs0msN/+nKt1QyredHvFDYml4m4MqnaHGOmGuPd2CvZKB94Uzm1zVfwglBvUgSb/F+v19dXnkiKSXK1iYk/qLwwKrSLvO698qlbbOsKap1lABp0jj+K/qiAHub2TK8jAoceWmLxQeQSBADlMaCADLWR+6skfIANPn3rtqYJ8jyZQBqFsqEXIj4D28GYfY/DBSYzAXtsahuzhxPbGwLDNgDUbOSicU39KmkFOPQXX2UBBwl1eXMjHuaZv+7KkEPa9UTejMWRQdpbJcgThnNFlrHT5IYjPcQZxhIHeoxSjmdqYy3nhh3XKLLXramY/FnIOe1XBkJ14+o51KjLZJ7v/Ttha7oYMelchU3WH/K3s75KVKhiGIZvMut0HKvlM0M1UeDZXf5bkHIYgmH5OWsRWV3Ech0GyPbu8fCziezS6ihKpK+ndf6PguVIVMWoj356Q0nTl2e0PUK4tU00ehu2JralhpnpfdQmgcNCG+lv0v6YQsI56Tbo/ZRrJGNX9jlJSV0o92s20OdGb2H26jN8Tff+UbDKR3fMoYNK2JW462SY1cdwj3EAOy3GtBPL3007pvYEcGmYX8uWyNF1fLbAmy7hAvUSn4evdXcRb9VxMY/8HKnuv+RDjFderl3R4QViRh3yznRu8lVZJ4p+bnabf6ZckNIyrrui56bz+Oyk67rsmeUmd2u676D71fsL2G5Yfycq0SJ0J9sXil8XFU8WMDqq4Xo0dj2Xs+AlHNJORO6PrgSjIgNrY7sCE7a+nC0HpusAt2lm80yx+Beh4Grf/H8n5Izk2CNNnXjqDOs53VOM4tnfckfG/RtMFlt/H4w4DvfNmGOQ4vZPSpJUOuDOzl/IjHc4f1ns0veSLDX6PXot97Ods6j8JlPyKUmqbeT3akxLlYSb/uFrDMHwBH3EdIg==
sidebar_class_name: "get api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
-
-
-
-
-
+>
+
Retrieve a list of apps filtered by app_name.
Args:
- app_name (Optional[str]): The name of the app to filter by.
+app_name (Optional[str]): The name of the app to filter by.
Returns:
- List[App]: A list of apps filtered by app_name.
+List[App]: A list of apps filtered by app_name.
Raises:
- HTTPException: If there was an error retrieving the list of apps.
+HTTPException: If there was an error retrieving the list of apps.
-
+>
-
-
-
-
-
-
-
-
-
-
-
-
-
-
\ No newline at end of file
+
+
diff --git a/docs/docs/reference/api/list-bases.api.mdx b/docs/docs/reference/api/list-bases.api.mdx
index 14d885bbc8..acab4a92c1 100644
--- a/docs/docs/reference/api/list-bases.api.mdx
+++ b/docs/docs/reference/api/list-bases.api.mdx
@@ -5,78 +5,363 @@ description: "Retrieve a list of bases filtered by app_id and base_name."
sidebar_label: "List Bases"
hide_title: true
hide_table_of_contents: true
-api: eJztVt9v2zYQ/leIe0oBwc6CPelpHhKsRrsmSL29OEZAS2eJHUWy5CmrZ+h/L46UbCme2+1xwJ5s36/v7rvjnQ9AsgqQr+FnGTDAJoMSQ+GVI2UN5PCI5BW+oJBCq0DC7sSWLcVOaUKPpdjuhXTuWZVCmjIqn41scPZknszCVyF/MkII4fFzi4HE1WP68iYXqxqFMoVtlKkG/SxZ9xGvAvnecHnL2FQj6wTZPgGx3fcuR2RxdR+zl3odyG8yYfuffaRo08dip2kwcYs72WoKLP5gTSrkEan1ZqjlvQq0ZsLuW3ItbXKxOLJzEgu7/YQFBeHReQxoiOtk1BN3TGUCkCrgEP/tavVw96XAmHculjFXj+JPGYQ0Ar23XvjUmSFmCgUZWIdesuOyhBw4reeogwyc9LJBQs8dPwATATkkriEDxQ3/3KLfQwbcD+WxhJx8ixmEosZGQn4A2jt2C+SVqSADUqRZsHBOLEvouuwY+tiTy9F3UodJeGn297uY3xSIw/YS02oN3eYEzaSLD4zTsdhjcNZwzfkBbq6v+WM61h/bosAQdq0Wj70xZFBYQ2goZuGcVkXkcf4psM/hnALpveRqFGETsZxn9kkl5Fi9Kr/FWUycSctGXH3Pvi80G6zSnE1YXR/Bx4FfMZbGlLEH6cBFnHCRVkLXMdSPNzfnLP4utSojR+KOZ/LfUDilqkSSSvO3C1xqW0y0/2BKlCGs0EO3OVE1dGwo+L1NCTIJTai+Rf2vGIKsEI7BLptGMsSKtd9rE9eVoHu7UY9O9CZ2L5dxm+j7O7DBhHfKWcDU2waptrwsKqS4JaiGHObD1gjoX4aV0XoNOdRELuTzeaFtW85khYbkTKq5dCo+y4BF6xXto8viYfkO929RlughX2/GBh95HFKDp2anJ+bUO+RS+42yaKm2Xv2VutZvlTp5cS3K7Gx0H5ZSTE4sHpaQAZeRJvd69sPsGl4fu4kxT7Is4iQP2FEN2SsWjvVDBtjEMQZC2fx0UnBmzgZqpBnFGz2yV5kcTu/o/wv8n7vA/egSfqG501LF7RJn5tA/rrSe49+t2gZiweHAot+87joWp0PJ76dUQW716FRenJTYWF4sb8SlLP7A/fjiv0jdslFcm2dAyXh8w0/2G/7hFTvER50Nj5BTTo6LgvkbeZ0dBI5yXD6/3K2g674CcIabKg==
+api: eJztVktv2zgQ/ivEnFJAsNOgJ53W3QSt0UeC1LsXxwhoaWyxpUiWHGXrNfTfF0NKlhyv291jgZ5sz+ObmW8e9B5IbgPkS3gtAwZYZVBiKLxypKyBHO6RvMInFFJoFUjYjVizpdgoTeixFOudkM49qlJIU0blo5E1Th7Mg5n5bcgfjBBCePzaYCBxcZ++vMjFokKhTGFrZba9fpKsO8SLQL4znF9zbKqQdYJsl4BY7zqXQ2RxcRuzl3oZyK8yYbufHVK06bDY6RhMXONGNpoCiz9akwq5R2q86Wt5rwItmbDbhlxDq1zMDuwMYmHXn7GgIDw6jwENcZ0cdeCOqUwBpArY479dLO5uvhUY887FPObqUfwlg5BGoPfWC58602MmKMjAOvSSHecl5MBpPUYdZOCklzUSeu74HpgIyCFxDRkobvjXBv0OMuB+KI8l5OQbzCAUFdYS8j3QzrFbIK/MFjIgRZoFM+fEvIS2zQ7Qh56cR99IHY7gpdndbmJ+x4EYtpOYRmtoV0NoJl185Dgtiz0GZw3XnO/h6vKSP47H+lNTFBjCptHivjOGDAprCA3FLJzTqog8Tj8H9tmfUiC9l1yNIqxjLOeZfVIpcqxeld/jLCbOpGUjrn5k3xWa9VZpzo5YXR6Cj4GfMZbGlGP30p6LOOEinYS25VCvrq5OWfxTalVGjsQNz+T/ofCYqhJJKs3fznCpbXGk/Q9TogzhFj20q4GqvmN9we9tSpBJqMP2e9R/wBDkFuEAdt40kiEWrP1Rm7iuFLqzG/VooDexe76M60TfvwXrTfimnACm3tZIleVjsUWKV4IqyGHaX42A/qk/GY3XkENF5EI+nRbaNuVEbtGQnEg1lU7B8wdkFrXidzaNOxuwaLyiXcSb3c3f4e4tyhI95MvV2OATz0rq/rHZsH9OvUPmoTs3s4Yq69XfqaXdyamSFxeqzMZG9/5ipdxey+ILGl4VrjOlfTl5Obk8V0znIGZ3827kZRFHvs8jmkH2jK4DUZAB1nHegVDWvw0KztLZQLU0I7zRNj7LaD8s3K+n+qd7qrsxJvxGU6elimcozsy+28J0x+P/ssoGYsF+z6I/vG5bFqcXlXepVEGu9ehNPTspsbF8gV6Ic1l8wd34r8GT1A0bxft6EigZjx/7wX7FP7xih7jgWb+QnHJynBXM38jr5OVglMOVenOzgLb9B82nqbk=
sidebar_class_name: "get api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
-
-
-
-
-
+>
+
Retrieve a list of bases filtered by app_id and base_name.
Args:
- request (Request): The incoming request.
- app_id (str): The ID of the app to filter by.
- base_name (Optional[str], optional): The name of the base to filter by. Defaults to None.
+request (Request): The incoming request.
+app_id (str): The ID of the app to filter by.
+base_name (Optional[str], optional): The name of the base to filter by. Defaults to None.
Returns:
- List[BaseOutput]: A list of BaseOutput objects representing the filtered bases.
+List[BaseOutput]: A list of BaseOutput objects representing the filtered bases.
Raises:
- HTTPException: If there was an error retrieving the bases.
+HTTPException: If there was an error retrieving the bases.
-
+>
-
-
-
-
-
-
-
-
-
-
-
-
-
-
\ No newline at end of file
+
+
diff --git a/docs/docs/reference/api/list-environments.api.mdx b/docs/docs/reference/api/list-environments.api.mdx
index 6ae59d178c..03a799802d 100644
--- a/docs/docs/reference/api/list-environments.api.mdx
+++ b/docs/docs/reference/api/list-environments.api.mdx
@@ -5,73 +5,569 @@ description: "Retrieve a list of environments for a given app ID."
sidebar_label: "List Environments"
hide_title: true
hide_table_of_contents: true
-api: eJzdVk2P2zYQ/SvEnBJAsLeLnnSqgV00RtLuwnHTg9dYcKWxxEQiGXLk1hX034MRRcsf602K7akny+LMm5nHeTNqgWThIV3BzFoP6wRy9JlTlpTRkMICySncopCiUp6E2QjUW+WMrlGTFxvjhBSF2qIW0loxv5k86Ac9c4VPH7QQgt8+qly88eTepmJZopjfMAyV2HuQES4GOYXusRZIjdMR7oPytLod7e4asg2tUzF7LkFhnj5jRn4CCRiLTnJZ8xxSYOPHw3iQgJVO1kjomJAWtKwRUggFQAKKCbGSSkjA4ddGOcwhJddgAj4rsZaQtkA7y16enNIFJECKKn4xY3Zy6Lo1e3trtEfPDtdXV/xzzPvHJsvQ+01TicVgDAlkRhNqYnNpbaWyvp7pZ88+7XkS0jm548wJ6z6WdcwCqRA5FHg55d/5vEsiAz9QXMIRmPHv2N8Hq8EnR1uZHeaPHGgrnZI6Aki9u9v0t3EM1SX7N7qpKmBWI/jNACc4q08B7jRSjBIpeFWcGCPy9Ww9DrfKK6P/68IWA+5QYQzzbAylCQt0LwaJeAxmXCG1+qfvslfkfXcAM6T5l3FfvJUZvgL2z4gRdJVEw6D5I5GuQq8no5gP+vQA8mywPHKyIzVBif0QEreHw6PrOIGfr6/PpfxJVioPtd86Z9y/0fGxXnMkqSp+uiDoymRHpz/A6r4l1iOBcWzEuj+YkCBzUfviJWH/ht7LoldBMLls2pMhlnz6vcvjukLowe7gykZ6A7uXy7gJ9D0XLJq8Wy7vzwDD3dZIpeHNUSD1u4JKSGEqrfXTNrRVNz1ZKB7dNm6TxlWQQklkfTqdZpVp8oksUJOcSDWVVvW97TFrnKJd7zK7n7/H3TuUOTpIV+tDg4/cI+HWj83G4W/Ve+T6hz02a6g0bpBhXGdl8OICld6Y3j3O9D45MbufQwJcRmjnq8lPkys4/U44Mub2lhmNK2Y4huSEhX39kADWfW8Doax/GQ84M2s81VIf4J0L8CShdtTY/+kbZrhawr9paiupekn2nLZDR654ojAd6X7UHTXlOoHSeGLDtn2SHv9wVdfx668NOu67dQL9znriLli1kCvPzzmkG1l5fIHoN4tBs2/FpVxjb2puzK2sGv4HCXzB3fip1c+iMrZ9OxzOsgwtHbidjU7Wx16mv94uoeu+Aamtvfc=
+api: eJzdVk1v20YQ/SuLOSUAIblGTzxVrY1GSFobipoeZMFYkyNxY3J3sztUqxL878FwuaI+LMeBc+pJFHfmzczbeTNsgOTaQ7qAibUelgnk6DOnLCmjIYUZklO4QSFFqTwJsxKoN8oZXaEmL1bGCSnWaoNaSGvF9Gp0p+/0xK19eqeFEPz2XuXijSf3NhXzAsX0imGowM6DjHAxyDF0hzVDqp2OcB+Up8X1YHdTk61pmYrJUwkK8/AZM/IjSMBYdJLLmuaQAhvf78eDBKx0skJCx4Q0oGWFkEIoABJQTIiVVEACDr/UymEOKbkaE/BZgZWEtAHaWvby5JReQwKkqOQXE2Ynh7Zdsre3Rnv07HB5ccE/h7x/rLMMvV/VpZj1xpBAZjShJjaX1pYq6+oZf/bs05wmIZ2TW86csOpiWccskAqRQ4HnU/6Tz9skMvCC4hKOwIx/w/42WPU+OdrSbDG/50Ab6ZTUEUDq7c2qu41DqDbZvdF1WQKzGsGvejjBWX0KcMeRYpRIwavixBiRryfrcbhRXhn9owub9bh9hTHMkzGUJlyjezZIxGMw49ZSq/+6LntF3jd7MH2a/xj36K3M8BWwf0eMoKskGgbNH4h0EXo9GcS816fnG/Bcw7zgivduYi/lk8F1z2QM1Aeld0NOXO8Pp7blAn++vDwdFZ9kqfLA7bVzxn3PnDicBzmSVCU/nRkYpckOTl9wa7uWWw4XFMdSrPuDCQkyF5VfPzc4/kDv5bpTWTA5b9qRIeZ8+q3m4LpC6N5u78oGegO758u4CvQ9FSyavJvPb08Aw91WSIXhzbRG6nYRFZDCWFrrx01o23Z8tLA8uk3cVrUrIYWCyPp0PM5KU+cjuUZNciTVWFoFx7t90p2K39i0E5bHrHaKth3e5Hb6HrfvUOboIF0s9w0+cgOFljg0GzaPVe+RyemX6KSmwrh+BsRdWgQvrl7plenc40IJuf0qs0fUrCauM6R9MfppdHGumN5BTG6nvQ5kRsOu680gOaJrRxQkgFUnAiCU1S/DAWdpjadK6j28U6UeJdYMYvw/fUz110z4L41tKVWn3Y7Tpm/dBY8epiPdzdyD7l0mUBhPbNg0D9LjX65sW379pUbHPbhMoJusD9wRiwZy5fk5h3QlS4/PEP1m1ov7rTiXa+xTzU26kWXN/yCBR9wO33zd0CqiBJr+cJJlaGnP7WTGslZ2ev79eg5t+xUUaPEZ
sidebar_class_name: "get api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
+>
-
-
-
-
+>
Retrieve a list of environments for a given app ID.
Args:
- app_id (str): The ID of the app to retrieve environments for.
+app_id (str): The ID of the app to retrieve environments for.
Returns:
- List[EnvironmentOutput]: A list of environment objects.
+List[EnvironmentOutput]: A list of environment objects.
-
+>
-
-
-
-
-
-
-
-
-
-
-
-
-
-
\ No newline at end of file
+
+
diff --git a/docs/docs/reference/api/list-organizations.api.mdx b/docs/docs/reference/api/list-organizations.api.mdx
index 8b2a7c2d04..de65436e3a 100644
--- a/docs/docs/reference/api/list-organizations.api.mdx
+++ b/docs/docs/reference/api/list-organizations.api.mdx
@@ -5,65 +5,184 @@ description: "Returns a list of organizations associated with the user's session
sidebar_label: "List Organizations"
hide_title: true
hide_table_of_contents: true
-api: eJztVE1v2zAM/SsCL7sISbajTwuwYg06LEWbYYc0GFibidXakivKbT3D/32gYidxtvWy63JoUvLx6/FRLQTcMSRrWPodWvMTg3EWNhoy4tSbKv6bwA2F2ltWqArDQbmtcid4VsjsUoOBMvViQq5CTqpm8u9YMTEbZyd39s72aZI7q5SKqdandTeJmv9jATRMQ/7L1er64jWlOESiFluJ8qQMK7SKvHdeeQre0LOxu5hyXHTrXRnNGQa8R6YJaHAV+ehfZJCAdPtjFAUaPHHlLBND0sKH2Uy+xnze1mlKzNu6UDc9GDSkzgayQeBYVYVJY8bpA0tMC5zmVKL8MoHKmLzy0k4w+1Imk7+hqQgS4OCN3YGGYEIhhkUGnQaLJb2F+ir+ToN7seTfAi4joDtTyt/xn05gne5hLaBtlltI1ueBnT5YbF0U0G2OqVZi7zS8OP/IFab99AMp54kGA3qPzUlH34/h3RHl7h8oDXGLT7XxlMl1mAx66gZmxnOfNDc6pL8XH9auvojelyMFdfLRUFLInYhsR9JPhSGHBKbnamPyz+Q5clj7AhLIQ6g4mU7TwtXZBHdkA07QTLEykUemtPYmNDFkfr24ouaSMJN9rzengFtR3J7dMexAMlbmipqBnATmdcidH8bXYETt+T5KZjJ262J4T8M8Nqfm1wvQIGPs72M2eT+ZwfkrNALLsWAaj2WoHd2gz1g4zA8aqEQjvkBYfjw6pLPKcSjRnuT7w2Z+U/vhZP8/kccnstdGoNcwrQo08Q7iUtpexWsYq3ijIXccxNG2kuabL7pOzE81eRHqRsMzeoP3Ipv1ptODqkTDj9TI/lMZRHSERb0X59kjKto+XNXnixV03S/QHnFC
+api: eJztVMFu2zAM/RWBl12EpNvRp2VbsQYdlqLNsEMaDKzMxGptyZXodlngfx+o2Emcrb3sOh9ih3ykyMdHbYFxHSFbwCys0dlfyNY7WGrIKZpg6/Q3g2viJrioUJU2svIr5Y/wUWGM3lhkytWz5UJxQaqJFN5EFSlG693o1t26Lk1265RSKdXi+Nxlpib/eADaSH3+i/n86vynodREpqYriQqkbFToFIXggwrEwdKTdeuUcnjoKvgqmXNkvMNII9DgawrJP80hA6n2xyAKNASKtXeRImRbeHd2Jq8hnzeNMRTjqinVdQcGDcY7JscCx7ourUkZx/dRYrYQTUEVypdlqlLyOkg5bHdH2Vx+eVMTZBA5WLcGDWy5FMM0h1aDw4peQ30Vf6vBPzsKrwFnCdCeKOVl/KcjWKs72BbQbWYryBanga3eW1xTltAuD6nmYm81PPvwEGs0Xfc9KaeJegOGgJujir4fwtsDyt/dk+E0xcfGBsplO2wOHXU9M8O+j4obLNLLh/djV19E77OBglp5NFTEhReRrUnqqZELyGB8qrZI4YlCTBw2oYQMCuY6ZuOxKX2Tj3BNjnGEdoy1hdPNniSv+ijQRHIk0wTLm5RvcjW9pM0FYS5iWCyPATcixx31Q9h+AljbS9r0zGUwabjwoedGg5UCil2UNGzdyqfwjqOutg9oHsjJBKTPXdlno7ejs5ea6QLU5GrabRWatFV9HQkG+oSuPVGggSq04mPC6v3BIVXWPnKF7ijfX0b4x1rsd/v/XXq4SzudMP3kcV2iTQuThrLt5L6AodyXGgofWRzbraT5Fsq2FfNjQ0FEu9TwhMHinUhosWx1rzDR8wNtZP5GGhE9YdnshHpy24rO9+v3+XwObfsb30Z/0Q==
sidebar_class_name: "get api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
-
-
-
-
-
+>
+
Returns a list of organizations associated with the user's session.
Returns:
- list[Organization]: A list of organizations associated with the user's session.
+list[Organization]: A list of organizations associated with the user's session.
Raises:
- HTTPException: If there is an error retrieving the organizations from the database.
-
-
-
-
-
-
-
-
-
-
-
-
-
+HTTPException: If there is an error retrieving the organizations from the database.
-
\ No newline at end of file
+
+
+
+
+
Successful Response
+
+
+
+
+
+
+
+ Schema
+
+
+
+
+
+ Array [
+
+
+
+
+
+
+
+
+
+ type
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ ]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/docs/docs/reference/api/list-secrets.api.mdx b/docs/docs/reference/api/list-secrets.api.mdx
index e9ca7faa8d..2a388bfe7a 100644
--- a/docs/docs/reference/api/list-secrets.api.mdx
+++ b/docs/docs/reference/api/list-secrets.api.mdx
@@ -5,59 +5,617 @@ description: "List Secrets"
sidebar_label: "List Secrets"
hide_title: true
hide_table_of_contents: true
-api: eJzNV01v20gM/SsGz0Kc7FGn9TZFN3C6DdbNXozAoCXammakmcxQRtRA/33BkeRISlabuIfWF0nz8cj3yOHQT8C49xCvYUWJI/ZwF4Gx5JCVKa5SiEErzxvfzkbgyFtTePIQP8Fv5+fySMknTlnZAjGsyiQh73elnv3dLoYIElMwFSzL0VqtkmBh/s3LnifwSUY5yptiygO4deIIq8bUvSpSeXJlCWLw7FSxhwioKHPx3zpzUCm5zT1VYq70bPJNNyq0WLGWrQ3TpeDVEaTIwSwW1ZcdxOv32jWWClSBYEZOmGJR+QS1vKZEVhU7hzKsyWaobSYfe2ceIIJceXaow34sOHPGqgQisOSspkfFVZhisyfOyG3Cl1h0pmRyAkS5KtSAHmORoktvWuod0aMUL6Wl6jWGHeCSKqjrqJs322+UcMiEh1I5SkUFgZjwYUXMqtj7y69f/h9K/O25OwEb4KL3RqxJDVH8exkCtqXUmeQeIvC4pxzvg7IHckyPz5L/UkH+EDi8PcSl08MkH4ojIrYjRak11D1Tt04L9oGcV81ZPQXln3Z7HXXpdgpKSMUI6JEd+iEIpqmSioL6psecXUnjdJsy8LFBfi1HX5d+kNkR5CYl7SfKmNflfuqwrWT+Z1HsH8Pg6IuM+yz8RpxbFHQOqx6Tz40UvbTcdORPCX0n+KxTKEgt1X6k0lH4MfbQ0SlbwfXZUqDfW62OKfCfhzWo1pu9lAuojkClU9JEsDMuR4YYylKlk+5fhWKQEb5aCgrM6dQg/CV75dLsX/inQV32IKaO258NjToCrXaUVImmqes6cYRM6Qb5VMc+NAizhRwjKG36g3i3DcIIb1ttpgP+Fsg/qlkT62fUH4ecjMY17TGprrtIdBfwEPWNZyZ0Xsc8fdGgdb3jdJXpVs2uledZ18TW8hulKQxWRJATZ0Ya3D2JgxY5gxjmByw1zw8X87blnUtbQE6uvyBpuEkhY7Y+ns8Tbcr0DPdUMJ6hmqNVQVZPSekUV2HL4uZqSVWbyPH6rr9gJX1vk7rDZUdR0aplaGmbgwuLkjPj1PfQP0MESri1IgppVexM2N4qtAjOzRY3V9C7xeH87OLsHMYaDRZLy44JPxeNdhqikQpH/tJl5ahkjgnz358nxDNrPOdY9PBGIRmVlVYApkeeW40qdA9tJ9OEaw0hXELsIsSpQ5J0yoxnWfL0tEVPt07XtQw/lOQkLncRHNAp3IpK67texVy3LTEskoRsQEddNrEY/XORUB4T6dPHr1DX/wLHrYz1
+api: eJydVE1v2zAM/SsBz0Kc7ujTsnbYghZrsXS7BEGhykyiVpZUiQqWBf7vA2UncT5WYPPFhkU+Pr5HagsklxHKGfyUyRDMBTiPQZJ2dlJBCUZHeoqoAlIEAQGjdzZihHILH0YjflUYVdCeU6CEaVIKY1wkM/jeBYMA5SyhJQ6X3hutcoXiJXLOFqJaYS35SxPWGdwHJkK6LfWqbcVv2niEEiIFbZcgAG2qmb0Pbq0rDE+vuOFyKZKrn3Z/uS3SZDh1mnu5ZbxGQCUpl5V2c7+Acrav4J5fUFHu+C3pgBVXySwEXIQlaSsZqofu7ObxHhrxb3ACalehiX3c69zJEWrv9IbpNwJ0dbGJvUwLF2pJUEJKuurzssmYI8RJlmWFkgmd2WBlje9Veg/5G+ey5P1x+T+omx5E04gzkXdxX9s2GgFGL1BtlLnM/izxbhd+YmPH5FLNC8bm4dqLeTaDu/Voa+wAZQhy06Oyixrc6UiDabeJDT8nWsJRhIAaaeV4h5fIBL2kFZRQrHnRi/VVcdjqiGGNIWZJUjBQworIx7IolHGpGsolWpJDqQvpNZyWHefTwTWHZpsiqhQ0bTLe+GFyi5vOiXI27wdMee3byToO2wsuvb7NG91OHowTrVzQv/P1AQI0E+gEZkG0Xbic3qnXcfsk1StmT7jPlvZoeDUc/a2ZLmEwfph0t5dUdNiALgzEiVx7ofhqqqXmM0JZfzwcMEvvItXS9vBOrDvZkU4Mwl9UeCO15XnJlbedrTPItnKDV9nQFmkuYOUi8fl2+ywj/gimafj3W8LABs0FrGXQ8pnlms17uz/bAl+mJYyVQp+hpUmtKSc3OHu6n7Yvnx+haf4AofknIQ==
sidebar_class_name: "get api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
-
-
-
-
-
-
+
+
List Secrets
-
-
-
-
-
-
-
-
-
-
-
-
-
-
\ No newline at end of file
+
diff --git a/docs/docs/reference/api/list-workflow-revisions.api.mdx b/docs/docs/reference/api/list-workflow-revisions.api.mdx
new file mode 100644
index 0000000000..967aecd1bc
--- /dev/null
+++ b/docs/docs/reference/api/list-workflow-revisions.api.mdx
@@ -0,0 +1,2062 @@
+---
+id: list-workflow-revisions
+title: "Query Workflow Revisions"
+description: "Query Workflow Revisions"
+sidebar_label: "Query Workflow Revisions"
+hide_title: true
+hide_table_of_contents: true
+api: eJzNWE1z2zYQ/SucPbUziOR6euKpip00TprYtR334Hg0ELmUEIMEA4ByVQ3/ewdfNCVRlOzITU8ei9i3b98uFgssQdOpgvgW/hLyPuPiQcEdAVGipJqJ4iyFGDhTevzgv48lzpliolBAoKSS5qhRGoglFDRHiGFOJaOFHkvMgAArIIZvFcoFEJD4rWISU4gzyhUSUMkMcwrxEmixOM8sjF6UBkZpyYop1KT5pag4h/qOQIoqkaw0DCGG91fnnyK3OhJZJDEjEQ6mg2j5Be5x8QXiaE55hTUQ0ExzA3XjOEaXmEH9fEBDzkcddPlfh33pSb5E3DlqmlJNf0TwwfcTFPgY6O6SYRd2lxYZN9vqBwhhHT9BhbeW6C4JelFb8bMi4VWKYyqTGZtj+j0KTITgSItOCUIYZ85fNAr+avNVoipFoVAZ3OOjI/NnNbyrKklQqazi0aVfDAQSUWgstGVTlpwltgUOvypjs2xxLaVpkJo5D4monJFnyQqNU5QttU/siprAY++Ml8A05moTjlZ6JmRfURDIhMyphhiqiqW9Co0cmkkw1bgnqln6SrMce6FPDWBNIEel6LQXuw/mozevia+i58F8opbuaqKfB3XagqgJJBKpxnRM9UHlO3Gw0chWRlWmL+Hks4P1TlLk+AJOTh2sdxLkmizGLD1UGQexXi+is7St10G9BLUaL0Gwg3oJcjVe5ijVd5TrjTevCSheTZ8Lc2VsawKHC9RF547CLkgx+YqJbvXJMIOGQ2kd2jYbf2qvANI0ZWa7Un6x2kt36xC69Yq3fLL6S/ss6vYFCZNJxan86W3F+XslilfnlS4r/bOJbyVeA+J7/z5WVEq66NV5zdaIvulxe/ttjUEbwu7K1Kmz7EgUlZplNNEH3DgjD+m3TfCwH+Ng3M02XFkORzZcMPwWd/89TVwP0cV4I8Wb1mHEa5WDK6XNKdBMqilmtOIa4ttO9MfJ7dYPPHfbXapmqKprA/br8fHmDHZDOUvthBW9kVLI5w9gKWrKeM9IxUWy8vUpbeFuu3x/CEfQDkFq2hoBm3rpGHLcku1LrRjRtfm6KxEmLufar1upvyCvU3d7GKdOvr6aend9fbEB6HK7mtQ/zaAfhXqI2gWWo54J85wwRW2fDvQMYhiWZibGh+GEKhyGRwY1bCblIRBQKOfhjaGSHGKYaV2qeDhMuKjSAZ1ioemAsiEtGaxzGtmv0YlZareowqSSTC8s3uji7AMu3iFNUdriby24MgXnSmh1WZM9WrIPaMT0FyA3brN/XF3428/MWRm1WJEJax66kuP2mib3WJjrUjMLwNHgl8HRtmC8QTS6OPP7xjfCwMMuA7ImVyMUEMDcbhrQSPPfHj8YlqVQOqdFC68nr2sDtxdG4996WHLK7O6wLJY+57fgcw4ETNaBQJN3aN+RCJiCnglluhIsl2bxZ8nr2vzsrpQmgylTdMJbl8o1Rk1XedqLSVcc97jYeN+yFhCD7R7/OZe1V6fDkNnjNWUPRq33oMPQ2vW2sQen8C7TT8gZdTxmPJrd+TN9YrbxrTkkZqGFLL39KEmw1C2rjTPNoDRt8fc311DX/wJDY1ap
+sidebar_class_name: "get api-method"
+info_path: reference/api/agenta-backend
+custom_edit_url: null
+---
+
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
+import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
+import OperationTabs from "@theme/OperationTabs";
+import TabItem from "@theme/TabItem";
+
+
+
+
+
+Query Workflow Revisions
+
+
+
+
+
+
Query Parameters
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Successful Response
+
+
+
+
+
+
+
+ Schema
+
+
+
+
+
+
+
+
+ revisions
+ object[]
+
+
+
+
+
+ Array [
+
+
+
+
+
+ author
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ date
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ message
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ name
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ description
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ version
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ slug
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ flags
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ is_custom
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_evaluator
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_human
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ metadata
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+ data
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ service
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+ configuration
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+
+
+
+
+
+ artifact_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ artifact
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ name
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ description
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ slug
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ flags
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+ metadata
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+
+
+
+
+
+ variant_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ variant
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ name
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ description
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ slug
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ flags
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ is_custom
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_evaluator
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_human
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ metadata
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+ artifact_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ artifact
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ name
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ description
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_at
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_at
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_at
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_by_id
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_by_id
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_by_id
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ slug
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ id
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ flags
+
+ object
+
+
+
+
+ anyOf
+
+
+
+
+
+ is_custom
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_evaluator
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_human
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ metadata
+
+ object
+
+
+
+
+ anyOf
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ ]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Validation Error
+
+
+
+
+
+
+
+ Schema
+
+
+
+
+
+
+
+ detail
+ object[]
+
+
+
+
+
+ Array [
+
+
+
+
+
+
+ loc
+ object[]
+
+ required
+
+
+
+
+
+ Array [
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+ integer
+
+
+
+
+
+
+ ]
+
+
+
+
+
+
+
+
+
+ ]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/docs/docs/reference/api/list-workflow-variants.api.mdx b/docs/docs/reference/api/list-workflow-variants.api.mdx
new file mode 100644
index 0000000000..4470841c06
--- /dev/null
+++ b/docs/docs/reference/api/list-workflow-variants.api.mdx
@@ -0,0 +1,1092 @@
+---
+id: list-workflow-variants
+title: "Query Workflow Variants"
+description: "Query Workflow Variants"
+sidebar_label: "Query Workflow Variants"
+hide_title: true
+hide_table_of_contents: true
+api: eJzVWNty2zYQ/RXOPrUzjOR6+sSnKnbSKGljN3bSB0ejgcClhBgkGFzsqhr+ewc3mrrLjjyePmlGwJ49e7DYxXIBmkwVZDfwt5C3BRf3CkYpiBol0UxUwxwy4Ezp8X1YH98RyUilFaRQE0lK1CgtwgIqUiJk0O6UWEAKrIIMvhuUc0hB4nfDJOaQFYQrTEHRGZYEsgWQan5ROBw9ry2O0pJVU2jS9p/KcA7NKIUcFZWstgwhg/dXFx8TvzsRRSKxQIkVxTTB3rSXLL7CLc6/QpbcEW6wgRQ009wCxqCTT1hA86O4lmnQIIj0P5Dgi2f6bAqUqElONHkJGaLvw1X4M7LdJ8U+6A1KFNxetReQwTk+XIO3juc+AXaCdqJnFeUmxzGRdMbuMP8RASZCcCTVRgViFEPvLxlEf41dlahqUSlUFvf05MT+LId3ZShFpQrDk09hM6RARaWx0o5NXXNGXVXsf1PWZtHhWktbMzXzHqgw3iiwZJXGKcqO2GduR5NCW06zBTCNpVpH80o+LiGin4/WdvUsnwZ13oFoUqASicZ8TPQuwBQKIUu7B3Ki8ZVmjs92L2ceNhk4dUydP4eTzx42OMmR4zM4OfewwUmUazIfs/xAP8aw/CCxXs+TYd7V66heolqtlyjYUb1EuVovipvpU3P1yto2KRyPnufka/gmSDH5hlRveFvEcroK3aTQ9sYlQJLnzF4ywi+XysABOsQ6s+StnCz/062im30BZZIaTuRPbw3n75WoXl0YXRv9s41vKV4LEsrWIVZESjLfqfOKrRV93eM24277JlKzglB9xBQdBMiQoNHD4/IhgmxKibVg181Dl+7o4jVd6+P2qZFjQQzXkN1sxH7ovTehZY22OlRtV2wai/Xr6el6E/1COMtdi0zeSCnk0ztojpowvqMpckGXVh9zO0bbxftDeIJW31JNOz28zZeHZFOKTPHhJLZvdWIk13Z13znYuLzrsG/UPdkor1d3exjnXr5dCfXu+vpyDdCf7fKh/mVfakk7KnXSq0Q9E3ZCnKJ246CeQQb9WuIdw/v+hCjsx2lQ9eNLpw8pKJR3cWw0kkMGM61rlfX7lAuT98gUK016hPVJzWCV0cCtJmd2q7ugCqmRTM8d3uBy+AHn75DkKF3mdzZc2XTzCbS8rT07UrMPaKUM79eB0TMh2b8+K8LjdeatrFasKoQzjzXCc3tN6C1W9rVr4/S0T3q/9E62BRMMksHlMNyaUFkiD7cN0hW5WqEgBSzdlQGNpPztYcGyrIXSJak6eNtPdeWpGHTR+I/u15wwdzUciUU48RsIJw4p2DOHtP0GYPE6HwxsMs+EsgUJFgu797PkTWP/9vOAPb+cKTLhnYlghVBbUZ4y9W4K5hbn658tnAlk4ArIC1Fa/opwHEYHzMT7CXWG+uOw2jei7qcUp+vdfLzNhpn0wWwUMnZir/ONbRWzWEoWwX5AKda6Y7XW2SxKWx1/f3MNTfMfDcGeBg==
+sidebar_class_name: "get api-method"
+info_path: reference/api/agenta-backend
+custom_edit_url: null
+---
+
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
+import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
+import OperationTabs from "@theme/OperationTabs";
+import TabItem from "@theme/TabItem";
+
+
+
+
+
+Query Workflow Variants
+
+
+
+
+
+
Query Parameters
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Successful Response
+
+
+
+
+
+
+
+ Schema
+
+
+
+
+
+
+
+
+ variants
+ object[]
+
+
+
+
+
+ Array [
+
+
+
+
+
+ name
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ description
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ slug
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ flags
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ is_custom
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_evaluator
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_human
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ metadata
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+ artifact_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ artifact
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ name
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ description
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ slug
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ flags
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ is_custom
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_evaluator
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_human
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ metadata
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+
+
+
+
+ ]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Validation Error
+
+
+
+
+
+
+
+ Schema
+
+
+
+
+
+
+
+ detail
+ object[]
+
+
+
+
+
+ Array [
+
+
+
+
+
+
+ loc
+ object[]
+
+ required
+
+
+
+
+
+ Array [
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+ integer
+
+
+
+
+
+
+ ]
+
+
+
+
+
+
+
+
+
+ ]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/docs/docs/reference/api/list-workflows.api.mdx b/docs/docs/reference/api/list-workflows.api.mdx
new file mode 100644
index 0000000000..a54fd1239f
--- /dev/null
+++ b/docs/docs/reference/api/list-workflows.api.mdx
@@ -0,0 +1,668 @@
+---
+id: list-workflows
+title: "Query Workflows"
+description: "Query Workflows"
+sidebar_label: "Query Workflows"
+hide_title: true
+hide_table_of_contents: true
+api: eJzFV9ty2zYQ/RXOPjUziOR6+sSnKnYuTtrYjZ32wdF4IGIpIQYJBhc7qob/3lmApKirZdeZPGlE7J49exbYBRbg+NRCeg3/aHObK31vYcxAV2i4k7o8E5CCktbd3HfrDCpueIEODXkuoOQFQgqtxY3BHBjIElL45tHMgYHBb14aFJDmXFlkYLMZFhzSBfByfp4HHDevCMc6I8sp1Kz7UnqloB4zEGgzIytiBim8vzz/mETrROeJwZwlOJgOksUXuMX5F0iTO6481sDASacIqk0z+YQ51E9HJHbreeeKtPwJmYfAj8j9TSD6UPZ7UbflX6Djgjv+MyRoYz9ChT9bug8J8RB2TwtZZsoLvOEmm8k7FP9HionWCnm5VYs2mbMYLxm18WpaNWgrXVq0hHt8dEQ/qxle+ixDa3Ovkk+NMTDIdOmwdIFNVSmZhR4w/GrJZ9HjWhnqEE7GCJn20alhKUuHUzQ9zU+CRc1g2UXSBUiHhd2Ei1I+bm+0gT6S73o9nwZ12oOoGWQGuUNxw90+QAa5NgXZgOAOXzoZ+OyOchJhk1GQx1fiRwT5HGGbIAIV/oAgpxG2CdLKNZnfSHFgHO+lOEisV/PkTPT1etYorVpdlFawZ43SytVFscpPn7pXL8m3ZvB89CKnONG2QerJV8zclrbazpZ16JpBNx9WALkQkg4ZVxcrbeAAHdpGsxKtmKx+6bfR7bEgkybziptf3nil3ltdvjz3rvLuBeW3ki+BNG3rEC9uDJ/v1XnNl0TfjLjLeTnBNtw2CzMyTuY864eI9DZN6RIjMOdeOUivx9vQl4Psuun/4y043WypawL57fh4cxT9zZUUYdAkr43R5ulzSKDjUu2ZLEpnK6uP2WLj3bL9oSNBUraw094k7A7esmLW8ikua7DbNIiRXNHqQwWgvGLoxq5Xi6W8Ud3daZxG+fbtpXdXVxcbgLG2q0X9i+47SX8/Fehmml4TU3ThCeFmkMKwMngn8X444RaH3e1gCAwsmrv2heGNghRmzlU2HQ4zpb0Y8CmWjg+4HPJKwjqBUVhNTsg0nBmLmTfSzQPe6OLsA87fIRdowg7vGVzS7or7ZdWsKxWv5Ack5ZpL38i7mTby37gJmhvfLHqRNLLMdXBvVGy4veLZLZZ0RaQ8I+2jwa+Do13JNA7J6OKsOSR0nLsbU2MGbE2uTihggEU4IeCQF78vF4hlpa0reNnD2yzi2rWq0cPhdzesFJfhBITgi6bA19AUGBhQiaF/BWRAG3WmLXUZWCzI4LNRdU2f442ZiiWk5RPVuzOvsei6xeMehNu43+J88yEbXCCF0BaeTOahN9oBdNr35fMQOuDFdACn3ptvP63ot+V9tHQb0x8jyS8cStYeItoF0X+UZVi5ntfGfCCUrte8fX0Fdf0fACLBAw==
+sidebar_class_name: "get api-method"
+info_path: reference/api/agenta-backend
+custom_edit_url: null
+---
+
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
+import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
+import OperationTabs from "@theme/OperationTabs";
+import TabItem from "@theme/TabItem";
+
+
+
+
+
+Query Workflows
+
+
+
+
+
+
Query Parameters
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Successful Response
+
+
+
+
+
+
+
+ Schema
+
+
+
+
+
+
+
+
+ workflows
+ object[]
+
+
+
+
+
+ Array [
+
+
+
+
+
+ name
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ description
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ slug
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ flags
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ is_custom
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_evaluator
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_human
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ metadata
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ ]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Validation Error
+
+
+
+
+
+
+
+ Schema
+
+
+
+
+
+
+
+ detail
+ object[]
+
+
+
+
+
+ Array [
+
+
+
+
+
+
+ loc
+ object[]
+
+ required
+
+
+
+
+
+ Array [
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+ integer
+
+
+
+
+
+
+ ]
+
+
+
+
+
+
+
+
+
+ ]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/docs/docs/reference/api/log-evaluator-revisions.api.mdx b/docs/docs/reference/api/log-evaluator-revisions.api.mdx
deleted file mode 100644
index b4ff537bc8..0000000000
--- a/docs/docs/reference/api/log-evaluator-revisions.api.mdx
+++ /dev/null
@@ -1,71 +0,0 @@
----
-id: log-evaluator-revisions
-title: "Log Evaluator Revisions"
-description: "Log Evaluator Revisions"
-sidebar_label: "Log Evaluator Revisions"
-hide_title: true
-hide_table_of_contents: true
-api: eJztWd9v2zYQ/leMe9oAOU7TNt38tDRp0axpGyRp92AYwVmibLaUqJJUWs/Q/z4cKcmSbKu2agMZtjwkjsT7zvfdL/K4AINTDcMRvHpAkaKRSsPYA5kwhYbL+DKAIQg5vWfF+3vFHrjmMtbggWJfU6bNSxnMYbgAX8aGxYY+YpII7luMwWctY3qm/RmLkD4lijQYzjT9V2K3vCrV3vPA4sfzDyEMRwsw84TBELRRPJ6CB6FUERoYQpryADKvXBGnQkA29sBwI+hBaXPvJgfvXZIEfJPqSyjkt0Mo/SvHbupcWvqAimNsDmLoJ4fdtHP/Kksz6xqXVh7CuoZVB7DGaThAWDSiYf8OqfsBleEh+vtUcJZD5hoClpjZWmweGzZlqhXswkpnmVeskJPPzDewxusFc/pKTtdKUJHiigVU5palZl0AVaFuXGmDLMuqEEalzD7QiYy1K1Inx8f0J2DaVzyhkgdDuE19n2kdpqJ3ky8Gr2uF9GXqhBoULm04tyuI9hBTYWB4XMu2ZdEeLoAbFum2Wvt/BXrsFejfXx8wNTPX8fcC7tCo7qBhW6LS0r7hEWsvRQSYeRAxrXHait0G8y4XzzwIhd10VXHqmcj1vZ9qI6O1yiZSCoZxq7ZL3Tt3CJlHcLUtVlfEMsxz0FkaYfwzgG8swFZV/rXlbBXPOsZg/VtgEHCqqyiua7xu4be1zSma1J9UzVuvC3yu/FSg+uV1KsSfWsb9D6lJUvMrNI0lkLwibyOFSuG8ldiGLKzQ2x6mBq3ESog+JlLfyyuuzQ9p3WTkivhuHN25UIQYo87l4D3aslPfNHSDuqhAZB74iqFhwT2avZbBcwfbO7NkpUlwCCUfHWyuJGCCHUDJhYPNlRR0TeZ77HUFWS/nebcr+NqrloKtUktB2F61FHSVWh6Y0j8Rrp9y8cwDLdJpV5hbkqU+tDdDcw6x2U3qvXl/1qeKd0X5qLhDEN0RBCHMGAZM7VzpD0XJ4wuIlr3JDQuZYrFfqzbFFx3v1FHe5G4gCuwRcFuPPKKe/Cg3Ojv44DYnnnaUmCQ83nYD1KR4l91Wocg6npp45+h30pkHCSqMmOmQ1f/xWNksfL2klBzF1AP32Tbs0rRopxjMoTM7KQr5NHVj+EPoOq8p2GnUdoH2kLByFtsFo+IB5722ubyuTrZGuynS5fgts7O8Zycnq9O6Tyh4YJnovVJKqu6juoAZ5KJlzCakX3u7S/6NN5N2JX0sjgCRnq4WprWjCLdk81JLRu+O3v5otEp2OdX5utpwqaDXsbvZjAtHX5uH39zdXa8AOt/WnXolp731sRQxM5N0tZVITdAJ0qwaBgnNSdm3QTku0YNycjoQkkyj7Le1dbRw+y+YGZPo4WDgC5kGRzhlscEj5ANMuE04zfxUcTO3ImfXl2/Z3PV7G8qVBbYDuSCpLyv9gwl/y4gud/TMx178b+d5DziZ7fZ0lkIK1pvlHd2r7xglgq3cuW24ZYOnIf72PDx91n/+4smL/rPnpyf9ydPQ75/4v58+DU9PMcRT2HRdtq30+rHzzrq7CNdnwTur3EWoEzVdbKpNbrcVyi9rjilmeBxKG3LFaNUGdO/s+hIqZz84PnpydAzNlKstpgqKvq2gRbza1+A1MqfMGfJJZMsnGIbRH8sX9M0oWfOJY463OcMbc5U8fwz7bgaJQB5Xjk8u+UeQJ381LnTFc/SZSsDYgxlVjeEIFosJavZRiSyjx19TpijLx7nnJsTfaFwes2wB+MLmrvlSc+nbykrLReryu9FoqAk4iTPfZ4lpXTuuFLbrD7d34MEkv5mPZEAyCsk8+j0EoGt+U9wJ2WcLEBhPUzvlBodJP/8AqWvzVA==
-sidebar_class_name: "post api-method"
-info_path: reference/api/agenta-api
-custom_edit_url: null
----
-
-import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
-import OperationTabs from "@theme/OperationTabs";
-import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
-
-
-
-
-
-
-
-
-
-Log Evaluator Revisions
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
\ No newline at end of file
diff --git a/docs/docs/reference/api/log-query-revisions.api.mdx b/docs/docs/reference/api/log-query-revisions.api.mdx
deleted file mode 100644
index 48f890fc3b..0000000000
--- a/docs/docs/reference/api/log-query-revisions.api.mdx
+++ /dev/null
@@ -1,71 +0,0 @@
----
-id: log-query-revisions
-title: "Log Query Revisions"
-description: "Log Query Revisions"
-sidebar_label: "Log Query Revisions"
-hide_title: true
-hide_table_of_contents: true
-api: eJztWV9T2zgQ/yqZfbrOGEJpS+/ydLTQKXcUOKDcA5NhFHuTqJUlI8lALuPvfrOSbezYcUIuvXs5HoKjaH/7R7ur3fUcLJsYGNzCHylqjgaGAagENbNcyZMIBiDU5O4+RT270/jADVfSQAAa71M09oOKZjCYQ6ikRWnpkSWJ4KGj738zStKaCacYM3pKNKFb4jSYwyLuig13PHIc5Ox8DIPbOdhZgjAAYzWXEwhgrHTMLAwgTXkEWVDukKkQkA0DsNwKWiB1Z73LHLh3QrvhB/BZ4OD1eWCaM2m3rs6Nx815bZ9LHd/rsm0dPDbTlo9ZuE3hD3PInEOEiZ22YnNpcYK6E+zIUWdZUOxQo28YWljQpjh9c6omrbspjLjGiCJwMRgWTVMFu/ThB1mWVUGsTtEtmERJ40Nof2+P/kVoQs0TCksYwFUahmjMOBW9y3wzBJtGcahST7RgwGf5P7odZPQxS4WFwV7pP7Xg5xbjpVng/6j576OGpXaq9NbAPRpFI7O4Jipt3bE8xu4AJcAsgBiNYZNO7C6YLzl5FsBYuJuyisOiiFOEMHFR89cGp5FSApmssiqYDxtZoUueT04Ip5dlGwrTVLs148Wj+kpVh3ZeEHIdpoLpnz6lQvxmlNw5T22S2lfQpmMe7OtQMa3ZrNswdVp4kVW/kDGJYvMT/heMeqZOubErzbpMyQb5y2x0nTueZPHG0XTGXNTW76LNoI4qEFkAoUZmMbpjdqtZ5KOH7R06Y6VJ9COYfPWwOZMIBf4AJkceNmdSmGu0zZuoMNaH4joq7LVVLoW1Si6FwbbKpTBXyeUBtfkH7nqTk2cBGJFONoW5ItosgO0pmtuQLd4m9QLskctIPRJ2xyaJj1SSbtNrzzxkFoAS0bbBzz0k5TR8Whd6pUHPCCsLQPCYt4Ou01+cOmrSW0e4ot5CmcbUPzATooz8GmXY/Muw0wYOnjxKWtQPTGws8UkBQP3HsmqueQE2G2ZXuHU0Vn+WvtiEoTKNC4t6hav6CYcvZMueBJiMnm/Upnndr4raGqlIIOlXpNLVVu1UTXjIxHnBIXNtlb/Z6z3OMuHGHEVUaadKScpax23IAviOs00zye84892OSNe9zl9c0BSaZi11XHuxQ/3rS4qSGyd+FtQOdHWkcJpgcXNH51i9wFScMM2NkpXTW46C93T67nNC/iCse0L3SJ8j+yir+GdpjJqHa4Eby7Q1j9xO3WJUPFKLzrgbwcXMhlOkJ8G/Y5XRlUNbiw+X3p/veE1UKhXXop8yEmDKGsY84uF6CPjEjTW5FPmXCs4xraAMsQ2smc+KTZVZAx23c5EyBJdFXsgM3hmUhlv+0B4WLZ1csxBiBntXJUxFlDEThgY0+ERdujvAjbkcE0bvi8NosOjIn9f4ZM9zWxB+3QRMtF8B60h0KMSLJPFO9izJ8kPN96wYoPnEWQvoPPG6LFy2mcUN8QqGi6mphdZUdboddilUIrddTevOCo+Ya0Y3poflOtWH3i9QrD5/LCeGmRs/vt3fbw4Yb5jgkRsf9o61dgG54XQxQsu46JgOChUuuVdXX2UdDnCqvIBu2mImXRdyZUrktyzf6ozRu6ZfV7kz6eVZ5/vqF19uXm/d5WocefN1ne7n6+uLBqA/2/qhnqpJr+lDMdqpojdFiTIEmzAarEM/odEuPvbv/Zulfjnp7QtFOhnU1FW5k0q1gAFMrU3MoN8PhUqjXTZBadku432WcJcQDIap5nbmSA4vTn7H2WdkrkC+HVY3XJEvee+obysPhiWcSqBinpGPIvlf/sipHiZ5PBUZgrz08vmd1/ETixOBS95htby1gjdj9vO78cHbnXfvX7/fefvuYH9n9GYc7uyHvxy8GR8csDE7gIVXUOsSNefj61JuQvM8wV6XojaXXpcof0Gzl7nuZKzc4RU3jXON3uHFCVRac9jbfb27B4teW9vs6qfQJaHi5N3PECz4YOl9VKrELgOBRRb/+vwDSUY+HzNZwWsPkoWRV+6FFp9sPxGMuyzjBJjn8XMLefzkFudoKu7hqj41oZQwpagb3MJ8PmIGv2qRZbTsToniIj/kERnulvLdtAiZuW8g6KqjxLzjslLZEzSTNCVQT3EYhpjYzr3DSmK4OL+6pno4f1kcq4hoNCPd6HMAUKvP3NocBJOT1A3vwWPS399UmGkU
-sidebar_class_name: "post api-method"
-info_path: reference/api/agenta-api
-custom_edit_url: null
----
-
-import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
-import OperationTabs from "@theme/OperationTabs";
-import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
-
-
-
-
-
-
-
-
-
-Log Query Revisions
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
\ No newline at end of file
diff --git a/docs/docs/reference/api/log-testset-revisions.api.mdx b/docs/docs/reference/api/log-testset-revisions.api.mdx
deleted file mode 100644
index 96473caed6..0000000000
--- a/docs/docs/reference/api/log-testset-revisions.api.mdx
+++ /dev/null
@@ -1,71 +0,0 @@
----
-id: log-testset-revisions
-title: "Log Testset Revisions"
-description: "Log Testset Revisions"
-sidebar_label: "Log Testset Revisions"
-hide_title: true
-hide_table_of_contents: true
-api: eJztWd9v2zYQ/leMe9oAOXbzq5ueljYtkjVrgsTNi2EEZ4mS2VGiSlJZPUP/+3Ck5NiS7DiuPQxD+tC60t13vI93x9NxBgZjDf4QBkwbzYyGkQcyYwoNl+llCD4IGT8Y9/ZBsUeuuUw1eKDYt5xp806GU/BnEMjUsNTQT8wywQOL0PuqZUrPdDBhCdKvTBG+4UzT/0rklS/mJh94aLHT6XUE/nAGZpox8EEbxdMYPIikStCAD3nOQyi8uUSaCwHFyAPDjaAHpa+d2xK6c0nysAdLNQuVT4+oOKZmDy7dO+TS3u7tLOOjMjzCYJcGzkrI0kLIMjNpxeapYTFTa8HOrXZBzDsJOf7KAgMN3q5k3CpGMc4VCylDqkhtsn4l41uXDFAUxaKWUTmzD3QmU+0C+7Dfp39CpgPFM0oS8OEuDwKmdZSLzm0pDN62ORXI3CnVqHpa+HsrQfRGmAsDfn8hOp+S3J8BNyzRq7Pz/xLJlT+79+PfyhTMzUSqnYE7NMpANGxDVBLtGp6w9UlJgIUHCdMa47XY62D+KNULDyJhD7FFnOVonaC2R1iAZQ42DI6lFAzTtRYvUHcGc5TCc7AKgx/FdBAblKmP1tEmnmXT4PIqMAw5lQsUN0tkbEB2a21NxstPFt1rtwUBV0EuUP30MRfidy3T7nVustz8DHVXCaQsNZtooVI4XUtsTRca5K6PLYNWoxFX/yVSP8srrs2ztK5ysqH+Mo4GLhQhxWTrHP6MtlYsn4XbQZ0vQBQeBIqhYeEDmp3WrvcOtnNmycqzcB9GvjjY0kjIBNuDkXMHWxqp6BpPd3hAVWS9m5ZHVMXXTq1UbM2tVITt1EpF19zKI1P6B8L1vlQvPNAij7eFuSPdwoPdOVpyiPXTpNn+0TH4wMNa+Zv3i88tYfNSXp25nctQV71ay0n+TKe6r85ut9h3c9zXCvZawfZawXZdM1q68I27pZZWuap+oxd1JVWH/Nq6bdC6vX4z7OGboXl0vpK6NannROZzH8bUC2zVUGz0zV2NcN1Smp/emyM01rh6Iq0XZ3PDlxjR8/FhYWeRx4eHzWnjPQoe2lli54NSUm0/agyZQS7WDAqFDJbeviToR6sJu5JugbaM6bit4WwZFDmR1aKWjM6A3j43DSa/nOlSbmm0WNHr2F3txrmjb93+XgwGNw1At7fLm3ol405bHCXMTCRd5GRSE3CGNFGHXkZzXvZXr2yQdW8+9+0JSW5ppugrx25WrgT4MDEm036vFwiZhwcYs9TgAfIeZtwmmGZBrriZWpWzm8tPbHrBMGTKBvGCwB2FkwuQZbH53mDGPzGiys0XyoEk/9vtugecXJ44LeKCAvX26Ubqw3dMMsFqN0ytd0pwFOEvJ9Hpcffk7Zu33eOT08Pu+CgKuofBr6dH0ekpRngKteuhTZXa5uSb6m6jszRn3lSpvGTpE488jaSlqhoE203unN1cwsJHL/QP3hz0oR6CS8JUUTCwvFd7aF+DV4umeRyBByyx5QQMw+S3pxe0MgrfBNMFvFURXxsnlRFl2HfTywRyWzTsEmZlMgyhTIanHdMLO06/KSVGHkwoh/whzGZj1OyLEkVBj7/lTFHUj8pdGxN3Q6pfkyr+Z/Anm9qrF1tou7bKkLjIXbzXii4VRKdxFgQsM2tlRwtpfnN9NwAPxuW9bCJD0lFIztHfPgBd8Zrqhsc+m4HANM7tPB4cJv35B+MYQPk=
-sidebar_class_name: "post api-method"
-info_path: reference/api/agenta-api
-custom_edit_url: null
----
-
-import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
-import OperationTabs from "@theme/OperationTabs";
-import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
-
-
-
-
-
-
-
-
-
-Log Testset Revisions
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
\ No newline at end of file
diff --git a/docs/docs/reference/api/log-workflow-revisions.api.mdx b/docs/docs/reference/api/log-workflow-revisions.api.mdx
index 600c4f676d..2d29bca338 100644
--- a/docs/docs/reference/api/log-workflow-revisions.api.mdx
+++ b/docs/docs/reference/api/log-workflow-revisions.api.mdx
@@ -5,67 +5,2214 @@ description: "Log Workflow Revisions"
sidebar_label: "Log Workflow Revisions"
hide_title: true
hide_table_of_contents: true
-api: eJztWW1P20gQ/ivRfLqTHEKhpXf5dBRalSvXIqDchyhCE3udbLv2uvsCzUX+76fZtY2dFzdJEwnpjg8Q7JlnMs+87O7sDAyONfQH8LdUX2MhHzUMA5AZU2i4TC8i6IOQ4/vH4vW9Yg9cc5lqCECxb5Zp80ZGU+jPIJSpYamhj5hlgocOovdFy5Se6XDCEqRPmSIDhjNN/5XQq99URu955NDT6acY+oMZmGnGoA/aKJ6OIYBYqgQN9MFaHkEeVBKpFQLyYQCGG0EPSn871wV254IUYA+m5ixUXj2g4piafTh156ELi7s31MSvPNqDJ94CKsNjDHfpwmkBWViIWGYmS7F5atiYqVawc6ed50EpIUdfWGhg0aEyGfSlHC9VoKLiikVUlFVpLGGmDnTtCxHyPK8DGGWZe6AzmWpfU0eHh/QnYjpUPKMChT7c2DBkWsdWdK4LYQi2redQWq80x9+TC2dOgjiP0QoD/cN6Ej11mP4MuGGJbukM/9fQM6ohtGYi1c7APRrVJhq2JiqJdg1PWHu5EmAeQMK0xnErdhvMX4V6HkAs3Dpax2lmLNf3odVGJkuNjaQUDNNWaxe6c+YR8oDg2AMKi2YF4Wsivq1APOjEJpj+DOB7B7BOJ3znKFuEc3Ex2PwSGEWcug+Kqwata4Rtaf9ORs0nde+W24KQq9AKVL+8s0L8qWXa/WRNZs2vMO8rgRSNax0tVAqnrbzO6cICu+1ZatBpLGTocyL1o7zk2vyQ1lVOLqhvxtGtT0VIMdm6G3xE13WaS+t2UOc1iDyAUDE0LLpHs9MueOZhO6eOLJtF+zDy2cMWRiIm2B6MnHvYwkhJ12i6w6WuJOvNtFjsSr52aqVkq7JSErZTKyVdlZUHpvRPpOtdoZ4HoIUdbwtzQ7q0DO3M0YJDnF9Nmkvz7ry3im+L8llxjyC2RxCEMGEYMbVxp98XJc8vIVq2JtcsZoqlYaPblF90uNGK8r4IA1HgDkrrRuQZrcnPcqOzQQxuCuJpR4lZxtN1N0DzFG+y2yoNucDTIr519nvtPIAMFSbMbFHV//FcWa189UQpBYqpBx6yddilmcpGOVhA526eEvOx9ZPVfdg6axjYZBp1ju6MsHAU2wCixr+PXcusVdeHP4ONzOhqQJW7adfLo6PFedYdCh45FjpvlZJq+2FWxAxy0TKJEjJsvN2k9oarKbuUIZbb/0SPF5vS0imEF1kt6sjo3NLbH40eyS9vupBrTK5Kej27q9049/S1Bfj97e3VAqCPbTOol3LcWZpJCTMTSRcVmdSEnCFNcqGX0SSRPfbK0ZruVaPFnpDkGNW966qDmd95wcSYTPd7vVBIGx3gmKUGD5D3MOOu1DQLreJm6lROry4+sKlf6V0e1wTc2uNTpClWRQcz/oERWf7QWcy7+D8+7gFwctrv5hyBlKrXTzcub79jkgk2f4Oy/M4EjmP87VV88rL76vWL192Xr06OuqPjOOwehb+fHMcnJxjjCczdfqyrtHQYu67yNjqNWem6So3x57pKxa3AIfHP01i68JXzSZccndOrC6idoODw4MXBIcwnb0OYehGGrheVsXevIZjLwir/IACWuEYEhmHyx9ML+maU98XYrsBbWStzw4kiFQ37bnqZQJ7WziC+jAZQlFGNd13LFPpMxTQMYELl1x/AbDZCzT4rkef0+JtliuplWAR7ROwNhtVRxZXSVzb1Cxg16a7rUCQurK+UuYZNzdRrnIYhy0yr7LDWIa4+3dxCAKPixjKREekoJO/odx+Abj9Nef3gns1AYDq2blAMHpN+/gVvRfqO
+api: eJy1WFFv2zYQ/ivGPW2AHKdp6256mpu0aNZuCZK0ezAC4yydbLaUqJJUUs/Qfx+OpGzZkd00c/OQIBTvu+N3H49HLsHizEA8hn+U/pJJdW/gNgJVkkYrVHGeQgxSzSb34fNE050wQhUGIihRY06WNCMsocCcIIaUSjuHCEQBMXytSC8gAk1fK6EphThDaSgCk8wpR4iXgMXiInMAdlEygCgszUhDHa2GikpKqG8jsMJKHjhzTmoeYmgy9rVKFwyXqMJSYR1yWUqRuIUMPhtV8Njab6l5mVaQ4f/uUAss7ERT1hmUmn6mxMI6givKSFORUEecNUfleToQYB3ttOOFT3bkqK5rF4spVWH8Qk+Oj/lPSibRomRuIIbrKknImKySvaswGaKnUpmoyhtt5XMd8qmb0WLJ2QlLuXkIh5WdK93JobFaFDOIIFM6RwsxVJVI9wpn5NHqCFK09EhUntq3Iu9KTUuTDFhHkJMxONuLvQ/mr2BeR2FHPQ3mb3Thbib6aVBnLYg6gkQTWkonaA9K36mH7Y2cMqoy/RlOPnrY4CQlST/ByZmHDU4auqaLiUgPJeOGrNeL3nna5uugXhq2Vl4awg7qpaFr5eWOtPkfcv0UzOsIjKxmT4W5Zts6gsMt1K8uk+7IfcyR0BzKb51J5zmTk8UU7dZJimkqeLuivNyspd/nofP0zaebI1OlJGHhhrp9QSJ0UknUv7ytpPzTqKJ/Udmysr/C9lHGIKH2P8YKtcbFXp63bOHB4bm//AZC/RHR3aLszNSZt+xIFGorMkzsATfOKECGbdN4eFzEjXF3tE07dLhgP3nEZov7/36M3ADxg/1RY30Veo2WHLyU2i3YurlNKcNKWojHnejrhnYcGp7b3S7NqqnyDdmLk5OHPdgnlCJ1HVbvjdZKP70BS8mikHtaKqmSja8/UhZud9P3QfkAXRNkZq0WcKWXjibHT9k91ZHRu+Gv30sEr8u7DvM29NfQ69ndvYwzT98+Tb27ubl8AOhzu5nUD2rWa9TQa8srJztXfLsqlbHuLmXnEMOg5JaY7gdTNDRoOnozWDXKA6l4dYb0XXPvqrSEGObWliYeDBKpqvQIZ1RYPEIxwFLAdlQj97V3ylPdJjWUVFrYhcMbXZ6/p8U7wpS0k39rwjVLzotoc9oqf1iK98R0huugb7jFv14Z4Vo491bMF4v5an2Fe/MN81JSx5WMSxE8z/C3l9nwRf/lq2ev+i9eDk/60+dZ0j9Jfh8+z4ZDzHAIzbm/1tKqo1gPobVaTCvrFlN33Nh+rj9uLIpMOeKaiuyz8hqTL1SkGyjHR8+OjnelMRj0RpfnoWaEQ6DJgJsG0ZZQVhKBCCh3BQMsYf7H+gNHyfrMsWjh7dT01lUjCMLSNzsoJQpXF1wMyyD3MQS5QwQseIhgJXlo3w4jfoTg3Tzn3RKPYbnk+R+1rGse9s8MLN5UGJzK1kPDF1q0HiXuUFYclFO9E9iUuR9zVZs3il8Go1Nffvuu9qxtH5RiLpPeYpQkVNq9c29be//y4vqGlx6eLnKVso1G5oN/xwD8GGObG7IbW4LEYla5GyZ4TP75D6h2D5I=
sidebar_class_name: "post api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
-
-
-
-
-
-
+
+
Log Workflow Revisions
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
+
-
\ No newline at end of file
+
+
+
Query Parameters
+
+
+
+
+
+
+
+
+
+
+
+
Body
+
+
+
+
+
+
+ variant_ref
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ slug
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ version
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ attributes
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+
+
+
+
+
+ revision_ref
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ slug
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ version
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ attributes
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Successful Response
+
+
+
+
+
+
+
+ Schema
+
+
+
+
+
+
+
+
+ revisions
+ object[]
+
+
+
+
+
+ Array [
+
+
+
+
+
+ author
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ date
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ message
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ name
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ description
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ version
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ slug
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ flags
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ is_custom
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_evaluator
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_human
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ metadata
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+ data
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ service
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+ configuration
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+
+
+
+
+
+ artifact_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ artifact
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ name
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ description
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ slug
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ flags
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+ metadata
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+
+
+
+
+
+ variant_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ variant
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ name
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ description
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ slug
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ flags
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ is_custom
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_evaluator
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_human
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ metadata
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+ artifact_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ artifact
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ name
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ description
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_at
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_at
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_at
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_by_id
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_by_id
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_by_id
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ slug
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ id
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ flags
+
+ object
+
+
+
+
+ anyOf
+
+
+
+
+
+ is_custom
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_evaluator
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_human
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ metadata
+
+ object
+
+
+
+
+ anyOf
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ ]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Validation Error
+
+
+
+
+
+
+
+ Schema
+
+
+
+
+
+
+
+ detail
+ object[]
+
+
+
+
+
+ Array [
+
+
+
+
+
+
+ loc
+ object[]
+
+ required
+
+
+
+
+
+ Array [
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+ integer
+
+
+
+
+
+
+ ]
+
+
+
+
+
+
+
+
+
+ ]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/docs/docs/reference/api/mark-variant-as-hidden.api.mdx b/docs/docs/reference/api/mark-variant-as-hidden.api.mdx
index d097a92669..8611cbb5db 100644
--- a/docs/docs/reference/api/mark-variant-as-hidden.api.mdx
+++ b/docs/docs/reference/api/mark-variant-as-hidden.api.mdx
@@ -5,73 +5,298 @@ description: "Mark a variant as hidden from the UI."
sidebar_label: "Remove Variant"
hide_title: true
hide_table_of_contents: true
-api: eJzVVMtu2zAQ/BViTy3A2mnQk041UAM1kqBBkubiGMFGWltMJJIlV0FcQf9eLCX50TQFeuzJMjmzj9nhtsC4iZAt4RaDQcsRVhoKinkwno2zkMEFhieF6rkHKIyqNEVBVq2DqxWXpL4vJnf2zs7CpqnJcszurFJKoff3I+vDBzXzfkii2KlAtXsmoV2hiTRyvt7cXM5fckrJM7VYS4JAykSFygf3UFHdc43dpOTo/VgbaHCeAgp3UUAGNYansYR7jPd94aDBY8CamIL03oLFmiCDEWkK0GCkd49cgoZAPxoTqICMQ0MaYl5SjZC1wFsvzMjB2A1oYMOVHIydLgroupVEiN7ZSFFIpycn8nMs83WT5xTjuqnU1QAGDbmzTJYFjt5XJk/NTR+jcNp9IV3XdRo+nZ6+DnyLlSkSTc1DcOEfooIPIiibvu6CGE0lX4apjq8BlcuPbtFuv62Twsc6dXp3YizThgJ0q06PZxgCbg/EPHd9gdBpqOPmb7pfUIy4IdgFexuaxFA3ctvtc7uHR8r5aObL1FefesCtDic9ytur+3YbX3r5/pRshIj9XwXsZ1sTl05MXVBFTMnEXEIG08G2cdruDdyBhkjheXR4EyrIoGT2MZtO88o1xQQ3ZBknaKboDYhLI+VNMLxNlNnl4oy2XwkLCpAtV4eAa3FIP/Nj2E5u9OaMpPvhbc0aLl0wP/tBDs+r7FnSnrFrl+iDErNUnJpdLkCDtNGb+WTycXICv6+oI7CYG/Nk7jF3ugb9mwq7/kED1cnZwIT15/2FVOZd5BrtQbyrtLvU7W7rHFXT7p/X/7U7h8kxvfDUV2jSe0uStYPXluOKjKAhO1iXKw2liyyItn3ASN9D1XVy/KOhIH5a6Z77INNdtlCYKN8FZGusIv1Fw3dXw0t8r94qcvScFcM9Y9XIP9DwRNvjtZ62TDlauh0As1xEO6C+Wori/d0D/DI/n9/Moet+AfNTesA=
+api: eJzVVE1v00AQ/SurOYG0JKXi5BMBIhHRiqotvaRRNbUn8VJ7d9kdVw2W/zuatZ0PSpE4coqz+958vH0zLTBuImRLuMFg0HKElYaCYh6MZ+MsZHCO4UGheuwBCqMqTVGQVevgasUlqW+Lya29tbOwaWqyHLNbq5RS6P3dyHrzRs28H5IodipQ7R5JaJdoIo2cz9fXF/OnnFLyTC3WkiCQMlGh8sHdV1T3XGM3KTl6P9YGGpyngMJdFJBBjeFhLOEO411fOGjwGLAmpiC9t2CxJshgRJoCNBjp3SOXoCHQj8YEKiDj0JCGmJdUI2Qt8NYLM3IwdgMa2HAlB2OniwK6biURonc2UhTS6cmJ/BzLfNXkOcW4bip1OYBBQ+4sk2WBo/eVyVNz0+9ROO2+kK7rOg3vTk+fB77ByhSJpuYhuPAPUcEHEZRNX3dBjKaSL8NUx+eAyuVHt2i3X9dJ4WOdOr07MZZpQwG6VafHMwwBtwdinrm+QOg01HHzN93PKUbcEOyCvQxNYqhrue32ud39d8r56M2Xqa8+9YBbHb70KG+v7sttfOrl+1OyESL2fxawf9uauHRi6oIqYkom5hIymA62jdN2b+AONEQKj6PDm1BBBiWzj9l0mleuKSa4Ics4QTNFb+D3sZ+lW/VRoCAWjpQ3wfA2xZtdLL7Q9jNhQQGy5eoQcCX26Q1xDNu9BXrzhUSaYfBmDZcumJ/9Kw+zV/Ys6d3YtUv0Qaahtg+YP5CVYZU++7JPJm8nJy81MxDU7GIxTAHmaQrGOhIM9G9y7YQCDVSnEQAmrN/vL6RK7yLXaA/iXaYlp2526+moqnY/h//Xkh1ekemJp75CkwYzSdYOplyOuzSChuxgr640lC6yINr2HiN9C1XXyfGPhoJ4a6V77r289LKFwkT5LiBbYxXpLxq+uhxG9rV6qcjRf1bM94hVI/9AwwNtj/d/WkflaO92AMxyEe2A+mx7yhzsJvXT/Gx+PYeu+wWvvYlP
sidebar_class_name: "delete api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
+>
-
-
-
-
+>
Mark a variant as hidden from the UI.
Arguments:
- app_variant -- AppVariant to remove
+app_variant -- AppVariant to remove
Raises:
- HTTPException: If there is a problem removing the app variant
+HTTPException: If there is a problem removing the app variant
-
+>
-
-
-
-
-
-
-
-
-
-
-
-
-
-
\ No newline at end of file
+
+
diff --git a/docs/docs/reference/api/mark-variant-revision-as-hidden.api.mdx b/docs/docs/reference/api/mark-variant-revision-as-hidden.api.mdx
index f213a24683..db85bb0196 100644
--- a/docs/docs/reference/api/mark-variant-revision-as-hidden.api.mdx
+++ b/docs/docs/reference/api/mark-variant-revision-as-hidden.api.mdx
@@ -5,74 +5,308 @@ description: "Mark a variant revision as hidden from the UI."
sidebar_label: "Remove Variant Revision"
hide_title: true
hide_table_of_contents: true
-api: eJzdVU1v2zgQ/SvEnHYB1soGe9KpBmIgRlNs4Ka9OEYwkcYWE4lkScqoK+i/L4YSbTlpCiwW6KEn2+SbD755b9xBwJ2HfA1f0CnUwcNGQkm+cMoGZTTk8BHds0CxHwDC0V55ZbRALypVlqTF1plGhIrE5+XsXt/rudu1Deng83sthBBo7UMKf/dOzK0dq4lghKPG7GkAptwPqmTgKpVaXk2R93qFylPKfn13d7v4VlDsNxfLLbfiSCgvUFhnHmtqhlild7FNtDY9ByQYSw45dllCDg2659Tsw7Ef9A/DW0GCRYcNBXLMWwcaG4IcUogqQYJi3iyGCiQ4+toqRyXkwbUkwRcVNQh5B+FgOdIHp/QOJAQVaj5I5CxL6Ht5rDAh53+XODHLNTacwlujPXmOury44I9zGXxqi4K837a1WI1gkFAYHUgHhqO1tSoik9mT55ju1Enf972Evy8vXyf+grUqY5hYOGfcf8gK1vH0ghr6LimgqvmbCtT414DaFGe3qA//bOMUz4li1scTpQPtyEG/6WU6Q+fwMGHzxgwNQi+h8bufEf+RvMcdwTHZ29BIhrjj2/5U2zw+URHOhr6O7xpKj7jNVE2J3oHdt59xNdD3o2IJwl57lXCYbUOhMuygkmoKFI0SKsghG63hs+5kkj5LcvZZN1F2DxI8uX1yV+tqyKEKwfo8y4ratOUMd6QDzlBlaBWwej0VrVPhEEPmt8sPdLgmLMlBvt5MAZ9YOYMWzmHHMaBVH4hZGV03b0NlnPo+DHj0XTVE8bOV3poYPjI0j82J+e0SJPAzBpFfzP6aXcDL1XoGZtFjEUWfasdrkC9YOL4fJFATFQ+BsHl/uuDOrPGhQT3Jt4oLVKT9kpbAy7a6k/9+++U/Tj3Qt5DZGlX0cKS7G/W7Tqvdg4T8bM0fJcw30/W8kVAZHzi46x7R02dX9z0ff23JsUw3ckj7yKJZd1Aqz99LyLdYe/rJRP5Yjcb/U7zVf5KyZh3vsW75F0h4psP5PxXvuV9YeUpR3KdVMmk3IuYFj3IS+2r9s5uPq+ZqcbO4W0Df/ws+jAPM
+api: eJzdVU1v2zgQ/SvEnLoAa6XBnnRad2OgRlts4Ka9OEYwkcYWG4lkScqoV9B/L4YSbTnZBCgK7KEny+Sbrzfzhh0E3HnI1/AFnUIdPGwklOQLp2xQRkMOH9E9CBT7ASAc7ZVXRgv0olJlSVpsnWlEqEh8Xs5u9a2eu13bkA4+v9VCCIHW3iXz16/F3NoxmghGOGrMngZg8n2nSgauUqjl1RR5q1eoPCXv725urhffC4r55mK55VQcCeUFCuvMfU3NYKv0LqaJ1qZyQIKx5JBtlyXk0KB7SMneHfNBfzfUChIsOmwokGPeOtDYEOSQTFQJEhTzZjFUIMHRt1Y5KiEPriUJvqioQcg7CAfLlj44pXcgIahQ80EiZ1lC38tjhAk5vxzixCzH2LALb4325Nnq8uKCf87H4FNbFOT9tq3FagSDhMLoQDowHK2tVRGZzL56tulOmfR930v48/LyqeMvWKsymomFc8b9hFewjrsX1JB3SQFVzV8qUOOfAmpTnN2iPvyzjV08J4pZH0+UDrQjB/2ml+kMncPDhM0PZkgQegmN371E/EfyHncER2fPQyMZ4oZv+1Nsc/+VinDW9HWsawg94jbTaUr0Duw+X8bVQN9/BUsQ1toTh0NvGwqVYQWVVFOgKJRQQQ7ZKA2fdSeR9FkaZ591k8nuQYInt0/qal0NOVQhWJ9nWVGbtpzhjnTAGaoMrYLH62oeb8XfDAUebU9F61Q4RH/z6+V7OrwjLMlBvt5MAZ94rIZBOYcde4RWvSembJTkvA2VcerfofujKKvBijlRemui+UjfmNtbLB5Is4q5ziHti9mb2cVzxYwGYn69HNWBRVRHyiPCQD6i60gUSKAmSgMCYfPX6YKztMaHBvXE3ypuWpEWUdoWj9PrTkL97V+JcQICfQ+ZrVFFsUe6u3HQ1+kN8CAhP3sPjrPON9M9vpFQGR/YuOvu0dNnV/c9H39ryfHIbuTg9p4HaN1BqTx/l5Bvsfb0QkdercYN8Yd4Lv801ppneo91y/9AwgMdzp80Xoj/Y+QpRXHxVkmw3YiYF9zKie2Td4KVfdxJV4sPi5sF9P0PWesSWw==
sidebar_class_name: "delete api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
+>
-
-
-
-
+>
Mark a variant revision as hidden from the UI.
Arguments:
- app_variant -- AppVariant to remove
- revision_id -- Revision ID to remove
+app_variant -- AppVariant to remove
+revision_id -- Revision ID to remove
Raises:
- HTTPException: If there is a problem removing the app variant
+HTTPException: If there is a problem removing the app variant
-
+>
-
-
-
-
-
-
-
-
-
-
-
-
-
-
\ No newline at end of file
+
+
diff --git a/docs/docs/reference/api/remove-trace.api.mdx b/docs/docs/reference/api/remove-trace.api.mdx
new file mode 100644
index 0000000000..09926b99b6
--- /dev/null
+++ b/docs/docs/reference/api/remove-trace.api.mdx
@@ -0,0 +1,324 @@
+---
+id: remove-trace
+title: "Remove Trace"
+description: "Remove Trace"
+sidebar_label: "Remove Trace"
+hide_title: true
+hide_table_of_contents: true
+api: eJydVU1v2zAM/SsGTxsgJF2xk0/L2gAN2qFFm/USBIUqM7FaWVYlOVtm+L8PlGXH+eiG7WRbfCSfHj9cg+drB+kC5pYLqdewZFAatNzLUs8ySMFiUW7wyVsuEBgYbnmBHi151aB5gZBCsD7JDBhIDSkY7nNgYPGtkhYzSL2tkIETORYc0hq43t6uQgS/NRTBeUvpG9afSO1xjRaaJQMvvaIzYonJLIOGTi06U2qHjiKen53TI0MnrDREH1J4qIRA51aVSu4jGBiIUnvUPvAwRkkRbjt+ceRTD1gaS1p42WbYoHWyheyT3vF7jJCGklRtioPr7MAXAdEwUFK/upOqcGv5diiKrpTaU+Qm+DYN6xDl8wsKvyf+oqfe0RoEuJ2jCkF6gZqGwn0+P6HnI1cyC2olU2tL+/9iZui5VPQmPRbuGKBKsWf9p47ZydEqONCrbAmS7oVb/6mY39A5vkbog70PDWIkc7L+rRR0rzZ1xA1qsZO3Vff9a1y28p1K1kGu5vO7o4BtbfeLeh8mPJnHCS/Q5yVNfoYKfTvzPocUxsbiRuKPsW93RXiiG9fd9DfAwKHddMuhsgpSyL03Lh2PhSqrbMTXqD0fcTnmRsIhlUmwJhcEDW3uUFRW+m2IN7mbXeP2CnmGFtLFcgh4oD5rO2cftpslI6+RNIw7a1L5vLTyV9sOcXHlrReJJPWqDO5Rz8jtKxevqGnT9fsAzkafRmfvXSY6JJO7WRwXLsK4dDwCDNiBXL1QwACLMCvgkRdfdgZiaUrnC64H8Q7KucepF8PjTz82isswCCFzHeu8gFhn6qX4V2jf0AGjVR53/ZJBXjpPHnX9zB1+t6pp6PitQkslWzLYcCv5Mwm4qCGTjt4zSFdcOTwi1+8S+HAfR+ZjAuw06a6smmq64aqiL2DwitvhHyksg7zrmTqaJ0Kg8QPHo91FzdWPwuX0ZjqfQtP8BqNPclE=
+sidebar_class_name: "delete api-method"
+info_path: reference/api/agenta-backend
+custom_edit_url: null
+---
+
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
+import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
+import OperationTabs from "@theme/OperationTabs";
+import TabItem from "@theme/TabItem";
+
+
+
+
+
+Remove Trace
+
+
+
+
+
+
Path Parameters
+
+
+
+
+
+
+
+
+
+
+
+
Successful Response
+
+
+
+
+
+
+
+ Schema
+
+
+
+
+
+
+
+
+ links
+ object
+
+
+
+ anyOf
+
+
+
+
+ Array [
+
+
+
+
+
+
+
+ attributes
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ ]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Validation Error
+
+
+
+
+
+
+
+ Schema
+
+
+
+
+
+
+
+ detail
+ object[]
+
+
+
+
+
+ Array [
+
+
+
+
+
+
+ loc
+ object[]
+
+ required
+
+
+
+
+
+ Array [
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+ integer
+
+
+
+
+
+
+ ]
+
+
+
+
+
+
+
+
+
+ ]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/docs/docs/reference/api/remove-user-from-workspace.api.mdx b/docs/docs/reference/api/remove-user-from-workspace.api.mdx
index fb62afac86..80fad58a99 100644
--- a/docs/docs/reference/api/remove-user-from-workspace.api.mdx
+++ b/docs/docs/reference/api/remove-user-from-workspace.api.mdx
@@ -5,71 +5,321 @@ description: "Remove a user from a workspace."
sidebar_label: "Remove User From Workspace"
hide_title: true
hide_table_of_contents: true
-api: eJzNVE1v00AQ/SurOYG0SkrFySciNagRRVQlwKGNook9ibfYXrM7DgRr/zsaf8VpaSUQB06W1m/ezLyZNzUw7jxEt/DFuq++xJhgpSEhHztTsrEFRHBDud2TQlV5cmrrbK5Qfe/xk7virpi5nY/uCqWUohxNpl54di8jtUype8AkceS9slvFKbVUbNWGlGvokzZ6oF2bZEyyuOgjj4lBgy3JoZS5SCCClmkt3Gspcz1gQUOJDnNictJuDQXmBBGM84EGI/2WyClocPStMo4SiNhVpMHHKeUIUQ18KCXWszPFDjSw4UweBg3VIoEQ9JClkaCn/1aRO/wV/7yhCWElwb60hScv+POzM/mcTu1jFcfk/bbK1E0HBg2xLZgKFjiWZWbiRr3pvZeY+lhDCCFoeH1+/pj4M2YmacLU3Dnr/oAVSicTY9PWnRBLQ1ENhin3jwGZjU/+YnH4sG3GdyqRSN29mIJpRw7CKuj+DZ3Dw0jHK9sWCEFD7nfPSf6evMcdwUD2zHREDLWUv+GY227uKeaTcd82fbWpO9zqSHOUt1X36TYuWvl+l6yHXC6X148I29nmxKkV1ySUEbcO4RQimA6e8NN67I8wFWd50ODJ7XsjVS6DCFLm0kfTaZzZKpngjgrGCZoplgZkXz3FlTN8aEJm14t3dLgkTMhBdLsaAz7KrrTTP4UNwmNp3pHo0JlrVnFqnfnZjrQzWdpGSaOm2NomvNNk1hSnZtcL0CBttGt9Nnk1OYOHt+8ELGuOcbPmfe7mN+gHKgz9g+68HwET5m+OP6Sy0nrOsRjxdaf2k1zHt3Jov4wu2Ell9dF0//OB7mbG9IOnZYam8VwjVt3t2+3xBstqRQ8ucrtyKw2p9Szout6gp08uC0Ge22sqW5UYj5tMDLbFzNMzer246bz4Uj1V4lc6jO72HrNKMM0q79EZSfTvk/YLXhzGOftiToRpzlvaO6juILM4ppJHwY+usTQwOP9ifjVfziGEX3c8vLw=
+api: eJzNVE1P20AQ/SurOYFkJSnqyaemJRURVEUQygEiNNiTeMH2mt1x2tTa/16Nv+JAQWrVQ0+W1m/ezLyZNxUwrh2EN3Bt7KMrMCJYBhCTi6wuWJscQrigzGxIoSodWbWyJlOovnf40W1+m0/t2oW3uVJKUYY6VQeO7WGoFgm1DxjHlpxTZqU4oYaKjbonZWv6uInuae90PCSZH3eRu8QQgCnIopQ5jyGEhulOuO+kzLseCwEUaDEjJivtVpBjRhDCMB8EoKXfAjmBACw9ldpSDCHbkgJwUUIZQlgBbwuJdWx1voYAWHMqD72Gah6D90GfpZago38qyW7/in9W03i/lGBXmNyRE/zRZCKf/aldllFEzq3KVF20YAggMjlTzgLHokh1VKs3fnASU+1q8N77AN4fHb0k/oapjuswNbPW2D9ghcLKxFg3dcfE0lBYgWbK3EtAaqK9v5hvv67q8e1LJFK3LzpnWpMFv/RB94bW4nag45lpCgQfQObWb0n+hZzDNUFP9sZ0RAy1kL9+l9vcP1DEe+O+qftqUre45Y5mJ2+j7uttHDfy/S5ZBzlZLM5fEDazzYgTI66JKSVuHMIJhDDuPeHG1dAffizOchCAI7vpjFTaFEJImAsXjsdRasp4hGvKGUeox1hoeH5PpvVf9UmgIMvsKCqt5m3NNz2fn9L2hDAmC+HNcgi4lEVqVmMf1k8FC31KIlLrvGnJibH6ZzPv1oFJEyUq6Hxl6vBWsLa2jxg9Ui4XQfpsyp6M3o0mrzXTBqjp+bz1A0a1H7o6ahgEz+TqhYKgPRIhMGH2YfdDqiyM4wzzAV97k6/kjH6Wi3w9OHV7FVY7d/7Pl7ydH9MPHhcp6tqctVhVu5g3u2MtOxg+O93Nbi4DSIxjQVfVPTq6sqn38tycXdmwWDu8T8WJK0wdvaHXwUVr2kP1WomPtB0c+A2mpWDqtd6g1ZLo3yftlj3fDnN2xewJU9/BpHNT1UKmUUQFD4JfnG1poD8Rx7Oz2WIG3v8CS2nLSw==
sidebar_class_name: "delete api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
+>
-
-
-
-
+>
Remove a user from a workspace.
Args:
- email (str): The email address of the user to be removed
- workspace_id (str): The ID of the workspace.
+email (str): The email address of the user to be removed
+workspace_id (str): The ID of the workspace.
-
-
-
-
-
+>
-
-
-
-
-
+
diff --git a/docs/docs/reference/api/resend-invitation.api.mdx b/docs/docs/reference/api/resend-invitation.api.mdx
index 7d07068c69..bfd9be5d4e 100644
--- a/docs/docs/reference/api/resend-invitation.api.mdx
+++ b/docs/docs/reference/api/resend-invitation.api.mdx
@@ -5,75 +5,337 @@ description: "Resend an invitation to a user to an Organization."
sidebar_label: "Resend User Invitation To Organization"
hide_title: true
hide_table_of_contents: true
-api: eJztVlFv40QQ/iureQLJ1KE6XswLASpdOKBVmgOhNIom9iTeO3vXt7tuGyz/dzS7duwkbQAJeCJPsXf2m5lv5ptxAw53FpIl3JodKvk7OqkVrCLIyKZGVv4xgTlZUplAJaR6lM5bCacFitqS8f+UGENcPagHNUdpySYPSggh3i4WdzfPKXnIRKxHDtZfC+vQ1Xad6owS8dVk8uKd2eBbaSe2ulaZ0EbkaAU9V9JQdoL05hWk33QtsDCE2V5sqNBqxzm4XFqhR1m8hPag5uRqo/q8fri//XlOttLKUiI8Ue6EJeboBOp6MoEIdEXGW80ySMB4ktfDXYigQoMlOTJcpAYUlgQJjGNcywwikFylCl0OERj6VDMXkDhTUwQ2zalESBpw+4qvW2ek2kEETrqCX4wrJ2YZtG108PWkzUdbYUr/hKNfe7DgZRUwyLpvdbbni6eQqVaOlOMjrKpCpj7G+IPltmxGHivDZDpJlp+oRFlcCuTGG7Rt1FvozQdK3VFSyw5mNVwLOvB9SPMQObQtw5iuBbx7rm7SnGjovk5TsnZbF6LvF/jrCQYvb66vz4F/wUJmoXY3xmjzN1BPaMvIdbxJR6U9Nyh0enSKan+79Z15TDT3T/dGKkc7MtCuBrLRGNyPqvGjDgFCG0Fpd5cK9xNZizuCA9iFGjMZYsGnf1Zoziu47uxGNR/oDey+nsb3gb6XnPUmPIfOAENtS3K55jlQaeu89F0OCcRjsdu4OdF+Gx8EauNmLNY29pOE4jBXIAJL5rGfJLUpIIHcucomcZwWus6ucEfK4RXKGCsJrE5LaW2k2/sr07vZO9q/JczIQLJcjQ3uuaNCjxybHcqDlXxHzFY3V6a1y7XpN043WPJwi+ngXp0Pw+HmGcuqoJG4h14Dqbbau+pYnvpExPRuBhFwykEok6svr3jsHsvnyJiFg6kXTh+nP4bohLEDVxAdAnKE5TfDAWfBtSxRjfC6Vfqe9+ZooS300f48jbIZJP3/Mv53lnHXp46eXVwVKP008kVvOi0ujxavhQiS8008yJHPz7YnK9IPH6/JVQQ5qz1ZQtNs0NJ7U7Qtv/5Uk2HZrSJ4RCNxw429bCCTlv9nkGyxsHShTT6bdwPuc/Fadr00FevyEYuanyCCj7R/4SuDh/p/6P6IOr898n70NJ3Jd8HbF37GDxBnK48jDzemKbfdRdvVaBTf3d4vIIJN93VS6ozvGHziEuJTCFdXoR3484XfNVCg2tW8pRIImPz7A8Vm9Bw=
+api: eJztVlFv40QQ/iureQLJ1KE6XswLvaPSlQNSpTkQSqNoYk/tvdq7vt1122D5v6PZtWMnbQNIwBN5itcz3+x8M9+MW3CYW0hWMDc5Kvk7OqkVrCPIyKZG1v4xgQVZUplAJaR6kM5bCacFisaS8f+UmEKc3apbtUBpySa3Sggh3i+X15dPKXnIRGwmATbfCuvQNXaT6owS8c1s9qLP1RhbaSfudKMyoY0o0Ap6qqWh7AjpzStIv+lGYGkIs53YUqlVzjm4QlqhJ1m8hHarFuQao4a8friZ/7wgW2tlKRGeKHfEEnN0BHU+m0EEuibjra4ySMB4kjejL0RQo8GKHBkuUgsKK4IEtMk3MoMIJBenRldABIY+N0wBJM40FIFNC6oQkhbcrmYv64xUOUTgpCv5YG5ycZVB10V75Edt7m2NKf0T+L8OYCHKOmCQdW91tmPHY8hUK0fK8Sus61Kmnob4k+UmbCcRa8PUOUmWn6hCWZ66yKU36LposNDbT5S6g6RWPcx6dAtd77uOFuHm0HUMY/qC+/Bcy6Q9UsxNk6Zk7V1TiqE74K8nGKK8OT9/DvwLljILrXVpjDZ/A/WItoxcz5t0VNnnBqVOD96i2s3vfB8eEs39059I5SgnA916JBuNwd2kGj/qcEHoIqhsfqpwP5G1mBPswU7UmMkQS377Z4XmvELo3m5S85HewO7raXwf6Hsp2GDCU+cZYKhtRa7QrPpaW+eF7gpIIJ6OHxu3QeldvNeljdupRrvYjwuKw/CACCyZh2FcNKaEBArnapvEcVrqJjvDnJTDM5Qx1hKOB/2FfyvesSmwYi2ljZFu5/Eurq8+0O49YUYGktV6anDDXRb65tBsXzKs5QdiBvtZc9G4Qpth5/TDpgheTBH372IcGJdPWNUlTQQ/9h9Idad9qJ75Po+3mN4HWpiTkOLs7Ouz2WuJ9w7i4vqqFxamXljDnb0ZREfU7kmFaH85R1h9N77gjLjWFaoJXr9YP/IWnay3pT7Ypse3bUfJ/7+a/53V3PesoycX1yVKP6180dteqys40CpEkOz38ihXPn62VFmxfiZ5za4jKHgIJCto2y1a+mjKruPjzw0ZVt46ggc0Erfc26sWMmn5fwbJHZaWTnTHF4t+7n0pXktqUKdiaT5g2fATRHBPu/FTg0f8fxj1gDG/S4ph6LS9ybsQ7Ss/8UeIZwuQbx48LlJuspO268lgvp7fLCGCbf+tUumMfQw+cuXwMVxX16H4/DHDZy2UqPKGd1YCAZN/fwBwe++q
sidebar_class_name: "post api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
+>
-
-
-
-
+ path={"/organizations/{org_id}/workspaces/{workspace_id}/invite/resend"}
+>
Resend an invitation to a user to an Organization.
Raises:
- HTTPException: _description_; status_code: 500
- HTTPException: Invitation not found or has expired; status_code: 400
- HTTPException: You already belong to this organization; status_code: 400
+HTTPException: _description_; status_code: 500
+HTTPException: Invitation not found or has expired; status_code: 400
+HTTPException: You already belong to this organization; status_code: 400
Returns:
- JSONResponse: Resent invitation to user; status_code: 200
+JSONResponse: Resent invitation to user; status_code: 200
-
+>
-
-
-
-
-
-
-
-
-
-
-
-
-
-
\ No newline at end of file
+
+
diff --git a/docs/docs/reference/api/unarchive-workflow-revision-rpc.api.mdx b/docs/docs/reference/api/unarchive-workflow-revision-rpc.api.mdx
index 92d1c5fa3d..129828ef6f 100644
--- a/docs/docs/reference/api/unarchive-workflow-revision-rpc.api.mdx
+++ b/docs/docs/reference/api/unarchive-workflow-revision-rpc.api.mdx
@@ -5,67 +5,1999 @@ description: "Unarchive Workflow Revision"
sidebar_label: "Unarchive Workflow Revision"
hide_title: true
hide_table_of_contents: true
-api: eJzNWdtuGkkQ/RVUT4k0MV5rn+ZpvXYie5ONkW/7gJBVzBTQSc8lfcFL0Pz7qm8DwwALBEs8YZiqU12nqrt6juegcCwh7sM/hfg+4sWrhEEERUkCFSvy2xRi0DmKZMKm9PLqjV4ETZlkRf4iygQiKFFgRoqEgZpDjhlBDG1rlkIELIcYSlQTiEDQD80EpRAroSkCmUwoQ4jnoGalwZBKsHwMEYwKkaEyq9EWRTHFjUFYd+feB+ncplBVA4MtyyKXJA3cxfm5+UhJJoKVJjWI4UEnCUk50rxz740hgqTIFeXKmGNZcpZYJrrfpPGZLy2xFIYnxVyEpNDOya+c5YrGJJaWemUtIkhphJoriM+rqE2SjZvP7kaWymaM2niKgmGuDKEN8+2sVVFtkWvOwbDU4vHZQVsaIzh+oCZ+ndEbZOIioFBshMkxU7j0kCGCVpNCHA3coVURpKhoR1Rj+kGxjLZCXxvAKoKMpMTxVuxtMH979yqCEbfHx+aOZfIl0VIV2dpgw6LghPnWaLeyc+UQqsjA0RS5RrWB8B0RP9YgDnSiM8x/BfDGAlRVFIyK4TdK1JqD6pOlrA1n66KwuQhMU2ZOH+S9Bq07lC2cP41I2bD5y3J262NBwkSiOYp3nzTnf8ki/3CnVanVe1jN1YAwRdmOXigEzrbyuuILLXa3d6lC69Hq0FMi9WvxhUn1v7RuSrLlvh9Hj64V/cA+7DT4ivbUaY7Ww6CulyCqCBJBqCh9QXXUU/DKwXYuLVm6TN8iyJOD9UFS4vQGQa4drA8S6BrOjjjqAll/zvywC3wdNUpgq44SCDtqlEBXHWVKon3j2r1dn717FYHkenwozIPxNWPoaIl6DnF1mjRH8/Gy14IdivIkmEPghyNwgzAhTO37x34n/VtRcnoNseVqck8jEpQnjdMmLHSw10S58WWowrvcrhU5oZl8khedPWrw4Ik3N0osS5bvegFapXif21YIZAtvhvjB3e+8q6aqcKr3t5Pslc3OvQWlplAkpiyhXdh18swePeihK6unjNhYO0HpLWJdNQLs8g4WtKJrtO8IrVexPSB+0b3WnarKuP1+cdGWqZ6Rs9Qm1/koRCEO16hSUsjspPUtuWrAi6TxdJ8ttTQoXBcvsv5SJBhu9Zkcr9P31ogLzmSzqSWj82ierqN8oS32bV4utLdrCFKBXsfu5jSuHX3b6nvz+NhrAbraNov6FETVTkvBBKsETAqjvZaFVFZgVROIoVsalZBeu0E2k90gG8rufJ3eWnVr9Rbchg8irb1ywUSpUsbdbsILnZ7hmHKFZ8i6WDK7xyQlWjA1sy6XvdvPNHMjHuL+YNnADh3XRE2zun5Yss9k6PTysBO62E/XGV4Xdtc4SzHLR4V1D8KYXVznsncLS1d3OD/77ewcVultGJvdgondLSG2fQzRCgt1/hABZXargCLM/lg8MCszRfF6kcfbXs2VV2PPh6J/VbfkyPKlG7ArdB98oWGhkErb0r7YEEG8QV5f1HsQwcS0T9yH+XyIkp4Eryrz8w9NwpR04AXeoSG4P4eUSfN3CvEIuaTWyutjB97d+931vrMYXs2MQtlzU3Ojt5lvEMF3mm3654A9Qyahv+be9DJJqFRLIK0jzzRivWV6dw+PUFX/AaoBjX0=
+api: eJydVcFu2zAM/RWDpw0Q4qzYyadl3YAG3dCgTbdDEBSqzMRqbUmV5HSZ4X8fKMuO07QbsFOU6JF6fHxkGvB86yBbwU9tHzelfnawZqANWu6lVvMcMqgVt6KQO7x7jqA7izvppFZ31ghgYLjlFXq0lKoBxSuEDAaQzIGBVJCB4b4ABhafamkxh8zbGhk4UWDFIWvA7w2FOm+l2gKDjbYV90SiDlm89CUBrmPuZJ5D264ppTNaOXSU5Ww6pY8cnbDSUCGQwU0tBDq3qcvkOoKBgdDKo/IE58aUUoS60wdHMc2ImbGkipfdC0LXXVAkLJXHLdoRw/OAaNkgQ3hC7a82QaMYp+8fUPhRWN+HvkBo2QBWdVlCu25bdhJ+UHQVua3fzjmU37aU6+PZ2alaP3gp86BF8tVabf9fqhw9lyWdpMfKnQJKLY5uT0SKbhgp0cvdrg9acGv5fqTkN90RpB5Ubvuau3rod3SObxGGZG9DgxjJkm7/1Qeqq3s64kYtOcjbqft2GV86+V57rIdcLJeLk4Rdb4+bettPctJbIhl8xqBCX2gaeKOdD1PtC8ggNeRgfE7vucO0XwEu7X3t0mY06W06rAtg4NDu+q1Q2xIyKLw3LktTUeo6n/AtKs8nXKbcSHhJdxZuk3OCAg25Q1Fb6fch32wxv8T9BfIcLWSr9RhwQ17s3HUMGxrLjbxE0jkuq1ntC23l784ycV0VXRQJKdVGh/CoeeT2mYtHVLSZqM6O9nTyYTJ9q5gYkMwW8zhSXISR6nkEGLAXcg1CAQOswjyBR159OlwQS+pcxdUo399bfkRx0MbjL5+aksswO4FIE92wgugGYEB+AAaDI+Cw6+icHe//gy3WDApyWLaCpqEkt7ZsW/r5qUZLzV0z2HEr+T1JvWogl47OOWQbXjo84T1sJnh3HQfwfQLs9Xp6Ayjq/o6XNX0DBo+4f/GnFbZL0RusiYiZEGj8KPZkGZITh2FaXN0soW3/AFlgjuA=
sidebar_class_name: "post api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
-
-
-
-
-
-
+
+
Unarchive Workflow Revision
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
+
-
\ No newline at end of file
+
+
+
Path Parameters
+
+
+
+
+
+
+
+
+
+
+
+
Successful Response
+
+
+
+
+
+
+
+ Schema
+
+
+
+
+
+
+
+ revision
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ author
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ date
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ message
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ name
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ description
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ version
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ slug
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ flags
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ is_custom
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_evaluator
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_human
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ metadata
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+ data
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ service
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+ configuration
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+
+
+
+
+
+ artifact_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ artifact
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ name
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ description
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ slug
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ flags
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+ metadata
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+
+
+
+
+
+ variant_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ variant
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ name
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ description
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ slug
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ flags
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ is_custom
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_evaluator
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_human
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ metadata
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+ artifact_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ artifact
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ name
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ description
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_at
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_at
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_at
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_by_id
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_by_id
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_by_id
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ slug
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ id
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ flags
+
+ object
+
+
+
+
+ anyOf
+
+
+
+
+
+ is_custom
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_evaluator
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_human
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ metadata
+
+ object
+
+
+
+
+ anyOf
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Validation Error
+
+
+
+
+
+
+
+ Schema
+
+
+
+
+
+
+
+ detail
+ object[]
+
+
+
+
+
+ Array [
+
+
+
+
+
+
+ loc
+ object[]
+
+ required
+
+
+
+
+
+ Array [
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+ integer
+
+
+
+
+
+
+ ]
+
+
+
+
+
+
+
+
+
+ ]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/docs/docs/reference/api/unarchive-workflow-variant.api.mdx b/docs/docs/reference/api/unarchive-workflow-variant.api.mdx
index e855606aae..78a04ea5c0 100644
--- a/docs/docs/reference/api/unarchive-workflow-variant.api.mdx
+++ b/docs/docs/reference/api/unarchive-workflow-variant.api.mdx
@@ -5,67 +5,1008 @@ description: "Unarchive Workflow Variant"
sidebar_label: "Unarchive Workflow Variant"
hide_title: true
hide_table_of_contents: true
-api: eJzNV01v20YQ/SvCnBJgY6lGTzxVtRNYTVoLsZweBEEYkSNxkyWX2Q+5qsD/XuxySYmipDiOAvTkD868N/NmdnZ2CwZXGqIp/C3Vl6WQTxpmDGRBCg2X+SiBCGyOKk75muZPwWi+RsUxN8CgQIUZGVIOZQs5ZgQRHBrOeQIMeA4RFGhSYKDoq+WKEoiMssRAxyllCNEWzKZwENoonq+AwVKqDI2Lw3oUw41wBnXEvU8VR2+UQFnOHLQuZK5JO7TrwcD9SEjHihcuJ4jgwcYxab20ovcxGAODWOaGcuPMsSgEj70E/c/a+Wz3IiyUE8jwiiGWtnIKgfPc0IrUXqQ33oJBQku0wkA0KFlHIk+bb+6XXsc2RWPLk7bZea1K1ljkVghw4nTUc6oxQGX4EmNzQYZhgAwMS+E77XSSXM9jq43MjgawkFIQ5mcZR7p3UyGUzMHRGoVFI9WPIL5tQCrQ1GaY/wjgnQcoS1YbycVnis2Rzn7nJevClQwyMtgOApOEu35FMW7JeqqUe7B1x7aYskX7P/vZHeeCmKvYClSv3lkh/tAyf3NvTWHNazjM1YFwQ9kzvVAp3JzV9cAXOuqec/7Tiek8Oh36fxL1L/mBa/NNWU8l2XH/Po0mVSuGAf8sBbohON+StYfxy6Bu9yBKBrEiNJTM0TxzeiVo6I3hPp7TLDcVbG/oxbJF8jNIHivYQJKQoJ9AclvBBpJarsXmgvO+Fuv3TZj4tV4XZanValhqwS7KUsvVsGhhVy/t1Qfn6y6Pi4XnN51v3x9hMTp2gzzfu1mQytJ5/Xp93d2nPqHgid+Wem+Vkurly1RCBrnw60B1PRwaCBm3vn7PJJ6VBzfKLukPsgrQ1SnTq2N76O620BpXtBufp029GL2J+3pM8d0OPPV5VdTBbq/cO3krdU+ncVvJd668d5PJuANY1bZd1Md67e8dbtrg949UusdBIXX1DDApRNAvFK05PfXrjVX3w3qr+9sjj4Ky3zwugIEmta4fElYJiCA1ptBRvx8LaZMrXFFu8Ap5HwvuD4Sm2CpuNt5lOB69p80dYUIKouls3+DB9VvVQW2zpnhY8PfktAxPmKE1qVT836otwuMlrbycWDxfSu9er7s+uN5wPAIGLo1KxcHVL1cDONS2ZeyOCsb+qNTc/jOwAxWa/IEBZf6cgCHMftt9cJG5koQdNeCdLeXBbRzkMPSP6RcCuT8SPo5tqPIUQpVh94rRLulQaWAQHX8A7oo9Y5C6zommsN0uUNOjEmXp/v3VknL1nAXAhVN3uoWEa/d7AtEShaZO3M3AgVcfw7l63dttSO186prnGx+3sO4vYPCFNider354pHVvbYPlMI6pMHsYnVnnmrA5LOP7hwmU5X9LvE7O
+api: eJydVU1v2zAM/SsGTxsg1Fmxk0/LugELuqFBv3YIjEKVmVitLamS3C4z/N8HyrLjNM0G7BQleqQeHx+ZFjzfOMhW8FPbx3WlXxzkDLRBy73UalFABo3iVpTyGe9eIujumVvJlQcGhlteo0dLWVpQvEbIIN7fyQIYSAUZGO5LYGDxqZEWC8i8bZCBEyXWHLIW/NZQpPNWqg0wWGtbc0/PNyGLl74iwG2fOlkU0HU5ZXRGK4eOkpzOZvRRoBNWGqoAMrhqhEDn1k2VXEYwMBBaeVSe4NyYSopQcPrgKKadEDOW5PCyf0Hopg+KfKXyuEE7IXgWEB0bRAgvqO3FOggUw/T9Awo/iRr0j+VBx0asaqoKurzr2EH0Ts5VZJYfTTnW3nWU6uPp6aFUt7ySRRAi+Wqttv+vU4Gey4pO0mPtDgGVFnu3BxJFJ0yEGLTu8p0U3Fq+nej4XfcEqQG127zlrAH6A53jG4Qx2XFoECO5ptt/tYHq6p+OuHzq3UHeXt3jZXzp5XvrsQHy7fp6eZCw7+1+U2+G+U0GRyS34/zW6EtNU2606+fZl5BBaiw+S3xJ77nDdJh7l0ZPu7TdjXiXjhsCGDi0z8M2aGwFGZTeG5elqah0U5zwDSrPT7hMuZHwmus83CZnBAUab4eisdJvQ775cnGO22/IC7SQrfIp4IqM2FtrHzZ2lRt5jiRyXFLzxpfayt+9X+KeKvsoUlGqtQ7hUfDI7TMXj6hoJVGdPe3ZyYeT2bFiYkAyXy7iPHER5mngEWDAXsk1CgUMsA7DBB55/Wl3QSypbzVXk3x/7fcew1Eaj798aiouw9wEHm20wgqiFYABmQEYjHaAccnRMdvb+jtP5AxKMle2gralFDe26jr6+alBS53NY5570nnVQiEdnQvI1rxyeMB63Enw7jKO3vsE2NvVDN1X20C3augbMHjE7f4/VVgr5WCuNgLmQqDxk9CDLUguHMdoeXF1DV33B0hBhwo=
sidebar_class_name: "post api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
-
-
-
-
-
-
+
+
Unarchive Workflow Variant
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
+
-
\ No newline at end of file
+
+
+
Path Parameters
+
+
+
+
+
+
+
+
+
+
+
+
Successful Response
+
+
+
+
+
+
+
+ Schema
+
+
+
+
+
+
+
+ variant
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ name
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ description
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ slug
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ flags
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ is_custom
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_evaluator
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_human
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ metadata
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+ artifact_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ artifact
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ name
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ description
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ slug
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ flags
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ is_custom
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_evaluator
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_human
+
+ object
+
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ metadata
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Validation Error
+
+
+
+
+
+
+
+ Schema
+
+
+
+
+
+
+
+ detail
+ object[]
+
+
+
+
+
+ Array [
+
+
+
+
+
+
+ loc
+ object[]
+
+ required
+
+
+
+
+
+ Array [
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+ integer
+
+
+
+
+
+
+ ]
+
+
+
+
+
+
+
+
+
+ ]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/docs/docs/reference/api/unarchive-workflow.api.mdx b/docs/docs/reference/api/unarchive-workflow.api.mdx
index c015a7c5ee..27a7b9151d 100644
--- a/docs/docs/reference/api/unarchive-workflow.api.mdx
+++ b/docs/docs/reference/api/unarchive-workflow.api.mdx
@@ -5,67 +5,598 @@ description: "Unarchive Workflow"
sidebar_label: "Unarchive Workflow"
hide_title: true
hide_table_of_contents: true
-api: eJzNV01v20YQ/SvCnBJgY6lGTzxVtRNYTVoLsdwcBMFYkSNxkyWX2Q+5KsH/XszyW5QcxXGAniRxZ97MvBnuPOVg+dZAsIRPSn/ZSPVoYMVAZai5FSqdRRCAS7kOY7HDh8fKCBhkXPMELWryziHlCUIAtcGDiICBSCGAjNsYGGj86oTGCAKrHTIwYYwJhyAHu8/I1Vgt0i0w2CidcEtxnUexwkoyqDMczSIoihVBmkylBg2hXE4m9BGhCbXIKHcI4M6FIRqzcXL0sTIGBqFKLaaWzHmWSRH6UsefDfnkncwyTURYUUYIlSudqoRFanGLupPhlbdgEOGGO2khmBSsocSHS/e3G89XH3ojfRdOGwjzEDpjVdI3qjJZKyWRp1Cw5lHqpAQiqc5tZkZXJULBCA53XDpulf4RxLcNSAkau4SnPwJ44wGKgtVGav0ZQ3tkCt55yoZwBYMELe8nwaNIUI+5nPdoHaRZDWEHtu5yL1Ky7j/pVnc8FoRCh05y/eqdk/IPo9I3t85mzr6Gw1oJRFhMzvTiWvP9k7we+MKA3aec/yQyyWMwof8nUv9SH4Sx36T1VJED9+/jaFGOYnUJnsXAMAXyLVj/Anse1HUHomAQauQWowdunwLs3LsRt/jGCp/P6ShXJexo6slyWfQzgtyXsFWQCCX+hCDXJWwVpKZrvacldl4cv6nOIev3vd9eLV8vGqVmq4lSE/aiUWq6mihGuu1zZ/WOfGl5vFh6Xh18e38cWx1nuDUyoijI/NfLy6Hq+JtLEXlNMXqrtdLPlxwRWi6kFwDlQjg0kCrsnX7P3bsqDnZIW+0HVSZInUnM9phKa/eDMXyL7YV52tSTMVrQ6TGqW4W49HWVoSu7ToNbekt2T5dxXdL3VF9vFov5ALDsbb+p97UIHn1qRXCCNlYkkTNlrBfFNoYAxpnGncDHca39zDjvKONi3ChqYGBQ72oV7bSEAGJrMxOMx6FULrrgW0wtv+BizDPhJ91g6LSwe+8ync/e4/4GeYQaguWqa3BHY1UOSt+s6RHPxHskyir9PnU2Vlr8W3a/UvBx6UWciHSjvHtF39QnN5rOZ8CAyijJmlz8cjGBQwp7xvRG8NC/EXVsfwzsgIWmfmCAiX8dwCJPfmsPKDPqQCU+K7yjHTtYrxUNFv+x40xy4Sfex8+rZi6haia0Ut4Ag6D/V6ft6IpBTNMQLCHP19zgvZZFQY+/OtTUtBWDHdeCr4nCZQ6RMPQ9gmDDpcFBks3lAa8+Vu/I61Grb/rJ141Nqaskz+kXMPiC+4P/Z/4CiOvBySuLaRhiZju+g/uKJqwZ/Pnt3QKK4j/5nOJ5
+api: eJydVU1v2zAM/SsGTxsgxFmxk0/LugENuqFBm66HIAhYmYnVypYryekyw/99oL/i1CkG7JQgeiQfH/mYEjzuHEQreDD2eavNq4O1AJOTRa9MNo8hgiJDKxO1p81rCwIBOVpMyZPl6BIyTAkiQOvVFqXfqBgEqAwieCnIHkCApZdCWYoh8rYgAU4mlCJEJfhDzrHOW5XtQMDW2BQ9Fy7qNF55zYBZmzyYx1BVa07pcpM5cpzlYjrlj5ictCpn8hDBXSElObctdHDbgkGANJmnzDMc81wrWfcaPjmOKQfMcstKeNVUkKZoglrCKvO0IztgeFkjKgG9UFwiO9xsa5HaOPP4RNIPwjrtuwahEj04K7SGal1VYhR+VHTVcluPc/ZtVxXn+HxxMVbpF2oV1xoE36019v8lismj0vxNeUrdGKCNPHkdidNuwUCBTuZqfdQArcXDQMEfpiHI2qdud26rOuhPcg53BH2y96G1GMGSX/+lP/fVlG5xg1Ec5W3Ufb+Nb41854p1kKvlcjFK2Mz2dKj3nWuDh6NrU/KJYU/nxvnaxT6BCMLc0l7Ra/iIjsJueV1Ydl83Kq7C/g6AAEd233m/sBoiSLzPXRSGUpsinuCOMo8TVCHmCt5ym9WvwSVDgZ3sSBZW+UOdb7aYX9PhijAmC9FqPQTc8eI1q3QK66eIubomFrU9SbPCJ8aqP81+tEcpaaJYNZVtTR3eXZmG21eUz5Tx+eE+G9rTyafJ9L1m2oBgtpi3/mEnR/1pbGAg3sjVCwUCKK3NA54w/XJ8YJY8rxSzQb6z8z1h1kvi6bcPc42q9kddv2xHv4J29CCAhw/H2+VAQDRYAKbeb8BaQMIrFK2gLDnw3uqq4p+be89zjJXDR83+2KJ2NGLX3xj4cNta6WMA4jzrZzqM/l/2qAtG1iu0R6u4XL0xopsw82hCZ1JS7gdRo7vGWXqDLG7ullBVfwELanMP
sidebar_class_name: "post api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
-
-
-
-
-
-
+
+
Unarchive Workflow
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
+
-
\ No newline at end of file
+
+
+
Query Parameters
+
+
+
+
+
+
+
+
+
+
+
+
Successful Response
+
+
+
+
+
+
+
+ Schema
+
+
+
+
+
+
+
+ workflow
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ name
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ description
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_at
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ created_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ updated_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ deleted_by_id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ slug
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ id
+ object
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+
+
+
+
+
+ flags
+ object
+
+
+
+ anyOf
+
+
+
+
+
+ is_custom
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_evaluator
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+ is_human
+ object
+
+
+
+ anyOf
+
+
+
+ boolean
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ metadata
+ object
+
+
+
+ anyOf
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Validation Error
+
+
+
+
+
+
+
+ Schema
+
+
+
+
+
+
+
+ detail
+ object[]
+
+
+
+
+
+ Array [
+
+
+
+
+
+
+ loc
+ object[]
+
+ required
+
+
+
+
+
+ Array [
+
+
+
+ anyOf
+
+
+
+ string
+
+
+
+
+ integer
+
+
+
+
+
+
+ ]
+
+
+
+
+
+
+
+
+
+ ]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/docs/docs/reference/api/unassign-role-from-user.api.mdx b/docs/docs/reference/api/unassign-role-from-user.api.mdx
index c76f108f6f..7330953d5b 100644
--- a/docs/docs/reference/api/unassign-role-from-user.api.mdx
+++ b/docs/docs/reference/api/unassign-role-from-user.api.mdx
@@ -5,81 +5,349 @@ description: "Delete a role assignment from a user in a workspace."
sidebar_label: "Unassign Role From User"
hide_title: true
hide_table_of_contents: true
-api: eJztVt9v20YM/lcOfFkLCHYW7ElPMxAXNdqhQepsD4kRMBJtXSvdqcdTWk/Q/z7wTrLkeAm2dXnrk607kh/58cexhZw4c7r22hpI4YJK8qRQOVuSQma9MxUZr7bOVgpVw+SUNgrVV+s+c40ZzW7NrVm4Hae3Rik1XtzpXL1i716nal2QWl0ou1W+oCNV0aAKdTkVjQe9dID0Vjmq7AOFo+CceNQbsG6HRv+JEsSTqFOhXi/YmQiH7xEqxDy4MKjQl4bYq1dX8U+v+QbZLy5Xh2t7/4kyH6i5It84M7Bzb22ZqrVrSOntGMyE6a/IipssI+ZtU5Z7lYec5NEYaqbB1tv1+nL5LaOQvFStgj1HSrNCo8g5G3IVQAa/3MhpbomVsV4V+ECqJldpZm2NMFCT21oXo8dsQtk/xGzqHL02uwPaTxwC5RkkYGtyIQ2rHFJoTAz+Tu7vhPM7UYAEanRYkSfHkN60YLAiSGFaX5CAlrKt0ReQgESpHeWQetdQApwVVCGkLfh9LbrsnTY7SMBrX8rBH4Mxtcqh65IDSijBwfyXhtz+P9lfBjNTw49q9fshPkwMPo5CKP1+hCux0nUb0eXaGiYW8fOzM/k5HiAfD5WrrnphSCCzxpPxIo51XeoseDv/xKLTji50Xdcl8Mv5+anh37HUeQxyKVX2L6xC7aTkvI5+5+QlKWkL2lPFpwKlzY5u0ew/bEMJHjMkRPcn2njakYNu0yXDGTqH+wmN7210ELoEKt49x/hvxIw7goOxZyostNxabrsRO86fo2zfhLgidC+3Gc2M9EZ2nw7jItL3d2CDiAyJE4MxtxX5wkrbx6kWutwXkML80Nc8b6c93s3D4IAEmNzDMAwaV0IKhfc1p/N5Vtomn+GOjMcZ6jnWGqRembLGab8PKovL1TvavyXMyUF6s5kKfJRaidk/FjsQj7V+R8JD31qLxhfW9X039FgRtSRQbbY2qPecLIJzanG5ggQkjFjWZ7OfZ2eQPCr2I2Epc8xCmQ/Y4RqSRywc4oekn18peMLq1/FCPKst+wrNxN51P4KVdLp6I8/edRzBR261Y8f9WBR+LAovvyj0refpm5/XJeowOkPNt/3YuBnXAZkQ6aPlIE6OTQKFZS/SbXuPTNeu7Do5jm+iDIdcM96XMie3WDI9U/khoTJSX6unXPxM+8kK8YBlIzLhvXgBnNON4qUR+7VihNnIh9OC8/+zOQxgs59iDr4cZTw8v8Uw4dteZJFJGU6UT7YFCeDwMl0s3y/XS+i6vwDtNX7O
+api: eJztVktv20YQ/iuLuTQBCMk1euKpaq0gQlLEcOT2YAvGmByRG5O7zM7SiUrwvxezfMqujbapbzlJ3J3nNzPfTgMpceJ05bU1EMMZFeRJoXK2IIXMOjMlGa/2zpYKVc3klDYK1Rfr7rjChBbX5tqsXMbxtVFKTRc3OlWv2LvXsdrmpDZnyu6Vz+lIVTSoRF3MRbuDXjq49FY5Ku09haMQnETUG7Aue9KZdRka/SdKgr14UJ8Jh+/JQ0h18Dyo0Oea2KtXF92fXvMNsl+db8Zre/uJEh8QuSBfOzOAcmttEautq0np/ZTDDOAvyIrrJCHmfV0UB5WGUqSdMdRMg6232+35+mtCoWax2gR7jpRmhUaRczaUKDgZ4nITlKklVsZ6leM9qYpcqZm1NYJARW5vXZc9JjPI/qHPukrRa5ON3n7gkCgvIAJbkQtl2KQQQ2265G/k/kYwvxEFiKBChyV5cgzxVQMGS4IY5m0FEWjp1gp9DhFIltpRCrF3NUXASU4lQtyAP1Siy95pk0EEXvtCDv4YjKlNCm0bjV5C5w3mP9fkDv/J/jqYmRvuWvTbLX9w2cOYBcBvN3whVtp2J7pcWcPEIn56ciI/xyzxcexTddELQwSJNZ6MF3GsqkInodjLTyw6zRRC27ZtBD+dnj42/DsWOg1qai099S+sQuWkwbzu4k7JSwniBrSnkh8LFDY5ukVz+LAPDXeMkADdn2jjKSMH7a6NhjN0Dg8zGN/bLkBoIyg5ew7x34gZM4LR2DP9FAZsK7ft5Ltjm6NqX4W8Ote93G4yM8Hboft0GmcdfH/nbBARSnhksKttST63MuQdh4WZ9jnEsBynmJfNfKLbZaAJiIDJ3Q+jX7sCYsi9rzheLpPC1ukCMzIeF6iXWGmIHjTQKtyqX0UUpJmZktppfwj2Vuebd3R4S5iSg/hqNxf4KI3Utcax2FgVrPQ7EpD6uVvVPreuf1qGAcw7LUFBm70N6j1gfWy/YHJHRqhA8uzCPln8uDh5KpleQa3ON/08YBLmYYgjiEH0AK4RKIh6WovBE5Y/TxcSZWXZl2hm9i57ZlZCCeqNvIaXHTMfhddMo/l9bfi+NrzY2tBPn6evflkVqAO1hlZvelq5mpYDYZD4warQMcsugtyyF+mmuUWmS1e0rRx3b6bwQ6oZbwvh0T0WTM80fCioUO5r9VSId3SYLRT3WNQiE96TF/Az7hcv7ajfNiY3O/lwWvz8/yAO1GsOc59DLEeFDq9yPnB704usEum+mfKjJUISGB+ss/X79XYNbfsXsWB6XA==
sidebar_class_name: "delete api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
+>
-
-
-
-
+>
Delete a role assignment from a user in a workspace.
Args:
- workspace_id (str): The ID of the workspace.
- email (str): The email of the user to remove the role from.
- organization_id (str): The ID of the organization.
- role (str): The role to remove from the user.
- request (Request): The FastAPI request object.
+workspace_id (str): The ID of the workspace.
+email (str): The email of the user to remove the role from.
+org_id (str): The ID of the organization.
+role (str): The role to remove from the user.
+request (Request): The FastAPI request object.
Returns:
- bool: True if the role assignment was successfully deleted.
+bool: True if the role assignment was successfully deleted.
Raises:
- HTTPException: If there is an error in the request or the user does not have permission to perform the action.
- HTTPException: If there is an error in updating the user's roles.
+HTTPException: If there is an error in the request or the user does not have permission to perform the action.
+HTTPException: If there is an error in updating the user's roles.
-
-
-
-
-
+>
-
-
-
-
-
+
diff --git a/docs/docs/reference/api/update-app.api.mdx b/docs/docs/reference/api/update-app.api.mdx
index 4a27ce4da7..0ba076482c 100644
--- a/docs/docs/reference/api/update-app.api.mdx
+++ b/docs/docs/reference/api/update-app.api.mdx
@@ -5,77 +5,355 @@ description: "Update an app for a user or organization."
sidebar_label: "Update App"
hide_title: true
hide_table_of_contents: true
-api: eJztVk1v4zYQ/SvEXJoFVDsNetKp6m6AGNt2g2y2l8QIJuJY4lYiuSSVxBX03xdDSpadNG7RFuilhgHTmu83M4/qIWDlIb+BwloP6wwk+dIpG5TRkMMnKzGQQC3QWrExTqDoPDlh+FuhVr8jqy5u9a0uXOXzWy2EYO07JcWJD+5NLq5rEqt3wmxEqIlli6RlcdsYlOIkhSmsHZUnQWl0QKWVriZLobGlGO2KQuf0FHDn4UMXbBeSGxPPz71oemy2onSEgST7/MZzdqjlnnNUnibfF9fXl+dPJUVQcrGKZTgSyjMw5Jxxyd1+nsbFYwRLGvJCmyBqfCBhybXKe2W0CEZgWZL3O2AgA2PJRVBXEnLoYmF3aC1kYNFhS4Ect6wHThdySGBDBopbZjHUkIGjL51yJCEPrqMMfFlTi5D3ELaWrXxwSleQQVCh4QeFtWIlYRjWyZp8+NHILZs8d8aIkg4sQmsbVcZ8l589D02/F8s6riYo8qPuXcr5eBa/sM4wZJOSuf9MZTgo6mZ2tp6Nd1MAw8D2jrw12qfoZ6en/HM44B+7iP+ma8TVqAz/qD4l/wrG2b+PRZyAo6Ck1Rih+f7s7CUav2KjZKxVnPNY/30oJAVUDZ9UoNa/VGhMeSBFvf2wiVN9iMaQ7Z4oHagiB8N6hgOdw+0eZD+ZlCBD3PrqGLo/k/dYEeycva4awRDXLP2zVnBdKfSot9eJGd6E7utlvEvw/VGwSYVZ6YXD1NuWQm1k4oKyjrwRashhidb6ZZ+mZYAMPLmHiUw610AOdQjW58tl2ZhOLrAiHXCBaolWARODp7JzKmyjSXG5ek/bC0JJDvKb9b7CRx6M1OpDtR3KaNV74qJHGiu6UBs3XigTm9XJiqvikbuaeen8CVvb0CGvzFMDSm9MjDYtU6xFFJcryICrTiN/uvhucQrP770D5Xh/lHEFplSjGLJnoO3gggyojfMPgbD9YRZwIdb40KLe8zdes0Uk+YNM+nkB/7+M/4PLeJzVQE9haRtUkVhi1/txqyL7esggH1l4nUFtfGBJ39+jp0+uGQZ+/KUjx5uzzuABncJ7HsybHqTyfJaQb7DxdGQETq5GqnkjXktu2i7Nq/WATcf/IIPfaDu/K0QKrafF7Ufh2xTn20h0s/EL3mdOThZFyZ04qrve46PL4vrtBWRwP75XtEaykcNHZlF8TJmaWHikjvishwZ11TFX55Cc8ucrw6GrnA==
+api: eJztVk1v4zYQ/SvEXJoFVDsNetKp2myAGNt2g6y3l8QIJuJY4q5EckkqiSvovy+GlPyRNG7RFuilRgArmjdfjzOP7iFg5SG/gcJaD6sMJPnSKRuU0ZDDJysxkEAt0FqxNk6g6Dw5YfivQq1+R4bObvWtLlzl81sthGD0nZLixAf3JhfLmsTinTBrEWpi2yyhLG4ag1KcpDSFtSN4MpRGB1Ra6WryFBpbitmuKXROTwm3ET50wXYhhTHx+XkUTY/NRpSOMJDkmN95rg613AuOytMU+3K5vLp4KimSkotFbMORUJ6JIeeMS+H26zQuPkaypCEvtAmixgcSllyrvFdGi2AEliV5vyUGMjCWXCR1ISGHLjZ2h9ZCBhYdthTI8ZH1wOVCDolsyEDxkVkMNWTg6GunHEnIg+soA1/W1CLkPYSNZS8fnNIVZBBUaPhFYa1YSBiGVfImH94auWGX58GYUdKBTWhto8pY7/yz56Hp93JZx90ERX7E3qWaj1fxK2OGIZtA5v4zleGgqZtdsNXOeTsFMAzs78hbo33KfnZ6yl+HA/6xi/yvu0Zcj2D4R/0p+Vc4zv59LuIEHCUlrcZIzY9nZy/Z+A0bJWOv4oLH+u9TISmgavhJBWr9S0BjygMr6s2HdZzqQzaGbPtG6UAVORhWOzrQOdzsUfazSQUyxa2vjrH7C3mPFcE22OvQSIZYsvXPjoL7SqlH3N5J7OhN7L7exrtE3x8lmyCsSi8CprNtKdRGJi0o66gboYYc5mitn/dpWgbIwJN7mMSkcw3kUIdgfT6fl43p5Awr0gFnqOZoFTy/HIpoFecMBVYNT2XnVNjEeMXV4j1tLgklOchvVvuAjzw1aQ4OYdsjQKveEzMyalzRhdq48baZpK5OXtwyz+P1TrQunrC1DR2Kzm6kQOm1idmmTUutvMXyC2leJKYldXk6+2F2+lrvo4MorhbjrmAZd2UqO8Ige8bullfIgNq4KBAI2592Bm7KGh9a1Hvxxvu4iLfBQUX9blP/v7X/g1t7nNtAT2FuG1RRgeKp9+P6RZn2kEE+yvUqg9r4wJa+v0dPn1wzDPz6a0eOt2iVwQM6hfc8pDc9SOX5WUK+xsbTkRE4uR416Y14rbhp0zSv2QM2Hf8HGXyhze5HRdTaelrifjSepzzfR0XcOb+4IFi8k0dR8kkcxa72hOuqWJ5fQgb34w+Q1kh2cvjIcouPqVITG48yEt/10KCuOhb1HFJQ/nwDDBi6Kw==
sidebar_class_name: "patch api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
-
-
-
-
-
+>
+
Update an app for a user or organization.
Args:
- app_id (str): The ID of the app.
- payload (UpdateApp): The payload containing the app name.
+app_id (str): The ID of the app.
+payload (UpdateApp): The payload containing the app name.
Returns:
- UpdateAppOutput: The output containing the newly created app's ID and name.
+UpdateAppOutput: The output containing the newly created app's ID and name.
Raises:
- HTTPException: If there is an error creating the app or the user does not have permission to access the app.
+HTTPException: If there is an error creating the app or the user does not have permission to access the app.
-
+>
-
-
-
-
-
-
-
-
-
-
-
-
-
-
\ No newline at end of file
+
+
diff --git a/docs/docs/reference/api/update-evaluation-scenario-router-human-evaluations-evaluation-id-evaluation-scenario-evaluation-scenario-id-evaluation-type-put.api.mdx b/docs/docs/reference/api/update-evaluation-scenario-router-human-evaluations-evaluation-id-evaluation-scenario-evaluation-scenario-id-evaluation-type-put.api.mdx
index f6f00a0c9a..2669342041 100644
--- a/docs/docs/reference/api/update-evaluation-scenario-router-human-evaluations-evaluation-id-evaluation-scenario-evaluation-scenario-id-evaluation-type-put.api.mdx
+++ b/docs/docs/reference/api/update-evaluation-scenario-router-human-evaluations-evaluation-id-evaluation-scenario-evaluation-scenario-id-evaluation-type-put.api.mdx
@@ -5,73 +5,552 @@ description: "Updates an evaluation scenario's vote or score based on its type."
sidebar_label: "Update Evaluation Scenario Router"
hide_title: true
hide_table_of_contents: true
-api: eJzdV0tv4zYQ/ivEXNoCWjsN9qRT3W2AGNsmRuLk4hgCLY1tbiVSS1JJXEH/fTGkZD3sOMm2wALNJQk5L37zzUMlWL4xEC7gssi4/HDxyNOCW6GkgWUACZpYi5z+hxDu8oRbNIxLhns5ZmKUXAv1k2GPyiJTmplYaWQrbjBhSjJhDbO7HEcP8kHecGHQhA+SMcYu5/PZxXOMzkPIpmtWOB9szUVqyFQheWG3Sot/MPH6aAstGwNXSmLIzs8+sivFPilpUVpmLLeFYbFKkBU5hVjEMRqzLtLa/ggCUDlq94JpAiH486h9V9S8K9KqsKijLQHUETBRV1ok0VHlo4ck3L0gcKIoLywEkHPNM7SoKSslSJ4hhNDzBAEIykfO7RYC0Pi1EBoTCK0uMAATbzHjEJZAdiEEY7WQGwjACpvSQZtlNk2gqoJjfjrR/pcOb2uzJzw7K9/rEmWREZ99uni0iiwa6++MkJsUo0wlmLpj4vhhjHMyWVVL7xaN/V0lO/I1jCL2hKMrnuepiJ36+Iuhcik7Qeaa2GYFGvqPysTpyN312mW5/whCpT6RRZpC1YnynnQrAkDptxoR0uIG9Um7t85eRW/SGmMbcWmeUH9vmJ+8FTbxVqoAVGHzwpq+QWExM0cA4lpwaYl5J0h176UckYK9jvfzFr1rL1lVQSOqVl8wtj26LbrBHHjpvNj1z5ZDDc0bJ3sfXGu+O4nddY1URSXwZsycaOSL6eW3T0mKXZFUYz6imN+gdO/EXkOrE0bf/utQOTfvQmoq90CZKBdSYnKUsCulUuTytC3DZt5CFYD8FxV65Sr0CEyvPN9PV6gq0tVociWNT+352Rn96g/j23am3dTC8PaO5L18PD8/NHzPU5H4Xn2htdLvsDqgZIKWi9SR8zhnUxX3bt/Ty5ZDorQA/6l8gJTJzGxOUfsvNIZvsGXdiRlGYLB6NpwuAnqXd13LdRv4Hl6P7svP+MPDd5JK8/nswKDPbYZ2q2izadYKu4UQxm4sfuhsMeOyt1tU4yMrQE+msxdUvQsKsqIpi/qxWV8KnUIIW2tzE47HcaqKZMQ3KC0fcTHmuXClYzAutLA7pzKZTT/j7hJ5QuNnsewK3BLTPHf6Yvu08Vx8RkKxXism9fboCVEvFVuvRTARh2/aKX/xzLM8xXZKtzyoJ257MJyU7c1+1i36s6yVGE6rPdOXbdNf9Jt6q9xr2z3Ntgv6BUX23uA6/lo5tGoCTVwu2GQ2paBQG98Dzka/js5guP/3hKkn8Nj1hAZqdw3BIOn7dNNylrmGABZ59lt7QYnIlbEZlx17viGyY7vjjVvJhwGWbaP6n32r1OS2+GzHecqFa20O5rKu7Hrh7VY2BBAOvxuO1PFArL/2h8O1fBnAVhlLDsuSULvTaVXR8dcCNVXwsmb3igi2KCERhv5OIFzz1OCJnP18U/fQX9hLb26qXO5cEfkKgAD+xt3BVxJNjR/jvIvhj4vCCbo5uW2aaVlL1ez7MPdfWo2Vg+FOwXuNSUzMPym77Ayd2d0cAljV3030wQUhaP5Eg5I/+YBV7mlKH1Z0VkLK5aagcRyCN0k/3wD7w9F0
+api: eJzdVktv4zYQ/ivEXNoCWjsN9qRTvWmAGNtmjcTZi2MItDS2uSuRWj6SuIL+ezEkbcmPGPsosEBziUzOfDOcxzfTgOUrA+kMblzF5ZvrJ146boWSBuYJFGhyLWr6DSk81AW3aBiXDHdyzOQouRbqF8OelEWmNDO50sgW3GDBlGTCGmY3NQ4e5aO848KgSR8lY4zdTKeT65ccvYWUjZfMeRtsyUVpCMpJ7uxaafEPFkEfrdNyC3CrJKbs8uItu1XsSkmL0jJjuXWG5apA5mpy0eU5GrN0ZcQfQAKqRu1fMC4ghXCede/Ktu/KtHIWdbamAPUETNaXFkV2UvnkIQn3Lyg4WVY7CwnUXPMKLWrKSgOSVwgp7FmCBATlo+Z2DQlo/OKExgJSqx0mYPI1VhzSBggXUjBWC7mCBKywJR10WWbjAto2OWWn5+1/afA+wp6x7FG+1yRKV1E9h3TxbJFZNDbcGSFXJWaVKrD0x1Tjxz5OCbJt58EsGvtOFRuydehFHgqOrnhdlyL36sNPhtql6TlZa6o2K9DQL2oTryM3H5Y+y/uPoKjEE+nKEtqelx9Jt6UAKP21IEJaXKE+i3vv8Vp6k9aY24xL84z6e928CihsFFDaBJSztbPmJCDXmm/O4n2I2i2VxQ/gjOUOxmS1kJKSeQJpoVSJXJ7HMmwSENoE5A/k9NbntE22AmrxCXPbax/PzV19blso8DG0LelqNLWSJlTY5cUF/dun7/uOBe+iMHx9DQcrby8vj4E/8lIUobuvtVb6G1APOqNAy0VJX8JiZY4FSpXv3X5L9c+7CIdC6QL8lwoOUiYrszpHZX+jMXzlWyWInGE9CgaLbHKc3Y5NZv5dwXSU67f8Lrwhuq8/488QvrOlNJ1OjgBDbiu0a0WzcDuI7BpSGHoifdObe8Nmbxq1wxNDY0+mN0navQtysiVeRv20HXhOl5DC2trapMNhXipXDPgKpeUDLoa8FnC4loz8LbsiUd9XBnOnhd14vNFk/B43N8gLYrPZvC9wT2UYCmtfrGOUWrxHCnGcUqO4jIRqiTNqHbQohlTgd93QuH7hVV1iR/pdkUQC7w4Oibe72VHnrIEnrgWXloZyT2J7GiR7bTDv+HLWhM8sPmWnHE4pK3ig2VFkmHdy7w2ei5fKRytWV0zFO55/RklrA6U1ZOli8Pvg4rXcRQU2mowjefDck8c27F4MkoPq2NUFzf3KMwdY5NUf3QUlpVbGVlz28AJzslNryZ3f9g4dbTpG+5+twbHQLb7YYV1y4TnQh7mJFBB3qT4FQALp4Up6ouEPxPY3yvRw45snsFbGksGmoag96LJt6fiLQ03dPI+VvqBimzVQCEPfBaRLXho8k7Nf7yLZ/sZee/O24+XGN1ToBkjgM26OFnAaLz/HeD+GP88LL+gH6npLrE2UitX3ZhqW+C3K0RZAzgeNUU6Vf1Z23ptOk4cpJLCIKznt8pCC5s80UflzcFjVoUxpZ6ezBkouV47mdgoBkv7+BV6vPIg=
sidebar_class_name: "put api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
+>
-
-
-
-
+ path={
+ "/human-evaluations/{evaluation_id}/evaluation_scenario/{evaluation_scenario_id}/{evaluation_type}"
+ }
+>
Updates an evaluation scenario's vote or score based on its type.
Raises:
- HTTPException: If update fails or unauthorized.
+HTTPException: If update fails or unauthorized.
Returns:
- None: 204 No Content status code upon successful update.
+None: 204 No Content status code upon successful update.
-
+>
-
-
-
-
-
-
-
-
-
-
-
-
-
-
\ No newline at end of file
+
+
diff --git a/docs/docs/reference/api/update-evaluation-scenario-score-router-human-evaluations-evaluation-scenario-evaluation-scenario-id-score-put.api.mdx b/docs/docs/reference/api/update-evaluation-scenario-score-router-human-evaluations-evaluation-scenario-evaluation-scenario-id-score-put.api.mdx
index 0b747730b6..6191952abf 100644
--- a/docs/docs/reference/api/update-evaluation-scenario-score-router-human-evaluations-evaluation-scenario-evaluation-scenario-id-score-put.api.mdx
+++ b/docs/docs/reference/api/update-evaluation-scenario-score-router-human-evaluations-evaluation-scenario-evaluation-scenario-id-score-put.api.mdx
@@ -5,73 +5,326 @@ description: "Updates the score of an evaluation scenario."
sidebar_label: "Update Evaluation Scenario Score Router"
hide_title: true
hide_table_of_contents: true
-api: eJzdVU1v4zYQ/SvEnFpAsd1gTzo1bQMk2HYb2EkvjmGMpbHFrURy+ZGNK+i/F0PKtvyR7PZaX2yTw8eZN28eW/C4cZDP4S40qK5uX7AO6KVWDhYZlOQKKw3/hxyeTImenPAVCVdoS0KvBSpB+0PCFaTQSj16Vs9qitKRy5+VEELcPT4+3L4WFMFyMSP7QlaQtdoKuY6YA5wQrxJrlLVLWOSDVTuwT1pRLq4nH8QnLX7VypPywnn0wYlClySC4WRCUZBz61D3eCPIQBuy8Y77EnJI68vDzctdBctY4dLq4MkuK2ZnEOYuHrm4KMtlj2WChwwMWmzIk2XaW1DYEORw+SRkIJl5g76CDCx9CdJSCbm3gTJwRUUNQt6C3xpGcd5KtYEMvPQ1Lxz6KWY9rLgvoesWCY2c/0WXW4Y4BS8SrbyFxtSyiDDjz4610A7uNpY59ZJcWteWBimp0KzIDlKaxYCuy3YRevWZCn9U3ryHWVyqZFdIBEqahK5jQEvOaOVSIteTCX8dS3h2kMS0D4bvLzXd8uH6+hz4L6xlmYi+ZU3/B9QTAkvyKGv+JT017jyg1sXRLqrtn+uopWMVdNl+RSpPG7LQLQ60o7W4HfTld50ShC6Dxm3eU9Uf5BxuCPZg7wgwDvgj736r5VxXurqPG3T/QG9i9+0yfkv0XbpsF8JWdAaYetuQrzQbw25WfQU5jOP4Xw3Gf3xhXsft5SHuxknMGbhoemnug60hh8p74/LxuKh1KEe4IeVxhHKMRgLPqKMiWOm38cjNw/1H2t4RlmQhny+GATNWU9LHcdi+NWjkR2Kmese5Cb7SVv6Tmt4bTZVOMRWs0+nBIm5fsTE1DUZ80vGhtY539NTexArEzcM9ZMC1pumYjH4aTeD0PTkK5mnBIk7LLsG4DdkJVXuSIANq4qiAJ2x+Pmxw+kY736Aa4CWrEJcsMVqJmEavP02zPQzy/+MF7PXg6dWPTY0yTnzkuO0FP4czwTPZ5+qGDPI3X669hVfaeQZt2xU6erJ11/Hyl0CWhb3I4AWtxBUraN5CKR3/LiFfY+3onXb8MO3t40fxVl078StWPmfK/yCDv2n79qsbbbLazVnbB/csX0UzO4CdeTv7bjpxU3C3341dDDzn4ekRMlj173GjSz5i8Sv7JH5NeWuT2sEPNq+1UKPaBHbjHBIkf/4F55hy1w==
+api: eJzdVU1v2zgQ/SvEnLaAYrtBTzqt2wZI0N1uYCe9OIYxlsYWW4lkSSqNV9B/L4aUbfkr3b3WF9vk8HHmzbzHBjyuHaQzuK0rVFc3z1jW6KVWDuYJ5OQyKw3/hxQeTY6enPAFCZdpS0KvBCpBu0PCZaTQSj14Uk9qgtKRS5+UEELcPjzc37xkFMBSMSX7TFaQtdoKuQqYPZw6XCVWKEsXscjXVm3BPmtFqbgevROftfiglSflhfPoaycynZOoDSdTZxk5t6rLDm8ACWhDNtxxl0MKcX2xv3mxrWARKlxYXXuyi4LZ6YW5s0fOLsp80WGZ2kMCBi1W5Mky7Q0orAhSOH8SEpDMvEFfQAKWvtfSUg6ptzUl4LKCKoS0Ab8xjOK8lWoNCXjpS17Y91NMO1hxl0PbziMaOf9e5xuGOAbPIq28hcaUMgsww6+OZ6Hp3W0sc+olubiuLfVSUnW1JNtLaRoC2jbZRujlV8r8QXmzDmZ+rpJtIQEoziS0LQNackYrFxO5Ho3463CEp/uRmHTB8N9Ljbe8u74+Bf6Cpcwj0Tc80/8D9YjAnDzKkn9JT5U7DSh1drCLavPPKszS4RS0yW5FKk9rstDO97Sjtbjp9eUvHROENoHKrV+bqr/JOVwT7MBeGcAg8Afe/VXLua54dRfX6/6e3sju5TI+RvrOXbYNYSs6AYy9rcgXmo1hq1VfQArDIP+rnvyHZ/Q6bM6LuB3GYU7ABdOLuq9tCSkU3huXDodZqet8gGtSHgcoh2gkHNvvOOyKDxwKLGBHWW2l3wS88f3dJ9rcEuZkIZ3N+wFTHrU4PIdhu76hkZ+IaezsaFz7Qlv5b5yIzoWKeIp54iGe7P3j5gUrU1JP/6OWD610uKPjvSvgPWbfSLG3MRmxttHg7WB0qeLugBjf33WywizIaptsCIPkiNMdm5AAVUFT4AmrP/cbXIrRzleoenjRU8Q57wyeIybhUThOt9kr/vd4KrvZ8PTih6ZEGawhcNx0ypjBiTKY7FMZQALpxSdu5/WFdp5Bm2aJjh5t2ba8/L0my0M+T+AZrcQlT9OsgVw6/p1DusLS0Svt+GPS+cwbcamurRAUq4Az5X+QwDfaXH6eg58WW801XXDH8lVwvT3YySPABh1PjDPu9qux85453T8+QALL7uGudM5HLP5gQ8UfMW9tYjv4Zee1BkpU65ptO4UIyZ+fGoWBZg==
sidebar_class_name: "put api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
+>
-
-
-
-
+>
Updates the score of an evaluation scenario.
Raises:
- HTTPException: Server error if the evaluation update fails.
+HTTPException: Server error if the evaluation update fails.
Returns:
- None: 204 No Content status code upon successful update.
+None: 204 No Content status code upon successful update.
-
+>
-
-
-
-
-
-
-
-
-
-
-
-
-
-
\ No newline at end of file
+
+
diff --git a/docs/docs/reference/api/update-evaluator-config-evaluators-configs-evaluator-config-id-put.api.mdx b/docs/docs/reference/api/update-evaluator-config-evaluators-configs-evaluator-config-id-put.api.mdx
index 414b3175c8..c5c9ddba0d 100644
--- a/docs/docs/reference/api/update-evaluator-config-evaluators-configs-evaluator-config-id-put.api.mdx
+++ b/docs/docs/reference/api/update-evaluator-config-evaluators-configs-evaluator-config-id-put.api.mdx
@@ -5,70 +5,366 @@ description: "Endpoint to update evaluator configurations for a specific app."
sidebar_label: "Update Evaluator Config"
hide_title: true
hide_table_of_contents: true
-api: eJzNVktv2zgQ/ivEnLaAameDPelUNw3QIN1dI49eHMNgpLHNrkSy5CitV9B/L4aULcl2nCKX1hfb5Dy/+WaGNZBceUhncPkki0qScR7mCeToM6csKaMhhUudW6M0CTKisrkkFLgVF5nRS7WqnGRhL5bGCSm8xUwtVSaktaMH/aBvkCqnffqghRDik/I023m8CBY+vJ+nYiIK5UmY5XMOhHn8ghn5ESRgLMbDqxxSiIEtdnqLqNcd+PbELw6FVL5Y2IogASudLJHQMSo1aFkipHBEARJQDI6VtIYEHH6tlMMcUnIVJuCzNZYS0hpoY9mEJ6f0ChIgRQUf7PIXEQBxlUPTzKMp9PTe5BvW37ecGU2oia+ktYXKAgbjL55rVfccW8cIkULP/2ImNUi9+XcZchsG1iS7E10VBXAk21D/Yd0m6cHwH25ea6zL+xo3bNUjkdIrv+CLGGxnV+a54vxkMe2lE5FoPURKnPR527oQn6OLptnX7ldwdhBSz9R94NkeeaFp2KRDb432MYfzszP+GrbSbZVl6P2yKsRNKwyvLqnKT/GL6ZTsyv6c1La01hnGYXHa5jRKiWj7gA4vU/0XljyBzKEkzBeSTgV7EaXEhO23Y+UlnUiKqPMCtcLkCFUZgL6P5iDaQRjHWmnAwr/Ozw+J91kWKo8j9NI5417PuhxJqiLwj7D0hwKFyQa3PzEklCZcoYNm3qEnnZObHsafTAyQq1L61aly/I3ey1XgdRQ5QU0GQ9zx7UuV47yi61auV4kO3oju82l8iPAdc7YV+Xh3Nz0wGGtbIq0Nr7vttqI1pDDulty4XXLj+sjOaoBbzz1tt1vlCkhhTWR9Oh5nhanykVyhJjmSaiytCh3lMaucok1QmUyvrnHzEWWODtLZvC9wy4yJHBiK7eCXVl0Hcrd7dVLR2jj1fyxsu1HXUYvTZS7edOvw8rssbYHdOuvquTeLuovDUcOGlV6aEFcL+SRkLSbTK0iA8Yldczb6c3QG+++hgTB3kcyoF1O85qYdwLsDlqMtQwsBoSzfdRccmTWeSql79uJwEfvvhf2w6q6hf9cXW8sCwu80toVUoZcDSnVL5VlXSB/BZTJDAumxJ9g8gbXxxGp1/Sg93ruiafj4a4WOCTtP4Ek6JR+5yrMacuX5dw7pUhYeT0D4x03b+m/Ec5FvSa2Z0YFekAIkEBl4LOAw39bb5qlbyYvo9G2YQp2lg6HMAzNqTLIMLZ2UnfeGxfT+DhJ4bB+UpclZxclvPODktxi0CRiE5g1nNRRSryoeoylEk/z5ATGJP5A=
+api: eJzNVktv4zYQ/ivEnFpAtdOgJ53qZANskG1r5LEXxzAYaWxzVyK55Ci7rqD/XgwpW5JfKXJpc0lCzovf982MaiC58pDO4OZVFpUk4zzME8jRZ05ZUkZDCjc6t0ZpEmREZXNJKHBrLjKjl2pVOcnGXiyNE1J4i5laqkxIa0fP+lnfI1VO+/RZCyHEJ+Vptst4HSJ8uJqnYiIK5UmY5akEwrx8wYz8CBIwFuPhbQ4pxMIWO79F9OsOfHviF4dGKl8sbEWQgJVOlkjoGJUatCwRUjjiAAkoBsdKWkMCDr9VymEOKbkKE/DZGksJaQ20sRzCk1N6BQmQooIPdu8XEQBxm0PTzGMo9HRl8g3770fOjCbUxFfS2kJlAYPxF89c1b3E1jFCpNDzf/ElNUi9+WsZ3jYsrEl2J7oqCuBKtqX+yb5N0oPhK27eG6x79x1uOKpHIqVXfsEXsdgursxzxe+TxbT3nIhEmyFK4mzOhzaF+BxTNM2+d5/B2UFJvVBPQWd74oWm4ZAOvTXaxzdcXlzwr2ErPVRZht4vq0Lct8bwbkpVfk5fLKdkR/spqy211hnGYXE+5jRaiRj7QA5vS/0/pDyBzKEkzBeSzhV7Ha3EhOO3Y+UtnyiK6POGtMLkCKwMQN9Hc1DtoIxjrTRQ4W+Xl4fC+ywLlccReuOcce9XXY4kVRH0R1j6Q4PCZIPbfzEklCZcoYNm3qEnnZObHsafTCyQWSn96hwdf6D3chV0HU3OSJPBEI98+xZz/K6YurXrMdHBG9E9/YwPEb5jybYmHx8fpwcBI7cl0trwuttuK1pDCuNuyY3bJTeuj+ysBrj13Ot2u1WugBTWRNan43FWmCofyRVqkiOpxtIq2P8UmIRbcc2mod08ZpVTtAnxJtPbO9x8RJmjg3Q27xs8sJyiQIZmO26kVXdB+e3SnVS0Nk79HVlv1+06ejEWLNT7blfe/JClLbDbdR3Ze4OquzicQxxY6aUJdbV8tI++ktlX1NyqDGDE42L06+jiFEqtg5hMb9t2kxn16otm3N0DHnYMcOVl6DUglOXv3QVXaY2nUupevDiFxP6HxX55ddf5/9dPu1YRhD9obAupQtMHlOpW87OOVB/BZdVDAumxb7V5Amvjid3q+kV6fHJF0/Dxtwodi3eewKt0Sr4w47MacuX57xzSpSw8noHwp/t2RvwsTlW+FbhmdQepQQqQQFTjsYLDIFxvG6luLa9j0l/CuOoiHUxvnqzRY5JlaOms7bw3VaZPj5DAS/vlWZqcXZz8zpNQfo9Fm4BBaORwVkMh9arieZtCDMk//wD3SU4f
sidebar_class_name: "put api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
-
-
-
-
-
-
+
+
Endpoint to update evaluator configurations for a specific app.
Returns:
- List[EvaluatorConfigDB]: A list of evaluator configuration objects.
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
+List[EvaluatorConfigDB]: A list of evaluator configuration objects.
+
-
\ No newline at end of file
+
+
+
diff --git a/docs/docs/reference/api/update-human-evaluation.api.mdx b/docs/docs/reference/api/update-human-evaluation.api.mdx
index d18c3e78e4..f9cdfce2dd 100644
--- a/docs/docs/reference/api/update-human-evaluation.api.mdx
+++ b/docs/docs/reference/api/update-human-evaluation.api.mdx
@@ -5,73 +5,346 @@ description: "Updates an evaluation's status."
sidebar_label: "Update Human Evaluation"
hide_title: true
hide_table_of_contents: true
-api: eJzlVt9vo0YQ/ldW89JWInYa3RNPpT03QZcmkU2uUh3L2sDE7BV2ud0hiQ/xv1ezYIPtS9Q+1y+G+bUz38x8SwMkNw7CJVzVpdRns2dZ1JKU0Q5WAWToUqsqfocQ7qtMEjohtcC93Q9OOJJUu8mDftBzqRy68EELIcRVktzNXlP0/qGInwTlKFJT1KV2Qmn/SuhIOCSRGaENiVJSmosXRblXK13VtDd+llZJTd1RSLXVu7NujMZQROLi/IO4MeI3owk19amJ1GQYCKUzlUpSeiMol+Qj1r4m8SKdcHWaonNPdTGBAEyF1hcYZxBCZ7bOGaT1UDwEUEkrSyS0jGIDWpYIIQwma5VBAIrxqyTlEIDFr7WymEFItsYAXJpjKSFsgLYVOzuySm8gAFJUsGDoiogzaNtVFwQd/WqyLXsex0y7+lklq6rwZRs9/eK4kc3oyMpyoaTQebmHy3vp7e2Tr+g4KdR1yQMz+xxd30dJfHuzjm/iJI6u479mHyEYKxZJNE+Ohb/HN/Hi6i3p+s84uVrP5vPb+eLIIoqvj72iy8v57PJAv/oebgtf2IxTb4N9TbouCmhXbRvsJObxC6Y0gt6vxRCnWwFoW/ax6CqjXQfdxfk5/x1uzGI/UmLeG8O/b053yoeLi9PAn2Whsm4iZtYa+x+iHrU8Q5Kq4CdFWLpTg8KkB9q3JmOEq9KEG7QM7R5Zaa3cjoC9Nl2C0AZQus174/8HOic3CPtg72wKgyES1n6vq8OaLH1d3dG93WhwBng7dN8u42MH37sjlCR3JwG73pZIuWF+qWryXEI5hDD1NHM2cIibNgeE0kIADu3zjnRqW0AIOVHlwuk0LUydTeQGNcmJVFNZKWDKcJjWVtHWu0R38SfcXqHM0EK4XI0NFjwqXfMPzfa4y0p9Qoahp7uoptxY9W1Hip7u8s6L6+QhnA+MNXuVZVXgmHHeopOWgz0Zf3aPZ+QrE9FdDAEwBt1KnE9+npzD8Z11YMwrIlO/IrvEvRqCIwj34DHflX4/gFCWvwwKLqsyjkqpR/E6fhCeNcRsfE8cpNUM2/r/ulX7+SF8pWlVSOXX32Pf9NO/hJPphwDCwwt1FUBuHLF10zxKh/e2aFsWf63R8oSvAvBlPfLILBvIlOPnDMInWTh8px8/znuS+Em8lfBuCzSvACfGbxDA37g9ufs9B+a7PWt6mx7JM89UQ4wT4mZS7TyilFv+ru1qRCh39wkE8Nh/HpQmYxcrX5gE5UuXrqk6ePn7gWUNFFJvaqbaELqQ/PsHaeR2tA==
+api: eJzlVk1v20YQ/SuLubQFGMk1cuKpTKLaRFzbkOgUqC0Ia3IsbkLuMrtD2yrB/17MkhIpKTLac32xuPOx897MPLIBkmsH4T1c1qXU72bPsqglKaMdLAPI0KVWVfwMIdxVmSR0QmqBO7+fnHAkqXaTB/2g51I5dOGDFkKIyyS5nb2m6ONDET8JylGkpqhL7YTS/pHQkXBIIjNCGxKlpDQXL4pyb1a6qmnn/Cytkpq6q5Bqq7d3XRuNoYjE+dl7cW3ER6MJNfWlidRkGAilM5VKUnotKJfkM9Yek3iRTrg6TdG5p7qYQACmQusBxhmE0LmtciZpNYCHACppZYmElllsQMsSIYTBZaUyCEAxf5WkHAKw+L1WFjMIydYYgEtzLCWEDdCm4mBHVuk1BECKCj4YuiLiDNp22SVBRx9MtuHIw5xph59NsqoKD9vo6VfHjWxGV1aWgZJC5889XT5Kb26ePKLDolDXJQ/M7Et0dRcl8c31Kr6Okzi6iv+afYJgbFgk0Tw5PPw9vo4Xl6dOV3/GyeVqNp/fzBcHHlF8dRgVXVzMZxd79uWPeFt4YDMuvQ12mHRdFNAu2zbYnpjHr5jSiHq/FkOebgWgbTnGoquMdh1152dn/G9/Yxa7kRLz3hn+fXO6W96fnx8n/iILlXUTMbPW2P+Q9aDlGZJUBf9ShKU7dihMumc9NRkjXpUmXKNlanfMSmvlZkTslekKhDaA0q3fGv8/0Dm5Rtgle2NTmAyRsPVHXR3W5N7j6q7u/UaDM9DbsXsaxqeOvjdHKElujxJ2vS2RcsP6UtXktYRyCGHqZebdoCFu2uwJSgsBOLTPW9GpbQEh5ESVC6fTtDB1NpFr1CQnUk1lpeBQyiNvFR/ZFVhPHKa1VbTx+aLb+DNuLlFmaCG8X44dFjxH3WTsu+2aIiv1GZmjXgujmnJj1d9bxfRamHdRTAJP6HyQs9mrLKsCx3J0SmtaTvZk/N092T2wDzL9hpqVl0nqMJ9Nfp2cnWKiDxDRbdzvkkz9Lm1BeDcIDrjesczCWPpFAkJZ/jYYGGJlHJVSj/J1QiK8vIjZ+IWyV14zrPX/6/XbzxLhK02rQiqvE577pl+TezhaEwgg3H/zLgPIjSP2bppH6fDOFm3Lx99rtDztywA8rEcen/sGMuX4dwbhkywcvtGPn+e9mvwiThW83QjN68CF8RME8A03Rx8JXizz7c41vU/P5DsvaUOOI4Vn9e0iopRb/qbvcqQ8t3cJBPDYf0eUJuMQK19YLeVLV66pOnr5Q4PPGiikXtesySF0KfnvH7lMhUM=
sidebar_class_name: "put api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
+>
-
-
-
-
+>
Updates an evaluation's status.
Raises:
- HTTPException: If the columns in the test set do not match with the inputs in the variant.
+HTTPException: If the columns in the test set do not match with the inputs in the variant.
Returns:
- None: A 204 No Content status code, indicating that the update was successful.
+None: A 204 No Content status code, indicating that the update was successful.
-
+>
-
-
-
-
-
-
-
-
-
-
-
-
-
-
\ No newline at end of file
+
+
diff --git a/docs/docs/reference/api/update-testset.api.mdx b/docs/docs/reference/api/update-testset.api.mdx
index fd669ceab9..11a5adda82 100644
--- a/docs/docs/reference/api/update-testset.api.mdx
+++ b/docs/docs/reference/api/update-testset.api.mdx
@@ -5,36 +5,33 @@ description: "Update a testset with given id, update the testset in Postgres."
sidebar_label: "Update Testset"
hide_title: true
hide_table_of_contents: true
-api: eJzlVU2P2zgM/SsCTzOAN5kOevJps90CHXQ7G0wze8kEhWIxtlpbciU6adbwfy9oKbGTaWa35+aSRCQfycevFkjmHtIlLNCTR/KwSkChz5yuSVsDKTzWShIKKSioiJ2mQuR6i0ZolYgmyKnAo4Y2Ym495Q795Mk8mZnLffpkoviTVuLKk7tOhVbCbo6mgm3JijVGUDV5MpnfKklSXN3jLgZ5nYp73In+maxwWJcyCxHYUh2i6D0/IDXOsHNPLhWLAn/k8+ANErA1OsmZ3ylIIQg+RURIoJZOVkjomLQWjKwQUhgSgwQ0k1ZLKiABh18b7VBBSq7BBHxWYCUhbYH2NVt6ctrkkABpKvkhpijuFHTdKiCgpz+s2rPZOWBmDaEhFsm6LnXWxz797Ll07chf7Tgz0uj5X4j7chT3LO8SiOSzqiaseluplGYnspyPMEM4Ec+uP2NGDBAfpHNyP8J/E3G77txmzNkyxDmEsRpFeOwG6DqGcehra3zI7/bmhr9OG/ljk2Xo/aYpxUNUhv/PYPDy+vb2OfA/stSqNxNvnbPuJ1DP6qKQpC5P6D5VKG12Wgyz/3vTt+JpKbvk+KINYY4OutXlevxlQ4BcssrnL7XGB/Re5jgU97JqT4ZYsPS/Cs15BddRb1Tqgd7A7uU0/gz0/cjZQeXdYjF/BhhqWyEVlqe+bsKoUwEpTONw+2k7jHkHCXh028MeaFwJKRREtU+n06y0jZrIHA3JidRTWWvgWfaYNU7TvjeZze/e4/4dSoUO0uVqrPCR2yMU/FTtyLWs9Xvk1OMGmjVUWKf/DVWMS6gIVpwbN97DsErefpNVXeKwCobiHUd+2fYdo83G9o4jgbM+LTGb30ECTECYgZvJq8kNnN+OE2WeCZnRyGkQQ3LG35E5SACrfiCAUFa/DwLOqbaeKmlGePFULY7r+iSadpjKX+yoxaYh/EbTupS6n/Oe8za2+fJwwzwkkI7u2SqBwnpijbZdS4+Pruw6fv7aoONWXiWwlU7LNbfHsgWlPf9WkG5k6fGFIlw9xA1wLS4FeWh3w72+lWXD/yCBL7g/vbt9rxaHaWqjwpvg67d+Bw0Az1Yyr8tgMcsyrOlF3dVoVcwfF5DAOl7nyio2cXLH603uQqy2T72f5v6thVKavOElmkKA5M93EkZMcw==
+api: eJzlVVFv2zYQ/ivEPSWAZmfBnvQ0NyvQoFtmpM5eHKOgxbPEViJV8mTXE/TfixNpS3bqtHueX2yTd9/dfXf3sQWSuYd0CQv05JE8rBJQ6DOna9LWQApPtZKEQgoKJmKnqRC53qIRWiWiCfdU4NFCGzG3nnKHfvJsns3M5T59NvH6o1biypO7ToVWwm6OroJ9yYo1RlA1eTaZ3ypJUlw94C4meZ2KB9yJ/piscFiXMgsZ2FIdsugjPyI1znBwTy4ViwK/F/MQDRKwNTrJld8rSCFcfIyIkEAtnayQ0DFpLRhZIaQwFAYJaCatllRAAg6/NNqhgpRcgwn4rMBKQtoC7Wv29OS0ySEB0lTyQSxR3CvoulVAQE9vrNqz2zlgZg2hIb6SdV3qrM99+slz69pRvNpxZaTR87+Q9+UsHvi+SyCSz6aasOp9pVKag8hyPsIM6UQ8u/6EGTFAPJDOyf0I/y7idt25z5izZchzSGM1yvA4DdB1DOPQ19b4UN/tzQ1/nQ7yhybL0PtNU4rHaAw/z2CI8tvt7Uvgf2SpVe8m3jpn3X9APeuLQpK6PKH71KC02WkzzP7vTT+Kp63skuOJNoQ5OuhWl/vxpw0Jcssqn782Gn+h9zLHobmXTXsyxIJvf9RoriuEjnajVg/0BnYvl/FHoO97wQ4m7xaL+QvA0NsKqbC89XUTVp0KSGEal9tP22HNO0jAo9sedKBxJaRQENU+nU6z0jZqInM0JCdST2Wt4VxVZ/2tuGNT4EX3mDVO077Hm83v3+P+HUqFDtLlamzwgWcnTMOp2bERstbvkXmJ8jRrqLBO/xtaHBWqCF5cOE/l46Azb7/Kqi5x0Imhs0c9WLb9OGmzsX3gyG6s6o3MPqNhMWSGQsE3k18nN5doiA5iNr+PyyMzGiUQzCA5I/pIMSSAVb85QCir34cLrq+2nippRnjxTVscdf0kq3ZY3//Z6xcHiPArTetS6l4Qes7buA/Lw2PnIYF09PCtEiisJ7Zo27X0+OTKruPjLw06HutVAlvptFzzqCxbUNrzbwXpRpYeX2nC1WOUimtxKcnD6Bue+60sG/4HCXzG/ekD3c9tcdisNhrchVi/9GI1ALzQbtbV4DHLMqzpVdvVSFPmTwtIYB2f8coqdnFyxzoodyFX25feb3Z/1kIpTd6w2qYQIPnzDU5rWwI=
sidebar_class_name: "put api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
-
-
-
-
-
+>
+
Update a testset with given id, update the testset in Postgres.
@@ -50,29 +47,294 @@ str: The id of the test set updated.
as={"h2"}
className={"openapi-tabs__heading"}
children={"Request"}
->
-
+>
-
-
-
-
-
-
-
-
-
-
-
-
-
-
\ No newline at end of file
+
+
diff --git a/docs/docs/reference/api/update-variant-parameters.api.mdx b/docs/docs/reference/api/update-variant-parameters.api.mdx
index d3d0f14354..9792a11377 100644
--- a/docs/docs/reference/api/update-variant-parameters.api.mdx
+++ b/docs/docs/reference/api/update-variant-parameters.api.mdx
@@ -5,77 +5,370 @@ description: "Updates the parameters for an app variant."
sidebar_label: "Update Variant Parameters"
hide_title: true
hide_table_of_contents: true
-api: eJztVk1v4zYQ/SvEnLKAaqdBTzrV+QDW3e6ukY+9OIbBSGOLW4nkklQSV9B/L4YULSneeIugQHtoLo7I4eO8mTczbMDxrYV0CV+4EVw6C6sEcrSZEdoJJSGFO51zh5a5Apnmhlfo0Fi2UYZxybjW7DGcndzLezkzW5veS8ZYXF6LnJ1YZ96l7LZANr9kauPBBkeZU6z290zCWc13peI5Owm3d94t4vWLsN1BRuNMSceFFHLr8QNgPnDae3jNhcXo4/vb28XVc4aebMrm3jODTFgih8Yow54KUSJzZueBo6MvKQRsdLWREfy3m8+frtFqJS2mbOa/mekWXvN2GJWB55CA0mg4+TnPIYVgv45B7k0hgcFHumxA8gohhT4fkICg1GruCkjA4LdaGMwhdabGBGxWYMUhbcDtNJ20zgi5hQSccCUtdPlg8xzadhUQ0Lpzle/o2EtAoorS0RbXuhSZpzH9aklgzeA+bYikE2j914BGAzzPBR3j5WJgFS7o/FQPXzFzAz8XPUJLblSVcOsKreVb9Jhy93njYzRm2ib7FVmXJRDHiHnhUdjHDqVtD6/v+S+HJAYgR2UNbUuoUSme/dnpKf2MS/OmzjK0dlOXLOoM3hxtkb81IqQCcvdRWBHu6CyFdLhFM0jIdTRqE6hULjYC8/XD7pjWPnZm7HwXkig3YnvoflhfB62/jnbhzdgnMmuTf1hiP5DC0MXVS5cuzz07g9QF1twdJRGs2Mz9a7LeZ3ucx31+RlQG18y07lTfS8Gr/Zezs0OBf+GlyL182RX14rerO0fHRel17rCyhwalyka7fyOGUd7tqg8WN4bvBqn6XQUHveDt9qjQY+gj2OumPhjslnZ/lCjiFa7u7FbDNh7DG6L7Oo3LEL7vXRZNaJQeAIbcVugKRUNL186PJ1dACtNuINlp04+mdjqaZBbNYxxjtSkhhcI5bdPpNCtVnU/4FqXjEy6mXAuvZ4tZbYTb+SOzxfwD7t4jz9FAulwNDW5IKSH3Y7N92LkWH5Ci0A3QWe0KZcSfIaHdDC3CKaJJGrzuJ+HVM690iQeT7Dsl24sLhNwo70MsGM+QzRZzSIBiESrjdPLz5BRevtVGxv6BkflKiQT8NiQvQrkPIiSAlS8TcMirX/sNoqeVdRWXA7wwxVh8DSyGmRs51vRl+/978r/ynuxU7vDZTXXJhe9RXhlNV6LL+GakhKaj9+P4WVMo68i8aR64xTtTti0tf6vRUCGukgD0QIpeNpALS//nkG54afGIWE6uu1b2jr3mcSxWSZX6yMuaviCBP3A3fvT6Nl3EXtB0Bhfhrp98M+0BDmYL9f1wYpZRfo/argY9b3F3Cwk8dE/jSuV0xPAn6tP8KfiqPHXfi/xaAyWX2zr0hgBJf38B1oTAMw==
+api: eJztVk1z2zYQ/SuYPTkzrOR6euKpiuOZqGkbjS3nIms8a3IlIiEBBABtqxz+984CpElJkZvccqguIoHdh3378cAGPG4dpCv4hFai8g7WCeTkMiuNl1pBCrcmR09O+IKEQYsVebJObLQVqAQaIx6j7+RO3amZ3br0Tgkh+uV7mYsz5+2bVCwLEvN3Qm8C2MhVeC3qcM4k+hrclRpzcRZP76Jb9Mcv4nYH2RtnWnmUSqptwI+A+SjoEOE1Skd9jO+Xy8XVc0aBbCrmITJLQjomR9ZqK54KWZLwdheA+0APKURs8rVVPfgfNx//viZntHKUill4F7ZbOBXtOCujyCEBbcgixznPIYVof98neTCFBEYv6aoBhRVBCkM9IAHJpTXoC0jA0tdaWsoh9bamBFxWUIWQNuB3hj2dt1JtIQEvfckLXT3EPIe2XUcEcv6tznfsdgjIVEl53kJjSpkFGtPPjhusGZ1nLJP0klx4G9FoAPNcshuWi5FVPKCLUz98psyP4lwMCC2HUVXS31fkHG4pYKrdx03I0T7TNnlZUXVZAnPsMS8DivirQ2nb4+MH/qsxiRHIq20NbcuofacE9hfn5/y3P5o3dZaRc5u6FH2fwfdnO57y28XFMfAnLGUe3MQVz8APoB7UMCePsuQn6alyxwalzvZ2v6MkUnnakoV2PeQercXdqPJ/6hgg171y29ea+aWSPdhp05AMseTd/6o784pHd3br8fj06Y3ZPU3jXUzftw7rTVjCjgBjbSvyhWaxMLUPsuALSGHaCYGbNoMktNM9BXFkH3v5qG0JKRTeG5dOp1mp63yCW1IeJyinaCQcXhmzsCsu2TTMjqOsttLvAt5sMf9Au/eEOVlIV+uxwQ23UWyMfbOXmqCRH4hT1KnarPaFtvKfWO1O2IroxTngBr0e5OnqGStT0pG8fEMehs4DqTY6xNDlvCP4FrMvpFhOOVmR+/nk18n5qYx0DmK2mHcjhVkYqZ5MMIPkIOcv2YYEqArzBJ6w+n3YYKpGO1+hGuFFmRG9XC/GJd4LsBnm+/8L/2e58LuO9/Tsp6ZEGcQsdEbTzfKqv9S5oOneBb9/7xTaeTZvmgd0dGvLtuXlrzVZHsp1EoEeuLtXDeTS8XMO6QZLR680y9l1p3lvxKmI+8FVPLWPWNb8Bgl8od3+V0nQ86LXhaYzuIxn/RJUdwA4uoT4goges4zr+6rteiSOi9slJPDQfbtUOmcXi08s6PgUY9WBetClsNZAiWpbR52IkPz7F3A47+g=
sidebar_class_name: "put api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
+>
-
-
-
-
+>
Updates the parameters for an app variant.
Args:
- variant_id (str): The ID of the app variant to update.
- payload (UpdateVariantParameterPayload): The payload containing the updated parameters.
+variant_id (str): The ID of the app variant to update.
+payload (UpdateVariantParameterPayload): The payload containing the updated parameters.
Raises:
- HTTPException: If there is an error while trying to update the app variant.
+HTTPException: If there is an error while trying to update the app variant.
Returns:
- JSONResponse: A JSON response containing the updated app variant parameters.
+JSONResponse: A JSON response containing the updated app variant parameters.
-
+>
-
-
-
-
-
-
-
-
-
-
-
-
-
-
\ No newline at end of file
+
+
diff --git a/docs/docs/reference/api/update-variant-url.api.mdx b/docs/docs/reference/api/update-variant-url.api.mdx
index 3044551fb2..c16d6c0266 100644
--- a/docs/docs/reference/api/update-variant-url.api.mdx
+++ b/docs/docs/reference/api/update-variant-url.api.mdx
@@ -5,77 +5,332 @@ description: "Updates the URL used in an app variant."
sidebar_label: "Update Variant Url"
hide_title: true
hide_table_of_contents: true
-api: eJztVlFv2zYQ/ivEPW2AZqfBnvQ0bw1Qt+lqOHZfHCNgpLPEViJV8lRHE/TfiyMlS3bWrAP2uLxEou4+Hu/77qNbIJk5iHfwUVolNTnYR5CiS6yqSBkNMWyrVBI6QTmK7fpW1A5TobSQWsiqEl9D4uxe3+uFzVx8r4UQw/KDSsVPjuzPsdjkKJavhTl4pEmqICNqv8ks5Na2mCbxptOQe72WyuGw05vNZnXzlKCvNxbLA1eG1horTJLU1oljrgoUZBulsxHpsoyAjFRbPUC/vfvw5xpdZbTDWCz8u7D9glA6VYkkBj3mSDlaD9mjH6UTrk4SdO5QF8JYoQ3NIAJToZVc6zKFGEL0w9Cu2hYQgcUvNTr63aQNxC0kRhNq4kdZVYXf1Oj5J8f8tOCSHEvJT5VlcFLo+I2x4haoqRBicGSVziACUlTwwtYW0EUwEvVScC8PsUw5JzFlqeihROdkhr4u3Xw4QLy7ROii04quiwK6/Yj5h0cR73uUrouGUPP4CRPqG6EspqzQ0JpJuROooNG+yO36diWbwsgUOgYdQcjW6BcCg75L11dX/O9c83cjcQP/EP0wD53f9tfr6+fAH2WhUp8mblii/wL1gt0USSpPsCIs3fOAwiRnX3+AIqUJM7TQ7UcupLWymSjh1oQCWQely14SzYnZAez7ob4ZYsNf/0kHfK6wdR+3n8p0aG/o7veP8Tq07+82G0LYWJ4BBm5LpNzw+FY1Z1SScohh3ovTzdtRpt3cof2qEhYQP6F1ngQ/nZATVS6ez5PC1OlMZqhJzqSay0r5WXGY1FZR41MWq+U7bN6gTNFCvNtPA+5YJoH487BTz2Wl3iG3QMuS3xc15caqvwKbESjWZx6y+IwswPVoRDdPsqwKPBnLyOHUQcbVS48Y5QZKH4wvrG/0wh9bLFZLRkPrwqxczV7NruDyOjoL5tmRiZ+d4VT+M0QX/T11FiLA0g8OEMryt/EDn7kyjkqpJ3jBWcTgf1tvQmcVTfz5/7vyP74re+kSPtG8KqTyrtPfa2HodoP8HEQQT6QYpo3nbh9BbhxxbNs+SodbW3QdL3+p0fJo7XsRP7Icd2x++TBkLXzGxl9WnuNfvEVxeFGHobpwbHbTkLFIuM8vxu4nTrLabiCCx/7KL03KKVYe2f3kEWLwPx042Q+5X2uhkDqrw3wFSP77Bi81Yjs=
+api: eJztVsFu20YQ/ZXFnFqAlVyjJ56qJAaixm0E2cpFFowxORI3IXeZ3WFkVeC/F7NLirRcuynQY3URuZx5OzvvzSOPwLjzkK7hEzqNhj1sEsjJZ07XrK2BFFZ1jkxecUFqtbxWjadcaaPQKKxr9S0mTu7MnZm5nU/vjFKqX77XufrBs/sxVbcFqfk7ZbcBaZSq2KombDKJuY0rx0my6TjkzixRe+p3en97u7h6zCjUm6r5Vioj56xTNssa59W+0CUpdgdtdgPSeRkRmbhxpof+7ebjH0vytTWeUjUL98p1C0qbXGfIAroviAtyAbJD36NXvsky8n7blMo6ZSxPIAFbk0OpdZ5DCjH6vm9X40pIwNHXhjy/sfkB0iNk1jAZlkus6zJsas30sxd+juCzgiqUq9oJOGvycidY6RH4UBOk4Nlps4MEWHMpCytXQpvAQNRrwZ081DyXnMxWleb7irzHHYW6zOHjFtL1OUKbnFZMU5bQbgbMtwFF/d6htG3Sh9qHz5Rx1wjtKBeFxtaMyh1BRY12Ra6W1ws8lBZzaAV0AGHXUFiIDIYuXV5cyN9Tzd8MxPX8Q/LdPLRh218uL58Df8JS5yFNXYlE/wXqGbs5MepAsGaq/POA0mZPnn4HRdow7chBuxm4QOfwMFLCtY0Fig4qv3tNNCdme7CXQ0Mz1K08/ScdyLni1l3cZizTvr2xuy8f411s399t1oeIsTwDjNxWxIWV8a0byaiRC0hh2onTT4+DTNupJ/dNZyIguSLnAwlhOqFgrn06nWalbfIJ7sgwTlBPsdZwbsSz8FS9ldAwSJ6yxmk+BLzZYv6BDu8Jc3KQrjfjgBvRUFTF07ATIVjrDyT9MVjJ/azhwjr9Z6Q6AS0FFDFLGiDqXA4udfWIVV3SyXUGgsf2MqyeG8igRdBma0NhHQvdqd9g9oVMLojkfGzIxeTnycVLbeoS1Gwx74YMszBk/QlDGCRnRJwogASoChMGTFj9OjyQ89fWc4VmhBctSPVGuQpu9aSykZH//1L9j1+qnYyZHnlal6iDPXUvwDid616KHhJIR7KMYykDukmgsJ4l9nh8QE8rV7atLH9tyMmYbTpBP4g01+KSRT9wR/hCh/BWCxz/FLxMwssmDtiZtYvtxoxZJn1+NXYzspzF6hYSeOi+DSqbS4rDvdgk7iGF8I0hyWHgw9oRSjS7Js5ahJTfXzRScMo=
sidebar_class_name: "put api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
+>
-
-
-
-
+>
Updates the URL used in an app variant.
Args:
- variant_id (str): The ID of the app variant to update.
- url (str): The URL to update.
+variant_id (str): The ID of the app variant to update.
+url (str): The URL to update.
Raises:
- HTTPException: If an error occurs while trying to update the app variant.
+HTTPException: If an error occurs while trying to update the app variant.
Returns:
- JSONResponse: A JSON response indicating whether the update was successful or not.
+JSONResponse: A JSON response indicating whether the update was successful or not.
-
-
-
-
-
-
-
-
-
-
-
-
-
-
+>
-
\ No newline at end of file
+
+
+
+
+
diff --git a/docs/docs/reference/api/upload-file.api.mdx b/docs/docs/reference/api/upload-file.api.mdx
index 7e1762784d..d98d2edb79 100644
--- a/docs/docs/reference/api/upload-file.api.mdx
+++ b/docs/docs/reference/api/upload-file.api.mdx
@@ -5,75 +5,363 @@ description: "Uploads a CSV or JSON file and saves its data to Postgres."
sidebar_label: "Upload File"
hide_title: true
hide_table_of_contents: true
-api: eJztVsGOGzcM/RWBpwSY2JtFT3OqmybINm3WiJ1cvMZCHtEepRpJkThGXWP+vaA0tsfrzW7Qc04jS+Qj+fgoeQ8kNxHKBcwxUkSKsCxAYayC9qSdhRI+e+OkikKKN7MvwgXxx+z2o1hrg0JaJaLcYhSaolCSpCAnpi7SJmAc3dk7OwmbWN7ZNoHc086jKMVbTTUGIcXX6CxDVnGbEEd3VgiRwV/kwO+0wZelmNd4GZ+cyMC9H+Ui7q1sULy4TRVI87IUVKNIm26d1r2h0Gvhg9tqhSpl+wmpDZYTZjilK8qRA8bW0ME7x2TPCmMcQQHOY5Ac7UZBCX2xnCIUEPBbi5F+c2oH5R4qZwkt8bJpDWkvA43XLjSvmD/ejlWNTVr5wMCkMfKvAYf8M38hUtB2AwWQJoPHfok5H3cFpCweMeeQkqCElbYy7AYAzDh7DtlkBGl3t2soFw+xuuK4Y1tjoFuesHpZiY+M0TFoNnSrr1hRz44OqFiDKdWBM1N2PySzY4STB4UW00b0zsZM0vXVFX/OJTxrK+7UujXiU28MxbAT0nujq9TBMWvyqTZo9RT7N4qpO1D2PatMRwFVQEmo7iU9Zf0mW4kJPUuhVtBHPwN/0JEZ0kw33uCRjS5R+8v19SV5X6TRKlEj3obgwv9nTiFJbRKHhE28NDCuOjv9AcVpS7jBAN3yxIwMQQ4F/afLCTLjTdw8RfVfGKPcZPk/M2eJjH7MnukK15VD93aDfpzozex+v4zfM32PBTuYvJ/PpxeAubcNUu34dvIusouXVEMJ437K4zgPGhQQMWwxxMR7GwyUUBP5WI7HlXGtGskNWpIjqcfS6zTtEas2aNoll8n05gPu3qNUGKBcLIcGM1ZG7vW52ZFm6fUH3B1EXMKkpdoF/W9uYAGaJVlnLy5L27VL7j0Dk5ScmExvoAAuI4v4avR6dAUPn7YzYxa1rJKoD7HTMRQPWDjWDwVgkxQNhLL59XTAmTHPjbQDvP5mfpffhbNUBu/Czwf3Bx7cXi2E/9DYG6nTbKc27XtlLw7vV+QGZm0vC6hZ/eUC9vuVjPg5mK7j7W8tBpbvsoCtDFquWEwLvlPqg5D38Dfu+DbOjXqVJp/NTcupPPaY8z2VnSZVhZ4G5hf3Jo/JcUant7M5FLDq/zM0TrEPAyfc4rTkJLvuPxo/V5k=
+api: eJztVk2P20YM/SsDnhJAsbeLnnSqs02QbdqsETu5eI0FLdHWJKMZZYZa1DX03wvOyLa83o+g554kj8hH8vGR4x0wbgLkC5hT4EAcYJlBSaHwumHtLOTwpTEOy6BQXc2+KufVH7ObT2qtDSm0pQp4T0FpDqpERsVOTV3gjacwurW3duI3Ib+1bQS5421DKlfvNFfkFapvwVmBLMJ9RBzdWqVUAn+VAr/Xhl7nal7ReXx2KgH3fpyKuLNYk3p1EytA8zpXXJGKh24d33tDpdeq8e5el1TGbD8Tt95KwgJX6oJTZE+hNbz3TjHFs6AQRpCBa8ijRLsuIYe+WEkRMvD0o6XAb125hXwHhbNMluW1bg3rBj2P187Xb4Q/OQ5FRXV8a7wAs6YgvwYcys/0hMBe2w1kwJoNHfql5vK5yyBm8Yi5hESGHFbaot8OAIRx8RyyKQhotzdryBcPsbrscGJbY6BbHrF6WalPgtEJaDJ0q29UcM+O9lSKBmOqA2eh7G5IZicIRw/2LcWD0DgbEkmXFxfyOJXwrC2kU+vWqM+9MWTDTmDTGF3EDo5Fk8+1QZfPsX9dCnV7yp6ySnRkUHhCpvIO+Tnrq2SlJvwihbqEPvoJ+IOOzIhnum4MHdjoIrW/Xl6ek/cVjS4jNeqd987/d+ZKYtQmcshUh3MD44qTrz+hOG2ZNuShWx6ZQe9xKOg/XUpQGK/D5jmq/6IQcJPk/8KcRTL6MXuhK1JXCt3bDfpxpDex+3QZvyf6Hgu2N/kwn0/PAFNva+LKyXZqXBCXBrmCHMb9lIdxGjTIIJC/Jx8i7603kEPF3IR8PC6Ma8sRbsgyjlCPsdHw8LqYxK/qSkzjKghUtF7zNuJNptcfafuBsCQP+WI5NJiJbJIQTs0OPcBGf6TtXuE5TFqunNf/pO5moCWBKnlJzdquXXTv6elze4vFd7JSqtSZ0r4Y/TK6eKqY3kFNpte9+rGI6t/nEc0ge0DXgSjIgOoofWDC+rfjB8lSGlKjHeD1K/x9ukBOUhpcIP/fzD9xM/fKYfqbx41BHZdAbNOuH4HF/qIL0sA0BMsMKhmTfAG73QoDffGm6+T4R0tepLzM4B69xpUIayHLp9qLegffaStrOzXqTVwRYm5aSeWxW18WWnKaFAU1PDA/W7AyModhnt7M5pDBqv9zUbtSfAQ44mbHV0my6/4Fa4RmKA==
sidebar_class_name: "post api-method"
-info_path: reference/api/agenta-api
+info_path: reference/api/agenta-backend
custom_edit_url: null
---
+import ApiTabs from "@theme/ApiTabs";
+import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
-import ParamsDetails from "@theme/ParamsDetails";
-import RequestSchema from "@theme/RequestSchema";
-import StatusCodes from "@theme/StatusCodes";
+import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
+import MimeTabs from "@theme/MimeTabs";
+import ParamsItem from "@theme/ParamsItem";
+import ResponseSamples from "@theme/ResponseSamples";
+import SchemaItem from "@theme/SchemaItem";
+import SchemaTabs from "@theme/SchemaTabs";
+import Markdown from "@theme/Markdown";
+import Heading from "@theme/Heading";
import OperationTabs from "@theme/OperationTabs";
import TabItem from "@theme/TabItem";
-import Heading from "@theme/Heading";
-
-
-
-
-
-
+>
+
Uploads a CSV or JSON file and saves its data to Postgres.
Args:
upload_type : Either a json or csv file.
- file (UploadFile): The CSV or JSON file to upload.
- testset_name (Optional): the name of the testset if provided.
+file (UploadFile): The CSV or JSON file to upload.
+testset_name (Optional): the name of the testset if provided.
Returns:
- dict: The result of the upload process.
+dict: The result of the upload process.
-
-
-
-
-
-
-
-
-
-
-
-
-
-
+>
-
\ No newline at end of file
+
+
+
+
+
- );
-}
-
-
-```
-
-## Direct HTML Approach (Alternative)
-
-If you prefer not to use the component, you can add custom classes directly to regular DocCard:
-
-### Using data attribute for emoji:
-```mdx
-
-
-
-```
-
-### Using className for no icon:
-```mdx
-
-
-
-```
-
-## Supported Icon Types
-
-| Type | Method | Example |
-|------|--------|---------|
-| Emoji | `icon` prop | `icon="🚀"` |
-| Unicode | `icon` prop | `icon="★"` |
-| SVG File | `imagePath` prop | `imagePath="/img/icon.svg"` |
-| PNG/JPG | `imagePath` prop | `imagePath="/img/icon.png"` |
-| None | `noIcon` prop | `noIcon={true}` |
-
-## Icon Best Practices
-
-1. **Emoji Size**: Emojis are automatically sized at 24px
-2. **Image Icons**: Should be square (24x24px recommended) for best results
-3. **SVG Icons**: Preferred for crisp rendering at any resolution
-4. **Consistency**: Use similar icon styles across related cards
-5. **Accessibility**: Icons are decorative - ensure card titles are descriptive
-
-## CSS Classes Reference
-
-- `.custom-icon` - Applied when using emoji/text icons
-- `.icon-img` - Applied when using image/SVG icons
-- `.no-icon` - Applied when hiding the default icon
-- `[data-card-icon]` - Attribute used to pass emoji/text content
-
-## Troubleshooting
-
-**Icons not showing?**
-- Clear browser cache (Ctrl+Shift+R / Cmd+Shift+R)
-- Verify image paths are correct (relative to /static folder)
-- Check that CSS custom.css has been updated
-
-**Default arrow still showing?**
-- Ensure you're using `CustomDocCard` component or proper class names
-- Verify the `noIcon` prop is set to `true`
diff --git a/docs/src/components/CustomCardExample.mdx b/docs/src/components/CustomCardExample.mdx
deleted file mode 100644
index fc696bb2e7..0000000000
--- a/docs/src/components/CustomCardExample.mdx
+++ /dev/null
@@ -1,109 +0,0 @@
----
-title: Custom Card Icons Example
-description: Example page showing different card icon options
----
-
-import CustomDocCard from '@site/src/components/CustomDocCard';
-import DocCard from '@theme/DocCard';
-
-# Custom Card Icons Example
-
-This page demonstrates the different ways to use custom icons with DocCard components.
-
-## Default Card (Standard Arrow)
-
-
-
-## Card with Emoji Icon
-
-
-
-## Card with Different Emoji
-
-
-
-
-
-
-
-
-
-
-## Card Without Icon
-
-
-
-## More Examples
-
-
-
-
-
-
-
-
-
-
-
-
diff --git a/docs/src/components/CustomDocCard.tsx b/docs/src/components/CustomDocCard.tsx
deleted file mode 100644
index a03abbf428..0000000000
--- a/docs/src/components/CustomDocCard.tsx
+++ /dev/null
@@ -1,67 +0,0 @@
-import React from 'react';
-import DocCard from '@theme/DocCard';
-import type { PropSidebarItem } from '@docusaurus/plugin-content-docs';
-
-interface CustomDocCardProps {
- item: PropSidebarItem;
- icon?: string; // Emoji or text icon
- imagePath?: string; // Path to SVG/image icon
- noIcon?: boolean; // Set to true to hide the default icon
-}
-
-/**
- * CustomDocCard - A wrapper around Docusaurus DocCard with custom icon support
- *
- * Usage examples:
- *
- * 1. With emoji icon:
- *
- *
- * 2. With image/SVG icon:
- *
- *
- * 3. Without icon:
- *
- *
- * 4. Default (standard arrow):
- *
- */
-export default function CustomDocCard({
- item,
- icon,
- imagePath,
- noIcon
-}: CustomDocCardProps) {
- const getClassName = () => {
- if (noIcon) return 'no-icon';
- if (imagePath) return 'icon-img';
- if (icon) return 'custom-icon';
- return '';
- };
-
- const getStyle = () => {
- if (imagePath) {
- return {
- '--card-icon-image': `url(${imagePath})`,
- } as React.CSSProperties;
- }
- return {};
- };
-
- // For image icons, we need to apply the background via inline style
- const cardProps: any = {
- item,
- };
-
- return (
-
- );
-}
-
-export default function RoadmapPage() {
- const { siteConfig } = useDocusaurusContext();
- const pageTitle = 'Roadmap';
- const pageDescription = 'What we shipped, what we are building next, and what we plan to build.';
-
- return (
-
-
-
-
- );
-}
diff --git a/docs/src/theme/NavbarItem/DefaultNavbarItem/index.tsx b/docs/src/theme/NavbarItem/DefaultNavbarItem/index.tsx
deleted file mode 100644
index 825ceaa899..0000000000
--- a/docs/src/theme/NavbarItem/DefaultNavbarItem/index.tsx
+++ /dev/null
@@ -1,37 +0,0 @@
-import React from "react";
-import DefaultNavbarItemMobile from "./Mobile";
-import DefaultNavbarItemDesktop from "./Desktop";
-
-export type IconConfig = {
- name: string;
- library?: string;
-};
-
-export type ExtendedNavbarItemProps = {
- mobile?: boolean;
- customProps?: {
- icon?: IconConfig;
- [key: string]: unknown;
- };
- label?: React.ReactNode;
- html?: string;
- [key: string]: unknown;
-};
-
-export default function DefaultNavbarItem({
- mobile = false,
- position, // Avoid passing position on to DOM elements
- ...props
-}: ExtendedNavbarItemProps) {
- const Comp = mobile ? DefaultNavbarItemMobile : DefaultNavbarItemDesktop;
-
- return (
-
- );
-}
diff --git a/docs/src/theme/NavbarItem/DefaultNavbarItem/styles.module.css b/docs/src/theme/NavbarItem/DefaultNavbarItem/styles.module.css
deleted file mode 100644
index 7d3b42e8ae..0000000000
--- a/docs/src/theme/NavbarItem/DefaultNavbarItem/styles.module.css
+++ /dev/null
@@ -1,20 +0,0 @@
-.labelWithIcon {
- display: inline-flex;
- align-items: center;
- gap: 0.4rem;
-}
-
-.icon {
- display: inline-flex;
-}
-
-.text {
- display: inline-block;
- line-height: 1.2;
-}
-
-.navItem {
- display: inline-flex;
- align-items: center;
- gap: 0.4rem;
-}
diff --git a/docs/src/theme/ResponseSamples/index.tsx b/docs/src/theme/ResponseSamples/index.tsx
deleted file mode 100644
index 65184088d7..0000000000
--- a/docs/src/theme/ResponseSamples/index.tsx
+++ /dev/null
@@ -1,47 +0,0 @@
-import React from "react";
-import CodeBlock from "@theme/CodeBlock";
-
-type ResponseSamplesProps = {
- responseExample?: string | Record | Array;
- language?: string;
- title?: string;
- className?: string;
-};
-
-function formatExample(
- example: ResponseSamplesProps["responseExample"]
-): string {
- if (example == null) {
- return "";
- }
-
- if (typeof example === "string") {
- return example;
- }
-
- try {
- return JSON.stringify(example, null, 2);
- } catch (error) {
- console.warn("Failed to stringify responseExample:", error);
- return "";
- }
-}
-
-export default function ResponseSamples({
- responseExample,
- language = "json",
- title,
- className,
-}: ResponseSamplesProps) {
- const formattedExample = formatExample(responseExample);
-
- if (!formattedExample) {
- return null;
- }
-
- return (
-
- {formattedExample}
-
- );
-}
diff --git a/docs/static/examples/sentiment140_first50.csv b/docs/static/examples/sentiment140_first50.csv
deleted file mode 100644
index c8321a8598..0000000000
--- a/docs/static/examples/sentiment140_first50.csv
+++ /dev/null
@@ -1,51 +0,0 @@
-correct_answer,tweet
-negative,"@switchfoot http://twitpic.com/2y1zl - Awww, that's a bummer. You shoulda got David Carr of Third Day to do it. ;D"
-negative,is upset that he can't update his Facebook by texting it... and might cry as a result School today also. Blah!
-negative,@Kenichan I dived many times for the ball. Managed to save 50% The rest go out of bounds
-negative,my whole body feels itchy and like its on fire
-negative,"@nationwideclass no, it's not behaving at all. i'm mad. why am i here? because I can't see you all over there. "
-negative,@Kwesidei not the whole crew
-negative,Need a hug
-negative,"@LOLTrish hey long time no see! Yes.. Rains a bit ,only a bit LOL , I'm fine thanks , how's you ?"
-negative,@Tatiana_K nope they didn't have it
-negative,@twittera que me muera ?
-negative,spring break in plain city... it's snowing
-negative,I just re-pierced my ears
-negative,@caregiving I couldn't bear to watch it. And I thought the UA loss was embarrassing . . . . .
-negative,"@octolinz16 It it counts, idk why I did either. you never talk to me anymore "
-negative,"@smarrison i would've been the first, but i didn't have a gun. not really though, zac snyder's just a doucheclown."
-negative,@iamjazzyfizzle I wish I got to watch it with you!! I miss you and @iamlilnicki how was the premiere?!
-negative,Hollis' death scene will hurt me severely to watch on film wry is directors cut not out now?
-negative,about to file taxes
-negative,@LettyA ahh ive always wanted to see rent love the soundtrack!!
-negative,@FakerPattyPattz Oh dear. Were you drinking out of the forgotten table drinks?
-negative,@alydesigns i was out most of the day so didn't get much done
-negative,"one of my friend called me, and asked to meet with her at Mid Valley today...but i've no time *sigh* "
-negative,@angry_barista I baked you a cake but I ated it
-negative,this week is not going as i had hoped
-negative,blagh class at 8 tomorrow
-negative,I hate when I have to call and wake people up
-negative,Just going to cry myself to sleep after watching Marley and Me.
-negative,im sad now Miss.Lilly
-negative,ooooh.... LOL that leslie.... and ok I won't do it again so leslie won't get mad again
-negative,Meh... Almost Lover is the exception... this track gets me depressed every time.
-negative,some1 hacked my account on aim now i have to make a new one
-negative,@alielayus I want to go to promote GEAR AND GROOVE but unfornately no ride there I may b going to the one in Anaheim in May though
-negative,thought sleeping in was an option tomorrow but realizing that it now is not. evaluations in the morning and work in the afternoon!
-negative,@julieebaby awe i love you too!!!! 1 am here i miss you
-negative,@HumpNinja I cry my asian eyes to sleep at night
-negative,ok I'm sick and spent an hour sitting in the shower cause I was too sick to stand and held back the puke like a champ. BED now
-negative,@cocomix04 ill tell ya the story later not a good day and ill be workin for like three more hours...
-negative,@MissXu sorry! bed time came here (GMT+1) http://is.gd/fNge
-negative,@fleurylis I don't either. Its depressing. I don't think I even want to know about the kids in suitcases.
-negative,Bed. Class 8-12. Work 12-3. Gym 3-5 or 6. Then class 6-10. Another day that's gonna fly by. I miss my girlfriend
-negative,really don't feel like getting up today... but got to study to for tomorrows practical exam...
-negative,He's the reason for the teardrops on my guitar the only one who has enough of me to break my heart
-negative,"Sad, sad, sad. I don't know why but I hate this feeling I wanna sleep and I still can't!"
-negative,@JonathanRKnight Awww I soo wish I was there to see you finally comfortable! Im sad that I missed it
-negative,Falling asleep. Just heard about that Tracy girl's body being found. How sad My heart breaks for that family.
-negative,@Viennah Yay! I'm happy for you with your job! But that also means less time for me and you...
-negative,"Just checked my user timeline on my blackberry, it looks like the twanking is still happening Are ppl still having probs w/ BGs and UIDs?"
-negative,Oh man...was ironing @jeancjumbe's fave top to wear to a meeting. Burnt it
-negative,is strangely sad about LiLo and SamRo breaking up.
-negative,@tea oh! i'm so sorry i didn't think about that before retweeting.
diff --git a/docs/static/images/Agenta-logo-full-dark-accent.png b/docs/static/images/Agenta-logo-full-dark-accent.png
deleted file mode 100644
index a270afc094..0000000000
Binary files a/docs/static/images/Agenta-logo-full-dark-accent.png and /dev/null differ
diff --git a/docs/static/images/Agenta-logo-full-light.png b/docs/static/images/Agenta-logo-full-light.png
deleted file mode 100644
index bddc2359bd..0000000000
Binary files a/docs/static/images/Agenta-logo-full-light.png and /dev/null differ
diff --git a/docs/static/images/changelog/agenta_askai.png b/docs/static/images/changelog/agenta_askai.png
deleted file mode 100644
index eee2ba8ff0..0000000000
Binary files a/docs/static/images/changelog/agenta_askai.png and /dev/null differ
diff --git a/docs/static/images/changelog/agenta_dspy.png b/docs/static/images/changelog/agenta_dspy.png
deleted file mode 100644
index ea3d16b89f..0000000000
Binary files a/docs/static/images/changelog/agenta_dspy.png and /dev/null differ
diff --git a/docs/static/images/changelog/changelog-annotation-filter.png b/docs/static/images/changelog/changelog-annotation-filter.png
deleted file mode 100644
index 172c97eac5..0000000000
Binary files a/docs/static/images/changelog/changelog-annotation-filter.png and /dev/null differ
diff --git a/docs/static/images/changelog/changelog-filters-observability.png b/docs/static/images/changelog/changelog-filters-observability.png
deleted file mode 100644
index 6a2da97c18..0000000000
Binary files a/docs/static/images/changelog/changelog-filters-observability.png and /dev/null differ
diff --git a/docs/static/images/dark-complete-transparent-CROPPED.png b/docs/static/images/dark-complete-transparent-CROPPED.png
new file mode 100644
index 0000000000..df1ed7e261
Binary files /dev/null and b/docs/static/images/dark-complete-transparent-CROPPED.png differ
diff --git a/docs/static/images/dark-logo.svg b/docs/static/images/dark-logo.svg
new file mode 100644
index 0000000000..6cb8ef3330
--- /dev/null
+++ b/docs/static/images/dark-logo.svg
@@ -0,0 +1 @@
+
\ No newline at end of file
diff --git a/docs/static/images/evaluation/comparing-evaluations.gif b/docs/static/images/evaluation/comparing-evaluations.gif
new file mode 100644
index 0000000000..87cfd4b4f6
Binary files /dev/null and b/docs/static/images/evaluation/comparing-evaluations.gif differ
diff --git a/docs/static/images/evaluation/comparing-evaluations.png b/docs/static/images/evaluation/comparing-evaluations.png
deleted file mode 100644
index 6031a4cd6b..0000000000
Binary files a/docs/static/images/evaluation/comparing-evaluations.png and /dev/null differ
diff --git a/docs/static/images/evaluation/comparison-view-configuration.png b/docs/static/images/evaluation/comparison-view-configuration.png
deleted file mode 100644
index ae9c692e9e..0000000000
Binary files a/docs/static/images/evaluation/comparison-view-configuration.png and /dev/null differ
diff --git a/docs/static/images/evaluation/comparison-view-drawer.png b/docs/static/images/evaluation/comparison-view-drawer.png
deleted file mode 100644
index 3bd4b21c40..0000000000
Binary files a/docs/static/images/evaluation/comparison-view-drawer.png and /dev/null differ
diff --git a/docs/static/images/evaluation/comparison-view-testset.png b/docs/static/images/evaluation/comparison-view-testset.png
deleted file mode 100644
index e45b8eb4f2..0000000000
Binary files a/docs/static/images/evaluation/comparison-view-testset.png and /dev/null differ
diff --git a/docs/static/images/evaluation/configure-evaluators-1.png b/docs/static/images/evaluation/configure-evaluators-1.png
index b202b1e5ec..fc61f18144 100644
Binary files a/docs/static/images/evaluation/configure-evaluators-1.png and b/docs/static/images/evaluation/configure-evaluators-1.png differ
diff --git a/docs/static/images/evaluation/configure-evaluators-3.png b/docs/static/images/evaluation/configure-evaluators-3.png
index 5c8f8001ac..5e36236191 100644
Binary files a/docs/static/images/evaluation/configure-evaluators-3.png and b/docs/static/images/evaluation/configure-evaluators-3.png differ
diff --git a/docs/static/images/evaluation/detailed-evaluation-drawer.png b/docs/static/images/evaluation/detailed-evaluation-drawer.png
deleted file mode 100644
index e4c131ca30..0000000000
Binary files a/docs/static/images/evaluation/detailed-evaluation-drawer.png and /dev/null differ
diff --git a/docs/static/images/evaluation/detailed-evaluation-results.png b/docs/static/images/evaluation/detailed-evaluation-results.png
index 0c40be89a2..e72e4715bc 100644
Binary files a/docs/static/images/evaluation/detailed-evaluation-results.png and b/docs/static/images/evaluation/detailed-evaluation-results.png differ
diff --git a/docs/static/images/evaluation/evaluate-sdk.png b/docs/static/images/evaluation/evaluate-sdk.png
new file mode 100644
index 0000000000..dc91bc0379
Binary files /dev/null and b/docs/static/images/evaluation/evaluate-sdk.png differ
diff --git a/docs/static/images/evaluation/evaluation-from-ui/01-evaluation-ui-prompt.png b/docs/static/images/evaluation/evaluation-from-ui/01-evaluation-ui-prompt.png
deleted file mode 100644
index 151caefb2e..0000000000
Binary files a/docs/static/images/evaluation/evaluation-from-ui/01-evaluation-ui-prompt.png and /dev/null differ
diff --git a/docs/static/images/evaluation/evaluation-from-ui/02-running-evaluation.png b/docs/static/images/evaluation/evaluation-from-ui/02-running-evaluation.png
deleted file mode 100644
index 28ff95d629..0000000000
Binary files a/docs/static/images/evaluation/evaluation-from-ui/02-running-evaluation.png and /dev/null differ
diff --git a/docs/static/images/evaluation/evaluation-from-ui/03-results-overview.png b/docs/static/images/evaluation/evaluation-from-ui/03-results-overview.png
deleted file mode 100644
index 21828bab9d..0000000000
Binary files a/docs/static/images/evaluation/evaluation-from-ui/03-results-overview.png and /dev/null differ
diff --git a/docs/static/images/evaluation/evaluation-from-ui/04-results-testcase.png b/docs/static/images/evaluation/evaluation-from-ui/04-results-testcase.png
deleted file mode 100644
index 7cacec545e..0000000000
Binary files a/docs/static/images/evaluation/evaluation-from-ui/04-results-testcase.png and /dev/null differ
diff --git a/docs/static/images/evaluation/evaluation-from-ui/05-results-testcase-drawer.png b/docs/static/images/evaluation/evaluation-from-ui/05-results-testcase-drawer.png
deleted file mode 100644
index 21ae52e5b9..0000000000
Binary files a/docs/static/images/evaluation/evaluation-from-ui/05-results-testcase-drawer.png and /dev/null differ
diff --git a/docs/static/images/evaluation/evaluation-from-ui/06-comparison-view.png b/docs/static/images/evaluation/evaluation-from-ui/06-comparison-view.png
deleted file mode 100644
index b3abf9af9b..0000000000
Binary files a/docs/static/images/evaluation/evaluation-from-ui/06-comparison-view.png and /dev/null differ
diff --git a/docs/static/images/evaluation/evaluation-prompt-config.png b/docs/static/images/evaluation/evaluation-prompt-config.png
deleted file mode 100644
index 6d94eaa57d..0000000000
Binary files a/docs/static/images/evaluation/evaluation-prompt-config.png and /dev/null differ
diff --git a/docs/static/images/evaluation/evaluators-inout.png b/docs/static/images/evaluation/evaluators-inout.png
new file mode 100644
index 0000000000..1cd8bc76ff
Binary files /dev/null and b/docs/static/images/evaluation/evaluators-inout.png differ
diff --git a/docs/static/images/evaluation/human-evaluation/configuring-evaluators.png b/docs/static/images/evaluation/human-evaluation/configuring-evaluators.png
deleted file mode 100644
index 117694e912..0000000000
Binary files a/docs/static/images/evaluation/human-evaluation/configuring-evaluators.png and /dev/null differ
diff --git a/docs/static/images/evaluation/human-evaluation/human-evaluation-config-1.png b/docs/static/images/evaluation/human-evaluation/human-evaluation-config-1.png
deleted file mode 100644
index 82dd8f9f90..0000000000
Binary files a/docs/static/images/evaluation/human-evaluation/human-evaluation-config-1.png and /dev/null differ
diff --git a/docs/static/images/evaluation/human-evaluation/human-evaluation-view.png b/docs/static/images/evaluation/human-evaluation/human-evaluation-view.png
deleted file mode 100644
index c075f4a45a..0000000000
Binary files a/docs/static/images/evaluation/human-evaluation/human-evaluation-view.png and /dev/null differ
diff --git a/docs/static/images/evaluation/human-evaluation/starting-human-evaluation.png b/docs/static/images/evaluation/human-evaluation/starting-human-evaluation.png
deleted file mode 100644
index c83f886e41..0000000000
Binary files a/docs/static/images/evaluation/human-evaluation/starting-human-evaluation.png and /dev/null differ
diff --git a/docs/static/images/evaluation/new-evaluation-modal.png b/docs/static/images/evaluation/new-evaluation-modal.png
index 309ce88c90..cb805118e8 100644
Binary files a/docs/static/images/evaluation/new-evaluation-modal.png and b/docs/static/images/evaluation/new-evaluation-modal.png differ
diff --git a/docs/static/images/evaluation/overview-results.png b/docs/static/images/evaluation/overview-results.png
deleted file mode 100644
index 640bb31abc..0000000000
Binary files a/docs/static/images/evaluation/overview-results.png and /dev/null differ
diff --git a/docs/static/images/evaluation/start-new-evaluation.png b/docs/static/images/evaluation/start-new-evaluation.png
index 5d77da863d..a26a72a045 100644
Binary files a/docs/static/images/evaluation/start-new-evaluation.png and b/docs/static/images/evaluation/start-new-evaluation.png differ
diff --git a/docs/static/images/favicon.ico b/docs/static/images/favicon.ico
index dad02fe072..4dc8619b1d 100644
Binary files a/docs/static/images/favicon.ico and b/docs/static/images/favicon.ico differ
diff --git a/docs/static/images/google_collab.png b/docs/static/images/google_collab.png
deleted file mode 100644
index c5b77f1626..0000000000
Binary files a/docs/static/images/google_collab.png and /dev/null differ
diff --git a/docs/static/images/integrations/agenta-openinference-agno-trace.png b/docs/static/images/integrations/agenta-openinference-agno-trace.png
deleted file mode 100644
index 204c694f7a..0000000000
Binary files a/docs/static/images/integrations/agenta-openinference-agno-trace.png and /dev/null differ
diff --git a/docs/static/images/light-complete-transparent-CROPPED.png b/docs/static/images/light-complete-transparent-CROPPED.png
new file mode 100644
index 0000000000..6be2e99e08
Binary files /dev/null and b/docs/static/images/light-complete-transparent-CROPPED.png differ
diff --git a/docs/static/images/light-logo.svg b/docs/static/images/light-logo.svg
new file mode 100644
index 0000000000..9c795f8e88
--- /dev/null
+++ b/docs/static/images/light-logo.svg
@@ -0,0 +1 @@
+
\ No newline at end of file
diff --git a/docs/static/images/observability/observability-mockup.png b/docs/static/images/observability/observability-mockup.png
deleted file mode 100644
index 9df9e7c022..0000000000
Binary files a/docs/static/images/observability/observability-mockup.png and /dev/null differ
diff --git a/docs/static/images/observability/observability_quickstart.png b/docs/static/images/observability/observability_quickstart.png
deleted file mode 100644
index 128a1dee28..0000000000
Binary files a/docs/static/images/observability/observability_quickstart.png and /dev/null differ
diff --git a/docs/static/images/prompt_management/deploy-api.gif b/docs/static/images/prompt_management/deploy-api.gif
index 9438605d5e..e7a367b9af 100644
Binary files a/docs/static/images/prompt_management/deploy-api.gif and b/docs/static/images/prompt_management/deploy-api.gif differ
diff --git a/docs/static/images/social-card.png b/docs/static/images/social-card.png
index 49fe2b893e..d62f2f99b9 100644
Binary files a/docs/static/images/social-card.png and b/docs/static/images/social-card.png differ
diff --git a/ee/LICENSE b/ee/LICENSE
deleted file mode 100644
index ae7a2f38f4..0000000000
--- a/ee/LICENSE
+++ /dev/null
@@ -1,37 +0,0 @@
-Agenta Enterprise License (the “Enterprise License”)
-Copyright (c) 2023–2025
-Agentatech UG (haftungsbeschränkt), doing business as “Agenta” (“Agenta”)
-
-With regard to the Agenta Software:
-
-This software and associated documentation files (the "Software") may only be
-used in production, if you (and any entity that you represent) have agreed to,
-and are in compliance with, the Agenta Subscription Terms of Service, available
-at https://agenta.ai/terms (the “Enterprise Terms”), or other
-agreement governing the use of the Software, as agreed by you and Agenta,
-and otherwise have a valid Agenta Enterprise License.
-
-Subject to the foregoing sentence, you are free to modify this Software and
-publish patches to the Software. You agree that Agenta and/or its licensors
-(as applicable) retain all right, title and interest in and to all such
-modifications and/or patches, and all such modifications and/or patches may
-only be used, copied, modified, displayed, distributed, or otherwise exploited
-with a valid Agenta Enterprise License. Notwithstanding the foregoing, you may
-copy and modify the Software for development and testing purposes, without
-requiring a subscription. You agree that Agenta and/or its licensors (as
-applicable) retain all right, title and interest in and to all such
-modifications. You are not granted any other rights beyond what is expressly
-stated herein. Subject to the foregoing, it is forbidden to copy, merge,
-publish, distribute, sublicense, and/or sell the Software.
-
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-SOFTWARE.
-
-For all third party components incorporated into the Agenta Software, those
-components are licensed under the original license provided by the owner of the
-applicable component.
diff --git a/examples/jupyter/capture_user_feedback.ipynb b/examples/jupyter/capture_user_feedback.ipynb
index a8dcb2e146..9693d437ef 100644
--- a/examples/jupyter/capture_user_feedback.ipynb
+++ b/examples/jupyter/capture_user_feedback.ipynb
@@ -75,9 +75,7 @@
"# Note: Replace these with your actual API keys\n",
"\n",
"os.environ[\"AGENTA_API_KEY\"] = \"your_agenta_api_key_here\"\n",
- "os.environ[\n",
- " \"AGENTA_HOST\"\n",
- "] = \"https://cloud.agenta.ai\" # Use your self-hosted URL if applicable\n",
+ "os.environ[\"AGENTA_HOST\"] = \"https://cloud.agenta.ai\" # Use your self-hosted URL if applicable\n",
"os.environ[\"OPENAI_API_KEY\"] = \"your_openai_api_key_here\""
]
},
@@ -122,13 +120,13 @@
"source": [
"def annotate(trace_id, span_id, annotation, evaluator_slug):\n",
" \"\"\"Create an annotation for a specific trace/span with evaluation data.\n",
- "\n",
+ " \n",
" Args:\n",
" trace_id: The ID of the trace to annotate\n",
" span_id: The ID of the span to annotate\n",
" annotation: Dictionary containing evaluation data (scores, comments, etc.)\n",
" evaluator_slug: Identifier for the evaluator (creates one if it doesn't exist)\n",
- "\n",
+ " \n",
" Returns:\n",
" The annotation response data if successful, None otherwise\n",
" \"\"\"\n",
@@ -142,12 +140,8 @@
" annotation_data = {\n",
" \"annotation\": {\n",
" \"data\": {\"outputs\": annotation}, # Your feedback data goes here\n",
- " \"references\": {\n",
- " \"evaluator\": {\"slug\": evaluator_slug}\n",
- " }, # Evaluator reference\n",
- " \"links\": {\n",
- " \"invocation\": {\"trace_id\": trace_id, \"span_id\": span_id}\n",
- " }, # Link to the trace\n",
+ " \"references\": {\"evaluator\": {\"slug\": evaluator_slug}}, # Evaluator reference\n",
+ " \"links\": {\"invocation\": {\"trace_id\": trace_id, \"span_id\": span_id}}, # Link to the trace\n",
" }\n",
" }\n",
"\n",
@@ -213,10 +207,10 @@
"@ag.instrument() # This decorator creates a root span for tracking the entire function\n",
"def generate(topic: str):\n",
" \"\"\"Generate a story about the given topic and add feedback as an annotation.\n",
- "\n",
+ " \n",
" Args:\n",
" topic: The subject of the story to generate\n",
- "\n",
+ " \n",
" Returns:\n",
" The generated story text\n",
" \"\"\"\n",
@@ -234,35 +228,34 @@
" },\n",
" ],\n",
" )\n",
- "\n",
+ " \n",
" # In a real application, you would obtain trace_id and span_id dynamically\n",
" # using one of these methods:\n",
- "\n",
+ " \n",
" # Method 1: Get the current span context\n",
" # span_ctx = ag.tracing.get_span_context()\n",
" # trace_id = f\"{span_ctx.trace_id:032x}\" # Format as hexadecimal\n",
" # span_id = f\"{span_ctx.span_id:016x}\" # Format as hexadecimal\n",
- "\n",
+ " \n",
" # Method 2: Use the helper function\n",
" link = ag.tracing.build_invocation_link()\n",
" trace_id = link.trace_id\n",
" span_id = link.span_id\n",
- "\n",
+ " \n",
" # Add feedback annotation (simulating user feedback)\n",
" annotate(\n",
" trace_id=trace_id,\n",
" span_id=span_id,\n",
" annotation={\n",
- " \"score\": 5, # Numerical score (1-5)\n",
- " \"comment\": \"This is a comment\", # Text feedback\n",
+ " \"score\": 5, # Numerical score (1-5)\n",
+ " \"comment\": \"This is a comment\" # Text feedback\n",
" },\n",
- " evaluator_slug=\"score-evaluator\", # Creates this evaluator if it doesn't exist\n",
- " )\n",
- "\n",
+ " evaluator_slug=\"score-evaluator\" # Creates this evaluator if it doesn't exist\n",
+ " ) \n",
+ " \n",
" # Return the generated story\n",
" return response.choices[0].message.content\n",
"\n",
- "\n",
"# Test our function by generating a story about AI\n",
"generate(topic=\"AI\")"
]
diff --git a/examples/jupyter/evaluations_with_sdk.ipynb b/examples/jupyter/evaluations_with_sdk.ipynb
index 74e3d9ff9e..1c3e91069d 100644
--- a/examples/jupyter/evaluations_with_sdk.ipynb
+++ b/examples/jupyter/evaluations_with_sdk.ipynb
@@ -9,7 +9,7 @@
"\n",
"We will do the following:\n",
"\n",
- "- Create a testset\n",
+ "- Create a test set\n",
"- Create and configure an evaluator\n",
"- Run an evaluation\n",
"- Retrieve the results of evaluations\n",
@@ -18,7 +18,7 @@
"\n",
"\n",
"### Architectural Overview:\n",
- "In this scenario, evaluations are executed on the Agenta backend. Specifically, Agenta invokes the LLM application for each row in the testset and subsequently processes the output using the designated evaluator. \n",
+ "In this scenario, evaluations are executed on the Agenta backend. Specifically, Agenta invokes the LLM application for each row in the test set and subsequently processes the output using the designated evaluator. \n",
"This operation is managed through Celery tasks. The interactions with the LLM application are asynchronous, batched, and include retry mechanisms. Additionally, the batching configuration can be adjusted to avoid exceeding the rate limits imposed by the LLM provider.\n"
]
},
@@ -84,7 +84,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "## Create a testset"
+ "## Create a test set"
]
},
{
@@ -101,9 +101,9 @@
"]\n",
"\n",
"response = client.testsets.create_testset(\n",
- " request=NewTestset(name=\"testset\", csvdata=csvdata)\n",
+ " request=NewTestset(name=\"test set\", csvdata=csvdata)\n",
")\n",
- "testset_id = response.id\n",
+ "test_set_id = response.id\n",
"\n",
"# let's now update it\n",
"\n",
@@ -113,7 +113,7 @@
"]\n",
"\n",
"client.testsets.update_testset(\n",
- " testset_id=testset_id, request=NewTestset(name=\"testset\", csvdata=csvdata)\n",
+ " testset_id=test_set_id, request=NewTestset(name=\"test set\", csvdata=csvdata)\n",
")"
]
},
@@ -204,7 +204,7 @@
"response = client.evaluations.create_evaluation(\n",
" app_id=app_id,\n",
" variant_ids=[myvariant_id],\n",
- " testset_id=testsetid,\n",
+ " testset_id=test_set_id,\n",
" evaluators_configs=[exact_match_eval_id, letter_match_eval_id],\n",
" rate_limit=LlmRunRateLimit(\n",
" batch_size=10, # number of rows to call in parallel\n",
diff --git a/examples/jupyter/integrations/agno-integration.ipynb b/examples/jupyter/integrations/agno-integration.ipynb
deleted file mode 100644
index f00ab1788d..0000000000
--- a/examples/jupyter/integrations/agno-integration.ipynb
+++ /dev/null
@@ -1,825 +0,0 @@
-{
- "cells": [
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "# Integrate Agenta with Agno\n",
- "\n",
- "This notebook demonstrates how to connect **Agenta** with **Agno** for comprehensive observability and debugging of your AI agent applications.\n",
- "\n",
- "> **What is Agenta?** [Agenta](https://agenta.ai) is an open-source LLMOps platform designed to streamline the deployment, management, and scaling of large language models. It offers comprehensive observability, testing, and deployment capabilities for AI applications.\n",
- "\n",
- "> **What is Agno?** [Agno](https://github.com/agno-agi/agno) is a Python framework for building AI agents with tools, structured outputs, and efficient workflows. It provides a simple yet powerful interface for creating intelligent agents that can interact with external systems and perform complex tasks."
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Implementation Guide\n",
- "\n",
- "Follow this tutorial to set up Agno with Agenta's observability platform for real-time application insights."
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### Step 1: Install Required Dependencies\n",
- "\n",
- "Install the necessary Python packages for this integration:"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "!pip install agenta openinference-instrumentation-agno agno"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "**Package Descriptions:**\n",
- "- `agenta`: Core SDK for Agenta's prompt engineering and observability platform\n",
- "- `agno`: Framework for building AI agents with tools and structured workflows\n",
- "- `openinference-instrumentation-agno`: Automatic instrumentation library for Agno operations"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### Step 2: Setup and Configuration\n",
- "\n",
- "Configure your environment and initialize the Agenta SDK:"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "import os\n",
- "import re\n",
- "from itertools import permutations\n",
- "import agenta as ag\n",
- "from agno.agent import Agent\n",
- "from agno.models.openai import OpenAIChat\n",
- "from openinference.instrumentation.agno import AgnoInstrumentor\n",
- "\n",
- "\n",
- "# Load configuration from environment\n",
- "os.environ[\"AGENTA_API_KEY\"] = \"your_agenta_api_key\"\n",
- "os.environ[\"AGENTA_HOST\"] = (\n",
- " \"https://cloud.agenta.ai\" # Optional, defaults to the Agenta cloud API\n",
- ")\n",
- "\n",
- "# Start Agenta SDK\n",
- "ag.init()"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "**What does `ag.init()` do?**\n",
- "This function initializes the Agenta SDK and sets up the necessary configuration for observability. It establishes connection to the Agenta platform, configures tracing and logging settings, and prepares the instrumentation context for your application."
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### Step 3: Enable Agno Monitoring\n",
- "\n",
- "Initialize the OpenInference Agno instrumentation to automatically capture agent operations:"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "# Enable Agno instrumentation\n",
- "AgnoInstrumentor().instrument()"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### Step 4: Build Your Instrumented Agno Application\n",
- "\n",
- "Here's a complete example showcasing a logistics dispatch agent with Agenta instrumentation:\n",
- "\n",
- "#### Setup Logistics Data"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "# Simulated logistics data\n",
- "tracking_data = {\n",
- " \"TRK10001\": \"In transit at Berlin Friedrichshain Distribution Center\",\n",
- " \"TRK10002\": \"Delivered on 2025-06-14 at 18:32 in Charlottenburg\",\n",
- " \"TRK10003\": \"Out for delivery — last scanned near Tempelhofer Feld\",\n",
- " \"TRK10004\": \"Held at customs near Berlin Brandenburg Airport (BER)\",\n",
- " \"TRK10005\": \"Awaiting pickup at Berlin Hauptbahnhof Parcel Station\",\n",
- "}\n",
- "\n",
- "distance_matrix = {\n",
- " \"Warehouse\": {\"A\": 10, \"B\": 15, \"C\": 20},\n",
- " \"A\": {\"Warehouse\": 10, \"B\": 12, \"C\": 5},\n",
- " \"B\": {\"Warehouse\": 15, \"A\": 12, \"C\": 8},\n",
- " \"C\": {\"Warehouse\": 20, \"A\": 5, \"B\": 8},\n",
- "}\n",
- "\n",
- "driver_load = {\"Alice\": 2, \"Bob\": 3, \"Charlie\": 1}"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "#### Create Tools for the Agent"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "# Tool: TrackingTool\n",
- "class TrackingTool:\n",
- " def __init__(self):\n",
- " self.name = \"TrackingTool\"\n",
- " self.description = \"Provides shipment status updates given a tracking ID.\"\n",
- "\n",
- " def run(self, query: str) -> str:\n",
- " match = re.search(r\"\\bTRK\\d+\\b\", query.upper())\n",
- " if not match:\n",
- " return \"Please provide a valid tracking ID.\"\n",
- " tid = match.group(0)\n",
- " status = tracking_data.get(tid)\n",
- " return f\"Status for {tid}: {status}\" if status else f\"No information for {tid}.\""
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "# Tool: RouteTool\n",
- "class RouteTool:\n",
- " def __init__(self):\n",
- " self.name = \"RouteTool\"\n",
- " self.description = \"Computes the best delivery route given a start and destinations.\"\n",
- "\n",
- " def run(self, query: str) -> str:\n",
- " m = re.search(r\"from\\s+([\\w\\s]+)\\s+to\\s+(.+)\", query, re.IGNORECASE)\n",
- " if not m:\n",
- " return \"Specify route as 'from to , , ...'.\"\n",
- " origin = m.group(1).strip()\n",
- " dests = [d.strip() for d in re.split(r\",| and \", m.group(2)) if d.strip()]\n",
- " \n",
- " if origin not in distance_matrix:\n",
- " return f\"Unknown origin: {origin}.\"\n",
- "\n",
- " for loc in dests:\n",
- " if loc not in distance_matrix:\n",
- " return f\"Unknown destination: {loc}.\"\n",
- " \n",
- " best_distance = float(\"inf\")\n",
- " best_order = None\n",
- " for perm in permutations(dests):\n",
- " total = 0\n",
- " cur = origin\n",
- "\n",
- " for nxt in perm:\n",
- " total += distance_matrix[cur][nxt]\n",
- " cur = nxt\n",
- "\n",
- " if total < best_distance:\n",
- " best_distance = total\n",
- " best_order = perm\n",
- " \n",
- " route_plan = \" → \".join([origin] + list(best_order)) if best_order else origin\n",
- " return f\"Optimal route: {route_plan} (Total distance: {best_distance} km)\""
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "# Tool: WorkloadBalancerTool\n",
- "class WorkloadBalancerTool:\n",
- " def __init__(self):\n",
- " self.name = \"WorkloadBalancerTool\"\n",
- " self.description = \"Assigns delivery locations to the least busy driver.\"\n",
- " self.drivers = driver_load.copy()\n",
- "\n",
- " def run(self, query: str) -> str:\n",
- " m = re.search(r\"deliver(?:y|ies)? to (.+)\", query, re.IGNORECASE)\n",
- " if not m:\n",
- " return \"Please specify delivery locations like 'deliver to A, B, C'.\"\n",
- "\n",
- " locations = [\n",
- " loc.strip() for loc in re.split(r\",| and \", m.group(1)) if loc.strip()\n",
- " ]\n",
- " assignments = []\n",
- " for loc in locations:\n",
- " least_loaded = min(self.drivers, key=lambda d: self.drivers[d])\n",
- " assignments.append(f\"{loc} → {least_loaded}\")\n",
- " self.drivers[least_loaded] += 1\n",
- "\n",
- " return \"Delivery assignments:\\n\" + \"\\n\".join(assignments)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "#### Create and Configure the Agent"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "# Create the dispatch agent\n",
- "agent = Agent(\n",
- " model=OpenAIChat(id=\"gpt-4o\"),\n",
- " description=\"You are a smart dispatch assistant for a logistics team.\",\n",
- " instructions=[\n",
- " \"Use TrackingTool for shipment queries.\",\n",
- " \"Use RouteTool for route optimization.\",\n",
- " \"Use WorkloadBalancerTool to assign deliveries to drivers.\",\n",
- " \"Always return concise, formatted answers with relevant detail.\",\n",
- " ],\n",
- " tools=[TrackingTool(), RouteTool(), WorkloadBalancerTool()],\n",
- " show_tool_calls=False,\n",
- ")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "#### Create Instrumented Handler Function"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "@ag.instrument()\n",
- "def handle_dispatch_request(query: str):\n",
- " result = agent.run(query)\n",
- " return result.content"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "#### Test the Logistics Agent"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "# Test individual functionalities\n",
- "print(\"Testing Shipment Tracking:\")\n",
- "tracking_response = handle_dispatch_request(\"Where is shipment TRK10001?\")\n",
- "print(tracking_response)"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "print(\"\\nTesting Route Optimization:\")\n",
- "route_response = handle_dispatch_request(\"Find the best route from Warehouse to A, B and C\")\n",
- "print(route_response)"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "print(\"\\nTesting Workload Assignment:\")\n",
- "workload_response = handle_dispatch_request(\"Assign deliveries to A, B, and C\")\n",
- "print(workload_response)"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "print(\"\\nTesting Combined Multi-Tool Request:\")\n",
- "combined_response = handle_dispatch_request(\n",
- " \"Where is shipment TRK10001? Also, find the best route from Warehouse to A, B and C, \"\n",
- " \"and assign deliveries to the least busy drivers.\"\n",
- ")\n",
- "print(combined_response)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### Step 5: Understanding the @ag.instrument() Decorator\n",
- "\n",
- "The `@ag.instrument()` decorator automatically captures all input and output data from your function, enabling comprehensive observability without manual instrumentation.\n",
- "\n",
- "**Span Type Configuration:**\n",
- "Use the `spankind` parameter to categorize operations in Agenta WebUI. Available options:\n",
- "\n",
- "- `agent` - Autonomous agent behaviors\n",
- "- `chain` - Sequential processing workflows\n",
- "- `workflow` - Complete application processes (default)\n",
- "- `tool` - Utility and helper functions\n",
- "- `embedding` - Vector embedding operations\n",
- "- `query` - Search and retrieval tasks\n",
- "- `completion` - Text generation operations\n",
- "- `chat` - Conversational interfaces\n",
- "- `rerank` - Result ordering operations\n",
- "\n",
- "**Standard Behavior:**\n",
- "By default, when `spankind` is not specified, the operation becomes a root-level span, categorized as a `workflow` in Agenta."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "# Example with custom span classification:\n",
- "@ag.instrument(spankind=\"agent\")\n",
- "def logistics_agent_handler(query: str):\n",
- " # Agent-specific logic implementation\n",
- " pass"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### Step 6: View Traces in Agenta\n",
- "\n",
- "After running your application, access detailed execution traces through Agenta's dashboard. The observability data includes:\n",
- "\n",
- "- Complete agent workflow execution timeline\n",
- "- Agno agent initialization and tool configuration\n",
- "- Tool execution sequences and decision-making processes\n",
- "- LLM interactions and response generation\n",
- "- Tool call chains and data flow between tools\n",
- "- Performance metrics and timing analysis\n",
- "\n",
- "\n",
- "\n",
- "The observability interface provides insights for:\n",
- "- Debug complex agent interactions and tool usage patterns\n",
- "- Monitor tool execution performance and decision logic\n",
- "- Analyze LLM reasoning and response quality\n",
- "- Track multi-tool workflows and coordination"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Advanced Usage\n",
- "\n",
- "### Custom Span Configuration\n",
- "\n",
- "Customize instrumentation for different application components:"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "@ag.instrument(spankind=\"workflow\")\n",
- "def logistics_pipeline(request: str):\n",
- " return handle_dispatch_request(request)\n",
- "\n",
- "\n",
- "@ag.instrument(spankind=\"tool\")\n",
- "def external_system_integration(data: dict):\n",
- " # External system call logic\n",
- " pass\n",
- "\n",
- "\n",
- "@ag.instrument(spankind=\"agent\")\n",
- "def specialized_logistics_agent(context: dict):\n",
- " # Specialized agent implementation\n",
- " return context"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### Real-world Examples\n",
- "\n",
- "#### Customer Service Agent System"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "# Customer service data\n",
- "customer_db = {\n",
- " \"CUST001\": {\"name\": \"John Doe\", \"tier\": \"Premium\", \"issues\": []},\n",
- " \"CUST002\": {\"name\": \"Jane Smith\", \"tier\": \"Standard\", \"issues\": [\"billing\"]},\n",
- "}\n",
- "\n",
- "ticket_system = {\n",
- " \"TKT001\": {\"customer\": \"CUST001\", \"status\": \"open\", \"priority\": \"high\"},\n",
- " \"TKT002\": {\"customer\": \"CUST002\", \"status\": \"resolved\", \"priority\": \"medium\"},\n",
- "}\n",
- "\n",
- "class CustomerLookupTool:\n",
- " def __init__(self):\n",
- " self.name = \"CustomerLookupTool\"\n",
- " self.description = \"Retrieves customer information by ID.\"\n",
- " \n",
- " def run(self, query: str) -> str:\n",
- " match = re.search(r\"\\bCUST\\d+\\b\", query.upper())\n",
- " if not match:\n",
- " return \"Please provide a valid customer ID.\"\n",
- "\n",
- " cust_id = match.group(0)\n",
- " customer = customer_db.get(cust_id)\n",
- " if customer:\n",
- " return f\"Customer {cust_id}: {customer['name']}, Tier: {customer['tier']}\"\n",
- " return f\"No customer found for {cust_id}.\"\n",
- "\n",
- "class TicketManagementTool:\n",
- " def __init__(self):\n",
- " self.name = \"TicketManagementTool\"\n",
- " self.description = \"Manages support tickets and escalations.\"\n",
- " \n",
- " def run(self, query: str) -> str:\n",
- " if \"create ticket\" in query.lower():\n",
- " return \"New ticket TKT003 created successfully.\"\n",
- "\n",
- " elif \"escalate\" in query.lower():\n",
- " return \"Ticket escalated to senior support team.\"\n",
- "\n",
- " match = re.search(r\"\\bTKT\\d+\\b\", query.upper())\n",
- " if match:\n",
- " ticket_id = match.group(0)\n",
- " ticket = ticket_system.get(ticket_id)\n",
- " if ticket:\n",
- " return f\"Ticket {ticket_id}: Status: {ticket['status']}, Priority: {ticket['priority']}\"\n",
- "\n",
- " return \"Please specify a valid ticket action or ID.\"\n",
- "\n",
- "\n",
- "customer_service_agent = Agent(\n",
- " model=OpenAIChat(id=\"gpt-4o\"),\n",
- " description=\"You are a customer service agent helping with inquiries and ticket management.\",\n",
- " instructions=[\n",
- " \"Use CustomerLookupTool to find customer information.\",\n",
- " \"Use TicketManagementTool to manage support tickets.\",\n",
- " \"Always be helpful and professional in your responses.\",\n",
- " ],\n",
- " tools=[CustomerLookupTool(), TicketManagementTool()],\n",
- " show_tool_calls=False,\n",
- ")\n",
- "\n",
- "\n",
- "@ag.instrument(spankind=\"workflow\")\n",
- "def handle_customer_request(query: str):\n",
- " result = customer_service_agent.run(query)\n",
- " return result.content"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "#### Test Customer Service Agent"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "# Test customer service functionality\n",
- "customer_response = handle_customer_request(\"Look up customer CUST001 and check ticket TKT001\")\n",
- "print(\"Customer Service Response:\", customer_response)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "#### E-commerce Order Management System"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "# E-commerce data\n",
- "inventory = {\n",
- " \"PROD001\": {\"name\": \"Laptop\", \"stock\": 5, \"price\": 999.99},\n",
- " \"PROD002\": {\"name\": \"Mouse\", \"stock\": 25, \"price\": 29.99},\n",
- " \"PROD003\": {\"name\": \"Keyboard\", \"stock\": 0, \"price\": 79.99},\n",
- "}\n",
- "\n",
- "orders = {\n",
- " \"ORD001\": {\"customer\": \"CUST001\", \"items\": [\"PROD001\"], \"status\": \"shipped\"},\n",
- " \"ORD002\": {\"customer\": \"CUST002\", \"items\": [\"PROD002\"], \"status\": \"processing\"},\n",
- "}\n",
- "\n",
- "class InventoryTool:\n",
- " def __init__(self):\n",
- " self.name = \"InventoryTool\"\n",
- " self.description = \"Checks product availability and stock levels.\"\n",
- " \n",
- " def run(self, query: str) -> str:\n",
- " match = re.search(r\"\\bPROD\\d+\\b\", query.upper())\n",
- " if not match:\n",
- " return \"Please provide a valid product ID.\"\n",
- "\n",
- " prod_id = match.group(0)\n",
- " product = inventory.get(prod_id)\n",
- " if product:\n",
- " status = \"In Stock\" if product[\"stock\"] > 0 else \"Out of Stock\"\n",
- " return f\"Product {prod_id}: {product['name']}, Stock: {product['stock']}, Price: ${product['price']}, Status: {status}\"\n",
- "\n",
- " return f\"Product {prod_id} not found.\"\n",
- "\n",
- "\n",
- "class OrderManagementTool:\n",
- " def __init__(self):\n",
- " self.name = \"OrderManagementTool\"\n",
- " self.description = \"Manages order status and updates.\"\n",
- " \n",
- " def run(self, query: str) -> str:\n",
- " match = re.search(r\"\\bORD\\d+\\b\", query.upper())\n",
- " if not match:\n",
- " return \"Please provide a valid order ID.\"\n",
- "\n",
- " order_id = match.group(0)\n",
- " order = orders.get(order_id)\n",
- " if order:\n",
- " return f\"Order {order_id}: Customer: {order['customer']}, Items: {order['items']}, Status: {order['status']}\"\n",
- " return f\"Order {order_id} not found.\"\n",
- "\n",
- "\n",
- "ecommerce_agent = Agent(\n",
- " model=OpenAIChat(id=\"gpt-4o\"),\n",
- " description=\"You are an e-commerce assistant helping with inventory and order management.\",\n",
- " instructions=[\n",
- " \"Use InventoryTool to check product availability.\",\n",
- " \"Use OrderManagementTool to track orders.\",\n",
- " \"Provide clear and helpful information about products and orders.\",\n",
- " ],\n",
- " tools=[InventoryTool(), OrderManagementTool()],\n",
- " show_tool_calls=False,\n",
- ")\n",
- "\n",
- "\n",
- "@ag.instrument(spankind=\"agent\")\n",
- "def handle_ecommerce_request(query: str):\n",
- " result = ecommerce_agent.run(query)\n",
- " return result.content"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "#### Test E-commerce Agent"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "# Test e-commerce functionality\n",
- "ecommerce_response = handle_ecommerce_request(\"Check availability of PROD001 and status of order ORD001\")\n",
- "print(\"E-commerce Response:\", ecommerce_response)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "#### Financial Trading Assistant"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "# Financial data\n",
- "portfolio = {\n",
- " \"AAPL\": {\"shares\": 100, \"avg_price\": 150.00, \"current_price\": 175.50},\n",
- " \"GOOGL\": {\"shares\": 50, \"avg_price\": 2800.00, \"current_price\": 2950.00},\n",
- " \"TSLA\": {\"shares\": 25, \"avg_price\": 800.00, \"current_price\": 725.00},\n",
- "}\n",
- "\n",
- "market_data = {\n",
- " \"AAPL\": {\"price\": 175.50, \"change\": \"+2.5%\", \"volume\": \"45M\"},\n",
- " \"GOOGL\": {\"price\": 2950.00, \"change\": \"+1.8%\", \"volume\": \"12M\"},\n",
- " \"TSLA\": {\"price\": 725.00, \"change\": \"-3.2%\", \"volume\": \"28M\"},\n",
- "}\n",
- "\n",
- "class PortfolioTool:\n",
- " def __init__(self):\n",
- " self.name = \"PortfolioTool\"\n",
- " self.description = \"Provides portfolio information and performance analysis.\"\n",
- " \n",
- " def run(self, query: str) -> str:\n",
- " if \"portfolio\" in query.lower():\n",
- " total_value = sum(stock[\"shares\"] * stock[\"current_price\"] for stock in portfolio.values())\n",
- " return f\"Portfolio Total Value: ${total_value:,.2f}\"\n",
- " \n",
- " match = re.search(r\"\\b[A-Z]{2,5}\\b\", query.upper())\n",
- " if match:\n",
- " symbol = match.group(0)\n",
- " if symbol in portfolio:\n",
- " stock = portfolio[symbol]\n",
- " current_value = stock[\"shares\"] * stock[\"current_price\"]\n",
- " return f\"{symbol}: {stock['shares']} shares @ ${stock['current_price']}, Value: ${current_value:,.2f}\"\n",
- "\n",
- " return \"Please specify a valid stock symbol or ask about your portfolio.\"\n",
- "\n",
- "\n",
- "class MarketDataTool:\n",
- " def __init__(self):\n",
- " self.name = \"MarketDataTool\"\n",
- " self.description = \"Provides real-time market data and stock information.\"\n",
- " \n",
- " def run(self, query: str) -> str:\n",
- " match = re.search(r\"\\b[A-Z]{2,5}\\b\", query.upper())\n",
- " if not match:\n",
- " return \"Please provide a valid stock symbol.\"\n",
- "\n",
- " symbol = match.group(0)\n",
- " if symbol in market_data:\n",
- " data = market_data[symbol]\n",
- " return f\"{symbol}: ${data['price']}, Change: {data['change']}, Volume: {data['volume']}\"\n",
- " return f\"No market data available for {symbol}.\"\n",
- "\n",
- "\n",
- "trading_agent = Agent(\n",
- " model=OpenAIChat(id=\"gpt-4o\"),\n",
- " description=\"You are a financial trading assistant providing portfolio and market analysis.\",\n",
- " instructions=[\n",
- " \"Use PortfolioTool to analyze portfolio holdings.\",\n",
- " \"Use MarketDataTool to get current market information.\",\n",
- " \"Provide clear financial insights and recommendations.\",\n",
- " ],\n",
- " tools=[PortfolioTool(), MarketDataTool()],\n",
- " show_tool_calls=False,\n",
- ")\n",
- "\n",
- "\n",
- "@ag.instrument(spankind=\"workflow\")\n",
- "def handle_trading_request(query: str):\n",
- " result = trading_agent.run(query)\n",
- " return result.content"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "#### Test Trading Agent"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "# Test trading functionality\n",
- "trading_response = handle_trading_request(\"Show me my AAPL holdings and current market data for GOOGL\")\n",
- "print(\"Trading Response:\", trading_response)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### Additional Testing Examples"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "# Test portfolio overview\n",
- "portfolio_overview = handle_trading_request(\"What is my total portfolio value?\")\n",
- "print(\"Portfolio Overview:\", portfolio_overview)"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "# Test customer service ticket creation\n",
- "ticket_creation = handle_customer_request(\"Create a new ticket for billing issues\")\n",
- "print(\"Ticket Creation:\", ticket_creation)"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "# Test inventory check for out of stock item\n",
- "inventory_check = handle_ecommerce_request(\"Check if PROD003 is available for purchase\")\n",
- "print(\"Inventory Check:\", inventory_check)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Next Steps\n",
- "\n",
- "For more detailed information about Agenta's observability features and advanced configuration options, visit the [Agenta Observability SDK Documentation](/observability/observability-sdk)."
- ]
- }
- ],
- "metadata": {
- "kernelspec": {
- "display_name": "Python 3",
- "language": "python",
- "name": "python3"
- },
- "language_info": {
- "codemirror_mode": {
- "name": "ipython",
- "version": 3
- },
- "file_extension": ".py",
- "mimetype": "text/x-python",
- "name": "python",
- "nbconvert_exporter": "python",
- "pygments_lexer": "ipython3",
- "version": "3.8.0"
- }
- },
- "nbformat": 4,
- "nbformat_minor": 4
-}
diff --git a/examples/jupyter/integrations/dspy-integration.ipynb b/examples/jupyter/integrations/dspy-integration.ipynb
index 6909808cf5..ee91151c3e 100644
--- a/examples/jupyter/integrations/dspy-integration.ipynb
+++ b/examples/jupyter/integrations/dspy-integration.ipynb
@@ -73,9 +73,9 @@
"\n",
"# Load configuration from environment\n",
"os.environ[\"AGENTA_API_KEY\"] = \"your_agenta_api_key\"\n",
- "os.environ[\n",
- " \"AGENTA_HOST\"\n",
- "] = \"https://cloud.agenta.ai\" # Optional, defaults to the Agenta cloud API\n",
+ "os.environ[\"AGENTA_HOST\"] = (\n",
+ " \"https://cloud.agenta.ai\" # Optional, defaults to the Agenta cloud API\n",
+ ")\n",
"os.environ[\"OPENAI_API_KEY\"] = \"your_openai_api_key\" # Required for OpenAI models\n",
"\n",
"\n",
@@ -197,7 +197,7 @@
"source": [
"class Outline(dspy.Signature):\n",
" \"\"\"Outline a thorough overview of a topic.\"\"\"\n",
- "\n",
+ " \n",
" topic: str = dspy.InputField()\n",
" title: str = dspy.OutputField()\n",
" sections: list[str] = dspy.OutputField()\n",
@@ -208,7 +208,7 @@
"\n",
"class DraftSection(dspy.Signature):\n",
" \"\"\"Draft a top-level section of an article.\"\"\"\n",
- "\n",
+ " \n",
" topic: str = dspy.InputField()\n",
" section_heading: str = dspy.InputField()\n",
" section_subheadings: list[str] = dspy.InputField()\n",
@@ -219,7 +219,7 @@
" def __init__(self):\n",
" self.build_outline = dspy.ChainOfThought(Outline)\n",
" self.draft_section = dspy.ChainOfThought(DraftSection)\n",
- "\n",
+ " \n",
" def forward(self, topic):\n",
" outline = self.build_outline(topic=topic)\n",
" sections = []\n",
diff --git a/examples/jupyter/integrations/images/agenta-openinference-agno-trace.png b/examples/jupyter/integrations/images/agenta-openinference-agno-trace.png
deleted file mode 100644
index 204c694f7a..0000000000
Binary files a/examples/jupyter/integrations/images/agenta-openinference-agno-trace.png and /dev/null differ
diff --git a/examples/jupyter/integrations/langgraph-integration.ipynb b/examples/jupyter/integrations/langgraph-integration.ipynb
index 30797cc017..4737aaf71e 100644
--- a/examples/jupyter/integrations/langgraph-integration.ipynb
+++ b/examples/jupyter/integrations/langgraph-integration.ipynb
@@ -81,9 +81,9 @@
"\n",
"# Load configuration from environment\n",
"os.environ[\"AGENTA_API_KEY\"] = \"your_agenta_api_key\"\n",
- "os.environ[\n",
- " \"AGENTA_HOST\"\n",
- "] = \"https://cloud.agenta.ai\" # Optional, defaults to the Agenta cloud API\n",
+ "os.environ[\"AGENTA_HOST\"] = (\n",
+ " \"https://cloud.agenta.ai\" # Optional, defaults to the Agenta cloud API\n",
+ ")\n",
"os.environ[\"OPENAI_API_KEY\"] = \"your_openai_api_key\" # Required for OpenAI Agents SDK\n",
"\n",
"\n",
@@ -256,12 +256,12 @@
" builder.add_node(\"segment\", RunnableLambda(segment_by_speaker))\n",
" builder.add_node(\"summarize\", RunnableLambda(summarize_per_speaker))\n",
" builder.add_node(\"extract_actions\", RunnableLambda(extract_actions))\n",
- "\n",
+ " \n",
" builder.set_entry_point(\"segment\")\n",
" builder.add_edge(\"segment\", \"summarize\")\n",
" builder.add_edge(\"summarize\", \"extract_actions\")\n",
" builder.add_edge(\"extract_actions\", END)\n",
- "\n",
+ " \n",
" graph = builder.compile()\n",
" result = graph.invoke({\"input\": transcript})\n",
" return result"
@@ -428,18 +428,14 @@
"def determine_priority(state):\n",
" sentiment = state[\"sentiment\"]\n",
" categories = state[\"categories\"]\n",
- " result = llm.invoke(\n",
- " f\"Determine priority (high/medium/low) based on sentiment: {sentiment} and categories: {categories}\"\n",
- " )\n",
+ " result = llm.invoke(f\"Determine priority (high/medium/low) based on sentiment: {sentiment} and categories: {categories}\")\n",
" return {**state, \"priority\": result.content}\n",
"\n",
"\n",
"def draft_response(state):\n",
" feedback = state[\"input\"]\n",
" sentiment = state[\"sentiment\"]\n",
- " result = llm.invoke(\n",
- " f\"Draft a professional response to this {sentiment} feedback: {feedback}\"\n",
- " )\n",
+ " result = llm.invoke(f\"Draft a professional response to this {sentiment} feedback: {feedback}\")\n",
" return {**state, \"response_draft\": result.content}\n",
"\n",
"\n",
@@ -450,13 +446,13 @@
" builder.add_node(\"categorize\", RunnableLambda(categorize_feedback))\n",
" builder.add_node(\"priority\", RunnableLambda(determine_priority))\n",
" builder.add_node(\"response\", RunnableLambda(draft_response))\n",
- "\n",
+ " \n",
" builder.set_entry_point(\"sentiment\")\n",
" builder.add_edge(\"sentiment\", \"categorize\")\n",
" builder.add_edge(\"categorize\", \"priority\")\n",
" builder.add_edge(\"priority\", \"response\")\n",
" builder.add_edge(\"response\", END)\n",
- "\n",
+ " \n",
" graph = builder.compile()\n",
" return graph.invoke({\"input\": feedback_text})"
]
@@ -510,51 +506,47 @@
"def research_analyzer(paper_text: str):\n",
" def extract_abstract(state):\n",
" paper = state[\"input\"]\n",
- " result = llm.invoke(\n",
- " f\"Extract and summarize the abstract from this research paper: {paper}\"\n",
- " )\n",
+ " result = llm.invoke(f\"Extract and summarize the abstract from this research paper: {paper}\")\n",
" return {**state, \"abstract_summary\": result.content}\n",
- "\n",
+ " \n",
" def identify_findings(state):\n",
" paper = state[\"input\"]\n",
" result = llm.invoke(f\"List the key findings from this research paper: {paper}\")\n",
" return {**state, \"key_findings\": result.content.split(\"\\n\")}\n",
- "\n",
+ " \n",
" def analyze_methodology(state):\n",
" paper = state[\"input\"]\n",
" result = llm.invoke(f\"Describe the methodology used in this research: {paper}\")\n",
" return {**state, \"methodology\": result.content}\n",
- "\n",
+ " \n",
" def assess_limitations(state):\n",
" paper = state[\"input\"]\n",
" result = llm.invoke(f\"Identify limitations mentioned in this research: {paper}\")\n",
" return {**state, \"limitations\": result.content}\n",
- "\n",
+ " \n",
" def score_relevance(state):\n",
" abstract = state[\"abstract_summary\"]\n",
- " result = llm.invoke(\n",
- " f\"Rate the relevance of this research on a scale of 0-10: {abstract}\"\n",
- " )\n",
+ " result = llm.invoke(f\"Rate the relevance of this research on a scale of 0-10: {abstract}\")\n",
" try:\n",
" score = float(result.content.strip())\n",
" except:\n",
" score = 5.0\n",
" return {**state, \"relevance_score\": score}\n",
- "\n",
+ " \n",
" builder = StateGraph(ResearchState)\n",
" builder.add_node(\"abstract\", RunnableLambda(extract_abstract))\n",
" builder.add_node(\"findings\", RunnableLambda(identify_findings))\n",
" builder.add_node(\"methodology\", RunnableLambda(analyze_methodology))\n",
" builder.add_node(\"limitations\", RunnableLambda(assess_limitations))\n",
" builder.add_node(\"relevance\", RunnableLambda(score_relevance))\n",
- "\n",
+ " \n",
" builder.set_entry_point(\"abstract\")\n",
" builder.add_edge(\"abstract\", \"findings\")\n",
" builder.add_edge(\"findings\", \"methodology\")\n",
" builder.add_edge(\"methodology\", \"limitations\")\n",
" builder.add_edge(\"limitations\", \"relevance\")\n",
" builder.add_edge(\"relevance\", END)\n",
- "\n",
+ " \n",
" graph = builder.compile()\n",
" return graph.invoke({\"input\": paper_text})"
]
@@ -590,14 +582,12 @@
" except:\n",
" score = 0.0\n",
" return {**state, \"toxicity_score\": score}\n",
- "\n",
+ " \n",
" def categorize_content(state):\n",
" content = state[\"input\"]\n",
- " result = llm.invoke(\n",
- " f\"Categorize this content (spam, harassment, hate speech, etc.): {content}\"\n",
- " )\n",
+ " result = llm.invoke(f\"Categorize this content (spam, harassment, hate speech, etc.): {content}\")\n",
" return {**state, \"content_categories\": result.content.split(\", \")}\n",
- "\n",
+ " \n",
" def determine_action(state):\n",
" toxicity = state[\"toxicity_score\"]\n",
" categories = state[\"content_categories\"]\n",
@@ -607,21 +597,19 @@
" action = \"flag_for_review\"\n",
" else:\n",
" action = \"approve\"\n",
- " explanation = (\n",
- " f\"Decision based on toxicity score: {toxicity} and categories: {categories}\"\n",
- " )\n",
+ " explanation = f\"Decision based on toxicity score: {toxicity} and categories: {categories}\"\n",
" return {**state, \"action_required\": action, \"explanation\": explanation}\n",
- "\n",
+ " \n",
" builder = StateGraph(ModerationState)\n",
" builder.add_node(\"toxicity\", RunnableLambda(assess_toxicity))\n",
" builder.add_node(\"categorize\", RunnableLambda(categorize_content))\n",
" builder.add_node(\"action\", RunnableLambda(determine_action))\n",
- "\n",
+ " \n",
" builder.set_entry_point(\"toxicity\")\n",
" builder.add_edge(\"toxicity\", \"categorize\")\n",
" builder.add_edge(\"categorize\", \"action\")\n",
" builder.add_edge(\"action\", END)\n",
- "\n",
+ " \n",
" graph = builder.compile()\n",
" return graph.invoke({\"input\": user_content})"
]
diff --git a/examples/jupyter/integrations/observability-openinference-llamaindex.ipynb b/examples/jupyter/integrations/observability-openinference-llamaindex.ipynb
index 9b4dfe2aa1..4d033df3f0 100644
--- a/examples/jupyter/integrations/observability-openinference-llamaindex.ipynb
+++ b/examples/jupyter/integrations/observability-openinference-llamaindex.ipynb
@@ -71,9 +71,9 @@
"\n",
"# Load configuration from environment\n",
"os.environ[\"AGENTA_API_KEY\"] = \"your_agenta_api_key\"\n",
- "os.environ[\n",
- " \"AGENTA_HOST\"\n",
- "] = \"https://cloud.agenta.ai\" # Optional, defaults to the Agenta cloud API\n",
+ "os.environ[\"AGENTA_HOST\"] = (\n",
+ " \"https://cloud.agenta.ai\" # Optional, defaults to the Agenta cloud API\n",
+ ")\n",
"\n",
"\n",
"# Start Agenta SDK\n",
@@ -132,9 +132,9 @@
"\n",
"# Configuration setup\n",
"os.environ[\"AGENTA_API_KEY\"] = \"your_agenta_api_key\"\n",
- "os.environ[\n",
- " \"AGENTA_HOST\"\n",
- "] = \"https://cloud.agenta.ai\" # Optional, defaults to the Agenta cloud API\n",
+ "os.environ[\"AGENTA_HOST\"] = (\n",
+ " \"https://cloud.agenta.ai\" # Optional, defaults to the Agenta cloud API\n",
+ ")\n",
"\n",
"# Initialize observability\n",
"ag.init()\n",
@@ -151,16 +151,16 @@
" \"\"\"\n",
" # Load documents from local directory\n",
" docs = SimpleDirectoryReader(\"data\").load_data()\n",
- "\n",
+ " \n",
" # Build vector search index\n",
" search_index = VectorStoreIndex.from_documents(docs)\n",
- "\n",
+ " \n",
" # Initialize query processor\n",
" query_processor = search_index.as_query_engine()\n",
- "\n",
+ " \n",
" # Process user query\n",
" answer = query_processor.query(user_query)\n",
- "\n",
+ " \n",
" return answer\n",
"\n",
"\n",
diff --git a/examples/jupyter/integrations/openai-agents-integration.ipynb b/examples/jupyter/integrations/openai-agents-integration.ipynb
index 976b0d3160..605e482325 100644
--- a/examples/jupyter/integrations/openai-agents-integration.ipynb
+++ b/examples/jupyter/integrations/openai-agents-integration.ipynb
@@ -74,9 +74,9 @@
"\n",
"# Load configuration from environment\n",
"os.environ[\"AGENTA_API_KEY\"] = \"your_agenta_api_key\"\n",
- "os.environ[\n",
- " \"AGENTA_HOST\"\n",
- "] = \"https://cloud.agenta.ai\" # Optional, defaults to the Agenta cloud API\n",
+ "os.environ[\"AGENTA_HOST\"] = (\n",
+ " \"https://cloud.agenta.ai\" # Optional, defaults to the Agenta cloud API\n",
+ ")\n",
"os.environ[\"OPENAI_API_KEY\"] = \"your_openai_api_key\" # Required for OpenAI Agents SDK\n",
"\n",
"\n",
@@ -203,7 +203,6 @@
" response = await duolingo(\"What is 'What's your name' in French?\")\n",
" print(\"Response:\", response)\n",
"\n",
- "\n",
"# Run the example\n",
"await main()"
]
@@ -331,7 +330,7 @@
")\n",
"\n",
"billing_agent = Agent(\n",
- " name=\"Billing Agent\",\n",
+ " name=\"Billing Agent\", \n",
" instructions=\"You handle billing-related questions and payment issues\",\n",
")\n",
"\n",
@@ -340,25 +339,18 @@
" instructions=\"You provide technical support and troubleshooting assistance\",\n",
")\n",
"\n",
- "\n",
"@ag.instrument(spankind=\"workflow\")\n",
"async def customer_support_system(customer_query: str):\n",
" orchestrator = Agent(\n",
" name=\"Support Orchestrator\",\n",
" instructions=\"Route customer queries to the appropriate specialist agent\",\n",
" tools=[\n",
- " support_agent.as_tool(\n",
- " \"general_support\", \"Handle general customer inquiries\"\n",
- " ),\n",
- " billing_agent.as_tool(\n",
- " \"billing_support\", \"Handle billing and payment issues\"\n",
- " ),\n",
- " technical_agent.as_tool(\n",
- " \"technical_support\", \"Provide technical assistance\"\n",
- " ),\n",
- " ],\n",
+ " support_agent.as_tool(\"general_support\", \"Handle general customer inquiries\"),\n",
+ " billing_agent.as_tool(\"billing_support\", \"Handle billing and payment issues\"),\n",
+ " technical_agent.as_tool(\"technical_support\", \"Provide technical assistance\"),\n",
+ " ]\n",
" )\n",
- "\n",
+ " \n",
" result = await Runner.run(orchestrator, input=customer_query)\n",
" return result.final_output"
]
@@ -382,21 +374,21 @@
" name=\"Data Collector\",\n",
" instructions=\"Gather relevant information and data on the given topic\",\n",
" )\n",
- "\n",
+ " \n",
" analyst = Agent(\n",
- " name=\"Research Analyst\",\n",
+ " name=\"Research Analyst\", \n",
" instructions=\"Analyze collected data and provide insights\",\n",
" )\n",
- "\n",
+ " \n",
" reporter = Agent(\n",
" name=\"Report Writer\",\n",
" instructions=\"Create comprehensive reports based on analysis\",\n",
" tools=[\n",
" data_collector.as_tool(\"collect_data\", \"Gather relevant research data\"),\n",
" analyst.as_tool(\"analyze_data\", \"Perform data analysis and insights\"),\n",
- " ],\n",
+ " ]\n",
" )\n",
- "\n",
+ " \n",
" result = await Runner.run(reporter, input=research_topic)\n",
" return result.final_output"
]
@@ -420,17 +412,17 @@
" name=\"Content Writer\",\n",
" instructions=\"Create engaging content based on the brief\",\n",
" )\n",
- "\n",
+ " \n",
" editor = Agent(\n",
" name=\"Content Editor\",\n",
" instructions=\"Review and improve content quality and style\",\n",
" )\n",
- "\n",
+ " \n",
" seo_specialist = Agent(\n",
" name=\"SEO Specialist\",\n",
" instructions=\"Optimize content for search engines\",\n",
" )\n",
- "\n",
+ " \n",
" content_manager = Agent(\n",
" name=\"Content Manager\",\n",
" instructions=\"Coordinate the content creation process and ensure quality\",\n",
@@ -438,9 +430,9 @@
" writer.as_tool(\"write_content\", \"Generate initial content draft\"),\n",
" editor.as_tool(\"edit_content\", \"Review and edit content\"),\n",
" seo_specialist.as_tool(\"optimize_seo\", \"Optimize content for SEO\"),\n",
- " ],\n",
+ " ]\n",
" )\n",
- "\n",
+ " \n",
" result = await Runner.run(content_manager, input=content_brief)\n",
" return result.final_output"
]
@@ -459,9 +451,7 @@
"outputs": [],
"source": [
"# Test customer support system\n",
- "support_response = await customer_support_system(\n",
- " \"I have a billing question about my subscription\"\n",
- ")\n",
+ "support_response = await customer_support_system(\"I have a billing question about my subscription\")\n",
"print(\"Support Response:\", support_response)"
]
},
@@ -483,9 +473,7 @@
"outputs": [],
"source": [
"# Test content creation system\n",
- "content_response = await content_creation_system(\n",
- " \"Write a blog post about sustainable energy solutions\"\n",
- ")\n",
+ "content_response = await content_creation_system(\"Write a blog post about sustainable energy solutions\")\n",
"print(\"Content Response:\", content_response)"
]
},
diff --git a/examples/jupyter/integrations/pydanticai-integration.ipynb b/examples/jupyter/integrations/pydanticai-integration.ipynb
index 630bcddcee..6da49ce54c 100644
--- a/examples/jupyter/integrations/pydanticai-integration.ipynb
+++ b/examples/jupyter/integrations/pydanticai-integration.ipynb
@@ -76,16 +76,16 @@
"\n",
"# Load configuration from environment\n",
"os.environ[\"AGENTA_API_KEY\"] = \"your_agenta_api_key\"\n",
- "os.environ[\n",
- " \"AGENTA_HOST\"\n",
- "] = \"https://cloud.agenta.ai\" # Optional, defaults to the Agenta cloud API\n",
+ "os.environ[\"AGENTA_HOST\"] = \"https://cloud.agenta.ai\" # Optional, defaults to the Agenta cloud API\n",
"\n",
"# Initialize Agenta SDK\n",
"ag.init()\n",
"\n",
"# Configure Logfire for local development\n",
"logfire.configure(\n",
- " service_name=\"my_logfire_service\", send_to_logfire=False, scrubbing=False\n",
+ " service_name=\"my_logfire_service\", \n",
+ " send_to_logfire=False, \n",
+ " scrubbing=False\n",
")"
]
},
@@ -139,13 +139,13 @@
"# Mock database for demonstration\n",
"class DatabaseConn:\n",
" \"\"\"Fake database for example purposes.\"\"\"\n",
- "\n",
+ " \n",
" @classmethod\n",
" async def customer_name(cls, *, id: int) -> str | None:\n",
" if id == 123:\n",
" return \"John\"\n",
" return None\n",
- "\n",
+ " \n",
" @classmethod\n",
" async def customer_balance(cls, *, id: int, include_pending: bool) -> float:\n",
" if id == 123 and include_pending:\n",
@@ -464,12 +464,12 @@
"def route_customer_query(customer_id: int, query: str, query_type: str = \"general\"):\n",
" \"\"\"Route customer queries to appropriate specialized agents.\"\"\"\n",
" deps = CustomerServiceDeps(customer_id=customer_id, db=DatabaseConn())\n",
- "\n",
+ " \n",
" if query_type == \"technical\":\n",
" result = technical_agent.run_sync(query, deps=deps)\n",
" else:\n",
" result = general_agent.run_sync(query, deps=deps)\n",
- "\n",
+ " \n",
" return result"
]
},
@@ -561,13 +561,11 @@
"\n",
"\n",
"@ag.instrument(spankind=\"agent\")\n",
- "def analyze_transaction(\n",
- " transaction_id: str, customer_id: int, amount: float, location: str\n",
- "):\n",
+ "def analyze_transaction(transaction_id: str, customer_id: int, amount: float, location: str):\n",
" \"\"\"Analyze a transaction for potential fraud.\"\"\"\n",
" deps = FraudDeps(transaction_id=transaction_id, customer_id=customer_id)\n",
" query = f\"Analyze transaction {transaction_id}: ${amount} in {location}\"\n",
- "\n",
+ " \n",
" result = fraud_agent.run_sync(query, deps=deps)\n",
" return result"
]
@@ -666,9 +664,7 @@
"outputs": [],
"source": [
"# Test investment advice\n",
- "investment_result = get_investment_advice(\n",
- " 123, 50000.0, \"I want to diversify my portfolio for retirement\"\n",
- ")\n",
+ "investment_result = get_investment_advice(123, 50000.0, \"I want to diversify my portfolio for retirement\")\n",
"print(\"Investment Advisory Result:\")\n",
"print(f\"Recommendation: {investment_result.output.recommendation}\")\n",
"print(f\"Risk Level: {investment_result.output.risk_level}\")\n",
diff --git a/examples/jupyter/observability/analytics-api-tutorial.ipynb b/examples/jupyter/observability/analytics-api-tutorial.ipynb
deleted file mode 100644
index d5097dfbf6..0000000000
--- a/examples/jupyter/observability/analytics-api-tutorial.ipynb
+++ /dev/null
@@ -1,714 +0,0 @@
-{
- "cells": [
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "# Analytics API - Tutorial\n",
- "\n",
- "This tutorial demonstrates how to use the Agenta Analytics API to analyze LLM performance metrics. You'll learn how to:\n",
- "\n",
- "- Retrieve aggregated metrics over time\n",
- "- Analyze costs, latency, and token usage\n",
- "- Filter analytics by status and other attributes\n",
- "- Track error trends and failure rates\n",
- "- Compare performance across different time periods\n",
- "\n",
- "## What You'll Build\n",
- "\n",
- "We'll create analytics queries that:\n",
- "1. Track daily LLM costs and spending trends\n",
- "2. Monitor error rates and identify peak error times\n",
- "3. Analyze token usage patterns\n",
- "4. Compare performance metrics over time\n",
- "5. Generate cost reports and visualizations"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Setup\n",
- "\n",
- "Before using the API, you need your Agenta API key. You can create API keys from the Settings page in your Agenta workspace."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "import os\n",
- "import requests\n",
- "import json\n",
- "from datetime import datetime, timedelta, timezone\n",
- "from getpass import getpass\n",
- "\n",
- "# Configuration\n",
- "AGENTA_HOST = os.getenv(\"AGENTA_HOST\", \"https://cloud.agenta.ai\")\n",
- "api_key = os.getenv(\"AGENTA_API_KEY\")\n",
- "if not api_key:\n",
- " api_key = getpass(\"Enter your Agenta API key: \")\n",
- " os.environ[\"AGENTA_API_KEY\"] = api_key\n",
- "\n",
- "# Setup base configuration\n",
- "BASE_URL = f\"{AGENTA_HOST}/api/preview/tracing/spans/analytics\"\n",
- "HEADERS = {\n",
- " \"Authorization\": f\"ApiKey {api_key}\",\n",
- " \"Content-Type\": \"application/json\"\n",
- "}\n",
- "\n",
- "print(\"✅ Setup complete!\")\n",
- "print(f\"API endpoint: {BASE_URL}\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Part 1: Get Recent Metrics\n",
- "\n",
- "Let's start by retrieving metrics for the last 7 days with daily buckets. Each bucket contains aggregated metrics for all traces within that day."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "# Get analytics for last 7 days with daily buckets\n",
- "newest = datetime.now(timezone.utc)\n",
- "oldest = newest - timedelta(days=7)\n",
- "\n",
- "payload = {\n",
- " \"focus\": \"trace\",\n",
- " \"interval\": 1440, # 1440 minutes = daily buckets\n",
- " \"windowing\": {\n",
- " \"oldest\": oldest.isoformat(),\n",
- " \"newest\": newest.isoformat()\n",
- " }\n",
- "}\n",
- "\n",
- "response = requests.post(BASE_URL, headers=HEADERS, json=payload)\n",
- "data = response.json()\n",
- "\n",
- "print(f\"📊 Found {data['count']} daily buckets\\n\")\n",
- "\n",
- "# Show all days with activity\n",
- "for bucket in data['buckets']:\n",
- " if bucket['total']['count'] > 0:\n",
- " date = bucket['timestamp'][:10]\n",
- " print(f\"Date: {date}\")\n",
- " print(f\" Traces: {bucket['total']['count']}\")\n",
- " print(f\" Cost: ${bucket['total']['costs']:.4f}\")\n",
- " print(f\" Tokens: {bucket['total']['tokens']:,.0f}\")\n",
- " print(f\" Errors: {bucket['errors']['count']}\\n\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Part 2: Track Daily Costs\n",
- "\n",
- "Calculate total costs and generate summary statistics over a time period."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "# Get daily metrics for last 30 days\n",
- "newest = datetime.now(timezone.utc)\n",
- "oldest = newest - timedelta(days=30)\n",
- "\n",
- "payload = {\n",
- " \"focus\": \"trace\",\n",
- " \"interval\": 1440, # Daily buckets\n",
- " \"windowing\": {\n",
- " \"oldest\": oldest.isoformat(),\n",
- " \"newest\": newest.isoformat()\n",
- " }\n",
- "}\n",
- "\n",
- "response = requests.post(BASE_URL, headers=HEADERS, json=payload)\n",
- "data = response.json()\n",
- "\n",
- "# Calculate totals\n",
- "total_traces = sum(b['total']['count'] for b in data['buckets'])\n",
- "total_cost = sum(b['total']['costs'] for b in data['buckets'])\n",
- "total_tokens = sum(b['total']['tokens'] for b in data['buckets'])\n",
- "total_errors = sum(b['errors']['count'] for b in data['buckets'])\n",
- "\n",
- "print(\"💰 Cost Summary (Last 30 Days)\")\n",
- "print(\"=\" * 50)\n",
- "print(f\"Total Cost: ${total_cost:.2f}\")\n",
- "print(f\"Total Requests: {total_traces:,}\")\n",
- "if total_traces > 0:\n",
- " print(f\"Average Cost per Request: ${total_cost/total_traces:.6f}\")\n",
- " print(f\"Total Tokens: {total_tokens:,.0f}\")\n",
- " print(f\"Average Tokens per Request: {total_tokens/total_traces:.1f}\")\n",
- " print(f\"Error Rate: {(total_errors/total_traces)*100:.2f}%\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Part 3: Analyze Error Trends\n",
- "\n",
- "Monitor error rates over time to identify patterns and peak error times."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "# Get hourly metrics for last 7 days\n",
- "newest = datetime.now(timezone.utc)\n",
- "oldest = newest - timedelta(days=7)\n",
- "\n",
- "payload = {\n",
- " \"focus\": \"trace\",\n",
- " \"interval\": 60, # Hourly buckets\n",
- " \"windowing\": {\n",
- " \"oldest\": oldest.isoformat(),\n",
- " \"newest\": newest.isoformat()\n",
- " }\n",
- "}\n",
- "\n",
- "response = requests.post(BASE_URL, headers=HEADERS, json=payload)\n",
- "data = response.json()\n",
- "\n",
- "print(\"🚨 Error Analysis\")\n",
- "print(\"=\" * 50)\n",
- "\n",
- "# Find hours with high error rates\n",
- "high_error_periods = []\n",
- "for bucket in data['buckets']:\n",
- " if bucket['total']['count'] > 0:\n",
- " error_rate = (bucket['errors']['count'] / bucket['total']['count']) * 100\n",
- " if error_rate > 5: # Flag periods with > 5% errors\n",
- " high_error_periods.append({\n",
- " 'time': bucket['timestamp'],\n",
- " 'error_rate': error_rate,\n",
- " 'total': bucket['total']['count'],\n",
- " 'errors': bucket['errors']['count']\n",
- " })\n",
- "\n",
- "if high_error_periods:\n",
- " print(f\"\\nFound {len(high_error_periods)} periods with high error rates (>5%):\\n\")\n",
- " for period in high_error_periods[:10]: # Show top 10\n",
- " print(f\" {period['time']}\")\n",
- " print(f\" Error Rate: {period['error_rate']:.1f}%\")\n",
- " print(f\" Total: {period['total']}, Errors: {period['errors']}\\n\")\n",
- "else:\n",
- " print(\"✅ No high error rates detected in the last 7 days\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Part 4: Filter by Status Code\n",
- "\n",
- "Analyze only successful traces by filtering on status code."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "# Get successful traces only\n",
- "newest = datetime.now(timezone.utc)\n",
- "oldest = newest - timedelta(days=7)\n",
- "\n",
- "payload = {\n",
- " \"focus\": \"trace\",\n",
- " \"interval\": 1440, # Daily buckets\n",
- " \"windowing\": {\n",
- " \"oldest\": oldest.isoformat(),\n",
- " \"newest\": newest.isoformat()\n",
- " },\n",
- " \"filter\": {\n",
- " \"conditions\": [\n",
- " {\n",
- " \"field\": \"status.code\",\n",
- " \"operator\": \"eq\",\n",
- " \"value\": \"STATUS_CODE_OK\"\n",
- " }\n",
- " ]\n",
- " }\n",
- "}\n",
- "\n",
- "response = requests.post(BASE_URL, headers=HEADERS, json=payload)\n",
- "data = response.json()\n",
- "\n",
- "# Calculate success metrics\n",
- "total_count = sum(b['total']['count'] for b in data['buckets'])\n",
- "total_cost = sum(b['total']['costs'] for b in data['buckets'])\n",
- "total_duration = sum(b['total']['duration'] for b in data['buckets'])\n",
- "\n",
- "print(\"✅ Successful Traces (Last 7 Days)\")\n",
- "print(\"=\" * 50)\n",
- "print(f\"Count: {total_count:,}\")\n",
- "print(f\"Total Cost: ${total_cost:.4f}\")\n",
- "if total_count > 0:\n",
- " print(f\"Avg Duration: {total_duration/total_count:.0f}ms\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Part 5: Track Token Usage\n",
- "\n",
- "Monitor token consumption patterns over time."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "# Get daily token usage for last 7 days\n",
- "newest = datetime.now(timezone.utc)\n",
- "oldest = newest - timedelta(days=7)\n",
- "\n",
- "payload = {\n",
- " \"focus\": \"trace\",\n",
- " \"interval\": 1440, # Daily buckets\n",
- " \"windowing\": {\n",
- " \"oldest\": oldest.isoformat(),\n",
- " \"newest\": newest.isoformat()\n",
- " }\n",
- "}\n",
- "\n",
- "response = requests.post(BASE_URL, headers=HEADERS, json=payload)\n",
- "data = response.json()\n",
- "\n",
- "print(\"🎯 Token Usage Analysis\")\n",
- "print(\"=\" * 50)\n",
- "print(\"\\nDaily Token Usage:\\n\")\n",
- "\n",
- "for bucket in data['buckets']:\n",
- " if bucket['total']['count'] > 0:\n",
- " date = bucket['timestamp'][:10]\n",
- " avg_tokens = bucket['total']['tokens'] / bucket['total']['count']\n",
- " print(f\" {date}: {bucket['total']['tokens']:>8,.0f} total ({avg_tokens:>6.0f} avg)\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Part 6: Analyze Performance\n",
- "\n",
- "Track latency trends over time to identify performance changes."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "# Get hourly performance for last 24 hours\n",
- "newest = datetime.now(timezone.utc)\n",
- "oldest = newest - timedelta(days=1)\n",
- "\n",
- "payload = {\n",
- " \"focus\": \"trace\",\n",
- " \"interval\": 60, # Hourly buckets\n",
- " \"windowing\": {\n",
- " \"oldest\": oldest.isoformat(),\n",
- " \"newest\": newest.isoformat()\n",
- " }\n",
- "}\n",
- "\n",
- "response = requests.post(BASE_URL, headers=HEADERS, json=payload)\n",
- "data = response.json()\n",
- "\n",
- "print(\"⚡ Performance Analysis (Last 24 Hours)\")\n",
- "print(\"=\" * 50)\n",
- "print(\"\\nHourly Average Latency:\\n\")\n",
- "\n",
- "latencies = []\n",
- "for bucket in data['buckets']:\n",
- " if bucket['total']['count'] > 0:\n",
- " avg_duration = bucket['total']['duration'] / bucket['total']['count']\n",
- " latencies.append(avg_duration)\n",
- " hour = bucket['timestamp'][11:16] # Extract HH:MM\n",
- " print(f\" {hour}: {avg_duration:7.0f}ms\")\n",
- "\n",
- "if latencies:\n",
- " print(f\"\\n📈 Statistics:\")\n",
- " print(f\" Min: {min(latencies):.0f}ms\")\n",
- " print(f\" Max: {max(latencies):.0f}ms\")\n",
- " print(f\" Avg: {sum(latencies)/len(latencies):.0f}ms\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Part 7: Generate Monthly Cost Report\n",
- "\n",
- "Create a comprehensive monthly report with cost breakdown and usage statistics."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "# Get monthly metrics\n",
- "newest = datetime.now(timezone.utc)\n",
- "oldest = newest - timedelta(days=30)\n",
- "\n",
- "payload = {\n",
- " \"focus\": \"trace\",\n",
- " \"interval\": 1440, # Daily buckets\n",
- " \"windowing\": {\n",
- " \"oldest\": oldest.isoformat(),\n",
- " \"newest\": newest.isoformat()\n",
- " }\n",
- "}\n",
- "\n",
- "response = requests.post(BASE_URL, headers=HEADERS, json=payload)\n",
- "data = response.json()\n",
- "\n",
- "# Calculate totals\n",
- "total_traces = sum(b['total']['count'] for b in data['buckets'])\n",
- "total_cost = sum(b['total']['costs'] for b in data['buckets'])\n",
- "total_tokens = sum(b['total']['tokens'] for b in data['buckets'])\n",
- "total_duration = sum(b['total']['duration'] for b in data['buckets'])\n",
- "total_errors = sum(b['errors']['count'] for b in data['buckets'])\n",
- "\n",
- "print(\"📊 MONTHLY COST REPORT\")\n",
- "print(\"=\" * 60)\n",
- "print(f\"Period: {oldest.strftime('%Y-%m-%d')} to {newest.strftime('%Y-%m-%d')}\")\n",
- "print(\"=\" * 60)\n",
- "\n",
- "print(\"\\n💰 Cost Summary:\")\n",
- "print(f\" Total Cost: ${total_cost:.2f}\")\n",
- "if total_traces > 0:\n",
- " print(f\" Average Cost per Request: ${total_cost/total_traces:.6f}\")\n",
- "daily_cost = total_cost / 30\n",
- "print(f\" Average Daily Cost: ${daily_cost:.2f}\")\n",
- "print(f\" Projected Monthly Cost: ${daily_cost * 30:.2f}\")\n",
- "\n",
- "print(\"\\n📊 Usage Statistics:\")\n",
- "print(f\" Total Requests: {total_traces:,}\")\n",
- "successful = total_traces - total_errors\n",
- "print(f\" Successful: {successful:,}\")\n",
- "print(f\" Failed: {total_errors:,}\")\n",
- "if total_traces > 0:\n",
- " print(f\" Failure Rate: {(total_errors/total_traces)*100:.2f}%\")\n",
- " print(f\" Average Daily Requests: {total_traces/30:.0f}\")\n",
- "\n",
- "print(\"\\n🎯 Performance Metrics:\")\n",
- "if total_traces > 0:\n",
- " print(f\" Average Latency: {total_duration/total_traces:.0f}ms\")\n",
- "print(f\" Total Tokens: {total_tokens:,.0f}\")\n",
- "if total_traces > 0:\n",
- " print(f\" Average Tokens per Request: {total_tokens/total_traces:.1f}\")\n",
- " print(f\" Average Daily Tokens: {total_tokens/30:,.0f}\")\n",
- "\n",
- "# Cost per 1K tokens\n",
- "if total_tokens > 0:\n",
- " cost_per_1k = (total_cost / total_tokens) * 1000\n",
- " print(f\" Cost per 1K Tokens: ${cost_per_1k:.4f}\")\n",
- "\n",
- "# Find most expensive days\n",
- "print(\"\\n📅 Top 5 Most Expensive Days:\")\n",
- "days_with_data = [(b['timestamp'][:10], b['total']['costs'], b['total']['count']) \n",
- " for b in data['buckets'] if b['total']['count'] > 0]\n",
- "sorted_days = sorted(days_with_data, key=lambda x: x[1], reverse=True)\n",
- "for i, (date, cost, count) in enumerate(sorted_days[:5], 1):\n",
- " print(f\" {i}. {date}: ${cost:.4f} ({count} requests)\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Part 8: Compare Week-over-Week Performance\n",
- "\n",
- "Analyze how metrics change from one week to the next."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "# Helper function to get weekly metrics\n",
- "def get_weekly_metrics(weeks_ago=0):\n",
- " newest = datetime.now(timezone.utc) - timedelta(weeks=weeks_ago)\n",
- " oldest = newest - timedelta(days=7)\n",
- " \n",
- " payload = {\n",
- " \"focus\": \"trace\",\n",
- " \"interval\": 10080, # Weekly bucket\n",
- " \"windowing\": {\n",
- " \"oldest\": oldest.isoformat(),\n",
- " \"newest\": newest.isoformat()\n",
- " }\n",
- " }\n",
- " \n",
- " response = requests.post(BASE_URL, headers=HEADERS, json=payload)\n",
- " data = response.json()\n",
- " \n",
- " if data['buckets']:\n",
- " bucket = data['buckets'][0]\n",
- " return {\n",
- " 'count': bucket['total']['count'],\n",
- " 'costs': bucket['total']['costs'],\n",
- " 'duration': bucket['total']['duration'],\n",
- " 'tokens': bucket['total']['tokens'],\n",
- " 'errors': bucket['errors']['count']\n",
- " }\n",
- " return None\n",
- "\n",
- "this_week = get_weekly_metrics(0)\n",
- "last_week = get_weekly_metrics(1)\n",
- "\n",
- "def calc_change(current, previous):\n",
- " if previous == 0:\n",
- " return \"N/A\"\n",
- " change = ((current - previous) / previous) * 100\n",
- " symbol = \"📈\" if change > 0 else \"📉\" if change < 0 else \"➡️\"\n",
- " return f\"{symbol} {change:+.1f}%\"\n",
- "\n",
- "print(\"📊 Week-over-Week Comparison\")\n",
- "print(\"=\" * 60)\n",
- "\n",
- "if this_week and last_week:\n",
- " print(\"\\n💰 Cost:\")\n",
- " print(f\" Last Week: ${last_week['costs']:.4f}\")\n",
- " print(f\" This Week: ${this_week['costs']:.4f}\")\n",
- " print(f\" Change: {calc_change(this_week['costs'], last_week['costs'])}\")\n",
- "\n",
- " print(\"\\n📊 Volume:\")\n",
- " print(f\" Last Week: {last_week['count']:,} requests\")\n",
- " print(f\" This Week: {this_week['count']:,} requests\")\n",
- " print(f\" Change: {calc_change(this_week['count'], last_week['count'])}\")\n",
- "\n",
- " print(\"\\n⚡ Performance:\")\n",
- " last_avg = last_week['duration'] / last_week['count'] if last_week['count'] > 0 else 0\n",
- " this_avg = this_week['duration'] / this_week['count'] if this_week['count'] > 0 else 0\n",
- " print(f\" Last Week: {last_avg:.0f}ms\")\n",
- " print(f\" This Week: {this_avg:.0f}ms\")\n",
- " print(f\" Change: {calc_change(this_avg, last_avg)}\")\n",
- "\n",
- " print(\"\\n🚨 Error Rate:\")\n",
- " last_err_rate = (last_week['errors'] / last_week['count'] * 100) if last_week['count'] > 0 else 0\n",
- " this_err_rate = (this_week['errors'] / this_week['count'] * 100) if this_week['count'] > 0 else 0\n",
- " print(f\" Last Week: {last_err_rate:.2f}%\")\n",
- " print(f\" This Week: {this_err_rate:.2f}%\")\n",
- " print(f\" Change: {calc_change(this_err_rate, last_err_rate)}\")\n",
- "else:\n",
- " print(\"\\n⚠️ Not enough data for comparison\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Part 9: Create Visualizations\n",
- "\n",
- "Visualize cost and usage trends using matplotlib."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "try:\n",
- " import matplotlib.pyplot as plt\n",
- " import matplotlib.dates as mdates\n",
- " from datetime import datetime\n",
- " \n",
- " # Get daily metrics for last 30 days\n",
- " newest = datetime.now(timezone.utc)\n",
- " oldest = newest - timedelta(days=30)\n",
- "\n",
- " payload = {\n",
- " \"focus\": \"trace\",\n",
- " \"interval\": 1440, # Daily buckets\n",
- " \"windowing\": {\n",
- " \"oldest\": oldest.isoformat(),\n",
- " \"newest\": newest.isoformat()\n",
- " }\n",
- " }\n",
- "\n",
- " response = requests.post(BASE_URL, headers=HEADERS, json=payload)\n",
- " data = response.json()\n",
- "\n",
- " # Extract dates and metrics\n",
- " dates = [datetime.fromisoformat(b['timestamp'].replace('Z', '+00:00')) \n",
- " for b in data['buckets'] if b['total']['count'] > 0]\n",
- " costs = [b['total']['costs'] for b in data['buckets'] if b['total']['count'] > 0]\n",
- " counts = [b['total']['count'] for b in data['buckets'] if b['total']['count'] > 0]\n",
- "\n",
- " # Create figure with two subplots\n",
- " fig, (ax1, ax2) = plt.subplots(2, 1, figsize=(12, 8))\n",
- "\n",
- " # Plot 1: Daily Cost\n",
- " ax1.plot(dates, costs, marker='o', linewidth=2, markersize=4, color='#2563eb')\n",
- " ax1.set_title('Daily LLM Costs (Last 30 Days)', fontsize=14, fontweight='bold')\n",
- " ax1.set_ylabel('Cost ($)', fontsize=12)\n",
- " ax1.grid(True, alpha=0.3)\n",
- " ax1.xaxis.set_major_formatter(mdates.DateFormatter('%m/%d'))\n",
- " plt.setp(ax1.xaxis.get_majorticklabels(), rotation=45)\n",
- "\n",
- " # Plot 2: Daily Request Volume\n",
- " ax2.bar(dates, counts, alpha=0.7, color='steelblue')\n",
- " ax2.set_title('Daily Request Volume (Last 30 Days)', fontsize=14, fontweight='bold')\n",
- " ax2.set_xlabel('Date', fontsize=12)\n",
- " ax2.set_ylabel('Requests', fontsize=12)\n",
- " ax2.grid(True, alpha=0.3, axis='y')\n",
- " ax2.xaxis.set_major_formatter(mdates.DateFormatter('%m/%d'))\n",
- " plt.setp(ax2.xaxis.get_majorticklabels(), rotation=45)\n",
- "\n",
- " plt.tight_layout()\n",
- " plt.show()\n",
- " \n",
- " print(\"✅ Visualizations created successfully!\")\n",
- " \n",
- "except ImportError:\n",
- " print(\"⚠️ matplotlib not installed. Run: pip install matplotlib\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Part 10: Export Data to DataFrame\n",
- "\n",
- "Convert analytics data to a pandas DataFrame for further analysis."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "try:\n",
- " import pandas as pd\n",
- " \n",
- " # Get daily metrics for last 30 days\n",
- " newest = datetime.now(timezone.utc)\n",
- " oldest = newest - timedelta(days=30)\n",
- "\n",
- " payload = {\n",
- " \"focus\": \"trace\",\n",
- " \"interval\": 1440, # Daily buckets\n",
- " \"windowing\": {\n",
- " \"oldest\": oldest.isoformat(),\n",
- " \"newest\": newest.isoformat()\n",
- " }\n",
- " }\n",
- "\n",
- " response = requests.post(BASE_URL, headers=HEADERS, json=payload)\n",
- " data = response.json()\n",
- "\n",
- " # Convert to DataFrame\n",
- " rows = []\n",
- " for bucket in data['buckets']:\n",
- " if bucket['total']['count'] > 0: # Only include days with data\n",
- " rows.append({\n",
- " 'timestamp': bucket['timestamp'],\n",
- " 'total_count': bucket['total']['count'],\n",
- " 'total_cost': bucket['total']['costs'],\n",
- " 'total_duration': bucket['total']['duration'],\n",
- " 'total_tokens': bucket['total']['tokens'],\n",
- " 'error_count': bucket['errors']['count'],\n",
- " 'error_duration': bucket['errors']['duration'],\n",
- " 'avg_duration': bucket['total']['duration'] / bucket['total']['count'],\n",
- " 'avg_cost': bucket['total']['costs'] / bucket['total']['count'],\n",
- " 'error_rate': (bucket['errors']['count'] / bucket['total']['count'] * 100)\n",
- " })\n",
- "\n",
- " df = pd.DataFrame(rows)\n",
- " df['timestamp'] = pd.to_datetime(df['timestamp'])\n",
- "\n",
- " print(\"📊 Analytics Data Summary\\n\")\n",
- " print(df.describe())\n",
- " \n",
- " print(\"\\n📅 Recent Days:\")\n",
- " print(df.tail(10).to_string())\n",
- " \n",
- " # Optional: Save to CSV\n",
- " # df.to_csv('analytics_export.csv', index=False)\n",
- " # print(\"\\n✅ Data exported to analytics_export.csv\")\n",
- " \n",
- "except ImportError:\n",
- " print(\"⚠️ pandas not installed. Run: pip install pandas\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Summary\n",
- "\n",
- "In this tutorial, you learned how to:\n",
- "\n",
- "1. ✅ **Retrieve aggregated metrics** using the Analytics API\n",
- "2. ✅ **Track daily costs** and generate spending reports\n",
- "3. ✅ **Analyze error trends** to identify reliability issues\n",
- "4. ✅ **Filter by status code** to analyze successful vs failed traces\n",
- "5. ✅ **Track token usage** patterns over time\n",
- "6. ✅ **Monitor performance** and latency trends\n",
- "7. ✅ **Generate monthly reports** with comprehensive cost breakdowns\n",
- "8. ✅ **Compare week-over-week** metrics to identify trends\n",
- "9. ✅ **Visualize data** using matplotlib\n",
- "10. ✅ **Export to DataFrame** for further analysis\n",
- "\n",
- "## Next Steps\n",
- "\n",
- "- Learn about [Query API](/observability/query-data/query-api) for detailed trace analysis\n",
- "- Explore [Using the UI](/observability/using-the-ui/filtering-traces) for visual analytics\n",
- "- Check out [Semantic Conventions](/observability/concepts/semantic-conventions) for available metrics\n",
- "- Read about [Cost Tracking](/observability/trace-with-python-sdk/track-costs) for automatic cost calculation"
- ]
- }
- ],
- "metadata": {
- "kernelspec": {
- "display_name": "Python 3",
- "language": "python",
- "name": "python3"
- },
- "language_info": {
- "codemirror_mode": {
- "name": "ipython",
- "version": 3
- },
- "file_extension": ".py",
- "mimetype": "text/x-python",
- "name": "python",
- "nbconvert_exporter": "python",
- "pygments_lexer": "ipython3",
- "version": "3.9.0"
- }
- },
- "nbformat": 4,
- "nbformat_minor": 4
-}
diff --git a/examples/jupyter/observability/annotate-traces-tutorial.ipynb b/examples/jupyter/observability/annotate-traces-tutorial.ipynb
deleted file mode 100644
index 4fccc69fd3..0000000000
--- a/examples/jupyter/observability/annotate-traces-tutorial.ipynb
+++ /dev/null
@@ -1,438 +0,0 @@
-{
- "cells": [
- {
- "cell_type": "markdown",
- "id": "intro",
- "metadata": {},
- "source": [
- "# Annotate Traces Tutorial\n",
- "\n",
- "Annotations in Agenta let you enrich the traces created by your LLM applications. You can add scores, comments, expected answers and other metrics to help evaluate your application's performance.\n",
- "\n",
- "In this tutorial, we'll:\n",
- "1. Set up the Agenta SDK and create a traced LLM application\n",
- "2. Run the application to generate traces\n",
- "3. Add annotations to those traces programmatically\n",
- "4. Query and view the annotations\n",
- "\n",
- "## What You Can Do With Annotations\n",
- "\n",
- "- Collect user feedback on LLM responses\n",
- "- Run custom evaluation workflows\n",
- "- Measure application performance in real-time"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "install",
- "metadata": {},
- "source": [
- "## Step 1: Install Required Packages\n",
- "\n",
- "First, install the Agenta SDK, OpenAI, and the OpenTelemetry instrumentor:"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "install-deps",
- "metadata": {},
- "outputs": [],
- "source": [
- "%pip install -U agenta openai opentelemetry-instrumentation-openai requests"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "setup",
- "metadata": {},
- "source": [
- "## Step 2: Configure Environment Variables\n",
- "\n",
- "To start tracing your application and adding annotations, you'll need an API key:\n",
- "\n",
- "1. Visit the Agenta API Keys page under settings\n",
- "2. Click on **Create New API Key** and follow the prompts\n",
- "\n",
- "Then set your environment variables:"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "env-setup",
- "metadata": {},
- "outputs": [],
- "source": [
- "import os\n",
- "\n",
- "# Set your API keys here\n",
- "os.environ[\"AGENTA_API_KEY\"] = \"\"\n",
- "os.environ[\"AGENTA_HOST\"] = \"https://cloud.agenta.ai\" # Change for self-hosted\n",
- "os.environ[\"OPENAI_API_KEY\"] = \"\""
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "init-sdk",
- "metadata": {},
- "outputs": [],
- "source": [
- "import agenta as ag\n",
- "from getpass import getpass\n",
- "\n",
- "# Initialize the SDK with your API key\n",
- "api_key = os.getenv(\"AGENTA_API_KEY\")\n",
- "if not api_key:\n",
- " os.environ[\"AGENTA_API_KEY\"] = getpass(\"Enter your Agenta API key: \")\n",
- "\n",
- "openai_api_key = os.getenv(\"OPENAI_API_KEY\")\n",
- "if not openai_api_key:\n",
- " os.environ[\"OPENAI_API_KEY\"] = getpass(\"Enter your OpenAI API key: \")\n",
- "\n",
- "# Initialize Agenta\n",
- "ag.init()"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "instrument",
- "metadata": {},
- "source": [
- "## Step 3: Create and Instrument an LLM Application\n",
- "\n",
- "Let's create a simple LLM application that we can trace and annotate:"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "setup-openai",
- "metadata": {},
- "outputs": [],
- "source": [
- "import openai\n",
- "from opentelemetry.instrumentation.openai import OpenAIInstrumentor\n",
- "\n",
- "# Instrument OpenAI to automatically capture traces\n",
- "OpenAIInstrumentor().instrument()"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "create-function",
- "metadata": {},
- "outputs": [],
- "source": [
- "@ag.instrument()\n",
- "def answer_question(question: str) -> tuple[str, str, str]:\n",
- " \"\"\"A simple question-answering function that we'll trace and annotate.\n",
- " \n",
- " Returns:\n",
- " Tuple of (answer, trace_id, span_id)\n",
- " \"\"\"\n",
- " response = openai.chat.completions.create(\n",
- " model=\"gpt-3.5-turbo\",\n",
- " messages=[\n",
- " {\"role\": \"system\", \"content\": \"You are a helpful assistant that answers questions concisely.\"},\n",
- " {\"role\": \"user\", \"content\": question},\n",
- " ],\n",
- " )\n",
- " \n",
- " # Automatically get the trace_id and span_id from the current span\n",
- " link = ag.tracing.build_invocation_link()\n",
- " \n",
- " return response.choices[0].message.content, link.trace_id, link.span_id"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "generate-trace",
- "metadata": {},
- "source": [
- "## Step 4: Generate a Trace\n",
- "\n",
- "Let's run our function to generate a trace. The function will automatically capture the trace_id and span_id using `ag.tracing.build_invocation_link()`."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "run-function",
- "metadata": {},
- "outputs": [],
- "source": [
- "# Run the function to create a trace and get the IDs automatically\n",
- "question = \"What is the capital of France?\"\n",
- "result, trace_id, span_id = answer_question(question)\n",
- "\n",
- "print(f\"Question: {question}\")\n",
- "print(f\"Answer: {result}\")\n",
- "print(f\"\\n✅ Trace captured!\")\n",
- "print(f\"Trace ID: {trace_id}\")\n",
- "print(f\"Span ID: {span_id}\")"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "create-annotation",
- "metadata": {},
- "source": [
- "## Step 5: Create an Annotation\n",
- "\n",
- "Now let's add an annotation to the trace we just created. We'll use the trace_id and span_id that were automatically captured."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "annotate-trace",
- "metadata": {},
- "outputs": [],
- "source": "import requests\n\nbase_url = os.environ.get(\"AGENTA_HOST\", \"https://cloud.agenta.ai\")\napi_key = os.environ[\"AGENTA_API_KEY\"]\n\nheaders = {\n \"Content-Type\": \"application/json\",\n \"Authorization\": f\"ApiKey {api_key}\"\n}\n\n# Create an annotation with a score and reasoning\nannotation_data = {\n \"annotation\": {\n \"data\": {\n \"outputs\": {\n \"score\": 90,\n \"normalized_score\": 0.9,\n \"reasoning\": \"The answer is correct and concise\",\n \"expected_answer\": \"The capital of France is Paris\"\n }\n },\n \"references\": {\n \"evaluator\": {\n \"slug\": \"accuracy_evaluator\"\n }\n },\n \"links\": {\n \"invocation\": {\n \"trace_id\": trace_id,\n \"span_id\": span_id\n }\n },\n \"metadata\": {\n \"annotator\": \"tutorial_user\",\n \"timestamp\": \"2025-10-30T00:00:00Z\"\n }\n }\n}\n\n# Make the API request (note the trailing slash!)\nresponse = requests.post(\n f\"{base_url}/api/preview/annotations/\",\n headers=headers,\n json=annotation_data\n)\n\n# Process the response\nif response.status_code == 200:\n print(\"✅ Annotation created successfully!\")\n annotation_response = response.json()\n print(f\"\\nAnnotation ID: {annotation_response['annotation']['trace_id']}\")\n print(f\"Span ID: {annotation_response['annotation']['span_id']}\")\n print(f\"\\nAnnotation data:\")\n print(annotation_response)\nelse:\n print(f\"❌ Error: {response.status_code}\")\n print(response.text)"
- },
- {
- "cell_type": "markdown",
- "id": "multiple-annotations",
- "metadata": {},
- "source": [
- "## Step 6: Create Additional Annotations\n",
- "\n",
- "You can add multiple annotations to the same trace from different evaluators:"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "create-second-annotation",
- "metadata": {},
- "outputs": [],
- "source": "# Create another annotation for quality assessment\nquality_annotation = {\n \"annotation\": {\n \"data\": {\n \"outputs\": {\n \"score\": 85,\n \"reasoning\": \"Response is helpful and well-formatted\",\n \"labels\": [\"Helpful\", \"Accurate\", \"Concise\"]\n }\n },\n \"references\": {\n \"evaluator\": {\n \"slug\": \"quality_evaluator\"\n }\n },\n \"links\": {\n \"invocation\": {\n \"trace_id\": trace_id,\n \"span_id\": span_id\n }\n }\n }\n}\n\nresponse = requests.post(\n f\"{base_url}/api/preview/annotations/\", # Note the trailing slash!\n headers=headers,\n json=quality_annotation\n)\n\nif response.status_code == 200:\n print(\"✅ Quality annotation created successfully!\")\nelse:\n print(f\"❌ Error: {response.status_code}\")\n print(response.text)"
- },
- {
- "cell_type": "markdown",
- "id": "query-annotations",
- "metadata": {},
- "source": [
- "## Step 7: Query Annotations\n",
- "\n",
- "Now let's query all annotations for our invocation:"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "query-by-invocation",
- "metadata": {},
- "outputs": [],
- "source": [
- "# Query all annotations for the invocation\n",
- "query_data = {\n",
- " \"annotation\": {\n",
- " \"links\": {\n",
- " \"invocation\": {\n",
- " \"trace_id\": trace_id,\n",
- " \"span_id\": span_id\n",
- " }\n",
- " }\n",
- " }\n",
- "}\n",
- "\n",
- "response = requests.post(\n",
- " f\"{base_url}/api/preview/annotations/query\",\n",
- " headers=headers,\n",
- " json=query_data\n",
- ")\n",
- "\n",
- "if response.status_code == 200:\n",
- " print(\"✅ Annotations retrieved successfully!\")\n",
- " annotations = response.json()\n",
- " print(f\"\\nFound {len(annotations.get('annotations', []))} annotation(s)\")\n",
- " print(\"\\nAnnotations:\")\n",
- " for idx, ann in enumerate(annotations.get('annotations', []), 1):\n",
- " print(f\"\\n--- Annotation {idx} ---\")\n",
- " print(f\"Evaluator: {ann['references']['evaluator']['slug']}\")\n",
- " print(f\"Data: {ann['data']}\")\n",
- "else:\n",
- " print(f\"❌ Error: {response.status_code}\")\n",
- " print(response.text)"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "view-ui",
- "metadata": {},
- "source": [
- "## Step 8: View Annotations in the UI\n",
- "\n",
- "You can see all annotations for a trace in the Agenta UI:\n",
- "\n",
- "1. Log in to your Agenta dashboard\n",
- "2. Navigate to the **Observability** section\n",
- "3. Find your trace\n",
- "4. Check the **Annotations** tab to see detailed information\n",
- "\n",
- "The right sidebar will show average metrics for each evaluator."
- ]
- },
- {
- "cell_type": "markdown",
- "id": "understanding-structure",
- "metadata": {},
- "source": [
- "## Understanding Automatic Trace Capture\n",
- "\n",
- "The `ag.tracing.build_invocation_link()` function is a helper that automatically:\n",
- "1. Gets the current span context from the active trace\n",
- "2. Formats the trace_id and span_id as hex strings\n",
- "3. Returns a Link object with both IDs ready to use\n",
- "\n",
- "This is much more convenient than manually querying the UI for trace IDs!\n",
- "\n",
- "**Alternative Method:**\n",
- "You can also use `ag.tracing.get_span_context()` if you need more control:\n",
- "\n",
- "```python\n",
- "span_ctx = ag.tracing.get_span_context()\n",
- "trace_id = f\"{span_ctx.trace_id:032x}\" # Format as hexadecimal\n",
- "span_id = f\"{span_ctx.span_id:016x}\" # Format as hexadecimal\n",
- "```\n",
- "\n",
- "## Understanding Annotation Structure\n",
- "\n",
- "An annotation has four main parts:\n",
- "\n",
- "1. **Data**: The actual evaluation content (scores, comments)\n",
- "2. **References**: Which evaluator to use (will be created automatically if it doesn't exist)\n",
- "3. **Links**: Which trace and span you're annotating\n",
- "4. **Metadata** (optional): Any extra information you want to include\n",
- "\n",
- "### Annotation Data Examples\n",
- "\n",
- "You can include various types of data in your annotations:"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "annotation-examples",
- "metadata": {},
- "outputs": [],
- "source": [
- "# Example 1: Simple score\n",
- "simple_annotation = {\n",
- " \"outputs\": {\n",
- " \"score\": 3\n",
- " }\n",
- "}\n",
- "\n",
- "# Example 2: Score with explanation\n",
- "detailed_annotation = {\n",
- " \"outputs\": {\n",
- " \"score\": 3,\n",
- " \"comment\": \"The response is not grounded\"\n",
- " }\n",
- "}\n",
- "\n",
- "# Example 3: Multiple metrics with reference information\n",
- "comprehensive_annotation = {\n",
- " \"outputs\": {\n",
- " \"score\": 3,\n",
- " \"normalized_score\": 0.5,\n",
- " \"comment\": \"The response is not grounded\",\n",
- " \"expected_answer\": \"The capital of France is Paris\",\n",
- " \"labels\": [\"factual\", \"concise\"]\n",
- " }\n",
- "}\n",
- "\n",
- "print(\"Annotation data can include:\")\n",
- "print(\"- Numbers (scores, ratings)\")\n",
- "print(\"- Categories (labels, classifications)\")\n",
- "print(\"- Text (comments, reasoning)\")\n",
- "print(\"- Booleans (true/false values)\")"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "cleanup",
- "metadata": {},
- "source": [
- "## Optional: Remove an Annotation\n",
- "\n",
- "If you need to remove an annotation, you can delete it by its trace_id and span_id:"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "delete-annotation",
- "metadata": {},
- "outputs": [],
- "source": [
- "# Uncomment and replace with your annotation's trace_id and span_id to delete\n",
- "# annotation_trace_id = \"your_annotation_trace_id\"\n",
- "# annotation_span_id = \"your_annotation_span_id\"\n",
- "\n",
- "# response = requests.delete(\n",
- "# f\"{base_url}/api/preview/annotations/{annotation_trace_id}/{annotation_span_id}\",\n",
- "# headers=headers\n",
- "# )\n",
- "\n",
- "# if response.status_code == 200:\n",
- "# print(\"✅ Annotation deleted successfully\")\n",
- "# else:\n",
- "# print(f\"❌ Error: {response.status_code}\")\n",
- "# print(response.text)"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "summary",
- "metadata": {},
- "source": [
- "## Summary\n",
- "\n",
- "In this tutorial, we've covered:\n",
- "\n",
- "1. ✅ Setting up the Agenta SDK and instrumenting an LLM application\n",
- "2. ✅ Generating traces by running the application\n",
- "3. ✅ Creating annotations with scores, reasoning, and metadata\n",
- "4. ✅ Adding multiple annotations from different evaluators\n",
- "5. ✅ Querying annotations programmatically\n",
- "6. ✅ Understanding annotation structure and capabilities\n",
- "\n",
- "## Next Steps\n",
- "\n",
- "Now that you know how to annotate traces, you can:\n",
- "\n",
- "- Integrate annotation creation into your evaluation workflows\n",
- "- Build custom evaluators that automatically annotate traces\n",
- "- Use annotations to track user feedback in production\n",
- "- Analyze annotation data to improve your LLM applications"
- ]
- }
- ],
- "metadata": {
- "kernelspec": {
- "display_name": ".venv",
- "language": "python",
- "name": "python3"
- },
- "language_info": {
- "codemirror_mode": {
- "name": "ipython",
- "version": 3
- },
- "file_extension": ".py",
- "mimetype": "text/x-python",
- "name": "python",
- "nbconvert_exporter": "python",
- "pygments_lexer": "ipython3",
- "version": "3.12.3"
- }
- },
- "nbformat": 4,
- "nbformat_minor": 5
-}
\ No newline at end of file
diff --git a/examples/jupyter/observability/query-data-api-tutorial.ipynb b/examples/jupyter/observability/query-data-api-tutorial.ipynb
deleted file mode 100644
index 09ad0a7ecc..0000000000
--- a/examples/jupyter/observability/query-data-api-tutorial.ipynb
+++ /dev/null
@@ -1,643 +0,0 @@
-{
- "cells": [
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "# Query Data API - Tutorial\n",
- "\n",
- "This tutorial shows you how to use the Agenta Query Data API to retrieve and analyze your LLM traces. You'll learn how to:\n",
- "\n",
- "- Set up the API client with authentication\n",
- "- Query spans and traces with filters\n",
- "- Filter by attributes, time ranges, and status codes\n",
- "- Use advanced filters with logical operators\n",
- "- Analyze trace data to calculate costs and latencies\n",
- "\n",
- "## What You'll Build\n",
- "\n",
- "We'll create scripts that:\n",
- "1. Query recent traces from your applications\n",
- "2. Filter traces by type, status, and custom attributes\n",
- "3. Analyze cost and performance metrics\n",
- "4. Find problematic traces (errors, slow responses)\n",
- "5. Export trace data for further analysis"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Install Dependencies"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "pip install -U requests pandas"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": "## Setup\n\nBefore using the API, you need your Agenta API key. You can create API keys from the Settings page in your Agenta workspace."
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "import os\n",
- "os.environ[\"AGENTA_HOST\"] = \"https://cloud.agenta.ai\" # Default value, change for self-hosted\n",
- "os.environ[\"AGENTA_API_KEY\"] = \"\""
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "import os\n",
- "import requests\n",
- "from getpass import getpass\n",
- "from datetime import datetime, timedelta, timezone\n",
- "import json\n",
- "\n",
- "# Get API credentials\n",
- "AGENTA_HOST = os.getenv(\"AGENTA_HOST\", \"https://cloud.agenta.ai\")\n",
- "api_key = os.getenv(\"AGENTA_API_KEY\")\n",
- "if not api_key:\n",
- " api_key = getpass(\"Enter your Agenta API key: \")\n",
- " os.environ[\"AGENTA_API_KEY\"] = api_key\n",
- "\n",
- "# Setup base configuration\n",
- "BASE_URL = f\"{AGENTA_HOST}/api/preview/tracing/spans/query\"\n",
- "HEADERS = {\n",
- " \"Authorization\": f\"ApiKey {api_key}\",\n",
- " \"Content-Type\": \"application/json\"\n",
- "}\n",
- "\n",
- "print(f\"\u2713 Connected to {AGENTA_HOST}\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Part 1: Query Recent Traces\n",
- "\n",
- "Let's start by querying traces from the last 7 days. We'll use the `focus=trace` parameter to get complete trace trees instead of individual spans."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "# Query traces from the last 7 days\n",
- "now = datetime.now(timezone.utc)\n",
- "week_ago = now - timedelta(days=7)\n",
- "\n",
- "query = {\n",
- " \"focus\": \"trace\",\n",
- " \"oldest\": week_ago.isoformat(),\n",
- " \"newest\": now.isoformat(),\n",
- " \"limit\": 5\n",
- "}\n",
- "\n",
- "response = requests.post(BASE_URL, headers=HEADERS, json=query)\n",
- "data = response.json()\n",
- "\n",
- "print(f\"Found {data['count']} traces\")\n",
- "print(f\"Trace IDs: {list(data.get('traces', {}).keys())}\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Part 2: Query Spans with Filters\n",
- "\n",
- "Now let's query individual spans and filter by type. We'll look for LLM spans specifically."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "# Query LLM spans\n",
- "query = {\n",
- " \"focus\": \"span\",\n",
- " \"limit\": 10,\n",
- " \"filter\": {\n",
- " \"operator\": \"and\",\n",
- " \"conditions\": [\n",
- " {\n",
- " \"field\": \"attributes\",\n",
- " \"key\": \"ag.type.span\",\n",
- " \"operator\": \"is\",\n",
- " \"value\": \"llm\"\n",
- " }\n",
- " ]\n",
- " }\n",
- "}\n",
- "\n",
- "response = requests.post(BASE_URL, headers=HEADERS, json=query)\n",
- "data = response.json()\n",
- "\n",
- "print(f\"Found {data['count']} LLM spans\")\n",
- "\n",
- "# Display first span details\n",
- "if data.get('spans'):\n",
- " span = data['spans'][0]\n",
- " print(f\"\\nFirst span:\")\n",
- " print(f\" Name: {span.get('span_name')}\")\n",
- " print(f\" Status: {span.get('status_code')}\")\n",
- " print(f\" Start: {span.get('start_time')}\")\n",
- " print(f\" End: {span.get('end_time')}\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Part 3: Filter by Status Code\n",
- "\n",
- "Let's find traces that encountered errors. This is useful for debugging and monitoring application health."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "# Find error traces\n",
- "query = {\n",
- " \"focus\": \"trace\",\n",
- " \"limit\": 10,\n",
- " \"filter\": {\n",
- " \"operator\": \"and\",\n",
- " \"conditions\": [\n",
- " {\n",
- " \"field\": \"status_code\",\n",
- " \"operator\": \"is\",\n",
- " \"value\": \"STATUS_CODE_ERROR\"\n",
- " }\n",
- " ]\n",
- " }\n",
- "}\n",
- "\n",
- "response = requests.post(BASE_URL, headers=HEADERS, json=query)\n",
- "data = response.json()\n",
- "\n",
- "print(f\"Found {data['count']} traces with errors\")\n",
- "\n",
- "if data.get('traces'):\n",
- " for trace_id in list(data['traces'].keys())[:3]:\n",
- " print(f\"\\nError trace: {trace_id}\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Part 4: Advanced Filtering with Multiple Conditions\n",
- "\n",
- "Let's use multiple filters to find specific traces. We'll look for successful LLM calls that took longer than 2 seconds."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "# Find slow but successful LLM calls\n",
- "query = {\n",
- " \"focus\": \"span\",\n",
- " \"limit\": 10,\n",
- " \"filter\": {\n",
- " \"operator\": \"and\",\n",
- " \"conditions\": [\n",
- " {\n",
- " \"field\": \"attributes\",\n",
- " \"key\": \"ag.type.span\",\n",
- " \"operator\": \"is\",\n",
- " \"value\": \"llm\"\n",
- " },\n",
- " {\n",
- " \"field\": \"status_code\",\n",
- " \"operator\": \"is_not\",\n",
- " \"value\": \"STATUS_CODE_ERROR\"\n",
- " },\n",
- " {\n",
- " \"field\": \"attributes\",\n",
- " \"key\": \"ag.metrics.unit.duration\",\n",
- " \"operator\": \"gt\",\n",
- " \"value\": 2000 # milliseconds\n",
- " }\n",
- " ]\n",
- " }\n",
- "}\n",
- "\n",
- "response = requests.post(BASE_URL, headers=HEADERS, json=query)\n",
- "data = response.json()\n",
- "\n",
- "print(f\"Found {data['count']} slow LLM spans (>2s)\")\n",
- "\n",
- "if data.get('spans'):\n",
- " for span in data['spans'][:3]:\n",
- " duration = span.get('attributes', {}).get('ag', {}).get('metrics', {}).get('duration', {}).get('cumulative', 'N/A')\n",
- " print(f\" {span.get('span_name')}: {duration}ms\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Part 5: Nested Logical Operators\n",
- "\n",
- "Let's create a more complex query using nested logical operators. We'll find spans that are either errors OR slow responses."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "# Find problematic spans (errors OR slow)\n",
- "query = {\n",
- " \"focus\": \"span\",\n",
- " \"limit\": 10,\n",
- " \"filter\": {\n",
- " \"operator\": \"and\",\n",
- " \"conditions\": [\n",
- " {\n",
- " \"field\": \"attributes\",\n",
- " \"key\": \"ag.type.span\",\n",
- " \"operator\": \"is\",\n",
- " \"value\": \"llm\"\n",
- " },\n",
- " {\n",
- " \"operator\": \"or\",\n",
- " \"conditions\": [\n",
- " {\n",
- " \"field\": \"status_code\",\n",
- " \"value\": \"STATUS_CODE_ERROR\"\n",
- " },\n",
- " {\n",
- " \"field\": \"attributes\",\n",
- " \"key\": \"ag.metrics.unit.duration\",\n",
- " \"operator\": \"gt\",\n",
- " \"value\": 5000\n",
- " }\n",
- " ]\n",
- " }\n",
- " ]\n",
- " }\n",
- "}\n",
- "\n",
- "response = requests.post(BASE_URL, headers=HEADERS, json=query)\n",
- "data = response.json()\n",
- "\n",
- "print(f\"Found {data['count']} problematic spans\")\n",
- "\n",
- "if data.get('spans'):\n",
- " for span in data['spans'][:5]:\n",
- " status = span.get('status_code')\n",
- " duration = span.get('attributes', {}).get('ag', {}).get('metrics', {}).get('duration', {}).get('cumulative', 'N/A')\n",
- " print(f\" {span.get('span_name')}: Status={status}, Duration={duration}ms\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Part 6: Analyze Cost and Token Usage\n",
- "\n",
- "Let's query LLM spans and analyze their costs and token usage. This helps you understand your LLM spending."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "# Query LLM spans with cost tracking\n",
- "query = {\n",
- " \"focus\": \"span\",\n",
- " \"limit\": 50,\n",
- " \"filter\": {\n",
- " \"operator\": \"and\",\n",
- " \"conditions\": [\n",
- " {\n",
- " \"field\": \"attributes\",\n",
- " \"key\": \"ag.type.span\",\n",
- " \"operator\": \"is\",\n",
- " \"value\": \"llm\"\n",
- " },\n",
- " {\n",
- " \"field\": \"attributes\",\n",
- " \"key\": \"ag.metrics.unit.cost\",\n",
- " \"operator\": \"exists\"\n",
- " }\n",
- " ]\n",
- " }\n",
- "}\n",
- "\n",
- "response = requests.post(BASE_URL, headers=HEADERS, json=query)\n",
- "data = response.json()\n",
- "\n",
- "print(f\"Analyzing {data['count']} LLM spans with cost data\\n\")\n",
- "\n",
- "if data.get('spans'):\n",
- " total_cost = 0\n",
- " total_tokens = 0\n",
- " total_duration = 0\n",
- " \n",
- " for span in data['spans']:\n",
- " metrics = span.get('attributes', {}).get('ag', {}).get('metrics', {})\n",
- " \n",
- " # Extract cost\n",
- " cost = metrics.get('costs', {}).get('cumulative', {}).get('total', 0)\n",
- " total_cost += cost\n",
- " \n",
- " # Extract tokens\n",
- " tokens = metrics.get('tokens', {}).get('cumulative', {}).get('total', 0)\n",
- " total_tokens += tokens\n",
- " \n",
- " # Extract duration\n",
- " duration = metrics.get('duration', {}).get('cumulative', 0)\n",
- " total_duration += duration\n",
- " \n",
- " print(f\"Summary:\")\n",
- " print(f\" Total Cost: ${total_cost:.4f}\")\n",
- " print(f\" Total Tokens: {total_tokens:,}\")\n",
- " print(f\" Average Cost per Span: ${(total_cost/len(data['spans'])):.4f}\")\n",
- " print(f\" Average Tokens per Span: {int(total_tokens/len(data['spans']))}\")\n",
- " print(f\" Average Duration: {int(total_duration/len(data['spans']))}ms\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Part 7: Filter by Span Name Pattern\n",
- "\n",
- "Let's use string matching operators to find specific types of operations. We'll search for OpenAI-related spans."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "# Find OpenAI spans using pattern matching\n",
- "query = {\n",
- " \"focus\": \"span\",\n",
- " \"limit\": 10,\n",
- " \"filter\": {\n",
- " \"operator\": \"and\",\n",
- " \"conditions\": [\n",
- " {\n",
- " \"field\": \"span_name\",\n",
- " \"operator\": \"contains\",\n",
- " \"value\": \"openai\"\n",
- " }\n",
- " ]\n",
- " }\n",
- "}\n",
- "\n",
- "response = requests.post(BASE_URL, headers=HEADERS, json=query)\n",
- "data = response.json()\n",
- "\n",
- "print(f\"Found {data['count']} OpenAI spans\")\n",
- "\n",
- "if data.get('spans'):\n",
- " span_names = set(span.get('span_name') for span in data['spans'])\n",
- " print(f\"\\nUnique span names:\")\n",
- " for name in span_names:\n",
- " print(f\" - {name}\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Part 8: Export Trace Data to DataFrame\n",
- "\n",
- "Let's export our trace data to a pandas DataFrame for further analysis and visualization."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "import pandas as pd\n",
- "\n",
- "# Query recent spans\n",
- "query = {\n",
- " \"focus\": \"span\",\n",
- " \"limit\": 100,\n",
- " \"filter\": {\n",
- " \"operator\": \"and\",\n",
- " \"conditions\": [\n",
- " {\n",
- " \"field\": \"attributes\",\n",
- " \"key\": \"ag.type.span\",\n",
- " \"operator\": \"is\",\n",
- " \"value\": \"llm\"\n",
- " }\n",
- " ]\n",
- " }\n",
- "}\n",
- "\n",
- "response = requests.post(BASE_URL, headers=HEADERS, json=query)\n",
- "data = response.json()\n",
- "\n",
- "# Convert to DataFrame\n",
- "records = []\n",
- "for span in data.get('spans', []):\n",
- " metrics = span.get('attributes', {}).get('ag', {}).get('metrics', {})\n",
- " \n",
- " record = {\n",
- " 'trace_id': span.get('trace_id'),\n",
- " 'span_id': span.get('span_id'),\n",
- " 'span_name': span.get('span_name'),\n",
- " 'status': span.get('status_code'),\n",
- " 'start_time': span.get('start_time'),\n",
- " 'end_time': span.get('end_time'),\n",
- " 'duration_ms': metrics.get('duration', {}).get('cumulative', 0),\n",
- " 'cost': metrics.get('costs', {}).get('cumulative', {}).get('total', 0),\n",
- " 'total_tokens': metrics.get('tokens', {}).get('cumulative', {}).get('total', 0),\n",
- " 'prompt_tokens': metrics.get('tokens', {}).get('cumulative', {}).get('prompt', 0),\n",
- " 'completion_tokens': metrics.get('tokens', {}).get('cumulative', {}).get('completion', 0),\n",
- " }\n",
- " records.append(record)\n",
- "\n",
- "df = pd.DataFrame(records)\n",
- "\n",
- "print(f\"Created DataFrame with {len(df)} rows\\n\")\n",
- "print(\"First 5 rows:\")\n",
- "print(df.head())\n",
- "\n",
- "print(\"\\nBasic statistics:\")\n",
- "print(df[['duration_ms', 'cost', 'total_tokens']].describe())"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Part 9: Time-Based Analysis\n",
- "\n",
- "Let's analyze how your costs and latencies change over time."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "# Convert timestamps to datetime\n",
- "df['start_time'] = pd.to_datetime(df['start_time'])\n",
- "df['end_time'] = pd.to_datetime(df['end_time'])\n",
- "\n",
- "# Group by hour\n",
- "df['hour'] = df['start_time'].dt.floor('H')\n",
- "hourly_stats = df.groupby('hour').agg({\n",
- " 'span_id': 'count',\n",
- " 'duration_ms': 'mean',\n",
- " 'cost': 'sum',\n",
- " 'total_tokens': 'sum'\n",
- "}).rename(columns={'span_id': 'num_calls'})\n",
- "\n",
- "print(\"Hourly statistics:\")\n",
- "print(hourly_stats)\n",
- "\n",
- "print(f\"\\nPeak usage hour: {hourly_stats['num_calls'].idxmax()}\")\n",
- "print(f\"Highest cost hour: {hourly_stats['cost'].idxmax()} (${hourly_stats['cost'].max():.4f})\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Part 10: Filter by Time Range\n",
- "\n",
- "Let's query traces from a specific time window. This is useful for analyzing specific incidents or time periods."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "# Query last 24 hours\n",
- "now = datetime.now(timezone.utc)\n",
- "yesterday = now - timedelta(days=1)\n",
- "\n",
- "query = {\n",
- " \"focus\": \"span\",\n",
- " \"oldest\": yesterday.isoformat(),\n",
- " \"newest\": now.isoformat(),\n",
- " \"limit\": 100,\n",
- " \"filter\": {\n",
- " \"operator\": \"and\",\n",
- " \"conditions\": [\n",
- " {\n",
- " \"field\": \"attributes\",\n",
- " \"key\": \"ag.type.span\",\n",
- " \"operator\": \"is\",\n",
- " \"value\": \"llm\"\n",
- " }\n",
- " ]\n",
- " }\n",
- "}\n",
- "\n",
- "response = requests.post(BASE_URL, headers=HEADERS, json=query)\n",
- "data = response.json()\n",
- "\n",
- "print(f\"Last 24 hours: {data['count']} LLM spans\")\n",
- "\n",
- "if data.get('spans'):\n",
- " # Calculate totals\n",
- " total_cost = sum(\n",
- " span.get('attributes', {}).get('ag', {}).get('metrics', {}).get('costs', {}).get('cumulative', {}).get('total', 0)\n",
- " for span in data['spans']\n",
- " )\n",
- " \n",
- " error_count = sum(\n",
- " 1 for span in data['spans']\n",
- " if span.get('status_code') == 'STATUS_CODE_ERROR'\n",
- " )\n",
- " \n",
- " print(f\"\\nSummary for last 24 hours:\")\n",
- " print(f\" Total Cost: ${total_cost:.4f}\")\n",
- " print(f\" Error Rate: {(error_count/len(data['spans'])*100):.2f}%\")\n",
- " print(f\" Success Rate: {((len(data['spans'])-error_count)/len(data['spans'])*100):.2f}%\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Summary\n",
- "\n",
- "In this tutorial, you learned how to:\n",
- "\n",
- "1. \u2713 Set up the Agenta Query Data API client\n",
- "2. \u2713 Query traces and spans with filters\n",
- "3. \u2713 Filter by attributes, status codes, and time ranges\n",
- "4. \u2713 Use advanced filters with logical operators\n",
- "5. \u2713 Analyze cost and performance metrics\n",
- "6. \u2713 Export trace data to pandas DataFrames\n",
- "7. \u2713 Perform time-based analysis\n",
- "\n",
- "## Next Steps\n",
- "\n",
- "- Learn about the [Analytics Data API](/observability/query-data/analytics-data) for aggregated metrics\n",
- "- Explore [filtering in the UI](/observability/using-the-ui/filtering-traces) for visual query building\n",
- "- Check out [trace annotations](/observability/trace-with-python-sdk/annotate-traces) for adding feedback data\n",
- "- Read the complete [API reference](/reference/api) for all available endpoints"
- ]
- }
- ],
- "metadata": {
- "kernelspec": {
- "display_name": "Python 3",
- "language": "python",
- "name": "python3"
- },
- "language_info": {
- "codemirror_mode": {
- "name": "ipython",
- "version": 3
- },
- "file_extension": ".py",
- "mimetype": "text/x-python",
- "name": "python",
- "nbconvert_exporter": "python",
- "pygments_lexer": "ipython3",
- "version": "3.9.0"
- }
- },
- "nbformat": 4,
- "nbformat_minor": 4
-}
\ No newline at end of file
diff --git a/examples/jupyter/observability/quickstart.ipynb b/examples/jupyter/observability/quickstart.ipynb
deleted file mode 100644
index a16891c606..0000000000
--- a/examples/jupyter/observability/quickstart.ipynb
+++ /dev/null
@@ -1,328 +0,0 @@
-{
- "cells": [
- {
- "cell_type": "markdown",
- "id": "a1b2c3d4",
- "metadata": {},
- "source": [
- "# Quick Start: Observability with Agenta\n",
- "\n",
- "Agenta enables you to capture all inputs, outputs, and metadata from your LLM applications, **whether they're hosted within Agenta or running in your own environment**.\n",
- "\n",
- "This guide will walk you through setting up observability for an OpenAI application running locally.\n",
- "\n",
- "**Note:** If you create an application through the Agenta UI, tracing is enabled by default. No additional setup is required—simply go to the observability view to see all your requests."
- ]
- },
- {
- "cell_type": "markdown",
- "id": "b2c3d4e5",
- "metadata": {},
- "source": [
- "## Step 1: Install Required Packages\n",
- "\n",
- "First, install the Agenta SDK, OpenAI, and the OpenTelemetry instrumentor for OpenAI:"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 1,
- "id": "c3d4e5f6",
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "Requirement already satisfied: agenta in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (0.51.6)\n",
- "Collecting agenta\n",
- " Downloading agenta-0.59.6-py3-none-any.whl.metadata (31 kB)\n",
- "Requirement already satisfied: openai in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (1.107.1)\n",
- "Collecting openai\n",
- " Downloading openai-2.6.1-py3-none-any.whl.metadata (29 kB)\n",
- "Requirement already satisfied: opentelemetry-instrumentation-openai in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (0.46.2)\n",
- "Collecting opentelemetry-instrumentation-openai\n",
- " Downloading opentelemetry_instrumentation_openai-0.47.5-py3-none-any.whl.metadata (2.2 kB)\n",
- "Requirement already satisfied: decorator<6.0.0,>=5.2.1 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (5.2.1)\n",
- "Requirement already satisfied: fastapi<0.117.0,>=0.116.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (0.116.1)\n",
- "Requirement already satisfied: google-auth<3,>=2.23 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (2.40.3)\n",
- "Requirement already satisfied: h11<0.17.0,>=0.16.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (0.16.0)\n",
- "Requirement already satisfied: httpx<0.29.0,>=0.28.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (0.28.1)\n",
- "Requirement already satisfied: huggingface-hub<0.31.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (0.30.2)\n",
- "Requirement already satisfied: importlib-metadata<9.0,>=8.0.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (8.7.0)\n",
- "Requirement already satisfied: jinja2<4.0.0,>=3.1.6 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (3.1.6)\n",
- "Collecting litellm==1.78.7 (from agenta)\n",
- " Downloading litellm-1.78.7-py3-none-any.whl.metadata (42 kB)\n",
- "Collecting openai\n",
- " Downloading openai-1.109.1-py3-none-any.whl.metadata (29 kB)\n",
- "Requirement already satisfied: opentelemetry-api<2.0.0,>=1.27.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (1.36.0)\n",
- "Requirement already satisfied: opentelemetry-exporter-otlp-proto-http<2.0.0,>=1.27.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (1.36.0)\n",
- "Requirement already satisfied: opentelemetry-instrumentation>=0.56b0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (0.57b0)\n",
- "Requirement already satisfied: opentelemetry-sdk<2.0.0,>=1.27.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (1.36.0)\n",
- "Requirement already satisfied: pydantic<3,>=2 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (2.11.7)\n",
- "Requirement already satisfied: python-dotenv<2.0.0,>=1.0.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (1.1.1)\n",
- "Requirement already satisfied: pyyaml<7.0.0,>=6.0.2 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (6.0.2)\n",
- "Requirement already satisfied: starlette<0.48.0,>=0.47.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (0.47.3)\n",
- "Requirement already satisfied: structlog<26.0.0,>=25.2.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (25.4.0)\n",
- "Requirement already satisfied: tiktoken==0.11.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (0.11.0)\n",
- "Requirement already satisfied: toml<0.11.0,>=0.10.2 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (0.10.2)\n",
- "Requirement already satisfied: aiohttp>=3.10 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from litellm==1.78.7->agenta) (3.12.15)\n",
- "Requirement already satisfied: click in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from litellm==1.78.7->agenta) (8.2.1)\n",
- "Collecting fastuuid>=0.13.0 (from litellm==1.78.7->agenta)\n",
- " Downloading fastuuid-0.14.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (1.1 kB)\n",
- "Requirement already satisfied: jsonschema<5.0.0,>=4.22.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from litellm==1.78.7->agenta) (4.25.1)\n",
- "Requirement already satisfied: tokenizers in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from litellm==1.78.7->agenta) (0.22.0)\n",
- "Requirement already satisfied: regex>=2022.1.18 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from tiktoken==0.11.0->agenta) (2025.9.1)\n",
- "Requirement already satisfied: requests>=2.26.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from tiktoken==0.11.0->agenta) (2.32.5)\n",
- "Requirement already satisfied: anyio<5,>=3.5.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from openai) (4.10.0)\n",
- "Requirement already satisfied: distro<2,>=1.7.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from openai) (1.9.0)\n",
- "Requirement already satisfied: jiter<1,>=0.4.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from openai) (0.10.0)\n",
- "Requirement already satisfied: sniffio in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from openai) (1.3.1)\n",
- "Requirement already satisfied: tqdm>4 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from openai) (4.67.1)\n",
- "Requirement already satisfied: typing-extensions<5,>=4.11 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from openai) (4.15.0)\n",
- "Requirement already satisfied: idna>=2.8 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from anyio<5,>=3.5.0->openai) (3.10)\n",
- "Requirement already satisfied: cachetools<6.0,>=2.0.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from google-auth<3,>=2.23->agenta) (5.5.2)\n",
- "Requirement already satisfied: pyasn1-modules>=0.2.1 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from google-auth<3,>=2.23->agenta) (0.4.2)\n",
- "Requirement already satisfied: rsa<5,>=3.1.4 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from google-auth<3,>=2.23->agenta) (4.9.1)\n",
- "Requirement already satisfied: certifi in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from httpx<0.29.0,>=0.28.0->agenta) (2025.8.3)\n",
- "Requirement already satisfied: httpcore==1.* in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from httpx<0.29.0,>=0.28.0->agenta) (1.0.9)\n",
- "Requirement already satisfied: filelock in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from huggingface-hub<0.31.0->agenta) (3.19.1)\n",
- "Requirement already satisfied: fsspec>=2023.5.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from huggingface-hub<0.31.0->agenta) (2025.7.0)\n",
- "Requirement already satisfied: packaging>=20.9 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from huggingface-hub<0.31.0->agenta) (25.0)\n",
- "Requirement already satisfied: zipp>=3.20 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from importlib-metadata<9.0,>=8.0.0->agenta) (3.23.0)\n",
- "Requirement already satisfied: MarkupSafe>=2.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from jinja2<4.0.0,>=3.1.6->agenta) (3.0.2)\n",
- "Requirement already satisfied: attrs>=22.2.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from jsonschema<5.0.0,>=4.22.0->litellm==1.78.7->agenta) (25.3.0)\n",
- "Requirement already satisfied: jsonschema-specifications>=2023.03.6 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from jsonschema<5.0.0,>=4.22.0->litellm==1.78.7->agenta) (2025.4.1)\n",
- "Requirement already satisfied: referencing>=0.28.4 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from jsonschema<5.0.0,>=4.22.0->litellm==1.78.7->agenta) (0.36.2)\n",
- "Requirement already satisfied: rpds-py>=0.7.1 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from jsonschema<5.0.0,>=4.22.0->litellm==1.78.7->agenta) (0.27.1)\n",
- "Requirement already satisfied: googleapis-common-protos~=1.52 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from opentelemetry-exporter-otlp-proto-http<2.0.0,>=1.27.0->agenta) (1.70.0)\n",
- "Requirement already satisfied: opentelemetry-exporter-otlp-proto-common==1.36.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from opentelemetry-exporter-otlp-proto-http<2.0.0,>=1.27.0->agenta) (1.36.0)\n",
- "Requirement already satisfied: opentelemetry-proto==1.36.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from opentelemetry-exporter-otlp-proto-http<2.0.0,>=1.27.0->agenta) (1.36.0)\n",
- "Requirement already satisfied: protobuf<7.0,>=5.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from opentelemetry-proto==1.36.0->opentelemetry-exporter-otlp-proto-http<2.0.0,>=1.27.0->agenta) (6.32.0)\n",
- "Requirement already satisfied: opentelemetry-semantic-conventions==0.57b0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from opentelemetry-sdk<2.0.0,>=1.27.0->agenta) (0.57b0)\n",
- "Requirement already satisfied: annotated-types>=0.6.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from pydantic<3,>=2->agenta) (0.7.0)\n",
- "Requirement already satisfied: pydantic-core==2.33.2 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from pydantic<3,>=2->agenta) (2.33.2)\n",
- "Requirement already satisfied: typing-inspection>=0.4.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from pydantic<3,>=2->agenta) (0.4.1)\n",
- "Requirement already satisfied: charset_normalizer<4,>=2 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from requests>=2.26.0->tiktoken==0.11.0->agenta) (3.4.3)\n",
- "Requirement already satisfied: urllib3<3,>=1.21.1 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from requests>=2.26.0->tiktoken==0.11.0->agenta) (2.5.0)\n",
- "Requirement already satisfied: pyasn1>=0.1.3 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from rsa<5,>=3.1.4->google-auth<3,>=2.23->agenta) (0.6.1)\n",
- "Requirement already satisfied: opentelemetry-semantic-conventions-ai<0.5.0,>=0.4.13 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from opentelemetry-instrumentation-openai) (0.4.13)\n",
- "Requirement already satisfied: aiohappyeyeballs>=2.5.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from aiohttp>=3.10->litellm==1.78.7->agenta) (2.6.1)\n",
- "Requirement already satisfied: aiosignal>=1.4.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from aiohttp>=3.10->litellm==1.78.7->agenta) (1.4.0)\n",
- "Requirement already satisfied: frozenlist>=1.1.1 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from aiohttp>=3.10->litellm==1.78.7->agenta) (1.7.0)\n",
- "Requirement already satisfied: multidict<7.0,>=4.5 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from aiohttp>=3.10->litellm==1.78.7->agenta) (6.6.4)\n",
- "Requirement already satisfied: propcache>=0.2.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from aiohttp>=3.10->litellm==1.78.7->agenta) (0.3.2)\n",
- "Requirement already satisfied: yarl<2.0,>=1.17.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from aiohttp>=3.10->litellm==1.78.7->agenta) (1.20.1)\n",
- "Requirement already satisfied: wrapt<2.0.0,>=1.0.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from opentelemetry-instrumentation>=0.56b0->agenta) (1.17.3)\n",
- "Downloading agenta-0.59.6-py3-none-any.whl (339 kB)\n",
- "Downloading litellm-1.78.7-py3-none-any.whl (9.9 MB)\n",
- "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m9.9/9.9 MB\u001b[0m \u001b[31m80.7 MB/s\u001b[0m \u001b[33m0:00:00\u001b[0m\n",
- "\u001b[?25hDownloading openai-1.109.1-py3-none-any.whl (948 kB)\n",
- "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m948.6/948.6 kB\u001b[0m \u001b[31m26.3 MB/s\u001b[0m \u001b[33m0:00:00\u001b[0m\n",
- "\u001b[?25hDownloading opentelemetry_instrumentation_openai-0.47.5-py3-none-any.whl (35 kB)\n",
- "Downloading fastuuid-0.14.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (278 kB)\n",
- "Installing collected packages: fastuuid, openai, litellm, opentelemetry-instrumentation-openai, agenta\n",
- "\u001b[2K Attempting uninstall: openai\n",
- "\u001b[2K Found existing installation: openai 1.107.1\n",
- "\u001b[2K Uninstalling openai-1.107.1:\n",
- "\u001b[2K Successfully uninstalled openai-1.107.1\n",
- "\u001b[2K Attempting uninstall: litellm[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1/5\u001b[0m [openai]\n",
- "\u001b[2K Found existing installation: litellm 1.76.0━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1/5\u001b[0m [openai]\n",
- "\u001b[2K Uninstalling litellm-1.76.0:m╺\u001b[0m\u001b[90m━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2/5\u001b[0m [litellm]\n",
- "\u001b[2K Successfully uninstalled litellm-1.76.0━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2/5\u001b[0m [litellm]\n",
- "\u001b[2K Attempting uninstall: opentelemetry-instrumentation-openai━━━━━━\u001b[0m \u001b[32m2/5\u001b[0m [litellm]\n",
- "\u001b[2K Found existing installation: opentelemetry-instrumentation-openai 0.46.2[0m [litellm]\n",
- "\u001b[2K Uninstalling opentelemetry-instrumentation-openai-0.46.2:━\u001b[0m \u001b[32m2/5\u001b[0m [litellm]\n",
- "\u001b[2K Successfully uninstalled opentelemetry-instrumentation-openai-0.46.25\u001b[0m [litellm]\n",
- "\u001b[2K Attempting uninstall: agenta\u001b[0m\u001b[90m━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2/5\u001b[0m [litellm]\n",
- "\u001b[2K Found existing installation: agenta 0.51.6━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2/5\u001b[0m [litellm]\n",
- "\u001b[2K Uninstalling agenta-0.51.6:[0m\u001b[90m━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2/5\u001b[0m [litellm]\n",
- "\u001b[2K Successfully uninstalled agenta-0.51.6━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2/5\u001b[0m [litellm]\n",
- "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m5/5\u001b[0m [agenta]2m4/5\u001b[0m [agenta]\n",
- "\u001b[1A\u001b[2KSuccessfully installed agenta-0.59.6 fastuuid-0.14.0 litellm-1.78.7 openai-1.109.1 opentelemetry-instrumentation-openai-0.47.5\n",
- "Note: you may need to restart the kernel to use updated packages.\n"
- ]
- }
- ],
- "source": [
- "pip install -U agenta openai opentelemetry-instrumentation-openai"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "d4e5f6g7",
- "metadata": {},
- "source": [
- "## Step 2: Configure Environment Variables\n",
- "\n",
- "To start tracing your application, you'll need an API key:\n",
- "\n",
- "1. Visit the Agenta API Keys page under settings.\n",
- "2. Click on **Create New API Key** and follow the prompts.\n",
- "\n",
- "Then set your environment variables:"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "e5f6g7h8",
- "metadata": {},
- "outputs": [],
- "source": [
- "import os\n",
- "\n",
- "# Set your API key here\n",
- "os.environ[\"AGENTA_API_KEY\"] = \"\"\n",
- "os.environ[\"AGENTA_HOST\"] = \"https://cloud.agenta.ai\" # Change for self-hosted\n",
- "os.environ[\"OPENAI_API_KEY\"] = \"\""
- ]
- },
- {
- "cell_type": "markdown",
- "id": "f6g7h8i9",
- "metadata": {},
- "source": [
- "## Step 3: Instrument Your Application\n",
- "\n",
- "Below is a sample script to instrument an OpenAI application:"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 3,
- "id": "g7h8i9j0",
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "2025-10-28T10:24:18.411Z \u001b[38;5;70m[INFO.]\u001b[0m Agenta - SDK version: 0.59.6 \u001b[38;5;245m[agenta.sdk.agenta_init]\u001b[0m \n",
- "2025-10-28T10:24:18.411Z \u001b[38;5;70m[INFO.]\u001b[0m Agenta - Host: https://cloud.agenta.ai \u001b[38;5;245m[agenta.sdk.agenta_init]\u001b[0m \n",
- "2025-10-28T10:24:18.412Z \u001b[38;5;70m[INFO.]\u001b[0m Agenta - OLTP URL: https://cloud.agenta.ai/api/otlp/v1/traces \u001b[38;5;245m[agenta.sdk.tracing.tracing]\u001b[0m \n"
- ]
- }
- ],
- "source": [
- "import agenta as ag\n",
- "from opentelemetry.instrumentation.openai import OpenAIInstrumentor\n",
- "import openai\n",
- "\n",
- "# Initialize Agenta\n",
- "ag.init()\n",
- "\n",
- "# Instrument OpenAI\n",
- "OpenAIInstrumentor().instrument()"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "h8i9j0k1",
- "metadata": {},
- "source": [
- "## Step 4: Create an Instrumented Function\n",
- "\n",
- "Decorate your function with `@ag.instrument()` to enable tracing:"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 5,
- "id": "i9j0k1l2",
- "metadata": {},
- "outputs": [],
- "source": [
- "@ag.instrument()\n",
- "def generate():\n",
- " response = openai.chat.completions.create(\n",
- " model=\"gpt-3.5-turbo\",\n",
- " messages=[\n",
- " {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n",
- " {\"role\": \"user\", \"content\": \"Write a short story about AI Engineering.\"},\n",
- " ],\n",
- " )\n",
- " return response.choices[0].message.content"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "j0k1l2m3",
- "metadata": {},
- "source": [
- "## Step 5: Run Your Application\n",
- "\n",
- "Call your instrumented function to generate a trace:"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 6,
- "id": "k1l2m3n4",
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "In a future where artificial intelligence had revolutionized society, a young engineer named Maya was fascinated by the endless possibilities of AI technology. She dedicated her life to mastering the complexities of AI engineering, pushing the boundaries of what was thought possible.\n",
- "\n",
- "Maya worked tirelessly, constantly experimenting and innovating to create AI systems that could change the world for the better. Her creations helped streamline businesses, improve healthcare outcomes, and enhance everyday tasks for people around the globe.\n",
- "\n",
- "However, Maya soon realized that not all AI technology was being used ethically. Some companies exploited AI for profit, using it to manipulate data or invade people's privacy. Troubled by this misuse of technology, Maya made it her mission to advocate for responsible AI engineering practices.\n",
- "\n",
- "Through her dedication and leadership, Maya became a voice for ethical AI engineering, pushing for regulations and guidelines to ensure that AI technology was used for the greater good of humanity. Her passion inspired others in the field to prioritize ethics and social responsibility in their work.\n",
- "\n",
- "As the world grappled with the emerging challenges of AI technology, Maya stood at the forefront, a shining example of how innovation and integrity could go hand in hand in the exciting world of AI engineering.\n"
- ]
- }
- ],
- "source": [
- "if __name__ == \"__main__\":\n",
- " result = generate()\n",
- " print(result)"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "l2m3n4o5",
- "metadata": {},
- "source": [
- "## Step 6: View Traces in the Agenta UI\n",
- "\n",
- "After running your application, you can view the captured traces in Agenta:\n",
- "\n",
- "1. Log in to your Agenta dashboard.\n",
- "2. Navigate to the **Observability** section.\n",
- "3. You'll see a list of traces corresponding to your application's requests.\n",
- "\n",
- "Each trace will show you the inputs, outputs, and metadata from your LLM application, including:\n",
- "- Function execution time\n",
- "- OpenAI API calls and responses\n",
- "- Token usage\n",
- "- Any errors or exceptions"
- ]
- }
- ],
- "metadata": {
- "kernelspec": {
- "display_name": ".venv",
- "language": "python",
- "name": "python3"
- },
- "language_info": {
- "codemirror_mode": {
- "name": "ipython",
- "version": 3
- },
- "file_extension": ".py",
- "mimetype": "text/x-python",
- "name": "python",
- "nbconvert_exporter": "python",
- "pygments_lexer": "ipython3",
- "version": "3.12.3"
- }
- },
- "nbformat": 4,
- "nbformat_minor": 5
-}
diff --git a/examples/jupyter/observability/trace-with-python-sdk-tutorial.ipynb b/examples/jupyter/observability/trace-with-python-sdk-tutorial.ipynb
deleted file mode 100644
index d40d9dc9af..0000000000
--- a/examples/jupyter/observability/trace-with-python-sdk-tutorial.ipynb
+++ /dev/null
@@ -1,524 +0,0 @@
-{
- "cells": [
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "# Trace with Python SDK - Tutorial\n",
- "\n",
- "This tutorial demonstrates how to use the Agenta Python SDK to trace your LLM applications. You'll learn how to:\n",
- "\n",
- "- Set up tracing with the Agenta SDK\n",
- "- Instrument functions and OpenAI calls automatically\n",
- "- Start and end spans manually to capture internals\n",
- "- Reference prompt versions in your traces\n",
- "- Redact sensitive data from traces\n",
- "\n",
- "## What You'll Build\n",
- "\n",
- "We'll create a simple LLM application that:\n",
- "1. Uses OpenAI auto-instrumentation to trace API calls\n",
- "2. Instruments custom functions to capture workflow steps\n",
- "3. Stores internal data like retrieved context\n",
- "4. Links traces to deployed prompt versions\n",
- "5. Redacts sensitive information from traces"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Install Dependencies"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "pip install -U agenta openai opentelemetry-instrumentation-openai"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Setup\n",
- "\n",
- "Before using the SDK, we need to initialize it with your API keys. The SDK requires:\n",
- "- **Agenta API Key**: For sending traces to Agenta\n",
- "- **OpenAI API Key**: For making LLM calls\n",
- "\n",
- "You can get your Agenta API key from the [API Keys page](https://cloud.agenta.ai/settings?tab=apiKeys)."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "import os\n",
- "os.environ[\"AGENTA_HOST\"] = \"https://cloud.agenta.ai/\" # Default value, change for self-hosted\n",
- "os.environ[\"AGENTA_API_KEY\"] = \"\"\n",
- "os.environ[\"OPENAI_API_KEY\"] = \"\""
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "import os\n",
- "import agenta as ag\n",
- "from getpass import getpass\n",
- "\n",
- "# Initialize the SDK with your API key\n",
- "api_key = os.getenv(\"AGENTA_API_KEY\")\n",
- "if not api_key:\n",
- " os.environ[\"AGENTA_API_KEY\"] = getpass(\"Enter your Agenta API key: \")\n",
- "\n",
- "openai_api_key = os.getenv(\"OPENAI_API_KEY\")\n",
- "if not openai_api_key:\n",
- " os.environ[\"OPENAI_API_KEY\"] = getpass(\"Enter your OpenAI API key: \")\n",
- "\n",
- "# Initialize the SDK\n",
- "ag.init()"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Part 1: Setup Tracing with OpenAI Auto-Instrumentation\n",
- "\n",
- "The Agenta SDK provides two powerful mechanisms for tracing:\n",
- "\n",
- "1. **Auto-instrumentation**: Automatically traces third-party libraries like OpenAI\n",
- "2. **Function decorators**: Manually instrument your custom functions\n",
- "\n",
- "Let's start by setting up OpenAI auto-instrumentation, which will capture all OpenAI API calls automatically."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "from opentelemetry.instrumentation.openai import OpenAIInstrumentor\n",
- "import openai\n",
- "\n",
- "# Instrument OpenAI to automatically trace all API calls\n",
- "OpenAIInstrumentor().instrument()\n",
- "\n",
- "print(\"OpenAI auto-instrumentation enabled!\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Part 2: Instrument Functions\n",
- "\n",
- "Now let's create a simple function and instrument it using the `@ag.instrument()` decorator. This will create a span for the function and automatically capture its inputs and outputs.\n",
- "\n",
- "The decorator accepts a `spankind` parameter to categorize the span. Available types include: `agent`, `chain`, `workflow`, `tool`, `embedding`, `query`, `completion`, `chat`, `rerank`."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "@ag.instrument(spankind=\"workflow\")\n",
- "def generate_story(topic: str):\n",
- " \"\"\"Generate a short story about the given topic.\"\"\"\n",
- " response = openai.chat.completions.create(\n",
- " model=\"gpt-3.5-turbo\",\n",
- " messages=[\n",
- " {\"role\": \"system\", \"content\": \"You are a creative storyteller.\"},\n",
- " {\"role\": \"user\", \"content\": f\"Write a short story about {topic}.\"},\n",
- " ],\n",
- " )\n",
- " return response.choices[0].message.content\n",
- "\n",
- "# Test the instrumented function\n",
- "story = generate_story(\"AI Engineering\")\n",
- "print(\"Generated story:\")\n",
- "print(story)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Part 3: Starting Spans and Storing Internals\n",
- "\n",
- "Sometimes you need to capture intermediate data that isn't part of the function's inputs or outputs. The SDK provides two methods:\n",
- "\n",
- "- `ag.tracing.store_meta()`: Add metadata to a span (saved under `ag.meta`)\n",
- "- `ag.tracing.store_internals()`: Store internal data (saved under `ag.data.internals`)\n",
- "\n",
- "Internals are especially useful because they:\n",
- "1. Are searchable using plain text queries\n",
- "2. Appear in the overview tab of the observability drawer\n",
- "\n",
- "Let's create a RAG (Retrieval-Augmented Generation) example that captures the retrieved context:"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "@ag.instrument(spankind=\"tool\")\n",
- "def retrieve_context(query: str):\n",
- " \"\"\"Simulate retrieving context from a knowledge base.\"\"\"\n",
- " # In a real application, this would query a vector database\n",
- " context = [\n",
- " \"Agenta is an open-source LLM developer platform.\",\n",
- " \"Agenta provides tools for prompt management, evaluation, and observability.\",\n",
- " \"The Agenta SDK supports tracing with OpenTelemetry.\",\n",
- " ]\n",
- " \n",
- " # Store metadata about the retrieval\n",
- " ag.tracing.store_meta({\n",
- " \"retrieval_method\": \"vector_search\",\n",
- " \"num_results\": len(context)\n",
- " })\n",
- " \n",
- " return context\n",
- "\n",
- "@ag.instrument(spankind=\"workflow\")\n",
- "def rag_workflow(query: str):\n",
- " \"\"\"Answer a question using retrieved context.\"\"\"\n",
- " # Retrieve context\n",
- " context = retrieve_context(query)\n",
- " \n",
- " # Store the retrieved context as internals\n",
- " # This makes it visible in the UI and searchable\n",
- " ag.tracing.store_internals({\"retrieved_context\": context})\n",
- " \n",
- " # Generate answer using context\n",
- " context_str = \"\\n\".join(context)\n",
- " prompt = f\"Answer the following question based on the context:\\n\\nContext:\\n{context_str}\\n\\nQuestion: {query}\"\n",
- " \n",
- " response = openai.chat.completions.create(\n",
- " model=\"gpt-3.5-turbo\",\n",
- " messages=[\n",
- " {\"role\": \"system\", \"content\": \"You are a helpful assistant. Answer questions based only on the provided context.\"},\n",
- " {\"role\": \"user\", \"content\": prompt},\n",
- " ],\n",
- " )\n",
- " \n",
- " return response.choices[0].message.content\n",
- "\n",
- "# Test the RAG workflow\n",
- "answer = rag_workflow(\"What is Agenta?\")\n",
- "print(\"Answer:\")\n",
- "print(answer)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Part 4: Reference Prompt Versions\n",
- "\n",
- "One of Agenta's powerful features is linking traces to specific prompt versions. This allows you to:\n",
- "- Filter traces by application, variant, or environment\n",
- "- Compare performance across different variants\n",
- "- Track production behavior\n",
- "\n",
- "Let's create a prompt using the SDK, deploy it, and then reference it in our traces."
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### Create and Deploy a Prompt"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": "from agenta.sdk.types import PromptTemplate, Message, ModelConfig\nfrom pydantic import BaseModel\n\n# Define the prompt configuration\nclass Config(BaseModel):\n prompt: PromptTemplate\n\nconfig = Config(\n prompt=PromptTemplate(\n messages=[\n Message(role=\"system\", content=\"You are a helpful assistant that explains topics clearly.\"),\n Message(role=\"user\", content=\"Explain {{topic}} in simple terms.\"),\n ],\n llm_config=ModelConfig(\n model=\"gpt-3.5-turbo\",\n max_tokens=200,\n temperature=0.7,\n top_p=1.0,\n frequency_penalty=0.0,\n presence_penalty=0.0,\n ),\n template_format=\"curly\"\n )\n)\n\n# Create an application and variant\napp = ag.AppManager.create(\n app_slug=\"topic-explainer-traced\",\n template_key=\"SERVICE:completion\",\n)\n\nprint(f\"Created application: {app.app_name}\")\n\n# Create a variant with the prompt configuration\nvariant = ag.VariantManager.create(\n parameters=config.model_dump(),\n app_slug=\"topic-explainer-traced\",\n variant_slug=\"production-variant\"\n)\n\nprint(f\"Created variant: {variant.variant_slug} (version {variant.variant_version})\")\n\n# Deploy to production environment\ndeployment = ag.DeploymentManager.deploy(\n app_slug=\"topic-explainer-traced\",\n variant_slug=\"production-variant\",\n environment_slug=\"production\",\n)\n\nprint(f\"Deployed to {deployment.environment_slug} environment\")"
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### Reference the Prompt in Traces\n",
- "\n",
- "Now we'll create a function that uses the deployed prompt and links its traces to the application and environment."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "@ag.instrument(spankind=\"workflow\")\n",
- "def explain_topic_with_prompt(topic: str):\n",
- " \"\"\"Explain a topic using the deployed prompt configuration.\"\"\"\n",
- " \n",
- " # Fetch the prompt configuration from production\n",
- " prompt_config = ag.ConfigManager.get_from_registry(\n",
- " app_slug=\"topic-explainer-traced\",\n",
- " environment_slug=\"production\"\n",
- " )\n",
- " \n",
- " # Format the prompt with the topic\n",
- " prompt_template = PromptTemplate(**prompt_config[\"prompt\"])\n",
- " formatted_prompt = prompt_template.format(topic=topic)\n",
- " \n",
- " # Make the OpenAI call\n",
- " response = openai.chat.completions.create(\n",
- " **formatted_prompt.to_openai_kwargs()\n",
- " )\n",
- " \n",
- " # Link this trace to the application and environment\n",
- " ag.tracing.store_refs({\n",
- " \"application.slug\": \"topic-explainer-traced\",\n",
- " \"variant.slug\": \"production-variant\",\n",
- " \"environment.slug\": \"production\",\n",
- " })\n",
- " \n",
- " return response.choices[0].message.content\n",
- "\n",
- "# Test the function\n",
- "explanation = explain_topic_with_prompt(\"machine learning\")\n",
- "print(\"Explanation:\")\n",
- "print(explanation)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Part 5: Redact Sensitive Data\n",
- "\n",
- "When working with production data, you often need to exclude sensitive information from traces. The Agenta SDK provides several ways to redact data:\n",
- "\n",
- "1. **Simple redaction**: Ignore all inputs/outputs\n",
- "2. **Selective redaction**: Ignore specific fields\n",
- "3. **Custom redaction**: Use a callback function for fine-grained control\n",
- "4. **Global redaction**: Apply rules across all instrumented functions\n",
- "\n",
- "Let's explore these different approaches."
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### Simple Redaction: Ignore All Inputs/Outputs"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "@ag.instrument(\n",
- " spankind=\"workflow\",\n",
- " ignore_inputs=True,\n",
- " ignore_outputs=True\n",
- ")\n",
- "def process_sensitive_data(user_email: str, credit_card: str):\n",
- " \"\"\"Process sensitive data without logging inputs/outputs.\"\"\"\n",
- " # The function inputs and outputs won't be captured in the trace\n",
- " result = f\"Processed data for {user_email}\"\n",
- " return result\n",
- "\n",
- "# This trace will not contain inputs or outputs\n",
- "result = process_sensitive_data(\"user@example.com\", \"4111-1111-1111-1111\")\n",
- "print(result)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### Selective Redaction: Ignore Specific Fields"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "@ag.instrument(\n",
- " spankind=\"workflow\",\n",
- " ignore_inputs=[\"api_key\", \"password\"],\n",
- " ignore_outputs=[\"internal_token\"]\n",
- ")\n",
- "def authenticate_user(username: str, password: str, api_key: str):\n",
- " \"\"\"Authenticate a user (password and api_key will be redacted).\"\"\"\n",
- " # Simulate authentication\n",
- " return {\n",
- " \"username\": username,\n",
- " \"authenticated\": True,\n",
- " \"internal_token\": \"secret-token-12345\", # This will be redacted\n",
- " }\n",
- "\n",
- "# The trace will show username but not password or api_key\n",
- "auth_result = authenticate_user(\"john_doe\", \"secret123\", \"sk-abc123\")\n",
- "print(f\"Authenticated: {auth_result['authenticated']}\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### Custom Redaction: Use a Callback Function"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "import re\n",
- "\n",
- "def redact_pii(name: str, field: str, data: dict):\n",
- " \"\"\"Custom redaction function that removes PII.\"\"\"\n",
- " if field == \"inputs\":\n",
- " # Redact email addresses\n",
- " if \"email\" in data:\n",
- " data[\"email\"] = \"[REDACTED]\"\n",
- " # Redact phone numbers\n",
- " if \"phone\" in data:\n",
- " data[\"phone\"] = \"[REDACTED]\"\n",
- " \n",
- " if field == \"outputs\":\n",
- " # Redact any credit card patterns\n",
- " if isinstance(data, dict):\n",
- " for key, value in data.items():\n",
- " if isinstance(value, str):\n",
- " # Simple credit card pattern\n",
- " data[key] = re.sub(r'\\d{4}[-\\s]?\\d{4}[-\\s]?\\d{4}[-\\s]?\\d{4}', '[CARD-REDACTED]', value)\n",
- " \n",
- " return data\n",
- "\n",
- "@ag.instrument(\n",
- " spankind=\"workflow\",\n",
- " redact=redact_pii,\n",
- " redact_on_error=False # Don't apply redaction if it raises an error\n",
- ")\n",
- "def process_customer_order(name: str, email: str, phone: str, card_number: str):\n",
- " \"\"\"Process a customer order with PII redaction.\"\"\"\n",
- " return {\n",
- " \"status\": \"processed\",\n",
- " \"customer\": name,\n",
- " \"payment_info\": f\"Charged card ending in {card_number[-4:]}\",\n",
- " \"full_card\": card_number # This will be redacted\n",
- " }\n",
- "\n",
- "# Test with sample data\n",
- "order = process_customer_order(\n",
- " name=\"Jane Smith\",\n",
- " email=\"jane@example.com\", # Will be redacted\n",
- " phone=\"555-1234\", # Will be redacted\n",
- " card_number=\"4111-1111-1111-1111\" # Will be redacted in output\n",
- ")\n",
- "print(f\"Order status: {order['status']}\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### Global Redaction: Apply Rules Across All Functions\n",
- "\n",
- "For organization-wide policies, you can set up global redaction rules during initialization. Note: Since we already called `ag.init()`, this is just for demonstration. In a real application, you would set this during the initial setup."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "# Example of global redaction setup (would be done during ag.init())\n",
- "from typing import Dict, Any\n",
- "\n",
- "def global_redact_function(name: str, field: str, data: Dict[str, Any]):\n",
- " \"\"\"Global redaction that applies to all instrumented functions.\"\"\"\n",
- " # Remove any field containing 'api_key' or 'secret'\n",
- " if isinstance(data, dict):\n",
- " keys_to_redact = [k for k in data.keys() if 'api_key' in k.lower() or 'secret' in k.lower()]\n",
- " for key in keys_to_redact:\n",
- " data[key] = \"[REDACTED]\"\n",
- " \n",
- " return data\n",
- "\n",
- "# In production, you would initialize like this:\n",
- "# ag.init(\n",
- "# redact=global_redact_function,\n",
- "# redact_on_error=True\n",
- "# )\n",
- "\n",
- "print(\"Global redaction would be configured during ag.init()\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Summary\n",
- "\n",
- "In this tutorial, you learned how to:\n",
- "\n",
- "1. ✅ **Set up tracing** with the Agenta SDK and OpenAI auto-instrumentation\n",
- "2. ✅ **Instrument functions** using the `@ag.instrument()` decorator\n",
- "3. ✅ **Store internals and metadata** to capture intermediate data in your workflows\n",
- "4. ✅ **Reference prompt versions** by creating, deploying, and linking traces to applications\n",
- "5. ✅ **Redact sensitive data** using multiple approaches for privacy protection\n",
- "\n",
- "## Next Steps\n",
- "\n",
- "- Explore [distributed tracing](/observability/trace-with-python-sdk/distributed-tracing) for multi-service applications\n",
- "- Learn about [cost tracking](/observability/trace-with-python-sdk/track-costs) to monitor LLM expenses\n",
- "- Understand [trace annotations](/observability/trace-with-python-sdk/annotate-traces) for collecting feedback\n",
- "- Check out the [Agenta UI guide](/observability/using-the-ui/filtering-traces) for filtering and analyzing traces"
- ]
- }
- ],
- "metadata": {
- "kernelspec": {
- "display_name": "Python 3",
- "language": "python",
- "name": "python3"
- },
- "language_info": {
- "codemirror_mode": {
- "name": "ipython",
- "version": 3
- },
- "file_extension": ".py",
- "mimetype": "text/x-python",
- "name": "python",
- "nbconvert_exporter": "python",
- "pygments_lexer": "ipython3",
- "version": "3.9.0"
- },
- "colab": {
- "provenance": []
- }
- },
- "nbformat": 4,
- "nbformat_minor": 0
-}
\ No newline at end of file
diff --git a/examples/jupyter/observability_langchain.ipynb b/examples/jupyter/observability_langchain.ipynb
index 0fa0e070e0..f2b6321392 100644
--- a/examples/jupyter/observability_langchain.ipynb
+++ b/examples/jupyter/observability_langchain.ipynb
@@ -74,10 +74,10 @@
"import os\n",
"import getpass\n",
"\n",
- "os.environ[\"OPENAI_API_KEY\"] = getpass.getpass(prompt=\"OpenAI API key:\")\n",
+ "os.environ[\"OPENAI_API_KEY\"] = getpass.getpass(prompt=\"OpenAI API key:\") \n",
"# Create an Agenta API key under https://cloud.agenta.ai/settings?tab=apiKeys\n",
- "os.environ[\"AGENTA_API_KEY\"] = getpass.getpass(prompt=\"Agenta API key:\")\n",
- "os.environ[\"AGENTA_HOST\"] = \"https://cloud.agenta.ai\" # Change for self-hosted"
+ "os.environ[\"AGENTA_API_KEY\"] = getpass.getpass(prompt=\"Agenta API key:\") \n",
+ "os.environ[\"AGENTA_HOST\"] = \"https://cloud.agenta.ai\" # Change for self-hosted\n"
]
},
{
@@ -115,7 +115,7 @@
"\n",
"ag.init()\n",
"\n",
- "LangchainInstrumentor().instrument()"
+ "LangchainInstrumentor().instrument()\n"
]
},
{
@@ -174,19 +174,17 @@
"Answer:\n",
"\"\"\"\n",
"\n",
- "prompt_template = ChatPromptTemplate(\n",
- " [\n",
- " (\"human\", prompt),\n",
- " ]\n",
- ")\n",
+ "prompt_template = ChatPromptTemplate([\n",
+ " (\"human\", prompt),\n",
+ "])\n",
"\n",
"llm = ChatOpenAI(model=\"gpt-4o-mini\")\n",
"\n",
"loader = WebBaseLoader(\n",
- " web_paths=(\n",
- " \"https://docs.agenta.ai/prompt-engineering/managing-prompts-programatically/create-and-commit\",\n",
+ " web_paths=(\"https://docs.agenta.ai/prompt-engineering/prompt-management/prompt-management-sdk\",),\n",
+ " bs_kwargs=dict(\n",
+ " parse_only=bs4.SoupStrainer('article') # Only parse the core\n",
" ),\n",
- " bs_kwargs=dict(parse_only=bs4.SoupStrainer(\"article\")), # Only parse the core\n",
")\n",
"docs = loader.load()\n",
"\n",
@@ -205,7 +203,8 @@
" | StrOutputParser()\n",
")\n",
"\n",
- "rag_chain.invoke(\"How can I save a new version of a prompt in Agenta?\")"
+ "rag_chain.invoke(\"How can I save a new version of a prompt in Agenta?\")\n",
+ "\n"
]
}
],
diff --git a/examples/jupyter/prompt-management/how-to-prompt-management.ipynb b/examples/jupyter/prompt-management/how-to-prompt-management.ipynb
deleted file mode 100644
index abc071f818..0000000000
--- a/examples/jupyter/prompt-management/how-to-prompt-management.ipynb
+++ /dev/null
@@ -1,826 +0,0 @@
-{
- "cells": [
- {
- "cell_type": "markdown",
- "id": "fe036c83",
- "metadata": {},
- "source": [
- "# A How-To Guide for Prompt Management with the SDK in Agenta\n",
- "\n",
- "This guide covers all prompt management operations using the Agenta SDK: creating variants, committing changes, deploying to environments, and fetching configurations.\n",
- "\n"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "d96a3129",
- "metadata": {
- "vscode": {
- "languageId": "bat"
- }
- },
- "source": [
- "## Install dependencies"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "35bbfebe",
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "Requirement already satisfied: agenta in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (0.51.6)\n",
- "Requirement already satisfied: decorator<6.0.0,>=5.2.1 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (5.2.1)\n",
- "Requirement already satisfied: fastapi<0.117.0,>=0.116.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (0.116.1)\n",
- "Requirement already satisfied: h11>=0.16.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (0.16.0)\n",
- "Requirement already satisfied: httpx>=0.28.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (0.28.1)\n",
- "Requirement already satisfied: huggingface-hub<0.31.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (0.30.2)\n",
- "Requirement already satisfied: importlib-metadata<9.0,>=8.0.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (8.7.0)\n",
- "Requirement already satisfied: jinja2<4.0.0,>=3.1.6 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (3.1.6)\n",
- "Requirement already satisfied: litellm==1.76.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (1.76.0)\n",
- "Requirement already satisfied: openai>=1.100.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (1.106.1)\n",
- "Requirement already satisfied: opentelemetry-api<2.0.0,>=1.27.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (1.36.0)\n",
- "Requirement already satisfied: opentelemetry-exporter-otlp-proto-http<2.0.0,>=1.27.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (1.36.0)\n",
- "Requirement already satisfied: opentelemetry-instrumentation>=0.56b0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (0.57b0)\n",
- "Requirement already satisfied: opentelemetry-sdk<2.0.0,>=1.27.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (1.36.0)\n",
- "Requirement already satisfied: pydantic>=2 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (2.11.7)\n",
- "Requirement already satisfied: python-dotenv<2.0.0,>=1.0.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (1.1.1)\n",
- "Requirement already satisfied: pyyaml<7.0.0,>=6.0.2 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (6.0.2)\n",
- "Requirement already satisfied: starlette<0.48.0,>=0.47.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (0.47.3)\n",
- "Requirement already satisfied: structlog<26.0.0,>=25.2.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (25.4.0)\n",
- "Requirement already satisfied: toml<0.11.0,>=0.10.2 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (0.10.2)\n",
- "Requirement already satisfied: aiohttp>=3.10 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from litellm==1.76.0->agenta) (3.12.15)\n",
- "Requirement already satisfied: click in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from litellm==1.76.0->agenta) (8.2.1)\n",
- "Requirement already satisfied: jsonschema<5.0.0,>=4.22.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from litellm==1.76.0->agenta) (4.25.1)\n",
- "Requirement already satisfied: tiktoken>=0.7.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from litellm==1.76.0->agenta) (0.11.0)\n",
- "Requirement already satisfied: tokenizers in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from litellm==1.76.0->agenta) (0.22.0)\n",
- "Requirement already satisfied: typing-extensions>=4.8.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from fastapi<0.117.0,>=0.116.0->agenta) (4.15.0)\n",
- "Requirement already satisfied: filelock in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from huggingface-hub<0.31.0->agenta) (3.19.1)\n",
- "Requirement already satisfied: fsspec>=2023.5.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from huggingface-hub<0.31.0->agenta) (2025.7.0)\n",
- "Requirement already satisfied: packaging>=20.9 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from huggingface-hub<0.31.0->agenta) (25.0)\n",
- "Requirement already satisfied: requests in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from huggingface-hub<0.31.0->agenta) (2.32.5)\n",
- "Requirement already satisfied: tqdm>=4.42.1 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from huggingface-hub<0.31.0->agenta) (4.67.1)\n",
- "Requirement already satisfied: zipp>=3.20 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from importlib-metadata<9.0,>=8.0.0->agenta) (3.23.0)\n",
- "Requirement already satisfied: MarkupSafe>=2.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from jinja2<4.0.0,>=3.1.6->agenta) (3.0.2)\n",
- "Requirement already satisfied: attrs>=22.2.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from jsonschema<5.0.0,>=4.22.0->litellm==1.76.0->agenta) (25.3.0)\n",
- "Requirement already satisfied: jsonschema-specifications>=2023.03.6 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from jsonschema<5.0.0,>=4.22.0->litellm==1.76.0->agenta) (2025.4.1)\n",
- "Requirement already satisfied: referencing>=0.28.4 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from jsonschema<5.0.0,>=4.22.0->litellm==1.76.0->agenta) (0.36.2)\n",
- "Requirement already satisfied: rpds-py>=0.7.1 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from jsonschema<5.0.0,>=4.22.0->litellm==1.76.0->agenta) (0.27.1)\n",
- "Requirement already satisfied: googleapis-common-protos~=1.52 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from opentelemetry-exporter-otlp-proto-http<2.0.0,>=1.27.0->agenta) (1.70.0)\n",
- "Requirement already satisfied: opentelemetry-exporter-otlp-proto-common==1.36.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from opentelemetry-exporter-otlp-proto-http<2.0.0,>=1.27.0->agenta) (1.36.0)\n",
- "Requirement already satisfied: opentelemetry-proto==1.36.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from opentelemetry-exporter-otlp-proto-http<2.0.0,>=1.27.0->agenta) (1.36.0)\n",
- "Requirement already satisfied: protobuf<7.0,>=5.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from opentelemetry-proto==1.36.0->opentelemetry-exporter-otlp-proto-http<2.0.0,>=1.27.0->agenta) (6.32.0)\n",
- "Requirement already satisfied: opentelemetry-semantic-conventions==0.57b0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from opentelemetry-sdk<2.0.0,>=1.27.0->agenta) (0.57b0)\n",
- "Requirement already satisfied: annotated-types>=0.6.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from pydantic>=2->agenta) (0.7.0)\n",
- "Requirement already satisfied: pydantic-core==2.33.2 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from pydantic>=2->agenta) (2.33.2)\n",
- "Requirement already satisfied: typing-inspection>=0.4.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from pydantic>=2->agenta) (0.4.1)\n",
- "Requirement already satisfied: charset_normalizer<4,>=2 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from requests->huggingface-hub<0.31.0->agenta) (3.4.3)\n",
- "Requirement already satisfied: idna<4,>=2.5 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from requests->huggingface-hub<0.31.0->agenta) (3.10)\n",
- "Requirement already satisfied: urllib3<3,>=1.21.1 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from requests->huggingface-hub<0.31.0->agenta) (2.5.0)\n",
- "Requirement already satisfied: certifi>=2017.4.17 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from requests->huggingface-hub<0.31.0->agenta) (2025.8.3)\n",
- "Requirement already satisfied: anyio<5,>=3.6.2 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from starlette<0.48.0,>=0.47.0->agenta) (4.10.0)\n",
- "Requirement already satisfied: sniffio>=1.1 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from anyio<5,>=3.6.2->starlette<0.48.0,>=0.47.0->agenta) (1.3.1)\n",
- "Requirement already satisfied: aiohappyeyeballs>=2.5.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from aiohttp>=3.10->litellm==1.76.0->agenta) (2.6.1)\n",
- "Requirement already satisfied: aiosignal>=1.4.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from aiohttp>=3.10->litellm==1.76.0->agenta) (1.4.0)\n",
- "Requirement already satisfied: frozenlist>=1.1.1 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from aiohttp>=3.10->litellm==1.76.0->agenta) (1.7.0)\n",
- "Requirement already satisfied: multidict<7.0,>=4.5 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from aiohttp>=3.10->litellm==1.76.0->agenta) (6.6.4)\n",
- "Requirement already satisfied: propcache>=0.2.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from aiohttp>=3.10->litellm==1.76.0->agenta) (0.3.2)\n",
- "Requirement already satisfied: yarl<2.0,>=1.17.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from aiohttp>=3.10->litellm==1.76.0->agenta) (1.20.1)\n",
- "Requirement already satisfied: httpcore==1.* in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from httpx>=0.28.0->agenta) (1.0.9)\n",
- "Requirement already satisfied: distro<2,>=1.7.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from openai>=1.100.0->agenta) (1.9.0)\n",
- "Requirement already satisfied: jiter<1,>=0.4.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from openai>=1.100.0->agenta) (0.10.0)\n",
- "Requirement already satisfied: wrapt<2.0.0,>=1.0.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from opentelemetry-instrumentation>=0.56b0->agenta) (1.17.3)\n",
- "Requirement already satisfied: regex>=2022.1.18 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from tiktoken>=0.7.0->litellm==1.76.0->agenta) (2025.9.1)\n",
- "Note: you may need to restart the kernel to use updated packages.\n"
- ]
- }
- ],
- "source": [
- "pip install -U agenta"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "09579f66",
- "metadata": {},
- "source": [
- "## Setup"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "02fa644e",
- "metadata": {},
- "outputs": [],
- "source": "import os\n\nos.environ[\"AGENTA_API_KEY\"] = \"\"\nos.environ[\"AGENTA_HOST\"] = \"https://cloud.agenta.ai/\""
- },
- {
- "cell_type": "code",
- "execution_count": 16,
- "id": "e43db2b5",
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "2025-09-11T13:55:47.580Z \u001b[38;5;70m[INFO.]\u001b[0m Agenta - SDK version: 0.51.6 \u001b[38;5;245m[agenta.sdk.agenta_init]\u001b[0m \n",
- "2025-09-11T13:55:47.582Z \u001b[38;5;70m[INFO.]\u001b[0m Agenta - Host: http://144.76.237.122 \u001b[38;5;245m[agenta.sdk.agenta_init]\u001b[0m \n",
- "2025-09-11T13:55:47.583Z \u001b[38;5;70m[INFO.]\u001b[0m Agenta - OLTP URL: https://http://144.76.237.122/api/otlp/v1/traces \u001b[38;5;245m[agenta.sdk.tracing.tracing]\u001b[0m \n"
- ]
- },
- {
- "name": "stderr",
- "output_type": "stream",
- "text": [
- "Overriding of current TracerProvider is not allowed\n"
- ]
- }
- ],
- "source": [
- "import agenta as ag\n",
- "from getpass import getpass\n",
- "\n",
- "# Get API key from environment or prompt user\n",
- "api_key = os.getenv(\"AGENTA_API_KEY\")\n",
- "if not api_key:\n",
- " os.environ[\"AGENTA_API_KEY\"] = getpass(\"Enter your Agenta API key: \")\n",
- "\n",
- "\n",
- "# Initialize the Agenta client\n",
- "ag.init()\n"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "564c90d6",
- "metadata": {},
- "source": [
- "## Create new prompts"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 36,
- "id": "05786812",
- "metadata": {},
- "outputs": [],
- "source": [
- "\n",
- "from agenta.sdk.types import PromptTemplate, Message, ModelConfig\n",
- "from pydantic import BaseModel\n",
- "\n",
- "# Creates an empty application\n",
- "app = ag.AppManager.create(\n",
- " app_slug=\"my-completion\",\n",
- " template_key=\"SERVICE:completion\", # we define here the app type\n",
- " # template_key=\"SERVICE:chat\" # chat prompts\n",
- " # template_key=\"CUSTOM\" # custom configuration (schema-less, however unless you provide a URI, you can only use the registry but not the playground)\n",
- ")\n",
- "\n",
- "# Define your configuration model it should alway be of this format for completion and chat apps\n",
- "class Config(BaseModel):\n",
- " prompt: PromptTemplate\n",
- "\n",
- "# Create the initial configuration\n",
- "config = Config(\n",
- " prompt=PromptTemplate(\n",
- " messages=[\n",
- " Message(role=\"system\", content=\"You are an assistant that provides concise answers\"),\n",
- " Message(role=\"user\", content=\"Explain {{topic}} in simple terms\"),\n",
- " ],\n",
- " llm_config=ModelConfig(\n",
- " model=\"gpt-3.5-turbo\",\n",
- " max_tokens=150,\n",
- " temperature=0.7,\n",
- " top_p=1.0,\n",
- " frequency_penalty=0.0,\n",
- " presence_penalty=0.0,\n",
- " ),\n",
- " )\n",
- ")\n",
- "\n",
- "# Create a new variant with the first revision \n",
- "variant = ag.VariantManager.create(\n",
- " parameters=config.model_dump(),\n",
- " app_slug=\"my-completion\",\n",
- " variant_slug=\"default\",\n",
- ")\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 37,
- "id": "25ae5452",
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "{\n",
- " \"app_id\": \"0199396b-aba7-7451-8db4-6fb5e4a1c8d2\",\n",
- " \"app_slug\": \"my-completion\",\n",
- " \"variant_id\": \"0199396b-ac32-7783-a385-09d0a1d084af\",\n",
- " \"variant_slug\": \"default\",\n",
- " \"variant_version\": 1,\n",
- " \"environment_id\": null,\n",
- " \"environment_slug\": null,\n",
- " \"environment_version\": null,\n",
- " \"committed_at\": \"2025-09-11T15:36:21.728674+00:00\",\n",
- " \"committed_by\": \"mahmoud@agenta.ai\",\n",
- " \"committed_by_id\": \"0198eb1b-012a-7891-955e-15ef49443ece\",\n",
- " \"deployed_at\": null,\n",
- " \"deployed_by\": null,\n",
- " \"deployed_by_id\": null,\n",
- " \"params\": {\n",
- " \"prompt\": {\n",
- " \"messages\": [\n",
- " {\n",
- " \"role\": \"system\",\n",
- " \"content\": \"You are an assistant that provides concise answers\",\n",
- " \"name\": null,\n",
- " \"tool_calls\": null,\n",
- " \"tool_call_id\": null\n",
- " },\n",
- " {\n",
- " \"role\": \"user\",\n",
- " \"content\": \"Explain {{topic}} in simple terms\",\n",
- " \"name\": null,\n",
- " \"tool_calls\": null,\n",
- " \"tool_call_id\": null\n",
- " }\n",
- " ],\n",
- " \"system_prompt\": null,\n",
- " \"user_prompt\": null,\n",
- " \"template_format\": \"curly\",\n",
- " \"input_keys\": null,\n",
- " \"llm_config\": {\n",
- " \"model\": \"gpt-3.5-turbo\",\n",
- " \"temperature\": 0.7,\n",
- " \"max_tokens\": 150,\n",
- " \"top_p\": 1.0,\n",
- " \"frequency_penalty\": 0.0,\n",
- " \"presence_penalty\": 0.0,\n",
- " \"response_format\": null,\n",
- " \"stream\": null,\n",
- " \"tools\": null,\n",
- " \"tool_choice\": null\n",
- " }\n",
- " }\n",
- " }\n",
- "}\n"
- ]
- }
- ],
- "source": [
- "\n",
- "print(variant)\n"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "d36a543a",
- "metadata": {},
- "source": [
- "## Creating a new revision"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 38,
- "id": "ff9382d2",
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "{\n",
- " \"app_id\": \"0199396b-aba7-7451-8db4-6fb5e4a1c8d2\",\n",
- " \"app_slug\": \"my-completion\",\n",
- " \"variant_id\": \"0199396b-cbea-7ca2-bc6e-2920316958c3\",\n",
- " \"variant_slug\": \"default\",\n",
- " \"variant_version\": 2,\n",
- " \"environment_id\": null,\n",
- " \"environment_slug\": null,\n",
- " \"environment_version\": null,\n",
- " \"committed_at\": \"2025-09-11T15:36:21.728674+00:00\",\n",
- " \"committed_by\": \"mahmoud@agenta.ai\",\n",
- " \"committed_by_id\": \"0198eb1b-012a-7891-955e-15ef49443ece\",\n",
- " \"deployed_at\": null,\n",
- " \"deployed_by\": null,\n",
- " \"deployed_by_id\": null,\n",
- " \"params\": {\n",
- " \"prompt\": {\n",
- " \"messages\": [\n",
- " {\n",
- " \"role\": \"system\",\n",
- " \"content\": \"You are an assistant that provides VERY concise answers\",\n",
- " \"name\": null,\n",
- " \"tool_calls\": null,\n",
- " \"tool_call_id\": null\n",
- " },\n",
- " {\n",
- " \"role\": \"user\",\n",
- " \"content\": \"Explain {{topic}} in simple terms\",\n",
- " \"name\": null,\n",
- " \"tool_calls\": null,\n",
- " \"tool_call_id\": null\n",
- " }\n",
- " ],\n",
- " \"system_prompt\": null,\n",
- " \"user_prompt\": null,\n",
- " \"template_format\": \"curly\",\n",
- " \"input_keys\": null,\n",
- " \"llm_config\": {\n",
- " \"model\": \"anthropic/claude-3-5-sonnet-20240620\",\n",
- " \"temperature\": 0.7,\n",
- " \"max_tokens\": 150,\n",
- " \"top_p\": 1.0,\n",
- " \"frequency_penalty\": 0.0,\n",
- " \"presence_penalty\": 0.0,\n",
- " \"response_format\": null,\n",
- " \"stream\": null,\n",
- " \"tools\": null,\n",
- " \"tool_choice\": null\n",
- " }\n",
- " }\n",
- " }\n",
- "}\n"
- ]
- }
- ],
- "source": [
- "config2=Config(\n",
- " prompt=PromptTemplate(\n",
- " messages=[\n",
- " Message(role=\"system\", content=\"You are an assistant that provides VERY concise answers\"),\n",
- " Message(role=\"user\", content=\"Explain {{topic}} in simple terms\"),\n",
- " ],\n",
- " llm_config=ModelConfig(\n",
- " model=\"anthropic/claude-3-5-sonnet-20240620\",\n",
- " max_tokens=150,\n",
- " temperature=0.7,\n",
- " top_p=1.0,\n",
- " frequency_penalty=0.0,\n",
- " presence_penalty=0.0,\n",
- " ),\n",
- " )\n",
- ")\n",
- "\n",
- "\n",
- "# Commit the new version\n",
- "variant = ag.VariantManager.commit(\n",
- " parameters=config2.model_dump(),\n",
- " app_slug=\"my-completion\",\n",
- " variant_slug=\"default\",\n",
- ")\n",
- "\n",
- "print(variant)"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "79553cec",
- "metadata": {},
- "source": [
- "## Deploying to an environment"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 43,
- "id": "5fbd3d75",
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "Deployed to staging\n"
- ]
- }
- ],
- "source": [
- "deployment = ag.DeploymentManager.deploy(\n",
- " app_slug=\"my-completion\",\n",
- " variant_slug=\"default\",\n",
- " variant_version=None, # Deploys latest version if not specified\n",
- " environment_slug=\"staging\" # Options: development, staging, production\n",
- ")\n",
- "\n",
- "print(f\"Deployed to {deployment.environment_slug}\")\n",
- "\n",
- "\n"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "7fbae284",
- "metadata": {},
- "source": [
- "## Fetching configurations"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 45,
- "id": "6ab6bab6",
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "Fetched configuration for variant default version 2:\n",
- "{'prompt': {'messages': [{'role': 'system', 'content': 'You are an assistant that provides VERY concise answers', 'name': None, 'tool_calls': None, 'tool_call_id': None}, {'role': 'user', 'content': 'Explain {{topic}} in simple terms', 'name': None, 'tool_calls': None, 'tool_call_id': None}], 'system_prompt': None, 'user_prompt': None, 'template_format': 'curly', 'input_keys': None, 'llm_config': {'model': 'anthropic/claude-3-5-sonnet-20240620', 'temperature': 0.7, 'max_tokens': 150, 'top_p': 1.0, 'frequency_penalty': 0.0, 'presence_penalty': 0.0, 'response_format': None, 'stream': None, 'tools': None, 'tool_choice': None}}}\n",
- "Fetched configuration for staging environment version 1:\n",
- "{'prompt': {'messages': [{'role': 'system', 'content': 'You are an assistant that provides VERY concise answers', 'name': None, 'tool_calls': None, 'tool_call_id': None}, {'role': 'user', 'content': 'Explain {{topic}} in simple terms', 'name': None, 'tool_calls': None, 'tool_call_id': None}], 'system_prompt': None, 'user_prompt': None, 'template_format': 'curly', 'input_keys': None, 'llm_config': {'model': 'anthropic/claude-3-5-sonnet-20240620', 'temperature': 0.7, 'max_tokens': 150, 'top_p': 1.0, 'frequency_penalty': 0.0, 'presence_penalty': 0.0, 'response_format': None, 'stream': None, 'tools': None, 'tool_choice': None}}}\n"
- ]
- }
- ],
- "source": [
- "# Fetching by variant reference\n",
- "config = ag.ConfigManager.get_from_registry(\n",
- " app_slug=\"my-completion\",\n",
- " variant_slug=\"default\",\n",
- " variant_version=2 # Optional: fetches latest if not provided\n",
- ")\n",
- "print(\"Fetched configuration for variant default version 2:\")\n",
- "print(config)\n",
- "\n",
- "# Fetching by environment reference (from staging)\n",
- "config = ag.ConfigManager.get_from_registry(\n",
- " app_slug=\"my-completion\",\n",
- " environment_slug=\"staging\",\n",
- " environment_version=1 # Optional: If not provided, fetches the latest version\n",
- ")\n",
- "\n",
- "print(f\"Fetched configuration for staging environment version 1:\")\n",
- "print(config)\n",
- "\n",
- "\n"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "4e298bae",
- "metadata": {},
- "source": [
- "## Listing all variants"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 46,
- "id": "792e7804",
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "[{\n",
- " \"app_id\": \"0199396b-aba7-7451-8db4-6fb5e4a1c8d2\",\n",
- " \"app_slug\": \"my-completion\",\n",
- " \"variant_id\": \"0199396b-cbea-7ca2-bc6e-2920316958c3\",\n",
- " \"variant_slug\": \"default\",\n",
- " \"variant_version\": 2,\n",
- " \"environment_id\": null,\n",
- " \"environment_slug\": null,\n",
- " \"environment_version\": null,\n",
- " \"committed_at\": \"2025-09-11T15:36:29.930593+00:00\",\n",
- " \"committed_by\": \"mahmoud@agenta.ai\",\n",
- " \"committed_by_id\": \"0198eb1b-012a-7891-955e-15ef49443ece\",\n",
- " \"deployed_at\": null,\n",
- " \"deployed_by\": null,\n",
- " \"deployed_by_id\": null,\n",
- " \"params\": {\n",
- " \"prompt\": {\n",
- " \"messages\": [\n",
- " {\n",
- " \"role\": \"system\",\n",
- " \"content\": \"You are an assistant that provides VERY concise answers\",\n",
- " \"name\": null,\n",
- " \"tool_calls\": null,\n",
- " \"tool_call_id\": null\n",
- " },\n",
- " {\n",
- " \"role\": \"user\",\n",
- " \"content\": \"Explain {{topic}} in simple terms\",\n",
- " \"name\": null,\n",
- " \"tool_calls\": null,\n",
- " \"tool_call_id\": null\n",
- " }\n",
- " ],\n",
- " \"system_prompt\": null,\n",
- " \"user_prompt\": null,\n",
- " \"template_format\": \"curly\",\n",
- " \"input_keys\": null,\n",
- " \"llm_config\": {\n",
- " \"model\": \"anthropic/claude-3-5-sonnet-20240620\",\n",
- " \"temperature\": 0.7,\n",
- " \"max_tokens\": 150,\n",
- " \"top_p\": 1.0,\n",
- " \"frequency_penalty\": 0.0,\n",
- " \"presence_penalty\": 0.0,\n",
- " \"response_format\": null,\n",
- " \"stream\": null,\n",
- " \"tools\": null,\n",
- " \"tool_choice\": null\n",
- " }\n",
- " }\n",
- " }\n",
- "}]\n"
- ]
- }
- ],
- "source": [
- "variants = ag.VariantManager.list(\n",
- " app_slug=\"my-completion\"\n",
- ")\n",
- "\n",
- "print(variants)\n",
- "\n"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "23503c53",
- "metadata": {},
- "source": [
- "## Fetching variant history"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 47,
- "id": "9fcf9a8b",
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "[{\n",
- " \"app_id\": \"0199396b-aba7-7451-8db4-6fb5e4a1c8d2\",\n",
- " \"app_slug\": \"my-completion\",\n",
- " \"variant_id\": \"0199396b-abe0-74a1-9e4c-80779f36c9e5\",\n",
- " \"variant_slug\": \"default\",\n",
- " \"variant_version\": 2,\n",
- " \"environment_id\": null,\n",
- " \"environment_slug\": null,\n",
- " \"environment_version\": null,\n",
- " \"committed_at\": \"2025-09-11T15:36:21.733212+00:00\",\n",
- " \"committed_by\": \"mahmoud@agenta.ai\",\n",
- " \"committed_by_id\": \"0198eb1b-012a-7891-955e-15ef49443ece\",\n",
- " \"deployed_at\": null,\n",
- " \"deployed_by\": null,\n",
- " \"deployed_by_id\": null,\n",
- " \"params\": {}\n",
- "}, {\n",
- " \"app_id\": \"0199396b-aba7-7451-8db4-6fb5e4a1c8d2\",\n",
- " \"app_slug\": \"my-completion\",\n",
- " \"variant_id\": \"0199396b-abe0-74a1-9e4c-80779f36c9e5\",\n",
- " \"variant_slug\": \"default\",\n",
- " \"variant_version\": 2,\n",
- " \"environment_id\": null,\n",
- " \"environment_slug\": null,\n",
- " \"environment_version\": null,\n",
- " \"committed_at\": \"2025-09-11T15:36:21.810617+00:00\",\n",
- " \"committed_by\": \"mahmoud@agenta.ai\",\n",
- " \"committed_by_id\": \"0198eb1b-012a-7891-955e-15ef49443ece\",\n",
- " \"deployed_at\": null,\n",
- " \"deployed_by\": null,\n",
- " \"deployed_by_id\": null,\n",
- " \"params\": {\n",
- " \"prompt\": {\n",
- " \"messages\": [\n",
- " {\n",
- " \"role\": \"system\",\n",
- " \"content\": \"You are an assistant that provides concise answers\",\n",
- " \"name\": null,\n",
- " \"tool_calls\": null,\n",
- " \"tool_call_id\": null\n",
- " },\n",
- " {\n",
- " \"role\": \"user\",\n",
- " \"content\": \"Explain {{topic}} in simple terms\",\n",
- " \"name\": null,\n",
- " \"tool_calls\": null,\n",
- " \"tool_call_id\": null\n",
- " }\n",
- " ],\n",
- " \"system_prompt\": null,\n",
- " \"user_prompt\": null,\n",
- " \"template_format\": \"curly\",\n",
- " \"input_keys\": null,\n",
- " \"llm_config\": {\n",
- " \"model\": \"gpt-3.5-turbo\",\n",
- " \"temperature\": 0.7,\n",
- " \"max_tokens\": 150,\n",
- " \"top_p\": 1.0,\n",
- " \"frequency_penalty\": 0.0,\n",
- " \"presence_penalty\": 0.0,\n",
- " \"response_format\": null,\n",
- " \"stream\": null,\n",
- " \"tools\": null,\n",
- " \"tool_choice\": null\n",
- " }\n",
- " }\n",
- " }\n",
- "}, {\n",
- " \"app_id\": \"0199396b-aba7-7451-8db4-6fb5e4a1c8d2\",\n",
- " \"app_slug\": \"my-completion\",\n",
- " \"variant_id\": \"0199396b-abe0-74a1-9e4c-80779f36c9e5\",\n",
- " \"variant_slug\": \"default\",\n",
- " \"variant_version\": 2,\n",
- " \"environment_id\": null,\n",
- " \"environment_slug\": null,\n",
- " \"environment_version\": null,\n",
- " \"committed_at\": \"2025-09-11T15:36:29.930593+00:00\",\n",
- " \"committed_by\": \"mahmoud@agenta.ai\",\n",
- " \"committed_by_id\": \"0198eb1b-012a-7891-955e-15ef49443ece\",\n",
- " \"deployed_at\": null,\n",
- " \"deployed_by\": null,\n",
- " \"deployed_by_id\": null,\n",
- " \"params\": {\n",
- " \"prompt\": {\n",
- " \"messages\": [\n",
- " {\n",
- " \"role\": \"system\",\n",
- " \"content\": \"You are an assistant that provides VERY concise answers\",\n",
- " \"name\": null,\n",
- " \"tool_calls\": null,\n",
- " \"tool_call_id\": null\n",
- " },\n",
- " {\n",
- " \"role\": \"user\",\n",
- " \"content\": \"Explain {{topic}} in simple terms\",\n",
- " \"name\": null,\n",
- " \"tool_calls\": null,\n",
- " \"tool_call_id\": null\n",
- " }\n",
- " ],\n",
- " \"system_prompt\": null,\n",
- " \"user_prompt\": null,\n",
- " \"template_format\": \"curly\",\n",
- " \"input_keys\": null,\n",
- " \"llm_config\": {\n",
- " \"model\": \"anthropic/claude-3-5-sonnet-20240620\",\n",
- " \"temperature\": 0.7,\n",
- " \"max_tokens\": 150,\n",
- " \"top_p\": 1.0,\n",
- " \"frequency_penalty\": 0.0,\n",
- " \"presence_penalty\": 0.0,\n",
- " \"response_format\": null,\n",
- " \"stream\": null,\n",
- " \"tools\": null,\n",
- " \"tool_choice\": null\n",
- " }\n",
- " }\n",
- " }\n",
- "}]\n"
- ]
- }
- ],
- "source": [
- "versions = ag.VariantManager.history(\n",
- " variant_slug=\"default\",\n",
- " app_slug=\"my-completion\"\n",
- ")\n",
- "\n",
- "print(versions)\n",
- "\n"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "e8f52205",
- "metadata": {},
- "source": [
- "## Deleting a variant"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 50,
- "id": "9d8d2e39",
- "metadata": {},
- "outputs": [
- {
- "data": {
- "text/plain": [
- "204"
- ]
- },
- "execution_count": 50,
- "metadata": {},
- "output_type": "execute_result"
- }
- ],
- "source": [
- "variant = ag.VariantManager.create(\n",
- " parameters=config,\n",
- " app_slug=\"my-completion\",\n",
- " variant_slug=\"to-remove\",\n",
- ")\n",
- "\n",
- "ag.VariantManager.delete(\n",
- " app_slug=\"my-completion\",\n",
- " variant_slug=\"to-remove\"\n",
- ")\n",
- "\n"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "efad781d",
- "metadata": {},
- "source": [
- "## Creating an app with a custom configuration (schema-less)"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 52,
- "id": "dfb8437e",
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "{\n",
- " \"app_id\": \"01993970-5fa6-7561-bd77-edb117ee6788\",\n",
- " \"app_slug\": \"my-rag-config\",\n",
- " \"variant_id\": \"01993970-6008-7ca3-abb8-d5bfdf798fc4\",\n",
- " \"variant_slug\": \"default\",\n",
- " \"variant_version\": 1,\n",
- " \"environment_id\": null,\n",
- " \"environment_slug\": null,\n",
- " \"environment_version\": null,\n",
- " \"committed_at\": \"2025-09-11T15:41:29.923484+00:00\",\n",
- " \"committed_by\": \"mahmoud@agenta.ai\",\n",
- " \"committed_by_id\": \"0198eb1b-012a-7891-955e-15ef49443ece\",\n",
- " \"deployed_at\": null,\n",
- " \"deployed_by\": null,\n",
- " \"deployed_by_id\": null,\n",
- " \"params\": {\n",
- " \"rag-prompt\": \"You are a helpful assistant that can answer questions and help with tasks.\",\n",
- " \"rag-chunk_size\": 100,\n",
- " \"rag-chunk_overlap\": 20\n",
- " }\n",
- "}\n"
- ]
- }
- ],
- "source": [
- "app = ag.AppManager.create(\n",
- " app_slug=\"my-rag-config\",\n",
- " template_key=\"CUSTOM\", # we define here the app type\n",
- ")\n",
- "\n",
- "rag_config = {\n",
- " \"rag-prompt\": \"You are a helpful assistant that can answer questions and help with tasks.\",\n",
- " \"rag-chunk_size\": 100,\n",
- " \"rag-chunk_overlap\": 20,\n",
- "}\n",
- "variant = ag.VariantManager.create(\n",
- " parameters=rag_config,\n",
- " app_slug=\"my-rag-config\",\n",
- " variant_slug=\"default\",\n",
- ")\n",
- "\n",
- "print(variant)\n"
- ]
- }
- ],
- "metadata": {
- "kernelspec": {
- "display_name": ".venv (3.12.3)",
- "language": "python",
- "name": "python3"
- },
- "language_info": {
- "codemirror_mode": {
- "name": "ipython",
- "version": 3
- },
- "file_extension": ".py",
- "mimetype": "text/x-python",
- "name": "python",
- "nbconvert_exporter": "python",
- "pygments_lexer": "ipython3",
- "version": "3.12.3"
- }
- },
- "nbformat": 4,
- "nbformat_minor": 5
-}
\ No newline at end of file
diff --git a/examples/jupyter/prompt-management/manage-prompts-with-sdk-tutorial.ipynb b/examples/jupyter/prompt-management/manage-prompts-with-sdk-tutorial.ipynb
deleted file mode 100644
index 3eaa4c44e7..0000000000
--- a/examples/jupyter/prompt-management/manage-prompts-with-sdk-tutorial.ipynb
+++ /dev/null
@@ -1,634 +0,0 @@
-{
- "cells": [
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "# Manage Prompts with SDK - Tutorial\n",
- "\n",
- "You can manage prompts easily through Agenta's web UI. But sometimes you might want to do things programmatically instead of using the interface.\n",
- "\n",
- "In this tutorial, we'll use the Agenta SDK to create a new prompt, commit changes, deploy them to production, then fetch their configuration and use it to call the LLM.\n",
- "\n",
- "## Tutorial Overview\n",
- "\n",
- "Before we begin, let's quickly review how Agenta versions prompts:\n",
- "\n",
- "Agenta follows a structure similar to **git** for prompt versioning. Instead of having one commit history, it uses **multiple branches (called variants)** where changes can be committed, and **environments** where these changes can be deployed (and used in your application).\n",
- "\n",
- "The workflow for deploying a change to production that we'll follow in this tutorial is:\n",
- "\n",
- "1. Create a new variant\n",
- "2. Commit a change to that variant\n",
- "3. Deploy that commit (variant/version) to the production environment\n",
- "4. Commit a new change to that variant\n",
- "5. Fetch the config from that environment\n"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Install dependencies\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 1,
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "Requirement already satisfied: agenta in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (0.51.6)\n",
- "Requirement already satisfied: openai in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (1.106.1)\n",
- "Collecting openai\n",
- " Downloading openai-1.107.1-py3-none-any.whl.metadata (29 kB)\n",
- "Requirement already satisfied: decorator<6.0.0,>=5.2.1 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (5.2.1)\n",
- "Requirement already satisfied: fastapi<0.117.0,>=0.116.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (0.116.1)\n",
- "Requirement already satisfied: h11>=0.16.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (0.16.0)\n",
- "Requirement already satisfied: httpx>=0.28.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (0.28.1)\n",
- "Requirement already satisfied: huggingface-hub<0.31.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (0.30.2)\n",
- "Requirement already satisfied: importlib-metadata<9.0,>=8.0.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (8.7.0)\n",
- "Requirement already satisfied: jinja2<4.0.0,>=3.1.6 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (3.1.6)\n",
- "Requirement already satisfied: litellm==1.76.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (1.76.0)\n",
- "Requirement already satisfied: opentelemetry-api<2.0.0,>=1.27.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (1.36.0)\n",
- "Requirement already satisfied: opentelemetry-exporter-otlp-proto-http<2.0.0,>=1.27.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (1.36.0)\n",
- "Requirement already satisfied: opentelemetry-instrumentation>=0.56b0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (0.57b0)\n",
- "Requirement already satisfied: opentelemetry-sdk<2.0.0,>=1.27.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (1.36.0)\n",
- "Requirement already satisfied: pydantic>=2 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (2.11.7)\n",
- "Requirement already satisfied: python-dotenv<2.0.0,>=1.0.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (1.1.1)\n",
- "Requirement already satisfied: pyyaml<7.0.0,>=6.0.2 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (6.0.2)\n",
- "Requirement already satisfied: starlette<0.48.0,>=0.47.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (0.47.3)\n",
- "Requirement already satisfied: structlog<26.0.0,>=25.2.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (25.4.0)\n",
- "Requirement already satisfied: toml<0.11.0,>=0.10.2 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (0.10.2)\n",
- "Requirement already satisfied: aiohttp>=3.10 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from litellm==1.76.0->agenta) (3.12.15)\n",
- "Requirement already satisfied: click in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from litellm==1.76.0->agenta) (8.2.1)\n",
- "Requirement already satisfied: jsonschema<5.0.0,>=4.22.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from litellm==1.76.0->agenta) (4.25.1)\n",
- "Requirement already satisfied: tiktoken>=0.7.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from litellm==1.76.0->agenta) (0.11.0)\n",
- "Requirement already satisfied: tokenizers in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from litellm==1.76.0->agenta) (0.22.0)\n",
- "Requirement already satisfied: typing-extensions>=4.8.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from fastapi<0.117.0,>=0.116.0->agenta) (4.15.0)\n",
- "Requirement already satisfied: filelock in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from huggingface-hub<0.31.0->agenta) (3.19.1)\n",
- "Requirement already satisfied: fsspec>=2023.5.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from huggingface-hub<0.31.0->agenta) (2025.7.0)\n",
- "Requirement already satisfied: packaging>=20.9 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from huggingface-hub<0.31.0->agenta) (25.0)\n",
- "Requirement already satisfied: requests in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from huggingface-hub<0.31.0->agenta) (2.32.5)\n",
- "Requirement already satisfied: tqdm>=4.42.1 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from huggingface-hub<0.31.0->agenta) (4.67.1)\n",
- "Requirement already satisfied: zipp>=3.20 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from importlib-metadata<9.0,>=8.0.0->agenta) (3.23.0)\n",
- "Requirement already satisfied: MarkupSafe>=2.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from jinja2<4.0.0,>=3.1.6->agenta) (3.0.2)\n",
- "Requirement already satisfied: attrs>=22.2.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from jsonschema<5.0.0,>=4.22.0->litellm==1.76.0->agenta) (25.3.0)\n",
- "Requirement already satisfied: jsonschema-specifications>=2023.03.6 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from jsonschema<5.0.0,>=4.22.0->litellm==1.76.0->agenta) (2025.4.1)\n",
- "Requirement already satisfied: referencing>=0.28.4 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from jsonschema<5.0.0,>=4.22.0->litellm==1.76.0->agenta) (0.36.2)\n",
- "Requirement already satisfied: rpds-py>=0.7.1 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from jsonschema<5.0.0,>=4.22.0->litellm==1.76.0->agenta) (0.27.1)\n",
- "Requirement already satisfied: googleapis-common-protos~=1.52 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from opentelemetry-exporter-otlp-proto-http<2.0.0,>=1.27.0->agenta) (1.70.0)\n",
- "Requirement already satisfied: opentelemetry-exporter-otlp-proto-common==1.36.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from opentelemetry-exporter-otlp-proto-http<2.0.0,>=1.27.0->agenta) (1.36.0)\n",
- "Requirement already satisfied: opentelemetry-proto==1.36.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from opentelemetry-exporter-otlp-proto-http<2.0.0,>=1.27.0->agenta) (1.36.0)\n",
- "Requirement already satisfied: protobuf<7.0,>=5.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from opentelemetry-proto==1.36.0->opentelemetry-exporter-otlp-proto-http<2.0.0,>=1.27.0->agenta) (6.32.0)\n",
- "Requirement already satisfied: opentelemetry-semantic-conventions==0.57b0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from opentelemetry-sdk<2.0.0,>=1.27.0->agenta) (0.57b0)\n",
- "Requirement already satisfied: annotated-types>=0.6.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from pydantic>=2->agenta) (0.7.0)\n",
- "Requirement already satisfied: pydantic-core==2.33.2 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from pydantic>=2->agenta) (2.33.2)\n",
- "Requirement already satisfied: typing-inspection>=0.4.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from pydantic>=2->agenta) (0.4.1)\n",
- "Requirement already satisfied: charset_normalizer<4,>=2 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from requests->huggingface-hub<0.31.0->agenta) (3.4.3)\n",
- "Requirement already satisfied: idna<4,>=2.5 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from requests->huggingface-hub<0.31.0->agenta) (3.10)\n",
- "Requirement already satisfied: urllib3<3,>=1.21.1 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from requests->huggingface-hub<0.31.0->agenta) (2.5.0)\n",
- "Requirement already satisfied: certifi>=2017.4.17 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from requests->huggingface-hub<0.31.0->agenta) (2025.8.3)\n",
- "Requirement already satisfied: anyio<5,>=3.6.2 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from starlette<0.48.0,>=0.47.0->agenta) (4.10.0)\n",
- "Requirement already satisfied: sniffio>=1.1 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from anyio<5,>=3.6.2->starlette<0.48.0,>=0.47.0->agenta) (1.3.1)\n",
- "Requirement already satisfied: distro<2,>=1.7.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from openai) (1.9.0)\n",
- "Requirement already satisfied: jiter<1,>=0.4.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from openai) (0.10.0)\n",
- "Requirement already satisfied: httpcore==1.* in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from httpx>=0.28.0->agenta) (1.0.9)\n",
- "Requirement already satisfied: aiohappyeyeballs>=2.5.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from aiohttp>=3.10->litellm==1.76.0->agenta) (2.6.1)\n",
- "Requirement already satisfied: aiosignal>=1.4.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from aiohttp>=3.10->litellm==1.76.0->agenta) (1.4.0)\n",
- "Requirement already satisfied: frozenlist>=1.1.1 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from aiohttp>=3.10->litellm==1.76.0->agenta) (1.7.0)\n",
- "Requirement already satisfied: multidict<7.0,>=4.5 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from aiohttp>=3.10->litellm==1.76.0->agenta) (6.6.4)\n",
- "Requirement already satisfied: propcache>=0.2.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from aiohttp>=3.10->litellm==1.76.0->agenta) (0.3.2)\n",
- "Requirement already satisfied: yarl<2.0,>=1.17.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from aiohttp>=3.10->litellm==1.76.0->agenta) (1.20.1)\n",
- "Requirement already satisfied: wrapt<2.0.0,>=1.0.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from opentelemetry-instrumentation>=0.56b0->agenta) (1.17.3)\n",
- "Requirement already satisfied: regex>=2022.1.18 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from tiktoken>=0.7.0->litellm==1.76.0->agenta) (2025.9.1)\n",
- "Downloading openai-1.107.1-py3-none-any.whl (945 kB)\n",
- "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m945.2/945.2 kB\u001b[0m \u001b[31m22.6 MB/s\u001b[0m \u001b[33m0:00:00\u001b[0m\n",
- "\u001b[?25hInstalling collected packages: openai\n",
- " Attempting uninstall: openai\n",
- " Found existing installation: openai 1.106.1\n",
- " Uninstalling openai-1.106.1:\n",
- " Successfully uninstalled openai-1.106.1\n",
- "Successfully installed openai-1.107.1\n",
- "Note: you may need to restart the kernel to use updated packages.\n"
- ]
- }
- ],
- "source": [
- "pip install -U agenta openai\n"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Setup\n",
- "\n",
- "Before using the SDK, we need to initialize it using the `ag.init()` method, which takes the host (default `cloud.agenta.ai`) and the API key (not required for community editions):\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": "import os\nos.environ[\"AGENTA_HOST\"] = \"https://cloud.agenta.ai/\" # Default value, no need to set explicitly unless self-hosted\nos.environ[\"AGENTA_API_KEY\"] = \"\"\nos.environ[\"OPENAI_API_KEY\"] = \"\""
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "2025-09-11T15:56:10.922Z \u001b[38;5;70m[INFO.]\u001b[0m Agenta - SDK version: 0.51.6 \u001b[38;5;245m[agenta.sdk.agenta_init]\u001b[0m \n",
- "2025-09-11T15:56:10.923Z \u001b[38;5;70m[INFO.]\u001b[0m Agenta - Host: http://144.76.237.122 \u001b[38;5;245m[agenta.sdk.agenta_init]\u001b[0m \n",
- "2025-09-11T15:56:10.923Z \u001b[38;5;70m[INFO.]\u001b[0m Agenta - OLTP URL: http://144.76.237.122/api/otlp/v1/traces \u001b[38;5;245m[agenta.sdk.tracing.tracing]\u001b[0m \n"
- ]
- },
- {
- "name": "stderr",
- "output_type": "stream",
- "text": [
- "Overriding of current TracerProvider is not allowed\n"
- ]
- }
- ],
- "source": [
- "import os\n",
- "import agenta as ag\n",
- "from getpass import getpass\n",
- "\n",
- "# Initialize the SDK with your API key\n",
- "api_key = os.getenv(\"AGENTA_API_KEY\")\n",
- "if not api_key:\n",
- " os.environ[\"AGENTA_API_KEY\"] = getpass(\"Enter your Agenta API key: \")\n",
- "\n",
- "openai_api_key = os.getenv(\"OPENAI_API_KEY\")\n",
- "if not openai_api_key:\n",
- " os.environ[\"OPENAI_API_KEY\"] = getpass(\"Enter your OpenAI API key: \")\n",
- "\n",
- "# Initialize the SDK\n",
- "ag.init()\n"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Creating a new prompt\n",
- "\n",
- "We're going to create a new completion prompt called `topic-explainer`.\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 3,
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "Created application:\n",
- "app_id='0199397d-14f4-7bb2-a9c5-9a12934fb6c3' app_name='topic-explainer'\n"
- ]
- }
- ],
- "source": [
- "# Creates an empty application\n",
- "app = ag.AppManager.create(\n",
- " app_slug=\"topic-explainer\",\n",
- " template_key=\"SERVICE:completion\", # we define here the app type, choices are SERVICE:completion, SERVICE:chat, CUSTOM\n",
- ")\n",
- "\n",
- "print(\"Created application:\")\n",
- "print(app)\n"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- ":::warning\n",
- "The app created until now is empty. You cannot use it from the UI yet. You need to create a variant and commit changes to it to be able to use it (next section).\n",
- ":::\n"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Creating a new variant\n",
- "\n",
- "Variants are similar to branches in **git**. Any change to the prompt must first be committed to a variant. Here, we'll create a new variant and make our first commit to it using the `VariantManager.create` method:\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 4,
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "Created variant:\n",
- "{\n",
- " \"app_id\": \"0199397d-14f4-7bb2-a9c5-9a12934fb6c3\",\n",
- " \"app_slug\": \"topic-explainer\",\n",
- " \"variant_id\": \"0199397d-1d46-74c3-bc3f-e63f2357c7dc\",\n",
- " \"variant_slug\": \"new-variant\",\n",
- " \"variant_version\": 1,\n",
- " \"environment_id\": null,\n",
- " \"environment_slug\": null,\n",
- " \"environment_version\": null,\n",
- " \"committed_at\": \"2025-09-11T15:55:24.842469+00:00\",\n",
- " \"committed_by\": \"mahmoud@agenta.ai\",\n",
- " \"committed_by_id\": \"0198eb1b-012a-7891-955e-15ef49443ece\",\n",
- " \"deployed_at\": null,\n",
- " \"deployed_by\": null,\n",
- " \"deployed_by_id\": null,\n",
- " \"params\": {\n",
- " \"prompt\": {\n",
- " \"messages\": [\n",
- " {\n",
- " \"role\": \"system\",\n",
- " \"content\": \"You are an assistant that provides concise answers\",\n",
- " \"name\": null,\n",
- " \"tool_calls\": null,\n",
- " \"tool_call_id\": null\n",
- " },\n",
- " {\n",
- " \"role\": \"user\",\n",
- " \"content\": \"Explain {{topic}} in simple terms\",\n",
- " \"name\": null,\n",
- " \"tool_calls\": null,\n",
- " \"tool_call_id\": null\n",
- " }\n",
- " ],\n",
- " \"system_prompt\": null,\n",
- " \"user_prompt\": null,\n",
- " \"template_format\": \"curly\",\n",
- " \"input_keys\": null,\n",
- " \"llm_config\": {\n",
- " \"model\": \"gpt-3.5-turbo\",\n",
- " \"temperature\": 0.6,\n",
- " \"max_tokens\": 150,\n",
- " \"top_p\": 1.0,\n",
- " \"frequency_penalty\": 0.0,\n",
- " \"presence_penalty\": 0.0,\n",
- " \"response_format\": null,\n",
- " \"stream\": null,\n",
- " \"tools\": null,\n",
- " \"tool_choice\": null\n",
- " }\n",
- " }\n",
- " }\n",
- "}\n"
- ]
- }
- ],
- "source": [
- "from agenta.sdk.types import PromptTemplate, Message, ModelConfig\n",
- "from pydantic import BaseModel\n",
- "\n",
- "# We need to create a Pydantic Model with a `prompt` field of type `PromptTemplate`\n",
- "class Config(BaseModel):\n",
- " prompt: PromptTemplate\n",
- "\n",
- "config = Config(\n",
- " prompt=PromptTemplate(\n",
- " messages=[\n",
- " Message(role=\"system\", content=\"You are an assistant that provides concise answers\"),\n",
- " Message(role=\"user\", content=\"Explain {{topic}} in simple terms\"),\n",
- " ],\n",
- " llm_config=ModelConfig(\n",
- " model=\"gpt-3.5-turbo\",\n",
- " max_tokens=150,\n",
- " temperature=0.6,\n",
- " top_p=1.0,\n",
- " frequency_penalty=0.0,\n",
- " presence_penalty=0.0,\n",
- " ),\n",
- " template_format=\"curly\"\n",
- " )\n",
- ")\n",
- "\n",
- "# Create a new variant\n",
- "variant = ag.VariantManager.create(\n",
- " parameters=config.model_dump(),\n",
- " app_slug=\"topic-explainer\",\n",
- " variant_slug=\"new-variant\"\n",
- ")\n",
- "\n",
- "print(\"Created variant:\")\n",
- "print(variant)\n"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "This command will create a new variant and initialize it with the first commit containing the provided parameters.\n",
- "\n",
- "- **Parameters:**\n",
- " - `app_slug`: The unique slug of your application.\n",
- " - `variant_slug`: The unique slug of the new variant.\n",
- " - `parameters`: A dictionary containing the initial configuration parameters.\n"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Deploying changes to the production environment\n",
- "\n",
- "To deploy our commit to an environment, use the `DeploymentManager.deploy` method.\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 5,
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "Deployed variant to environment:\n",
- "{\n",
- " \"app_id\": \"0199397d-14f4-7bb2-a9c5-9a12934fb6c3\",\n",
- " \"app_slug\": \"topic-explainer\",\n",
- " \"variant_id\": \"0199397d-1d46-74c3-bc3f-e63f2357c7dc\",\n",
- " \"variant_slug\": \"new-variant\",\n",
- " \"variant_version\": 1,\n",
- " \"environment_id\": \"0199397d-23fe-7621-8d30-29426339a777\",\n",
- " \"environment_slug\": \"production\",\n",
- " \"environment_version\": 1,\n",
- " \"committed_at\": \"2025-09-11T15:55:24.842469+00:00\",\n",
- " \"committed_by\": \"mahmoud@agenta.ai\",\n",
- " \"committed_by_id\": \"0198eb1b-012a-7891-955e-15ef49443ece\",\n",
- " \"deployed_at\": \"2025-09-11T15:55:26.590624+00:00\",\n",
- " \"deployed_by\": \"mahmoud@agenta.ai\",\n",
- " \"deployed_by_id\": \"0198eb1b-012a-7891-955e-15ef49443ece\"\n",
- "}\n"
- ]
- }
- ],
- "source": [
- "# Deploy the variant to the production environment\n",
- "deployment = ag.DeploymentManager.deploy(\n",
- " app_slug=\"topic-explainer\",\n",
- " variant_slug=\"new-variant\",\n",
- " environment_slug=\"production\",\n",
- ")\n",
- "\n",
- "print(\"Deployed variant to environment:\")\n",
- "print(deployment)\n"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "- **Parameters:**\n",
- " - `environment_slug`: The slug of the environment (`development`, `staging`, or `production`).\n",
- "\n",
- "- **Notes:**\n",
- " - Deploying a variant without specifying a `variant_version` deploys the latest version.\n"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Committing a change to variant\n",
- "\n",
- "We're now going to commit changes to our variant. Note that this will not modify the version in deployment!\n",
- "\n",
- "To save changes to a variant (creating a new version), we are going to use the `VariantManager.commit` method with explicit parameters.\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 6,
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "Committed new version of variant:\n",
- "{\n",
- " \"app_id\": \"0199397d-14f4-7bb2-a9c5-9a12934fb6c3\",\n",
- " \"app_slug\": \"topic-explainer\",\n",
- " \"variant_id\": \"0199397d-2a70-7f20-ba60-2b810e512400\",\n",
- " \"variant_slug\": \"new-variant\",\n",
- " \"variant_version\": 2,\n",
- " \"environment_id\": null,\n",
- " \"environment_slug\": null,\n",
- " \"environment_version\": null,\n",
- " \"committed_at\": \"2025-09-11T15:55:24.842469+00:00\",\n",
- " \"committed_by\": \"mahmoud@agenta.ai\",\n",
- " \"committed_by_id\": \"0198eb1b-012a-7891-955e-15ef49443ece\",\n",
- " \"deployed_at\": null,\n",
- " \"deployed_by\": null,\n",
- " \"deployed_by_id\": null,\n",
- " \"params\": {\n",
- " \"prompt\": {\n",
- " \"messages\": [\n",
- " {\n",
- " \"role\": \"system\",\n",
- " \"content\": \"You are an assistant that provides concise answers\",\n",
- " \"name\": null,\n",
- " \"tool_calls\": null,\n",
- " \"tool_call_id\": null\n",
- " },\n",
- " {\n",
- " \"role\": \"user\",\n",
- " \"content\": \"Use Paul Graham style to explain {{topic}} in simple terms.\",\n",
- " \"name\": null,\n",
- " \"tool_calls\": null,\n",
- " \"tool_call_id\": null\n",
- " }\n",
- " ],\n",
- " \"system_prompt\": null,\n",
- " \"user_prompt\": null,\n",
- " \"template_format\": \"curly\",\n",
- " \"input_keys\": null,\n",
- " \"llm_config\": {\n",
- " \"model\": \"gpt-3.5-turbo\",\n",
- " \"temperature\": 0.9,\n",
- " \"max_tokens\": 150,\n",
- " \"top_p\": 1.0,\n",
- " \"frequency_penalty\": 0.0,\n",
- " \"presence_penalty\": 0.0,\n",
- " \"response_format\": null,\n",
- " \"stream\": null,\n",
- " \"tools\": null,\n",
- " \"tool_choice\": null\n",
- " }\n",
- " }\n",
- " }\n",
- "}\n"
- ]
- }
- ],
- "source": [
- "config2 = Config(\n",
- " prompt=PromptTemplate(\n",
- " messages=[\n",
- " Message(role=\"system\", content=\"You are an assistant that provides concise answers\"),\n",
- " Message(role=\"user\", content=\"Use Paul Graham style to explain {{topic}} in simple terms.\"),\n",
- " ],\n",
- " llm_config=ModelConfig(\n",
- " model=\"gpt-3.5-turbo\",\n",
- " max_tokens=150,\n",
- " temperature=0.9,\n",
- " top_p=1.0,\n",
- " frequency_penalty=0.0,\n",
- " presence_penalty=0.0,\n",
- " ),\n",
- " template_format=\"curly\"\n",
- " )\n",
- ")\n",
- "\n",
- "# Commit the new version\n",
- "variant = ag.VariantManager.commit(\n",
- " parameters=config2.model_dump(),\n",
- " app_slug=\"topic-explainer\",\n",
- " variant_slug=\"new-variant\"\n",
- ")\n",
- "\n",
- "print(\"Committed new version of variant:\")\n",
- "print(variant)\n"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- ":::info Immutability\n",
- "Each commit creates a new version of the variant. Versions are immutable once created.\n",
- ":::\n"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Fetching the prompt in production\n",
- "\n",
- "Now we'll fetch and use the prompt that's in production. Keep in mind that the production environment still references the first version of our variant. If we want it to reflect the latest changes, we'll need to deploy it again.\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 7,
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "Fetched configuration from production:\n",
- "{'prompt': {'messages': [{'role': 'system', 'content': 'You are an assistant that provides concise answers', 'name': None, 'tool_calls': None, 'tool_call_id': None}, {'role': 'user', 'content': 'Explain {{topic}} in simple terms', 'name': None, 'tool_calls': None, 'tool_call_id': None}], 'system_prompt': None, 'user_prompt': None, 'template_format': 'curly', 'input_keys': None, 'llm_config': {'model': 'gpt-3.5-turbo', 'temperature': 0.6, 'max_tokens': 150, 'top_p': 1.0, 'frequency_penalty': 0.0, 'presence_penalty': 0.0, 'response_format': None, 'stream': None, 'tools': None, 'tool_choice': None}}}\n"
- ]
- }
- ],
- "source": [
- "# Fetch configuration from the production environment\n",
- "config = ag.ConfigManager.get_from_registry(\n",
- " app_slug=\"topic-explainer\",\n",
- " environment_slug=\"production\"\n",
- ")\n",
- "\n",
- "print(\"Fetched configuration from production:\")\n",
- "print(config)\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 10,
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "Response for topic 'artificial intelligence':\n",
- "Artificial intelligence (AI) is the ability of a computer or machine to perform tasks that typically require human intelligence, such as learning, problem-solving, and decision-making. It involves creating algorithms and systems that can process information, recognize patterns, and make predictions without explicit programming.\n"
- ]
- }
- ],
- "source": [
- "# Using the configuration with OpenAI client\n",
- "from openai import OpenAI\n",
- "\n",
- "# Format the prompt with your topic\n",
- "topic = \"artificial intelligence\"\n",
- "prompt = PromptTemplate(**config[\"prompt\"]).format(topic=topic)\n",
- "client = OpenAI()\n",
- "\n",
- "response = client.chat.completions.create(\n",
- " **prompt.to_openai_kwargs()\n",
- ")\n",
- "\n",
- "print(f\"Response for topic '{topic}':\")\n",
- "print(response.choices[0].message.content)\n"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Next Steps\n",
- "\n",
- "Now that you've learned how to manage configurations using the SDK, you can:\n",
- "\n",
- "- Read the guide to explore more advanced features of the SDK\n",
- "- Read how to reference prompts in your traces\n",
- "- Read how to manage configuration for your workflows (chain of prompts, RAG..)\n",
- "\n",
- "## Summary\n",
- "\n",
- "In this tutorial, we've covered the complete workflow for managing prompts with the Agenta SDK:\n",
- "\n",
- "1. ✅ Created a new application\n",
- "2. ✅ Created a variant with initial configuration\n",
- "3. ✅ Deployed the variant to production environment\n",
- "4. ✅ Committed changes to create a new version\n",
- "5. ✅ Fetched configuration from production and used it with OpenAI\n",
- "\n",
- "This demonstrates the Git-like workflow that Agenta uses for prompt versioning, allowing you to manage different variants and deploy specific versions to different environments.\n"
- ]
- }
- ],
- "metadata": {
- "kernelspec": {
- "display_name": ".venv (3.12.3)",
- "language": "python",
- "name": "python3"
- },
- "language_info": {
- "codemirror_mode": {
- "name": "ipython",
- "version": 3
- },
- "file_extension": ".py",
- "mimetype": "text/x-python",
- "name": "python",
- "nbconvert_exporter": "python",
- "pygments_lexer": "ipython3",
- "version": "3.12.3"
- }
- },
- "nbformat": 4,
- "nbformat_minor": 2
-}
\ No newline at end of file
diff --git a/examples/node/observability-opentelemetry/.gitignore b/examples/node/observability-opentelemetry/.gitignore
deleted file mode 100644
index 2cd0f44931..0000000000
--- a/examples/node/observability-opentelemetry/.gitignore
+++ /dev/null
@@ -1,4 +0,0 @@
-node_modules/
-.env
-*.log
-
diff --git a/examples/node/observability-opentelemetry/README.md b/examples/node/observability-opentelemetry/README.md
deleted file mode 100644
index 9d61b4984f..0000000000
--- a/examples/node/observability-opentelemetry/README.md
+++ /dev/null
@@ -1,68 +0,0 @@
-# OpenTelemetry Quick Start Example
-
-This example demonstrates how to instrument a Node.js application with OpenTelemetry and send traces to Agenta.
-
-## Prerequisites
-
-- Node.js 18+ installed
-- Agenta API key ([get one here](https://cloud.agenta.ai))
-- OpenAI API key
-
-## Version Compatibility
-
-This example uses the latest stable versions:
-- **OpenAI SDK** (`latest` - v6.x)
-- **OpenInference Instrumentation** (`latest` - v3.x)
-- **OpenInference Semantic Conventions** (`latest` - v2.x)
-
-## Setup
-
-1. Install dependencies:
-```bash
-npm install
-```
-
-2. Create a `.env` file with your credentials:
-```bash
-cp .env.example .env
-# Edit .env and add your API keys
-```
-
-3. Run the example:
-```bash
-npm start
-```
-
-## What's Happening?
-
-1. **instrumentation.js** - Configures OpenTelemetry to:
- - Send traces to Agenta via OTLP
- - Automatically instrument OpenAI calls using OpenInference
- - Use SimpleSpanProcessor for immediate export (ideal for short scripts)
-
-2. **app.js** - A simple application that:
- - Creates a manual span using Agenta's semantic conventions
- - Calls OpenAI's chat completion API (auto-instrumented)
- - Demonstrates proper use of `ag.data.inputs`, `ag.data.outputs`, and `ag.data.internals`
-
-3. All traces are sent to Agenta where you can:
- - View the complete trace timeline
- - See inputs/outputs with proper formatting
- - Monitor costs and latency
- - Debug issues
-
-## Semantic Conventions
-
-This example follows Agenta's semantic conventions for proper trace display:
-
-- **`ag.type.node`** - Defines the operation type (workflow, task, tool, etc.)
-- **`ag.data.inputs`** - Stores input parameters as JSON
-- **`ag.data.outputs`** - Stores output results as JSON
-- **`ag.data.internals`** - Stores intermediate values and metadata
-
-See [SEMANTIC_CONVENTIONS.md](./SEMANTIC_CONVENTIONS.md) for detailed documentation.
-
-## View Your Traces
-
-After running the example, log in to [Agenta](https://cloud.agenta.ai) and navigate to the Observability section to see your traces!
-
diff --git a/examples/node/observability-opentelemetry/app.js b/examples/node/observability-opentelemetry/app.js
deleted file mode 100644
index b3dce7e794..0000000000
--- a/examples/node/observability-opentelemetry/app.js
+++ /dev/null
@@ -1,80 +0,0 @@
-// app.js
-import OpenAI from "openai";
-import { trace } from "@opentelemetry/api";
-
-const openai = new OpenAI({
- apiKey: process.env.OPENAI_API_KEY,
-});
-
-const tracer = trace.getTracer("test-app", "1.0.0");
-
-async function generate() {
- // Create a manual span using Agenta's semantic conventions
- // This demonstrates how to manually instrument functions with proper attributes
- return tracer.startActiveSpan("generate", async (span) => {
- try {
- // Define the messages for the chat completion
- const messages = [
- { role: "system", content: "You are a helpful assistant." },
- { role: "user", content: "Write a short story about AI Engineering." },
- ];
-
- // Agenta Semantic Convention: ag.type.node
- // Defines the type of operation (workflow, task, tool, etc.)
- span.setAttribute("ag.type.node", "workflow");
-
- // Agenta Semantic Convention: ag.data.inputs
- // Stores the input parameters as JSON
- span.setAttribute("ag.data.inputs", JSON.stringify({
- messages: messages,
- model: "gpt-3.5-turbo"
- }));
-
- const response = await openai.chat.completions.create({
- model: "gpt-3.5-turbo",
- messages: messages,
- });
-
- const content = response.choices[0].message.content;
-
- // Agenta Semantic Convention: ag.data.internals
- // Stores intermediate values and metadata (optional)
- span.setAttribute("ag.data.internals", JSON.stringify({
- response_length: content.length
- }));
-
- // Agenta Semantic Convention: ag.data.outputs
- // Stores the output results as JSON
- span.setAttribute("ag.data.outputs", JSON.stringify({
- content: content
- }));
-
- return content;
- } finally {
- span.end();
- }
- });
-}
-
-async function main() {
- try {
- const result = await generate();
- console.log("\n" + result);
-
- console.log("\n⏳ Flushing traces...");
- // Ensure traces are flushed before exit
- const tracerProvider = trace.getTracerProvider();
- if (tracerProvider && typeof tracerProvider.forceFlush === 'function') {
- await tracerProvider.forceFlush();
- }
- // Extra wait to ensure export completes
- await new Promise(resolve => setTimeout(resolve, 1000));
- console.log("✅ Done!");
- } catch (error) {
- console.error("❌ Error:", error.message);
- process.exit(1);
- }
-}
-
-main();
-
diff --git a/examples/node/observability-opentelemetry/instrumentation.js b/examples/node/observability-opentelemetry/instrumentation.js
deleted file mode 100644
index 4921de9bf7..0000000000
--- a/examples/node/observability-opentelemetry/instrumentation.js
+++ /dev/null
@@ -1,74 +0,0 @@
-// instrumentation.js
-import { registerInstrumentations } from "@opentelemetry/instrumentation";
-import { OpenAIInstrumentation } from "@arizeai/openinference-instrumentation-openai";
-import { diag, DiagConsoleLogger, DiagLogLevel } from "@opentelemetry/api";
-import { OTLPTraceExporter } from "@opentelemetry/exporter-trace-otlp-proto";
-import { Resource } from "@opentelemetry/resources";
-import { BatchSpanProcessor, SimpleSpanProcessor } from "@opentelemetry/sdk-trace-base";
-import { NodeTracerProvider } from "@opentelemetry/sdk-trace-node";
-import { ATTR_SERVICE_NAME } from "@opentelemetry/semantic-conventions";
-import { SEMRESATTRS_PROJECT_NAME } from "@arizeai/openinference-semantic-conventions";
-import OpenAI from "openai";
-
-// For troubleshooting, set the log level to DiagLogLevel.DEBUG
-diag.setLogger(new DiagConsoleLogger(), DiagLogLevel.INFO);
-
-// Get Agenta configuration from environment variables
-const AGENTA_HOST = process.env.AGENTA_HOST || "https://cloud.agenta.ai";
-const AGENTA_API_KEY = process.env.AGENTA_API_KEY;
-
-if (!AGENTA_API_KEY) {
- console.error("❌ AGENTA_API_KEY environment variable is required");
- process.exit(1);
-}
-
-// Configure the OTLP exporter to send traces to Agenta
-const otlpExporter = new OTLPTraceExporter({
- url: `${AGENTA_HOST}/api/otlp/v1/traces`,
- headers: {
- Authorization: `ApiKey ${AGENTA_API_KEY}`,
- },
- timeoutMillis: 5000, // 5 second timeout
-});
-
-// Add logging to the exporter for debugging
-const originalExport = otlpExporter.export.bind(otlpExporter);
-otlpExporter.export = function (spans, resultCallback) {
- console.log(`📤 Exporting ${spans.length} span(s)...`);
- originalExport(spans, (result) => {
- if (result.code === 0) {
- console.log('✅ Spans exported successfully');
- } else {
- console.error('❌ Export failed:', result.error);
- }
- resultCallback(result);
- });
-};
-
-// Create and configure the tracer provider
-const tracerProvider = new NodeTracerProvider({
- resource: new Resource({
- [ATTR_SERVICE_NAME]: "openai-quickstart",
- // Project name in Agenta, defaults to "default"
- [SEMRESATTRS_PROJECT_NAME]: "openai-quickstart",
- }),
-});
-
-// Use SimpleSpanProcessor for immediate export (better for short-lived scripts)
-// For long-running services, use: new BatchSpanProcessor(otlpExporter)
-tracerProvider.addSpanProcessor(new SimpleSpanProcessor(otlpExporter));
-
-// Register the tracer provider
-tracerProvider.register();
-
-// Register OpenAI instrumentation with manual instrumentation
-// This is required for OpenInference to properly instrument the OpenAI client
-const instrumentation = new OpenAIInstrumentation();
-instrumentation.manuallyInstrument(OpenAI);
-
-registerInstrumentations({
- instrumentations: [instrumentation],
-});
-
-console.log("✅ OpenTelemetry instrumentation initialized");
-
diff --git a/examples/node/observability-opentelemetry/package-lock.json b/examples/node/observability-opentelemetry/package-lock.json
deleted file mode 100644
index c9c0ef0ae5..0000000000
--- a/examples/node/observability-opentelemetry/package-lock.json
+++ /dev/null
@@ -1,935 +0,0 @@
-{
- "name": "agenta-opentelemetry-quickstart",
- "version": "1.0.0",
- "lockfileVersion": 3,
- "requires": true,
- "packages": {
- "": {
- "name": "agenta-opentelemetry-quickstart",
- "version": "1.0.0",
- "dependencies": {
- "@arizeai/openinference-instrumentation-openai": "^3.2.3",
- "@arizeai/openinference-semantic-conventions": "^2.1.2",
- "@opentelemetry/api": "^1.9.0",
- "@opentelemetry/exporter-trace-otlp-proto": "^0.54.0",
- "@opentelemetry/instrumentation": "^0.54.0",
- "@opentelemetry/resources": "^1.28.0",
- "@opentelemetry/sdk-trace-base": "^1.28.0",
- "@opentelemetry/sdk-trace-node": "^1.28.0",
- "@opentelemetry/semantic-conventions": "^1.28.0",
- "openai": "^6.7.0"
- },
- "devDependencies": {
- "@types/node": "^22.0.0",
- "typescript": "^5.7.0"
- }
- },
- "node_modules/@arizeai/openinference-core": {
- "version": "1.0.7",
- "resolved": "https://registry.npmjs.org/@arizeai/openinference-core/-/openinference-core-1.0.7.tgz",
- "integrity": "sha512-O9WYkrHNh/0mGTV+T9SWC3tkxVrT16gBrFiByG3aukBsqdOfSzoRj6QINk+Oi+VEDNIoQUzVQPFh81/gEL/thA==",
- "license": "Apache-2.0",
- "dependencies": {
- "@arizeai/openinference-semantic-conventions": "2.1.2",
- "@opentelemetry/api": "^1.9.0",
- "@opentelemetry/core": "^1.25.1"
- }
- },
- "node_modules/@arizeai/openinference-instrumentation-openai": {
- "version": "3.2.3",
- "resolved": "https://registry.npmjs.org/@arizeai/openinference-instrumentation-openai/-/openinference-instrumentation-openai-3.2.3.tgz",
- "integrity": "sha512-4GiUyUIafNkAf7milnsS4Xjh0rZwjPvhHu+xkcj2MQ19a0Teu6PcFDa7DCt7YV/xw2o+wcxeaLfVURmGGLAHOg==",
- "license": "Apache-2.0",
- "dependencies": {
- "@arizeai/openinference-core": "1.0.7",
- "@arizeai/openinference-semantic-conventions": "2.1.2",
- "@opentelemetry/api": "^1.9.0",
- "@opentelemetry/core": "^1.25.1",
- "@opentelemetry/instrumentation": "^0.46.0"
- }
- },
- "node_modules/@arizeai/openinference-instrumentation-openai/node_modules/@opentelemetry/instrumentation": {
- "version": "0.46.0",
- "resolved": "https://registry.npmjs.org/@opentelemetry/instrumentation/-/instrumentation-0.46.0.tgz",
- "integrity": "sha512-a9TijXZZbk0vI5TGLZl+0kxyFfrXHhX6Svtz7Pp2/VBlCSKrazuULEyoJQrOknJyFWNMEmbbJgOciHCCpQcisw==",
- "license": "Apache-2.0",
- "dependencies": {
- "@types/shimmer": "^1.0.2",
- "import-in-the-middle": "1.7.1",
- "require-in-the-middle": "^7.1.1",
- "semver": "^7.5.2",
- "shimmer": "^1.2.1"
- },
- "engines": {
- "node": ">=14"
- },
- "peerDependencies": {
- "@opentelemetry/api": "^1.3.0"
- }
- },
- "node_modules/@arizeai/openinference-instrumentation-openai/node_modules/import-in-the-middle": {
- "version": "1.7.1",
- "resolved": "https://registry.npmjs.org/import-in-the-middle/-/import-in-the-middle-1.7.1.tgz",
- "integrity": "sha512-1LrZPDtW+atAxH42S6288qyDFNQ2YCty+2mxEPRtfazH6Z5QwkaBSTS2ods7hnVJioF6rkRfNoA6A/MstpFXLg==",
- "license": "Apache-2.0",
- "dependencies": {
- "acorn": "^8.8.2",
- "acorn-import-assertions": "^1.9.0",
- "cjs-module-lexer": "^1.2.2",
- "module-details-from-path": "^1.0.3"
- }
- },
- "node_modules/@arizeai/openinference-semantic-conventions": {
- "version": "2.1.2",
- "resolved": "https://registry.npmjs.org/@arizeai/openinference-semantic-conventions/-/openinference-semantic-conventions-2.1.2.tgz",
- "integrity": "sha512-u7UeuU9bJ1LxzHk0MPWb+1ZcotCcJwPnKDXi7Rl2cPs1pWMFg9Ogq7zzYZX+sDcibD2AEa1U+ElyOD8DwZc9gw==",
- "license": "Apache-2.0"
- },
- "node_modules/@opentelemetry/api": {
- "version": "1.9.0",
- "resolved": "https://registry.npmjs.org/@opentelemetry/api/-/api-1.9.0.tgz",
- "integrity": "sha512-3giAOQvZiH5F9bMlMiv8+GSPMeqg0dbaeo58/0SlA9sxSqZhnUtxzX9/2FzyhS9sWQf5S0GJE0AKBrFqjpeYcg==",
- "license": "Apache-2.0",
- "engines": {
- "node": ">=8.0.0"
- }
- },
- "node_modules/@opentelemetry/api-logs": {
- "version": "0.54.2",
- "resolved": "https://registry.npmjs.org/@opentelemetry/api-logs/-/api-logs-0.54.2.tgz",
- "integrity": "sha512-4MTVwwmLgUh5QrJnZpYo6YRO5IBLAggf2h8gWDblwRagDStY13aEvt7gGk3jewrMaPlHiF83fENhIx0HO97/cQ==",
- "license": "Apache-2.0",
- "dependencies": {
- "@opentelemetry/api": "^1.3.0"
- },
- "engines": {
- "node": ">=14"
- }
- },
- "node_modules/@opentelemetry/context-async-hooks": {
- "version": "1.30.1",
- "resolved": "https://registry.npmjs.org/@opentelemetry/context-async-hooks/-/context-async-hooks-1.30.1.tgz",
- "integrity": "sha512-s5vvxXPVdjqS3kTLKMeBMvop9hbWkwzBpu+mUO2M7sZtlkyDJGwFe33wRKnbaYDo8ExRVBIIdwIGrqpxHuKttA==",
- "license": "Apache-2.0",
- "engines": {
- "node": ">=14"
- },
- "peerDependencies": {
- "@opentelemetry/api": ">=1.0.0 <1.10.0"
- }
- },
- "node_modules/@opentelemetry/core": {
- "version": "1.27.0",
- "resolved": "https://registry.npmjs.org/@opentelemetry/core/-/core-1.27.0.tgz",
- "integrity": "sha512-yQPKnK5e+76XuiqUH/gKyS8wv/7qITd5ln56QkBTf3uggr0VkXOXfcaAuG330UfdYu83wsyoBwqwxigpIG+Jkg==",
- "license": "Apache-2.0",
- "dependencies": {
- "@opentelemetry/semantic-conventions": "1.27.0"
- },
- "engines": {
- "node": ">=14"
- },
- "peerDependencies": {
- "@opentelemetry/api": ">=1.0.0 <1.10.0"
- }
- },
- "node_modules/@opentelemetry/core/node_modules/@opentelemetry/semantic-conventions": {
- "version": "1.27.0",
- "resolved": "https://registry.npmjs.org/@opentelemetry/semantic-conventions/-/semantic-conventions-1.27.0.tgz",
- "integrity": "sha512-sAay1RrB+ONOem0OZanAR1ZI/k7yDpnOQSQmTMuGImUQb2y8EbSaCJ94FQluM74xoU03vlb2d2U90hZluL6nQg==",
- "license": "Apache-2.0",
- "engines": {
- "node": ">=14"
- }
- },
- "node_modules/@opentelemetry/exporter-trace-otlp-proto": {
- "version": "0.54.2",
- "resolved": "https://registry.npmjs.org/@opentelemetry/exporter-trace-otlp-proto/-/exporter-trace-otlp-proto-0.54.2.tgz",
- "integrity": "sha512-XSmm1N2wAhoWDXP1q/N6kpLebWaxl6VIADv4WA5QWKHLRpF3gLz5NAWNJBR8ygsvv8jQcrwnXgwfnJ18H3v1fg==",
- "license": "Apache-2.0",
- "dependencies": {
- "@opentelemetry/core": "1.27.0",
- "@opentelemetry/otlp-exporter-base": "0.54.2",
- "@opentelemetry/otlp-transformer": "0.54.2",
- "@opentelemetry/resources": "1.27.0",
- "@opentelemetry/sdk-trace-base": "1.27.0"
- },
- "engines": {
- "node": ">=14"
- },
- "peerDependencies": {
- "@opentelemetry/api": "^1.3.0"
- }
- },
- "node_modules/@opentelemetry/exporter-trace-otlp-proto/node_modules/@opentelemetry/resources": {
- "version": "1.27.0",
- "resolved": "https://registry.npmjs.org/@opentelemetry/resources/-/resources-1.27.0.tgz",
- "integrity": "sha512-jOwt2VJ/lUD5BLc+PMNymDrUCpm5PKi1E9oSVYAvz01U/VdndGmrtV3DU1pG4AwlYhJRHbHfOUIlpBeXCPw6QQ==",
- "license": "Apache-2.0",
- "dependencies": {
- "@opentelemetry/core": "1.27.0",
- "@opentelemetry/semantic-conventions": "1.27.0"
- },
- "engines": {
- "node": ">=14"
- },
- "peerDependencies": {
- "@opentelemetry/api": ">=1.0.0 <1.10.0"
- }
- },
- "node_modules/@opentelemetry/exporter-trace-otlp-proto/node_modules/@opentelemetry/sdk-trace-base": {
- "version": "1.27.0",
- "resolved": "https://registry.npmjs.org/@opentelemetry/sdk-trace-base/-/sdk-trace-base-1.27.0.tgz",
- "integrity": "sha512-btz6XTQzwsyJjombpeqCX6LhiMQYpzt2pIYNPnw0IPO/3AhT6yjnf8Mnv3ZC2A4eRYOjqrg+bfaXg9XHDRJDWQ==",
- "license": "Apache-2.0",
- "dependencies": {
- "@opentelemetry/core": "1.27.0",
- "@opentelemetry/resources": "1.27.0",
- "@opentelemetry/semantic-conventions": "1.27.0"
- },
- "engines": {
- "node": ">=14"
- },
- "peerDependencies": {
- "@opentelemetry/api": ">=1.0.0 <1.10.0"
- }
- },
- "node_modules/@opentelemetry/exporter-trace-otlp-proto/node_modules/@opentelemetry/semantic-conventions": {
- "version": "1.27.0",
- "resolved": "https://registry.npmjs.org/@opentelemetry/semantic-conventions/-/semantic-conventions-1.27.0.tgz",
- "integrity": "sha512-sAay1RrB+ONOem0OZanAR1ZI/k7yDpnOQSQmTMuGImUQb2y8EbSaCJ94FQluM74xoU03vlb2d2U90hZluL6nQg==",
- "license": "Apache-2.0",
- "engines": {
- "node": ">=14"
- }
- },
- "node_modules/@opentelemetry/instrumentation": {
- "version": "0.54.2",
- "resolved": "https://registry.npmjs.org/@opentelemetry/instrumentation/-/instrumentation-0.54.2.tgz",
- "integrity": "sha512-go6zpOVoZVztT9r1aPd79Fr3OWiD4N24bCPJsIKkBses8oyFo12F/Ew3UBTdIu6hsW4HC4MVEJygG6TEyJI/lg==",
- "license": "Apache-2.0",
- "dependencies": {
- "@opentelemetry/api-logs": "0.54.2",
- "@types/shimmer": "^1.2.0",
- "import-in-the-middle": "^1.8.1",
- "require-in-the-middle": "^7.1.1",
- "semver": "^7.5.2",
- "shimmer": "^1.2.1"
- },
- "engines": {
- "node": ">=14"
- },
- "peerDependencies": {
- "@opentelemetry/api": "^1.3.0"
- }
- },
- "node_modules/@opentelemetry/otlp-exporter-base": {
- "version": "0.54.2",
- "resolved": "https://registry.npmjs.org/@opentelemetry/otlp-exporter-base/-/otlp-exporter-base-0.54.2.tgz",
- "integrity": "sha512-NrNyxu6R/bGAwanhz1HI0aJWKR6xUED4TjCH4iWMlAfyRukGbI9Kt/Akd2sYLwRKNhfS+sKetKGCUQPMDyYYMA==",
- "license": "Apache-2.0",
- "dependencies": {
- "@opentelemetry/core": "1.27.0",
- "@opentelemetry/otlp-transformer": "0.54.2"
- },
- "engines": {
- "node": ">=14"
- },
- "peerDependencies": {
- "@opentelemetry/api": "^1.3.0"
- }
- },
- "node_modules/@opentelemetry/otlp-transformer": {
- "version": "0.54.2",
- "resolved": "https://registry.npmjs.org/@opentelemetry/otlp-transformer/-/otlp-transformer-0.54.2.tgz",
- "integrity": "sha512-2tIjahJlMRRUz0A2SeE+qBkeBXBFkSjR0wqJ08kuOqaL8HNGan5iZf+A8cfrfmZzPUuMKCyY9I+okzFuFs6gKQ==",
- "license": "Apache-2.0",
- "dependencies": {
- "@opentelemetry/api-logs": "0.54.2",
- "@opentelemetry/core": "1.27.0",
- "@opentelemetry/resources": "1.27.0",
- "@opentelemetry/sdk-logs": "0.54.2",
- "@opentelemetry/sdk-metrics": "1.27.0",
- "@opentelemetry/sdk-trace-base": "1.27.0",
- "protobufjs": "^7.3.0"
- },
- "engines": {
- "node": ">=14"
- },
- "peerDependencies": {
- "@opentelemetry/api": "^1.3.0"
- }
- },
- "node_modules/@opentelemetry/otlp-transformer/node_modules/@opentelemetry/resources": {
- "version": "1.27.0",
- "resolved": "https://registry.npmjs.org/@opentelemetry/resources/-/resources-1.27.0.tgz",
- "integrity": "sha512-jOwt2VJ/lUD5BLc+PMNymDrUCpm5PKi1E9oSVYAvz01U/VdndGmrtV3DU1pG4AwlYhJRHbHfOUIlpBeXCPw6QQ==",
- "license": "Apache-2.0",
- "dependencies": {
- "@opentelemetry/core": "1.27.0",
- "@opentelemetry/semantic-conventions": "1.27.0"
- },
- "engines": {
- "node": ">=14"
- },
- "peerDependencies": {
- "@opentelemetry/api": ">=1.0.0 <1.10.0"
- }
- },
- "node_modules/@opentelemetry/otlp-transformer/node_modules/@opentelemetry/sdk-trace-base": {
- "version": "1.27.0",
- "resolved": "https://registry.npmjs.org/@opentelemetry/sdk-trace-base/-/sdk-trace-base-1.27.0.tgz",
- "integrity": "sha512-btz6XTQzwsyJjombpeqCX6LhiMQYpzt2pIYNPnw0IPO/3AhT6yjnf8Mnv3ZC2A4eRYOjqrg+bfaXg9XHDRJDWQ==",
- "license": "Apache-2.0",
- "dependencies": {
- "@opentelemetry/core": "1.27.0",
- "@opentelemetry/resources": "1.27.0",
- "@opentelemetry/semantic-conventions": "1.27.0"
- },
- "engines": {
- "node": ">=14"
- },
- "peerDependencies": {
- "@opentelemetry/api": ">=1.0.0 <1.10.0"
- }
- },
- "node_modules/@opentelemetry/otlp-transformer/node_modules/@opentelemetry/semantic-conventions": {
- "version": "1.27.0",
- "resolved": "https://registry.npmjs.org/@opentelemetry/semantic-conventions/-/semantic-conventions-1.27.0.tgz",
- "integrity": "sha512-sAay1RrB+ONOem0OZanAR1ZI/k7yDpnOQSQmTMuGImUQb2y8EbSaCJ94FQluM74xoU03vlb2d2U90hZluL6nQg==",
- "license": "Apache-2.0",
- "engines": {
- "node": ">=14"
- }
- },
- "node_modules/@opentelemetry/propagator-b3": {
- "version": "1.30.1",
- "resolved": "https://registry.npmjs.org/@opentelemetry/propagator-b3/-/propagator-b3-1.30.1.tgz",
- "integrity": "sha512-oATwWWDIJzybAZ4pO76ATN5N6FFbOA1otibAVlS8v90B4S1wClnhRUk7K+2CHAwN1JKYuj4jh/lpCEG5BAqFuQ==",
- "license": "Apache-2.0",
- "dependencies": {
- "@opentelemetry/core": "1.30.1"
- },
- "engines": {
- "node": ">=14"
- },
- "peerDependencies": {
- "@opentelemetry/api": ">=1.0.0 <1.10.0"
- }
- },
- "node_modules/@opentelemetry/propagator-b3/node_modules/@opentelemetry/core": {
- "version": "1.30.1",
- "resolved": "https://registry.npmjs.org/@opentelemetry/core/-/core-1.30.1.tgz",
- "integrity": "sha512-OOCM2C/QIURhJMuKaekP3TRBxBKxG/TWWA0TL2J6nXUtDnuCtccy49LUJF8xPFXMX+0LMcxFpCo8M9cGY1W6rQ==",
- "license": "Apache-2.0",
- "dependencies": {
- "@opentelemetry/semantic-conventions": "1.28.0"
- },
- "engines": {
- "node": ">=14"
- },
- "peerDependencies": {
- "@opentelemetry/api": ">=1.0.0 <1.10.0"
- }
- },
- "node_modules/@opentelemetry/propagator-b3/node_modules/@opentelemetry/semantic-conventions": {
- "version": "1.28.0",
- "resolved": "https://registry.npmjs.org/@opentelemetry/semantic-conventions/-/semantic-conventions-1.28.0.tgz",
- "integrity": "sha512-lp4qAiMTD4sNWW4DbKLBkfiMZ4jbAboJIGOQr5DvciMRI494OapieI9qiODpOt0XBr1LjIDy1xAGAnVs5supTA==",
- "license": "Apache-2.0",
- "engines": {
- "node": ">=14"
- }
- },
- "node_modules/@opentelemetry/propagator-jaeger": {
- "version": "1.30.1",
- "resolved": "https://registry.npmjs.org/@opentelemetry/propagator-jaeger/-/propagator-jaeger-1.30.1.tgz",
- "integrity": "sha512-Pj/BfnYEKIOImirH76M4hDaBSx6HyZ2CXUqk+Kj02m6BB80c/yo4BdWkn/1gDFfU+YPY+bPR2U0DKBfdxCKwmg==",
- "license": "Apache-2.0",
- "dependencies": {
- "@opentelemetry/core": "1.30.1"
- },
- "engines": {
- "node": ">=14"
- },
- "peerDependencies": {
- "@opentelemetry/api": ">=1.0.0 <1.10.0"
- }
- },
- "node_modules/@opentelemetry/propagator-jaeger/node_modules/@opentelemetry/core": {
- "version": "1.30.1",
- "resolved": "https://registry.npmjs.org/@opentelemetry/core/-/core-1.30.1.tgz",
- "integrity": "sha512-OOCM2C/QIURhJMuKaekP3TRBxBKxG/TWWA0TL2J6nXUtDnuCtccy49LUJF8xPFXMX+0LMcxFpCo8M9cGY1W6rQ==",
- "license": "Apache-2.0",
- "dependencies": {
- "@opentelemetry/semantic-conventions": "1.28.0"
- },
- "engines": {
- "node": ">=14"
- },
- "peerDependencies": {
- "@opentelemetry/api": ">=1.0.0 <1.10.0"
- }
- },
- "node_modules/@opentelemetry/propagator-jaeger/node_modules/@opentelemetry/semantic-conventions": {
- "version": "1.28.0",
- "resolved": "https://registry.npmjs.org/@opentelemetry/semantic-conventions/-/semantic-conventions-1.28.0.tgz",
- "integrity": "sha512-lp4qAiMTD4sNWW4DbKLBkfiMZ4jbAboJIGOQr5DvciMRI494OapieI9qiODpOt0XBr1LjIDy1xAGAnVs5supTA==",
- "license": "Apache-2.0",
- "engines": {
- "node": ">=14"
- }
- },
- "node_modules/@opentelemetry/resources": {
- "version": "1.30.1",
- "resolved": "https://registry.npmjs.org/@opentelemetry/resources/-/resources-1.30.1.tgz",
- "integrity": "sha512-5UxZqiAgLYGFjS4s9qm5mBVo433u+dSPUFWVWXmLAD4wB65oMCoXaJP1KJa9DIYYMeHu3z4BZcStG3LC593cWA==",
- "license": "Apache-2.0",
- "dependencies": {
- "@opentelemetry/core": "1.30.1",
- "@opentelemetry/semantic-conventions": "1.28.0"
- },
- "engines": {
- "node": ">=14"
- },
- "peerDependencies": {
- "@opentelemetry/api": ">=1.0.0 <1.10.0"
- }
- },
- "node_modules/@opentelemetry/resources/node_modules/@opentelemetry/core": {
- "version": "1.30.1",
- "resolved": "https://registry.npmjs.org/@opentelemetry/core/-/core-1.30.1.tgz",
- "integrity": "sha512-OOCM2C/QIURhJMuKaekP3TRBxBKxG/TWWA0TL2J6nXUtDnuCtccy49LUJF8xPFXMX+0LMcxFpCo8M9cGY1W6rQ==",
- "license": "Apache-2.0",
- "dependencies": {
- "@opentelemetry/semantic-conventions": "1.28.0"
- },
- "engines": {
- "node": ">=14"
- },
- "peerDependencies": {
- "@opentelemetry/api": ">=1.0.0 <1.10.0"
- }
- },
- "node_modules/@opentelemetry/resources/node_modules/@opentelemetry/semantic-conventions": {
- "version": "1.28.0",
- "resolved": "https://registry.npmjs.org/@opentelemetry/semantic-conventions/-/semantic-conventions-1.28.0.tgz",
- "integrity": "sha512-lp4qAiMTD4sNWW4DbKLBkfiMZ4jbAboJIGOQr5DvciMRI494OapieI9qiODpOt0XBr1LjIDy1xAGAnVs5supTA==",
- "license": "Apache-2.0",
- "engines": {
- "node": ">=14"
- }
- },
- "node_modules/@opentelemetry/sdk-logs": {
- "version": "0.54.2",
- "resolved": "https://registry.npmjs.org/@opentelemetry/sdk-logs/-/sdk-logs-0.54.2.tgz",
- "integrity": "sha512-yIbYqDLS/AtBbPjCjh6eSToGNRMqW2VR8RrKEy+G+J7dFG7pKoptTH5T+XlKPleP9NY8JZYIpgJBlI+Osi0rFw==",
- "license": "Apache-2.0",
- "dependencies": {
- "@opentelemetry/api-logs": "0.54.2",
- "@opentelemetry/core": "1.27.0",
- "@opentelemetry/resources": "1.27.0"
- },
- "engines": {
- "node": ">=14"
- },
- "peerDependencies": {
- "@opentelemetry/api": ">=1.4.0 <1.10.0"
- }
- },
- "node_modules/@opentelemetry/sdk-logs/node_modules/@opentelemetry/resources": {
- "version": "1.27.0",
- "resolved": "https://registry.npmjs.org/@opentelemetry/resources/-/resources-1.27.0.tgz",
- "integrity": "sha512-jOwt2VJ/lUD5BLc+PMNymDrUCpm5PKi1E9oSVYAvz01U/VdndGmrtV3DU1pG4AwlYhJRHbHfOUIlpBeXCPw6QQ==",
- "license": "Apache-2.0",
- "dependencies": {
- "@opentelemetry/core": "1.27.0",
- "@opentelemetry/semantic-conventions": "1.27.0"
- },
- "engines": {
- "node": ">=14"
- },
- "peerDependencies": {
- "@opentelemetry/api": ">=1.0.0 <1.10.0"
- }
- },
- "node_modules/@opentelemetry/sdk-logs/node_modules/@opentelemetry/semantic-conventions": {
- "version": "1.27.0",
- "resolved": "https://registry.npmjs.org/@opentelemetry/semantic-conventions/-/semantic-conventions-1.27.0.tgz",
- "integrity": "sha512-sAay1RrB+ONOem0OZanAR1ZI/k7yDpnOQSQmTMuGImUQb2y8EbSaCJ94FQluM74xoU03vlb2d2U90hZluL6nQg==",
- "license": "Apache-2.0",
- "engines": {
- "node": ">=14"
- }
- },
- "node_modules/@opentelemetry/sdk-metrics": {
- "version": "1.27.0",
- "resolved": "https://registry.npmjs.org/@opentelemetry/sdk-metrics/-/sdk-metrics-1.27.0.tgz",
- "integrity": "sha512-JzWgzlutoXCydhHWIbLg+r76m+m3ncqvkCcsswXAQ4gqKS+LOHKhq+t6fx1zNytvLuaOUBur7EvWxECc4jPQKg==",
- "license": "Apache-2.0",
- "dependencies": {
- "@opentelemetry/core": "1.27.0",
- "@opentelemetry/resources": "1.27.0"
- },
- "engines": {
- "node": ">=14"
- },
- "peerDependencies": {
- "@opentelemetry/api": ">=1.3.0 <1.10.0"
- }
- },
- "node_modules/@opentelemetry/sdk-metrics/node_modules/@opentelemetry/resources": {
- "version": "1.27.0",
- "resolved": "https://registry.npmjs.org/@opentelemetry/resources/-/resources-1.27.0.tgz",
- "integrity": "sha512-jOwt2VJ/lUD5BLc+PMNymDrUCpm5PKi1E9oSVYAvz01U/VdndGmrtV3DU1pG4AwlYhJRHbHfOUIlpBeXCPw6QQ==",
- "license": "Apache-2.0",
- "dependencies": {
- "@opentelemetry/core": "1.27.0",
- "@opentelemetry/semantic-conventions": "1.27.0"
- },
- "engines": {
- "node": ">=14"
- },
- "peerDependencies": {
- "@opentelemetry/api": ">=1.0.0 <1.10.0"
- }
- },
- "node_modules/@opentelemetry/sdk-metrics/node_modules/@opentelemetry/semantic-conventions": {
- "version": "1.27.0",
- "resolved": "https://registry.npmjs.org/@opentelemetry/semantic-conventions/-/semantic-conventions-1.27.0.tgz",
- "integrity": "sha512-sAay1RrB+ONOem0OZanAR1ZI/k7yDpnOQSQmTMuGImUQb2y8EbSaCJ94FQluM74xoU03vlb2d2U90hZluL6nQg==",
- "license": "Apache-2.0",
- "engines": {
- "node": ">=14"
- }
- },
- "node_modules/@opentelemetry/sdk-trace-base": {
- "version": "1.30.1",
- "resolved": "https://registry.npmjs.org/@opentelemetry/sdk-trace-base/-/sdk-trace-base-1.30.1.tgz",
- "integrity": "sha512-jVPgBbH1gCy2Lb7X0AVQ8XAfgg0pJ4nvl8/IiQA6nxOsPvS+0zMJaFSs2ltXe0J6C8dqjcnpyqINDJmU30+uOg==",
- "license": "Apache-2.0",
- "dependencies": {
- "@opentelemetry/core": "1.30.1",
- "@opentelemetry/resources": "1.30.1",
- "@opentelemetry/semantic-conventions": "1.28.0"
- },
- "engines": {
- "node": ">=14"
- },
- "peerDependencies": {
- "@opentelemetry/api": ">=1.0.0 <1.10.0"
- }
- },
- "node_modules/@opentelemetry/sdk-trace-base/node_modules/@opentelemetry/core": {
- "version": "1.30.1",
- "resolved": "https://registry.npmjs.org/@opentelemetry/core/-/core-1.30.1.tgz",
- "integrity": "sha512-OOCM2C/QIURhJMuKaekP3TRBxBKxG/TWWA0TL2J6nXUtDnuCtccy49LUJF8xPFXMX+0LMcxFpCo8M9cGY1W6rQ==",
- "license": "Apache-2.0",
- "dependencies": {
- "@opentelemetry/semantic-conventions": "1.28.0"
- },
- "engines": {
- "node": ">=14"
- },
- "peerDependencies": {
- "@opentelemetry/api": ">=1.0.0 <1.10.0"
- }
- },
- "node_modules/@opentelemetry/sdk-trace-base/node_modules/@opentelemetry/semantic-conventions": {
- "version": "1.28.0",
- "resolved": "https://registry.npmjs.org/@opentelemetry/semantic-conventions/-/semantic-conventions-1.28.0.tgz",
- "integrity": "sha512-lp4qAiMTD4sNWW4DbKLBkfiMZ4jbAboJIGOQr5DvciMRI494OapieI9qiODpOt0XBr1LjIDy1xAGAnVs5supTA==",
- "license": "Apache-2.0",
- "engines": {
- "node": ">=14"
- }
- },
- "node_modules/@opentelemetry/sdk-trace-node": {
- "version": "1.30.1",
- "resolved": "https://registry.npmjs.org/@opentelemetry/sdk-trace-node/-/sdk-trace-node-1.30.1.tgz",
- "integrity": "sha512-cBjYOINt1JxXdpw1e5MlHmFRc5fgj4GW/86vsKFxJCJ8AL4PdVtYH41gWwl4qd4uQjqEL1oJVrXkSy5cnduAnQ==",
- "license": "Apache-2.0",
- "dependencies": {
- "@opentelemetry/context-async-hooks": "1.30.1",
- "@opentelemetry/core": "1.30.1",
- "@opentelemetry/propagator-b3": "1.30.1",
- "@opentelemetry/propagator-jaeger": "1.30.1",
- "@opentelemetry/sdk-trace-base": "1.30.1",
- "semver": "^7.5.2"
- },
- "engines": {
- "node": ">=14"
- },
- "peerDependencies": {
- "@opentelemetry/api": ">=1.0.0 <1.10.0"
- }
- },
- "node_modules/@opentelemetry/sdk-trace-node/node_modules/@opentelemetry/core": {
- "version": "1.30.1",
- "resolved": "https://registry.npmjs.org/@opentelemetry/core/-/core-1.30.1.tgz",
- "integrity": "sha512-OOCM2C/QIURhJMuKaekP3TRBxBKxG/TWWA0TL2J6nXUtDnuCtccy49LUJF8xPFXMX+0LMcxFpCo8M9cGY1W6rQ==",
- "license": "Apache-2.0",
- "dependencies": {
- "@opentelemetry/semantic-conventions": "1.28.0"
- },
- "engines": {
- "node": ">=14"
- },
- "peerDependencies": {
- "@opentelemetry/api": ">=1.0.0 <1.10.0"
- }
- },
- "node_modules/@opentelemetry/sdk-trace-node/node_modules/@opentelemetry/semantic-conventions": {
- "version": "1.28.0",
- "resolved": "https://registry.npmjs.org/@opentelemetry/semantic-conventions/-/semantic-conventions-1.28.0.tgz",
- "integrity": "sha512-lp4qAiMTD4sNWW4DbKLBkfiMZ4jbAboJIGOQr5DvciMRI494OapieI9qiODpOt0XBr1LjIDy1xAGAnVs5supTA==",
- "license": "Apache-2.0",
- "engines": {
- "node": ">=14"
- }
- },
- "node_modules/@opentelemetry/semantic-conventions": {
- "version": "1.37.0",
- "resolved": "https://registry.npmjs.org/@opentelemetry/semantic-conventions/-/semantic-conventions-1.37.0.tgz",
- "integrity": "sha512-JD6DerIKdJGmRp4jQyX5FlrQjA4tjOw1cvfsPAZXfOOEErMUHjPcPSICS+6WnM0nB0efSFARh0KAZss+bvExOA==",
- "license": "Apache-2.0",
- "engines": {
- "node": ">=14"
- }
- },
- "node_modules/@protobufjs/aspromise": {
- "version": "1.1.2",
- "resolved": "https://registry.npmjs.org/@protobufjs/aspromise/-/aspromise-1.1.2.tgz",
- "integrity": "sha512-j+gKExEuLmKwvz3OgROXtrJ2UG2x8Ch2YZUxahh+s1F2HZ+wAceUNLkvy6zKCPVRkU++ZWQrdxsUeQXmcg4uoQ==",
- "license": "BSD-3-Clause"
- },
- "node_modules/@protobufjs/base64": {
- "version": "1.1.2",
- "resolved": "https://registry.npmjs.org/@protobufjs/base64/-/base64-1.1.2.tgz",
- "integrity": "sha512-AZkcAA5vnN/v4PDqKyMR5lx7hZttPDgClv83E//FMNhR2TMcLUhfRUBHCmSl0oi9zMgDDqRUJkSxO3wm85+XLg==",
- "license": "BSD-3-Clause"
- },
- "node_modules/@protobufjs/codegen": {
- "version": "2.0.4",
- "resolved": "https://registry.npmjs.org/@protobufjs/codegen/-/codegen-2.0.4.tgz",
- "integrity": "sha512-YyFaikqM5sH0ziFZCN3xDC7zeGaB/d0IUb9CATugHWbd1FRFwWwt4ld4OYMPWu5a3Xe01mGAULCdqhMlPl29Jg==",
- "license": "BSD-3-Clause"
- },
- "node_modules/@protobufjs/eventemitter": {
- "version": "1.1.0",
- "resolved": "https://registry.npmjs.org/@protobufjs/eventemitter/-/eventemitter-1.1.0.tgz",
- "integrity": "sha512-j9ednRT81vYJ9OfVuXG6ERSTdEL1xVsNgqpkxMsbIabzSo3goCjDIveeGv5d03om39ML71RdmrGNjG5SReBP/Q==",
- "license": "BSD-3-Clause"
- },
- "node_modules/@protobufjs/fetch": {
- "version": "1.1.0",
- "resolved": "https://registry.npmjs.org/@protobufjs/fetch/-/fetch-1.1.0.tgz",
- "integrity": "sha512-lljVXpqXebpsijW71PZaCYeIcE5on1w5DlQy5WH6GLbFryLUrBD4932W/E2BSpfRJWseIL4v/KPgBFxDOIdKpQ==",
- "license": "BSD-3-Clause",
- "dependencies": {
- "@protobufjs/aspromise": "^1.1.1",
- "@protobufjs/inquire": "^1.1.0"
- }
- },
- "node_modules/@protobufjs/float": {
- "version": "1.0.2",
- "resolved": "https://registry.npmjs.org/@protobufjs/float/-/float-1.0.2.tgz",
- "integrity": "sha512-Ddb+kVXlXst9d+R9PfTIxh1EdNkgoRe5tOX6t01f1lYWOvJnSPDBlG241QLzcyPdoNTsblLUdujGSE4RzrTZGQ==",
- "license": "BSD-3-Clause"
- },
- "node_modules/@protobufjs/inquire": {
- "version": "1.1.0",
- "resolved": "https://registry.npmjs.org/@protobufjs/inquire/-/inquire-1.1.0.tgz",
- "integrity": "sha512-kdSefcPdruJiFMVSbn801t4vFK7KB/5gd2fYvrxhuJYg8ILrmn9SKSX2tZdV6V+ksulWqS7aXjBcRXl3wHoD9Q==",
- "license": "BSD-3-Clause"
- },
- "node_modules/@protobufjs/path": {
- "version": "1.1.2",
- "resolved": "https://registry.npmjs.org/@protobufjs/path/-/path-1.1.2.tgz",
- "integrity": "sha512-6JOcJ5Tm08dOHAbdR3GrvP+yUUfkjG5ePsHYczMFLq3ZmMkAD98cDgcT2iA1lJ9NVwFd4tH/iSSoe44YWkltEA==",
- "license": "BSD-3-Clause"
- },
- "node_modules/@protobufjs/pool": {
- "version": "1.1.0",
- "resolved": "https://registry.npmjs.org/@protobufjs/pool/-/pool-1.1.0.tgz",
- "integrity": "sha512-0kELaGSIDBKvcgS4zkjz1PeddatrjYcmMWOlAuAPwAeccUrPHdUqo/J6LiymHHEiJT5NrF1UVwxY14f+fy4WQw==",
- "license": "BSD-3-Clause"
- },
- "node_modules/@protobufjs/utf8": {
- "version": "1.1.0",
- "resolved": "https://registry.npmjs.org/@protobufjs/utf8/-/utf8-1.1.0.tgz",
- "integrity": "sha512-Vvn3zZrhQZkkBE8LSuW3em98c0FwgO4nxzv6OdSxPKJIEKY2bGbHn+mhGIPerzI4twdxaP8/0+06HBpwf345Lw==",
- "license": "BSD-3-Clause"
- },
- "node_modules/@types/node": {
- "version": "22.18.12",
- "resolved": "https://registry.npmjs.org/@types/node/-/node-22.18.12.tgz",
- "integrity": "sha512-BICHQ67iqxQGFSzfCFTT7MRQ5XcBjG5aeKh5Ok38UBbPe5fxTyE+aHFxwVrGyr8GNlqFMLKD1D3P2K/1ks8tog==",
- "license": "MIT",
- "dependencies": {
- "undici-types": "~6.21.0"
- }
- },
- "node_modules/@types/shimmer": {
- "version": "1.2.0",
- "resolved": "https://registry.npmjs.org/@types/shimmer/-/shimmer-1.2.0.tgz",
- "integrity": "sha512-UE7oxhQLLd9gub6JKIAhDq06T0F6FnztwMNRvYgjeQSBeMc1ZG/tA47EwfduvkuQS8apbkM/lpLpWsaCeYsXVg==",
- "license": "MIT"
- },
- "node_modules/acorn": {
- "version": "8.15.0",
- "resolved": "https://registry.npmjs.org/acorn/-/acorn-8.15.0.tgz",
- "integrity": "sha512-NZyJarBfL7nWwIq+FDL6Zp/yHEhePMNnnJ0y3qfieCrmNvYct8uvtiV41UvlSe6apAfk0fY1FbWx+NwfmpvtTg==",
- "license": "MIT",
- "bin": {
- "acorn": "bin/acorn"
- },
- "engines": {
- "node": ">=0.4.0"
- }
- },
- "node_modules/acorn-import-assertions": {
- "version": "1.9.0",
- "resolved": "https://registry.npmjs.org/acorn-import-assertions/-/acorn-import-assertions-1.9.0.tgz",
- "integrity": "sha512-cmMwop9x+8KFhxvKrKfPYmN6/pKTYYHBqLa0DfvVZcKMJWNyWLnaqND7dx/qn66R7ewM1UX5XMaDVP5wlVTaVA==",
- "deprecated": "package has been renamed to acorn-import-attributes",
- "license": "MIT",
- "peerDependencies": {
- "acorn": "^8"
- }
- },
- "node_modules/acorn-import-attributes": {
- "version": "1.9.5",
- "resolved": "https://registry.npmjs.org/acorn-import-attributes/-/acorn-import-attributes-1.9.5.tgz",
- "integrity": "sha512-n02Vykv5uA3eHGM/Z2dQrcD56kL8TyDb2p1+0P83PClMnC/nc+anbQRhIOWnSq4Ke/KvDPrY3C9hDtC/A3eHnQ==",
- "license": "MIT",
- "peerDependencies": {
- "acorn": "^8"
- }
- },
- "node_modules/cjs-module-lexer": {
- "version": "1.4.3",
- "resolved": "https://registry.npmjs.org/cjs-module-lexer/-/cjs-module-lexer-1.4.3.tgz",
- "integrity": "sha512-9z8TZaGM1pfswYeXrUpzPrkx8UnWYdhJclsiYMm6x/w5+nN+8Tf/LnAgfLGQCm59qAOxU8WwHEq2vNwF6i4j+Q==",
- "license": "MIT"
- },
- "node_modules/debug": {
- "version": "4.4.3",
- "resolved": "https://registry.npmjs.org/debug/-/debug-4.4.3.tgz",
- "integrity": "sha512-RGwwWnwQvkVfavKVt22FGLw+xYSdzARwm0ru6DhTVA3umU5hZc28V3kO4stgYryrTlLpuvgI9GiijltAjNbcqA==",
- "license": "MIT",
- "dependencies": {
- "ms": "^2.1.3"
- },
- "engines": {
- "node": ">=6.0"
- },
- "peerDependenciesMeta": {
- "supports-color": {
- "optional": true
- }
- }
- },
- "node_modules/function-bind": {
- "version": "1.1.2",
- "resolved": "https://registry.npmjs.org/function-bind/-/function-bind-1.1.2.tgz",
- "integrity": "sha512-7XHNxH7qX9xG5mIwxkhumTox/MIRNcOgDrxWsMt2pAr23WHp6MrRlN7FBSFpCpr+oVO0F744iUgR82nJMfG2SA==",
- "license": "MIT",
- "funding": {
- "url": "https://github.com/sponsors/ljharb"
- }
- },
- "node_modules/hasown": {
- "version": "2.0.2",
- "resolved": "https://registry.npmjs.org/hasown/-/hasown-2.0.2.tgz",
- "integrity": "sha512-0hJU9SCPvmMzIBdZFqNPXWa6dqh7WdH0cII9y+CyS8rG3nL48Bclra9HmKhVVUHyPWNH5Y7xDwAB7bfgSjkUMQ==",
- "license": "MIT",
- "dependencies": {
- "function-bind": "^1.1.2"
- },
- "engines": {
- "node": ">= 0.4"
- }
- },
- "node_modules/import-in-the-middle": {
- "version": "1.15.0",
- "resolved": "https://registry.npmjs.org/import-in-the-middle/-/import-in-the-middle-1.15.0.tgz",
- "integrity": "sha512-bpQy+CrsRmYmoPMAE/0G33iwRqwW4ouqdRg8jgbH3aKuCtOc8lxgmYXg2dMM92CRiGP660EtBcymH/eVUpCSaA==",
- "license": "Apache-2.0",
- "dependencies": {
- "acorn": "^8.14.0",
- "acorn-import-attributes": "^1.9.5",
- "cjs-module-lexer": "^1.2.2",
- "module-details-from-path": "^1.0.3"
- }
- },
- "node_modules/is-core-module": {
- "version": "2.16.1",
- "resolved": "https://registry.npmjs.org/is-core-module/-/is-core-module-2.16.1.tgz",
- "integrity": "sha512-UfoeMA6fIJ8wTYFEUjelnaGI67v6+N7qXJEvQuIGa99l4xsCruSYOVSQ0uPANn4dAzm8lkYPaKLrrijLq7x23w==",
- "license": "MIT",
- "dependencies": {
- "hasown": "^2.0.2"
- },
- "engines": {
- "node": ">= 0.4"
- },
- "funding": {
- "url": "https://github.com/sponsors/ljharb"
- }
- },
- "node_modules/long": {
- "version": "5.3.2",
- "resolved": "https://registry.npmjs.org/long/-/long-5.3.2.tgz",
- "integrity": "sha512-mNAgZ1GmyNhD7AuqnTG3/VQ26o760+ZYBPKjPvugO8+nLbYfX6TVpJPseBvopbdY+qpZ/lKUnmEc1LeZYS3QAA==",
- "license": "Apache-2.0"
- },
- "node_modules/module-details-from-path": {
- "version": "1.0.4",
- "resolved": "https://registry.npmjs.org/module-details-from-path/-/module-details-from-path-1.0.4.tgz",
- "integrity": "sha512-EGWKgxALGMgzvxYF1UyGTy0HXX/2vHLkw6+NvDKW2jypWbHpjQuj4UMcqQWXHERJhVGKikolT06G3bcKe4fi7w==",
- "license": "MIT"
- },
- "node_modules/ms": {
- "version": "2.1.3",
- "resolved": "https://registry.npmjs.org/ms/-/ms-2.1.3.tgz",
- "integrity": "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==",
- "license": "MIT"
- },
- "node_modules/openai": {
- "version": "6.7.0",
- "resolved": "https://registry.npmjs.org/openai/-/openai-6.7.0.tgz",
- "integrity": "sha512-mgSQXa3O/UXTbA8qFzoa7aydbXBJR5dbLQXCRapAOtoNT+v69sLdKMZzgiakpqhclRnhPggPAXoniVGn2kMY2A==",
- "license": "Apache-2.0",
- "bin": {
- "openai": "bin/cli"
- },
- "peerDependencies": {
- "ws": "^8.18.0",
- "zod": "^3.25 || ^4.0"
- },
- "peerDependenciesMeta": {
- "ws": {
- "optional": true
- },
- "zod": {
- "optional": true
- }
- }
- },
- "node_modules/path-parse": {
- "version": "1.0.7",
- "resolved": "https://registry.npmjs.org/path-parse/-/path-parse-1.0.7.tgz",
- "integrity": "sha512-LDJzPVEEEPR+y48z93A0Ed0yXb8pAByGWo/k5YYdYgpY2/2EsOsksJrq7lOHxryrVOn1ejG6oAp8ahvOIQD8sw==",
- "license": "MIT"
- },
- "node_modules/protobufjs": {
- "version": "7.5.4",
- "resolved": "https://registry.npmjs.org/protobufjs/-/protobufjs-7.5.4.tgz",
- "integrity": "sha512-CvexbZtbov6jW2eXAvLukXjXUW1TzFaivC46BpWc/3BpcCysb5Vffu+B3XHMm8lVEuy2Mm4XGex8hBSg1yapPg==",
- "hasInstallScript": true,
- "license": "BSD-3-Clause",
- "dependencies": {
- "@protobufjs/aspromise": "^1.1.2",
- "@protobufjs/base64": "^1.1.2",
- "@protobufjs/codegen": "^2.0.4",
- "@protobufjs/eventemitter": "^1.1.0",
- "@protobufjs/fetch": "^1.1.0",
- "@protobufjs/float": "^1.0.2",
- "@protobufjs/inquire": "^1.1.0",
- "@protobufjs/path": "^1.1.2",
- "@protobufjs/pool": "^1.1.0",
- "@protobufjs/utf8": "^1.1.0",
- "@types/node": ">=13.7.0",
- "long": "^5.0.0"
- },
- "engines": {
- "node": ">=12.0.0"
- }
- },
- "node_modules/require-in-the-middle": {
- "version": "7.5.2",
- "resolved": "https://registry.npmjs.org/require-in-the-middle/-/require-in-the-middle-7.5.2.tgz",
- "integrity": "sha512-gAZ+kLqBdHarXB64XpAe2VCjB7rIRv+mU8tfRWziHRJ5umKsIHN2tLLv6EtMw7WCdP19S0ERVMldNvxYCHnhSQ==",
- "license": "MIT",
- "dependencies": {
- "debug": "^4.3.5",
- "module-details-from-path": "^1.0.3",
- "resolve": "^1.22.8"
- },
- "engines": {
- "node": ">=8.6.0"
- }
- },
- "node_modules/resolve": {
- "version": "1.22.11",
- "resolved": "https://registry.npmjs.org/resolve/-/resolve-1.22.11.tgz",
- "integrity": "sha512-RfqAvLnMl313r7c9oclB1HhUEAezcpLjz95wFH4LVuhk9JF/r22qmVP9AMmOU4vMX7Q8pN8jwNg/CSpdFnMjTQ==",
- "license": "MIT",
- "dependencies": {
- "is-core-module": "^2.16.1",
- "path-parse": "^1.0.7",
- "supports-preserve-symlinks-flag": "^1.0.0"
- },
- "bin": {
- "resolve": "bin/resolve"
- },
- "engines": {
- "node": ">= 0.4"
- },
- "funding": {
- "url": "https://github.com/sponsors/ljharb"
- }
- },
- "node_modules/semver": {
- "version": "7.7.3",
- "resolved": "https://registry.npmjs.org/semver/-/semver-7.7.3.tgz",
- "integrity": "sha512-SdsKMrI9TdgjdweUSR9MweHA4EJ8YxHn8DFaDisvhVlUOe4BF1tLD7GAj0lIqWVl+dPb/rExr0Btby5loQm20Q==",
- "license": "ISC",
- "bin": {
- "semver": "bin/semver.js"
- },
- "engines": {
- "node": ">=10"
- }
- },
- "node_modules/shimmer": {
- "version": "1.2.1",
- "resolved": "https://registry.npmjs.org/shimmer/-/shimmer-1.2.1.tgz",
- "integrity": "sha512-sQTKC1Re/rM6XyFM6fIAGHRPVGvyXfgzIDvzoq608vM+jeyVD0Tu1E6Np0Kc2zAIFWIj963V2800iF/9LPieQw==",
- "license": "BSD-2-Clause"
- },
- "node_modules/supports-preserve-symlinks-flag": {
- "version": "1.0.0",
- "resolved": "https://registry.npmjs.org/supports-preserve-symlinks-flag/-/supports-preserve-symlinks-flag-1.0.0.tgz",
- "integrity": "sha512-ot0WnXS9fgdkgIcePe6RHNk1WA8+muPa6cSjeR3V8K27q9BB1rTE3R1p7Hv0z1ZyAc8s6Vvv8DIyWf681MAt0w==",
- "license": "MIT",
- "engines": {
- "node": ">= 0.4"
- },
- "funding": {
- "url": "https://github.com/sponsors/ljharb"
- }
- },
- "node_modules/typescript": {
- "version": "5.9.3",
- "resolved": "https://registry.npmjs.org/typescript/-/typescript-5.9.3.tgz",
- "integrity": "sha512-jl1vZzPDinLr9eUt3J/t7V6FgNEw9QjvBPdysz9KfQDD41fQrC2Y4vKQdiaUpFT4bXlb1RHhLpp8wtm6M5TgSw==",
- "dev": true,
- "license": "Apache-2.0",
- "bin": {
- "tsc": "bin/tsc",
- "tsserver": "bin/tsserver"
- },
- "engines": {
- "node": ">=14.17"
- }
- },
- "node_modules/undici-types": {
- "version": "6.21.0",
- "resolved": "https://registry.npmjs.org/undici-types/-/undici-types-6.21.0.tgz",
- "integrity": "sha512-iwDZqg0QAGrg9Rav5H4n0M64c3mkR59cJ6wQp+7C4nI0gsmExaedaYLNO44eT4AtBBwjbTiGPMlt2Md0T9H9JQ==",
- "license": "MIT"
- }
- }
-}
diff --git a/examples/node/observability-opentelemetry/package.json b/examples/node/observability-opentelemetry/package.json
deleted file mode 100644
index bc74d98dae..0000000000
--- a/examples/node/observability-opentelemetry/package.json
+++ /dev/null
@@ -1,25 +0,0 @@
-{
- "name": "agenta-opentelemetry-quickstart",
- "version": "1.0.0",
- "description": "Quick start example for using OpenTelemetry with Agenta",
- "type": "module",
- "scripts": {
- "start": "node --import ./instrumentation.js app.js"
- },
- "dependencies": {
- "@arizeai/openinference-instrumentation-openai": "^3.2.3",
- "@arizeai/openinference-semantic-conventions": "^2.1.2",
- "@opentelemetry/api": "^1.9.0",
- "@opentelemetry/exporter-trace-otlp-proto": "^0.54.0",
- "@opentelemetry/instrumentation": "^0.54.0",
- "@opentelemetry/resources": "^1.28.0",
- "@opentelemetry/sdk-trace-base": "^1.28.0",
- "@opentelemetry/sdk-trace-node": "^1.28.0",
- "@opentelemetry/semantic-conventions": "^1.28.0",
- "openai": "^6.7.0"
- },
- "devDependencies": {
- "@types/node": "^22.0.0",
- "typescript": "^5.7.0"
- }
-}
diff --git a/examples/node/observability-opentelemetry/test.sh b/examples/node/observability-opentelemetry/test.sh
deleted file mode 100755
index 56fb36d592..0000000000
--- a/examples/node/observability-opentelemetry/test.sh
+++ /dev/null
@@ -1,29 +0,0 @@
-#!/bin/bash
-# Test script for OpenTelemetry example
-
-# Check if required environment variables are set
-if [ -z "$AGENTA_API_KEY" ]; then
- echo "❌ Error: AGENTA_API_KEY is not set"
- echo " Set it with: export AGENTA_API_KEY='your_key_here'"
- exit 1
-fi
-
-if [ -z "$OPENAI_API_KEY" ]; then
- echo "❌ Error: OPENAI_API_KEY is not set"
- echo " Set it with: export OPENAI_API_KEY='your_key_here'"
- exit 1
-fi
-
-# Set default AGENTA_HOST if not provided
-if [ -z "$AGENTA_HOST" ]; then
- export AGENTA_HOST="https://cloud.staging.agenta.ai"
- echo "ℹ️ Using default AGENTA_HOST: $AGENTA_HOST"
-fi
-
-echo "✅ Environment variables configured"
-echo ""
-echo "🚀 Running example..."
-echo ""
-
-npm start
-
diff --git a/examples/python/custom_workflows/ai-code-reviewer/ai-code-reviewer.py b/examples/python/custom_workflows/ai-code-reviewer/ai-code-reviewer.py
index 2703c420d2..4ab89887a1 100644
--- a/examples/python/custom_workflows/ai-code-reviewer/ai-code-reviewer.py
+++ b/examples/python/custom_workflows/ai-code-reviewer/ai-code-reviewer.py
@@ -89,4 +89,6 @@ def generate_critique(pr_url: str):
if __name__ == "__main__":
import uvicorn
- uvicorn.run("agenta:app", host="0.0.0.0", port=8000, reload=True)
+ uvicorn.run(
+ "agenta.sdk.decorators.routing:app", host="0.0.0.0", port=8000, reload=True
+ )
diff --git a/examples/python/custom_workflows/chain_of_prompts/cop.py b/examples/python/custom_workflows/chain_of_prompts/cop.py
index 1af034895d..163daf542a 100644
--- a/examples/python/custom_workflows/chain_of_prompts/cop.py
+++ b/examples/python/custom_workflows/chain_of_prompts/cop.py
@@ -38,4 +38,6 @@ def generate(blog_post: str):
if __name__ == "__main__":
import uvicorn
- uvicorn.run("agenta:app", host="0.0.0.0", port=8000, reload=True)
+ uvicorn.run(
+ "agenta.sdk.decorators.routing:app", host="0.0.0.0", port=8000, reload=True
+ )
diff --git a/examples/python/custom_workflows/rag-docs-qa/README.md b/examples/python/custom_workflows/rag-docs-qa/README.md
index e02e4dd77c..fe844b9bd3 100644
--- a/examples/python/custom_workflows/rag-docs-qa/README.md
+++ b/examples/python/custom_workflows/rag-docs-qa/README.md
@@ -61,4 +61,4 @@ python query.py
## Notes:
-- `generate_testset.py` is used to generate a testset of questions based on the documentation for evaluation.
+- `generate_test_set.py` is used to generate a test set of questions based on the documentation for evaluation.
diff --git a/examples/python/custom_workflows/rag-docs-qa/generate_test_set.py b/examples/python/custom_workflows/rag-docs-qa/generate_test_set.py
new file mode 100644
index 0000000000..3210a8c244
--- /dev/null
+++ b/examples/python/custom_workflows/rag-docs-qa/generate_test_set.py
@@ -0,0 +1,99 @@
+import os
+import glob
+from pathlib import Path
+import pandas as pd
+from dotenv import load_dotenv
+from litellm import completion
+import frontmatter
+import tqdm
+import json
+
+# Load environment variables
+load_dotenv()
+
+
+def get_files(docs_path):
+ """Get all markdown files recursively."""
+ return
+
+
+def extract_content(file_path):
+ """Extract content from markdown file."""
+ with open(file_path, "r", encoding="utf-8") as f:
+ post = frontmatter.load(f)
+ # Get title from frontmatter or filename
+ title = post.get("title", Path(file_path).stem)
+ # Get content without frontmatter
+ content = post.content
+ return title, content
+
+
+def generate_questions(title, content):
+ """Generate questions using OpenAI."""
+ system_prompt = """You are a helpful assistant that generates questions based on documentation content.
+ Generate 5 questions that could be answered using the provided documentation.
+ Your response must be a JSON object with a single key "questions" containing an array of strings."""
+
+ user_prompt = f"""
+ Title: {title}
+
+ Content: {content} # Limit content length to avoid token limits
+
+ Generate 5 questions about this documentation. Put yourself in the shoes of a user attempting to 1) figure how to use the product for a use case 2) troubleshoot an issue 3) learn about the features of the product.
+ The user in this case is a technical user (AI engineer) who is trying to build an llm application.
+ The user would write the questions they would ask in a chat with a human. Therefore, not all questions will be clear and well written.
+ """
+
+ try:
+ response = completion(
+ model="gpt-3.5-turbo-0125", # Using the latest model that supports JSON mode
+ messages=[
+ {"role": "system", "content": system_prompt},
+ {"role": "user", "content": user_prompt},
+ ],
+ response_format={"type": "json_object"},
+ )
+
+ # Check if the response was complete
+ if response.choices[0].finish_reason == "length":
+ print(f"Warning: Response was truncated for {title}")
+ return []
+
+ # Parse JSON response - no need for eval()
+ result = json.loads(response.choices[0].message.content)
+ return result["questions"]
+
+ except Exception as e:
+ print(f"Error generating questions for {title}: {str(e)}")
+ return []
+
+
+def main():
+ docs_path = os.getenv("DOCS_PATH")
+ if not docs_path:
+ raise ValueError("DOCS_PATH environment variable not set")
+
+ # Get all files
+ files = glob.glob(os.path.join(docs_path, "**/*.mdx"), recursive=True)
+ all_questions = []
+ # Process each file
+ for file_path in tqdm.tqdm(files, desc="Processing documentation files"):
+ if "/reference/api" in file_path:
+ # skip api docs
+ continue
+ try:
+ title, content = extract_content(file_path)
+ questions = generate_questions(title, content)
+ all_questions.extend(questions)
+ except Exception as e:
+ print(f"Error processing {file_path}: {str(e)}")
+ continue
+
+ # Save to CSV
+ df = pd.DataFrame({"query": all_questions})
+ df.to_csv("test_set.csv", index=False, lineterminator="\n")
+ print(f"Generated {len(all_questions)} questions and saved to test_set.csv")
+
+
+if __name__ == "__main__":
+ main()
diff --git a/examples/python/custom_workflows/rag-docs-qa/generate_testset.py b/examples/python/custom_workflows/rag-docs-qa/generate_testset.py
deleted file mode 100644
index 0eb362fd6d..0000000000
--- a/examples/python/custom_workflows/rag-docs-qa/generate_testset.py
+++ /dev/null
@@ -1,99 +0,0 @@
-import os
-import glob
-from pathlib import Path
-import pandas as pd
-from dotenv import load_dotenv
-from litellm import completion
-import frontmatter
-import tqdm
-import json
-
-# Load environment variables
-load_dotenv()
-
-
-def get_files(docs_path):
- """Get all markdown files recursively."""
- return
-
-
-def extract_content(file_path):
- """Extract content from markdown file."""
- with open(file_path, "r", encoding="utf-8") as f:
- post = frontmatter.load(f)
- # Get title from frontmatter or filename
- title = post.get("title", Path(file_path).stem)
- # Get content without frontmatter
- content = post.content
- return title, content
-
-
-def generate_questions(title, content):
- """Generate questions using OpenAI."""
- system_prompt = """You are a helpful assistant that generates questions based on documentation content.
- Generate 5 questions that could be answered using the provided documentation.
- Your response must be a JSON object with a single key "questions" containing an array of strings."""
-
- user_prompt = f"""
- Title: {title}
-
- Content: {content} # Limit content length to avoid token limits
-
- Generate 5 questions about this documentation. Put yourself in the shoes of a user attempting to 1) figure how to use the product for a use case 2) troubleshoot an issue 3) learn about the features of the product.
- The user in this case is a technical user (AI engineer) who is trying to build an llm application.
- The user would write the questions they would ask in a chat with a human. Therefore, not all questions will be clear and well written.
- """
-
- try:
- response = completion(
- model="gpt-3.5-turbo-0125", # Using the latest model that supports JSON mode
- messages=[
- {"role": "system", "content": system_prompt},
- {"role": "user", "content": user_prompt},
- ],
- response_format={"type": "json_object"},
- )
-
- # Check if the response was complete
- if response.choices[0].finish_reason == "length":
- print(f"Warning: Response was truncated for {title}")
- return []
-
- # Parse JSON response - no need for eval()
- result = json.loads(response.choices[0].message.content)
- return result["questions"]
-
- except Exception as e:
- print(f"Error generating questions for {title}: {str(e)}")
- return []
-
-
-def main():
- docs_path = os.getenv("DOCS_PATH")
- if not docs_path:
- raise ValueError("DOCS_PATH environment variable not set")
-
- # Get all files
- files = glob.glob(os.path.join(docs_path, "**/*.mdx"), recursive=True)
- all_questions = []
- # Process each file
- for file_path in tqdm.tqdm(files, desc="Processing documentation files"):
- if "/reference/api" in file_path:
- # skip api docs
- continue
- try:
- title, content = extract_content(file_path)
- questions = generate_questions(title, content)
- all_questions.extend(questions)
- except Exception as e:
- print(f"Error processing {file_path}: {str(e)}")
- continue
-
- # Save to CSV
- df = pd.DataFrame({"query": all_questions})
- df.to_csv("testset.csv", index=False, lineterminator="\n")
- print(f"Generated {len(all_questions)} questions and saved to testset.csv")
-
-
-if __name__ == "__main__":
- main()
diff --git a/examples/python/custom_workflows/rag-docs-qa/query.py b/examples/python/custom_workflows/rag-docs-qa/query.py
index 2bee1789e3..f5d494bac7 100644
--- a/examples/python/custom_workflows/rag-docs-qa/query.py
+++ b/examples/python/custom_workflows/rag-docs-qa/query.py
@@ -164,4 +164,6 @@ def generate(query: str):
if __name__ == "__main__":
import uvicorn
- uvicorn.run("agenta:app", host="0.0.0.0", port=8000, reload=True)
+ uvicorn.run(
+ "agenta.sdk.decorators.routing:app", host="0.0.0.0", port=8000, reload=True
+ )
diff --git a/examples/python/custom_workflows/rag-docs-qa/test_set.csv b/examples/python/custom_workflows/rag-docs-qa/test_set.csv
new file mode 100644
index 0000000000..9c7d6028b4
--- /dev/null
+++ b/examples/python/custom_workflows/rag-docs-qa/test_set.csv
@@ -0,0 +1,311 @@
+query,correct_answer
+How can I deploy Agenta locally in production mode?,
+What command should I run to update Agenta to the latest version?,
+How do I start Agenta in development mode?,
+What steps should I take to troubleshoot if I encounter a port conflict issue?,
+Where can I seek help or report issues related to Agenta?,
+How can I deploy Agenta on Kubernetes for my AI model hosting use case?,
+"I'm having trouble accessing the Kubernetes deployment feature in Agenta Enterprise, how can I troubleshoot this?",
+Can Agenta Enterprise on Kubernetes support scaling my AI models automatically based on demand?,
+What tools are provided by Agenta Enterprise to manage multiple users and teams?,
+Can you provide more details about Agenta Enterprise's early access stage for select partners?,
+How can I deploy Agenta on AWS EC2 using Terraform?,
+What are the open ports created by the Terraform module for Agenta on AWS EC2?,
+How do I serve a variant to an instance after hosting Agenta?,
+What steps are involved in SSH-ing into the instance where Agenta is hosted?,
+How can I delete all the resources created by Terraform for Agenta on AWS?,
+How can I deploy Agenta on Google Cloud Engine using Terraform?,
+What are the prerequisites for deploying Agenta on Google Cloud Engine?,
+How do I SSH into the instance after deploying Agenta on Google Cloud Engine?,
+What security considerations should I keep in mind when allowing SSH access to the instance?,
+How can I delete all the resources created by Terraform for Agenta on Google Cloud Engine?,
+How can I host Agenta on a remote server like an AWS EC2 instance?,
+What are the prerequisites for deploying Agenta on a remote server?,
+How do I obtain the public IP of my AWS EC2 instance?,
+What environment variables do I need to set before launching the Agenta server?,
+"After starting the Agenta server, how can I verify that it is running correctly on the remote server?",
+How can I apply schema migrations to my PostgreSQL database using the provided instructions?,
+What command should I run to ensure that Alembic looks for the configuration file and other necessary files in the specified directory before executing?,
+"If I need to revert a schema migration due to encountered issues, how can I do that according to the documentation?",
+"Is there a way to automate the application of migrations without manually running the upgrade command? If so, how can I achieve this?",
+"After completing the migration, what should I do to verify data integrity in PostgreSQL and ensure that everything works fine?",
+How can I back up my MongoDB database before running the migration script?,
+What Docker command should I use to start the local instance of Agenta for migration?,
+"After initiating the migration, how can I access the running docker containers?",
+How can I verify the data integrity in PostgreSQL post migration?,
+"In case I encounter issues during migration, how can I revert the migration and what should I do next?",
+How can I upgrade to the latest version of Agenta using Docker?,
+What database does Agenta now use starting from version 0.19?,
+How can I access the backend Docker container for Agenta?,
+What is Beanie and how does it relate to MongoDB ODM for Python?,
+What are the steps involved in performing a database migration using Beanie with Agenta backend system?,
+Where can I find answers to common questions about Agenta?,
+How can I get community support for Agenta? Is there a Slack community?,
+What is the recommended channel for reporting bugs in Agenta?,
+"As a Pro plan cloud user, how can I access direct team support?",
+"If I'm interested in demos or have sales inquiries, how can I schedule a call with the founders?",
+How can I add new users to my workspace?,
+What are the different user roles available and what are their respective rights?,
+"As a Workspace Admin, what are the limitations in terms of managing the workspace?",
+What is the role of a Deployment Manager and what tasks do they handle within the workspace?,
+How can I switch between different workspaces if I am a member of multiple workspaces?,
+How do I format backend and CLI code using Black in specific directories?,
+"What are the main steps to contribute to Agenta, especially when picking an issue?",
+Can you guide me through running backend tests locally before making a pull request?,
+How can I update the .env.local file in the frontend directory to include my OpenAI API Key?,
+What are the rules regarding issue assignment and PR activity to prevent zombie issues in the Agenta project?,
+How can I use my own version of Agenta CLI or SDK instead of the installed one?,
+What steps can I take if I'm unable to run Agenta in my terminal even after following the provided instructions?,
+How can I quickly work and test a new type of parameter like IntParam in the SDK?,
+Are there any specific steps needed for working on the backend code in Agenta?,
+How can I efficiently debug the backend code in Agenta using Docker or Visual Studio Code?,
+How can I report a bug using the bug report template?,
+Where can I find the network logs in the browser and why are they important when reporting an issue?,
+What command should I run to get information about Docker containers and how can I provide this information when filing an issue?,
+"When reporting an issue related to a specific llm app, what are the steps to provide container information using Docker Desktop or the terminal?",
+Why is providing network logs and Docker container information crucial when filing an issue?,
+How can I start monitoring and understanding the behavior of my LLM application with Agenta?,
+What are the advantages of using OpenTelemetry (OTel) with Agenta's observability features?,
+What is the relationship between traces and spans in the context of Agenta?,
+Where can I find a Quick Start guide to get started with observability in Agenta?,
+How can I learn to instrument specific applications like OpenAI or LangChain with Agenta?,
+How can I enable tracing for OpenAI calls using Agenta SDK?,
+What packages do I need to install to set up observability for an OpenAI application running locally?,
+How can I configure environment variables for Agenta OSS running locally?,
+Where can I find the captured traces of my application's requests in the Agenta UI?,
+Is tracing enabled by default if I create an application through the Agenta UI?,
+How can I install the Agenta SDK?,
+What is the purpose of the @ag.instrument() decorator in Agenta SDK?,
+How can I add additional metadata to a span in Agenta?,
+"Can you explain how to link spans to applications, variants, and environments in Agenta?",
+What options are available for redacting sensitive data in Agenta SDK?,
+How do I install LiteLLM and Agenta SDK?,
+What are the different environments I can configure while setting up LiteLLM with Agenta?,
+How can I initialize Agenta SDK in my LiteLLM application?,
+What does the @ag.instrument() decorator do in the code example provided?,
+Why is it important to set up the callback handler for LiteLLM while using Agenta?,
+How can I install the required packages for using LangChain with Agenta?,
+What environment variables do I need to configure when using Agenta Cloud or Enterprise?,
+What steps are involved in the code example provided for a LangChain application?,
+How do I initialize Agenta in my LangChain application?,
+Why is it important to call 'LangchainInstrumentor().instrument()' before running my LangChain application?,
+How can I instrument OpenAI API calls with Agenta using `opentelemetry-instrumentation-openai` package?,
+What packages do I need to install to instrument OpenAI API calls with Agenta using `opentelemetry-instrumentation-openai` package?,
+How do I configure environment variables for Agenta Cloud or Enterprise when instrumenting OpenAI API calls?,
+What decorator can I use to monitor multiple calls in a function or workflow as a single trace?,
+How can I associate traces with specific parts of an Agenta project when instrumenting functions?,
+How do I install the required packages for using Instructor with Agenta?,
+What environment variables do I need to configure for Agenta Cloud or Enterprise setup?,
+What must I ensure the order of when instrumenting OpenAI and creating the Instructor client?,
+Why is it mentioned to instrument OpenAI before creating the Instructor client?,
+"What does the @ag.instrument(spankind=""WORKFLOW"") decorator do in the code example?",
+How can I experiment and compare prompts using Agenta?,
+What types of workflows does Agenta support for prompt management and evaluation?,
+How can I collaborate with product teams using Agenta for prompt engineering and evaluation?,
+What are the different ways to deploy an application with Agenta?,
+"Is Agenta compatible with various LLM app architectures and model providers? If so, how?",
+How can I create a new LLM app using an existing template in Agenta?,
+Where can I find the API endpoint after deploying my application in Agenta?,
+What steps do I need to follow to test the application in the playground?,
+How do I add my OpenAI API keys when self-hosting Agenta?,
+What are some next steps after creating my first LLM application in Agenta?,
+How can I create custom templates for my LLM-powered applications in Agenta?,
+What are the predefined environments in Agenta and their purposes?,
+"Explain the relationship between variants, versions, and commits in Agenta.",
+Can you describe the difference between Completion Application Template and Chat Application Template in Agenta?,
+What can I do if the deployment of a variant to an environment does not update automatically in Agenta?,
+How does Agenta enable rapid experimentation and evaluation for LLM applications?,
+What is the significance of Agenta treating each application as a microservice?,
+How does Agenta handle the separation of application logic and configuration?,
+What role does Agenta's backend play in managing applications and configurations?,
+Can you explain the purpose and functionality of Agenta's SDK for a Python library?,
+How can I collaborate with subject matter experts using custom workflows in Agenta?,
+What is the main problem with traditional prompt playgrounds?,
+Can I trust the outputs of traditional prompt playgrounds?,
+How does Agenta's Custom Workflows simplify debugging for AI engineers?,
+What frameworks and models are compatible with Agenta?,
+How can I create a custom workflow with two prompts using Agenta?,
+What are the steps to serve an application in Agenta using the CLI?,
+How do I initialize the Agenta SDK in my Python code?,
+What is the purpose of the `CoPConfig` class in the provided code snippet?,
+Can you explain the concept of entry points in Agenta and how they are used in the code example?,
+How can I configure evaluators for my LLM application?,
+What tools are available for creating test sets for evaluation?,
+Can I run evaluations directly from the web UI in Agenta?,
+What types of evaluators are available for classification/entity extraction in LLM applications?,
+How can I evaluate the faithfulness of outputs in RAG workflows?,
+How can I create a test set for evaluation in agenta SDK?,
+What is the purpose of creating evaluators in the agenta SDK?,
+How can I run an evaluation job using the agenta SDK?,
+What is the function of the rate limit configuration in evaluation jobs?,
+How can I retrieve the detailed results of an evaluation using the agenta SDK?,
+How can I initiate a single model evaluation in the Human Evaluation feature?,
+What steps are involved in starting a new evaluation with the single model test?,
+How do I compare the performance of two different variants manually using A/B Test in the Human Evaluation feature?,
+Can collaborators be invited to collaborate on an A/B Test evaluation in the Human Evaluation feature?,
+Is there a way to switch between card and table view in the evaluation process? How can this be done?,
+How do I create a test set in Agenta using a CSV or JSON file?,
+What is the default column name for the reference answer in a CSV test set?,
+What is the structure of a valid JSON file for a test set in Agenta?,
+How can I add data to a test set from the playground in Agenta?,
+"In Agenta, how can I upload a test set using the API?",
+How do I start a new evaluation in the UI?,
+What parameters can be specified when setting up a new evaluation?,
+What are the advanced configuration options available for adjusting batching and retry parameters?,
+How can I analyze the results of an evaluation in more detail?,
+"Can I compare multiple evaluations from the same test set in the UI? If yes, how?",
+How can I configure evaluators for my LLM application?,
+What are the inputs that evaluators typically take?,
+Which button should I click to create a new evaluator?,
+Can I create custom evaluators for my LLM application?,
+What is the purpose of mappings evaluator's inputs to the LLM data?,
+How can I create a webhook evaluator for a specific use case in Agenta?,
+What are the limitations in terms of security measures when using webhook evaluators?,
+What input parameters are required for a webhook evaluator in Agenta?,
+What is the expected format of the webhook request body for a webhook evaluator?,
+What should the webhook response body contain in order to be considered properly-formatted?,
+How does the Exact Match evaluator determine if the model's output is correct?,
+What does the Contains JSON evaluator check for in the model's output?,
+How does the JSON Field Match evaluator compare specific fields within JSON data?,
+Can you explain the process of JSON Diff Match evaluation?,
+What configuration options are available for the JSON Diff Match evaluator and how do they affect the comparison process?,
+How can I assess the performance of LLMs by identifying specific patterns within the output generated by the model?,
+What is the purpose of the 'Regex Test' evaluator in Agenta and how does it work?,
+Could you provide an example of using the 'Starts With' evaluator in Agenta and its significance?,
+How does the 'Contains Any' evaluator in Agenta differ from the 'Contains All' evaluator?,
+In what situations would I use the 'Ends With' evaluator in Agenta and how does it handle case sensitivity?,
+How can I create a custom evaluator in Agenta?,
+What is the function signature for the 'evaluate' function in a custom evaluator?,
+What is the purpose of the 'evaluate' function in a custom evaluator?,
+Which language is used for writing custom evaluators in Agenta?,
+What ranges of scores can the 'evaluate' function return?,
+How can I configure and use RAG evaluators in my custom-built application?,
+What version of the Python SDK is required to access internal variables and intermediate outputs for RAG Evaluators?,
+Where can I find the source code for a simple RAG Application that fetches movies and generates summaries?,
+How can I troubleshoot if the RAG Evaluator and `view trace` are not accessible in Agenta OSS?,
+Which utility can I use in Agenta to add internal variables (internals) to stages of a workflow?,
+How does the Similarity Match evaluator determine a match between the generated output and the correct answer?,
+What configuration parameter is needed for the Semantic Similarity Match evaluator?,
+What is the cost associated with using the Semantic Similarity Match evaluator that uses OpenAI embeddings?,
+In what use cases is the Levenshtein Distance evaluator particularly useful?,
+Can you explain how the Levenshtein Distance evaluator works for comparing text outputs?,
+How can I configure the prompt for LLM-as-a-Judge evaluation?,
+"What variables can be used to reference inputs, outputs, and reference answers in the prompt?",
+What is the default prompt used for the country expert demo application?,
+"Which models are supported by LLM-as-a-Judge, and how can I select a specific model?",
+Where do I need to set my OpenAI or Anthropic API key to use LLM-as-a-Judge?,
+How can I create a test set using the SDK?,
+What is the process for creating and configuring an evaluator programmatically?,
+How can I check the status of an evaluation run?,
+Where can I find the list of evaluator keys and configurations?,
+What should I do if I encounter rate limits during the evaluation process?,
+How can I create a new prompt using the Agenta SDK?,
+How do I deploy changes to the production environment using the SDK?,
+"What is the structure used by Agenta for prompt versioning, and how does it differ from traditional versioning systems?",
+"In case of an issue with deploying changes using the SDK, what steps can I take to troubleshoot?",
+Can you explain the concept of variants in Agenta and how they are similar to branches in Git?,
+How can I set up tracing for a RAG application in LangChain using Agenta?,
+What is the purpose of tracing in LLM applications?,
+"How do I install the necessary dependencies for LangChain, Agenta, and instrumentation?",
+Can you provide an example of a Q&A RAG application in LangChain and how it works?,
+How can I retrieve and generate answers using relevant snippets of documentation in Agenta?,
+How can I fetch the diff for a GitHub pull request using Python?,
+What role does LiteLLM play in the AI assistant workflow described in the tutorial?,
+How can I add observability to my LLM application using Agenta?,
+What is the purpose of creating an LLM playground for the application?,
+How can I deploy my AI assistant to production using Agenta?,
+How can I version prompts in Agenta?,
+What capabilities does Agenta provide for prompt management?,
+Why do I need a prompt management system?,
+What is the purpose of configuration management in Agenta?,
+How can I publish a prompt to an endpoint from the web UI?,
+How can I create a prompt using the web UI?,
+Why do I need to publish a variant to a deployment?,
+How can I integrate the prompt configuration with my Python code using the Agenta SDK?,
+"Can I revert to a previous deployment version, and if so, how?",
+What is the purpose of using Pydantic for schema validation in the SDK integration process?,
+How can I create reusable prompts using prompt templates in the agenta playground?,
+Where can I find the option to add new inputs to the LLM app in the playground?,
+"What is the process for creating a new variant of an application in agenta, and where can I provide a new name for the variant?",
+"How can I test a variant dynamically in the playground, and is there a way to run all inputs in a test set?",
+"In the agenta playground, how do I compare variants side by side? Can I interact with different variants simultaneously in a chat application?",
+How can I create a new prompt using the SDK?,
+What is the structure followed by Agenta for prompt versioning?,
+How can I delete a variant in Agenta? Is this action reversible?,
+Can I list all variants of an application using the SDK? How can I do that?,
+What is the default behavior for fetching configurations in Agenta?,
+How can I use the Agenta SDK to invoke the deployed version of my prompt?,
+Where can I find the call to invoke the deployed version of my prompt within the Agenta UI?,
+What parameters do I need to provide when invoking a deployed prompt through the REST API?,
+What does the 'inputs' dictionary in the parameters contain and how should it be structured?,
+How can I control which environment version of my prompt is being called when using the REST API?,
+How do I fetch the deployed version of my prompt in my code using the Agenta SDK?,
+What Python SDK do I need to install for using the Agenta SDK?,
+What environment variables need to be set up for fetching prompts?,
+"In the sample output, what is the value of 'model' fetched from staging configuration?",
+Which slugs should I provide when fetching prompts using Agenta SDK?,
+How can Agenta help me manage prompts in my application?,
+What are the advantages of using Agenta as a prompt management system?,
+What considerations should I keep in mind when integrating observability manually with Agenta?,
+When should I consider using Agenta as a middleware/model proxy?,
+Does using Agenta as a proxy add any latency to the response?,
+How can I install the agenta CLI tool?,
+What is the command to install agenta using pip?,
+Where can I find a quick usage guide for the agenta CLI?,
+How do I get an overview of the main commands and capabilities of agenta CLI?,
+Is there a tutorial available for deploying an LLM app from code using agenta CLI?,
+How can I initialize a new Agenta project using the CLI?,
+"What options can be provided with the 'agenta init' command, and how do they affect the project initialization process?",
+"For the 'agenta variant list' command, where should it be executed, and what information does it provide?",
+What are the steps involved in removing a variant from an application using the 'agenta variant remove' command?,
+"How can I deploy an application variant to the Agenta platform using the 'agenta variant serve' command, and what additional options are available for deployment?",
+How can I create a new variant with initial configuration parameters?,
+What method should I use to commit changes to an existing variant?,
+Which method is used to deploy a variant to a specific environment?,
+How can I retrieve the version history of a variant?,
+What are the available predefined environments for deployment?,
+How can I store additional data within the current span in a trace?,
+What utility function can I use to validate if a string is a valid attribute key?,
+How can I link spans to specific Agenta resources using the SDK?,
+What method should I use to set the status of a span in the CustomSpan class?,
+When should I apply the @ag.instrument() decorator in my Python code?,
+How can I expose specific stages of LLM workflows as API endpoints using the Custom Workflows SDK?,
+What types of fields are accepted in the configuration schema when defining a configuration for a function?,
+How can I define fields with constraints and defaults in the configuration schema when using `@ag.route` or `@ag.entrypoint` decorators?,
+"How are different field types represented in the Agenta playground UI, and what are the input methods for each type?",
+How can I retrieve configuration information from the route context using `ag.ConfigManager.get_from_route()` and how is this configuration used in a function?,
+How do I experiment with AI applications using the agenta SDK?,
+What is the purpose of the deprecated SDK v2 mentioned on the page?,
+How can I initialize my variant using the agenta SDK?,
+What function should I use to set the default configuration in the agenta SDK?,
+Can the agenta SDK be used with any Foundational Model?,
+How can I display a text area widget in the playground using the SDK v2?,
+What is the difference between IntParam and FloatParam in terms of the widgets they display in the playground?,
+Can you provide an example of how to use BinaryParam in the SDK v2 configuration?,
+What is the purpose of the GroupedMultipleChoiceParam in the configuration settings?,
+What is the important note mentioned regarding the initialization of BinaryParam in the SDK v2?,
+How do I access the parameters in the configuration in my code?,
+What is the purpose of the 'config' object in the SDK v2?,
+Can I experiment with different parameters using the 'config' object?,
+What is the significance of calling 'agenta.init()' before using the 'config' object?,
+What are some examples of parameters that can be saved in the 'config' object for an LLM variant?,
+Why is it recommended to call `agenta.init()` only once at the entry point of the code?,
+What happens if `agenta.init()` is called multiple times in the code?,
+How does calling `agenta.init()` help in initializing the variant?,
+Is there a different method to initialize the variant in the deprecated SDK v2?,
+What could be the potential impact of using the deprecated SDK v2 for building an llm application?,
+How can I set the default configuration for my variant using the register_default method?,
+What happens if I set the default prompt value to 'Hello World' in my configuration?,
+When should I use the register_default method in my llm application development?,
+What parameters can I set using the register_default method for my llm application?,
+How can I access the prompt1 parameter value from the configuration in the backend?,
+How do I push configuration using agenta.config.push()?,
+What happens if I set overwrite to True when pushing a configuration?,
+How can I avoid overwriting an existing configuration when using agenta.config.push()?,
+Where does the pushed configuration get stored for a specific code base?,
+Can I push configurations for multiple variants using agenta.config.push()?,
+How can I pull a configuration with a specific name?,
+What happens when I pull a configuration using the 'production' environment?,
+How can I access the parameters after pulling the configuration?,
+Can I pull configurations for different variants using this function?,
+Why does the documentation mention that this page is for the deprecated SDK v2?,
\ No newline at end of file
diff --git a/examples/python/custom_workflows/rag-docs-qa/testset_small.csv b/examples/python/custom_workflows/rag-docs-qa/test_set_small.csv
similarity index 100%
rename from examples/python/custom_workflows/rag-docs-qa/testset_small.csv
rename to examples/python/custom_workflows/rag-docs-qa/test_set_small.csv
diff --git a/examples/python/custom_workflows/rag-docs-qa/testset.csv b/examples/python/custom_workflows/rag-docs-qa/testset.csv
deleted file mode 100644
index e10dd18aac..0000000000
--- a/examples/python/custom_workflows/rag-docs-qa/testset.csv
+++ /dev/null
@@ -1,311 +0,0 @@
-query,correct_answer
-How can I deploy Agenta locally in production mode?,
-What command should I run to update Agenta to the latest version?,
-How do I start Agenta in development mode?,
-What steps should I take to troubleshoot if I encounter a port conflict issue?,
-Where can I seek help or report issues related to Agenta?,
-How can I deploy Agenta on Kubernetes for my AI model hosting use case?,
-"I'm having trouble accessing the Kubernetes deployment feature in Agenta Enterprise, how can I troubleshoot this?",
-Can Agenta Enterprise on Kubernetes support scaling my AI models automatically based on demand?,
-What tools are provided by Agenta Enterprise to manage multiple users and teams?,
-Can you provide more details about Agenta Enterprise's early access stage for select partners?,
-How can I deploy Agenta on AWS EC2 using Terraform?,
-What are the open ports created by the Terraform module for Agenta on AWS EC2?,
-How do I serve a variant to an instance after hosting Agenta?,
-What steps are involved in SSH-ing into the instance where Agenta is hosted?,
-How can I delete all the resources created by Terraform for Agenta on AWS?,
-How can I deploy Agenta on Google Cloud Engine using Terraform?,
-What are the prerequisites for deploying Agenta on Google Cloud Engine?,
-How do I SSH into the instance after deploying Agenta on Google Cloud Engine?,
-What security considerations should I keep in mind when allowing SSH access to the instance?,
-How can I delete all the resources created by Terraform for Agenta on Google Cloud Engine?,
-How can I host Agenta on a remote server like an AWS EC2 instance?,
-What are the prerequisites for deploying Agenta on a remote server?,
-How do I obtain the public IP of my AWS EC2 instance?,
-What environment variables do I need to set before launching the Agenta server?,
-"After starting the Agenta server, how can I verify that it is running correctly on the remote server?",
-How can I apply schema migrations to my PostgreSQL database using the provided instructions?,
-What command should I run to ensure that Alembic looks for the configuration file and other necessary files in the specified directory before executing?,
-"If I need to revert a schema migration due to encountered issues, how can I do that according to the documentation?",
-"Is there a way to automate the application of migrations without manually running the upgrade command? If so, how can I achieve this?",
-"After completing the migration, what should I do to verify data integrity in PostgreSQL and ensure that everything works fine?",
-How can I back up my MongoDB database before running the migration script?,
-What Docker command should I use to start the local instance of Agenta for migration?,
-"After initiating the migration, how can I access the running docker containers?",
-How can I verify the data integrity in PostgreSQL post migration?,
-"In case I encounter issues during migration, how can I revert the migration and what should I do next?",
-How can I upgrade to the latest version of Agenta using Docker?,
-What database does Agenta now use starting from version 0.19?,
-How can I access the backend Docker container for Agenta?,
-What is Beanie and how does it relate to MongoDB ODM for Python?,
-What are the steps involved in performing a database migration using Beanie with Agenta backend system?,
-Where can I find answers to common questions about Agenta?,
-How can I get community support for Agenta? Is there a Slack community?,
-What is the recommended channel for reporting bugs in Agenta?,
-"As a Pro plan cloud user, how can I access direct team support?",
-"If I'm interested in demos or have sales inquiries, how can I schedule a call with the founders?",
-How can I add new users to my workspace?,
-What are the different user roles available and what are their respective rights?,
-"As a Workspace Admin, what are the limitations in terms of managing the workspace?",
-What is the role of a Deployment Manager and what tasks do they handle within the workspace?,
-How can I switch between different workspaces if I am a member of multiple workspaces?,
-How do I format backend and CLI code using Black in specific directories?,
-"What are the main steps to contribute to Agenta, especially when picking an issue?",
-Can you guide me through running backend tests locally before making a pull request?,
-How can I update the .env.local file in the frontend directory to include my OpenAI API Key?,
-What are the rules regarding issue assignment and PR activity to prevent zombie issues in the Agenta project?,
-How can I use my own version of Agenta CLI or SDK instead of the installed one?,
-What steps can I take if I'm unable to run Agenta in my terminal even after following the provided instructions?,
-How can I quickly work and test a new type of parameter like IntParam in the SDK?,
-Are there any specific steps needed for working on the backend code in Agenta?,
-How can I efficiently debug the backend code in Agenta using Docker or Visual Studio Code?,
-How can I report a bug using the bug report template?,
-Where can I find the network logs in the browser and why are they important when reporting an issue?,
-What command should I run to get information about Docker containers and how can I provide this information when filing an issue?,
-"When reporting an issue related to a specific llm app, what are the steps to provide container information using Docker Desktop or the terminal?",
-Why is providing network logs and Docker container information crucial when filing an issue?,
-How can I start monitoring and understanding the behavior of my LLM application with Agenta?,
-What are the advantages of using OpenTelemetry (OTel) with Agenta's observability features?,
-What is the relationship between traces and spans in the context of Agenta?,
-Where can I find a Quick Start guide to get started with observability in Agenta?,
-How can I learn to instrument specific applications like OpenAI or LangChain with Agenta?,
-How can I enable tracing for OpenAI calls using Agenta SDK?,
-What packages do I need to install to set up observability for an OpenAI application running locally?,
-How can I configure environment variables for Agenta OSS running locally?,
-Where can I find the captured traces of my application's requests in the Agenta UI?,
-Is tracing enabled by default if I create an application through the Agenta UI?,
-How can I install the Agenta SDK?,
-What is the purpose of the @ag.instrument() decorator in Agenta SDK?,
-How can I add additional metadata to a span in Agenta?,
-"Can you explain how to link spans to applications, variants, and environments in Agenta?",
-What options are available for redacting sensitive data in Agenta SDK?,
-How do I install LiteLLM and Agenta SDK?,
-What are the different environments I can configure while setting up LiteLLM with Agenta?,
-How can I initialize Agenta SDK in my LiteLLM application?,
-What does the @ag.instrument() decorator do in the code example provided?,
-Why is it important to set up the callback handler for LiteLLM while using Agenta?,
-How can I install the required packages for using LangChain with Agenta?,
-What environment variables do I need to configure when using Agenta Cloud or Enterprise?,
-What steps are involved in the code example provided for a LangChain application?,
-How do I initialize Agenta in my LangChain application?,
-Why is it important to call 'LangchainInstrumentor().instrument()' before running my LangChain application?,
-How can I instrument OpenAI API calls with Agenta using `opentelemetry-instrumentation-openai` package?,
-What packages do I need to install to instrument OpenAI API calls with Agenta using `opentelemetry-instrumentation-openai` package?,
-How do I configure environment variables for Agenta Cloud or Enterprise when instrumenting OpenAI API calls?,
-What decorator can I use to monitor multiple calls in a function or workflow as a single trace?,
-How can I associate traces with specific parts of an Agenta project when instrumenting functions?,
-How do I install the required packages for using Instructor with Agenta?,
-What environment variables do I need to configure for Agenta Cloud or Enterprise setup?,
-What must I ensure the order of when instrumenting OpenAI and creating the Instructor client?,
-Why is it mentioned to instrument OpenAI before creating the Instructor client?,
-"What does the @ag.instrument(spankind=""WORKFLOW"") decorator do in the code example?",
-How can I experiment and compare prompts using Agenta?,
-What types of workflows does Agenta support for prompt management and evaluation?,
-How can I collaborate with product teams using Agenta for prompt engineering and evaluation?,
-What are the different ways to deploy an application with Agenta?,
-"Is Agenta compatible with various LLM app architectures and model providers? If so, how?",
-How can I create a new LLM app using an existing template in Agenta?,
-Where can I find the API endpoint after deploying my application in Agenta?,
-What steps do I need to follow to test the application in the playground?,
-How do I add my OpenAI API keys when self-hosting Agenta?,
-What are some next steps after creating my first LLM application in Agenta?,
-How can I create custom templates for my LLM-powered applications in Agenta?,
-What are the predefined environments in Agenta and their purposes?,
-"Explain the relationship between variants, versions, and commits in Agenta.",
-Can you describe the difference between Completion Application Template and Chat Application Template in Agenta?,
-What can I do if the deployment of a variant to an environment does not update automatically in Agenta?,
-How does Agenta enable rapid experimentation and evaluation for LLM applications?,
-What is the significance of Agenta treating each application as a microservice?,
-How does Agenta handle the separation of application logic and configuration?,
-What role does Agenta's backend play in managing applications and configurations?,
-Can you explain the purpose and functionality of Agenta's SDK for a Python library?,
-How can I collaborate with subject matter experts using custom workflows in Agenta?,
-What is the main problem with traditional prompt playgrounds?,
-Can I trust the outputs of traditional prompt playgrounds?,
-How does Agenta's Custom Workflows simplify debugging for AI engineers?,
-What frameworks and models are compatible with Agenta?,
-How can I create a custom workflow with two prompts using Agenta?,
-What are the steps to serve an application in Agenta using the CLI?,
-How do I initialize the Agenta SDK in my Python code?,
-What is the purpose of the `CoPConfig` class in the provided code snippet?,
-Can you explain the concept of entry points in Agenta and how they are used in the code example?,
-How can I configure evaluators for my LLM application?,
-What tools are available for creating testsets for evaluation?,
-Can I run evaluations directly from the web UI in Agenta?,
-What types of evaluators are available for classification/entity extraction in LLM applications?,
-How can I evaluate the faithfulness of outputs in RAG workflows?,
-How can I create a testset for evaluation in agenta SDK?,
-What is the purpose of creating evaluators in the agenta SDK?,
-How can I run an evaluation job using the agenta SDK?,
-What is the function of the rate limit configuration in evaluation jobs?,
-How can I retrieve the detailed results of an evaluation using the agenta SDK?,
-How can I initiate a single model evaluation in the Human Evaluation feature?,
-What steps are involved in starting a new evaluation with the single model test?,
-How do I compare the performance of two different variants manually using A/B Test in the Human Evaluation feature?,
-Can collaborators be invited to collaborate on an A/B Test evaluation in the Human Evaluation feature?,
-Is there a way to switch between card and table view in the evaluation process? How can this be done?,
-How do I create a testset in Agenta using a CSV or JSON file?,
-What is the default column name for the reference answer in a CSV testset?,
-What is the structure of a valid JSON file for a testset in Agenta?,
-How can I add data to a testset from the playground in Agenta?,
-"In Agenta, how can I upload a testset using the API?",
-How do I start a new evaluation in the UI?,
-What parameters can be specified when setting up a new evaluation?,
-What are the advanced configuration options available for adjusting batching and retry parameters?,
-How can I analyze the results of an evaluation in more detail?,
-"Can I compare multiple evaluations from the same testset in the UI? If yes, how?",
-How can I configure evaluators for my LLM application?,
-What are the inputs that evaluators typically take?,
-Which button should I click to create a new evaluator?,
-Can I create custom evaluators for my LLM application?,
-What is the purpose of mappings evaluator's inputs to the LLM data?,
-How can I create a webhook evaluator for a specific use case in Agenta?,
-What are the limitations in terms of security measures when using webhook evaluators?,
-What input parameters are required for a webhook evaluator in Agenta?,
-What is the expected format of the webhook request body for a webhook evaluator?,
-What should the webhook response body contain in order to be considered properly-formatted?,
-How does the Exact Match evaluator determine if the model's output is correct?,
-What does the Contains JSON evaluator check for in the model's output?,
-How does the JSON Field Match evaluator compare specific fields within JSON data?,
-Can you explain the process of JSON Diff Match evaluation?,
-What configuration options are available for the JSON Diff Match evaluator and how do they affect the comparison process?,
-How can I assess the performance of LLMs by identifying specific patterns within the output generated by the model?,
-What is the purpose of the 'Regex Test' evaluator in Agenta and how does it work?,
-Could you provide an example of using the 'Starts With' evaluator in Agenta and its significance?,
-How does the 'Contains Any' evaluator in Agenta differ from the 'Contains All' evaluator?,
-In what situations would I use the 'Ends With' evaluator in Agenta and how does it handle case sensitivity?,
-How can I create a custom evaluator in Agenta?,
-What is the function signature for the 'evaluate' function in a custom evaluator?,
-What is the purpose of the 'evaluate' function in a custom evaluator?,
-Which language is used for writing custom evaluators in Agenta?,
-What ranges of scores can the 'evaluate' function return?,
-How can I configure and use RAG evaluators in my custom-built application?,
-What version of the Python SDK is required to access internal variables and intermediate outputs for RAG Evaluators?,
-Where can I find the source code for a simple RAG Application that fetches movies and generates summaries?,
-How can I troubleshoot if the RAG Evaluator and `view trace` are not accessible in Agenta OSS?,
-Which utility can I use in Agenta to add internal variables (internals) to stages of a workflow?,
-How does the Similarity Match evaluator determine a match between the generated output and the correct answer?,
-What configuration parameter is needed for the Semantic Similarity Match evaluator?,
-What is the cost associated with using the Semantic Similarity Match evaluator that uses OpenAI embeddings?,
-In what use cases is the Levenshtein Distance evaluator particularly useful?,
-Can you explain how the Levenshtein Distance evaluator works for comparing text outputs?,
-How can I configure the prompt for LLM-as-a-Judge evaluation?,
-"What variables can be used to reference inputs, outputs, and reference answers in the prompt?",
-What is the default prompt used for the country expert demo application?,
-"Which models are supported by LLM-as-a-Judge, and how can I select a specific model?",
-Where do I need to set my OpenAI or Anthropic API key to use LLM-as-a-Judge?,
-How can I create a testset using the SDK?,
-What is the process for creating and configuring an evaluator programmatically?,
-How can I check the status of an evaluation run?,
-Where can I find the list of evaluator keys and configurations?,
-What should I do if I encounter rate limits during the evaluation process?,
-How can I create a new prompt using the Agenta SDK?,
-How do I deploy changes to the production environment using the SDK?,
-"What is the structure used by Agenta for prompt versioning, and how does it differ from traditional versioning systems?",
-"In case of an issue with deploying changes using the SDK, what steps can I take to troubleshoot?",
-Can you explain the concept of variants in Agenta and how they are similar to branches in Git?,
-How can I set up tracing for a RAG application in LangChain using Agenta?,
-What is the purpose of tracing in LLM applications?,
-"How do I install the necessary dependencies for LangChain, Agenta, and instrumentation?",
-Can you provide an example of a Q&A RAG application in LangChain and how it works?,
-How can I retrieve and generate answers using relevant snippets of documentation in Agenta?,
-How can I fetch the diff for a GitHub pull request using Python?,
-What role does LiteLLM play in the AI assistant workflow described in the tutorial?,
-How can I add observability to my LLM application using Agenta?,
-What is the purpose of creating an LLM playground for the application?,
-How can I deploy my AI assistant to production using Agenta?,
-How can I version prompts in Agenta?,
-What capabilities does Agenta provide for prompt management?,
-Why do I need a prompt management system?,
-What is the purpose of configuration management in Agenta?,
-How can I publish a prompt to an endpoint from the web UI?,
-How can I create a prompt using the web UI?,
-Why do I need to publish a variant to a deployment?,
-How can I integrate the prompt configuration with my Python code using the Agenta SDK?,
-"Can I revert to a previous deployment version, and if so, how?",
-What is the purpose of using Pydantic for schema validation in the SDK integration process?,
-How can I create reusable prompts using prompt templates in the agenta playground?,
-Where can I find the option to add new inputs to the LLM app in the playground?,
-"What is the process for creating a new variant of an application in agenta, and where can I provide a new name for the variant?",
-"How can I test a variant dynamically in the playground, and is there a way to run all inputs in a testset?",
-"In the agenta playground, how do I compare variants side by side? Can I interact with different variants simultaneously in a chat application?",
-How can I create a new prompt using the SDK?,
-What is the structure followed by Agenta for prompt versioning?,
-How can I delete a variant in Agenta? Is this action reversible?,
-Can I list all variants of an application using the SDK? How can I do that?,
-What is the default behavior for fetching configurations in Agenta?,
-How can I use the Agenta SDK to invoke the deployed version of my prompt?,
-Where can I find the call to invoke the deployed version of my prompt within the Agenta UI?,
-What parameters do I need to provide when invoking a deployed prompt through the REST API?,
-What does the 'inputs' dictionary in the parameters contain and how should it be structured?,
-How can I control which environment version of my prompt is being called when using the REST API?,
-How do I fetch the deployed version of my prompt in my code using the Agenta SDK?,
-What Python SDK do I need to install for using the Agenta SDK?,
-What environment variables need to be set up for fetching prompts?,
-"In the sample output, what is the value of 'model' fetched from staging configuration?",
-Which slugs should I provide when fetching prompts using Agenta SDK?,
-How can Agenta help me manage prompts in my application?,
-What are the advantages of using Agenta as a prompt management system?,
-What considerations should I keep in mind when integrating observability manually with Agenta?,
-When should I consider using Agenta as a middleware/model proxy?,
-Does using Agenta as a proxy add any latency to the response?,
-How can I install the agenta CLI tool?,
-What is the command to install agenta using pip?,
-Where can I find a quick usage guide for the agenta CLI?,
-How do I get an overview of the main commands and capabilities of agenta CLI?,
-Is there a tutorial available for deploying an LLM app from code using agenta CLI?,
-How can I initialize a new Agenta project using the CLI?,
-"What options can be provided with the 'agenta init' command, and how do they affect the project initialization process?",
-"For the 'agenta variant list' command, where should it be executed, and what information does it provide?",
-What are the steps involved in removing a variant from an application using the 'agenta variant remove' command?,
-"How can I deploy an application variant to the Agenta platform using the 'agenta variant serve' command, and what additional options are available for deployment?",
-How can I create a new variant with initial configuration parameters?,
-What method should I use to commit changes to an existing variant?,
-Which method is used to deploy a variant to a specific environment?,
-How can I retrieve the version history of a variant?,
-What are the available predefined environments for deployment?,
-How can I store additional data within the current span in a trace?,
-What utility function can I use to validate if a string is a valid attribute key?,
-How can I link spans to specific Agenta resources using the SDK?,
-What method should I use to set the status of a span in the CustomSpan class?,
-When should I apply the @ag.instrument() decorator in my Python code?,
-How can I expose specific stages of LLM workflows as API endpoints using the Custom Workflows SDK?,
-What types of fields are accepted in the configuration schema when defining a configuration for a function?,
-How can I define fields with constraints and defaults in the configuration schema when using `@ag.route` or `@ag.entrypoint` decorators?,
-"How are different field types represented in the Agenta playground UI, and what are the input methods for each type?",
-How can I retrieve configuration information from the route context using `ag.ConfigManager.get_from_route()` and how is this configuration used in a function?,
-How do I experiment with AI applications using the agenta SDK?,
-What is the purpose of the deprecated SDK v2 mentioned on the page?,
-How can I initialize my variant using the agenta SDK?,
-What function should I use to set the default configuration in the agenta SDK?,
-Can the agenta SDK be used with any Foundational Model?,
-How can I display a text area widget in the playground using the SDK v2?,
-What is the difference between IntParam and FloatParam in terms of the widgets they display in the playground?,
-Can you provide an example of how to use BinaryParam in the SDK v2 configuration?,
-What is the purpose of the GroupedMultipleChoiceParam in the configuration settings?,
-What is the important note mentioned regarding the initialization of BinaryParam in the SDK v2?,
-How do I access the parameters in the configuration in my code?,
-What is the purpose of the 'config' object in the SDK v2?,
-Can I experiment with different parameters using the 'config' object?,
-What is the significance of calling 'agenta.init()' before using the 'config' object?,
-What are some examples of parameters that can be saved in the 'config' object for an LLM variant?,
-Why is it recommended to call `agenta.init()` only once at the entry point of the code?,
-What happens if `agenta.init()` is called multiple times in the code?,
-How does calling `agenta.init()` help in initializing the variant?,
-Is there a different method to initialize the variant in the deprecated SDK v2?,
-What could be the potential impact of using the deprecated SDK v2 for building an llm application?,
-How can I set the default configuration for my variant using the register_default method?,
-What happens if I set the default prompt value to 'Hello World' in my configuration?,
-When should I use the register_default method in my llm application development?,
-What parameters can I set using the register_default method for my llm application?,
-How can I access the prompt1 parameter value from the configuration in the backend?,
-How do I push configuration using agenta.config.push()?,
-What happens if I set overwrite to True when pushing a configuration?,
-How can I avoid overwriting an existing configuration when using agenta.config.push()?,
-Where does the pushed configuration get stored for a specific code base?,
-Can I push configurations for multiple variants using agenta.config.push()?,
-How can I pull a configuration with a specific name?,
-What happens when I pull a configuration using the 'production' environment?,
-How can I access the parameters after pulling the configuration?,
-Can I pull configurations for different variants using this function?,
-Why does the documentation mention that this page is for the deprecated SDK v2?,
\ No newline at end of file
diff --git a/hooks/setup.sh b/hooks/setup.sh
deleted file mode 100755
index 287d31bab6..0000000000
--- a/hooks/setup.sh
+++ /dev/null
@@ -1,50 +0,0 @@
-#!/usr/bin/env bash
-set -euo pipefail
-
-echo "🔧 Setting up Git hooks with pre-commit + gitleaks..."
-
-# --- check dependencies ---
-if ! command -v python3 >/dev/null 2>&1; then
- echo "❌ Python3 is required but not installed."
- exit 1
-fi
-if ! command -v pip3 >/dev/null 2>&1; then
- echo "❌ pip3 is required but not installed."
- exit 1
-fi
-
-# --- install pre-commit globally if missing ---
-if ! command -v pre-commit >/dev/null 2>&1; then
- echo "📦 Installing pre-commit..."
- pip3 install pre-commit
-fi
-
-# --- install gitleaks globally if missing ---
-if ! command -v gitleaks >/dev/null 2>&1; then
- echo "📦 Installing gitleaks..."
- if command -v brew >/dev/null 2>&1; then
- brew install gitleaks
- else
- # fallback: go install (requires Go installed)
- go install github.com/zricethezav/gitleaks/v8@latest
- export PATH="$PATH:$(go env GOPATH)/bin"
- echo "----------------------------------------------------------------------"
- echo "⚠️⚠️ PLEASE ADD $(go env GOPATH)/bin TO YOUR PATH IN YOUR .bashrc OR .zshrc ⚠️⚠️"
- echo "----------------------------------------------------------------------"
- fi
-fi
-
-# --- install hooks into .git/hooks/ ---
-echo "⚙️ Installing pre-commit hooks..."
-pre-commit install --install-hooks
-pre-commit install --hook-type pre-push
-
-# --- one-time full scans ---
-echo "🔍 Running one-time gitleaks scans..."
-
-gitleaks --config .gitleaks.toml --exit-code 1 --verbose detect --no-git --source . || {
- echo "❌ Gitleaks detected potential secrets in the working directory."
- exit 1
-}
-
-echo "✅ Setup complete! Hooks installed and initial scan passed. You are safe to commit."
diff --git a/hosting/old/aws/agenta_instance.tf b/hosting/aws/agenta_instance.tf
similarity index 100%
rename from hosting/old/aws/agenta_instance.tf
rename to hosting/aws/agenta_instance.tf
diff --git a/hosting/old/aws/agenta_instance_sg.tf b/hosting/aws/agenta_instance_sg.tf
similarity index 100%
rename from hosting/old/aws/agenta_instance_sg.tf
rename to hosting/aws/agenta_instance_sg.tf
diff --git a/hosting/old/aws/instance-setup.sh b/hosting/aws/instance-setup.sh
similarity index 100%
rename from hosting/old/aws/instance-setup.sh
rename to hosting/aws/instance-setup.sh
diff --git a/hosting/old/aws/main.tf b/hosting/aws/main.tf
similarity index 100%
rename from hosting/old/aws/main.tf
rename to hosting/aws/main.tf
diff --git a/hosting/docker-compose/ee/LICENSE b/hosting/docker-compose/ee/LICENSE
deleted file mode 100644
index ae7a2f38f4..0000000000
--- a/hosting/docker-compose/ee/LICENSE
+++ /dev/null
@@ -1,37 +0,0 @@
-Agenta Enterprise License (the “Enterprise License”)
-Copyright (c) 2023–2025
-Agentatech UG (haftungsbeschränkt), doing business as “Agenta” (“Agenta”)
-
-With regard to the Agenta Software:
-
-This software and associated documentation files (the "Software") may only be
-used in production, if you (and any entity that you represent) have agreed to,
-and are in compliance with, the Agenta Subscription Terms of Service, available
-at https://agenta.ai/terms (the “Enterprise Terms”), or other
-agreement governing the use of the Software, as agreed by you and Agenta,
-and otherwise have a valid Agenta Enterprise License.
-
-Subject to the foregoing sentence, you are free to modify this Software and
-publish patches to the Software. You agree that Agenta and/or its licensors
-(as applicable) retain all right, title and interest in and to all such
-modifications and/or patches, and all such modifications and/or patches may
-only be used, copied, modified, displayed, distributed, or otherwise exploited
-with a valid Agenta Enterprise License. Notwithstanding the foregoing, you may
-copy and modify the Software for development and testing purposes, without
-requiring a subscription. You agree that Agenta and/or its licensors (as
-applicable) retain all right, title and interest in and to all such
-modifications. You are not granted any other rights beyond what is expressly
-stated herein. Subject to the foregoing, it is forbidden to copy, merge,
-publish, distribute, sublicense, and/or sell the Software.
-
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-SOFTWARE.
-
-For all third party components incorporated into the Agenta Software, those
-components are licensed under the original license provided by the owner of the
-applicable component.
diff --git a/hosting/docker-compose/ee/docker-compose.dev.yml b/hosting/docker-compose/ee/docker-compose.dev.yml
index 09861d7b20..3889ef61e1 100644
--- a/hosting/docker-compose/ee/docker-compose.dev.yml
+++ b/hosting/docker-compose/ee/docker-compose.dev.yml
@@ -49,7 +49,6 @@ services:
volumes:
- ../../../api:/app
- - ../../../sdk:/sdk
env_file:
- ${ENV_FILE:-./.env.ee.dev}
@@ -96,7 +95,6 @@ services:
volumes:
- ../../../api:/app
- - ../../../sdk:/sdk
env_file:
- ${ENV_FILE:-./.env.ee.dev}
@@ -115,14 +113,13 @@ services:
- agenta-network
command: >
- watchmedo auto-restart --directory=/app/ --pattern=*.py --recursive -- celery -A entrypoint.celery_app worker --concurrency=1 --max-tasks-per-child=1 --prefetch-multiplier=1 --loglevel=DEBUG
+ watchmedo auto-restart --directory=/app/ --pattern=*.py --recursive -- celery -A entrypoint.celery_app worker --concurrency=1 --max-tasks-per-child=1 --prefetch-multiplier=1
cron:
image: agenta-ee-dev-api:latest
volumes:
- ../../../api/ee/src/crons/meters.sh:/meters.sh
- - ../../../api/oss/src/crons/queries.sh:/queries.sh
env_file:
- ${ENV_FILE:-./.env.ee.dev}
@@ -145,8 +142,7 @@ services:
image: agenta-ee-dev-api:latest
volumes:
- - ../../../api:/app
- - ../../../sdk:/sdk
+ - ../../../api:/app/
env_file:
- ${ENV_FILE:-./.env.ee.dev}
@@ -157,12 +153,12 @@ services:
networks:
- agenta-network
- command: sh -c "python -m ee.databases.postgres.migrations.runner"
+ command: sh -c "python -c 'from ee.databases.postgres.migrations.utils import split_core_and_tracing as split; split(); from ee.databases.postgres.migrations.core.utils import run_alembic_migration as run; run(); from ee.databases.postgres.migrations.tracing.utils import run_alembic_migration as run; run(); from ee.databases.postgres.migrations.utils import copy_nodes_from_core_to_tracing as copy; copy();'"
completion:
build:
context: ../../../services/completion
- dockerfile: ee/docker/Dockerfile.dev
+ dockerfile: oss/docker/Dockerfile.dev
volumes:
- ../../../services/completion:/app
@@ -191,7 +187,7 @@ services:
chat:
build:
context: ../../../services/chat
- dockerfile: ee/docker/Dockerfile.dev
+ dockerfile: oss/docker/Dockerfile.dev
volumes:
- ../../../services/chat:/app
diff --git a/hosting/docker-compose/ee/env.ee.dev.example b/hosting/docker-compose/ee/env.ee.dev.example
deleted file mode 100644
index c42666965b..0000000000
--- a/hosting/docker-compose/ee/env.ee.dev.example
+++ /dev/null
@@ -1,91 +0,0 @@
-# First-party (required)
-AGENTA_LICENSE=ee
-AGENTA_STAGE=dev
-AGENTA_PROVIDER=local
-AGENTA_WEB_URL=http://localhost
-AGENTA_API_URL=http://localhost/api
-AGENTA_SERVICES_URL=http://localhost/services
-AGENTA_AUTH_KEY=change-me
-AGENTA_CRYPT_KEY=change-me
-AGENTA_API_IMAGE_NAME=agenta-api
-AGENTA_API_IMAGE_TAG=latest
-AGENTA_WEB_IMAGE_NAME=agenta-web
-AGENTA_WEB_IMAGE_TAG=latest
-
-# First-party (registry & service)
-DOCKER_NETWORK_MODE=bridge
-POSTGRES_USERNAME=username
-POSTGRES_PASSWORD=password
-
-# First-party (optional)
-AGENTA_AUTO_MIGRATIONS=true
-AGENTA_PRICING=
-AGENTA_DEMOS=
-AGENTA_RUNTIME_PREFIX=
-AGENTA_API_INTERNAL_URL=
-AGENTA_LITELLM_MOCK=
-POSTGRES_USERNAME_ADMIN=
-POSTGRES_PASSWORD_ADMIN=
-AGENTA_SERVICE_MIDDLEWARE_CACHE_ENABLED=true
-AGENTA_OTLP_MAX_BATCH_BYTES=10485760
-
-# Third-party (required)
-TRAEFIK_DOMAIN=
-TRAEFIK_PROTOCOL=
-TRAEFIK_PORT=
-
-REDIS_URL=redis://redis:6379/0
-RABBITMQ_DEFAULT_PASS=guest
-RABBITMQ_DEFAULT_USER=guest
-
-CELERY_BROKER_URL=amqp://guest@rabbitmq//
-CELERY_RESULT_BACKEND=redis://redis:6379/0
-
-POSTGRES_URI_SUPERTOKENS="postgresql://username:password@postgres:5432/agenta_ee_supertokens"
-POSTGRES_URI_CORE="postgresql+asyncpg://username:password@postgres:5432/agenta_ee_core"
-POSTGRES_URI_TRACING="postgresql+asyncpg://username:password@postgres:5432/agenta_ee_tracing"
-
-ALEMBIC_CFG_PATH_CORE=/app/ee/databases/postgres/migrations/core/alembic.ini
-ALEMBIC_CFG_PATH_TRACING=/app/ee/databases/postgres/migrations/tracing/alembic.ini
-
-SUPERTOKENS_CONNECTION_URI=http://supertokens:3567
-
-# Third-party (optional)
-AWS_ECR_URL=
-AWS_RDS_SECRET=
-
-POSTHOG_API_KEY=phc_3urGRy5TL1HhaHnRYL0JSHxJxigRVackhphHtozUmdp
-
-GITHUB_OAUTH_CLIENT_ID=
-GITHUB_OAUTH_CLIENT_SECRET=
-GOOGLE_OAUTH_CLIENT_ID=
-GOOGLE_OAUTH_CLIENT_SECRET=
-
-SUPERTOKENS_API_KEY=replace-me
-
-NEW_RELIC_LICENSE_KEY=
-NRIA_LICENSE_KEY=
-
-LOOPS_API_KEY=
-
-SENDGRID_API_KEY=
-
-CRISP_WEBSITE_ID=
-
-STRIPE_API_KEY=
-STRIPE_WEBHOOK_SECRET=
-STRIPE_TARGET=
-
-# Third-party - LLM (optional)
-ALEPHALPHA_API_KEY=
-ANTHROPIC_API_KEY=
-ANYSCALE_API_KEY=
-COHERE_API_KEY=
-DEEPINFRA_API_KEY=
-GEMINI_API_KEY=
-GROQ_API_KEY=
-MISTRAL_API_KEY=
-OPENAI_API_KEY=
-OPENROUTER_API_KEY=
-PERPLEXITYAI_API_KEY=
-TOGETHERAI_API_KEY=
diff --git a/hosting/docker-compose/ee/env.ee.gh.example b/hosting/docker-compose/ee/env.ee.gh.example
deleted file mode 100644
index 5cba09c18c..0000000000
--- a/hosting/docker-compose/ee/env.ee.gh.example
+++ /dev/null
@@ -1,80 +0,0 @@
-# First-party (required)
-AGENTA_LICENSE=ee
-AGENTA_STAGE=dev
-AGENTA_PROVIDER=local
-AGENTA_API_URL=http://localhost/api
-AGENTA_WEB_URL=http://localhost
-AGENTA_SERVICES_URL=http://localhost/services
-AGENTA_AUTH_KEY=change-me
-AGENTA_CRYPT_KEY=change-me
-
-# First-party (registry & service)
-DOCKER_NETWORK_MODE=bridge
-POSTGRES_PASSWORD=password
-POSTGRES_USERNAME=username
-
-# First-party (optional)
-AGENTA_AUTO_MIGRATIONS=true
-AGENTA_PRICING=
-AGENTA_DEMOS=
-AGENTA_RUNTIME_PREFIX=
-AGENTA_API_INTERNAL_URL=
-AGENTA_SERVICE_MIDDLEWARE_CACHE_ENABLED=true
-AGENTA_OTLP_MAX_BATCH_BYTES=10485760
-
-# Third-party (required)
-TRAEFIK_DOMAIN=
-TRAEFIK_PROTOCOL=
-TRAEFIK_PORT=
-
-REDIS_URL=redis://redis:6379/0
-RABBITMQ_DEFAULT_PASS=guest
-RABBITMQ_DEFAULT_USER=guest
-
-CELERY_BROKER_URL=amqp://guest@rabbitmq//
-CELERY_RESULT_BACKEND=redis://redis:6379/0
-
-POSTGRES_URI_SUPERTOKENS="postgresql://username:password@postgres:5432/agenta_ee_supertokens"
-POSTGRES_URI_CORE="postgresql+asyncpg://username:password@postgres:5432/agenta_ee_core"
-POSTGRES_URI_TRACING="postgresql+asyncpg://username:password@postgres:5432/agenta_ee_tracing"
-
-ALEMBIC_CFG_PATH_CORE=/app/ee/databases/postgres/migrations/core/alembic.ini
-ALEMBIC_CFG_PATH_TRACING=/app/ee/databases/postgres/migrations/tracing/alembic.ini
-
-SUPERTOKENS_API_KEY=replace-me
-SUPERTOKENS_CONNECTION_URI=http://supertokens:3567
-
-# Third-party (optional)
-POSTHOG_API_KEY=phc_3urGRy5TL1HhaHnRYL0JSHxJxigRVackhphHtozUmdp
-
-GITHUB_OAUTH_CLIENT_ID=
-GITHUB_OAUTH_CLIENT_SECRET=
-
-GOOGLE_OAUTH_CLIENT_ID=
-GOOGLE_OAUTH_CLIENT_SECRET=
-
-NEW_RELIC_LICENSE_KEY=
-NRIA_LICENSE_KEY=
-
-LOOPS_API_KEY=
-
-SENDGRID_API_KEY=
-
-CRISP_WEBSITE_ID=
-
-STRIPE_API_KEY=
-STRIPE_WEBHOOK_SECRET=
-
-# Third-party - LLM (optional)
-ALEPHALPHA_API_KEY=
-ANTHROPIC_API_KEY=
-ANYSCALE_API_KEY=
-COHERE_API_KEY=
-DEEPINFRA_API_KEY=
-GEMINI_API_KEY=
-GROQ_API_KEY=
-MISTRAL_API_KEY=
-OPENAI_API_KEY=
-OPENROUTER_API_KEY=
-PERPLEXITYAI_API_KEY=
-TOGETHERAI_API_KEY=
\ No newline at end of file
diff --git a/hosting/docker-compose/oss/docker-compose.dev.yml b/hosting/docker-compose/oss/docker-compose.dev.yml
index 53c0eaef52..8489bc0f01 100644
--- a/hosting/docker-compose/oss/docker-compose.dev.yml
+++ b/hosting/docker-compose/oss/docker-compose.dev.yml
@@ -37,7 +37,6 @@ services:
volumes:
- ../../../api/entrypoint.py:/app/entrypoint.py
- ../../../api/oss:/app/oss
- - ../../../sdk:/sdk
env_file:
- ${ENV_FILE:-./.env.oss.dev}
@@ -84,7 +83,6 @@ services:
volumes:
- ../../../api/entrypoint.py:/app/entrypoint.py
- ../../../api/oss:/app/oss
- - ../../../sdk:/sdk
env_file:
- ${ENV_FILE:-./.env.oss.dev}
@@ -103,30 +101,6 @@ services:
command: >
watchmedo auto-restart --directory=/app/ --pattern=*.py --recursive -- celery -A entrypoint.celery_app worker --concurrency=1 --max-tasks-per-child=1 --prefetch-multiplier=1
- cron:
- image: agenta-oss-dev-api:latest
-
- volumes:
- #
- - ../../../api/oss/src/crons/queries.sh:/queries.sh
-
- env_file:
- - ${ENV_FILE:-./.env.oss.dev}
-
- depends_on:
- - postgres
- - api
-
- extra_hosts:
- - "host.docker.internal:host-gateway"
-
- restart: always
-
- networks:
- - agenta-network
-
- command: cron -f
-
alembic:
build:
context: ../../../api
@@ -135,7 +109,6 @@ services:
volumes:
- ../../../api/entrypoint.py:/app/entrypoint.py
- ../../../api/oss:/app/oss
- - ../../../sdk:/sdk
env_file:
- ${ENV_FILE:-./.env.oss.dev}
@@ -146,7 +119,7 @@ services:
networks:
- agenta-network
- command: sh -c "python -m oss.databases.postgres.migrations.runner"
+ command: sh -c "python -c 'from oss.databases.postgres.migrations.utils import split_core_and_tracing as split; split(); from oss.databases.postgres.migrations.core.utils import run_alembic_migration as run; run(); from oss.databases.postgres.migrations.tracing.utils import run_alembic_migration as run; run(); from oss.databases.postgres.migrations.utils import copy_nodes_from_core_to_tracing as copy; copy();'"
completion:
build:
diff --git a/hosting/docker-compose/oss/docker-compose.gh.ssl.yml b/hosting/docker-compose/oss/docker-compose.gh.ssl.yml
index c7dbd5c730..65101d4c0b 100644
--- a/hosting/docker-compose/oss/docker-compose.gh.ssl.yml
+++ b/hosting/docker-compose/oss/docker-compose.gh.ssl.yml
@@ -104,26 +104,6 @@ services:
condition: service_healthy
restart: always
- cron:
- image: agenta-oss-dev-api:latest
-
- env_file:
- - ${ENV_FILE:-./.env.oss.dev}
-
- depends_on:
- - postgres
- - api
-
- extra_hosts:
- - "host.docker.internal:host-gateway"
-
- restart: always
-
- networks:
- - agenta-network
-
- command: cron -f
-
alembic:
build:
context: ../../../api
@@ -137,8 +117,7 @@ services:
env_file:
- ${ENV_FILE:-./.env.oss.gh}
- command: sh -c "python -m oss.databases.postgres.migrations.runner"
-
+ command: sh -c "python -c 'from oss.databases.postgres.migrations.utils import split_core_and_tracing as split; split(); from oss.databases.postgres.migrations.core.utils import run_alembic_migration as run; run(); from oss.databases.postgres.migrations.tracing.utils import run_alembic_migration as run; run(); from oss.databases.postgres.migrations.utils import copy_nodes_from_core_to_tracing as copy; copy();'"
depends_on:
postgres:
condition: service_healthy
diff --git a/hosting/docker-compose/oss/docker-compose.gh.yml b/hosting/docker-compose/oss/docker-compose.gh.yml
index e079346a68..867296bb2f 100644
--- a/hosting/docker-compose/oss/docker-compose.gh.yml
+++ b/hosting/docker-compose/oss/docker-compose.gh.yml
@@ -80,26 +80,6 @@ services:
- redis
restart: always
- cron:
- image: agenta-oss-dev-api:latest
-
- env_file:
- - ${ENV_FILE:-./.env.oss.dev}
-
- depends_on:
- - postgres
- - api
-
- extra_hosts:
- - "host.docker.internal:host-gateway"
-
- restart: always
-
- networks:
- - agenta-network
-
- command: cron -f
-
alembic:
build:
context: ../../../api
@@ -113,7 +93,7 @@ services:
env_file:
- ${ENV_FILE:-./.env.oss.gh}
- command: sh -c "python -m oss.databases.postgres.migrations.runner"
+ command: sh -c "python -c 'from oss.databases.postgres.migrations.utils import split_core_and_tracing as split; split(); from oss.databases.postgres.migrations.core.utils import run_alembic_migration as run; run(); from oss.databases.postgres.migrations.tracing.utils import run_alembic_migration as run; run(); from oss.databases.postgres.migrations.utils import copy_nodes_from_core_to_tracing as copy; copy();'"
depends_on:
postgres:
diff --git a/hosting/docker-compose/oss/env.oss.dev.example b/hosting/docker-compose/oss/env.oss.dev.example
deleted file mode 100644
index 05e16eda8a..0000000000
--- a/hosting/docker-compose/oss/env.oss.dev.example
+++ /dev/null
@@ -1,103 +0,0 @@
-# First-party (required)
-AGENTA_LICENSE=oss
-AGENTA_API_URL=http://localhost/api
-AGENTA_WEB_URL=http://localhost
-AGENTA_SERVICES_URL=http://localhost/services
-AGENTA_AUTH_KEY=replace-me
-AGENTA_CRYPT_KEY=replace-me
-
-# First-party (registry & service)
-POSTGRES_PASSWORD=password
-POSTGRES_USERNAME=username
-
-# First-party (optional)
-AGENTA_AUTO_MIGRATIONS=true
-AGENTA_PRICING=
-AGENTA_DEMOS=
-DOCKER_NETWORK_MODE=bridge
-AGENTA_RUNTIME_PREFIX=
-AGENTA_SEND_EMAIL_FROM_ADDRESS=mail@example.com
-AGENTA_API_INTERNAL_URL=
-AGENTA_TELEMETRY_ENABLED=true
-AGENTA_SERVICE_MIDDLEWARE_CACHE_ENABLED=true
-AGENTA_OTLP_MAX_BATCH_BYTES=10485760
-
-# Third-party (required)
-TRAEFIK_DOMAIN=localhost
-TRAEFIK_PORT=80
-TRAEFIK_PROTOCOL=http
-TRAEFIK_UI_PORT=8080
-TRAEFIK_HTTPS_PORT=443
-
-REDIS_URL=redis://redis:6379/0
-RABBITMQ_DEFAULT_USER=guest
-RABBITMQ_DEFAULT_PASS=guest
-
-CELERY_BROKER_URL=amqp://guest@rabbitmq//
-CELERY_RESULT_BACKEND=redis://redis:6379/0
-
-
-POSTGRES_URI_SUPERTOKENS=postgresql://username:password@postgres:5432/agenta_oss_supertokens
-POSTGRES_URI_CORE=postgresql+asyncpg://username:password@postgres:5432/agenta_oss_core
-POSTGRES_URI_TRACING=postgresql+asyncpg://username:password@postgres:5432/agenta_oss_tracing
-
-ALEMBIC_CFG_PATH_CORE=/app/oss/databases/postgres/migrations/core/alembic.ini
-ALEMBIC_CFG_PATH_TRACING=/app/oss/databases/postgres/migrations/tracing/alembic.ini
-
-SUPERTOKENS_CONNECTION_URI=http://supertokens:3567
-
-POSTHOG_API_KEY=phc_hmVSxIjTW1REBHXgj2aw4HW9X6CXb6FzerBgP9XenC7
-
-# Third-party (required for TLS/SSL)
-AGENTA_SSL_DIR=
-
-# Third-party (optional)
-NGINX_PORT=80
-SUPERTOKENS_API_KEY=
-RABBITMQ_PORT=
-RABBITMQ_UI_PORT=
-REDIS_CACHE_HOST=
-REDIS_CACHE_PORT=
-POSTGRES_PORT=
-SUPERTOKENS_PORT=
-
-GOOGLE_OAUTH_CLIENT_ID=
-GOOGLE_OAUTH_CLIENT_SECRET=
-GITHUB_OAUTH_CLIENT_ID=
-GITHUB_OAUTH_CLIENT_SECRET=
-
-NEW_RELIC_LICENSE_KEY=
-NRIA_LICENSE_KEY=
-
-LOOPS_API_KEY=
-
-SENDGRID_API_KEY=
-
-CRISP_WEBSITE_ID=
-
-STRIPE_API_KEY=
-STRIPE_WEBHOOK_SECRET=
-
-# Third-party — LLM (optional)
-ALEPHALPHA_API_KEY=
-ANTHROPIC_API_KEY=
-ANYSCALE_API_KEY=
-COHERE_API_KEY=
-DEEPINFRA_API_KEY=
-GEMINI_API_KEY=
-GROQ_API_KEY=
-MISTRAL_API_KEY=
-OPENAI_API_KEY=
-OPENROUTER_API_KEY=
-PERPLEXITYAI_API_KEY=
-TOGETHERAI_API_KEY=
-
-# Legacy (deprecated, to be removed)
-AGENTA_PORT=80
-BARE_DOMAIN_NAME=localhost
-DOMAIN_NAME=http://localhost
-WEBSITE_DOMAIN_NAME=http://localhost
-SERVICE_URL_TEMPLATE=http://localhost:80/services/{path}
-POSTGRES_DB=agenta_oss
-POSTGRES_URI=postgresql+asyncpg://username:password@postgres:5432/agenta_oss
-ALEMBIC_CFG_PATH=/app/oss/databases/postgres/migrations/alembic.oss.ini
diff --git a/hosting/docker-compose/oss/env.oss.gh.example b/hosting/docker-compose/oss/env.oss.gh.example
deleted file mode 100644
index 05e16eda8a..0000000000
--- a/hosting/docker-compose/oss/env.oss.gh.example
+++ /dev/null
@@ -1,103 +0,0 @@
-# First-party (required)
-AGENTA_LICENSE=oss
-AGENTA_API_URL=http://localhost/api
-AGENTA_WEB_URL=http://localhost
-AGENTA_SERVICES_URL=http://localhost/services
-AGENTA_AUTH_KEY=replace-me
-AGENTA_CRYPT_KEY=replace-me
-
-# First-party (registry & service)
-POSTGRES_PASSWORD=password
-POSTGRES_USERNAME=username
-
-# First-party (optional)
-AGENTA_AUTO_MIGRATIONS=true
-AGENTA_PRICING=
-AGENTA_DEMOS=
-DOCKER_NETWORK_MODE=bridge
-AGENTA_RUNTIME_PREFIX=
-AGENTA_SEND_EMAIL_FROM_ADDRESS=mail@example.com
-AGENTA_API_INTERNAL_URL=
-AGENTA_TELEMETRY_ENABLED=true
-AGENTA_SERVICE_MIDDLEWARE_CACHE_ENABLED=true
-AGENTA_OTLP_MAX_BATCH_BYTES=10485760
-
-# Third-party (required)
-TRAEFIK_DOMAIN=localhost
-TRAEFIK_PORT=80
-TRAEFIK_PROTOCOL=http
-TRAEFIK_UI_PORT=8080
-TRAEFIK_HTTPS_PORT=443
-
-REDIS_URL=redis://redis:6379/0
-RABBITMQ_DEFAULT_USER=guest
-RABBITMQ_DEFAULT_PASS=guest
-
-CELERY_BROKER_URL=amqp://guest@rabbitmq//
-CELERY_RESULT_BACKEND=redis://redis:6379/0
-
-
-POSTGRES_URI_SUPERTOKENS=postgresql://username:password@postgres:5432/agenta_oss_supertokens
-POSTGRES_URI_CORE=postgresql+asyncpg://username:password@postgres:5432/agenta_oss_core
-POSTGRES_URI_TRACING=postgresql+asyncpg://username:password@postgres:5432/agenta_oss_tracing
-
-ALEMBIC_CFG_PATH_CORE=/app/oss/databases/postgres/migrations/core/alembic.ini
-ALEMBIC_CFG_PATH_TRACING=/app/oss/databases/postgres/migrations/tracing/alembic.ini
-
-SUPERTOKENS_CONNECTION_URI=http://supertokens:3567
-
-POSTHOG_API_KEY=phc_hmVSxIjTW1REBHXgj2aw4HW9X6CXb6FzerBgP9XenC7
-
-# Third-party (required for TLS/SSL)
-AGENTA_SSL_DIR=
-
-# Third-party (optional)
-NGINX_PORT=80
-SUPERTOKENS_API_KEY=
-RABBITMQ_PORT=
-RABBITMQ_UI_PORT=
-REDIS_CACHE_HOST=
-REDIS_CACHE_PORT=
-POSTGRES_PORT=
-SUPERTOKENS_PORT=
-
-GOOGLE_OAUTH_CLIENT_ID=
-GOOGLE_OAUTH_CLIENT_SECRET=
-GITHUB_OAUTH_CLIENT_ID=
-GITHUB_OAUTH_CLIENT_SECRET=
-
-NEW_RELIC_LICENSE_KEY=
-NRIA_LICENSE_KEY=
-
-LOOPS_API_KEY=
-
-SENDGRID_API_KEY=
-
-CRISP_WEBSITE_ID=
-
-STRIPE_API_KEY=
-STRIPE_WEBHOOK_SECRET=
-
-# Third-party — LLM (optional)
-ALEPHALPHA_API_KEY=
-ANTHROPIC_API_KEY=
-ANYSCALE_API_KEY=
-COHERE_API_KEY=
-DEEPINFRA_API_KEY=
-GEMINI_API_KEY=
-GROQ_API_KEY=
-MISTRAL_API_KEY=
-OPENAI_API_KEY=
-OPENROUTER_API_KEY=
-PERPLEXITYAI_API_KEY=
-TOGETHERAI_API_KEY=
-
-# Legacy (deprecated, to be removed)
-AGENTA_PORT=80
-BARE_DOMAIN_NAME=localhost
-DOMAIN_NAME=http://localhost
-WEBSITE_DOMAIN_NAME=http://localhost
-SERVICE_URL_TEMPLATE=http://localhost:80/services/{path}
-POSTGRES_DB=agenta_oss
-POSTGRES_URI=postgresql+asyncpg://username:password@postgres:5432/agenta_oss
-ALEMBIC_CFG_PATH=/app/oss/databases/postgres/migrations/alembic.oss.ini
diff --git a/hosting/old/gcp/agenta-instance.tf b/hosting/gcp/agenta-instance.tf
similarity index 100%
rename from hosting/old/gcp/agenta-instance.tf
rename to hosting/gcp/agenta-instance.tf
diff --git a/hosting/old/gcp/credentials.json b/hosting/gcp/credentials.json
similarity index 100%
rename from hosting/old/gcp/credentials.json
rename to hosting/gcp/credentials.json
diff --git a/hosting/old/gcp/main.tf b/hosting/gcp/main.tf
similarity index 100%
rename from hosting/old/gcp/main.tf
rename to hosting/gcp/main.tf
diff --git a/sdk/README.md b/sdk/README.md
index d07ac59e52..4424f046d2 100644
--- a/sdk/README.md
+++ b/sdk/README.md
@@ -2,12 +2,11 @@
+ )
+}
+
+export default memo(InvocationRun)
diff --git a/web/oss/src/components/EvalRunDetails/HumanEvalRun/components/EvalRunScenarioCard/assets/KeyValue.tsx b/web/ee/src/components/EvalRunDetails/components/EvalRunScenarioCard/assets/KeyValue.tsx
similarity index 100%
rename from web/oss/src/components/EvalRunDetails/HumanEvalRun/components/EvalRunScenarioCard/assets/KeyValue.tsx
rename to web/ee/src/components/EvalRunDetails/components/EvalRunScenarioCard/assets/KeyValue.tsx
diff --git a/web/oss/src/components/EvalRunDetails/HumanEvalRun/components/EvalRunScenarioCard/assets/utils.tsx b/web/ee/src/components/EvalRunDetails/components/EvalRunScenarioCard/assets/utils.tsx
similarity index 100%
rename from web/oss/src/components/EvalRunDetails/HumanEvalRun/components/EvalRunScenarioCard/assets/utils.tsx
rename to web/ee/src/components/EvalRunDetails/components/EvalRunScenarioCard/assets/utils.tsx
diff --git a/web/ee/src/components/EvalRunDetails/components/EvalRunScenarioCard/index.tsx b/web/ee/src/components/EvalRunDetails/components/EvalRunScenarioCard/index.tsx
new file mode 100644
index 0000000000..1cca7fc3ac
--- /dev/null
+++ b/web/ee/src/components/EvalRunDetails/components/EvalRunScenarioCard/index.tsx
@@ -0,0 +1,62 @@
+import {memo, useMemo} from "react"
+
+import {Card} from "antd"
+import deepEqual from "fast-deep-equal"
+import {useAtomValue} from "jotai"
+import {selectAtom} from "jotai/utils"
+
+import {evaluationRunStateAtom} from "@/oss/lib/hooks/useEvaluationRunData/assets/atoms"
+import {EvaluationRunState} from "@/oss/lib/hooks/useEvaluationRunData/types"
+
+import EvalRunScenarioCardTitle from "../EvalRunScenarioCardTitle"
+import RunEvalScenarioButton from "../RunEvalScenarioButton"
+
+import EvalRunScenarioCardBody from "./EvalRunScenarioCardBody"
+import {EvalRunScenarioCardProps} from "./types"
+
+/**
+ * Component that renders a card view for a specific evaluation run scenario.
+ * Depending on the `viewType`, it can display the scenario in a card format
+ * or a full-width format. Utilizes data from Jotai atoms to display scenario
+ * details, including loading state and error handling.
+ *
+ * @param {string} scenarioId - The unique identifier for the scenario to be displayed.
+ * @param {ViewType} [viewType="list"] - Determines the layout of the scenario display,
+ * either as a "list" (card format) or "single" (full-width).
+ */
+const EvalRunScenarioCard = ({scenarioId, viewType = "list"}: EvalRunScenarioCardProps) => {
+ /* scenario index for card title */
+ const scenarioIndex = useAtomValue(
+ useMemo(
+ () =>
+ selectAtom(
+ evaluationRunStateAtom,
+ (state: EvaluationRunState) =>
+ state.scenarios?.find((s) => s.id === scenarioId)?.scenarioIndex,
+ deepEqual,
+ ),
+ [scenarioId],
+ ),
+ )
+
+ if (scenarioIndex === undefined) return null
+
+ return viewType === "list" ? (
+
+ }
+ style={{width: 400}}
+ className="self-stretch"
+ actions={[]}
+ >
+
+
+ ) : (
+
-
- {/* Text - centered in the remaining space */}
- ![\"Agenta](\"./images/agenta-openinference-agno-trace.png\"\n",)
\n",
- "\n",
- "The observability interface provides insights for:\n",
- "- Debug complex agent interactions and tool usage patterns\n",
- "- Monitor tool execution performance and decision logic\n",
- "- Analyze LLM reasoning and response quality\n",
- "- Track multi-tool workflows and coordination"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Advanced Usage\n",
- "\n",
- "### Custom Span Configuration\n",
- "\n",
- "Customize instrumentation for different application components:"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "@ag.instrument(spankind=\"workflow\")\n",
- "def logistics_pipeline(request: str):\n",
- " return handle_dispatch_request(request)\n",
- "\n",
- "\n",
- "@ag.instrument(spankind=\"tool\")\n",
- "def external_system_integration(data: dict):\n",
- " # External system call logic\n",
- " pass\n",
- "\n",
- "\n",
- "@ag.instrument(spankind=\"agent\")\n",
- "def specialized_logistics_agent(context: dict):\n",
- " # Specialized agent implementation\n",
- " return context"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### Real-world Examples\n",
- "\n",
- "#### Customer Service Agent System"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "# Customer service data\n",
- "customer_db = {\n",
- " \"CUST001\": {\"name\": \"John Doe\", \"tier\": \"Premium\", \"issues\": []},\n",
- " \"CUST002\": {\"name\": \"Jane Smith\", \"tier\": \"Standard\", \"issues\": [\"billing\"]},\n",
- "}\n",
- "\n",
- "ticket_system = {\n",
- " \"TKT001\": {\"customer\": \"CUST001\", \"status\": \"open\", \"priority\": \"high\"},\n",
- " \"TKT002\": {\"customer\": \"CUST002\", \"status\": \"resolved\", \"priority\": \"medium\"},\n",
- "}\n",
- "\n",
- "class CustomerLookupTool:\n",
- " def __init__(self):\n",
- " self.name = \"CustomerLookupTool\"\n",
- " self.description = \"Retrieves customer information by ID.\"\n",
- " \n",
- " def run(self, query: str) -> str:\n",
- " match = re.search(r\"\\bCUST\\d+\\b\", query.upper())\n",
- " if not match:\n",
- " return \"Please provide a valid customer ID.\"\n",
- "\n",
- " cust_id = match.group(0)\n",
- " customer = customer_db.get(cust_id)\n",
- " if customer:\n",
- " return f\"Customer {cust_id}: {customer['name']}, Tier: {customer['tier']}\"\n",
- " return f\"No customer found for {cust_id}.\"\n",
- "\n",
- "class TicketManagementTool:\n",
- " def __init__(self):\n",
- " self.name = \"TicketManagementTool\"\n",
- " self.description = \"Manages support tickets and escalations.\"\n",
- " \n",
- " def run(self, query: str) -> str:\n",
- " if \"create ticket\" in query.lower():\n",
- " return \"New ticket TKT003 created successfully.\"\n",
- "\n",
- " elif \"escalate\" in query.lower():\n",
- " return \"Ticket escalated to senior support team.\"\n",
- "\n",
- " match = re.search(r\"\\bTKT\\d+\\b\", query.upper())\n",
- " if match:\n",
- " ticket_id = match.group(0)\n",
- " ticket = ticket_system.get(ticket_id)\n",
- " if ticket:\n",
- " return f\"Ticket {ticket_id}: Status: {ticket['status']}, Priority: {ticket['priority']}\"\n",
- "\n",
- " return \"Please specify a valid ticket action or ID.\"\n",
- "\n",
- "\n",
- "customer_service_agent = Agent(\n",
- " model=OpenAIChat(id=\"gpt-4o\"),\n",
- " description=\"You are a customer service agent helping with inquiries and ticket management.\",\n",
- " instructions=[\n",
- " \"Use CustomerLookupTool to find customer information.\",\n",
- " \"Use TicketManagementTool to manage support tickets.\",\n",
- " \"Always be helpful and professional in your responses.\",\n",
- " ],\n",
- " tools=[CustomerLookupTool(), TicketManagementTool()],\n",
- " show_tool_calls=False,\n",
- ")\n",
- "\n",
- "\n",
- "@ag.instrument(spankind=\"workflow\")\n",
- "def handle_customer_request(query: str):\n",
- " result = customer_service_agent.run(query)\n",
- " return result.content"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "#### Test Customer Service Agent"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "# Test customer service functionality\n",
- "customer_response = handle_customer_request(\"Look up customer CUST001 and check ticket TKT001\")\n",
- "print(\"Customer Service Response:\", customer_response)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "#### E-commerce Order Management System"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "# E-commerce data\n",
- "inventory = {\n",
- " \"PROD001\": {\"name\": \"Laptop\", \"stock\": 5, \"price\": 999.99},\n",
- " \"PROD002\": {\"name\": \"Mouse\", \"stock\": 25, \"price\": 29.99},\n",
- " \"PROD003\": {\"name\": \"Keyboard\", \"stock\": 0, \"price\": 79.99},\n",
- "}\n",
- "\n",
- "orders = {\n",
- " \"ORD001\": {\"customer\": \"CUST001\", \"items\": [\"PROD001\"], \"status\": \"shipped\"},\n",
- " \"ORD002\": {\"customer\": \"CUST002\", \"items\": [\"PROD002\"], \"status\": \"processing\"},\n",
- "}\n",
- "\n",
- "class InventoryTool:\n",
- " def __init__(self):\n",
- " self.name = \"InventoryTool\"\n",
- " self.description = \"Checks product availability and stock levels.\"\n",
- " \n",
- " def run(self, query: str) -> str:\n",
- " match = re.search(r\"\\bPROD\\d+\\b\", query.upper())\n",
- " if not match:\n",
- " return \"Please provide a valid product ID.\"\n",
- "\n",
- " prod_id = match.group(0)\n",
- " product = inventory.get(prod_id)\n",
- " if product:\n",
- " status = \"In Stock\" if product[\"stock\"] > 0 else \"Out of Stock\"\n",
- " return f\"Product {prod_id}: {product['name']}, Stock: {product['stock']}, Price: ${product['price']}, Status: {status}\"\n",
- "\n",
- " return f\"Product {prod_id} not found.\"\n",
- "\n",
- "\n",
- "class OrderManagementTool:\n",
- " def __init__(self):\n",
- " self.name = \"OrderManagementTool\"\n",
- " self.description = \"Manages order status and updates.\"\n",
- " \n",
- " def run(self, query: str) -> str:\n",
- " match = re.search(r\"\\bORD\\d+\\b\", query.upper())\n",
- " if not match:\n",
- " return \"Please provide a valid order ID.\"\n",
- "\n",
- " order_id = match.group(0)\n",
- " order = orders.get(order_id)\n",
- " if order:\n",
- " return f\"Order {order_id}: Customer: {order['customer']}, Items: {order['items']}, Status: {order['status']}\"\n",
- " return f\"Order {order_id} not found.\"\n",
- "\n",
- "\n",
- "ecommerce_agent = Agent(\n",
- " model=OpenAIChat(id=\"gpt-4o\"),\n",
- " description=\"You are an e-commerce assistant helping with inventory and order management.\",\n",
- " instructions=[\n",
- " \"Use InventoryTool to check product availability.\",\n",
- " \"Use OrderManagementTool to track orders.\",\n",
- " \"Provide clear and helpful information about products and orders.\",\n",
- " ],\n",
- " tools=[InventoryTool(), OrderManagementTool()],\n",
- " show_tool_calls=False,\n",
- ")\n",
- "\n",
- "\n",
- "@ag.instrument(spankind=\"agent\")\n",
- "def handle_ecommerce_request(query: str):\n",
- " result = ecommerce_agent.run(query)\n",
- " return result.content"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "#### Test E-commerce Agent"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "# Test e-commerce functionality\n",
- "ecommerce_response = handle_ecommerce_request(\"Check availability of PROD001 and status of order ORD001\")\n",
- "print(\"E-commerce Response:\", ecommerce_response)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "#### Financial Trading Assistant"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "# Financial data\n",
- "portfolio = {\n",
- " \"AAPL\": {\"shares\": 100, \"avg_price\": 150.00, \"current_price\": 175.50},\n",
- " \"GOOGL\": {\"shares\": 50, \"avg_price\": 2800.00, \"current_price\": 2950.00},\n",
- " \"TSLA\": {\"shares\": 25, \"avg_price\": 800.00, \"current_price\": 725.00},\n",
- "}\n",
- "\n",
- "market_data = {\n",
- " \"AAPL\": {\"price\": 175.50, \"change\": \"+2.5%\", \"volume\": \"45M\"},\n",
- " \"GOOGL\": {\"price\": 2950.00, \"change\": \"+1.8%\", \"volume\": \"12M\"},\n",
- " \"TSLA\": {\"price\": 725.00, \"change\": \"-3.2%\", \"volume\": \"28M\"},\n",
- "}\n",
- "\n",
- "class PortfolioTool:\n",
- " def __init__(self):\n",
- " self.name = \"PortfolioTool\"\n",
- " self.description = \"Provides portfolio information and performance analysis.\"\n",
- " \n",
- " def run(self, query: str) -> str:\n",
- " if \"portfolio\" in query.lower():\n",
- " total_value = sum(stock[\"shares\"] * stock[\"current_price\"] for stock in portfolio.values())\n",
- " return f\"Portfolio Total Value: ${total_value:,.2f}\"\n",
- " \n",
- " match = re.search(r\"\\b[A-Z]{2,5}\\b\", query.upper())\n",
- " if match:\n",
- " symbol = match.group(0)\n",
- " if symbol in portfolio:\n",
- " stock = portfolio[symbol]\n",
- " current_value = stock[\"shares\"] * stock[\"current_price\"]\n",
- " return f\"{symbol}: {stock['shares']} shares @ ${stock['current_price']}, Value: ${current_value:,.2f}\"\n",
- "\n",
- " return \"Please specify a valid stock symbol or ask about your portfolio.\"\n",
- "\n",
- "\n",
- "class MarketDataTool:\n",
- " def __init__(self):\n",
- " self.name = \"MarketDataTool\"\n",
- " self.description = \"Provides real-time market data and stock information.\"\n",
- " \n",
- " def run(self, query: str) -> str:\n",
- " match = re.search(r\"\\b[A-Z]{2,5}\\b\", query.upper())\n",
- " if not match:\n",
- " return \"Please provide a valid stock symbol.\"\n",
- "\n",
- " symbol = match.group(0)\n",
- " if symbol in market_data:\n",
- " data = market_data[symbol]\n",
- " return f\"{symbol}: ${data['price']}, Change: {data['change']}, Volume: {data['volume']}\"\n",
- " return f\"No market data available for {symbol}.\"\n",
- "\n",
- "\n",
- "trading_agent = Agent(\n",
- " model=OpenAIChat(id=\"gpt-4o\"),\n",
- " description=\"You are a financial trading assistant providing portfolio and market analysis.\",\n",
- " instructions=[\n",
- " \"Use PortfolioTool to analyze portfolio holdings.\",\n",
- " \"Use MarketDataTool to get current market information.\",\n",
- " \"Provide clear financial insights and recommendations.\",\n",
- " ],\n",
- " tools=[PortfolioTool(), MarketDataTool()],\n",
- " show_tool_calls=False,\n",
- ")\n",
- "\n",
- "\n",
- "@ag.instrument(spankind=\"workflow\")\n",
- "def handle_trading_request(query: str):\n",
- " result = trading_agent.run(query)\n",
- " return result.content"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "#### Test Trading Agent"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "# Test trading functionality\n",
- "trading_response = handle_trading_request(\"Show me my AAPL holdings and current market data for GOOGL\")\n",
- "print(\"Trading Response:\", trading_response)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### Additional Testing Examples"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "# Test portfolio overview\n",
- "portfolio_overview = handle_trading_request(\"What is my total portfolio value?\")\n",
- "print(\"Portfolio Overview:\", portfolio_overview)"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "# Test customer service ticket creation\n",
- "ticket_creation = handle_customer_request(\"Create a new ticket for billing issues\")\n",
- "print(\"Ticket Creation:\", ticket_creation)"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "# Test inventory check for out of stock item\n",
- "inventory_check = handle_ecommerce_request(\"Check if PROD003 is available for purchase\")\n",
- "print(\"Inventory Check:\", inventory_check)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Next Steps\n",
- "\n",
- "For more detailed information about Agenta's observability features and advanced configuration options, visit the [Agenta Observability SDK Documentation](/observability/observability-sdk)."
- ]
- }
- ],
- "metadata": {
- "kernelspec": {
- "display_name": "Python 3",
- "language": "python",
- "name": "python3"
- },
- "language_info": {
- "codemirror_mode": {
- "name": "ipython",
- "version": 3
- },
- "file_extension": ".py",
- "mimetype": "text/x-python",
- "name": "python",
- "nbconvert_exporter": "python",
- "pygments_lexer": "ipython3",
- "version": "3.8.0"
- }
- },
- "nbformat": 4,
- "nbformat_minor": 4
-}
diff --git a/examples/jupyter/integrations/dspy-integration.ipynb b/examples/jupyter/integrations/dspy-integration.ipynb
index 6909808cf5..ee91151c3e 100644
--- a/examples/jupyter/integrations/dspy-integration.ipynb
+++ b/examples/jupyter/integrations/dspy-integration.ipynb
@@ -73,9 +73,9 @@
"\n",
"# Load configuration from environment\n",
"os.environ[\"AGENTA_API_KEY\"] = \"your_agenta_api_key\"\n",
- "os.environ[\n",
- " \"AGENTA_HOST\"\n",
- "] = \"https://cloud.agenta.ai\" # Optional, defaults to the Agenta cloud API\n",
+ "os.environ[\"AGENTA_HOST\"] = (\n",
+ " \"https://cloud.agenta.ai\" # Optional, defaults to the Agenta cloud API\n",
+ ")\n",
"os.environ[\"OPENAI_API_KEY\"] = \"your_openai_api_key\" # Required for OpenAI models\n",
"\n",
"\n",
@@ -197,7 +197,7 @@
"source": [
"class Outline(dspy.Signature):\n",
" \"\"\"Outline a thorough overview of a topic.\"\"\"\n",
- "\n",
+ " \n",
" topic: str = dspy.InputField()\n",
" title: str = dspy.OutputField()\n",
" sections: list[str] = dspy.OutputField()\n",
@@ -208,7 +208,7 @@
"\n",
"class DraftSection(dspy.Signature):\n",
" \"\"\"Draft a top-level section of an article.\"\"\"\n",
- "\n",
+ " \n",
" topic: str = dspy.InputField()\n",
" section_heading: str = dspy.InputField()\n",
" section_subheadings: list[str] = dspy.InputField()\n",
@@ -219,7 +219,7 @@
" def __init__(self):\n",
" self.build_outline = dspy.ChainOfThought(Outline)\n",
" self.draft_section = dspy.ChainOfThought(DraftSection)\n",
- "\n",
+ " \n",
" def forward(self, topic):\n",
" outline = self.build_outline(topic=topic)\n",
" sections = []\n",
diff --git a/examples/jupyter/integrations/images/agenta-openinference-agno-trace.png b/examples/jupyter/integrations/images/agenta-openinference-agno-trace.png
deleted file mode 100644
index 204c694f7a..0000000000
Binary files a/examples/jupyter/integrations/images/agenta-openinference-agno-trace.png and /dev/null differ
diff --git a/examples/jupyter/integrations/langgraph-integration.ipynb b/examples/jupyter/integrations/langgraph-integration.ipynb
index 30797cc017..4737aaf71e 100644
--- a/examples/jupyter/integrations/langgraph-integration.ipynb
+++ b/examples/jupyter/integrations/langgraph-integration.ipynb
@@ -81,9 +81,9 @@
"\n",
"# Load configuration from environment\n",
"os.environ[\"AGENTA_API_KEY\"] = \"your_agenta_api_key\"\n",
- "os.environ[\n",
- " \"AGENTA_HOST\"\n",
- "] = \"https://cloud.agenta.ai\" # Optional, defaults to the Agenta cloud API\n",
+ "os.environ[\"AGENTA_HOST\"] = (\n",
+ " \"https://cloud.agenta.ai\" # Optional, defaults to the Agenta cloud API\n",
+ ")\n",
"os.environ[\"OPENAI_API_KEY\"] = \"your_openai_api_key\" # Required for OpenAI Agents SDK\n",
"\n",
"\n",
@@ -256,12 +256,12 @@
" builder.add_node(\"segment\", RunnableLambda(segment_by_speaker))\n",
" builder.add_node(\"summarize\", RunnableLambda(summarize_per_speaker))\n",
" builder.add_node(\"extract_actions\", RunnableLambda(extract_actions))\n",
- "\n",
+ " \n",
" builder.set_entry_point(\"segment\")\n",
" builder.add_edge(\"segment\", \"summarize\")\n",
" builder.add_edge(\"summarize\", \"extract_actions\")\n",
" builder.add_edge(\"extract_actions\", END)\n",
- "\n",
+ " \n",
" graph = builder.compile()\n",
" result = graph.invoke({\"input\": transcript})\n",
" return result"
@@ -428,18 +428,14 @@
"def determine_priority(state):\n",
" sentiment = state[\"sentiment\"]\n",
" categories = state[\"categories\"]\n",
- " result = llm.invoke(\n",
- " f\"Determine priority (high/medium/low) based on sentiment: {sentiment} and categories: {categories}\"\n",
- " )\n",
+ " result = llm.invoke(f\"Determine priority (high/medium/low) based on sentiment: {sentiment} and categories: {categories}\")\n",
" return {**state, \"priority\": result.content}\n",
"\n",
"\n",
"def draft_response(state):\n",
" feedback = state[\"input\"]\n",
" sentiment = state[\"sentiment\"]\n",
- " result = llm.invoke(\n",
- " f\"Draft a professional response to this {sentiment} feedback: {feedback}\"\n",
- " )\n",
+ " result = llm.invoke(f\"Draft a professional response to this {sentiment} feedback: {feedback}\")\n",
" return {**state, \"response_draft\": result.content}\n",
"\n",
"\n",
@@ -450,13 +446,13 @@
" builder.add_node(\"categorize\", RunnableLambda(categorize_feedback))\n",
" builder.add_node(\"priority\", RunnableLambda(determine_priority))\n",
" builder.add_node(\"response\", RunnableLambda(draft_response))\n",
- "\n",
+ " \n",
" builder.set_entry_point(\"sentiment\")\n",
" builder.add_edge(\"sentiment\", \"categorize\")\n",
" builder.add_edge(\"categorize\", \"priority\")\n",
" builder.add_edge(\"priority\", \"response\")\n",
" builder.add_edge(\"response\", END)\n",
- "\n",
+ " \n",
" graph = builder.compile()\n",
" return graph.invoke({\"input\": feedback_text})"
]
@@ -510,51 +506,47 @@
"def research_analyzer(paper_text: str):\n",
" def extract_abstract(state):\n",
" paper = state[\"input\"]\n",
- " result = llm.invoke(\n",
- " f\"Extract and summarize the abstract from this research paper: {paper}\"\n",
- " )\n",
+ " result = llm.invoke(f\"Extract and summarize the abstract from this research paper: {paper}\")\n",
" return {**state, \"abstract_summary\": result.content}\n",
- "\n",
+ " \n",
" def identify_findings(state):\n",
" paper = state[\"input\"]\n",
" result = llm.invoke(f\"List the key findings from this research paper: {paper}\")\n",
" return {**state, \"key_findings\": result.content.split(\"\\n\")}\n",
- "\n",
+ " \n",
" def analyze_methodology(state):\n",
" paper = state[\"input\"]\n",
" result = llm.invoke(f\"Describe the methodology used in this research: {paper}\")\n",
" return {**state, \"methodology\": result.content}\n",
- "\n",
+ " \n",
" def assess_limitations(state):\n",
" paper = state[\"input\"]\n",
" result = llm.invoke(f\"Identify limitations mentioned in this research: {paper}\")\n",
" return {**state, \"limitations\": result.content}\n",
- "\n",
+ " \n",
" def score_relevance(state):\n",
" abstract = state[\"abstract_summary\"]\n",
- " result = llm.invoke(\n",
- " f\"Rate the relevance of this research on a scale of 0-10: {abstract}\"\n",
- " )\n",
+ " result = llm.invoke(f\"Rate the relevance of this research on a scale of 0-10: {abstract}\")\n",
" try:\n",
" score = float(result.content.strip())\n",
" except:\n",
" score = 5.0\n",
" return {**state, \"relevance_score\": score}\n",
- "\n",
+ " \n",
" builder = StateGraph(ResearchState)\n",
" builder.add_node(\"abstract\", RunnableLambda(extract_abstract))\n",
" builder.add_node(\"findings\", RunnableLambda(identify_findings))\n",
" builder.add_node(\"methodology\", RunnableLambda(analyze_methodology))\n",
" builder.add_node(\"limitations\", RunnableLambda(assess_limitations))\n",
" builder.add_node(\"relevance\", RunnableLambda(score_relevance))\n",
- "\n",
+ " \n",
" builder.set_entry_point(\"abstract\")\n",
" builder.add_edge(\"abstract\", \"findings\")\n",
" builder.add_edge(\"findings\", \"methodology\")\n",
" builder.add_edge(\"methodology\", \"limitations\")\n",
" builder.add_edge(\"limitations\", \"relevance\")\n",
" builder.add_edge(\"relevance\", END)\n",
- "\n",
+ " \n",
" graph = builder.compile()\n",
" return graph.invoke({\"input\": paper_text})"
]
@@ -590,14 +582,12 @@
" except:\n",
" score = 0.0\n",
" return {**state, \"toxicity_score\": score}\n",
- "\n",
+ " \n",
" def categorize_content(state):\n",
" content = state[\"input\"]\n",
- " result = llm.invoke(\n",
- " f\"Categorize this content (spam, harassment, hate speech, etc.): {content}\"\n",
- " )\n",
+ " result = llm.invoke(f\"Categorize this content (spam, harassment, hate speech, etc.): {content}\")\n",
" return {**state, \"content_categories\": result.content.split(\", \")}\n",
- "\n",
+ " \n",
" def determine_action(state):\n",
" toxicity = state[\"toxicity_score\"]\n",
" categories = state[\"content_categories\"]\n",
@@ -607,21 +597,19 @@
" action = \"flag_for_review\"\n",
" else:\n",
" action = \"approve\"\n",
- " explanation = (\n",
- " f\"Decision based on toxicity score: {toxicity} and categories: {categories}\"\n",
- " )\n",
+ " explanation = f\"Decision based on toxicity score: {toxicity} and categories: {categories}\"\n",
" return {**state, \"action_required\": action, \"explanation\": explanation}\n",
- "\n",
+ " \n",
" builder = StateGraph(ModerationState)\n",
" builder.add_node(\"toxicity\", RunnableLambda(assess_toxicity))\n",
" builder.add_node(\"categorize\", RunnableLambda(categorize_content))\n",
" builder.add_node(\"action\", RunnableLambda(determine_action))\n",
- "\n",
+ " \n",
" builder.set_entry_point(\"toxicity\")\n",
" builder.add_edge(\"toxicity\", \"categorize\")\n",
" builder.add_edge(\"categorize\", \"action\")\n",
" builder.add_edge(\"action\", END)\n",
- "\n",
+ " \n",
" graph = builder.compile()\n",
" return graph.invoke({\"input\": user_content})"
]
diff --git a/examples/jupyter/integrations/observability-openinference-llamaindex.ipynb b/examples/jupyter/integrations/observability-openinference-llamaindex.ipynb
index 9b4dfe2aa1..4d033df3f0 100644
--- a/examples/jupyter/integrations/observability-openinference-llamaindex.ipynb
+++ b/examples/jupyter/integrations/observability-openinference-llamaindex.ipynb
@@ -71,9 +71,9 @@
"\n",
"# Load configuration from environment\n",
"os.environ[\"AGENTA_API_KEY\"] = \"your_agenta_api_key\"\n",
- "os.environ[\n",
- " \"AGENTA_HOST\"\n",
- "] = \"https://cloud.agenta.ai\" # Optional, defaults to the Agenta cloud API\n",
+ "os.environ[\"AGENTA_HOST\"] = (\n",
+ " \"https://cloud.agenta.ai\" # Optional, defaults to the Agenta cloud API\n",
+ ")\n",
"\n",
"\n",
"# Start Agenta SDK\n",
@@ -132,9 +132,9 @@
"\n",
"# Configuration setup\n",
"os.environ[\"AGENTA_API_KEY\"] = \"your_agenta_api_key\"\n",
- "os.environ[\n",
- " \"AGENTA_HOST\"\n",
- "] = \"https://cloud.agenta.ai\" # Optional, defaults to the Agenta cloud API\n",
+ "os.environ[\"AGENTA_HOST\"] = (\n",
+ " \"https://cloud.agenta.ai\" # Optional, defaults to the Agenta cloud API\n",
+ ")\n",
"\n",
"# Initialize observability\n",
"ag.init()\n",
@@ -151,16 +151,16 @@
" \"\"\"\n",
" # Load documents from local directory\n",
" docs = SimpleDirectoryReader(\"data\").load_data()\n",
- "\n",
+ " \n",
" # Build vector search index\n",
" search_index = VectorStoreIndex.from_documents(docs)\n",
- "\n",
+ " \n",
" # Initialize query processor\n",
" query_processor = search_index.as_query_engine()\n",
- "\n",
+ " \n",
" # Process user query\n",
" answer = query_processor.query(user_query)\n",
- "\n",
+ " \n",
" return answer\n",
"\n",
"\n",
diff --git a/examples/jupyter/integrations/openai-agents-integration.ipynb b/examples/jupyter/integrations/openai-agents-integration.ipynb
index 976b0d3160..605e482325 100644
--- a/examples/jupyter/integrations/openai-agents-integration.ipynb
+++ b/examples/jupyter/integrations/openai-agents-integration.ipynb
@@ -74,9 +74,9 @@
"\n",
"# Load configuration from environment\n",
"os.environ[\"AGENTA_API_KEY\"] = \"your_agenta_api_key\"\n",
- "os.environ[\n",
- " \"AGENTA_HOST\"\n",
- "] = \"https://cloud.agenta.ai\" # Optional, defaults to the Agenta cloud API\n",
+ "os.environ[\"AGENTA_HOST\"] = (\n",
+ " \"https://cloud.agenta.ai\" # Optional, defaults to the Agenta cloud API\n",
+ ")\n",
"os.environ[\"OPENAI_API_KEY\"] = \"your_openai_api_key\" # Required for OpenAI Agents SDK\n",
"\n",
"\n",
@@ -203,7 +203,6 @@
" response = await duolingo(\"What is 'What's your name' in French?\")\n",
" print(\"Response:\", response)\n",
"\n",
- "\n",
"# Run the example\n",
"await main()"
]
@@ -331,7 +330,7 @@
")\n",
"\n",
"billing_agent = Agent(\n",
- " name=\"Billing Agent\",\n",
+ " name=\"Billing Agent\", \n",
" instructions=\"You handle billing-related questions and payment issues\",\n",
")\n",
"\n",
@@ -340,25 +339,18 @@
" instructions=\"You provide technical support and troubleshooting assistance\",\n",
")\n",
"\n",
- "\n",
"@ag.instrument(spankind=\"workflow\")\n",
"async def customer_support_system(customer_query: str):\n",
" orchestrator = Agent(\n",
" name=\"Support Orchestrator\",\n",
" instructions=\"Route customer queries to the appropriate specialist agent\",\n",
" tools=[\n",
- " support_agent.as_tool(\n",
- " \"general_support\", \"Handle general customer inquiries\"\n",
- " ),\n",
- " billing_agent.as_tool(\n",
- " \"billing_support\", \"Handle billing and payment issues\"\n",
- " ),\n",
- " technical_agent.as_tool(\n",
- " \"technical_support\", \"Provide technical assistance\"\n",
- " ),\n",
- " ],\n",
+ " support_agent.as_tool(\"general_support\", \"Handle general customer inquiries\"),\n",
+ " billing_agent.as_tool(\"billing_support\", \"Handle billing and payment issues\"),\n",
+ " technical_agent.as_tool(\"technical_support\", \"Provide technical assistance\"),\n",
+ " ]\n",
" )\n",
- "\n",
+ " \n",
" result = await Runner.run(orchestrator, input=customer_query)\n",
" return result.final_output"
]
@@ -382,21 +374,21 @@
" name=\"Data Collector\",\n",
" instructions=\"Gather relevant information and data on the given topic\",\n",
" )\n",
- "\n",
+ " \n",
" analyst = Agent(\n",
- " name=\"Research Analyst\",\n",
+ " name=\"Research Analyst\", \n",
" instructions=\"Analyze collected data and provide insights\",\n",
" )\n",
- "\n",
+ " \n",
" reporter = Agent(\n",
" name=\"Report Writer\",\n",
" instructions=\"Create comprehensive reports based on analysis\",\n",
" tools=[\n",
" data_collector.as_tool(\"collect_data\", \"Gather relevant research data\"),\n",
" analyst.as_tool(\"analyze_data\", \"Perform data analysis and insights\"),\n",
- " ],\n",
+ " ]\n",
" )\n",
- "\n",
+ " \n",
" result = await Runner.run(reporter, input=research_topic)\n",
" return result.final_output"
]
@@ -420,17 +412,17 @@
" name=\"Content Writer\",\n",
" instructions=\"Create engaging content based on the brief\",\n",
" )\n",
- "\n",
+ " \n",
" editor = Agent(\n",
" name=\"Content Editor\",\n",
" instructions=\"Review and improve content quality and style\",\n",
" )\n",
- "\n",
+ " \n",
" seo_specialist = Agent(\n",
" name=\"SEO Specialist\",\n",
" instructions=\"Optimize content for search engines\",\n",
" )\n",
- "\n",
+ " \n",
" content_manager = Agent(\n",
" name=\"Content Manager\",\n",
" instructions=\"Coordinate the content creation process and ensure quality\",\n",
@@ -438,9 +430,9 @@
" writer.as_tool(\"write_content\", \"Generate initial content draft\"),\n",
" editor.as_tool(\"edit_content\", \"Review and edit content\"),\n",
" seo_specialist.as_tool(\"optimize_seo\", \"Optimize content for SEO\"),\n",
- " ],\n",
+ " ]\n",
" )\n",
- "\n",
+ " \n",
" result = await Runner.run(content_manager, input=content_brief)\n",
" return result.final_output"
]
@@ -459,9 +451,7 @@
"outputs": [],
"source": [
"# Test customer support system\n",
- "support_response = await customer_support_system(\n",
- " \"I have a billing question about my subscription\"\n",
- ")\n",
+ "support_response = await customer_support_system(\"I have a billing question about my subscription\")\n",
"print(\"Support Response:\", support_response)"
]
},
@@ -483,9 +473,7 @@
"outputs": [],
"source": [
"# Test content creation system\n",
- "content_response = await content_creation_system(\n",
- " \"Write a blog post about sustainable energy solutions\"\n",
- ")\n",
+ "content_response = await content_creation_system(\"Write a blog post about sustainable energy solutions\")\n",
"print(\"Content Response:\", content_response)"
]
},
diff --git a/examples/jupyter/integrations/pydanticai-integration.ipynb b/examples/jupyter/integrations/pydanticai-integration.ipynb
index 630bcddcee..6da49ce54c 100644
--- a/examples/jupyter/integrations/pydanticai-integration.ipynb
+++ b/examples/jupyter/integrations/pydanticai-integration.ipynb
@@ -76,16 +76,16 @@
"\n",
"# Load configuration from environment\n",
"os.environ[\"AGENTA_API_KEY\"] = \"your_agenta_api_key\"\n",
- "os.environ[\n",
- " \"AGENTA_HOST\"\n",
- "] = \"https://cloud.agenta.ai\" # Optional, defaults to the Agenta cloud API\n",
+ "os.environ[\"AGENTA_HOST\"] = \"https://cloud.agenta.ai\" # Optional, defaults to the Agenta cloud API\n",
"\n",
"# Initialize Agenta SDK\n",
"ag.init()\n",
"\n",
"# Configure Logfire for local development\n",
"logfire.configure(\n",
- " service_name=\"my_logfire_service\", send_to_logfire=False, scrubbing=False\n",
+ " service_name=\"my_logfire_service\", \n",
+ " send_to_logfire=False, \n",
+ " scrubbing=False\n",
")"
]
},
@@ -139,13 +139,13 @@
"# Mock database for demonstration\n",
"class DatabaseConn:\n",
" \"\"\"Fake database for example purposes.\"\"\"\n",
- "\n",
+ " \n",
" @classmethod\n",
" async def customer_name(cls, *, id: int) -> str | None:\n",
" if id == 123:\n",
" return \"John\"\n",
" return None\n",
- "\n",
+ " \n",
" @classmethod\n",
" async def customer_balance(cls, *, id: int, include_pending: bool) -> float:\n",
" if id == 123 and include_pending:\n",
@@ -464,12 +464,12 @@
"def route_customer_query(customer_id: int, query: str, query_type: str = \"general\"):\n",
" \"\"\"Route customer queries to appropriate specialized agents.\"\"\"\n",
" deps = CustomerServiceDeps(customer_id=customer_id, db=DatabaseConn())\n",
- "\n",
+ " \n",
" if query_type == \"technical\":\n",
" result = technical_agent.run_sync(query, deps=deps)\n",
" else:\n",
" result = general_agent.run_sync(query, deps=deps)\n",
- "\n",
+ " \n",
" return result"
]
},
@@ -561,13 +561,11 @@
"\n",
"\n",
"@ag.instrument(spankind=\"agent\")\n",
- "def analyze_transaction(\n",
- " transaction_id: str, customer_id: int, amount: float, location: str\n",
- "):\n",
+ "def analyze_transaction(transaction_id: str, customer_id: int, amount: float, location: str):\n",
" \"\"\"Analyze a transaction for potential fraud.\"\"\"\n",
" deps = FraudDeps(transaction_id=transaction_id, customer_id=customer_id)\n",
" query = f\"Analyze transaction {transaction_id}: ${amount} in {location}\"\n",
- "\n",
+ " \n",
" result = fraud_agent.run_sync(query, deps=deps)\n",
" return result"
]
@@ -666,9 +664,7 @@
"outputs": [],
"source": [
"# Test investment advice\n",
- "investment_result = get_investment_advice(\n",
- " 123, 50000.0, \"I want to diversify my portfolio for retirement\"\n",
- ")\n",
+ "investment_result = get_investment_advice(123, 50000.0, \"I want to diversify my portfolio for retirement\")\n",
"print(\"Investment Advisory Result:\")\n",
"print(f\"Recommendation: {investment_result.output.recommendation}\")\n",
"print(f\"Risk Level: {investment_result.output.risk_level}\")\n",
diff --git a/examples/jupyter/observability/analytics-api-tutorial.ipynb b/examples/jupyter/observability/analytics-api-tutorial.ipynb
deleted file mode 100644
index d5097dfbf6..0000000000
--- a/examples/jupyter/observability/analytics-api-tutorial.ipynb
+++ /dev/null
@@ -1,714 +0,0 @@
-{
- "cells": [
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "# Analytics API - Tutorial\n",
- "\n",
- "This tutorial demonstrates how to use the Agenta Analytics API to analyze LLM performance metrics. You'll learn how to:\n",
- "\n",
- "- Retrieve aggregated metrics over time\n",
- "- Analyze costs, latency, and token usage\n",
- "- Filter analytics by status and other attributes\n",
- "- Track error trends and failure rates\n",
- "- Compare performance across different time periods\n",
- "\n",
- "## What You'll Build\n",
- "\n",
- "We'll create analytics queries that:\n",
- "1. Track daily LLM costs and spending trends\n",
- "2. Monitor error rates and identify peak error times\n",
- "3. Analyze token usage patterns\n",
- "4. Compare performance metrics over time\n",
- "5. Generate cost reports and visualizations"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Setup\n",
- "\n",
- "Before using the API, you need your Agenta API key. You can create API keys from the Settings page in your Agenta workspace."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "import os\n",
- "import requests\n",
- "import json\n",
- "from datetime import datetime, timedelta, timezone\n",
- "from getpass import getpass\n",
- "\n",
- "# Configuration\n",
- "AGENTA_HOST = os.getenv(\"AGENTA_HOST\", \"https://cloud.agenta.ai\")\n",
- "api_key = os.getenv(\"AGENTA_API_KEY\")\n",
- "if not api_key:\n",
- " api_key = getpass(\"Enter your Agenta API key: \")\n",
- " os.environ[\"AGENTA_API_KEY\"] = api_key\n",
- "\n",
- "# Setup base configuration\n",
- "BASE_URL = f\"{AGENTA_HOST}/api/preview/tracing/spans/analytics\"\n",
- "HEADERS = {\n",
- " \"Authorization\": f\"ApiKey {api_key}\",\n",
- " \"Content-Type\": \"application/json\"\n",
- "}\n",
- "\n",
- "print(\"✅ Setup complete!\")\n",
- "print(f\"API endpoint: {BASE_URL}\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Part 1: Get Recent Metrics\n",
- "\n",
- "Let's start by retrieving metrics for the last 7 days with daily buckets. Each bucket contains aggregated metrics for all traces within that day."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "# Get analytics for last 7 days with daily buckets\n",
- "newest = datetime.now(timezone.utc)\n",
- "oldest = newest - timedelta(days=7)\n",
- "\n",
- "payload = {\n",
- " \"focus\": \"trace\",\n",
- " \"interval\": 1440, # 1440 minutes = daily buckets\n",
- " \"windowing\": {\n",
- " \"oldest\": oldest.isoformat(),\n",
- " \"newest\": newest.isoformat()\n",
- " }\n",
- "}\n",
- "\n",
- "response = requests.post(BASE_URL, headers=HEADERS, json=payload)\n",
- "data = response.json()\n",
- "\n",
- "print(f\"📊 Found {data['count']} daily buckets\\n\")\n",
- "\n",
- "# Show all days with activity\n",
- "for bucket in data['buckets']:\n",
- " if bucket['total']['count'] > 0:\n",
- " date = bucket['timestamp'][:10]\n",
- " print(f\"Date: {date}\")\n",
- " print(f\" Traces: {bucket['total']['count']}\")\n",
- " print(f\" Cost: ${bucket['total']['costs']:.4f}\")\n",
- " print(f\" Tokens: {bucket['total']['tokens']:,.0f}\")\n",
- " print(f\" Errors: {bucket['errors']['count']}\\n\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Part 2: Track Daily Costs\n",
- "\n",
- "Calculate total costs and generate summary statistics over a time period."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "# Get daily metrics for last 30 days\n",
- "newest = datetime.now(timezone.utc)\n",
- "oldest = newest - timedelta(days=30)\n",
- "\n",
- "payload = {\n",
- " \"focus\": \"trace\",\n",
- " \"interval\": 1440, # Daily buckets\n",
- " \"windowing\": {\n",
- " \"oldest\": oldest.isoformat(),\n",
- " \"newest\": newest.isoformat()\n",
- " }\n",
- "}\n",
- "\n",
- "response = requests.post(BASE_URL, headers=HEADERS, json=payload)\n",
- "data = response.json()\n",
- "\n",
- "# Calculate totals\n",
- "total_traces = sum(b['total']['count'] for b in data['buckets'])\n",
- "total_cost = sum(b['total']['costs'] for b in data['buckets'])\n",
- "total_tokens = sum(b['total']['tokens'] for b in data['buckets'])\n",
- "total_errors = sum(b['errors']['count'] for b in data['buckets'])\n",
- "\n",
- "print(\"💰 Cost Summary (Last 30 Days)\")\n",
- "print(\"=\" * 50)\n",
- "print(f\"Total Cost: ${total_cost:.2f}\")\n",
- "print(f\"Total Requests: {total_traces:,}\")\n",
- "if total_traces > 0:\n",
- " print(f\"Average Cost per Request: ${total_cost/total_traces:.6f}\")\n",
- " print(f\"Total Tokens: {total_tokens:,.0f}\")\n",
- " print(f\"Average Tokens per Request: {total_tokens/total_traces:.1f}\")\n",
- " print(f\"Error Rate: {(total_errors/total_traces)*100:.2f}%\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Part 3: Analyze Error Trends\n",
- "\n",
- "Monitor error rates over time to identify patterns and peak error times."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "# Get hourly metrics for last 7 days\n",
- "newest = datetime.now(timezone.utc)\n",
- "oldest = newest - timedelta(days=7)\n",
- "\n",
- "payload = {\n",
- " \"focus\": \"trace\",\n",
- " \"interval\": 60, # Hourly buckets\n",
- " \"windowing\": {\n",
- " \"oldest\": oldest.isoformat(),\n",
- " \"newest\": newest.isoformat()\n",
- " }\n",
- "}\n",
- "\n",
- "response = requests.post(BASE_URL, headers=HEADERS, json=payload)\n",
- "data = response.json()\n",
- "\n",
- "print(\"🚨 Error Analysis\")\n",
- "print(\"=\" * 50)\n",
- "\n",
- "# Find hours with high error rates\n",
- "high_error_periods = []\n",
- "for bucket in data['buckets']:\n",
- " if bucket['total']['count'] > 0:\n",
- " error_rate = (bucket['errors']['count'] / bucket['total']['count']) * 100\n",
- " if error_rate > 5: # Flag periods with > 5% errors\n",
- " high_error_periods.append({\n",
- " 'time': bucket['timestamp'],\n",
- " 'error_rate': error_rate,\n",
- " 'total': bucket['total']['count'],\n",
- " 'errors': bucket['errors']['count']\n",
- " })\n",
- "\n",
- "if high_error_periods:\n",
- " print(f\"\\nFound {len(high_error_periods)} periods with high error rates (>5%):\\n\")\n",
- " for period in high_error_periods[:10]: # Show top 10\n",
- " print(f\" {period['time']}\")\n",
- " print(f\" Error Rate: {period['error_rate']:.1f}%\")\n",
- " print(f\" Total: {period['total']}, Errors: {period['errors']}\\n\")\n",
- "else:\n",
- " print(\"✅ No high error rates detected in the last 7 days\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Part 4: Filter by Status Code\n",
- "\n",
- "Analyze only successful traces by filtering on status code."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "# Get successful traces only\n",
- "newest = datetime.now(timezone.utc)\n",
- "oldest = newest - timedelta(days=7)\n",
- "\n",
- "payload = {\n",
- " \"focus\": \"trace\",\n",
- " \"interval\": 1440, # Daily buckets\n",
- " \"windowing\": {\n",
- " \"oldest\": oldest.isoformat(),\n",
- " \"newest\": newest.isoformat()\n",
- " },\n",
- " \"filter\": {\n",
- " \"conditions\": [\n",
- " {\n",
- " \"field\": \"status.code\",\n",
- " \"operator\": \"eq\",\n",
- " \"value\": \"STATUS_CODE_OK\"\n",
- " }\n",
- " ]\n",
- " }\n",
- "}\n",
- "\n",
- "response = requests.post(BASE_URL, headers=HEADERS, json=payload)\n",
- "data = response.json()\n",
- "\n",
- "# Calculate success metrics\n",
- "total_count = sum(b['total']['count'] for b in data['buckets'])\n",
- "total_cost = sum(b['total']['costs'] for b in data['buckets'])\n",
- "total_duration = sum(b['total']['duration'] for b in data['buckets'])\n",
- "\n",
- "print(\"✅ Successful Traces (Last 7 Days)\")\n",
- "print(\"=\" * 50)\n",
- "print(f\"Count: {total_count:,}\")\n",
- "print(f\"Total Cost: ${total_cost:.4f}\")\n",
- "if total_count > 0:\n",
- " print(f\"Avg Duration: {total_duration/total_count:.0f}ms\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Part 5: Track Token Usage\n",
- "\n",
- "Monitor token consumption patterns over time."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "# Get daily token usage for last 7 days\n",
- "newest = datetime.now(timezone.utc)\n",
- "oldest = newest - timedelta(days=7)\n",
- "\n",
- "payload = {\n",
- " \"focus\": \"trace\",\n",
- " \"interval\": 1440, # Daily buckets\n",
- " \"windowing\": {\n",
- " \"oldest\": oldest.isoformat(),\n",
- " \"newest\": newest.isoformat()\n",
- " }\n",
- "}\n",
- "\n",
- "response = requests.post(BASE_URL, headers=HEADERS, json=payload)\n",
- "data = response.json()\n",
- "\n",
- "print(\"🎯 Token Usage Analysis\")\n",
- "print(\"=\" * 50)\n",
- "print(\"\\nDaily Token Usage:\\n\")\n",
- "\n",
- "for bucket in data['buckets']:\n",
- " if bucket['total']['count'] > 0:\n",
- " date = bucket['timestamp'][:10]\n",
- " avg_tokens = bucket['total']['tokens'] / bucket['total']['count']\n",
- " print(f\" {date}: {bucket['total']['tokens']:>8,.0f} total ({avg_tokens:>6.0f} avg)\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Part 6: Analyze Performance\n",
- "\n",
- "Track latency trends over time to identify performance changes."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "# Get hourly performance for last 24 hours\n",
- "newest = datetime.now(timezone.utc)\n",
- "oldest = newest - timedelta(days=1)\n",
- "\n",
- "payload = {\n",
- " \"focus\": \"trace\",\n",
- " \"interval\": 60, # Hourly buckets\n",
- " \"windowing\": {\n",
- " \"oldest\": oldest.isoformat(),\n",
- " \"newest\": newest.isoformat()\n",
- " }\n",
- "}\n",
- "\n",
- "response = requests.post(BASE_URL, headers=HEADERS, json=payload)\n",
- "data = response.json()\n",
- "\n",
- "print(\"⚡ Performance Analysis (Last 24 Hours)\")\n",
- "print(\"=\" * 50)\n",
- "print(\"\\nHourly Average Latency:\\n\")\n",
- "\n",
- "latencies = []\n",
- "for bucket in data['buckets']:\n",
- " if bucket['total']['count'] > 0:\n",
- " avg_duration = bucket['total']['duration'] / bucket['total']['count']\n",
- " latencies.append(avg_duration)\n",
- " hour = bucket['timestamp'][11:16] # Extract HH:MM\n",
- " print(f\" {hour}: {avg_duration:7.0f}ms\")\n",
- "\n",
- "if latencies:\n",
- " print(f\"\\n📈 Statistics:\")\n",
- " print(f\" Min: {min(latencies):.0f}ms\")\n",
- " print(f\" Max: {max(latencies):.0f}ms\")\n",
- " print(f\" Avg: {sum(latencies)/len(latencies):.0f}ms\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Part 7: Generate Monthly Cost Report\n",
- "\n",
- "Create a comprehensive monthly report with cost breakdown and usage statistics."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "# Get monthly metrics\n",
- "newest = datetime.now(timezone.utc)\n",
- "oldest = newest - timedelta(days=30)\n",
- "\n",
- "payload = {\n",
- " \"focus\": \"trace\",\n",
- " \"interval\": 1440, # Daily buckets\n",
- " \"windowing\": {\n",
- " \"oldest\": oldest.isoformat(),\n",
- " \"newest\": newest.isoformat()\n",
- " }\n",
- "}\n",
- "\n",
- "response = requests.post(BASE_URL, headers=HEADERS, json=payload)\n",
- "data = response.json()\n",
- "\n",
- "# Calculate totals\n",
- "total_traces = sum(b['total']['count'] for b in data['buckets'])\n",
- "total_cost = sum(b['total']['costs'] for b in data['buckets'])\n",
- "total_tokens = sum(b['total']['tokens'] for b in data['buckets'])\n",
- "total_duration = sum(b['total']['duration'] for b in data['buckets'])\n",
- "total_errors = sum(b['errors']['count'] for b in data['buckets'])\n",
- "\n",
- "print(\"📊 MONTHLY COST REPORT\")\n",
- "print(\"=\" * 60)\n",
- "print(f\"Period: {oldest.strftime('%Y-%m-%d')} to {newest.strftime('%Y-%m-%d')}\")\n",
- "print(\"=\" * 60)\n",
- "\n",
- "print(\"\\n💰 Cost Summary:\")\n",
- "print(f\" Total Cost: ${total_cost:.2f}\")\n",
- "if total_traces > 0:\n",
- " print(f\" Average Cost per Request: ${total_cost/total_traces:.6f}\")\n",
- "daily_cost = total_cost / 30\n",
- "print(f\" Average Daily Cost: ${daily_cost:.2f}\")\n",
- "print(f\" Projected Monthly Cost: ${daily_cost * 30:.2f}\")\n",
- "\n",
- "print(\"\\n📊 Usage Statistics:\")\n",
- "print(f\" Total Requests: {total_traces:,}\")\n",
- "successful = total_traces - total_errors\n",
- "print(f\" Successful: {successful:,}\")\n",
- "print(f\" Failed: {total_errors:,}\")\n",
- "if total_traces > 0:\n",
- " print(f\" Failure Rate: {(total_errors/total_traces)*100:.2f}%\")\n",
- " print(f\" Average Daily Requests: {total_traces/30:.0f}\")\n",
- "\n",
- "print(\"\\n🎯 Performance Metrics:\")\n",
- "if total_traces > 0:\n",
- " print(f\" Average Latency: {total_duration/total_traces:.0f}ms\")\n",
- "print(f\" Total Tokens: {total_tokens:,.0f}\")\n",
- "if total_traces > 0:\n",
- " print(f\" Average Tokens per Request: {total_tokens/total_traces:.1f}\")\n",
- " print(f\" Average Daily Tokens: {total_tokens/30:,.0f}\")\n",
- "\n",
- "# Cost per 1K tokens\n",
- "if total_tokens > 0:\n",
- " cost_per_1k = (total_cost / total_tokens) * 1000\n",
- " print(f\" Cost per 1K Tokens: ${cost_per_1k:.4f}\")\n",
- "\n",
- "# Find most expensive days\n",
- "print(\"\\n📅 Top 5 Most Expensive Days:\")\n",
- "days_with_data = [(b['timestamp'][:10], b['total']['costs'], b['total']['count']) \n",
- " for b in data['buckets'] if b['total']['count'] > 0]\n",
- "sorted_days = sorted(days_with_data, key=lambda x: x[1], reverse=True)\n",
- "for i, (date, cost, count) in enumerate(sorted_days[:5], 1):\n",
- " print(f\" {i}. {date}: ${cost:.4f} ({count} requests)\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Part 8: Compare Week-over-Week Performance\n",
- "\n",
- "Analyze how metrics change from one week to the next."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "# Helper function to get weekly metrics\n",
- "def get_weekly_metrics(weeks_ago=0):\n",
- " newest = datetime.now(timezone.utc) - timedelta(weeks=weeks_ago)\n",
- " oldest = newest - timedelta(days=7)\n",
- " \n",
- " payload = {\n",
- " \"focus\": \"trace\",\n",
- " \"interval\": 10080, # Weekly bucket\n",
- " \"windowing\": {\n",
- " \"oldest\": oldest.isoformat(),\n",
- " \"newest\": newest.isoformat()\n",
- " }\n",
- " }\n",
- " \n",
- " response = requests.post(BASE_URL, headers=HEADERS, json=payload)\n",
- " data = response.json()\n",
- " \n",
- " if data['buckets']:\n",
- " bucket = data['buckets'][0]\n",
- " return {\n",
- " 'count': bucket['total']['count'],\n",
- " 'costs': bucket['total']['costs'],\n",
- " 'duration': bucket['total']['duration'],\n",
- " 'tokens': bucket['total']['tokens'],\n",
- " 'errors': bucket['errors']['count']\n",
- " }\n",
- " return None\n",
- "\n",
- "this_week = get_weekly_metrics(0)\n",
- "last_week = get_weekly_metrics(1)\n",
- "\n",
- "def calc_change(current, previous):\n",
- " if previous == 0:\n",
- " return \"N/A\"\n",
- " change = ((current - previous) / previous) * 100\n",
- " symbol = \"📈\" if change > 0 else \"📉\" if change < 0 else \"➡️\"\n",
- " return f\"{symbol} {change:+.1f}%\"\n",
- "\n",
- "print(\"📊 Week-over-Week Comparison\")\n",
- "print(\"=\" * 60)\n",
- "\n",
- "if this_week and last_week:\n",
- " print(\"\\n💰 Cost:\")\n",
- " print(f\" Last Week: ${last_week['costs']:.4f}\")\n",
- " print(f\" This Week: ${this_week['costs']:.4f}\")\n",
- " print(f\" Change: {calc_change(this_week['costs'], last_week['costs'])}\")\n",
- "\n",
- " print(\"\\n📊 Volume:\")\n",
- " print(f\" Last Week: {last_week['count']:,} requests\")\n",
- " print(f\" This Week: {this_week['count']:,} requests\")\n",
- " print(f\" Change: {calc_change(this_week['count'], last_week['count'])}\")\n",
- "\n",
- " print(\"\\n⚡ Performance:\")\n",
- " last_avg = last_week['duration'] / last_week['count'] if last_week['count'] > 0 else 0\n",
- " this_avg = this_week['duration'] / this_week['count'] if this_week['count'] > 0 else 0\n",
- " print(f\" Last Week: {last_avg:.0f}ms\")\n",
- " print(f\" This Week: {this_avg:.0f}ms\")\n",
- " print(f\" Change: {calc_change(this_avg, last_avg)}\")\n",
- "\n",
- " print(\"\\n🚨 Error Rate:\")\n",
- " last_err_rate = (last_week['errors'] / last_week['count'] * 100) if last_week['count'] > 0 else 0\n",
- " this_err_rate = (this_week['errors'] / this_week['count'] * 100) if this_week['count'] > 0 else 0\n",
- " print(f\" Last Week: {last_err_rate:.2f}%\")\n",
- " print(f\" This Week: {this_err_rate:.2f}%\")\n",
- " print(f\" Change: {calc_change(this_err_rate, last_err_rate)}\")\n",
- "else:\n",
- " print(\"\\n⚠️ Not enough data for comparison\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Part 9: Create Visualizations\n",
- "\n",
- "Visualize cost and usage trends using matplotlib."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "try:\n",
- " import matplotlib.pyplot as plt\n",
- " import matplotlib.dates as mdates\n",
- " from datetime import datetime\n",
- " \n",
- " # Get daily metrics for last 30 days\n",
- " newest = datetime.now(timezone.utc)\n",
- " oldest = newest - timedelta(days=30)\n",
- "\n",
- " payload = {\n",
- " \"focus\": \"trace\",\n",
- " \"interval\": 1440, # Daily buckets\n",
- " \"windowing\": {\n",
- " \"oldest\": oldest.isoformat(),\n",
- " \"newest\": newest.isoformat()\n",
- " }\n",
- " }\n",
- "\n",
- " response = requests.post(BASE_URL, headers=HEADERS, json=payload)\n",
- " data = response.json()\n",
- "\n",
- " # Extract dates and metrics\n",
- " dates = [datetime.fromisoformat(b['timestamp'].replace('Z', '+00:00')) \n",
- " for b in data['buckets'] if b['total']['count'] > 0]\n",
- " costs = [b['total']['costs'] for b in data['buckets'] if b['total']['count'] > 0]\n",
- " counts = [b['total']['count'] for b in data['buckets'] if b['total']['count'] > 0]\n",
- "\n",
- " # Create figure with two subplots\n",
- " fig, (ax1, ax2) = plt.subplots(2, 1, figsize=(12, 8))\n",
- "\n",
- " # Plot 1: Daily Cost\n",
- " ax1.plot(dates, costs, marker='o', linewidth=2, markersize=4, color='#2563eb')\n",
- " ax1.set_title('Daily LLM Costs (Last 30 Days)', fontsize=14, fontweight='bold')\n",
- " ax1.set_ylabel('Cost ($)', fontsize=12)\n",
- " ax1.grid(True, alpha=0.3)\n",
- " ax1.xaxis.set_major_formatter(mdates.DateFormatter('%m/%d'))\n",
- " plt.setp(ax1.xaxis.get_majorticklabels(), rotation=45)\n",
- "\n",
- " # Plot 2: Daily Request Volume\n",
- " ax2.bar(dates, counts, alpha=0.7, color='steelblue')\n",
- " ax2.set_title('Daily Request Volume (Last 30 Days)', fontsize=14, fontweight='bold')\n",
- " ax2.set_xlabel('Date', fontsize=12)\n",
- " ax2.set_ylabel('Requests', fontsize=12)\n",
- " ax2.grid(True, alpha=0.3, axis='y')\n",
- " ax2.xaxis.set_major_formatter(mdates.DateFormatter('%m/%d'))\n",
- " plt.setp(ax2.xaxis.get_majorticklabels(), rotation=45)\n",
- "\n",
- " plt.tight_layout()\n",
- " plt.show()\n",
- " \n",
- " print(\"✅ Visualizations created successfully!\")\n",
- " \n",
- "except ImportError:\n",
- " print(\"⚠️ matplotlib not installed. Run: pip install matplotlib\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Part 10: Export Data to DataFrame\n",
- "\n",
- "Convert analytics data to a pandas DataFrame for further analysis."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "try:\n",
- " import pandas as pd\n",
- " \n",
- " # Get daily metrics for last 30 days\n",
- " newest = datetime.now(timezone.utc)\n",
- " oldest = newest - timedelta(days=30)\n",
- "\n",
- " payload = {\n",
- " \"focus\": \"trace\",\n",
- " \"interval\": 1440, # Daily buckets\n",
- " \"windowing\": {\n",
- " \"oldest\": oldest.isoformat(),\n",
- " \"newest\": newest.isoformat()\n",
- " }\n",
- " }\n",
- "\n",
- " response = requests.post(BASE_URL, headers=HEADERS, json=payload)\n",
- " data = response.json()\n",
- "\n",
- " # Convert to DataFrame\n",
- " rows = []\n",
- " for bucket in data['buckets']:\n",
- " if bucket['total']['count'] > 0: # Only include days with data\n",
- " rows.append({\n",
- " 'timestamp': bucket['timestamp'],\n",
- " 'total_count': bucket['total']['count'],\n",
- " 'total_cost': bucket['total']['costs'],\n",
- " 'total_duration': bucket['total']['duration'],\n",
- " 'total_tokens': bucket['total']['tokens'],\n",
- " 'error_count': bucket['errors']['count'],\n",
- " 'error_duration': bucket['errors']['duration'],\n",
- " 'avg_duration': bucket['total']['duration'] / bucket['total']['count'],\n",
- " 'avg_cost': bucket['total']['costs'] / bucket['total']['count'],\n",
- " 'error_rate': (bucket['errors']['count'] / bucket['total']['count'] * 100)\n",
- " })\n",
- "\n",
- " df = pd.DataFrame(rows)\n",
- " df['timestamp'] = pd.to_datetime(df['timestamp'])\n",
- "\n",
- " print(\"📊 Analytics Data Summary\\n\")\n",
- " print(df.describe())\n",
- " \n",
- " print(\"\\n📅 Recent Days:\")\n",
- " print(df.tail(10).to_string())\n",
- " \n",
- " # Optional: Save to CSV\n",
- " # df.to_csv('analytics_export.csv', index=False)\n",
- " # print(\"\\n✅ Data exported to analytics_export.csv\")\n",
- " \n",
- "except ImportError:\n",
- " print(\"⚠️ pandas not installed. Run: pip install pandas\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Summary\n",
- "\n",
- "In this tutorial, you learned how to:\n",
- "\n",
- "1. ✅ **Retrieve aggregated metrics** using the Analytics API\n",
- "2. ✅ **Track daily costs** and generate spending reports\n",
- "3. ✅ **Analyze error trends** to identify reliability issues\n",
- "4. ✅ **Filter by status code** to analyze successful vs failed traces\n",
- "5. ✅ **Track token usage** patterns over time\n",
- "6. ✅ **Monitor performance** and latency trends\n",
- "7. ✅ **Generate monthly reports** with comprehensive cost breakdowns\n",
- "8. ✅ **Compare week-over-week** metrics to identify trends\n",
- "9. ✅ **Visualize data** using matplotlib\n",
- "10. ✅ **Export to DataFrame** for further analysis\n",
- "\n",
- "## Next Steps\n",
- "\n",
- "- Learn about [Query API](/observability/query-data/query-api) for detailed trace analysis\n",
- "- Explore [Using the UI](/observability/using-the-ui/filtering-traces) for visual analytics\n",
- "- Check out [Semantic Conventions](/observability/concepts/semantic-conventions) for available metrics\n",
- "- Read about [Cost Tracking](/observability/trace-with-python-sdk/track-costs) for automatic cost calculation"
- ]
- }
- ],
- "metadata": {
- "kernelspec": {
- "display_name": "Python 3",
- "language": "python",
- "name": "python3"
- },
- "language_info": {
- "codemirror_mode": {
- "name": "ipython",
- "version": 3
- },
- "file_extension": ".py",
- "mimetype": "text/x-python",
- "name": "python",
- "nbconvert_exporter": "python",
- "pygments_lexer": "ipython3",
- "version": "3.9.0"
- }
- },
- "nbformat": 4,
- "nbformat_minor": 4
-}
diff --git a/examples/jupyter/observability/annotate-traces-tutorial.ipynb b/examples/jupyter/observability/annotate-traces-tutorial.ipynb
deleted file mode 100644
index 4fccc69fd3..0000000000
--- a/examples/jupyter/observability/annotate-traces-tutorial.ipynb
+++ /dev/null
@@ -1,438 +0,0 @@
-{
- "cells": [
- {
- "cell_type": "markdown",
- "id": "intro",
- "metadata": {},
- "source": [
- "# Annotate Traces Tutorial\n",
- "\n",
- "Annotations in Agenta let you enrich the traces created by your LLM applications. You can add scores, comments, expected answers and other metrics to help evaluate your application's performance.\n",
- "\n",
- "In this tutorial, we'll:\n",
- "1. Set up the Agenta SDK and create a traced LLM application\n",
- "2. Run the application to generate traces\n",
- "3. Add annotations to those traces programmatically\n",
- "4. Query and view the annotations\n",
- "\n",
- "## What You Can Do With Annotations\n",
- "\n",
- "- Collect user feedback on LLM responses\n",
- "- Run custom evaluation workflows\n",
- "- Measure application performance in real-time"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "install",
- "metadata": {},
- "source": [
- "## Step 1: Install Required Packages\n",
- "\n",
- "First, install the Agenta SDK, OpenAI, and the OpenTelemetry instrumentor:"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "install-deps",
- "metadata": {},
- "outputs": [],
- "source": [
- "%pip install -U agenta openai opentelemetry-instrumentation-openai requests"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "setup",
- "metadata": {},
- "source": [
- "## Step 2: Configure Environment Variables\n",
- "\n",
- "To start tracing your application and adding annotations, you'll need an API key:\n",
- "\n",
- "1. Visit the Agenta API Keys page under settings\n",
- "2. Click on **Create New API Key** and follow the prompts\n",
- "\n",
- "Then set your environment variables:"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "env-setup",
- "metadata": {},
- "outputs": [],
- "source": [
- "import os\n",
- "\n",
- "# Set your API keys here\n",
- "os.environ[\"AGENTA_API_KEY\"] = \"\"\n",
- "os.environ[\"AGENTA_HOST\"] = \"https://cloud.agenta.ai\" # Change for self-hosted\n",
- "os.environ[\"OPENAI_API_KEY\"] = \"\""
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "init-sdk",
- "metadata": {},
- "outputs": [],
- "source": [
- "import agenta as ag\n",
- "from getpass import getpass\n",
- "\n",
- "# Initialize the SDK with your API key\n",
- "api_key = os.getenv(\"AGENTA_API_KEY\")\n",
- "if not api_key:\n",
- " os.environ[\"AGENTA_API_KEY\"] = getpass(\"Enter your Agenta API key: \")\n",
- "\n",
- "openai_api_key = os.getenv(\"OPENAI_API_KEY\")\n",
- "if not openai_api_key:\n",
- " os.environ[\"OPENAI_API_KEY\"] = getpass(\"Enter your OpenAI API key: \")\n",
- "\n",
- "# Initialize Agenta\n",
- "ag.init()"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "instrument",
- "metadata": {},
- "source": [
- "## Step 3: Create and Instrument an LLM Application\n",
- "\n",
- "Let's create a simple LLM application that we can trace and annotate:"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "setup-openai",
- "metadata": {},
- "outputs": [],
- "source": [
- "import openai\n",
- "from opentelemetry.instrumentation.openai import OpenAIInstrumentor\n",
- "\n",
- "# Instrument OpenAI to automatically capture traces\n",
- "OpenAIInstrumentor().instrument()"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "create-function",
- "metadata": {},
- "outputs": [],
- "source": [
- "@ag.instrument()\n",
- "def answer_question(question: str) -> tuple[str, str, str]:\n",
- " \"\"\"A simple question-answering function that we'll trace and annotate.\n",
- " \n",
- " Returns:\n",
- " Tuple of (answer, trace_id, span_id)\n",
- " \"\"\"\n",
- " response = openai.chat.completions.create(\n",
- " model=\"gpt-3.5-turbo\",\n",
- " messages=[\n",
- " {\"role\": \"system\", \"content\": \"You are a helpful assistant that answers questions concisely.\"},\n",
- " {\"role\": \"user\", \"content\": question},\n",
- " ],\n",
- " )\n",
- " \n",
- " # Automatically get the trace_id and span_id from the current span\n",
- " link = ag.tracing.build_invocation_link()\n",
- " \n",
- " return response.choices[0].message.content, link.trace_id, link.span_id"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "generate-trace",
- "metadata": {},
- "source": [
- "## Step 4: Generate a Trace\n",
- "\n",
- "Let's run our function to generate a trace. The function will automatically capture the trace_id and span_id using `ag.tracing.build_invocation_link()`."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "run-function",
- "metadata": {},
- "outputs": [],
- "source": [
- "# Run the function to create a trace and get the IDs automatically\n",
- "question = \"What is the capital of France?\"\n",
- "result, trace_id, span_id = answer_question(question)\n",
- "\n",
- "print(f\"Question: {question}\")\n",
- "print(f\"Answer: {result}\")\n",
- "print(f\"\\n✅ Trace captured!\")\n",
- "print(f\"Trace ID: {trace_id}\")\n",
- "print(f\"Span ID: {span_id}\")"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "create-annotation",
- "metadata": {},
- "source": [
- "## Step 5: Create an Annotation\n",
- "\n",
- "Now let's add an annotation to the trace we just created. We'll use the trace_id and span_id that were automatically captured."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "annotate-trace",
- "metadata": {},
- "outputs": [],
- "source": "import requests\n\nbase_url = os.environ.get(\"AGENTA_HOST\", \"https://cloud.agenta.ai\")\napi_key = os.environ[\"AGENTA_API_KEY\"]\n\nheaders = {\n \"Content-Type\": \"application/json\",\n \"Authorization\": f\"ApiKey {api_key}\"\n}\n\n# Create an annotation with a score and reasoning\nannotation_data = {\n \"annotation\": {\n \"data\": {\n \"outputs\": {\n \"score\": 90,\n \"normalized_score\": 0.9,\n \"reasoning\": \"The answer is correct and concise\",\n \"expected_answer\": \"The capital of France is Paris\"\n }\n },\n \"references\": {\n \"evaluator\": {\n \"slug\": \"accuracy_evaluator\"\n }\n },\n \"links\": {\n \"invocation\": {\n \"trace_id\": trace_id,\n \"span_id\": span_id\n }\n },\n \"metadata\": {\n \"annotator\": \"tutorial_user\",\n \"timestamp\": \"2025-10-30T00:00:00Z\"\n }\n }\n}\n\n# Make the API request (note the trailing slash!)\nresponse = requests.post(\n f\"{base_url}/api/preview/annotations/\",\n headers=headers,\n json=annotation_data\n)\n\n# Process the response\nif response.status_code == 200:\n print(\"✅ Annotation created successfully!\")\n annotation_response = response.json()\n print(f\"\\nAnnotation ID: {annotation_response['annotation']['trace_id']}\")\n print(f\"Span ID: {annotation_response['annotation']['span_id']}\")\n print(f\"\\nAnnotation data:\")\n print(annotation_response)\nelse:\n print(f\"❌ Error: {response.status_code}\")\n print(response.text)"
- },
- {
- "cell_type": "markdown",
- "id": "multiple-annotations",
- "metadata": {},
- "source": [
- "## Step 6: Create Additional Annotations\n",
- "\n",
- "You can add multiple annotations to the same trace from different evaluators:"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "create-second-annotation",
- "metadata": {},
- "outputs": [],
- "source": "# Create another annotation for quality assessment\nquality_annotation = {\n \"annotation\": {\n \"data\": {\n \"outputs\": {\n \"score\": 85,\n \"reasoning\": \"Response is helpful and well-formatted\",\n \"labels\": [\"Helpful\", \"Accurate\", \"Concise\"]\n }\n },\n \"references\": {\n \"evaluator\": {\n \"slug\": \"quality_evaluator\"\n }\n },\n \"links\": {\n \"invocation\": {\n \"trace_id\": trace_id,\n \"span_id\": span_id\n }\n }\n }\n}\n\nresponse = requests.post(\n f\"{base_url}/api/preview/annotations/\", # Note the trailing slash!\n headers=headers,\n json=quality_annotation\n)\n\nif response.status_code == 200:\n print(\"✅ Quality annotation created successfully!\")\nelse:\n print(f\"❌ Error: {response.status_code}\")\n print(response.text)"
- },
- {
- "cell_type": "markdown",
- "id": "query-annotations",
- "metadata": {},
- "source": [
- "## Step 7: Query Annotations\n",
- "\n",
- "Now let's query all annotations for our invocation:"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "query-by-invocation",
- "metadata": {},
- "outputs": [],
- "source": [
- "# Query all annotations for the invocation\n",
- "query_data = {\n",
- " \"annotation\": {\n",
- " \"links\": {\n",
- " \"invocation\": {\n",
- " \"trace_id\": trace_id,\n",
- " \"span_id\": span_id\n",
- " }\n",
- " }\n",
- " }\n",
- "}\n",
- "\n",
- "response = requests.post(\n",
- " f\"{base_url}/api/preview/annotations/query\",\n",
- " headers=headers,\n",
- " json=query_data\n",
- ")\n",
- "\n",
- "if response.status_code == 200:\n",
- " print(\"✅ Annotations retrieved successfully!\")\n",
- " annotations = response.json()\n",
- " print(f\"\\nFound {len(annotations.get('annotations', []))} annotation(s)\")\n",
- " print(\"\\nAnnotations:\")\n",
- " for idx, ann in enumerate(annotations.get('annotations', []), 1):\n",
- " print(f\"\\n--- Annotation {idx} ---\")\n",
- " print(f\"Evaluator: {ann['references']['evaluator']['slug']}\")\n",
- " print(f\"Data: {ann['data']}\")\n",
- "else:\n",
- " print(f\"❌ Error: {response.status_code}\")\n",
- " print(response.text)"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "view-ui",
- "metadata": {},
- "source": [
- "## Step 8: View Annotations in the UI\n",
- "\n",
- "You can see all annotations for a trace in the Agenta UI:\n",
- "\n",
- "1. Log in to your Agenta dashboard\n",
- "2. Navigate to the **Observability** section\n",
- "3. Find your trace\n",
- "4. Check the **Annotations** tab to see detailed information\n",
- "\n",
- "The right sidebar will show average metrics for each evaluator."
- ]
- },
- {
- "cell_type": "markdown",
- "id": "understanding-structure",
- "metadata": {},
- "source": [
- "## Understanding Automatic Trace Capture\n",
- "\n",
- "The `ag.tracing.build_invocation_link()` function is a helper that automatically:\n",
- "1. Gets the current span context from the active trace\n",
- "2. Formats the trace_id and span_id as hex strings\n",
- "3. Returns a Link object with both IDs ready to use\n",
- "\n",
- "This is much more convenient than manually querying the UI for trace IDs!\n",
- "\n",
- "**Alternative Method:**\n",
- "You can also use `ag.tracing.get_span_context()` if you need more control:\n",
- "\n",
- "```python\n",
- "span_ctx = ag.tracing.get_span_context()\n",
- "trace_id = f\"{span_ctx.trace_id:032x}\" # Format as hexadecimal\n",
- "span_id = f\"{span_ctx.span_id:016x}\" # Format as hexadecimal\n",
- "```\n",
- "\n",
- "## Understanding Annotation Structure\n",
- "\n",
- "An annotation has four main parts:\n",
- "\n",
- "1. **Data**: The actual evaluation content (scores, comments)\n",
- "2. **References**: Which evaluator to use (will be created automatically if it doesn't exist)\n",
- "3. **Links**: Which trace and span you're annotating\n",
- "4. **Metadata** (optional): Any extra information you want to include\n",
- "\n",
- "### Annotation Data Examples\n",
- "\n",
- "You can include various types of data in your annotations:"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "annotation-examples",
- "metadata": {},
- "outputs": [],
- "source": [
- "# Example 1: Simple score\n",
- "simple_annotation = {\n",
- " \"outputs\": {\n",
- " \"score\": 3\n",
- " }\n",
- "}\n",
- "\n",
- "# Example 2: Score with explanation\n",
- "detailed_annotation = {\n",
- " \"outputs\": {\n",
- " \"score\": 3,\n",
- " \"comment\": \"The response is not grounded\"\n",
- " }\n",
- "}\n",
- "\n",
- "# Example 3: Multiple metrics with reference information\n",
- "comprehensive_annotation = {\n",
- " \"outputs\": {\n",
- " \"score\": 3,\n",
- " \"normalized_score\": 0.5,\n",
- " \"comment\": \"The response is not grounded\",\n",
- " \"expected_answer\": \"The capital of France is Paris\",\n",
- " \"labels\": [\"factual\", \"concise\"]\n",
- " }\n",
- "}\n",
- "\n",
- "print(\"Annotation data can include:\")\n",
- "print(\"- Numbers (scores, ratings)\")\n",
- "print(\"- Categories (labels, classifications)\")\n",
- "print(\"- Text (comments, reasoning)\")\n",
- "print(\"- Booleans (true/false values)\")"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "cleanup",
- "metadata": {},
- "source": [
- "## Optional: Remove an Annotation\n",
- "\n",
- "If you need to remove an annotation, you can delete it by its trace_id and span_id:"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "delete-annotation",
- "metadata": {},
- "outputs": [],
- "source": [
- "# Uncomment and replace with your annotation's trace_id and span_id to delete\n",
- "# annotation_trace_id = \"your_annotation_trace_id\"\n",
- "# annotation_span_id = \"your_annotation_span_id\"\n",
- "\n",
- "# response = requests.delete(\n",
- "# f\"{base_url}/api/preview/annotations/{annotation_trace_id}/{annotation_span_id}\",\n",
- "# headers=headers\n",
- "# )\n",
- "\n",
- "# if response.status_code == 200:\n",
- "# print(\"✅ Annotation deleted successfully\")\n",
- "# else:\n",
- "# print(f\"❌ Error: {response.status_code}\")\n",
- "# print(response.text)"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "summary",
- "metadata": {},
- "source": [
- "## Summary\n",
- "\n",
- "In this tutorial, we've covered:\n",
- "\n",
- "1. ✅ Setting up the Agenta SDK and instrumenting an LLM application\n",
- "2. ✅ Generating traces by running the application\n",
- "3. ✅ Creating annotations with scores, reasoning, and metadata\n",
- "4. ✅ Adding multiple annotations from different evaluators\n",
- "5. ✅ Querying annotations programmatically\n",
- "6. ✅ Understanding annotation structure and capabilities\n",
- "\n",
- "## Next Steps\n",
- "\n",
- "Now that you know how to annotate traces, you can:\n",
- "\n",
- "- Integrate annotation creation into your evaluation workflows\n",
- "- Build custom evaluators that automatically annotate traces\n",
- "- Use annotations to track user feedback in production\n",
- "- Analyze annotation data to improve your LLM applications"
- ]
- }
- ],
- "metadata": {
- "kernelspec": {
- "display_name": ".venv",
- "language": "python",
- "name": "python3"
- },
- "language_info": {
- "codemirror_mode": {
- "name": "ipython",
- "version": 3
- },
- "file_extension": ".py",
- "mimetype": "text/x-python",
- "name": "python",
- "nbconvert_exporter": "python",
- "pygments_lexer": "ipython3",
- "version": "3.12.3"
- }
- },
- "nbformat": 4,
- "nbformat_minor": 5
-}
\ No newline at end of file
diff --git a/examples/jupyter/observability/query-data-api-tutorial.ipynb b/examples/jupyter/observability/query-data-api-tutorial.ipynb
deleted file mode 100644
index 09ad0a7ecc..0000000000
--- a/examples/jupyter/observability/query-data-api-tutorial.ipynb
+++ /dev/null
@@ -1,643 +0,0 @@
-{
- "cells": [
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "# Query Data API - Tutorial\n",
- "\n",
- "This tutorial shows you how to use the Agenta Query Data API to retrieve and analyze your LLM traces. You'll learn how to:\n",
- "\n",
- "- Set up the API client with authentication\n",
- "- Query spans and traces with filters\n",
- "- Filter by attributes, time ranges, and status codes\n",
- "- Use advanced filters with logical operators\n",
- "- Analyze trace data to calculate costs and latencies\n",
- "\n",
- "## What You'll Build\n",
- "\n",
- "We'll create scripts that:\n",
- "1. Query recent traces from your applications\n",
- "2. Filter traces by type, status, and custom attributes\n",
- "3. Analyze cost and performance metrics\n",
- "4. Find problematic traces (errors, slow responses)\n",
- "5. Export trace data for further analysis"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Install Dependencies"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "pip install -U requests pandas"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": "## Setup\n\nBefore using the API, you need your Agenta API key. You can create API keys from the Settings page in your Agenta workspace."
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "import os\n",
- "os.environ[\"AGENTA_HOST\"] = \"https://cloud.agenta.ai\" # Default value, change for self-hosted\n",
- "os.environ[\"AGENTA_API_KEY\"] = \"\""
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "import os\n",
- "import requests\n",
- "from getpass import getpass\n",
- "from datetime import datetime, timedelta, timezone\n",
- "import json\n",
- "\n",
- "# Get API credentials\n",
- "AGENTA_HOST = os.getenv(\"AGENTA_HOST\", \"https://cloud.agenta.ai\")\n",
- "api_key = os.getenv(\"AGENTA_API_KEY\")\n",
- "if not api_key:\n",
- " api_key = getpass(\"Enter your Agenta API key: \")\n",
- " os.environ[\"AGENTA_API_KEY\"] = api_key\n",
- "\n",
- "# Setup base configuration\n",
- "BASE_URL = f\"{AGENTA_HOST}/api/preview/tracing/spans/query\"\n",
- "HEADERS = {\n",
- " \"Authorization\": f\"ApiKey {api_key}\",\n",
- " \"Content-Type\": \"application/json\"\n",
- "}\n",
- "\n",
- "print(f\"\u2713 Connected to {AGENTA_HOST}\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Part 1: Query Recent Traces\n",
- "\n",
- "Let's start by querying traces from the last 7 days. We'll use the `focus=trace` parameter to get complete trace trees instead of individual spans."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "# Query traces from the last 7 days\n",
- "now = datetime.now(timezone.utc)\n",
- "week_ago = now - timedelta(days=7)\n",
- "\n",
- "query = {\n",
- " \"focus\": \"trace\",\n",
- " \"oldest\": week_ago.isoformat(),\n",
- " \"newest\": now.isoformat(),\n",
- " \"limit\": 5\n",
- "}\n",
- "\n",
- "response = requests.post(BASE_URL, headers=HEADERS, json=query)\n",
- "data = response.json()\n",
- "\n",
- "print(f\"Found {data['count']} traces\")\n",
- "print(f\"Trace IDs: {list(data.get('traces', {}).keys())}\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Part 2: Query Spans with Filters\n",
- "\n",
- "Now let's query individual spans and filter by type. We'll look for LLM spans specifically."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "# Query LLM spans\n",
- "query = {\n",
- " \"focus\": \"span\",\n",
- " \"limit\": 10,\n",
- " \"filter\": {\n",
- " \"operator\": \"and\",\n",
- " \"conditions\": [\n",
- " {\n",
- " \"field\": \"attributes\",\n",
- " \"key\": \"ag.type.span\",\n",
- " \"operator\": \"is\",\n",
- " \"value\": \"llm\"\n",
- " }\n",
- " ]\n",
- " }\n",
- "}\n",
- "\n",
- "response = requests.post(BASE_URL, headers=HEADERS, json=query)\n",
- "data = response.json()\n",
- "\n",
- "print(f\"Found {data['count']} LLM spans\")\n",
- "\n",
- "# Display first span details\n",
- "if data.get('spans'):\n",
- " span = data['spans'][0]\n",
- " print(f\"\\nFirst span:\")\n",
- " print(f\" Name: {span.get('span_name')}\")\n",
- " print(f\" Status: {span.get('status_code')}\")\n",
- " print(f\" Start: {span.get('start_time')}\")\n",
- " print(f\" End: {span.get('end_time')}\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Part 3: Filter by Status Code\n",
- "\n",
- "Let's find traces that encountered errors. This is useful for debugging and monitoring application health."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "# Find error traces\n",
- "query = {\n",
- " \"focus\": \"trace\",\n",
- " \"limit\": 10,\n",
- " \"filter\": {\n",
- " \"operator\": \"and\",\n",
- " \"conditions\": [\n",
- " {\n",
- " \"field\": \"status_code\",\n",
- " \"operator\": \"is\",\n",
- " \"value\": \"STATUS_CODE_ERROR\"\n",
- " }\n",
- " ]\n",
- " }\n",
- "}\n",
- "\n",
- "response = requests.post(BASE_URL, headers=HEADERS, json=query)\n",
- "data = response.json()\n",
- "\n",
- "print(f\"Found {data['count']} traces with errors\")\n",
- "\n",
- "if data.get('traces'):\n",
- " for trace_id in list(data['traces'].keys())[:3]:\n",
- " print(f\"\\nError trace: {trace_id}\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Part 4: Advanced Filtering with Multiple Conditions\n",
- "\n",
- "Let's use multiple filters to find specific traces. We'll look for successful LLM calls that took longer than 2 seconds."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "# Find slow but successful LLM calls\n",
- "query = {\n",
- " \"focus\": \"span\",\n",
- " \"limit\": 10,\n",
- " \"filter\": {\n",
- " \"operator\": \"and\",\n",
- " \"conditions\": [\n",
- " {\n",
- " \"field\": \"attributes\",\n",
- " \"key\": \"ag.type.span\",\n",
- " \"operator\": \"is\",\n",
- " \"value\": \"llm\"\n",
- " },\n",
- " {\n",
- " \"field\": \"status_code\",\n",
- " \"operator\": \"is_not\",\n",
- " \"value\": \"STATUS_CODE_ERROR\"\n",
- " },\n",
- " {\n",
- " \"field\": \"attributes\",\n",
- " \"key\": \"ag.metrics.unit.duration\",\n",
- " \"operator\": \"gt\",\n",
- " \"value\": 2000 # milliseconds\n",
- " }\n",
- " ]\n",
- " }\n",
- "}\n",
- "\n",
- "response = requests.post(BASE_URL, headers=HEADERS, json=query)\n",
- "data = response.json()\n",
- "\n",
- "print(f\"Found {data['count']} slow LLM spans (>2s)\")\n",
- "\n",
- "if data.get('spans'):\n",
- " for span in data['spans'][:3]:\n",
- " duration = span.get('attributes', {}).get('ag', {}).get('metrics', {}).get('duration', {}).get('cumulative', 'N/A')\n",
- " print(f\" {span.get('span_name')}: {duration}ms\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Part 5: Nested Logical Operators\n",
- "\n",
- "Let's create a more complex query using nested logical operators. We'll find spans that are either errors OR slow responses."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "# Find problematic spans (errors OR slow)\n",
- "query = {\n",
- " \"focus\": \"span\",\n",
- " \"limit\": 10,\n",
- " \"filter\": {\n",
- " \"operator\": \"and\",\n",
- " \"conditions\": [\n",
- " {\n",
- " \"field\": \"attributes\",\n",
- " \"key\": \"ag.type.span\",\n",
- " \"operator\": \"is\",\n",
- " \"value\": \"llm\"\n",
- " },\n",
- " {\n",
- " \"operator\": \"or\",\n",
- " \"conditions\": [\n",
- " {\n",
- " \"field\": \"status_code\",\n",
- " \"value\": \"STATUS_CODE_ERROR\"\n",
- " },\n",
- " {\n",
- " \"field\": \"attributes\",\n",
- " \"key\": \"ag.metrics.unit.duration\",\n",
- " \"operator\": \"gt\",\n",
- " \"value\": 5000\n",
- " }\n",
- " ]\n",
- " }\n",
- " ]\n",
- " }\n",
- "}\n",
- "\n",
- "response = requests.post(BASE_URL, headers=HEADERS, json=query)\n",
- "data = response.json()\n",
- "\n",
- "print(f\"Found {data['count']} problematic spans\")\n",
- "\n",
- "if data.get('spans'):\n",
- " for span in data['spans'][:5]:\n",
- " status = span.get('status_code')\n",
- " duration = span.get('attributes', {}).get('ag', {}).get('metrics', {}).get('duration', {}).get('cumulative', 'N/A')\n",
- " print(f\" {span.get('span_name')}: Status={status}, Duration={duration}ms\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Part 6: Analyze Cost and Token Usage\n",
- "\n",
- "Let's query LLM spans and analyze their costs and token usage. This helps you understand your LLM spending."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "# Query LLM spans with cost tracking\n",
- "query = {\n",
- " \"focus\": \"span\",\n",
- " \"limit\": 50,\n",
- " \"filter\": {\n",
- " \"operator\": \"and\",\n",
- " \"conditions\": [\n",
- " {\n",
- " \"field\": \"attributes\",\n",
- " \"key\": \"ag.type.span\",\n",
- " \"operator\": \"is\",\n",
- " \"value\": \"llm\"\n",
- " },\n",
- " {\n",
- " \"field\": \"attributes\",\n",
- " \"key\": \"ag.metrics.unit.cost\",\n",
- " \"operator\": \"exists\"\n",
- " }\n",
- " ]\n",
- " }\n",
- "}\n",
- "\n",
- "response = requests.post(BASE_URL, headers=HEADERS, json=query)\n",
- "data = response.json()\n",
- "\n",
- "print(f\"Analyzing {data['count']} LLM spans with cost data\\n\")\n",
- "\n",
- "if data.get('spans'):\n",
- " total_cost = 0\n",
- " total_tokens = 0\n",
- " total_duration = 0\n",
- " \n",
- " for span in data['spans']:\n",
- " metrics = span.get('attributes', {}).get('ag', {}).get('metrics', {})\n",
- " \n",
- " # Extract cost\n",
- " cost = metrics.get('costs', {}).get('cumulative', {}).get('total', 0)\n",
- " total_cost += cost\n",
- " \n",
- " # Extract tokens\n",
- " tokens = metrics.get('tokens', {}).get('cumulative', {}).get('total', 0)\n",
- " total_tokens += tokens\n",
- " \n",
- " # Extract duration\n",
- " duration = metrics.get('duration', {}).get('cumulative', 0)\n",
- " total_duration += duration\n",
- " \n",
- " print(f\"Summary:\")\n",
- " print(f\" Total Cost: ${total_cost:.4f}\")\n",
- " print(f\" Total Tokens: {total_tokens:,}\")\n",
- " print(f\" Average Cost per Span: ${(total_cost/len(data['spans'])):.4f}\")\n",
- " print(f\" Average Tokens per Span: {int(total_tokens/len(data['spans']))}\")\n",
- " print(f\" Average Duration: {int(total_duration/len(data['spans']))}ms\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Part 7: Filter by Span Name Pattern\n",
- "\n",
- "Let's use string matching operators to find specific types of operations. We'll search for OpenAI-related spans."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "# Find OpenAI spans using pattern matching\n",
- "query = {\n",
- " \"focus\": \"span\",\n",
- " \"limit\": 10,\n",
- " \"filter\": {\n",
- " \"operator\": \"and\",\n",
- " \"conditions\": [\n",
- " {\n",
- " \"field\": \"span_name\",\n",
- " \"operator\": \"contains\",\n",
- " \"value\": \"openai\"\n",
- " }\n",
- " ]\n",
- " }\n",
- "}\n",
- "\n",
- "response = requests.post(BASE_URL, headers=HEADERS, json=query)\n",
- "data = response.json()\n",
- "\n",
- "print(f\"Found {data['count']} OpenAI spans\")\n",
- "\n",
- "if data.get('spans'):\n",
- " span_names = set(span.get('span_name') for span in data['spans'])\n",
- " print(f\"\\nUnique span names:\")\n",
- " for name in span_names:\n",
- " print(f\" - {name}\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Part 8: Export Trace Data to DataFrame\n",
- "\n",
- "Let's export our trace data to a pandas DataFrame for further analysis and visualization."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "import pandas as pd\n",
- "\n",
- "# Query recent spans\n",
- "query = {\n",
- " \"focus\": \"span\",\n",
- " \"limit\": 100,\n",
- " \"filter\": {\n",
- " \"operator\": \"and\",\n",
- " \"conditions\": [\n",
- " {\n",
- " \"field\": \"attributes\",\n",
- " \"key\": \"ag.type.span\",\n",
- " \"operator\": \"is\",\n",
- " \"value\": \"llm\"\n",
- " }\n",
- " ]\n",
- " }\n",
- "}\n",
- "\n",
- "response = requests.post(BASE_URL, headers=HEADERS, json=query)\n",
- "data = response.json()\n",
- "\n",
- "# Convert to DataFrame\n",
- "records = []\n",
- "for span in data.get('spans', []):\n",
- " metrics = span.get('attributes', {}).get('ag', {}).get('metrics', {})\n",
- " \n",
- " record = {\n",
- " 'trace_id': span.get('trace_id'),\n",
- " 'span_id': span.get('span_id'),\n",
- " 'span_name': span.get('span_name'),\n",
- " 'status': span.get('status_code'),\n",
- " 'start_time': span.get('start_time'),\n",
- " 'end_time': span.get('end_time'),\n",
- " 'duration_ms': metrics.get('duration', {}).get('cumulative', 0),\n",
- " 'cost': metrics.get('costs', {}).get('cumulative', {}).get('total', 0),\n",
- " 'total_tokens': metrics.get('tokens', {}).get('cumulative', {}).get('total', 0),\n",
- " 'prompt_tokens': metrics.get('tokens', {}).get('cumulative', {}).get('prompt', 0),\n",
- " 'completion_tokens': metrics.get('tokens', {}).get('cumulative', {}).get('completion', 0),\n",
- " }\n",
- " records.append(record)\n",
- "\n",
- "df = pd.DataFrame(records)\n",
- "\n",
- "print(f\"Created DataFrame with {len(df)} rows\\n\")\n",
- "print(\"First 5 rows:\")\n",
- "print(df.head())\n",
- "\n",
- "print(\"\\nBasic statistics:\")\n",
- "print(df[['duration_ms', 'cost', 'total_tokens']].describe())"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Part 9: Time-Based Analysis\n",
- "\n",
- "Let's analyze how your costs and latencies change over time."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "# Convert timestamps to datetime\n",
- "df['start_time'] = pd.to_datetime(df['start_time'])\n",
- "df['end_time'] = pd.to_datetime(df['end_time'])\n",
- "\n",
- "# Group by hour\n",
- "df['hour'] = df['start_time'].dt.floor('H')\n",
- "hourly_stats = df.groupby('hour').agg({\n",
- " 'span_id': 'count',\n",
- " 'duration_ms': 'mean',\n",
- " 'cost': 'sum',\n",
- " 'total_tokens': 'sum'\n",
- "}).rename(columns={'span_id': 'num_calls'})\n",
- "\n",
- "print(\"Hourly statistics:\")\n",
- "print(hourly_stats)\n",
- "\n",
- "print(f\"\\nPeak usage hour: {hourly_stats['num_calls'].idxmax()}\")\n",
- "print(f\"Highest cost hour: {hourly_stats['cost'].idxmax()} (${hourly_stats['cost'].max():.4f})\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Part 10: Filter by Time Range\n",
- "\n",
- "Let's query traces from a specific time window. This is useful for analyzing specific incidents or time periods."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "# Query last 24 hours\n",
- "now = datetime.now(timezone.utc)\n",
- "yesterday = now - timedelta(days=1)\n",
- "\n",
- "query = {\n",
- " \"focus\": \"span\",\n",
- " \"oldest\": yesterday.isoformat(),\n",
- " \"newest\": now.isoformat(),\n",
- " \"limit\": 100,\n",
- " \"filter\": {\n",
- " \"operator\": \"and\",\n",
- " \"conditions\": [\n",
- " {\n",
- " \"field\": \"attributes\",\n",
- " \"key\": \"ag.type.span\",\n",
- " \"operator\": \"is\",\n",
- " \"value\": \"llm\"\n",
- " }\n",
- " ]\n",
- " }\n",
- "}\n",
- "\n",
- "response = requests.post(BASE_URL, headers=HEADERS, json=query)\n",
- "data = response.json()\n",
- "\n",
- "print(f\"Last 24 hours: {data['count']} LLM spans\")\n",
- "\n",
- "if data.get('spans'):\n",
- " # Calculate totals\n",
- " total_cost = sum(\n",
- " span.get('attributes', {}).get('ag', {}).get('metrics', {}).get('costs', {}).get('cumulative', {}).get('total', 0)\n",
- " for span in data['spans']\n",
- " )\n",
- " \n",
- " error_count = sum(\n",
- " 1 for span in data['spans']\n",
- " if span.get('status_code') == 'STATUS_CODE_ERROR'\n",
- " )\n",
- " \n",
- " print(f\"\\nSummary for last 24 hours:\")\n",
- " print(f\" Total Cost: ${total_cost:.4f}\")\n",
- " print(f\" Error Rate: {(error_count/len(data['spans'])*100):.2f}%\")\n",
- " print(f\" Success Rate: {((len(data['spans'])-error_count)/len(data['spans'])*100):.2f}%\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Summary\n",
- "\n",
- "In this tutorial, you learned how to:\n",
- "\n",
- "1. \u2713 Set up the Agenta Query Data API client\n",
- "2. \u2713 Query traces and spans with filters\n",
- "3. \u2713 Filter by attributes, status codes, and time ranges\n",
- "4. \u2713 Use advanced filters with logical operators\n",
- "5. \u2713 Analyze cost and performance metrics\n",
- "6. \u2713 Export trace data to pandas DataFrames\n",
- "7. \u2713 Perform time-based analysis\n",
- "\n",
- "## Next Steps\n",
- "\n",
- "- Learn about the [Analytics Data API](/observability/query-data/analytics-data) for aggregated metrics\n",
- "- Explore [filtering in the UI](/observability/using-the-ui/filtering-traces) for visual query building\n",
- "- Check out [trace annotations](/observability/trace-with-python-sdk/annotate-traces) for adding feedback data\n",
- "- Read the complete [API reference](/reference/api) for all available endpoints"
- ]
- }
- ],
- "metadata": {
- "kernelspec": {
- "display_name": "Python 3",
- "language": "python",
- "name": "python3"
- },
- "language_info": {
- "codemirror_mode": {
- "name": "ipython",
- "version": 3
- },
- "file_extension": ".py",
- "mimetype": "text/x-python",
- "name": "python",
- "nbconvert_exporter": "python",
- "pygments_lexer": "ipython3",
- "version": "3.9.0"
- }
- },
- "nbformat": 4,
- "nbformat_minor": 4
-}
\ No newline at end of file
diff --git a/examples/jupyter/observability/quickstart.ipynb b/examples/jupyter/observability/quickstart.ipynb
deleted file mode 100644
index a16891c606..0000000000
--- a/examples/jupyter/observability/quickstart.ipynb
+++ /dev/null
@@ -1,328 +0,0 @@
-{
- "cells": [
- {
- "cell_type": "markdown",
- "id": "a1b2c3d4",
- "metadata": {},
- "source": [
- "# Quick Start: Observability with Agenta\n",
- "\n",
- "Agenta enables you to capture all inputs, outputs, and metadata from your LLM applications, **whether they're hosted within Agenta or running in your own environment**.\n",
- "\n",
- "This guide will walk you through setting up observability for an OpenAI application running locally.\n",
- "\n",
- "**Note:** If you create an application through the Agenta UI, tracing is enabled by default. No additional setup is required—simply go to the observability view to see all your requests."
- ]
- },
- {
- "cell_type": "markdown",
- "id": "b2c3d4e5",
- "metadata": {},
- "source": [
- "## Step 1: Install Required Packages\n",
- "\n",
- "First, install the Agenta SDK, OpenAI, and the OpenTelemetry instrumentor for OpenAI:"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 1,
- "id": "c3d4e5f6",
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "Requirement already satisfied: agenta in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (0.51.6)\n",
- "Collecting agenta\n",
- " Downloading agenta-0.59.6-py3-none-any.whl.metadata (31 kB)\n",
- "Requirement already satisfied: openai in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (1.107.1)\n",
- "Collecting openai\n",
- " Downloading openai-2.6.1-py3-none-any.whl.metadata (29 kB)\n",
- "Requirement already satisfied: opentelemetry-instrumentation-openai in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (0.46.2)\n",
- "Collecting opentelemetry-instrumentation-openai\n",
- " Downloading opentelemetry_instrumentation_openai-0.47.5-py3-none-any.whl.metadata (2.2 kB)\n",
- "Requirement already satisfied: decorator<6.0.0,>=5.2.1 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (5.2.1)\n",
- "Requirement already satisfied: fastapi<0.117.0,>=0.116.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (0.116.1)\n",
- "Requirement already satisfied: google-auth<3,>=2.23 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (2.40.3)\n",
- "Requirement already satisfied: h11<0.17.0,>=0.16.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (0.16.0)\n",
- "Requirement already satisfied: httpx<0.29.0,>=0.28.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (0.28.1)\n",
- "Requirement already satisfied: huggingface-hub<0.31.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (0.30.2)\n",
- "Requirement already satisfied: importlib-metadata<9.0,>=8.0.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (8.7.0)\n",
- "Requirement already satisfied: jinja2<4.0.0,>=3.1.6 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (3.1.6)\n",
- "Collecting litellm==1.78.7 (from agenta)\n",
- " Downloading litellm-1.78.7-py3-none-any.whl.metadata (42 kB)\n",
- "Collecting openai\n",
- " Downloading openai-1.109.1-py3-none-any.whl.metadata (29 kB)\n",
- "Requirement already satisfied: opentelemetry-api<2.0.0,>=1.27.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (1.36.0)\n",
- "Requirement already satisfied: opentelemetry-exporter-otlp-proto-http<2.0.0,>=1.27.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (1.36.0)\n",
- "Requirement already satisfied: opentelemetry-instrumentation>=0.56b0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (0.57b0)\n",
- "Requirement already satisfied: opentelemetry-sdk<2.0.0,>=1.27.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (1.36.0)\n",
- "Requirement already satisfied: pydantic<3,>=2 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (2.11.7)\n",
- "Requirement already satisfied: python-dotenv<2.0.0,>=1.0.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (1.1.1)\n",
- "Requirement already satisfied: pyyaml<7.0.0,>=6.0.2 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (6.0.2)\n",
- "Requirement already satisfied: starlette<0.48.0,>=0.47.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (0.47.3)\n",
- "Requirement already satisfied: structlog<26.0.0,>=25.2.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (25.4.0)\n",
- "Requirement already satisfied: tiktoken==0.11.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (0.11.0)\n",
- "Requirement already satisfied: toml<0.11.0,>=0.10.2 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (0.10.2)\n",
- "Requirement already satisfied: aiohttp>=3.10 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from litellm==1.78.7->agenta) (3.12.15)\n",
- "Requirement already satisfied: click in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from litellm==1.78.7->agenta) (8.2.1)\n",
- "Collecting fastuuid>=0.13.0 (from litellm==1.78.7->agenta)\n",
- " Downloading fastuuid-0.14.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (1.1 kB)\n",
- "Requirement already satisfied: jsonschema<5.0.0,>=4.22.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from litellm==1.78.7->agenta) (4.25.1)\n",
- "Requirement already satisfied: tokenizers in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from litellm==1.78.7->agenta) (0.22.0)\n",
- "Requirement already satisfied: regex>=2022.1.18 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from tiktoken==0.11.0->agenta) (2025.9.1)\n",
- "Requirement already satisfied: requests>=2.26.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from tiktoken==0.11.0->agenta) (2.32.5)\n",
- "Requirement already satisfied: anyio<5,>=3.5.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from openai) (4.10.0)\n",
- "Requirement already satisfied: distro<2,>=1.7.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from openai) (1.9.0)\n",
- "Requirement already satisfied: jiter<1,>=0.4.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from openai) (0.10.0)\n",
- "Requirement already satisfied: sniffio in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from openai) (1.3.1)\n",
- "Requirement already satisfied: tqdm>4 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from openai) (4.67.1)\n",
- "Requirement already satisfied: typing-extensions<5,>=4.11 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from openai) (4.15.0)\n",
- "Requirement already satisfied: idna>=2.8 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from anyio<5,>=3.5.0->openai) (3.10)\n",
- "Requirement already satisfied: cachetools<6.0,>=2.0.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from google-auth<3,>=2.23->agenta) (5.5.2)\n",
- "Requirement already satisfied: pyasn1-modules>=0.2.1 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from google-auth<3,>=2.23->agenta) (0.4.2)\n",
- "Requirement already satisfied: rsa<5,>=3.1.4 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from google-auth<3,>=2.23->agenta) (4.9.1)\n",
- "Requirement already satisfied: certifi in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from httpx<0.29.0,>=0.28.0->agenta) (2025.8.3)\n",
- "Requirement already satisfied: httpcore==1.* in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from httpx<0.29.0,>=0.28.0->agenta) (1.0.9)\n",
- "Requirement already satisfied: filelock in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from huggingface-hub<0.31.0->agenta) (3.19.1)\n",
- "Requirement already satisfied: fsspec>=2023.5.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from huggingface-hub<0.31.0->agenta) (2025.7.0)\n",
- "Requirement already satisfied: packaging>=20.9 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from huggingface-hub<0.31.0->agenta) (25.0)\n",
- "Requirement already satisfied: zipp>=3.20 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from importlib-metadata<9.0,>=8.0.0->agenta) (3.23.0)\n",
- "Requirement already satisfied: MarkupSafe>=2.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from jinja2<4.0.0,>=3.1.6->agenta) (3.0.2)\n",
- "Requirement already satisfied: attrs>=22.2.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from jsonschema<5.0.0,>=4.22.0->litellm==1.78.7->agenta) (25.3.0)\n",
- "Requirement already satisfied: jsonschema-specifications>=2023.03.6 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from jsonschema<5.0.0,>=4.22.0->litellm==1.78.7->agenta) (2025.4.1)\n",
- "Requirement already satisfied: referencing>=0.28.4 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from jsonschema<5.0.0,>=4.22.0->litellm==1.78.7->agenta) (0.36.2)\n",
- "Requirement already satisfied: rpds-py>=0.7.1 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from jsonschema<5.0.0,>=4.22.0->litellm==1.78.7->agenta) (0.27.1)\n",
- "Requirement already satisfied: googleapis-common-protos~=1.52 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from opentelemetry-exporter-otlp-proto-http<2.0.0,>=1.27.0->agenta) (1.70.0)\n",
- "Requirement already satisfied: opentelemetry-exporter-otlp-proto-common==1.36.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from opentelemetry-exporter-otlp-proto-http<2.0.0,>=1.27.0->agenta) (1.36.0)\n",
- "Requirement already satisfied: opentelemetry-proto==1.36.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from opentelemetry-exporter-otlp-proto-http<2.0.0,>=1.27.0->agenta) (1.36.0)\n",
- "Requirement already satisfied: protobuf<7.0,>=5.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from opentelemetry-proto==1.36.0->opentelemetry-exporter-otlp-proto-http<2.0.0,>=1.27.0->agenta) (6.32.0)\n",
- "Requirement already satisfied: opentelemetry-semantic-conventions==0.57b0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from opentelemetry-sdk<2.0.0,>=1.27.0->agenta) (0.57b0)\n",
- "Requirement already satisfied: annotated-types>=0.6.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from pydantic<3,>=2->agenta) (0.7.0)\n",
- "Requirement already satisfied: pydantic-core==2.33.2 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from pydantic<3,>=2->agenta) (2.33.2)\n",
- "Requirement already satisfied: typing-inspection>=0.4.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from pydantic<3,>=2->agenta) (0.4.1)\n",
- "Requirement already satisfied: charset_normalizer<4,>=2 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from requests>=2.26.0->tiktoken==0.11.0->agenta) (3.4.3)\n",
- "Requirement already satisfied: urllib3<3,>=1.21.1 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from requests>=2.26.0->tiktoken==0.11.0->agenta) (2.5.0)\n",
- "Requirement already satisfied: pyasn1>=0.1.3 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from rsa<5,>=3.1.4->google-auth<3,>=2.23->agenta) (0.6.1)\n",
- "Requirement already satisfied: opentelemetry-semantic-conventions-ai<0.5.0,>=0.4.13 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from opentelemetry-instrumentation-openai) (0.4.13)\n",
- "Requirement already satisfied: aiohappyeyeballs>=2.5.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from aiohttp>=3.10->litellm==1.78.7->agenta) (2.6.1)\n",
- "Requirement already satisfied: aiosignal>=1.4.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from aiohttp>=3.10->litellm==1.78.7->agenta) (1.4.0)\n",
- "Requirement already satisfied: frozenlist>=1.1.1 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from aiohttp>=3.10->litellm==1.78.7->agenta) (1.7.0)\n",
- "Requirement already satisfied: multidict<7.0,>=4.5 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from aiohttp>=3.10->litellm==1.78.7->agenta) (6.6.4)\n",
- "Requirement already satisfied: propcache>=0.2.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from aiohttp>=3.10->litellm==1.78.7->agenta) (0.3.2)\n",
- "Requirement already satisfied: yarl<2.0,>=1.17.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from aiohttp>=3.10->litellm==1.78.7->agenta) (1.20.1)\n",
- "Requirement already satisfied: wrapt<2.0.0,>=1.0.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from opentelemetry-instrumentation>=0.56b0->agenta) (1.17.3)\n",
- "Downloading agenta-0.59.6-py3-none-any.whl (339 kB)\n",
- "Downloading litellm-1.78.7-py3-none-any.whl (9.9 MB)\n",
- "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m9.9/9.9 MB\u001b[0m \u001b[31m80.7 MB/s\u001b[0m \u001b[33m0:00:00\u001b[0m\n",
- "\u001b[?25hDownloading openai-1.109.1-py3-none-any.whl (948 kB)\n",
- "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m948.6/948.6 kB\u001b[0m \u001b[31m26.3 MB/s\u001b[0m \u001b[33m0:00:00\u001b[0m\n",
- "\u001b[?25hDownloading opentelemetry_instrumentation_openai-0.47.5-py3-none-any.whl (35 kB)\n",
- "Downloading fastuuid-0.14.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (278 kB)\n",
- "Installing collected packages: fastuuid, openai, litellm, opentelemetry-instrumentation-openai, agenta\n",
- "\u001b[2K Attempting uninstall: openai\n",
- "\u001b[2K Found existing installation: openai 1.107.1\n",
- "\u001b[2K Uninstalling openai-1.107.1:\n",
- "\u001b[2K Successfully uninstalled openai-1.107.1\n",
- "\u001b[2K Attempting uninstall: litellm[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1/5\u001b[0m [openai]\n",
- "\u001b[2K Found existing installation: litellm 1.76.0━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1/5\u001b[0m [openai]\n",
- "\u001b[2K Uninstalling litellm-1.76.0:m╺\u001b[0m\u001b[90m━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2/5\u001b[0m [litellm]\n",
- "\u001b[2K Successfully uninstalled litellm-1.76.0━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2/5\u001b[0m [litellm]\n",
- "\u001b[2K Attempting uninstall: opentelemetry-instrumentation-openai━━━━━━\u001b[0m \u001b[32m2/5\u001b[0m [litellm]\n",
- "\u001b[2K Found existing installation: opentelemetry-instrumentation-openai 0.46.2[0m [litellm]\n",
- "\u001b[2K Uninstalling opentelemetry-instrumentation-openai-0.46.2:━\u001b[0m \u001b[32m2/5\u001b[0m [litellm]\n",
- "\u001b[2K Successfully uninstalled opentelemetry-instrumentation-openai-0.46.25\u001b[0m [litellm]\n",
- "\u001b[2K Attempting uninstall: agenta\u001b[0m\u001b[90m━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2/5\u001b[0m [litellm]\n",
- "\u001b[2K Found existing installation: agenta 0.51.6━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2/5\u001b[0m [litellm]\n",
- "\u001b[2K Uninstalling agenta-0.51.6:[0m\u001b[90m━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2/5\u001b[0m [litellm]\n",
- "\u001b[2K Successfully uninstalled agenta-0.51.6━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2/5\u001b[0m [litellm]\n",
- "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m5/5\u001b[0m [agenta]2m4/5\u001b[0m [agenta]\n",
- "\u001b[1A\u001b[2KSuccessfully installed agenta-0.59.6 fastuuid-0.14.0 litellm-1.78.7 openai-1.109.1 opentelemetry-instrumentation-openai-0.47.5\n",
- "Note: you may need to restart the kernel to use updated packages.\n"
- ]
- }
- ],
- "source": [
- "pip install -U agenta openai opentelemetry-instrumentation-openai"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "d4e5f6g7",
- "metadata": {},
- "source": [
- "## Step 2: Configure Environment Variables\n",
- "\n",
- "To start tracing your application, you'll need an API key:\n",
- "\n",
- "1. Visit the Agenta API Keys page under settings.\n",
- "2. Click on **Create New API Key** and follow the prompts.\n",
- "\n",
- "Then set your environment variables:"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "e5f6g7h8",
- "metadata": {},
- "outputs": [],
- "source": [
- "import os\n",
- "\n",
- "# Set your API key here\n",
- "os.environ[\"AGENTA_API_KEY\"] = \"\"\n",
- "os.environ[\"AGENTA_HOST\"] = \"https://cloud.agenta.ai\" # Change for self-hosted\n",
- "os.environ[\"OPENAI_API_KEY\"] = \"\""
- ]
- },
- {
- "cell_type": "markdown",
- "id": "f6g7h8i9",
- "metadata": {},
- "source": [
- "## Step 3: Instrument Your Application\n",
- "\n",
- "Below is a sample script to instrument an OpenAI application:"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 3,
- "id": "g7h8i9j0",
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "2025-10-28T10:24:18.411Z \u001b[38;5;70m[INFO.]\u001b[0m Agenta - SDK version: 0.59.6 \u001b[38;5;245m[agenta.sdk.agenta_init]\u001b[0m \n",
- "2025-10-28T10:24:18.411Z \u001b[38;5;70m[INFO.]\u001b[0m Agenta - Host: https://cloud.agenta.ai \u001b[38;5;245m[agenta.sdk.agenta_init]\u001b[0m \n",
- "2025-10-28T10:24:18.412Z \u001b[38;5;70m[INFO.]\u001b[0m Agenta - OLTP URL: https://cloud.agenta.ai/api/otlp/v1/traces \u001b[38;5;245m[agenta.sdk.tracing.tracing]\u001b[0m \n"
- ]
- }
- ],
- "source": [
- "import agenta as ag\n",
- "from opentelemetry.instrumentation.openai import OpenAIInstrumentor\n",
- "import openai\n",
- "\n",
- "# Initialize Agenta\n",
- "ag.init()\n",
- "\n",
- "# Instrument OpenAI\n",
- "OpenAIInstrumentor().instrument()"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "h8i9j0k1",
- "metadata": {},
- "source": [
- "## Step 4: Create an Instrumented Function\n",
- "\n",
- "Decorate your function with `@ag.instrument()` to enable tracing:"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 5,
- "id": "i9j0k1l2",
- "metadata": {},
- "outputs": [],
- "source": [
- "@ag.instrument()\n",
- "def generate():\n",
- " response = openai.chat.completions.create(\n",
- " model=\"gpt-3.5-turbo\",\n",
- " messages=[\n",
- " {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n",
- " {\"role\": \"user\", \"content\": \"Write a short story about AI Engineering.\"},\n",
- " ],\n",
- " )\n",
- " return response.choices[0].message.content"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "j0k1l2m3",
- "metadata": {},
- "source": [
- "## Step 5: Run Your Application\n",
- "\n",
- "Call your instrumented function to generate a trace:"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 6,
- "id": "k1l2m3n4",
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "In a future where artificial intelligence had revolutionized society, a young engineer named Maya was fascinated by the endless possibilities of AI technology. She dedicated her life to mastering the complexities of AI engineering, pushing the boundaries of what was thought possible.\n",
- "\n",
- "Maya worked tirelessly, constantly experimenting and innovating to create AI systems that could change the world for the better. Her creations helped streamline businesses, improve healthcare outcomes, and enhance everyday tasks for people around the globe.\n",
- "\n",
- "However, Maya soon realized that not all AI technology was being used ethically. Some companies exploited AI for profit, using it to manipulate data or invade people's privacy. Troubled by this misuse of technology, Maya made it her mission to advocate for responsible AI engineering practices.\n",
- "\n",
- "Through her dedication and leadership, Maya became a voice for ethical AI engineering, pushing for regulations and guidelines to ensure that AI technology was used for the greater good of humanity. Her passion inspired others in the field to prioritize ethics and social responsibility in their work.\n",
- "\n",
- "As the world grappled with the emerging challenges of AI technology, Maya stood at the forefront, a shining example of how innovation and integrity could go hand in hand in the exciting world of AI engineering.\n"
- ]
- }
- ],
- "source": [
- "if __name__ == \"__main__\":\n",
- " result = generate()\n",
- " print(result)"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "l2m3n4o5",
- "metadata": {},
- "source": [
- "## Step 6: View Traces in the Agenta UI\n",
- "\n",
- "After running your application, you can view the captured traces in Agenta:\n",
- "\n",
- "1. Log in to your Agenta dashboard.\n",
- "2. Navigate to the **Observability** section.\n",
- "3. You'll see a list of traces corresponding to your application's requests.\n",
- "\n",
- "Each trace will show you the inputs, outputs, and metadata from your LLM application, including:\n",
- "- Function execution time\n",
- "- OpenAI API calls and responses\n",
- "- Token usage\n",
- "- Any errors or exceptions"
- ]
- }
- ],
- "metadata": {
- "kernelspec": {
- "display_name": ".venv",
- "language": "python",
- "name": "python3"
- },
- "language_info": {
- "codemirror_mode": {
- "name": "ipython",
- "version": 3
- },
- "file_extension": ".py",
- "mimetype": "text/x-python",
- "name": "python",
- "nbconvert_exporter": "python",
- "pygments_lexer": "ipython3",
- "version": "3.12.3"
- }
- },
- "nbformat": 4,
- "nbformat_minor": 5
-}
diff --git a/examples/jupyter/observability/trace-with-python-sdk-tutorial.ipynb b/examples/jupyter/observability/trace-with-python-sdk-tutorial.ipynb
deleted file mode 100644
index d40d9dc9af..0000000000
--- a/examples/jupyter/observability/trace-with-python-sdk-tutorial.ipynb
+++ /dev/null
@@ -1,524 +0,0 @@
-{
- "cells": [
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "# Trace with Python SDK - Tutorial\n",
- "\n",
- "This tutorial demonstrates how to use the Agenta Python SDK to trace your LLM applications. You'll learn how to:\n",
- "\n",
- "- Set up tracing with the Agenta SDK\n",
- "- Instrument functions and OpenAI calls automatically\n",
- "- Start and end spans manually to capture internals\n",
- "- Reference prompt versions in your traces\n",
- "- Redact sensitive data from traces\n",
- "\n",
- "## What You'll Build\n",
- "\n",
- "We'll create a simple LLM application that:\n",
- "1. Uses OpenAI auto-instrumentation to trace API calls\n",
- "2. Instruments custom functions to capture workflow steps\n",
- "3. Stores internal data like retrieved context\n",
- "4. Links traces to deployed prompt versions\n",
- "5. Redacts sensitive information from traces"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Install Dependencies"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "pip install -U agenta openai opentelemetry-instrumentation-openai"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Setup\n",
- "\n",
- "Before using the SDK, we need to initialize it with your API keys. The SDK requires:\n",
- "- **Agenta API Key**: For sending traces to Agenta\n",
- "- **OpenAI API Key**: For making LLM calls\n",
- "\n",
- "You can get your Agenta API key from the [API Keys page](https://cloud.agenta.ai/settings?tab=apiKeys)."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "import os\n",
- "os.environ[\"AGENTA_HOST\"] = \"https://cloud.agenta.ai/\" # Default value, change for self-hosted\n",
- "os.environ[\"AGENTA_API_KEY\"] = \"\"\n",
- "os.environ[\"OPENAI_API_KEY\"] = \"\""
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "import os\n",
- "import agenta as ag\n",
- "from getpass import getpass\n",
- "\n",
- "# Initialize the SDK with your API key\n",
- "api_key = os.getenv(\"AGENTA_API_KEY\")\n",
- "if not api_key:\n",
- " os.environ[\"AGENTA_API_KEY\"] = getpass(\"Enter your Agenta API key: \")\n",
- "\n",
- "openai_api_key = os.getenv(\"OPENAI_API_KEY\")\n",
- "if not openai_api_key:\n",
- " os.environ[\"OPENAI_API_KEY\"] = getpass(\"Enter your OpenAI API key: \")\n",
- "\n",
- "# Initialize the SDK\n",
- "ag.init()"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Part 1: Setup Tracing with OpenAI Auto-Instrumentation\n",
- "\n",
- "The Agenta SDK provides two powerful mechanisms for tracing:\n",
- "\n",
- "1. **Auto-instrumentation**: Automatically traces third-party libraries like OpenAI\n",
- "2. **Function decorators**: Manually instrument your custom functions\n",
- "\n",
- "Let's start by setting up OpenAI auto-instrumentation, which will capture all OpenAI API calls automatically."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "from opentelemetry.instrumentation.openai import OpenAIInstrumentor\n",
- "import openai\n",
- "\n",
- "# Instrument OpenAI to automatically trace all API calls\n",
- "OpenAIInstrumentor().instrument()\n",
- "\n",
- "print(\"OpenAI auto-instrumentation enabled!\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Part 2: Instrument Functions\n",
- "\n",
- "Now let's create a simple function and instrument it using the `@ag.instrument()` decorator. This will create a span for the function and automatically capture its inputs and outputs.\n",
- "\n",
- "The decorator accepts a `spankind` parameter to categorize the span. Available types include: `agent`, `chain`, `workflow`, `tool`, `embedding`, `query`, `completion`, `chat`, `rerank`."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "@ag.instrument(spankind=\"workflow\")\n",
- "def generate_story(topic: str):\n",
- " \"\"\"Generate a short story about the given topic.\"\"\"\n",
- " response = openai.chat.completions.create(\n",
- " model=\"gpt-3.5-turbo\",\n",
- " messages=[\n",
- " {\"role\": \"system\", \"content\": \"You are a creative storyteller.\"},\n",
- " {\"role\": \"user\", \"content\": f\"Write a short story about {topic}.\"},\n",
- " ],\n",
- " )\n",
- " return response.choices[0].message.content\n",
- "\n",
- "# Test the instrumented function\n",
- "story = generate_story(\"AI Engineering\")\n",
- "print(\"Generated story:\")\n",
- "print(story)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Part 3: Starting Spans and Storing Internals\n",
- "\n",
- "Sometimes you need to capture intermediate data that isn't part of the function's inputs or outputs. The SDK provides two methods:\n",
- "\n",
- "- `ag.tracing.store_meta()`: Add metadata to a span (saved under `ag.meta`)\n",
- "- `ag.tracing.store_internals()`: Store internal data (saved under `ag.data.internals`)\n",
- "\n",
- "Internals are especially useful because they:\n",
- "1. Are searchable using plain text queries\n",
- "2. Appear in the overview tab of the observability drawer\n",
- "\n",
- "Let's create a RAG (Retrieval-Augmented Generation) example that captures the retrieved context:"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "@ag.instrument(spankind=\"tool\")\n",
- "def retrieve_context(query: str):\n",
- " \"\"\"Simulate retrieving context from a knowledge base.\"\"\"\n",
- " # In a real application, this would query a vector database\n",
- " context = [\n",
- " \"Agenta is an open-source LLM developer platform.\",\n",
- " \"Agenta provides tools for prompt management, evaluation, and observability.\",\n",
- " \"The Agenta SDK supports tracing with OpenTelemetry.\",\n",
- " ]\n",
- " \n",
- " # Store metadata about the retrieval\n",
- " ag.tracing.store_meta({\n",
- " \"retrieval_method\": \"vector_search\",\n",
- " \"num_results\": len(context)\n",
- " })\n",
- " \n",
- " return context\n",
- "\n",
- "@ag.instrument(spankind=\"workflow\")\n",
- "def rag_workflow(query: str):\n",
- " \"\"\"Answer a question using retrieved context.\"\"\"\n",
- " # Retrieve context\n",
- " context = retrieve_context(query)\n",
- " \n",
- " # Store the retrieved context as internals\n",
- " # This makes it visible in the UI and searchable\n",
- " ag.tracing.store_internals({\"retrieved_context\": context})\n",
- " \n",
- " # Generate answer using context\n",
- " context_str = \"\\n\".join(context)\n",
- " prompt = f\"Answer the following question based on the context:\\n\\nContext:\\n{context_str}\\n\\nQuestion: {query}\"\n",
- " \n",
- " response = openai.chat.completions.create(\n",
- " model=\"gpt-3.5-turbo\",\n",
- " messages=[\n",
- " {\"role\": \"system\", \"content\": \"You are a helpful assistant. Answer questions based only on the provided context.\"},\n",
- " {\"role\": \"user\", \"content\": prompt},\n",
- " ],\n",
- " )\n",
- " \n",
- " return response.choices[0].message.content\n",
- "\n",
- "# Test the RAG workflow\n",
- "answer = rag_workflow(\"What is Agenta?\")\n",
- "print(\"Answer:\")\n",
- "print(answer)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Part 4: Reference Prompt Versions\n",
- "\n",
- "One of Agenta's powerful features is linking traces to specific prompt versions. This allows you to:\n",
- "- Filter traces by application, variant, or environment\n",
- "- Compare performance across different variants\n",
- "- Track production behavior\n",
- "\n",
- "Let's create a prompt using the SDK, deploy it, and then reference it in our traces."
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### Create and Deploy a Prompt"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": "from agenta.sdk.types import PromptTemplate, Message, ModelConfig\nfrom pydantic import BaseModel\n\n# Define the prompt configuration\nclass Config(BaseModel):\n prompt: PromptTemplate\n\nconfig = Config(\n prompt=PromptTemplate(\n messages=[\n Message(role=\"system\", content=\"You are a helpful assistant that explains topics clearly.\"),\n Message(role=\"user\", content=\"Explain {{topic}} in simple terms.\"),\n ],\n llm_config=ModelConfig(\n model=\"gpt-3.5-turbo\",\n max_tokens=200,\n temperature=0.7,\n top_p=1.0,\n frequency_penalty=0.0,\n presence_penalty=0.0,\n ),\n template_format=\"curly\"\n )\n)\n\n# Create an application and variant\napp = ag.AppManager.create(\n app_slug=\"topic-explainer-traced\",\n template_key=\"SERVICE:completion\",\n)\n\nprint(f\"Created application: {app.app_name}\")\n\n# Create a variant with the prompt configuration\nvariant = ag.VariantManager.create(\n parameters=config.model_dump(),\n app_slug=\"topic-explainer-traced\",\n variant_slug=\"production-variant\"\n)\n\nprint(f\"Created variant: {variant.variant_slug} (version {variant.variant_version})\")\n\n# Deploy to production environment\ndeployment = ag.DeploymentManager.deploy(\n app_slug=\"topic-explainer-traced\",\n variant_slug=\"production-variant\",\n environment_slug=\"production\",\n)\n\nprint(f\"Deployed to {deployment.environment_slug} environment\")"
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### Reference the Prompt in Traces\n",
- "\n",
- "Now we'll create a function that uses the deployed prompt and links its traces to the application and environment."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "@ag.instrument(spankind=\"workflow\")\n",
- "def explain_topic_with_prompt(topic: str):\n",
- " \"\"\"Explain a topic using the deployed prompt configuration.\"\"\"\n",
- " \n",
- " # Fetch the prompt configuration from production\n",
- " prompt_config = ag.ConfigManager.get_from_registry(\n",
- " app_slug=\"topic-explainer-traced\",\n",
- " environment_slug=\"production\"\n",
- " )\n",
- " \n",
- " # Format the prompt with the topic\n",
- " prompt_template = PromptTemplate(**prompt_config[\"prompt\"])\n",
- " formatted_prompt = prompt_template.format(topic=topic)\n",
- " \n",
- " # Make the OpenAI call\n",
- " response = openai.chat.completions.create(\n",
- " **formatted_prompt.to_openai_kwargs()\n",
- " )\n",
- " \n",
- " # Link this trace to the application and environment\n",
- " ag.tracing.store_refs({\n",
- " \"application.slug\": \"topic-explainer-traced\",\n",
- " \"variant.slug\": \"production-variant\",\n",
- " \"environment.slug\": \"production\",\n",
- " })\n",
- " \n",
- " return response.choices[0].message.content\n",
- "\n",
- "# Test the function\n",
- "explanation = explain_topic_with_prompt(\"machine learning\")\n",
- "print(\"Explanation:\")\n",
- "print(explanation)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Part 5: Redact Sensitive Data\n",
- "\n",
- "When working with production data, you often need to exclude sensitive information from traces. The Agenta SDK provides several ways to redact data:\n",
- "\n",
- "1. **Simple redaction**: Ignore all inputs/outputs\n",
- "2. **Selective redaction**: Ignore specific fields\n",
- "3. **Custom redaction**: Use a callback function for fine-grained control\n",
- "4. **Global redaction**: Apply rules across all instrumented functions\n",
- "\n",
- "Let's explore these different approaches."
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### Simple Redaction: Ignore All Inputs/Outputs"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "@ag.instrument(\n",
- " spankind=\"workflow\",\n",
- " ignore_inputs=True,\n",
- " ignore_outputs=True\n",
- ")\n",
- "def process_sensitive_data(user_email: str, credit_card: str):\n",
- " \"\"\"Process sensitive data without logging inputs/outputs.\"\"\"\n",
- " # The function inputs and outputs won't be captured in the trace\n",
- " result = f\"Processed data for {user_email}\"\n",
- " return result\n",
- "\n",
- "# This trace will not contain inputs or outputs\n",
- "result = process_sensitive_data(\"user@example.com\", \"4111-1111-1111-1111\")\n",
- "print(result)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### Selective Redaction: Ignore Specific Fields"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "@ag.instrument(\n",
- " spankind=\"workflow\",\n",
- " ignore_inputs=[\"api_key\", \"password\"],\n",
- " ignore_outputs=[\"internal_token\"]\n",
- ")\n",
- "def authenticate_user(username: str, password: str, api_key: str):\n",
- " \"\"\"Authenticate a user (password and api_key will be redacted).\"\"\"\n",
- " # Simulate authentication\n",
- " return {\n",
- " \"username\": username,\n",
- " \"authenticated\": True,\n",
- " \"internal_token\": \"secret-token-12345\", # This will be redacted\n",
- " }\n",
- "\n",
- "# The trace will show username but not password or api_key\n",
- "auth_result = authenticate_user(\"john_doe\", \"secret123\", \"sk-abc123\")\n",
- "print(f\"Authenticated: {auth_result['authenticated']}\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### Custom Redaction: Use a Callback Function"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "import re\n",
- "\n",
- "def redact_pii(name: str, field: str, data: dict):\n",
- " \"\"\"Custom redaction function that removes PII.\"\"\"\n",
- " if field == \"inputs\":\n",
- " # Redact email addresses\n",
- " if \"email\" in data:\n",
- " data[\"email\"] = \"[REDACTED]\"\n",
- " # Redact phone numbers\n",
- " if \"phone\" in data:\n",
- " data[\"phone\"] = \"[REDACTED]\"\n",
- " \n",
- " if field == \"outputs\":\n",
- " # Redact any credit card patterns\n",
- " if isinstance(data, dict):\n",
- " for key, value in data.items():\n",
- " if isinstance(value, str):\n",
- " # Simple credit card pattern\n",
- " data[key] = re.sub(r'\\d{4}[-\\s]?\\d{4}[-\\s]?\\d{4}[-\\s]?\\d{4}', '[CARD-REDACTED]', value)\n",
- " \n",
- " return data\n",
- "\n",
- "@ag.instrument(\n",
- " spankind=\"workflow\",\n",
- " redact=redact_pii,\n",
- " redact_on_error=False # Don't apply redaction if it raises an error\n",
- ")\n",
- "def process_customer_order(name: str, email: str, phone: str, card_number: str):\n",
- " \"\"\"Process a customer order with PII redaction.\"\"\"\n",
- " return {\n",
- " \"status\": \"processed\",\n",
- " \"customer\": name,\n",
- " \"payment_info\": f\"Charged card ending in {card_number[-4:]}\",\n",
- " \"full_card\": card_number # This will be redacted\n",
- " }\n",
- "\n",
- "# Test with sample data\n",
- "order = process_customer_order(\n",
- " name=\"Jane Smith\",\n",
- " email=\"jane@example.com\", # Will be redacted\n",
- " phone=\"555-1234\", # Will be redacted\n",
- " card_number=\"4111-1111-1111-1111\" # Will be redacted in output\n",
- ")\n",
- "print(f\"Order status: {order['status']}\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### Global Redaction: Apply Rules Across All Functions\n",
- "\n",
- "For organization-wide policies, you can set up global redaction rules during initialization. Note: Since we already called `ag.init()`, this is just for demonstration. In a real application, you would set this during the initial setup."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "# Example of global redaction setup (would be done during ag.init())\n",
- "from typing import Dict, Any\n",
- "\n",
- "def global_redact_function(name: str, field: str, data: Dict[str, Any]):\n",
- " \"\"\"Global redaction that applies to all instrumented functions.\"\"\"\n",
- " # Remove any field containing 'api_key' or 'secret'\n",
- " if isinstance(data, dict):\n",
- " keys_to_redact = [k for k in data.keys() if 'api_key' in k.lower() or 'secret' in k.lower()]\n",
- " for key in keys_to_redact:\n",
- " data[key] = \"[REDACTED]\"\n",
- " \n",
- " return data\n",
- "\n",
- "# In production, you would initialize like this:\n",
- "# ag.init(\n",
- "# redact=global_redact_function,\n",
- "# redact_on_error=True\n",
- "# )\n",
- "\n",
- "print(\"Global redaction would be configured during ag.init()\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Summary\n",
- "\n",
- "In this tutorial, you learned how to:\n",
- "\n",
- "1. ✅ **Set up tracing** with the Agenta SDK and OpenAI auto-instrumentation\n",
- "2. ✅ **Instrument functions** using the `@ag.instrument()` decorator\n",
- "3. ✅ **Store internals and metadata** to capture intermediate data in your workflows\n",
- "4. ✅ **Reference prompt versions** by creating, deploying, and linking traces to applications\n",
- "5. ✅ **Redact sensitive data** using multiple approaches for privacy protection\n",
- "\n",
- "## Next Steps\n",
- "\n",
- "- Explore [distributed tracing](/observability/trace-with-python-sdk/distributed-tracing) for multi-service applications\n",
- "- Learn about [cost tracking](/observability/trace-with-python-sdk/track-costs) to monitor LLM expenses\n",
- "- Understand [trace annotations](/observability/trace-with-python-sdk/annotate-traces) for collecting feedback\n",
- "- Check out the [Agenta UI guide](/observability/using-the-ui/filtering-traces) for filtering and analyzing traces"
- ]
- }
- ],
- "metadata": {
- "kernelspec": {
- "display_name": "Python 3",
- "language": "python",
- "name": "python3"
- },
- "language_info": {
- "codemirror_mode": {
- "name": "ipython",
- "version": 3
- },
- "file_extension": ".py",
- "mimetype": "text/x-python",
- "name": "python",
- "nbconvert_exporter": "python",
- "pygments_lexer": "ipython3",
- "version": "3.9.0"
- },
- "colab": {
- "provenance": []
- }
- },
- "nbformat": 4,
- "nbformat_minor": 0
-}
\ No newline at end of file
diff --git a/examples/jupyter/observability_langchain.ipynb b/examples/jupyter/observability_langchain.ipynb
index 0fa0e070e0..f2b6321392 100644
--- a/examples/jupyter/observability_langchain.ipynb
+++ b/examples/jupyter/observability_langchain.ipynb
@@ -74,10 +74,10 @@
"import os\n",
"import getpass\n",
"\n",
- "os.environ[\"OPENAI_API_KEY\"] = getpass.getpass(prompt=\"OpenAI API key:\")\n",
+ "os.environ[\"OPENAI_API_KEY\"] = getpass.getpass(prompt=\"OpenAI API key:\") \n",
"# Create an Agenta API key under https://cloud.agenta.ai/settings?tab=apiKeys\n",
- "os.environ[\"AGENTA_API_KEY\"] = getpass.getpass(prompt=\"Agenta API key:\")\n",
- "os.environ[\"AGENTA_HOST\"] = \"https://cloud.agenta.ai\" # Change for self-hosted"
+ "os.environ[\"AGENTA_API_KEY\"] = getpass.getpass(prompt=\"Agenta API key:\") \n",
+ "os.environ[\"AGENTA_HOST\"] = \"https://cloud.agenta.ai\" # Change for self-hosted\n"
]
},
{
@@ -115,7 +115,7 @@
"\n",
"ag.init()\n",
"\n",
- "LangchainInstrumentor().instrument()"
+ "LangchainInstrumentor().instrument()\n"
]
},
{
@@ -174,19 +174,17 @@
"Answer:\n",
"\"\"\"\n",
"\n",
- "prompt_template = ChatPromptTemplate(\n",
- " [\n",
- " (\"human\", prompt),\n",
- " ]\n",
- ")\n",
+ "prompt_template = ChatPromptTemplate([\n",
+ " (\"human\", prompt),\n",
+ "])\n",
"\n",
"llm = ChatOpenAI(model=\"gpt-4o-mini\")\n",
"\n",
"loader = WebBaseLoader(\n",
- " web_paths=(\n",
- " \"https://docs.agenta.ai/prompt-engineering/managing-prompts-programatically/create-and-commit\",\n",
+ " web_paths=(\"https://docs.agenta.ai/prompt-engineering/prompt-management/prompt-management-sdk\",),\n",
+ " bs_kwargs=dict(\n",
+ " parse_only=bs4.SoupStrainer('article') # Only parse the core\n",
" ),\n",
- " bs_kwargs=dict(parse_only=bs4.SoupStrainer(\"article\")), # Only parse the core\n",
")\n",
"docs = loader.load()\n",
"\n",
@@ -205,7 +203,8 @@
" | StrOutputParser()\n",
")\n",
"\n",
- "rag_chain.invoke(\"How can I save a new version of a prompt in Agenta?\")"
+ "rag_chain.invoke(\"How can I save a new version of a prompt in Agenta?\")\n",
+ "\n"
]
}
],
diff --git a/examples/jupyter/prompt-management/how-to-prompt-management.ipynb b/examples/jupyter/prompt-management/how-to-prompt-management.ipynb
deleted file mode 100644
index abc071f818..0000000000
--- a/examples/jupyter/prompt-management/how-to-prompt-management.ipynb
+++ /dev/null
@@ -1,826 +0,0 @@
-{
- "cells": [
- {
- "cell_type": "markdown",
- "id": "fe036c83",
- "metadata": {},
- "source": [
- "# A How-To Guide for Prompt Management with the SDK in Agenta\n",
- "\n",
- "This guide covers all prompt management operations using the Agenta SDK: creating variants, committing changes, deploying to environments, and fetching configurations.\n",
- "\n"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "d96a3129",
- "metadata": {
- "vscode": {
- "languageId": "bat"
- }
- },
- "source": [
- "## Install dependencies"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "35bbfebe",
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "Requirement already satisfied: agenta in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (0.51.6)\n",
- "Requirement already satisfied: decorator<6.0.0,>=5.2.1 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (5.2.1)\n",
- "Requirement already satisfied: fastapi<0.117.0,>=0.116.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (0.116.1)\n",
- "Requirement already satisfied: h11>=0.16.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (0.16.0)\n",
- "Requirement already satisfied: httpx>=0.28.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (0.28.1)\n",
- "Requirement already satisfied: huggingface-hub<0.31.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (0.30.2)\n",
- "Requirement already satisfied: importlib-metadata<9.0,>=8.0.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (8.7.0)\n",
- "Requirement already satisfied: jinja2<4.0.0,>=3.1.6 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (3.1.6)\n",
- "Requirement already satisfied: litellm==1.76.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (1.76.0)\n",
- "Requirement already satisfied: openai>=1.100.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (1.106.1)\n",
- "Requirement already satisfied: opentelemetry-api<2.0.0,>=1.27.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (1.36.0)\n",
- "Requirement already satisfied: opentelemetry-exporter-otlp-proto-http<2.0.0,>=1.27.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (1.36.0)\n",
- "Requirement already satisfied: opentelemetry-instrumentation>=0.56b0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (0.57b0)\n",
- "Requirement already satisfied: opentelemetry-sdk<2.0.0,>=1.27.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (1.36.0)\n",
- "Requirement already satisfied: pydantic>=2 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (2.11.7)\n",
- "Requirement already satisfied: python-dotenv<2.0.0,>=1.0.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (1.1.1)\n",
- "Requirement already satisfied: pyyaml<7.0.0,>=6.0.2 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (6.0.2)\n",
- "Requirement already satisfied: starlette<0.48.0,>=0.47.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (0.47.3)\n",
- "Requirement already satisfied: structlog<26.0.0,>=25.2.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (25.4.0)\n",
- "Requirement already satisfied: toml<0.11.0,>=0.10.2 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (0.10.2)\n",
- "Requirement already satisfied: aiohttp>=3.10 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from litellm==1.76.0->agenta) (3.12.15)\n",
- "Requirement already satisfied: click in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from litellm==1.76.0->agenta) (8.2.1)\n",
- "Requirement already satisfied: jsonschema<5.0.0,>=4.22.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from litellm==1.76.0->agenta) (4.25.1)\n",
- "Requirement already satisfied: tiktoken>=0.7.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from litellm==1.76.0->agenta) (0.11.0)\n",
- "Requirement already satisfied: tokenizers in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from litellm==1.76.0->agenta) (0.22.0)\n",
- "Requirement already satisfied: typing-extensions>=4.8.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from fastapi<0.117.0,>=0.116.0->agenta) (4.15.0)\n",
- "Requirement already satisfied: filelock in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from huggingface-hub<0.31.0->agenta) (3.19.1)\n",
- "Requirement already satisfied: fsspec>=2023.5.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from huggingface-hub<0.31.0->agenta) (2025.7.0)\n",
- "Requirement already satisfied: packaging>=20.9 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from huggingface-hub<0.31.0->agenta) (25.0)\n",
- "Requirement already satisfied: requests in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from huggingface-hub<0.31.0->agenta) (2.32.5)\n",
- "Requirement already satisfied: tqdm>=4.42.1 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from huggingface-hub<0.31.0->agenta) (4.67.1)\n",
- "Requirement already satisfied: zipp>=3.20 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from importlib-metadata<9.0,>=8.0.0->agenta) (3.23.0)\n",
- "Requirement already satisfied: MarkupSafe>=2.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from jinja2<4.0.0,>=3.1.6->agenta) (3.0.2)\n",
- "Requirement already satisfied: attrs>=22.2.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from jsonschema<5.0.0,>=4.22.0->litellm==1.76.0->agenta) (25.3.0)\n",
- "Requirement already satisfied: jsonschema-specifications>=2023.03.6 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from jsonschema<5.0.0,>=4.22.0->litellm==1.76.0->agenta) (2025.4.1)\n",
- "Requirement already satisfied: referencing>=0.28.4 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from jsonschema<5.0.0,>=4.22.0->litellm==1.76.0->agenta) (0.36.2)\n",
- "Requirement already satisfied: rpds-py>=0.7.1 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from jsonschema<5.0.0,>=4.22.0->litellm==1.76.0->agenta) (0.27.1)\n",
- "Requirement already satisfied: googleapis-common-protos~=1.52 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from opentelemetry-exporter-otlp-proto-http<2.0.0,>=1.27.0->agenta) (1.70.0)\n",
- "Requirement already satisfied: opentelemetry-exporter-otlp-proto-common==1.36.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from opentelemetry-exporter-otlp-proto-http<2.0.0,>=1.27.0->agenta) (1.36.0)\n",
- "Requirement already satisfied: opentelemetry-proto==1.36.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from opentelemetry-exporter-otlp-proto-http<2.0.0,>=1.27.0->agenta) (1.36.0)\n",
- "Requirement already satisfied: protobuf<7.0,>=5.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from opentelemetry-proto==1.36.0->opentelemetry-exporter-otlp-proto-http<2.0.0,>=1.27.0->agenta) (6.32.0)\n",
- "Requirement already satisfied: opentelemetry-semantic-conventions==0.57b0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from opentelemetry-sdk<2.0.0,>=1.27.0->agenta) (0.57b0)\n",
- "Requirement already satisfied: annotated-types>=0.6.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from pydantic>=2->agenta) (0.7.0)\n",
- "Requirement already satisfied: pydantic-core==2.33.2 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from pydantic>=2->agenta) (2.33.2)\n",
- "Requirement already satisfied: typing-inspection>=0.4.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from pydantic>=2->agenta) (0.4.1)\n",
- "Requirement already satisfied: charset_normalizer<4,>=2 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from requests->huggingface-hub<0.31.0->agenta) (3.4.3)\n",
- "Requirement already satisfied: idna<4,>=2.5 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from requests->huggingface-hub<0.31.0->agenta) (3.10)\n",
- "Requirement already satisfied: urllib3<3,>=1.21.1 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from requests->huggingface-hub<0.31.0->agenta) (2.5.0)\n",
- "Requirement already satisfied: certifi>=2017.4.17 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from requests->huggingface-hub<0.31.0->agenta) (2025.8.3)\n",
- "Requirement already satisfied: anyio<5,>=3.6.2 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from starlette<0.48.0,>=0.47.0->agenta) (4.10.0)\n",
- "Requirement already satisfied: sniffio>=1.1 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from anyio<5,>=3.6.2->starlette<0.48.0,>=0.47.0->agenta) (1.3.1)\n",
- "Requirement already satisfied: aiohappyeyeballs>=2.5.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from aiohttp>=3.10->litellm==1.76.0->agenta) (2.6.1)\n",
- "Requirement already satisfied: aiosignal>=1.4.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from aiohttp>=3.10->litellm==1.76.0->agenta) (1.4.0)\n",
- "Requirement already satisfied: frozenlist>=1.1.1 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from aiohttp>=3.10->litellm==1.76.0->agenta) (1.7.0)\n",
- "Requirement already satisfied: multidict<7.0,>=4.5 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from aiohttp>=3.10->litellm==1.76.0->agenta) (6.6.4)\n",
- "Requirement already satisfied: propcache>=0.2.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from aiohttp>=3.10->litellm==1.76.0->agenta) (0.3.2)\n",
- "Requirement already satisfied: yarl<2.0,>=1.17.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from aiohttp>=3.10->litellm==1.76.0->agenta) (1.20.1)\n",
- "Requirement already satisfied: httpcore==1.* in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from httpx>=0.28.0->agenta) (1.0.9)\n",
- "Requirement already satisfied: distro<2,>=1.7.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from openai>=1.100.0->agenta) (1.9.0)\n",
- "Requirement already satisfied: jiter<1,>=0.4.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from openai>=1.100.0->agenta) (0.10.0)\n",
- "Requirement already satisfied: wrapt<2.0.0,>=1.0.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from opentelemetry-instrumentation>=0.56b0->agenta) (1.17.3)\n",
- "Requirement already satisfied: regex>=2022.1.18 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from tiktoken>=0.7.0->litellm==1.76.0->agenta) (2025.9.1)\n",
- "Note: you may need to restart the kernel to use updated packages.\n"
- ]
- }
- ],
- "source": [
- "pip install -U agenta"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "09579f66",
- "metadata": {},
- "source": [
- "## Setup"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "02fa644e",
- "metadata": {},
- "outputs": [],
- "source": "import os\n\nos.environ[\"AGENTA_API_KEY\"] = \"\"\nos.environ[\"AGENTA_HOST\"] = \"https://cloud.agenta.ai/\""
- },
- {
- "cell_type": "code",
- "execution_count": 16,
- "id": "e43db2b5",
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "2025-09-11T13:55:47.580Z \u001b[38;5;70m[INFO.]\u001b[0m Agenta - SDK version: 0.51.6 \u001b[38;5;245m[agenta.sdk.agenta_init]\u001b[0m \n",
- "2025-09-11T13:55:47.582Z \u001b[38;5;70m[INFO.]\u001b[0m Agenta - Host: http://144.76.237.122 \u001b[38;5;245m[agenta.sdk.agenta_init]\u001b[0m \n",
- "2025-09-11T13:55:47.583Z \u001b[38;5;70m[INFO.]\u001b[0m Agenta - OLTP URL: https://http://144.76.237.122/api/otlp/v1/traces \u001b[38;5;245m[agenta.sdk.tracing.tracing]\u001b[0m \n"
- ]
- },
- {
- "name": "stderr",
- "output_type": "stream",
- "text": [
- "Overriding of current TracerProvider is not allowed\n"
- ]
- }
- ],
- "source": [
- "import agenta as ag\n",
- "from getpass import getpass\n",
- "\n",
- "# Get API key from environment or prompt user\n",
- "api_key = os.getenv(\"AGENTA_API_KEY\")\n",
- "if not api_key:\n",
- " os.environ[\"AGENTA_API_KEY\"] = getpass(\"Enter your Agenta API key: \")\n",
- "\n",
- "\n",
- "# Initialize the Agenta client\n",
- "ag.init()\n"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "564c90d6",
- "metadata": {},
- "source": [
- "## Create new prompts"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 36,
- "id": "05786812",
- "metadata": {},
- "outputs": [],
- "source": [
- "\n",
- "from agenta.sdk.types import PromptTemplate, Message, ModelConfig\n",
- "from pydantic import BaseModel\n",
- "\n",
- "# Creates an empty application\n",
- "app = ag.AppManager.create(\n",
- " app_slug=\"my-completion\",\n",
- " template_key=\"SERVICE:completion\", # we define here the app type\n",
- " # template_key=\"SERVICE:chat\" # chat prompts\n",
- " # template_key=\"CUSTOM\" # custom configuration (schema-less, however unless you provide a URI, you can only use the registry but not the playground)\n",
- ")\n",
- "\n",
- "# Define your configuration model it should alway be of this format for completion and chat apps\n",
- "class Config(BaseModel):\n",
- " prompt: PromptTemplate\n",
- "\n",
- "# Create the initial configuration\n",
- "config = Config(\n",
- " prompt=PromptTemplate(\n",
- " messages=[\n",
- " Message(role=\"system\", content=\"You are an assistant that provides concise answers\"),\n",
- " Message(role=\"user\", content=\"Explain {{topic}} in simple terms\"),\n",
- " ],\n",
- " llm_config=ModelConfig(\n",
- " model=\"gpt-3.5-turbo\",\n",
- " max_tokens=150,\n",
- " temperature=0.7,\n",
- " top_p=1.0,\n",
- " frequency_penalty=0.0,\n",
- " presence_penalty=0.0,\n",
- " ),\n",
- " )\n",
- ")\n",
- "\n",
- "# Create a new variant with the first revision \n",
- "variant = ag.VariantManager.create(\n",
- " parameters=config.model_dump(),\n",
- " app_slug=\"my-completion\",\n",
- " variant_slug=\"default\",\n",
- ")\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 37,
- "id": "25ae5452",
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "{\n",
- " \"app_id\": \"0199396b-aba7-7451-8db4-6fb5e4a1c8d2\",\n",
- " \"app_slug\": \"my-completion\",\n",
- " \"variant_id\": \"0199396b-ac32-7783-a385-09d0a1d084af\",\n",
- " \"variant_slug\": \"default\",\n",
- " \"variant_version\": 1,\n",
- " \"environment_id\": null,\n",
- " \"environment_slug\": null,\n",
- " \"environment_version\": null,\n",
- " \"committed_at\": \"2025-09-11T15:36:21.728674+00:00\",\n",
- " \"committed_by\": \"mahmoud@agenta.ai\",\n",
- " \"committed_by_id\": \"0198eb1b-012a-7891-955e-15ef49443ece\",\n",
- " \"deployed_at\": null,\n",
- " \"deployed_by\": null,\n",
- " \"deployed_by_id\": null,\n",
- " \"params\": {\n",
- " \"prompt\": {\n",
- " \"messages\": [\n",
- " {\n",
- " \"role\": \"system\",\n",
- " \"content\": \"You are an assistant that provides concise answers\",\n",
- " \"name\": null,\n",
- " \"tool_calls\": null,\n",
- " \"tool_call_id\": null\n",
- " },\n",
- " {\n",
- " \"role\": \"user\",\n",
- " \"content\": \"Explain {{topic}} in simple terms\",\n",
- " \"name\": null,\n",
- " \"tool_calls\": null,\n",
- " \"tool_call_id\": null\n",
- " }\n",
- " ],\n",
- " \"system_prompt\": null,\n",
- " \"user_prompt\": null,\n",
- " \"template_format\": \"curly\",\n",
- " \"input_keys\": null,\n",
- " \"llm_config\": {\n",
- " \"model\": \"gpt-3.5-turbo\",\n",
- " \"temperature\": 0.7,\n",
- " \"max_tokens\": 150,\n",
- " \"top_p\": 1.0,\n",
- " \"frequency_penalty\": 0.0,\n",
- " \"presence_penalty\": 0.0,\n",
- " \"response_format\": null,\n",
- " \"stream\": null,\n",
- " \"tools\": null,\n",
- " \"tool_choice\": null\n",
- " }\n",
- " }\n",
- " }\n",
- "}\n"
- ]
- }
- ],
- "source": [
- "\n",
- "print(variant)\n"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "d36a543a",
- "metadata": {},
- "source": [
- "## Creating a new revision"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 38,
- "id": "ff9382d2",
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "{\n",
- " \"app_id\": \"0199396b-aba7-7451-8db4-6fb5e4a1c8d2\",\n",
- " \"app_slug\": \"my-completion\",\n",
- " \"variant_id\": \"0199396b-cbea-7ca2-bc6e-2920316958c3\",\n",
- " \"variant_slug\": \"default\",\n",
- " \"variant_version\": 2,\n",
- " \"environment_id\": null,\n",
- " \"environment_slug\": null,\n",
- " \"environment_version\": null,\n",
- " \"committed_at\": \"2025-09-11T15:36:21.728674+00:00\",\n",
- " \"committed_by\": \"mahmoud@agenta.ai\",\n",
- " \"committed_by_id\": \"0198eb1b-012a-7891-955e-15ef49443ece\",\n",
- " \"deployed_at\": null,\n",
- " \"deployed_by\": null,\n",
- " \"deployed_by_id\": null,\n",
- " \"params\": {\n",
- " \"prompt\": {\n",
- " \"messages\": [\n",
- " {\n",
- " \"role\": \"system\",\n",
- " \"content\": \"You are an assistant that provides VERY concise answers\",\n",
- " \"name\": null,\n",
- " \"tool_calls\": null,\n",
- " \"tool_call_id\": null\n",
- " },\n",
- " {\n",
- " \"role\": \"user\",\n",
- " \"content\": \"Explain {{topic}} in simple terms\",\n",
- " \"name\": null,\n",
- " \"tool_calls\": null,\n",
- " \"tool_call_id\": null\n",
- " }\n",
- " ],\n",
- " \"system_prompt\": null,\n",
- " \"user_prompt\": null,\n",
- " \"template_format\": \"curly\",\n",
- " \"input_keys\": null,\n",
- " \"llm_config\": {\n",
- " \"model\": \"anthropic/claude-3-5-sonnet-20240620\",\n",
- " \"temperature\": 0.7,\n",
- " \"max_tokens\": 150,\n",
- " \"top_p\": 1.0,\n",
- " \"frequency_penalty\": 0.0,\n",
- " \"presence_penalty\": 0.0,\n",
- " \"response_format\": null,\n",
- " \"stream\": null,\n",
- " \"tools\": null,\n",
- " \"tool_choice\": null\n",
- " }\n",
- " }\n",
- " }\n",
- "}\n"
- ]
- }
- ],
- "source": [
- "config2=Config(\n",
- " prompt=PromptTemplate(\n",
- " messages=[\n",
- " Message(role=\"system\", content=\"You are an assistant that provides VERY concise answers\"),\n",
- " Message(role=\"user\", content=\"Explain {{topic}} in simple terms\"),\n",
- " ],\n",
- " llm_config=ModelConfig(\n",
- " model=\"anthropic/claude-3-5-sonnet-20240620\",\n",
- " max_tokens=150,\n",
- " temperature=0.7,\n",
- " top_p=1.0,\n",
- " frequency_penalty=0.0,\n",
- " presence_penalty=0.0,\n",
- " ),\n",
- " )\n",
- ")\n",
- "\n",
- "\n",
- "# Commit the new version\n",
- "variant = ag.VariantManager.commit(\n",
- " parameters=config2.model_dump(),\n",
- " app_slug=\"my-completion\",\n",
- " variant_slug=\"default\",\n",
- ")\n",
- "\n",
- "print(variant)"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "79553cec",
- "metadata": {},
- "source": [
- "## Deploying to an environment"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 43,
- "id": "5fbd3d75",
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "Deployed to staging\n"
- ]
- }
- ],
- "source": [
- "deployment = ag.DeploymentManager.deploy(\n",
- " app_slug=\"my-completion\",\n",
- " variant_slug=\"default\",\n",
- " variant_version=None, # Deploys latest version if not specified\n",
- " environment_slug=\"staging\" # Options: development, staging, production\n",
- ")\n",
- "\n",
- "print(f\"Deployed to {deployment.environment_slug}\")\n",
- "\n",
- "\n"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "7fbae284",
- "metadata": {},
- "source": [
- "## Fetching configurations"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 45,
- "id": "6ab6bab6",
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "Fetched configuration for variant default version 2:\n",
- "{'prompt': {'messages': [{'role': 'system', 'content': 'You are an assistant that provides VERY concise answers', 'name': None, 'tool_calls': None, 'tool_call_id': None}, {'role': 'user', 'content': 'Explain {{topic}} in simple terms', 'name': None, 'tool_calls': None, 'tool_call_id': None}], 'system_prompt': None, 'user_prompt': None, 'template_format': 'curly', 'input_keys': None, 'llm_config': {'model': 'anthropic/claude-3-5-sonnet-20240620', 'temperature': 0.7, 'max_tokens': 150, 'top_p': 1.0, 'frequency_penalty': 0.0, 'presence_penalty': 0.0, 'response_format': None, 'stream': None, 'tools': None, 'tool_choice': None}}}\n",
- "Fetched configuration for staging environment version 1:\n",
- "{'prompt': {'messages': [{'role': 'system', 'content': 'You are an assistant that provides VERY concise answers', 'name': None, 'tool_calls': None, 'tool_call_id': None}, {'role': 'user', 'content': 'Explain {{topic}} in simple terms', 'name': None, 'tool_calls': None, 'tool_call_id': None}], 'system_prompt': None, 'user_prompt': None, 'template_format': 'curly', 'input_keys': None, 'llm_config': {'model': 'anthropic/claude-3-5-sonnet-20240620', 'temperature': 0.7, 'max_tokens': 150, 'top_p': 1.0, 'frequency_penalty': 0.0, 'presence_penalty': 0.0, 'response_format': None, 'stream': None, 'tools': None, 'tool_choice': None}}}\n"
- ]
- }
- ],
- "source": [
- "# Fetching by variant reference\n",
- "config = ag.ConfigManager.get_from_registry(\n",
- " app_slug=\"my-completion\",\n",
- " variant_slug=\"default\",\n",
- " variant_version=2 # Optional: fetches latest if not provided\n",
- ")\n",
- "print(\"Fetched configuration for variant default version 2:\")\n",
- "print(config)\n",
- "\n",
- "# Fetching by environment reference (from staging)\n",
- "config = ag.ConfigManager.get_from_registry(\n",
- " app_slug=\"my-completion\",\n",
- " environment_slug=\"staging\",\n",
- " environment_version=1 # Optional: If not provided, fetches the latest version\n",
- ")\n",
- "\n",
- "print(f\"Fetched configuration for staging environment version 1:\")\n",
- "print(config)\n",
- "\n",
- "\n"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "4e298bae",
- "metadata": {},
- "source": [
- "## Listing all variants"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 46,
- "id": "792e7804",
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "[{\n",
- " \"app_id\": \"0199396b-aba7-7451-8db4-6fb5e4a1c8d2\",\n",
- " \"app_slug\": \"my-completion\",\n",
- " \"variant_id\": \"0199396b-cbea-7ca2-bc6e-2920316958c3\",\n",
- " \"variant_slug\": \"default\",\n",
- " \"variant_version\": 2,\n",
- " \"environment_id\": null,\n",
- " \"environment_slug\": null,\n",
- " \"environment_version\": null,\n",
- " \"committed_at\": \"2025-09-11T15:36:29.930593+00:00\",\n",
- " \"committed_by\": \"mahmoud@agenta.ai\",\n",
- " \"committed_by_id\": \"0198eb1b-012a-7891-955e-15ef49443ece\",\n",
- " \"deployed_at\": null,\n",
- " \"deployed_by\": null,\n",
- " \"deployed_by_id\": null,\n",
- " \"params\": {\n",
- " \"prompt\": {\n",
- " \"messages\": [\n",
- " {\n",
- " \"role\": \"system\",\n",
- " \"content\": \"You are an assistant that provides VERY concise answers\",\n",
- " \"name\": null,\n",
- " \"tool_calls\": null,\n",
- " \"tool_call_id\": null\n",
- " },\n",
- " {\n",
- " \"role\": \"user\",\n",
- " \"content\": \"Explain {{topic}} in simple terms\",\n",
- " \"name\": null,\n",
- " \"tool_calls\": null,\n",
- " \"tool_call_id\": null\n",
- " }\n",
- " ],\n",
- " \"system_prompt\": null,\n",
- " \"user_prompt\": null,\n",
- " \"template_format\": \"curly\",\n",
- " \"input_keys\": null,\n",
- " \"llm_config\": {\n",
- " \"model\": \"anthropic/claude-3-5-sonnet-20240620\",\n",
- " \"temperature\": 0.7,\n",
- " \"max_tokens\": 150,\n",
- " \"top_p\": 1.0,\n",
- " \"frequency_penalty\": 0.0,\n",
- " \"presence_penalty\": 0.0,\n",
- " \"response_format\": null,\n",
- " \"stream\": null,\n",
- " \"tools\": null,\n",
- " \"tool_choice\": null\n",
- " }\n",
- " }\n",
- " }\n",
- "}]\n"
- ]
- }
- ],
- "source": [
- "variants = ag.VariantManager.list(\n",
- " app_slug=\"my-completion\"\n",
- ")\n",
- "\n",
- "print(variants)\n",
- "\n"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "23503c53",
- "metadata": {},
- "source": [
- "## Fetching variant history"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 47,
- "id": "9fcf9a8b",
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "[{\n",
- " \"app_id\": \"0199396b-aba7-7451-8db4-6fb5e4a1c8d2\",\n",
- " \"app_slug\": \"my-completion\",\n",
- " \"variant_id\": \"0199396b-abe0-74a1-9e4c-80779f36c9e5\",\n",
- " \"variant_slug\": \"default\",\n",
- " \"variant_version\": 2,\n",
- " \"environment_id\": null,\n",
- " \"environment_slug\": null,\n",
- " \"environment_version\": null,\n",
- " \"committed_at\": \"2025-09-11T15:36:21.733212+00:00\",\n",
- " \"committed_by\": \"mahmoud@agenta.ai\",\n",
- " \"committed_by_id\": \"0198eb1b-012a-7891-955e-15ef49443ece\",\n",
- " \"deployed_at\": null,\n",
- " \"deployed_by\": null,\n",
- " \"deployed_by_id\": null,\n",
- " \"params\": {}\n",
- "}, {\n",
- " \"app_id\": \"0199396b-aba7-7451-8db4-6fb5e4a1c8d2\",\n",
- " \"app_slug\": \"my-completion\",\n",
- " \"variant_id\": \"0199396b-abe0-74a1-9e4c-80779f36c9e5\",\n",
- " \"variant_slug\": \"default\",\n",
- " \"variant_version\": 2,\n",
- " \"environment_id\": null,\n",
- " \"environment_slug\": null,\n",
- " \"environment_version\": null,\n",
- " \"committed_at\": \"2025-09-11T15:36:21.810617+00:00\",\n",
- " \"committed_by\": \"mahmoud@agenta.ai\",\n",
- " \"committed_by_id\": \"0198eb1b-012a-7891-955e-15ef49443ece\",\n",
- " \"deployed_at\": null,\n",
- " \"deployed_by\": null,\n",
- " \"deployed_by_id\": null,\n",
- " \"params\": {\n",
- " \"prompt\": {\n",
- " \"messages\": [\n",
- " {\n",
- " \"role\": \"system\",\n",
- " \"content\": \"You are an assistant that provides concise answers\",\n",
- " \"name\": null,\n",
- " \"tool_calls\": null,\n",
- " \"tool_call_id\": null\n",
- " },\n",
- " {\n",
- " \"role\": \"user\",\n",
- " \"content\": \"Explain {{topic}} in simple terms\",\n",
- " \"name\": null,\n",
- " \"tool_calls\": null,\n",
- " \"tool_call_id\": null\n",
- " }\n",
- " ],\n",
- " \"system_prompt\": null,\n",
- " \"user_prompt\": null,\n",
- " \"template_format\": \"curly\",\n",
- " \"input_keys\": null,\n",
- " \"llm_config\": {\n",
- " \"model\": \"gpt-3.5-turbo\",\n",
- " \"temperature\": 0.7,\n",
- " \"max_tokens\": 150,\n",
- " \"top_p\": 1.0,\n",
- " \"frequency_penalty\": 0.0,\n",
- " \"presence_penalty\": 0.0,\n",
- " \"response_format\": null,\n",
- " \"stream\": null,\n",
- " \"tools\": null,\n",
- " \"tool_choice\": null\n",
- " }\n",
- " }\n",
- " }\n",
- "}, {\n",
- " \"app_id\": \"0199396b-aba7-7451-8db4-6fb5e4a1c8d2\",\n",
- " \"app_slug\": \"my-completion\",\n",
- " \"variant_id\": \"0199396b-abe0-74a1-9e4c-80779f36c9e5\",\n",
- " \"variant_slug\": \"default\",\n",
- " \"variant_version\": 2,\n",
- " \"environment_id\": null,\n",
- " \"environment_slug\": null,\n",
- " \"environment_version\": null,\n",
- " \"committed_at\": \"2025-09-11T15:36:29.930593+00:00\",\n",
- " \"committed_by\": \"mahmoud@agenta.ai\",\n",
- " \"committed_by_id\": \"0198eb1b-012a-7891-955e-15ef49443ece\",\n",
- " \"deployed_at\": null,\n",
- " \"deployed_by\": null,\n",
- " \"deployed_by_id\": null,\n",
- " \"params\": {\n",
- " \"prompt\": {\n",
- " \"messages\": [\n",
- " {\n",
- " \"role\": \"system\",\n",
- " \"content\": \"You are an assistant that provides VERY concise answers\",\n",
- " \"name\": null,\n",
- " \"tool_calls\": null,\n",
- " \"tool_call_id\": null\n",
- " },\n",
- " {\n",
- " \"role\": \"user\",\n",
- " \"content\": \"Explain {{topic}} in simple terms\",\n",
- " \"name\": null,\n",
- " \"tool_calls\": null,\n",
- " \"tool_call_id\": null\n",
- " }\n",
- " ],\n",
- " \"system_prompt\": null,\n",
- " \"user_prompt\": null,\n",
- " \"template_format\": \"curly\",\n",
- " \"input_keys\": null,\n",
- " \"llm_config\": {\n",
- " \"model\": \"anthropic/claude-3-5-sonnet-20240620\",\n",
- " \"temperature\": 0.7,\n",
- " \"max_tokens\": 150,\n",
- " \"top_p\": 1.0,\n",
- " \"frequency_penalty\": 0.0,\n",
- " \"presence_penalty\": 0.0,\n",
- " \"response_format\": null,\n",
- " \"stream\": null,\n",
- " \"tools\": null,\n",
- " \"tool_choice\": null\n",
- " }\n",
- " }\n",
- " }\n",
- "}]\n"
- ]
- }
- ],
- "source": [
- "versions = ag.VariantManager.history(\n",
- " variant_slug=\"default\",\n",
- " app_slug=\"my-completion\"\n",
- ")\n",
- "\n",
- "print(versions)\n",
- "\n"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "e8f52205",
- "metadata": {},
- "source": [
- "## Deleting a variant"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 50,
- "id": "9d8d2e39",
- "metadata": {},
- "outputs": [
- {
- "data": {
- "text/plain": [
- "204"
- ]
- },
- "execution_count": 50,
- "metadata": {},
- "output_type": "execute_result"
- }
- ],
- "source": [
- "variant = ag.VariantManager.create(\n",
- " parameters=config,\n",
- " app_slug=\"my-completion\",\n",
- " variant_slug=\"to-remove\",\n",
- ")\n",
- "\n",
- "ag.VariantManager.delete(\n",
- " app_slug=\"my-completion\",\n",
- " variant_slug=\"to-remove\"\n",
- ")\n",
- "\n"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "efad781d",
- "metadata": {},
- "source": [
- "## Creating an app with a custom configuration (schema-less)"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 52,
- "id": "dfb8437e",
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "{\n",
- " \"app_id\": \"01993970-5fa6-7561-bd77-edb117ee6788\",\n",
- " \"app_slug\": \"my-rag-config\",\n",
- " \"variant_id\": \"01993970-6008-7ca3-abb8-d5bfdf798fc4\",\n",
- " \"variant_slug\": \"default\",\n",
- " \"variant_version\": 1,\n",
- " \"environment_id\": null,\n",
- " \"environment_slug\": null,\n",
- " \"environment_version\": null,\n",
- " \"committed_at\": \"2025-09-11T15:41:29.923484+00:00\",\n",
- " \"committed_by\": \"mahmoud@agenta.ai\",\n",
- " \"committed_by_id\": \"0198eb1b-012a-7891-955e-15ef49443ece\",\n",
- " \"deployed_at\": null,\n",
- " \"deployed_by\": null,\n",
- " \"deployed_by_id\": null,\n",
- " \"params\": {\n",
- " \"rag-prompt\": \"You are a helpful assistant that can answer questions and help with tasks.\",\n",
- " \"rag-chunk_size\": 100,\n",
- " \"rag-chunk_overlap\": 20\n",
- " }\n",
- "}\n"
- ]
- }
- ],
- "source": [
- "app = ag.AppManager.create(\n",
- " app_slug=\"my-rag-config\",\n",
- " template_key=\"CUSTOM\", # we define here the app type\n",
- ")\n",
- "\n",
- "rag_config = {\n",
- " \"rag-prompt\": \"You are a helpful assistant that can answer questions and help with tasks.\",\n",
- " \"rag-chunk_size\": 100,\n",
- " \"rag-chunk_overlap\": 20,\n",
- "}\n",
- "variant = ag.VariantManager.create(\n",
- " parameters=rag_config,\n",
- " app_slug=\"my-rag-config\",\n",
- " variant_slug=\"default\",\n",
- ")\n",
- "\n",
- "print(variant)\n"
- ]
- }
- ],
- "metadata": {
- "kernelspec": {
- "display_name": ".venv (3.12.3)",
- "language": "python",
- "name": "python3"
- },
- "language_info": {
- "codemirror_mode": {
- "name": "ipython",
- "version": 3
- },
- "file_extension": ".py",
- "mimetype": "text/x-python",
- "name": "python",
- "nbconvert_exporter": "python",
- "pygments_lexer": "ipython3",
- "version": "3.12.3"
- }
- },
- "nbformat": 4,
- "nbformat_minor": 5
-}
\ No newline at end of file
diff --git a/examples/jupyter/prompt-management/manage-prompts-with-sdk-tutorial.ipynb b/examples/jupyter/prompt-management/manage-prompts-with-sdk-tutorial.ipynb
deleted file mode 100644
index 3eaa4c44e7..0000000000
--- a/examples/jupyter/prompt-management/manage-prompts-with-sdk-tutorial.ipynb
+++ /dev/null
@@ -1,634 +0,0 @@
-{
- "cells": [
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "# Manage Prompts with SDK - Tutorial\n",
- "\n",
- "You can manage prompts easily through Agenta's web UI. But sometimes you might want to do things programmatically instead of using the interface.\n",
- "\n",
- "In this tutorial, we'll use the Agenta SDK to create a new prompt, commit changes, deploy them to production, then fetch their configuration and use it to call the LLM.\n",
- "\n",
- "## Tutorial Overview\n",
- "\n",
- "Before we begin, let's quickly review how Agenta versions prompts:\n",
- "\n",
- "Agenta follows a structure similar to **git** for prompt versioning. Instead of having one commit history, it uses **multiple branches (called variants)** where changes can be committed, and **environments** where these changes can be deployed (and used in your application).\n",
- "\n",
- "The workflow for deploying a change to production that we'll follow in this tutorial is:\n",
- "\n",
- "1. Create a new variant\n",
- "2. Commit a change to that variant\n",
- "3. Deploy that commit (variant/version) to the production environment\n",
- "4. Commit a new change to that variant\n",
- "5. Fetch the config from that environment\n"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Install dependencies\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 1,
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "Requirement already satisfied: agenta in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (0.51.6)\n",
- "Requirement already satisfied: openai in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (1.106.1)\n",
- "Collecting openai\n",
- " Downloading openai-1.107.1-py3-none-any.whl.metadata (29 kB)\n",
- "Requirement already satisfied: decorator<6.0.0,>=5.2.1 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (5.2.1)\n",
- "Requirement already satisfied: fastapi<0.117.0,>=0.116.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (0.116.1)\n",
- "Requirement already satisfied: h11>=0.16.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (0.16.0)\n",
- "Requirement already satisfied: httpx>=0.28.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (0.28.1)\n",
- "Requirement already satisfied: huggingface-hub<0.31.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (0.30.2)\n",
- "Requirement already satisfied: importlib-metadata<9.0,>=8.0.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (8.7.0)\n",
- "Requirement already satisfied: jinja2<4.0.0,>=3.1.6 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (3.1.6)\n",
- "Requirement already satisfied: litellm==1.76.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (1.76.0)\n",
- "Requirement already satisfied: opentelemetry-api<2.0.0,>=1.27.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (1.36.0)\n",
- "Requirement already satisfied: opentelemetry-exporter-otlp-proto-http<2.0.0,>=1.27.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (1.36.0)\n",
- "Requirement already satisfied: opentelemetry-instrumentation>=0.56b0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (0.57b0)\n",
- "Requirement already satisfied: opentelemetry-sdk<2.0.0,>=1.27.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (1.36.0)\n",
- "Requirement already satisfied: pydantic>=2 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (2.11.7)\n",
- "Requirement already satisfied: python-dotenv<2.0.0,>=1.0.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (1.1.1)\n",
- "Requirement already satisfied: pyyaml<7.0.0,>=6.0.2 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (6.0.2)\n",
- "Requirement already satisfied: starlette<0.48.0,>=0.47.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (0.47.3)\n",
- "Requirement already satisfied: structlog<26.0.0,>=25.2.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (25.4.0)\n",
- "Requirement already satisfied: toml<0.11.0,>=0.10.2 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from agenta) (0.10.2)\n",
- "Requirement already satisfied: aiohttp>=3.10 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from litellm==1.76.0->agenta) (3.12.15)\n",
- "Requirement already satisfied: click in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from litellm==1.76.0->agenta) (8.2.1)\n",
- "Requirement already satisfied: jsonschema<5.0.0,>=4.22.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from litellm==1.76.0->agenta) (4.25.1)\n",
- "Requirement already satisfied: tiktoken>=0.7.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from litellm==1.76.0->agenta) (0.11.0)\n",
- "Requirement already satisfied: tokenizers in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from litellm==1.76.0->agenta) (0.22.0)\n",
- "Requirement already satisfied: typing-extensions>=4.8.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from fastapi<0.117.0,>=0.116.0->agenta) (4.15.0)\n",
- "Requirement already satisfied: filelock in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from huggingface-hub<0.31.0->agenta) (3.19.1)\n",
- "Requirement already satisfied: fsspec>=2023.5.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from huggingface-hub<0.31.0->agenta) (2025.7.0)\n",
- "Requirement already satisfied: packaging>=20.9 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from huggingface-hub<0.31.0->agenta) (25.0)\n",
- "Requirement already satisfied: requests in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from huggingface-hub<0.31.0->agenta) (2.32.5)\n",
- "Requirement already satisfied: tqdm>=4.42.1 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from huggingface-hub<0.31.0->agenta) (4.67.1)\n",
- "Requirement already satisfied: zipp>=3.20 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from importlib-metadata<9.0,>=8.0.0->agenta) (3.23.0)\n",
- "Requirement already satisfied: MarkupSafe>=2.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from jinja2<4.0.0,>=3.1.6->agenta) (3.0.2)\n",
- "Requirement already satisfied: attrs>=22.2.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from jsonschema<5.0.0,>=4.22.0->litellm==1.76.0->agenta) (25.3.0)\n",
- "Requirement already satisfied: jsonschema-specifications>=2023.03.6 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from jsonschema<5.0.0,>=4.22.0->litellm==1.76.0->agenta) (2025.4.1)\n",
- "Requirement already satisfied: referencing>=0.28.4 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from jsonschema<5.0.0,>=4.22.0->litellm==1.76.0->agenta) (0.36.2)\n",
- "Requirement already satisfied: rpds-py>=0.7.1 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from jsonschema<5.0.0,>=4.22.0->litellm==1.76.0->agenta) (0.27.1)\n",
- "Requirement already satisfied: googleapis-common-protos~=1.52 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from opentelemetry-exporter-otlp-proto-http<2.0.0,>=1.27.0->agenta) (1.70.0)\n",
- "Requirement already satisfied: opentelemetry-exporter-otlp-proto-common==1.36.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from opentelemetry-exporter-otlp-proto-http<2.0.0,>=1.27.0->agenta) (1.36.0)\n",
- "Requirement already satisfied: opentelemetry-proto==1.36.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from opentelemetry-exporter-otlp-proto-http<2.0.0,>=1.27.0->agenta) (1.36.0)\n",
- "Requirement already satisfied: protobuf<7.0,>=5.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from opentelemetry-proto==1.36.0->opentelemetry-exporter-otlp-proto-http<2.0.0,>=1.27.0->agenta) (6.32.0)\n",
- "Requirement already satisfied: opentelemetry-semantic-conventions==0.57b0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from opentelemetry-sdk<2.0.0,>=1.27.0->agenta) (0.57b0)\n",
- "Requirement already satisfied: annotated-types>=0.6.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from pydantic>=2->agenta) (0.7.0)\n",
- "Requirement already satisfied: pydantic-core==2.33.2 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from pydantic>=2->agenta) (2.33.2)\n",
- "Requirement already satisfied: typing-inspection>=0.4.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from pydantic>=2->agenta) (0.4.1)\n",
- "Requirement already satisfied: charset_normalizer<4,>=2 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from requests->huggingface-hub<0.31.0->agenta) (3.4.3)\n",
- "Requirement already satisfied: idna<4,>=2.5 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from requests->huggingface-hub<0.31.0->agenta) (3.10)\n",
- "Requirement already satisfied: urllib3<3,>=1.21.1 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from requests->huggingface-hub<0.31.0->agenta) (2.5.0)\n",
- "Requirement already satisfied: certifi>=2017.4.17 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from requests->huggingface-hub<0.31.0->agenta) (2025.8.3)\n",
- "Requirement already satisfied: anyio<5,>=3.6.2 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from starlette<0.48.0,>=0.47.0->agenta) (4.10.0)\n",
- "Requirement already satisfied: sniffio>=1.1 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from anyio<5,>=3.6.2->starlette<0.48.0,>=0.47.0->agenta) (1.3.1)\n",
- "Requirement already satisfied: distro<2,>=1.7.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from openai) (1.9.0)\n",
- "Requirement already satisfied: jiter<1,>=0.4.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from openai) (0.10.0)\n",
- "Requirement already satisfied: httpcore==1.* in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from httpx>=0.28.0->agenta) (1.0.9)\n",
- "Requirement already satisfied: aiohappyeyeballs>=2.5.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from aiohttp>=3.10->litellm==1.76.0->agenta) (2.6.1)\n",
- "Requirement already satisfied: aiosignal>=1.4.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from aiohttp>=3.10->litellm==1.76.0->agenta) (1.4.0)\n",
- "Requirement already satisfied: frozenlist>=1.1.1 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from aiohttp>=3.10->litellm==1.76.0->agenta) (1.7.0)\n",
- "Requirement already satisfied: multidict<7.0,>=4.5 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from aiohttp>=3.10->litellm==1.76.0->agenta) (6.6.4)\n",
- "Requirement already satisfied: propcache>=0.2.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from aiohttp>=3.10->litellm==1.76.0->agenta) (0.3.2)\n",
- "Requirement already satisfied: yarl<2.0,>=1.17.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from aiohttp>=3.10->litellm==1.76.0->agenta) (1.20.1)\n",
- "Requirement already satisfied: wrapt<2.0.0,>=1.0.0 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from opentelemetry-instrumentation>=0.56b0->agenta) (1.17.3)\n",
- "Requirement already satisfied: regex>=2022.1.18 in /home/mahmoud/code/agenta_cloud/.venv/lib/python3.12/site-packages (from tiktoken>=0.7.0->litellm==1.76.0->agenta) (2025.9.1)\n",
- "Downloading openai-1.107.1-py3-none-any.whl (945 kB)\n",
- "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m945.2/945.2 kB\u001b[0m \u001b[31m22.6 MB/s\u001b[0m \u001b[33m0:00:00\u001b[0m\n",
- "\u001b[?25hInstalling collected packages: openai\n",
- " Attempting uninstall: openai\n",
- " Found existing installation: openai 1.106.1\n",
- " Uninstalling openai-1.106.1:\n",
- " Successfully uninstalled openai-1.106.1\n",
- "Successfully installed openai-1.107.1\n",
- "Note: you may need to restart the kernel to use updated packages.\n"
- ]
- }
- ],
- "source": [
- "pip install -U agenta openai\n"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Setup\n",
- "\n",
- "Before using the SDK, we need to initialize it using the `ag.init()` method, which takes the host (default `cloud.agenta.ai`) and the API key (not required for community editions):\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": "import os\nos.environ[\"AGENTA_HOST\"] = \"https://cloud.agenta.ai/\" # Default value, no need to set explicitly unless self-hosted\nos.environ[\"AGENTA_API_KEY\"] = \"\"\nos.environ[\"OPENAI_API_KEY\"] = \"\""
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "2025-09-11T15:56:10.922Z \u001b[38;5;70m[INFO.]\u001b[0m Agenta - SDK version: 0.51.6 \u001b[38;5;245m[agenta.sdk.agenta_init]\u001b[0m \n",
- "2025-09-11T15:56:10.923Z \u001b[38;5;70m[INFO.]\u001b[0m Agenta - Host: http://144.76.237.122 \u001b[38;5;245m[agenta.sdk.agenta_init]\u001b[0m \n",
- "2025-09-11T15:56:10.923Z \u001b[38;5;70m[INFO.]\u001b[0m Agenta - OLTP URL: http://144.76.237.122/api/otlp/v1/traces \u001b[38;5;245m[agenta.sdk.tracing.tracing]\u001b[0m \n"
- ]
- },
- {
- "name": "stderr",
- "output_type": "stream",
- "text": [
- "Overriding of current TracerProvider is not allowed\n"
- ]
- }
- ],
- "source": [
- "import os\n",
- "import agenta as ag\n",
- "from getpass import getpass\n",
- "\n",
- "# Initialize the SDK with your API key\n",
- "api_key = os.getenv(\"AGENTA_API_KEY\")\n",
- "if not api_key:\n",
- " os.environ[\"AGENTA_API_KEY\"] = getpass(\"Enter your Agenta API key: \")\n",
- "\n",
- "openai_api_key = os.getenv(\"OPENAI_API_KEY\")\n",
- "if not openai_api_key:\n",
- " os.environ[\"OPENAI_API_KEY\"] = getpass(\"Enter your OpenAI API key: \")\n",
- "\n",
- "# Initialize the SDK\n",
- "ag.init()\n"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Creating a new prompt\n",
- "\n",
- "We're going to create a new completion prompt called `topic-explainer`.\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 3,
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "Created application:\n",
- "app_id='0199397d-14f4-7bb2-a9c5-9a12934fb6c3' app_name='topic-explainer'\n"
- ]
- }
- ],
- "source": [
- "# Creates an empty application\n",
- "app = ag.AppManager.create(\n",
- " app_slug=\"topic-explainer\",\n",
- " template_key=\"SERVICE:completion\", # we define here the app type, choices are SERVICE:completion, SERVICE:chat, CUSTOM\n",
- ")\n",
- "\n",
- "print(\"Created application:\")\n",
- "print(app)\n"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- ":::warning\n",
- "The app created until now is empty. You cannot use it from the UI yet. You need to create a variant and commit changes to it to be able to use it (next section).\n",
- ":::\n"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Creating a new variant\n",
- "\n",
- "Variants are similar to branches in **git**. Any change to the prompt must first be committed to a variant. Here, we'll create a new variant and make our first commit to it using the `VariantManager.create` method:\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 4,
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "Created variant:\n",
- "{\n",
- " \"app_id\": \"0199397d-14f4-7bb2-a9c5-9a12934fb6c3\",\n",
- " \"app_slug\": \"topic-explainer\",\n",
- " \"variant_id\": \"0199397d-1d46-74c3-bc3f-e63f2357c7dc\",\n",
- " \"variant_slug\": \"new-variant\",\n",
- " \"variant_version\": 1,\n",
- " \"environment_id\": null,\n",
- " \"environment_slug\": null,\n",
- " \"environment_version\": null,\n",
- " \"committed_at\": \"2025-09-11T15:55:24.842469+00:00\",\n",
- " \"committed_by\": \"mahmoud@agenta.ai\",\n",
- " \"committed_by_id\": \"0198eb1b-012a-7891-955e-15ef49443ece\",\n",
- " \"deployed_at\": null,\n",
- " \"deployed_by\": null,\n",
- " \"deployed_by_id\": null,\n",
- " \"params\": {\n",
- " \"prompt\": {\n",
- " \"messages\": [\n",
- " {\n",
- " \"role\": \"system\",\n",
- " \"content\": \"You are an assistant that provides concise answers\",\n",
- " \"name\": null,\n",
- " \"tool_calls\": null,\n",
- " \"tool_call_id\": null\n",
- " },\n",
- " {\n",
- " \"role\": \"user\",\n",
- " \"content\": \"Explain {{topic}} in simple terms\",\n",
- " \"name\": null,\n",
- " \"tool_calls\": null,\n",
- " \"tool_call_id\": null\n",
- " }\n",
- " ],\n",
- " \"system_prompt\": null,\n",
- " \"user_prompt\": null,\n",
- " \"template_format\": \"curly\",\n",
- " \"input_keys\": null,\n",
- " \"llm_config\": {\n",
- " \"model\": \"gpt-3.5-turbo\",\n",
- " \"temperature\": 0.6,\n",
- " \"max_tokens\": 150,\n",
- " \"top_p\": 1.0,\n",
- " \"frequency_penalty\": 0.0,\n",
- " \"presence_penalty\": 0.0,\n",
- " \"response_format\": null,\n",
- " \"stream\": null,\n",
- " \"tools\": null,\n",
- " \"tool_choice\": null\n",
- " }\n",
- " }\n",
- " }\n",
- "}\n"
- ]
- }
- ],
- "source": [
- "from agenta.sdk.types import PromptTemplate, Message, ModelConfig\n",
- "from pydantic import BaseModel\n",
- "\n",
- "# We need to create a Pydantic Model with a `prompt` field of type `PromptTemplate`\n",
- "class Config(BaseModel):\n",
- " prompt: PromptTemplate\n",
- "\n",
- "config = Config(\n",
- " prompt=PromptTemplate(\n",
- " messages=[\n",
- " Message(role=\"system\", content=\"You are an assistant that provides concise answers\"),\n",
- " Message(role=\"user\", content=\"Explain {{topic}} in simple terms\"),\n",
- " ],\n",
- " llm_config=ModelConfig(\n",
- " model=\"gpt-3.5-turbo\",\n",
- " max_tokens=150,\n",
- " temperature=0.6,\n",
- " top_p=1.0,\n",
- " frequency_penalty=0.0,\n",
- " presence_penalty=0.0,\n",
- " ),\n",
- " template_format=\"curly\"\n",
- " )\n",
- ")\n",
- "\n",
- "# Create a new variant\n",
- "variant = ag.VariantManager.create(\n",
- " parameters=config.model_dump(),\n",
- " app_slug=\"topic-explainer\",\n",
- " variant_slug=\"new-variant\"\n",
- ")\n",
- "\n",
- "print(\"Created variant:\")\n",
- "print(variant)\n"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "This command will create a new variant and initialize it with the first commit containing the provided parameters.\n",
- "\n",
- "- **Parameters:**\n",
- " - `app_slug`: The unique slug of your application.\n",
- " - `variant_slug`: The unique slug of the new variant.\n",
- " - `parameters`: A dictionary containing the initial configuration parameters.\n"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Deploying changes to the production environment\n",
- "\n",
- "To deploy our commit to an environment, use the `DeploymentManager.deploy` method.\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 5,
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "Deployed variant to environment:\n",
- "{\n",
- " \"app_id\": \"0199397d-14f4-7bb2-a9c5-9a12934fb6c3\",\n",
- " \"app_slug\": \"topic-explainer\",\n",
- " \"variant_id\": \"0199397d-1d46-74c3-bc3f-e63f2357c7dc\",\n",
- " \"variant_slug\": \"new-variant\",\n",
- " \"variant_version\": 1,\n",
- " \"environment_id\": \"0199397d-23fe-7621-8d30-29426339a777\",\n",
- " \"environment_slug\": \"production\",\n",
- " \"environment_version\": 1,\n",
- " \"committed_at\": \"2025-09-11T15:55:24.842469+00:00\",\n",
- " \"committed_by\": \"mahmoud@agenta.ai\",\n",
- " \"committed_by_id\": \"0198eb1b-012a-7891-955e-15ef49443ece\",\n",
- " \"deployed_at\": \"2025-09-11T15:55:26.590624+00:00\",\n",
- " \"deployed_by\": \"mahmoud@agenta.ai\",\n",
- " \"deployed_by_id\": \"0198eb1b-012a-7891-955e-15ef49443ece\"\n",
- "}\n"
- ]
- }
- ],
- "source": [
- "# Deploy the variant to the production environment\n",
- "deployment = ag.DeploymentManager.deploy(\n",
- " app_slug=\"topic-explainer\",\n",
- " variant_slug=\"new-variant\",\n",
- " environment_slug=\"production\",\n",
- ")\n",
- "\n",
- "print(\"Deployed variant to environment:\")\n",
- "print(deployment)\n"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "- **Parameters:**\n",
- " - `environment_slug`: The slug of the environment (`development`, `staging`, or `production`).\n",
- "\n",
- "- **Notes:**\n",
- " - Deploying a variant without specifying a `variant_version` deploys the latest version.\n"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Committing a change to variant\n",
- "\n",
- "We're now going to commit changes to our variant. Note that this will not modify the version in deployment!\n",
- "\n",
- "To save changes to a variant (creating a new version), we are going to use the `VariantManager.commit` method with explicit parameters.\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 6,
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "Committed new version of variant:\n",
- "{\n",
- " \"app_id\": \"0199397d-14f4-7bb2-a9c5-9a12934fb6c3\",\n",
- " \"app_slug\": \"topic-explainer\",\n",
- " \"variant_id\": \"0199397d-2a70-7f20-ba60-2b810e512400\",\n",
- " \"variant_slug\": \"new-variant\",\n",
- " \"variant_version\": 2,\n",
- " \"environment_id\": null,\n",
- " \"environment_slug\": null,\n",
- " \"environment_version\": null,\n",
- " \"committed_at\": \"2025-09-11T15:55:24.842469+00:00\",\n",
- " \"committed_by\": \"mahmoud@agenta.ai\",\n",
- " \"committed_by_id\": \"0198eb1b-012a-7891-955e-15ef49443ece\",\n",
- " \"deployed_at\": null,\n",
- " \"deployed_by\": null,\n",
- " \"deployed_by_id\": null,\n",
- " \"params\": {\n",
- " \"prompt\": {\n",
- " \"messages\": [\n",
- " {\n",
- " \"role\": \"system\",\n",
- " \"content\": \"You are an assistant that provides concise answers\",\n",
- " \"name\": null,\n",
- " \"tool_calls\": null,\n",
- " \"tool_call_id\": null\n",
- " },\n",
- " {\n",
- " \"role\": \"user\",\n",
- " \"content\": \"Use Paul Graham style to explain {{topic}} in simple terms.\",\n",
- " \"name\": null,\n",
- " \"tool_calls\": null,\n",
- " \"tool_call_id\": null\n",
- " }\n",
- " ],\n",
- " \"system_prompt\": null,\n",
- " \"user_prompt\": null,\n",
- " \"template_format\": \"curly\",\n",
- " \"input_keys\": null,\n",
- " \"llm_config\": {\n",
- " \"model\": \"gpt-3.5-turbo\",\n",
- " \"temperature\": 0.9,\n",
- " \"max_tokens\": 150,\n",
- " \"top_p\": 1.0,\n",
- " \"frequency_penalty\": 0.0,\n",
- " \"presence_penalty\": 0.0,\n",
- " \"response_format\": null,\n",
- " \"stream\": null,\n",
- " \"tools\": null,\n",
- " \"tool_choice\": null\n",
- " }\n",
- " }\n",
- " }\n",
- "}\n"
- ]
- }
- ],
- "source": [
- "config2 = Config(\n",
- " prompt=PromptTemplate(\n",
- " messages=[\n",
- " Message(role=\"system\", content=\"You are an assistant that provides concise answers\"),\n",
- " Message(role=\"user\", content=\"Use Paul Graham style to explain {{topic}} in simple terms.\"),\n",
- " ],\n",
- " llm_config=ModelConfig(\n",
- " model=\"gpt-3.5-turbo\",\n",
- " max_tokens=150,\n",
- " temperature=0.9,\n",
- " top_p=1.0,\n",
- " frequency_penalty=0.0,\n",
- " presence_penalty=0.0,\n",
- " ),\n",
- " template_format=\"curly\"\n",
- " )\n",
- ")\n",
- "\n",
- "# Commit the new version\n",
- "variant = ag.VariantManager.commit(\n",
- " parameters=config2.model_dump(),\n",
- " app_slug=\"topic-explainer\",\n",
- " variant_slug=\"new-variant\"\n",
- ")\n",
- "\n",
- "print(\"Committed new version of variant:\")\n",
- "print(variant)\n"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- ":::info Immutability\n",
- "Each commit creates a new version of the variant. Versions are immutable once created.\n",
- ":::\n"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Fetching the prompt in production\n",
- "\n",
- "Now we'll fetch and use the prompt that's in production. Keep in mind that the production environment still references the first version of our variant. If we want it to reflect the latest changes, we'll need to deploy it again.\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 7,
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "Fetched configuration from production:\n",
- "{'prompt': {'messages': [{'role': 'system', 'content': 'You are an assistant that provides concise answers', 'name': None, 'tool_calls': None, 'tool_call_id': None}, {'role': 'user', 'content': 'Explain {{topic}} in simple terms', 'name': None, 'tool_calls': None, 'tool_call_id': None}], 'system_prompt': None, 'user_prompt': None, 'template_format': 'curly', 'input_keys': None, 'llm_config': {'model': 'gpt-3.5-turbo', 'temperature': 0.6, 'max_tokens': 150, 'top_p': 1.0, 'frequency_penalty': 0.0, 'presence_penalty': 0.0, 'response_format': None, 'stream': None, 'tools': None, 'tool_choice': None}}}\n"
- ]
- }
- ],
- "source": [
- "# Fetch configuration from the production environment\n",
- "config = ag.ConfigManager.get_from_registry(\n",
- " app_slug=\"topic-explainer\",\n",
- " environment_slug=\"production\"\n",
- ")\n",
- "\n",
- "print(\"Fetched configuration from production:\")\n",
- "print(config)\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 10,
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "Response for topic 'artificial intelligence':\n",
- "Artificial intelligence (AI) is the ability of a computer or machine to perform tasks that typically require human intelligence, such as learning, problem-solving, and decision-making. It involves creating algorithms and systems that can process information, recognize patterns, and make predictions without explicit programming.\n"
- ]
- }
- ],
- "source": [
- "# Using the configuration with OpenAI client\n",
- "from openai import OpenAI\n",
- "\n",
- "# Format the prompt with your topic\n",
- "topic = \"artificial intelligence\"\n",
- "prompt = PromptTemplate(**config[\"prompt\"]).format(topic=topic)\n",
- "client = OpenAI()\n",
- "\n",
- "response = client.chat.completions.create(\n",
- " **prompt.to_openai_kwargs()\n",
- ")\n",
- "\n",
- "print(f\"Response for topic '{topic}':\")\n",
- "print(response.choices[0].message.content)\n"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Next Steps\n",
- "\n",
- "Now that you've learned how to manage configurations using the SDK, you can:\n",
- "\n",
- "- Read the guide to explore more advanced features of the SDK\n",
- "- Read how to reference prompts in your traces\n",
- "- Read how to manage configuration for your workflows (chain of prompts, RAG..)\n",
- "\n",
- "## Summary\n",
- "\n",
- "In this tutorial, we've covered the complete workflow for managing prompts with the Agenta SDK:\n",
- "\n",
- "1. ✅ Created a new application\n",
- "2. ✅ Created a variant with initial configuration\n",
- "3. ✅ Deployed the variant to production environment\n",
- "4. ✅ Committed changes to create a new version\n",
- "5. ✅ Fetched configuration from production and used it with OpenAI\n",
- "\n",
- "This demonstrates the Git-like workflow that Agenta uses for prompt versioning, allowing you to manage different variants and deploy specific versions to different environments.\n"
- ]
- }
- ],
- "metadata": {
- "kernelspec": {
- "display_name": ".venv (3.12.3)",
- "language": "python",
- "name": "python3"
- },
- "language_info": {
- "codemirror_mode": {
- "name": "ipython",
- "version": 3
- },
- "file_extension": ".py",
- "mimetype": "text/x-python",
- "name": "python",
- "nbconvert_exporter": "python",
- "pygments_lexer": "ipython3",
- "version": "3.12.3"
- }
- },
- "nbformat": 4,
- "nbformat_minor": 2
-}
\ No newline at end of file
diff --git a/examples/node/observability-opentelemetry/.gitignore b/examples/node/observability-opentelemetry/.gitignore
deleted file mode 100644
index 2cd0f44931..0000000000
--- a/examples/node/observability-opentelemetry/.gitignore
+++ /dev/null
@@ -1,4 +0,0 @@
-node_modules/
-.env
-*.log
-
diff --git a/examples/node/observability-opentelemetry/README.md b/examples/node/observability-opentelemetry/README.md
deleted file mode 100644
index 9d61b4984f..0000000000
--- a/examples/node/observability-opentelemetry/README.md
+++ /dev/null
@@ -1,68 +0,0 @@
-# OpenTelemetry Quick Start Example
-
-This example demonstrates how to instrument a Node.js application with OpenTelemetry and send traces to Agenta.
-
-## Prerequisites
-
-- Node.js 18+ installed
-- Agenta API key ([get one here](https://cloud.agenta.ai))
-- OpenAI API key
-
-## Version Compatibility
-
-This example uses the latest stable versions:
-- **OpenAI SDK** (`latest` - v6.x)
-- **OpenInference Instrumentation** (`latest` - v3.x)
-- **OpenInference Semantic Conventions** (`latest` - v2.x)
-
-## Setup
-
-1. Install dependencies:
-```bash
-npm install
-```
-
-2. Create a `.env` file with your credentials:
-```bash
-cp .env.example .env
-# Edit .env and add your API keys
-```
-
-3. Run the example:
-```bash
-npm start
-```
-
-## What's Happening?
-
-1. **instrumentation.js** - Configures OpenTelemetry to:
- - Send traces to Agenta via OTLP
- - Automatically instrument OpenAI calls using OpenInference
- - Use SimpleSpanProcessor for immediate export (ideal for short scripts)
-
-2. **app.js** - A simple application that:
- - Creates a manual span using Agenta's semantic conventions
- - Calls OpenAI's chat completion API (auto-instrumented)
- - Demonstrates proper use of `ag.data.inputs`, `ag.data.outputs`, and `ag.data.internals`
-
-3. All traces are sent to Agenta where you can:
- - View the complete trace timeline
- - See inputs/outputs with proper formatting
- - Monitor costs and latency
- - Debug issues
-
-## Semantic Conventions
-
-This example follows Agenta's semantic conventions for proper trace display:
-
-- **`ag.type.node`** - Defines the operation type (workflow, task, tool, etc.)
-- **`ag.data.inputs`** - Stores input parameters as JSON
-- **`ag.data.outputs`** - Stores output results as JSON
-- **`ag.data.internals`** - Stores intermediate values and metadata
-
-See [SEMANTIC_CONVENTIONS.md](./SEMANTIC_CONVENTIONS.md) for detailed documentation.
-
-## View Your Traces
-
-After running the example, log in to [Agenta](https://cloud.agenta.ai) and navigate to the Observability section to see your traces!
-
diff --git a/examples/node/observability-opentelemetry/app.js b/examples/node/observability-opentelemetry/app.js
deleted file mode 100644
index b3dce7e794..0000000000
--- a/examples/node/observability-opentelemetry/app.js
+++ /dev/null
@@ -1,80 +0,0 @@
-// app.js
-import OpenAI from "openai";
-import { trace } from "@opentelemetry/api";
-
-const openai = new OpenAI({
- apiKey: process.env.OPENAI_API_KEY,
-});
-
-const tracer = trace.getTracer("test-app", "1.0.0");
-
-async function generate() {
- // Create a manual span using Agenta's semantic conventions
- // This demonstrates how to manually instrument functions with proper attributes
- return tracer.startActiveSpan("generate", async (span) => {
- try {
- // Define the messages for the chat completion
- const messages = [
- { role: "system", content: "You are a helpful assistant." },
- { role: "user", content: "Write a short story about AI Engineering." },
- ];
-
- // Agenta Semantic Convention: ag.type.node
- // Defines the type of operation (workflow, task, tool, etc.)
- span.setAttribute("ag.type.node", "workflow");
-
- // Agenta Semantic Convention: ag.data.inputs
- // Stores the input parameters as JSON
- span.setAttribute("ag.data.inputs", JSON.stringify({
- messages: messages,
- model: "gpt-3.5-turbo"
- }));
-
- const response = await openai.chat.completions.create({
- model: "gpt-3.5-turbo",
- messages: messages,
- });
-
- const content = response.choices[0].message.content;
-
- // Agenta Semantic Convention: ag.data.internals
- // Stores intermediate values and metadata (optional)
- span.setAttribute("ag.data.internals", JSON.stringify({
- response_length: content.length
- }));
-
- // Agenta Semantic Convention: ag.data.outputs
- // Stores the output results as JSON
- span.setAttribute("ag.data.outputs", JSON.stringify({
- content: content
- }));
-
- return content;
- } finally {
- span.end();
- }
- });
-}
-
-async function main() {
- try {
- const result = await generate();
- console.log("\n" + result);
-
- console.log("\n⏳ Flushing traces...");
- // Ensure traces are flushed before exit
- const tracerProvider = trace.getTracerProvider();
- if (tracerProvider && typeof tracerProvider.forceFlush === 'function') {
- await tracerProvider.forceFlush();
- }
- // Extra wait to ensure export completes
- await new Promise(resolve => setTimeout(resolve, 1000));
- console.log("✅ Done!");
- } catch (error) {
- console.error("❌ Error:", error.message);
- process.exit(1);
- }
-}
-
-main();
-
diff --git a/examples/node/observability-opentelemetry/instrumentation.js b/examples/node/observability-opentelemetry/instrumentation.js
deleted file mode 100644
index 4921de9bf7..0000000000
--- a/examples/node/observability-opentelemetry/instrumentation.js
+++ /dev/null
@@ -1,74 +0,0 @@
-// instrumentation.js
-import { registerInstrumentations } from "@opentelemetry/instrumentation";
-import { OpenAIInstrumentation } from "@arizeai/openinference-instrumentation-openai";
-import { diag, DiagConsoleLogger, DiagLogLevel } from "@opentelemetry/api";
-import { OTLPTraceExporter } from "@opentelemetry/exporter-trace-otlp-proto";
-import { Resource } from "@opentelemetry/resources";
-import { BatchSpanProcessor, SimpleSpanProcessor } from "@opentelemetry/sdk-trace-base";
-import { NodeTracerProvider } from "@opentelemetry/sdk-trace-node";
-import { ATTR_SERVICE_NAME } from "@opentelemetry/semantic-conventions";
-import { SEMRESATTRS_PROJECT_NAME } from "@arizeai/openinference-semantic-conventions";
-import OpenAI from "openai";
-
-// For troubleshooting, set the log level to DiagLogLevel.DEBUG
-diag.setLogger(new DiagConsoleLogger(), DiagLogLevel.INFO);
-
-// Get Agenta configuration from environment variables
-const AGENTA_HOST = process.env.AGENTA_HOST || "https://cloud.agenta.ai";
-const AGENTA_API_KEY = process.env.AGENTA_API_KEY;
-
-if (!AGENTA_API_KEY) {
- console.error("❌ AGENTA_API_KEY environment variable is required");
- process.exit(1);
-}
-
-// Configure the OTLP exporter to send traces to Agenta
-const otlpExporter = new OTLPTraceExporter({
- url: `${AGENTA_HOST}/api/otlp/v1/traces`,
- headers: {
- Authorization: `ApiKey ${AGENTA_API_KEY}`,
- },
- timeoutMillis: 5000, // 5 second timeout
-});
-
-// Add logging to the exporter for debugging
-const originalExport = otlpExporter.export.bind(otlpExporter);
-otlpExporter.export = function (spans, resultCallback) {
- console.log(`📤 Exporting ${spans.length} span(s)...`);
- originalExport(spans, (result) => {
- if (result.code === 0) {
- console.log('✅ Spans exported successfully');
- } else {
- console.error('❌ Export failed:', result.error);
- }
- resultCallback(result);
- });
-};
-
-// Create and configure the tracer provider
-const tracerProvider = new NodeTracerProvider({
- resource: new Resource({
- [ATTR_SERVICE_NAME]: "openai-quickstart",
- // Project name in Agenta, defaults to "default"
- [SEMRESATTRS_PROJECT_NAME]: "openai-quickstart",
- }),
-});
-
-// Use SimpleSpanProcessor for immediate export (better for short-lived scripts)
-// For long-running services, use: new BatchSpanProcessor(otlpExporter)
-tracerProvider.addSpanProcessor(new SimpleSpanProcessor(otlpExporter));
-
-// Register the tracer provider
-tracerProvider.register();
-
-// Register OpenAI instrumentation with manual instrumentation
-// This is required for OpenInference to properly instrument the OpenAI client
-const instrumentation = new OpenAIInstrumentation();
-instrumentation.manuallyInstrument(OpenAI);
-
-registerInstrumentations({
- instrumentations: [instrumentation],
-});
-
-console.log("✅ OpenTelemetry instrumentation initialized");
-
diff --git a/examples/node/observability-opentelemetry/package-lock.json b/examples/node/observability-opentelemetry/package-lock.json
deleted file mode 100644
index c9c0ef0ae5..0000000000
--- a/examples/node/observability-opentelemetry/package-lock.json
+++ /dev/null
@@ -1,935 +0,0 @@
-{
- "name": "agenta-opentelemetry-quickstart",
- "version": "1.0.0",
- "lockfileVersion": 3,
- "requires": true,
- "packages": {
- "": {
- "name": "agenta-opentelemetry-quickstart",
- "version": "1.0.0",
- "dependencies": {
- "@arizeai/openinference-instrumentation-openai": "^3.2.3",
- "@arizeai/openinference-semantic-conventions": "^2.1.2",
- "@opentelemetry/api": "^1.9.0",
- "@opentelemetry/exporter-trace-otlp-proto": "^0.54.0",
- "@opentelemetry/instrumentation": "^0.54.0",
- "@opentelemetry/resources": "^1.28.0",
- "@opentelemetry/sdk-trace-base": "^1.28.0",
- "@opentelemetry/sdk-trace-node": "^1.28.0",
- "@opentelemetry/semantic-conventions": "^1.28.0",
- "openai": "^6.7.0"
- },
- "devDependencies": {
- "@types/node": "^22.0.0",
- "typescript": "^5.7.0"
- }
- },
- "node_modules/@arizeai/openinference-core": {
- "version": "1.0.7",
- "resolved": "https://registry.npmjs.org/@arizeai/openinference-core/-/openinference-core-1.0.7.tgz",
- "integrity": "sha512-O9WYkrHNh/0mGTV+T9SWC3tkxVrT16gBrFiByG3aukBsqdOfSzoRj6QINk+Oi+VEDNIoQUzVQPFh81/gEL/thA==",
- "license": "Apache-2.0",
- "dependencies": {
- "@arizeai/openinference-semantic-conventions": "2.1.2",
- "@opentelemetry/api": "^1.9.0",
- "@opentelemetry/core": "^1.25.1"
- }
- },
- "node_modules/@arizeai/openinference-instrumentation-openai": {
- "version": "3.2.3",
- "resolved": "https://registry.npmjs.org/@arizeai/openinference-instrumentation-openai/-/openinference-instrumentation-openai-3.2.3.tgz",
- "integrity": "sha512-4GiUyUIafNkAf7milnsS4Xjh0rZwjPvhHu+xkcj2MQ19a0Teu6PcFDa7DCt7YV/xw2o+wcxeaLfVURmGGLAHOg==",
- "license": "Apache-2.0",
- "dependencies": {
- "@arizeai/openinference-core": "1.0.7",
- "@arizeai/openinference-semantic-conventions": "2.1.2",
- "@opentelemetry/api": "^1.9.0",
- "@opentelemetry/core": "^1.25.1",
- "@opentelemetry/instrumentation": "^0.46.0"
- }
- },
- "node_modules/@arizeai/openinference-instrumentation-openai/node_modules/@opentelemetry/instrumentation": {
- "version": "0.46.0",
- "resolved": "https://registry.npmjs.org/@opentelemetry/instrumentation/-/instrumentation-0.46.0.tgz",
- "integrity": "sha512-a9TijXZZbk0vI5TGLZl+0kxyFfrXHhX6Svtz7Pp2/VBlCSKrazuULEyoJQrOknJyFWNMEmbbJgOciHCCpQcisw==",
- "license": "Apache-2.0",
- "dependencies": {
- "@types/shimmer": "^1.0.2",
- "import-in-the-middle": "1.7.1",
- "require-in-the-middle": "^7.1.1",
- "semver": "^7.5.2",
- "shimmer": "^1.2.1"
- },
- "engines": {
- "node": ">=14"
- },
- "peerDependencies": {
- "@opentelemetry/api": "^1.3.0"
- }
- },
- "node_modules/@arizeai/openinference-instrumentation-openai/node_modules/import-in-the-middle": {
- "version": "1.7.1",
- "resolved": "https://registry.npmjs.org/import-in-the-middle/-/import-in-the-middle-1.7.1.tgz",
- "integrity": "sha512-1LrZPDtW+atAxH42S6288qyDFNQ2YCty+2mxEPRtfazH6Z5QwkaBSTS2ods7hnVJioF6rkRfNoA6A/MstpFXLg==",
- "license": "Apache-2.0",
- "dependencies": {
- "acorn": "^8.8.2",
- "acorn-import-assertions": "^1.9.0",
- "cjs-module-lexer": "^1.2.2",
- "module-details-from-path": "^1.0.3"
- }
- },
- "node_modules/@arizeai/openinference-semantic-conventions": {
- "version": "2.1.2",
- "resolved": "https://registry.npmjs.org/@arizeai/openinference-semantic-conventions/-/openinference-semantic-conventions-2.1.2.tgz",
- "integrity": "sha512-u7UeuU9bJ1LxzHk0MPWb+1ZcotCcJwPnKDXi7Rl2cPs1pWMFg9Ogq7zzYZX+sDcibD2AEa1U+ElyOD8DwZc9gw==",
- "license": "Apache-2.0"
- },
- "node_modules/@opentelemetry/api": {
- "version": "1.9.0",
- "resolved": "https://registry.npmjs.org/@opentelemetry/api/-/api-1.9.0.tgz",
- "integrity": "sha512-3giAOQvZiH5F9bMlMiv8+GSPMeqg0dbaeo58/0SlA9sxSqZhnUtxzX9/2FzyhS9sWQf5S0GJE0AKBrFqjpeYcg==",
- "license": "Apache-2.0",
- "engines": {
- "node": ">=8.0.0"
- }
- },
- "node_modules/@opentelemetry/api-logs": {
- "version": "0.54.2",
- "resolved": "https://registry.npmjs.org/@opentelemetry/api-logs/-/api-logs-0.54.2.tgz",
- "integrity": "sha512-4MTVwwmLgUh5QrJnZpYo6YRO5IBLAggf2h8gWDblwRagDStY13aEvt7gGk3jewrMaPlHiF83fENhIx0HO97/cQ==",
- "license": "Apache-2.0",
- "dependencies": {
- "@opentelemetry/api": "^1.3.0"
- },
- "engines": {
- "node": ">=14"
- }
- },
- "node_modules/@opentelemetry/context-async-hooks": {
- "version": "1.30.1",
- "resolved": "https://registry.npmjs.org/@opentelemetry/context-async-hooks/-/context-async-hooks-1.30.1.tgz",
- "integrity": "sha512-s5vvxXPVdjqS3kTLKMeBMvop9hbWkwzBpu+mUO2M7sZtlkyDJGwFe33wRKnbaYDo8ExRVBIIdwIGrqpxHuKttA==",
- "license": "Apache-2.0",
- "engines": {
- "node": ">=14"
- },
- "peerDependencies": {
- "@opentelemetry/api": ">=1.0.0 <1.10.0"
- }
- },
- "node_modules/@opentelemetry/core": {
- "version": "1.27.0",
- "resolved": "https://registry.npmjs.org/@opentelemetry/core/-/core-1.27.0.tgz",
- "integrity": "sha512-yQPKnK5e+76XuiqUH/gKyS8wv/7qITd5ln56QkBTf3uggr0VkXOXfcaAuG330UfdYu83wsyoBwqwxigpIG+Jkg==",
- "license": "Apache-2.0",
- "dependencies": {
- "@opentelemetry/semantic-conventions": "1.27.0"
- },
- "engines": {
- "node": ">=14"
- },
- "peerDependencies": {
- "@opentelemetry/api": ">=1.0.0 <1.10.0"
- }
- },
- "node_modules/@opentelemetry/core/node_modules/@opentelemetry/semantic-conventions": {
- "version": "1.27.0",
- "resolved": "https://registry.npmjs.org/@opentelemetry/semantic-conventions/-/semantic-conventions-1.27.0.tgz",
- "integrity": "sha512-sAay1RrB+ONOem0OZanAR1ZI/k7yDpnOQSQmTMuGImUQb2y8EbSaCJ94FQluM74xoU03vlb2d2U90hZluL6nQg==",
- "license": "Apache-2.0",
- "engines": {
- "node": ">=14"
- }
- },
- "node_modules/@opentelemetry/exporter-trace-otlp-proto": {
- "version": "0.54.2",
- "resolved": "https://registry.npmjs.org/@opentelemetry/exporter-trace-otlp-proto/-/exporter-trace-otlp-proto-0.54.2.tgz",
- "integrity": "sha512-XSmm1N2wAhoWDXP1q/N6kpLebWaxl6VIADv4WA5QWKHLRpF3gLz5NAWNJBR8ygsvv8jQcrwnXgwfnJ18H3v1fg==",
- "license": "Apache-2.0",
- "dependencies": {
- "@opentelemetry/core": "1.27.0",
- "@opentelemetry/otlp-exporter-base": "0.54.2",
- "@opentelemetry/otlp-transformer": "0.54.2",
- "@opentelemetry/resources": "1.27.0",
- "@opentelemetry/sdk-trace-base": "1.27.0"
- },
- "engines": {
- "node": ">=14"
- },
- "peerDependencies": {
- "@opentelemetry/api": "^1.3.0"
- }
- },
- "node_modules/@opentelemetry/exporter-trace-otlp-proto/node_modules/@opentelemetry/resources": {
- "version": "1.27.0",
- "resolved": "https://registry.npmjs.org/@opentelemetry/resources/-/resources-1.27.0.tgz",
- "integrity": "sha512-jOwt2VJ/lUD5BLc+PMNymDrUCpm5PKi1E9oSVYAvz01U/VdndGmrtV3DU1pG4AwlYhJRHbHfOUIlpBeXCPw6QQ==",
- "license": "Apache-2.0",
- "dependencies": {
- "@opentelemetry/core": "1.27.0",
- "@opentelemetry/semantic-conventions": "1.27.0"
- },
- "engines": {
- "node": ">=14"
- },
- "peerDependencies": {
- "@opentelemetry/api": ">=1.0.0 <1.10.0"
- }
- },
- "node_modules/@opentelemetry/exporter-trace-otlp-proto/node_modules/@opentelemetry/sdk-trace-base": {
- "version": "1.27.0",
- "resolved": "https://registry.npmjs.org/@opentelemetry/sdk-trace-base/-/sdk-trace-base-1.27.0.tgz",
- "integrity": "sha512-btz6XTQzwsyJjombpeqCX6LhiMQYpzt2pIYNPnw0IPO/3AhT6yjnf8Mnv3ZC2A4eRYOjqrg+bfaXg9XHDRJDWQ==",
- "license": "Apache-2.0",
- "dependencies": {
- "@opentelemetry/core": "1.27.0",
- "@opentelemetry/resources": "1.27.0",
- "@opentelemetry/semantic-conventions": "1.27.0"
- },
- "engines": {
- "node": ">=14"
- },
- "peerDependencies": {
- "@opentelemetry/api": ">=1.0.0 <1.10.0"
- }
- },
- "node_modules/@opentelemetry/exporter-trace-otlp-proto/node_modules/@opentelemetry/semantic-conventions": {
- "version": "1.27.0",
- "resolved": "https://registry.npmjs.org/@opentelemetry/semantic-conventions/-/semantic-conventions-1.27.0.tgz",
- "integrity": "sha512-sAay1RrB+ONOem0OZanAR1ZI/k7yDpnOQSQmTMuGImUQb2y8EbSaCJ94FQluM74xoU03vlb2d2U90hZluL6nQg==",
- "license": "Apache-2.0",
- "engines": {
- "node": ">=14"
- }
- },
- "node_modules/@opentelemetry/instrumentation": {
- "version": "0.54.2",
- "resolved": "https://registry.npmjs.org/@opentelemetry/instrumentation/-/instrumentation-0.54.2.tgz",
- "integrity": "sha512-go6zpOVoZVztT9r1aPd79Fr3OWiD4N24bCPJsIKkBses8oyFo12F/Ew3UBTdIu6hsW4HC4MVEJygG6TEyJI/lg==",
- "license": "Apache-2.0",
- "dependencies": {
- "@opentelemetry/api-logs": "0.54.2",
- "@types/shimmer": "^1.2.0",
- "import-in-the-middle": "^1.8.1",
- "require-in-the-middle": "^7.1.1",
- "semver": "^7.5.2",
- "shimmer": "^1.2.1"
- },
- "engines": {
- "node": ">=14"
- },
- "peerDependencies": {
- "@opentelemetry/api": "^1.3.0"
- }
- },
- "node_modules/@opentelemetry/otlp-exporter-base": {
- "version": "0.54.2",
- "resolved": "https://registry.npmjs.org/@opentelemetry/otlp-exporter-base/-/otlp-exporter-base-0.54.2.tgz",
- "integrity": "sha512-NrNyxu6R/bGAwanhz1HI0aJWKR6xUED4TjCH4iWMlAfyRukGbI9Kt/Akd2sYLwRKNhfS+sKetKGCUQPMDyYYMA==",
- "license": "Apache-2.0",
- "dependencies": {
- "@opentelemetry/core": "1.27.0",
- "@opentelemetry/otlp-transformer": "0.54.2"
- },
- "engines": {
- "node": ">=14"
- },
- "peerDependencies": {
- "@opentelemetry/api": "^1.3.0"
- }
- },
- "node_modules/@opentelemetry/otlp-transformer": {
- "version": "0.54.2",
- "resolved": "https://registry.npmjs.org/@opentelemetry/otlp-transformer/-/otlp-transformer-0.54.2.tgz",
- "integrity": "sha512-2tIjahJlMRRUz0A2SeE+qBkeBXBFkSjR0wqJ08kuOqaL8HNGan5iZf+A8cfrfmZzPUuMKCyY9I+okzFuFs6gKQ==",
- "license": "Apache-2.0",
- "dependencies": {
- "@opentelemetry/api-logs": "0.54.2",
- "@opentelemetry/core": "1.27.0",
- "@opentelemetry/resources": "1.27.0",
- "@opentelemetry/sdk-logs": "0.54.2",
- "@opentelemetry/sdk-metrics": "1.27.0",
- "@opentelemetry/sdk-trace-base": "1.27.0",
- "protobufjs": "^7.3.0"
- },
- "engines": {
- "node": ">=14"
- },
- "peerDependencies": {
- "@opentelemetry/api": "^1.3.0"
- }
- },
- "node_modules/@opentelemetry/otlp-transformer/node_modules/@opentelemetry/resources": {
- "version": "1.27.0",
- "resolved": "https://registry.npmjs.org/@opentelemetry/resources/-/resources-1.27.0.tgz",
- "integrity": "sha512-jOwt2VJ/lUD5BLc+PMNymDrUCpm5PKi1E9oSVYAvz01U/VdndGmrtV3DU1pG4AwlYhJRHbHfOUIlpBeXCPw6QQ==",
- "license": "Apache-2.0",
- "dependencies": {
- "@opentelemetry/core": "1.27.0",
- "@opentelemetry/semantic-conventions": "1.27.0"
- },
- "engines": {
- "node": ">=14"
- },
- "peerDependencies": {
- "@opentelemetry/api": ">=1.0.0 <1.10.0"
- }
- },
- "node_modules/@opentelemetry/otlp-transformer/node_modules/@opentelemetry/sdk-trace-base": {
- "version": "1.27.0",
- "resolved": "https://registry.npmjs.org/@opentelemetry/sdk-trace-base/-/sdk-trace-base-1.27.0.tgz",
- "integrity": "sha512-btz6XTQzwsyJjombpeqCX6LhiMQYpzt2pIYNPnw0IPO/3AhT6yjnf8Mnv3ZC2A4eRYOjqrg+bfaXg9XHDRJDWQ==",
- "license": "Apache-2.0",
- "dependencies": {
- "@opentelemetry/core": "1.27.0",
- "@opentelemetry/resources": "1.27.0",
- "@opentelemetry/semantic-conventions": "1.27.0"
- },
- "engines": {
- "node": ">=14"
- },
- "peerDependencies": {
- "@opentelemetry/api": ">=1.0.0 <1.10.0"
- }
- },
- "node_modules/@opentelemetry/otlp-transformer/node_modules/@opentelemetry/semantic-conventions": {
- "version": "1.27.0",
- "resolved": "https://registry.npmjs.org/@opentelemetry/semantic-conventions/-/semantic-conventions-1.27.0.tgz",
- "integrity": "sha512-sAay1RrB+ONOem0OZanAR1ZI/k7yDpnOQSQmTMuGImUQb2y8EbSaCJ94FQluM74xoU03vlb2d2U90hZluL6nQg==",
- "license": "Apache-2.0",
- "engines": {
- "node": ">=14"
- }
- },
- "node_modules/@opentelemetry/propagator-b3": {
- "version": "1.30.1",
- "resolved": "https://registry.npmjs.org/@opentelemetry/propagator-b3/-/propagator-b3-1.30.1.tgz",
- "integrity": "sha512-oATwWWDIJzybAZ4pO76ATN5N6FFbOA1otibAVlS8v90B4S1wClnhRUk7K+2CHAwN1JKYuj4jh/lpCEG5BAqFuQ==",
- "license": "Apache-2.0",
- "dependencies": {
- "@opentelemetry/core": "1.30.1"
- },
- "engines": {
- "node": ">=14"
- },
- "peerDependencies": {
- "@opentelemetry/api": ">=1.0.0 <1.10.0"
- }
- },
- "node_modules/@opentelemetry/propagator-b3/node_modules/@opentelemetry/core": {
- "version": "1.30.1",
- "resolved": "https://registry.npmjs.org/@opentelemetry/core/-/core-1.30.1.tgz",
- "integrity": "sha512-OOCM2C/QIURhJMuKaekP3TRBxBKxG/TWWA0TL2J6nXUtDnuCtccy49LUJF8xPFXMX+0LMcxFpCo8M9cGY1W6rQ==",
- "license": "Apache-2.0",
- "dependencies": {
- "@opentelemetry/semantic-conventions": "1.28.0"
- },
- "engines": {
- "node": ">=14"
- },
- "peerDependencies": {
- "@opentelemetry/api": ">=1.0.0 <1.10.0"
- }
- },
- "node_modules/@opentelemetry/propagator-b3/node_modules/@opentelemetry/semantic-conventions": {
- "version": "1.28.0",
- "resolved": "https://registry.npmjs.org/@opentelemetry/semantic-conventions/-/semantic-conventions-1.28.0.tgz",
- "integrity": "sha512-lp4qAiMTD4sNWW4DbKLBkfiMZ4jbAboJIGOQr5DvciMRI494OapieI9qiODpOt0XBr1LjIDy1xAGAnVs5supTA==",
- "license": "Apache-2.0",
- "engines": {
- "node": ">=14"
- }
- },
- "node_modules/@opentelemetry/propagator-jaeger": {
- "version": "1.30.1",
- "resolved": "https://registry.npmjs.org/@opentelemetry/propagator-jaeger/-/propagator-jaeger-1.30.1.tgz",
- "integrity": "sha512-Pj/BfnYEKIOImirH76M4hDaBSx6HyZ2CXUqk+Kj02m6BB80c/yo4BdWkn/1gDFfU+YPY+bPR2U0DKBfdxCKwmg==",
- "license": "Apache-2.0",
- "dependencies": {
- "@opentelemetry/core": "1.30.1"
- },
- "engines": {
- "node": ">=14"
- },
- "peerDependencies": {
- "@opentelemetry/api": ">=1.0.0 <1.10.0"
- }
- },
- "node_modules/@opentelemetry/propagator-jaeger/node_modules/@opentelemetry/core": {
- "version": "1.30.1",
- "resolved": "https://registry.npmjs.org/@opentelemetry/core/-/core-1.30.1.tgz",
- "integrity": "sha512-OOCM2C/QIURhJMuKaekP3TRBxBKxG/TWWA0TL2J6nXUtDnuCtccy49LUJF8xPFXMX+0LMcxFpCo8M9cGY1W6rQ==",
- "license": "Apache-2.0",
- "dependencies": {
- "@opentelemetry/semantic-conventions": "1.28.0"
- },
- "engines": {
- "node": ">=14"
- },
- "peerDependencies": {
- "@opentelemetry/api": ">=1.0.0 <1.10.0"
- }
- },
- "node_modules/@opentelemetry/propagator-jaeger/node_modules/@opentelemetry/semantic-conventions": {
- "version": "1.28.0",
- "resolved": "https://registry.npmjs.org/@opentelemetry/semantic-conventions/-/semantic-conventions-1.28.0.tgz",
- "integrity": "sha512-lp4qAiMTD4sNWW4DbKLBkfiMZ4jbAboJIGOQr5DvciMRI494OapieI9qiODpOt0XBr1LjIDy1xAGAnVs5supTA==",
- "license": "Apache-2.0",
- "engines": {
- "node": ">=14"
- }
- },
- "node_modules/@opentelemetry/resources": {
- "version": "1.30.1",
- "resolved": "https://registry.npmjs.org/@opentelemetry/resources/-/resources-1.30.1.tgz",
- "integrity": "sha512-5UxZqiAgLYGFjS4s9qm5mBVo433u+dSPUFWVWXmLAD4wB65oMCoXaJP1KJa9DIYYMeHu3z4BZcStG3LC593cWA==",
- "license": "Apache-2.0",
- "dependencies": {
- "@opentelemetry/core": "1.30.1",
- "@opentelemetry/semantic-conventions": "1.28.0"
- },
- "engines": {
- "node": ">=14"
- },
- "peerDependencies": {
- "@opentelemetry/api": ">=1.0.0 <1.10.0"
- }
- },
- "node_modules/@opentelemetry/resources/node_modules/@opentelemetry/core": {
- "version": "1.30.1",
- "resolved": "https://registry.npmjs.org/@opentelemetry/core/-/core-1.30.1.tgz",
- "integrity": "sha512-OOCM2C/QIURhJMuKaekP3TRBxBKxG/TWWA0TL2J6nXUtDnuCtccy49LUJF8xPFXMX+0LMcxFpCo8M9cGY1W6rQ==",
- "license": "Apache-2.0",
- "dependencies": {
- "@opentelemetry/semantic-conventions": "1.28.0"
- },
- "engines": {
- "node": ">=14"
- },
- "peerDependencies": {
- "@opentelemetry/api": ">=1.0.0 <1.10.0"
- }
- },
- "node_modules/@opentelemetry/resources/node_modules/@opentelemetry/semantic-conventions": {
- "version": "1.28.0",
- "resolved": "https://registry.npmjs.org/@opentelemetry/semantic-conventions/-/semantic-conventions-1.28.0.tgz",
- "integrity": "sha512-lp4qAiMTD4sNWW4DbKLBkfiMZ4jbAboJIGOQr5DvciMRI494OapieI9qiODpOt0XBr1LjIDy1xAGAnVs5supTA==",
- "license": "Apache-2.0",
- "engines": {
- "node": ">=14"
- }
- },
- "node_modules/@opentelemetry/sdk-logs": {
- "version": "0.54.2",
- "resolved": "https://registry.npmjs.org/@opentelemetry/sdk-logs/-/sdk-logs-0.54.2.tgz",
- "integrity": "sha512-yIbYqDLS/AtBbPjCjh6eSToGNRMqW2VR8RrKEy+G+J7dFG7pKoptTH5T+XlKPleP9NY8JZYIpgJBlI+Osi0rFw==",
- "license": "Apache-2.0",
- "dependencies": {
- "@opentelemetry/api-logs": "0.54.2",
- "@opentelemetry/core": "1.27.0",
- "@opentelemetry/resources": "1.27.0"
- },
- "engines": {
- "node": ">=14"
- },
- "peerDependencies": {
- "@opentelemetry/api": ">=1.4.0 <1.10.0"
- }
- },
- "node_modules/@opentelemetry/sdk-logs/node_modules/@opentelemetry/resources": {
- "version": "1.27.0",
- "resolved": "https://registry.npmjs.org/@opentelemetry/resources/-/resources-1.27.0.tgz",
- "integrity": "sha512-jOwt2VJ/lUD5BLc+PMNymDrUCpm5PKi1E9oSVYAvz01U/VdndGmrtV3DU1pG4AwlYhJRHbHfOUIlpBeXCPw6QQ==",
- "license": "Apache-2.0",
- "dependencies": {
- "@opentelemetry/core": "1.27.0",
- "@opentelemetry/semantic-conventions": "1.27.0"
- },
- "engines": {
- "node": ">=14"
- },
- "peerDependencies": {
- "@opentelemetry/api": ">=1.0.0 <1.10.0"
- }
- },
- "node_modules/@opentelemetry/sdk-logs/node_modules/@opentelemetry/semantic-conventions": {
- "version": "1.27.0",
- "resolved": "https://registry.npmjs.org/@opentelemetry/semantic-conventions/-/semantic-conventions-1.27.0.tgz",
- "integrity": "sha512-sAay1RrB+ONOem0OZanAR1ZI/k7yDpnOQSQmTMuGImUQb2y8EbSaCJ94FQluM74xoU03vlb2d2U90hZluL6nQg==",
- "license": "Apache-2.0",
- "engines": {
- "node": ">=14"
- }
- },
- "node_modules/@opentelemetry/sdk-metrics": {
- "version": "1.27.0",
- "resolved": "https://registry.npmjs.org/@opentelemetry/sdk-metrics/-/sdk-metrics-1.27.0.tgz",
- "integrity": "sha512-JzWgzlutoXCydhHWIbLg+r76m+m3ncqvkCcsswXAQ4gqKS+LOHKhq+t6fx1zNytvLuaOUBur7EvWxECc4jPQKg==",
- "license": "Apache-2.0",
- "dependencies": {
- "@opentelemetry/core": "1.27.0",
- "@opentelemetry/resources": "1.27.0"
- },
- "engines": {
- "node": ">=14"
- },
- "peerDependencies": {
- "@opentelemetry/api": ">=1.3.0 <1.10.0"
- }
- },
- "node_modules/@opentelemetry/sdk-metrics/node_modules/@opentelemetry/resources": {
- "version": "1.27.0",
- "resolved": "https://registry.npmjs.org/@opentelemetry/resources/-/resources-1.27.0.tgz",
- "integrity": "sha512-jOwt2VJ/lUD5BLc+PMNymDrUCpm5PKi1E9oSVYAvz01U/VdndGmrtV3DU1pG4AwlYhJRHbHfOUIlpBeXCPw6QQ==",
- "license": "Apache-2.0",
- "dependencies": {
- "@opentelemetry/core": "1.27.0",
- "@opentelemetry/semantic-conventions": "1.27.0"
- },
- "engines": {
- "node": ">=14"
- },
- "peerDependencies": {
- "@opentelemetry/api": ">=1.0.0 <1.10.0"
- }
- },
- "node_modules/@opentelemetry/sdk-metrics/node_modules/@opentelemetry/semantic-conventions": {
- "version": "1.27.0",
- "resolved": "https://registry.npmjs.org/@opentelemetry/semantic-conventions/-/semantic-conventions-1.27.0.tgz",
- "integrity": "sha512-sAay1RrB+ONOem0OZanAR1ZI/k7yDpnOQSQmTMuGImUQb2y8EbSaCJ94FQluM74xoU03vlb2d2U90hZluL6nQg==",
- "license": "Apache-2.0",
- "engines": {
- "node": ">=14"
- }
- },
- "node_modules/@opentelemetry/sdk-trace-base": {
- "version": "1.30.1",
- "resolved": "https://registry.npmjs.org/@opentelemetry/sdk-trace-base/-/sdk-trace-base-1.30.1.tgz",
- "integrity": "sha512-jVPgBbH1gCy2Lb7X0AVQ8XAfgg0pJ4nvl8/IiQA6nxOsPvS+0zMJaFSs2ltXe0J6C8dqjcnpyqINDJmU30+uOg==",
- "license": "Apache-2.0",
- "dependencies": {
- "@opentelemetry/core": "1.30.1",
- "@opentelemetry/resources": "1.30.1",
- "@opentelemetry/semantic-conventions": "1.28.0"
- },
- "engines": {
- "node": ">=14"
- },
- "peerDependencies": {
- "@opentelemetry/api": ">=1.0.0 <1.10.0"
- }
- },
- "node_modules/@opentelemetry/sdk-trace-base/node_modules/@opentelemetry/core": {
- "version": "1.30.1",
- "resolved": "https://registry.npmjs.org/@opentelemetry/core/-/core-1.30.1.tgz",
- "integrity": "sha512-OOCM2C/QIURhJMuKaekP3TRBxBKxG/TWWA0TL2J6nXUtDnuCtccy49LUJF8xPFXMX+0LMcxFpCo8M9cGY1W6rQ==",
- "license": "Apache-2.0",
- "dependencies": {
- "@opentelemetry/semantic-conventions": "1.28.0"
- },
- "engines": {
- "node": ">=14"
- },
- "peerDependencies": {
- "@opentelemetry/api": ">=1.0.0 <1.10.0"
- }
- },
- "node_modules/@opentelemetry/sdk-trace-base/node_modules/@opentelemetry/semantic-conventions": {
- "version": "1.28.0",
- "resolved": "https://registry.npmjs.org/@opentelemetry/semantic-conventions/-/semantic-conventions-1.28.0.tgz",
- "integrity": "sha512-lp4qAiMTD4sNWW4DbKLBkfiMZ4jbAboJIGOQr5DvciMRI494OapieI9qiODpOt0XBr1LjIDy1xAGAnVs5supTA==",
- "license": "Apache-2.0",
- "engines": {
- "node": ">=14"
- }
- },
- "node_modules/@opentelemetry/sdk-trace-node": {
- "version": "1.30.1",
- "resolved": "https://registry.npmjs.org/@opentelemetry/sdk-trace-node/-/sdk-trace-node-1.30.1.tgz",
- "integrity": "sha512-cBjYOINt1JxXdpw1e5MlHmFRc5fgj4GW/86vsKFxJCJ8AL4PdVtYH41gWwl4qd4uQjqEL1oJVrXkSy5cnduAnQ==",
- "license": "Apache-2.0",
- "dependencies": {
- "@opentelemetry/context-async-hooks": "1.30.1",
- "@opentelemetry/core": "1.30.1",
- "@opentelemetry/propagator-b3": "1.30.1",
- "@opentelemetry/propagator-jaeger": "1.30.1",
- "@opentelemetry/sdk-trace-base": "1.30.1",
- "semver": "^7.5.2"
- },
- "engines": {
- "node": ">=14"
- },
- "peerDependencies": {
- "@opentelemetry/api": ">=1.0.0 <1.10.0"
- }
- },
- "node_modules/@opentelemetry/sdk-trace-node/node_modules/@opentelemetry/core": {
- "version": "1.30.1",
- "resolved": "https://registry.npmjs.org/@opentelemetry/core/-/core-1.30.1.tgz",
- "integrity": "sha512-OOCM2C/QIURhJMuKaekP3TRBxBKxG/TWWA0TL2J6nXUtDnuCtccy49LUJF8xPFXMX+0LMcxFpCo8M9cGY1W6rQ==",
- "license": "Apache-2.0",
- "dependencies": {
- "@opentelemetry/semantic-conventions": "1.28.0"
- },
- "engines": {
- "node": ">=14"
- },
- "peerDependencies": {
- "@opentelemetry/api": ">=1.0.0 <1.10.0"
- }
- },
- "node_modules/@opentelemetry/sdk-trace-node/node_modules/@opentelemetry/semantic-conventions": {
- "version": "1.28.0",
- "resolved": "https://registry.npmjs.org/@opentelemetry/semantic-conventions/-/semantic-conventions-1.28.0.tgz",
- "integrity": "sha512-lp4qAiMTD4sNWW4DbKLBkfiMZ4jbAboJIGOQr5DvciMRI494OapieI9qiODpOt0XBr1LjIDy1xAGAnVs5supTA==",
- "license": "Apache-2.0",
- "engines": {
- "node": ">=14"
- }
- },
- "node_modules/@opentelemetry/semantic-conventions": {
- "version": "1.37.0",
- "resolved": "https://registry.npmjs.org/@opentelemetry/semantic-conventions/-/semantic-conventions-1.37.0.tgz",
- "integrity": "sha512-JD6DerIKdJGmRp4jQyX5FlrQjA4tjOw1cvfsPAZXfOOEErMUHjPcPSICS+6WnM0nB0efSFARh0KAZss+bvExOA==",
- "license": "Apache-2.0",
- "engines": {
- "node": ">=14"
- }
- },
- "node_modules/@protobufjs/aspromise": {
- "version": "1.1.2",
- "resolved": "https://registry.npmjs.org/@protobufjs/aspromise/-/aspromise-1.1.2.tgz",
- "integrity": "sha512-j+gKExEuLmKwvz3OgROXtrJ2UG2x8Ch2YZUxahh+s1F2HZ+wAceUNLkvy6zKCPVRkU++ZWQrdxsUeQXmcg4uoQ==",
- "license": "BSD-3-Clause"
- },
- "node_modules/@protobufjs/base64": {
- "version": "1.1.2",
- "resolved": "https://registry.npmjs.org/@protobufjs/base64/-/base64-1.1.2.tgz",
- "integrity": "sha512-AZkcAA5vnN/v4PDqKyMR5lx7hZttPDgClv83E//FMNhR2TMcLUhfRUBHCmSl0oi9zMgDDqRUJkSxO3wm85+XLg==",
- "license": "BSD-3-Clause"
- },
- "node_modules/@protobufjs/codegen": {
- "version": "2.0.4",
- "resolved": "https://registry.npmjs.org/@protobufjs/codegen/-/codegen-2.0.4.tgz",
- "integrity": "sha512-YyFaikqM5sH0ziFZCN3xDC7zeGaB/d0IUb9CATugHWbd1FRFwWwt4ld4OYMPWu5a3Xe01mGAULCdqhMlPl29Jg==",
- "license": "BSD-3-Clause"
- },
- "node_modules/@protobufjs/eventemitter": {
- "version": "1.1.0",
- "resolved": "https://registry.npmjs.org/@protobufjs/eventemitter/-/eventemitter-1.1.0.tgz",
- "integrity": "sha512-j9ednRT81vYJ9OfVuXG6ERSTdEL1xVsNgqpkxMsbIabzSo3goCjDIveeGv5d03om39ML71RdmrGNjG5SReBP/Q==",
- "license": "BSD-3-Clause"
- },
- "node_modules/@protobufjs/fetch": {
- "version": "1.1.0",
- "resolved": "https://registry.npmjs.org/@protobufjs/fetch/-/fetch-1.1.0.tgz",
- "integrity": "sha512-lljVXpqXebpsijW71PZaCYeIcE5on1w5DlQy5WH6GLbFryLUrBD4932W/E2BSpfRJWseIL4v/KPgBFxDOIdKpQ==",
- "license": "BSD-3-Clause",
- "dependencies": {
- "@protobufjs/aspromise": "^1.1.1",
- "@protobufjs/inquire": "^1.1.0"
- }
- },
- "node_modules/@protobufjs/float": {
- "version": "1.0.2",
- "resolved": "https://registry.npmjs.org/@protobufjs/float/-/float-1.0.2.tgz",
- "integrity": "sha512-Ddb+kVXlXst9d+R9PfTIxh1EdNkgoRe5tOX6t01f1lYWOvJnSPDBlG241QLzcyPdoNTsblLUdujGSE4RzrTZGQ==",
- "license": "BSD-3-Clause"
- },
- "node_modules/@protobufjs/inquire": {
- "version": "1.1.0",
- "resolved": "https://registry.npmjs.org/@protobufjs/inquire/-/inquire-1.1.0.tgz",
- "integrity": "sha512-kdSefcPdruJiFMVSbn801t4vFK7KB/5gd2fYvrxhuJYg8ILrmn9SKSX2tZdV6V+ksulWqS7aXjBcRXl3wHoD9Q==",
- "license": "BSD-3-Clause"
- },
- "node_modules/@protobufjs/path": {
- "version": "1.1.2",
- "resolved": "https://registry.npmjs.org/@protobufjs/path/-/path-1.1.2.tgz",
- "integrity": "sha512-6JOcJ5Tm08dOHAbdR3GrvP+yUUfkjG5ePsHYczMFLq3ZmMkAD98cDgcT2iA1lJ9NVwFd4tH/iSSoe44YWkltEA==",
- "license": "BSD-3-Clause"
- },
- "node_modules/@protobufjs/pool": {
- "version": "1.1.0",
- "resolved": "https://registry.npmjs.org/@protobufjs/pool/-/pool-1.1.0.tgz",
- "integrity": "sha512-0kELaGSIDBKvcgS4zkjz1PeddatrjYcmMWOlAuAPwAeccUrPHdUqo/J6LiymHHEiJT5NrF1UVwxY14f+fy4WQw==",
- "license": "BSD-3-Clause"
- },
- "node_modules/@protobufjs/utf8": {
- "version": "1.1.0",
- "resolved": "https://registry.npmjs.org/@protobufjs/utf8/-/utf8-1.1.0.tgz",
- "integrity": "sha512-Vvn3zZrhQZkkBE8LSuW3em98c0FwgO4nxzv6OdSxPKJIEKY2bGbHn+mhGIPerzI4twdxaP8/0+06HBpwf345Lw==",
- "license": "BSD-3-Clause"
- },
- "node_modules/@types/node": {
- "version": "22.18.12",
- "resolved": "https://registry.npmjs.org/@types/node/-/node-22.18.12.tgz",
- "integrity": "sha512-BICHQ67iqxQGFSzfCFTT7MRQ5XcBjG5aeKh5Ok38UBbPe5fxTyE+aHFxwVrGyr8GNlqFMLKD1D3P2K/1ks8tog==",
- "license": "MIT",
- "dependencies": {
- "undici-types": "~6.21.0"
- }
- },
- "node_modules/@types/shimmer": {
- "version": "1.2.0",
- "resolved": "https://registry.npmjs.org/@types/shimmer/-/shimmer-1.2.0.tgz",
- "integrity": "sha512-UE7oxhQLLd9gub6JKIAhDq06T0F6FnztwMNRvYgjeQSBeMc1ZG/tA47EwfduvkuQS8apbkM/lpLpWsaCeYsXVg==",
- "license": "MIT"
- },
- "node_modules/acorn": {
- "version": "8.15.0",
- "resolved": "https://registry.npmjs.org/acorn/-/acorn-8.15.0.tgz",
- "integrity": "sha512-NZyJarBfL7nWwIq+FDL6Zp/yHEhePMNnnJ0y3qfieCrmNvYct8uvtiV41UvlSe6apAfk0fY1FbWx+NwfmpvtTg==",
- "license": "MIT",
- "bin": {
- "acorn": "bin/acorn"
- },
- "engines": {
- "node": ">=0.4.0"
- }
- },
- "node_modules/acorn-import-assertions": {
- "version": "1.9.0",
- "resolved": "https://registry.npmjs.org/acorn-import-assertions/-/acorn-import-assertions-1.9.0.tgz",
- "integrity": "sha512-cmMwop9x+8KFhxvKrKfPYmN6/pKTYYHBqLa0DfvVZcKMJWNyWLnaqND7dx/qn66R7ewM1UX5XMaDVP5wlVTaVA==",
- "deprecated": "package has been renamed to acorn-import-attributes",
- "license": "MIT",
- "peerDependencies": {
- "acorn": "^8"
- }
- },
- "node_modules/acorn-import-attributes": {
- "version": "1.9.5",
- "resolved": "https://registry.npmjs.org/acorn-import-attributes/-/acorn-import-attributes-1.9.5.tgz",
- "integrity": "sha512-n02Vykv5uA3eHGM/Z2dQrcD56kL8TyDb2p1+0P83PClMnC/nc+anbQRhIOWnSq4Ke/KvDPrY3C9hDtC/A3eHnQ==",
- "license": "MIT",
- "peerDependencies": {
- "acorn": "^8"
- }
- },
- "node_modules/cjs-module-lexer": {
- "version": "1.4.3",
- "resolved": "https://registry.npmjs.org/cjs-module-lexer/-/cjs-module-lexer-1.4.3.tgz",
- "integrity": "sha512-9z8TZaGM1pfswYeXrUpzPrkx8UnWYdhJclsiYMm6x/w5+nN+8Tf/LnAgfLGQCm59qAOxU8WwHEq2vNwF6i4j+Q==",
- "license": "MIT"
- },
- "node_modules/debug": {
- "version": "4.4.3",
- "resolved": "https://registry.npmjs.org/debug/-/debug-4.4.3.tgz",
- "integrity": "sha512-RGwwWnwQvkVfavKVt22FGLw+xYSdzARwm0ru6DhTVA3umU5hZc28V3kO4stgYryrTlLpuvgI9GiijltAjNbcqA==",
- "license": "MIT",
- "dependencies": {
- "ms": "^2.1.3"
- },
- "engines": {
- "node": ">=6.0"
- },
- "peerDependenciesMeta": {
- "supports-color": {
- "optional": true
- }
- }
- },
- "node_modules/function-bind": {
- "version": "1.1.2",
- "resolved": "https://registry.npmjs.org/function-bind/-/function-bind-1.1.2.tgz",
- "integrity": "sha512-7XHNxH7qX9xG5mIwxkhumTox/MIRNcOgDrxWsMt2pAr23WHp6MrRlN7FBSFpCpr+oVO0F744iUgR82nJMfG2SA==",
- "license": "MIT",
- "funding": {
- "url": "https://github.com/sponsors/ljharb"
- }
- },
- "node_modules/hasown": {
- "version": "2.0.2",
- "resolved": "https://registry.npmjs.org/hasown/-/hasown-2.0.2.tgz",
- "integrity": "sha512-0hJU9SCPvmMzIBdZFqNPXWa6dqh7WdH0cII9y+CyS8rG3nL48Bclra9HmKhVVUHyPWNH5Y7xDwAB7bfgSjkUMQ==",
- "license": "MIT",
- "dependencies": {
- "function-bind": "^1.1.2"
- },
- "engines": {
- "node": ">= 0.4"
- }
- },
- "node_modules/import-in-the-middle": {
- "version": "1.15.0",
- "resolved": "https://registry.npmjs.org/import-in-the-middle/-/import-in-the-middle-1.15.0.tgz",
- "integrity": "sha512-bpQy+CrsRmYmoPMAE/0G33iwRqwW4ouqdRg8jgbH3aKuCtOc8lxgmYXg2dMM92CRiGP660EtBcymH/eVUpCSaA==",
- "license": "Apache-2.0",
- "dependencies": {
- "acorn": "^8.14.0",
- "acorn-import-attributes": "^1.9.5",
- "cjs-module-lexer": "^1.2.2",
- "module-details-from-path": "^1.0.3"
- }
- },
- "node_modules/is-core-module": {
- "version": "2.16.1",
- "resolved": "https://registry.npmjs.org/is-core-module/-/is-core-module-2.16.1.tgz",
- "integrity": "sha512-UfoeMA6fIJ8wTYFEUjelnaGI67v6+N7qXJEvQuIGa99l4xsCruSYOVSQ0uPANn4dAzm8lkYPaKLrrijLq7x23w==",
- "license": "MIT",
- "dependencies": {
- "hasown": "^2.0.2"
- },
- "engines": {
- "node": ">= 0.4"
- },
- "funding": {
- "url": "https://github.com/sponsors/ljharb"
- }
- },
- "node_modules/long": {
- "version": "5.3.2",
- "resolved": "https://registry.npmjs.org/long/-/long-5.3.2.tgz",
- "integrity": "sha512-mNAgZ1GmyNhD7AuqnTG3/VQ26o760+ZYBPKjPvugO8+nLbYfX6TVpJPseBvopbdY+qpZ/lKUnmEc1LeZYS3QAA==",
- "license": "Apache-2.0"
- },
- "node_modules/module-details-from-path": {
- "version": "1.0.4",
- "resolved": "https://registry.npmjs.org/module-details-from-path/-/module-details-from-path-1.0.4.tgz",
- "integrity": "sha512-EGWKgxALGMgzvxYF1UyGTy0HXX/2vHLkw6+NvDKW2jypWbHpjQuj4UMcqQWXHERJhVGKikolT06G3bcKe4fi7w==",
- "license": "MIT"
- },
- "node_modules/ms": {
- "version": "2.1.3",
- "resolved": "https://registry.npmjs.org/ms/-/ms-2.1.3.tgz",
- "integrity": "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==",
- "license": "MIT"
- },
- "node_modules/openai": {
- "version": "6.7.0",
- "resolved": "https://registry.npmjs.org/openai/-/openai-6.7.0.tgz",
- "integrity": "sha512-mgSQXa3O/UXTbA8qFzoa7aydbXBJR5dbLQXCRapAOtoNT+v69sLdKMZzgiakpqhclRnhPggPAXoniVGn2kMY2A==",
- "license": "Apache-2.0",
- "bin": {
- "openai": "bin/cli"
- },
- "peerDependencies": {
- "ws": "^8.18.0",
- "zod": "^3.25 || ^4.0"
- },
- "peerDependenciesMeta": {
- "ws": {
- "optional": true
- },
- "zod": {
- "optional": true
- }
- }
- },
- "node_modules/path-parse": {
- "version": "1.0.7",
- "resolved": "https://registry.npmjs.org/path-parse/-/path-parse-1.0.7.tgz",
- "integrity": "sha512-LDJzPVEEEPR+y48z93A0Ed0yXb8pAByGWo/k5YYdYgpY2/2EsOsksJrq7lOHxryrVOn1ejG6oAp8ahvOIQD8sw==",
- "license": "MIT"
- },
- "node_modules/protobufjs": {
- "version": "7.5.4",
- "resolved": "https://registry.npmjs.org/protobufjs/-/protobufjs-7.5.4.tgz",
- "integrity": "sha512-CvexbZtbov6jW2eXAvLukXjXUW1TzFaivC46BpWc/3BpcCysb5Vffu+B3XHMm8lVEuy2Mm4XGex8hBSg1yapPg==",
- "hasInstallScript": true,
- "license": "BSD-3-Clause",
- "dependencies": {
- "@protobufjs/aspromise": "^1.1.2",
- "@protobufjs/base64": "^1.1.2",
- "@protobufjs/codegen": "^2.0.4",
- "@protobufjs/eventemitter": "^1.1.0",
- "@protobufjs/fetch": "^1.1.0",
- "@protobufjs/float": "^1.0.2",
- "@protobufjs/inquire": "^1.1.0",
- "@protobufjs/path": "^1.1.2",
- "@protobufjs/pool": "^1.1.0",
- "@protobufjs/utf8": "^1.1.0",
- "@types/node": ">=13.7.0",
- "long": "^5.0.0"
- },
- "engines": {
- "node": ">=12.0.0"
- }
- },
- "node_modules/require-in-the-middle": {
- "version": "7.5.2",
- "resolved": "https://registry.npmjs.org/require-in-the-middle/-/require-in-the-middle-7.5.2.tgz",
- "integrity": "sha512-gAZ+kLqBdHarXB64XpAe2VCjB7rIRv+mU8tfRWziHRJ5umKsIHN2tLLv6EtMw7WCdP19S0ERVMldNvxYCHnhSQ==",
- "license": "MIT",
- "dependencies": {
- "debug": "^4.3.5",
- "module-details-from-path": "^1.0.3",
- "resolve": "^1.22.8"
- },
- "engines": {
- "node": ">=8.6.0"
- }
- },
- "node_modules/resolve": {
- "version": "1.22.11",
- "resolved": "https://registry.npmjs.org/resolve/-/resolve-1.22.11.tgz",
- "integrity": "sha512-RfqAvLnMl313r7c9oclB1HhUEAezcpLjz95wFH4LVuhk9JF/r22qmVP9AMmOU4vMX7Q8pN8jwNg/CSpdFnMjTQ==",
- "license": "MIT",
- "dependencies": {
- "is-core-module": "^2.16.1",
- "path-parse": "^1.0.7",
- "supports-preserve-symlinks-flag": "^1.0.0"
- },
- "bin": {
- "resolve": "bin/resolve"
- },
- "engines": {
- "node": ">= 0.4"
- },
- "funding": {
- "url": "https://github.com/sponsors/ljharb"
- }
- },
- "node_modules/semver": {
- "version": "7.7.3",
- "resolved": "https://registry.npmjs.org/semver/-/semver-7.7.3.tgz",
- "integrity": "sha512-SdsKMrI9TdgjdweUSR9MweHA4EJ8YxHn8DFaDisvhVlUOe4BF1tLD7GAj0lIqWVl+dPb/rExr0Btby5loQm20Q==",
- "license": "ISC",
- "bin": {
- "semver": "bin/semver.js"
- },
- "engines": {
- "node": ">=10"
- }
- },
- "node_modules/shimmer": {
- "version": "1.2.1",
- "resolved": "https://registry.npmjs.org/shimmer/-/shimmer-1.2.1.tgz",
- "integrity": "sha512-sQTKC1Re/rM6XyFM6fIAGHRPVGvyXfgzIDvzoq608vM+jeyVD0Tu1E6Np0Kc2zAIFWIj963V2800iF/9LPieQw==",
- "license": "BSD-2-Clause"
- },
- "node_modules/supports-preserve-symlinks-flag": {
- "version": "1.0.0",
- "resolved": "https://registry.npmjs.org/supports-preserve-symlinks-flag/-/supports-preserve-symlinks-flag-1.0.0.tgz",
- "integrity": "sha512-ot0WnXS9fgdkgIcePe6RHNk1WA8+muPa6cSjeR3V8K27q9BB1rTE3R1p7Hv0z1ZyAc8s6Vvv8DIyWf681MAt0w==",
- "license": "MIT",
- "engines": {
- "node": ">= 0.4"
- },
- "funding": {
- "url": "https://github.com/sponsors/ljharb"
- }
- },
- "node_modules/typescript": {
- "version": "5.9.3",
- "resolved": "https://registry.npmjs.org/typescript/-/typescript-5.9.3.tgz",
- "integrity": "sha512-jl1vZzPDinLr9eUt3J/t7V6FgNEw9QjvBPdysz9KfQDD41fQrC2Y4vKQdiaUpFT4bXlb1RHhLpp8wtm6M5TgSw==",
- "dev": true,
- "license": "Apache-2.0",
- "bin": {
- "tsc": "bin/tsc",
- "tsserver": "bin/tsserver"
- },
- "engines": {
- "node": ">=14.17"
- }
- },
- "node_modules/undici-types": {
- "version": "6.21.0",
- "resolved": "https://registry.npmjs.org/undici-types/-/undici-types-6.21.0.tgz",
- "integrity": "sha512-iwDZqg0QAGrg9Rav5H4n0M64c3mkR59cJ6wQp+7C4nI0gsmExaedaYLNO44eT4AtBBwjbTiGPMlt2Md0T9H9JQ==",
- "license": "MIT"
- }
- }
-}
diff --git a/examples/node/observability-opentelemetry/package.json b/examples/node/observability-opentelemetry/package.json
deleted file mode 100644
index bc74d98dae..0000000000
--- a/examples/node/observability-opentelemetry/package.json
+++ /dev/null
@@ -1,25 +0,0 @@
-{
- "name": "agenta-opentelemetry-quickstart",
- "version": "1.0.0",
- "description": "Quick start example for using OpenTelemetry with Agenta",
- "type": "module",
- "scripts": {
- "start": "node --import ./instrumentation.js app.js"
- },
- "dependencies": {
- "@arizeai/openinference-instrumentation-openai": "^3.2.3",
- "@arizeai/openinference-semantic-conventions": "^2.1.2",
- "@opentelemetry/api": "^1.9.0",
- "@opentelemetry/exporter-trace-otlp-proto": "^0.54.0",
- "@opentelemetry/instrumentation": "^0.54.0",
- "@opentelemetry/resources": "^1.28.0",
- "@opentelemetry/sdk-trace-base": "^1.28.0",
- "@opentelemetry/sdk-trace-node": "^1.28.0",
- "@opentelemetry/semantic-conventions": "^1.28.0",
- "openai": "^6.7.0"
- },
- "devDependencies": {
- "@types/node": "^22.0.0",
- "typescript": "^5.7.0"
- }
-}
diff --git a/examples/node/observability-opentelemetry/test.sh b/examples/node/observability-opentelemetry/test.sh
deleted file mode 100755
index 56fb36d592..0000000000
--- a/examples/node/observability-opentelemetry/test.sh
+++ /dev/null
@@ -1,29 +0,0 @@
-#!/bin/bash
-# Test script for OpenTelemetry example
-
-# Check if required environment variables are set
-if [ -z "$AGENTA_API_KEY" ]; then
- echo "❌ Error: AGENTA_API_KEY is not set"
- echo " Set it with: export AGENTA_API_KEY='your_key_here'"
- exit 1
-fi
-
-if [ -z "$OPENAI_API_KEY" ]; then
- echo "❌ Error: OPENAI_API_KEY is not set"
- echo " Set it with: export OPENAI_API_KEY='your_key_here'"
- exit 1
-fi
-
-# Set default AGENTA_HOST if not provided
-if [ -z "$AGENTA_HOST" ]; then
- export AGENTA_HOST="https://cloud.staging.agenta.ai"
- echo "ℹ️ Using default AGENTA_HOST: $AGENTA_HOST"
-fi
-
-echo "✅ Environment variables configured"
-echo ""
-echo "🚀 Running example..."
-echo ""
-
-npm start
-
diff --git a/examples/python/custom_workflows/ai-code-reviewer/ai-code-reviewer.py b/examples/python/custom_workflows/ai-code-reviewer/ai-code-reviewer.py
index 2703c420d2..4ab89887a1 100644
--- a/examples/python/custom_workflows/ai-code-reviewer/ai-code-reviewer.py
+++ b/examples/python/custom_workflows/ai-code-reviewer/ai-code-reviewer.py
@@ -89,4 +89,6 @@ def generate_critique(pr_url: str):
if __name__ == "__main__":
import uvicorn
- uvicorn.run("agenta:app", host="0.0.0.0", port=8000, reload=True)
+ uvicorn.run(
+ "agenta.sdk.decorators.routing:app", host="0.0.0.0", port=8000, reload=True
+ )
diff --git a/examples/python/custom_workflows/chain_of_prompts/cop.py b/examples/python/custom_workflows/chain_of_prompts/cop.py
index 1af034895d..163daf542a 100644
--- a/examples/python/custom_workflows/chain_of_prompts/cop.py
+++ b/examples/python/custom_workflows/chain_of_prompts/cop.py
@@ -38,4 +38,6 @@ def generate(blog_post: str):
if __name__ == "__main__":
import uvicorn
- uvicorn.run("agenta:app", host="0.0.0.0", port=8000, reload=True)
+ uvicorn.run(
+ "agenta.sdk.decorators.routing:app", host="0.0.0.0", port=8000, reload=True
+ )
diff --git a/examples/python/custom_workflows/rag-docs-qa/README.md b/examples/python/custom_workflows/rag-docs-qa/README.md
index e02e4dd77c..fe844b9bd3 100644
--- a/examples/python/custom_workflows/rag-docs-qa/README.md
+++ b/examples/python/custom_workflows/rag-docs-qa/README.md
@@ -61,4 +61,4 @@ python query.py
## Notes:
-- `generate_testset.py` is used to generate a testset of questions based on the documentation for evaluation.
+- `generate_test_set.py` is used to generate a test set of questions based on the documentation for evaluation.
diff --git a/examples/python/custom_workflows/rag-docs-qa/generate_test_set.py b/examples/python/custom_workflows/rag-docs-qa/generate_test_set.py
new file mode 100644
index 0000000000..3210a8c244
--- /dev/null
+++ b/examples/python/custom_workflows/rag-docs-qa/generate_test_set.py
@@ -0,0 +1,99 @@
+import os
+import glob
+from pathlib import Path
+import pandas as pd
+from dotenv import load_dotenv
+from litellm import completion
+import frontmatter
+import tqdm
+import json
+
+# Load environment variables
+load_dotenv()
+
+
+def get_files(docs_path):
+ """Get all markdown files recursively."""
+ return
+
+
+def extract_content(file_path):
+ """Extract content from markdown file."""
+ with open(file_path, "r", encoding="utf-8") as f:
+ post = frontmatter.load(f)
+ # Get title from frontmatter or filename
+ title = post.get("title", Path(file_path).stem)
+ # Get content without frontmatter
+ content = post.content
+ return title, content
+
+
+def generate_questions(title, content):
+ """Generate questions using OpenAI."""
+ system_prompt = """You are a helpful assistant that generates questions based on documentation content.
+ Generate 5 questions that could be answered using the provided documentation.
+ Your response must be a JSON object with a single key "questions" containing an array of strings."""
+
+ user_prompt = f"""
+ Title: {title}
+
+ Content: {content} # Limit content length to avoid token limits
+
+ Generate 5 questions about this documentation. Put yourself in the shoes of a user attempting to 1) figure how to use the product for a use case 2) troubleshoot an issue 3) learn about the features of the product.
+ The user in this case is a technical user (AI engineer) who is trying to build an llm application.
+ The user would write the questions they would ask in a chat with a human. Therefore, not all questions will be clear and well written.
+ """
+
+ try:
+ response = completion(
+ model="gpt-3.5-turbo-0125", # Using the latest model that supports JSON mode
+ messages=[
+ {"role": "system", "content": system_prompt},
+ {"role": "user", "content": user_prompt},
+ ],
+ response_format={"type": "json_object"},
+ )
+
+ # Check if the response was complete
+ if response.choices[0].finish_reason == "length":
+ print(f"Warning: Response was truncated for {title}")
+ return []
+
+ # Parse JSON response - no need for eval()
+ result = json.loads(response.choices[0].message.content)
+ return result["questions"]
+
+ except Exception as e:
+ print(f"Error generating questions for {title}: {str(e)}")
+ return []
+
+
+def main():
+ docs_path = os.getenv("DOCS_PATH")
+ if not docs_path:
+ raise ValueError("DOCS_PATH environment variable not set")
+
+ # Get all files
+ files = glob.glob(os.path.join(docs_path, "**/*.mdx"), recursive=True)
+ all_questions = []
+ # Process each file
+ for file_path in tqdm.tqdm(files, desc="Processing documentation files"):
+ if "/reference/api" in file_path:
+ # skip api docs
+ continue
+ try:
+ title, content = extract_content(file_path)
+ questions = generate_questions(title, content)
+ all_questions.extend(questions)
+ except Exception as e:
+ print(f"Error processing {file_path}: {str(e)}")
+ continue
+
+ # Save to CSV
+ df = pd.DataFrame({"query": all_questions})
+ df.to_csv("test_set.csv", index=False, lineterminator="\n")
+ print(f"Generated {len(all_questions)} questions and saved to test_set.csv")
+
+
+if __name__ == "__main__":
+ main()
diff --git a/examples/python/custom_workflows/rag-docs-qa/generate_testset.py b/examples/python/custom_workflows/rag-docs-qa/generate_testset.py
deleted file mode 100644
index 0eb362fd6d..0000000000
--- a/examples/python/custom_workflows/rag-docs-qa/generate_testset.py
+++ /dev/null
@@ -1,99 +0,0 @@
-import os
-import glob
-from pathlib import Path
-import pandas as pd
-from dotenv import load_dotenv
-from litellm import completion
-import frontmatter
-import tqdm
-import json
-
-# Load environment variables
-load_dotenv()
-
-
-def get_files(docs_path):
- """Get all markdown files recursively."""
- return
-
-
-def extract_content(file_path):
- """Extract content from markdown file."""
- with open(file_path, "r", encoding="utf-8") as f:
- post = frontmatter.load(f)
- # Get title from frontmatter or filename
- title = post.get("title", Path(file_path).stem)
- # Get content without frontmatter
- content = post.content
- return title, content
-
-
-def generate_questions(title, content):
- """Generate questions using OpenAI."""
- system_prompt = """You are a helpful assistant that generates questions based on documentation content.
- Generate 5 questions that could be answered using the provided documentation.
- Your response must be a JSON object with a single key "questions" containing an array of strings."""
-
- user_prompt = f"""
- Title: {title}
-
- Content: {content} # Limit content length to avoid token limits
-
- Generate 5 questions about this documentation. Put yourself in the shoes of a user attempting to 1) figure how to use the product for a use case 2) troubleshoot an issue 3) learn about the features of the product.
- The user in this case is a technical user (AI engineer) who is trying to build an llm application.
- The user would write the questions they would ask in a chat with a human. Therefore, not all questions will be clear and well written.
- """
-
- try:
- response = completion(
- model="gpt-3.5-turbo-0125", # Using the latest model that supports JSON mode
- messages=[
- {"role": "system", "content": system_prompt},
- {"role": "user", "content": user_prompt},
- ],
- response_format={"type": "json_object"},
- )
-
- # Check if the response was complete
- if response.choices[0].finish_reason == "length":
- print(f"Warning: Response was truncated for {title}")
- return []
-
- # Parse JSON response - no need for eval()
- result = json.loads(response.choices[0].message.content)
- return result["questions"]
-
- except Exception as e:
- print(f"Error generating questions for {title}: {str(e)}")
- return []
-
-
-def main():
- docs_path = os.getenv("DOCS_PATH")
- if not docs_path:
- raise ValueError("DOCS_PATH environment variable not set")
-
- # Get all files
- files = glob.glob(os.path.join(docs_path, "**/*.mdx"), recursive=True)
- all_questions = []
- # Process each file
- for file_path in tqdm.tqdm(files, desc="Processing documentation files"):
- if "/reference/api" in file_path:
- # skip api docs
- continue
- try:
- title, content = extract_content(file_path)
- questions = generate_questions(title, content)
- all_questions.extend(questions)
- except Exception as e:
- print(f"Error processing {file_path}: {str(e)}")
- continue
-
- # Save to CSV
- df = pd.DataFrame({"query": all_questions})
- df.to_csv("testset.csv", index=False, lineterminator="\n")
- print(f"Generated {len(all_questions)} questions and saved to testset.csv")
-
-
-if __name__ == "__main__":
- main()
diff --git a/examples/python/custom_workflows/rag-docs-qa/query.py b/examples/python/custom_workflows/rag-docs-qa/query.py
index 2bee1789e3..f5d494bac7 100644
--- a/examples/python/custom_workflows/rag-docs-qa/query.py
+++ b/examples/python/custom_workflows/rag-docs-qa/query.py
@@ -164,4 +164,6 @@ def generate(query: str):
if __name__ == "__main__":
import uvicorn
- uvicorn.run("agenta:app", host="0.0.0.0", port=8000, reload=True)
+ uvicorn.run(
+ "agenta.sdk.decorators.routing:app", host="0.0.0.0", port=8000, reload=True
+ )
diff --git a/examples/python/custom_workflows/rag-docs-qa/test_set.csv b/examples/python/custom_workflows/rag-docs-qa/test_set.csv
new file mode 100644
index 0000000000..9c7d6028b4
--- /dev/null
+++ b/examples/python/custom_workflows/rag-docs-qa/test_set.csv
@@ -0,0 +1,311 @@
+query,correct_answer
+How can I deploy Agenta locally in production mode?,
+What command should I run to update Agenta to the latest version?,
+How do I start Agenta in development mode?,
+What steps should I take to troubleshoot if I encounter a port conflict issue?,
+Where can I seek help or report issues related to Agenta?,
+How can I deploy Agenta on Kubernetes for my AI model hosting use case?,
+"I'm having trouble accessing the Kubernetes deployment feature in Agenta Enterprise, how can I troubleshoot this?",
+Can Agenta Enterprise on Kubernetes support scaling my AI models automatically based on demand?,
+What tools are provided by Agenta Enterprise to manage multiple users and teams?,
+Can you provide more details about Agenta Enterprise's early access stage for select partners?,
+How can I deploy Agenta on AWS EC2 using Terraform?,
+What are the open ports created by the Terraform module for Agenta on AWS EC2?,
+How do I serve a variant to an instance after hosting Agenta?,
+What steps are involved in SSH-ing into the instance where Agenta is hosted?,
+How can I delete all the resources created by Terraform for Agenta on AWS?,
+How can I deploy Agenta on Google Cloud Engine using Terraform?,
+What are the prerequisites for deploying Agenta on Google Cloud Engine?,
+How do I SSH into the instance after deploying Agenta on Google Cloud Engine?,
+What security considerations should I keep in mind when allowing SSH access to the instance?,
+How can I delete all the resources created by Terraform for Agenta on Google Cloud Engine?,
+How can I host Agenta on a remote server like an AWS EC2 instance?,
+What are the prerequisites for deploying Agenta on a remote server?,
+How do I obtain the public IP of my AWS EC2 instance?,
+What environment variables do I need to set before launching the Agenta server?,
+"After starting the Agenta server, how can I verify that it is running correctly on the remote server?",
+How can I apply schema migrations to my PostgreSQL database using the provided instructions?,
+What command should I run to ensure that Alembic looks for the configuration file and other necessary files in the specified directory before executing?,
+"If I need to revert a schema migration due to encountered issues, how can I do that according to the documentation?",
+"Is there a way to automate the application of migrations without manually running the upgrade command? If so, how can I achieve this?",
+"After completing the migration, what should I do to verify data integrity in PostgreSQL and ensure that everything works fine?",
+How can I back up my MongoDB database before running the migration script?,
+What Docker command should I use to start the local instance of Agenta for migration?,
+"After initiating the migration, how can I access the running docker containers?",
+How can I verify the data integrity in PostgreSQL post migration?,
+"In case I encounter issues during migration, how can I revert the migration and what should I do next?",
+How can I upgrade to the latest version of Agenta using Docker?,
+What database does Agenta now use starting from version 0.19?,
+How can I access the backend Docker container for Agenta?,
+What is Beanie and how does it relate to MongoDB ODM for Python?,
+What are the steps involved in performing a database migration using Beanie with Agenta backend system?,
+Where can I find answers to common questions about Agenta?,
+How can I get community support for Agenta? Is there a Slack community?,
+What is the recommended channel for reporting bugs in Agenta?,
+"As a Pro plan cloud user, how can I access direct team support?",
+"If I'm interested in demos or have sales inquiries, how can I schedule a call with the founders?",
+How can I add new users to my workspace?,
+What are the different user roles available and what are their respective rights?,
+"As a Workspace Admin, what are the limitations in terms of managing the workspace?",
+What is the role of a Deployment Manager and what tasks do they handle within the workspace?,
+How can I switch between different workspaces if I am a member of multiple workspaces?,
+How do I format backend and CLI code using Black in specific directories?,
+"What are the main steps to contribute to Agenta, especially when picking an issue?",
+Can you guide me through running backend tests locally before making a pull request?,
+How can I update the .env.local file in the frontend directory to include my OpenAI API Key?,
+What are the rules regarding issue assignment and PR activity to prevent zombie issues in the Agenta project?,
+How can I use my own version of Agenta CLI or SDK instead of the installed one?,
+What steps can I take if I'm unable to run Agenta in my terminal even after following the provided instructions?,
+How can I quickly work and test a new type of parameter like IntParam in the SDK?,
+Are there any specific steps needed for working on the backend code in Agenta?,
+How can I efficiently debug the backend code in Agenta using Docker or Visual Studio Code?,
+How can I report a bug using the bug report template?,
+Where can I find the network logs in the browser and why are they important when reporting an issue?,
+What command should I run to get information about Docker containers and how can I provide this information when filing an issue?,
+"When reporting an issue related to a specific llm app, what are the steps to provide container information using Docker Desktop or the terminal?",
+Why is providing network logs and Docker container information crucial when filing an issue?,
+How can I start monitoring and understanding the behavior of my LLM application with Agenta?,
+What are the advantages of using OpenTelemetry (OTel) with Agenta's observability features?,
+What is the relationship between traces and spans in the context of Agenta?,
+Where can I find a Quick Start guide to get started with observability in Agenta?,
+How can I learn to instrument specific applications like OpenAI or LangChain with Agenta?,
+How can I enable tracing for OpenAI calls using Agenta SDK?,
+What packages do I need to install to set up observability for an OpenAI application running locally?,
+How can I configure environment variables for Agenta OSS running locally?,
+Where can I find the captured traces of my application's requests in the Agenta UI?,
+Is tracing enabled by default if I create an application through the Agenta UI?,
+How can I install the Agenta SDK?,
+What is the purpose of the @ag.instrument() decorator in Agenta SDK?,
+How can I add additional metadata to a span in Agenta?,
+"Can you explain how to link spans to applications, variants, and environments in Agenta?",
+What options are available for redacting sensitive data in Agenta SDK?,
+How do I install LiteLLM and Agenta SDK?,
+What are the different environments I can configure while setting up LiteLLM with Agenta?,
+How can I initialize Agenta SDK in my LiteLLM application?,
+What does the @ag.instrument() decorator do in the code example provided?,
+Why is it important to set up the callback handler for LiteLLM while using Agenta?,
+How can I install the required packages for using LangChain with Agenta?,
+What environment variables do I need to configure when using Agenta Cloud or Enterprise?,
+What steps are involved in the code example provided for a LangChain application?,
+How do I initialize Agenta in my LangChain application?,
+Why is it important to call 'LangchainInstrumentor().instrument()' before running my LangChain application?,
+How can I instrument OpenAI API calls with Agenta using `opentelemetry-instrumentation-openai` package?,
+What packages do I need to install to instrument OpenAI API calls with Agenta using `opentelemetry-instrumentation-openai` package?,
+How do I configure environment variables for Agenta Cloud or Enterprise when instrumenting OpenAI API calls?,
+What decorator can I use to monitor multiple calls in a function or workflow as a single trace?,
+How can I associate traces with specific parts of an Agenta project when instrumenting functions?,
+How do I install the required packages for using Instructor with Agenta?,
+What environment variables do I need to configure for Agenta Cloud or Enterprise setup?,
+What must I ensure the order of when instrumenting OpenAI and creating the Instructor client?,
+Why is it mentioned to instrument OpenAI before creating the Instructor client?,
+"What does the @ag.instrument(spankind=""WORKFLOW"") decorator do in the code example?",
+How can I experiment and compare prompts using Agenta?,
+What types of workflows does Agenta support for prompt management and evaluation?,
+How can I collaborate with product teams using Agenta for prompt engineering and evaluation?,
+What are the different ways to deploy an application with Agenta?,
+"Is Agenta compatible with various LLM app architectures and model providers? If so, how?",
+How can I create a new LLM app using an existing template in Agenta?,
+Where can I find the API endpoint after deploying my application in Agenta?,
+What steps do I need to follow to test the application in the playground?,
+How do I add my OpenAI API keys when self-hosting Agenta?,
+What are some next steps after creating my first LLM application in Agenta?,
+How can I create custom templates for my LLM-powered applications in Agenta?,
+What are the predefined environments in Agenta and their purposes?,
+"Explain the relationship between variants, versions, and commits in Agenta.",
+Can you describe the difference between Completion Application Template and Chat Application Template in Agenta?,
+What can I do if the deployment of a variant to an environment does not update automatically in Agenta?,
+How does Agenta enable rapid experimentation and evaluation for LLM applications?,
+What is the significance of Agenta treating each application as a microservice?,
+How does Agenta handle the separation of application logic and configuration?,
+What role does Agenta's backend play in managing applications and configurations?,
+Can you explain the purpose and functionality of Agenta's SDK for a Python library?,
+How can I collaborate with subject matter experts using custom workflows in Agenta?,
+What is the main problem with traditional prompt playgrounds?,
+Can I trust the outputs of traditional prompt playgrounds?,
+How does Agenta's Custom Workflows simplify debugging for AI engineers?,
+What frameworks and models are compatible with Agenta?,
+How can I create a custom workflow with two prompts using Agenta?,
+What are the steps to serve an application in Agenta using the CLI?,
+How do I initialize the Agenta SDK in my Python code?,
+What is the purpose of the `CoPConfig` class in the provided code snippet?,
+Can you explain the concept of entry points in Agenta and how they are used in the code example?,
+How can I configure evaluators for my LLM application?,
+What tools are available for creating test sets for evaluation?,
+Can I run evaluations directly from the web UI in Agenta?,
+What types of evaluators are available for classification/entity extraction in LLM applications?,
+How can I evaluate the faithfulness of outputs in RAG workflows?,
+How can I create a test set for evaluation in agenta SDK?,
+What is the purpose of creating evaluators in the agenta SDK?,
+How can I run an evaluation job using the agenta SDK?,
+What is the function of the rate limit configuration in evaluation jobs?,
+How can I retrieve the detailed results of an evaluation using the agenta SDK?,
+How can I initiate a single model evaluation in the Human Evaluation feature?,
+What steps are involved in starting a new evaluation with the single model test?,
+How do I compare the performance of two different variants manually using A/B Test in the Human Evaluation feature?,
+Can collaborators be invited to collaborate on an A/B Test evaluation in the Human Evaluation feature?,
+Is there a way to switch between card and table view in the evaluation process? How can this be done?,
+How do I create a test set in Agenta using a CSV or JSON file?,
+What is the default column name for the reference answer in a CSV test set?,
+What is the structure of a valid JSON file for a test set in Agenta?,
+How can I add data to a test set from the playground in Agenta?,
+"In Agenta, how can I upload a test set using the API?",
+How do I start a new evaluation in the UI?,
+What parameters can be specified when setting up a new evaluation?,
+What are the advanced configuration options available for adjusting batching and retry parameters?,
+How can I analyze the results of an evaluation in more detail?,
+"Can I compare multiple evaluations from the same test set in the UI? If yes, how?",
+How can I configure evaluators for my LLM application?,
+What are the inputs that evaluators typically take?,
+Which button should I click to create a new evaluator?,
+Can I create custom evaluators for my LLM application?,
+What is the purpose of mappings evaluator's inputs to the LLM data?,
+How can I create a webhook evaluator for a specific use case in Agenta?,
+What are the limitations in terms of security measures when using webhook evaluators?,
+What input parameters are required for a webhook evaluator in Agenta?,
+What is the expected format of the webhook request body for a webhook evaluator?,
+What should the webhook response body contain in order to be considered properly-formatted?,
+How does the Exact Match evaluator determine if the model's output is correct?,
+What does the Contains JSON evaluator check for in the model's output?,
+How does the JSON Field Match evaluator compare specific fields within JSON data?,
+Can you explain the process of JSON Diff Match evaluation?,
+What configuration options are available for the JSON Diff Match evaluator and how do they affect the comparison process?,
+How can I assess the performance of LLMs by identifying specific patterns within the output generated by the model?,
+What is the purpose of the 'Regex Test' evaluator in Agenta and how does it work?,
+Could you provide an example of using the 'Starts With' evaluator in Agenta and its significance?,
+How does the 'Contains Any' evaluator in Agenta differ from the 'Contains All' evaluator?,
+In what situations would I use the 'Ends With' evaluator in Agenta and how does it handle case sensitivity?,
+How can I create a custom evaluator in Agenta?,
+What is the function signature for the 'evaluate' function in a custom evaluator?,
+What is the purpose of the 'evaluate' function in a custom evaluator?,
+Which language is used for writing custom evaluators in Agenta?,
+What ranges of scores can the 'evaluate' function return?,
+How can I configure and use RAG evaluators in my custom-built application?,
+What version of the Python SDK is required to access internal variables and intermediate outputs for RAG Evaluators?,
+Where can I find the source code for a simple RAG Application that fetches movies and generates summaries?,
+How can I troubleshoot if the RAG Evaluator and `view trace` are not accessible in Agenta OSS?,
+Which utility can I use in Agenta to add internal variables (internals) to stages of a workflow?,
+How does the Similarity Match evaluator determine a match between the generated output and the correct answer?,
+What configuration parameter is needed for the Semantic Similarity Match evaluator?,
+What is the cost associated with using the Semantic Similarity Match evaluator that uses OpenAI embeddings?,
+In what use cases is the Levenshtein Distance evaluator particularly useful?,
+Can you explain how the Levenshtein Distance evaluator works for comparing text outputs?,
+How can I configure the prompt for LLM-as-a-Judge evaluation?,
+"What variables can be used to reference inputs, outputs, and reference answers in the prompt?",
+What is the default prompt used for the country expert demo application?,
+"Which models are supported by LLM-as-a-Judge, and how can I select a specific model?",
+Where do I need to set my OpenAI or Anthropic API key to use LLM-as-a-Judge?,
+How can I create a test set using the SDK?,
+What is the process for creating and configuring an evaluator programmatically?,
+How can I check the status of an evaluation run?,
+Where can I find the list of evaluator keys and configurations?,
+What should I do if I encounter rate limits during the evaluation process?,
+How can I create a new prompt using the Agenta SDK?,
+How do I deploy changes to the production environment using the SDK?,
+"What is the structure used by Agenta for prompt versioning, and how does it differ from traditional versioning systems?",
+"In case of an issue with deploying changes using the SDK, what steps can I take to troubleshoot?",
+Can you explain the concept of variants in Agenta and how they are similar to branches in Git?,
+How can I set up tracing for a RAG application in LangChain using Agenta?,
+What is the purpose of tracing in LLM applications?,
+"How do I install the necessary dependencies for LangChain, Agenta, and instrumentation?",
+Can you provide an example of a Q&A RAG application in LangChain and how it works?,
+How can I retrieve and generate answers using relevant snippets of documentation in Agenta?,
+How can I fetch the diff for a GitHub pull request using Python?,
+What role does LiteLLM play in the AI assistant workflow described in the tutorial?,
+How can I add observability to my LLM application using Agenta?,
+What is the purpose of creating an LLM playground for the application?,
+How can I deploy my AI assistant to production using Agenta?,
+How can I version prompts in Agenta?,
+What capabilities does Agenta provide for prompt management?,
+Why do I need a prompt management system?,
+What is the purpose of configuration management in Agenta?,
+How can I publish a prompt to an endpoint from the web UI?,
+How can I create a prompt using the web UI?,
+Why do I need to publish a variant to a deployment?,
+How can I integrate the prompt configuration with my Python code using the Agenta SDK?,
+"Can I revert to a previous deployment version, and if so, how?",
+What is the purpose of using Pydantic for schema validation in the SDK integration process?,
+How can I create reusable prompts using prompt templates in the agenta playground?,
+Where can I find the option to add new inputs to the LLM app in the playground?,
+"What is the process for creating a new variant of an application in agenta, and where can I provide a new name for the variant?",
+"How can I test a variant dynamically in the playground, and is there a way to run all inputs in a test set?",
+"In the agenta playground, how do I compare variants side by side? Can I interact with different variants simultaneously in a chat application?",
+How can I create a new prompt using the SDK?,
+What is the structure followed by Agenta for prompt versioning?,
+How can I delete a variant in Agenta? Is this action reversible?,
+Can I list all variants of an application using the SDK? How can I do that?,
+What is the default behavior for fetching configurations in Agenta?,
+How can I use the Agenta SDK to invoke the deployed version of my prompt?,
+Where can I find the call to invoke the deployed version of my prompt within the Agenta UI?,
+What parameters do I need to provide when invoking a deployed prompt through the REST API?,
+What does the 'inputs' dictionary in the parameters contain and how should it be structured?,
+How can I control which environment version of my prompt is being called when using the REST API?,
+How do I fetch the deployed version of my prompt in my code using the Agenta SDK?,
+What Python SDK do I need to install for using the Agenta SDK?,
+What environment variables need to be set up for fetching prompts?,
+"In the sample output, what is the value of 'model' fetched from staging configuration?",
+Which slugs should I provide when fetching prompts using Agenta SDK?,
+How can Agenta help me manage prompts in my application?,
+What are the advantages of using Agenta as a prompt management system?,
+What considerations should I keep in mind when integrating observability manually with Agenta?,
+When should I consider using Agenta as a middleware/model proxy?,
+Does using Agenta as a proxy add any latency to the response?,
+How can I install the agenta CLI tool?,
+What is the command to install agenta using pip?,
+Where can I find a quick usage guide for the agenta CLI?,
+How do I get an overview of the main commands and capabilities of agenta CLI?,
+Is there a tutorial available for deploying an LLM app from code using agenta CLI?,
+How can I initialize a new Agenta project using the CLI?,
+"What options can be provided with the 'agenta init' command, and how do they affect the project initialization process?",
+"For the 'agenta variant list' command, where should it be executed, and what information does it provide?",
+What are the steps involved in removing a variant from an application using the 'agenta variant remove' command?,
+"How can I deploy an application variant to the Agenta platform using the 'agenta variant serve' command, and what additional options are available for deployment?",
+How can I create a new variant with initial configuration parameters?,
+What method should I use to commit changes to an existing variant?,
+Which method is used to deploy a variant to a specific environment?,
+How can I retrieve the version history of a variant?,
+What are the available predefined environments for deployment?,
+How can I store additional data within the current span in a trace?,
+What utility function can I use to validate if a string is a valid attribute key?,
+How can I link spans to specific Agenta resources using the SDK?,
+What method should I use to set the status of a span in the CustomSpan class?,
+When should I apply the @ag.instrument() decorator in my Python code?,
+How can I expose specific stages of LLM workflows as API endpoints using the Custom Workflows SDK?,
+What types of fields are accepted in the configuration schema when defining a configuration for a function?,
+How can I define fields with constraints and defaults in the configuration schema when using `@ag.route` or `@ag.entrypoint` decorators?,
+"How are different field types represented in the Agenta playground UI, and what are the input methods for each type?",
+How can I retrieve configuration information from the route context using `ag.ConfigManager.get_from_route()` and how is this configuration used in a function?,
+How do I experiment with AI applications using the agenta SDK?,
+What is the purpose of the deprecated SDK v2 mentioned on the page?,
+How can I initialize my variant using the agenta SDK?,
+What function should I use to set the default configuration in the agenta SDK?,
+Can the agenta SDK be used with any Foundational Model?,
+How can I display a text area widget in the playground using the SDK v2?,
+What is the difference between IntParam and FloatParam in terms of the widgets they display in the playground?,
+Can you provide an example of how to use BinaryParam in the SDK v2 configuration?,
+What is the purpose of the GroupedMultipleChoiceParam in the configuration settings?,
+What is the important note mentioned regarding the initialization of BinaryParam in the SDK v2?,
+How do I access the parameters in the configuration in my code?,
+What is the purpose of the 'config' object in the SDK v2?,
+Can I experiment with different parameters using the 'config' object?,
+What is the significance of calling 'agenta.init()' before using the 'config' object?,
+What are some examples of parameters that can be saved in the 'config' object for an LLM variant?,
+Why is it recommended to call `agenta.init()` only once at the entry point of the code?,
+What happens if `agenta.init()` is called multiple times in the code?,
+How does calling `agenta.init()` help in initializing the variant?,
+Is there a different method to initialize the variant in the deprecated SDK v2?,
+What could be the potential impact of using the deprecated SDK v2 for building an llm application?,
+How can I set the default configuration for my variant using the register_default method?,
+What happens if I set the default prompt value to 'Hello World' in my configuration?,
+When should I use the register_default method in my llm application development?,
+What parameters can I set using the register_default method for my llm application?,
+How can I access the prompt1 parameter value from the configuration in the backend?,
+How do I push configuration using agenta.config.push()?,
+What happens if I set overwrite to True when pushing a configuration?,
+How can I avoid overwriting an existing configuration when using agenta.config.push()?,
+Where does the pushed configuration get stored for a specific code base?,
+Can I push configurations for multiple variants using agenta.config.push()?,
+How can I pull a configuration with a specific name?,
+What happens when I pull a configuration using the 'production' environment?,
+How can I access the parameters after pulling the configuration?,
+Can I pull configurations for different variants using this function?,
+Why does the documentation mention that this page is for the deprecated SDK v2?,
\ No newline at end of file
diff --git a/examples/python/custom_workflows/rag-docs-qa/testset_small.csv b/examples/python/custom_workflows/rag-docs-qa/test_set_small.csv
similarity index 100%
rename from examples/python/custom_workflows/rag-docs-qa/testset_small.csv
rename to examples/python/custom_workflows/rag-docs-qa/test_set_small.csv
diff --git a/examples/python/custom_workflows/rag-docs-qa/testset.csv b/examples/python/custom_workflows/rag-docs-qa/testset.csv
deleted file mode 100644
index e10dd18aac..0000000000
--- a/examples/python/custom_workflows/rag-docs-qa/testset.csv
+++ /dev/null
@@ -1,311 +0,0 @@
-query,correct_answer
-How can I deploy Agenta locally in production mode?,
-What command should I run to update Agenta to the latest version?,
-How do I start Agenta in development mode?,
-What steps should I take to troubleshoot if I encounter a port conflict issue?,
-Where can I seek help or report issues related to Agenta?,
-How can I deploy Agenta on Kubernetes for my AI model hosting use case?,
-"I'm having trouble accessing the Kubernetes deployment feature in Agenta Enterprise, how can I troubleshoot this?",
-Can Agenta Enterprise on Kubernetes support scaling my AI models automatically based on demand?,
-What tools are provided by Agenta Enterprise to manage multiple users and teams?,
-Can you provide more details about Agenta Enterprise's early access stage for select partners?,
-How can I deploy Agenta on AWS EC2 using Terraform?,
-What are the open ports created by the Terraform module for Agenta on AWS EC2?,
-How do I serve a variant to an instance after hosting Agenta?,
-What steps are involved in SSH-ing into the instance where Agenta is hosted?,
-How can I delete all the resources created by Terraform for Agenta on AWS?,
-How can I deploy Agenta on Google Cloud Engine using Terraform?,
-What are the prerequisites for deploying Agenta on Google Cloud Engine?,
-How do I SSH into the instance after deploying Agenta on Google Cloud Engine?,
-What security considerations should I keep in mind when allowing SSH access to the instance?,
-How can I delete all the resources created by Terraform for Agenta on Google Cloud Engine?,
-How can I host Agenta on a remote server like an AWS EC2 instance?,
-What are the prerequisites for deploying Agenta on a remote server?,
-How do I obtain the public IP of my AWS EC2 instance?,
-What environment variables do I need to set before launching the Agenta server?,
-"After starting the Agenta server, how can I verify that it is running correctly on the remote server?",
-How can I apply schema migrations to my PostgreSQL database using the provided instructions?,
-What command should I run to ensure that Alembic looks for the configuration file and other necessary files in the specified directory before executing?,
-"If I need to revert a schema migration due to encountered issues, how can I do that according to the documentation?",
-"Is there a way to automate the application of migrations without manually running the upgrade command? If so, how can I achieve this?",
-"After completing the migration, what should I do to verify data integrity in PostgreSQL and ensure that everything works fine?",
-How can I back up my MongoDB database before running the migration script?,
-What Docker command should I use to start the local instance of Agenta for migration?,
-"After initiating the migration, how can I access the running docker containers?",
-How can I verify the data integrity in PostgreSQL post migration?,
-"In case I encounter issues during migration, how can I revert the migration and what should I do next?",
-How can I upgrade to the latest version of Agenta using Docker?,
-What database does Agenta now use starting from version 0.19?,
-How can I access the backend Docker container for Agenta?,
-What is Beanie and how does it relate to MongoDB ODM for Python?,
-What are the steps involved in performing a database migration using Beanie with Agenta backend system?,
-Where can I find answers to common questions about Agenta?,
-How can I get community support for Agenta? Is there a Slack community?,
-What is the recommended channel for reporting bugs in Agenta?,
-"As a Pro plan cloud user, how can I access direct team support?",
-"If I'm interested in demos or have sales inquiries, how can I schedule a call with the founders?",
-How can I add new users to my workspace?,
-What are the different user roles available and what are their respective rights?,
-"As a Workspace Admin, what are the limitations in terms of managing the workspace?",
-What is the role of a Deployment Manager and what tasks do they handle within the workspace?,
-How can I switch between different workspaces if I am a member of multiple workspaces?,
-How do I format backend and CLI code using Black in specific directories?,
-"What are the main steps to contribute to Agenta, especially when picking an issue?",
-Can you guide me through running backend tests locally before making a pull request?,
-How can I update the .env.local file in the frontend directory to include my OpenAI API Key?,
-What are the rules regarding issue assignment and PR activity to prevent zombie issues in the Agenta project?,
-How can I use my own version of Agenta CLI or SDK instead of the installed one?,
-What steps can I take if I'm unable to run Agenta in my terminal even after following the provided instructions?,
-How can I quickly work and test a new type of parameter like IntParam in the SDK?,
-Are there any specific steps needed for working on the backend code in Agenta?,
-How can I efficiently debug the backend code in Agenta using Docker or Visual Studio Code?,
-How can I report a bug using the bug report template?,
-Where can I find the network logs in the browser and why are they important when reporting an issue?,
-What command should I run to get information about Docker containers and how can I provide this information when filing an issue?,
-"When reporting an issue related to a specific llm app, what are the steps to provide container information using Docker Desktop or the terminal?",
-Why is providing network logs and Docker container information crucial when filing an issue?,
-How can I start monitoring and understanding the behavior of my LLM application with Agenta?,
-What are the advantages of using OpenTelemetry (OTel) with Agenta's observability features?,
-What is the relationship between traces and spans in the context of Agenta?,
-Where can I find a Quick Start guide to get started with observability in Agenta?,
-How can I learn to instrument specific applications like OpenAI or LangChain with Agenta?,
-How can I enable tracing for OpenAI calls using Agenta SDK?,
-What packages do I need to install to set up observability for an OpenAI application running locally?,
-How can I configure environment variables for Agenta OSS running locally?,
-Where can I find the captured traces of my application's requests in the Agenta UI?,
-Is tracing enabled by default if I create an application through the Agenta UI?,
-How can I install the Agenta SDK?,
-What is the purpose of the @ag.instrument() decorator in Agenta SDK?,
-How can I add additional metadata to a span in Agenta?,
-"Can you explain how to link spans to applications, variants, and environments in Agenta?",
-What options are available for redacting sensitive data in Agenta SDK?,
-How do I install LiteLLM and Agenta SDK?,
-What are the different environments I can configure while setting up LiteLLM with Agenta?,
-How can I initialize Agenta SDK in my LiteLLM application?,
-What does the @ag.instrument() decorator do in the code example provided?,
-Why is it important to set up the callback handler for LiteLLM while using Agenta?,
-How can I install the required packages for using LangChain with Agenta?,
-What environment variables do I need to configure when using Agenta Cloud or Enterprise?,
-What steps are involved in the code example provided for a LangChain application?,
-How do I initialize Agenta in my LangChain application?,
-Why is it important to call 'LangchainInstrumentor().instrument()' before running my LangChain application?,
-How can I instrument OpenAI API calls with Agenta using `opentelemetry-instrumentation-openai` package?,
-What packages do I need to install to instrument OpenAI API calls with Agenta using `opentelemetry-instrumentation-openai` package?,
-How do I configure environment variables for Agenta Cloud or Enterprise when instrumenting OpenAI API calls?,
-What decorator can I use to monitor multiple calls in a function or workflow as a single trace?,
-How can I associate traces with specific parts of an Agenta project when instrumenting functions?,
-How do I install the required packages for using Instructor with Agenta?,
-What environment variables do I need to configure for Agenta Cloud or Enterprise setup?,
-What must I ensure the order of when instrumenting OpenAI and creating the Instructor client?,
-Why is it mentioned to instrument OpenAI before creating the Instructor client?,
-"What does the @ag.instrument(spankind=""WORKFLOW"") decorator do in the code example?",
-How can I experiment and compare prompts using Agenta?,
-What types of workflows does Agenta support for prompt management and evaluation?,
-How can I collaborate with product teams using Agenta for prompt engineering and evaluation?,
-What are the different ways to deploy an application with Agenta?,
-"Is Agenta compatible with various LLM app architectures and model providers? If so, how?",
-How can I create a new LLM app using an existing template in Agenta?,
-Where can I find the API endpoint after deploying my application in Agenta?,
-What steps do I need to follow to test the application in the playground?,
-How do I add my OpenAI API keys when self-hosting Agenta?,
-What are some next steps after creating my first LLM application in Agenta?,
-How can I create custom templates for my LLM-powered applications in Agenta?,
-What are the predefined environments in Agenta and their purposes?,
-"Explain the relationship between variants, versions, and commits in Agenta.",
-Can you describe the difference between Completion Application Template and Chat Application Template in Agenta?,
-What can I do if the deployment of a variant to an environment does not update automatically in Agenta?,
-How does Agenta enable rapid experimentation and evaluation for LLM applications?,
-What is the significance of Agenta treating each application as a microservice?,
-How does Agenta handle the separation of application logic and configuration?,
-What role does Agenta's backend play in managing applications and configurations?,
-Can you explain the purpose and functionality of Agenta's SDK for a Python library?,
-How can I collaborate with subject matter experts using custom workflows in Agenta?,
-What is the main problem with traditional prompt playgrounds?,
-Can I trust the outputs of traditional prompt playgrounds?,
-How does Agenta's Custom Workflows simplify debugging for AI engineers?,
-What frameworks and models are compatible with Agenta?,
-How can I create a custom workflow with two prompts using Agenta?,
-What are the steps to serve an application in Agenta using the CLI?,
-How do I initialize the Agenta SDK in my Python code?,
-What is the purpose of the `CoPConfig` class in the provided code snippet?,
-Can you explain the concept of entry points in Agenta and how they are used in the code example?,
-How can I configure evaluators for my LLM application?,
-What tools are available for creating testsets for evaluation?,
-Can I run evaluations directly from the web UI in Agenta?,
-What types of evaluators are available for classification/entity extraction in LLM applications?,
-How can I evaluate the faithfulness of outputs in RAG workflows?,
-How can I create a testset for evaluation in agenta SDK?,
-What is the purpose of creating evaluators in the agenta SDK?,
-How can I run an evaluation job using the agenta SDK?,
-What is the function of the rate limit configuration in evaluation jobs?,
-How can I retrieve the detailed results of an evaluation using the agenta SDK?,
-How can I initiate a single model evaluation in the Human Evaluation feature?,
-What steps are involved in starting a new evaluation with the single model test?,
-How do I compare the performance of two different variants manually using A/B Test in the Human Evaluation feature?,
-Can collaborators be invited to collaborate on an A/B Test evaluation in the Human Evaluation feature?,
-Is there a way to switch between card and table view in the evaluation process? How can this be done?,
-How do I create a testset in Agenta using a CSV or JSON file?,
-What is the default column name for the reference answer in a CSV testset?,
-What is the structure of a valid JSON file for a testset in Agenta?,
-How can I add data to a testset from the playground in Agenta?,
-"In Agenta, how can I upload a testset using the API?",
-How do I start a new evaluation in the UI?,
-What parameters can be specified when setting up a new evaluation?,
-What are the advanced configuration options available for adjusting batching and retry parameters?,
-How can I analyze the results of an evaluation in more detail?,
-"Can I compare multiple evaluations from the same testset in the UI? If yes, how?",
-How can I configure evaluators for my LLM application?,
-What are the inputs that evaluators typically take?,
-Which button should I click to create a new evaluator?,
-Can I create custom evaluators for my LLM application?,
-What is the purpose of mappings evaluator's inputs to the LLM data?,
-How can I create a webhook evaluator for a specific use case in Agenta?,
-What are the limitations in terms of security measures when using webhook evaluators?,
-What input parameters are required for a webhook evaluator in Agenta?,
-What is the expected format of the webhook request body for a webhook evaluator?,
-What should the webhook response body contain in order to be considered properly-formatted?,
-How does the Exact Match evaluator determine if the model's output is correct?,
-What does the Contains JSON evaluator check for in the model's output?,
-How does the JSON Field Match evaluator compare specific fields within JSON data?,
-Can you explain the process of JSON Diff Match evaluation?,
-What configuration options are available for the JSON Diff Match evaluator and how do they affect the comparison process?,
-How can I assess the performance of LLMs by identifying specific patterns within the output generated by the model?,
-What is the purpose of the 'Regex Test' evaluator in Agenta and how does it work?,
-Could you provide an example of using the 'Starts With' evaluator in Agenta and its significance?,
-How does the 'Contains Any' evaluator in Agenta differ from the 'Contains All' evaluator?,
-In what situations would I use the 'Ends With' evaluator in Agenta and how does it handle case sensitivity?,
-How can I create a custom evaluator in Agenta?,
-What is the function signature for the 'evaluate' function in a custom evaluator?,
-What is the purpose of the 'evaluate' function in a custom evaluator?,
-Which language is used for writing custom evaluators in Agenta?,
-What ranges of scores can the 'evaluate' function return?,
-How can I configure and use RAG evaluators in my custom-built application?,
-What version of the Python SDK is required to access internal variables and intermediate outputs for RAG Evaluators?,
-Where can I find the source code for a simple RAG Application that fetches movies and generates summaries?,
-How can I troubleshoot if the RAG Evaluator and `view trace` are not accessible in Agenta OSS?,
-Which utility can I use in Agenta to add internal variables (internals) to stages of a workflow?,
-How does the Similarity Match evaluator determine a match between the generated output and the correct answer?,
-What configuration parameter is needed for the Semantic Similarity Match evaluator?,
-What is the cost associated with using the Semantic Similarity Match evaluator that uses OpenAI embeddings?,
-In what use cases is the Levenshtein Distance evaluator particularly useful?,
-Can you explain how the Levenshtein Distance evaluator works for comparing text outputs?,
-How can I configure the prompt for LLM-as-a-Judge evaluation?,
-"What variables can be used to reference inputs, outputs, and reference answers in the prompt?",
-What is the default prompt used for the country expert demo application?,
-"Which models are supported by LLM-as-a-Judge, and how can I select a specific model?",
-Where do I need to set my OpenAI or Anthropic API key to use LLM-as-a-Judge?,
-How can I create a testset using the SDK?,
-What is the process for creating and configuring an evaluator programmatically?,
-How can I check the status of an evaluation run?,
-Where can I find the list of evaluator keys and configurations?,
-What should I do if I encounter rate limits during the evaluation process?,
-How can I create a new prompt using the Agenta SDK?,
-How do I deploy changes to the production environment using the SDK?,
-"What is the structure used by Agenta for prompt versioning, and how does it differ from traditional versioning systems?",
-"In case of an issue with deploying changes using the SDK, what steps can I take to troubleshoot?",
-Can you explain the concept of variants in Agenta and how they are similar to branches in Git?,
-How can I set up tracing for a RAG application in LangChain using Agenta?,
-What is the purpose of tracing in LLM applications?,
-"How do I install the necessary dependencies for LangChain, Agenta, and instrumentation?",
-Can you provide an example of a Q&A RAG application in LangChain and how it works?,
-How can I retrieve and generate answers using relevant snippets of documentation in Agenta?,
-How can I fetch the diff for a GitHub pull request using Python?,
-What role does LiteLLM play in the AI assistant workflow described in the tutorial?,
-How can I add observability to my LLM application using Agenta?,
-What is the purpose of creating an LLM playground for the application?,
-How can I deploy my AI assistant to production using Agenta?,
-How can I version prompts in Agenta?,
-What capabilities does Agenta provide for prompt management?,
-Why do I need a prompt management system?,
-What is the purpose of configuration management in Agenta?,
-How can I publish a prompt to an endpoint from the web UI?,
-How can I create a prompt using the web UI?,
-Why do I need to publish a variant to a deployment?,
-How can I integrate the prompt configuration with my Python code using the Agenta SDK?,
-"Can I revert to a previous deployment version, and if so, how?",
-What is the purpose of using Pydantic for schema validation in the SDK integration process?,
-How can I create reusable prompts using prompt templates in the agenta playground?,
-Where can I find the option to add new inputs to the LLM app in the playground?,
-"What is the process for creating a new variant of an application in agenta, and where can I provide a new name for the variant?",
-"How can I test a variant dynamically in the playground, and is there a way to run all inputs in a testset?",
-"In the agenta playground, how do I compare variants side by side? Can I interact with different variants simultaneously in a chat application?",
-How can I create a new prompt using the SDK?,
-What is the structure followed by Agenta for prompt versioning?,
-How can I delete a variant in Agenta? Is this action reversible?,
-Can I list all variants of an application using the SDK? How can I do that?,
-What is the default behavior for fetching configurations in Agenta?,
-How can I use the Agenta SDK to invoke the deployed version of my prompt?,
-Where can I find the call to invoke the deployed version of my prompt within the Agenta UI?,
-What parameters do I need to provide when invoking a deployed prompt through the REST API?,
-What does the 'inputs' dictionary in the parameters contain and how should it be structured?,
-How can I control which environment version of my prompt is being called when using the REST API?,
-How do I fetch the deployed version of my prompt in my code using the Agenta SDK?,
-What Python SDK do I need to install for using the Agenta SDK?,
-What environment variables need to be set up for fetching prompts?,
-"In the sample output, what is the value of 'model' fetched from staging configuration?",
-Which slugs should I provide when fetching prompts using Agenta SDK?,
-How can Agenta help me manage prompts in my application?,
-What are the advantages of using Agenta as a prompt management system?,
-What considerations should I keep in mind when integrating observability manually with Agenta?,
-When should I consider using Agenta as a middleware/model proxy?,
-Does using Agenta as a proxy add any latency to the response?,
-How can I install the agenta CLI tool?,
-What is the command to install agenta using pip?,
-Where can I find a quick usage guide for the agenta CLI?,
-How do I get an overview of the main commands and capabilities of agenta CLI?,
-Is there a tutorial available for deploying an LLM app from code using agenta CLI?,
-How can I initialize a new Agenta project using the CLI?,
-"What options can be provided with the 'agenta init' command, and how do they affect the project initialization process?",
-"For the 'agenta variant list' command, where should it be executed, and what information does it provide?",
-What are the steps involved in removing a variant from an application using the 'agenta variant remove' command?,
-"How can I deploy an application variant to the Agenta platform using the 'agenta variant serve' command, and what additional options are available for deployment?",
-How can I create a new variant with initial configuration parameters?,
-What method should I use to commit changes to an existing variant?,
-Which method is used to deploy a variant to a specific environment?,
-How can I retrieve the version history of a variant?,
-What are the available predefined environments for deployment?,
-How can I store additional data within the current span in a trace?,
-What utility function can I use to validate if a string is a valid attribute key?,
-How can I link spans to specific Agenta resources using the SDK?,
-What method should I use to set the status of a span in the CustomSpan class?,
-When should I apply the @ag.instrument() decorator in my Python code?,
-How can I expose specific stages of LLM workflows as API endpoints using the Custom Workflows SDK?,
-What types of fields are accepted in the configuration schema when defining a configuration for a function?,
-How can I define fields with constraints and defaults in the configuration schema when using `@ag.route` or `@ag.entrypoint` decorators?,
-"How are different field types represented in the Agenta playground UI, and what are the input methods for each type?",
-How can I retrieve configuration information from the route context using `ag.ConfigManager.get_from_route()` and how is this configuration used in a function?,
-How do I experiment with AI applications using the agenta SDK?,
-What is the purpose of the deprecated SDK v2 mentioned on the page?,
-How can I initialize my variant using the agenta SDK?,
-What function should I use to set the default configuration in the agenta SDK?,
-Can the agenta SDK be used with any Foundational Model?,
-How can I display a text area widget in the playground using the SDK v2?,
-What is the difference between IntParam and FloatParam in terms of the widgets they display in the playground?,
-Can you provide an example of how to use BinaryParam in the SDK v2 configuration?,
-What is the purpose of the GroupedMultipleChoiceParam in the configuration settings?,
-What is the important note mentioned regarding the initialization of BinaryParam in the SDK v2?,
-How do I access the parameters in the configuration in my code?,
-What is the purpose of the 'config' object in the SDK v2?,
-Can I experiment with different parameters using the 'config' object?,
-What is the significance of calling 'agenta.init()' before using the 'config' object?,
-What are some examples of parameters that can be saved in the 'config' object for an LLM variant?,
-Why is it recommended to call `agenta.init()` only once at the entry point of the code?,
-What happens if `agenta.init()` is called multiple times in the code?,
-How does calling `agenta.init()` help in initializing the variant?,
-Is there a different method to initialize the variant in the deprecated SDK v2?,
-What could be the potential impact of using the deprecated SDK v2 for building an llm application?,
-How can I set the default configuration for my variant using the register_default method?,
-What happens if I set the default prompt value to 'Hello World' in my configuration?,
-When should I use the register_default method in my llm application development?,
-What parameters can I set using the register_default method for my llm application?,
-How can I access the prompt1 parameter value from the configuration in the backend?,
-How do I push configuration using agenta.config.push()?,
-What happens if I set overwrite to True when pushing a configuration?,
-How can I avoid overwriting an existing configuration when using agenta.config.push()?,
-Where does the pushed configuration get stored for a specific code base?,
-Can I push configurations for multiple variants using agenta.config.push()?,
-How can I pull a configuration with a specific name?,
-What happens when I pull a configuration using the 'production' environment?,
-How can I access the parameters after pulling the configuration?,
-Can I pull configurations for different variants using this function?,
-Why does the documentation mention that this page is for the deprecated SDK v2?,
\ No newline at end of file
diff --git a/hooks/setup.sh b/hooks/setup.sh
deleted file mode 100755
index 287d31bab6..0000000000
--- a/hooks/setup.sh
+++ /dev/null
@@ -1,50 +0,0 @@
-#!/usr/bin/env bash
-set -euo pipefail
-
-echo "🔧 Setting up Git hooks with pre-commit + gitleaks..."
-
-# --- check dependencies ---
-if ! command -v python3 >/dev/null 2>&1; then
- echo "❌ Python3 is required but not installed."
- exit 1
-fi
-if ! command -v pip3 >/dev/null 2>&1; then
- echo "❌ pip3 is required but not installed."
- exit 1
-fi
-
-# --- install pre-commit globally if missing ---
-if ! command -v pre-commit >/dev/null 2>&1; then
- echo "📦 Installing pre-commit..."
- pip3 install pre-commit
-fi
-
-# --- install gitleaks globally if missing ---
-if ! command -v gitleaks >/dev/null 2>&1; then
- echo "📦 Installing gitleaks..."
- if command -v brew >/dev/null 2>&1; then
- brew install gitleaks
- else
- # fallback: go install (requires Go installed)
- go install github.com/zricethezav/gitleaks/v8@latest
- export PATH="$PATH:$(go env GOPATH)/bin"
- echo "----------------------------------------------------------------------"
- echo "⚠️⚠️ PLEASE ADD $(go env GOPATH)/bin TO YOUR PATH IN YOUR .bashrc OR .zshrc ⚠️⚠️"
- echo "----------------------------------------------------------------------"
- fi
-fi
-
-# --- install hooks into .git/hooks/ ---
-echo "⚙️ Installing pre-commit hooks..."
-pre-commit install --install-hooks
-pre-commit install --hook-type pre-push
-
-# --- one-time full scans ---
-echo "🔍 Running one-time gitleaks scans..."
-
-gitleaks --config .gitleaks.toml --exit-code 1 --verbose detect --no-git --source . || {
- echo "❌ Gitleaks detected potential secrets in the working directory."
- exit 1
-}
-
-echo "✅ Setup complete! Hooks installed and initial scan passed. You are safe to commit."
diff --git a/hosting/old/aws/agenta_instance.tf b/hosting/aws/agenta_instance.tf
similarity index 100%
rename from hosting/old/aws/agenta_instance.tf
rename to hosting/aws/agenta_instance.tf
diff --git a/hosting/old/aws/agenta_instance_sg.tf b/hosting/aws/agenta_instance_sg.tf
similarity index 100%
rename from hosting/old/aws/agenta_instance_sg.tf
rename to hosting/aws/agenta_instance_sg.tf
diff --git a/hosting/old/aws/instance-setup.sh b/hosting/aws/instance-setup.sh
similarity index 100%
rename from hosting/old/aws/instance-setup.sh
rename to hosting/aws/instance-setup.sh
diff --git a/hosting/old/aws/main.tf b/hosting/aws/main.tf
similarity index 100%
rename from hosting/old/aws/main.tf
rename to hosting/aws/main.tf
diff --git a/hosting/docker-compose/ee/LICENSE b/hosting/docker-compose/ee/LICENSE
deleted file mode 100644
index ae7a2f38f4..0000000000
--- a/hosting/docker-compose/ee/LICENSE
+++ /dev/null
@@ -1,37 +0,0 @@
-Agenta Enterprise License (the “Enterprise License”)
-Copyright (c) 2023–2025
-Agentatech UG (haftungsbeschränkt), doing business as “Agenta” (“Agenta”)
-
-With regard to the Agenta Software:
-
-This software and associated documentation files (the "Software") may only be
-used in production, if you (and any entity that you represent) have agreed to,
-and are in compliance with, the Agenta Subscription Terms of Service, available
-at https://agenta.ai/terms (the “Enterprise Terms”), or other
-agreement governing the use of the Software, as agreed by you and Agenta,
-and otherwise have a valid Agenta Enterprise License.
-
-Subject to the foregoing sentence, you are free to modify this Software and
-publish patches to the Software. You agree that Agenta and/or its licensors
-(as applicable) retain all right, title and interest in and to all such
-modifications and/or patches, and all such modifications and/or patches may
-only be used, copied, modified, displayed, distributed, or otherwise exploited
-with a valid Agenta Enterprise License. Notwithstanding the foregoing, you may
-copy and modify the Software for development and testing purposes, without
-requiring a subscription. You agree that Agenta and/or its licensors (as
-applicable) retain all right, title and interest in and to all such
-modifications. You are not granted any other rights beyond what is expressly
-stated herein. Subject to the foregoing, it is forbidden to copy, merge,
-publish, distribute, sublicense, and/or sell the Software.
-
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-SOFTWARE.
-
-For all third party components incorporated into the Agenta Software, those
-components are licensed under the original license provided by the owner of the
-applicable component.
diff --git a/hosting/docker-compose/ee/docker-compose.dev.yml b/hosting/docker-compose/ee/docker-compose.dev.yml
index 09861d7b20..3889ef61e1 100644
--- a/hosting/docker-compose/ee/docker-compose.dev.yml
+++ b/hosting/docker-compose/ee/docker-compose.dev.yml
@@ -49,7 +49,6 @@ services:
volumes:
- ../../../api:/app
- - ../../../sdk:/sdk
env_file:
- ${ENV_FILE:-./.env.ee.dev}
@@ -96,7 +95,6 @@ services:
volumes:
- ../../../api:/app
- - ../../../sdk:/sdk
env_file:
- ${ENV_FILE:-./.env.ee.dev}
@@ -115,14 +113,13 @@ services:
- agenta-network
command: >
- watchmedo auto-restart --directory=/app/ --pattern=*.py --recursive -- celery -A entrypoint.celery_app worker --concurrency=1 --max-tasks-per-child=1 --prefetch-multiplier=1 --loglevel=DEBUG
+ watchmedo auto-restart --directory=/app/ --pattern=*.py --recursive -- celery -A entrypoint.celery_app worker --concurrency=1 --max-tasks-per-child=1 --prefetch-multiplier=1
cron:
image: agenta-ee-dev-api:latest
volumes:
- ../../../api/ee/src/crons/meters.sh:/meters.sh
- - ../../../api/oss/src/crons/queries.sh:/queries.sh
env_file:
- ${ENV_FILE:-./.env.ee.dev}
@@ -145,8 +142,7 @@ services:
image: agenta-ee-dev-api:latest
volumes:
- - ../../../api:/app
- - ../../../sdk:/sdk
+ - ../../../api:/app/
env_file:
- ${ENV_FILE:-./.env.ee.dev}
@@ -157,12 +153,12 @@ services:
networks:
- agenta-network
- command: sh -c "python -m ee.databases.postgres.migrations.runner"
+ command: sh -c "python -c 'from ee.databases.postgres.migrations.utils import split_core_and_tracing as split; split(); from ee.databases.postgres.migrations.core.utils import run_alembic_migration as run; run(); from ee.databases.postgres.migrations.tracing.utils import run_alembic_migration as run; run(); from ee.databases.postgres.migrations.utils import copy_nodes_from_core_to_tracing as copy; copy();'"
completion:
build:
context: ../../../services/completion
- dockerfile: ee/docker/Dockerfile.dev
+ dockerfile: oss/docker/Dockerfile.dev
volumes:
- ../../../services/completion:/app
@@ -191,7 +187,7 @@ services:
chat:
build:
context: ../../../services/chat
- dockerfile: ee/docker/Dockerfile.dev
+ dockerfile: oss/docker/Dockerfile.dev
volumes:
- ../../../services/chat:/app
diff --git a/hosting/docker-compose/ee/env.ee.dev.example b/hosting/docker-compose/ee/env.ee.dev.example
deleted file mode 100644
index c42666965b..0000000000
--- a/hosting/docker-compose/ee/env.ee.dev.example
+++ /dev/null
@@ -1,91 +0,0 @@
-# First-party (required)
-AGENTA_LICENSE=ee
-AGENTA_STAGE=dev
-AGENTA_PROVIDER=local
-AGENTA_WEB_URL=http://localhost
-AGENTA_API_URL=http://localhost/api
-AGENTA_SERVICES_URL=http://localhost/services
-AGENTA_AUTH_KEY=change-me
-AGENTA_CRYPT_KEY=change-me
-AGENTA_API_IMAGE_NAME=agenta-api
-AGENTA_API_IMAGE_TAG=latest
-AGENTA_WEB_IMAGE_NAME=agenta-web
-AGENTA_WEB_IMAGE_TAG=latest
-
-# First-party (registry & service)
-DOCKER_NETWORK_MODE=bridge
-POSTGRES_USERNAME=username
-POSTGRES_PASSWORD=password
-
-# First-party (optional)
-AGENTA_AUTO_MIGRATIONS=true
-AGENTA_PRICING=
-AGENTA_DEMOS=
-AGENTA_RUNTIME_PREFIX=
-AGENTA_API_INTERNAL_URL=
-AGENTA_LITELLM_MOCK=
-POSTGRES_USERNAME_ADMIN=
-POSTGRES_PASSWORD_ADMIN=
-AGENTA_SERVICE_MIDDLEWARE_CACHE_ENABLED=true
-AGENTA_OTLP_MAX_BATCH_BYTES=10485760
-
-# Third-party (required)
-TRAEFIK_DOMAIN=
-TRAEFIK_PROTOCOL=
-TRAEFIK_PORT=
-
-REDIS_URL=redis://redis:6379/0
-RABBITMQ_DEFAULT_PASS=guest
-RABBITMQ_DEFAULT_USER=guest
-
-CELERY_BROKER_URL=amqp://guest@rabbitmq//
-CELERY_RESULT_BACKEND=redis://redis:6379/0
-
-POSTGRES_URI_SUPERTOKENS="postgresql://username:password@postgres:5432/agenta_ee_supertokens"
-POSTGRES_URI_CORE="postgresql+asyncpg://username:password@postgres:5432/agenta_ee_core"
-POSTGRES_URI_TRACING="postgresql+asyncpg://username:password@postgres:5432/agenta_ee_tracing"
-
-ALEMBIC_CFG_PATH_CORE=/app/ee/databases/postgres/migrations/core/alembic.ini
-ALEMBIC_CFG_PATH_TRACING=/app/ee/databases/postgres/migrations/tracing/alembic.ini
-
-SUPERTOKENS_CONNECTION_URI=http://supertokens:3567
-
-# Third-party (optional)
-AWS_ECR_URL=
-AWS_RDS_SECRET=
-
-POSTHOG_API_KEY=phc_3urGRy5TL1HhaHnRYL0JSHxJxigRVackhphHtozUmdp
-
-GITHUB_OAUTH_CLIENT_ID=
-GITHUB_OAUTH_CLIENT_SECRET=
-GOOGLE_OAUTH_CLIENT_ID=
-GOOGLE_OAUTH_CLIENT_SECRET=
-
-SUPERTOKENS_API_KEY=replace-me
-
-NEW_RELIC_LICENSE_KEY=
-NRIA_LICENSE_KEY=
-
-LOOPS_API_KEY=
-
-SENDGRID_API_KEY=
-
-CRISP_WEBSITE_ID=
-
-STRIPE_API_KEY=
-STRIPE_WEBHOOK_SECRET=
-STRIPE_TARGET=
-
-# Third-party - LLM (optional)
-ALEPHALPHA_API_KEY=
-ANTHROPIC_API_KEY=
-ANYSCALE_API_KEY=
-COHERE_API_KEY=
-DEEPINFRA_API_KEY=
-GEMINI_API_KEY=
-GROQ_API_KEY=
-MISTRAL_API_KEY=
-OPENAI_API_KEY=
-OPENROUTER_API_KEY=
-PERPLEXITYAI_API_KEY=
-TOGETHERAI_API_KEY=
diff --git a/hosting/docker-compose/ee/env.ee.gh.example b/hosting/docker-compose/ee/env.ee.gh.example
deleted file mode 100644
index 5cba09c18c..0000000000
--- a/hosting/docker-compose/ee/env.ee.gh.example
+++ /dev/null
@@ -1,80 +0,0 @@
-# First-party (required)
-AGENTA_LICENSE=ee
-AGENTA_STAGE=dev
-AGENTA_PROVIDER=local
-AGENTA_API_URL=http://localhost/api
-AGENTA_WEB_URL=http://localhost
-AGENTA_SERVICES_URL=http://localhost/services
-AGENTA_AUTH_KEY=change-me
-AGENTA_CRYPT_KEY=change-me
-
-# First-party (registry & service)
-DOCKER_NETWORK_MODE=bridge
-POSTGRES_PASSWORD=password
-POSTGRES_USERNAME=username
-
-# First-party (optional)
-AGENTA_AUTO_MIGRATIONS=true
-AGENTA_PRICING=
-AGENTA_DEMOS=
-AGENTA_RUNTIME_PREFIX=
-AGENTA_API_INTERNAL_URL=
-AGENTA_SERVICE_MIDDLEWARE_CACHE_ENABLED=true
-AGENTA_OTLP_MAX_BATCH_BYTES=10485760
-
-# Third-party (required)
-TRAEFIK_DOMAIN=
-TRAEFIK_PROTOCOL=
-TRAEFIK_PORT=
-
-REDIS_URL=redis://redis:6379/0
-RABBITMQ_DEFAULT_PASS=guest
-RABBITMQ_DEFAULT_USER=guest
-
-CELERY_BROKER_URL=amqp://guest@rabbitmq//
-CELERY_RESULT_BACKEND=redis://redis:6379/0
-
-POSTGRES_URI_SUPERTOKENS="postgresql://username:password@postgres:5432/agenta_ee_supertokens"
-POSTGRES_URI_CORE="postgresql+asyncpg://username:password@postgres:5432/agenta_ee_core"
-POSTGRES_URI_TRACING="postgresql+asyncpg://username:password@postgres:5432/agenta_ee_tracing"
-
-ALEMBIC_CFG_PATH_CORE=/app/ee/databases/postgres/migrations/core/alembic.ini
-ALEMBIC_CFG_PATH_TRACING=/app/ee/databases/postgres/migrations/tracing/alembic.ini
-
-SUPERTOKENS_API_KEY=replace-me
-SUPERTOKENS_CONNECTION_URI=http://supertokens:3567
-
-# Third-party (optional)
-POSTHOG_API_KEY=phc_3urGRy5TL1HhaHnRYL0JSHxJxigRVackhphHtozUmdp
-
-GITHUB_OAUTH_CLIENT_ID=
-GITHUB_OAUTH_CLIENT_SECRET=
-
-GOOGLE_OAUTH_CLIENT_ID=
-GOOGLE_OAUTH_CLIENT_SECRET=
-
-NEW_RELIC_LICENSE_KEY=
-NRIA_LICENSE_KEY=
-
-LOOPS_API_KEY=
-
-SENDGRID_API_KEY=
-
-CRISP_WEBSITE_ID=
-
-STRIPE_API_KEY=
-STRIPE_WEBHOOK_SECRET=
-
-# Third-party - LLM (optional)
-ALEPHALPHA_API_KEY=
-ANTHROPIC_API_KEY=
-ANYSCALE_API_KEY=
-COHERE_API_KEY=
-DEEPINFRA_API_KEY=
-GEMINI_API_KEY=
-GROQ_API_KEY=
-MISTRAL_API_KEY=
-OPENAI_API_KEY=
-OPENROUTER_API_KEY=
-PERPLEXITYAI_API_KEY=
-TOGETHERAI_API_KEY=
\ No newline at end of file
diff --git a/hosting/docker-compose/oss/docker-compose.dev.yml b/hosting/docker-compose/oss/docker-compose.dev.yml
index 53c0eaef52..8489bc0f01 100644
--- a/hosting/docker-compose/oss/docker-compose.dev.yml
+++ b/hosting/docker-compose/oss/docker-compose.dev.yml
@@ -37,7 +37,6 @@ services:
volumes:
- ../../../api/entrypoint.py:/app/entrypoint.py
- ../../../api/oss:/app/oss
- - ../../../sdk:/sdk
env_file:
- ${ENV_FILE:-./.env.oss.dev}
@@ -84,7 +83,6 @@ services:
volumes:
- ../../../api/entrypoint.py:/app/entrypoint.py
- ../../../api/oss:/app/oss
- - ../../../sdk:/sdk
env_file:
- ${ENV_FILE:-./.env.oss.dev}
@@ -103,30 +101,6 @@ services:
command: >
watchmedo auto-restart --directory=/app/ --pattern=*.py --recursive -- celery -A entrypoint.celery_app worker --concurrency=1 --max-tasks-per-child=1 --prefetch-multiplier=1
- cron:
- image: agenta-oss-dev-api:latest
-
- volumes:
- #
- - ../../../api/oss/src/crons/queries.sh:/queries.sh
-
- env_file:
- - ${ENV_FILE:-./.env.oss.dev}
-
- depends_on:
- - postgres
- - api
-
- extra_hosts:
- - "host.docker.internal:host-gateway"
-
- restart: always
-
- networks:
- - agenta-network
-
- command: cron -f
-
alembic:
build:
context: ../../../api
@@ -135,7 +109,6 @@ services:
volumes:
- ../../../api/entrypoint.py:/app/entrypoint.py
- ../../../api/oss:/app/oss
- - ../../../sdk:/sdk
env_file:
- ${ENV_FILE:-./.env.oss.dev}
@@ -146,7 +119,7 @@ services:
networks:
- agenta-network
- command: sh -c "python -m oss.databases.postgres.migrations.runner"
+ command: sh -c "python -c 'from oss.databases.postgres.migrations.utils import split_core_and_tracing as split; split(); from oss.databases.postgres.migrations.core.utils import run_alembic_migration as run; run(); from oss.databases.postgres.migrations.tracing.utils import run_alembic_migration as run; run(); from oss.databases.postgres.migrations.utils import copy_nodes_from_core_to_tracing as copy; copy();'"
completion:
build:
diff --git a/hosting/docker-compose/oss/docker-compose.gh.ssl.yml b/hosting/docker-compose/oss/docker-compose.gh.ssl.yml
index c7dbd5c730..65101d4c0b 100644
--- a/hosting/docker-compose/oss/docker-compose.gh.ssl.yml
+++ b/hosting/docker-compose/oss/docker-compose.gh.ssl.yml
@@ -104,26 +104,6 @@ services:
condition: service_healthy
restart: always
- cron:
- image: agenta-oss-dev-api:latest
-
- env_file:
- - ${ENV_FILE:-./.env.oss.dev}
-
- depends_on:
- - postgres
- - api
-
- extra_hosts:
- - "host.docker.internal:host-gateway"
-
- restart: always
-
- networks:
- - agenta-network
-
- command: cron -f
-
alembic:
build:
context: ../../../api
@@ -137,8 +117,7 @@ services:
env_file:
- ${ENV_FILE:-./.env.oss.gh}
- command: sh -c "python -m oss.databases.postgres.migrations.runner"
-
+ command: sh -c "python -c 'from oss.databases.postgres.migrations.utils import split_core_and_tracing as split; split(); from oss.databases.postgres.migrations.core.utils import run_alembic_migration as run; run(); from oss.databases.postgres.migrations.tracing.utils import run_alembic_migration as run; run(); from oss.databases.postgres.migrations.utils import copy_nodes_from_core_to_tracing as copy; copy();'"
depends_on:
postgres:
condition: service_healthy
diff --git a/hosting/docker-compose/oss/docker-compose.gh.yml b/hosting/docker-compose/oss/docker-compose.gh.yml
index e079346a68..867296bb2f 100644
--- a/hosting/docker-compose/oss/docker-compose.gh.yml
+++ b/hosting/docker-compose/oss/docker-compose.gh.yml
@@ -80,26 +80,6 @@ services:
- redis
restart: always
- cron:
- image: agenta-oss-dev-api:latest
-
- env_file:
- - ${ENV_FILE:-./.env.oss.dev}
-
- depends_on:
- - postgres
- - api
-
- extra_hosts:
- - "host.docker.internal:host-gateway"
-
- restart: always
-
- networks:
- - agenta-network
-
- command: cron -f
-
alembic:
build:
context: ../../../api
@@ -113,7 +93,7 @@ services:
env_file:
- ${ENV_FILE:-./.env.oss.gh}
- command: sh -c "python -m oss.databases.postgres.migrations.runner"
+ command: sh -c "python -c 'from oss.databases.postgres.migrations.utils import split_core_and_tracing as split; split(); from oss.databases.postgres.migrations.core.utils import run_alembic_migration as run; run(); from oss.databases.postgres.migrations.tracing.utils import run_alembic_migration as run; run(); from oss.databases.postgres.migrations.utils import copy_nodes_from_core_to_tracing as copy; copy();'"
depends_on:
postgres:
diff --git a/hosting/docker-compose/oss/env.oss.dev.example b/hosting/docker-compose/oss/env.oss.dev.example
deleted file mode 100644
index 05e16eda8a..0000000000
--- a/hosting/docker-compose/oss/env.oss.dev.example
+++ /dev/null
@@ -1,103 +0,0 @@
-# First-party (required)
-AGENTA_LICENSE=oss
-AGENTA_API_URL=http://localhost/api
-AGENTA_WEB_URL=http://localhost
-AGENTA_SERVICES_URL=http://localhost/services
-AGENTA_AUTH_KEY=replace-me
-AGENTA_CRYPT_KEY=replace-me
-
-# First-party (registry & service)
-POSTGRES_PASSWORD=password
-POSTGRES_USERNAME=username
-
-# First-party (optional)
-AGENTA_AUTO_MIGRATIONS=true
-AGENTA_PRICING=
-AGENTA_DEMOS=
-DOCKER_NETWORK_MODE=bridge
-AGENTA_RUNTIME_PREFIX=
-AGENTA_SEND_EMAIL_FROM_ADDRESS=mail@example.com
-AGENTA_API_INTERNAL_URL=
-AGENTA_TELEMETRY_ENABLED=true
-AGENTA_SERVICE_MIDDLEWARE_CACHE_ENABLED=true
-AGENTA_OTLP_MAX_BATCH_BYTES=10485760
-
-# Third-party (required)
-TRAEFIK_DOMAIN=localhost
-TRAEFIK_PORT=80
-TRAEFIK_PROTOCOL=http
-TRAEFIK_UI_PORT=8080
-TRAEFIK_HTTPS_PORT=443
-
-REDIS_URL=redis://redis:6379/0
-RABBITMQ_DEFAULT_USER=guest
-RABBITMQ_DEFAULT_PASS=guest
-
-CELERY_BROKER_URL=amqp://guest@rabbitmq//
-CELERY_RESULT_BACKEND=redis://redis:6379/0
-
-
-POSTGRES_URI_SUPERTOKENS=postgresql://username:password@postgres:5432/agenta_oss_supertokens
-POSTGRES_URI_CORE=postgresql+asyncpg://username:password@postgres:5432/agenta_oss_core
-POSTGRES_URI_TRACING=postgresql+asyncpg://username:password@postgres:5432/agenta_oss_tracing
-
-ALEMBIC_CFG_PATH_CORE=/app/oss/databases/postgres/migrations/core/alembic.ini
-ALEMBIC_CFG_PATH_TRACING=/app/oss/databases/postgres/migrations/tracing/alembic.ini
-
-SUPERTOKENS_CONNECTION_URI=http://supertokens:3567
-
-POSTHOG_API_KEY=phc_hmVSxIjTW1REBHXgj2aw4HW9X6CXb6FzerBgP9XenC7
-
-# Third-party (required for TLS/SSL)
-AGENTA_SSL_DIR=
-
-# Third-party (optional)
-NGINX_PORT=80
-SUPERTOKENS_API_KEY=
-RABBITMQ_PORT=
-RABBITMQ_UI_PORT=
-REDIS_CACHE_HOST=
-REDIS_CACHE_PORT=
-POSTGRES_PORT=
-SUPERTOKENS_PORT=
-
-GOOGLE_OAUTH_CLIENT_ID=
-GOOGLE_OAUTH_CLIENT_SECRET=
-GITHUB_OAUTH_CLIENT_ID=
-GITHUB_OAUTH_CLIENT_SECRET=
-
-NEW_RELIC_LICENSE_KEY=
-NRIA_LICENSE_KEY=
-
-LOOPS_API_KEY=
-
-SENDGRID_API_KEY=
-
-CRISP_WEBSITE_ID=
-
-STRIPE_API_KEY=
-STRIPE_WEBHOOK_SECRET=
-
-# Third-party — LLM (optional)
-ALEPHALPHA_API_KEY=
-ANTHROPIC_API_KEY=
-ANYSCALE_API_KEY=
-COHERE_API_KEY=
-DEEPINFRA_API_KEY=
-GEMINI_API_KEY=
-GROQ_API_KEY=
-MISTRAL_API_KEY=
-OPENAI_API_KEY=
-OPENROUTER_API_KEY=
-PERPLEXITYAI_API_KEY=
-TOGETHERAI_API_KEY=
-
-# Legacy (deprecated, to be removed)
-AGENTA_PORT=80
-BARE_DOMAIN_NAME=localhost
-DOMAIN_NAME=http://localhost
-WEBSITE_DOMAIN_NAME=http://localhost
-SERVICE_URL_TEMPLATE=http://localhost:80/services/{path}
-POSTGRES_DB=agenta_oss
-POSTGRES_URI=postgresql+asyncpg://username:password@postgres:5432/agenta_oss
-ALEMBIC_CFG_PATH=/app/oss/databases/postgres/migrations/alembic.oss.ini
diff --git a/hosting/docker-compose/oss/env.oss.gh.example b/hosting/docker-compose/oss/env.oss.gh.example
deleted file mode 100644
index 05e16eda8a..0000000000
--- a/hosting/docker-compose/oss/env.oss.gh.example
+++ /dev/null
@@ -1,103 +0,0 @@
-# First-party (required)
-AGENTA_LICENSE=oss
-AGENTA_API_URL=http://localhost/api
-AGENTA_WEB_URL=http://localhost
-AGENTA_SERVICES_URL=http://localhost/services
-AGENTA_AUTH_KEY=replace-me
-AGENTA_CRYPT_KEY=replace-me
-
-# First-party (registry & service)
-POSTGRES_PASSWORD=password
-POSTGRES_USERNAME=username
-
-# First-party (optional)
-AGENTA_AUTO_MIGRATIONS=true
-AGENTA_PRICING=
-AGENTA_DEMOS=
-DOCKER_NETWORK_MODE=bridge
-AGENTA_RUNTIME_PREFIX=
-AGENTA_SEND_EMAIL_FROM_ADDRESS=mail@example.com
-AGENTA_API_INTERNAL_URL=
-AGENTA_TELEMETRY_ENABLED=true
-AGENTA_SERVICE_MIDDLEWARE_CACHE_ENABLED=true
-AGENTA_OTLP_MAX_BATCH_BYTES=10485760
-
-# Third-party (required)
-TRAEFIK_DOMAIN=localhost
-TRAEFIK_PORT=80
-TRAEFIK_PROTOCOL=http
-TRAEFIK_UI_PORT=8080
-TRAEFIK_HTTPS_PORT=443
-
-REDIS_URL=redis://redis:6379/0
-RABBITMQ_DEFAULT_USER=guest
-RABBITMQ_DEFAULT_PASS=guest
-
-CELERY_BROKER_URL=amqp://guest@rabbitmq//
-CELERY_RESULT_BACKEND=redis://redis:6379/0
-
-
-POSTGRES_URI_SUPERTOKENS=postgresql://username:password@postgres:5432/agenta_oss_supertokens
-POSTGRES_URI_CORE=postgresql+asyncpg://username:password@postgres:5432/agenta_oss_core
-POSTGRES_URI_TRACING=postgresql+asyncpg://username:password@postgres:5432/agenta_oss_tracing
-
-ALEMBIC_CFG_PATH_CORE=/app/oss/databases/postgres/migrations/core/alembic.ini
-ALEMBIC_CFG_PATH_TRACING=/app/oss/databases/postgres/migrations/tracing/alembic.ini
-
-SUPERTOKENS_CONNECTION_URI=http://supertokens:3567
-
-POSTHOG_API_KEY=phc_hmVSxIjTW1REBHXgj2aw4HW9X6CXb6FzerBgP9XenC7
-
-# Third-party (required for TLS/SSL)
-AGENTA_SSL_DIR=
-
-# Third-party (optional)
-NGINX_PORT=80
-SUPERTOKENS_API_KEY=
-RABBITMQ_PORT=
-RABBITMQ_UI_PORT=
-REDIS_CACHE_HOST=
-REDIS_CACHE_PORT=
-POSTGRES_PORT=
-SUPERTOKENS_PORT=
-
-GOOGLE_OAUTH_CLIENT_ID=
-GOOGLE_OAUTH_CLIENT_SECRET=
-GITHUB_OAUTH_CLIENT_ID=
-GITHUB_OAUTH_CLIENT_SECRET=
-
-NEW_RELIC_LICENSE_KEY=
-NRIA_LICENSE_KEY=
-
-LOOPS_API_KEY=
-
-SENDGRID_API_KEY=
-
-CRISP_WEBSITE_ID=
-
-STRIPE_API_KEY=
-STRIPE_WEBHOOK_SECRET=
-
-# Third-party — LLM (optional)
-ALEPHALPHA_API_KEY=
-ANTHROPIC_API_KEY=
-ANYSCALE_API_KEY=
-COHERE_API_KEY=
-DEEPINFRA_API_KEY=
-GEMINI_API_KEY=
-GROQ_API_KEY=
-MISTRAL_API_KEY=
-OPENAI_API_KEY=
-OPENROUTER_API_KEY=
-PERPLEXITYAI_API_KEY=
-TOGETHERAI_API_KEY=
-
-# Legacy (deprecated, to be removed)
-AGENTA_PORT=80
-BARE_DOMAIN_NAME=localhost
-DOMAIN_NAME=http://localhost
-WEBSITE_DOMAIN_NAME=http://localhost
-SERVICE_URL_TEMPLATE=http://localhost:80/services/{path}
-POSTGRES_DB=agenta_oss
-POSTGRES_URI=postgresql+asyncpg://username:password@postgres:5432/agenta_oss
-ALEMBIC_CFG_PATH=/app/oss/databases/postgres/migrations/alembic.oss.ini
diff --git a/hosting/old/gcp/agenta-instance.tf b/hosting/gcp/agenta-instance.tf
similarity index 100%
rename from hosting/old/gcp/agenta-instance.tf
rename to hosting/gcp/agenta-instance.tf
diff --git a/hosting/old/gcp/credentials.json b/hosting/gcp/credentials.json
similarity index 100%
rename from hosting/old/gcp/credentials.json
rename to hosting/gcp/credentials.json
diff --git a/hosting/old/gcp/main.tf b/hosting/gcp/main.tf
similarity index 100%
rename from hosting/old/gcp/main.tf
rename to hosting/gcp/main.tf
diff --git a/sdk/README.md b/sdk/README.md
index d07ac59e52..4424f046d2 100644
--- a/sdk/README.md
+++ b/sdk/README.md
@@ -2,12 +2,11 @@