Add reference / best practice for AWS + Azure infrastructure #1403

katmayb · 2025-11-13T17:26:42Z

Fixes DOC-442

Adds reference / best practice pages on AWS + Azure infrastructure setup for self-hosted and hybrid in LangSmith.

Decided on a different placement from initial discussion, for the following reasons:

These pages belong under "Self-hosted cloud architecture" rather than "Setup guides" because they provide reference architecture and best practices, not step-by-step installation instructions.

Structure allows for:

Cloud-specific best practices before beginning setup
Reference throughout their deployment lifecycle
Architectural guidance separate from procedural installation steps
Content on using self-hosted or hybrid models

Preview

AWS reference
Azure reference

github-actions · 2025-11-13T17:27:53Z

Mintlify preview ID generated: preview-selfho-1763054841-914688a

github-actions · 2025-11-13T17:39:19Z

Mintlify preview ID generated: preview-selfho-1763055531-4b07c43

katmayb · 2025-11-13T18:19:14Z

src/langsmith/aws-self-hosted.mdx

+- <Icon icon="globe" /> **Ingress & networking**: Requests enter via [Amazon Application Load Balancer (ALB)](https://aws.amazon.com/elasticloadbalancing/application-load-balancer/) within your [VPC](https://aws.amazon.com/vpc/), secured using [AWS WAF](https://aws.amazon.com/waf/) and [IAM](https://aws.amazon.com/iam/)-based authentication.
+- <Icon icon="cube" /> **Frontend & backend services:** Containers run on [Amazon EKS](https://aws.amazon.com/eks/), orchestrated behind the ALB. Nginx routes requests to the LangSmith frontend, backend, and queue workers.
+- <Icon icon="database" /> **Storage & databases:**
+  - [Amazon RDS for PostgreSQL](https://aws.amazon.com/rds/postgresql/): metadata, projects, users.
+  - [Amazon ElastiCache (Redis)](https://aws.amazon.com/elasticache/redis/): caching and job queues.
+  - ClickHouse + [Amazon EBS](https://aws.amazon.com/ebs/): analytics and trace storage.
+  - [Amazon S3](https://aws.amazon.com/s3/): object storage for trace artifacts and telemetry.
+- <Icon icon="sparkles" /> **LLM integration:** Optionally proxy requests to [Amazon Bedrock](https://aws.amazon.com/bedrock/) or [Amazon SageMaker](https://aws.amazon.com/sagemaker/) for LLM inference.
+- <Icon icon="chart-line" /> **Monitoring & observability:** Integrated with [Amazon CloudWatch](https://aws.amazon.com/cloudwatch/) and [LangSmith Beacon](/langsmith/self-host-egress) (for self-hosted telemetry opt-in).


@rahul-langchain I noticed you had a section like this in the Azure google doc. I've essentially copied that into AWS but adapted — can you check whether this is all correct?

github-actions · 2025-11-14T16:59:12Z

Mintlify preview ID generated: preview-selfho-1763139517-b0f2587

github-actions · 2025-11-18T03:07:05Z

Mintlify preview ID generated: preview-selfho-1763435197-a35c7ce

katmayb · 2025-11-18T13:31:56Z

src/langsmith/aws-self-hosted.mdx

+- <Icon icon="sparkles" /> **LLM integration:** Optionally proxy requests to [Amazon Bedrock](https://aws.amazon.com/bedrock/) or [Amazon SageMaker](https://aws.amazon.com/sagemaker/) for LLM inference.
+- <Icon icon="chart-line" /> **Monitoring & observability:** Integrate with [Amazon CloudWatch](https://aws.amazon.com/cloudwatch/)

- [LangSmith Observability and Evaluation](/langsmith/self-hosted#self-host-langsmith-observability-and-evaluation): Deploy the UI and API services (frontend, backend, platform backend, playground, queue workers, and ACE). Use external AWS managed services for RDS PostgreSQL, ElastiCache, and S3.


I think we should consider linking out, or having briefer mentions of the self-hosted types — while it is in other places in the docs, users can land on a page from search and then they have no context.

Comment applies to Azure + AWS pages. In my opinion you could actually keep the page description and add a small addendum:

This page provides AWS-specific architecture patterns, service recommendations, and best practices for deploying and operating LangSmith on AWS. For more details on LangSmith self-hosted models, refer to the self-hosted overview page.

katmayb · 2025-11-18T13:34:58Z

src/langsmith/aws-self-hosted.mdx


-When running LangSmith on [Amazon Web Services (AWS)](https://aws.amazon.com/), you can set up in either [self-hosted](/langsmith/self-hosted) or [hybrid](/langsmith/hybrid) mode. In both cases, your workloads run on AWS infrastructure within your account, allowing you to use AWS managed services while maintaining control over your data and compute resources.
-
-This page provides AWS-specific architecture patterns, service recommendations, and best practices for deploying and operating LangSmith on AWS.


See comment below re this paragraph

github-actions · 2025-11-18T13:47:46Z

Mintlify preview ID generated: preview-selfho-1763473632-8f0ba30

github-actions · 2025-11-19T02:11:00Z

Mintlify preview ID generated: preview-selfho-1763518232-00a0eab

romain-priour-lc · 2025-11-20T17:03:43Z

src/langsmith/aws-self-hosted.mdx


-When running LangSmith on [Amazon Web Services (AWS)](https://aws.amazon.com/), you can set up in either [full self-hosted](/langsmith/self-hosted) or [hybrid](/langsmith/hybrid) mode. This page provides AWS-specific architecture patterns, service recommendations, and best practices for deploying and operating LangSmith on AWS.
+When running LangSmith on [Amazon Web Services (AWS)](https://aws.amazon.com/), you can set up in either [full self-hosted](/langsmith/self-hosted) or [hybrid](/langsmith/hybrid) mode. Full self-hosted mode deploys a complete LangSmith platform with observability functionality as well as the option to create agent deployments. Hybrid mode entails just the infrastructure to run LangSmith-managed agents in a data plane within your cloud, while our SaaS provides the control plane and observability functionality.
+


What does "LangSmith-managed agent" mean?

romain-priour-lc · 2025-11-20T17:04:35Z

src/langsmith/aws-self-hosted.mdx

 - <Icon icon="database" /> **Storage & databases:**
-  - [Amazon RDS for PostgreSQL](https://aws.amazon.com/rds/postgresql/): metadata, projects, users.
-  - [Amazon ElastiCache (Redis)](https://aws.amazon.com/elasticache/redis/): caching and job queues.
+  - [Amazon RDS for PostgreSQL](https://aws.amazon.com/rds/postgresql/), optionally using [Aurora](https://aws.amazon.com/rds/aurora/): metadata, projects, users, and short-term and long-term memory for deployed agents. LangSmith supports Postgres version 14 or higher.


"or Aurora" is more clear to me, "optionally using" sounds like you have an option to use both at once

I thought Aurora was on top of RDS?

romain-priour-lc · 2025-11-20T17:05:19Z

src/langsmith/aws-self-hosted.mdx

-  - [Amazon RDS for PostgreSQL](https://aws.amazon.com/rds/postgresql/): metadata, projects, users.
-  - [Amazon ElastiCache (Redis)](https://aws.amazon.com/elasticache/redis/): caching and job queues.
+  - [Amazon RDS for PostgreSQL](https://aws.amazon.com/rds/postgresql/), optionally using [Aurora](https://aws.amazon.com/rds/aurora/): metadata, projects, users, and short-term and long-term memory for deployed agents. LangSmith supports Postgres version 14 or higher.
+  - [Amazon ElastiCache (Redis)](https://aws.amazon.com/elasticache/redis/): caching and job queues. ElastiCache must be in single instance mode, running open-source Redis version 5 or higher.


nit: official elasticache redis open source product branch is "Redis OSS"

"running Redis OSS version 5 or higher"

katmayb requested a review from lnhsingh as a code owner November 13, 2025 17:26

github-actions bot added the langsmith For docs changes to LangSmith label Nov 13, 2025

Add reference / best practice for AWS + Azure infrastructure

cedd495

katmayb force-pushed the self-hosted-aws-azure-options branch from 57baa47 to cedd495 Compare November 13, 2025 17:38

katmayb commented Nov 13, 2025

View reviewed changes

Update diagrams

a307c73

katmayb requested review from langchain-infra and rahul-langchain November 14, 2025 16:58

[Rahul] Content touch ups

accb018

katmayb commented Nov 18, 2025

View reviewed changes

Updated images

fdb0b5d

Tweaks

8c88702

romain-priour-lc reviewed Nov 20, 2025

View reviewed changes


		When running LangSmith on [Amazon Web Services (AWS)](https://aws.amazon.com/), you can set up in either [self-hosted](/langsmith/self-hosted) or [hybrid](/langsmith/hybrid) mode. In both cases, your workloads run on AWS infrastructure within your account, allowing you to use AWS managed services while maintaining control over your data and compute resources.

		This page provides AWS-specific architecture patterns, service recommendations, and best practices for deploying and operating LangSmith on AWS.

Add reference / best practice for AWS + Azure infrastructure #1403

Are you sure you want to change the base?

Add reference / best practice for AWS + Azure infrastructure #1403

Conversation

katmayb commented Nov 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Preview

Uh oh!

github-actions bot commented Nov 13, 2025

Uh oh!

github-actions bot commented Nov 13, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Nov 14, 2025

Uh oh!

github-actions bot commented Nov 18, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Nov 18, 2025

Uh oh!

github-actions bot commented Nov 19, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

romain-priour-lc Nov 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

katmayb commented Nov 13, 2025 •

edited

Loading

romain-priour-lc Nov 20, 2025 •

edited

Loading