From b9b0df5058a767604ebe5b40baa1027d1f45b7d7 Mon Sep 17 00:00:00 2001 From: Maedah Batool Date: Thu, 3 Apr 2025 14:07:46 -0700 Subject: [PATCH 01/14] Docs for Sourcegraph Architecture --- docs/admin/architecture.mdx | 183 ++++++++++++++++++++++++++++++++++++ src/data/navigation.ts | 1 + 2 files changed, 184 insertions(+) create mode 100644 docs/admin/architecture.mdx diff --git a/docs/admin/architecture.mdx b/docs/admin/architecture.mdx new file mode 100644 index 000000000..2712cdaf2 --- /dev/null +++ b/docs/admin/architecture.mdx @@ -0,0 +1,183 @@ +# Sourcegraph Architecture + +

This document provides a high-level overview of Sourcegraph's architecture so you can understand how our systems fit together.

+ +## Diagram + +![sourcegraph-architecture](https://storage.googleapis.com/sourcegraph-assets/Docs/sg-architecture.svg) + +## Repository syncing + +At its core, Sourcegraph maintains a persistent cache of all repositories that are connected to it. It is persistent because this data is critical for Sourcegraph to function. Still, it is ultimately a cache because the code host is the source of truth, and our cache is eventually consistent. + +- `gitserver` is the sharded service that stores repositories and makes them accessible to other Sourcegraph services +- `repo-updater` is the singleton service responsible for ensuring all repositories in gitserver are as up-to-date as possible while respecting code host rate limits. It is also responsible for syncing repository metadata from the code host that is stored in the repo table of our Postgres database + +## Permission syncing + +Repository permissions are mirrored from code hosts to Sourcegraph by default. This builds the foundation of Sourcegraph authorization for repositories to ensure users see consistent content on code hosts. Currently, the background permissions syncer resides in the repo-updater. + +To learn more about how repository permissions are synchronized in the background, read about [permission syncing](/admin/permissions/syncing). + +## Settings cascade + +Sourcegraph offers the flexibility of customizing user settings. A single user's settings are generally the result of merging user settings, organization settings, and global settings. Each of these is referred to as a settings subject, which is part of the settings cascade. They are all exposed to GraphQL. + +## Search + +Developers can search for the entire codebase that is connected to their Sourcegraph instance. + +By default, Sourcegraph uses `zoekt` to create a trigram index of the default branch of every repository, which makes searches fast. This trigram index is why Sourcegraph search is more powerful and faster than what is usually provided by code hosts. + +- [zoekt-indexserver](https://sourcegraph.com/github.com/sourcegraph/zoekt/-/tree/cmd/zoekt-sourcegraph-indexserver) +- [zoekt-webserver](https://sourcegraph.com/github.com/sourcegraph/zoekt/-/tree/cmd/zoekt-webserver) + +Sourcegraph also has a fast search path for code that isn't indexed yet or will never be indexed (for example, code that is not on a default branch). Indexing every branch of every repository isn't a pragmatic use of resources for most customers, so this decision balances optimizing the common case (searching all default branches) with space savings (not indexing everything). + +- `searcher` implements the non-indexed search +- Syntax highlighting for any code view, including search results, is provided by `Syntect` server + +You can learn more about search in the [Code Search docs](/code-search). + +## Code Navigation + +Code Navigation surfaces data (for example, doc comments for a symbol) and actions (for example, go to definition, find references) based on our semantic understanding of code (unlike search, which is completely text-based). + +By default, Sourcegraph provides [search-based code navigation](/code-search/code-navigation/search_based_code_navigation). This reuses all the architecture that makes search fast, but it can result in false positives (for example, finding two definitions for a symbol or references that aren't actually references) or false negatives (for example, not being able to find the definition or all references). + +This is the default because it works with no extra configuration and is good for many use cases and languages. We support many languages this way because it only requires writing a few regular expressions. + +With some setup, customers can enable [precise code navigation](/code-search/code-navigation/precise_code_navigation). Repositories add a step to their build pipeline that computes the index for that code revision and uploads it to Sourcegraph. We must write language-specific indexers, so adding precise code navigation support for new languages is a non-trivial task. + +To learn more about [Code Navigation, read the docs here](/code-search/code-navigation). + +### Dependencies + +- Search: Symbol search is used for basic code navigation +- Sourcegraph extension API: Hover and definition providers +- Native integrations (for code hosts): UI of hover tooltips on code hosts + +## Batch Changes + +Batch Changes creates and manages large-scale code changes across projects, repositories, and code hosts. + +To create a batch change, users write a [batch spec](/batch-changes/batch-spec-yaml-reference), which is a YAML file that specifies the changes that should be performed and the repositories that they should be performed upon — either through a Sourcegraph search or by declaring them directly. This spec is then executed by [src-cli](/cli/references/batch) on the user's machine (or in CI or some other environment controlled by the user), which results in changeset specs that are sent to Sourcegraph. Sourcegraph then applies these changeset specs to create one or more changesets per repository. (Depending on the code host, a changeset is a pull request or merge request.) + +Once created, Sourcegraph monitors changesets, and their current review and CI status can be viewed on the batch change page. This provides a single pane of glass view of all the changesets created as part of the batch change. The batch change can be updated at any time by re-applying the original batch spec: this will transparently add or remove changesets in repositories that now match or don't match the original search as needed. + +Read the [Batch Changes](/batch-changes) docs to learn more. + +### Dependencies + +- src-cli: Batch changes are currently executed client-side through the `src` CLI +- Search: Repositories in which batch specs need to be executed are resolved through the search API + +## Code insights + +Code insights surface higher-level, aggregated information to leaders in engineering organizations in dashboards. For example, code insights can track the number of matches of a search query over time, the number of code navigation diagnostic warnings in a codebase, or the usage of different programming languages. Sample use cases for this are tracking migrations, the usage of libraries across an organization, tech debt, code base health, and much more. + +Code Insights persist in a separate database called `codeinsights-db`. The web application interacts with the backend through a [GraphQL API](/api/graphql). + +Code Insights uses data from the `frontend` database for repository metadata and repository permissions to filter time series data. + +Code Insights can generate data in the background or just-in-time when viewing charts. This decision is currently enforced in the product, depending on the type and scope of the insight. For code insights being run just-in-time in the client, the performance of code insights is bound to the performance of the underlying data source. These insights are relatively fast as long as the scope doesn't include many repositories (or large monorepos), but performance degrades when trying to include a lot of repositories. Insights that are processed in the background are rate-limited and will perform approximately 28,000 queries per hour when fully saturated on default settings. + +There is also a feature flag left over from the original development of the early-stage product that we retained in case a customer who doesn't purchase it ever has a justified need to disable insights. You can set `"experimentalFeatures": { "codeInsights": false }` in your settings to disable insights. + +You can learn more in the [Code Insights](/code_insights) docs. + +### Dependencies + +- Search: + - GraphQL API for text search, in particular `search()`, `matchCount`, `stats.languages` + - Query syntax: Code insights "construct" search queries programmatically + - Exhaustive search (with `count:all/count:999999` operator) + - Historical search (= unindexed search, currently) + - Commit search to find historical commits to search over +- Repository Syncing: The code insights backend has direct dependencies on `gitserver` and `repo-updater` +- Permission syncing: The code insights backend depends on synced repository permissions for access control +- Settings cascade: + - Insights and dashboard configuration are stored in user, organization, and global settings. This will change in the future and is planned to be moved to the database + - Insights contributed by extensions are configured through settings (this will stay the same) + +## Code Monitoring + +Code Monitoring allows users to get notified of changes to their codebase. + +Users can view, edit, and create code monitors through the code monitoring UI. A code monitor comprises a trigger and one or more actions. + +The **trigger** watches for new data; if there is new data, we call this an event. The only supported trigger is a search query of `type:diff` or `type:commit`, run every five minutes by the Go backend with an automatically added `after:` parameter narrowing down the diffs/commits that should be searched. The monitor's configured actions are run when this query returns a non-zero number of results. + +The actions are run in response to a trigger event. For now, the only supported action is an email notification to the primary email address of the code monitor's owner. To work, `email.address` and `email.smtp` must be configured in the site configuration. Code monitoring actions will be extended in the future to support webhooks. + +Learn more in the [Code Monitoring](/code_monitoring) docs. + +### Dependencies + +- Search: Diff and commit search triggers + +## Browser extensions + +The [Sourcegraph browser extensions](/integration/browser_extension) bring the features of Sourcegraph directly into the UI of code hosts such as GitHub, GitLab, and Bitbucket. + +With the Sourcegraph browser extension installed, users get Sourcegraph features (including code navigation) on their code host while browsing code, viewing diffs, or reviewing pull requests. + +This lets users get value from Sourcegraph without leaving existing workflows on their code host. It also gives them a convenient way to jump into Sourcegraph anytime (by using the **Open in Sourcegraph** button on any repository or file). The browser extension also adds an address bar search shortcut, allowing you to search on Sourcegraph directly from the browser address bar. + +## Native integrations (for code hosts) + +Native integrations bring Sourcegraph features directly into the UI of code hosts, similar to browser extensions. + +Instead of requiring a browser extension, native integrations inject a script by extending the code host directly (for example, using the code host's plugin architecture). The advantage is that Sourcegraph can be enabled for all users of a code host instance without any action required from each user. + +Learn more about our [Code host integrations in the docs](/admin/code_hosts). + +#### Dependencies + +- Repository Syncing: Uses the GraphQL API to resolve repositories and revisions on code hosts +- Search: Query transformer API hooks into search in the web app +- Settings cascade: Which extensions are enabled, and which configurations for extensions are stored in the settings. Extensions may also change settings + +## src-cli + +`src-cli`, or `src`, is a command line tool that users can run locally to interact with Sourcegraph. + +`src-cli` is written in Go and distributed as a standalone binary for Windows, macOS, and Linux. Its features include running searches, managing Sourcegraph, and executing batch changes. `src-cli` is integral to the batch changes product. + +Note that `src-cli` is not contained within the Sourcegraph monorepo and has its release cadence. + +Learn more about `src-cli` in the [docs](/cli) and [GitHub repo](https://github.com/sourcegraph/src-cli). + +### Dependencies + +- Search: GraphQL API +- Batch Changes: GraphQL API + +## Deployment + +Sourcegraph is deployable via three supported methods: + +- **[Kubernetes](/admin/deploy/kubernetes)** is intended for all medium to large-scale production deployments that require fault tolerance and high availability. For advanced users only with significant kubernetes experience required. This deployment method is developed in deploy-sourcegraph +- **[Docker Compose](/admin/deploy/docker-compose)** is intended to be used for small to medium production deployments, with some customization available. Easy to set up with basic infrastructure and docker knowledge required. A variation on this is the pure-Docker option. Both of these deployment methods are developed in deploy-sourcegraph-docker +- **[Docker Single Container](/admin/deploy/docker-single-container)** for small environments on a single server. It is the easiest and quickest to set up with a single command. Little infrastructure knowledge is required. This deployment method is developed in cmd/server + +The [resource estimator](/admin/deploy/resource_estimator#sourcegraph-resource-estimator) can guide you on the requirements for each deployment type. + +## Observability + +Observability encapsulates the monitoring and debugging of Sourcegraph deployments. Sourcegraph is designed and ships several observability tools and out-of-the-box capabilities to enable visibility into the health and state of a Sourcegraph deployment. + +Monitoring includes [metrics and dashboards](/admin/observability/metrics), [alerting](/admin/observability/alerting), and [health checking](/admin/observability/health_checks) capabilities. + +- grafana is the frontend for service metrics and ships with customized dashboards for Sourcegraph services +- prometheus handles the scraping of service metrics and ships with recording rules, alert rules, and alerting capabilities +- cadvisor provides per-container performance metrics (scraped by Prometheus) in most Sourcegraph environments +- Each Sourcegraph service provides health checks + +Debugging includes [tracing](/admin/observability/tracing) and [logging](/admin/observability/logs). + +- jaeger is the distributed tracing service used by Sourcegraph +- jaeger-agent is a sidecar used in Kubernetes deployments for collecting traces from services +- Each Sourcegraph service provides logs + +Learn more in our [Observability docs](/admin/observability). diff --git a/src/data/navigation.ts b/src/data/navigation.ts index ca6881ad1..e84763ae9 100644 --- a/src/data/navigation.ts +++ b/src/data/navigation.ts @@ -235,6 +235,7 @@ export const navigation: NavigationItem[] = [ href: "/admin", sections: [ { title: "Deploy", href: "/admin/deploy" }, + { title: "Architecture", href: "/admin/architecture" }, { title: "Upgrade", href: "/admin/updates" }, { title: "Configuration", href: "/admin/config" }, { title: "Licensing", href: "/admin/licensing" }, From 252a7f9dc6ba49041d01f78c59161bed1469c5cf Mon Sep 17 00:00:00 2001 From: Maedah Batool Date: Fri, 4 Apr 2025 08:58:42 -0700 Subject: [PATCH 02/14] Update docs/admin/architecture.mdx Co-authored-by: Louis Jarvis <31862633+loujar@users.noreply.github.com> --- docs/admin/architecture.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/admin/architecture.mdx b/docs/admin/architecture.mdx index 2712cdaf2..aa6d736cd 100644 --- a/docs/admin/architecture.mdx +++ b/docs/admin/architecture.mdx @@ -41,7 +41,7 @@ You can learn more about search in the [Code Search docs](/code-search). ## Code Navigation -Code Navigation surfaces data (for example, doc comments for a symbol) and actions (for example, go to definition, find references) based on our semantic understanding of code (unlike search, which is completely text-based). +Unlike Search (which is completely text-based), Code Navigation surfaces data such as doc comments for a symbol and actions such as the "go to definition" or "find references" features based on our semantic understanding of code. By default, Sourcegraph provides [search-based code navigation](/code-search/code-navigation/search_based_code_navigation). This reuses all the architecture that makes search fast, but it can result in false positives (for example, finding two definitions for a symbol or references that aren't actually references) or false negatives (for example, not being able to find the definition or all references). From 37f0209bb35f399e9af09feb27e61ca570af3cf6 Mon Sep 17 00:00:00 2001 From: Maedah Batool Date: Fri, 4 Apr 2025 10:41:21 -0700 Subject: [PATCH 03/14] Update docs/admin/architecture.mdx Co-authored-by: Andrew Norrish <110418926+anorrish@users.noreply.github.com> --- docs/admin/architecture.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/admin/architecture.mdx b/docs/admin/architecture.mdx index aa6d736cd..3c1fc45ba 100644 --- a/docs/admin/architecture.mdx +++ b/docs/admin/architecture.mdx @@ -61,7 +61,7 @@ To learn more about [Code Navigation, read the docs here](/code-search/code-navi Batch Changes creates and manages large-scale code changes across projects, repositories, and code hosts. -To create a batch change, users write a [batch spec](/batch-changes/batch-spec-yaml-reference), which is a YAML file that specifies the changes that should be performed and the repositories that they should be performed upon — either through a Sourcegraph search or by declaring them directly. This spec is then executed by [src-cli](/cli/references/batch) on the user's machine (or in CI or some other environment controlled by the user), which results in changeset specs that are sent to Sourcegraph. Sourcegraph then applies these changeset specs to create one or more changesets per repository. (Depending on the code host, a changeset is a pull request or merge request.) +To create a batch change, users write a [batch spec](/batch-changes/batch-spec-yaml-reference), which is a YAML file that specifies the changes that should be performed and the repositories that they should be performed upon — either through a Sourcegraph search or by declaring them directly. This spec is then executed by [src-cli](/cli/references/batch) on the user's machine, in CI or some other environment controlled by the user, or directly within the Sourcegraph UI by enabling Server-Side Batch Changes via executors. This results in changeset specs that are sent to Sourcegraph. Sourcegraph then applies these changeset specs to create one or more changesets per repository. (Depending on the code host, a changeset is a pull request or merge request.) Once created, Sourcegraph monitors changesets, and their current review and CI status can be viewed on the batch change page. This provides a single pane of glass view of all the changesets created as part of the batch change. The batch change can be updated at any time by re-applying the original batch spec: this will transparently add or remove changesets in repositories that now match or don't match the original search as needed. From a72bac2b3b62c754a6d4708095929b7f940e3fab Mon Sep 17 00:00:00 2001 From: Maedah Batool Date: Tue, 22 Apr 2025 10:45:50 -0700 Subject: [PATCH 04/14] Update docs/admin/architecture.mdx Co-authored-by: Louis Jarvis <31862633+loujar@users.noreply.github.com> --- docs/admin/architecture.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/admin/architecture.mdx b/docs/admin/architecture.mdx index 3c1fc45ba..b8255d29f 100644 --- a/docs/admin/architecture.mdx +++ b/docs/admin/architecture.mdx @@ -110,7 +110,7 @@ The **trigger** watches for new data; if there is new data, we call this an even The actions are run in response to a trigger event. For now, the only supported action is an email notification to the primary email address of the code monitor's owner. To work, `email.address` and `email.smtp` must be configured in the site configuration. Code monitoring actions will be extended in the future to support webhooks. -Learn more in the [Code Monitoring](/code_monitoring) docs. +Learn more in the [Code Monitoring docs](/code_monitoring) ### Dependencies From ce466c28c6b3f867257da0306e3b20a56b49bbce Mon Sep 17 00:00:00 2001 From: Maedah Batool Date: Tue, 22 Apr 2025 10:46:03 -0700 Subject: [PATCH 05/14] Update docs/admin/architecture.mdx Co-authored-by: Louis Jarvis <31862633+loujar@users.noreply.github.com> --- docs/admin/architecture.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/admin/architecture.mdx b/docs/admin/architecture.mdx index b8255d29f..550a4e3a0 100644 --- a/docs/admin/architecture.mdx +++ b/docs/admin/architecture.mdx @@ -17,7 +17,7 @@ At its core, Sourcegraph maintains a persistent cache of all repositories that a Repository permissions are mirrored from code hosts to Sourcegraph by default. This builds the foundation of Sourcegraph authorization for repositories to ensure users see consistent content on code hosts. Currently, the background permissions syncer resides in the repo-updater. -To learn more about how repository permissions are synchronized in the background, read about [permission syncing](/admin/permissions/syncing). +Learn more in the [Permission Syncing docs](/admin/permissions/syncing) ## Settings cascade From 1d8c9ddccbdc5b0fda7e0e5a8ae79de4dae65f01 Mon Sep 17 00:00:00 2001 From: Maedah Batool Date: Tue, 22 Apr 2025 10:46:15 -0700 Subject: [PATCH 06/14] Update docs/admin/architecture.mdx Co-authored-by: Louis Jarvis <31862633+loujar@users.noreply.github.com> --- docs/admin/architecture.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/admin/architecture.mdx b/docs/admin/architecture.mdx index 550a4e3a0..5a1bdf90a 100644 --- a/docs/admin/architecture.mdx +++ b/docs/admin/architecture.mdx @@ -37,7 +37,7 @@ Sourcegraph also has a fast search path for code that isn't indexed yet or will - `searcher` implements the non-indexed search - Syntax highlighting for any code view, including search results, is provided by `Syntect` server -You can learn more about search in the [Code Search docs](/code-search). +Learn more in the [Code Search docs](/code-search) ## Code Navigation From a750912cd4d017814663abb00bc5ccd1045a1a77 Mon Sep 17 00:00:00 2001 From: Maedah Batool Date: Tue, 22 Apr 2025 10:46:23 -0700 Subject: [PATCH 07/14] Update docs/admin/architecture.mdx Co-authored-by: Louis Jarvis <31862633+loujar@users.noreply.github.com> --- docs/admin/architecture.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/admin/architecture.mdx b/docs/admin/architecture.mdx index 5a1bdf90a..e7f4caeed 100644 --- a/docs/admin/architecture.mdx +++ b/docs/admin/architecture.mdx @@ -49,7 +49,7 @@ This is the default because it works with no extra configuration and is good for With some setup, customers can enable [precise code navigation](/code-search/code-navigation/precise_code_navigation). Repositories add a step to their build pipeline that computes the index for that code revision and uploads it to Sourcegraph. We must write language-specific indexers, so adding precise code navigation support for new languages is a non-trivial task. -To learn more about [Code Navigation, read the docs here](/code-search/code-navigation). +Learn more in the [Code Navigation docs](/code-search/code-navigation) ### Dependencies From b1bd28aa252eaf6c3fa64fe3d32526f8d8b6a297 Mon Sep 17 00:00:00 2001 From: Maedah Batool Date: Tue, 22 Apr 2025 10:46:34 -0700 Subject: [PATCH 08/14] Update docs/admin/architecture.mdx Co-authored-by: Louis Jarvis <31862633+loujar@users.noreply.github.com> --- docs/admin/architecture.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/admin/architecture.mdx b/docs/admin/architecture.mdx index e7f4caeed..622f56b67 100644 --- a/docs/admin/architecture.mdx +++ b/docs/admin/architecture.mdx @@ -120,7 +120,7 @@ The actions are run in response to a trigger event. For now, the only supported The [Sourcegraph browser extensions](/integration/browser_extension) bring the features of Sourcegraph directly into the UI of code hosts such as GitHub, GitLab, and Bitbucket. -With the Sourcegraph browser extension installed, users get Sourcegraph features (including code navigation) on their code host while browsing code, viewing diffs, or reviewing pull requests. +With the Sourcegraph browser extension installed, users get Sourcegraph features (including Code Navigation) on their code host while browsing code, viewing diffs, or reviewing pull requests. This lets users get value from Sourcegraph without leaving existing workflows on their code host. It also gives them a convenient way to jump into Sourcegraph anytime (by using the **Open in Sourcegraph** button on any repository or file). The browser extension also adds an address bar search shortcut, allowing you to search on Sourcegraph directly from the browser address bar. From 0b8de554f926636dbf09d7c27cec9ca8407780d7 Mon Sep 17 00:00:00 2001 From: Maedah Batool Date: Tue, 22 Apr 2025 10:46:49 -0700 Subject: [PATCH 09/14] Update docs/admin/architecture.mdx Co-authored-by: Louis Jarvis <31862633+loujar@users.noreply.github.com> --- docs/admin/architecture.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/admin/architecture.mdx b/docs/admin/architecture.mdx index 622f56b67..38fd8dbb8 100644 --- a/docs/admin/architecture.mdx +++ b/docs/admin/architecture.mdx @@ -130,7 +130,7 @@ Native integrations bring Sourcegraph features directly into the UI of code host Instead of requiring a browser extension, native integrations inject a script by extending the code host directly (for example, using the code host's plugin architecture). The advantage is that Sourcegraph can be enabled for all users of a code host instance without any action required from each user. -Learn more about our [Code host integrations in the docs](/admin/code_hosts). +Learn more in the [Code host integrations docs](/admin/code_hosts) #### Dependencies From 830f6ee1105cdc73760a90a1eecb61a7db1aeb80 Mon Sep 17 00:00:00 2001 From: Maedah Batool Date: Tue, 22 Apr 2025 10:47:06 -0700 Subject: [PATCH 10/14] Update docs/admin/architecture.mdx Co-authored-by: Louis Jarvis <31862633+loujar@users.noreply.github.com> --- docs/admin/architecture.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/admin/architecture.mdx b/docs/admin/architecture.mdx index 38fd8dbb8..39a663f58 100644 --- a/docs/admin/architecture.mdx +++ b/docs/admin/architecture.mdx @@ -146,7 +146,7 @@ Instead of requiring a browser extension, native integrations inject a script by Note that `src-cli` is not contained within the Sourcegraph monorepo and has its release cadence. -Learn more about `src-cli` in the [docs](/cli) and [GitHub repo](https://github.com/sourcegraph/src-cli). +Learn more in the [src-cli docs](/admin/code_hosts) and [GitHub repo](https://github.com/sourcegraph/src-cli) ### Dependencies From c3f12054c8295568af2d1d8c4c7811f50b2a5987 Mon Sep 17 00:00:00 2001 From: Maedah Batool Date: Tue, 22 Apr 2025 10:47:22 -0700 Subject: [PATCH 11/14] Update docs/admin/architecture.mdx Co-authored-by: Louis Jarvis <31862633+loujar@users.noreply.github.com> --- docs/admin/architecture.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/admin/architecture.mdx b/docs/admin/architecture.mdx index 39a663f58..e37be4f2d 100644 --- a/docs/admin/architecture.mdx +++ b/docs/admin/architecture.mdx @@ -144,7 +144,7 @@ Instead of requiring a browser extension, native integrations inject a script by `src-cli` is written in Go and distributed as a standalone binary for Windows, macOS, and Linux. Its features include running searches, managing Sourcegraph, and executing batch changes. `src-cli` is integral to the batch changes product. -Note that `src-cli` is not contained within the Sourcegraph monorepo and has its release cadence. +`src-cli` is a standalone client, maintained and released separately from Sourcegraph Learn more in the [src-cli docs](/admin/code_hosts) and [GitHub repo](https://github.com/sourcegraph/src-cli) From 43ea6582dcfc0a13983eef577950b2af5e87ce40 Mon Sep 17 00:00:00 2001 From: Maedah Batool Date: Tue, 22 Apr 2025 10:47:32 -0700 Subject: [PATCH 12/14] Update docs/admin/architecture.mdx Co-authored-by: Louis Jarvis <31862633+loujar@users.noreply.github.com> --- docs/admin/architecture.mdx | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/admin/architecture.mdx b/docs/admin/architecture.mdx index e37be4f2d..f814c209d 100644 --- a/docs/admin/architecture.mdx +++ b/docs/admin/architecture.mdx @@ -152,6 +152,7 @@ Instead of requiring a browser extension, native integrations inject a script by - Search: GraphQL API - Batch Changes: GraphQL API +- Code Intelligence: GraphQL API ## Deployment From c0bdf8269d9af0fbe9d89472464a58ba56710506 Mon Sep 17 00:00:00 2001 From: Maedah Batool Date: Tue, 22 Apr 2025 10:47:42 -0700 Subject: [PATCH 13/14] Update docs/admin/architecture.mdx Co-authored-by: Louis Jarvis <31862633+loujar@users.noreply.github.com> --- docs/admin/architecture.mdx | 1 - 1 file changed, 1 deletion(-) diff --git a/docs/admin/architecture.mdx b/docs/admin/architecture.mdx index f814c209d..16efb5bf5 100644 --- a/docs/admin/architecture.mdx +++ b/docs/admin/architecture.mdx @@ -160,7 +160,6 @@ Sourcegraph is deployable via three supported methods: - **[Kubernetes](/admin/deploy/kubernetes)** is intended for all medium to large-scale production deployments that require fault tolerance and high availability. For advanced users only with significant kubernetes experience required. This deployment method is developed in deploy-sourcegraph - **[Docker Compose](/admin/deploy/docker-compose)** is intended to be used for small to medium production deployments, with some customization available. Easy to set up with basic infrastructure and docker knowledge required. A variation on this is the pure-Docker option. Both of these deployment methods are developed in deploy-sourcegraph-docker -- **[Docker Single Container](/admin/deploy/docker-single-container)** for small environments on a single server. It is the easiest and quickest to set up with a single command. Little infrastructure knowledge is required. This deployment method is developed in cmd/server The [resource estimator](/admin/deploy/resource_estimator#sourcegraph-resource-estimator) can guide you on the requirements for each deployment type. From 72fec9b424612beefd404bb25c543f82f039c70b Mon Sep 17 00:00:00 2001 From: Maedah Batool Date: Tue, 22 Apr 2025 11:13:58 -0700 Subject: [PATCH 14/14] Add feedback from team --- docs/admin/architecture.mdx | 44 +++++++++++++++++++++++++++++-------- 1 file changed, 35 insertions(+), 9 deletions(-) diff --git a/docs/admin/architecture.mdx b/docs/admin/architecture.mdx index 16efb5bf5..af5eb56c8 100644 --- a/docs/admin/architecture.mdx +++ b/docs/admin/architecture.mdx @@ -72,9 +72,9 @@ Read the [Batch Changes](/batch-changes) docs to learn more. - src-cli: Batch changes are currently executed client-side through the `src` CLI - Search: Repositories in which batch specs need to be executed are resolved through the search API -## Code insights +## Code Insights -Code insights surface higher-level, aggregated information to leaders in engineering organizations in dashboards. For example, code insights can track the number of matches of a search query over time, the number of code navigation diagnostic warnings in a codebase, or the usage of different programming languages. Sample use cases for this are tracking migrations, the usage of libraries across an organization, tech debt, code base health, and much more. +Code Insights surface higher-level, aggregated information to leaders in engineering organizations in dashboards. For example, code insights can track the number of matches of a search query over time, the number of code navigation diagnostic warnings in a codebase, or the usage of different programming languages. Sample use cases for this are tracking migrations, the usage of libraries across an organization, tech debt, code base health, and much more. Code Insights persist in a separate database called `codeinsights-db`. The web application interacts with the backend through a [GraphQL API](/api/graphql). @@ -156,28 +156,54 @@ Instead of requiring a browser extension, native integrations inject a script by ## Deployment -Sourcegraph is deployable via three supported methods: +Sourcegraph's recommended deployment methods are: -- **[Kubernetes](/admin/deploy/kubernetes)** is intended for all medium to large-scale production deployments that require fault tolerance and high availability. For advanced users only with significant kubernetes experience required. This deployment method is developed in deploy-sourcegraph -- **[Docker Compose](/admin/deploy/docker-compose)** is intended to be used for small to medium production deployments, with some customization available. Easy to set up with basic infrastructure and docker knowledge required. A variation on this is the pure-Docker option. Both of these deployment methods are developed in deploy-sourcegraph-docker +1. Sourcegraph Cloud: This provides a fully managed solution where Sourcegraph handles all of the maintenance, monitoring, and upgrading tasks to give you an optimal Sourcegraph experience while immediately getting the latest features into your users' hands. This solution does require your code hosts to be connected to the Sourcegraph managed environment. +2. Kubernetes Helm: Sourcegraph's Kubernetes deployment provides the most robust, scalable, and vetted self-hosted solution. This solution is ideal across many self-hosted customers capable of deploying a multi-node instance, and can be supported by all mainstream managed Kubernetes platforms. +3. Docker Compose: Docker Compose provides the preferred single-node deployment solution for Sourcegraph. It can be a good option when the complexities and flexibility provided by Kubernetes Helm are not needed. +4. Kubernetes Kustomize: Helm is Sourcegraph's more standardized and vetted approach to deploying with Kubernetes, but if Kustomize is your preferred deployment method it is a viable and supported approach. +5. Machine Images: Sourcegraph can be deployed using dedicated Machine Images for specific Cloud providers. This can be a simple solution in specific circumstances, though has its own considerations. If you are considering this path, please discuss with your account team. The [resource estimator](/admin/deploy/resource_estimator#sourcegraph-resource-estimator) can guide you on the requirements for each deployment type. +Learn more in the [deployment docs](/admin/deploy) + ## Observability Observability encapsulates the monitoring and debugging of Sourcegraph deployments. Sourcegraph is designed and ships several observability tools and out-of-the-box capabilities to enable visibility into the health and state of a Sourcegraph deployment. Monitoring includes [metrics and dashboards](/admin/observability/metrics), [alerting](/admin/observability/alerting), and [health checking](/admin/observability/health_checks) capabilities. -- grafana is the frontend for service metrics and ships with customized dashboards for Sourcegraph services +- `grafana` is the frontend for service metrics and ships with customized dashboards for Sourcegraph services - prometheus handles the scraping of service metrics and ships with recording rules, alert rules, and alerting capabilities -- cadvisor provides per-container performance metrics (scraped by Prometheus) in most Sourcegraph environments +- `cadvisor` provides per-container performance metrics (scraped by Prometheus) in most Sourcegraph environments - Each Sourcegraph service provides health checks Debugging includes [tracing](/admin/observability/tracing) and [logging](/admin/observability/logs). - jaeger is the distributed tracing service used by Sourcegraph -- jaeger-agent is a sidecar used in Kubernetes deployments for collecting traces from services - Each Sourcegraph service provides logs -Learn more in our [Observability docs](/admin/observability). +Learn more in the [Observability docs](/admin/observability). + +## Cody + +This section covers the Enterprise architecture of our AI assistant, Cody. [Cody Enterprise](/cody/clients/enable-cody-enterprise) can be deployed via the Sourcegraph Cloud or on your self-hosted infrastructure. + +### Cody with Sourcegraph Cloud deployment + +This is a recommended deployment for Cody Enterprise. It uses the Sourcegraph Cloud infrastructure and Cody gateway. + + + +### Sourcegraph Enterprise Server (self-hosted) on Amazon Bedrock + +This is an example of a more complex deployment that uses Sourcegraph Enterprise Server (self-hosted) and Amazon Bedrock. + + + +### Data flow + +The following diagram describes the data flow between the different components of Cody Enterprise. + +