Skip to content

Commit dd14907

Browse files
committed
Cleanup documentation around symbols and repo-updater
Those two services are removed from our architecture, and workloads are moved to worker and searcher. This PR removes (hopefully) all remaining references to those services in our docs. Test plan: Review.
1 parent 8ee048f commit dd14907

25 files changed

+65
-177
lines changed

docs/admin/architecture.mdx

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -11,11 +11,11 @@
1111
At its core, Sourcegraph maintains a persistent cache of all repositories that are connected to it. It is persistent because this data is critical for Sourcegraph to function. Still, it is ultimately a cache because the code host is the source of truth, and our cache is eventually consistent.
1212

1313
- `gitserver` is the sharded service that stores repositories and makes them accessible to other Sourcegraph services
14-
- `repo-updater` is the singleton service responsible for ensuring all repositories in gitserver are as up-to-date as possible while respecting code host rate limits. It is also responsible for syncing repository metadata from the code host that is stored in the repo table of our Postgres database
14+
- `worker` is responsible for ensuring all repositories in gitserver are as up-to-date as possible while respecting code host rate limits. It is also responsible for syncing repository metadata from the code host that is stored in the repo table of our Postgres database
1515

1616
## Permission syncing
1717

18-
Repository permissions are mirrored from code hosts to Sourcegraph by default. This builds the foundation of Sourcegraph authorization for repositories to ensure users see consistent content on code hosts. Currently, the background permissions syncer resides in the repo-updater.
18+
Repository permissions are mirrored from code hosts to Sourcegraph by default. This builds the foundation of Sourcegraph authorization for repositories to ensure users see consistent content on code hosts. Currently, the background permissions syncer resides in the `worker`.
1919

2020
<Callout type="note">Learn more in the [Permission Syncing docs](/admin/permissions/syncing)</Callout>
2121

@@ -94,7 +94,7 @@ You can learn more in the [Code Insights](/code_insights) docs.
9494
- Exhaustive search (with `count:all/count:999999` operator)
9595
- Historical search (= unindexed search, currently)
9696
- Commit search to find historical commits to search over
97-
- Repository Syncing: The code insights backend has direct dependencies on `gitserver` and `repo-updater`
97+
- Repository Syncing: The code insights backend has a direct dependency on `gitserver`
9898
- Permission syncing: The code insights backend depends on synced repository permissions for access control
9999
- Settings cascade:
100100
- Insights and dashboard configuration are stored in user, organization, and global settings. This will change in the future and is planned to be moved to the database

docs/admin/code_hosts/rate_limits.mdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -52,7 +52,7 @@ Requests to the configured code host will be staggered as to not exceed `"reques
5252
- For Sourcegraph `<=3.38`, if rate limiting is configured more than once for the same code host instance, the most restrictive limit will be used.
5353
- For Sourcegraph >=3.39, rate limiting should be enabled and configured for each individual code host connection.
5454

55-
To see the status of configured internal rate limits, visit **Site admin > Instrumentation > repo-updater > Rate Limiter State**. This page lists internal rate limits by code host, for example:
55+
To see the status of configured internal rate limits, visit **Site admin > Instrumentation > worker > Rate Limiter State**. This page lists internal rate limits by code host, for example:
5656

5757
```json
5858
{

docs/admin/config/postgres-conf.mdx

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -53,7 +53,6 @@ The setting `max_connections` determines the number of active connections that c
5353
| --------------------------- | ------------------------------------------ |
5454
| `frontend` | `pgsql`, `codeintel-db`, `codeinsights-db` |
5555
| `gitserver` | `pgsql` |
56-
| `repo-updater` | `pgsql` |
5756
| `precise-code-intel-worker` | `codeintel-db`, `pgsql` |
5857
| `worker` | `codeintel-db`, `pgsql`, `codeinsights-db` |
5958

docs/admin/config/private-network.mdx

Lines changed: 4 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,6 @@ services hosted within an organization's private network
1919
* Connecting to external [LLM providers](../../cody/capabilities/supported-models) with Cody
2020
- **gitserver**: Executes git commands against externally hosted [code hosts](../external_service)
2121
- **migrator**: Connects to Postgres instances (which may be [externally hosted](../external_services/postgres)) to process database migrations
22-
- **repo-updater**: Communicates with [code hosts](../external_service) APIs to coordinate repository synchronization
2322
- **worker**: Sourcegraph [Worker](../workers) run various background jobs that may require establishing connections to
2423
services hosted within an organization's private network
2524

@@ -34,22 +33,22 @@ variables will depend on your Sourcegraph deployment method.
3433
Add the proxy environment variables to your Sourcegraph Helm chart [override file](https://github.com/sourcegraph/deploy-sourcegraph-helm/blob/main/charts/sourcegraph/values.yaml):
3534

3635
```yaml
37-
executor|frontend|gitserver|migrator|repo-updater|worker:
36+
executor|frontend|gitserver|migrator|worker:
3837
env:
3938
- name: HTTP_PROXY
4039
value: http://proxy.example.com:8080
4140
- name: HTTPS_PROXY
4241
value: http://proxy.example.com:8080
4342
- name: NO_PROXY
44-
value: "blobstore,codeinsights-db,codeintel-db,sourcegraph-frontend-internal,sourcegraph-frontend,github-proxy,gitserver,grafana,indexed-search-indexer,indexed-search,jaeger-query,pgsql,precise-code-intel-worker,prometheus,redis-cache,redis-store,repo-updater,searcher,symbols,syntect-server,worker-executors,worker,cloud-sql-proxy,localhost,127.0.0.1,.svc,.svc.cluster.local,kubernetes.default.svc"
43+
value: "blobstore,codeinsights-db,codeintel-db,sourcegraph-frontend-internal,sourcegraph-frontend,github-proxy,gitserver,grafana,indexed-search-indexer,indexed-search,jaeger-query,pgsql,precise-code-intel-worker,prometheus,redis-cache,redis-store,searcher,syntect-server,worker-executors,worker,cloud-sql-proxy,localhost,127.0.0.1,.svc,.svc.cluster.local,kubernetes.default.svc"
4544
```
4645
4746
<Callout type="info">
4847
If the updated Sourcegraph pods fail to pass their readiness or health checks after configuring the HTTP proxy environment variables, you may also need to add your k8s cluster pod & service CIDR ranges to the `NO_PROXY` environment variable. Example:
4948

5049
```yaml
5150
- name: NO_PROXY
52-
value: "blobstore,codeinsights-db,codeintel-db,sourcegraph-frontend-internal,sourcegraph-frontend,github-proxy,gitserver,grafana,indexed-search-indexer,indexed-search,jaeger-query,pgsql,precise-code-intel-worker,prometheus,redis-cache,redis-store,repo-updater,searcher,symbols,syntect-server,worker-executors,worker,cloud-sql-proxy,localhost,127.0.0.1,.svc,.svc.cluster.local,kubernetes.default.svc,10.10.0.0/16,10.20.0.0/16"
51+
value: "blobstore,codeinsights-db,codeintel-db,sourcegraph-frontend-internal,sourcegraph-frontend,github-proxy,gitserver,grafana,indexed-search-indexer,indexed-search,jaeger-query,pgsql,precise-code-intel-worker,prometheus,redis-cache,redis-store,searcher,syntect-server,worker-executors,worker,cloud-sql-proxy,localhost,127.0.0.1,.svc,.svc.cluster.local,kubernetes.default.svc,10.10.0.0/16,10.20.0.0/16"
5352
```
5453
</Callout>
5554

@@ -62,7 +61,7 @@ services:
6261
environment:
6362
- HTTP_PROXY=http://proxy.example.com:8080
6463
- HTTPS_PROXY=http://proxy.example.com:8080
65-
- NO_PROXY='blobstore,caddy,cadvisor,codeintel-db,codeintel-db-exporter,codeinsights-db,codeinsights-db-exporter,sourcegraph-frontend-0,sourcegraph-frontend-internal,gitserver-0,grafana,migrator,node-exporter,otel-collector,pgsql,pgsql-exporter,precise-code-intel-worker,prometheus,redis-cache,redis-store,repo-updater,searcher-0,symbols-0,syntect-server,worker,zoekt-indexserver-0,zoekt-webserver-0,localhost,127.0.0.1'
64+
- NO_PROXY='blobstore,caddy,cadvisor,codeintel-db,codeintel-db-exporter,codeinsights-db,codeinsights-db-exporter,sourcegraph-frontend-0,sourcegraph-frontend-internal,gitserver-0,grafana,migrator,node-exporter,otel-collector,pgsql,pgsql-exporter,precise-code-intel-worker,prometheus,redis-cache,redis-store,searcher-0,syntect-server,worker,zoekt-indexserver-0,zoekt-webserver-0,localhost,127.0.0.1'
6665
```
6766

6867
<Callout type="warning">Failure to configure `NO_PROXY` correctly can cause the proxy configuration to interfere with

docs/admin/deploy/docker-compose/configuration.mdx

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -125,7 +125,7 @@ If you must use a `.netrc` file to store these credentials instead, follow the p
125125

126126
## Add replicas
127127

128-
When adding replicas for `gitserver`, `indexed-search`, `searcher`, or `symbols`, you must update the corresponding environment variable on each of the frontend services in your docker-compose.override.yaml file, `SRC_GIT_SERVERS`, `INDEXED_SEARCH_SERVERS`, `SEARCHER_URL`, and `SYMBOLS_URL` to the number of replicas for each respective service. Sourcegraph will then automatically infer the endpoints for each replica.
128+
When adding replicas for `gitserver`, `indexed-search`, or `searcher`, you must update the corresponding environment variable on each of the frontend services in your docker-compose.override.yaml file, `SRC_GIT_SERVERS`, `INDEXED_SEARCH_SERVERS`, and `SEARCHER_URL` to the number of replicas for each respective service. Sourcegraph will then automatically infer the endpoints for each replica.
129129

130130
```yaml
131131
# docker-compose.override.yaml
@@ -136,14 +136,12 @@ services:
136136
- 'SRC_GIT_SERVERS=2'
137137
- 'INDEXED_SEARCH_SERVERS=2'
138138
- 'SEARCHER_URL=1'
139-
- 'SYMBOLS_URL=1'
140139
141140
sourcegraph-frontend-internal:
142141
environment:
143142
- 'SRC_GIT_SERVERS=2'
144143
- 'INDEXED_SEARCH_SERVERS=2'
145144
- 'SEARCHER_URL=1'
146-
- 'SYMBOLS_URL=1'
147145
```
148146

149147
## Shard gitserver

docs/admin/deploy/docker-compose/operations.mdx

Lines changed: 0 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -71,9 +71,7 @@ prometheus /bin/prom-wrapper Up
7171
query-runner /sbin/tini -- /usr/local/b ... Up
7272
redis-cache /sbin/tini -- redis-server ... Up 6379/tcp
7373
redis-store /sbin/tini -- redis-server ... Up 6379/tcp
74-
repo-updater /sbin/tini -- /usr/local/b ... Up
7574
searcher-0 /sbin/tini -- /usr/local/b ... Up (healthy)
76-
symbols-0 /sbin/tini -- /usr/local/b ... Up (healthy) 3184/tcp
7775
syntect-server sh -c /http-server-stabili ... Up (healthy) 9238/tcp
7876
worker /sbin/tini -- /usr/local/b ... Up 3189/tcp
7977
zoekt-indexserver-0 /sbin/tini -- zoekt-source ... Up
@@ -151,9 +149,7 @@ prometheus /bin/prom-wrapper Up
151149
query-runner /sbin/tini -- /usr/local/b ... Up
152150
redis-cache /sbin/tini -- redis-server ... Up 6379/tcp
153151
redis-store /sbin/tini -- redis-server ... Up 6379/tcp
154-
repo-updater /sbin/tini -- /usr/local/b ... Up
155152
searcher-0 /sbin/tini -- /usr/local/b ... Up (healthy)
156-
symbols-0 /sbin/tini -- /usr/local/b ... Up (healthy) 3184/tcp
157153
syntect-server sh -c /http-server-stabili ... Up (healthy) 9238/tcp
158154
worker /sbin/tini -- /usr/local/b ... Up 3189/tcp
159155
zoekt-indexserver-0 /sbin/tini -- zoekt-source ... Up
@@ -221,9 +217,7 @@ prometheus /bin/prom-wrapper Up
221217
query-runner /sbin/tini -- /usr/local/b ... Up
222218
redis-cache /sbin/tini -- redis-server ... Up 6379/tcp
223219
redis-store /sbin/tini -- redis-server ... Up 6379/tcp
224-
repo-updater /sbin/tini -- /usr/local/b ... Up
225220
searcher-0 /sbin/tini -- /usr/local/b ... Up (healthy)
226-
symbols-0 /sbin/tini -- /usr/local/b ... Up (healthy) 3184/tcp
227221
syntect-server sh -c /http-server-stabili ... Up (healthy) 9238/tcp
228222
worker /sbin/tini -- /usr/local/b ... Up 3189/tcp
229223
zoekt-indexserver-0 /sbin/tini -- zoekt-source ... Up

docs/admin/deploy/kubernetes/configure.mdx

Lines changed: 1 addition & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -994,7 +994,7 @@ patches:
994994

995995
You can update environment variables for **searcher** with `patches`.
996996

997-
For example, to update the value for `SEARCHER_CACHE_SIZE_MB`:
997+
For example, to update the value for `SEARCHER_CACHE_SIZE_MB` and `SEARCHER_CACHE_SIZE_MB`:
998998

999999
```yaml
10001000
# instances/$INSTANCE_NAME/kustomization.yaml
@@ -1008,21 +1008,6 @@ For example, to update the value for `SEARCHER_CACHE_SIZE_MB`:
10081008
value:
10091009
name: SEARCHER_CACHE_SIZE_MB
10101010
value: "50000"
1011-
```
1012-
1013-
### Symbols
1014-
1015-
You can update environment variables for **searcher** with `patches`.
1016-
1017-
For example, to update the value for `SYMBOLS_CACHE_SIZE_MB`:
1018-
1019-
```yaml
1020-
# instances/$INSTANCE_NAME/kustomization.yaml
1021-
patches:
1022-
- target:
1023-
name: symbols
1024-
kind: StatefulSet|Deployment
1025-
patch: |-
10261011
- op: replace
10271012
path: /spec/template/spec/containers/0/env/0
10281013
value:
@@ -1098,12 +1083,9 @@ Sourcegraph supports specifying an external Redis server with these environment
10981083

10991084
When using an external Redis server, the corresponding environment variable must also be added to the following services:
11001085

1101-
11021086
- `sourcegraph-frontend`
1103-
- `repo-updater`
11041087
- `gitserver`
11051088
- `searcher`
1106-
- `symbols`
11071089
- `worker`
11081090

11091091
**Step 1**: Include the `services/redis` component in your components:

docs/admin/deploy/kubernetes/index.mdx

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -944,11 +944,9 @@ Scale down `deployments` and `statefulSets` that access the database, _this step
944944
The following services must have their replicas scaled to 0:
945945
- Deployments (e.g., `kubectl scale deployment <name> --replicas=0`)
946946
- precise-code-intel-worker
947-
- repo-updater
948947
- searcher
949948
- sourcegraph-frontend
950949
- sourcegraph-frontend-internal
951-
- symbols
952950
- worker
953951
- Stateful sets (e.g., `kubectl scale sts <name> --replicas=0`):
954952
- gitserver

docs/admin/deploy/kubernetes/kustomize/migrate.mdx

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@ Here are the benefits of the new base cluster with the new Kustomize setup compa
1515
- Streamlined resource allocation process:
1616
* Allocates resources based on the size of the instance
1717
* Optimized through load testing
18-
* The searcher and symbols use StatefulSets and do not require ephemeral storage
18+
* The searcher StatefulSet does not require ephemeral storage
1919
- Utilizes the Kubernetes-native tool Kustomize:
2020
* Built into kubectl
2121
* No additional scripting required
@@ -192,6 +192,8 @@ If your instance was deployed using the non-privileged overlay, you can follow t
192192

193193
## Step 9: Build and review new manifests
194194

195+
> NOTE: Symbols has been removed in Sourcegraph 6.4.
196+
195197
`pgsql`, `codeinsights-db`, `searcher`, `symbols`, and `codeintel-db` have been changed from `Deployments` to `StatefulSets`. However, redeploying these services as StatefulSets should not affect your existing deployment as they are all configured to use the same PVCs.
196198

197199
### From Deployment to StatefulSet

docs/admin/deploy/kubernetes/operations.mdx

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -429,11 +429,9 @@ precise-code-intel-worker ClusterIP 10.72.11.102 <none> 3188/TC
429429
prometheus ClusterIP 10.72.12.201 <none> 30090/TCP 25h
430430
redis-cache ClusterIP 10.72.15.138 <none> 6379/TCP,9121/TCP 25h
431431
redis-store ClusterIP 10.72.4.162 <none> 6379/TCP,9121/TCP 25h
432-
repo-updater ClusterIP 10.72.11.176 <none> 3182/TCP,6060/TCP 25h
433432
searcher ClusterIP None <none> 3181/TCP,6060/TCP 23h
434433
sourcegraph-frontend ClusterIP 10.72.12.103 <none> 30080/TCP,6060/TCP 25h
435434
sourcegraph-frontend-internal ClusterIP 10.72.9.155 <none> 80/TCP 25h
436-
symbols ClusterIP None <none> 3184/TCP,6060/TCP 23h
437435
syntect-server ClusterIP 10.72.14.49 <none> 9238/TCP,6060/TCP 25h
438436
worker ClusterIP 10.72.7.72 <none> 3189/TCP,6060/TCP 25h
439437
```

0 commit comments

Comments
 (0)