Skip to content

Commit f284e90

Browse files
authored
Cleanup documentation around symbols and repo-updater (#1132)
Those two services are removed from our architecture, and workloads are moved to worker and searcher. This PR removes (hopefully) all remaining references to those services in our docs. Test plan: Review.
1 parent f628b02 commit f284e90

25 files changed

+65
-177
lines changed

docs/admin/architecture.mdx

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -11,11 +11,11 @@
1111
At its core, Sourcegraph maintains a persistent cache of all repositories that are connected to it. It is persistent because this data is critical for Sourcegraph to function. Still, it is ultimately a cache because the code host is the source of truth, and our cache is eventually consistent.
1212

1313
- `gitserver` is the sharded service that stores repositories and makes them accessible to other Sourcegraph services
14-
- `repo-updater` is the singleton service responsible for ensuring all repositories in gitserver are as up-to-date as possible while respecting code host rate limits. It is also responsible for syncing repository metadata from the code host that is stored in the repo table of our Postgres database
14+
- `worker` is responsible for ensuring all repositories in gitserver are as up-to-date as possible while respecting code host rate limits. It is also responsible for syncing repository metadata from the code host that is stored in the repo table of our Postgres database
1515

1616
## Permission syncing
1717

18-
Repository permissions are mirrored from code hosts to Sourcegraph by default. This builds the foundation of Sourcegraph authorization for repositories to ensure users see consistent content on code hosts. Currently, the background permissions syncer resides in the repo-updater.
18+
Repository permissions are mirrored from code hosts to Sourcegraph by default. This builds the foundation of Sourcegraph authorization for repositories to ensure users see consistent content on code hosts. Currently, the background permissions syncer resides in the `worker`.
1919

2020
<Callout type="note">Learn more in the [Permission Syncing docs](/admin/permissions/syncing)</Callout>
2121

@@ -94,7 +94,7 @@ You can learn more in the [Code Insights](/code_insights) docs.
9494
- Exhaustive search (with `count:all/count:999999` operator)
9595
- Historical search (= unindexed search, currently)
9696
- Commit search to find historical commits to search over
97-
- Repository Syncing: The code insights backend has direct dependencies on `gitserver` and `repo-updater`
97+
- Repository Syncing: The code insights backend has a direct dependency on `gitserver`
9898
- Permission syncing: The code insights backend depends on synced repository permissions for access control
9999
- Settings cascade:
100100
- Insights and dashboard configuration are stored in user, organization, and global settings. This will change in the future and is planned to be moved to the database

docs/admin/code_hosts/rate_limits.mdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -52,7 +52,7 @@ Requests to the configured code host will be staggered as to not exceed `"reques
5252
- For Sourcegraph `<=3.38`, if rate limiting is configured more than once for the same code host instance, the most restrictive limit will be used.
5353
- For Sourcegraph >=3.39, rate limiting should be enabled and configured for each individual code host connection.
5454

55-
To see the status of configured internal rate limits, visit **Site admin > Instrumentation > repo-updater > Rate Limiter State**. This page lists internal rate limits by code host, for example:
55+
To see the status of configured internal rate limits, visit **Site admin > Instrumentation > worker > Rate Limiter State**. This page lists internal rate limits by code host, for example:
5656

5757
```json
5858
{

docs/admin/config/postgres-conf.mdx

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -53,7 +53,6 @@ The setting `max_connections` determines the number of active connections that c
5353
| --------------------------- | ------------------------------------------ |
5454
| `frontend` | `pgsql`, `codeintel-db`, `codeinsights-db` |
5555
| `gitserver` | `pgsql` |
56-
| `repo-updater` | `pgsql` |
5756
| `precise-code-intel-worker` | `codeintel-db`, `pgsql` |
5857
| `worker` | `codeintel-db`, `pgsql`, `codeinsights-db` |
5958

docs/admin/config/private-network.mdx

Lines changed: 4 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,6 @@ services hosted within an organization's private network
1919
* Connecting to external [LLM providers](../../cody/capabilities/supported-models) with Cody
2020
- **gitserver**: Executes git commands against externally hosted [code hosts](../external_service)
2121
- **migrator**: Connects to Postgres instances (which may be [externally hosted](../external_services/postgres)) to process database migrations
22-
- **repo-updater**: Communicates with [code hosts](../external_service) APIs to coordinate repository synchronization
2322
- **worker**: Sourcegraph [Worker](../workers) run various background jobs that may require establishing connections to
2423
services hosted within an organization's private network
2524

@@ -34,22 +33,22 @@ variables will depend on your Sourcegraph deployment method.
3433
Add the proxy environment variables to your Sourcegraph Helm chart [override file](https://github.com/sourcegraph/deploy-sourcegraph-helm/blob/main/charts/sourcegraph/values.yaml):
3534

3635
```yaml
37-
executor|frontend|gitserver|migrator|repo-updater|worker:
36+
executor|frontend|gitserver|migrator|worker:
3837
env:
3938
- name: HTTP_PROXY
4039
value: http://proxy.example.com:8080
4140
- name: HTTPS_PROXY
4241
value: http://proxy.example.com:8080
4342
- name: NO_PROXY
44-
value: "blobstore,codeinsights-db,codeintel-db,sourcegraph-frontend-internal,sourcegraph-frontend,github-proxy,gitserver,grafana,indexed-search-indexer,indexed-search,jaeger-query,pgsql,precise-code-intel-worker,prometheus,redis-cache,redis-store,repo-updater,searcher,symbols,syntect-server,worker-executors,worker,cloud-sql-proxy,localhost,127.0.0.1,.svc,.svc.cluster.local,kubernetes.default.svc"
43+
value: "blobstore,codeinsights-db,codeintel-db,sourcegraph-frontend-internal,sourcegraph-frontend,github-proxy,gitserver,grafana,indexed-search-indexer,indexed-search,jaeger-query,pgsql,precise-code-intel-worker,prometheus,redis-cache,redis-store,searcher,syntect-server,worker-executors,worker,cloud-sql-proxy,localhost,127.0.0.1,.svc,.svc.cluster.local,kubernetes.default.svc"
4544
```
4645
4746
<Callout type="info">
4847
If the updated Sourcegraph pods fail to pass their readiness or health checks after configuring the HTTP proxy environment variables, you may also need to add your k8s cluster pod & service CIDR ranges to the `NO_PROXY` environment variable. Example:
4948

5049
```yaml
5150
- name: NO_PROXY
52-
value: "blobstore,codeinsights-db,codeintel-db,sourcegraph-frontend-internal,sourcegraph-frontend,github-proxy,gitserver,grafana,indexed-search-indexer,indexed-search,jaeger-query,pgsql,precise-code-intel-worker,prometheus,redis-cache,redis-store,repo-updater,searcher,symbols,syntect-server,worker-executors,worker,cloud-sql-proxy,localhost,127.0.0.1,.svc,.svc.cluster.local,kubernetes.default.svc,10.10.0.0/16,10.20.0.0/16"
51+
value: "blobstore,codeinsights-db,codeintel-db,sourcegraph-frontend-internal,sourcegraph-frontend,github-proxy,gitserver,grafana,indexed-search-indexer,indexed-search,jaeger-query,pgsql,precise-code-intel-worker,prometheus,redis-cache,redis-store,searcher,syntect-server,worker-executors,worker,cloud-sql-proxy,localhost,127.0.0.1,.svc,.svc.cluster.local,kubernetes.default.svc,10.10.0.0/16,10.20.0.0/16"
5352
```
5453
</Callout>
5554

@@ -62,7 +61,7 @@ services:
6261
environment:
6362
- HTTP_PROXY=http://proxy.example.com:8080
6463
- HTTPS_PROXY=http://proxy.example.com:8080
65-
- NO_PROXY='blobstore,caddy,cadvisor,codeintel-db,codeintel-db-exporter,codeinsights-db,codeinsights-db-exporter,sourcegraph-frontend-0,sourcegraph-frontend-internal,gitserver-0,grafana,migrator,node-exporter,otel-collector,pgsql,pgsql-exporter,precise-code-intel-worker,prometheus,redis-cache,redis-store,repo-updater,searcher-0,symbols-0,syntect-server,worker,zoekt-indexserver-0,zoekt-webserver-0,localhost,127.0.0.1'
64+
- NO_PROXY='blobstore,caddy,cadvisor,codeintel-db,codeintel-db-exporter,codeinsights-db,codeinsights-db-exporter,sourcegraph-frontend-0,sourcegraph-frontend-internal,gitserver-0,grafana,migrator,node-exporter,otel-collector,pgsql,pgsql-exporter,precise-code-intel-worker,prometheus,redis-cache,redis-store,searcher-0,syntect-server,worker,zoekt-indexserver-0,zoekt-webserver-0,localhost,127.0.0.1'
6665
```
6766

6867
<Callout type="warning">Failure to configure `NO_PROXY` correctly can cause the proxy configuration to interfere with

docs/admin/deploy/docker-compose/configuration.mdx

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -125,7 +125,7 @@ If you must use a `.netrc` file to store these credentials instead, follow the p
125125

126126
## Add replicas
127127

128-
When adding replicas for `gitserver`, `searcher`, `symbols`, `zoekt-indexserver`, or `zoekt-webserver`, you must update the corresponding environment variable on each of the frontend services in your docker-compose.override.yaml file to the number of replicas for the respective service. Sourcegraph will then automatically infer the containers' endpoints for each replica.
128+
When adding replicas for `gitserver`, `searcher`, `zoekt-indexserver`, or `zoekt-webserver`, you must update the corresponding environment variable on each of the frontend services in your docker-compose.override.yaml file to the number of replicas for the respective service. Sourcegraph will then automatically infer the containers' endpoints for each replica.
129129

130130
```yaml
131131
# docker-compose.override.yaml
@@ -137,15 +137,13 @@ services:
137137
- 'INDEXED_SEARCH_SERVERS=1'
138138
- 'SEARCHER_URL=1'
139139
- 'SRC_GIT_SERVERS=1'
140-
- 'SYMBOLS_URL=1'
141140
142141
sourcegraph-frontend-internal:
143142
environment:
144143
- 'INDEXED_SEARCH_INDEXERS=1'
145144
- 'INDEXED_SEARCH_SERVERS=1'
146145
- 'SEARCHER_URL=1'
147146
- 'SRC_GIT_SERVERS=1'
148-
- 'SYMBOLS_URL=1'
149147
```
150148

151149
## Shard gitserver

docs/admin/deploy/docker-compose/operations.mdx

Lines changed: 0 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -71,9 +71,7 @@ prometheus /bin/prom-wrapper Up
7171
query-runner /sbin/tini -- /usr/local/b ... Up
7272
redis-cache /sbin/tini -- redis-server ... Up 6379/tcp
7373
redis-store /sbin/tini -- redis-server ... Up 6379/tcp
74-
repo-updater /sbin/tini -- /usr/local/b ... Up
7574
searcher-0 /sbin/tini -- /usr/local/b ... Up (healthy)
76-
symbols-0 /sbin/tini -- /usr/local/b ... Up (healthy) 3184/tcp
7775
syntect-server sh -c /http-server-stabili ... Up (healthy) 9238/tcp
7876
worker /sbin/tini -- /usr/local/b ... Up 3189/tcp
7977
zoekt-indexserver-0 /sbin/tini -- zoekt-source ... Up
@@ -151,9 +149,7 @@ prometheus /bin/prom-wrapper Up
151149
query-runner /sbin/tini -- /usr/local/b ... Up
152150
redis-cache /sbin/tini -- redis-server ... Up 6379/tcp
153151
redis-store /sbin/tini -- redis-server ... Up 6379/tcp
154-
repo-updater /sbin/tini -- /usr/local/b ... Up
155152
searcher-0 /sbin/tini -- /usr/local/b ... Up (healthy)
156-
symbols-0 /sbin/tini -- /usr/local/b ... Up (healthy) 3184/tcp
157153
syntect-server sh -c /http-server-stabili ... Up (healthy) 9238/tcp
158154
worker /sbin/tini -- /usr/local/b ... Up 3189/tcp
159155
zoekt-indexserver-0 /sbin/tini -- zoekt-source ... Up
@@ -221,9 +217,7 @@ prometheus /bin/prom-wrapper Up
221217
query-runner /sbin/tini -- /usr/local/b ... Up
222218
redis-cache /sbin/tini -- redis-server ... Up 6379/tcp
223219
redis-store /sbin/tini -- redis-server ... Up 6379/tcp
224-
repo-updater /sbin/tini -- /usr/local/b ... Up
225220
searcher-0 /sbin/tini -- /usr/local/b ... Up (healthy)
226-
symbols-0 /sbin/tini -- /usr/local/b ... Up (healthy) 3184/tcp
227221
syntect-server sh -c /http-server-stabili ... Up (healthy) 9238/tcp
228222
worker /sbin/tini -- /usr/local/b ... Up 3189/tcp
229223
zoekt-indexserver-0 /sbin/tini -- zoekt-source ... Up

docs/admin/deploy/kubernetes/configure.mdx

Lines changed: 1 addition & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -996,7 +996,7 @@ patches:
996996

997997
You can update environment variables for **searcher** with `patches`.
998998

999-
For example, to update the value for `SEARCHER_CACHE_SIZE_MB`:
999+
For example, to update the value for `SEARCHER_CACHE_SIZE_MB` and `SEARCHER_CACHE_SIZE_MB`:
10001000

10011001
```yaml
10021002
# instances/$INSTANCE_NAME/kustomization.yaml
@@ -1010,21 +1010,6 @@ For example, to update the value for `SEARCHER_CACHE_SIZE_MB`:
10101010
value:
10111011
name: SEARCHER_CACHE_SIZE_MB
10121012
value: "50000"
1013-
```
1014-
1015-
### Symbols
1016-
1017-
You can update environment variables for **searcher** with `patches`.
1018-
1019-
For example, to update the value for `SYMBOLS_CACHE_SIZE_MB`:
1020-
1021-
```yaml
1022-
# instances/$INSTANCE_NAME/kustomization.yaml
1023-
patches:
1024-
- target:
1025-
name: symbols
1026-
kind: StatefulSet|Deployment
1027-
patch: |-
10281013
- op: replace
10291014
path: /spec/template/spec/containers/0/env/0
10301015
value:
@@ -1100,12 +1085,9 @@ Sourcegraph supports specifying an external Redis server with these environment
11001085

11011086
When using an external Redis server, the corresponding environment variable must also be added to the following services:
11021087

1103-
11041088
- `sourcegraph-frontend`
1105-
- `repo-updater`
11061089
- `gitserver`
11071090
- `searcher`
1108-
- `symbols`
11091091
- `worker`
11101092

11111093
**Step 1**: Include the `services/redis` component in your components:

docs/admin/deploy/kubernetes/index.mdx

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -944,11 +944,9 @@ Scale down `deployments` and `statefulSets` that access the database, _this step
944944
The following services must have their replicas scaled to 0:
945945
- Deployments (e.g., `kubectl scale deployment <name> --replicas=0`)
946946
- precise-code-intel-worker
947-
- repo-updater
948947
- searcher
949948
- sourcegraph-frontend
950949
- sourcegraph-frontend-internal
951-
- symbols
952950
- worker
953951
- Stateful sets (e.g., `kubectl scale sts <name> --replicas=0`):
954952
- gitserver

docs/admin/deploy/kubernetes/kustomize/migrate.mdx

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@ Here are the benefits of the new base cluster with the new Kustomize setup compa
1515
- Streamlined resource allocation process:
1616
* Allocates resources based on the size of the instance
1717
* Optimized through load testing
18-
* The searcher and symbols use StatefulSets and do not require ephemeral storage
18+
* The searcher StatefulSet does not require ephemeral storage
1919
- Utilizes the Kubernetes-native tool Kustomize:
2020
* Built into kubectl
2121
* No additional scripting required
@@ -192,6 +192,8 @@ If your instance was deployed using the non-privileged overlay, you can follow t
192192

193193
## Step 9: Build and review new manifests
194194

195+
> NOTE: Symbols has been removed in Sourcegraph 6.4.
196+
195197
`pgsql`, `codeinsights-db`, `searcher`, `symbols`, and `codeintel-db` have been changed from `Deployments` to `StatefulSets`. However, redeploying these services as StatefulSets should not affect your existing deployment as they are all configured to use the same PVCs.
196198

197199
### From Deployment to StatefulSet

docs/admin/deploy/kubernetes/operations.mdx

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -429,11 +429,9 @@ precise-code-intel-worker ClusterIP 10.72.11.102 <none> 3188/TC
429429
prometheus ClusterIP 10.72.12.201 <none> 30090/TCP 25h
430430
redis-cache ClusterIP 10.72.15.138 <none> 6379/TCP,9121/TCP 25h
431431
redis-store ClusterIP 10.72.4.162 <none> 6379/TCP,9121/TCP 25h
432-
repo-updater ClusterIP 10.72.11.176 <none> 3182/TCP,6060/TCP 25h
433432
searcher ClusterIP None <none> 3181/TCP,6060/TCP 23h
434433
sourcegraph-frontend ClusterIP 10.72.12.103 <none> 30080/TCP,6060/TCP 25h
435434
sourcegraph-frontend-internal ClusterIP 10.72.9.155 <none> 80/TCP 25h
436-
symbols ClusterIP None <none> 3184/TCP,6060/TCP 23h
437435
syntect-server ClusterIP 10.72.14.49 <none> 9238/TCP,6060/TCP 25h
438436
worker ClusterIP 10.72.7.72 <none> 3189/TCP,6060/TCP 25h
439437
```

0 commit comments

Comments
 (0)