Skip to content

Commit 63790c5

Browse files
DaedalusGChickensoupwithriceMaedahBatooljdpleiness
authored
6.0.0 MVU upgrade docs update (#946)
This PR updates the upgrade docs embellishing instructions for the different deployment methods to include better coverage and handling of the Postgres 16 database upgrade when crossing over Sourcegraph 5.10.0 using migrator 6.0.0 or greater. The notable change here is the inclusion of deployment specific builtin Postgres upgrade instructions, as well as changes to the MVU upgrade docs suggesting that users bring down all services and merge upstream changes from their target version into their release branch **before** performing their migrator upgrade. Due to migrators `depends_on` clause this will ensure that Postgres upgrades occur **before** migrator attempts connection to a PG12 database. This gets around the connect bouncer added in 6.0.0 which blocks services connection to our dbs ## Testing - [x] docker-compose - [x] autoupgrade - [x] MVU (tested 5.9 -> 5.11 with migrator 6.0) - [x] kustomize - [x] autoupgrade - [x] MVU (tested 5.9 -> 5.11 with migrator 6.0) - [ ] Helm - [ ] autoupgrade - [ ] MVU --------- Co-authored-by: Anish Lakhwara <[email protected]> Co-authored-by: Maedah Batool <[email protected]> Co-authored-by: Jacob Pleiness <[email protected]>
1 parent 66f6b7c commit 63790c5

File tree

6 files changed

+223
-108
lines changed

6 files changed

+223
-108
lines changed

docs/admin/deploy/docker-compose/upgrade.mdx

Lines changed: 61 additions & 47 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ import { CURRENT_VERSION_STRING } from 'src/components/PreCodeBlock'
44

55
This document describes the process to update a Docker Compose Sourcegraph instance. If you are unfamiliar with sourcegraph versioning or releases see our [general concepts documentation](/admin/updates/).
66

7-
> ***⚠️ Attention: Always consult the [release notes](/admin/updates/docker_compose) for the versions your upgrade will pass over and end on.***
7+
> <Callout type="note">Always consult the [release notes](/admin/updates/docker_compose) for the versions your upgrade will pass over and end on.</Callout>
88
99
### Standard upgrades
1010

@@ -59,70 +59,84 @@ $ docker-compose up -d --remove-orphans
5959

6060
### Multi-version upgrades
6161

62-
If you are upgrading to **Sourcegraph 5.1 or later**, we encourage you to perform an [**automatic multi-version upgrade**](/admin/updates/automatic). The following procedure has been automated, but is still applicable should errors occur in an automated upgrade.
62+
> <Callout type="warning">Multiversion upgrades **require downtime**, please see our [cautionary note](/admin/updates/#best-practices) on upgrades, if you have any concerns about running a multiversion upgrade, please reach out to us at [[email protected]](mailto:[email protected]) for advisement. </Callout>
6363
64-
---
65-
66-
> **⚠️ Attention:** please see our [cautionary note](/admin/updates/#best-practices) on upgrades, if you have any concerns about running a multiversion upgrade, please reach out to us at [[email protected]](mailto:[email protected]) for advisement.
67-
68-
To perform a **manual** multi-version upgrade on a Sourcegraph instance running on Docker compose follow the procedure below:
64+
To perform a multi-version upgrade via migrators [upgrade](/admin/updates/migrator/migrator-operations#upgrade) command on a Sourcegraph instance running on Docker compose follow the procedure below:
6965

7066
1. **Check Upgrade Readiness**:
7167
- Check the [upgrade notes](/admin/updates/docker_compose#docker-compose-upgrade-notes) for the version range you're passing through.
7268
- Check the `Site Admin > Updates` page to determine [upgrade readiness](/admin/updates/#upgrade-readiness).
7369

74-
2. **Disable Connections to the Database**:
75-
- Run the following command in the directory containing your `docker-compose.yaml` file.
76-
```sh
77-
$ docker-compose stop && docker-compose up -d pgsql codeintel-db codeinsights-db
78-
```
79-
3. **Run Migrator with the `upgrade` command**:
80-
- The following procedure describes running migrator in brief, for more detailed instructions and available command flags see our [migrator docs](/admin/updates/migrator/migrator-operations#docker-compose).
81-
1. Set the migrator `image:` in your `docker-compose.yaml` to the **latest** release of `migrator`. **Example:**
70+
2. **Pull and merge upstream changes**:
71+
- Follow the [standard upgrade procedure](#standard-upgrades) to pull and merge upstream changes from the version you are upgrading to your `release` branch.
72+
73+
3. **Disable Connections to the Database**:
74+
- Stop running containers connected to the databases and the databases:
75+
```
76+
$ docker-compose down --remove-orphans
77+
```
78+
79+
4. **Run Migrator with the `upgrade` command**:
80+
- *For more detailed instructions and available command flags see our [migrator docs](/admin/updates/migrator/migrator-operations#upgrade).*
81+
1. If the migrator `image:` in your `docker-compose.yaml` wasn't updated to in the **latest** release of `migrator` in step 2 set migrator's image to the latest release.
82+
**Example:**
8283
```yaml
8384
migrator:
8485
container_name: migrator
85-
image: 'index.docker.io/sourcegraph/migrator:5.0.4'
86+
image: 'index.docker.io/sourcegraph/migrator:6.0.0'
8687
```
87-
> *Note: Always use the latest image version of migrator for migrator commands, except the startup command `up`*
88-
2. Set the migrator `command:` to `upgrade` you'll need to supply a `--to=` argument. **Example:**
88+
2. Set the migrator `command:` to `upgrade` you'll need to supply a `--to=` argument to the version you're upgrading to.
89+
**Example:**
8990
```yaml
90-
command: ['upgrade', '--from=v4.1.2', '--to=v4.4.0']
91+
command: ['upgrade', '--from=v5.9.0', '--to=v6.0.0']
9192
```
9293
> *Note: you may add the `--dry-run` flag to the `command:` to test things out before altering the dbs*
93-
3. Run migrator with `docker-compose up migrator` **Example:**
94+
3. Run migrator with `docker-compose up migrator`
95+
96+
- Migrator `depends_on:` will ensure the databases are ready before attempting to run the migrator. Ensuring that database entry point scripts are run before the migrator attempts to connect to the databases. For users upgrading from a version earlier than `5.10.0`, a PostgreSQL version is required and will be performed automatically here. For more details, see [Upgradeing PostgreSQL](/admin/postgresql#upgrading-postgresql).
97+
98+
**Example:**
9499
```sh
95100
$ ~/deploy-sourcegraph-docker/docker-compose/ docker-compose up migrator
96-
codeintel-db is up-to-date
97-
codeinsights-db is up-to-date
98-
pgsql is up-to-date
99-
Recreating migrator ... done
101+
✔ Network docker-compose_sourcegraph Create... 0.1s
102+
✔ Container pgsql Created 0.1s
103+
✔ Container codeinsights-db Created 0.1s
104+
✔ Container codeintel-db Created 0.1s
105+
! codeinsights-db The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested 0.0s
106+
! pgsql The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested 0.0s
107+
! codeintel-db The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested 0.0s
108+
✔ Container migrator Created 0.0s
109+
! migrator The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested 0.0s
100110
Attaching to migrator
101-
migrator | ❗️ An error was returned when detecting the terminal size and capabilities:
102-
migrator |
103-
migrator | GetWinsize: inappropriate ioctl for device
104-
migrator |
105-
migrator | Execution will continue, but please report this, along with your operating
106-
migrator | system, terminal, and any other details, to:
107-
migrator | https://github.com/sourcegraph/sourcegraph/issues/new
108-
migrator |
109-
migrator | ✱ Sourcegraph migrator 4.4.0
110-
migrator | 👉 Migrating to v4.3 (step 1 of 2)
111-
migrator | 👉 Running schema migrations
112-
migrator | ✅ Schema migrations complete
113-
migrator | 👉 Running out of band migrations [17 18]
114-
✅ Out of band migrations complete
115-
migrator | 👉 Migrating to v4.4 (step 2 of 2)
116-
migrator | 👉 Running schema migrations
117-
migrator | ✅ Schema migrations complete
118-
migrator | migrator exited with code 0
111+
migrator | ✱ Sourcegraph migrator 6.0.0
112+
migrator | ⚠️ Failed to check for migrator update: unexpected status code 404. Continuing...
113+
migrator | Attempting connection to frontend...
114+
migrator | ✅ Connection to frontend succeeded
115+
migrator | Attempting connection to frontend...
116+
migrator | ✅ Connection to frontend succeeded
117+
migrator | Attempting connection to codeintel...
118+
migrator | ✅ Connection to codeintel succeeded
119+
migrator | Attempting connection to codeinsights...
120+
migrator | ✅ Connection to codeinsights succeeded
121+
migrator | 👉 Migrating to v6.0 (step 1 of 1)
122+
migrator | 👉 Running schema migrations
123+
migrator | ✅ Schema migrations complete
124+
migrator exited with code 0
119125
```
126+
5. **Run Migrator with the `up` command**
127+
- Set your migrator `command:` to `up`
128+
> *Note: If you aren't upgrading to the latest release remember to set the migrator image back to the version you are upgrading to.*
129+
**Example:**
130+
```yaml
131+
migrator:
132+
container_name: migrator
133+
image: 'index.docker.io/sourcegraph/migrator:6.0.0@sha256:ec295eb0b743da6bf56777ca6524972267a5c442b0288095e2fe12fce38ebacc'
134+
cpus: 0.5
135+
mem_limit: '500m'
136+
command: ['up']
137+
```
120138

121-
4. **Pull and merge upstream changes**:
122-
- Follow the [standard upgrade procedure](#standard-upgrades) to pull and merge upstream changes from the version you are upgrading to to your `release` branch.
123-
- **⚠️ Attention:** *merging upstream changes should set the migrator `image:` version back to the release you are upgrading to, and the `command:` should be set back to `up`, this is necessary to start your instance again.*
124-
125-
5. **Start your containers again**:
139+
6. **Start your containers again**:
126140
- run `docker-compose up -d` in the folder containing your `docker-compose.yaml` file.
127141
```sh
128142
$ docker-compose up -d

docs/admin/deploy/kubernetes/index.mdx

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -928,10 +928,6 @@ When all pods have restarted and show as Running, you can browse to your Sourceg
928928

929929
### Multi-version upgrades
930930

931-
If you are upgrading to **Sourcegraph 5.1 or later**, we encourage you to perform an [**automatic multi-version upgrade**](/admin/updates/automatic). The following instructions are still applicable, just the manual version of what automatic multi-version upgrade do for you now (and therefore is valuable information during a bumpy upgrade).
932-
933-
---
934-
935931
<Callout type="warning">Please see our [cautionary note](/admin/updates/#best-practices) on upgrades, if you have any concerns about running a multi-version upgrade, please reach out to us at [[email protected]](mailto:[email protected]) for advisement.</Callout>
936932

937933
### Multi-version upgrade procedure
@@ -942,6 +938,8 @@ If you are upgrading to **Sourcegraph 5.1 or later**, we encourage you to perfor
942938
- Check the `Site Admin > Updates` page to determine [upgrade readiness](/admin/updates/#upgrade-readiness).
943939

944940
**Step 2:**
941+
> <Callout type="warning">The Kubernetes Helm deployment type does not support MVU from Sourcegraph `v5.9.45` versions and earlier to Sourcegraph `v6.0.0`. Admins seeking to upgrade to Sourcegraph `v6.0.0` should upgrade to `v5.10.3940` or `v5.11.6271` and then use the standard upgrade procedure to get to `v6.0.0`. This is because migrator v6.0.0 will no longer connect to Postgres 12 databases. For more info see our [PostgreSQL upgrade docs](/admin/postgres#requirements).</Callout>
942+
945943
Scale down `deployments` and `statefulSets` that access the database, _this step prevents services from accessing the database while schema migrations are in process._
946944
The following services must have their replicas scaled to 0:
947945
- Deployments (e.g., `kubectl scale deployment <name> --replicas=0`)

docs/admin/deploy/kubernetes/upgrade.mdx

Lines changed: 41 additions & 41 deletions
Original file line numberDiff line numberDiff line change
@@ -120,46 +120,41 @@ For example, if you use [overlays to make changes to the manifests](https://gith
120120

121121
## Multi-version upgrades
122122

123-
If you are upgrading to **Sourcegraph 5.1 or later**, we encourage you to perform an [**automatic multi-version upgrade**](/admin/updates/automatic). The following procedure has been automated, but is still applicable should errors occur in an automated upgrade.
124-
125-
---
126-
127123
> **⚠️ Attention:** please see our [cautionary note](/admin/updates/#best-practices) on upgrades, if you have any concerns about running a multiversion upgrade, please reach out to us at [[email protected]](mailto:[email protected]) for advisement.
128124
129-
To perform a **manual** multi-version upgrade on a Sourcegraph instance running on our **kubernetes** repo follow the procedure below:
125+
To perform a multi-version upgrade via migrators [upgrade](/admin/updates/migrator/migrator-operations#upgrade) command on a Sourcegraph instance deployed with our [kustomize repo](https://github.com/sourcegraph/deploy-sourcegraph-k8s) follow the procedure below:
130126

131127
1. **Check Upgrade Readiness**:
132128
- Check the [upgrade notes](/admin/updates/kubernetes#kubernetes-upgrade-notes) for the version range you're passing through.
133129
- Check the `Site Admin > Updates` page to determine [upgrade readiness](/admin/updates/#upgrade-readiness).
134130

135-
2. **Disable Connections to the Database**:
136-
- The following services must have their replicas scaled to 0:
137-
- Deployments (e.g., `kubectl scale deployment <name> --replicas=0`)
138-
- precise-code-intel-worker
139-
- repo-updater
140-
- searcher
141-
- sourcegraph-frontend
142-
- sourcegraph-frontend-internal
143-
- symbols
144-
- worker
145-
- Stateful sets (e.g., `kubectl scale sts <name> --replicas=0`):
146-
- gitserver
147-
- indexed-search
148-
149-
*convenience command:*
150-
```sh
151-
$ kubectl scale deployment precise-code-intel-worker repo-updater searcher sourcegraph-frontend sourcegraph-frontend-internal symbols worker --replicas=0 && kubectl scale sts gitserver indexed-search --replicas=0
152-
```
153-
154-
3. **Run Migrator with the `upgrade` command**:
155-
- The following procedure describes running migrator in brief, for more detailed instructions and available command flags see our [migrator docs](/admin/updates/migrator/migrator-operations#kubernetes-kustomize).
156-
1. In the `configure/migrator/migrator.Job.yaml` manifest ([kustomize](https://github.com/sourcegraph/deploy-sourcegraph/blob/master/configure/migrator/migrator.Job.yaml) or [legacy](https://github.com/sourcegraph/deploy-sourcegraph/blob/master/configure/migrator/migrator.Job.yaml)):
131+
2. **Pull and merge upstream changes**:
132+
- Follow the [standard legacy upgrade procedure](#upgrade-with-legacy-kubernetes) to pull and merge upstream changes from the version you are upgrading to to your `release` branch.
133+
134+
3. **Update cluster.yaml and scale down non-database deployments and replicas**:
135+
- In your cluster kustomization file (`instances/my-sourcegraph/kustomize.yaml`), uncomment the [multi-version-upgrade util](https://github.com/sourcegraph/deploy-sourcegraph-k8s/blob/main/instances/template/kustomization.template.yaml#L155). This will scale down all non-database deployments and statefulSets replicas to 0.
136+
```yaml
137+
# - ../../components/utils/uid # -- Run all Postgres database with valid users on host
138+
- ../../components/utils/multi-version-upgrade # -- Scale down non-database pods to 0 for multi-version upgrade
139+
# - ../../components/utils/migrate-to-nonprivileged # -- Component for migrating from privileged to non-privileged
140+
#
141+
```
142+
- Generate and apply a new cluster.yaml file.
143+
```sh
144+
kubectl kustomize instances/my-sourcegraph -o cluster.yaml
145+
kubectl apply --prune -l deploy=sourcegraph -f cluster.yaml
146+
```
147+
> Note: This step will ensure that any PostgreSQL upgrade performed as an entrypoint script will have a chance to execute before the migrator is run. For more information see [Upgradeing PostgreSQL](/admin/postgres#upgrading-postgresql).
148+
149+
4. **Run Migrator with the `upgrade` command**:
150+
- *For more detailed instructions and available command flags see our [migrator docs](/admin/updates/migrator/migrator-operations#upgrade).*
151+
1. In the `configure/migrator/migrator.Job.yaml` [manifest](https://github.com/sourcegraph/deploy-sourcegraph-k8s/blob/main/components/utils/migrator/resources/migrator.Job.yaml):
157152
- set the `image:` to the **latest** release of `migrator`
158153
- set the `args:` value to `upgrade`. **Example:**
159154
```yaml
160-
- name: migrator
161-
image: "index.docker.io/sourcegraph/migrator:5.0.3"
162-
args: ["upgrade", "--from=v3.41.0", "--to=v4.5.1"]
155+
- name: migrator
156+
image: "index.docker.io/sourcegraph/migrator:6.0.0" // here we use a later version of migrator than the version we're upgrading too.
157+
args: ["upgrade", "--from=v5.9.0", "--to=5.11.0"]
163158
env:
164159
```
165160
> *Note:*
@@ -168,11 +163,11 @@ To perform a **manual** multi-version upgrade on a Sourcegraph instance running
168163
2. Run the following commands to schedule the migrator job with the upgrade command and monitor its progress:
169164
```sh
170165
# To ensure no previous job invocations will conflict with our current invocation
171-
kubectl delete -f configure/migrator/migrator.Job.yaml
166+
kubectl delete job migrator
172167
# Start the migrator job
173-
kubectl apply -f configure/migrator/migrator.Job.yaml
168+
kubectl apply -f components/utils/migrator/resources/migrator.Job.yaml
174169
# Stream the migrator's stdout logs for progress
175-
kubectl logs job.batch/migrator -f
170+
kubectl logs job/migrator -f
176171
```
177172
**Example:**
178173
```sh
@@ -209,14 +204,19 @@ To perform a **manual** multi-version upgrade on a Sourcegraph instance running
209204
migrator 1/1 9s 35s
210205
```
211206

212-
4. **Pull and merge upstream changes**:
213-
- Follow the [standard legacy upgrade procedure](#upgrade-with-legacy-kubernetes) to pull and merge upstream changes from the version you are upgrading to to your `release` branch.
214-
215-
5. **Scale your replicas back up and apply new manifests**:
216-
- The [legacy kubernetes upgrade procedure](#upgrade-with-legacy-kubernetes) describes this step in more detail.
217-
- Ensure that the replica counts adjusted in the previous steps are turned back up.
218-
- Run `./kubectl-apply-all.sh` to deploy the new pods to the Kubernetes cluster.
219-
- Monitor the status of the deployment via `kubectl get pods -o wide --watch`.
207+
5. **Generate and apply a new cluster.yaml file**:
208+
1. Comment out the [multi-version-upgrade util](https://github.com/sourcegraph/deploy-sourcegraph-k8s/blob/main/instances/template/kustomization.template.yaml#L155) in your cluster kustomization file (`instances/my-sourcegraph/kustomize.yaml`).
209+
```yaml
210+
# - ../../components/utils/uid # -- Run all Postgres database with valid users on host
211+
# - ../../components/utils/multi-version-upgrade # -- Scale down non-database pods to 0 for multi-version upgrade
212+
# - ../../components/utils/migrate-to-nonprivileged # -- Component for migrating from privileged to non-privileged
213+
```
214+
2. Generate and apply a new cluster.yaml with the new images from Step 2.
215+
```sh
216+
kubectl kustomize instances/my-sourcegraph -o cluster.yaml
217+
kubectl apply --prune -l deploy=sourcegraph -f cluster.yaml
218+
```
219+
Your instance should now be up and running in the new version!
220220

221221
---
222222

0 commit comments

Comments
 (0)