-
Notifications
You must be signed in to change notification settings - Fork 78
chore(release): pg 16 update to docs #849
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 1 commit
Commits
Show all changes
8 commits
Select commit
Hold shift + click to select a range
baa3534
docs update
DaedalusG cf9d5d8
Update docs/admin/external_services/postgres.mdx
DaedalusG f5fb505
Update docs/admin/postgres.mdx
DaedalusG ceff06d
Update docs/admin/postgres.mdx
DaedalusG 264c01e
Update docs/admin/postgres.mdx
DaedalusG 3e38184
Update docs/admin/postgres.mdx
DaedalusG 99dcadb
Merge branch 'main' into wg/rel/pg16-doc-update
DaedalusG b6ddae4
address review feedback
DaedalusG File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -4,12 +4,11 @@ Sourcegraph uses several PostgreSQL databases to support various functionality. | |
|
|
||
| - `pgsql` or `primary`: responsible for user data and account information | ||
| - `codeintel-db`: provides support for lsif data and part of the code-intelligence | ||
| - `codeinsights-db`: provides support for code insights data | ||
|
|
||
| ## Version requirements | ||
|
|
||
| We support any version **starting from 12**. | ||
|
|
||
| > NOTE: ⚠️ Version **3.26** required only Postgres 9.6. You should check the [upgrade docs for your deployment type](/admin/updates/) as well as the [database upgrade docs](#upgrading-postgresql) below prior to upgrading to 3.27. | ||
| We support postgres 16 and above. Older versions of Sourcegraph supported 12 see our [postgres 12 deprecation notice](/admin/postgres12_end_of_life_notice) for more details. | ||
DaedalusG marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| ## Role requirements | ||
|
|
||
|
|
@@ -23,19 +22,23 @@ intarray | |
| pg_stat_statements | ||
| pg_trgm | ||
| pgcrypto | ||
| vector | ||
| pg_vector | ||
DaedalusG marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
DaedalusG marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| ``` | ||
|
|
||
| ## [Configuring PostgreSQL](/admin/config/postgres-conf) | ||
|
|
||
| ## Upgrading PostgreSQL | ||
| ### Upgrading Built-in PostgreSQL | ||
|
|
||
| Sourcegraph uses PostgreSQL as its main database and this documentation describes how to upgrade PostgreSQL between major versions. | ||
| Sourcegraph deployments come packaged with containers using the `postgresql-16` image for `pgsql` and `codeintel-db` deployments and `postgresql-16-codeinsights` for `codeinsights`. These images contain an entryscript which will detect and upgrade postgres instances below version 12. See the (postgres 12 end of life notice)[/admin/prostgres12_end_of_life_notice#] for more details. | ||
DaedalusG marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| > WARNING: Upgrading the PostgreSQL database requires stopping your Sourcegraph deployment which will result in **downtime**. | ||
| > | ||
| > Additionally, once started the upgrade process stopping the database container will corrupt sourcegraph data. We advise taking a backup before the upgrade. | ||
DaedalusG marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| The `PG_UPGRADE_EXTRA_ARGS` env var can be set in the built-in `postgresql-16` and `postgresql-16-codeinsights` deployments to provide extra arguments to the `pg_upgrade` command. | ||
|
|
||
|
|
||
| > NOTE: If you running PostgreSQL externally, see [Upgrading external PostgreSQL instances](/admin/postgres#upgrading-external-postgresql-instances) | ||
| Setting `PG_UPGRADE_EXTRA_ARGS=--link` provides the `--link` option to `pg_update`. Creating a hard link between the old and new data directories. This option is faster and requires less disk space, but if containers are killed during the upgrade, corrupted data will be unrecoverable without a backup. | ||
DaedalusG marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| ### Upgrading external PostgreSQL instances | ||
|
|
||
|
|
@@ -52,100 +55,11 @@ When running an [external PostgreSQL instance](/admin/external_services/postgres | |
| - [EnterpriseDB](https://www.enterprisedb.com/docs/en/9.6/pg/upgrading.html) | ||
| - [Citus](http://docs.citusdata.com/en/v8.1/admin_guide/upgrading_citus.html) | ||
| - [Aiven PostgreSQL](https://help.aiven.io/postgresql/operations/how-to-perform-a-postgresql-in-place-major-version-upgrade) | ||
| - [Your own PostgreSQL](https://www.postgresql.org/docs/11/pgupgrade.html) | ||
| - [Your own PostgreSQL](https://www.postgresql.org/docs/16/pgupgrade.html) | ||
| 4. Turn Sourcegraph back on connecting to the now-upgraded database. | ||
|
|
||
| > WARNING: **Do not allow Sourcegraph to run/connect to the new Postgres database until it has been fully populated with your data.** Doing so could result in Sourcegraph trying to create e.g. a new DB schema and partially migrating. If this happens to you, restore from the backup you previously took or contact us ([email protected]) | ||
|
|
||
| ### Upgrading internal PostgreSQL instances | ||
|
|
||
| Many of Sourcegraph's [deployment methods](/admin/deploy/) come with internal PostgreSQL instances that can be deployed as part of a Sourcegraph installation. This section outlines how to upgrade these databases. | ||
|
|
||
| #### Sourcegraph with Docker Compose | ||
|
|
||
| Generally, no additional steps are required to upgrade the databases shipped alongside [Sourcegraph with Docker Compose](/admin/deploy/docker-compose/). | ||
|
|
||
| #### Sourcegraph with Kubernetes | ||
|
|
||
| **The upgrade process is different for [Sourcegraph Kubernetes deployments](/admin/deploy/kubernetes/)** because [by default](https://github.com/sourcegraph/sourcegraph-public-snapshot/blob/main/docker-images/postgres-12/build.sh#L10), it uses `sourcegraph/postgres-12` which can be [customized with environment variables](https://github.com/sourcegraph/deploy-sourcegraph/blob/7edcadb/docs/configure#external-postgres). | ||
|
|
||
| If you have changed `PGUSER`, `PGDATABASE` or `PGDATA`, then the `PG*OLD` and `PG*NEW` environment variables are | ||
| required. Below are the defaults and documentation on what each variable is used for: | ||
|
|
||
| - `POSTGRES_PASSWORD=''`: Password of `PGUSERNEW` if it is newly created (i.e when `PGUSERNEW` didn't exist in the old | ||
| database). | ||
| - `PGUSEROLD=sg`: A user that exists in the old database that can be used to authenticate intermediate upgrade | ||
| operations. | ||
| - `PGUSERNEW=sg`: A user that must exist in the new database after the upgrade is done (i.e. it'll be created if it | ||
| didn't exist already). | ||
| - `PGDATABASEOLD=sg`: A database that exists in the old database that can be used to authenticate intermediate upgrade | ||
| operations. (e.g `psql -d`) | ||
| - `PGDATABASENEW=sg`: A database that must exist in the new database after the upgrade is done (i.e. it'll be created if | ||
| it didn't exist already). | ||
| - `PGDATAOLD=/data/pgdata-11`: The data directory containing the files of the old PostgreSQL database to be upgraded. | ||
| - `PGDATANEW=/data/pgdata-12`: The data directory containing the upgraded PostgreSQL data files, used by the new version | ||
| of PostgreSQL. | ||
|
|
||
| Additionally, the upgrade process assumes it can write to the parent directory of `PGDATAOLD`. | ||
|
|
||
| ##### Requirements for upgrading from Postgres 11 to 12 for Kubernetes | ||
|
|
||
| **This guide is for the Sourcegraph 3.27 release.** The migration to Postgres 12 using the Sourcegraph provided Postgres images is largely automated. | ||
| However, you should take some steps before the migration. For the `3.27` release, these apply to the `pgsql` & `codeintel-db` deployments. | ||
|
|
||
| 1. Inform your users prior to starting the upgrade that Sourcegraph will take downtime. This downtime scales with the | ||
| size and resources allocated to the Postgres database instances. | ||
| 1. Take snapshots of all Sourcegraph Postgres databases. You can find the backing disk in your cloud by describing the | ||
| relevant `PersistentVolumes`. | ||
| 1. Ensure that the size of the disks backing the databases are under 50% utilization. (You will need *up-to* 2x the size | ||
| of your current database to do the migration). In most cases Kubernetes can expand the size of the volume for you, | ||
| see the relevant kubernetes | ||
| docs [here](https://kubernetes.io/docs/concepts/storage/persistent-volumes/#expanding-persistent-volumes-claims). | ||
|
|
||
| You can use `kubectl exec -it $DB_POD_NAME -- df -h`. By default, the database will be mounted under `/data/`. | ||
|
|
||
| #### Single-container Sourcegraph | ||
|
|
||
| When running a new version of Sourcegraph, [single-container Sourcegraph with Docker instances](/admin/deploy/kubernetes/) will check if the PostgreSQL data needs upgrading upon initialization. | ||
|
|
||
| See the following steps to upgrade between major postgresql versions: | ||
|
|
||
| **1.** Stop the `sourcegraph/server` container. | ||
|
|
||
| **2.** Save this script and give it executable permissions (`chmod + x`). | ||
|
|
||
| > NOTE: The script presumes your data is being upgraded from `11` to `12` and your Sourcegraph directory is at `~/.sourcegraph/`. Change the values in the code below if that's not the case. | ||
|
|
||
| ```bash | ||
| #!/usr/bin/env bash | ||
|
|
||
| set -xeuo pipefail | ||
|
|
||
| export OLD=${OLD:-"11"} | ||
| export NEW=${NEW:-"12"} | ||
| export SRC_DIR=${SRC_DIR:-"$HOME/.sourcegraph"} | ||
|
|
||
| docker run \ | ||
| -w /tmp/upgrade \ | ||
| -v "$SRC_DIR/data/postgres-$NEW-upgrade:/tmp/upgrade" \ | ||
| -v "$SRC_DIR/data/postgresql:/var/lib/postgresql/$OLD/data" \ | ||
| -v "$SRC_DIR/data/postgresql-$NEW:/var/lib/postgresql/$NEW/data" \ | ||
| "tianon/postgres-upgrade:$OLD-to-$NEW" | ||
|
|
||
| mv "$SRC_DIR/data/"{postgresql,postgresql-$OLD} | ||
| mv "$SRC_DIR/data/"{postgresql-$NEW,postgresql} | ||
|
|
||
| curl -fsSL -o "$SRC_DIR/data/postgres-$NEW-upgrade/optimize.sh" https://raw.githubusercontent.com/sourcegraph/sourcegraph/master/cmd/server/rootfs/postgres-optimize.sh | ||
|
|
||
| docker run \ | ||
| --entrypoint "/bin/bash" \ | ||
| -w /tmp/upgrade \ | ||
| -v "$SRC_DIR/data/postgres-$NEW-upgrade:/tmp/upgrade" \ | ||
| -v "$SRC_DIR/data/postgresql:/var/lib/postgresql/data" \ | ||
| "postgres:$NEW" \ | ||
| -c 'chown -R postgres $PGDATA . && gosu postgres bash /admin/optimize.sh $PGDATA' | ||
| ``` | ||
|
|
||
| **3.** Execute the script. | ||
|
|
||
| **4.** Start the `sourcegraph/server` container. | ||
| Sourcegraph Single-container is not a supported production deployment and currently doesn't support upgrades to Postgres 16. Admins with production data in a single container deployment are advised to reach out to [email protected] for assistance. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.