sourcegraph 6 aware upgrade docs

docs/admin/how-to/index.mdx

@@ -28,3 +28,4 @@
 - [Migrating code intelligence data from LSIF to SCIP (Sourcegraph 4.5 -> 4.6)](/admin/how-to/lsif_scip_migration)
 - [How to export search results](/admin/how-to/export-search-results)
 - [How to debug / confirm blobstore is healthy](/admin/how-to/blobstore_debugging)
+- [How to handle postgresql 12 to 16 drift](/admin/how-to/postgresql-12-to-16-drift)

docs/admin/how-to/postgres_12_to_16_drift.mdx

@@ -0,0 +1,189 @@
+# PostgreSQL 12 to 16 Schema Drift
+
+In Sourcegraph versions `5.10.x` and `5.11.x` we support both PostgreSQL 12 and 16. However, Sourcegraph's database management tool `migrator` expects the database schema of the various Sourcegraph databases to be in an exact expected state. The upgrade from PostgreSQL 12 to 16 is opinionated and automatically mutates the schema without running our application defined migrations. Starting in Sourcegraph `5.10.0` we expect databases to be in PosttgresSQL 16 and as such our tooling will identify schema drift in PostgreSQL 12 databases. This drift does not impact the functionality of the Sourcegraph instance but will stop migrator's multiversion `upgrade` command and `autoupgrade` from executing.
+
+The drift takes the following general form, dropping table prefixes to columns in views, and changing `uuid` types to `gen_random_uuid()`:
+```diff
+Schema drift detected in 8 objects:
+
+webhooks.uuid:
+Problem: Unexpected properties of column webhooks."uuid"
+Solution: alter the column
+Statement: ALTER TABLE webhooks ALTER COLUMN uuid SET DEFAULT public.gen_random_uuid();
+
+batch_spec_workspace_execution_queue:
+Problem: Unexpected definition of view "batch_spec_workspace_execution_queue"
+Solution: redefine the view
+Diff:
+  (
+  	"""
+  	... // 6 identical lines
+  	          ORDER BY (rank() OVER (PARTITION BY queue.user_id ORDER BY exec.created_at, exec.id)), queue.latest_dequeue NULLS FIRST
+  	        )
+- 	 SELECT id,
++ 	 SELECT queue_candidates.id,
+  	    row_number() OVER () AS place_in_global_queue,
+- 	    place_in_user_queue
++ 	    queue_candidates.place_in_user_queue
+  	   FROM queue_candidates;
+  	"""
+  )
+
+codeintel_configuration_policies:
+Problem: Unexpected definition of view "codeintel_configuration_policies"
+Solution: redefine the view
+Diff:
+  (
+  	"""
+- 	 SELECT id,
+- 	    repository_id,
+- 	    name,
+- 	    type,
+- 	    pattern,
+- 	    retention_enabled,
+- 	    retention_duration_hours,
+- 	    retain_intermediate_commits,
+- 	    indexing_enabled,
+- 	    index_commit_max_age_hours,
+- 	    index_intermediate_commits,
+- 	    protected,
+- 	    repository_patterns,
+- 	    last_resolved_at,
+- 	    embeddings_enabled
++ 	 SELECT lsif_configuration_policies.id,
++ 	    lsif_configuration_policies.repository_id,
++ 	    lsif_configuration_policies.name,
++ 	    lsif_configuration_policies.type,
++ 	    lsif_configuration_policies.pattern,
++ 	    lsif_configuration_policies.retention_enabled,
++ 	    lsif_configuration_policies.retention_duration_hours,
++ 	    lsif_configuration_policies.retain_intermediate_commits,
++ 	    lsif_configuration_policies.indexing_enabled,
++ 	    lsif_configuration_policies.index_commit_max_age_hours,
++ 	    lsif_configuration_policies.index_intermediate_commits,
++ 	    lsif_configuration_policies.protected,
++ 	    lsif_configuration_policies.repository_patterns,
++ 	    lsif_configuration_policies.last_resolved_at,
++ 	    lsif_configuration_policies.embeddings_enabled
+  	   FROM lsif_configuration_policies;
+  	"""
+  )
+
+codeintel_configuration_policies_repository_pattern_lookup:
+Problem: Unexpected definition of view "codeintel_configuration_policies_repository_pattern_lookup"
+Solution: redefine the view
+Diff:
+  (
+  	"""
+- 	 SELECT policy_id,
+- 	    repo_id
++ 	 SELECT lsif_configuration_policies_repository_pattern_lookup.policy_id,
++ 	    lsif_configuration_policies_repository_pattern_lookup.repo_id
+  	   FROM lsif_configuration_policies_repository_pattern_lookup;
+  	"""
+  )
+
+lsif_dumps:
+Problem: Unexpected definition of view "lsif_dumps"
+Solution: redefine the view
+Diff:
+  strings.Join({
+  	" SELECT ",
++ 	"u.",
+  	"id,\n    ",
++ 	"u.",
+  	"commit,\n    ",
+- 	"root,\n    queued_at,\n    uploaded_at,\n    state,\n    ",
++ 	"u.root,\n    u.queued_at,\n    u.uploaded_at,\n    u.state,\n    u.",
+  	"failure_message,\n    ",
++ 	"u.",
+  	"started_at,\n    ",
++ 	"u.",
+  	"finished_at,\n    ",
++ 	"u.",
+  	"repository_id,\n    ",
++ 	"u.",
+  	"indexer,\n    ",
++ 	"u.",
+  	"indexer_version,\n    ",
++ 	"u.",
+  	"num_parts,\n    ",
++ 	"u.",
+  	"uploaded_parts,\n    ",
++ 	"u.",
+  	"process_after,\n    ",
++ 	"u.",
+  	"num_resets,\n    ",
++ 	"u.",
+  	"upload_size,\n    ",
++ 	"u.",
+  	"num_failures,\n    ",
++ 	"u.",
+  	"associated_index_id,\n    ",
++ 	"u.",
+  	"expired,\n    ",
++ 	"u.",
+  	"last_retention_scan_at,\n    ",
++ 	"u.",
+  	"finished_at AS processed_at\n   FROM lsif_uploads u\n  WHERE ((",
++ 	"u.",
+  	"state = 'completed'::text) OR (",
++ 	"u.",
+  	"state = 'deleting'::text));",
+  }, "")
+
+outbound_webhooks_with_event_types:
+Problem: Unexpected definition of view "outbound_webhooks_with_event_types"
+Solution: redefine the view
+Diff:
+  (
+  	"""
+- 	 SELECT id,
+- 	    created_by,
+- 	    created_at,
+- 	    updated_by,
+- 	    updated_at,
+- 	    encryption_key_id,
+- 	    url,
+- 	    secret,
++ 	 SELECT outbound_webhooks.id,
++ 	    outbound_webhooks.created_by,
++ 	    outbound_webhooks.created_at,
++ 	    outbound_webhooks.updated_by,
++ 	    outbound_webhooks.updated_at,
++ 	    outbound_webhooks.encryption_key_id,
++ 	    outbound_webhooks.url,
++ 	    outbound_webhooks.secret,
+  	    array_to_json(ARRAY( SELECT json_build_object('id', outbound_webhook_event_types.id, 'outbound_webhook_id', outbound_webhook_event_types.outbound_webhook_id, 'event_type', outbound_webhook_event_types.event_type, 'scope', outbound_webhook_event_types.scope) AS json_build_object
+  	           FROM outbound_webhook_event_types
+  	... // 2 identical lines
+  	"""
+  )
+
+site_config:
+Problem: Unexpected definition of view "site_config"
+Solution: redefine the view
+Diff:
+  (
+  	"""
+- 	 SELECT site_id,
+- 	    initialized
++ 	 SELECT global_state.site_id,
++ 	    global_state.initialized
+  	   FROM global_state;
+  	"""
+  )
+```
+
+## Solutions for Handling Schema Drift
+
+If you're confident that your instance is seeing database drift associated with the PG12 to PG16 upgrade, you can run a nultiversion upgrade via migrator `upgrade` or run `autoupgrade` using the following options.
+
+To run `autoupgrade` via the frontend, set the `SRC_AUTOUPGRADE_IGNORE_DRIFT=true` environment variable in the frontend container.
+
+To run migrators `upgrade` command add the `--skip-drift-check` flag to migrator's entrycommand as below:
+```yaml
+command: ['upgrade', '-from', '5.5.0', '-to', '5.10.0', '--skip-drift-check=true']             
+```
+
+If you have any concerns about the drift, please reach out to us at [email protected]

docs/admin/postgres.mdx

@@ -8,7 +8,14 @@ Sourcegraph uses several PostgreSQL databases to support various functionality.
 
 ## Version requirements
 
-We support Postgres 16 and above. Older versions of Sourcegraph supported 12 see our [Postgres 12 deprecation notice](/admin/postgres12_end_of_life_notice) for more details.
+We support Postgres 16 and above. Older versions of Sourcegraph supported 12 see our [Postgres 12 deprecation notice](/admin/postgres12_end_of_life_notice) for more details. Starting in Sourcegraph 6.0.0, sourcegraph services will no longer connect to a PostgreSQL 12 or lower database. Instead displaying an error message in container logs for services attempting to connect to a PostgreSQL database below version 16.
+```
+migrator  | ✱ Sourcegraph migrator 6.0.0
+migrator  | ⚠️ Failed to check for migrator update: unexpected status code 404. Continuing...
+migrator  | Attempting connection to frontend...
+migrator  | {"SeverityText":"FATAL","Timestamp":1738107940979396426,"InstrumentationScope":"migrator","Caller":"migrator/main.go:24","Function":"main.main","Body":"new db handle: Sourcegraph requires PostgreSQL 16+. For more information about PostgreSQL requirements and upgrade guides, visit https://sourcegraph.com/docs/admin/postgres","Resource":{"service.name":"migrator","service.version":"6.0.0","service.instance.id":"23097a97-50f4-4b42-80b5-83802baffeb4"},"Attributes":{}}
+migrator exited with code 1
+```
 
 ## Role requirements
 
@@ -29,11 +36,27 @@ vector
 
 ### Upgrading Built-in PostgreSQL
 
-Sourcegraph deployments come packaged with containers using the `postgresql-16` image for `pgsql` and `codeintel-db` deployments and `postgresql-16-codeinsights` for `codeinsights`. These images contain an entryscript which will detect and upgrade Postgres instances at version 12 (our previous images Postgres version). See the (Postgres 12 end of life notice)[/admin/prostgres12_end_of_life_notice#] for more details. 
+Starting in version `5.10.0`, Sourcegraph begins supporting PostgreSQL 16. For admins deploying our built-in PostgreSQL, we facilitate the upgrade of PostgreSQL using entrypoint scripts packaged in the new `postgresql-16` and `postgresql-16-codeinsights` container images which become standard in our deployment repos in `5.10.0`. The database container images are renamed as follows:
+
+**postgres-12-alpine → postgresql-16**
+
+**codeintel-db → postgresql-16**
+
+**codeinsights-db → postgresql-16-codeinsights**
 
 > WARNING: Upgrading the PostgreSQL database requires stopping your Sourcegraph deployment which will result in **downtime**.
 >
-> Additionally, once the upgrade process is started via the database container, interrupting the container before the upgrade is complete could result in corrupting the underlying Postgres database. We strongly advise taking a backup before the upgrade. 
+> Once the database upgrade process is started, containers running the upgrade should not be stopped or restarted.
+
+When our `postgresql-16` images start, if they detect a PostgresSQL 12 database in their attached volume, they will automatically attempt to upgrade the database to PostgreSQL 16.
+
+We have the following advisements to admins:
+- Container orchestration management systems (e.g. Kubernetes) may restart containers at any time, **it is recommended that you take a backup of your database before starting the upgrade process.**
+- Review your deployment types [upgrade notes](/admin/updates#upgrades-index) before upgrading. The PostgreSQL upgrade process mutates the database schema in a way that may Sourcegraph `v5.10.0` and `v5.11.0` may not recognize. [Learn more about pg12 to pg16 schema drift here](/admin/how-to/postgres_12_to_16_drift).
+
+For additional assistance with PostgreSQL upgrades, please contact [email protected].
+
+#### Upgrade entryscript `--link` option
 
 The `PG_UPGRADE_EXTRA_ARGS` env var can be set in the built-in `postgresql-16` and `postgresql-16-codeinsights` deployments to provide extra arguments to the `pg_upgrade` command. 
 

docs/admin/updates/docker_compose.mdx

@@ -11,7 +11,9 @@ For upgrade procedures or general info about sourcegraph versioning see the link
 >
 > ***If the notes indicate a patch release exists, target the highest one.***
 
-## Unreleased
+## v6.0.0
+
+- Sourcegraph 6.0.0 no longer supports PostgreSQL 12, admins must upgrade to PostgreSQL 16. See our [postgres 12 end of life](/admin/postgres12_end_of_life_notice) notice! As well as [supporting documentation](/admin/postgres) and advisements on how to upgrade.
 
 ## v5.9.0 ➔ v5.10.1164
 

docs/admin/updates/kubernetes.mdx

@@ -12,8 +12,9 @@ For upgrade procedures or general info about sourcegraph versioning see the link
 >
 > ***If the notes indicate a patch release exists, target the highest one.***
 
+## v6.0.0
 
-## Unreleased
+- Sourcegraph 6.0.0 no longer supports PostgreSQL 12, admins must upgrade to PostgreSQL 16. See our [postgres 12 end of life](/admin/postgres12_end_of_life_notice) notice! As well as [supporting documentation](/admin/postgres) and advisements on how to upgrade.
 
 ## v5.9.0 ➔ v5.10.1164