You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
It’s possible for the Sourcegraph instance to deny access to hosts by setting the environment variable `EXTERNAL_DENY_LIST` on the deployment. If you want to only prevent codemonitors and other frontend services to connect to arbitrary hosts, you can set the variable only on the frontend deployment.
3
+
The external denylist supports a comma separated list of IP ranges, hostnames and keywords. To block all the internal connections use the “private” keyword, this would block all RFC 1918 (10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16) and RFC 4193 (FC00::/7) IP addresses. Keywords can be combined with ranges and IP addresses so it's very customizable.
4
+
5
+
The default denylist is set up to only block localhost and the Cloud metadata service IP address. Expanding the denylist could interfere with internal authentication providers, and they might need to be excluded from the denylist.
6
+
7
+
## Example Configuration
8
+
9
+
```
10
+
EXTERNAL_DENY_LIST="private,github.com"
11
+
```
12
+
13
+
This would deny all connections to hosts in the private network and github.com.
14
+
15
+
## Implementation Details
16
+
To achieve this, we use [gitea's hostmatcher](https://github.com/go-gitea/gitea/blob/v1.22.6/modules/hostmatcher/hostmatcher.go#L39). This is configured by default for the `ExternalClient`, which is used for all external requests. The common options and configuration can be found [here](https://github.com/sourcegraph/sourcegraph-public-snapshot/blob/main/internal/httpcli/client.go#L406C1-L423C2).
Copy file name to clipboardExpand all lines: docs/admin/deploy/kubernetes/index.mdx
+4-6Lines changed: 4 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -21,20 +21,18 @@ Helm offers a simple deployment process on Kubernetes using well known and stand
21
21
22
22
</QuickLinks>
23
23
24
-
<Callouttype="warning">Sourcegraph currently does not support migration from an existing Sourcegraph deployment without Helm to a Sourcegraph deployment using Helm. The information you are looking at is currently recommended for a new install of Sourcegraph. We are currently working to provide migration guidance from a non-Helm deployment. If you are inquiring about performing such a migration please email `[email protected]`.</Callout>
25
-
26
24
## Why use Helm
27
25
28
26
Our Helm chart has a lot of sensible defaults baked into the values.yaml so that when an override file is used to make the changes, you _never_ have to deal with merge conflicts during upgrades (see more about customizations in the [configuration](#configuration) section).
29
27
30
28
## High-level overview of how to use Helm with Sourcegraph
31
29
32
30
1. Prepare any required customizations
33
-
- Most environments are likely to need changes from the defaults—use the guidance in [Configuration](#configuration).
34
-
-These changes are inclusive of updates to resourcing for each service. See our [resource estimator](/admin/deploy/resource_estimator) for more information.
35
-
1. Review the changes
31
+
- Most environments will likely require some customizations to the default Helm chart values. See _[Configuration](#configuration)_ for more information.
32
+
-Additionally, resource allocations for individual services may need to be adjusted. See our _[Resource Estimator](/admin/deploy/resource_estimator)_ for more information.
33
+
2. Review the customized Helm chart
36
34
- There are [three mechanisms](#reviewing-changes) that can be used to review any customizations made, this is an optional step, but may be useful the first time you deploy Sourcegraph, for peace of mind.
37
-
1. Select your deployment method and follow the guidance:
35
+
3. Follow the relevant platform-specific guide below to deploy Sourcegraph to your environment:
You can use your own PostgreSQL v12+ server with Sourcegraph if you wish. For example, you may prefer this if you already have existing backup infrastructure around your own PostgreSQL server, wish to use Amazon RDS, etc.
5
+
You can use your own PostgreSQL v16+ server with Sourcegraph if you wish. For example, you may prefer this if you already have existing backup infrastructure around your own PostgreSQL server, wish to use Amazon RDS, etc.
6
6
7
7
Please review the [PostgreSQL](/admin/postgres) documentation for a complete list of requirements.
8
8
9
-
<Callouttype="note">As of version 3.39.0, codeinsights-db no longer relies on the internal TimescaleDB and can be externalized.</Callout>
9
+
<Callouttype="note">As of version 6.0.0 The Sourcegraph application will require PostgreSQL v16+, earlier versions worked with 12 or greater.</Callout>
Copy file name to clipboardExpand all lines: docs/admin/postgres.mdx
+13-97Lines changed: 13 additions & 97 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -4,12 +4,11 @@ Sourcegraph uses several PostgreSQL databases to support various functionality.
4
4
5
5
-`pgsql` or `primary`: responsible for user data and account information
6
6
-`codeintel-db`: provides support for lsif data and part of the code-intelligence
7
+
-`codeinsights-db`: provides support for code insights data
7
8
8
9
## Version requirements
9
10
10
-
We support any version **starting from 12**.
11
-
12
-
> NOTE: ⚠️ Version **3.26** required only Postgres 9.6. You should check the [upgrade docs for your deployment type](/admin/updates/) as well as the [database upgrade docs](#upgrading-postgresql) below prior to upgrading to 3.27.
11
+
We support Postgres 16 and above. Older versions of Sourcegraph supported 12 see our [Postgres 12 deprecation notice](/admin/postgres12_end_of_life_notice) for more details.
Sourcegraph uses PostgreSQL as its main database and this documentation describes how to upgrade PostgreSQL between major versions.
32
+
Sourcegraph deployments come packaged with containers using the `postgresql-16` image for `pgsql`and `codeintel-db` deployments and `postgresql-16-codeinsights` for `codeinsights`. These images contain an entryscript which will detect and upgrade Postgres instances at version 12 (our previous images Postgres version). See the (Postgres 12 end of life notice)[/admin/prostgres12_end_of_life_notice#] for more details.
33
33
34
34
> WARNING: Upgrading the PostgreSQL database requires stopping your Sourcegraph deployment which will result in **downtime**.
35
+
>
36
+
> Additionally, once the upgrade process is started via the database container, interrupting the container before the upgrade is complete could result in corrupting the underlying Postgres database. We strongly advise taking a backup before the upgrade.
37
+
38
+
The `PG_UPGRADE_EXTRA_ARGS` env var can be set in the built-in `postgresql-16` and `postgresql-16-codeinsights` deployments to provide extra arguments to the `pg_upgrade` command.
35
39
40
+
Setting the env var `PG_UPGRADE_EXTRA_ARGS=--link` provides the `--link` option to the underlying `pg_update`; creating a hard links between the old and new data directories instead of copying data. This option is faster and requires less disk space, however if the database containers are interrupted during the upgrade process there is a strong possibility of data corruption. This can still be a viable choice for users who are confident in their container scheduling and backups and would like to avoid longer downtimes.
36
41
42
+
If not using the `--link` option, you will need enough disk space available for Postgres to make a copy of the existing data. For example, if your database is currently using 100GiB of storage, you will need an additional 100GiB free space in order for the upgrade process to succeed.
37
43
38
-
> NOTE: If you running PostgreSQL externally, see [Upgrading external PostgreSQL instances](/admin/postgres#upgrading-external-postgresql-instances)
39
44
40
45
### Upgrading external PostgreSQL instances
41
46
@@ -52,100 +57,11 @@ When running an [external PostgreSQL instance](/admin/external_services/postgres
-[Your own PostgreSQL](https://www.postgresql.org/docs/11/pgupgrade.html)
60
+
-[Your own PostgreSQL](https://www.postgresql.org/docs/16/pgupgrade.html)
56
61
4. Turn Sourcegraph back on connecting to the now-upgraded database.
57
62
58
63
> WARNING: **Do not allow Sourcegraph to run/connect to the new Postgres database until it has been fully populated with your data.** Doing so could result in Sourcegraph trying to create e.g. a new DB schema and partially migrating. If this happens to you, restore from the backup you previously took or contact us ([email protected])
59
64
60
-
### Upgrading internal PostgreSQL instances
61
-
62
-
Many of Sourcegraph's [deployment methods](/admin/deploy/) come with internal PostgreSQL instances that can be deployed as part of a Sourcegraph installation. This section outlines how to upgrade these databases.
63
-
64
-
#### Sourcegraph with Docker Compose
65
-
66
-
Generally, no additional steps are required to upgrade the databases shipped alongside [Sourcegraph with Docker Compose](/admin/deploy/docker-compose/).
67
-
68
-
#### Sourcegraph with Kubernetes
69
-
70
-
**The upgrade process is different for [Sourcegraph Kubernetes deployments](/admin/deploy/kubernetes/)** because [by default](https://github.com/sourcegraph/sourcegraph-public-snapshot/blob/main/docker-images/postgres-12/build.sh#L10), it uses `sourcegraph/postgres-12` which can be [customized with environment variables](https://github.com/sourcegraph/deploy-sourcegraph/blob/7edcadb/docs/configure#external-postgres).
71
-
72
-
If you have changed `PGUSER`, `PGDATABASE` or `PGDATA`, then the `PG*OLD` and `PG*NEW` environment variables are
73
-
required. Below are the defaults and documentation on what each variable is used for:
74
-
75
-
-`POSTGRES_PASSWORD=''`: Password of `PGUSERNEW` if it is newly created (i.e when `PGUSERNEW` didn't exist in the old
76
-
database).
77
-
-`PGUSEROLD=sg`: A user that exists in the old database that can be used to authenticate intermediate upgrade
78
-
operations.
79
-
-`PGUSERNEW=sg`: A user that must exist in the new database after the upgrade is done (i.e. it'll be created if it
80
-
didn't exist already).
81
-
-`PGDATABASEOLD=sg`: A database that exists in the old database that can be used to authenticate intermediate upgrade
82
-
operations. (e.g `psql -d`)
83
-
-`PGDATABASENEW=sg`: A database that must exist in the new database after the upgrade is done (i.e. it'll be created if
84
-
it didn't exist already).
85
-
-`PGDATAOLD=/data/pgdata-11`: The data directory containing the files of the old PostgreSQL database to be upgraded.
86
-
-`PGDATANEW=/data/pgdata-12`: The data directory containing the upgraded PostgreSQL data files, used by the new version
87
-
of PostgreSQL.
88
-
89
-
Additionally, the upgrade process assumes it can write to the parent directory of `PGDATAOLD`.
90
-
91
-
##### Requirements for upgrading from Postgres 11 to 12 for Kubernetes
92
-
93
-
**This guide is for the Sourcegraph 3.27 release.** The migration to Postgres 12 using the Sourcegraph provided Postgres images is largely automated.
94
-
However, you should take some steps before the migration. For the `3.27` release, these apply to the `pgsql` & `codeintel-db` deployments.
95
-
96
-
1. Inform your users prior to starting the upgrade that Sourcegraph will take downtime. This downtime scales with the
97
-
size and resources allocated to the Postgres database instances.
98
-
1. Take snapshots of all Sourcegraph Postgres databases. You can find the backing disk in your cloud by describing the
99
-
relevant `PersistentVolumes`.
100
-
1. Ensure that the size of the disks backing the databases are under 50% utilization. (You will need *up-to* 2x the size
101
-
of your current database to do the migration). In most cases Kubernetes can expand the size of the volume for you,
You can use `kubectl exec -it $DB_POD_NAME -- df -h`. By default, the database will be mounted under `/data/`.
106
-
107
65
#### Single-container Sourcegraph
108
66
109
-
When running a new version of Sourcegraph, [single-container Sourcegraph with Docker instances](/admin/deploy/kubernetes/) will check if the PostgreSQL data needs upgrading upon initialization.
110
-
111
-
See the following steps to upgrade between major postgresql versions:
112
-
113
-
**1.** Stop the `sourcegraph/server` container.
114
-
115
-
**2.** Save this script and give it executable permissions (`chmod + x`).
116
-
117
-
> NOTE: The script presumes your data is being upgraded from `11` to `12` and your Sourcegraph directory is at `~/.sourcegraph/`. Change the values in the code below if that's not the case.
Sourcegraph Single-container is not a supported production deployment and currently doesn't support upgrades to Postgres 16. Admins with production data in a single container deployment are advised to reach out to [email protected] for assistance.
Copy file name to clipboardExpand all lines: docs/admin/updates/index.mdx
+7-4Lines changed: 7 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -22,9 +22,9 @@ Sourcegraph releases have three digits, for example: `v5.0.3`
22
22
| minor |`0`|
23
23
| patch |`3`|
24
24
25
-
For more details on our release cadence, refer to the [Sourcegraph handbook](https://handbook.sourcegraph.com/departments/engineering/dev/process/releases/#sourcegraph-releases).
25
+
For more details on previous releases see [Sourcegraph Releases](/releases).
26
26
27
-
Generally, we ship a new version of Sourcegraph with new features and bug fixes monthly, followed by a patch release later that month. Major and minor releases happen less frequently and contain substantial changes to Sourcegraph.
27
+
Generally, we ship a new minor version of Sourcegraph with new features and bug fixes monthly, followed by a patch release later that month.
28
28
29
29
Check the [Sourcegraph blog](https://about.sourcegraph.com/blog) for release announcements and details on the latest version.
30
30
@@ -78,14 +78,16 @@ It is highly recommended to:
78
78
Sourcegraph upgrades take the following general form:
79
79
1. Determine if your instance is ready to Upgrade (check upgrade notes)
80
80
2. Merge the latest Sourcegraph release into your deployment manifests
81
-
3. If updating more than a single minor version, perform an [**automatic multi-version upgrade**](/admin/updates/automatic) if targeting **Sourcegraph 5.1 or later**; [manual multi-version upgrades]/admin/updates/migrator/migrator-operations) are required if upgrading to an earlier version, which requires shutting off the instance and invoking the `migrator` container or job to perform the database rewrite and application of unfinished out-of-band migrations
81
+
3. Select an upgrade method:
82
+
- If updating more than a single minor version, perform a multiversion upgrade with either the migrator [upgrade command](/admin/updates/migrator/migrator-operations#upgrade) or an [automatic upgrade](/admin/updates/automatic). *This method requires downtime.*
83
+
- If updating a single minor version, perform a standard upgrade with the [migrator up command](/admin/updates/migrator/migrator-operations#up) or an [automatic upgrade](/admin/updates/automatic). `up` is the default migrator entry command and runs on `frontend` startup.
82
84
4. With upstream changes to your manifests merged, start the new instance
83
85
84
86
> Note: For more explicit steps, specific to your deployment see the operations guides linked below.
85
87
86
88
### Upgrade Readiness
87
89
88
-
Starting in v5.0.0, as an admin you are able to check instance upgrade readiness by navigating to the `Site admin > Updates` page. Here you'll be notified if your instance has any **schema drift** or unfinished **out of band migrations**.
90
+
Navigate to the `Site admin > Updates` page. Here you'll be notified if your instance has any **schema drift** or unfinished **out of band migrations**.
89
91
90
92
91
93
@@ -117,6 +119,7 @@ If your instance has schema drift or unfinished oob migrations you may need to a
117
119
-[**Sourcegraph AWS AMI instances**](/admin/deploy/machine-images/aws-ami#upgrade)
118
120
119
121
## Other helpful links
122
+
-[Postgres 12 End Of Life Notice](admin/postgres12_end_of_life_notice#postgres-12-end-of-life)
0 commit comments