Skip to content

Commit 6778cd7

Browse files
marcleblanc2marcleblanc2
committed
Add examples and best practices
1 parent 4f878d7 commit 6778cd7

File tree

3 files changed

+206
-0
lines changed

3 files changed

+206
-0
lines changed

‎docker-compose/examples/README.md‎

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
# Docker Compose Examples
2+
3+
## Best Practices
4+
5+
These best practices reduce the operational cost of keeping your Sourcegraph instance up to date, and are consistent across the customers we see keeping up to date at low cost, with happy users, vs customers who are often many versions behind, with users asking, "Hey, where's the new features?"
6+
7+
- All of your customizations should be in one docker-compose.override.yaml file, applied on top of one or more unmodified base docker-compose.yaml file(s) from this deploy-sourcegraph-docker repo
8+
9+
- Your docker-compose.override.yaml file should be stored, maintained, and fetched from a Git repo / code version control system, with meaningful commit messages, so that future you, and future admins, can understand why changes needed to be made
10+
11+
- Bonus points if your override file is stored outside of your clone of this deploy-sourcegraph-docker repo, so that this deploy-sourcegraph-docker repo can be deleted / recreated / `git reset --hard` as needed, without losing your customizations
12+
13+
- The examples in this directory use the current compose syntax, used by `docker compose`, the docker-compose-plugin. Some parts of this syntax may not be valid for the old `docker-compose` standalone binary. It is highly recommended to switch to using the new compose plugin. See Docker docs:
14+
15+
- Install Compose plugin
16+
- https://docs.docker.com/compose/install
17+
18+
- Merging multiple compose files
19+
- https://docs.docker.com/compose/how-tos/multiple-compose-files/merge/
20+
- https://docs.docker.com/reference/compose-file/merge/#reset-value
21+
22+
- Using multiple compose / override files
23+
24+
- Old method:
25+
26+
- Store your docker-compose.override.yaml file inside this deploy-sourcegraph-docker repo, under the /docker-compose directory
27+
28+
- `docker compose -f docker-compose.yaml -f executors/executor.docker-compose.yaml -f docker-compose.override.yaml up -d --remove-orphans`
29+
30+
- New method:
31+
32+
- Store your docker-compose.override.yaml file in the parent directory / beside this deploy-sourcegraph-docker repo
33+
34+
- Create an `.env` file, containing the below
35+
36+
- `docker compose up -d --remove-orphans`
37+
38+
- Note that any file paths (ex. mounting TLS CAs) are relative to the first compose file, so the mount path will likely start with `../../cert-chain.pem` if `cert-chain.pem` is in the parent directory / beside this deploy-sourcegraph-docker repo
39+
40+
```env
41+
SOURCEGRAPH=./deploy-sourcegraph-docker/docker-compose/docker-compose.yaml
42+
EXECUTORS=./deploy-sourcegraph-docker/docker-compose/executors/executor.docker-compose.yaml
43+
OVERRIDE=./docker-compose.override.yaml
44+
45+
COMPOSE_FILE=${SOURCEGRAPH}:${EXECUTORS}:${OVERRIDE}
46+
```

‎docker-compose/examples/aws-alb/docker-compose.override.yaml‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
# Docker Compose override snippet for customers using AWS ALBs to terminate TLS,
2+
# and do not need TLS between the ALB and their Sourcegraph intance
3+
4+
5+
## Overrides for services defined in the base docker-compose.yaml files
6+
services:
7+
8+
sourcegraph-frontend-0:
9+
10+
# Add port 80, for ALB target group, in place of Caddy
11+
ports:
12+
- "0.0.0.0:80:3080"
13+
14+
15+
## Disable services
16+
17+
# Disable Caddy, because we're using an ALB for TLS termination
18+
caddy: !reset {}

‎docker-compose/examples/aws-rds-databases/docker-compose.override.yaml‎

Lines changed: 142 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,142 @@
1+
# Docker Compose override file for customers using AWS IAM authentication to databases on RDS
2+
# See https://sourcegraph.com/docs/admin/external_services/postgres#usage-with-aws-rds-iam-auth
3+
4+
5+
6+
## Reusable blocks of environment variables
7+
8+
# For all containers, which connect to any of the databases
9+
x-database-env-var-rds-iam-auth: &database-env-var-rds-iam-auth
10+
11+
# Using AWS RDS IAM authentication, not a password, so instruct the psql library to refresh the token
12+
PG_CONNECTION_UPDATER: EC2_ROLE_CREDENTIALS
13+
14+
# For both migrator and frontend
15+
x-database-env-vars: &database-env-vars
16+
17+
<<: *database-env-var-rds-iam-auth
18+
19+
# NOTE: Must clear / override the PGDATASOURCE env vars, otherwise frontends don't start up
20+
PGHOST: example-database-pgsql.cluster-example.us-west-2.rds.amazonaws.com
21+
PGPORT: 5432
22+
PGSSLMODE: require
23+
PGDATABASE: sg
24+
PGUSER: sg
25+
PGPASSWORD: ""
26+
PGDATASOURCE: ""
27+
28+
CODEINTEL_PGHOST: example-database-codeintel.cluster-example.us-west-2.rds.amazonaws.com
29+
CODEINTEL_PGPORT: 5432
30+
CODEINTEL_PGSSLMODE: require
31+
CODEINTEL_PGDATABASE: codeintel
32+
CODEINTEL_PGUSER: sg
33+
CODEINTEL_PGPASSWORD: ""
34+
CODEINTEL_PGDATASOURCE: ""
35+
36+
CODEINSIGHTS_PGHOST: example-database-codeinsights.cluster-example.us-west-2.rds.amazonaws.com
37+
CODEINSIGHTS_PGPORT: 5432
38+
CODEINSIGHTS_PGSSLMODE: require
39+
CODEINSIGHTS_PGDATABASE: codeinsights
40+
CODEINSIGHTS_PGUSER: sg
41+
CODEINSIGHTS_PGPASSWORD: ""
42+
CODEINSIGHTS_PGDATASOURCE: ""
43+
44+
45+
46+
## Overrides for services defined in the base docker-compose.yaml files
47+
services:
48+
49+
50+
gitserver-0:
51+
# Add / merge the env vars, to only overwrite the ones we need, and leave the others intact
52+
environment:
53+
<<: *database-env-var-rds-iam-auth
54+
55+
56+
migrator:
57+
58+
# !reset drops the depends_on config entirely, so that migrator will start up without the database containers
59+
depends_on: !reset []
60+
61+
# !override drops all env vars from earlier compose files, then defines new ones, to ensure no other env vars are in the mix
62+
environment: !override
63+
<<: *database-env-vars
64+
65+
# Normal startup of migrator should only ever use the default "up" command, no additional args
66+
# Overriding the command should only to be used to make specific changes, on single startups, only when needed, only when directed by Sourcegraph Customer Support Engineering
67+
# If you find that your migrator needs (an) additional arg(s) to run, there's something wrong with your databases; please open a support ticket by emailing [email protected]
68+
69+
# AWS RDS requires super admin permissions to install some of the extensions that Sourcegraph requires, ex. pg_stat_statements
70+
# Install the needed extensions by running these queries, as an admin user, against their respective databases
71+
# https://github.com/sourcegraph/artifacts/blob/main/migrations/frontend/1680296731_squashed_migrations_privileged/up.sql
72+
# https://github.com/sourcegraph/artifacts/blob/main/migrations/codeintel/1679010276_squashed_migrations_privileged/up.sql
73+
# https://github.com/sourcegraph/artifacts/blob/main/migrations/codeinsights/1675347548_squashed_migrations_privileged/up.sql
74+
# After the extensions are successfully installed, manually mark these migrations as completed,
75+
# by uncommenting these commands, one at a time, and running `docker compose up migrator` each time,
76+
# then comment the last one out, and run the normal `docker compose up -d --remove-orphans` command, for a normal startup
77+
# command: ["add-log", "-db=frontend", "-version=1680296731"]
78+
# command: ["add-log", "-db=codeintel", "-version=1679010276"]
79+
# command: ["add-log", "-db=codeinsights", "-version=1675347548"]
80+
81+
# If you do fall a few versions behind, here's the command to run a multi-version upgrade
82+
# See https://sourcegraph.com/docs/admin/deploy/docker-compose/upgrade#multi-version-upgrades
83+
# command: ["upgrade", "--from=v6.4.0", "--to=v6.9.1277"]
84+
85+
# Do not skip the drift check, or use other args, without instruction from Sourcegraph Customer Support Engineering
86+
# If migrator fails to start up due to drift detected, or other issues, please open a support ticket by emailing [email protected]
87+
88+
89+
precise-code-intel-worker:
90+
# Add / merge the env vars, to only overwrite the ones we need, and leave the others intact
91+
environment:
92+
<<: *database-env-var-rds-iam-auth
93+
94+
95+
searcher-0:
96+
# Add / merge the env vars, to only overwrite the ones we need, and leave the others intact
97+
environment:
98+
<<: *database-env-var-rds-iam-auth
99+
100+
101+
sourcegraph-frontend-0:
102+
# Add / merge the env vars, to only overwrite the ones we need, and leave the others intact
103+
environment:
104+
<<: *database-env-vars
105+
106+
107+
sourcegraph-frontend-internal:
108+
# !override drops all depends_on from earlier compose files, then defines new ones,
109+
# to ensure it doesn't wait for the database containers, but also doesn't skip the dependency on migrator
110+
depends_on: !override
111+
migrator:
112+
condition: service_completed_successfully
113+
# Add / merge the env vars, to only overwrite the ones we need, and leave the others intact
114+
environment:
115+
<<: *database-env-vars
116+
117+
118+
syntactic-code-intel-worker:
119+
# Add / merge the env vars, to only overwrite the ones we need, and leave the others intact
120+
environment:
121+
<<: *database-env-var-rds-iam-auth
122+
123+
124+
worker:
125+
# Add / merge the env vars, to only overwrite the ones we need, and leave the others intact
126+
environment:
127+
<<: *database-env-var-rds-iam-auth
128+
# Add additional env vars, as needed
129+
# TELEMETRY_GATEWAY_EXPORTER_USERMETADATA_EXPORT_ENABLED: true
130+
# TELEMETRY_GATEWAY_EXPORTER_USERMETADATA_EXPORT_ORGANIZATIONS: true
131+
# TELEMETRY_GATEWAY_EXPORTER_USERMETADATA_EXPORT_USERS: true
132+
133+
134+
## Disable services
135+
136+
# Drop built-in database services and volumes from the base compose file
137+
pgsql: !reset {}
138+
pgsql-exporter: !reset {}
139+
codeintel-db: !reset {}
140+
codeintel-db-exporter: !reset {}
141+
codeinsights-db: !reset {}
142+
codeinsights-db-exporter: !reset {}

0 commit comments

Comments
 (0)