Skip to content

Commit 07df2f9

Browse files
ChlodAlejandroChlodAlejandro
committed
Refactor Docker setup
Re-add ability to use Docker for development post-switch to Cloud VPS. Support for Trove has been added, along with a few tweaks (like moving execution to a non-root user, as required by Symfony). Also moved off of Toolforge images and onto global `php` images, since CopyPatrol isn't expected to run on Toolforge anymore.
1 parent 6a9361d commit 07df2f9

File tree

6 files changed

+248
-154
lines changed

6 files changed

+248
-154
lines changed

‎.env‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,7 @@ TOOLSDB_PORT=4720
2929
TOOLSDB_USERNAME=
3030
TOOLSDB_PASSWORD=
3131
TROVE_HOST=127.0.0.1
32+
TROVE_REMOTE_HOST=hxmnwriu2vm.svc.trove.eqiad1.wikimedia.cloud
3233
TROVE_PORT=4721
3334
TROVE_USERNAME=
3435
TROVE_PASSWORD=

‎.github/workflows/ci.yml‎

Lines changed: 56 additions & 71 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ jobs:
1212
name: Build and test
1313
runs-on: ubuntu-latest
1414
steps:
15-
- uses: actions/checkout@v3
15+
- uses: actions/checkout@v4
1616
with:
1717
fetch-depth: 0
1818
- name: Setup PHP
@@ -29,74 +29,59 @@ jobs:
2929
./bin/console lint:yaml ./config
3030
./vendor/bin/minus-x check .
3131
./bin/phpunit --exclude-group=integration
32+
build_image:
33+
name: Build Docker image
34+
runs-on: ubuntu-latest
35+
needs: build
36+
steps:
37+
- name: Checkout code
38+
uses: actions/checkout@v4
39+
40+
- name: Set up QEMU
41+
uses: docker/setup-qemu-action@v3
42+
43+
- name: Set up Docker Buildx
44+
id: buildx
45+
uses: docker/setup-buildx-action@v3
46+
47+
- name: Build image
48+
id: docker_build
49+
uses: docker/build-push-action@v6
50+
with:
51+
context: .
52+
file: docker/Dockerfile
53+
target: production
54+
tags: wikimedia/copypatrol:latest
55+
outputs: type=docker,dest=/tmp/copypatrol-production.image.tar
56+
cache-from: type=gha
57+
cache-to: type=gha,mode=max
58+
59+
- name: Image digest
60+
run: echo ${{ steps.docker_build.outputs.digest }}
61+
62+
- name: Upload Docker image to artifacts
63+
uses: actions/upload-artifact@v4
64+
with:
65+
name: image-production
66+
path: /tmp/copypatrol-production.image.tar
67+
analysis:
68+
name: Analyze Docker images
69+
runs-on: ubuntu-latest
70+
needs: build_image
71+
72+
steps:
73+
- name: Download Docker image from artifacts
74+
uses: actions/download-artifact@v4
75+
with:
76+
name: image-production
77+
path: /tmp
3278

33-
# build_image:
34-
# name: Build Docker image
35-
# runs-on: ubuntu-latest
36-
# needs: build
37-
# strategy:
38-
# matrix:
39-
# targets:
40-
# - name: production
41-
# tag: wikimedia/copypatrol
42-
# - name: development
43-
# tag: wikimedia/copypatrol-development
44-
# steps:
45-
# - name: Checkout code
46-
# uses: actions/checkout@v2
47-
#
48-
# - name: Set up QEMU
49-
# uses: docker/setup-qemu-action@v2
50-
#
51-
# - name: Set up Docker Buildx
52-
# id: buildx
53-
# uses: docker/setup-buildx-action@v2
54-
#
55-
# - name: Build image
56-
# id: docker_build
57-
# uses: docker/build-push-action@v4
58-
# with:
59-
# context: .
60-
# file: docker/Dockerfile
61-
# target: ${{ matrix.targets.name }}
62-
# tags: ${{ matrix.targets.tag }}:latest
63-
# outputs: type=docker,dest=/tmp/copypatrol-${{ matrix.targets.name }}.image.tar
64-
# cache-from: type=gha
65-
# cache-to: type=gha,mode=max
66-
#
67-
# - name: Image digest
68-
# run: echo ${{ steps.docker_build.outputs.digest }}
69-
#
70-
# - name: Upload Docker image to artifacts
71-
# uses: actions/upload-artifact@v2
72-
# with:
73-
# name: image-${{ matrix.targets.name }}
74-
# path: /tmp/copypatrol-${{ matrix.targets.name }}.image.tar
75-
# analysis:
76-
# name: Analyze Docker images
77-
# runs-on: ubuntu-latest
78-
# needs: build_image
79-
# strategy:
80-
# matrix:
81-
# targets:
82-
# - name: production
83-
# tag: wikimedia/copypatrol
84-
# - name: development
85-
# tag: wikimedia/copypatrol-development
86-
#
87-
# steps:
88-
# - name: Download Docker image from artifacts
89-
# uses: actions/download-artifact@v2
90-
# with:
91-
# name: image-${{ matrix.targets.name }}
92-
# path: /tmp
93-
#
94-
# - name: Load image
95-
# run: |
96-
# docker load --input /tmp/copypatrol-${{ matrix.targets.name }}.image.tar
97-
# docker image ls -a
98-
# - name: Dive
99-
# uses: yuichielectric/dive-action@0.0.4
100-
# with:
101-
# image: ${{ matrix.targets.tag }}:latest
102-
# github-token: ${{ secrets.GITHUB_TOKEN }}
79+
- name: Load image
80+
run: |
81+
docker load --input /tmp/copypatrol-production.image.tar
82+
docker image ls -a
83+
- name: Dive
84+
uses: MaxymVlasov/dive-action@v1.0.1
85+
with:
86+
image: wikimedia/copypatrol:latest
87+
github-token: ${{ secrets.GITHUB_TOKEN }}

‎README.md‎

Lines changed: 76 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -65,8 +65,82 @@ Assets are compiled using Webpack Encore. The compiled assets **must** be commit
6565

6666
## Installing using Docker
6767

68-
_A new Docker image needs to be created following the move to Wikimedia VPS._
69-
_You can use the manual installation instructions above in the meantime._
68+
Development through Docker is suggested if you have a different version of PHP locally
69+
installed, or if you wish to keep an isolated installation of PHP 8.2 for CopyPatrol.
70+
71+
1. Copy [.env](.env) to [.env.local](.env.local) and fill in the appropriate details.
72+
1. Set `REPLICAS_HOST_*` and `TROVE_HOST` to `127.0.0.1`.
73+
* To change the Trove host to be used, change the `TROVE_REMOTE_HOST` environmental variable.
74+
2. Use the credentials in your `replica.my.cnf` file in the home directory of your
75+
Toolforge account for `REPLICAS_USERNAME` and `REPLICAS_PASSWORD`.
76+
3. Set the rest of the `TROVE_*` variables to that of the installation of the CopyPatrol
77+
database (`COPYPATROL_DB_NAME`).
78+
4. If you need to test OAuth, obtain tokens by registering a new consumer on Meta at
79+
[Special:OAuthConsumerRegistration](https://meta.wikimedia.org/wiki/Special:OAuthConsumerRegistration).
80+
Alternatively, you can set `LOGGED_IN_USER` to any value to simulate being that user
81+
after clicking on 'Login'.
82+
5. If you need to test the "iThenticate report" functionality, set `TCA_DOMAIN` and `TCA_KEY`.
83+
Reports older than `AppController::ITHENTICATE_V2_TIMESTAMP` need to connect to the older
84+
iThenticate API, using the credentials set by `ITHENTICATE_USERNAME` and `ITHENTICATE_PASSWORD`.
85+
2. Build the development image once and install Composer dependencies with the following
86+
```bash
87+
docker compose build
88+
# On Windows, use `%CD%` instead of `$(pwd)`.
89+
docker run --rm -ti -v $(pwd):/app wikimedia/copypatrol:development composer install
90+
```
91+
Run the second command again every time you change `composer.json`, or when `composer.json`
92+
is changed in an upstream commit. This can take a while on Windows.
93+
3. (*Windows only*) Set the `HOME` environment variable to your user profile directory.
94+
```cmd
95+
setx HOME %UserProfile%
96+
set HOME=%UserProfile%
97+
```
98+
The first command sets `HOME` for future shells. The second command sets `HOME` for the current shell.
99+
4. Open a new terminal and start the development container with
100+
```bash
101+
docker compose up
102+
```
103+
Starting the local development server will be delayed until the next
104+
step is finished.
105+
5. Open up an SSH tunnel to access the databases on Toolforge.
106+
```bash
107+
# Your SSH config at $HOME/.ssh will be mounted into the container.
108+
# Your passphrase will be requested if your private key is protected.
109+
docker compose exec copypatrol start ssh
110+
# OR if your Toolforge shell name is not in your SSH config with a
111+
# `User <username>` line, you need to specify this manually.
112+
docker compose exec copypatrol start ssh <username>
113+
```
114+
This terminal will stay open as long as SSH is connected. No successful
115+
connection message is shown, but Symfony will start immediately once the
116+
ports are open. This extra step is required for you to be able to enter
117+
your SSH key password through a TTY.
118+
119+
Changes to this folder will automatically be applied to the running Docker container. This includes
120+
changes to `src` files, `.env.local`, etc. XDebug is set up to connect to the host machine
121+
(the computer running the Docker container) on port 9003 upon request ([more info](https://xdebug.org/docs/step_debug)).
122+
123+
If the Trove host changes, you must set the `TROVE_REMOTE_HOST` environment variable to the correct host.
124+
Review [OpenStack browser](https://openstack-browser.toolforge.org/project/copypatrol/database/copypatrol-dev-db-01) for
125+
the latest host.
126+
127+
An XDebug configuration is provided by default. To customize this config, mount a
128+
`xdebug.ini` file to `/usr/local/etc/php/conf.d/xdebug.ini` in the container.
129+
130+
<details>
131+
<summary>Production image</summary>
132+
133+
A production image can be built with the following command:
134+
```bash
135+
docker build -t wikimedia/copypatrol:latest -f docker/Dockerfile .
136+
```
137+
This image does not contain XDebug or OpenSSH, and does not have an SSH tunnel to Toolforge.
138+
You can test it out by running the following command:
139+
```bash
140+
# On Windows, use `%CD%` instead of `$(pwd)`.
141+
docker run -ti --rm -p 8000:8000 wikimedia/copypatrol:latest
142+
```
143+
</details>
70144

71145
## Adding new languages
72146

‎docker-compose.yml‎

Lines changed: 10 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,3 @@
1-
version: '3.8'
2-
31
# =========================================================
42
#
53
# This file allows anyone to start the CopyPatrol web interface
@@ -26,12 +24,17 @@ services:
2624
command: serve
2725
stdin_open: true
2826
tty: true
27+
environment:
28+
- TROVE_REMOTE_HOST
2929
extra_hosts:
3030
- host.docker.internal:host-gateway
3131
ports:
3232
- "8000:8000"
3333
volumes:
3434
# This binds your SSH configuration into the container.
35+
# If you don't want to do this, comment this entry out.
36+
# TODO: Disable `copypatrol` user access to /ssh, when that becomes possible.
37+
# https://github.com/docker/roadmap/issues/398
3538
- type: bind
3639
source: "$HOME/.ssh"
3740
target: "/ssh"
@@ -40,4 +43,8 @@ services:
4043
- type: bind
4144
source: "."
4245
target: "/app"
43-
stop_signal: SIGINT
46+
read_only: true
47+
- type: bind
48+
source: "./var"
49+
target: "/app/var"
50+
stop_signal: SIGINT

0 commit comments

Comments
 (0)