Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 29 additions & 19 deletions .dockerfiles/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,28 +1,38 @@
FROM python:3.11-slim as base
ENV PYTHONPATH /app
ARG PYTHON_VERSION=3.13
ARG UID=1000
ARG GID=1000

FROM python:${PYTHON_VERSION}-slim as base

Check warning on line 5 in .dockerfiles/Dockerfile

View workflow job for this annotation

GitHub Actions / docker

The 'as' keyword should match the case of the 'from' keyword

FromAsCasing: 'as' and 'FROM' keywords' casing do not match More info: https://docs.docker.com/go/dockerfile/rule/from-as-casing/
ENV PYTHONDONTWRITEBYTECODE 1

Check warning on line 6 in .dockerfiles/Dockerfile

View workflow job for this annotation

GitHub Actions / docker

Legacy key/value format with whitespace separator should not be used

LegacyKeyValueFormat: "ENV key=value" should be used instead of legacy "ENV key value" format More info: https://docs.docker.com/go/dockerfile/rule/legacy-key-value-format/
ENV PYTHONUNBUFFERED 1

Check warning on line 7 in .dockerfiles/Dockerfile

View workflow job for this annotation

GitHub Actions / docker

Legacy key/value format with whitespace separator should not be used

LegacyKeyValueFormat: "ENV key=value" should be used instead of legacy "ENV key value" format More info: https://docs.docker.com/go/dockerfile/rule/legacy-key-value-format/
ENV DEBUG False
RUN mkdir -p /app
WORKDIR /app

ENV UV_LINK_MODE copy

Check warning on line 8 in .dockerfiles/Dockerfile

View workflow job for this annotation

GitHub Actions / docker

Legacy key/value format with whitespace separator should not be used

LegacyKeyValueFormat: "ENV key=value" should be used instead of legacy "ENV key value" format More info: https://docs.docker.com/go/dockerfile/rule/legacy-key-value-format/
ENV UV_COMPILE_BYTECODE 1

Check warning on line 9 in .dockerfiles/Dockerfile

View workflow job for this annotation

GitHub Actions / docker

Legacy key/value format with whitespace separator should not be used

LegacyKeyValueFormat: "ENV key=value" should be used instead of legacy "ENV key value" format More info: https://docs.docker.com/go/dockerfile/rule/legacy-key-value-format/
ENV UV_PYTHON_DOWNLOADS never

Check warning on line 10 in .dockerfiles/Dockerfile

View workflow job for this annotation

GitHub Actions / docker

Legacy key/value format with whitespace separator should not be used

LegacyKeyValueFormat: "ENV key=value" should be used instead of legacy "ENV key value" format More info: https://docs.docker.com/go/dockerfile/rule/legacy-key-value-format/
ENV UV_PYTHON python${PYTHON_VERSION}

Check warning on line 11 in .dockerfiles/Dockerfile

View workflow job for this annotation

GitHub Actions / docker

Legacy key/value format with whitespace separator should not be used

LegacyKeyValueFormat: "ENV key=value" should be used instead of legacy "ENV key value" format More info: https://docs.docker.com/go/dockerfile/rule/legacy-key-value-format/
ENV UV_PROJECT_ENVIRONMENT /app

Check warning on line 12 in .dockerfiles/Dockerfile

View workflow job for this annotation

GitHub Actions / docker

Legacy key/value format with whitespace separator should not be used

LegacyKeyValueFormat: "ENV key=value" should be used instead of legacy "ENV key value" format More info: https://docs.docker.com/go/dockerfile/rule/legacy-key-value-format/
ENV PATH /app/bin:$PATH
COPY --from=ghcr.io/astral-sh/uv:latest /uv /usr/local/bin/uv

FROM base as py
COPY src /app/src
COPY LICENSE pyproject.toml README.md /app/
RUN python -m pip install --upgrade pip \
&& python -m pip install '.[hc,psycopg,relay]'


FROM base as app
COPY src/service /app/service
FROM base as builder

Check warning on line 17 in .dockerfiles/Dockerfile

View workflow job for this annotation

GitHub Actions / docker

The 'as' keyword should match the case of the 'from' keyword

FromAsCasing: 'as' and 'FROM' keywords' casing do not match More info: https://docs.docker.com/go/dockerfile/rule/from-as-casing/
RUN --mount=type=cache,target=/root/.cache \
--mount=type=bind,source=uv.lock,target=uv.lock \
--mount=type=bind,source=pyproject.toml,target=pyproject.toml \
uv sync --locked --no-dev --no-install-project
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have been struggling with these options. Maybe you have the correct combo but mine was trying to re-download when I run uv run ...

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh this is totally lifted from Hynek's article about how he builds his Dockerfiles with uv. https://hynek.me/articles/docker-uv/

Just giving it a try to see if it works before rolling it out anywhere else.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

doesn't seem like it's redownloading -- first pass took 13s, second one didn't even clock

➜ just docker smoke
docker build --file .dockerfiles/Dockerfile --tag docker-email-relay:local .
[+] Building 12.5s (17/17) FINISHED                                                                                                                                                                   docker:default
 => [internal] load build definition from Dockerfile                                                                                                                                                            0.0s
 => => transferring dockerfile: 1.19kB                                                                                                                                                                          0.0s
 => WARN: FromAsCasing: 'as' and 'FROM' keywords' casing do not match (line 5)                                                                                                                                  0.0s
 => WARN: FromAsCasing: 'as' and 'FROM' keywords' casing do not match (line 17)                                                                                                                                 0.0s
 => WARN: FromAsCasing: 'as' and 'FROM' keywords' casing do not match (line 28)                                                                                                                                 0.0s
 => [internal] load metadata for ghcr.io/astral-sh/uv:latest                                                                                                                                                    0.7s
 => [internal] load metadata for docker.io/library/python:3.13-slim                                                                                                                                             0.8s
 => [internal] load .dockerignore                                                                                                                                                                               0.0s
 => => transferring context: 153B                                                                                                                                                                               0.0s
 => [internal] load build context                                                                                                                                                                               0.0s
 => => transferring context: 12.72kB                                                                                                                                                                            0.0s
 => FROM ghcr.io/astral-sh/uv:latest@sha256:3362a526af7eca2fcd8604e6a07e873fb6e4286d8837cb753503558ce1213664                                                                                                    0.0s
 => [base 1/2] FROM docker.io/library/python:3.13-slim@sha256:21e39cf1815802d4c6f89a0d3a166cc67ce58f95b6d1639e68a394c99310d2e5                                                                                  0.0s
 => CACHED [base 2/2] COPY --from=ghcr.io/astral-sh/uv:latest /uv /usr/local/bin/uv                                                                                                                             0.0s
 => CACHED [builder 1/4] RUN --mount=type=cache,target=/root/.cache     --mount=type=bind,source=uv.lock,target=uv.lock     --mount=type=bind,source=pyproject.toml,target=pyproject.toml     uv sync --locked  0.0s
 => [builder 2/4] COPY . /src                                                                                                                                                                                   0.2s
 => [builder 3/4] WORKDIR /src                                                                                                                                                                                  0.1s
 => [builder 4/4] RUN --mount=type=cache,target=/root/.cache     uv sync --locked --no-dev --no-editable --extra hc --extra psycopg --extra relay                                                               2.3s
 => CACHED [final 1/4] RUN mkdir -p /app                                                                                                                                                                        0.0s
 => [final 2/4] COPY --from=builder /app /app                                                                                                                                                                   1.2s
 => [final 3/4] RUN addgroup -gid "1000" --system django   && adduser -uid "1000" -gid "1000" --home /home/django --system django   && chown -R django:django /app                                              5.6s
 => [final 4/4] WORKDIR /app                                                                                                                                                                                    0.1s
 => exporting to image                                                                                                                                                                                          0.9s
 => => exporting layers                                                                                                                                                                                         0.9s
 => => writing image sha256:7fe7610cb55734c197c5089f347c1890aaf495e2cbf6c1c74edcf37bb77ee235                                                                                                                    0.0s
 => => naming to docker.io/library/docker-email-relay:local                                                                                                                                                     0.0s
docker run --rm docker-email-relay:local uv run -m email_relay.service --help
usage: service.py [-h]

Run the Django Email Relay service.

options:
  -h, --help  show this help message and exit

django-email-relay on  uv  v3.13.0 (django-email-relay) took 13s
➜ just docker smoke
docker build --file .dockerfiles/Dockerfile --tag docker-email-relay:local .
[+] Building 0.5s (17/17) FINISHED                                                                                                                                                                    docker:default
 => [internal] load build definition from Dockerfile                                                                                                                                                            0.0s
 => => transferring dockerfile: 1.19kB                                                                                                                                                                          0.0s
 => WARN: FromAsCasing: 'as' and 'FROM' keywords' casing do not match (line 5)                                                                                                                                  0.0s
 => WARN: FromAsCasing: 'as' and 'FROM' keywords' casing do not match (line 17)                                                                                                                                 0.0s
 => WARN: FromAsCasing: 'as' and 'FROM' keywords' casing do not match (line 28)                                                                                                                                 0.0s
 => [internal] load metadata for ghcr.io/astral-sh/uv:latest                                                                                                                                                    0.1s
 => [internal] load metadata for docker.io/library/python:3.13-slim                                                                                                                                             0.3s
 => [internal] load .dockerignore                                                                                                                                                                               0.0s
 => => transferring context: 153B                                                                                                                                                                               0.0s
 => [internal] load build context                                                                                                                                                                               0.0s
 => => transferring context: 6.92kB                                                                                                                                                                             0.0s
 => FROM ghcr.io/astral-sh/uv:latest@sha256:3362a526af7eca2fcd8604e6a07e873fb6e4286d8837cb753503558ce1213664                                                                                                    0.0s
 => [base 1/2] FROM docker.io/library/python:3.13-slim@sha256:21e39cf1815802d4c6f89a0d3a166cc67ce58f95b6d1639e68a394c99310d2e5                                                                                  0.0s
 => CACHED [base 2/2] COPY --from=ghcr.io/astral-sh/uv:latest /uv /usr/local/bin/uv                                                                                                                             0.0s
 => CACHED [final 1/4] RUN mkdir -p /app                                                                                                                                                                        0.0s
 => CACHED [builder 1/4] RUN --mount=type=cache,target=/root/.cache     --mount=type=bind,source=uv.lock,target=uv.lock     --mount=type=bind,source=pyproject.toml,target=pyproject.toml     uv sync --locked  0.0s
 => CACHED [builder 2/4] COPY . /src                                                                                                                                                                            0.0s
 => CACHED [builder 3/4] WORKDIR /src                                                                                                                                                                           0.0s
 => CACHED [builder 4/4] RUN --mount=type=cache,target=/root/.cache     uv sync --locked --no-dev --no-editable --extra hc --extra psycopg --extra relay                                                        0.0s
 => CACHED [final 2/4] COPY --from=builder /app /app                                                                                                                                                            0.0s
 => CACHED [final 3/4] RUN addgroup -gid "1000" --system django   && adduser -uid "1000" -gid "1000" --home /home/django --system django   && chown -R django:django /app                                       0.0s
 => CACHED [final 4/4] WORKDIR /app                                                                                                                                                                             0.0s
 => exporting to image                                                                                                                                                                                          0.0s
 => => exporting layers                                                                                                                                                                                         0.0s
 => => writing image sha256:7fe7610cb55734c197c5089f347c1890aaf495e2cbf6c1c74edcf37bb77ee235                                                                                                                    0.0s
 => => naming to docker.io/library/docker-email-relay:local                                                                                                                                                     0.0s
docker run --rm docker-email-relay:local uv run -m email_relay.service --help
usage: service.py [-h]

Run the Django Email Relay service.

options:
  -h, --help  show this help message and exit

django-email-relay on  uv  v3.13.0 (django-email-relay)
➜

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, it did, it just didn't show up in my prompt. 12.5s build time first pass, 0.5s second pass.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or wait, are you talking about uv run redownloading later in the same build? I've confused myself.

COPY . /src
WORKDIR /src
RUN --mount=type=cache,target=/root/.cache \
uv sync --locked --no-dev --no-editable --extra hc --extra psycopg --extra relay


FROM base as final

Check warning on line 28 in .dockerfiles/Dockerfile

View workflow job for this annotation

GitHub Actions / docker

The 'as' keyword should match the case of the 'from' keyword

FromAsCasing: 'as' and 'FROM' keywords' casing do not match More info: https://docs.docker.com/go/dockerfile/rule/from-as-casing/
COPY --from=py /usr/local /usr/local
COPY --from=app /app /app
RUN addgroup --system django \
&& adduser --system --ingroup django django \
ARG UID
ARG GID
RUN mkdir -p /app
COPY --from=builder /app /app
RUN addgroup -gid "${GID}" --system django \
&& adduser -uid "${UID}" -gid "${GID}" --home /home/django --system django \
&& chown -R django:django /app
USER django
CMD ["python", "-m", "service"]
WORKDIR /app
CMD ["uv", "run", "-m", "email_relay.service"]
20 changes: 7 additions & 13 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -1,13 +1,7 @@
fly.toml
.git/
*.sqlite3
.editorconfig
.env*
.gitignore
.pre-commit-config.yaml
.python-version
.yarnrc.yml
Dockerfile
Justfile
requirements.in
__pycache__
*

!src/
!LICENSE
!README.md
!pyproject.toml
!uv.lock
62 changes: 20 additions & 42 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,69 +2,47 @@ name: release

on:
release:
types: [released]
types: [published]

jobs:
check:
runs-on: ubuntu-latest
permissions:
actions: read
contents: read
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
steps:
- uses: actions/checkout@v4

- name: Check most recent test run on `main`
id: latest-test-result
run: |
echo "result=$(gh run list \
--branch=main \
--workflow=test.yml \
--json headBranch,workflowName,conclusion \
--jq '.[] | select(.headBranch=="main" and .conclusion=="success") | .conclusion' \
| head -n 1)" >> $GITHUB_OUTPUT
Comment on lines -21 to -26
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🍺 pours one out for this code.


- name: OK
if: ${{ (contains(steps.latest-test-result.outputs.result, 'success')) }}
run: exit 0

- name: Fail
if: ${{ !contains(steps.latest-test-result.outputs.result, 'success') }}
run: exit 1
test:
uses: ./.github/workflows/test.yml
secrets: inherit

pypi:
if: ${{ github.event_name == 'release' }}
runs-on: ubuntu-latest
needs: check
needs: test
environment: release
permissions:
contents: read
contents: write
id-token: write
steps:
- uses: actions/checkout@v4
with:
persist-credentials: false

- uses: westerveltco/setup-ci-action@v0
- name: Install uv
uses: astral-sh/setup-uv@v5
with:
python-version: 3.12
extra-python-dependencies: hatch
use-uv: true
enable-cache: true
pyproject-file: pyproject.toml

- name: Build package
run: |
hatch build
uv build

- name: Upload release assets to GitHub
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
gh release upload ${{ github.event.release.tag_name }} ./dist/*

- name: Publish to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
run: |
uv publish

docker:
runs-on: ubuntu-latest
needs: check
environment: release
permissions:
contents: read
contents: write
packages: write
steps:
- uses: actions/checkout@v4
Expand Down
75 changes: 45 additions & 30 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,18 +20,16 @@ jobs:
matrix: ${{ steps.set-matrix.outputs.matrix }}
steps:
- uses: actions/checkout@v4
with:
persist-credentials: false

- uses: westerveltco/setup-ci-action@v0
- name: Install uv
uses: astral-sh/setup-uv@v5
with:
python-version: 3.9
extra-python-dependencies: nox
use-uv: true
enable-cache: true
pyproject-file: pyproject.toml

- id: set-matrix
run: |
echo "matrix=$(python -m nox -l --json | jq -c '[.[] | select(.name == "tests") | {"python-version": .python, "django-version": .call_spec.django}] | {include: .}')" >> $GITHUB_OUTPUT
uv run nox --session "gha_matrix"
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's amazing how much this cleans stuff up.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, much easier to see what's going on here:

@nox.session
def gha_matrix(session):
    sessions = session.run("nox", "-l", "--json", silent=True)
    matrix = {
        "include": [
            {
                "python-version": session["python"],
                "django-version": session["call_spec"]["django"],
            }
            for session in json.loads(sessions)
            if session["name"] == "tests"
        ]
    }
    with Path(os.environ["GITHUB_OUTPUT"]).open("a") as fh:
        print(f"matrix={matrix}", file=fh)

Arguably you could unroll the comprehension too to help readability.


test:
name: Python ${{ matrix.python-version }}, Django ${{ matrix.django-version }}
Expand All @@ -42,18 +40,16 @@ jobs:
matrix: ${{ fromJSON(needs.generate-matrix.outputs.matrix) }}
steps:
- uses: actions/checkout@v4
with:
persist-credentials: false

- uses: westerveltco/setup-ci-action@v0
- name: Install uv
uses: astral-sh/setup-uv@v5
with:
python-version: ${{ matrix.python-version }}
extra-python-dependencies: nox
use-uv: true
enable-cache: true
pyproject-file: pyproject.toml

- name: Run tests
run: |
python -m nox --session "tests(python='${{ matrix.python-version }}', django='${{ matrix.django-version }}')"
uv run nox --session "tests(python='${{ matrix.python-version }}', django='${{ matrix.django-version }}')"

tests:
runs-on: ubuntu-latest
Expand All @@ -71,32 +67,51 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
persist-credentials: false

- uses: westerveltco/setup-ci-action@v0
- name: Install uv
uses: astral-sh/setup-uv@v5
with:
python-version: 3.9
extra-python-dependencies: nox
use-uv: true
enable-cache: true
pyproject-file: pyproject.toml

- name: Run mypy
- name: Run type checks
run: |
python -m nox --session "mypy"
uv run nox --session "mypy"

coverage:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
persist-credentials: false

- uses: westerveltco/setup-ci-action@v0
- name: Install uv
uses: astral-sh/setup-uv@v5
with:
python-version: 3.9
extra-python-dependencies: nox
use-uv: true
enable-cache: true
pyproject-file: pyproject.toml

- name: Generate code coverage
run: |
uv run nox --session "coverage"

- name: Run coverage
docker:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Build Docker image
uses: docker/build-push-action@v6
with:
context: .
file: .dockerfiles/Dockerfile
load: true
tags: django-email-relay-test:latest
push: false
cache-from: type=gha
cache-to: type=gha,mode=max
Comment on lines +112 to +113
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👀 oh, is this the trick?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, mode=max caches everything about PRs indefinitely across GitHub causing the whole service to come to a standstill.


- name: Run container and check
run: |
python -m nox --session "coverage"
docker run --rm --name test-container django-email-relay-test:latest uv run -m email_relay.service --help
36 changes: 36 additions & 0 deletions .just/copier.just
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
set unstable := true

justfile := justfile_directory() + "/.just/copier.just"

[private]
default:
@just --list --justfile {{ justfile }}

[private]
fmt:
@just --fmt --justfile {{ justfile }}

# Create a copier answers file
[no-cd]
copy TEMPLATE_PATH DESTINATION_PATH=".":
uv run copier copy --trust {{ TEMPLATE_PATH }} {{ DESTINATION_PATH }}

# Recopy the project from the original template
[no-cd]
recopy ANSWERS_FILE *ARGS:
uv run copier recopy --trust --answers-file {{ ANSWERS_FILE }} {{ ARGS }}

# Loop through all answers files and recopy the project using copier
[no-cd]
@recopy-all *ARGS:
for file in `ls .copier/`; do just copier recopy .copier/$file "{{ ARGS }}"; done

# Update the project using a copier answers file
[no-cd]
update ANSWERS_FILE *ARGS:
uv run copier update --trust --answers-file {{ ANSWERS_FILE }} {{ ARGS }}

# Loop through all answers files and update the project using copier
[no-cd]
@update-all *ARGS:
for file in `ls .copier/`; do just copier update .copier/$file "{{ ARGS }}"; done
23 changes: 23 additions & 0 deletions .just/docker.just
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
set unstable := true

justfile := justfile_directory() + "/.just/docker.just"

[private]
default:
@just --list --justfile {{ justfile }}

[private]
fmt:
@just --fmt --justfile {{ justfile }}

[no-cd]
build:
docker build --file .dockerfiles/Dockerfile --tag docker-email-relay:local .

[no-cd]
run *ARGS: build
docker run --rm docker-email-relay:local {{ ARGS }}

[no-cd]
smoke:
@just docker run uv run -m email_relay.service --help
31 changes: 31 additions & 0 deletions .just/documentation.just
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
set unstable := true

justfile := justfile_directory() + "/.just/documentation.just"

[private]
default:
@just --list --justfile {{ justfile }}

[private]
fmt:
@just --fmt --justfile {{ justfile }}

# Build documentation using Sphinx
[no-cd]
build LOCATION="docs/_build/html": cog
uv run --group docs sphinx-build docs {{ LOCATION }}

# Serve documentation locally
[no-cd]
serve PORT="8000": cog
#!/usr/bin/env sh
HOST="localhost"
if [ -f "/.dockerenv" ]; then
HOST="0.0.0.0"
fi
uv run --group docs sphinx-autobuild docs docs/_build/html --host "$HOST" --port {{ PORT }}

[no-cd]
[private]
cog:
uv run --with cogapp cog -r CONTRIBUTING.md docs/development/just.md
Loading