Skip to content

Use uv instead of poetry #224

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 11 commits into from
Dec 27, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 7 additions & 5 deletions .github/workflows/github-pages.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,12 +32,14 @@ jobs:
with:
python-version: "3.13"

- name: Install uv
uses: astral-sh/setup-uv@v5
with:
# Install a specific version of uv.
version: "0.5.13"

- name: Install dependencies
run: |
python -m pip install --upgrade pip
python -m pip install poetry
poetry config virtualenvs.create false
make dev-dependencies
run: make dev-dependencies

- name: Build static pages
run: make docs-build
Expand Down
14 changes: 7 additions & 7 deletions .github/workflows/python-code-style.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,12 +21,12 @@ jobs:
uses: actions/setup-python@v5
with:
python-version: "3.13"
- name: Install uv
uses: astral-sh/setup-uv@v5
with:
# Install a specific version of uv.
version: "0.5.13"
- name: Install dependencies
run: |
python -m pip install --upgrade pip
python -m pip install poetry
poetry config virtualenvs.create false
make dev-dependencies
run: make dev-dependencies
- name: Check code style
run: |
make format
run: make format
11 changes: 6 additions & 5 deletions .github/workflows/python-lint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,11 +21,12 @@ jobs:
uses: actions/setup-python@v5
with:
python-version: "3.13"
- name: Install uv
uses: astral-sh/setup-uv@v5
with:
# Install a specific version of uv.
version: "0.5.13"
- name: Install dependencies
run: |
python -m pip install --upgrade pip
python -m pip install poetry
poetry config virtualenvs.create false
make dev-dependencies
run: make dev-dependencies
- name: Lint with ruff
run: make lint
11 changes: 6 additions & 5 deletions .github/workflows/python-quality.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,12 +21,13 @@ jobs:
uses: actions/setup-python@v5
with:
python-version: "3.13"
- name: Install uv
uses: astral-sh/setup-uv@v5
with:
# Install a specific version of uv.
version: "0.5.13"
- name: Install dependencies
run: |
python -m pip install --upgrade pip
python -m pip install poetry
poetry config virtualenvs.create false
make dev-dependencies
run: make dev-dependencies
- name: Test & publish code coverage
uses: paambaati/[email protected]
env:
Expand Down
11 changes: 6 additions & 5 deletions .github/workflows/python-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,12 +24,13 @@ jobs:
uses: actions/setup-python@v5
with:
python-version: "${{ matrix.version }}"
- name: Install uv
uses: astral-sh/setup-uv@v5
with:
# Install a specific version of uv.
version: "0.5.13"
- name: Install dependencies
run: |
python -m pip install --upgrade pip
python -m pip install poetry
poetry config virtualenvs.create false
make dev-dependencies
run: make dev-dependencies
- name: Test with pytest
run: |
make ci-test
Expand Down
11 changes: 6 additions & 5 deletions .github/workflows/python-typing.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,11 +21,12 @@ jobs:
uses: actions/setup-python@v5
with:
python-version: "3.13"
- name: Install uv
uses: astral-sh/setup-uv@v5
with:
# Install a specific version of uv.
version: "0.5.13"
- name: Install dependencies
run: |
python -m pip install --upgrade pip
python -m pip install poetry
poetry config virtualenvs.create false
make dev-dependencies
run: make dev-dependencies
- name: Check typing
run: make typing
2 changes: 1 addition & 1 deletion .idea/bootstrap-python-fastapi.iml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion .idea/misc.xml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

11 changes: 11 additions & 0 deletions .idea/ryecharm-overrides.xml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 6 additions & 0 deletions .idea/ryecharm.xml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

46 changes: 22 additions & 24 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
ARG PYTHON_VERSION=3.12
ARG PYTHON_VERSION=3.13
FROM python:$PYTHON_VERSION-slim AS base
ARG UID=2000
ARG GID=2000
Expand All @@ -10,78 +10,76 @@ RUN chown nonroot:nonroot /app
# Creating a separate directory for venvs allows to easily
# copy them from the builder and to mount the application
# for local development
RUN mkdir /poetryvenvs && chown nonroot:nonroot /poetryvenvs
RUN mkdir /venv && chown nonroot:nonroot /venv
ENV PATH="/venv/bin:$PATH"

# Install necessary runtime libraries (e.g. libmysql)
RUN apt-get update \
&& apt-get install -y --no-install-recommends \
make \
&& rm -rf /var/lib/apt/lists/*

# Update pip and install poetry
RUN pip install --no-cache-dir -U pip
RUN pip install --no-cache-dir -U poetry

# We run everything by poetry run from now on, so that PATH will be handled
# for binaries installed in virtual environments
ENTRYPOINT ["poetry", "run"]

FROM base AS base_builder
ENV UV_PROJECT_ENVIRONMENT=/venv
# Enable bytecode compilation
ENV UV_COMPILE_BYTECODE=1

# Install build system requirements (gcc, library headers, etc.)
# for compiled Python requirements like psycopg2
RUN apt-get update \
&& apt-get install -y --no-install-recommends \
build-essential gcc git \
&& rm -rf /var/lib/apt/lists/*

COPY --from=ghcr.io/astral-sh/uv:0.5.13 /uv /uvx /bin/

# From here we shouldn't need anymore a root user
# Switch to nonroot and config poetry
# Switch to nonroot and config uv
USER nonroot
RUN poetry config virtualenvs.path /poetryvenvs

COPY --chown=nonroot:nonroot pyproject.toml .
COPY --chown=nonroot:nonroot poetry.lock .
COPY --chown=nonroot:nonroot uv.lock .
COPY --chown=nonroot:nonroot Makefile .

# Test image, contains all files and dependencies
# Dev image, contains all files and dependencies
FROM base_builder AS dev
COPY --chown=nonroot:nonroot . .
RUN make dev-dependencies
RUN --mount=type=cache,target=~/.cache/uv \
make dev-dependencies

# Note that opentelemetry doesn't play well together with uvicorn reloader
# when signals are propagated, we disable it in dev image default CMD
CMD ["uvicorn", "http_app:create_app", "--host", "0.0.0.0", "--port", "8000", "--factory", "--reload"]

# Installs requirements to run production dramatiq application
FROM base_builder AS dramatiq_builder
RUN poetry install --no-root
RUN --mount=type=cache,target=~/.cache/uv \
uv sync --no-dev --no-install-project --frozen --no-editable

# Installs requirements to run production http application
FROM base_builder AS http_builder
RUN poetry install --no-root --with http
RUN --mount=type=cache,target=~/.cache/uv \
uv sync --no-dev --group http --no-install-project --frozen --no-editable

# Copy the shared python packages
# Create the base app with the common python packages
FROM base AS base_app
USER nonroot
RUN poetry config virtualenvs.path /poetryvenvs
COPY --chown=nonroot:nonroot pyproject.toml .
COPY --chown=nonroot:nonroot poetry.lock .
COPY --chown=nonroot:nonroot src/alembic ./alembic
COPY --chown=nonroot:nonroot src/domains ./domains
COPY --chown=nonroot:nonroot src/gateways ./gateways
COPY --chown=nonroot:nonroot src/common ./common
COPY --chown=nonroot:nonroot src/alembic.ini .
COPY --chown=nonroot:nonroot Makefile .

# Copy the http python package and requirements from relevant builder
FROM base_app AS http_app
COPY --from=http_builder /poetryvenvs /poetryvenvs
COPY --from=http_builder /venv /venv
COPY --chown=nonroot:nonroot src/http_app ./http_app
# Run CMD using array syntax, so it's uses `exec` and runs as PID1
CMD ["opentelemetry-instrument", "uvicorn", "http_app:create_app", "--host", "0.0.0.0", "--port", "8000", "--factory"]

# Copy the dramatiq python package and requirements from relevant builder
FROM base_app AS dramatiq_app
COPY --from=dramatiq_builder /poetryvenvs /poetryvenvs
COPY --from=dramatiq_builder /venv /venv
COPY --chown=nonroot:nonroot src/dramatiq_worker ./dramatiq_worker
# Run CMD using array syntax, so it's uses `exec` and runs as PID1
# TODO: Review processes/threads
Expand Down
36 changes: 18 additions & 18 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -9,56 +9,56 @@ containers:
docker compose build --build-arg UID=`id -u`

dev:
poetry run uvicorn http_app:create_app --host 0.0.0.0 --port 8000 --factory --reload
uv run uvicorn http_app:create_app --host 0.0.0.0 --port 8000 --factory --reload

otel:
OTEL_SERVICE_NAME=bootstrap-fastapi OTEL_TRACES_EXPORTER=none OTEL_METRICS_EXPORTER=none OTEL_LOGS_EXPORTER=none poetry run opentelemetry-instrument uvicorn http_app:create_app --host 0.0.0.0 --port 8000 --factory
OTEL_SERVICE_NAME=bootstrap-fastapi OTEL_TRACES_EXPORTER=none OTEL_METRICS_EXPORTER=none OTEL_LOGS_EXPORTER=none uv run opentelemetry-instrument uvicorn http_app:create_app --host 0.0.0.0 --port 8000 --factory

run:
poetry run uvicorn http_app:create_app --host 0.0.0.0 --port 8000 --factory
uv run uvicorn http_app:create_app --host 0.0.0.0 --port 8000 --factory

test:
poetry run pytest -n auto --cov
uv run pytest -n auto --cov

ci-test:
poetry run pytest
uv run pytest

ci-coverage:
poetry run pytest --cov --cov-report lcov
uv run pytest --cov --cov-report lcov

typing:
poetry run mypy
uv run mypy

install-dependencies:
poetry install --no-root --with http
uv sync --all-groups --no-dev --no-install-project --frozen

dev-dependencies:
poetry install --with http,dev
uv sync --all-groups --frozen

update-dependencies:
poetry update --with http,dev
uv lock --upgrade

migrate:
poetry run alembic upgrade heads
uv run alembic upgrade heads

format:
poetry run ruff format --check .
uv run ruff format --check .

lint:
poetry run ruff check .
uv run ruff check .

fix:
poetry run ruff format .
poetry run ruff check . --fix
poetry run ruff format .
uv run ruff format .
uv run ruff check . --fix
uv run ruff format .

check: lint format typing test

docs:
poetry run mkdocs serve
uv run mkdocs serve

adr:
adr-viewer --serve --adr-path docs/adr

docs-build:
poetry run mkdocs build
uv run mkdocs build
5 changes: 2 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,10 +69,9 @@ Python docker image tend to become large after installing the application requir
to minimise the image size, even if it produces a slightly more complex multistage
Dockerfile.

The following setup makes sure the production image will keep to a minimal size ("only" 390MB):
The following setup makes sure the production image will keep to a minimal size ("only" 360MB):
* 150MB base image
* 165MB python installed dependencies
* 73MB poetry + updated pip
* 210MB python installed dependencies

Using the following pipeline the "test" image is instead ~850MB, more than 400MB that would
end up as a cost in traffic on each image pull.
Expand Down
10 changes: 3 additions & 7 deletions docker-compose.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
services:

otel-collector:
image: otel/opentelemetry-collector-contrib
volumes:
Expand All @@ -13,11 +12,9 @@ services:
dockerfile: Dockerfile
context: .
target: dramatiq_app
env_file: local.env
environment:
WATCHFILES_FORCE_POLLING: true
OTEL_SERVICE_NAME: "bootstrap-fastapi-dramatiq-worker"
OTEL_EXPORTER_OTLP_ENDPOINT: "http://otel-collector:4317"
DRAMATIQ__REDIS_URL: "redis://redis:6379/0"
working_dir: "/app/src"
volumes:
- '.:/app'
Expand All @@ -40,10 +37,9 @@ services:
dockerfile: Dockerfile
context: .
target: dev
env_file: local.env
environment:
WATCHFILES_FORCE_POLLING: true
OTEL_SERVICE_NAME: "bootstrap-fastapi-dev"
OTEL_EXPORTER_OTLP_ENDPOINT: "http://otel-collector:4317"
ports:
- '8000:8000'
working_dir: "/app/src"
Expand Down Expand Up @@ -72,9 +68,9 @@ services:
depends_on:
- redis
- otel-collector
env_file: local.env
environment:
OTEL_SERVICE_NAME: "bootstrap-fastapi-http"
OTEL_EXPORTER_OTLP_ENDPOINT: "http://otel-collector:4317"
ports:
- '8001:8000'
volumes:
Expand Down
Loading
Loading