Skip to content

Use the binary version of psycopg for development #2212

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Oct 1, 2025
Merged

Use the binary version of psycopg for development #2212

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 4 additions & 12 deletions Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ ENV PYTHONDONTWRITEBYTECODE=1
ENV PYTHONUNBUFFERED=1

# install deb packages
# PostgreSQL dependencies are needed for dbshell and backup process
RUN apt-get update \
&& apt-get install --assume-yes --no-install-recommends \
gettext \
Expand All @@ -23,23 +24,14 @@ RUN apt-get update \
&& apt-get distclean

ARG REQ_FILE=requirements/prod.txt
ARG BUILD_DEPENDENCIES="g++ gcc libc6-dev libpq-dev zlib1g-dev"

# install python dependencies
COPY ./requirements ./requirements
RUN apt-get update \
&& apt-get install --assume-yes --no-install-recommends \
g++ \
gcc \
libc6-dev \
libpq-dev \
zlib1g-dev \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If finally we're going to use psycopg[binary] check if this depenencies are not needed to the compilation of other dependencies (eg: pillow).

If not, and we don't need any more of these system dependencies to compile python packages, I'll invite you to get rid of the whole section of installing debian packages (also compilers), and removing an leaving only the pip install line.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

gcc and g++ are required for libsass. I looked for an alternative that doesn't require compiling but couldn't find one yet.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is libsass requiring both gcc and g++ ?
Since you studied it, can we open a PR for the libsass repo to ask a wheel for recent python versions?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or we can find an alternate package to replace it.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll check this again.

https://github.com/sass/libsass-python says

Provides prebuilt wheel binaries for Linux, Windows, and Mac.

but our build fails with

77.32   × Building wheel for libsass (pyproject.toml) did not run successfully.
...
77.32       error: command 'gcc' failed: No such file or directory

when those dependencies are not installed.

This feels more a like bug, rather than a missing feature in the libsass package.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[binary] works well these years and it's supported on major platforms but it's not supported everywhere and wheels creation can't be guaranteed, so I advise against using it as a library dependency. It's only suggested when you know exactly where an app will run (a requirements file, not an abstract dependency).

I was answering about the matter here too: psycopg/psycopg#1156 (comment)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd add that this is not a library, this is the project for the Django website.
I'm not sure the website needs the same level of compatibility as the Django framework.

Copy link
Member Author

@ulgens ulgens Oct 6, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so I advise against using it as a library dependency

This repo is not a library.

It's only suggested when you know exactly where an app will run

I'd say this is a given for a website project. I agree with @alexgmin

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This repo is not a library.

Oh sorry, I didn't understand it. In this case no problem at all to use paycopg[binary] and you can drop the libpq5/libpq-dev dependencies too.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @dvarrazzo, thank you for joining this conversation and for the in-depth discussion on psycopg/psycopg#1156!

To clarify one point, with this PR, we segmented the requirements so we can use psycopg[binary] for local development (to simplify the local setup process) and psycopg[c] in production, as this is how I interpreted the recommendation in the local installation instructions:

This is the preferred way to install Psycopg for a production site.

Could you clarify if psycopg[binary] is also intended for use in production environments? We control the environment (currently python:3.12-slim-trixie, plus pgdg for the correct postgresql-client version if that matters), but that will of course change as we upgrade Python & Debian releases over time.

Thanks so much for your feedback!

&& apt-get install --assume-yes --no-install-recommends ${BUILD_DEPENDENCIES} \
&& python3 -m pip install --no-cache-dir -r ${REQ_FILE} \
&& apt-get purge --assume-yes --auto-remove \
g++ \
gcc \
libc6-dev \
libpq-dev \
zlib1g-dev \
&& apt-get purge --assume-yes --auto-remove ${BUILD_DEPENDENCIES} \
&& apt-get distclean

# copy project
Expand Down
1 change: 1 addition & 0 deletions docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ services:
dockerfile: Dockerfile
args:
- REQ_FILE=requirements/tests.txt
- BUILD_DEPENDENCIES=g++ gcc
entrypoint: ./docker-entrypoint.dev.sh
command: python manage.py runserver 0.0.0.0:8000
volumes:
Expand Down
1 change: 0 additions & 1 deletion requirements/common.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,6 @@ Jinja2==3.1.6
libsass==0.23.0
Markdown==3.9
Pillow==11.3.0
psycopg[c]==3.2.9
Pygments==2.19.2
pykismet3==0.1.1
requests==2.32.5
Expand Down
1 change: 1 addition & 0 deletions requirements/dev.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
-r common.txt
django-debug-toolbar==6.0.0
pre-commit~=4.3.0
psycopg[binary]==3.2.10
watchdog==6.0.0
1 change: 1 addition & 0 deletions requirements/prod.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
-r common.txt
gunicorn==23.0.0
psycopg[c]==3.2.10
redis==6.4.0
sentry-sdk==2.38.0