Skip to content

Conversation

@tuhaihe
Copy link
Member

@tuhaihe tuhaihe commented Aug 1, 2025

Replace pip3 download commands with curl for downloading psutil,
PyYAML and PyGreSQL packages in gpMgmt/bin/Makefile.

This change unifies the download mechanism to use curl consistently
for all Python dependencies, avoiding potential pip3 download
compatibility issues across different environments.

Changes:

  • Remove pip3 download commands for psutil, PyYAML and PyGreSQL
  • Use curl to download packages directly from PyPI
  • Maintain existing wheel and cython installation for PyYAML build
  • Keep the same package versions and download locations

Fixes #ISSUE_Number

What does this PR do?

Type of Change

  • Bug fix (non-breaking change)
  • New feature (non-breaking change)
  • Breaking change (fix or feature with breaking changes)
  • Documentation update

Breaking Changes

Test Plan

  • Unit tests added/updated
  • Integration tests added/updated
  • Passed make installcheck
  • Passed make -C src/test installcheck-cbdb-parallel

Impact

Performance:

User-facing changes:

Dependencies:

Checklist

Additional Context

CI Skip Instructions


@tuhaihe
Copy link
Member Author

tuhaihe commented Aug 1, 2025

We will have a failure during the build process if enabling the configure option --with-pythonsrc-ext without pre-installing the python3-pip package in Ubuntu. python3-pip will be installed as a dependency by Python in Rocky Linux, but will not be installed in Ubuntu Docker Image.

The following is how to fix it:

  • Pre-install python3-pip as a required dependency;
  • Check if the python3-pip is installed; if not, will install this package to resolve the failure.

@tuhaihe tuhaihe force-pushed the fix-pip-ubuntu branch 5 times, most recently from 2f15c6b to 09851d4 Compare August 3, 2025 04:00
@tuhaihe tuhaihe changed the title Add automatic pip3 check based on package manager Replace pip3 download with curl for Python deps Aug 3, 2025
@tuhaihe
Copy link
Member Author

tuhaihe commented Aug 4, 2025

Updates: will add the curl, pip3 check in the configure.ac.

Why?

  • curl and pip3 are not default-installed in the Ubuntu 20.04/22.04 Docker image. So for a better user experience, we need to check these dependencies before make install the Cloudberry. When users run the configure with the option --with-pythonsrc-ext, it will check the dependencies earlier.

Why use curl instead of pip3 download?

  • pip3 download has some compatibility issues between different Python versions (3.6 under Rocky8, 3.9 under Rocky9), especially for downloading pyyaml - pyaml 5.7 is a little old and has different behavior across Ubuntu and Rockylinux. So for a simple way, change the way of downloading these Python packages.

@tuhaihe tuhaihe force-pushed the fix-pip-ubuntu branch 2 times, most recently from a5b64d0 to 96f4788 Compare August 4, 2025 10:02
@tuhaihe
Copy link
Member Author

tuhaihe commented Aug 4, 2025

Already tested successfully on Ubuntu 20.04/22.04 + Rocky Linux 8/9.

@tuhaihe
Copy link
Member Author

tuhaihe commented Aug 4, 2025

Note

This commit will also be cherry-picked into the branch REL_2_STABLE.

@tuhaihe tuhaihe requested a review from gfphoenix78 August 4, 2025 10:15
Copy link
Contributor

@jiaqizho jiaqizho left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Replace pip3 download commands with curl for downloading psutil,
PyYAML, PyGreSQL and mock packages in gpMgmt/bin/Makefile.

This change unifies the download mechanism to use curl consistently
for all Python dependencies, avoiding potential pip3 download
compatibility issues across different environments.

Changes:
- Remove pip3 download commands for psutil, PyYAML, PyGreSQL & mock
- Use curl to download packages directly from PyPI
- Maintain existing wheel and cython installation for PyYAML build
- Keep the same package versions and download locations

Also, add curl and pip3 dependency detection in configure.ac file for
better user experience. When `--with-pythonsrc-ext` option is enabled,
configure can check the required dependencies early.
Copy link

@xtangcode xtangcode left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Configure updates to support deps.

@tuhaihe tuhaihe merged commit d5967fd into apache:main Aug 6, 2025
27 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants