jiveplot in kernsuite?

[haavee]

Hi Kernsuite maintainers!

I have received a request if we could integrate jiveplot (https://github.com/haavee/jiveplot) with kernsuite as some users are running into Python/numpy incompatibilities when combining ppgplot and jiveplot and python-casacore.

Recently I've made both ppgplot (https://github.com/haavee/ppgplot) and jiveplot "pip installable" in a virtual environment, and possibly in other environments too.

• ppgplot: pip-installable already in main branch (still called "master", sorry)
• jiveplot: pip-installable using the modern-building branch

Let me know what I need to do to help this/these modules be included!
Thanks in advance.

[Athanaseus]

Hi @haavee,

Thank you so much for working on making both ppgplot and jiveplot "pip-installable"!

One more thing that needs to be done is to create a release tag on GitHub—KERN will build from this.
Eventually, a PyPI release would also be great so that users can simply run pip install ppgplot jiveplot without needing to clone the source code.

Let me know if I can assist with these.

Best regards,

[haavee]

Hi Athanaseus!
Let's see if we can get this going step by step. I've created a ppgplot tag (v1.5) and also built a release out of that.

Next up is a PyPI release then? New territory for me!

[Athanaseus]

Awesome — great start!

Next up is a PyPI release then? New territory for me!

GitHub Actions would be a good option here — it can help automate the build and publish process whenever you push a new tag. Let me know if you'd like a hand setting that up!

Best regards

[haavee]

OK I have gone through the "test-pypi" hoops and pushed a version there:

https://test.pypi.org/project/ppgplot/

But when I try to install it in a fresh venv on my system I get this:

(venv) $> pip install -i https://test.pypi.org/simple/ ppgplot
Looking in indexes: https://test.pypi.org/simple/
Collecting ppgplot
  Downloading https://test-files.pythonhosted.org/packages/eb/b4/d830b249a248ad61659685abb841fc803d8340893dcc276b9b63c77ab48b/ppgplot-1.5-cp313-cp313-macosx_14_0_arm64.whl.metadata (3.8 kB)
INFO: pip is looking at multiple versions of ppgplot to determine which version is compatible with other requirements. This could take a while.
ERROR: Could not find a version that satisfies the requirement numpy>=1.21.0 (from ppgplot) (from versions: 1.9.3)
ERROR: No matching distribution found for numpy>=1.21.0

[Athanaseus]

This error occurs because pip tries to find numpy>=1.21.0 only on TestPyPI, which doesn't host it.
You need to add the main PyPI as a fallback source for dependencies like numpy.

Please try:

pip install --index-url https://test.pypi.org/simple/ \
            --extra-index-url https://pypi.org/simple \
            ppgplot

This will:

• Look for ppgplot on TestPyPI.
• Look for dependencies like numpy on the main PyPI.

[haavee]

Aha! That worked like a charm!

Installing collected packages: numpy, ppgplot
Successfully installed numpy-2.2.4 ppgplot-1.5

And then:

(venv) $> which python3
/private/tmp/venv/bin/python3
(venv) $> python3
Python 3.13.2 (main, Feb  4 2025, 14:51:09) [Clang 16.0.0 (clang-1600.0.26.6)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import ppgplot
>>> ppgplot.pgopen("42/xw")
1
>>> ppgplot.pgenv(0, 10, 0, 10)
>>> ppgplot.pgpt(range(1,8), range(1,8))
>>>

resulting in the attached plot!

So how would this work for other architectures than MacOSX/AMD64?

[haavee]

Hmmm. When I tried to upload to pypi I got an error - there is already a package called ppgplot (it's in this totally unmaintained one: https://pypi.org/project/rea/)

So now what? Rename the package? Can do it easily in pyproject.toml but than the package to install and Python module name would be different, in order not to break existing code that relies on being able to

import ppgplot

.... sigh

[Athanaseus]

So how would this work for other architectures than MacOSX/AMD64?

Python packages can be distributed as:

• Wheels (.whl): precompiled binaries for specific platforms (e.g., macosx_10_9_x86_64, manylinux_x86_64, win_amd64)
• Source distributions (.tar.gz): must be built locally, often requiring system-level libraries and compilers.

For example: https://pypi.org/project/scipy/#files
SciPy uploads a whole fleet of wheels, each tagged for a specific OS/platform and Architecture.

[Athanaseus]

So now what? Rename the package? Can do it easily in pyproject.toml but than the package to install and Python module name would be different, in order not to break existing code that relies on being able to

You're right that renaming the distribution package name (the one on PyPI and installable via pip install) doesn’t have to match the Python import name. But it can feel a little awkward.

Yes, you can do this cleanly using pyproject.toml or setup.py, by renaming only the install package name, not the module name.

[project]
name = "ppgplot2"  # <-- this is the name on PyPI
...

[tool.setuptools.packages.find]
include = ["ppgplot"]  # <-- this keeps `import ppgplot` working

or

setup(
    name="ppgplot2",  # <-- on PyPI
    packages=["ppgplot"],  # <-- Python module name
    ...
)

Now users would:

pip install ppgplot2

and still write:

import ppgplot

Also this can be mentioned in the README to avoid confusion:
“This package was renamed to ppgplot2 on PyPI due to a name conflict, but it still provides ppgplot as the importable module."

Let me know if you want help setting up a rename-friendly pyproject.toml.

[haavee]

So now what? Rename the package? Can do it easily in pyproject.toml but than the package to install and Python module name would be different, in order not to break existing code that relies on being able to

You're right that renaming the distribution package name (the one on PyPI and installable via pip install) doesn’t have to match the Python import name. But it can feel a little awkward.

Yes, you can do this cleanly using pyproject.toml or setup.py, by renaming only the install package name, not the module name.
<<< snip >>>

Also this can be mentioned in the README to avoid confusion: “This package was renamed to ppgplot2 on PyPI due to a name conflict, but it still provides ppgplot as the importable module."

OK that's what I experimented with locally - just modifying the pyproject.toml a bit, need to do the README too.

Then it comes to choosing a package name, would "python-pgplot" not be a reasonable package name? (I've read this SO answer and it seems legit)

By the way, in the setup.py this is what I currently have - should I add name = "..." to that?
(A bit hesitant to change these things, given that it's been quite a journey with "pyproject.toml" and "various build backends" - jeez)

###########################################################
#             This triggers the whole build               #
###########################################################
setup(
        ext_modules=[
            set_extension_config( Extension('ppgplot._ppgplot',
                                            sources=[os.path.join('src', '_ppgplot.c')]) ),
        ]
)

[haavee]

So how would this work for other architectures than MacOSX/AMD64?

Python packages can be distributed as:

• Wheels (.whl): precompiled binaries for specific platforms (e.g., macosx_10_9_x86_64, manylinux_x86_64, win_amd64)
• Source distributions (.tar.gz): must be built locally, often requiring system-level libraries and compilers.

Claro - that is mostly what I'd expected.
Then the follow up is: this is probably where your suggestion of "incorporating git workflows" comes in? Using build farm(s) to compile the wheels on various platforms?
If you happen to have some config(s) lying around that help with that that'd be awsome!

[Athanaseus]

Then it comes to choosing a package name, would "python-pgplot" not be a reasonable package name?

Sounds good to me.

By the way, in the setup.py this is what I currently have - should I add name = "..." to that?

Yes, in pyproject.toml or setup.py.

setup(
    name='python-pgplot',
    ...
)

Then the follow up is: this is probably where your suggestion of "incorporating git workflows" comes in? Using build farm(s) to compile the wheels on various platforms?

Yes, that's correct.

Please have a look at the examples here: https://github.com/scipy/scipy/blob/main/.github/workflows/

[Athanaseus]

GitHub Actions

Configure your pyproject.toml to specify build requirements and metadata:​

[build-system]
requires = ["setuptools>=42", "wheel", "cibuildwheel"]
build-backend = "setuptools.build_meta"

[project]
name = "python-pgplot"
version = "1.5.0"
description = "Python bindings for the PGPLOT graphics library"

Set up GitHub Actions to build wheels across multiple platforms:​

name: Build and Publish Wheels

on:
  push:
    tags:
      - "v*.*.*"

jobs:
  build_wheels:
    name: Build wheels on ${{ matrix.os }}
    runs-on: ${{ matrix.os }}
    strategy:
      matrix:
        os: [ubuntu-latest, macos-latest, windows-latest]

    steps:
      - uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.11"

      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          pip install cibuildwheel

      - name: Build wheels
        run: cibuildwheel --output-dir dist

      - name: Upload wheels
        uses: actions/upload-artifact@v4
        with:
          name: python-pgplot-wheels
          path: dist/

NB: This workflow will build wheels for Linux, macOS, and Windows whenever you push a new tag matching the pattern v*...

PyPI publishing

  publish_pypi:
    name: Publish to PyPI
    needs: build_wheels
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v4

      - name: Download built wheels
        uses: actions/download-artifact@v4
        with:
          name: built-wheels
          path: dist/

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.11"

      - name: Install Twine
        run: pip install twine

      - name: Publish to PyPI
        env:
          TWINE_USERNAME: __token__
          TWINE_PASSWORD: ${{ secrets.PYPI_API_TOKEN }}
        run: twine upload dist/*

PyPI Token Setup

• Go to https://pypi.org/manage/account/token/
• Create a new "Scoped token" with permission to upload to your package (e.g., python-pgplot)
• Copy the token value.
• In your GitHub repo:
  • Go to Settings → Secrets and variables → Actions
  • Add a new secret:
    Name: PYPI_API_TOKEN
    Value: (paste your PyPI token)

[haavee]

Hi Athanaseus!

FINALLY I've had some time to dedicate to this - it's peak holiday, everyone's buggered off, which means I finally got some time to myself.

Both the pgplot binary dependencies and jiveplot are now available from PyPI:
https://pypi.org/project/jiveplot/
https://pypi.org/project/python-pgplot/

Probably not yet 100% compatible with everything, but at least I've got the projects (re)structured, configured CI/CD workflows to build and publish wheels. Now it's fine-tuning, I hope.

[Athanaseus]

Hi @haavee,

That's fantastic news - great work getting that all set up.

I just tried pip installing both releases, and everything went smoothly on Ubuntu24 with Py3.12.