Consider updating Docker CI flow between several p4lang projects to use Ubuntu 24.04, instead of current Ubuntu 20.04

[jafingerhut]

If I understand what we have now, it was created by Antonin Bas, and has the following "flow":

• This Dockerfile starts with a base ubuntu20.04 Docker image, and adds some packages like grpc, thrift, and a few others developed outside of the p4lang organization, and are major dependencies used by some p4lang projects.
  • Dockerfile: https://github.com/p4lang/third-party/blob/main/Dockerfile
  • CI script where this Dockerfile is used: https://github.com/p4lang/third-party/blob/main/.github/workflows/build.yml
  • output Docker image name: p4lang/third-party
• This Dockerfile starts from p4lang/third-party image, and adds built code from the p4lang/PI repo:
  • Dockerfile: https://github.com/p4lang/PI/blob/main/Dockerfile
  • CI script: https://github.com/p4lang/PI/blob/main/.github/workflows/build.yml
  • output Docker image name: p4lang/pi
• This Dockerfile starts from p4lang/pi, and adds built code from the p4lang/behavioral-model repo:
  • Dockerfile: https://github.com/p4lang/behavioral-model/blob/main/Dockerfile
  • CI script: https://github.com/p4lang/behavioral-model/blob/main/.github/workflows/build.yml
  • output Docker image name: p4ang/behavioral-model
• This Dockerfile starts from p4lang/behavioral-model,
  • Dockerfile: https://github.com/p4lang/p4c/blob/main/Dockerfile
  • CI script where this Dockerfile is used: I am not sure if this file uses that Docker image or not, or precisely how: https://github.com/p4lang/p4c/blob/main/.github/workflows/ci-container-image.yml
  • output DOcker image named: p4lang/p4c

At one time, the p4lang/p4c repo used that p4lang/behavioral-model Docker image to run p4c CI tests, but I think it has been a while since it switched to using Ubuntu deb packages installed via the apt-get command. Unfortunately those deb packages have not been updated since August 2023. The person who was generating them appears to have moved on to other things, and no one else has chosen to update them.

Recently I have propopsed to the P4 Governance Board an idea from Fabian Ruffy: hire someone part time whose main job will be to maintain the CI infrastructure of multiple p4lang projects, including at least all of the ones mentioned above. Sure, if volunteers are willing to do it, great, or if companies are willing to pay people to do this part-time, excellent, but that has been a very short list of people over the last 5 years (mostly Antonin Bas and Fabian Ruffy). The P4 Governance Board sounds open to the idea of paying some money for a qualified person to work on this, but selecting a qualified person has only barely started, and I do not know how long it will take.

All of that said, if there is some specific flow of compute jobs that you are interested in creating automation for, in a way that you believe others will be able to maintain and enhance later, let us know. There are many choices besides Docker, I am sure, but I don't have good enough knowledge of the alernatives to compare and contrast those choices.

Here are some PRs and branches started by Andy Fingerhut in early 2025 that are an attempt to create modifications to the Dockerfiles mentioned above, that start with ubuntu24.04 instead of ubuntu20.04. They do not yet achieve a working build for all of the p4lang repos involved:

• Use Ubuntu 24.04 instead of 20.04 for docker container third-party#36
• https://github.com/jafingerhut/PI/tree/use-jafingerhut-docker-image
• https://github.com/jafingerhut/behavioral-model/tree/dockerfile-updates-for-ubuntu-24.04
• https://github.com/jafingerhut/p4c/tree/dockerfile-updates-for-ubuntu-24.04

[jafingerhut]

@verios-google Above are my notes and work-in-progress attempts to modify the existing Dockerfile CI workflow to use ubuntu24.04 instead of ubuntu20.04.

[Vineet1101]

@jafingerhut I want to work on this task I know basic of docker. I might not be able to complete the whole but can solve some minor tasks and resolve errors Can you tell some tasks from where I can start?

[jafingerhut]

The end goal here is to get the sequence of 4 Dockerfiles for third-party, PI, behavioral-model, then p4c to start with an ubuntu24.04 image, and produce a Docker image with all of the code in those 4 repositories built and installed in the final Docker image such that it can run p4c tests and pass them.

I don't know if there are any minor tasks to complete for this overall goal, other than "try to make the current thing that isn't working, working". It could take days or weeks of effort, depending upon how familiar one is with all of the software packages being installed, the dependencies between them, what options there are for installing them, etc.

Notes on failures experienced trying the current Dockerfiles that once were able to create Docker images based on ubuntu20.04:

Without modifying anything in any p4lang repositories, you can try to use the current Dockerfile in the repository https://github.com/p4lang/third-party to create a Docker image, using instructions I have recorded here: https://github.com/jafingerhut/p4-guide/tree/master/dependencies/p4lang-docker-ubuntu-20.04

The result I got: A step during docker build ... fails, while attempting to create Python package files for grpc. You can see the error messages near the end of this output file that I recorded during my recent attempt: https://github.com/jafingerhut/p4-guide/blob/master/dependencies/p4lang-docker-ubuntu-20.04/output/out-docker-build-third-party-1.txt

I see a similar failure in log file failure messages on a recent PR in the third-party repo, here: p4lang/third-party#35 so it is not only on my local system that this error occurs.

I believe that these Dockerfiles once worked, but I do not know how long ago that was. I do not have any specific suggestions for small changes to try to make the current Dockerfile in the third-party work again. I personally think it is more important to get a similar Dockerfile setup that starts from an ubuntu24.04 image, than it is to fix the current one that uses ubuntu20.04, but if someone makes the ubuntu20.04 one work, that might also be nice to have.

Notes on creating updated Dockerfiles that start with ubuntu24.04:

I did a lot of experiments a month ago trying to rewrite the current 4 Dockerfiles to start with ubuntu24.04. I have an updated version of Dockerfile for the third-party and PI repositories that do not give any errors while they are being built, but the resulting Docker image fails to build behavioral-model code. It seems likely that the root cause is in one of my modified third-party or PI Dockerfiles, but I do not know for sure.

See my notes on steps to reproduce what I currently have, here: https://github.com/jafingerhut/p4-guide/tree/master/dependencies/p4lang-docker-ubuntu-24.04

[p2peek]

I have opened a PR updating the Dockerfile to support Ubuntu 24.04.

PR: p4lang/third-party#37

Local testing confirmed successful builds for:

ubuntu:24.04
→ third-party
→ PI
→ behavioral-model

The p4c build still requires additional adjustments, which I will investigate next.