Skip to content

Commit aa66083

Browse files
hunschehunsche
authored
feat: Add base Dockerfile for Ubuntu 24.04` (#4938)
### **Context** This is the foundational first step in the project to upgrade ClusterFuzz's base images from Ubuntu 20.04 to 24.04. The primary motivation is to mitigate security and maintenance risks associated with the upcoming End-of-Life (EOL) of Ubuntu 20.04. This PR introduces the new, clean base image upon which all other specialized images (like the OSS-Fuzz image) will be built. ### **Description of Changes** * **New File:** Introduces a new `docker/base/Dockerfile`. * **Base Image:** Sets the parent image to `ubuntu:24.04`. * **Packages:** Installs a set of essential packages required for a basic fuzzing environment, including `build-essential`, `git`, `curl`, `wget`, `python3`, and `sudo`. * **Configuration:** Implements standard security best practices for the container image and sets up a non-root user. ### **How to Test** A reviewer can verify this change by building the Docker image locally. 1. Check out this branch. 2. From the root of the repository, run the build command: ```bash docker build -t clusterfuzz/base:ubuntu24.04 -f docker/base/Dockerfile . ``` 3. The build should complete successfully. You can also run the container to inspect it: ```bash docker run -it --rm clusterfuzz/base:ubuntu24.04 bash ``` ### **Related Issues** * This is part of the work for the main tracking issue: **b/395645705** * This PR specifically addresses the task: "Develop the base Dockerfile for Ubuntu 24.04". ### **Next Steps** Once this base image is approved and merged, the next task will be to create the `oss-fuzz` Dockerfile, which will use this image as its foundation.
1 parent 8383233 commit aa66083

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

49 files changed

+1638
-5
lines changed

‎docker/README.md‎

Lines changed: 63 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -2,15 +2,21 @@
22

33
## Local testing
44

5-
You can build an image locally as:
5+
You can build a specific image locally using the `docker build` command from the root of the **clusterfuzz** repository.
6+
7+
For example, to build the `base` image, run the following command from the root of the **clusterfuzz** repository:
68

79
```bash
8-
cd /path/to/image/dir
9-
docker build .
10+
docker build -f docker/base/Dockerfile docker/base
1011
```
1112

12-
where `/path/to/image/dir` is any image sub-directory in this directory that contains a
13-
'Dockerfile'.
13+
### Command Explanation
14+
15+
* `docker build`: The standard command to build a Docker image.
16+
* `-f docker/base/Dockerfile`: This flag specifies the path to the `Dockerfile` to be used, relative to the repository root.
17+
* `docker/base`: This is the "build context". The Docker daemon will have access to all files and folders within this path. This is necessary for the `COPY` instructions in the `Dockerfile`.
18+
19+
You can adapt this command to build other images by changing the path to the `Dockerfile` and the build context accordingly.
1420

1521
## Production
1622

@@ -22,3 +28,55 @@ To build all images on container builder, run:
2228

2329
Note that your checkout needs to be on the latest deployed commit.
2430
You also need to have access to the `clusterfuzz-images` project.
31+
## Docker Image Dependency Tree
32+
33+
```mermaid
34+
graph TD
35+
subgraph Base Images
36+
A(base)
37+
end
38+
39+
subgraph Chromium
40+
F(chromium/base)
41+
K(chromium/tester)
42+
L(chromium/python-profiler)
43+
M(chromium/high-end)
44+
E(chromium/tests-syncer)
45+
G(chromium/builder)
46+
end
47+
48+
subgraph OSS-Fuzz
49+
H(oss-fuzz/base)
50+
N(oss-fuzz/worker)
51+
O(oss-fuzz/host)
52+
P(oss-fuzz/host-high-end)
53+
end
54+
55+
subgraph General
56+
B(ci)
57+
C(utask-main-scheduler)
58+
D(fuchsia)
59+
I(tworker)
60+
J(high-end)
61+
end
62+
63+
64+
A --> B;
65+
A --> C;
66+
A --> D;
67+
A --> E;
68+
A --> F;
69+
A --> G;
70+
A --> H;
71+
A --> I;
72+
A --> J;
73+
74+
F --> K;
75+
F --> L;
76+
F --> M;
77+
78+
H --> N;
79+
H --> O;
80+
81+
O --> P;
82+
```

‎docker/base/CHANGELOG.md‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
# Docker Image Version Changelog: clusterfuzz/base
2+
3+
4+
## Analysis Summary
5+
6+
The `latest` and `ubuntu20` images are nearly identical, as they are both based on Ubuntu 20.04. The `ubuntu24` image, based on Ubuntu 24.04, introduces a number of package updates and changes. All three images were built successfully, but required workarounds to run and perform health checks due to a startup script that depends on a GCE metadata service. The `ubuntu24` image appears to be a viable replacement for the older images, but the package differences should be reviewed to ensure compatibility.
7+
8+
## Build Status
9+
10+
| Image Tag | Dockerfile | Status |
11+
| -------------------------- | ------------------------ | ------- |
12+
| `clusterfuzz/base:latest` | `Dockerfile` | Success |
13+
| `clusterfuzz/base:ubuntu20`| `ubuntu20-04.Dockerfile` | Success |
14+
| `clusterfuzz/base:ubuntu24`| `ubuntu24-04.Dockerfile` | Success |
15+
16+
## Package Comparison
17+
18+
### Key Differences (Ubuntu 20.04 vs. Ubuntu 24.04)
19+
20+
| Package | Ubuntu 20.04 Version | Ubuntu 24.04 Version | Notes |
21+
| ----------------------- | -------------------- | -------------------- | ----------------------------------- |
22+
| `libidn11` | `1.33-2.2ubuntu2` | - | Removed in Ubuntu 24.04 |
23+
| `libidn12` | - | `1.42-1build1` | Added in Ubuntu 24.04 |
24+
| `libncurses5-dev` | `6.2-0ubuntu2.1` | - | Removed in Ubuntu 24.04 |
25+
| `libncurses-dev` | `6.2-0ubuntu2.1` | `6.4+20240113-1ubuntu2` | Upgraded in Ubuntu 24.04 |
26+
| `libncursesw5` | `6.2-0ubuntu2.1` | - | Removed in Ubuntu 24.04 |
27+
| `libncursesw6` | `6.2-0ubuntu2.1` | `6.4+20240113-1ubuntu2` | Upgraded in Ubuntu 24.04 |
28+
| `libssl-dev` | `1.1.1f-1ubuntu2.24` | `3.0.13-0ubuntu3.5` | Upgraded in Ubuntu 24.04 |
29+
| `libssl1.1` | `1.1.1f-1ubuntu2.24` | - | Removed in Ubuntu 24.04 |
30+
| `libssl3t64` | - | `3.0.13-0ubuntu3.5` | Added in Ubuntu 24.04 |
31+
| `python3.8` | `3.8.10-0ubuntu1~20.04.18` | - | Removed in Ubuntu 24.04 |
32+
| `python3.12` | - | `3.12.3-1ubuntu0.8` | Added in Ubuntu 24.04 |
33+
34+
## Dockerfile Analysis
35+
36+
The main difference between the Dockerfiles is the base image:
37+
38+
* **`Dockerfile` (latest):** Uses `ubuntu:20.04` as the final base image, but also uses `ubuntu:16.04` in a multi-stage build to copy some older libraries.
39+
* **`ubuntu20-04.Dockerfile`:** Identical to the latest `Dockerfile`.
40+
* **`ubuntu24-04.Dockerfile`:** Uses `ubuntu:24.04` as the base image. It removes the multi-stage build with `ubuntu:16.04` and updates the package installation commands to reflect the changes in Ubuntu 24.04. For example, it installs `libidn12` instead of `libidn11`.

‎docker/base/ubuntu20-04.Dockerfile‎

Lines changed: 156 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,156 @@
1+
# Copyright 2025 Google LLC
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
# Copy some commonly linked library versions from xenial for backwards
16+
# compatibility with older builds.
17+
FROM ubuntu:16.04 as xenial
18+
19+
# Prevent interactive prompts during package installation. This seems to work
20+
# better than `ENV DEBIAN_FRONTEND=noninteractive` for some reason.
21+
RUN echo 'debconf debconf/frontend select Noninteractive' | debconf-set-selections
22+
23+
RUN apt-get update && \
24+
apt-get install -y \
25+
libcurl3-gnutls \
26+
libffi6 \
27+
libnettle6 \
28+
libssl1.0.0
29+
30+
FROM ubuntu:20.04
31+
32+
# And again with the newest ubuntu image.
33+
RUN echo 'debconf debconf/frontend select Noninteractive' | debconf-set-selections
34+
35+
RUN mkdir /data
36+
WORKDIR /data
37+
38+
RUN apt-get update && \
39+
apt-get upgrade -y && \
40+
apt-get autoremove -y && \
41+
apt-get install -y \
42+
apt-transport-https \
43+
build-essential \
44+
curl \
45+
gdb \
46+
iproute2 \
47+
libbz2-dev \
48+
libcurl4-openssl-dev \
49+
libffi-dev \
50+
libgdbm-dev \
51+
libidn11 \
52+
liblzma-dev \
53+
libncurses5-dev \
54+
libncursesw5 \
55+
libnss3-dev \
56+
libreadline-dev \
57+
libsqlite3-dev \
58+
libssl-dev \
59+
libtinfo5 \
60+
locales \
61+
lsb-release \
62+
net-tools \
63+
psmisc \
64+
socat \
65+
sudo \
66+
unzip \
67+
util-linux \
68+
wget \
69+
zip \
70+
zlib1g-dev
71+
72+
COPY --from=xenial \
73+
/lib/x86_64-linux-gnu/libcrypto.so.1.0.0 \
74+
/lib/x86_64-linux-gnu/libssl.so.1.0.0 \
75+
/lib/x86_64-linux-gnu/
76+
COPY --from=xenial \
77+
/usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.* \
78+
/usr/lib/x86_64-linux-gnu/libffi.so.6.* \
79+
/usr/lib/x86_64-linux-gnu/libnettle.so.* \
80+
/usr/lib/x86_64-linux-gnu/
81+
82+
# Install patchelf.
83+
RUN curl -sS https://releases.nixos.org/patchelf/patchelf-0.9/patchelf-0.9.tar.bz2 | tar -C /tmp -xj && \
84+
cd /tmp/patchelf-*/ && \
85+
./configure --prefix=/usr && \
86+
make install && \
87+
rm -rf /tmp/patchelf-*
88+
89+
# Install OpenJDK 17 for Jazzer (Java fuzzer).
90+
# Copied from gcr.io/oss-fuzz-base/base-runner.
91+
ENV JAVA_HOME /usr/lib/jvm/java-17-openjdk-amd64
92+
ENV JAVA_15_HOME /usr/lib/jvm/java-15-openjdk-amd64
93+
ENV JVM_LD_LIBRARY_PATH=$JAVA_HOME/lib/server
94+
ENV PATH=$PATH:$JAVA_HOME/bin
95+
96+
RUN cd /tmp && \
97+
curl --silent -L -O https://download.java.net/java/GA/jdk17.0.2/dfd4a8d0985749f896bed50d7138ee7f/8/GPL/openjdk-17.0.2_linux-x64_bin.tar.gz && \
98+
mkdir -p $JAVA_HOME && \
99+
tar -xzv --strip-components=1 -f openjdk-17.0.2_linux-x64_bin.tar.gz --directory $JAVA_HOME && \
100+
rm -rf openjdk*.tar.gz $JAVA_HOME/jmods $JAVA_HOME/lib/src.zip
101+
102+
# Install OpenJDK 15 and trim its size by removing unused components. Some projects only run with Java 15.
103+
RUN cd /tmp && \
104+
curl --silent -L -O https://download.java.net/java/GA/jdk15.0.2/0d1cfde4252546c6931946de8db48ee2/7/GPL/openjdk-15.0.2_linux-x64_bin.tar.gz && \
105+
mkdir -p $JAVA_15_HOME && \
106+
tar -xz --strip-components=1 -f openjdk-15.0.2_linux-x64_bin.tar.gz --directory $JAVA_15_HOME && \
107+
rm -rf openjdk*.tar.gz $JAVA_15_HOME/lib/src.zip
108+
109+
# Install Python 3.11
110+
RUN curl -sS https://www.python.org/ftp/python/3.11.4/Python-3.11.4.tgz | tar -C /tmp -xzv && \
111+
cd /tmp/Python-3.11.4 && \
112+
./configure --enable-optimizations --enable-loadable-sqlite-extensions && make altinstall && \
113+
rm -rf /tmp/Python-3.11.4 /tmp/Python-3.11.4.tar.xz
114+
RUN pip3.11 --no-cache-dir install pipenv==2022.8.5
115+
116+
# Install Node.js
117+
COPY setup_19.x /data
118+
RUN bash setup_19.x && apt-get update -y && apt-get install -y nodejs
119+
120+
RUN echo "deb https://packages.cloud.google.com/apt cloud-sdk main" \
121+
| tee -a /etc/apt/sources.list.d/google-cloud-sdk.list && \
122+
curl https://packages.cloud.google.com/apt/doc/apt-key.gpg \
123+
| apt-key add - && \
124+
apt-get update -y && \
125+
apt-get install -y google-cloud-sdk
126+
127+
# Common environment variables.
128+
ENV USER=clusterfuzz
129+
ENV INSTALL_DIRECTORY /mnt/scratch0
130+
ENV BOT_TMPDIR $INSTALL_DIRECTORY/tmp
131+
ENV ROOT_DIR $INSTALL_DIRECTORY/clusterfuzz
132+
ENV UPDATE_WEB_TESTS True
133+
ENV PYTHONPATH $INSTALL_DIRECTORY/clusterfuzz/src
134+
ENV RUN_CMD "python3.11 $ROOT_DIR/src/python/bot/startup/run.py"
135+
136+
# Passwordless sudo (needed for AFL launcher).
137+
RUN groupadd nopwsudo && \
138+
echo "%nopwsudo ALL=(ALL:ALL) NOPASSWD:ALL" > /etc/sudoers.d/mysudoers
139+
140+
# Make sure GSUtil uses the GCE service account.
141+
RUN echo '[GoogleCompute]\nservice_account = default' > /etc/boto.cfg
142+
143+
VOLUME $INSTALL_DIRECTORY
144+
WORKDIR $INSTALL_DIRECTORY
145+
146+
RUN locale-gen en_US.UTF-8
147+
ENV LANG en_US.UTF-8
148+
ENV PYTHONIOENCODING UTF-8
149+
150+
COPY setup_common.sh setup_clusterfuzz.sh start_clusterfuzz.sh setup_mock_metadata.sh Pipfile Pipfile.lock start.sh /data/
151+
RUN cd /data && \
152+
# Make pip3.11 the default so that pipenv install --system works.
153+
mv /usr/local/bin/pip3.11 /usr/local/bin/pip && \
154+
pipenv install --deploy --system
155+
156+
CMD ["bash", "-ex", "/data/start.sh"]

0 commit comments

Comments
 (0)