Skip to content

Commit f4b106b

Browse files
RazvanLiviuVarzaruRazvanLiviuVarzaru
committed
MDBF-866 - Create bintar builder from non-EOL distro
AlmaLinux 8 was chosen as the oldest non-eol distro for a new bintar which will build += 11.7 Buildbot configuration changes will follow after this PR is merged. (new builder,worker). A dedicated workflow was defined, with a workflow_call directive to allow us to rebuild the image anytime we like. About bintar.Dockerfile - it is based on the centos.Dockerfile with small changes to remove redundant code - use galera-4 from RHEL-8 because we no longer build on alma/rocky, neither the server or the galera library. Compatibility wise, should be ok, because we also test RHEL packages on alma/rocky VM's. About static libs: - static libraries are build in a separate stage to reduce the image size - the general rule is to install the -devel packages (provide headers) then to compile the static library and move it in a path used then by CMAKE - the strategy to compile static is to take the source rpm and then install deps + re-build. Different libs have different needs in terms of what custom modifications of the SPEC files are needed. Testing: - one can build the image from scratch, compile the server and check resulted binaries with LDD - the list of *.sh files introduced by this patch is the minimum I found that will produce a binary package similar with what we've released so far.
1 parent d7792d9 commit f4b106b

File tree

9 files changed

+209
-0
lines changed

9 files changed

+209
-0
lines changed

‎.github/workflows/build-bintar.yml‎

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
name: Build Bintar image
2+
3+
on:
4+
push:
5+
branches:
6+
- 'main'
7+
- 'dev'
8+
paths:
9+
- 'ci_build_images/bintar.Dockerfile'
10+
- 'ci_build_images/qpress.Dockerfile'
11+
- 'ci_build_images/buildbot-worker.Dockerfile'
12+
- 'ci_build_images/scripts/**'
13+
- .github/workflows/build-bintar.yml
14+
- .github/workflows/bbw_build_container_template.yml
15+
pull_request:
16+
paths:
17+
- 'ci_build_images/bintar.Dockerfile'
18+
- 'ci_build_images/qpress.Dockerfile'
19+
- 'ci_build_images/buildbot-worker.Dockerfile'
20+
- 'ci_build_images/scripts/**'
21+
- .github/workflows/build-bintar.yml
22+
- .github/workflows/bbw_build_container_template.yml
23+
workflow_call:
24+
25+
jobs:
26+
build-images:
27+
strategy:
28+
fail-fast: false
29+
matrix:
30+
include:
31+
- image: almalinux:8
32+
platforms: linux/amd64
33+
tag: almalinux8-bintar
34+
nogalera: false
35+
36+
uses: ./.github/workflows/bbw_build_container_template.yml
37+
with:
38+
dockerfile: bintar.Dockerfile
39+
image: ${{ matrix.image }}
40+
platforms: ${{ matrix.platforms }}
41+
tag: ${{ matrix.tag }}
42+
nogalera: ${{ matrix.nogalera }}
43+
secrets: inherit

‎ci_build_images/bintar.Dockerfile‎

Lines changed: 90 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,90 @@
1+
# Buildbot worker for building MariaDB
2+
#
3+
# Provides a bintar image based on AlmaLinux with build dependencies
4+
# and statically compiled libraries
5+
6+
ARG BASE_IMAGE
7+
FROM "$BASE_IMAGE" AS buildeps
8+
LABEL maintainer="MariaDB Buildbot maintainers"
9+
10+
# Install updates and required packages
11+
# hadolint ignore=SC2086
12+
RUN dnf -y install 'dnf-command(config-manager)' \
13+
&& source /etc/os-release \
14+
&& ARCH=amd64 \
15+
&& dnf -y --enablerepo=extras install epel-release \
16+
&& dnf config-manager --set-enabled powertools \
17+
&& dnf -y module enable mariadb-devel \
18+
&& dnf -y install almalinux-release-devel \
19+
&& VERSION_ID=-${VERSION_ID} \
20+
&& VERSION_ID=${VERSION_ID%%.*} \
21+
&& dnf config-manager --add-repo https://ci.mariadb.org/galera/mariadb-4.x-latest-gal-${ARCH}-rhel${VERSION_ID}.repo \
22+
&& dnf -y upgrade \
23+
&& dnf -y groupinstall "Development Tools" \
24+
&& dnf -y builddep mariadb-server \
25+
&& dnf -y install \
26+
asio-devel \
27+
buildbot-worker \
28+
bzip2 \
29+
bzip2-devel \
30+
ccache \
31+
check-devel \
32+
cracklib-devel \
33+
createrepo \
34+
curl-devel \
35+
eigen3-devel \
36+
flex \
37+
galera-4 \
38+
gnutls-devel \
39+
java-1.8.0-openjdk-devel \
40+
java-1.8.0-openjdk \
41+
jemalloc-devel \
42+
libcurl-devel \
43+
libevent-devel \
44+
libffi-devel \
45+
libpmem-devel \
46+
libxml2-devel \
47+
libzstd-devel \
48+
perl-autodie \
49+
perl-Net-SSLeay \
50+
python3-devel \
51+
readline-devel \
52+
rpmlint \
53+
ruby \
54+
snappy-devel \
55+
subversion \
56+
unixODBC \
57+
unixODBC-devel \
58+
wget \
59+
which \
60+
xz-devel \
61+
yum-utils \
62+
&& dnf clean all \
63+
&& curl -sL "https://github.com/Yelp/dumb-init/releases/download/v1.2.5/dumb-init_1.2.5_$(uname -m)" >/usr/local/bin/dumb-init \
64+
&& chmod +x /usr/local/bin/dumb-init
65+
ENV WSREP_PROVIDER=/usr/lib64/galera-4/libgalera_smm.so
66+
67+
## Build the static libraries in a separate stage so we save space
68+
FROM buildeps AS staticlibs
69+
70+
COPY scripts/* /scripts/
71+
WORKDIR /scripts
72+
RUN chmod +x ./*.sh && ls -l
73+
74+
# Build static libraries
75+
RUN mkdir -p ./local/lib/ \
76+
&& ./libaio.sh \
77+
&& ./liblz4.sh \
78+
&& ./xz.sh \
79+
&& ./ncurses.sh \
80+
&& ./libpmem.sh \
81+
&& ./libzstd.sh \
82+
&& ./gnutls.sh
83+
84+
FROM buildeps AS bintar
85+
COPY --from=staticlibs /scripts/local/lib /scripts/local/lib
86+
87+
### Other .Dockerfiles will be concatenated here (worker, qpress, etc)
88+
89+
90+

‎ci_build_images/scripts/gnutls.sh‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
#!/bin/bash
2+
3+
set -e
4+
5+
yumdownloader --source gnutls-devel
6+
yum-builddep -y gnutls-*.src.rpm
7+
rpm -ivh gnutls-*.src.rpm
8+
sed -i 's/--disable-static/--enable-static \\\n--disable-tests/g' ~/rpmbuild/SPECS/gnutls.spec
9+
sed -i 's/export LDFLAGS="-Wl,--no-add-needed"/export LDFLAGS="-Wl,--copy-dt-needed-entries -fpic -fPIC"/g' ~/rpmbuild/SPECS/gnutls.spec
10+
sed -i 's/make %{?_smp_mflags}/make %{?_smp_mflags} CFLAGS="-fpic -fPIC" CXX_FLAGS="-fpic -fPIC"/g' ~/rpmbuild/SPECS/gnutls.spec
11+
rpmbuild -bc ~/rpmbuild/SPECS/gnutls.spec
12+
mv -v ~/rpmbuild/BUILD/gnutls-*/lib/.libs/libgnutls.a local/lib
13+
rm -rf ~/rpmbuild gnutls-*.src.rpm

‎ci_build_images/scripts/libaio.sh‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
#!/bin/bash
2+
3+
set -e
4+
5+
yumdownloader --source libaio-devel
6+
yum-builddep -y libaio-*.src.rpm
7+
rpmbuild --recompile libaio-*.src.rpm
8+
mv -v ~/rpmbuild/BUILD/libaio-*/src/*.a local/lib
9+
rm -rf ~/rpmbuild libaio-*.src.rpm

‎ci_build_images/scripts/liblz4.sh‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
#!/bin/bash
2+
3+
set -e
4+
5+
yumdownloader --source lz4-devel
6+
yum-builddep -y lz4-*.src.rpm
7+
rpm -ivh lz4-*.src.rpm
8+
sed -i 's/%make_build/%make_build CFLAGS="-fpic -fPIC" CXX_FLAGS="-fpic -fPIC"/g' ~/rpmbuild/SPECS/lz4.spec
9+
rpmbuild -bc ~/rpmbuild/SPECS/lz4.spec
10+
mv -v ~/rpmbuild/BUILD/lz4-*/lib/liblz4.a local/lib
11+
rm -rf ~/rpmbuild lz4-*.src.rpm

‎ci_build_images/scripts/libpmem.sh‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
#!/bin/bash
2+
3+
set -e
4+
5+
yumdownloader --source libpmem-devel
6+
yum-builddep -y pmdk-*.src.rpm
7+
rpmbuild --recompile pmdk-*.src.rpm
8+
mv -v ~/rpmbuild/BUILD/pmdk-*/src/nondebug/libpmem.a local/lib
9+
rm -rf ~/rpmbuild pmdk-*.src.rpm

‎ci_build_images/scripts/libzstd.sh‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
#!/bin/bash
2+
3+
set -e
4+
5+
yumdownloader --source libzstd
6+
yum-builddep -y zstd-*.src.rpm
7+
rpm -ivh zstd-*.src.rpm
8+
rpmbuild -bp ~/rpmbuild/SPECS/zstd.spec
9+
make CFLAGS=-fPIC -C ~/rpmbuild/BUILD/zstd-*
10+
mv -v ~/rpmbuild/BUILD/zstd-*/lib/libzstd.a local/lib
11+
rm -rf ~/rpmbuild zstd-*.src.rpm

‎ci_build_images/scripts/ncurses.sh‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
#!/bin/bash
2+
3+
set -e
4+
5+
yumdownloader --source ncurses-devel
6+
yum-builddep -y ncurses-*.src.rpm
7+
rpmbuild --recompile ncurses-*.src.rpm
8+
mv -v ~/rpmbuild/BUILDROOT/ncurses-*/usr/lib64/*.a local/lib
9+
rm -rf ~/rpmbuild ncurses-*.src.rpm

‎ci_build_images/scripts/xz.sh‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
#!/bin/bash
2+
3+
set -e
4+
5+
yumdownloader --source xz-devel
6+
yum-builddep -y xz-*.src.rpm
7+
rpm -ivh xz-*.src.rpm
8+
sed -i 's/--disable-static/--disable-shared --enable-static --with-pic/g' ~/rpmbuild/SPECS/xz.spec
9+
sed -i 's/CFLAGS="/CFLAGS="-fpic -fPIC /g' ~/rpmbuild/SPECS/xz.spec
10+
sed -i 's/export CFLAGS/export CFLAGS\nCXXFLAGS="-fpic -fPIC"\nexport CXXFLAGS/g' ~/rpmbuild/SPECS/xz.spec
11+
sed -i '/\*\.a/d' ~/rpmbuild/SPECS/xz.spec
12+
rpmbuild -bc ~/rpmbuild/SPECS/xz.spec
13+
mv -v ~/rpmbuild/BUILD/xz-*/src/liblzma/.libs/liblzma.a local/lib
14+
rm -rf ~/rpmbuild xz-*.src.rpm

0 commit comments

Comments
 (0)