Skip to content

Update ffmpeg and change static build provider #64

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Sayan- merged 17 commits into from
Nov 4, 2025
Merged

Update ffmpeg and change static build provider #64

Show file tree
Hide file tree
Changes from 13 commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
68a1256
upd(build): reduce image size and build time
matthewjmarangoni Aug 22, 2025
4575a56
upd(build): use cached ffmpeg when host is unreachable
matthewjmarangoni Aug 27, 2025
56c8c09
upd(build): erase straggling metadata removal
matthewjmarangoni Aug 27, 2025
5f938da
upd(build): improve cache writer parallelism
matthewjmarangoni Aug 29, 2025
2692ad5
upd(build): install recommended chromium packages
matthewjmarangoni Sep 3, 2025
05a63a1
fix(build): whitespace and command order consistency
matthewjmarangoni Sep 9, 2025
a724b78
upd(build): upgrade ffmpeg and change provider
matthewjmarangoni Aug 29, 2025
f460cf2
fix(build): avoid potential of pattern evaluation
matthewjmarangoni Sep 10, 2025
f06f24d
upd(build): stage ffmpeg download
matthewjmarangoni Sep 18, 2025
95ab4be
upd(build): add extra space for checksum format
matthewjmarangoni Sep 30, 2025
0133b6f
upd(build): change ffmpeg cache access
matthewjmarangoni Sep 30, 2025
fe3e0e5
upd(build): add cache to go and npm invocations
matthewjmarangoni Oct 31, 2025
d4db75e
Merge branch 'main' into upd-change-ffmpeg-provider
matthewjmarangoni Oct 31, 2025
3cc12de
fix(build): limit ffmpeg to 7.1 latest releases
matthewjmarangoni Nov 1, 2025
c360d31
upd(build): handle arm64 and amd64 ffmpeg architectures
matthewjmarangoni Nov 3, 2025
9d94b9a
upd(build): simplify implementation
matthewjmarangoni Nov 3, 2025
2f19ef4
upd(build): support multi-platform builds, refine caching
matthewjmarangoni Nov 4, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
83 changes: 50 additions & 33 deletions images/chromium-headful/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,9 @@ RUN --mount=type=cache,target=/root/.cache/go-build \
go build -ldflags="-s -w" -o /out/kernel-images-api ./cmd/api

# Build chromium launcher
RUN GOOS=${TARGETOS:-linux} GOARCH=${TARGETARCH:-amd64} \
RUN --mount=type=cache,target=/root/.cache/go-build \
--mount=type=cache,target=/go/pkg/mod \
GOOS=${TARGETOS:-linux} GOARCH=${TARGETARCH:-amd64} \
go build -ldflags="-s -w" -o /out/chromium-launcher ./cmd/chromium-launcher

# webrtc client
Expand Down Expand Up @@ -60,6 +62,50 @@ RUN set -eux; \
make -j$(nproc); \
make install;

FROM docker.io/ubuntu:22.04 AS ffmpeg-downloader
Copy link
Contributor

@Sayan- Sayan- Nov 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
FROM docker.io/ubuntu:22.04 AS ffmpeg-downloader
FROM docker.io/ubuntu:22.04 AS ffmpeg-downloader
ARG TARGETARCH

Copy link
Contributor

@Sayan- Sayan- Nov 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For context the failure I was seeing:

[kernel-images-api] panic: ffmpeg not found or not executable: signal: trace/breakpoint trap
[kernel-images-api]
[kernel-images-api] goroutine 1 [running]:
[kernel-images-api] main.mustFFmpeg()
[kernel-images-api] 	/workspace/server/cmd/api/main.go:212 +0x90
[kernel-images-api] main.main()
[kernel-images-api] 	/workspace/server/cmd/api/main.go:48 +0x2b8
[kernel-images-api] time=2025-11-03T11:27:53.645-08:00 level=INFO msg="server configuration" config="&{Port:10001 FrameRate:10 DisplayNum:1 MaxSizeInMB:500 OutputDir:/recordings PathToFFmpeg:ffmpeg LogCDPMessages:false}"
[kernel-images-api] panic: ffmpeg not found or not executable: signal: trace/breakpoint trap
[kernel-images-api]
[kernel-images-api] goroutine 1 [running]:
[kernel-images-api] main.mustFFmpeg()
[kernel-images-api] 	/workspace/server/cmd/api/main.go:212 +0x90
[kernel-images-api] main.main()
[kernel-images-api] 	/workspace/server/cmd/api/main.go:48 +0x2b8
kernel-images-api: ERROR (spawn error)

under the hood when running build-docker.sh on my macbook I ended up with the wrong arch

root@5c30d0e00714:/# which ffmpeg
/usr/local/bin/ffmpeg
root@5c30d0e00714:/# ls -la /usr/local/bin/ffmpeg
-rwxr-xr-x 1 root root 137227080 Nov  3 19:23 /usr/local/bin/ffmpeg
root@5c30d0e00714:/# /usr/local/bin/ffmpeg -version
rosetta error: failed to open elf at /lib64/ld-linux-x86-64.so.2
 Trace/breakpoint trap
root@5c30d0e00714:/# ldd /usr/local/bin/ffmpeg
	not a dynamic executable
root@5c30d0e00714:/# uname -m
aarch64
root@5c30d0e00714:/# dpkg --print-architecture
arm64

RUN --mount=type=cache,target=/var/cache/apt,sharing=private,id=ubuntu2204-aptcache \
--mount=type=cache,target=/var/lib/apt,sharing=private,id=ubuntu2204-aptlib \
rm -f /etc/apt/apt.conf.d/docker-clean; \
echo 'Binary::apt::APT::Keep-Downloaded-Packages "true";' > /etc/apt/apt.conf.d/keep-cache; \
set -xe; \
apt-get -yqq update; \
apt-get -yqq --no-install-recommends install ca-certificates curl xz-utils;

# Download FFmpeg (latest static build) for the recording server
RUN --mount=type=cache,target=/tmp/cache/ffmpeg,sharing=private,id=ffmpeg \
<<-'EOT'
set -eux
ARCHIVE_NAME="ffmpeg-master-latest-linux64-gpl.tar.xz"
URL="https://github.com/BtbN/FFmpeg-Builds/releases/download/latest/$ARCHIVE_NAME"
echo "Downloading FFmpeg checksum"
if ! curl --connect-timeout 10 -fsSL "https://github.com/BtbN/FFmpeg-Builds/releases/download/latest/checksums.sha256" -o /tmp/cache/ffmpeg/ffmpeg.tar.xz.sha256; then
echo "Failed to connect to ffmpeg static build provider for checksum."
echo "Checking for cached version to use."
if [ ! -f /tmp/cache/ffmpeg/ffmpeg.tar.xz.sha256 ]; then
echo "Unable to locate cached checksum. Exiting."
exit 1
else
echo "Found cached checksum."
fi
fi
grep -F "$ARCHIVE_NAME" /tmp/cache/ffmpeg/ffmpeg.tar.xz.sha256 > /tmp/cache/ffmpeg/tmp_sha256_only
mv /tmp/cache/ffmpeg/tmp_sha256_only /tmp/cache/ffmpeg/ffmpeg.tar.xz.sha256
sed -i -e 's/ .*$/ \/tmp\/cache\/ffmpeg\/ffmpeg.tar.xz/' /tmp/cache/ffmpeg/ffmpeg.tar.xz.sha256
echo "Checking cache for FFmpeg archive and validating checksum"
if sha256sum --check /tmp/cache/ffmpeg/ffmpeg.tar.xz.sha256; then
echo "Checksum validated, using cached FFmpeg archive"
else
echo "Downloading FFmpeg static build from $URL"
curl -fsSL "$URL" -o /tmp/cache/ffmpeg/ffmpeg.tar.xz
echo "Validating checksum of FFmpeg static build download"
sha256sum --check /tmp/cache/ffmpeg/ffmpeg.tar.xz.sha256
fi
Copy link

@cursor cursor bot Sep 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: FFmpeg Build Failures Due to Checksum Validation Issues

The FFmpeg download and caching logic has a couple of issues that can cause builds to fail. If the remote checksum download fails, a newly fetched 'latest' archive is incorrectly validated against an outdated cached checksum. Separately, if the archive name isn't found in the checksum file, the file can be corrupted, preventing successful validation.

Additional Locations (1)

Fix in Cursor Fix in Web

tar -xJf /tmp/cache/ffmpeg/ffmpeg.tar.xz -C /tmp
install -m755 /tmp/ffmpeg-*/bin/ffmpeg /usr/local/bin/ffmpeg
install -m755 /tmp/ffmpeg-*/bin/ffprobe /usr/local/bin/ffprobe
rm -rf /tmp/ffmpeg*
EOT

FROM ghcr.io/onkernel/neko/base:3.0.8-v1.3.0 AS neko
# ^--- now has event.SYSTEM_PONG with legacy support to keepalive
FROM node:22-bullseye-slim AS node-22
Expand Down Expand Up @@ -119,40 +165,11 @@ RUN --mount=type=cache,target=/var/cache/apt,sharing=private,id=ubuntu2204-aptca
fonts-nanum \
fontconfig \
unzip && \
apt-get clean && fc-cache -f
fc-cache -f

# install ffmpeg manually since the version available in apt is from the 4.x branch due to #drama.
# as of writing these static builds will be the latest 7.0.x release.
RUN --mount=type=cache,target=/tmp/cache/ffmpeg,id=ffmpeg \
<<-'EOT'
set -eux
URL="https://johnvansickle.com/ffmpeg/releases/ffmpeg-release-amd64-static.tar.xz"
echo "Downloading FFmpeg MD5 checksum"
if ! curl --connect-timeout 10 -fsSL "${URL}.md5" -o /tmp/cache/ffmpeg/ffmpeg.tar.xz.md5; then
echo "Failed to connect to ffmpeg static build provider for MD5 checksum."
echo "Checking for cached version to use."
if [ ! -f /tmp/cache/ffmpeg/ffmpeg.tar.xz.md5 ]; then
echo "Unable to locate cached MD5 checksum. Exiting."
exit 1
else
echo "Found cached MD5 checksum."
fi
fi
sed -i -e 's/ .*$/ \/tmp\/cache\/ffmpeg\/ffmpeg.tar.xz/' /tmp/cache/ffmpeg/ffmpeg.tar.xz.md5
echo "Checking cache for FFmpeg archive and validating MD5 checksum"
if md5sum --check /tmp/cache/ffmpeg/ffmpeg.tar.xz.md5; then
echo "Checksum validated, using cached FFmpeg archive"
else
echo "Downloading FFmpeg static build from $URL"
curl -fsSL "$URL" -o /tmp/cache/ffmpeg/ffmpeg.tar.xz
echo "Validating MD5 checksum of FFmpeg static build download"
md5sum --check /tmp/cache/ffmpeg/ffmpeg.tar.xz.md5
fi
tar -xJf /tmp/cache/ffmpeg/ffmpeg.tar.xz -C /tmp
install -m755 /tmp/ffmpeg-*/ffmpeg /usr/local/bin/ffmpeg
install -m755 /tmp/ffmpeg-*/ffprobe /usr/local/bin/ffprobe
rm -rf /tmp/ffmpeg*
EOT
COPY --from=ffmpeg-downloader /usr/local/bin/ffmpeg /usr/local/bin/ffmpeg
COPY --from=ffmpeg-downloader /usr/local/bin/ffprobe /usr/local/bin/ffprobe

# runtime
ENV USERNAME=root
Expand Down
82 changes: 50 additions & 32 deletions images/chromium-headless/image/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,9 +22,55 @@ RUN --mount=type=cache,target=/root/.cache/go-build \
go build -ldflags="-s -w" -o /out/kernel-images-api ./cmd/api

# Build chromium launcher
RUN GOOS=${TARGETOS:-linux} GOARCH=${TARGETARCH:-amd64} \
RUN --mount=type=cache,target=/root/.cache/go-build \
--mount=type=cache,target=/go/pkg/mod \
GOOS=${TARGETOS:-linux} GOARCH=${TARGETARCH:-amd64} \
go build -ldflags="-s -w" -o /out/chromium-launcher ./cmd/chromium-launcher

FROM docker.io/ubuntu:22.04 AS ffmpeg-downloader
RUN --mount=type=cache,target=/var/cache/apt,sharing=private,id=ubuntu2204-aptcache \
--mount=type=cache,target=/var/lib/apt,sharing=private,id=ubuntu2204-aptlib \
rm -f /etc/apt/apt.conf.d/docker-clean; \
echo 'Binary::apt::APT::Keep-Downloaded-Packages "true";' > /etc/apt/apt.conf.d/keep-cache; \
set -xe; \
apt-get -yqq update; \
apt-get -yqq --no-install-recommends install ca-certificates curl xz-utils;

# Download FFmpeg (latest static build) for the recording server
RUN --mount=type=cache,target=/tmp/cache/ffmpeg,sharing=private,id=ffmpeg \
<<-'EOT'
set -eux
ARCHIVE_NAME="ffmpeg-master-latest-linux64-gpl.tar.xz"
URL="https://github.com/BtbN/FFmpeg-Builds/releases/download/latest/$ARCHIVE_NAME"
echo "Downloading FFmpeg checksum"
if ! curl --connect-timeout 10 -fsSL "https://github.com/BtbN/FFmpeg-Builds/releases/download/latest/checksums.sha256" -o /tmp/cache/ffmpeg/ffmpeg.tar.xz.sha256; then
echo "Failed to connect to ffmpeg static build provider for checksum."
echo "Checking for cached version to use."
if [ ! -f /tmp/cache/ffmpeg/ffmpeg.tar.xz.sha256 ]; then
echo "Unable to locate cached checksum. Exiting."
exit 1
else
echo "Found cached checksum."
fi
fi
grep -F "$ARCHIVE_NAME" /tmp/cache/ffmpeg/ffmpeg.tar.xz.sha256 > /tmp/cache/ffmpeg/tmp_sha256_only
mv /tmp/cache/ffmpeg/tmp_sha256_only /tmp/cache/ffmpeg/ffmpeg.tar.xz.sha256
sed -i -e 's/ .*$/ \/tmp\/cache\/ffmpeg\/ffmpeg.tar.xz/' /tmp/cache/ffmpeg/ffmpeg.tar.xz.sha256
echo "Checking cache for FFmpeg archive and validating checksum"
if sha256sum --check /tmp/cache/ffmpeg/ffmpeg.tar.xz.sha256; then
echo "Checksum validated, using cached FFmpeg archive"
else
echo "Downloading FFmpeg static build from $URL"
curl -fsSL "$URL" -o /tmp/cache/ffmpeg/ffmpeg.tar.xz
echo "Validating checksum of FFmpeg static build download"
sha256sum --check /tmp/cache/ffmpeg/ffmpeg.tar.xz.sha256
fi
tar -xJf /tmp/cache/ffmpeg/ffmpeg.tar.xz -C /tmp
install -m755 /tmp/ffmpeg-*/bin/ffmpeg /usr/local/bin/ffmpeg
install -m755 /tmp/ffmpeg-*/bin/ffprobe /usr/local/bin/ffprobe
rm -rf /tmp/ffmpeg*
EOT

FROM node:22-bullseye-slim AS node-22
FROM docker.io/ubuntu:22.04
RUN --mount=type=cache,target=/var/cache/apt,sharing=private,id=ubuntu2204-aptcache \
Expand Down Expand Up @@ -85,36 +131,8 @@ RUN mkdir -p /etc/chromium/policies/managed
COPY shared/chromium-policies/managed/policy.json /etc/chromium/policies/managed/policy.json

# Install FFmpeg (latest static build) for the recording server
RUN --mount=type=cache,target=/tmp/cache/ffmpeg,id=ffmpeg \
<<-'EOT'
set -eux
URL="https://johnvansickle.com/ffmpeg/releases/ffmpeg-release-amd64-static.tar.xz"
echo "Downloading FFmpeg MD5 checksum"
if ! curl --connect-timeout 10 -fsSL "${URL}.md5" -o /tmp/cache/ffmpeg/ffmpeg.tar.xz.md5; then
echo "Failed to connect to ffmpeg static build provider for MD5 checksum."
echo "Checking for cached version to use."
if [ ! -f /tmp/cache/ffmpeg/ffmpeg.tar.xz.md5 ]; then
echo "Unable to locate cached MD5 checksum. Exiting."
exit 1
else
echo "Found cached MD5 checksum."
fi
fi
sed -i -e 's/ .*$/ \/tmp\/cache\/ffmpeg\/ffmpeg.tar.xz/' /tmp/cache/ffmpeg/ffmpeg.tar.xz.md5
echo "Checking cache for FFmpeg archive and validating MD5 checksum"
if md5sum --check /tmp/cache/ffmpeg/ffmpeg.tar.xz.md5; then
echo "Checksum validated, using cached FFmpeg archive"
else
echo "Downloading FFmpeg static build from $URL"
curl -fsSL "$URL" -o /tmp/cache/ffmpeg/ffmpeg.tar.xz
echo "Validating MD5 checksum of FFmpeg static build download"
md5sum --check /tmp/cache/ffmpeg/ffmpeg.tar.xz.md5
fi
tar -xJf /tmp/cache/ffmpeg/ffmpeg.tar.xz -C /tmp
install -m755 /tmp/ffmpeg-*/ffmpeg /usr/local/bin/ffmpeg
install -m755 /tmp/ffmpeg-*/ffprobe /usr/local/bin/ffprobe
rm -rf /tmp/ffmpeg*
EOT
COPY --from=ffmpeg-downloader /usr/local/bin/ffmpeg /usr/local/bin/ffmpeg
COPY --from=ffmpeg-downloader /usr/local/bin/ffprobe /usr/local/bin/ffprobe

# Remove upower to prevent spurious D-Bus activations and logs
RUN --mount=type=cache,target=/var/cache/apt,sharing=private,id=ubuntu2204-aptcache \
Expand All @@ -133,7 +151,7 @@ RUN set -eux; \
fi

# Install TypeScript and Playwright globally
RUN npm install -g typescript playwright-core tsx
RUN --mount=type=cache,target=/root/.npm npm install -g typescript playwright-core tsx

ENV WITHDOCKER=true

Expand Down
Loading