Skip to content

Commit 5dff10a

Browse files
committed
OCPBUGS-62790: Resize /var tmpfs to 10GiB for ABI installations
Agent-based installations on vSphere with 16GB RAM were failing with "no space left on device" errors during ostree image operations. The live ISO environment uses a loopback filesystem mounted at /var that is backed by a tmpfs sized at 50% of available RAM. On systems with 16GB RAM, this provides only 8GB of tmpfs space. During the bootstrap process, node-image-pull.sh creates a temporary ostree repository in /var/ostree-container/repo to pull and apply the node image. This operation has a peak tmpfs usage of approximately 8.5-9GB, exceeding the available 8GB and causing ENOSPC errors. This fix resizes the /var tmpfs, the loopback backing file in it, and the filesystem on it to 10GiB before creating the temporary ostree repository, if the tmpfs was smaller than that. This provides sufficient space for the image operations while maintaining compatibility with the minimum 16GB RAM requirement. The resize is performed using systemd-run to escape the mount namespace of the node-image-pull service.
1 parent 54f4b77 commit 5dff10a

File tree

2 files changed

+38
-0
lines changed

2 files changed

+38
-0
lines changed

data/data/bootstrap/files/usr/local/bin/node-image-pull.sh.template

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,11 @@ if test -f /run/ostree-live; then
3434
mkdir -p "${ostree_repo}"
3535
echo "In live environment; creating temporary repo to pull node image"
3636
ostree init --mode=bare --repo="${ostree_repo}"
37+
38+
echo "Resizing /run/ephemeral_base tmpfs to 10GiB for ostree operations..."
39+
# Use systemd-run to avoid inheriting MountFlags
40+
systemd-run --wait --service-type=oneshot /usr/local/bin/resize-ephemeral 10G
41+
3742
# if there are layers, import all the content in the system repo for
3843
# layer-level deduping
3944
if [ -d /ostree/repo/refs/heads/ostree/container ]; then
Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
#!/bin/bash
2+
3+
set -eu
4+
5+
TARGET_SIZE="$1"
6+
BASE_DIR="/run/ephemeral_base"
7+
LOOPBACK_FILE="${BASE_DIR}/loopfs"
8+
9+
function get_size_bytes() {
10+
stat -f -c "%b * %s" "${BASE_DIR}"
11+
}
12+
13+
TARGET_SIZE_BYTES="$(numfmt --from=iec "${TARGET_SIZE}")"
14+
CURRENT_SIZE_BYTES="$(get_size_bytes)"
15+
CURRENT_SIZE="$(numfmt --to=iec "${CURRENT_SIZE_BYTES}")"
16+
17+
if ((TARGET_SIZE_BYTES > CURRENT_SIZE_BYTES)); then
18+
echo "Expanding ephemeral base dir from ${CURRENT_SIZE} to ${TARGET_SIZE}"
19+
mount -o remount,size="${TARGET_SIZE}" "${BASE_DIR}"
20+
21+
echo "Expanding ephemeral loopback"
22+
truncate -s "$(get_size_bytes)" "${LOOPBACK_FILE}"
23+
24+
LOOPBACK_DEVICE="$(losetup -j "${LOOPBACK_FILE}" -O NAME -n)"
25+
losetup -c "${LOOPBACK_DEVICE}"
26+
27+
echo "Expanding ephemeral filesystem"
28+
xfs_growfs -d "${LOOPBACK_DEVICE}"
29+
else
30+
echo "Ephemeral base dir size ${CURRENT_SIZE} is already larger than ${TARGET_SIZE}; not expanding"
31+
fi
32+
33+
df -h

0 commit comments

Comments
 (0)