Lack of Documentation Details on Signed Deltas

[parkingmeter]

Filing this as a documentation problem rather than a bug — as far as I can tell every behaviour below is intentional and long-standing. What's missing is that none of it is written down anywhere a user would look, and one case fails in a way that gives no hint of the cause.

Measured on 2026.4; all line references are against 1d5a312.

1. static-delta generate --sign-type=… produces a delta pull cannot consume

generate exits 0, static-delta show reports Signed: yes with a correct From/To, the client fetches the superblock 200 OK, and then the pull dies with:

error: Invalid checksum of length 0 expected 32

A signed delta's superblock is a different variant, OSTREE_STATIC_DELTA_SIGNED_FORMAT = (taya{sv}), magic OSTSGNDT an envelope wrapping the real superblock. Only ostree_repo_static_delta_execute_offline_with_signature() detects that magic and unwraps it. The pull path parses the fetched bytes directly as OSTREE_STATIC_DELTA_SUPERBLOCK_FORMAT with no envelope detection, so child 3 (the to checksum) parses as empty and ostree_validate_structureof_csum_v throws the above. OSTSGNDT does not appear in ostree-repo-pull.c at all.

So signed static deltas are a feature of apply-offline only. That looks deliberate, #1985 says as much in review ("Currently, we only use it for 'offline' updates"), with online updates resting on the signed summary instead. My problem is only that ostree-static-delta(1) documents --sign-type in three stanzas and none of them mentions the restriction, so the first indication is a checksum error against a real client.

2. core.sign-verify-deltas cannot require a delta to be signed

It's read in exactly one place, inside if (is_signed) on the offline path, and appears zero times in ostree-repo-pull.c. Two consequences:

• On a network pull it is inert. (Given §1 that's consistent, a pulled delta can never be signed, but the option gives no hint that it's scoped to offline use.)
• Offline it makes a key mandatory when a delta happens to be signed; it cannot make a signature mandatory. A client with it set applies an unsigned delta without complaint.

The API docstring is accurate about this ("If sign-verify-deltas configuration option is set and static delta is signed"), so this isn't news to the project — but the name reads like a requirement, and the docstring isn't where someone configuring a client is looking. Arms 3 and 4 below show it refusing correctly when a delta is signed and accepting when it isn't; the option does exactly what it's coded to do.

3. None of the ed25519 verification options appear in the man pages

$ grep -rn "sign-verify" man/ docs/
$

ostree.repo-config(5) documents gpg-verify and gpg-verify-summary, but not sign-verify, sign-verify-summary, or core.sign-verify-deltas; the three spellings in ot-remote-builtin-add.c, ostree-repo-pull-verify.c and ostree-repo-static-delta-core.c. A reader configuring ed25519 verification from the man page finds only the GPG pair. It cost me a client that couldn't pull at all before I found the right names in the source, and it's easy to conclude from the docs that summary verification simply isn't available for ed25519.

Reproducer

ostree, openssl, python3, curl. No root, confined to a temp dir. Arms 1 and 3 are controls.

#!/bin/bash
set -eu
WORK=$(mktemp -d); PORT=${PORT:-8731}; SRV=""
cleanup() { [ -n "$SRV" ] && kill "$SRV" 2>/dev/null; rm -rf "$WORK"; }
trap cleanup EXIT
cd "$WORK"
ostree --version | sed -n '1,3p'

# ed25519 keypair in ostree's format: base64(seed || public)
openssl genpkey -algorithm ed25519 -out k.pem 2>/dev/null
SECRET=$(printf '%s%s' \
  "$(openssl pkey -in k.pem -outform DER | tail -c 32 | xxd -p -c256)" \
  "$(openssl pkey -in k.pem -pubout -outform DER | tail -c 32 | xxd -p -c256)" \
  | xxd -r -p | base64 -w0)

ostree --repo=origin init --mode=archive >/dev/null
mkdir -p tree/usr/bin; printf 'one\n' > tree/usr/bin/hello
FROM=$(ostree --repo=origin commit -b demo --tree=dir=tree)
printf 'two\n' > tree/usr/bin/hello
head -c 200000 /dev/urandom > tree/usr/bin/blob
TO=$(ostree --repo=origin commit -b demo --tree=dir=tree)

python3 -m http.server "$PORT" --directory origin >/dev/null 2>&1 & SRV=$!
until curl -sf "http://127.0.0.1:$PORT/config" >/dev/null; do sleep 0.2; done

gen() { rm -rf origin/deltas origin/delta-indexes
        ostree --repo=origin static-delta generate --from="$FROM" --to="$TO" "$@" >/dev/null 2>&1
        echo "  generate rc=$?"
        ostree --repo=origin summary -u; }
deltadir() { dirname "$(find origin/deltas -name superblock)"; }
pull_arm() { rm -rf "$1"
        ostree --repo="$1" init --mode=bare-user >/dev/null
        ostree --repo="$1" remote add --no-gpg-verify o "http://127.0.0.1:$PORT" >/dev/null
        ostree --repo="$1" pull o "$FROM" >/dev/null 2>&1
        set +e; ostree --repo="$1" pull o demo 2>&1 | tail -2
        echo "  pull rc=${PIPESTATUS[0]}"; set -e; }
offline_arm() { rm -rf "$1"
        ostree --repo="$1" init --mode=bare-user >/dev/null
        ostree --repo="$1" config set core.sign-verify-deltas true
        ostree --repo="$1" pull-local origin "$FROM" >/dev/null 2>&1
        set +e; ostree --repo="$1" static-delta apply-offline "$(deltadir)" 2>&1 | tail -2
        echo "  apply rc=${PIPESTATUS[0]}"; set -e
        printf '  to-commit landed: '
        ostree --repo="$1" cat "$TO" /usr/bin/hello 2>/dev/null || echo '(absent)'; }

echo; echo "### 1. pull, delta generated UNSIGNED  (control: deltas do work)"
gen; pull_arm c1

echo; echo "### 2. pull, same delta generated SIGNED"
gen --sign-type=ed25519 --sign="$SECRET"
ostree --repo=origin static-delta show "$(ostree --repo=origin static-delta list | tail -1)" \
  2>/dev/null | grep -i '^Signed:' | sed 's/^/  show says: /'
pull_arm c2

echo; echo "### 3. apply-offline, SIGNED delta, core.sign-verify-deltas=true, no key"
echo "     (control: the option is not inert offline)"
offline_arm c3

echo; echo "### 4. apply-offline, UNSIGNED delta, core.sign-verify-deltas=true"
gen; offline_arm c4

Output here:

libostree:
 Version: '2026.4'
 Git: v2026.4

### 1. pull, delta generated UNSIGNED  (control: deltas do work)
  generate rc=0
1 delta parts, 2 loose fetched; 196 KiB transferred in 0 seconds; 0 bytes content written
  pull rc=0

### 2. pull, same delta generated SIGNED
  generate rc=0
  show says: Signed: yes
error: Invalid checksum of length 0 expected 32
  pull rc=1

### 3. apply-offline, SIGNED delta, core.sign-verify-deltas=true, no key
     (control: the option is not inert offline)
error: Key is mandatory to check delta signature
  apply rc=1
  to-commit landed: (absent)

### 4. apply-offline, UNSIGNED delta, core.sign-verify-deltas=true
  generate rc=0
  apply rc=0
  to-commit landed: two

Note that §1 only reproduces against a non-archive client, archive-mode repos skip static deltas entirely, so it survives host-side testing and first appears against a real client.

Why this survived

tests/test-delta-sign.sh is thorough about apply-offline and never exercises a signed delta over pull, its only pull is a pull-local used to seed the client repo. So there's no coverage gap in the sense of a broken test, just a path the suite was never pointed at.

What I'd suggest

I'll open a PR against man/ostree-static-delta.xml and man/ostree.repo-config.xml covering the three points above; documentation only, no behaviour change. Please reshape the wording however you like, or tell me to drop it if you'd rather write it yourselves.

Two things I deliberately haven't touched, because they're yours to decide:

• pull could detect the OSTSGNDT magic and say "this static delta is signed; signed deltas can only be applied with static-delta apply-offline" instead of Invalid checksum of length 0 expected 32. That's ~10 lines and would have saved me the entire investigation. Happy to write it if you want it.
• static-delta show could print Signed: yes (offline apply only).

I'm not asking for signed-delta support on the pull path. #1985 shows that was a considered scope decision, and the project I hit this in has designed around it, we ship deltas unsigned and rest the trust root on the commit signature plus content-addressing, which we're satisfied is sound. This is only about the documentation and the error message.