Set compaction thread to idle I/O priority on Linux (ionice)

[vicnaum]

Context

During fast-sync on slow storage (external HDD), shard compaction saturates disk I/O and starves WAL writes, causing the sync pipeline to stall. See speed wave pattern analysis below.

Proposal

Set the compaction thread to idle I/O scheduling priority on Linux using the ioprio_set syscall, so WAL writes (which feed the pipeline) are prioritized over background compaction.

Implementation

In sync/historical/db_writer.rs, inside the compaction spawn at line ~104, add before storage.compact_shard():

#[cfg(target_os = "linux")]
unsafe {
    // IOPRIO_CLASS_IDLE=3, shift 13 bits, prio=7
    libc::syscall(libc::SYS_ioprio_set, 1 /*IOPRIO_WHO_PROCESS*/, 0 /*self*/, (3 << 13) | 7);
}

~5 lines of code, #[cfg(target_os = "linux")] guarded.

Caveats

• Only effective with CFQ or BFQ I/O schedulers. On mq-deadline or none (common defaults on modern kernels), ioprio_set is a no-op.
• macOS: No equivalent API exists. setiopolicy_np is private/undocumented.
• Risk of compaction starvation if WAL writes are continuous — compaction would never get disk time. May need a minimum I/O class (best-effort low priority) instead of idle.
• This is a partial mitigation. The --defer-compaction flag (#TBD) is a more complete solution for HDD users.

Evidence from logs

• IOWait averages 44% during sync, spikes to 95% during compaction
• DB flush (WAL write) duration goes from 1s to 37s for the same 34MB when competing with compaction
• Compaction durations grow from 3s (early shards) to 121s (later shards with more tx data)

Labels

enhancement, performance