Skip to content

dd incorrectly counts partial reads toward count limit, causing data loss in pipelines #9119

@kpwtxt

Description

@kpwtxt

Summary

uutils dd has a critical bug where partial reads from pipes are counted toward the count=N limit, causing premature termination and significant data loss. GNU dd correctly only counts full blocks. This bug was discovered while debugging a failure of NVIDIA's cuda toolkit installer (e.g cuda_12.9.1_575.57.08_linux.run), which relies on dd while running on Ubuntu 25.10.

Environment

  • uutils version: 0.2.2 (rust-coreutils 0.2.2-0ubuntu2.1)
  • OS: Ubuntu 25.10 (Linux 6.17.0-6-generic)
  • Comparison: GNU coreutils 9.5

Bug Description

When dd reads from a pipe with ibs=N count=M, it should read until it receives M full blocks of size N. Partial reads (which are common when reading from pipes due to buffering) should not count toward the limit.

Current behavior (uutils dd): Counts every read operation (including partial reads) toward the count limit, stopping prematurely and losing data.

Expected behavior (GNU dd): Only counts full blocks toward the count limit, continuing to read until the requested number of full blocks is received.

Impact

This causes data loss in pipelines that use count parameter:

  • In testing with a 5.8GB file, uutils dd lost ~2GB of data (~33% data loss)
  • Breaks compatibility with scripts designed for GNU coreutils
  • Breaks real-world software that uses Makeself archives (e.g., NVIDIA CUDA installer)

Steps to Reproduce

Thus far I can only reproduce the bug using the data block in the NVIDIA installer. This test script demonstrates the correct GNU dd behavior and reproduces the error using uu dd, with these results:

=== uutils dd bug reproduction test ===
File: cuda_12.9.1_575.57.08_linux.run (5.8GB)

Step 1: Testing with uutils dd...
Command: dd ibs=14808 skip=1 obs=1024 conv=sync | dd ibs=1024 count=5722911
uutils results:
  1833991+3888920 records in
  Total bytes: 3883225862

Step 2: Testing with GNU dd (both stages)...
GNU dd results:
  5722911+0 records in
  Total bytes: 5860260864

Expected output: 5860260864 bytes (5722911 blocks × 1024 bytes)

=== RESULTS ===
✗ FAIL: Data loss detected!
  Expected: 5860260864 bytes
  GNU dd:   5860260864 bytes
  uutils:   3883225862 bytes
  Lost:     1977035002 bytes (33% data loss)

Real-world reproduction (NVIDIA CUDA installer):

# Download CUDA installer
wget https://developer.download.nvidia.com/compute/cuda/12.9.1/local_installers/cuda_12.9.1_575.57.08_linux.run

# Attempt to run (will fail with uutils dd)
bash cuda_12.9.1_575.57.08_linux.run --extract=/tmp/cuda_test

# Error: "gzip: stdin: unexpected end of file"

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions