Skip to content

Conversation

@AnuthaDev
Copy link
Contributor

@AnuthaDev AnuthaDev commented Jan 6, 2026

Replaced the vector allocation in filter_flags with direct iterator usage for reduced memory overhead.

Added new tests to cover edge cases encountered during previous implementation attempts.

Memory Usage Analysis:

Using /usr/bin/time (Linux)

  1. Create a test input file: printf "00 %.0s" {1..100000} > args.txt
  2. Run the binary:
 /usr/bin/time -v ./target/release/coreutils echo $(cat args.txt)

Results:

  1. main: Maximum resident set size (kbytes): 19364

  2. This PR: Maximum resident set size (kbytes): 12280

  3. System echo: Maximum resident set size (kbytes): 2544

Using dhat-rs:

main:

./target/release/coreutils echo $(cat args.txt) > /dev/null

dhat: Total:     10,065,406 bytes in 200,206 blocks
dhat: At t-gmax: 7,647,027 bytes in 200,135 blocks
dhat: At t-end:  2,648,051 bytes in 100,134 blocks

This PR:

./target/release/coreutils echo $(cat args.txt) > /dev/null

dhat: Total:     5,264,197 bytes in 200,188 blocks
dhat: At t-gmax: 5,000,140 bytes in 100,005 blocks
dhat: At t-end:  2,648,047 bytes in 100,134 blocks

@ChrisDryden
Copy link
Collaborator

This is great work, I'd love to see if we had a way to track these changes over time. What was the main method you used for determining the memory usage, did you set up a benchmark?

@github-actions
Copy link

github-actions bot commented Jan 6, 2026

GNU testsuite comparison:

Congrats! The gnu test tests/shuf/shuf-reservoir is no longer failing!
Congrats! The gnu test tests/sort/sort-stale-thread-mem is no longer failing!

2 similar comments
@github-actions
Copy link

github-actions bot commented Jan 6, 2026

GNU testsuite comparison:

Congrats! The gnu test tests/shuf/shuf-reservoir is no longer failing!
Congrats! The gnu test tests/sort/sort-stale-thread-mem is no longer failing!

@github-actions
Copy link

github-actions bot commented Jan 6, 2026

GNU testsuite comparison:

Congrats! The gnu test tests/shuf/shuf-reservoir is no longer failing!
Congrats! The gnu test tests/sort/sort-stale-thread-mem is no longer failing!

@codspeed-hq
Copy link

codspeed-hq bot commented Jan 6, 2026

CodSpeed Performance Report

Merging this PR will not alter performance

Comparing AnuthaDev:echo-alloc (db22117) with main (b804ca4)

Summary

✅ 140 untouched benchmarks
⏩ 37 skipped benchmarks1

Footnotes

  1. 37 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@AnuthaDev
Copy link
Contributor Author

Hey @ChrisDryden , for benchmarking I used /usr/bin/time in the beginning and later shifted to dhat.

I believe the dhat crate can be used in a CI pipeline to keep track of memory allocation changes over time.

.stdout_matches(&Regex::new(r"^echo \(uutils coreutils\) (\d+\.\d+\.\d+)\n$").unwrap());
}

#[test]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems unrelated to this pr ?!

Copy link
Contributor Author

@AnuthaDev AnuthaDev Jan 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR removes the length check from args in case of --help and --version and instead uses other logic to infer the length

These test cases serve as additional confirmation that the changes did not regress any behaviour.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it makes our life harder as reviewers
and given your current stack of patches:
image
we will have to squash it and it won't be only for memory usage anymore

so, yeah, splitting it or writing a nice stack of patches would be appreciated

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Improved the patchset. Please let me know if any further improvements are required.

}

#[test]
fn multiple_help_argument() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same, unrelated

@github-actions
Copy link

github-actions bot commented Jan 7, 2026

GNU testsuite comparison:

Skipping an intermittent issue tests/timeout/timeout (passes in this run but fails in the 'main' branch)

@AnuthaDev
Copy link
Contributor Author

The CI failure logs seem to indicate its unrelated to the PR changes? It was working with the same code earlier...

@sylvestre sylvestre merged commit 478daa1 into uutils:main Jan 8, 2026
131 of 132 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants