Skip to content

dirent: no longer cache the results of Stat and Info#71

Draft
charlievieth wants to merge 1 commit intomasterfrom
cev/remove-fileinfo-caching
Draft

dirent: no longer cache the results of Stat and Info#71
charlievieth wants to merge 1 commit intomasterfrom
cev/remove-fileinfo-caching

Conversation

@charlievieth
Copy link
Owner

@charlievieth charlievieth commented Nov 19, 2025

This commit changes fastwalk.DirEntry to no longer cache the result of
calling the Stat and Info methods. This reduces memory usage by reducing
the size of the underlying DirEntry type and by saving an allocation in
the call to Stat/Info.

Benchmarks:

goos: darwin
goarch: arm64
cpu: Apple M4 Pro
                           │ base.10.txt │             new.10.txt             │
                           │   sec/op    │   sec/op     vs base               │
UnixDirentInfo/Serial-14     648.6n ± 2%   616.6n ± 1%  -4.92% (p=0.000 n=10)
UnixDirentInfo/Parallel-14   969.4n ± 0%   932.5n ± 0%  -3.81% (p=0.000 n=10)
geomean                      792.9n        758.3n       -4.37%

                           │ base.10.txt │             new.10.txt             │
                           │    B/op     │    B/op     vs base                │
UnixDirentInfo/Serial-14      384.0 ± 0%   336.0 ± 0%  -12.50% (p=0.000 n=10)
UnixDirentInfo/Parallel-14    384.0 ± 0%   336.0 ± 0%  -12.50% (p=0.000 n=10)
geomean                       384.0        336.0       -12.50%

                           │ base.10.txt │             new.10.txt             │
                           │  allocs/op  │ allocs/op   vs base                │
UnixDirentInfo/Serial-14      4.000 ± 0%   3.000 ± 0%  -25.00% (p=0.000 n=10)
UnixDirentInfo/Parallel-14    4.000 ± 0%   3.000 ± 0%  -25.00% (p=0.000 n=10)
geomean                       4.000        3.000       -25.00%
goos: linux
goarch: arm64
                          │ base.10.txt │             new.10.txt             │
                          │   sec/op    │   sec/op     vs base               │
UnixDirentInfo/Serial-4     425.3n ± 1%   394.3n ± 0%  -7.29% (p=0.000 n=10)
UnixDirentInfo/Parallel-4   248.0n ± 2%   229.8n ± 2%  -7.36% (p=0.000 n=10)
geomean                     324.8n        301.0n       -7.32%

                          │ base.10.txt │             new.10.txt             │
                          │    B/op     │    B/op     vs base                │
UnixDirentInfo/Serial-4      368.0 ± 0%   320.0 ± 0%  -13.04% (p=0.000 n=10)
UnixDirentInfo/Parallel-4    368.0 ± 0%   320.0 ± 0%  -13.04% (p=0.000 n=10)
geomean                      368.0        320.0       -13.04%

                          │ base.10.txt │             new.10.txt             │
                          │  allocs/op  │ allocs/op   vs base                │
UnixDirentInfo/Serial-4      4.000 ± 0%   3.000 ± 0%  -25.00% (p=0.000 n=10)
UnixDirentInfo/Parallel-4    4.000 ± 0%   3.000 ± 0%  -25.00% (p=0.000 n=10)
geomean                      4.000        3.000       -25.00%
goos: linux
goarch: amd64
cpu: AMD EPYC-Milan Processor
                          │ base.10.txt │             new.10.txt             │
                          │   sec/op    │   sec/op     vs base               │
UnixDirentInfo/Serial-4     2.235µ ± 1%   2.162µ ± 3%  -3.24% (p=0.001 n=10)
UnixDirentInfo/Parallel-4   732.7n ± 2%   695.9n ± 2%  -5.03% (p=0.000 n=10)
geomean                     1.280µ        1.227µ       -4.14%

                          │ base.10.txt │             new.10.txt             │
                          │    B/op     │    B/op     vs base                │
UnixDirentInfo/Serial-4      384.0 ± 0%   336.0 ± 0%  -12.50% (p=0.000 n=10)
UnixDirentInfo/Parallel-4    384.0 ± 0%   336.0 ± 0%  -12.50% (p=0.000 n=10)
geomean                      384.0        336.0       -12.50%

                          │ base.10.txt │             new.10.txt             │
                          │  allocs/op  │ allocs/op   vs base                │
UnixDirentInfo/Serial-4      4.000 ± 0%   3.000 ± 0%  -25.00% (p=0.000 n=10)
UnixDirentInfo/Parallel-4    4.000 ± 0%   3.000 ± 0%  -25.00% (p=0.000 n=10)
geomean                      4.000        3.000       -25.00%

@codecov-commenter
Copy link

codecov-commenter commented Nov 19, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 57.30%. Comparing base (f8029ee) to head (6e67aac).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master      #71      +/-   ##
==========================================
- Coverage   59.24%   57.30%   -1.94%     
==========================================
  Files          13       13              
  Lines         633      602      -31     
==========================================
- Hits          375      345      -30     
+ Misses        244      243       -1     
  Partials       14       14              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@charlievieth charlievieth force-pushed the cev/remove-fileinfo-caching branch 2 times, most recently from 99f5440 to 4b11fec Compare November 19, 2025 18:41
@charlievieth charlievieth marked this pull request as draft November 19, 2025 19:02
@charlievieth charlievieth force-pushed the cev/remove-fileinfo-caching branch from 03c16d3 to 74b9ebf Compare November 19, 2025 20:59
This commit changes fastwalk.DirEntry to no longer cache the result of
calling the Stat and Info methods.

goos: darwin
goarch: arm64
pkg: github.com/charlievieth/fastwalk
cpu: Apple M4 Pro
                           │ base.10.txt │             new.10.txt             │
                           │   sec/op    │   sec/op     vs base               │
UnixDirentInfo/Serial-14     648.6n ± 2%   616.6n ± 1%  -4.92% (p=0.000 n=10)
UnixDirentInfo/Parallel-14   969.4n ± 0%   932.5n ± 0%  -3.81% (p=0.000 n=10)
geomean                      792.9n        758.3n       -4.37%

                           │ base.10.txt │             new.10.txt             │
                           │    B/op     │    B/op     vs base                │
UnixDirentInfo/Serial-14      384.0 ± 0%   336.0 ± 0%  -12.50% (p=0.000 n=10)
UnixDirentInfo/Parallel-14    384.0 ± 0%   336.0 ± 0%  -12.50% (p=0.000 n=10)
geomean                       384.0        336.0       -12.50%

                           │ base.10.txt │             new.10.txt             │
                           │  allocs/op  │ allocs/op   vs base                │
UnixDirentInfo/Serial-14      4.000 ± 0%   3.000 ± 0%  -25.00% (p=0.000 n=10)
UnixDirentInfo/Parallel-14    4.000 ± 0%   3.000 ± 0%  -25.00% (p=0.000 n=10)
geomean                       4.000        3.000       -25.00%

goos: linux
goarch: amd64
pkg: github.com/charlievieth/fastwalk
cpu: AMD EPYC-Milan Processor
                          │ base.10.txt │             new.10.txt             │
                          │   sec/op    │   sec/op     vs base               │
UnixDirentInfo/Serial-4     2.235µ ± 1%   2.162µ ± 3%  -3.24% (p=0.001 n=10)
UnixDirentInfo/Parallel-4   732.7n ± 2%   695.9n ± 2%  -5.03% (p=0.000 n=10)
geomean                     1.280µ        1.227µ       -4.14%

                          │ base.10.txt │             new.10.txt             │
                          │    B/op     │    B/op     vs base                │
UnixDirentInfo/Serial-4      384.0 ± 0%   336.0 ± 0%  -12.50% (p=0.000 n=10)
UnixDirentInfo/Parallel-4    384.0 ± 0%   336.0 ± 0%  -12.50% (p=0.000 n=10)
geomean                      384.0        336.0       -12.50%

                          │ base.10.txt │             new.10.txt             │
                          │  allocs/op  │ allocs/op   vs base                │
UnixDirentInfo/Serial-4      4.000 ± 0%   3.000 ± 0%  -25.00% (p=0.000 n=10)
UnixDirentInfo/Parallel-4    4.000 ± 0%   3.000 ± 0%  -25.00% (p=0.000 n=10)
geomean                      4.000        3.000       -25.00%
@charlievieth charlievieth force-pushed the cev/remove-fileinfo-caching branch from 74b9ebf to 6e67aac Compare November 19, 2025 21:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants