Skip to content

Remove unnecessary lseek syscall when using std::fs::read #106664

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jan 11, 2023
Merged

Remove unnecessary lseek syscall when using std::fs::read #106664

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
eae615d
Remove unnecessary lseek syscall when using std::fs::read
chenyukang Jan 7, 2023
f7bc68b
use with_capacity in read read_to_string
chenyukang Jan 10, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions library/std/src/fs.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -250,7 +250,9 @@ pub fn read<P: AsRef<Path>>(path: P) -> io::Result<Vec<u8>> {
fn inner(path: &Path) -> io::Result<Vec<u8>> {
let mut file = File::open(path)?;
let mut bytes = Vec::new();
file.read_to_end(&mut bytes)?;
let size = file.metadata().map(|m| m.len()).unwrap_or(0);
bytes.reserve(size as usize);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can do let mut bytes = Vec::with_capacity(size as usize) instead of two lines reserve and Vec::new()

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I thought on this, but I have a dig on the implementation of with_capacity and reserve, seems it's not simply share the same code path, so I'm not 100% sure is there any flaw to use with_capacity here.

I just have a simple bench test, seems with_capacity perform better:

cat@LAPTOP-V6U0QKD4:~/code/play/bench$ cargo bench
    Finished bench [optimized] target(s) in 0.00s
     Running unittests src/main.rs (target/release/deps/bench-b73bcfb176559323)

running 2 tests
test tests::bench_reserve_capacity ... bench:     677,683 ns/iter (+/- 828,000)
test tests::bench_with_capacity    ... bench:     469,924 ns/iter (+/- 319,682)

test result: ok. 0 passed; 0 failed; 0 ignored; 2 measured; 0 filtered out; finished in 10.56s

cat@LAPTOP-V6U0QKD4:~/code/play/bench$ cargo bench
    Finished bench [optimized] target(s) in 0.01s
     Running unittests src/main.rs (target/release/deps/bench-b73bcfb176559323)

running 2 tests
test tests::bench_reserve_capacity ... bench:     645,487 ns/iter (+/- 347,440)
test tests::bench_with_capacity    ... bench:     415,772 ns/iter (+/- 162,321)

test result: ok. 0 passed; 0 failed; 0 ignored; 2 measured; 0 filtered out; finished in 10.09s

with bench code:

#![feature(test)]
extern crate test;

pub fn test_with_capacity(size: usize) {
    let mut v: Vec<i32> = Vec::with_capacity(size);
    v.push(3);
    assert_eq!(v.len(), 1);
}

pub fn test_reserve_capacity(size: usize) {
    let mut v: Vec<i32> = Vec::new();
    v.reserve(size);
    v.push(3);
    assert_eq!(v.len(), 1);
}

#[cfg(test)]
mod tests {
    use super::*;
    use test::Bencher;

    #[bench]
    fn bench_with_capacity(b: &mut Bencher) {
        b.iter(|| {
            for s in 0..10000 {
                test_with_capacity(s);
            }
        });
    }

    #[bench]
    fn bench_reserve_capacity(b: &mut Bencher) {
        b.iter(|| {
            for s in 0..10000 {
                test_reserve_capacity(s);
            }
        });
    }
}

io::default_read_to_end(&mut file, &mut bytes)?;
Ok(bytes)
}
inner(path.as_ref())
Expand Down Expand Up @@ -289,7 +291,9 @@ pub fn read_to_string<P: AsRef<Path>>(path: P) -> io::Result<String> {
fn inner(path: &Path) -> io::Result<String> {
let mut file = File::open(path)?;
let mut string = String::new();
file.read_to_string(&mut string)?;
let size = file.metadata().map(|m| m.len()).unwrap_or(0);
string.reserve(size as usize);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here.

io::default_read_to_string(&mut file, &mut string)?;
Ok(string)
}
inner(path.as_ref())
Expand Down