-
Notifications
You must be signed in to change notification settings - Fork 13.9k
Use fstatat() in DirEntry::metadata on Apple platforms #147986
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
rustbot has assigned @Mark-Simulacrum. Use |
This comment has been minimized.
This comment has been minimized.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good to me. fstatat is in all of the various SDK's public headers and its more darwin functionality then anything platform specific. And yeah available before Rust's current minimum Apple OS versions.
|
Cool! |
…=joboet Use fstatat() in DirEntry::metadata on Apple platforms Apple supports `fstatat` on macOS >=10.10 ([source](https://gitlab.gnome.org/GNOME/glib/-/issues/2203)), and according to [Platform Support](https://doc.rust-lang.org/beta/rustc/platform-support.html) the oldest supported version is 10.12. Google says iOS >=10 supports `fstatat` but doesn't provide a source. [*-apple-ios](https://doc.rust-lang.org/beta/rustc/platform-support/apple-ios.html#os-version) says the minimum supported iOS version is 10.0. Unsure about tvOS, watchOS and visionOS, hoping CI can confirm this. I am testing with [fastdu](https://github.com/jesseschalken/fastdu) which is effectively a stress test for `DirEntry::metadata`. In one test this provides a **1.13x** speedup. ``` $ hyperfine --warmup 1 'target/release/fastdu testdir' 'fastdu testdir' Benchmark 1: target/release/fastdu testdir Time (mean ± σ): 154.6 ms ± 17.4 ms [User: 31.7 ms, System: 187.6 ms] Range (min … max): 148.4 ms … 225.5 ms 19 runs Benchmark 2: fastdu testdir Time (mean ± σ): 175.3 ms ± 15.8 ms [User: 50.0 ms, System: 196.2 ms] Range (min … max): 165.4 ms … 211.7 ms 17 runs Summary target/release/fastdu testdir ran 1.13 ± 0.16 times faster than fastdu testdir ``` You can also reproduce a speedup with a program like this (providing a directory with many entries): ```rust fn main() { let args: Vec<_> = std::env::args_os().collect(); let dir: PathBuf = args[1].clone().into(); for entry in dir.read_dir().as_mut().unwrap() { let entry = entry.as_ref().unwrap(); let metadata = entry.metadata(); let metadata = metadata.as_ref().unwrap(); println!("{} {}", metadata.len(), entry.file_name().display()); } } ``` ``` $ hyperfine './target/release/main testdir' './main testdir' Benchmark 1: ./target/release/main testdir Time (mean ± σ): 148.3 ms ± 5.2 ms [User: 23.1 ms, System: 122.9 ms] Range (min … max): 145.2 ms … 167.2 ms 19 runs Benchmark 2: ./main testdir Time (mean ± σ): 164.4 ms ± 9.5 ms [User: 32.6 ms, System: 128.8 ms] Range (min … max): 158.5 ms … 199.5 ms 17 runs Summary ./target/release/main testdir ran 1.11 ± 0.07 times faster than ./main testdir ```
Rollup of 9 pull requests Successful merges: - #139310 (add first HelenOS compilation targets) - #147161 (implement VecDeque extend_from_within and prepend_from_within) - #147622 (`unicode_data` refactors) - #147780 (Implement VecDeque::extract_if) - #147942 (Enable regression labeling aliases) - #147986 (Use fstatat() in DirEntry::metadata on Apple platforms) - #148103 (cg_llvm: Pass `debuginfo_compression` through FFI as an enum) - #148319 (docs: Fix argument names for `carrying_mul_add`) - #148322 (Enable file locking support in illumos) r? `@ghost` `@rustbot` modify labels: rollup
…=joboet Use fstatat() in DirEntry::metadata on Apple platforms Apple supports `fstatat` on macOS >=10.10 ([source](https://gitlab.gnome.org/GNOME/glib/-/issues/2203)), and according to [Platform Support](https://doc.rust-lang.org/beta/rustc/platform-support.html) the oldest supported version is 10.12. Google says iOS >=10 supports `fstatat` but doesn't provide a source. [*-apple-ios](https://doc.rust-lang.org/beta/rustc/platform-support/apple-ios.html#os-version) says the minimum supported iOS version is 10.0. Unsure about tvOS, watchOS and visionOS, hoping CI can confirm this. I am testing with [fastdu](https://github.com/jesseschalken/fastdu) which is effectively a stress test for `DirEntry::metadata`. In one test this provides a **1.13x** speedup. ``` $ hyperfine --warmup 1 'target/release/fastdu testdir' 'fastdu testdir' Benchmark 1: target/release/fastdu testdir Time (mean ± σ): 154.6 ms ± 17.4 ms [User: 31.7 ms, System: 187.6 ms] Range (min … max): 148.4 ms … 225.5 ms 19 runs Benchmark 2: fastdu testdir Time (mean ± σ): 175.3 ms ± 15.8 ms [User: 50.0 ms, System: 196.2 ms] Range (min … max): 165.4 ms … 211.7 ms 17 runs Summary target/release/fastdu testdir ran 1.13 ± 0.16 times faster than fastdu testdir ``` You can also reproduce a speedup with a program like this (providing a directory with many entries): ```rust fn main() { let args: Vec<_> = std::env::args_os().collect(); let dir: PathBuf = args[1].clone().into(); for entry in dir.read_dir().as_mut().unwrap() { let entry = entry.as_ref().unwrap(); let metadata = entry.metadata(); let metadata = metadata.as_ref().unwrap(); println!("{} {}", metadata.len(), entry.file_name().display()); } } ``` ``` $ hyperfine './target/release/main testdir' './main testdir' Benchmark 1: ./target/release/main testdir Time (mean ± σ): 148.3 ms ± 5.2 ms [User: 23.1 ms, System: 122.9 ms] Range (min … max): 145.2 ms … 167.2 ms 19 runs Benchmark 2: ./main testdir Time (mean ± σ): 164.4 ms ± 9.5 ms [User: 32.6 ms, System: 128.8 ms] Range (min … max): 158.5 ms … 199.5 ms 17 runs Summary ./target/release/main testdir ran 1.11 ± 0.07 times faster than ./main testdir ```
…=joboet Use fstatat() in DirEntry::metadata on Apple platforms Apple supports `fstatat` on macOS >=10.10 ([source](https://gitlab.gnome.org/GNOME/glib/-/issues/2203)), and according to [Platform Support](https://doc.rust-lang.org/beta/rustc/platform-support.html) the oldest supported version is 10.12. Google says iOS >=10 supports `fstatat` but doesn't provide a source. [*-apple-ios](https://doc.rust-lang.org/beta/rustc/platform-support/apple-ios.html#os-version) says the minimum supported iOS version is 10.0. Unsure about tvOS, watchOS and visionOS, hoping CI can confirm this. I am testing with [fastdu](https://github.com/jesseschalken/fastdu) which is effectively a stress test for `DirEntry::metadata`. In one test this provides a **1.13x** speedup. ``` $ hyperfine --warmup 1 'target/release/fastdu testdir' 'fastdu testdir' Benchmark 1: target/release/fastdu testdir Time (mean ± σ): 154.6 ms ± 17.4 ms [User: 31.7 ms, System: 187.6 ms] Range (min … max): 148.4 ms … 225.5 ms 19 runs Benchmark 2: fastdu testdir Time (mean ± σ): 175.3 ms ± 15.8 ms [User: 50.0 ms, System: 196.2 ms] Range (min … max): 165.4 ms … 211.7 ms 17 runs Summary target/release/fastdu testdir ran 1.13 ± 0.16 times faster than fastdu testdir ``` You can also reproduce a speedup with a program like this (providing a directory with many entries): ```rust fn main() { let args: Vec<_> = std::env::args_os().collect(); let dir: PathBuf = args[1].clone().into(); for entry in dir.read_dir().as_mut().unwrap() { let entry = entry.as_ref().unwrap(); let metadata = entry.metadata(); let metadata = metadata.as_ref().unwrap(); println!("{} {}", metadata.len(), entry.file_name().display()); } } ``` ``` $ hyperfine './target/release/main testdir' './main testdir' Benchmark 1: ./target/release/main testdir Time (mean ± σ): 148.3 ms ± 5.2 ms [User: 23.1 ms, System: 122.9 ms] Range (min … max): 145.2 ms … 167.2 ms 19 runs Benchmark 2: ./main testdir Time (mean ± σ): 164.4 ms ± 9.5 ms [User: 32.6 ms, System: 128.8 ms] Range (min … max): 158.5 ms … 199.5 ms 17 runs Summary ./target/release/main testdir ran 1.11 ± 0.07 times faster than ./main testdir ```
Rollup of 9 pull requests Successful merges: - #139310 (add first HelenOS compilation targets) - #147161 (implement VecDeque extend_from_within and prepend_from_within) - #147622 (`unicode_data` refactors) - #147780 (Implement VecDeque::extract_if) - #147942 (Enable regression labeling aliases) - #147986 (Use fstatat() in DirEntry::metadata on Apple platforms) - #148103 (cg_llvm: Pass `debuginfo_compression` through FFI as an enum) - #148319 (docs: Fix argument names for `carrying_mul_add`) - #148322 (Enable file locking support in illumos) r? `@ghost` `@rustbot` modify labels: rollup
Rollup merge of #147986 - jesseschalken:use-fstatat-macos, r=joboet Use fstatat() in DirEntry::metadata on Apple platforms Apple supports `fstatat` on macOS >=10.10 ([source](https://gitlab.gnome.org/GNOME/glib/-/issues/2203)), and according to [Platform Support](https://doc.rust-lang.org/beta/rustc/platform-support.html) the oldest supported version is 10.12. Google says iOS >=10 supports `fstatat` but doesn't provide a source. [*-apple-ios](https://doc.rust-lang.org/beta/rustc/platform-support/apple-ios.html#os-version) says the minimum supported iOS version is 10.0. Unsure about tvOS, watchOS and visionOS, hoping CI can confirm this. I am testing with [fastdu](https://github.com/jesseschalken/fastdu) which is effectively a stress test for `DirEntry::metadata`. In one test this provides a **1.13x** speedup. ``` $ hyperfine --warmup 1 'target/release/fastdu testdir' 'fastdu testdir' Benchmark 1: target/release/fastdu testdir Time (mean ± σ): 154.6 ms ± 17.4 ms [User: 31.7 ms, System: 187.6 ms] Range (min … max): 148.4 ms … 225.5 ms 19 runs Benchmark 2: fastdu testdir Time (mean ± σ): 175.3 ms ± 15.8 ms [User: 50.0 ms, System: 196.2 ms] Range (min … max): 165.4 ms … 211.7 ms 17 runs Summary target/release/fastdu testdir ran 1.13 ± 0.16 times faster than fastdu testdir ``` You can also reproduce a speedup with a program like this (providing a directory with many entries): ```rust fn main() { let args: Vec<_> = std::env::args_os().collect(); let dir: PathBuf = args[1].clone().into(); for entry in dir.read_dir().as_mut().unwrap() { let entry = entry.as_ref().unwrap(); let metadata = entry.metadata(); let metadata = metadata.as_ref().unwrap(); println!("{} {}", metadata.len(), entry.file_name().display()); } } ``` ``` $ hyperfine './target/release/main testdir' './main testdir' Benchmark 1: ./target/release/main testdir Time (mean ± σ): 148.3 ms ± 5.2 ms [User: 23.1 ms, System: 122.9 ms] Range (min … max): 145.2 ms … 167.2 ms 19 runs Benchmark 2: ./main testdir Time (mean ± σ): 164.4 ms ± 9.5 ms [User: 32.6 ms, System: 128.8 ms] Range (min … max): 158.5 ms … 199.5 ms 17 runs Summary ./target/release/main testdir ran 1.11 ± 0.07 times faster than ./main testdir ```
Apple supports
fstataton macOS >=10.10 (source), and according to Platform Support the oldest supported version is 10.12.Google says iOS >=10 supports
fstatatbut doesn't provide a source. *-apple-ios says the minimum supported iOS version is 10.0.Unsure about tvOS, watchOS and visionOS, hoping CI can confirm this.
I am testing with fastdu which is effectively a stress test for
DirEntry::metadata. In one test this provides a 1.13x speedup.You can also reproduce a speedup with a program like this (providing a directory with many entries):