-
-
Notifications
You must be signed in to change notification settings - Fork 218
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
bug: slow performance with many directories with files included #844
Comments
Indeed second run is faster due to size caching, have you tried, as you don't seem to wanna know whats inside the dirs, to use the |
Could you maybe specify how it takes? On my machine, it consistently takes about 100ms, which isn't all too bad. Even for 5000 files, I don't think the check for empty folders is super expensive, so I have a feeling something more is going on, like Martin already suggested. It would be awesome if you could provide some more info, like a flamegraph or a samply profile. Or maybe the first few lines of running |
@tertsdiepraam Thanks for the suggestions. With 10164 files (no folders) it took 3m49s. Here's the
The second run was faster (~1s)
|
Open these with https://profiler.firefox.com samply profile with samply profile with I made sure I ran those on a new directory with pretty much similar files to avoid caching. |
That's great (the info you provided, not the performance 😄)! 3 minutes for a few statx calls definitely seems strange. That's gotta be quite a slow drive then? Nevertheless, I think this ties in with my PR here, which aims to reduce the number of However, the fact that |
note that when using
Depending on the filesystem, this can be optimized. For example BTRFS shows the size of a directory as 0 if it is empty, on ZFS it is 2 if empty (because of the . and .. I'm assuming), and on XFS it is 6 (directory entry header I guess? this might change depending on the on-disk format version and the number of files that has previously existed there) if empty. Does also seem to work over NFS, but filesystem detection would need to account for that. diff --git a/src/fs/file.rs b/src/fs/file.rs
index ba2703d9..7e3c31dd 100644
--- a/src/fs/file.rs
+++ b/src/fs/file.rs
@@ -666,6 +666,8 @@ impl<'dir> File<'dir> {
/// based on directory size alone.
#[cfg(unix)]
pub fn is_empty_dir(&self) -> bool {
+ use std::ffi::CString;
+
if self.is_directory() {
if self.metadata.nlink() > 2 {
// Directories will have a link count of two if they do not have any subdirectories.
@@ -675,6 +677,25 @@ impl<'dir> File<'dir> {
// has subdirectories.
false
} else {
+ unsafe {
+ let Some(str_path) = self.path.to_str() else {
+ return self.is_empty_directory();
+ };
+ let Ok(c_path) = CString::new(str_path) else {
+ return self.is_empty_directory();
+ };
+
+ let mut statfs_res: libc::statfs = std::mem::zeroed();
+ if libc::statfs(c_path.as_ptr().cast::<libc::c_char>(), &mut statfs_res) == 0 {
+ match statfs_res.f_type {
+ 0x9123_683e => return self.metadata.size() == 0, // BTRFS
+ 0x2fc1_2fc1 => return self.metadata.size() == 2, // ZFS
+ 0x5846_5342 => return self.metadata.size() == 6, // XFS
+ _ => (),
+ };
+ };
+ };
+
self.is_empty_directory()
}
} else { (doesn't work on EXT4 because the directory size seems to be 4096 even with a few files) Only problem with this patch is that it is ever so slightly slower on non-BTRFS/ZFS/XFS. Maybe users could opt-in with a flag? ref @BrianCArnold from #745 |
eza --version
)0.18.3
eza --icons auto
Ubuntu 22.04, Azure with mounted drive (shared across 3 VMs but not network)
With many folders (~5000), eza takes a while to print with icons. It is much slower than
exa
, and I think it's becauseeza
also checks if the directory is empty.From the second run, it runs faster maybe due to caching of the file system.
Possible solutions:
The files look like this:
The text was updated successfully, but these errors were encountered: