bug-coreutils
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

bug#80106: Don't sort dirents (in du) for Lustre


From: Collin Funk
Subject: bug#80106: Don't sort dirents (in du) for Lustre
Date: Thu, 01 Jan 2026 21:07:06 -0800
User-agent: Gnus/5.13 (Gnus v5.13)

"Day, Timothy" <timday@amazon.com> writes:

>>> For context, Lustre is an open-source (GPLv2) out-of-tree Linux filesystem
>>> commonly used for HPC applications.
>>>
>>> Let me know if you need anymore context. Thanks!
>>
>>Could you check coreutils built with this patch and see if it fixes the
>>performance? I can push it to Gnulib afterwards:
>
> Works perfectly:
>
> $ echo 3 | sudo tee /proc/sys/vm/drop_caches 
> 3
> $ time coreutils-9.9-unpatched/src/du /fsx/test_100K > /dev/null
>
> real  1m54.268s
> user  0m0.229s
> sys   0m14.136s
> $ echo 3 | sudo tee /proc/sys/vm/drop_caches 
> 3
> $ time coreutils-9.9-patched/src/du /fsx/test_100K > /dev/null
>
> real  0m13.150s
> user  0m0.075s
> sys   0m1.956s
>
> That's around a 9x speedup. I haven't tested any other commands.
> Lustre statahead (which is giving the performance boost) is designed
> for "stat everything in a directory" type workloads. The other commands
> wouldn't do that, I think. Perhaps chown would? Either way, thanks for
> the help!

Great, thanks for your help investigating!

I think 'chown -R' and 'rm -r' would probably be affected as well:

    $ mkdir -p dir && (cd dir && mkdir $(seq 10000))
    $ strace chown -R collin:collin dir 2>&1 > /dev/null | grep -Fc stat
    20022
    $ strace rm -r dir 2>&1 > /dev/null | grep -Fc stat
    40011

Anyways, I pushed the Gnulib patch and updated the submodule in
coreutils with the attached patch.

Pádraig, I assume this warrants a NEWS entry under "Bug Fixes"?

Collin

Attachment: 0001-build-update-gnulib-submodule-to-latest.patch
Description: Text Data


reply via email to

[Prev in Thread] Current Thread [Next in Thread]