"Day, Timothy" <timday@amazon.com> writes:
>>> For context, Lustre is an open-source (GPLv2) out-of-tree Linux filesystem
>>> commonly used for HPC applications.
>>>
>>> Let me know if you need anymore context. Thanks!
>>
>>Could you check coreutils built with this patch and see if it fixes the
>>performance? I can push it to Gnulib afterwards:
>
> Works perfectly:
>
> $ echo 3 | sudo tee /proc/sys/vm/drop_caches
> 3
> $ time coreutils-9.9-unpatched/src/du /fsx/test_100K > /dev/null
>
> real 1m54.268s
> user 0m0.229s
> sys 0m14.136s
> $ echo 3 | sudo tee /proc/sys/vm/drop_caches
> 3
> $ time coreutils-9.9-patched/src/du /fsx/test_100K > /dev/null
>
> real 0m13.150s
> user 0m0.075s
> sys 0m1.956s
>
> That's around a 9x speedup. I haven't tested any other commands.
> Lustre statahead (which is giving the performance boost) is designed
> for "stat everything in a directory" type workloads. The other commands
> wouldn't do that, I think. Perhaps chown would? Either way, thanks for
> the help!
Great, thanks for your help investigating!
I think 'chown -R' and 'rm -r' would probably be affected as well:
$ mkdir -p dir && (cd dir && mkdir $(seq 10000))
$ strace chown -R collin:collin dir 2>&1 > /dev/null | grep -Fc stat
20022
$ strace rm -r dir 2>&1 > /dev/null | grep -Fc stat
40011
Anyways, I pushed the Gnulib patch and updated the submodule in
coreutils with the attached patch.
Pádraig, I assume this warrants a NEWS entry under "Bug Fixes"?
Collin