Re: [PATCH] cat, cp, mv, install, split: Set the minimum IO block size u
From:
Bernhard Voelker
Subject:
Re: [PATCH] cat, cp, mv, install, split: Set the minimum IO block size used, to 64KiB
Date:
Thu, 21 Jul 2011 16:25:04 +0200
User-agent:
Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.18) Gecko/20110616 SUSE/3.1.11 Thunderbird/3.1.11
On 07/21/2011 03:59 PM, PƔdraig Brady wrote:
On 21/07/11 13:15, Bernhard Voelker wrote:
On 07/21/2011 02:06 PM, Bernhard Voelker wrote:
On 07/21/2011 01:21 PM, PƔdraig Brady wrote:
I'm a little wary of this being too aggressive,
but it does give a noticeable (13%) boost on my new laptop.
Here are the numbers from dd bs=$blksize if=/dev/zero of=/dev/null
blksize system-1 system-2
----------------------------
1024 734 MB/s 1.7 GB/s
2048 1.3 GB/s 3.0 GB/s
4096 2.4 GB/s 5.1 GB/s
8192 3.5 GB/s 7.3 GB/s
16384 3.9 GB/s 9.4 GB/s
32768 5.2 GB/s 9.9 GB/s
65536 5.3 GB/s 11.2 GB/s
131072 5.5 GB/s 11.8 GB/s
262144 5.7 GB/s 11.6 GB/s
524288 5.7 GB/s 11.4 GB/s
1048576 5.8 GB/s 11.4 GB/s
Hmm, I've taken the script from the test, but I'm not
quite sure about the numbers on my system:
$ for i in $(seq 0 10); do bs=$((1024*2**$i)); printf "%7s=" $bs;
src/timeout --foreground -sINT 1 dd bs=$bs if=/dev/zero of=/dev/null
2>&1 | sed -n 's/.* \([0-9.]* [GM]B\/s\)/\1/p' ; done
1024=2.3 GB/s
2048=4.0 GB/s
4096=6.7 GB/s
8192=9.7 GB/s
16384=16.3 GB/s
32768=31.3 GB/s
65536=52.2 GB/s
131072=81.6 GB/s
262144=115 GB/s
524288=141 GB/s
1048576=157 GB/s
I have a "Intel(R) Core(TM)2 Quad CPU Q6600 @ 2.40GHz" system.
And I used `src/timeout` from the latest `git pull`.
Wow, what?
Total guess. Perhaps your hardware has facilities to allow the kernel
determine the buffers haven't been modified, thus avoiding
the data copy in the write to /dev/null ??
What kernel are you using?
The situation changes when I specify ibs and obs for dd
(I looped until 15) for testing):
berny@blackice:~/git/coreutils> for i in $(seq 0 15); do
bs=$((1024*2**$i)); printf "%8s=" $bs; src/timeout --foreground -sINT 1
src/dd ibs=$bs obs=$bs if=/dev/zero of=/dev/null 2>&1 | sed -n 's/.*
\([0-9.]* [GM]B\/s\)/\1/p' ; done
1024=1.9 GB/s
2048=3.0 GB/s
4096=4.5 GB/s
8192=5.9 GB/s
16384=7.2 GB/s
32768=7.8 GB/s
65536=8.7 GB/s
131072=9.3 GB/s
262144=9.6 GB/s
524288=9.8 GB/s
1048576=9.9 GB/s
2097152=6.0 GB/s
4194304=5.1 GB/s
8388608=4.6 GB/s
16777216=4.4 GB/s
33554432=4.3 GB/s
So you get more "standard" results with the data copy,
with a corresponding 11.5% increase when going from 32K -> 64K.
Interestingly, you only see a drop in performance at 1M -> 2M,
whereas I gradually drop performance starting at 64K -> 32K.
1024=1.5 GB/s
2048=2.7 GB/s
4096=4.3 GB/s
8192=5.9 GB/s
16384=6.1 GB/s
32768=5.9 GB/s
65536=6.4 GB/s
131072=6.1 GB/s
262144=5.2 GB/s
524288=4.2 GB/s
1048576=4.3 GB/s
2097152=4.2 GB/s
4194304=3.4 GB/s
8388608=3.3 GB/s
16777216=3.4 GB/s
33554432=3.4 GB/s
Do you think that the assumption - that the io block size of
a sample dd run can be transfered to cat, cp et al - is valid?
Have a nice day,
Berny