du counting files in hard-linked directories twice
From:
Michael Schwarz
Subject:
du counting files in hard-linked directories twice
Date:
Thu, 27 Aug 2009 18:09:11 +0200
Hello everybody
The Problem I see as a bug is that, in some circumstances, du is
counting the same files multiple times when they appear inside of hard-
linked directories.
I have tested this with GNU coreutils 7.4 (MacPorts) and 7.5 (vanilla)
on Mac OS X 10.5.8 on x86-64. I did all tests on an HFS+ volume,
formatted as being non-cases-sensitive.
Judging from du.c, the problem seems to be that du only inserts a file
into it's table of visited files if it's link count is greater than
one (du.c, Lines 513-517):
Replacing the 1 in the condition above with a 0 solves the problem,
although it would probably be a much better idea to insert hard-linked
directories into the hash table instead. This would also prevent du
from counting the size of directory inodes twice.
Appended is a simple script which makes three tests:
* One where there is a single file with two links.
* One where there is a directory with two links containing one file
* One where there is a directory with two links containing two links
to a single file.
After the script is it's output on my machine and the output of ls -
ilR for reference.
With all the tests there is exactly one file, 1 MiB in size and only
for the second test, du reports a total size of 2 MiB.
I hope you agree that this is a bug in du and that I did not miss a
problem on my side in evaluating this. I will gladly be of any help I
can on fixing this problem, e.g. testing patches for people who do not
have a setup that may be used to reproduce this problem.
Thank you
Michael
$ cat test.sh
#! /usr/bin/env bash
set -e -o pipefail
# Test with a single file with two links.
(
echo 'Test with file hard links:'
rm -rf 'test-d1-f2'
mkdir 'test-d1-f2'
cd 'test-d1-f2'
mkdir 'top-1'
mkdir 'top-2'
dd bs=1k count=1k < /dev/zero > 'top-1/file' 2> /dev/null
link 'top-1/file' 'top-2/file'
du -h
echo
)
# Test with a directory with two links containing a single file with
one link.
(
echo 'Test with directory hard links:'
rm -rf 'test-d2-f1'
mkdir 'test-d2-f1'
cd 'test-d2-f1'
# Because of a restriction in HFS+ or the Mac OS, we need to put the
two directory entries for the hard-linked directory into sepparate
directories.