bug#7489: [coreutils] over aggressive threads in sort
From:
Paul Eggert
Subject:
bug#7489: [coreutils] over aggressive threads in sort
Date:
Mon, 06 Dec 2010 18:23:38 -0800
User-agent:
Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.15) Gecko/20101027 Thunderbird/3.0.10
On 12/05/10 03:21, Jim Meyering wrote:
> seq -w 200000 > exp && tac exp > in
> PATH=.:$PATH ./sort --compress-program=dzip -S 1k in > out
>
> That gets stuck in waitpid (from sort.c's reap), waiting for a
> dzip invocation that appears will never terminate. This is also
> on that same 4-core system, and is relatively easy to reproduce,
> so it should be easy to identify the offending change, but I'm
> out of time, now.
>
I've verified this bug. It occurs regardless of whether the
December 3 changes are applied, so it looks like it was an
earlier change. I've also verified that it occurs with
--parallel=1 so it seems unlikely it would be thread-related.
What happens is that 'sort' creates a pipe to a compressor
but then never closes the pipe. When it waits for the compressor
to finish there is an obvious deadlock.
I'll see if I can track it down, but my guess it's somewhere
in the compressor/decompressor code. It's doing some lazy
evaluation and I wouldn't be surprised if that goes haywire
in some cases.
I'd guess there's also a bug in the thread code too,
having to do with your core dump (and Chen's; could be
two bugs).