Re: line buffering in pipes (was: RFC: Safely using xargs -P$NUM childre
From:
Egmont Koblinger
Subject:
Re: line buffering in pipes (was: RFC: Safely using xargs -P$NUM children's output? Need a new tool?)
Date:
Thu, 2 May 2019 21:22:23 +0200
Hi Assaf,
On Thu, May 2, 2019 at 9:14 PM Assaf Gordon <address@hidden> wrote:
> The easiest way to avoid that is to use "stdbuf" (from coreutils),
> forcing a flush after each line. Assuming the lines are short enough
> (and file's output should be short enough), it should work:
>
> find /usr -print0 | xargs -0r -P199 -n16 stdbuf -oL file | ...
I don't think this is robust enough. If many "stdbuf -oL file"
processes decide to produce a reasonably sized output pretty much at
the same time, it might still suddenly clog the pipe and result in a
short write in one of them. Or am I missing something?
cheers,
egmont