Readdelim (was Re: Examples of concurrent coproc usage)
From:
Martin D Kealey
Subject:
Readdelim (was Re: Examples of concurrent coproc usage)
Date:
Mon, 29 Apr 2024 04:59:58 +1200
On Sun, 28 Apr 2024, 05:03 Carl Edquist, <edquist@cs.wisc.edu> wrote:
>
> > I would hope that mapfile/readarray could do better, since it's not
> > obligated to leave anything in the input stream.
>
> That is an interesting thought, although mapfile seems to read a byte at a
> time also.
>
> [I'm not suggesting this should be changed though. And frankly I have no
> strong desire for a faster 'read' builtin in the shell, either. A byte at
> a time is relatively slow, but that's generally fine in most limited
> contexts where it's actually needed.]
>
I'm encountering contexts where the lack of speed is annoying though not
critical.
My thought is that (performance considerations aside), the real functional
> improvement with a new "readd" call would be with _competing_ readers
> (more than one read call waiting on the same pipe at the same time).
>
That's a very good point; I'll mention that when I write to the Linux
kernel team.
In that case a length-prefixed or type-tagged record wouldn't seem to work
> with the regular read(2), because a single reader would not be able to
> read the length/type _and_ the corresponding record together. You can't
> work around this by reading a byte at a time either. That's why I said it
> would only seem to work (with read(2)) if the records have a fixed size.
> (In order to grab a whole record atomically.)
>
This makes me wonder whether readv() could be encouraged to handle
length-prefix simply by targeting the length element of a subsequent
element of the iovec array? iov[0].ptr = &iov[1].len; iov[0].len = sizeof
iov[1].len; iov[1].ptr = malloc(MAX_RECORD_SIZE); I suspect this won't work
today because the kernel probably computes the total read size up front.
But maybe some day?
-Martin