Re: 'read' sometimes truncates trailing field delimiter
From:
MacBeth
Subject:
Re: 'read' sometimes truncates trailing field delimiter
Date:
Sun, 12 Jan 2025 08:58:01 -0600
On Sun, Jan 12, 2025 at 8:24 AM Félix Hauri <felix@f-hauri.ch> wrote:
> Le Sat, Jan 11, 2025 at 10:49:50AM -0500, Greg Wooledge a écrit :
> > https://mywiki.wooledge.org/BashPitfalls#pf47
> > "It's a feature."
> >
> -- how do we work around this nonsense?
>
> I've alredy noticed this, but never stictly tested them as:
>
> for row in k{:v1{:v2{:{:{:,},},},:{:{:,},},},:{:{:,},},}; do
> IFS=: read k v <<<"$row";
> printf "%-20s %s\n" "row='$row'" "k='$k', v='$v'";
> done
> row='k:v1:v2:::' k='k', v='v1:v2:::'
> row='k:v1:v2::' k='k', v='v1:v2::'
> row='k:v1:v2:' k='k', v='v1:v2:'
> row='k:v1:v2' k='k', v='v1:v2'
> row='k:v1:::' k='k', v='v1:::'
> row='k:v1::' k='k', v='v1::'
> row='k:v1:' k='k', v='v1'
> row='k:v1' k='k', v='v1'
> row='k:::' k='k', v='::'
> row='k::' k='k', v=''
> row='k:' k='k', v=''
> row='k' k='k', v=''
>
> Thanks for driving my attention to this *feature*!
>
> --
> Félix Hauri - <felix@f-hauri.ch> - http://www.f-hauri.ch
>
> Greg's link provides a workaround:
$ input='foo:bar:'
$ echo "$input:" | { IFS=: read -r f1 rest; rest=${rest%:}; declare -p
rest; }
declare -- rest="bar:"
However it only works for reading a single line. If you want to read a
file, I think `sed` might be worthwhile instead of doing this individually
for every line in a loop.
Here is a `sed` version:
delim=,
{
sed -r -e "s/\$/$delim/" |
while IFS="$delim" read -r k values; do
values="${values%$delim}"
echo "k='$k' v='$values'"
done
} <<EOT
a,b,c,d,e
a,b,c,d,
a,b,c,d
a,b,c,
a,b,c
a,b,
a,b
a,
a
EOT
And here is a version you would have to use without `sed` to get the line
first, before you can append the field terminator to it:
delim=,
{
while IFS= read -r row; do
IFS="$delim" read -r k values < <(echo "$row$delim")
values="${values%$delim}"
printf "%-20s %s\n" "row='$row'" "k='$k' v='$values'"
done
} <<EOT
a,b,c,d,e
a,b,c,d,
a,b,c,d
a,b,c,
a,b,c
a,b,
a,b
a,
a
EOT