Sent with Proton Mail secure email.
On Saturday, June 7th, 2025 at 5:20 AM, Greg Wooledge <greg@wooledge.org> wrote:
> On Fri, Jun 06, 2025 at 15:06:12 +0000, Heime wrote:
>
> > I want to have the File Name (FLNAME) followed by the Line Number (LNUM)
> > followed by Colon and Space (: ) and the Matching Line (MATCH)
> >
> > FLNAME
> > LNUM: MATCH
> > LNUM: MATCH
> > LNUM: MATCH
> >
> > And this for all filename.
>
>
> OK. Your Subject: header said "Storing grep results in variable", but it
> turns out you don't actually want to store anything. Your ultimate
> goal is just to have this human-readable report.
Correct, just to have a human-readable report, without the file name getting
repeated on every line. It became problematic when the directory path was very
long due to large number of nested paths.
> That changes pretty much everything.
>
> The way you wrote it originally isn't terrible, given this goal. You're
> using find to generate the list of filenames, which means you don't have
> to worry about parsing them out of a stream and dealing with newlines.
> You can just use find -print0 and read the NUL-delimited names.
>
> The alternative would be to use grep -r to do the recursion and matching
> in a single command, and parse the stream to produce the desired report.
>
> Let's take a look at both of those approaches.
>
> == Version 1: using find ==
>
> # This version uses one grep and sed PER file, plus one find overall.
>
> find "$search_dir" -type f -print0 |
> while IFS= LC_ALL=C read -r -d '' file; do
> output=$(grep -n -- "$search_str" < "$file" | sed 's/:/: /')
> if [[ $output ]]; then
> printf '%s\n%s\n\n' "$file" "$output"
> fi
> done
>
> That's very similar to what you wrote originally. I just added the
> NUL-termination stuff to make it safe for all filenames. Here's the
> output I get:
>
> hobbit:~$ mkdir /tmp/x
> hobbit:~$ touch /tmp/x/z /tmp/x/'s pace' /tmp/x/$'t\tab' /tmp/x/$'new\nline'
> hobbit:~$ for f in /tmp/x/*; do printf 'foo\nbar\nfoot\n' > "$f"; done
>
> hobbit:~$ find /tmp/x -type f -print0 | while IFS= LC_ALL=C read -r -d ''
> file; do output=$(grep -n -- foo < "$file" | sed 's/:/: /'); if [[ $output
> ]]; then printf '%s\n%s\n\n' "$file" "$output"; fi; done
> /tmp/x/t ab
> 1: foo
> 3: foot
>
> /tmp/x/z
> 1: foo
> 3: foot
>
> /tmp/x/s pace
> 1: foo
> 3: foot
>
> /tmp/x/new
> line
> 1: foo
> 3: foot
>
>
> You could replace the sed with a bash parameter expansion that would
> replace ':' with ': ' in the output. It probably won't make much
> difference in the performance.
>
>
> == Version 2: using grep -r ==
>
> # This version uses one grep, but does slow reading/parsing in bash.
> # This time, I'll use the parameter expansion instead of sed.
>
> prev=
> grep --null -r -n -- "$search_str" "$search_dir" |
> while IFS= LC_ALL=C read -r -d '' file; do
> if [[ $file != "$prev" ]]; then
> printf '\n%s\n' "$file"
> prev=$file
> fi
> IFS= read -r match
> printf '%s\n' "${match/:/: }"
> done
>
> And the output:
>
> hobbit:~$ prev=; grep --null -r -n -- foo /tmp/x | while IFS= LC_ALL=C read
> -r -d '' file; do if [[ $file != "$prev" ]]; then printf '\n%s\n' "$file";
> prev=$file; fi; IFS= read -r match; printf '%s\n' "${match/:/: }"; done
>
> /tmp/x/t ab
> 1: foo
> 3: foot
>
> /tmp/x/z
> 1: foo
> 3: foot
>
> /tmp/x/s pace
> 1: foo
> 3: foot
>
> /tmp/x/new
> line
> 1: foo
> 3: foot
>
> This time, the extra blank line is at the top instead of the bottom.
> If that's a problem, I'm sure you can figure out some ways to change
> that. An extra "first" flag variable would be one way. Equivalently,
> treating an empty $prev as a "first" flag would also work. Or you
> could store results in memory and only add the extra newlines when
> printing them to stdout, etc.