mapfile -d '' matches < <( grep -Z
On Fri, Jun 6, 2025, 5:09 PM Heime via <help-bash@gnu.org> wrote:
>
> Sent with Proton Mail secure email.
>
> On Saturday, June 7th, 2025 at 1:54 AM, Greg Wooledge <greg@wooledge.org>
> wrote:
>
> > On Fri, Jun 06, 2025 at 06:06:05 +0000, Heime via wrote:
> >
> > > Can I improve this?
> > >
> > > mysearch ()
> > > {
> > > local search_dir=$1
> > > local search_str=$2
> > >
> > > if [[ -z "$search_dir" || -z "$search_str" ]]; then
> > > echo "Usage: mysearch <directory> <search_string>"
> > > return 1
> > > fi
> > >
> > > find "$search_dir" -type f | while IFS= read -r file; do
> > > matches=$(grep -n -- "$search_str" "$file" | sed 's/:/: /1' )
> > > if [[ -n "$matches" ]]; then
> > > echo "$file"
> > > echo "$matches"
> > > echo
> > > fi
> > > done
> > > }
> >
> >
> > Storing a list in a string variable is never ideal. You want to store
> > lists in array variables. That's the first and most important improvement
> > I would recommend.
> >
> > After that, the primary issue is how you separate the filenames from
> > each other. Except... now that I've read your code a couple of times,
> > it appears you aren't storing just filenames. You appear to be storing
> > filename/line number pairs, with the possibility of multiple matches
> > per file. So, now we're back to the question of how to store these
> > things in a useful way. The answer to that is going to depend primarily
> > on what you intend to do with these pairs, and you didn't tell us that.
>
> I want to have the File Name (FLNAME) followed by the Line Number (LNUM)
> followed by Colon and Space (: ) and the Matching Line (MATCH)
>
> FLNAME
> LNUM: MATCH
> LNUM: MATCH
> LNUM: MATCH
>
> And this for all filename.
>
>
>
> > The most obvious way (to me) to store these pairs would use two arrays.
> > Both arrays would be indexed by a "match number" which exists only to
> > tie the two results together. The first array would store the filename,
> > and the second, the line number.
> >
> > So then the problem becomes how to parse out the line number and filename
> > from each result. The naive approach which you appear to be using right
> > now uses grep's line-oriented output, where the pairs are separated by
> > newlines. This will break down as soon as you encounter a filename with
> > a newline in it.
> >
> > hobbit:/tmp/x$ touch z 's pace' $'t\tab' $'new\nline'
> > hobbit:/tmp/x$ ls -l
> > total 0
> > -rw-r--r-- 1 greg greg 0 Jun 6 09:15 'new'$'\n''line'
> > -rw-r--r-- 1 greg greg 0 Jun 6 09:15 's pace'
> > -rw-r--r-- 1 greg greg 0 Jun 6 09:15 't'$'\t''ab'
> > -rw-r--r-- 1 greg greg 0 Jun 6 09:15 z
> > hobbit:/tmp/x$ for f in *; do echo payload >> "$f"; done
> >
> > hobbit:/tmp/x$ grep -n p *
> > new
> > line:1:payload
> > s pace:1:payload
> > t ab:1:payload
> > z:1:payload
> >
> > Obviously, that's not viable. However, if you're writing this function
> > to be used only on GNU/Linux systems, you can use GNU grep's --null
> > option to produce a stream with NULs in it. Normally, this would be
> > used to produce a parseable list of filenames. However, your desire
> > to include matching line numbers complicates this quite a bit.
> >
> > hobbit:/tmp/x$ grep --null -n p * | hd
> > 00000000 6e 65 77 0a 6c 69 6e 65 00 31 3a 70 61 79 6c 6f
> |new.line.1:paylo|
> > 00000010 61 64 0a 73 20 70 61 63 65 00 31 3a 70 61 79 6c |ad.s
> pace.1:payl|
> > 00000020 6f 61 64 0a 74 09 61 62 00 31 3a 70 61 79 6c 6f
> |oad.t.ab.1:paylo|
> > 00000030 61 64 0a 7a 00 31 3a 70 61 79 6c 6f 61 64 0a |ad.z.1:payload.|
> > 0000003f
> >
> > As far as I can tell, what grep is writing here is:
> >
> > File name
> > NUL
> > Line number
> > Colon
> > Matching line text
> > Newline
> >
> > This is repeated for each match, and the filename may contain a newline
> > character. Therefore, we can't just split the output on NULs or on
> > newlines. We have to use an iterative approach -- read until the
> > next NUL, then read until the next newline, and repeat.
> >
> > Also, just for the record, be warned that filenames may contain bytes
> > which are not valid characters in your locale. Several versions of
> > bash have bugs which crop up when reading filenames containing invalid
> > partial characters. The workaround for that is to override the locale
> > variable(s).[1]
> >
> > Since we're using GNU grep, we can also use its -r option to recurse
> > for us, and drop the unnecessary find command.
> >
> > So, in order to parse this into our two arrays, we would need something
> > like this:
> >
> > files=()
> > lines=()
> > i=-1
> > while IFS= LC_ALL=C read -r -d '' file; do
> > IFS=: read -r num text
> > files[++i]=$file
> > lines[i]=$num
> > done < <(grep --null -r -n -- "$search_str" "$seach_dir")
> >
> > The "text" variable is unused, so we don't need to worry about the issues
> > with using multiple variables with read.[2]
> >
> > This should handle any filename, in any locale, in any version of bash
> > that has indexed arrays and read -d ''. It looks like the minimum bash
> > version for those is 2.04. I don't know the minimum required version
> > of GNU grep to have the --null feature; you might need to research that.
> >
> > If your needs aren't exactly met by this, then please feel free to
> > explain more precisely what you need.
> >
> > [1] https://mywiki.wooledge.org/BashPitfalls#pf65
> > [2] https://mywiki.wooledge.org/BashPitfalls#pf47
>
>