Sent with Proton Mail secure email.
On Saturday, June 7th, 2025 at 1:54 AM, Greg Wooledge <greg@wooledge.org> wrote:
> On Fri, Jun 06, 2025 at 06:06:05 +0000, Heime via wrote:
>
> > Can I improve this?
> >
> > mysearch ()
> > {
> > local search_dir=$1
> > local search_str=$2
> >
> > if [[ -z "$search_dir" || -z "$search_str" ]]; then
> > echo "Usage: mysearch <directory> <search_string>"
> > return 1
> > fi
> >
> > find "$search_dir" -type f | while IFS= read -r file; do
> > matches=$(grep -n -- "$search_str" "$file" | sed 's/:/: /1' )
> > if [[ -n "$matches" ]]; then
> > echo "$file"
> > echo "$matches"
> > echo
> > fi
> > done
> > }
>
>
> Storing a list in a string variable is never ideal. You want to store
> lists in array variables. That's the first and most important improvement
> I would recommend.
>
> After that, the primary issue is how you separate the filenames from
> each other. Except... now that I've read your code a couple of times,
> it appears you aren't storing just filenames. You appear to be storing
> filename/line number pairs, with the possibility of multiple matches
> per file. So, now we're back to the question of how to store these
> things in a useful way. The answer to that is going to depend primarily
> on what you intend to do with these pairs, and you didn't tell us that.
I want to have the File Name (FLNAME) followed by the Line Number (LNUM)
followed by Colon and Space (: ) and the Matching Line (MATCH)
FLNAME
LNUM: MATCH
LNUM: MATCH
LNUM: MATCH
And this for all filename.
> The most obvious way (to me) to store these pairs would use two arrays.
> Both arrays would be indexed by a "match number" which exists only to
> tie the two results together. The first array would store the filename,
> and the second, the line number.
>
> So then the problem becomes how to parse out the line number and filename
> from each result. The naive approach which you appear to be using right
> now uses grep's line-oriented output, where the pairs are separated by
> newlines. This will break down as soon as you encounter a filename with
> a newline in it.
>
> hobbit:/tmp/x$ touch z 's pace' $'t\tab' $'new\nline'
> hobbit:/tmp/x$ ls -l
> total 0
> -rw-r--r-- 1 greg greg 0 Jun 6 09:15 'new'$'\n''line'
> -rw-r--r-- 1 greg greg 0 Jun 6 09:15 's pace'
> -rw-r--r-- 1 greg greg 0 Jun 6 09:15 't'$'\t''ab'
> -rw-r--r-- 1 greg greg 0 Jun 6 09:15 z
> hobbit:/tmp/x$ for f in *; do echo payload >> "$f"; done
>
> hobbit:/tmp/x$ grep -n p *
> new
> line:1:payload
> s pace:1:payload
> t ab:1:payload
> z:1:payload
>
> Obviously, that's not viable. However, if you're writing this function
> to be used only on GNU/Linux systems, you can use GNU grep's --null
> option to produce a stream with NULs in it. Normally, this would be
> used to produce a parseable list of filenames. However, your desire
> to include matching line numbers complicates this quite a bit.
>
> hobbit:/tmp/x$ grep --null -n p * | hd
> 00000000 6e 65 77 0a 6c 69 6e 65 00 31 3a 70 61 79 6c 6f |new.line.1:paylo|
> 00000010 61 64 0a 73 20 70 61 63 65 00 31 3a 70 61 79 6c |ad.s pace.1:payl|
> 00000020 6f 61 64 0a 74 09 61 62 00 31 3a 70 61 79 6c 6f |oad.t.ab.1:paylo|
> 00000030 61 64 0a 7a 00 31 3a 70 61 79 6c 6f 61 64 0a |ad.z.1:payload.|
> 0000003f
>
> As far as I can tell, what grep is writing here is:
>
> File name
> NUL
> Line number
> Colon
> Matching line text
> Newline
>
> This is repeated for each match, and the filename may contain a newline
> character. Therefore, we can't just split the output on NULs or on
> newlines. We have to use an iterative approach -- read until the
> next NUL, then read until the next newline, and repeat.
>
> Also, just for the record, be warned that filenames may contain bytes
> which are not valid characters in your locale. Several versions of
> bash have bugs which crop up when reading filenames containing invalid
> partial characters. The workaround for that is to override the locale
> variable(s).[1]
>
> Since we're using GNU grep, we can also use its -r option to recurse
> for us, and drop the unnecessary find command.
>
> So, in order to parse this into our two arrays, we would need something
> like this:
>
> files=()
> lines=()
> i=-1
> while IFS= LC_ALL=C read -r -d '' file; do
> IFS=: read -r num text
> files[++i]=$file
> lines[i]=$num
> done < <(grep --null -r -n -- "$search_str" "$seach_dir")
>
> The "text" variable is unused, so we don't need to worry about the issues
> with using multiple variables with read.[2]
>
> This should handle any filename, in any locale, in any version of bash
> that has indexed arrays and read -d ''. It looks like the minimum bash
> version for those is 2.04. I don't know the minimum required version
> of GNU grep to have the --null feature; you might need to research that.
>
> If your needs aren't exactly met by this, then please feel free to
> explain more precisely what you need.
>
> [1] https://mywiki.wooledge.org/BashPitfalls#pf65
> [2] https://mywiki.wooledge.org/BashPitfalls#pf47