From MAILER-DAEMON Fri Jun 01 11:05:36 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1SaTQ0-0005HH-Qp
	for mharc-grep-commit@gnu.org; Fri, 01 Jun 2012 11:05:36 -0400
Received: from eggs.gnu.org ([208.118.235.92]:33479)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eggert@cs.ucla.edu>) id 1SaTPv-0005Fs-DP
	for grep-commit@gnu.org; Fri, 01 Jun 2012 11:05:35 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eggert@cs.ucla.edu>) id 1SaTPp-0001Qc-1h
	for grep-commit@gnu.org; Fri, 01 Jun 2012 11:05:30 -0400
Received: from vcs.savannah.gnu.org ([140.186.70.72]:56697)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eggert@cs.ucla.edu>) id 1SaTPo-0001QV-U5
	for grep-commit@gnu.org; Fri, 01 Jun 2012 11:05:24 -0400
Received: from eggert by vcs.savannah.gnu.org with local (Exim 4.72)
	(envelope-from <eggert@cs.ucla.edu>) id 1SaTPn-0005vo-VV
	for grep-commit@gnu.org; Fri, 01 Jun 2012 15:05:24 +0000
Date: Fri, 01 Jun 2012 15:05:23 +0000
Message-Id: <E1SaTPn-0005vo-VV@vcs.savannah.gnu.org>
To: grep-commit@gnu.org
Subject: grep branch, master, updated. v2.12-15-g2665746
X-Git-Refname: refs/heads/master
X-Git-Reftype: branch
X-Git-Oldrev: 34e0adabf0cd98eb4d1c4a929acf0b8238fcfbbc
X-Git-Newrev: 2665746b756bd372ba856e165388dc98032362fd
From: Paul Eggert <eggert@cs.ucla.edu>
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 2)
X-Received-From: 140.186.70.72
X-BeenThere: grep-commit@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
Reply-To: bug-grep@gnu.org
List-Id: CVS commit notices for GNU grep <grep-commit.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/grep-commit>,
	<mailto:grep-commit-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/grep-commit>
List-Post: <mailto:grep-commit@gnu.org>
List-Help: <mailto:grep-commit-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/grep-commit>,
	<mailto:grep-commit-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 01 Jun 2012 15:05:36 -0000

This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "grep".

The branch, master has been updated
       via  2665746b756bd372ba856e165388dc98032362fd (commit)
      from  34e0adabf0cd98eb4d1c4a929acf0b8238fcfbbc (commit)

Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.

- Log -----------------------------------------------------------------
http://git.savannah.gnu.org/cgit/grep.git/commit/?id=2665746b756bd372ba856e165388dc98032362fd


commit 2665746b756bd372ba856e165388dc98032362fd
Author: Paul Eggert <eggert@cs.ucla.edu>
Date:   Fri Jun 1 08:04:44 2012 -0700

    grep: remove unnecessary "what-if-signal?" code
    
    * src/main.c (fillbuf): Don't worry about EINTR when closing --
    not possible, since we're not catching signals.

diff --git a/src/main.c b/src/main.c
index 10fbfac..dda7c9b 100644
--- a/src/main.c
+++ b/src/main.c
@@ -601,7 +601,7 @@ reset (int fd, struct stat const *st)
 static int
 fillbuf (size_t save, struct stat const *st)
 {
-  size_t fillsize = 0;
+  ssize_t fillsize;
   int cc = 1;
   char *readbuf;
   size_t readsize;
@@ -662,18 +662,9 @@ fillbuf (size_t save, struct stat const *st)
   readsize = buffer + bufalloc - readbuf;
   readsize -= readsize % pagesize;
 
-  if (! fillsize)
-    {
-      ssize_t bytesread;
-      while ((bytesread = read (bufdesc, readbuf, readsize)) < 0
-             && errno == EINTR)
-        continue;
-      if (bytesread < 0)
-        cc = 0;
-      else
-        fillsize = bytesread;
-    }
-
+  fillsize = read (bufdesc, readbuf, readsize);
+  if (fillsize < 0)
+    fillsize = cc = 0;
   bufoffset += fillsize;
 #if defined HAVE_DOS_FILE_CONTENTS
   if (fillsize)

-----------------------------------------------------------------------

Summary of changes:
 src/main.c |   17 ++++-------------
 1 files changed, 4 insertions(+), 13 deletions(-)


hooks/post-receive
-- 
grep


From MAILER-DAEMON Sat Jun 02 05:06:37 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1SakI9-0006qS-Fs
	for mharc-grep-commit@gnu.org; Sat, 02 Jun 2012 05:06:37 -0400
Received: from eggs.gnu.org ([208.118.235.92]:48754)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1SakI5-0006pV-AK
	for grep-commit@gnu.org; Sat, 02 Jun 2012 05:06:35 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1SakI2-00022Y-Cn
	for grep-commit@gnu.org; Sat, 02 Jun 2012 05:06:32 -0400
Received: from vcs.savannah.gnu.org ([140.186.70.72]:44018)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1SakI2-00021L-7k
	for grep-commit@gnu.org; Sat, 02 Jun 2012 05:06:30 -0400
Received: from meyering by vcs.savannah.gnu.org with local (Exim 4.72)
	(envelope-from <jim@meyering.net>) id 1SakHz-0004yp-62
	for grep-commit@gnu.org; Sat, 02 Jun 2012 09:06:27 +0000
Date: Sat, 02 Jun 2012 09:06:27 +0000
Message-Id: <E1SakHz-0004yp-62@vcs.savannah.gnu.org>
To: grep-commit@gnu.org
Subject: grep branch, master, updated. v2.12-16-g7aa698d
X-Git-Refname: refs/heads/master
X-Git-Reftype: branch
X-Git-Oldrev: 2665746b756bd372ba856e165388dc98032362fd
X-Git-Newrev: 7aa698d36b5b2eeb8e90e7a327eb7ebe46d59e87
From: Jim Meyering <jim@meyering.net>
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 2)
X-Received-From: 140.186.70.72
X-BeenThere: grep-commit@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
Reply-To: bug-grep@gnu.org
List-Id: CVS commit notices for GNU grep <grep-commit.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/grep-commit>,
	<mailto:grep-commit-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/grep-commit>
List-Post: <mailto:grep-commit@gnu.org>
List-Help: <mailto:grep-commit-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/grep-commit>,
	<mailto:grep-commit-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sat, 02 Jun 2012 09:06:35 -0000

This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "grep".

The branch, master has been updated
       via  7aa698d36b5b2eeb8e90e7a327eb7ebe46d59e87 (commit)
      from  2665746b756bd372ba856e165388dc98032362fd (commit)

Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.

- Log -----------------------------------------------------------------
http://git.savannah.gnu.org/cgit/grep.git/commit/?id=7aa698d36b5b2eeb8e90e7a327eb7ebe46d59e87


commit 7aa698d36b5b2eeb8e90e7a327eb7ebe46d59e87
Author: Jim Meyering <meyering@redhat.com>
Date:   Fri Jun 1 21:18:00 2012 +0200

    grep: fix how -i works with a match containing the Turkish I-with-dot
    
    Fix a long-standing problem in the way grep's -i interacts with
    data whose byte count changes when we convert it to lower case.
    For example, the UTF-8 Turkish I-with-dot (İ) occupies two bytes,
    but its lower case analog, i, occupies just one byte.  The code
    converts both search string and the haystack data to lower case,
    and then searches for the modified string in the modified buffer.
    The trouble arose when using a lowercase buffer <offset,length>
    pair to manipulate the original (longer) buffer.
    
    The solution is to change mbtolower to return additional information:
    a malloc'd mapping vector.  With that, the caller maps the lowercase-
    relative <offset,length> to numbers that refer to the original buffer.
    This mapping is used only when lengths actually differ, so the cost
    in general should be small.
    
    * src/searchutils.c (mbtolower): Add the new map parameter.
    * src/search.h (mb_case_map_apply): New function.
    * src/kwsearch.c (Fexecute): Update mbtolower caller, and upon
    success, apply the new map.
    * src/dfasearch.c (EGexecute): Likewise.
    * tests/Makefile.am (XFAIL_TESTS): Remove turkish-I from this list;
    that test is no longer expected to fail.
    * NEWS (Bug fixes): Mention it.
    Reported by Ilya Basin in
    http://thread.gmane.org/gmane.comp.gnu.grep.bugs/3413 and later
    by Strahinja Kustudic in http://savannah.gnu.org/bugs/?36567

diff --git a/NEWS b/NEWS
index 6926276..d0ea60a 100644
--- a/NEWS
+++ b/NEWS
@@ -4,6 +4,11 @@ GNU grep NEWS                                    -*- outline -*-
 
 ** Bug fixes
 
+  grep -i, in a multi-byte locale, when matching a line containing a character
+  like the UTF-8 Turkish I-with-dot (U+0130) (whose lower-case representation
+  occupies fewer bytes), would print an incomplete output line.
+  [bug introduced in grep-2.6]
+
   --include and --exclude can again be combined, and again apply to
   the command line, e.g., "grep --include='*.[ch]' --exclude='system.h'
   PATTERN *" again reads all *.c and *.h files except for system.h.
diff --git a/THANKS b/THANKS
index 80df5ec..1720232 100644
--- a/THANKS
+++ b/THANKS
@@ -84,6 +84,7 @@ Shannon Hill               <hill@synnet.com>
 Sotiris Vassilopoulos      <Sotiris.Vassilopoulos@betatech.gr>
 Standish Parsley           <adsspamtrap01@yahoo.com>
 Stewart Levin              <stew@sep.stanford.edu>
+Strahinja Kustudic         <kustodian@gmail.com>
 Sven Joachim               <svenjoac@gmx.de>
 Sydoruk Stepan             <step@unitex.kiev.ua>
 Tapani Tarvainen           <tt@mit.jyu.fi>
diff --git a/src/dfasearch.c b/src/dfasearch.c
index bd09aa6..a48333d 100644
--- a/src/dfasearch.c
+++ b/src/dfasearch.c
@@ -78,8 +78,9 @@ static char const *
 kwsincr_case (const char *must)
 {
   size_t n = strlen (must);
+  unsigned char *map = NULL;
   const char *buf = (match_icase && MB_CUR_MAX > 1
-                     ? mbtolower (must, &n)
+                     ? mbtolower (must, &n, &map)
                      : must);
   return kwsincr (kwset, buf, n);
 }
@@ -217,13 +218,15 @@ EGexecute (char const *buf, size_t size, size_t *match_size,
   ptrdiff_t len, best_len;
   struct kwsmatch kwsm;
   size_t i, ret_val;
+  unsigned char *map = NULL;
+
   if (MB_CUR_MAX > 1)
     {
       if (match_icase)
         {
           /* mbtolower adds a NUL byte at the end.  That will provide
              space for the sentinel byte dfaexec may add.  */
-          char *case_buf = mbtolower (buf, &size);
+          char *case_buf = mbtolower (buf, &size, &map);
           if (start_ptr)
             start_ptr = case_buf + (start_ptr - buf);
           buf = case_buf;
@@ -408,9 +411,11 @@ EGexecute (char const *buf, size_t size, size_t *match_size,
 
  success:
   len = end - beg;
- success_in_len:
+ success_in_len:;
+  size_t off = beg - buf;
+  mb_case_map_apply (map, &off, &len);
   *match_size = len;
-  ret_val = beg - buf;
+  ret_val = off;
  out:
   return ret_val;
 }
diff --git a/src/kwsearch.c b/src/kwsearch.c
index f1a802e..d0bb201 100644
--- a/src/kwsearch.c
+++ b/src/kwsearch.c
@@ -34,8 +34,9 @@ Fcompile (char const *pattern, size_t size)
 {
   char const *err;
   size_t psize = size;
+  unsigned char *map = NULL;
   char const *pat = (match_icase && MB_CUR_MAX > 1
-                     ? mbtolower (pattern, &psize)
+                     ? mbtolower (pattern, &psize, &map)
                      : pattern);
 
   kwsinit (&kwset);
@@ -83,11 +84,13 @@ Fexecute (char const *buf, size_t size, size_t *match_size,
   char eol = eolbyte;
   struct kwsmatch kwsmatch;
   size_t ret_val;
+  unsigned char *map = NULL;
+
   if (MB_CUR_MAX > 1)
     {
       if (match_icase)
         {
-          char *case_buf = mbtolower (buf, &size);
+          char *case_buf = mbtolower (buf, &size, &map);
           if (start_ptr)
             start_ptr = case_buf + (start_ptr - buf);
           buf = case_buf;
@@ -162,9 +165,12 @@ Fexecute (char const *buf, size_t size, size_t *match_size,
   while (buf < beg && beg[-1] != eol)
     --beg;
   len = end - beg;
- success_in_beg_and_len:
+ success_in_beg_and_len:;
+  size_t off = beg - buf;
+  mb_case_map_apply (map, &off, &len);
+
   *match_size = len;
-  ret_val = beg - buf;
+  ret_val = off;
  out:
   return ret_val;
 }
diff --git a/src/search.h b/src/search.h
index 3074407..529e7e2 100644
--- a/src/search.h
+++ b/src/search.h
@@ -38,7 +38,7 @@
 /* searchutils.c */
 extern void kwsinit (kwset_t *);
 
-extern char *mbtolower (const char *, size_t *);
+extern char *mbtolower (const char *, size_t *, unsigned char **);
 extern bool is_mb_middle (const char **, const char *, const char *, size_t);
 
 /* dfasearch.c */
@@ -53,4 +53,23 @@ extern size_t Fexecute (char const *, size_t, size_t *, char const *);
 extern void Pcompile (char const *, size_t);
 extern size_t Pexecute (char const *, size_t, size_t *, char const *);
 
+/* Apply a non-NULL MAP from mbtolower to the lowercase-buffer-relative
+   *OFF and *LEN, converting them to be relative to the original buffer.  */
+static inline void
+mb_case_map_apply (unsigned char const *map, size_t *off, size_t *len)
+{
+  if (map)
+    {
+      size_t off_incr = 0;
+      size_t len_incr = 0;
+      size_t k;
+      for (k = 0; k < *off; k++)
+        off_incr += map[k];
+      for (k = *off; k < *off + *len; k++)
+        len_incr += map[k];
+      *off += off_incr;
+      *len += len_incr;
+    }
+}
+
 #endif /* GREP_SEARCH_H */
diff --git a/src/searchutils.c b/src/searchutils.c
index b787fe6..4942c51 100644
--- a/src/searchutils.c
+++ b/src/searchutils.c
@@ -53,25 +53,38 @@ kwsinit (kwset_t *kwset)
    Note that while this function returns a pointer to malloc'd storage,
    the caller must not free it, since this function retains a pointer
    to the buffer and reuses it on any subsequent call.  As a consequence,
-   this function is not thread-safe.  */
+   this function is not thread-safe.
+
+   When the lowercase result string has the same length as the input string,
+   set *LEN_MAP_P to NULL.  Otherwise, set it to a malloc'd buffer (like the
+   returned buffer, this must not be freed by caller) of the same length as
+   the result string.  (*LEN_MAP_P)[J] is one less than the length-in-bytes
+   of the character in BEG that formed byte J of the result.  This map is
+   used by the caller to convert offset,length pairs that reference the
+   lowercase result to numbers that refer to the corresponding parts of
+   the original buffer.  */
 char *
-mbtolower (const char *beg, size_t *n)
+mbtolower (const char *beg, size_t *n, unsigned char **len_map_p)
 {
   static char *out;
+  static unsigned char *len_map;
   static size_t outalloc;
   size_t outlen, mb_cur_max;
   mbstate_t is, os;
   const char *end;
   char *p;
+  unsigned char *m;
 
   if (*n > outalloc || outalloc == 0)
     {
       outalloc = MAX(1, *n);
       out = xrealloc (out, outalloc);
+      len_map = xrealloc (len_map, outalloc);
     }
 
   /* appease clang-2.6 */
   assert (out);
+  assert (len_map);
   if (*n == 0)
     return out;
 
@@ -81,6 +94,7 @@ mbtolower (const char *beg, size_t *n)
 
   mb_cur_max = MB_CUR_MAX;
   p = out;
+  m = len_map;
   outlen = 0;
   while (beg < end)
     {
@@ -88,14 +102,18 @@ mbtolower (const char *beg, size_t *n)
       size_t mbclen = mbrtowc(&wc, beg, end - beg, &is);
       if (outlen + mb_cur_max >= outalloc)
         {
+          size_t dm = m - len_map;
           out = x2nrealloc (out, &outalloc, 1);
+          len_map = xrealloc (len_map, outalloc);
           p = out + outlen;
+          m = len_map + dm;
         }
 
       if (mbclen == (size_t) -1 || mbclen == (size_t) -2 || mbclen == 0)
         {
           /* An invalid sequence, or a truncated multi-octet character.
              We treat it as a single-octet character.  */
+          *m++ = 0;
           *p++ = *beg++;
           outlen++;
           memset (&is, 0, sizeof (is));
@@ -103,6 +121,7 @@ mbtolower (const char *beg, size_t *n)
         }
       else
         {
+          *m++ = mbclen - 1;
           beg += mbclen;
           mbclen = wcrtomb (p, towlower ((wint_t) wc), &os);
           p += mbclen;
@@ -110,6 +129,8 @@ mbtolower (const char *beg, size_t *n)
         }
     }
 
+  /* If the new length differs from the original, give caller the map.  */
+  *len_map_p = p - out == *n ? NULL : len_map;
   *n = p - out;
   *p = 0;
   return out;
diff --git a/tests/Makefile.am b/tests/Makefile.am
index 7be788c..167e318 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -32,7 +32,6 @@ XFAIL_TESTS = \
 if USE_INCLUDED_REGEX
 XFAIL_TESTS += equiv-classes
 endif
-XFAIL_TESTS += turkish-I
 
 TESTS =						\
   backref					\

-----------------------------------------------------------------------

Summary of changes:
 NEWS              |    5 +++++
 THANKS            |    1 +
 src/dfasearch.c   |   13 +++++++++----
 src/kwsearch.c    |   14 ++++++++++----
 src/search.h      |   21 ++++++++++++++++++++-
 src/searchutils.c |   25 +++++++++++++++++++++++--
 tests/Makefile.am |    1 -
 7 files changed, 68 insertions(+), 12 deletions(-)


hooks/post-receive
-- 
grep


From MAILER-DAEMON Tue Jun 05 05:17:54 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1Sbpti-0002zs-8F
	for mharc-grep-commit@gnu.org; Tue, 05 Jun 2012 05:17:54 -0400
Received: from eggs.gnu.org ([208.118.235.92]:57860)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Sbpta-0002z1-2f
	for grep-commit@gnu.org; Tue, 05 Jun 2012 05:17:53 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1SbptR-0003YZ-4g
	for grep-commit@gnu.org; Tue, 05 Jun 2012 05:17:45 -0400
Received: from vcs.savannah.gnu.org ([140.186.70.72]:45886)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1SbptR-0003YU-0h
	for grep-commit@gnu.org; Tue, 05 Jun 2012 05:17:37 -0400
Received: from meyering by vcs.savannah.gnu.org with local (Exim 4.72)
	(envelope-from <jim@meyering.net>) id 1SbptQ-0004Ew-Dv
	for grep-commit@gnu.org; Tue, 05 Jun 2012 09:17:36 +0000
Date: Tue, 05 Jun 2012 09:17:36 +0000
Message-Id: <E1SbptQ-0004Ew-Dv@vcs.savannah.gnu.org>
To: grep-commit@gnu.org
Subject: grep branch, master, updated. v2.12-18-g1e65a88
X-Git-Refname: refs/heads/master
X-Git-Reftype: branch
X-Git-Oldrev: 7aa698d36b5b2eeb8e90e7a327eb7ebe46d59e87
X-Git-Newrev: 1e65a88a9ebbb4db7cd17fea8382e22c136e2688
From: Jim Meyering <jim@meyering.net>
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 2)
X-Received-From: 140.186.70.72
X-BeenThere: grep-commit@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
Reply-To: bug-grep@gnu.org
List-Id: CVS commit notices for GNU grep <grep-commit.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/grep-commit>,
	<mailto:grep-commit-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/grep-commit>
List-Post: <mailto:grep-commit@gnu.org>
List-Help: <mailto:grep-commit-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/grep-commit>,
	<mailto:grep-commit-request@gnu.org?subject=subscribe>
X-List-Received-Date: Tue, 05 Jun 2012 09:17:53 -0000

This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "grep".

The branch, master has been updated
       via  1e65a88a9ebbb4db7cd17fea8382e22c136e2688 (commit)
       via  10646e6a936233daa1c77abb8902ba9bf3007ba0 (commit)
      from  7aa698d36b5b2eeb8e90e7a327eb7ebe46d59e87 (commit)

Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.

- Log -----------------------------------------------------------------
http://git.savannah.gnu.org/cgit/grep.git/commit/?id=1e65a88a9ebbb4db7cd17fea8382e22c136e2688


commit 1e65a88a9ebbb4db7cd17fea8382e22c136e2688
Author: Jim Meyering <meyering@redhat.com>
Date:   Mon Jun 4 10:17:12 2012 +0200

    build: fix a subtly twisted "make distcheck" failure
    
    "make distcheck" would fail when, during a test build,
    an attempt to overwrite the deliberately-write-protected
    $(srcdir)/grep.pot file would fail.
    * bootstrap.conf (bootstrap_epilogue): Don't let the existence of
    a large sparse file in the build directory induce "make distcheck"
    failure.  The existence of a large sparse test file named 8T-or-so
    would make po/Makefile.in.in's use of grep (to search for "GNU grep"
    as an indication that this is a GNU package) exit 2 without generating
    any output, which made the first xgettext use --package-name=grep,
    while that same search for "GNU grep" would succeed when run
    from a pristine from-tarball build, thus making the second
    xgettext invocation use --package-name='GNU grep'.
    That mismatch:
    -"Project-Id-Version: grep 2.12.18-1080\n"
    +"Project-Id-Version: GNU grep 2.12.18-1080\n"
    led to the attempt by Makefile.in.in's grep.pot-update rule to
    overwrite ../../grep.pot in the read-only po/ source directory.

diff --git a/bootstrap.conf b/bootstrap.conf
index 57749b4..2b3563c 100644
--- a/bootstrap.conf
+++ b/bootstrap.conf
@@ -129,4 +129,15 @@ tar        -
 bootstrap_epilogue()
 {
   perl -pi -e "s/\@PACKAGE\@/$package/g" README-release
+
+  # Since this is a "GNU" package, replace this line
+  #   if LC_ALL=C grep 'GNU @PACKAGE@' $(top_srcdir)/* 2>/dev/null \
+  #      | grep -v 'libtool:' >/dev/null; then
+  # with this:
+  #   if true; then
+  # Why?  That pipeline searches all files in $(top_srcdir), and if you
+  # happen to have large files (or apparently large sparse files), the
+  # first grep may well run out of memory.
+  perl -pi -e 's/ LC_ALL=C grep .GNU .PACKAGE.*; then/ true; then/' \
+    po/Makefile.in.in
 }

http://git.savannah.gnu.org/cgit/grep.git/commit/?id=10646e6a936233daa1c77abb8902ba9bf3007ba0


commit 1e65a88a9ebbb4db7cd17fea8382e22c136e2688
Author: Jim Meyering <meyering@redhat.com>
Date:   Mon Jun 4 10:17:12 2012 +0200

    build: fix a subtly twisted "make distcheck" failure
    
    "make distcheck" would fail when, during a test build,
    an attempt to overwrite the deliberately-write-protected
    $(srcdir)/grep.pot file would fail.
    * bootstrap.conf (bootstrap_epilogue): Don't let the existence of
    a large sparse file in the build directory induce "make distcheck"
    failure.  The existence of a large sparse test file named 8T-or-so
    would make po/Makefile.in.in's use of grep (to search for "GNU grep"
    as an indication that this is a GNU package) exit 2 without generating
    any output, which made the first xgettext use --package-name=grep,
    while that same search for "GNU grep" would succeed when run
    from a pristine from-tarball build, thus making the second
    xgettext invocation use --package-name='GNU grep'.
    That mismatch:
    -"Project-Id-Version: grep 2.12.18-1080\n"
    +"Project-Id-Version: GNU grep 2.12.18-1080\n"
    led to the attempt by Makefile.in.in's grep.pot-update rule to
    overwrite ../../grep.pot in the read-only po/ source directory.

diff --git a/bootstrap.conf b/bootstrap.conf
index 57749b4..2b3563c 100644
--- a/bootstrap.conf
+++ b/bootstrap.conf
@@ -129,4 +129,15 @@ tar        -
 bootstrap_epilogue()
 {
   perl -pi -e "s/\@PACKAGE\@/$package/g" README-release
+
+  # Since this is a "GNU" package, replace this line
+  #   if LC_ALL=C grep 'GNU @PACKAGE@' $(top_srcdir)/* 2>/dev/null \
+  #      | grep -v 'libtool:' >/dev/null; then
+  # with this:
+  #   if true; then
+  # Why?  That pipeline searches all files in $(top_srcdir), and if you
+  # happen to have large files (or apparently large sparse files), the
+  # first grep may well run out of memory.
+  perl -pi -e 's/ LC_ALL=C grep .GNU .PACKAGE.*; then/ true; then/' \
+    po/Makefile.in.in
 }

-----------------------------------------------------------------------

Summary of changes:
 bootstrap      |  159 ++++++++++++++++++++++++++++++++++++++------------------
 bootstrap.conf |   11 ++++
 cfg.mk         |    1 +
 gnulib         |    2 +-
 tests/init.sh  |    8 +++
 5 files changed, 130 insertions(+), 51 deletions(-)


hooks/post-receive
-- 
grep


From MAILER-DAEMON Fri Jun 08 02:36:13 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1Scsnt-0007hW-AY
	for mharc-grep-commit@gnu.org; Fri, 08 Jun 2012 02:36:13 -0400
Received: from eggs.gnu.org ([208.118.235.92]:37550)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Scsnq-0007h2-6V
	for grep-commit@gnu.org; Fri, 08 Jun 2012 02:36:11 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Scsno-0001l0-AH
	for grep-commit@gnu.org; Fri, 08 Jun 2012 02:36:09 -0400
Received: from vcs.savannah.gnu.org ([140.186.70.72]:59344)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Scsno-0001kS-6R
	for grep-commit@gnu.org; Fri, 08 Jun 2012 02:36:08 -0400
Received: from meyering by vcs.savannah.gnu.org with local (Exim 4.72)
	(envelope-from <jim@meyering.net>) id 1Scsng-0005yX-P3
	for grep-commit@gnu.org; Fri, 08 Jun 2012 06:36:06 +0000
Date: Fri, 08 Jun 2012 06:36:01 +0000
Message-Id: <E1Scsng-0005yX-P3@vcs.savannah.gnu.org>
To: grep-commit@gnu.org
Subject: grep branch, master, updated. v2.12-19-g02f1daa
X-Git-Refname: refs/heads/master
X-Git-Reftype: branch
X-Git-Oldrev: 1e65a88a9ebbb4db7cd17fea8382e22c136e2688
X-Git-Newrev: 02f1daa1e7dc80a544b7904d1d9d9841e02e3d8b
From: Jim Meyering <jim@meyering.net>
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 2)
X-Received-From: 140.186.70.72
X-BeenThere: grep-commit@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
Reply-To: bug-grep@gnu.org
List-Id: CVS commit notices for GNU grep <grep-commit.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/grep-commit>,
	<mailto:grep-commit-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/grep-commit>
List-Post: <mailto:grep-commit@gnu.org>
List-Help: <mailto:grep-commit-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/grep-commit>,
	<mailto:grep-commit-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 08 Jun 2012 06:36:11 -0000

This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "grep".

The branch, master has been updated
       via  02f1daa1e7dc80a544b7904d1d9d9841e02e3d8b (commit)
      from  1e65a88a9ebbb4db7cd17fea8382e22c136e2688 (commit)

Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.

- Log -----------------------------------------------------------------
http://git.savannah.gnu.org/cgit/grep.git/commit/?id=02f1daa1e7dc80a544b7904d1d9d9841e02e3d8b


commit 02f1daa1e7dc80a544b7904d1d9d9841e02e3d8b
Author: Jim Meyering <meyering@redhat.com>
Date:   Sun Jun 3 16:29:59 2012 +0200

    tests: extend coverage of dfa.c's match_mb_charset
    
    Add a test case to increase test coverage of part of dfa.c (the DFA
    matcher used by grep and gawk).  While thinking about removing the few
    remaining uses of strncpy in dfa.c, I found that none of the existing
    tests covered the 40+ lines of code at the end of match_mb_charset,
    so constructed this test case to demonstrate that it's not dead code
    * tests/dfa-coverage: New test, for improved coverage.
    * tests/Makefile.am (TESTS): Add it.

diff --git a/tests/Makefile.am b/tests/Makefile.am
index 167e318..388d48a 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -47,6 +47,7 @@ TESTS =						\
   case-fold-char-range				\
   case-fold-char-type				\
   char-class-multibyte				\
+  dfa-coverage					\
   dfa-heap-overrun				\
   dfaexec-multibyte				\
   empty						\
diff --git a/tests/dfa-coverage b/tests/dfa-coverage
new file mode 100755
index 0000000..3b4e7ed
--- /dev/null
+++ b/tests/dfa-coverage
@@ -0,0 +1,32 @@
+#!/bin/sh
+# Exercise the final reachable code in dfa.c's match_mb_charset.
+
+# Copyright (C) 2012 Free Software Foundation, Inc.
+
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+. "${srcdir=.}/init.sh"; path_prepend_ ../src
+
+require_en_utf8_locale_
+require_compiled_in_MB_support
+
+fail=0
+
+echo a > in || framework_failure_
+
+LC_ALL=en_US.UTF-8 grep -E '[^_]|$' in > out || fail=1
+
+compare out in || fail=1
+
+Exit $fail

-----------------------------------------------------------------------

Summary of changes:
 tests/Makefile.am                 |    1 +
 tests/{turkish-I => dfa-coverage} |    9 ++++-----
 2 files changed, 5 insertions(+), 5 deletions(-)
 copy tests/{turkish-I => dfa-coverage} (75%)


hooks/post-receive
-- 
grep


From MAILER-DAEMON Tue Jun 12 12:46:46 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1SeUEw-0006iJ-1z
	for mharc-grep-commit@gnu.org; Tue, 12 Jun 2012 12:46:46 -0400
Received: from eggs.gnu.org ([208.118.235.92]:54868)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eggert@cs.ucla.edu>) id 1SeUEo-0006i9-S8
	for grep-commit@gnu.org; Tue, 12 Jun 2012 12:46:45 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eggert@cs.ucla.edu>) id 1SeUEi-0007aO-Cj
	for grep-commit@gnu.org; Tue, 12 Jun 2012 12:46:38 -0400
Received: from vcs.savannah.gnu.org ([140.186.70.72]:60173)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eggert@cs.ucla.edu>) id 1SeUEi-0007aI-9U
	for grep-commit@gnu.org; Tue, 12 Jun 2012 12:46:32 -0400
Received: from eggert by vcs.savannah.gnu.org with local (Exim 4.72)
	(envelope-from <eggert@cs.ucla.edu>) id 1SeUEh-00038B-CS
	for grep-commit@gnu.org; Tue, 12 Jun 2012 16:46:31 +0000
Date: Tue, 12 Jun 2012 16:46:31 +0000
Message-Id: <E1SeUEh-00038B-CS@vcs.savannah.gnu.org>
To: grep-commit@gnu.org
Subject: grep branch, master, updated. v2.12-20-g074842d
X-Git-Refname: refs/heads/master
X-Git-Reftype: branch
X-Git-Oldrev: 02f1daa1e7dc80a544b7904d1d9d9841e02e3d8b
X-Git-Newrev: 074842d3e3054714a495252e582886f0e4ace4e4
From: Paul Eggert <eggert@cs.ucla.edu>
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 2)
X-Received-From: 140.186.70.72
X-BeenThere: grep-commit@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
Reply-To: bug-grep@gnu.org
List-Id: CVS commit notices for GNU grep <grep-commit.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/grep-commit>,
	<mailto:grep-commit-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/grep-commit>
List-Post: <mailto:grep-commit@gnu.org>
List-Help: <mailto:grep-commit-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/grep-commit>,
	<mailto:grep-commit-request@gnu.org?subject=subscribe>
X-List-Received-Date: Tue, 12 Jun 2012 16:46:45 -0000

This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "grep".

The branch, master has been updated
       via  074842d3e3054714a495252e582886f0e4ace4e4 (commit)
      from  02f1daa1e7dc80a544b7904d1d9d9841e02e3d8b (commit)

Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.

- Log -----------------------------------------------------------------
http://git.savannah.gnu.org/cgit/grep.git/commit/?id=074842d3e3054714a495252e582886f0e4ace4e4


commit 074842d3e3054714a495252e582886f0e4ace4e4
Author: Paul Eggert <eggert@cs.ucla.edu>
Date:   Tue Jun 12 08:46:18 2012 -0700

    grep: handle -i when chars differ in length but line does not
    
    * src/searchutils.c (mbtolower): Return the map back to the caller
    if any input character's length differs from the corresponding output
    character's, not merely if the total string length differs.
    Problem reported by Johannes Mercer in
    <http://lists.gnu.org/archive/html/bug-grep/2012-06/msg00029.html>.

diff --git a/src/searchutils.c b/src/searchutils.c
index 4942c51..c1fb656 100644
--- a/src/searchutils.c
+++ b/src/searchutils.c
@@ -55,7 +55,8 @@ kwsinit (kwset_t *kwset)
    to the buffer and reuses it on any subsequent call.  As a consequence,
    this function is not thread-safe.
 
-   When the lowercase result string has the same length as the input string,
+   When all the characters in the lowercase result string have the
+   same length as corresponding characters in the input string,
    set *LEN_MAP_P to NULL.  Otherwise, set it to a malloc'd buffer (like the
    returned buffer, this must not be freed by caller) of the same length as
    the result string.  (*LEN_MAP_P)[J] is one less than the length-in-bytes
@@ -74,6 +75,7 @@ mbtolower (const char *beg, size_t *n, unsigned char **len_map_p)
   const char *end;
   char *p;
   unsigned char *m;
+  bool lengths_differ = false;
 
   if (*n > outalloc || outalloc == 0)
     {
@@ -99,7 +101,7 @@ mbtolower (const char *beg, size_t *n, unsigned char **len_map_p)
   while (beg < end)
     {
       wchar_t wc;
-      size_t mbclen = mbrtowc(&wc, beg, end - beg, &is);
+      size_t mbclen = mbrtowc (&wc, beg, end - beg, &is);
       if (outlen + mb_cur_max >= outalloc)
         {
           size_t dm = m - len_map;
@@ -123,14 +125,14 @@ mbtolower (const char *beg, size_t *n, unsigned char **len_map_p)
         {
           *m++ = mbclen - 1;
           beg += mbclen;
-          mbclen = wcrtomb (p, towlower ((wint_t) wc), &os);
-          p += mbclen;
-          outlen += mbclen;
+          size_t ombclen = wcrtomb (p, towlower ((wint_t) wc), &os);
+          p += ombclen;
+          outlen += ombclen;
+          lengths_differ |= (mbclen != ombclen);
         }
     }
 
-  /* If the new length differs from the original, give caller the map.  */
-  *len_map_p = p - out == *n ? NULL : len_map;
+  *len_map_p = lengths_differ ? len_map : NULL;
   *n = p - out;
   *p = 0;
   return out;

-----------------------------------------------------------------------

Summary of changes:
 src/searchutils.c |   16 +++++++++-------
 1 files changed, 9 insertions(+), 7 deletions(-)


hooks/post-receive
-- 
grep


From MAILER-DAEMON Sat Jun 16 13:10:05 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1SfwVh-0002Aw-08
	for mharc-grep-commit@gnu.org; Sat, 16 Jun 2012 13:10:05 -0400
Received: from eggs.gnu.org ([208.118.235.92]:33499)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1SfwVd-00029L-22
	for grep-commit@gnu.org; Sat, 16 Jun 2012 13:10:03 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1SfwVa-00044h-Ab
	for grep-commit@gnu.org; Sat, 16 Jun 2012 13:10:00 -0400
Received: from vcs.savannah.gnu.org ([140.186.70.72]:56592)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1SfwVa-00044d-6W
	for grep-commit@gnu.org; Sat, 16 Jun 2012 13:09:58 -0400
Received: from meyering by vcs.savannah.gnu.org with local (Exim 4.72)
	(envelope-from <jim@meyering.net>) id 1SfwVY-0002OR-Un
	for grep-commit@gnu.org; Sat, 16 Jun 2012 17:09:58 +0000
Date: Sat, 16 Jun 2012 17:09:57 +0000
Message-Id: <E1SfwVY-0002OR-Un@vcs.savannah.gnu.org>
To: grep-commit@gnu.org
Subject: grep branch, master, updated. v2.12-21-g2be0c65
X-Git-Refname: refs/heads/master
X-Git-Reftype: branch
X-Git-Oldrev: 074842d3e3054714a495252e582886f0e4ace4e4
X-Git-Newrev: 2be0c6591ea5d56ee7fbc364228f24b85f569a5c
From: Jim Meyering <jim@meyering.net>
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 2)
X-Received-From: 140.186.70.72
X-BeenThere: grep-commit@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
Reply-To: bug-grep@gnu.org
List-Id: CVS commit notices for GNU grep <grep-commit.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/grep-commit>,
	<mailto:grep-commit-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/grep-commit>
List-Post: <mailto:grep-commit@gnu.org>
List-Help: <mailto:grep-commit-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/grep-commit>,
	<mailto:grep-commit-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sat, 16 Jun 2012 17:10:03 -0000

This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "grep".

The branch, master has been updated
       via  2be0c6591ea5d56ee7fbc364228f24b85f569a5c (commit)
      from  074842d3e3054714a495252e582886f0e4ace4e4 (commit)

Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.

- Log -----------------------------------------------------------------
http://git.savannah.gnu.org/cgit/grep.git/commit/?id=2be0c6591ea5d56ee7fbc364228f24b85f569a5c


commit 2be0c6591ea5d56ee7fbc364228f24b85f569a5c
Author: Jim Meyering <meyering@redhat.com>
Date:   Sat Jun 16 12:32:35 2012 +0200

    grep -i: work also when converting to lower-case inflates byte count
    
    Commit v2.12-16-g7aa698d addressed the case in which the lower-case
    representation of an input byte occupies fewer bytes than the original.
    However, even with commit v2.12-20-g074842d, grep -i would still
    misbehave when converting a character to lower-case increased its
    byte count.  The map-manipulation code assumed that the case conversion
    could only shrink the byte count.  With the consideration that it may
    also inflate it, the deltas recorded in the map array must be signed,
    and we must account for the one-to-two-or-more mapping when the
    original-to-lower-case conversion causes the byte count to increase.
    * src/searchutils.c (mbtolower): When a lower-case character occupies
    more than one byte, set its remaining map slots to zero.  Change the
    type of the map to be signed, and compute the change in character
    byte count as new_length - old_length.
    * src/search.h: Include <stdint.h>, for decl of intmax_t.
    (mb_case_map_apply): Adjust for signed increments:
    each map entry is now signed.
    (mb_len_map_t): Define type.  Thanks to Paul Eggert for noticing
    in review that using a bare "char" as the base type would be wrong on
    systems for which it is a signed type (as with gcc's -funsigned-char).
    * src/kwsearch.c (Fcompile, Fexecute): Likewise.
    * src/dfasearch.c (kwsincr_case, EGexecute): Likewise.
    * tests/turkish-I-without-dot: New test.  Thanks to Paolo Bonzini
    for the tip that in the tr_TR.utf8 locale, mapping "I" to lower case
    increases the character's byte count.
    * tests/Makefile.am (TESTS): Add it.
    * tests/init.cfg (require_tr_utf8_locale_): New function.
    * NEWS (Bug fixes): Expand the existing entry.

diff --git a/NEWS b/NEWS
index d0ea60a..499d18d 100644
--- a/NEWS
+++ b/NEWS
@@ -7,6 +7,9 @@ GNU grep NEWS                                    -*- outline -*-
   grep -i, in a multi-byte locale, when matching a line containing a character
   like the UTF-8 Turkish I-with-dot (U+0130) (whose lower-case representation
   occupies fewer bytes), would print an incomplete output line.
+  Similarly, with a matched line containing a character (e.g., the Latin
+  capital I in a Turkish UTF-8 locale), where the lower-case representation
+  occupies more bytes, grep could print garbage.
   [bug introduced in grep-2.6]
 
   --include and --exclude can again be combined, and again apply to
diff --git a/src/dfasearch.c b/src/dfasearch.c
index a48333d..ab30cd5 100644
--- a/src/dfasearch.c
+++ b/src/dfasearch.c
@@ -78,7 +78,7 @@ static char const *
 kwsincr_case (const char *must)
 {
   size_t n = strlen (must);
-  unsigned char *map = NULL;
+  mb_len_map_t *map = NULL;
   const char *buf = (match_icase && MB_CUR_MAX > 1
                      ? mbtolower (must, &n, &map)
                      : must);
@@ -218,7 +218,7 @@ EGexecute (char const *buf, size_t size, size_t *match_size,
   ptrdiff_t len, best_len;
   struct kwsmatch kwsm;
   size_t i, ret_val;
-  unsigned char *map = NULL;
+  mb_len_map_t *map = NULL;
 
   if (MB_CUR_MAX > 1)
     {
diff --git a/src/kwsearch.c b/src/kwsearch.c
index d0bb201..b56b465 100644
--- a/src/kwsearch.c
+++ b/src/kwsearch.c
@@ -34,7 +34,7 @@ Fcompile (char const *pattern, size_t size)
 {
   char const *err;
   size_t psize = size;
-  unsigned char *map = NULL;
+  mb_len_map_t *map = NULL;
   char const *pat = (match_icase && MB_CUR_MAX > 1
                      ? mbtolower (pattern, &psize, &map)
                      : pattern);
@@ -84,7 +84,7 @@ Fexecute (char const *buf, size_t size, size_t *match_size,
   char eol = eolbyte;
   struct kwsmatch kwsmatch;
   size_t ret_val;
-  unsigned char *map = NULL;
+  mb_len_map_t *map = NULL;
 
   if (MB_CUR_MAX > 1)
     {
diff --git a/src/search.h b/src/search.h
index 529e7e2..3820d7a 100644
--- a/src/search.h
+++ b/src/search.h
@@ -22,6 +22,7 @@
 #include <config.h>
 
 #include <sys/types.h>
+#include <stdint.h>
 
 #include "mbsupport.h"
 
@@ -35,10 +36,16 @@
 #include "kwset.h"
 #include "xalloc.h"
 
+/* This must be a signed type.  Each value is the difference in the size
+   of a character (in bytes) induced by converting to lower case.
+   The vast majority of values are 0, but a few are 1 or -1, so
+   technically, two bits may be sufficient.  */
+typedef signed char mb_len_map_t;
+
 /* searchutils.c */
 extern void kwsinit (kwset_t *);
 
-extern char *mbtolower (const char *, size_t *, unsigned char **);
+extern char *mbtolower (const char *, size_t *, mb_len_map_t **);
 extern bool is_mb_middle (const char **, const char *, const char *, size_t);
 
 /* dfasearch.c */
@@ -53,15 +60,15 @@ extern size_t Fexecute (char const *, size_t, size_t *, char const *);
 extern void Pcompile (char const *, size_t);
 extern size_t Pexecute (char const *, size_t, size_t *, char const *);
 
-/* Apply a non-NULL MAP from mbtolower to the lowercase-buffer-relative
+/* Apply the MAP (created by mbtolower) to the lowercase-buffer-relative
    *OFF and *LEN, converting them to be relative to the original buffer.  */
 static inline void
-mb_case_map_apply (unsigned char const *map, size_t *off, size_t *len)
+mb_case_map_apply (mb_len_map_t const *map, size_t *off, size_t *len)
 {
   if (map)
     {
-      size_t off_incr = 0;
-      size_t len_incr = 0;
+      intmax_t off_incr = 0;
+      intmax_t len_incr = 0;
       size_t k;
       for (k = 0; k < *off; k++)
         off_incr += map[k];
diff --git a/src/searchutils.c b/src/searchutils.c
index c1fb656..ca30134 100644
--- a/src/searchutils.c
+++ b/src/searchutils.c
@@ -43,10 +43,10 @@ kwsinit (kwset_t *kwset)
 }
 
 #if MBS_SUPPORT
-/* Convert the *N-byte string, BEG, to lowercase, and write the
+/* Convert the *N-byte string, BEG, to lower-case, and write the
    NUL-terminated result into malloc'd storage.  Upon success, set *N
    to the length (in bytes) of the resulting string (not including the
-   trailing NUL byte), and return a pointer to the lowercase string.
+   trailing NUL byte), and return a pointer to the lower-case string.
    Upon memory allocation failure, this function exits.
    Note that on input, *N must be larger than zero.
 
@@ -55,26 +55,35 @@ kwsinit (kwset_t *kwset)
    to the buffer and reuses it on any subsequent call.  As a consequence,
    this function is not thread-safe.
 
-   When all the characters in the lowercase result string have the
-   same length as corresponding characters in the input string,
-   set *LEN_MAP_P to NULL.  Otherwise, set it to a malloc'd buffer (like the
-   returned buffer, this must not be freed by caller) of the same length as
-   the result string.  (*LEN_MAP_P)[J] is one less than the length-in-bytes
-   of the character in BEG that formed byte J of the result.  This map is
-   used by the caller to convert offset,length pairs that reference the
-   lowercase result to numbers that refer to the corresponding parts of
-   the original buffer.  */
+   When each character in the lower-case result string has the same length
+   as the corresponding character in the input string, set *LEN_MAP_P
+   to NULL.  Otherwise, set it to a malloc'd buffer (like the returned
+   buffer, this must not be freed by caller) of the same length as the
+   result string.  (*LEN_MAP_P)[J] is the change in byte-length of the
+   character in BEG that formed byte J of the result as it was converted to
+   lower-case.  It is usually zero.  For the upper-case Turkish I-with-dot
+   it is -1, since the upper-case character occupies two bytes, while the
+   lower-case one occupies only one byte.  For the Turkish-I-without-dot
+   in the tr_TR.utf8 locale, it is 1 because the lower-case representation
+   is one byte longer than the original.  When that happens, we have two
+   or more slots in *LEN_MAP_P for each such character.  We store the
+   difference in the first one and 0's in any remaining slots.
+
+   This map is used by the caller to convert offset,length pairs that
+   reference the lower-case result to numbers that refer to the matched
+   part of the original buffer.  */
+
 char *
-mbtolower (const char *beg, size_t *n, unsigned char **len_map_p)
+mbtolower (const char *beg, size_t *n, mb_len_map_t **len_map_p)
 {
   static char *out;
-  static unsigned char *len_map;
+  static mb_len_map_t *len_map;
   static size_t outalloc;
   size_t outlen, mb_cur_max;
   mbstate_t is, os;
   const char *end;
   char *p;
-  unsigned char *m;
+  mb_len_map_t *m;
   bool lengths_differ = false;
 
   if (*n > outalloc || outalloc == 0)
@@ -123,9 +132,11 @@ mbtolower (const char *beg, size_t *n, unsigned char **len_map_p)
         }
       else
         {
-          *m++ = mbclen - 1;
           beg += mbclen;
           size_t ombclen = wcrtomb (p, towlower ((wint_t) wc), &os);
+          *m = mbclen - ombclen;
+          memset (m + 1, 0, ombclen - 1);
+          m += ombclen;
           p += ombclen;
           outlen += ombclen;
           lengths_differ |= (mbclen != ombclen);
diff --git a/tests/Makefile.am b/tests/Makefile.am
index 388d48a..7d95862 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -86,6 +86,7 @@ TESTS =						\
   status					\
   symlink					\
   turkish-I					\
+  turkish-I-without-dot				\
   warn-char-classes				\
   word-delim-multibyte				\
   word-multi-file				\
diff --git a/tests/init.cfg b/tests/init.cfg
index 408400c..3b6242f 100644
--- a/tests/init.cfg
+++ b/tests/init.cfg
@@ -62,6 +62,15 @@ require_en_utf8_locale_()
   esac
 }
 
+require_tr_utf8_locale_()
+{
+  path_prepend_ .
+  case $(get-mb-cur-max tr_TR.UTF-8) in
+    [3456]) ;;
+    *) skip_ 'en_US.UTF-8 locale not found' ;;
+  esac
+}
+
 require_ru_RU_koi8_r()
 {
   path_prepend_ .
diff --git a/tests/turkish-I-without-dot b/tests/turkish-I-without-dot
new file mode 100755
index 0000000..cab4ede
--- /dev/null
+++ b/tests/turkish-I-without-dot
@@ -0,0 +1,47 @@
+#!/bin/sh
+# grep -i would misbehave for any matched line containing a character
+# (like "I" in the tr_TR.utf8 locale) whose lower-case representation
+# occupies more bytes (two in this case, for 0xc4b1, aka U+0131).
+
+# Copyright (C) 2011-2012 Free Software Foundation, Inc.
+
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+. "${srcdir=.}/init.sh"; path_prepend_ ../src
+
+require_tr_utf8_locale_
+require_compiled_in_MB_support
+
+# Before this change, grep could print a lot of uninitialized memory:
+# $ printf "IIIIIII\n" > in
+# $ for i in $(seq 10); do LC_ALL=tr_TR.utf8 src/grep -i . in|wc -c; done
+# 760
+# 754
+# 585
+# 298
+# 273
+# 458
+# 660
+# 552
+# 936
+# 678
+
+fail=0
+
+printf "IIIIIII\n" > in || framework_failure_
+LC_ALL=tr_TR.utf8 grep -i .... in > out || fail=1
+
+compare out in || fail=1
+
+Exit $fail

-----------------------------------------------------------------------

Summary of changes:
 NEWS                                       |    3 ++
 src/dfasearch.c                            |    4 +-
 src/kwsearch.c                             |    4 +-
 src/search.h                               |   17 ++++++++---
 src/searchutils.c                          |   41 +++++++++++++++++----------
 tests/Makefile.am                          |    1 +
 tests/init.cfg                             |    9 ++++++
 tests/{turkish-I => turkish-I-without-dot} |   26 +++++++++++++----
 8 files changed, 75 insertions(+), 30 deletions(-)
 copy tests/{turkish-I => turkish-I-without-dot} (59%)


hooks/post-receive
-- 
grep


