[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
bug#81762: [PATCH] join,uniq: fold multibyte letters with -i
|
From: |
Collin Funk |
|
Subject: |
bug#81762: [PATCH] join,uniq: fold multibyte letters with -i |
|
Date: |
Mon, 31 Aug 2026 10:40:38 -0700 |
|
User-agent: |
Gnus/5.13 (Gnus v5.13) |
Iván Ezequiel Rodriguez <ivanrwcm25@gmail.com> writes:
> join -i and uniq -i used memcasecmp, which only folds ASCII A-Z, so
> in UTF-8 locales É and é did not match despite -i. That matches the
> long-standing FIXME in join's keycmp.
>
> In multibyte locales, compare c32tolower of each mcel character
> (same approach as gnulib exclude). Unibyte locales keep the historical
> memcasecmp + length behavior. As before, -i does not use LC_COLLATE.
>
> This is simple codepoint folding, not full Unicode case mapping:
> É matches é, but ß and SS remain distinct.
>
> * src/memcasecmp-mcel.h (memcasecmp_fields): New shared helper.
> * src/join.c (keycmp): Use it for -i.
> * src/uniq.c (different): Likewise.
> * src/local.mk: List the new header.
> * tests/join/join.pl: Cover É/é join and ß/SS non-join.
> * tests/uniq/uniq.pl: Cover É/é collapse and ß/SS distinct.
> * NEWS: Mention the improvement.
>
> Signed-off-by: Iván Ezequiel Rodriguez <ivanrwcm25@gmail.com>
> ---
> NEWS | 5 ++++
> src/join.c | 17 +++++---------
> src/local.mk | 1 +
> src/memcasecmp-mcel.h | 54 +++++++++++++++++++++++++++++++++++++++++++
> src/uniq.c | 6 ++---
> tests/join/join.pl | 18 +++++++++++++++
> tests/uniq/uniq.pl | 13 ++++++++++-
> 7 files changed, 99 insertions(+), 15 deletions(-)
> create mode 100644 src/memcasecmp-mcel.h
I think I messaged an equivelent patch for 'uniq -i' privately (or maybe
publically, I forget) to Padraig a few months ago. But we decided to
hold off for now.
Collin