From MAILER-DAEMON Thu Feb 22 11:21:05 2018
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1eotcH-0003sE-Bm
	for mharc-bug-diffutils@gnu.org; Thu, 22 Feb 2018 11:21:05 -0500
Received: from eggs.gnu.org ([2001:4830:134:3::10]:59980)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <Debian-debbugs@debbugs.gnu.org>) id 1eotcF-0003s0-Af
	for bug-diffutils@gnu.org; Thu, 22 Feb 2018 11:21:04 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <Debian-debbugs@debbugs.gnu.org>) id 1eotcE-0002Hi-Gs
	for bug-diffutils@gnu.org; Thu, 22 Feb 2018 11:21:03 -0500
Received: from debbugs.gnu.org ([208.118.235.43]:48801)
	by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)
	(Exim 4.71) (envelope-from <Debian-debbugs@debbugs.gnu.org>)
	id 1eotcE-0002Hc-DD
	for bug-diffutils@gnu.org; Thu, 22 Feb 2018 11:21:02 -0500
Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2)
	(envelope-from <Debian-debbugs@debbugs.gnu.org>) id 1eotcE-0002JR-7l
	for bug-diffutils@gnu.org; Thu, 22 Feb 2018 11:21:02 -0500
X-Loop: help-debbugs@gnu.org
Resent-From: Victor Porton <porton@narod.ru>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces@debbugs.gnu.org>
Resent-CC: bug-diffutils@gnu.org
Resent-Date: Thu, 22 Feb 2018 16:21:02 +0000
Resent-Message-ID: <handler.30574.B.15193164048788@debbugs.gnu.org>
Resent-Sender: help-debbugs@gnu.org
X-GNU-PR-Message: report 30574
X-GNU-PR-Package: diffutils
X-GNU-PR-Keywords: 
To: 30574@debbugs.gnu.org
X-Debbugs-Original-To: bug-diffutils@gnu.org
Received: via spool by submit@debbugs.gnu.org id=B.15193164048788
	(code B ref -1); Thu, 22 Feb 2018 16:21:02 +0000
Received: (at submit) by debbugs.gnu.org; 22 Feb 2018 16:20:04 +0000
Received: from localhost ([127.0.0.1]:56698 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces@debbugs.gnu.org>)
	id 1eotbH-0002Hf-Ku
	for submit@debbugs.gnu.org; Thu, 22 Feb 2018 11:20:03 -0500
Received: from eggs.gnu.org ([208.118.235.92]:40331)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <porton@narod.ru>) id 1eotbF-0002H5-Hp
	for submit@debbugs.gnu.org; Thu, 22 Feb 2018 11:20:02 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <porton@narod.ru>) id 1eotb9-000165-O1
	for submit@debbugs.gnu.org; Thu, 22 Feb 2018 11:19:56 -0500
Received: from lists.gnu.org ([2001:4830:134:3::11]:42791)
	by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32)
	(Exim 4.71) (envelope-from <porton@narod.ru>) id 1eotb9-00015o-L1
	for submit@debbugs.gnu.org; Thu, 22 Feb 2018 11:19:55 -0500
Received: from eggs.gnu.org ([2001:4830:134:3::10]:59504)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <porton@narod.ru>) id 1eotb8-0003MH-Kc
	for bug-diffutils@gnu.org; Thu, 22 Feb 2018 11:19:55 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <porton@narod.ru>) id 1eotb7-00013s-FS
	for bug-diffutils@gnu.org; Thu, 22 Feb 2018 11:19:54 -0500
Received: from forward100p.mail.yandex.net
	([2a02:6b8:0:1472:2741:0:8b7:100]:47305)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)
	(Exim 4.71) (envelope-from <porton@narod.ru>) id 1eotb7-000103-11
	for bug-diffutils@gnu.org; Thu, 22 Feb 2018 11:19:53 -0500
Received: from mxback14j.mail.yandex.net (mxback14j.mail.yandex.net
	[IPv6:2a02:6b8:0:1619::90])
	by forward100p.mail.yandex.net (Yandex) with ESMTP id 5A2465103AAD
	for <bug-diffutils@gnu.org>; Thu, 22 Feb 2018 19:19:48 +0300 (MSK)
Received: from smtp2j.mail.yandex.net (smtp2j.mail.yandex.net
	[2a02:6b8:0:801::ac])
	by mxback14j.mail.yandex.net (nwsmtp/Yandex) with ESMTP id
	QtLfFyEg4m-JmTewNoe; Thu, 22 Feb 2018 19:19:48 +0300
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=narod.ru; s=mail;
	t=1519316388; bh=93Iw8sH41pONgpmlTV4TC3D+uXgt5DLFhN2hNgf+xVg=;
	h=Message-ID:Subject:From:To:Date;
	b=Gqv0ig0dkllFAbDV0y777TV7jV2p3Evc4mxy+Va66rZSsuqBne0xfNR/xr38kRGcN
	fYVf2FOoH7GJm9YiFW8P+jcA5fMhuokXZ2Muu5YAHp0DAzF7o6C4OaiM1Q5CRUDMaq
	QrHHsvq3J7dd5xtu0ZDeR2QcTAoZo7ZAK9TvlCxE=
Received: by smtp2j.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id
	PKFXxGU8Us-JldS0Ro5; Thu, 22 Feb 2018 19:19:47 +0300
	(using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))
	(Client certificate not present)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=narod.ru; s=mail;
	t=1519316387; bh=93Iw8sH41pONgpmlTV4TC3D+uXgt5DLFhN2hNgf+xVg=;
	h=Message-ID:Subject:From:To:Date;
	b=dZFA8yTN1YlsIk6jga27X079Rvdm2Hn4pd/8FTPDb65GlMx35ehhEtOf3suIWiHhz
	AgwJxrvOQ0rIcwDPQXCaKeY2TQJCJEkJz+9u2McZb8Nyr3cKzTMC2o23FzkmGe10eu
	qnO7wl713usghYIl5ws63NUa7KVZgXrZDG7rhqLc=
Authentication-Results: smtp2j.mail.yandex.net; dkim=pass header.i=@narod.ru
Message-ID: <1519316385.2225.34.camel@narod.ru>
From: Victor Porton <porton@narod.ru>
Date: Thu, 22 Feb 2018 18:19:45 +0200
Content-Type: text/plain; charset="UTF-8"
X-Mailer: Evolution 3.26.5-1 
Mime-Version: 1.0
Content-Transfer-Encoding: 7bit
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x
X-BeenThere: debbugs-submit@debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
Errors-To: debbugs-submit-bounces@debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces@debbugs.gnu.org>
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
X-Received-From: 208.118.235.43
Subject: [bug-diffutils] bug#30574: Support different charsets
X-BeenThere: bug-diffutils@gnu.org
List-Id: "All diffutils discussion." <bug-diffutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/bug-diffutils>,
	<mailto:bug-diffutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/bug-diffutils/>
List-Post: <mailto:bug-diffutils@gnu.org>
List-Help: <mailto:bug-diffutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/bug-diffutils>,
	<mailto:bug-diffutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 22 Feb 2018 16:21:04 -0000

1. We should be able to do `diff` even for source and destination
having different encodings and/or charsets.

Add option for source encoding and destination encoding.

2. The conversion to common encoding should be even with the option to
not fail on wrong or unconvertible characters (like //IGNORE in GNU
iconv), that is replacing unknown characters with a placeholder
character. This is useful to compare wrongly encoded files.

3. More generally we can add filter for every compared file first to
pass through the filter. Item 1 can be implemented by passing `iconv`
or `recode` command as such a filter.




