From MAILER-DAEMON Mon Jul 17 09:27:59 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dX647-00040K-DN
	for mharc-groff@gnu.org; Mon, 17 Jul 2017 09:27:59 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:57088)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <Anindo.Burman@wellsfargo.com>) id 1dX48U-0001w6-Gk
	for groff@gnu.org; Mon, 17 Jul 2017 07:24:23 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <Anindo.Burman@wellsfargo.com>) id 1dX48R-0004QR-6e
	for groff@gnu.org; Mon, 17 Jul 2017 07:24:22 -0400
Received: from mxdcmx04.wellsfargo.com ([159.45.87.72]:17973)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)
	(Exim 4.71) (envelope-from <Anindo.Burman@wellsfargo.com>)
	id 1dX48Q-0004QD-Vp; Mon, 17 Jul 2017 07:24:19 -0400
Received: from mxidli04.wellsfargo.com (mxidli04.wellsfargo.com
	[162.103.181.28])
	by mxdcmx04.wellsfargo.com (Sentrion-MTA-4.3.2/Sentrion-MTA-4.3.2) with
	ESMTP id v6HBN5SR017871; Mon, 17 Jul 2017 11:24:17 GMT
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wellsfargo.com;
	s=2011-05-wfb; t=1500290657;
	bh=8zMNvuK3Di1K7500BbgoyXg7FrBeLCMiP68JlDKFye8=;
	h=From:To:Subject:Date;
	b=U6dk4cIOoV7ujNpSzLFPG5HcLR+NaAoRky8a4b3Qt/X2E1bTrZiup5K+aFpIfDeBM
	oKgaXi94s8JXkpEepLKlt8CYFW/gRX8/jNaDQKBafB29Ac9aJK4d4SDb3wrZySFzcq
	J87VGnuutMNdAdP4xhb2xwoly1dBg28jxvG3fsOM=
Received: from mxidlx01.wellsfargo.com ( [162.102.225.32])
	by mxidli04.wellsfargo.com (postmaster@wellsfargo.com) with SMTP id
	00.AA.18274.16E9C695; Mon, 17 Jul 2017 11:24:17 +0000 (GMT)
X-AuditID: a267b51c-b51ff70000004762-ba-596c9e61b2f5
Received: from mxicmi04.wellsfargo.com ( [162.103.232.105])
	by mxidlx01.wellsfargo.com (postmaster@wellsfargo.com) with SMTP id
	30.D4.34527.06E9C695; Mon, 17 Jul 2017 11:24:17 +0000 (GMT)
Received: from MSGEXSIL4111.ent.wfb.bank.corp (msgexsil4111.wellsfargo.com
	[162.103.224.43])
	by mxicmi04.wellsfargo.com (Sentrion-MTA-4.3.2/Sentrion-MTA-4.3.2) with
	ESMTP id v6HBOECu009850; Mon, 17 Jul 2017 11:24:15 GMT
Received: from MSGEXSIL1143.ent.wfb.bank.corp ([169.254.1.112]) by
	MSGEXSIL4111.ent.wfb.bank.corp ([162.103.224.43]) with mapi id
	14.03.0319.002; Mon, 17 Jul 2017 07:24:14 -0400
From: <Anindo.Burman@wellsfargo.com>
To: <groff@gnu.org>, <maintainers@gnu.org>
Thread-Topic: GNU troff (groff)  help
Thread-Index: AdL+7zcVEZbjSC5vQL+PmGdIUvWt3g==
Date: Mon, 17 Jul 2017 11:24:14 +0000
Message-ID: <754C2B80AFF3D1498BB7B101B6CE3D7A0F4DC8@MSGEXSIL1143.ent.wfb.bank.corp>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [170.13.180.72]
MIME-Version: 1.0
X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrNIsWRmVeSWpSXmKPExsWyKH3fHN3EeTmRBiv/8Vu8OHaf1aJ94Uxm
	ByaPtmlmAYxRijYpqTmZZalF+nY2SZUFicXFuslpCok5ObZKJUWlqUr6dgmKGRN3zGYtOCta
	sW7udtYGxu9CXYycHBICJhILJi5j7mLk4hAS2MMo8XJmOyNMYs/bNVCJvYwS9zatZIFw7jNK
	fPrxkhmkSkhgK6NE24ncLkYODjYBFYmrh0tAwiICWhJ9kzewg9jCAgoSM3/dYoeIq0p0XjnB
	AmHrSbSfeQq2jAUovmbjSSYQm1cgWOLXqsdgNYwCYhLfT60BizMLiEvcejKfCeI4AYkle84z
	Q9iiEi8f/2OFsBUlLvx7DFWfLzGx5xU7xExBiZMzn4DdLyHQzC4xd/ZGFpCbJQSEJE79MYPo
	lZQ4uOIGywRG8VlI1s1CMmoWklEQcR2JBbs/sUHY2hLLFr5mhrHPHHjMhCy+gJF9FaN4bkVm
	Sk6mgYleeWpOTnFaYlF6vl5yfu4mRnAcb5XZwXhynvMhRgEORiUe3sknz0cIsSaWFVfmHmI0
	BQbSRGYp0eR8YLLIK4k3NDGwMLY0MTUyNLU0URLn1ZlqGikkkJ5YkpqdmlqQWhRfVJqTWnyI
	kYmDU6qBsb/ll9Xs7uKe2GCx8IP+OnNTtqj3Lr5wK+Fg4cydF23U15ybGHPtysfb3b+LVNjE
	lr+xnfhetHGBR7Wwc3ubkskv9vrnC9svmlucZmn2X1L8ovstj5jBocp9qysV3wc+/nHl3pzL
	enMOHHM+/2vGuUk/LRj95I4bu54ry1OX3XR2QvaNWQ8OrVJiKc5INNRiLipOBADtTGHC3gIA
	AA==
X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrDIsWRmVeSWpSXmKPExsWyKP1Fpm7ivJxIg1k6Fi+O3We1aF84k9mB
	yaNtmlkAY5SDTUZqYkpqkUJqXnJ+SmZeuq2SZ7C/roWFqaWuoZJCXmJuqq1ShG64m5Oui0+A
	rqt7sJJCWWJOKVA00jVYSd8uwSFj4o7ZrAVnRSvWzd3O2sD4XaiLkZNDQsBEYs/bNcxdjFwc
	QgJ7GSXubVrJAuHcZ5T49OMlM0iVkMBWRom2E7ldjBwcbAIqElcPl4CERQS0JPomb2AHsYUF
	FCRm/rrFDhFXlei8coIFwtaTaD/zlBHEZgGKr9l4kgnE5hUIlvi16jFYDaOAmMT3U2vA4swC
	4hK3nsxngjhOQGLJnvPMELaoxMvH/1ghbEWJC/8eQ9XnS0zsecUOMVNQ4uTMJywTGIVmIRk1
	C0nZLCRlEHEdiQW7P7FB2NoSyxa+Zoaxzxx4zIQsvoCRfRWjeG5FZkpOhYGhXnlqTk5xWmJR
	er5ecn7uJkZQJKY9VNjB+GeR8yFGAQ5GJR7eySfPRwixJpYVV+YeYjQFBsBEZinR5HxgvOeV
	xBuaGFgYW5pZACPRyExJnPfqNNNIIYH0xJLU7NTUgtSi+KLSnNTiQ4xMHJxSDYxXxS+7bMhQ
	EHA4n1voU/7MkP3oItnWf/9+qaRO/h3E0R+6uzxsg6LYnZm3xA9LPHrW8ror8B7vQx2H8473
	fizb15cUtkvS5MDqTknLA8clAqP1uvVSA3Rt02Wl3ywoFfcW1lT5lBVb57KSo0LR/04z78IT
	ZjUJR5e3uQbOtOMwYD+mal0nqMRSnJFoqMVcVJwIAI7bAke/AgAA
X-WFB-DLP-EGS: YES
X-CFilter-Loop: Reflected
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 159.45.87.72
X-Mailman-Approved-At: Mon, 17 Jul 2017 09:27:58 -0400
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
X-Content-Filtered-By: Mailman/MimeDel 2.1.21
Subject: [Groff] GNU troff (groff)  help
X-BeenThere: groff@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: <groff.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/groff>,
	<mailto:groff-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/groff/>
List-Post: <mailto:groff@gnu.org>
List-Help: <mailto:groff-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/groff>,
	<mailto:groff-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 17 Jul 2017 11:24:23 -0000

Hi,
 Can you please tell me the latest version of GNU troff (groff) and what is=
 the EOL  for this version, also can you please tell me the EOL of all the =
previous versions.

Regards,
Anindo Burman
Business Systems Consultant
Program Delivery and Governance - Application Competency Center - Product M=
anagement
Wells Fargo Centre, Embassy Tech Village (SEZ), Outer Ring Road, Bangalore =
560 103
Office: (612) 2560 615 | Cell: +91 98867 17633

Anindo.Burman@wellsfargo.com<mailto:Anindo.Burman@wellsfargo.com>










From MAILER-DAEMON Mon Jul 17 09:38:44 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dX6EW-0000eM-7f
	for mharc-groff@gnu.org; Mon, 17 Jul 2017 09:38:44 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:44510)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <werner.lemberg@gmx.de>) id 1dX6ES-0000bB-HK
	for groff@gnu.org; Mon, 17 Jul 2017 09:38:42 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <werner.lemberg@gmx.de>) id 1dX6EN-0002rv-Kz
	for groff@gnu.org; Mon, 17 Jul 2017 09:38:40 -0400
Received: from mout.gmx.net ([212.227.15.18]:64643)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16)
	(Exim 4.71) (envelope-from <werner.lemberg@gmx.de>)
	id 1dX6EN-0002qA-As; Mon, 17 Jul 2017 09:38:35 -0400
Received: from gnu.org ([178.115.128.116]) by mail.gmx.com (mrgmx001
	[212.227.17.190]) with ESMTPSA (Nemesis) id 0LiDKt-1e2XAb3kGp-00nTo1;
	Mon, 17 Jul 2017 15:38:31 +0200
Date: Mon, 17 Jul 2017 15:38:27 +0200 (CEST)
Message-Id: <20170717.153827.475591772044476760.wl@gnu.org>
To: Anindo.Burman@wellsfargo.com
Cc: groff@gnu.org, maintainers@gnu.org
From: Werner LEMBERG <wl@gnu.org>
In-Reply-To: <754C2B80AFF3D1498BB7B101B6CE3D7A0F4DC8@MSGEXSIL1143.ent.wfb.bank.corp>
References: <754C2B80AFF3D1498BB7B101B6CE3D7A0F4DC8@MSGEXSIL1143.ent.wfb.bank.corp>
X-Mailer: Mew version 6.7 on Emacs 26.0.50 / Mule 6.0 (HANACHIRUSATO)
Mime-Version: 1.0
Content-Type: Text/Plain; charset=utf-8
Content-Transfer-Encoding: base64
Sender: <werner.lemberg@gmx.de>
X-Provags-ID: V03:K0:O2ZlxlNNqVd7qhCMML74r53RLlUhnToCC4n/I6aC5EhpjQh+eee
	eiUaXDJugNJDKG7LKfnSltzT84CXP3WQTTkGggvZHFnsE13LYkseTmoauv/3R217hMEhd3f
	p3TxLyj8XI3cNyAapSt5gP4k2HAP0IcLYRmGIYnSgPuTxiMV3kMMTyk/loVLWp95Lc22VIn
	vu2qL9buQEmaI5viFmD3Q==
X-UI-Out-Filterresults: notjunk:1;V01:K0:swk3f6Hmsg0=:SRK+p/U3GRRU+veWrYw0JL
	pC6vPrK0qraPrBmXTZ220kXiyufOMUStONs7dpr4WeMVXntWVUguag6zHRtAcxWmMMN2BpIua
	R+5wv16LphKqC64qPQt1xr7a2At1bz+JJixcdH+DkezR6T/oTGq3wab6bWcJ40tvR98gX+Usw
	U9X5Dhd01D/2cImqYga2MJDI9IcGE81A0oYKY351zJ7e406JzTJcd+ACTLKf1lZViumG0G8Kt
	Nvfl/KeraHx9aNV2ngETrHA2ewQ/QN3p06YOMI8K6LDPRTFK4tkg5Ect/3OLHNZ+GmP8ZsMZ3
	g+5gJLXxYjwWaaVu6Py0AcKd7c4Xbkjg01flOIwu+V6ZIxaw6Kn4pM7AvR4ZnxVi9mcCUc+Wu
	sEM24K1mQnB0HjgT7XcDARE0MTnS0Ior8FlH3mCw086y04HXfNlTyiqdzmEwcUT6Z/yte2J5L
	zKFr6dLS7reoHmMXpTs5f+Iu/0yf6KoQBQSvMHHNJ3BDFpuR1xFAG3Owf7GTYiNyomI2Z4PIW
	I6owITW6xJxzEMlE5T4kVH5zM1tTE80J0k0P9k1gRb4JHvqJBVRGUCXndrNRj/mgCVmeDwLf/
	RJaYTOWhrS9VVRw9T6BwOfsgBaKnzek+grM3Tw6nxPA0mp8cczt3RwNb/XPDsIq20mVHnSNwX
	dN0kuxxJ9NL87j3+5zPcctvn8fWlwX0oqwdmHHh/7s2dxaVkRmtrl8Qz8wR8oPfKucfY548MO
	h/sBDj76F1ia7KHzKKbJ40IXrDxCPoLVd6JHOhw0T/2zfF1Igjj3vfRoiErhWEKQEydjRj8Re
	M0fvc6okjwRq1MqYnS06nWUEstovMav0n6vXBetkLFB9NJn3oA=
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
	[fuzzy]
X-Received-From: 212.227.15.18
Subject: Re: [Groff] GNU troff (groff) help
X-BeenThere: groff@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: <groff.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/groff>,
	<mailto:groff-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/groff/>
List-Post: <mailto:groff@gnu.org>
List-Help: <mailto:groff-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/groff>,
	<mailto:groff-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 17 Jul 2017 13:38:43 -0000

DQpIZWxsbyBBbmluZG8hDQoNCg0KPiBDYW4geW91IHBsZWFzZSB0ZWxsIG1lIHRoZSBsYXRlc3Qg
dmVyc2lvbiBvZiBHTlUgdHJvZmYgKGdyb2ZmKQ0KDQpUaGUgbGF0ZXN0IHB1Ymxpc2hlZCB2ZXJz
aW9uIGlzIDEuMjIuMyAoZnJvbSBOb3YuIDIwMTQpOyBzZWUNCg0KICBodHRwczovL3d3dy5nbnUu
b3JnL3NvZnR3YXJlL2dyb2ZmLw0KDQo+IGFuZCB3aGF0IGlzIHRoZSBFT0wgZm9yIHRoaXMgdmVy
c2lvbiwgYWxzbyBjYW4geW91IHBsZWFzZSB0ZWxsIG1lDQo+IHRoZSBFT0wgb2YgYWxsIHRoZSBw
cmV2aW91cyB2ZXJzaW9ucy4NCg0KSGVyZSBJIHNpbXBseSBxdW90ZSB3aGF0IEVyaWsgQmxha2Ug
aGFzIHdyaXR0ZW4gZm9yIE00LCBhbnN3ZXJpbmcgeW91cg0KaWRlbnRpY2FsIHF1ZXN0aW9uIG9u
IHRoZSBNNCBtYWlsaW5nIGxpc3QuDQoNCiAgVGhlIEVPTCBvZiBvcGVuIHNvdXJjZSBzb2Z0d2Fy
ZSBpcyBhIHRyaWNreSB0aGluZyB0byBwcmVkaWN0IOKAkyB0aGVyZQ0KICBpcyBubyBmaXhlZCBy
ZWxlYXNlIHNjaGVkdWxlLCBhbmQgdGhlcmVmb3JlLCBhbGwgdGhlIG1vcmUgd2UgY2FuDQogIHBy
b21pc2UgaXMgdGhhdCBhIHZlcnNpb24gaXMgbm90IEVPTCB1bnRpbCB0aGUgbmV4dCB2ZXJzaW9u
IGlzDQogIHJlbGVhc2VkLg0KDQoNCiAgICAgV2VybmVyDQo=


From MAILER-DAEMON Mon Jul 17 10:39:07 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dX7Ax-0000sp-P2
	for mharc-groff@gnu.org; Mon, 17 Jul 2017 10:39:07 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:34338)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <damianm@esi.com.au>) id 1dX7Au-0000se-5Z
	for groff@gnu.org; Mon, 17 Jul 2017 10:39:05 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <damianm@esi.com.au>) id 1dX7Ap-0005PI-AC
	for groff@gnu.org; Mon, 17 Jul 2017 10:39:04 -0400
Received: from key0.esi.com.au ([192.84.230.32]:38512)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <damianm@esi.com.au>) id 1dX7Ao-0005NN-Tw
	for groff@gnu.org; Mon, 17 Jul 2017 10:38:59 -0400
Received: from key0.esi.com.au (key0l.esi.com.au [127.0.0.1])
	by key0.esi.com.au (Postfix) with ESMTP id D7D9D12000C
	for <groff@gnu.org>; Tue, 18 Jul 2017 00:38:53 +1000 (AEST)
Received: from localhost (damianm@localhost)
	by key0.esi.com.au (8.14.4/8.14.4/Submit) with ESMTP id v6HEcqVa026922
	for <groff@gnu.org>; Tue, 18 Jul 2017 00:38:53 +1000
X-Authentication-Warning: key0.esi.com.au: damianm owned process doing -bs
Date: Tue, 18 Jul 2017 00:38:52 +1000 (AEST)
From: Damian McGuckin <damianm@esi.com.au>
To: groff@gnu.org
Message-ID: <alpine.LRH.2.02.1707172257130.23319@key0.esi.com.au>
User-Agent: Alpine 2.02 (LRH 1266 2009-07-14)
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy]
X-Received-From: 192.84.230.32
Subject: [Groff] Refer with MM
X-BeenThere: groff@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: <groff.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/groff>,
	<mailto:groff-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/groff/>
List-Post: <mailto:groff@gnu.org>
List-Help: <mailto:groff-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/groff>,
	<mailto:groff-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 17 Jul 2017 14:39:05 -0000


Hi,

The last time I used refer with MM was over a decade ago so my brain might 
be playing tricks.

There is a command used in macro .]- to explicitly clear the fields

 	.rm [A [B [C [D [E [G [I [J [N [O [P [Q [R [S [T [V

This means that the fields from one reference do not pollute the fields 
for the next reference.

However, if I have a volume (or other fields)

 	%V

in one reference, it persists into the next with '-mm' but not 'ms'. So,
something is broken.

Now, the common macro in refer.tmac is

.de ]-
.       rm [A [B [C [D [E [G [I [J [N [O [P [Q [R [S [T [V
.       rm ref*string
.       if d ref*item-start-hook \
.               ref*item-start-hook
..

Now ref*item-start-hook is beyond my groff skills but it does not exist in 
'-ms'.. If I move the first '.rm' to be 3 lines further I get

.de ]-
.       rm ref*string
.       if d ref*item-start-hook \
.               ref*item-start-hook
.       rm [A [B [C [D [E [G [I [J [N [O [P [Q [R [S [T [V
..

this variations appears to work for both '-ms' and '-me'.

What will I have broken?

The word 'appears' is crucial. I have used 'mm' for a long time, not 
groff. So no expert with commands like '.als' and '.am'.

A sample file follows and ends at the text 'Last line of file.' but simply

 	refer FILE | groff -me > x

works with the original refer.tmac. But

 	refer FILE | groff -mm > x

pollutes a reference with field from the previous reference, i.e. fails 
with the original refer.tmac but works perfectly for my small example with 
the changed one.

Any clues?

Thanks - Damian

.R1
move-punctuation
bracket-label " [" "]" ", "
join-authors ", and " ", " ", and "
accumulate
.R2
Perth.
.[
%A Groenenboom P.H.L.
%A Cartwright B.K.
%A McGuckin D.
%T Numerical Simulation of ships in High Seas using a Coupled SPH FEA Approach
%J Innovation in High Speed Marine Vessels
%I Royal Institute of Naval Architects (RINA)
%C Fremantle, WA, Australia
%D 28-29 January 2009
.]
Early.
.[
%A Groenenboom P.H.L.
%A Cartwright B.K.
%T Hydrodynamics and fluid-structure interaction by coupled SPH-FE method
%J Journal of Hydraulic Research
%V 48 (Extra Issue)
%P 61-73
%D 2010
.]
Later.
.[
%A Groenenboom P.H.L.
%A Cartwright B.K.
%T Pressure-corrected SPH with innovative particle regularization algorithms and non-uniform initial particle distributions
%J Proceedings of the 8th Int'l SPHERIC Workshop
%C Trondheim, Norway
%D 4-6 June 2013
.]
Last line of file.


From MAILER-DAEMON Mon Jul 17 12:17:18 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dX8hy-00089r-2v
	for mharc-groff@gnu.org; Mon, 17 Jul 2017 12:17:18 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:38632)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <damianm@esi.com.au>) id 1dX8hw-00088t-H7
	for groff@gnu.org; Mon, 17 Jul 2017 12:17:17 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <damianm@esi.com.au>) id 1dX8hr-0002C3-KL
	for groff@gnu.org; Mon, 17 Jul 2017 12:17:16 -0400
Received: from key0.esi.com.au ([192.84.230.32]:39379)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <damianm@esi.com.au>) id 1dX8hr-00029v-82
	for groff@gnu.org; Mon, 17 Jul 2017 12:17:11 -0400
Received: from key0.esi.com.au (key0l.esi.com.au [127.0.0.1])
	by key0.esi.com.au (Postfix) with ESMTP id AA3D112000E
	for <groff@gnu.org>; Tue, 18 Jul 2017 02:17:03 +1000 (AEST)
Received: from localhost (damianm@localhost)
	by key0.esi.com.au (8.14.4/8.14.4/Submit) with ESMTP id v6HGH14I030539
	for <groff@gnu.org>; Tue, 18 Jul 2017 02:17:03 +1000
X-Authentication-Warning: key0.esi.com.au: damianm owned process doing -bs
Date: Tue, 18 Jul 2017 02:17:01 +1000 (AEST)
From: Damian McGuckin <damianm@esi.com.au>
To: groff@gnu.org
Message-ID: <alpine.LRH.2.02.1707180122070.28656@key0.esi.com.au>
User-Agent: Alpine 2.02 (LRH 1266 2009-07-14)
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy]
X-Received-From: 192.84.230.32
Subject: [Groff] Hypenation of References generated by 'refer' with MM macros
X-BeenThere: groff@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: <groff.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/groff>,
	<mailto:groff-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/groff/>
List-Post: <mailto:groff@gnu.org>
List-Help: <mailto:groff-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/groff>,
	<mailto:groff-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 17 Jul 2017 16:17:17 -0000


I noticed that in a document whose hypenation is nominally controlled by
the MM-defined register

 	Hy

this value is ignored when processing references. Are references not 
counted as being part of the text-body? Or was this an oversight?

Regards - Damian

Pacific Engineering Systems International, 277-279 Broadway, Glebe NSW 2037
Ph:+61-2-8571-0847 .. Fx:+61-2-9692-9623 | unsolicited email not wanted here
Views & opinions here are mine and not those of any past or present employer


From MAILER-DAEMON Tue Jul 18 09:25:23 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dXSV9-0007Oz-1L
	for mharc-groff@gnu.org; Tue, 18 Jul 2017 09:25:23 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:55370)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <damianm@esi.com.au>) id 1dXSV7-0007Ng-Bi
	for groff@gnu.org; Tue, 18 Jul 2017 09:25:22 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <damianm@esi.com.au>) id 1dXSV4-0007zi-6G
	for groff@gnu.org; Tue, 18 Jul 2017 09:25:21 -0400
Received: from key0.esi.com.au ([192.84.230.32]:51967)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <damianm@esi.com.au>) id 1dXSV3-0007yT-QZ
	for groff@gnu.org; Tue, 18 Jul 2017 09:25:18 -0400
Received: from key0.esi.com.au (key0l.esi.com.au [127.0.0.1])
	by key0.esi.com.au (Postfix) with ESMTP id 900D712001D
	for <groff@gnu.org>; Tue, 18 Jul 2017 23:25:12 +1000 (AEST)
Received: from localhost (damianm@localhost)
	by key0.esi.com.au (8.14.4/8.14.4/Submit) with ESMTP id v6IDPAhr007174
	for <groff@gnu.org>; Tue, 18 Jul 2017 23:25:12 +1000
X-Authentication-Warning: key0.esi.com.au: damianm owned process doing -bs
Date: Tue, 18 Jul 2017 23:25:10 +1000 (AEST)
From: Damian McGuckin <damianm@esi.com.au>
To: groff@gnu.org
In-Reply-To: <alpine.LRH.2.02.1707172257130.23319@key0.esi.com.au>
Message-ID: <alpine.LRH.2.02.1707182310001.6858@key0.esi.com.au>
References: <alpine.LRH.2.02.1707172257130.23319@key0.esi.com.au>
User-Agent: Alpine 2.02 (LRH 1266 2009-07-14)
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy]
X-Received-From: 192.84.230.32
Subject: Re: [Groff] Refer with MM
X-BeenThere: groff@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: <groff.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/groff>,
	<mailto:groff-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/groff/>
List-Post: <mailto:groff@gnu.org>
List-Help: <mailto:groff-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/groff>,
	<mailto:groff-request@gnu.org?subject=subscribe>
X-List-Received-Date: Tue, 18 Jul 2017 13:25:22 -0000


I needed more flexibility with references when using the MM macros. I 
needed to modify the heading display, the hyphenation/font/line-spacing 
during the display of the references, and so on.

I have introduced the concept of 2 new macros, RZ and RH.  The 'RH' is a 
newly defined macro which will display the heading in font 2, centered on 
the page with a single line gap to the line.  By default, no 'RZ' macro 
is defined.

A user can define a macro 'RZ' which can be used to change hyphenation.
point size and interline spacing used to put the list of references on
the pages(s). E.g.

 		.de RZ
 		.S 12 14
 		.nh
 		..

A user can redefine the macro 'RH' which can be used to control the way
the heading \\[Rp] is displayed and the spacing between it and the list
of references. This heading text comes in as the first argument. E.g.

 		.de RH
 		.ft HB
 		\s+2\\$1\s-2
 		.ft P
 		.sp 2
 		..

It is probably not a good idea to leave fonts, point size and spacing any
different at exit as when it entered.

I have changed 'refer.tmac' by moving the two 'rm' commands in ']-' two 
lines down from their original lines. That change will only affect the MM 
macros.

Thare are changes to 2 files, 'm.tmac' and 'refer-mm.tmac'

I am testing. If anybody thinks this is useful, let me know.

Regards - Damian

Pacific Engineering Systems International, 277-279 Broadway, Glebe NSW 2037
Ph:+61-2-8571-0847 .. Fx:+61-2-9692-9623 | unsolicited email not wanted here
Views & opinions here are mine and not those of any past or present employer


From MAILER-DAEMON Thu Jul 20 15:50:07 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dYHSY-0000E0-Tc
	for mharc-groff@gnu.org; Thu, 20 Jul 2017 15:50:06 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:33349)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <erichhoffmann@gmx.de>) id 1dYHSW-0000D0-6p
	for groff@gnu.org; Thu, 20 Jul 2017 15:50:05 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <erichhoffmann@gmx.de>) id 1dYHST-0005Wv-Kg
	for groff@gnu.org; Thu, 20 Jul 2017 15:50:04 -0400
Received: from mout.gmx.net ([212.227.15.15]:56813)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16)
	(Exim 4.71) (envelope-from <erichhoffmann@gmx.de>)
	id 1dYHST-0005W7-AD
	for groff@gnu.org; Thu, 20 Jul 2017 15:50:01 -0400
Received: from e.h ([46.81.254.171]) by mail.gmx.com (mrgmx002
	[212.227.17.190]) with ESMTPSA (Nemesis) id 0MDhba-1dOyr22jZW-00HA9w
	for <groff@gnu.org>; Thu, 20 Jul 2017 21:49:58 +0200
Date: Thu, 20 Jul 2017 21:53:01 +0200
From: "E. Hoffmann" <erichhoffmann@gmx.de>
To: groff@gnu.org
Message-ID: <20170720215301.7b111801@e.h>
X-Mailer: Claws Mail 3.15.0-dirty (GTK+ 2.24.31; x86_64-slackware-linux-gnu)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
X-Provags-ID: V03:K0:LED/tZTPlNuBak4xqiSpwJMWf/7VPI3tqtykP0lYqv7Du/eRHUc
	1ocL9eihcTDGfjRIOj94+2Tidx4G2QBUTeP8Y4txSK6LRrbf/vSa8xbZP38RDIJpyH4GDTV
	yxo2QOnbhzFE9cVV+yKdcF5VgFkgTeXGW8IDsiyLaTbffXBDouMSScdBzOIKSc9zf/3ql/4
	5ZWIIu3N866gbQkJoc5qA==
X-UI-Out-Filterresults: notjunk:1;V01:K0:fIlYGBlEgKw=:rZGBOyyTo0X0u8VNXcXFsb
	tkmSnghyz/d8Y9SN8/7HqA7DF20O/vzq6ciNcCAXjDgP7DBDtPgVBuQqgN2tOs7oHFcs/fxfn
	2S7Ec5LYgqbF36UI34eNkGJEp1xQ+rg9mWj6IhcNwiXh9xkdZ1dV3kDOU4gEp72Xo5tDRLeue
	O3RIYQA8Kyz1Kd0H97HJzskGCNAElkTQxdGA46oSxupTvSbLrmbkyWdjbOfVucuK7NN3OmobC
	NvQ4wjMHEZwR8mo7Wq1dZV8v2TB+e8Vlgs6Wqam4N8d+d2VmZqY8sJ4bORaYBWYpU+VAUBvtS
	ZFE3ISCjc36jh5itVx9joJMHmNCdt/2BihWExp7OtIxbsKrAF/OWXgGNfWH4cAN4SpOUeAuS+
	TlxbccFo9eMWh9BWyJzwOfKA4y2sXXrDnmIF1UNRG1JGhjpcCBjjfdQU3Q3YE9rttWya9gl9G
	+DAOp9NKu8TJPwyEcvPQAg5qgVkvxEga67XYOruVOqLZleydu9Hc22SMRV+JfCmAInFxacp46
	SwtunOKLgl91CQTCoEZ2PmlbWEdfYtE2PlsEYALUT3gqDkFiZt2JdI9dso0Ta2RCCe1ePiWkP
	6L1O2Od4eQJZPPThj+5BT8dYISBcRYl0ZxEG6Iu4+rfCzzNrSlyqQUm4MajrIrK2RT1GsYkVs
	mqJNMBwlFmvpw/h48epnsAyaASwwjUim38R7nV15ezfs2vtU9DNb9kovE9FSD3Xsq5XxH2Q/p
	3Mabzr5IHHJS+TMyxOCUgr/9R3ZNy0+cWFeRqXupYPqlSodhq2A20799TOhXIDLCoQlL3tL8z
	H8jyKA0mTo4/ofJA55t9YAwYHx2WU3U5uukj7AdiRjHetxRWik=
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
	[fuzzy]
X-Received-From: 212.227.15.15
Subject: [Groff] mom : unicode in .INCLUDE'd files
X-BeenThere: groff@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: <groff.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/groff>,
	<mailto:groff-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/groff/>
List-Post: <mailto:groff@gnu.org>
List-Help: <mailto:groff-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/groff>,
	<mailto:groff-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 20 Jul 2017 19:50:05 -0000

Hello.

I'm going my first steps with mom.

There is a problem that I can't solve using the documentation, which
   very well may be just silly because I don't get it (which happened
   before ;)

When I enter unicode, like:

                         =C3=84=C3=96=C3=9C SS =C3=92=C3=93=C3=94=C3=95=C5=
=8E=C5=8C =C3=87 =C3=A4=C3=B6=C3=BC =C3=9F =C3=B2=C3=B3=C3=B4=C3=B5=C5=8F=
=C5=8D =C3=A7

...and process them with pdfmom, they show up perfectly.

But if I include the same characters in a file with the .INCLUDE macro,
they disappear.  Processed with -P-bcu -Tutf8, they show up
like wrong encoded strings.

Here is a minimal source code example.mom file:

---------------------------------------------\

.TITLE   "unicode example"
.AUTHOR  "me"
.CHAPTER  1
.DOCTYPE    DEFAULT
.PRINTSTYLE TYPESET
.START
.PP
UNICODE: =C3=84=C3=96=C3=9C SS =C3=92=C3=93=C3=94=C3=95=C5=8E=C5=8C =C3=87 =
=C3=A4=C3=B6=C3=BC =C3=9F =C3=B2=C3=B3=C3=B4=C3=B5=C5=8F=C5=8D =C3=A7
.PP
.PP
INCLUDED FILE BEGINS:
.PP
.INCLUDE file_with_identical_unicode_strings.mom
.EL

---------------------------------------------/

Command line:

               groff -mom -Tutf8 -P-bcu example.mom > example.txt

Output:

---------------------------------------------\

                                   unicode example
                                         by
                                         me

          UNICODE: =C3=84=C3=96=C3=9C SS =C3=92=C3=93=C3=94=C3=95=C5=8E=C5=
=8C =C3=87 =C3=A4=C3=B6=C3=BC =C3=9F =C3=B2=C3=B3=C3=B4=C3=B5=C5=8F=C5=8D =
=C3=A7

            INCLUDED FILE BEGINS:

            UNICODE: =C3=83=C3=83=C3=83 SS =C3=83=C3=83=C3=83=C3=83=C3=85=
=C3=85 =C3=83 =C3=83=C2=A4=C3=83=C2=B6=C3=83=C2=BC =C3=83 =C3=83=C2=B2=C3=
=83=C2=B3=C3=83=C2=B4=C3=83=C2=B5=C3=85=C3=85 =C3=83=C2=A7

---------------------------------------------/

As I said, processing to pdf with

    pdfmom -c example.mom > example.pdf

is similiar, only that the not recognized chars in the includefile just don=
't show up.


What am I doing wrong?

TIA,

Erich

PS

I tried, in vain, the following pipe:

 soelim example.mom | preconv -eutf8 | groff -mom -Tutf8 -P-bcu  > example.=
txt





From MAILER-DAEMON Thu Jul 20 17:17:42 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dYIpK-0004I9-EX
	for mharc-groff@gnu.org; Thu, 20 Jul 2017 17:17:42 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:51529)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <SRS0=SWoEHO=6X=schaffter.ca=peter@eigbox.net>)
	id 1dYIpG-0004Hn-R2
	for groff@gnu.org; Thu, 20 Jul 2017 17:17:39 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <SRS0=SWoEHO=6X=schaffter.ca=peter@eigbox.net>)
	id 1dYIpB-0000T4-R9
	for groff@gnu.org; Thu, 20 Jul 2017 17:17:38 -0400
Received: from bosmailout09.eigbox.net ([66.96.184.9]:59822)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)
	(Exim 4.71)
	(envelope-from <SRS0=SWoEHO=6X=schaffter.ca=peter@eigbox.net>)
	id 1dYIpB-0000Sa-GW
	for groff@gnu.org; Thu, 20 Jul 2017 17:17:33 -0400
Received: from bosmailscan15.eigbox.net ([10.20.15.15])
	by bosmailout09.eigbox.net with esmtp (Exim) id 1dYIJf-00012v-IZ
	for groff@gnu.org; Thu, 20 Jul 2017 16:44:59 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=schaffter.ca; s=dkim; h=Sender:In-Reply-To:Content-Transfer-Encoding:
	Content-Type:MIME-Version:References:Message-ID:Subject:To:From:Date:Reply-To
	:Cc:Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender:
	Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:
	List-Subscribe:List-Post:List-Owner:List-Archive;
	bh=zaFQlcSUY17NbMWO43i/hJJNp8xR1GPxASG5AVWVoQs=;
	b=GLLyUr9k9+LQ/kpbsz3xaAUlSf
	kV04T1M1Vosbiy7FLLZsAO9vmgkLLKObJEFRsyY//qa4BKXloqLEKTIBHxIKhYIYCjdZAx8IJGM2D
	cn3ab3yK4um4UvXQYzrllaiRcNojY3W7ZRc9AD1oSpblb4l6l9s4AOfijmtTNUG8z7EMhG0KNSowE
	ONrmH0/TW2zoIcTBI5Dg2qcevEnIhMy3Xq/cKtVbuHLH1tKG6dlAUjU+iVt0M3XB89byRlv+Qvr+h
	2IZpZQ2aiSxlssrLpFIXvDgvgoPzL58eBDoETklCRzmADWGOE8TB0iR/f1xLKAcSXp/c/z37Ic0Km
	+fFv641g==;
Received: from [10.115.3.32] (helo=bosimpout12)
	by bosmailscan15.eigbox.net with esmtp (Exim) id 1dYIJf-000375-FU
	for groff@gnu.org; Thu, 20 Jul 2017 16:44:59 -0400
Received: from bosauthsmtp08.yourhostingaccount.com ([10.20.18.8])
	by bosimpout12 with 
	id n8kt1v00q0ASroS018kwLE; Thu, 20 Jul 2017 16:44:59 -0400
X-Authority-Analysis: v=2.2 cv=U6ziNaju c=1 sm=1 tr=0
	a=BF10AaGwQl41phDg7WSPyA==:117 a=oHlF9Z/KUwGuOtL1ipNB3Q==:17
	a=IkcTkHD0fZMA:10 a=G3gG6ho9WtcA:10 a=g_SW5-S5AAAA:8
	a=XAjlg_b4_u2GaDwil7sA:9
	a=QEXdDO2ut3YA:10 a=zQDKbV1y6e_a9NbkwXG_:22
Received: from [206.47.14.40] (port=33462 helo=aurelius)
	by bosauthsmtp08.eigbox.net with esmtpsa
	(TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128) (Exim) id 1dYIJZ-0007jS-6c
	for groff@gnu.org; Thu, 20 Jul 2017 16:44:53 -0400
Received: from peter by aurelius with local (Exim 4.86_2)
	(envelope-from <peter@schaffter.ca>) id 1dYIJW-0004kk-2g
	for groff@gnu.org; Thu, 20 Jul 2017 16:44:50 -0400
Date: Thu, 20 Jul 2017 16:44:49 -0400
From: Peter Schaffter <peter@schaffter.ca>
To: groff@gnu.org
Message-ID: <20170720204449.GA17747@augustus>
Mail-Followup-To: Peter Schaffter <peter@schaffter.ca>, groff@gnu.org
References: <20170720215301.7b111801@e.h>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <20170720215301.7b111801@e.h>
User-Agent: Mutt/1.5.24 (2015-08-30)
X-EN-UserInfo: de950a90bfbefba2fd690f5ecdd2f718:931c98230c6409dcc37fa7e93b490c27
X-EN-AuthUser: peter@schaffter.ca
Sender: Peter Schaffter <peter@schaffter.ca>
X-EN-OrigIP: 206.47.14.40
X-EN-OrigHost: unknown
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 66.96.184.9
Subject: Re: [Groff] mom : unicode in .INCLUDE'd files
X-BeenThere: groff@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: <groff.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/groff>,
	<mailto:groff-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/groff/>
List-Post: <mailto:groff@gnu.org>
List-Help: <mailto:groff-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/groff>,
	<mailto:groff-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 20 Jul 2017 21:17:39 -0000

On Thu, Jul 20, 2017, E. Hoffmann wrote:
> I'm going my first steps with mom.
> 
> There is a problem that I can't solve using the documentation, which
>    very well may be just silly because I don't get it (which happened
>    before ;)
> 
> When I enter unicode, like:
> 
>                          ÄÖÜ SS ÒÓÔÕŎŌ Ç äöü ß òóôõŏō ç
> 
> ...and process them with pdfmom, they show up perfectly.
> 
> But if I include the same characters in a file with the .INCLUDE macro,
> they disappear.  Processed with -P-bcu -Tutf8, they show up
> like wrong encoded strings.

This one's a mystery.  I created and processed your test file, with
the same result.  INCLUDE isn't a macro, it's an unmodified alias of
the .so request.  Clearly I, too, am missing something.

Does anyone know why .so isn't passing unicode characters through?

-- 
Peter Schaffter
http://www.schaffter.ca


From MAILER-DAEMON Fri Jul 21 01:51:18 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dYQqM-0000oE-O2
	for mharc-groff@gnu.org; Fri, 21 Jul 2017 01:51:18 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:52126)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <erichhoffmann@gmx.de>) id 1dYQqK-0000ng-88
	for groff@gnu.org; Fri, 21 Jul 2017 01:51:17 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <erichhoffmann@gmx.de>) id 1dYQqG-0005Gf-A1
	for groff@gnu.org; Fri, 21 Jul 2017 01:51:16 -0400
Received: from mout.gmx.net ([212.227.15.18]:54662)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16)
	(Exim 4.71) (envelope-from <erichhoffmann@gmx.de>)
	id 1dYQqG-0005Fx-02
	for groff@gnu.org; Fri, 21 Jul 2017 01:51:12 -0400
Received: from e.h ([46.81.254.171]) by mail.gmx.com (mrgmx001
	[212.227.17.190]) with ESMTPSA (Nemesis) id 0MYwVv-1d3zfa3dOV-00Vi4F
	for <groff@gnu.org>; Fri, 21 Jul 2017 07:51:09 +0200
Date: Fri, 21 Jul 2017 07:54:13 +0200
From: "E. Hoffmann" <erichhoffmann@gmx.de>
To: groff@gnu.org
Message-ID: <20170721075413.204b2af1@e.h>
In-Reply-To: <20170720204449.GA17747@augustus>
References: <20170720215301.7b111801@e.h>
	<20170720204449.GA17747@augustus>
X-Mailer: Claws Mail 3.15.0-dirty (GTK+ 2.24.31; x86_64-slackware-linux-gnu)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
X-Provags-ID: V03:K0:B281dZWY4PFwEm/DVEAqkxmn5d5mmgbattuCsL7V2K3GD/S0TDg
	gXXS5bpJuKqiDKTurffz/qee9vjiuTvn++MqwE2ojy0TUyM/qdO7htPHZORT1Yxi49vL0Dh
	Hb1oyUjXHzdrnGmSYh7JVku6+jFrq35TJuIhdLukRb0A8l3HxYgAEjptsENh7pGIemBwt+w
	YhDwAIdvgI3l+5b2fr/cg==
X-UI-Out-Filterresults: notjunk:1;V01:K0:XXpO04kZD/w=:WPzpSOeaTmgxtT2q+d8ycK
	XR24MmmuoWxnYXOhMHnHlTLQVfbWUF1l2e9TE51botTu5telIK7z2jGUr4oKRfBjXzkiquW29
	FOK3Pih+ASGuaOQvpm00rHfS6w/47NRKvqJQiPehqJi56weMCWwwxeHK6ZhV/qK1VjJ8jiO6W
	uPCDGwe/ASUnW500KpmZhXxcSBPpBm9WY4sw9kzjH5YrFxplYjOV9G4J/B35EIsFvU2kTp/X6
	etsA6u8OQM/kMDB9ZRjBcG9R5Dii3QaB6bDMDrGhtda+/tCaxspxqinfaxtUTW3KPfR7y+ReS
	dPCcEMzYm7gfIRX3Boy2GpKLpQAQnmFUtSbsxkIcn8ctFk/jTj3mDY5BGn2iRZoBDhD3/FNpA
	oJlq3M4CrnyWW7yQn/XbFfzctgP/j5pCDbHhkm0VIoA1ekD7KKPgOmwp47mBGSbtJb0eJwRCk
	8pHRDzc2+IPYpcYc+XQThdRdcGczoANCchVDyqJ+saKD60LAeBpYQE0cP4SZkMOL+zDFQWmXs
	bc2JIVHE1EAOgRnHMVbjPlvmI/a6Xj92pXEQL6pHuARRyoS3nfvxy41QCjE0jBzKFSsildEMP
	6RzR7r7b3WIGSiUhuP4bL7WCnv9vtDlMXhRZgRxLqbTOQX2I7Mbs+G3iV+0FR2NNnvwknaKUl
	daiCxd9HMzWusFrMh/qK7WwHhDXlad/dSrOI2X0+HV2CI7yDMQ2lfXwg6hAnGLZgYxwvly+38
	iGVoIqd/+sPazP9wP1d0wJMp/NeI77u7bd7Vxmxsu9QxfZyZYd/j5n3CKipAsJKz9CKqj8tXM
	E+prsu11nJERfyYV8gcmRI6M9/dnFv6pVcygQb5d+iA26i+Agg=
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
	[fuzzy]
X-Received-From: 212.227.15.18
Subject: Re: [Groff] mom : unicode in .INCLUDE'd files
X-BeenThere: groff@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: <groff.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/groff>,
	<mailto:groff-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/groff/>
List-Post: <mailto:groff@gnu.org>
List-Help: <mailto:groff-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/groff>,
	<mailto:groff-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 21 Jul 2017 05:51:17 -0000

Am Thu, 20 Jul 2017 16:44:49 -0400
schrieb Peter Schaffter <peter@schaffter.ca>:

> On Thu, Jul 20, 2017, E. Hoffmann wrote:
[...]
> > When I enter unicode, like:
> >
> >                          =C3=84=C3=96=C3=9C SS =C3=92=C3=93=C3=94=C3=95=
=C5=8E=C5=8C =C3=87 =C3=A4=C3=B6=C3=BC =C3=9F =C3=B2=C3=B3=C3=B4=C3=B5=C5=
=8F=C5=8D =C3=A7
> >
[...]
> > But if I include the same characters in a file with the .INCLUDE
> > macro, they disappear.  Processed with -P-bcu -Tutf8, they show up
> > like wrong encoded strings.
>
> This one's a mystery.  I created and processed your test file, with
> the same result.  INCLUDE isn't a macro, it's an unmodified alias of
> the .so request.  Clearly I, too, am missing something.
>
> Does anyone know why .so isn't passing unicode characters through?

Peter thank you for looking into it.

   I'm using groff for a long time now, always using plain groff (*.tr)
and the ms macro package.  IIRC when groff introduced the use of
unicode, it was with the -k switch.  Back then I had the same problem
with sourced files, and I processed my files with the soelim-pipe:

    soelim example.tr | preconv -eutf8 | groff -P-bcu -Tutf8

Created an always-to-be-used script and never changed that, because
of lack of need and sheer lazyness.

    Now when I take that:

------------------------------------example.tr-----------\

.pl 2v
UNICODE: =C3=84=C3=96=C3=9C SS =C3=92=C3=93=C3=94=C3=95=C5=8E=C5=8C =C3=87 =
=C3=A4=C3=B6=C3=BC =C3=9F =C3=B2=C3=B3=C3=B4=C3=B5=C5=8F=C5=8D =C3=A7
.br
.so identical_string.tr

---------------------------------------------------------/

Processed with plain

                        groff -Tutf8 example.tr

it yields

        UNICODE: =C3=84=C3=96=C3=9C SS =C3=92=C3=93=C3=94=C3=95=C5=8E=C5=8C=
 =C3=87 =C3=A4=C3=B6=C3=BC =C3=9F =C3=B2=C3=B3=C3=B4=C3=B5=C5=8F=C5=8D =C3=
=A7
        UNICODE: =C3=83=C3=83=C3=83 SS =C3=83=C3=83=C3=83=C3=83=C3=85=C3=85=
 =C3=83 =C3=83=C2=A4=C3=83=C2=B6=C3=83=C2=BC =C3=83 =C3=83=C2=B2=C3=83=C2=
=B3=C3=83=C2=B4=C3=83=C2=B5=C3=85=C3=85 =C3=83=C2=A7

but with

soelim example.tr | preconv -eutf8 | groff -P-bcu -Tutf8 > example.tr.soeli=
m.txt

I get a correct

        UNICODE: =C3=84=C3=96=C3=9C SS =C3=92=C3=93=C3=94=C3=95=C5=8E=C5=8C=
 =C3=87 =C3=A4=C3=B6=C3=BC =C3=9F =C3=B2=C3=B3=C3=B4=C3=B5=C5=8F=C5=8D =C3=
=A7
        UNICODE: =C3=84=C3=96=C3=9C SS =C3=92=C3=93=C3=94=C3=95=C5=8E=C5=8C=
 =C3=87 =C3=A4=C3=B6=C3=BC =C3=9F =C3=B2=C3=B3=C3=B4=C3=B5=C5=8F=C5=8D =C3=
=A7


With ms it behaves similiar, with mom it does not.

    I'm far from being an expert.  This said, could it be that the
    problem lies in the handling of soelim and preconv?


Cheers,

erich



From MAILER-DAEMON Fri Jul 21 02:09:28 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dYR7w-0007l8-8L
	for mharc-groff@gnu.org; Fri, 21 Jul 2017 02:09:28 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:60037)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <dmw@oxytropis.plus.com>) id 1dYR7t-0007km-2S
	for groff@gnu.org; Fri, 21 Jul 2017 02:09:26 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <dmw@oxytropis.plus.com>) id 1dYR7o-0007vP-7Y
	for groff@gnu.org; Fri, 21 Jul 2017 02:09:25 -0400
Received: from avasout08.plus.net ([212.159.14.20]:45273)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16)
	(Exim 4.71) (envelope-from <dmw@oxytropis.plus.com>)
	id 1dYR7o-0007rZ-0H
	for groff@gnu.org; Fri, 21 Jul 2017 02:09:20 -0400
Received: from dell-desktop.dmw.com ([84.92.111.67]) by avasout08 with smtp
	id nJ9A1v0051THptQ01J9B1G; Fri, 21 Jul 2017 07:09:12 +0100
X-CM-Score: 0.00
X-CNFS-Analysis: v=2.2 cv=FLJr/6gs c=1 sm=1 tr=0
	a=zahYBDPRSX6fECX55ANGMA==:117 a=zahYBDPRSX6fECX55ANGMA==:17
	a=IkcTkHD0fZMA:10 a=IepqNwWnDfZTbb1dj14A:9 a=QEXdDO2ut3YA:10
Date: Fri, 21 Jul 2017 07:09:21 +0100
From: "Denis M. Wilson" <dmw@oxytropis.plus.com>
To: groff@gnu.org
Message-ID: <20170721070921.2624dca5@dell-desktop.dmw.com>
In-Reply-To: <20170721075413.204b2af1@e.h>
References: <20170720215301.7b111801@e.h> <20170720204449.GA17747@augustus>
	<20170721075413.204b2af1@e.h>
X-Mailer: Claws Mail 3.7.4 (GTK+ 2.20.1; i486-pc-linux-gnu)
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x (no
	timestamps) [generic] [fuzzy]
X-Received-From: 212.159.14.20
Subject: Re: [Groff] mom : unicode in .INCLUDE'd files
X-BeenThere: groff@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: <groff.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/groff>,
	<mailto:groff-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/groff/>
List-Post: <mailto:groff@gnu.org>
List-Help: <mailto:groff-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/groff>,
	<mailto:groff-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 21 Jul 2017 06:09:26 -0000

soelim does not process .INCLUDE
Denis

On Fri, 21 Jul 2017 07:54:13 +0200
"E. Hoffmann" <erichhoffmann@gmx.de> wrote:

> Am Thu, 20 Jul 2017 16:44:49 -0400
> schrieb Peter Schaffter <peter@schaffter.ca>:
> 
> > On Thu, Jul 20, 2017, E. Hoffmann wrote:
> [...]
> > > When I enter unicode, like:
> > >
> > >                          ÄÖÜ SS ÒÓÔÕŎŌ Ç äöü ß òóôõŏō ç
> > >
> [...]
> > > But if I include the same characters in a file with the .INCLUDE
> > > macro, they disappear.  Processed with -P-bcu -Tutf8, they show up
> > > like wrong encoded strings.
> >
> > This one's a mystery.  I created and processed your test file, with
> > the same result.  INCLUDE isn't a macro, it's an unmodified alias of
> > the .so request.  Clearly I, too, am missing something.
> >
> > Does anyone know why .so isn't passing unicode characters through?
> 
> Peter thank you for looking into it.
> 
>    I'm using groff for a long time now, always using plain groff
> (*.tr) and the ms macro package.  IIRC when groff introduced the use
> of unicode, it was with the -k switch.  Back then I had the same
> problem with sourced files, and I processed my files with the
> soelim-pipe:
> 
>     soelim example.tr | preconv -eutf8 | groff -P-bcu -Tutf8
> 
> Created an always-to-be-used script and never changed that, because
> of lack of need and sheer lazyness.
> 
>     Now when I take that:
> 
> ------------------------------------example.tr-----------\
> 
> .pl 2v
> UNICODE: ÄÖÜ SS ÒÓÔÕŎŌ Ç äöü ß òóôõŏō ç
> .br
> .so identical_string.tr
> 
> ---------------------------------------------------------/
> 
> Processed with plain
> 
>                         groff -Tutf8 example.tr
> 
> it yields
> 
>         UNICODE: ÄÖÜ SS ÒÓÔÕŎŌ Ç äöü ß òóôõŏō ç
>         UNICODE: ÃÃÃ SS ÃÃÃÃÅÅ Ã Ã¤Ã¶Ã¼ Ã Ã²Ã³Ã´ÃµÅÅ Ã§
> 
> but with
> 
> soelim example.tr | preconv -eutf8 | groff -P-bcu -Tutf8 >
> example.tr.soelim.txt
> 
> I get a correct
> 
>         UNICODE: ÄÖÜ SS ÒÓÔÕŎŌ Ç äöü ß òóôõŏō ç
>         UNICODE: ÄÖÜ SS ÒÓÔÕŎŌ Ç äöü ß òóôõŏō ç
> 
> 
> With ms it behaves similiar, with mom it does not.
> 
>     I'm far from being an expert.  This said, could it be that the
>     problem lies in the handling of soelim and preconv?
> 
> 
> Cheers,
> 
> erich
> 
> 


-- 


From MAILER-DAEMON Fri Jul 21 06:30:09 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dYVCC-0002QA-U7
	for mharc-groff@gnu.org; Fri, 21 Jul 2017 06:30:08 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:60331)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <ralph@inputplus.co.uk>) id 1dYVCB-0002Oy-4J
	for groff@gnu.org; Fri, 21 Jul 2017 06:30:08 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <ralph@inputplus.co.uk>) id 1dYVC7-0004cL-3b
	for groff@gnu.org; Fri, 21 Jul 2017 06:30:07 -0400
Received: from relay01.pair.com ([209.68.5.15]:46094)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)
	(Exim 4.71) (envelope-from <ralph@inputplus.co.uk>)
	id 1dYVC6-0004bq-Vp
	for groff@gnu.org; Fri, 21 Jul 2017 06:30:03 -0400
Received: from orac.inputplus.co.uk (unknown [81.174.161.162])
	by relay01.pair.com (Postfix) with ESMTP id 89B18D00AEE;
	Fri, 21 Jul 2017 06:30:01 -0400 (EDT)
Received: from orac.inputplus.co.uk (orac.inputplus.co.uk [IPv6:::1])
	by orac.inputplus.co.uk (Postfix) with ESMTP id 889B01FA63;
	Fri, 21 Jul 2017 11:30:00 +0100 (BST)
To: "E. Hoffmann" <erichhoffmann@gmx.de>
cc: groff@gnu.org
mail-followup-to: groff@gnu.org
From: Ralph Corderoy <ralph@inputplus.co.uk>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
In-reply-to: <20170720215301.7b111801@e.h>
References: <20170720215301.7b111801@e.h>
Date: Fri, 21 Jul 2017 11:30:00 +0100
Message-Id: <20170721103000.889B01FA63@orac.inputplus.co.uk>
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy]
X-Received-From: 209.68.5.15
Subject: Re: [Groff] mom : unicode in .INCLUDE'd files
X-BeenThere: groff@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: <groff.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/groff>,
	<mailto:groff-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/groff/>
List-Post: <mailto:groff@gnu.org>
List-Help: <mailto:groff-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/groff>,
	<mailto:groff-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 21 Jul 2017 10:30:08 -0000

Hi Erich,

> When I enter unicode, like:
>
>                          =C3=84=C3=96=C3=9C SS =C3=92=C3=93=C3=94=C3=95=
=C5=8E=C5=8C =C3=87 =C3=A4=C3=B6=C3=BC =C3=9F =C3=B2=C3=B3=C3=B4=C3=B5=C5=
=8F=C5=8D =C3=A7
>
> ...and process them with pdfmom, they show up perfectly.  But if I
> include the same characters in a file with the .INCLUDE macro, they
> disappear.

Those are Unicode codepoints, but what encoding are you using to
represent them in a file as bytes?  Is it UTF-8?  Only `=C5=8E', U+014E,
isn't in ISO 8859-1, AKA Latin1.

> Processed with -P-bcu -Tutf8, they show up like wrong encoded strings.

troff(1) reads files of ISO 8859-1.  It sounds like, in this particular
test, you're giving it bytes of UTF-8 that it's trying to interpret as
ISO-8859-1.

U+00A3 is a `=C2=A3'.  In UTF-8, it's two bytes;  the 0a is the linefeed.

    $ hd <<<=C2=A3
    00000000  c2 a3 0a                                          |...|

iso-8859-1(7) shows c2 is `=C3=82' and a3 is `=C2=A3' and that's how grof=
f
interprets these bytes.

    $ groff -Tutf8 <<<=C2=A3 | grep .
    =C3=82=C2=A3

> I tried, in vain, the following pipe:
>
>     soelim example.mom | preconv -eutf8 |
>     groff -mom -Tutf8 -P-bcu  > example.txt

As Denis said, soelim(1) looks for `.so' lines.  `.INCLUDE' means
nothing to it.
http://git.savannah.gnu.org/cgit/groff.git/tree/src/preproc/soelim/soelim=
.cpp#n169
You could try replacing `.INCLUDE' with `.so'.

--=20
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy


From MAILER-DAEMON Fri Jul 21 12:20:51 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dYafb-0001GI-9o
	for mharc-groff@gnu.org; Fri, 21 Jul 2017 12:20:51 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:37658)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <SRS0=Gn0j56=6Y=schaffter.ca=peter@eigbox.net>)
	id 1dYafW-0001F8-86
	for groff@gnu.org; Fri, 21 Jul 2017 12:20:47 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <SRS0=Gn0j56=6Y=schaffter.ca=peter@eigbox.net>)
	id 1dYafR-0006Fx-Cl
	for groff@gnu.org; Fri, 21 Jul 2017 12:20:44 -0400
Received: from bosmailout04.eigbox.net ([66.96.184.4]:37745)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)
	(Exim 4.71)
	(envelope-from <SRS0=Gn0j56=6Y=schaffter.ca=peter@eigbox.net>)
	id 1dYafR-0006FZ-0k
	for groff@gnu.org; Fri, 21 Jul 2017 12:20:41 -0400
Received: from bosmailscan04.eigbox.net ([10.20.15.4])
	by bosmailout04.eigbox.net with esmtp (Exim) id 1dYaCI-0006uB-Sw
	for groff@gnu.org; Fri, 21 Jul 2017 11:50:34 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=schaffter.ca; s=dkim; h=Sender:In-Reply-To:Content-Type:MIME-Version:
	References:Message-ID:Subject:To:From:Date:Reply-To:Cc:
	Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date:
	Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:
	List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive;
	bh=SbZAqZHYmEcI1rPfB0eYXrTxSazHSAeufnSPt9fgLEg=;
	b=RRQ5fdlGXZSw+2lhyrqUzK93t
	y5N+qmhH8nyUQTRNIO06eSVIuedjqckLdiVzwTDuwmhkQKdZECIUf4pVx9K6kMJkLnmV1SYjAIMD2
	0VIHmo24k3arbnLckSb2PBQ+WBr9lMB5OV6/dg0yHfgg7mTKJbyUm2t39EBVkdvNOrLUmixIpEqFh
	pcN5Doq2EW3ltCu2rH3bhluCsf6rtUarGBY3ynZuJxO9O+o0u69sa4ATCohp5hbxicOc3HpkSakmX
	8xFotHzLLmEFMiXfdlnr5lYOuW305g+fm/Lpa0h0xgXBalDdriJ00nnGBBmSTm380kowLtV2qWkOB
	wA49EEIgQ==;
Received: from [10.115.3.33] (helo=bosimpout13)
	by bosmailscan04.eigbox.net with esmtp (Exim) id 1dYaCI-00066s-PG
	for groff@gnu.org; Fri, 21 Jul 2017 11:50:34 -0400
Received: from bosauthsmtp08.yourhostingaccount.com ([10.20.18.8])
	by bosimpout13 with 
	id nTqW1v00C0ASroS01TqZJ1; Fri, 21 Jul 2017 11:50:34 -0400
X-Authority-Analysis: v=2.2 cv=fpmsXBwf c=1 sm=1 tr=0
	a=BF10AaGwQl41phDg7WSPyA==:117 a=oHlF9Z/KUwGuOtL1ipNB3Q==:17
	a=kj9zAlcOel0A:10 a=G3gG6ho9WtcA:10 a=g_SW5-S5AAAA:8
	a=TRYWXzA_tExX0D-aWswA:9
	a=CjuIK1q_8ugA:10 a=zQDKbV1y6e_a9NbkwXG_:22
Received: from [206.47.14.40] (port=34354 helo=aurelius)
	by bosauthsmtp08.eigbox.net with esmtpsa
	(TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128) (Exim) id 1dYaCD-0001Ez-WE
	for groff@gnu.org; Fri, 21 Jul 2017 11:50:30 -0400
Received: from peter by aurelius with local (Exim 4.86_2)
	(envelope-from <peter@schaffter.ca>) id 1dYaCC-0003te-5t
	for groff@gnu.org; Fri, 21 Jul 2017 11:50:28 -0400
Date: Fri, 21 Jul 2017 11:50:28 -0400
From: Peter Schaffter <peter@schaffter.ca>
To: groff@gnu.org
Message-ID: <20170721155028.GA14220@augustus>
Mail-Followup-To: Peter Schaffter <peter@schaffter.ca>, groff@gnu.org
References: <20170720215301.7b111801@e.h>
	<20170721103000.889B01FA63@orac.inputplus.co.uk>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20170721103000.889B01FA63@orac.inputplus.co.uk>
User-Agent: Mutt/1.5.24 (2015-08-30)
X-EN-UserInfo: de950a90bfbefba2fd690f5ecdd2f718:931c98230c6409dcc37fa7e93b490c27
X-EN-AuthUser: peter@schaffter.ca
Sender: Peter Schaffter <peter@schaffter.ca>
X-EN-OrigIP: 206.47.14.40
X-EN-OrigHost: unknown
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 66.96.184.4
Subject: Re: [Groff] mom : unicode in .INCLUDE'd files
X-BeenThere: groff@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: <groff.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/groff>,
	<mailto:groff-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/groff/>
List-Post: <mailto:groff@gnu.org>
List-Help: <mailto:groff-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/groff>,
	<mailto:groff-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 21 Jul 2017 16:20:50 -0000

On Fri, Jul 21, 2017, Ralph Corderoy wrote:
> As Denis said, soelim(1) looks for `.so' lines.  `.INCLUDE' means
> nothing to it.

You've just reminded me.  I'll make a note of it in the momdocs.

> You could try replacing `.INCLUDE' with `.so'.

First thing I tried with the OP.  Same problem.  .so is definitely
not playing nice with unicode.

-- 
Peter Schaffter
http://www.schaffter.ca


From MAILER-DAEMON Fri Jul 21 12:32:51 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dYarD-0003zl-8F
	for mharc-groff@gnu.org; Fri, 21 Jul 2017 12:32:51 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:40195)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <ralph@inputplus.co.uk>) id 1dYarA-0003zd-Ok
	for groff@gnu.org; Fri, 21 Jul 2017 12:32:49 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <ralph@inputplus.co.uk>) id 1dYar7-0003vo-LW
	for groff@gnu.org; Fri, 21 Jul 2017 12:32:48 -0400
Received: from relay01.pair.com ([209.68.5.15]:57796)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)
	(Exim 4.71) (envelope-from <ralph@inputplus.co.uk>)
	id 1dYar7-0003tr-H7
	for groff@gnu.org; Fri, 21 Jul 2017 12:32:45 -0400
Received: from orac.inputplus.co.uk (unknown [81.174.161.162])
	by relay01.pair.com (Postfix) with ESMTP id 846EDD00AF6
	for <groff@gnu.org>; Fri, 21 Jul 2017 12:32:39 -0400 (EDT)
Received: from orac.inputplus.co.uk (orac.inputplus.co.uk [IPv6:::1])
	by orac.inputplus.co.uk (Postfix) with ESMTP id E54F8219C2
	for <groff@gnu.org>; Fri, 21 Jul 2017 17:32:38 +0100 (BST)
To: groff@gnu.org
From: Ralph Corderoy <ralph@inputplus.co.uk>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
In-reply-to: <20170721155028.GA14220@augustus>
References: <20170720215301.7b111801@e.h>
	<20170721103000.889B01FA63@orac.inputplus.co.uk>
	<20170721155028.GA14220@augustus>
Date: Fri, 21 Jul 2017 17:32:38 +0100
Message-Id: <20170721163238.E54F8219C2@orac.inputplus.co.uk>
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy]
X-Received-From: 209.68.5.15
Subject: Re: [Groff] mom : unicode in .INCLUDE'd files
X-BeenThere: groff@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: <groff.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/groff>,
	<mailto:groff-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/groff/>
List-Post: <mailto:groff@gnu.org>
List-Help: <mailto:groff-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/groff>,
	<mailto:groff-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 21 Jul 2017 16:32:50 -0000

Hi Peter,

> > You could try replacing `.INCLUDE' with `.so'.
>
> First thing I tried with the OP.  Same problem.  .so is definitely not
> playing nice with unicode.

Ah, well, in that case, I think this explains it.

    $ cat foo
    .nf
    foo: =C3=84=C3=96=C3=9C SS =C3=92=C3=93=C3=94=C3=95=C5=8E=C5=8C =C3=87=
 =C3=A4=C3=B6=C3=BC =C3=9F =C3=B2=C3=B3=C3=B4=C3=B5=C5=8F=C5=8D =C3=A7
    .so bar
    $
    $ cat bar
    bar: =C3=84=C3=96=C3=9C SS =C3=92=C3=93=C3=94=C3=95=C5=8E=C5=8C =C3=87=
 =C3=A4=C3=B6=C3=BC =C3=9F =C3=B2=C3=B3=C3=B4=C3=B5=C5=8F=C5=8D =C3=A7
    $
    $ groff -Tutf8 -sk foo | grep .
    foo: =C3=84=C3=96=C3=9C SS =C3=92=C3=93=C3=94=C3=95=C5=8E=C5=8C =C3=87=
 =C3=A4=C3=B6=C3=BC =C3=9F =C3=B2=C3=B3=C3=B4=C3=B5=C5=8F=C5=8D =C3=A7
    bar: =C3=83=C3=83=C3=83 SS =C3=83=C3=83=C3=83=C3=83=C3=85=C3=85 =C3=83=
 =C3=83=C2=A4=C3=83=C2=B6=C3=83=C2=BC =C3=83 =C3=83=C2=B2=C3=83=C2=B3=C3=83=
=C2=B4=C3=83=C2=B5=C3=85=C3=85 =C3=83=C2=A7
    $
    $ soelim foo
    .lf 1 ./foo
    .nf
    foo: =C3=84=C3=96=C3=9C SS =C3=92=C3=93=C3=94=C3=95=C5=8E=C5=8C =C3=87=
 =C3=A4=C3=B6=C3=BC =C3=9F =C3=B2=C3=B3=C3=B4=C3=B5=C5=8F=C5=8D =C3=A7
    .lf 1 ./bar
    bar: =C3=84=C3=96=C3=9C SS =C3=92=C3=93=C3=94=C3=95=C5=8E=C5=8C =C3=87=
 =C3=A4=C3=B6=C3=BC =C3=9F =C3=B2=C3=B3=C3=B4=C3=B5=C5=8F=C5=8D =C3=A7
    .lf 4 ./foo
    $
    $ soelim foo | preconv
    .lf 1 -
    .lf 1 ./foo
    .nf
    foo: \[u00C4]\[u00D6]\[u00DC] SS \[u00D2]\[u00D3]\[u00D4]\[u00D5]\[u0=
14E]\[u014C] \[u00C7] \[u00E4]\[u00F6]\[u00FC] \[u00DF] \[u00F2]\[u00F3]\=
[u00F4]\[u00F5]\[u014F]\[u014D] \[u00E7]
    .lf 1 ./bar
    bar: \[u00C4]\[u00D6]\[u00DC] SS \[u00D2]\[u00D3]\[u00D4]\[u00D5]\[u0=
14E]\[u014C] \[u00C7] \[u00E4]\[u00F6]\[u00FC] \[u00DF] \[u00F2]\[u00F3]\=
[u00F4]\[u00F5]\[u014F]\[u014D] \[u00E7]
    .lf 4 ./foo
    $
    $ soelim foo | preconv | groff -Tutf8 | grep .
    foo: =C3=84=C3=96=C3=9C SS =C3=92=C3=93=C3=94=C3=95=C5=8E=C5=8C =C3=87=
 =C3=A4=C3=B6=C3=BC =C3=9F =C3=B2=C3=B3=C3=B4=C3=B5=C5=8F=C5=8D =C3=A7
    bar: =C3=84=C3=96=C3=9C SS =C3=92=C3=93=C3=94=C3=95=C5=8E=C5=8C =C3=87=
 =C3=A4=C3=B6=C3=BC =C3=9F =C3=B2=C3=B3=C3=B4=C3=B5=C5=8F=C5=8D =C3=A7
    $
    $ groff -V -Tutf8 -sk foo
    preconv foo | soelim | troff -Tutf8 | grotty
    $

--=20
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy


From MAILER-DAEMON Fri Jul 21 16:28:21 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dYeX7-0007EL-Ti
	for mharc-groff@gnu.org; Fri, 21 Jul 2017 16:28:21 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:59556)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <SRS0=Gn0j56=6Y=schaffter.ca=peter@eigbox.net>)
	id 1dYeX5-0007EE-Nn
	for groff@gnu.org; Fri, 21 Jul 2017 16:28:20 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <SRS0=Gn0j56=6Y=schaffter.ca=peter@eigbox.net>)
	id 1dYeX2-0003Qy-Iq
	for groff@gnu.org; Fri, 21 Jul 2017 16:28:19 -0400
Received: from bosmailout10.eigbox.net ([66.96.184.10]:46772)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)
	(Exim 4.71)
	(envelope-from <SRS0=Gn0j56=6Y=schaffter.ca=peter@eigbox.net>)
	id 1dYeX2-0003Ql-6s
	for groff@gnu.org; Fri, 21 Jul 2017 16:28:16 -0400
Received: from bosmailscan10.eigbox.net ([10.20.15.10])
	by bosmailout10.eigbox.net with esmtp (Exim) id 1dYeWy-0002dY-U0
	for groff@gnu.org; Fri, 21 Jul 2017 16:28:12 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=schaffter.ca; s=dkim; h=Sender:In-Reply-To:Content-Transfer-Encoding:
	Content-Type:MIME-Version:References:Message-ID:Subject:To:From:Date:Reply-To
	:Cc:Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender:
	Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:
	List-Subscribe:List-Post:List-Owner:List-Archive;
	bh=0Oi97ktERDCU0Pnqa1cWPuXDKNvCtJX12LMc0xq0Sos=;
	b=iXZAVkS3ZmtoRn8POoEf3gkg7+
	5epLujsb+CADwr1nRDzPHDLRXn85KqeXFhNwfcCZmG5LKAuCjSQqsop/OOnEGCS5mLKe3WoS4anM8
	xHHjqZf8AtzGwwhlZLZd/pNXpgKK0Ag/OLG4QfsRbHhLSkQ1S3KdjPsFDFVEDnQhu7h0ouY/MDAZc
	EdM2DIrOrr75AnJMXcfsSfy6FcGWxCNr/57Mxfd/43KbIR5HUWBaj9lRAO4livtWuhw2GBQJwr/O1
	PKPeOL0akPbw+92o0QC3VaFr78glo3jcYV3m0+yHCl0bw/xqwtUuJt/0C/sHI3k0TtPCmKsS2JhJo
	eEkro5ig==;
Received: from [10.115.3.33] (helo=bosimpout13)
	by bosmailscan10.eigbox.net with esmtp (Exim) id 1dYeWy-0000hC-OB
	for groff@gnu.org; Fri, 21 Jul 2017 16:28:12 -0400
Received: from bosauthsmtp16.yourhostingaccount.com ([10.20.18.16])
	by bosimpout13 with 
	id nYU81v00E0LoEWa01YUBCC; Fri, 21 Jul 2017 16:28:12 -0400
X-Authority-Analysis: v=2.2 cv=fpmsXBwf c=1 sm=1 tr=0
	a=2OW49aEHms2tn/AeNJ+rfA==:117 a=LMOGXgwQ04WolAN7c33MKA==:17
	a=IkcTkHD0fZMA:10 a=G3gG6ho9WtcA:10 a=g_SW5-S5AAAA:8
	a=Bsm-PG9spof7mF98jpIA:9
	a=QEXdDO2ut3YA:10 a=zQDKbV1y6e_a9NbkwXG_:22
Received: from [206.47.31.229] (port=34726 helo=aurelius)
	by bosauthsmtp16.eigbox.net with esmtpsa
	(TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128) (Exim) id 1dYeWu-0000Qb-0P
	for groff@gnu.org; Fri, 21 Jul 2017 16:28:08 -0400
Received: from peter by aurelius with local (Exim 4.86_2)
	(envelope-from <peter@schaffter.ca>) id 1dYeWq-0005QM-IU
	for groff@gnu.org; Fri, 21 Jul 2017 16:28:04 -0400
Date: Fri, 21 Jul 2017 16:28:04 -0400
From: Peter Schaffter <peter@schaffter.ca>
To: groff@gnu.org
Message-ID: <20170721202804.GA20000@augustus>
Mail-Followup-To: Peter Schaffter <peter@schaffter.ca>, groff@gnu.org
References: <20170720215301.7b111801@e.h>
	<20170721103000.889B01FA63@orac.inputplus.co.uk>
	<20170721155028.GA14220@augustus>
	<20170721163238.E54F8219C2@orac.inputplus.co.uk>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <20170721163238.E54F8219C2@orac.inputplus.co.uk>
User-Agent: Mutt/1.5.24 (2015-08-30)
X-EN-UserInfo: de950a90bfbefba2fd690f5ecdd2f718:931c98230c6409dcc37fa7e93b490c27
X-EN-AuthUser: peter@schaffter.ca
Sender: Peter Schaffter <peter@schaffter.ca>
X-EN-OrigIP: 206.47.31.229
X-EN-OrigHost: unknown
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 66.96.184.10
Subject: Re: [Groff] mom : unicode in .INCLUDE'd files
X-BeenThere: groff@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: <groff.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/groff>,
	<mailto:groff-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/groff/>
List-Post: <mailto:groff@gnu.org>
List-Help: <mailto:groff-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/groff>,
	<mailto:groff-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 21 Jul 2017 20:28:20 -0000

Ralph --

On Fri, Jul 21, 2017, Ralph Corderoy wrote:
> Ah, well, in that case, I think this explains it.
>     <snip>
>     $ soelim foo | preconv | groff -Tutf8 | grep .
>     foo: ÄÖÜ SS ÒÓÔÕŎŌ Ç äöü ß òóôõŏō ç
>     bar: ÄÖÜ SS ÒÓÔÕŎŌ Ç äöü ß òóôõŏō ç
>     $
>     $ groff -V -Tutf8 -sk foo
>     preconv foo | soelim | troff -Tutf8 | grotty
>     $

Thanks for this.

If the order of the groff pipeline is responsible for Erich's
problem (soelim needs to come before preconv), is this something to
change or something to document?

-- 
Peter Schaffter
http://www.schaffter.ca


From MAILER-DAEMON Fri Jul 21 16:30:09 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dYeYr-0007gc-5s
	for mharc-groff@gnu.org; Fri, 21 Jul 2017 16:30:09 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:59794)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <steffen@sdaoden.eu>) id 1dYeYp-0007f1-8M
	for groff@gnu.org; Fri, 21 Jul 2017 16:30:08 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <steffen@sdaoden.eu>) id 1dYeYm-0003xR-3T
	for groff@gnu.org; Fri, 21 Jul 2017 16:30:07 -0400
Received: from sdaoden.eu ([217.144.132.164]:44790)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <steffen@sdaoden.eu>) id 1dYeYl-0003vr-Sd
	for groff@gnu.org; Fri, 21 Jul 2017 16:30:04 -0400
Received: by sdaoden.eu (Postfix, from userid 1000)
	id 63FB216051; Fri, 21 Jul 2017 22:30:01 +0200 (CEST)
Date: Fri, 21 Jul 2017 22:30:36 +0200
From: Steffen Nurpmeso <steffen@sdaoden.eu>
To: groff@gnu.org
Message-ID: <20170721203036.PeGAm%steffen@sdaoden.eu>
References: <20170720215301.7b111801@e.h>
	<20170721103000.889B01FA63@orac.inputplus.co.uk>
	<20170721155028.GA14220@augustus>
	<20170721163238.E54F8219C2@orac.inputplus.co.uk>
Mail-Followup-To: groff@gnu.org
User-Agent: s-nail v14.9.0
OpenPGP: id=232C220BCB5690A37BD22FFDEB66022795F382CE;
	url=https://www.sdaoden.eu/downloads/steffen.asc
BlahBlahBlah: Any stupid boy can crush a beetle. But all the professors in
	the world can make no bugs.
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: base64
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
	[fuzzy]
X-Received-From: 217.144.132.164
Subject: Re: [Groff] mom : unicode in .INCLUDE'd files
X-BeenThere: groff@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: <groff.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/groff>,
	<mailto:groff-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/groff/>
List-Post: <mailto:groff@gnu.org>
List-Help: <mailto:groff-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/groff>,
	<mailto:groff-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 21 Jul 2017 20:30:08 -0000

UmFscGggQ29yZGVyb3kgPHJhbHBoQGlucHV0cGx1cy5jby51az4gd3JvdGU6CiB8Pj4gWW91IGNv
dWxkIHRyeSByZXBsYWNpbmcgYC5JTkNMVURFJyB3aXRoIGAuc28nLgogfD4KIHw+IEZpcnN0IHRo
aW5nIEkgdHJpZWQgd2l0aCB0aGUgT1AuICBTYW1lIHByb2JsZW0uICAuc28gaXMgZGVmaW5pdGVs
eSBub3QKIHw+IHBsYXlpbmcgbmljZSB3aXRoIHVuaWNvZGUuCiB8CiB8QWgsIHdlbGwsIGluIHRo
YXQgY2FzZSwgSSB0aGluayB0aGlzIGV4cGxhaW5zIGl0LgogLi4KIHwgICAgJCBzb2VsaW0gZm9v
IHwgcHJlY29udiB8IGdyb2ZmIC1UdXRmOCB8IGdyZXAgLgogfCAgICBmb286IMOEw5bDnCBTUyDD
ksOTw5TDlcWOxYwgw4cgw6TDtsO8IMOfIMOyw7PDtMO1xY/FjSDDpwogfCAgICBiYXI6IMOEw5bD
nCBTUyDDksOTw5TDlcWOxYwgw4cgw6TDtsO8IMOfIMOyw7PDtMO1xY/FjSDDpwogfCAgICAkCiB8
ICAgICQgZ3JvZmYgLVYgLVR1dGY4IC1zayBmb28KIHwgICAgcHJlY29udiBmb28gfCBzb2VsaW0g
fCB0cm9mZiAtVHV0ZjggfCBncm90dHkKCkluIG15IGh1bWJsZSBvcGluaW9uIHByZWNvbnYgaGFz
IHRvIGdvIGFzIHN1Y2gsIGkganVzdCBkbyBub3Qga25vdwp5ZXQuICBKdXN0IHRhbGtpbmcuCgot
LXN0ZWZmZW4KfAp8RGVyIEtyYWdlbmJhZXIsICAgICAgICAgICAgICAgIFRoZSBtb29uIGJlYXIs
CnxkZXIgaG9sdCBzaWNoIG11bnRlciAgICAgICAgICAgaGUgY2hlZXJmdWxseSBhbmQgb25lIGJ5
IG9uZQp8ZWluZW4gbmFjaCBkZW0gYW5kZXJlbiBydW50ZXIgIHdhLmtzIGhpbXNlbGYgb2ZmCnwo
QnkgUm9iZXJ0IEdlcm5oYXJkdCkK


From MAILER-DAEMON Sat Jul 22 06:44:17 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dYrtR-0005ux-5K
	for mharc-groff@gnu.org; Sat, 22 Jul 2017 06:44:17 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:49422)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <ralph@inputplus.co.uk>) id 1dYrtP-0005ua-D0
	for groff@gnu.org; Sat, 22 Jul 2017 06:44:16 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <ralph@inputplus.co.uk>) id 1dYrtM-0004rZ-BB
	for groff@gnu.org; Sat, 22 Jul 2017 06:44:15 -0400
Received: from relay01.pair.com ([209.68.5.15]:48851)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)
	(Exim 4.71) (envelope-from <ralph@inputplus.co.uk>)
	id 1dYrtM-0004r6-6x
	for groff@gnu.org; Sat, 22 Jul 2017 06:44:12 -0400
Received: from orac.inputplus.co.uk (179.112.114.87.dyn.plus.net
	[87.114.112.179])
	by relay01.pair.com (Postfix) with ESMTP id 09CA6D00A91
	for <groff@gnu.org>; Sat, 22 Jul 2017 06:44:11 -0400 (EDT)
Received: from orac.inputplus.co.uk (orac.inputplus.co.uk [IPv6:::1])
	by orac.inputplus.co.uk (Postfix) with ESMTP id E0B55215AA
	for <groff@gnu.org>; Sat, 22 Jul 2017 11:44:09 +0100 (BST)
To: groff@gnu.org
From: Ralph Corderoy <ralph@inputplus.co.uk>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
In-reply-to: <20170721202804.GA20000@augustus>
References: <20170720215301.7b111801@e.h>
	<20170721103000.889B01FA63@orac.inputplus.co.uk>
	<20170721155028.GA14220@augustus>
	<20170721163238.E54F8219C2@orac.inputplus.co.uk>
	<20170721202804.GA20000@augustus>
Date: Sat, 22 Jul 2017 11:44:09 +0100
Message-Id: <20170722104409.E0B55215AA@orac.inputplus.co.uk>
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy]
X-Received-From: 209.68.5.15
Subject: Re: [Groff] mom : unicode in .INCLUDE'd files
X-BeenThere: groff@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: <groff.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/groff>,
	<mailto:groff-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/groff/>
List-Post: <mailto:groff@gnu.org>
List-Help: <mailto:groff-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/groff>,
	<mailto:groff-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sat, 22 Jul 2017 10:44:16 -0000

Hi Peter,

> If the order of the groff pipeline is responsible for Erich's problem
> (soelim needs to come before preconv), is this something to change or
> something to document?

Change has been mooted in the past, including by Werner.
Some old relevant threads.

    https://lists.gnu.org/archive/html/groff/2006-01/msg00022.html
    https://lists.gnu.org/archive/html/groff/2010-01/msg00058.html
    https://lists.gnu.org/archive/html/groff/2012-12/msg00030.html

--=20
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy


From MAILER-DAEMON Sat Jul 22 09:56:26 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dYutO-0000qz-KB
	for mharc-groff@gnu.org; Sat, 22 Jul 2017 09:56:26 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:44367)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <schwarze@usta.de>) id 1dYutL-0000ql-PW
	for groff@gnu.org; Sat, 22 Jul 2017 09:56:24 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <schwarze@usta.de>) id 1dYutF-0006p8-Ib
	for groff@gnu.org; Sat, 22 Jul 2017 09:56:23 -0400
Received: from scc-mailout-kit-01.scc.kit.edu
	([2a00:1398:9:f712::810d:e751]:52077)
	by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)
	(Exim 4.71) (envelope-from <schwarze@usta.de>) id 1dYutF-0006oj-C8
	for groff@gnu.org; Sat, 22 Jul 2017 09:56:17 -0400
Received: from asta-nat.asta.uni-karlsruhe.de ([172.22.63.82]
	helo=hekate.usta.de) by scc-mailout-kit-01.scc.kit.edu with esmtps
	(TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
	(envelope-from <schwarze@usta.de>)
	id 1dYutA-00068q-Av; Sat, 22 Jul 2017 15:56:13 +0200
Received: from donnerwolke.usta.de ([172.24.96.3])
	by hekate.usta.de with esmtp (Exim 4.77)
	(envelope-from <schwarze@usta.de>)
	id 1dYut9-0000CY-V2; Sat, 22 Jul 2017 15:56:11 +0200
Received: from athene.usta.de ([172.24.96.10])
	by donnerwolke.usta.de with esmtp (Exim 4.84_2)
	(envelope-from <schwarze@usta.de>)
	id 1dYut9-0002to-RV; Sat, 22 Jul 2017 15:56:11 +0200
Received: from localhost (athene.usta.de [local])
	by athene.usta.de (OpenSMTPD) with ESMTPA id 91d35a86;
	Sat, 22 Jul 2017 15:56:11 +0200 (CEST)
Date: Sat, 22 Jul 2017 15:56:11 +0200
From: Ingo Schwarze <schwarze@usta.de>
To: Steffen Nurpmeso <steffen@sdaoden.eu>
Cc: groff@gnu.org
Message-ID: <20170722135611.GC67926@athene.usta.de>
References: <20170720215301.7b111801@e.h>
	<20170721103000.889B01FA63@orac.inputplus.co.uk>
	<20170721155028.GA14220@augustus>
	<20170721163238.E54F8219C2@orac.inputplus.co.uk>
	<20170721203036.PeGAm%steffen@sdaoden.eu>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20170721203036.PeGAm%steffen@sdaoden.eu>
User-Agent: Mutt/1.6.2 (2016-07-01)
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
X-Received-From: 2a00:1398:9:f712::810d:e751
Subject: Re: [Groff] mom : unicode in .INCLUDE'd files
X-BeenThere: groff@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: <groff.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/groff>,
	<mailto:groff-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/groff/>
List-Post: <mailto:groff@gnu.org>
List-Help: <mailto:groff-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/groff>,
	<mailto:groff-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sat, 22 Jul 2017 13:56:24 -0000

Hi,

Steffen Nurpmeso wrote on Fri, Jul 21, 2017 at 10:30:36PM +0200:

> In my humble opinion preconv has to go as such,
> i just do not know yet.  Just talking.

So much talk...

In mandoc, i completed that work in October 2014:

  http://mandoc.bsd.lv/cgi-bin/cvsweb/preconv.c#rev1.9

  "commit message:
   integrate preconv(1) into mandoc(1); enhances functionality
   and reduces code and docs by more than 300 lines"

Admittedly, mandoc only handles UTF-8 and ISO-LATIN-1, and
requires the user to convert files using obsolete encodings
to UTF-8 using iconv(1) first.

The only reason for supporting ISO-LATIN-1 is that many old manual
pages in the wild still use it, most even without saying so.
Otherwise, mandoc would be UTF-8 only on the input side.

In the long run, i think that would be a reasonable direction
for groff, too.  Preconv is notorious for causing trouble for
casual users, see the several threads that were quoted earlier
in this thread, and even very experienced users often get
confused about how it works and where in the pipeline it is
supposed to be, see this thread itself.

"Groff input always has to be UTF-8", that would be a very simple,
fool-proof principle.

Yours,
  Ingo


From MAILER-DAEMON Sat Jul 22 10:07:14 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dYv3q-0002O4-T9
	for mharc-groff@gnu.org; Sat, 22 Jul 2017 10:07:14 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:45744)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <gardnerjohng@gmail.com>) id 1dYv3o-0002Nw-Ve
	for groff@gnu.org; Sat, 22 Jul 2017 10:07:14 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <gardnerjohng@gmail.com>) id 1dYv3n-0002R3-G6
	for groff@gnu.org; Sat, 22 Jul 2017 10:07:12 -0400
Received: from mail-oi0-x22d.google.com ([2607:f8b0:4003:c06::22d]:33108)
	by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)
	(Exim 4.71) (envelope-from <gardnerjohng@gmail.com>)
	id 1dYv3n-0002Pk-94
	for groff@gnu.org; Sat, 22 Jul 2017 10:07:11 -0400
Received: by mail-oi0-x22d.google.com with SMTP id p188so70984632oia.0
	for <groff@gnu.org>; Sat, 22 Jul 2017 07:07:09 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
	h=mime-version:in-reply-to:references:from:date:message-id:subject:to
	:cc; bh=tckGbuhztKsrQ4UB9OmkkjeJx0vMKG4oNNkoApRNR2I=;
	b=Jv/+vF4JRxY6QGLzRlQ7s6BALod12goUzWKMPL17YKQ8lrd+r7aYumjEt1sI3Sv1dX
	mDmJ3/g0suztCR/rVJAxxfaoo/wmMxR8NDKl7y9EnhVUpDLyBgAoJN1ioJY0M72m3jRa
	eCP+3zOdClvsQSkHr4gGm5+fKuhNnUA0V7LJtUNLQhgfW6/kLiRDhLi6Pov7XGk1r9cq
	97WDDxNs3jsy8b39fRu1GvRSr05DX7K+CXf1J0oh4ydQvWf/jb0bz201/GN38OPS2KhZ
	7V5ZPvRnSwBWsaBoNelXqAC3F2Brwv2XDzXjfps0YbeokzintdRq5hyPXXPIrmsByAgi
	Yw1Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
	d=1e100.net; s=20161025;
	h=x-gm-message-state:mime-version:in-reply-to:references:from:date
	:message-id:subject:to:cc;
	bh=tckGbuhztKsrQ4UB9OmkkjeJx0vMKG4oNNkoApRNR2I=;
	b=T8m4hTsY/K9YvUN+ZDScNorJcbM8L3tHWh22gEDrqcV8ihUSwKVRM2GuhJ9eK7EXGo
	+NAWv5qBhVHWf6Bpy8ebpTKjC61h8Wl00IuEXbPBQFnGsVFUQgBgSj6D35iEFsS/G+Rq
	6JyErc4HmBNt/ZndTC5+lp6qqxqy4Q/BExjoB9ydnskFyTUrvbAhNuaLM78j6gPCgXjy
	nCjTRaWn+oC/weazWje1d6nHbSo8cT/MpJ2XPsxaoxFpxcixJzfvThLVFzpPJ+X8KnG7
	3NVJOK/68sKhusCeD8JztXObXJvuvLYqc5ClftM2y0IU7Q2PvPaDPB49qdGz7y2GOYaO
	8EdQ==
X-Gm-Message-State: AIVw111cyrSXMJhDJ+h7a/aHgc7aQLrUht+gi2ULCmaTmVxVWS20y/jg
	uR6r37mXCkw74CPwIyFEIRgFL+EkJab9oGk=
X-Received: by 10.202.207.82 with SMTP id f79mr4484460oig.37.1500732428838;
	Sat, 22 Jul 2017 07:07:08 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.74.43.89 with HTTP; Sat, 22 Jul 2017 07:06:53 -0700 (PDT)
In-Reply-To: <20170722135611.GC67926@athene.usta.de>
References: <20170720215301.7b111801@e.h>
	<20170721103000.889B01FA63@orac.inputplus.co.uk>
	<20170721155028.GA14220@augustus>
	<20170721163238.E54F8219C2@orac.inputplus.co.uk>
	<20170721203036.PeGAm%steffen@sdaoden.eu>
	<20170722135611.GC67926@athene.usta.de>
From: John Gardner <gardnerjohng@gmail.com>
Date: Sun, 23 Jul 2017 00:06:53 +1000
Message-ID: <CAGcdajcrzUZ96Uh37MY2jbb+kotfU+-aY0r67WwRFxjeaOaN3w@mail.gmail.com>
To: Ingo Schwarze <schwarze@usta.de>
Cc: Steffen Nurpmeso <steffen@sdaoden.eu>, groff <groff@gnu.org>
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 2607:f8b0:4003:c06::22d
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
X-Content-Filtered-By: Mailman/MimeDel 2.1.21
Subject: Re: [Groff] mom : unicode in .INCLUDE'd files
X-BeenThere: groff@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: <groff.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/groff>,
	<mailto:groff-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/groff/>
List-Post: <mailto:groff@gnu.org>
List-Help: <mailto:groff-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/groff>,
	<mailto:groff-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sat, 22 Jul 2017 14:07:14 -0000

I was bitten by preconv(1) quite recently, actually. Gonna back Ingo here.
Can I semi-seriously implore the world to only use UTF-8, and pretend other
encodings don't exist? Squash everything into the same shuttle as EBCDIC
and blast it into the sun.

Heh, another recent (and unpleasant) experience was learning about the
harsh relationship between PostScript's "ISOLatin1Encoding" and
grave/accurate accents. I spent days trying to figure out why \` (U+0060) w=
as
being converted to =E2=80=98 (U+2018). Several headaches later, I learned t=
he PDF
viewer was actually doing this *to the PDF's source* when rendered. This
page tells the whole complex history.
<https://www.cl.cam.ac.uk/~mgk25/ucs/quotes.html#postscript>..

Ugh...

On 22 July 2017 at 23:56, Ingo Schwarze <schwarze@usta.de> wrote:

> Hi,
>
> Steffen Nurpmeso wrote on Fri, Jul 21, 2017 at 10:30:36PM +0200:
>
> > In my humble opinion preconv has to go as such,
> > i just do not know yet.  Just talking.
>
> So much talk...
>
> In mandoc, i completed that work in October 2014:
>
>   http://mandoc.bsd.lv/cgi-bin/cvsweb/preconv.c#rev1.9
>
>   "commit message:
>    integrate preconv(1) into mandoc(1); enhances functionality
>    and reduces code and docs by more than 300 lines"
>
> Admittedly, mandoc only handles UTF-8 and ISO-LATIN-1, and
> requires the user to convert files using obsolete encodings
> to UTF-8 using iconv(1) first.
>
> The only reason for supporting ISO-LATIN-1 is that many old manual
> pages in the wild still use it, most even without saying so.
> Otherwise, mandoc would be UTF-8 only on the input side.
>
> In the long run, i think that would be a reasonable direction
> for groff, too.  Preconv is notorious for causing trouble for
> casual users, see the several threads that were quoted earlier
> in this thread, and even very experienced users often get
> confused about how it works and where in the pipeline it is
> supposed to be, see this thread itself.
>
> "Groff input always has to be UTF-8", that would be a very simple,
> fool-proof principle.
>
> Yours,
>   Ingo
>
>

From MAILER-DAEMON Sat Jul 22 10:57:42 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dYvqg-0001zv-Ne
	for mharc-groff@gnu.org; Sat, 22 Jul 2017 10:57:42 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:52535)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <erichhoffmann@gmx.de>) id 1dYvqe-0001ze-E4
	for groff@gnu.org; Sat, 22 Jul 2017 10:57:41 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <erichhoffmann@gmx.de>) id 1dYvqb-0008DQ-Bz
	for groff@gnu.org; Sat, 22 Jul 2017 10:57:40 -0400
Received: from mout.gmx.net ([212.227.17.21]:52991)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16)
	(Exim 4.71) (envelope-from <erichhoffmann@gmx.de>)
	id 1dYvqb-0008Cr-0x
	for groff@gnu.org; Sat, 22 Jul 2017 10:57:37 -0400
Received: from e.h ([46.81.254.171]) by mail.gmx.com (mrgmx101
	[212.227.17.168]) with ESMTPSA (Nemesis) id 0Me4Ly-1dFdKx41EM-00PsJ1;
	Sat, 22 Jul 2017 16:57:30 +0200
Date: Sat, 22 Jul 2017 17:00:37 +0200
From: "E. Hoffmann" <erichhoffmann@gmx.de>
To: Peter Schaffter <peter@schaffter.ca>
Cc: groff@gnu.org
Message-ID: <20170722170037.2f615a5f@e.h>
In-Reply-To: <20170721202804.GA20000@augustus>
References: <20170720215301.7b111801@e.h>
	<20170721103000.889B01FA63@orac.inputplus.co.uk>
	<20170721155028.GA14220@augustus>
	<20170721163238.E54F8219C2@orac.inputplus.co.uk>
	<20170721202804.GA20000@augustus>
X-Mailer: Claws Mail 3.15.0-dirty (GTK+ 2.24.31; x86_64-slackware-linux-gnu)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
X-Provags-ID: V03:K0:4OmtfyzSRtX7IuS3VSsGc3pw85Cqid7g4KG5TfwOUsBF8vz5xiy
	t3xFjUForSUdR+5+UrAl912KFp5hngIz2H/gg18oewVmmt15ng7l4gvsUrRZWZ3e+eohzLI
	6PbdZEI0tKIlZ+/a7LdbKiILwzcKfACkGtN4pmtOiJLKeSeRJwUbr/8Tm8tZh5jx1PS6Y7A
	Pv1YQxTuobSp3PBsSphyA==
X-UI-Out-Filterresults: notjunk:1;V01:K0:gQ3jksFqHHg=:SmplDC18n9bQcjKD1i3vvO
	W3rWqXzNQhG2gvaWbCu2/M2iiaeFHvHEVuoQojcC2/2F/VOAtkYDudtl+mgdD4XaeoRZSlkQn
	iIR4ydJnS7z3teVEs4SrsDybcce6LFYpuIKGxh0XxJl0jMy9VuoCW0XKxGQC7qE5vG9Binlr7
	l0RZw68fBGRFoSKShteBUfJ7PKiPwQXRQIsqEP6Se8I4M1opdALpvdc3noE3oJNeGascDm0Hm
	BdkPfrBZx6VlxJtTzESjo8E8IB908OM0Um0r5EYjR/Vdr48d/CPmBnzpKbXRNBMTscTqtuGZn
	CXau8r1poRy/DPvNF1nnTI1HEYKHAWxM94LzxNRkHBdQC1I+ag1TQkBfbtmK8NvCdkACQmyyt
	WowC0xIs9qpfYhAqUy+SaYXpcLj8xPEYnx6XSyJWLmQxFSWfteFi7XDmGggg8vln/YStitNR4
	ISfu1Y+ofzx9MqSNcniHqedxy1fdPYbLswP48JRqWWtWw4JhR7r4xgkBc0vvO2Xfis582+WXZ
	0uOCD+yar5UgY4Eh26ZJemXxnjoIG++xgreLUNQfw41OEnAJirMCyQDQHkGvOEa4c2zdlPMLG
	K7aPJzoVdoB17F/cEx6XdutEefWgQASY7D3jiRVlM0zRgG6c4K4ARmDoj/8XtuDzqF6sVn6QV
	9cqsTOF6vpgn4fYjv/xS3vKN45QavI3JKSuVoeOfbodDob3+1NWk81GXG0zKhB8qjwZgh9cNZ
	NkYfSfjqZaLyYAbD4ooH9Ko89Kf5CONJCLz6NJIq8JV3BPYiRz9JWq6/CAOuDHDO/KbOFS3Q5
	Ch6r5vEYNNwL+3p/rWHhEQB9QiZuOMQpCkFMDAolulu8Ld227I=
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
	[fuzzy]
X-Received-From: 212.227.17.21
Subject: Re: [Groff] mom : unicode in .INCLUDE'd files
X-BeenThere: groff@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: <groff.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/groff>,
	<mailto:groff-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/groff/>
List-Post: <mailto:groff@gnu.org>
List-Help: <mailto:groff-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/groff>,
	<mailto:groff-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sat, 22 Jul 2017 14:57:41 -0000

Am Fri, 21 Jul 2017 16:28:04 -0400
schrieb Peter Schaffter <peter@schaffter.ca>:

[...]
> >     $ soelim foo | preconv | groff -Tutf8 | grep .
> >     foo: =C3=84=C3=96=C3=9C SS =C3=92=C3=93=C3=94=C3=95=C5=8E=C5=8C =C3=
=87 =C3=A4=C3=B6=C3=BC =C3=9F =C3=B2=C3=B3=C3=B4=C3=B5=C5=8F=C5=8D =C3=A7
> >     bar: =C3=84=C3=96=C3=9C SS =C3=92=C3=93=C3=94=C3=95=C5=8E=C5=8C =C3=
=87 =C3=A4=C3=B6=C3=BC =C3=9F =C3=B2=C3=B3=C3=B4=C3=B5=C5=8F=C5=8D =C3=A7
> >     $
> >     $ groff -V -Tutf8 -sk foo
> >     preconv foo | soelim | troff -Tutf8 | grotty

Thanks a lot.

So, the solution's first step is to replace the .INCLUDE macro with the
plain .so request, and the second step is the preconv-pipe.

Something remains.

    $groff -Tutf8 -V -sk example.mom

gives:

    >preconv -eutf8 example.mom | soelim | troff -Tutf8 | grotty

Output:

    >        Abschnitt.DOCTYPE     DEFAULT UNICODE: =C3=84=E2=80=90=C3=96=
=E2=80=90=C3=9C =C3=92=E2=80=90=C3=93=E2=80=90=C3=94=E2=80=90=C3=95=E2=80=
=90=C5=8E=E2=80=90=C5=8C=E2=80=90
    > =C3=87 =C3=A4=E2=80=90=C3=B6=E2=80=90=C3=BC=E2=80=90 =C3=9F =C3=B2=E2=
=80=90=C3=B3=E2=80=90=C3=B4=E2=80=90=C3=B5=E2=80=90=C5=8F=E2=80=90=C5=8D=E2=
=80=90 =C3=A7 so=E2=80=99eD FILE BEGINS: UNICODE: =C3=83=E2=80=90=C3=83=E2=
=80=90=C3=83  =C3=83=E2=80=90=C3=83=E2=80=90
    > =C3=83=E2=80=90=C3=83=E2=80=90=C3=85=E2=80=90=C3=85=E2=80=90 =C3=83 =
=C3=83=C2=A4=E2=80=90=C3=83=C2=B6=E2=80=90=C3=83=C2=BC=E2=80=90 =C3=83 =C3=
=83=C2=B2=E2=80=90=C3=83=C2=B3=E2=80=90=C3=83=C2=B4=E2=80=90=C3=83=C2=B5=E2=
=80=90=C3=85=E2=80=90=C3=85=E2=80=90 =C3=83=C2=A7

But I want pdfmom and change that to

preconv -eutf8 example.mom | soelim | pdfmom > example.preconv.pdf

   I get a nice pdf-file containing all latin1-characters.  This is
progress!

    But the pdf does not show the characters " =C5=8E =C5=8C  =C5=8F =C5=8D=
 " .

The error message:

    example.mom:8: warning: can't find special character `u004F_0306'
    example.mom:8: warning: can't find special character `u004F_0304'
    example.mom:8: warning: can't find special character `u006F_0306'
    example.mom:8: warning: can't find special character `u006F_0304'

As Ralph pointed out:

>>  Those are Unicode codepoints, but what encoding are you using to
>>  represent them in a file as bytes?  Is it UTF-8?  Only `=C5=8E', U+014E,
>>  isn't in ISO 8859-1, AKA Latin1.

I am using the gvim editor with encoding=3Dutf-8 .

The "ga" command gives:

    <=C5=8E> 334, Hex 014e, Oktal 516
    <=C5=8C> 332, Hex 014c, Oktal 514
    <=C5=8F> 335, Hex 014f, Oktal 517
    <=C5=8D> 333, Hex 014d, Oktal 515

As it seems, the preconv-pipe manages latin1, but not the "true" utf-8.

Is there something else I can do?

Cheers, Erich



From MAILER-DAEMON Sat Jul 22 12:02:18 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dYwrC-0003pP-Hz
	for mharc-groff@gnu.org; Sat, 22 Jul 2017 12:02:18 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:33382)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <steffen@sdaoden.eu>) id 1dYwr7-0003l5-MR
	for groff@gnu.org; Sat, 22 Jul 2017 12:02:17 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <steffen@sdaoden.eu>) id 1dYwr0-0005wG-Vm
	for groff@gnu.org; Sat, 22 Jul 2017 12:02:12 -0400
Received: from sdaoden.eu ([217.144.132.164]:44836)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <steffen@sdaoden.eu>) id 1dYwr0-0005vx-OH
	for groff@gnu.org; Sat, 22 Jul 2017 12:02:06 -0400
Received: by sdaoden.eu (Postfix, from userid 1000)
	id F02D916051; Sat, 22 Jul 2017 18:02:04 +0200 (CEST)
Date: Sat, 22 Jul 2017 18:02:41 +0200
From: Steffen Nurpmeso <steffen@sdaoden.eu>
To: Ingo Schwarze <schwarze@usta.de>
Cc: groff@gnu.org
Message-ID: <20170722160241.zn8Nx%steffen@sdaoden.eu>
References: <20170720215301.7b111801@e.h>
	<20170721103000.889B01FA63@orac.inputplus.co.uk>
	<20170721155028.GA14220@augustus>
	<20170721163238.E54F8219C2@orac.inputplus.co.uk>
	<20170721203036.PeGAm%steffen@sdaoden.eu>
	<20170722135611.GC67926@athene.usta.de>
Mail-Followup-To: Ingo Schwarze <schwarze@usta.de>, groff@gnu.org
User-Agent: s-nail v14.9.0
OpenPGP: id=232C220BCB5690A37BD22FFDEB66022795F382CE;
	url=https://www.sdaoden.eu/downloads/steffen.asc
BlahBlahBlah: Any stupid boy can crush a beetle. But all the professors in
	the world can make no bugs.
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
	[fuzzy]
X-Received-From: 217.144.132.164
Subject: Re: [Groff] mom : unicode in .INCLUDE'd files
X-BeenThere: groff@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: <groff.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/groff>,
	<mailto:groff-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/groff/>
List-Post: <mailto:groff@gnu.org>
List-Help: <mailto:groff-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/groff>,
	<mailto:groff-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sat, 22 Jul 2017 16:02:17 -0000

Hey, Ingo,

Ingo Schwarze <schwarze@usta.de> wrote:
 |Steffen Nurpmeso wrote on Fri, Jul 21, 2017 at 10:30:36PM +0200:
 |
 |> In my humble opinion preconv has to go as such,
 |> i just do not know yet.  Just talking.
 |
 |So much talk...

SIIGHH!!!

 |In mandoc, i completed that work in October 2014:
 |
 |  http://mandoc.bsd.lv/cgi-bin/cvsweb/preconv.c#rev1.9
 |
 |  "commit message:
 |   integrate preconv(1) into mandoc(1); enhances functionality
 |   and reduces code and docs by more than 300 lines"

I think that is the right thing, because

 |Admittedly, mandoc only handles UTF-8 and ISO-LATIN-1, and
 |requires the user to convert files using obsolete encodings
 |to UTF-8 using iconv(1) first.

there is iconv(1), and either preconv catches it all or it is not
needed.  Probably there should be an .iconv request, but that is
probably too late ... anyway, looking at emacs tags cannot be the
solution for roff, in my opinion.

 |The only reason for supporting ISO-LATIN-1 is that many old manual
 |pages in the wild still use it, most even without saying so.
 |Otherwise, mandoc would be UTF-8 only on the input side.

I do not know.. i mean, the really good guys sat down and thought
about it at a dinner and were ready to go, but this was about 25
years ago and Unix is still not en par, so i do not think this can
be a solution for roff, which needs to deal with documents other
than manuals.  I have no idea about non-english (and a few de)
manual pages (but those were often not up-to-date, so i tried to
get rid of them asap anytime i rememember; note this really is an
*unfortunately*, because the very high-quality translations of
ISO C that i have from Prof. Dr. A.-T Schreiner and
Dr. Ernst Janich, typeset in roff, made me forget i ever needed
anything else!), i seem to remember other character sets.  But
since it can be automatized i guess it is ok for mandoc to
require that systems which switch perform an iconv run.

 |In the long run, i think that would be a reasonable direction
 |for groff, too.  Preconv is notorious for causing trouble for
 |casual users, see the several threads that were quoted earlier
 |in this thread, and even very experienced users often get
 |confused about how it works and where in the pipeline it is
 |supposed to be, see this thread itself.
 |
 |"Groff input always has to be UTF-8", that would be a very simple,
 |fool-proof principle.

I do not think this will be the direction to go.  I have no idea,
i really like to get Unicode on the input side, but a conversion
in front of that may be fine, as long as it is picked up
automatically in a way that the author of the document specifies.
And the emacs tags are in the world, and thus need to stay for
quite a while, too. :(

--steffen
|
|Der Kragenbaer,                The moon bear,
|der holt sich munter           he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)


From MAILER-DAEMON Sat Jul 22 12:08:39 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dYwxL-0006fu-Mn
	for mharc-groff@gnu.org; Sat, 22 Jul 2017 12:08:39 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:35431)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <mbianchi@foveal.com>) id 1dYwxJ-0006bP-Va
	for groff@gnu.org; Sat, 22 Jul 2017 12:08:38 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <mbianchi@foveal.com>) id 1dYwxG-0000hU-Nn
	for groff@gnu.org; Sat, 22 Jul 2017 12:08:37 -0400
Received: from relay01.pair.com ([209.68.5.15]:52257)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)
	(Exim 4.71) (envelope-from <mbianchi@foveal.com>) id 1dYwxG-0000fs-Ii
	for groff@gnu.org; Sat, 22 Jul 2017 12:08:34 -0400
Received: from foveal8 (pool-74-102-212-139.nwrknj.fios.verizon.net
	[74.102.212.139])
	by relay01.pair.com (Postfix) with ESMTP id 5FBE5D008F8;
	Sat, 22 Jul 2017 12:08:30 -0400 (EDT)
Received: by foveal8 (Postfix, from userid 617)
	id 1FC8C11E0042; Sat, 22 Jul 2017 12:08:30 -0400 (EDT)
Date: Sat, 22 Jul 2017 12:08:30 -0400
From: Mike Bianchi <MBianchi@Foveal.com>
To: "E. Hoffmann" <erichhoffmann@gmx.de>
Cc: Peter Schaffter <peter@schaffter.ca>, groff@gnu.org
Message-ID: <20170722160829.GA18558@foveal5.foveal.com>
References: <20170720215301.7b111801@e.h>
	<20170721103000.889B01FA63@orac.inputplus.co.uk>
	<20170721155028.GA14220@augustus>
	<20170721163238.E54F8219C2@orac.inputplus.co.uk>
	<20170721202804.GA20000@augustus> <20170722170037.2f615a5f@e.h>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20170722170037.2f615a5f@e.h>
User-Agent: Mutt/1.5.23 (2014-03-12)
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy]
X-Received-From: 209.68.5.15
Subject: Re: [Groff] mom : unicode in .INCLUDE'd files
X-BeenThere: groff@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: <groff.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/groff>,
	<mailto:groff-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/groff/>
List-Post: <mailto:groff@gnu.org>
List-Help: <mailto:groff-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/groff>,
	<mailto:groff-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sat, 22 Jul 2017 16:08:38 -0000

On Sat, Jul 22, 2017 at 05:00:37PM +0200, E. Hoffmann wrote:
> Am Fri, 21 Jul 2017 16:28:04 -0400
> schrieb Peter Schaffter <peter@schaffter.ca>:
> 
> [...]
> > >     $ soelim foo | preconv | groff -Tutf8 | grep .


Is there a reason the preferred solution isn't
          $ groff -k -s  -Tutf8  foo
  ? 

see  man 1 groff :
 -k     Preprocess with  preconv .  ...
 -s     Preprocess with  soelim .


That way groff knows the crucial ordering, etc.

-- 
 Mike Bianchi


From MAILER-DAEMON Sat Jul 22 12:10:11 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dYwyo-00070y-Vt
	for mharc-groff@gnu.org; Sat, 22 Jul 2017 12:10:11 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:35576)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <mbianchi@foveal.com>) id 1dYwym-0006z4-N3
	for groff@gnu.org; Sat, 22 Jul 2017 12:10:09 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <mbianchi@foveal.com>) id 1dYwyj-0001iY-Gs
	for groff@gnu.org; Sat, 22 Jul 2017 12:10:08 -0400
Received: from relay01.pair.com ([209.68.5.15]:52268)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)
	(Exim 4.71) (envelope-from <mbianchi@foveal.com>) id 1dYwyj-0001hu-Bt
	for groff@gnu.org; Sat, 22 Jul 2017 12:10:05 -0400
Received: from foveal8 (pool-74-102-212-139.nwrknj.fios.verizon.net
	[74.102.212.139])
	by relay01.pair.com (Postfix) with ESMTP id D19EFD00828;
	Sat, 22 Jul 2017 12:10:04 -0400 (EDT)
Received: by foveal8 (Postfix, from userid 617)
	id 8E6E911E0042; Sat, 22 Jul 2017 12:10:04 -0400 (EDT)
Date: Sat, 22 Jul 2017 12:10:04 -0400
From: Mike Bianchi <MBianchi@Foveal.com>
To: "E. Hoffmann" <erichhoffmann@gmx.de>
Cc: Peter Schaffter <peter@schaffter.ca>, groff@gnu.org
Message-ID: <20170722161004.GA19473@foveal5.foveal.com>
References: <20170720215301.7b111801@e.h>
	<20170721103000.889B01FA63@orac.inputplus.co.uk>
	<20170721155028.GA14220@augustus>
	<20170721163238.E54F8219C2@orac.inputplus.co.uk>
	<20170721202804.GA20000@augustus> <20170722170037.2f615a5f@e.h>
	<20170722160829.GA18558@foveal5.foveal.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20170722160829.GA18558@foveal5.foveal.com>
User-Agent: Mutt/1.5.23 (2014-03-12)
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy]
X-Received-From: 209.68.5.15
Subject: Re: [Groff] mom : unicode in .INCLUDE'd files
X-BeenThere: groff@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: <groff.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/groff>,
	<mailto:groff-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/groff/>
List-Post: <mailto:groff@gnu.org>
List-Help: <mailto:groff-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/groff>,
	<mailto:groff-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sat, 22 Jul 2017 16:10:09 -0000

Woops.
Never mind.

								Mike


On Sat, Jul 22, 2017 at 12:08:29PM -0400, Mike Bianchi wrote:
> On Sat, Jul 22, 2017 at 05:00:37PM +0200, E. Hoffmann wrote:
> > Am Fri, 21 Jul 2017 16:28:04 -0400
> > schrieb Peter Schaffter <peter@schaffter.ca>:
> > 
> > [...]
> > > >     $ soelim foo | preconv | groff -Tutf8 | grep .
> 
> 
> Is there a reason the preferred solution isn't
>           $ groff -k -s  -Tutf8  foo
>   ? 
> 
> see  man 1 groff :
>  -k     Preprocess with  preconv .  ...
>  -s     Preprocess with  soelim .
> 
> 
> That way groff knows the crucial ordering, etc.
> 
> -- 
>  Mike Bianchi


From MAILER-DAEMON Sat Jul 22 13:19:41 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dYy44-0000bv-UH
	for mharc-groff@gnu.org; Sat, 22 Jul 2017 13:19:40 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:45670)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <kdmarsha111953@gmail.com>) id 1dYy43-0000Wn-0a
	for groff@gnu.org; Sat, 22 Jul 2017 13:19:39 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <kdmarsha111953@gmail.com>) id 1dYy3z-0005qZ-1u
	for groff@gnu.org; Sat, 22 Jul 2017 13:19:39 -0400
Received: from mail-wr0-f196.google.com ([209.85.128.196]:37706)
	by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)
	(Exim 4.71) (envelope-from <kdmarsha111953@gmail.com>)
	id 1dYy3y-0005pF-RM
	for groff@gnu.org; Sat, 22 Jul 2017 13:19:34 -0400
Received: by mail-wr0-f196.google.com with SMTP id 12so6311150wrb.4
	for <groff@gnu.org>; Sat, 22 Jul 2017 10:19:33 -0700 (PDT)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
	d=1e100.net; s=20161025;
	h=x-gm-message-state:subject:to:references:from:message-id:date
	:user-agent:mime-version:in-reply-to:content-transfer-encoding;
	bh=B/XbMUeZDo/1af9DGnDgnlr+T2PjoDyw0mJGzGSaNaI=;
	b=HUa6MXW3se3WXd4B4j7mAq1zYTHtqoxsnr4CFeQvaamwkFC1cQYBp3qaeTzbqnJ/UY
	QbmTMHEfUFUAua5R7m9PEJw21hfiXOC0cVqZSRCKb5DDlTL0m10kj6zkHNz4gbrZvP0e
	AODxktuUrYozbeAZFiyc6R4c/+CuqcF4+0+l/syyTW84obdP5hswQp2WvCFOjbdy0EYr
	L92BUS/izav5oN8vIUgPUzVkitt/c637dptZjmHXBziCua/KUKlrahcQcLPkVH9iQXfh
	ln7Um51g6aLaS6nmobvaJqFHC9OExM9uAr9JB1fnfNqUvgza4zWBqHxAZ9LhDRbJ3rUg
	5PiA==
X-Gm-Message-State: AIVw112CYyKLXm0WJRnA4oAjT8ahG5vJA47nRHeWT+E6VnAwhf19eXLu
	Cw0+oAHKR33G2Eqb
X-Received: by 10.223.144.106 with SMTP id h97mr12895706wrh.93.1500743971930; 
	Sat, 22 Jul 2017 10:19:31 -0700 (PDT)
Received: from [192.168.0.10]
	(cpc80379-grim18-2-0-cust356.12-3.cable.virginm.net. [86.17.75.101])
	by smtp.googlemail.com with ESMTPSA id
	l68sm9165797wrc.12.2017.07.22.10.19.29 for <groff@gnu.org>
	(version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
	Sat, 22 Jul 2017 10:19:30 -0700 (PDT)
To: groff@gnu.org
References: <20170720215301.7b111801@e.h>
	<20170721103000.889B01FA63@orac.inputplus.co.uk>
	<20170721155028.GA14220@augustus>
	<20170721163238.E54F8219C2@orac.inputplus.co.uk>
	<20170721203036.PeGAm%steffen@sdaoden.eu>
	<20170722135611.GC67926@athene.usta.de>
	<CAGcdajcrzUZ96Uh37MY2jbb+kotfU+-aY0r67WwRFxjeaOaN3w@mail.gmail.com>
From: Keith Marshall <keith.d.marshall@ntlworld.com>
Message-ID: <59738921.1010004@ntlworld.com>
Date: Sat, 22 Jul 2017 18:19:29 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101
	Thunderbird/38.4.0
MIME-Version: 1.0
In-Reply-To: <CAGcdajcrzUZ96Uh37MY2jbb+kotfU+-aY0r67WwRFxjeaOaN3w@mail.gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
	[fuzzy]
X-Received-From: 209.85.128.196
Subject: Re: [Groff] mom : unicode in .INCLUDE'd files
X-BeenThere: groff@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: <groff.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/groff>,
	<mailto:groff-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/groff/>
List-Post: <mailto:groff@gnu.org>
List-Help: <mailto:groff-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/groff>,
	<mailto:groff-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sat, 22 Jul 2017 17:19:40 -0000

On 22/07/17 15:06, John Gardner wrote:
> I was bitten by preconv(1) quite recently, actually. Gonna back Ingo
> here. Can I semi-seriously implore the world to only use UTF-8, and
> pretend other encodings don't exist?

Not really going to happen, for as long as MS-Windows remains the 
dominant OS for personal computer platforms.  Much as we might like 
to see this ghastly OS join other dinosaurs, in extinction, it seems 
unlikely that this will happen any time soon.

All MS-Windows internal text APIs are UTF-16LE, (which seems to be 
about the limit of Microsoft's comprehension of Unicode); the OS is 
aggressively hostile to UTF-8, (as it is to any encoding which may 
use more than two bytes to represent a single character/code-point).

-- 
Regards,
Keith.


From MAILER-DAEMON Sat Jul 22 13:31:09 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dYyFB-00027l-5z
	for mharc-groff@gnu.org; Sat, 22 Jul 2017 13:31:09 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:46758)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <gardnerjohng@gmail.com>) id 1dYyF8-00027e-IF
	for groff@gnu.org; Sat, 22 Jul 2017 13:31:07 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <gardnerjohng@gmail.com>) id 1dYyF7-0003cy-Ei
	for groff@gnu.org; Sat, 22 Jul 2017 13:31:06 -0400
Received: from mail-oi0-x22e.google.com ([2607:f8b0:4003:c06::22e]:33648)
	by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)
	(Exim 4.71) (envelope-from <gardnerjohng@gmail.com>)
	id 1dYyF7-0003cF-7u
	for groff@gnu.org; Sat, 22 Jul 2017 13:31:05 -0400
Received: by mail-oi0-x22e.google.com with SMTP id p188so73012713oia.0
	for <groff@gnu.org>; Sat, 22 Jul 2017 10:31:04 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
	h=mime-version:in-reply-to:references:from:date:message-id:subject:to
	:cc; bh=2reTjGOlGQ69MI6O1muhTAp0Sc/u7n+EQZWReYeEpRg=;
	b=mzXxf5ax+DVTjslel0wo5fLoqXT2v35YGR58VbtFrBown3+V8SXT1yXRIh0aKjpsXd
	FKpRlsHmKcoA4MB0yGAWiC+JVIGziJGh3bCFxxyRyuNSkiHVuqA6ljTjRXXFk+qlwpMA
	6kEs1+eiYp4PlKwqCrG16cq78fO62JpKoyQuTRufaM8vaUUfDmnDr0CgjhVCIlQ/K+yF
	3HHx268F+8HGJ/WivRbZYoC/NjiCmMVECDqR+WDx1UTlUco4y0noe1DZupAp4dKv6b6e
	+QtITz6/nNE2+BGsJcCyIPrWUj4guHSObIo/tB0jv2YLX/fxE9xl2EuqG7x0X9iHLnGj
	5sFw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
	d=1e100.net; s=20161025;
	h=x-gm-message-state:mime-version:in-reply-to:references:from:date
	:message-id:subject:to:cc;
	bh=2reTjGOlGQ69MI6O1muhTAp0Sc/u7n+EQZWReYeEpRg=;
	b=obh/I+VhbaJLToyEEU04GEv5W4RaYj5xS9OiAlocoao2r6uPzY/5oY5d/7W9hVCUsn
	Huke/wSSOyiDYXW745Addy1LONOwHalPvMNOoxXprgspKHIurcbZM9QUSajVljtCuKuD
	fNc8/d5SJ1fsV7JITAkN1PM6so11prfXFLKIv/6EoJfWBi9WO3QYo1dUZvvsf6dO0It5
	s/LhDTe9/u191a0kXT6+3wMFp7m/fW23XK8fb2ffvDRAcYLSQH3wQYqiPoWdW03p9z2k
	tPytsZoCyv8fhzKiyxsQfOacZeJ2l0ijAau9Ll1EyVHGFU5T0rT9ttbnbvqfFEyAtDD1
	Klcw==
X-Gm-Message-State: AIVw111gSFdAKTBRuGLrvafjL3lQRDoS8iKoxIhernKS+oZBvnphksjD
	3JI8hdDxTnPu/rOThnUrD0x639gfnw==
X-Received: by 10.202.193.85 with SMTP id r82mr5449541oif.200.1500744664337;
	Sat, 22 Jul 2017 10:31:04 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.74.43.89 with HTTP; Sat, 22 Jul 2017 10:30:48 -0700 (PDT)
In-Reply-To: <59738921.1010004@ntlworld.com>
References: <20170720215301.7b111801@e.h>
	<20170721103000.889B01FA63@orac.inputplus.co.uk>
	<20170721155028.GA14220@augustus>
	<20170721163238.E54F8219C2@orac.inputplus.co.uk>
	<20170721203036.PeGAm%steffen@sdaoden.eu>
	<20170722135611.GC67926@athene.usta.de>
	<CAGcdajcrzUZ96Uh37MY2jbb+kotfU+-aY0r67WwRFxjeaOaN3w@mail.gmail.com>
	<59738921.1010004@ntlworld.com>
From: John Gardner <gardnerjohng@gmail.com>
Date: Sun, 23 Jul 2017 03:30:48 +1000
Message-ID: <CAGcdajdZ=EiQxyJ1bTEZxGpj0xSFZ7O3+vx4gG5LMx+Go7=30g@mail.gmail.com>
To: Keith Marshall <keith.d.marshall@ntlworld.com>
Cc: groff <groff@gnu.org>
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 2607:f8b0:4003:c06::22e
Content-Type: text/plain; charset="UTF-8"
X-Content-Filtered-By: Mailman/MimeDel 2.1.21
Subject: Re: [Groff] mom : unicode in .INCLUDE'd files
X-BeenThere: groff@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: <groff.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/groff>,
	<mailto:groff-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/groff/>
List-Post: <mailto:groff@gnu.org>
List-Help: <mailto:groff-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/groff>,
	<mailto:groff-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sat, 22 Jul 2017 17:31:07 -0000

Really? I didn't know that. Although I haven't used a Windows machine for
any serious use in years, and I'm quite happy to uphold that streak.
VirtualBox-emulated environments not withstanding...

I'll spare this mailing list another aimless anti-Microsoft rant. Sure it's
heard plenty before, haha.





On 23 July 2017 at 03:19, Keith Marshall <keith.d.marshall@ntlworld.com>
wrote:

> On 22/07/17 15:06, John Gardner wrote:
> > I was bitten by preconv(1) quite recently, actually. Gonna back Ingo
> > here. Can I semi-seriously implore the world to only use UTF-8, and
> > pretend other encodings don't exist?
>
> Not really going to happen, for as long as MS-Windows remains the
> dominant OS for personal computer platforms.  Much as we might like
> to see this ghastly OS join other dinosaurs, in extinction, it seems
> unlikely that this will happen any time soon.
>
> All MS-Windows internal text APIs are UTF-16LE, (which seems to be
> about the limit of Microsoft's comprehension of Unicode); the OS is
> aggressively hostile to UTF-8, (as it is to any encoding which may
> use more than two bytes to represent a single character/code-point).
>
> --
> Regards,
> Keith.
>
>

From MAILER-DAEMON Sat Jul 22 15:32:57 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dZ093-0000gb-Sw
	for mharc-groff@gnu.org; Sat, 22 Jul 2017 15:32:57 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:60762)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <mbianchi@foveal.com>) id 1dZ092-0000gV-En
	for groff@gnu.org; Sat, 22 Jul 2017 15:32:57 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <mbianchi@foveal.com>) id 1dZ08x-0000YR-HZ
	for groff@gnu.org; Sat, 22 Jul 2017 15:32:56 -0400
Received: from relay01.pair.com ([209.68.5.15]:54828)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)
	(Exim 4.71) (envelope-from <mbianchi@foveal.com>) id 1dZ08x-0000Y9-CR
	for groff@gnu.org; Sat, 22 Jul 2017 15:32:51 -0400
Received: from foveal8 (pool-74-102-212-139.nwrknj.fios.verizon.net
	[74.102.212.139])
	by relay01.pair.com (Postfix) with ESMTP id 3F03DD004B9;
	Sat, 22 Jul 2017 15:32:50 -0400 (EDT)
Received: by foveal8 (Postfix, from userid 617)
	id F400111E0042; Sat, 22 Jul 2017 15:32:49 -0400 (EDT)
Date: Sat, 22 Jul 2017 15:32:49 -0400
From: Mike Bianchi <MBianchi@Foveal.com>
To: Keith Marshall <keith.d.marshall@ntlworld.com>
Cc: groff@gnu.org
Message-ID: <20170722193249.GA3619@foveal5.foveal.com>
References: <20170720215301.7b111801@e.h>
	<20170721103000.889B01FA63@orac.inputplus.co.uk>
	<20170721155028.GA14220@augustus>
	<20170721163238.E54F8219C2@orac.inputplus.co.uk>
	<20170721203036.PeGAm%steffen@sdaoden.eu>
	<20170722135611.GC67926@athene.usta.de>
	<CAGcdajcrzUZ96Uh37MY2jbb+kotfU+-aY0r67WwRFxjeaOaN3w@mail.gmail.com>
	<59738921.1010004@ntlworld.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <59738921.1010004@ntlworld.com>
User-Agent: Mutt/1.5.23 (2014-03-12)
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy]
X-Received-From: 209.68.5.15
Subject: Re: [Groff] mom : unicode in .INCLUDE'd files
X-BeenThere: groff@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: <groff.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/groff>,
	<mailto:groff-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/groff/>
List-Post: <mailto:groff@gnu.org>
List-Help: <mailto:groff-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/groff>,
	<mailto:groff-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sat, 22 Jul 2017 19:32:57 -0000

On Sat, Jul 22, 2017 at 06:19:29PM +0100, Keith Marshall wrote:
> On 22/07/17 15:06, John Gardner wrote:
> > ... Can I semi-seriously implore the world to only use UTF-8, and
> > pretend other encodings don't exist?
> 
> Not really going to happen, for as long as MS-Windows remains the 
> dominant OS for personal computer platforms.  

I have documents, nroff, troff and others (plus sh/ksh/awk/sed/... scripts),
dating back to the mid-1970s.  Many of those *roff documents still format
correctly.

The thing I _like_ about the *nix OSs is they don't demand I upconvert just
because a "better way" comes along.

Remember when the "modern" way to archive was to put everything onto
microfiche?

-- 
 Mike Bianchi


From MAILER-DAEMON Sat Jul 22 16:17:32 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dZ0qC-0008HV-7R
	for mharc-groff@gnu.org; Sat, 22 Jul 2017 16:17:32 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:39732)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <ted.harding@wlandres.net>) id 1dZ0q9-0008HM-LW
	for groff@gnu.org; Sat, 22 Jul 2017 16:17:30 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <ted.harding@wlandres.net>) id 1dZ0q6-0002xt-Eg
	for groff@gnu.org; Sat, 22 Jul 2017 16:17:29 -0400
Received: from smarthost03d.mail.zen.net.uk ([212.23.1.23]:51657)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <ted.harding@wlandres.net>) id 1dZ0q6-0002we-7h
	for groff@gnu.org; Sat, 22 Jul 2017 16:17:26 -0400
Received: from [88.96.44.206] (helo=[192.168.1.16])
	by smarthost03d.mail.zen.net.uk with esmtp (Exim 4.80)
	(envelope-from <ted.harding@wlandres.net>)
	id 1dZ0q3-0002jH-H7; Sat, 22 Jul 2017 20:17:23 +0000
From: Ted Harding <ted.harding@wlandres.net>
Reply-To: ted.harding@wlandres.net
To: groff@gnu.org
In-Reply-To: <20170722193249.GA3619@foveal5.foveal.com>
References: <20170720215301.7b111801@e.h>
	<20170721103000.889B01FA63@orac.inputplus.co.uk>
	<20170721155028.GA14220@augustus>
	<20170721163238.E54F8219C2@orac.inputplus.co.uk>
	<20170721203036.PeGAm%steffen@sdaoden.eu>
	<20170722135611.GC67926@athene.usta.de>
	<CAGcdajcrzUZ96Uh37MY2jbb+kotfU+-aY0r67WwRFxjeaOaN3w@mail.gmail.com>
	<59738921.1010004@ntlworld.com>
	<20170722193249.GA3619@foveal5.foveal.com>
Content-Type: text/plain
Date: Sat, 22 Jul 2017 21:17:22 +0100
Message-Id: <1500754642.3787.5.camel@deb2.fort.knox.uk>
Mime-Version: 1.0
X-Mailer: Evolution 2.22.3.1 
Content-Transfer-Encoding: 7bit
X-Originating-smarthost03d-IP: [88.96.44.206]
Feedback-ID: 88.96.44.206
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy]
X-Received-From: 212.23.1.23
Subject: Re: [Groff] mom : unicode in .INCLUDE'd files
X-BeenThere: groff@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: <groff.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/groff>,
	<mailto:groff-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/groff/>
List-Post: <mailto:groff@gnu.org>
List-Help: <mailto:groff-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/groff>,
	<mailto:groff-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sat, 22 Jul 2017 20:17:30 -0000

On Sat, 2017-07-22 at 15:32 -0400, Mike Bianchi wrote:
> On Sat, Jul 22, 2017 at 06:19:29PM +0100, Keith Marshall wrote:
> > On 22/07/17 15:06, John Gardner wrote:
> > > ... Can I semi-seriously implore the world to only use UTF-8, and
> > > pretend other encodings don't exist?
> > 
> > Not really going to happen, for as long as MS-Windows remains the 
> > dominant OS for personal computer platforms.  
> 
> I have documents, nroff, troff and others (plus sh/ksh/awk/sed/... scripts),
> dating back to the mid-1970s.  Many of those *roff documents still format
> correctly.
> 
> The thing I _like_ about the *nix OSs is they don't demand I upconvert just
> because a "better way" comes along.
> 
> Remember when the "modern" way to archive was to put everything onto
> microfiche?

I completely agree with Mike! Of course it would be a good thing
to *extend* groff's capabilities so that it can cope (optionally)
with recent developments, but in my view it *must* keep its
original capabilities, and those that have evolved since (say)
the 1980s (which is where many of my own troff source files
date back to).

Best wishes to all,
Ted.



From MAILER-DAEMON Sat Jul 22 16:23:05 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dZ0vZ-0000KL-R4
	for mharc-groff@gnu.org; Sat, 22 Jul 2017 16:23:05 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:40166)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <steffen@sdaoden.eu>) id 1dZ0vY-0000KD-1S
	for groff@gnu.org; Sat, 22 Jul 2017 16:23:05 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <steffen@sdaoden.eu>) id 1dZ0vU-0005PZ-QL
	for groff@gnu.org; Sat, 22 Jul 2017 16:23:03 -0400
Received: from sdaoden.eu ([217.144.132.164]:44862)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <steffen@sdaoden.eu>) id 1dZ0vU-0005P8-FG
	for groff@gnu.org; Sat, 22 Jul 2017 16:23:00 -0400
Received: by sdaoden.eu (Postfix, from userid 1000)
	id 5984016051; Sat, 22 Jul 2017 22:22:59 +0200 (CEST)
Date: Sat, 22 Jul 2017 22:23:39 +0200
From: Steffen Nurpmeso <steffen@sdaoden.eu>
To: "E. Hoffmann" <erichhoffmann@gmx.de>
Cc: Peter Schaffter <peter@schaffter.ca>, groff@gnu.org
Message-ID: <20170722202339.5IogZ%steffen@sdaoden.eu>
References: <20170720215301.7b111801@e.h>
	<20170721103000.889B01FA63@orac.inputplus.co.uk>
	<20170721155028.GA14220@augustus>
	<20170721163238.E54F8219C2@orac.inputplus.co.uk>
	<20170721202804.GA20000@augustus> <20170722170037.2f615a5f@e.h>
Mail-Followup-To: "E. Hoffmann" <erichhoffmann@gmx.de>,
	Peter Schaffter <peter@schaffter.ca>, groff@gnu.org
User-Agent: s-nail v14.9.0
OpenPGP: id=232C220BCB5690A37BD22FFDEB66022795F382CE;
	url=https://www.sdaoden.eu/downloads/steffen.asc
BlahBlahBlah: Any stupid boy can crush a beetle. But all the professors in
	the world can make no bugs.
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: base64
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
	[fuzzy]
X-Received-From: 217.144.132.164
Subject: Re: [Groff] mom : unicode in .INCLUDE'd files
X-BeenThere: groff@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: <groff.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/groff>,
	<mailto:groff-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/groff/>
List-Post: <mailto:groff@gnu.org>
List-Help: <mailto:groff-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/groff>,
	<mailto:groff-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sat, 22 Jul 2017 20:23:05 -0000

IkUuIEhvZmZtYW5uIiA8ZXJpY2hob2ZmbWFubkBnbXguZGU+IHdyb3RlOgogfEFtIEZyaSwgMjEg
SnVsIDIwMTcgMTY6Mjg6MDQgLTA0MDAKIHxzY2hyaWViIFBldGVyIFNjaGFmZnRlciA8cGV0ZXJA
c2NoYWZmdGVyLmNhPjoKIHwKIHxbLi4uXQogfD4+ICAgICAkIHNvZWxpbSBmb28gfCBwcmVjb252
IHwgZ3JvZmYgLVR1dGY4IHwgZ3JlcCAuCiB8Pj4gICAgIGZvbzogw4TDlsOcIFNTIMOSw5PDlMOV
xY7FjCDDhyDDpMO2w7wgw58gw7LDs8O0w7XFj8WNIMOnCiB8Pj4gICAgIGJhcjogw4TDlsOcIFNT
IMOSw5PDlMOVxY7FjCDDhyDDpMO2w7wgw58gw7LDs8O0w7XFj8WNIMOnCiB8Pj4gICAgICQKIHw+
PiAgICAgJCBncm9mZiAtViAtVHV0ZjggLXNrIGZvbwogfD4+ICAgICBwcmVjb252IGZvbyB8IHNv
ZWxpbSB8IHRyb2ZmIC1UdXRmOCB8IGdyb3R0eQogfAogfFRoYW5rcyBhIGxvdC4KIHwKIHxTbywg
dGhlIHNvbHV0aW9uJ3MgZmlyc3Qgc3RlcCBpcyB0byByZXBsYWNlIHRoZSAuSU5DTFVERSBtYWNy
byB3aXRoIHRoZQogfHBsYWluIC5zbyByZXF1ZXN0LCBhbmQgdGhlIHNlY29uZCBzdGVwIGlzIHRo
ZSBwcmVjb252LXBpcGUuCiB8CiB8U29tZXRoaW5nIHJlbWFpbnMuCiB8CiB8ICAgICRncm9mZiAt
VHV0ZjggLVYgLXNrIGV4YW1wbGUubW9tCiB8CiB8Z2l2ZXM6CiB8CiB8PnByZWNvbnYgLWV1dGY4
IGV4YW1wbGUubW9tIHwgc29lbGltIHwgdHJvZmYgLVR1dGY4IHwgZ3JvdHR5CgpIbW0sIHNvIHVu
bGVzcyBpIGdvdCBzb21ldGhpbmcgd3JvbmcgUmFscGggc2hvd2VkIHRoaXMgcGlwZWxpbmUKdXNl
cyB0aGUgZmFsc2Ugb3JkZXIsIGJlY2F1c2Ugc29lbGltIGluY2x1ZGVzIGV4dGVybmFsIGZpbGVz
IGFmdGVyCnRoZSBtYWluIGJvZHkgaGFzIGFscmVhZHkgYmVlbiBjb252ZXJ0ZWQsIHNvIHRoYXQg
dGhlIGluY2x1ZGVkCmZpbGVzIGFzIHN1Y2ggd2lsbCBub3QgYmUgY292ZXJlZCBieSBwcmVjb252
LiAgQ29ycmVjdCB3b3VsZCBiZQoKICBzb2VsaW0gfCBwcmVjb252IC1ldXRmOCBleGFtcGxlLm1v
bSB8IHRyb2ZmIC1UdXRmOCB8IGdyb3R0eQoKQnV0IHRoaXMgb2YgY291cnNlIHJlcXVpcmVzIHRo
YXQgc29lbGltIGtub3dzIGl0IGhhcyB0byBoYW5kbGUKb3RoZXIgaW5jbHVzaW9ucyB0aGFuIC5z
by4gIEkgdGhpbmsgdGhlIGJlc3Qgd291bGQgYmUgaWYgdGhlcmUKd291bGQgYmUgYSBmaWxlLWlu
Y2x1c2lvbiBjb21tYW5kIHdoaWNoIGNhbiBiZSBnaXZlbiBhIGZpbGUtCnNwZWNpZmljIGNoYXJh
Y3RlciBzZXQsIGFzIGluCgogIC5pbmNsdWRlIC1jaGFyc2V0PWxhdGluMTUgcmVzdC1vZi1saW5l
CgpvciB0aGF0IHNvZWxpbSBjYW4gbGVhcm4gbW9yZSBkaXJlY3RpdmVzIGFzIGFsdGVybmF0aXZl
IHNwZWxsaW5ncwp0byAuc28sIG1heWJlCgogIC5zb2xlYXJuIElOQ0xVREUKb3IKICAuc29wdXNo
IElOQ0xVREUKCm9yIHdoYXRldmVyLCBidXQgYWxsIHRoaXMgaXMgbm90IGF2YWlsYWJsZS4KCkFz
IGEgZ2VuZXJhbCB0aG91Z2h0IG15IGltcHJlc3Npb24gYWx3YXlzIGhhcyBiZWVuIHRoYXQgbm9y
bWFsCnVzZXJzIHNob3VsZCBub3QgYmUgYm90aGVyZWQgd2l0aCB0aGUgcGlwZWxpbmUgYXMgc3Vj
aCwgYmVjYXVzZQppIGNhbiByZW1lbWJlciBoYXZpbmcgc2VlbiAudHIgZG9jdW1lbnRzIGFuZCBi
ZWluZyB1bmFibGUgdG8gZ2V0Cml0IHJpZ2h0IGJlY2F1c2UgKGEpIGkgZGlkIG5vdCBrbm93IGFu
ZCAoYikgaSBkaWQgbm90IGtub3cgYWJvdXQKdGhlIGRvY3VtZW50LiAgTXkgaWRlYSB3YXMsIGlm
IGkgcmVjYWxsIHRoYXQgY29ycmVjdGx5LCBoYXZpbmcgbm90Cmxvb2tlZCBpbnRvIHRoaXMgZm9y
IHR3byB5ZWFycyAoYW5kIG9uZSBtb3JlIHdlZWsgdG8gY29tZSksIHRoYXQKb2YgYW4gaW50cm9k
dWN0aW9uYWwgZGVzY3JpcHRpb24gbGluZSB0aGF0IHJvZmYgY2FuIGxvb2sgYXQgYW5kCmNyZWF0
ZSB0aGUgbmVjZXNzYXJ5IHBpcGVsaW5lIGl0c2VsZi4gW0xvb2tpbmcgaW50byB0aGF0XSAgQWgg
amEsCmluZGVlZCAtLSBvaCwgaSBhbSBzb3JyeSBpZiBpIHRyYXNoIHlvdXIgdGhyZWFkIHdpdGgg
dGhpcywgYnR3ISAtLToKCiAgIEludHJvZHVjZSBhIG5ldyBjb21tYW5kIHRoYXQgY2FuIGJlIHVz
ZWQgdG8gc2VsZWN0IHRoZSBwcmVwcm9jZXNzb3JzCiAgIGV4cGxpY2l0bHksIGUuZy4sICIucHJl
cHJvY2Vzc29ycyB0YmwsZXFuIiBldGMuCiAgIElmIHNvLCBob3cgdG8gZGVhbCB3aXRoIGluY2x1
ZGVkIGZpbGVzPyAgT2ZmZXIgIi0iIG1vZGUgd2hpY2ggdHVybnMKICAgb2ZmIHByZXByb2Nlc3Nv
cnMgZm9yIGEgc3BlY2lmaWMgZmlsZT8KCiAgIEkgdGhpbmsgdGhlIGJlc3Qgd291bGQgYmUgc29t
ZSBraW5kIG9mICJzaGViYW5nIiwgZm9yIGNvbXBhdGliaWxpdGllcwogICBzYWtlIGl0IG11c3Qg
YmUgY29tbWVudC1iYXNlZCwgYXMgaW4gJy5cIiF0cm9mZiBlcW4gdGJsIGVuYz1sYXRpbjEnPwog
ICBGb3IgZXhhbXBsZSwgbmV3ZXIgbWFuKDEpcyByZWFkIHRoZSBmaXJzdCBsaW5lIG9mIHRoZSBt
YW51YWwgYW5kCiAgIGNoZWNrIGZvciBhIHN5bnRheCA8XidcIiA+Zm9sbG93ZWQgYnkgY29uY2F0
IG9mIFtlZ3BydHZdKyAoYW5kIGluCiAgIGZhY3QgICpqb2luIGluKiAkTUFOUk9GRlNFUSBlbnZp
cm9ubWVudCBbZWdwcnR2XSspCiAgICAgICAgICAgICAgICB3aGlsZSBnZXRvcHRzICdlZ3BydHYn
IHByZXByb2NfYXJnOyBkbwogICAgICAgICAgICAgICAgICAgICAgICBjYXNlICIke3ByZXByb2Nf
YXJnfSIgaW4KICAgICAgICAgICAgICAgICAgICAgICAgZSkgICAgICBwaXBlbGluZT0iJHBpcGVs
aW5lIHwgJEVRTiIgOzsKICAgICAgICAgICAgICAgICAgICAgICAgZykgICAgICBHUkFQICA7OyAj
IElnbm9yZSBmb3IgY29tcGF0aWJpbGl0eS4KICAgICAgICAgICAgICAgICAgICAgICAgcCkgICAg
ICBwaXBlbGluZT0iJHBpcGVsaW5lIHwgJFBJQyIgOzsKICAgICAgICAgICAgICAgICAgICAgICAg
cikgICAgICBwaXBlbGluZT0iJHBpcGVsaW5lIHwgJFJFRkVSIiA7OwogICAgICAgICAgICAgICAg
ICAgICAgICB0KSAgICAgIHBpcGVsaW5lPSIkcGlwZWxpbmUgfCAkVEJMIiA7OwogICAgICAgICAg
ICAgICAgICAgICAgICB2KSAgICAgIHBpcGVsaW5lPSIkcGlwZWxpbmUgfCAkVkdSSU5EIiA7Owog
ICAgICAgICAgICAgICAgICAgICAgICAqKSAgICAgIHVzYWdlIDs7CiAgICAgICAgICAgICAgICAg
ICAgICAgIGVzYWMKCiAgIFRoaXMgaXMgZ29vZCwgYnV0IHByZXR0eSBtdWNoIGNyeXB0byBhbmQg
bm90aGluZyBmb3Igbm9ybWFsIHVzZXJzLgogICBCdXQgZXh0ZW5kaW5nIHRoaXMsIHdoeSBub3Qg
Y2hlY2sgZm9yIDwnXCIgcHJlcHJvY2VzczogdGJsIGVxbiI+IGV0Yy4/CiAgIEFuZCBhcyBhYm92
ZS4KICAgQW5kIHRoaXMgaXMgYWxzbyBkb2N1bWVudGVkIGluIGdyb2ZmX3RtYWMubWFuLiAgQnV0
IGRvIG5vdCBiZXQgb24KICAgdGhpcy4KCk5vIHByZWNvbnYgaW4gdGhpcyBwaXBlbGluZSBzcGVj
LCB3ZWxsLi4uCgogfElzIHRoZXJlIHNvbWV0aGluZyBlbHNlIEkgY2FuIGRvPwoKSGFyZCBzdHVm
ZiBmb3Igbm93LgoKLS1zdGVmZmVuCnwKfERlciBLcmFnZW5iYWVyLCAgICAgICAgICAgICAgICBU
aGUgbW9vbiBiZWFyLAp8ZGVyIGhvbHQgc2ljaCBtdW50ZXIgICAgICAgICAgIGhlIGNoZWVyZnVs
bHkgYW5kIG9uZSBieSBvbmUKfGVpbmVuIG5hY2ggZGVtIGFuZGVyZW4gcnVudGVyICB3YS5rcyBo
aW1zZWxmIG9mZgp8KEJ5IFJvYmVydCBHZXJuaGFyZHQpCg==


From MAILER-DAEMON Sat Jul 22 16:47:59 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dZ1Jf-0004hH-7o
	for mharc-groff@gnu.org; Sat, 22 Jul 2017 16:47:59 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:44612)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <kdmarsha111953@gmail.com>) id 1dZ1Jd-0004hA-Pf
	for groff@gnu.org; Sat, 22 Jul 2017 16:47:58 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <kdmarsha111953@gmail.com>) id 1dZ1Ja-0006wF-N9
	for groff@gnu.org; Sat, 22 Jul 2017 16:47:57 -0400
Received: from mail-wr0-f196.google.com ([209.85.128.196]:35953)
	by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)
	(Exim 4.71) (envelope-from <kdmarsha111953@gmail.com>)
	id 1dZ1Ja-0006w3-D6
	for groff@gnu.org; Sat, 22 Jul 2017 16:47:54 -0400
Received: by mail-wr0-f196.google.com with SMTP id y67so13736429wrb.3
	for <groff@gnu.org>; Sat, 22 Jul 2017 13:47:54 -0700 (PDT)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
	d=1e100.net; s=20161025;
	h=x-gm-message-state:subject:to:references:from:message-id:date
	:user-agent:mime-version:in-reply-to:content-transfer-encoding;
	bh=n9THhShG6uvDYylUmUTSRnonzXDiCNoz/vOqat0HwaQ=;
	b=uXQw/xqMhXkB5y8deAhT2sabmbaDX8EkNut2sxbSTWerwn3FpDsqF8E8sNI5gvNjs8
	l+pB03ye2vryhDTR0uCPoSiCM4Jh7Iv6sMIx7fj8VraIljyQ6R6/OX0LTK+8+dh8jnhg
	xVtGEcIjylag/sGU2eNckut8IXIaDBhR9y5bCkUuaWBF98pl+mTeqh/KlfmIvuKpft5m
	0vwfonmk7p+mEfPnOOWaO701YVkfgSVyKBTYH3W7uFEAJ7L5MzwrPHEusheth9Z23LyF
	TD+AVBFcCO4hNjndpi5WcAA0i91FCQls84gHSFyOPButu9dg7+re6Agw9lVCWazcVJyB
	AsYw==
X-Gm-Message-State: AIVw111A6IFJdqJ56txnDRsA6b0yLkzuVsaYKAlZ0aeiD0k0w+BGklcS
	MARbCnHNLPOmaW8V
X-Received: by 10.223.171.234 with SMTP id s97mr10342721wrc.277.1500756472848; 
	Sat, 22 Jul 2017 13:47:52 -0700 (PDT)
Received: from [192.168.0.10]
	(cpc80379-grim18-2-0-cust356.12-3.cable.virginm.net. [86.17.75.101])
	by smtp.googlemail.com with ESMTPSA id
	189sm5322300wmh.0.2017.07.22.13.47.50 for <groff@gnu.org>
	(version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
	Sat, 22 Jul 2017 13:47:51 -0700 (PDT)
To: groff@gnu.org
References: <20170720215301.7b111801@e.h>
	<20170721103000.889B01FA63@orac.inputplus.co.uk>
	<20170721155028.GA14220@augustus>
	<20170721163238.E54F8219C2@orac.inputplus.co.uk>
	<20170721203036.PeGAm%steffen@sdaoden.eu>
	<20170722135611.GC67926@athene.usta.de>
	<CAGcdajcrzUZ96Uh37MY2jbb+kotfU+-aY0r67WwRFxjeaOaN3w@mail.gmail.com>
	<59738921.1010004@ntlworld.com>
	<20170722193249.GA3619@foveal5.foveal.com>
	<1500754642.3787.5.camel@deb2.fort.knox.uk>
From: Keith Marshall <keith.d.marshall@ntlworld.com>
Message-ID: <5973B9F6.7080708@ntlworld.com>
Date: Sat, 22 Jul 2017 21:47:50 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101
	Thunderbird/38.4.0
MIME-Version: 1.0
In-Reply-To: <1500754642.3787.5.camel@deb2.fort.knox.uk>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
	[fuzzy]
X-Received-From: 209.85.128.196
Subject: Re: [Groff] mom : unicode in .INCLUDE'd files
X-BeenThere: groff@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: <groff.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/groff>,
	<mailto:groff-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/groff/>
List-Post: <mailto:groff@gnu.org>
List-Help: <mailto:groff-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/groff>,
	<mailto:groff-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sat, 22 Jul 2017 20:47:58 -0000

On 22/07/17 21:17, Ted Harding wrote:
> On Sat, 2017-07-22 at 15:32 -0400, Mike Bianchi wrote:
>> On Sat, Jul 22, 2017 at 06:19:29PM +0100, Keith Marshall wrote:
>>> On 22/07/17 15:06, John Gardner wrote:
>>>> ... Can I semi-seriously implore the world to only use UTF-8, and
>>>> pretend other encodings don't exist?
>>>
>>> Not really going to happen, for as long as MS-Windows remains the 
>>> dominant OS for personal computer platforms.  
>>
>> I have documents, nroff, troff and others (plus sh/ksh/awk/sed/...
>> scripts), dating back to the mid-1970s.  Many of those *roff
>> documents still format correctly.
>>
>> The thing I _like_ about the *nix OSs is they don't demand I
>> upconvert just because a "better way" comes along.
>>
>> Remember when the "modern" way to archive was to put everything
>> onto microfiche?
> 
> I completely agree with Mike!

So do I, but my original point is completely orthogonal.  Sure, for 
those of us who are sufficiently enlightened to favour *nix platforms 
over MS-Windows, we have that luxury; unfortunately, the majority of 
computer user's today swallow the Microsoft hype, so lack any such 
enlightenment.

> Of course it would be a good thing to *extend* groff's capabilities
> so that it can cope (optionally) with recent developments, but in my
> view it *must* keep its original capabilities, and those that have
> evolved since (say) the 1980s (which is where many of my own troff
> source files date back to).

The very reason that I put significant effort into making groff run 
reliably on Windows was because, in the face of corporate stupidity 
forcing me dowm the MS-Windows path, I needed to maintain and update 
documentation for a legacy process control system, which I needed to 
keep operational up until around 2010.  Groff was a much better fit 
for that requirement than would have been MS-Word -- management's 
preference, but ultimately, entirely unsuitable.

The simple reality is that, if we wish to preserve groff's current 
utility on MS-Windows, insistence on UTF-8 only as an input encoding 
is not a viable option.

-- 
Regards,
Keith.


From MAILER-DAEMON Sat Jul 22 17:00:28 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dZ1Vh-0007Eq-Di
	for mharc-groff@gnu.org; Sat, 22 Jul 2017 17:00:28 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:45845)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <kdmarsha111953@gmail.com>) id 1dZ1Vf-0007Ej-OC
	for groff@gnu.org; Sat, 22 Jul 2017 17:00:24 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <kdmarsha111953@gmail.com>) id 1dZ1Va-0003fU-Ta
	for groff@gnu.org; Sat, 22 Jul 2017 17:00:23 -0400
Received: from mail-wr0-f179.google.com ([209.85.128.179]:35841)
	by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)
	(Exim 4.71) (envelope-from <kdmarsha111953@gmail.com>)
	id 1dZ1Va-0003fG-NO
	for groff@gnu.org; Sat, 22 Jul 2017 17:00:18 -0400
Received: by mail-wr0-f179.google.com with SMTP id y43so93085161wrd.3
	for <groff@gnu.org>; Sat, 22 Jul 2017 14:00:18 -0700 (PDT)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
	d=1e100.net; s=20161025;
	h=x-gm-message-state:subject:to:references:from:message-id:date
	:user-agent:mime-version:in-reply-to:content-transfer-encoding;
	bh=btPD/cRm1zQECXcGMAdTElj2E3TmpeKdIUWzLsLPQa0=;
	b=lXMtddAC3vt8u+lm2uFBTjtq6udXdPT6cc9GW3DadC3kHzvmTWuoSgToU8KCflyLjW
	08wdOJThmmom8Dh1Bn9lA8u4FFDCYLsj2cZufmbO/mt5rrzvYGqJc/5HXiEZvXKovwor
	gZKrF98cTSrHbgO64pf0vEc6WzMBupwnS4s7OLqa5+qBX8h0c04QBIm7ybSlsn2sZUy0
	Pa8kck6uLVecL0tLPwp1x4fKfTkC87Cfn/fEr5WmpQo2SmB5KzItElJ1tFBy0X9YAcsM
	B05AMnMzTZUxZzN0GghuHCuTwNd53pnk7SkMY0lJ8qmT7aA8fo6qaJKojyDmgb40zk6S
	Al1g==
X-Gm-Message-State: AIVw110I6I36QTXahLS/3FtQfBdrocAHXzu+s1rj1feprIjXFhvyGMIS
	3+E9bwQp99n2EjvY
X-Received: by 10.223.151.82 with SMTP id r76mr12625081wrb.6.1500757217336;
	Sat, 22 Jul 2017 14:00:17 -0700 (PDT)
Received: from [192.168.0.10]
	(cpc80379-grim18-2-0-cust356.12-3.cable.virginm.net. [86.17.75.101])
	by smtp.googlemail.com with ESMTPSA id
	l1sm181883wmb.20.2017.07.22.14.00.16 for <groff@gnu.org>
	(version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
	Sat, 22 Jul 2017 14:00:16 -0700 (PDT)
To: groff@gnu.org
References: <20170720215301.7b111801@e.h>
	<20170721103000.889B01FA63@orac.inputplus.co.uk>
	<20170721155028.GA14220@augustus>
	<20170721163238.E54F8219C2@orac.inputplus.co.uk>
	<20170721202804.GA20000@augustus>
	<20170722104409.E0B55215AA@orac.inputplus.co.uk>
From: Keith Marshall <keith.d.marshall@ntlworld.com>
Message-ID: <5973BCE0.1090500@ntlworld.com>
Date: Sat, 22 Jul 2017 22:00:16 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101
	Thunderbird/38.4.0
MIME-Version: 1.0
In-Reply-To: <20170722104409.E0B55215AA@orac.inputplus.co.uk>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
	[fuzzy]
X-Received-From: 209.85.128.179
Subject: Re: [Groff] mom : unicode in .INCLUDE'd files
X-BeenThere: groff@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: <groff.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/groff>,
	<mailto:groff-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/groff/>
List-Post: <mailto:groff@gnu.org>
List-Help: <mailto:groff-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/groff>,
	<mailto:groff-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sat, 22 Jul 2017 21:00:24 -0000

On 22/07/17 11:44, Ralph Corderoy wrote:
>> If the order of the groff pipeline is responsible for Erich's problem
>> (soelim needs to come before preconv), is this something to change or
>> something to document?
> 
> Change has been mooted in the past, including by Werner.
> Some old relevant threads.
> 
>     https://lists.gnu.org/archive/html/groff/2006-01/msg00022.html

I guess Werner's original proposal, that soelim and preconv should be 
folded into a single preprocessor, was never implemented; maybe that 
idea should be revisited?

>     https://lists.gnu.org/archive/html/groff/2010-01/msg00058.html
>     https://lists.gnu.org/archive/html/groff/2012-12/msg00030.html

-- 
Regards,
Keith.



From MAILER-DAEMON Sun Jul 23 07:38:26 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dZFDO-0003m2-EQ
	for mharc-groff@gnu.org; Sun, 23 Jul 2017 07:38:26 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:43950)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <ralph@inputplus.co.uk>) id 1dZFDM-0003lv-LE
	for groff@gnu.org; Sun, 23 Jul 2017 07:38:25 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <ralph@inputplus.co.uk>) id 1dZFDJ-0008Fd-Gu
	for groff@gnu.org; Sun, 23 Jul 2017 07:38:24 -0400
Received: from relay01.pair.com ([209.68.5.15]:32779)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)
	(Exim 4.71) (envelope-from <ralph@inputplus.co.uk>)
	id 1dZFDJ-0008FE-Bw
	for groff@gnu.org; Sun, 23 Jul 2017 07:38:21 -0400
Received: from orac.inputplus.co.uk (213.175.115.87.dyn.plus.net
	[87.115.175.213])
	by relay01.pair.com (Postfix) with ESMTP id C4FDFD0097E
	for <groff@gnu.org>; Sun, 23 Jul 2017 07:38:16 -0400 (EDT)
Received: from orac.inputplus.co.uk (orac.inputplus.co.uk [IPv6:::1])
	by orac.inputplus.co.uk (Postfix) with ESMTP id 0F5962014E
	for <groff@gnu.org>; Sun, 23 Jul 2017 12:38:16 +0100 (BST)
To: groff@gnu.org
From: Ralph Corderoy <ralph@inputplus.co.uk>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
In-reply-to: <1500754642.3787.5.camel@deb2.fort.knox.uk>
References: <20170720215301.7b111801@e.h>
	<20170721103000.889B01FA63@orac.inputplus.co.uk>
	<20170721155028.GA14220@augustus>
	<20170721163238.E54F8219C2@orac.inputplus.co.uk>
	<20170721203036.PeGAm%steffen@sdaoden.eu>
	<20170722135611.GC67926@athene.usta.de>
	<CAGcdajcrzUZ96Uh37MY2jbb+kotfU+-aY0r67WwRFxjeaOaN3w@mail.gmail.com>
	<59738921.1010004@ntlworld.com>
	<20170722193249.GA3619@foveal5.foveal.com>
	<1500754642.3787.5.camel@deb2.fort.knox.uk>
Date: Sun, 23 Jul 2017 12:38:16 +0100
Message-Id: <20170723113816.0F5962014E@orac.inputplus.co.uk>
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy]
X-Received-From: 209.68.5.15
Subject: Re: [Groff] mom : unicode in .INCLUDE'd files
X-BeenThere: groff@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: <groff.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/groff>,
	<mailto:groff-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/groff/>
List-Post: <mailto:groff@gnu.org>
List-Help: <mailto:groff-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/groff>,
	<mailto:groff-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 23 Jul 2017 11:38:25 -0000

Hi Ted,

> Mike Bianchi wrote:
> > The thing I _like_ about the *nix OSs is they don't demand I
> > upconvert just because a "better way" comes along.

Thompson and Pike knew that when designing UTF-8;  it's a superset of
ASCII and ensures a zero byte only means NUL so C strings continue as
before.

> I completely agree with Mike! Of course it would be a good thing to
> *extend* groff's capabilities so that it can cope (optionally) with
> recent developments, but in my view it *must* keep its original
> capabilities, and those that have evolved since (say) the 1980s (which
> is where many of my own troff source files date back to).

Isn't it groff's evolution that's the problem here?  Bell Labs troff
took ASCII, i.e. 7-bit.  groff added ISO-8859-1 support, another ASCII
superset, that was still one byte per rune but used 96 of the
top-bit-set bytes for more runes.  UTF-8 comes along and groff can't
adopt it because it's already taken an incompatible fork.  IIRC Bell
Labs plough on with Plan 9's troff taking UTF-8.

How many of our old documents are ISO-8859-1 instead of ASCII?  Could we
wind back the clock and make UTF-8, and thus ASCII, groff's input, with
ISO-8859-1 being the runt input character set that needs options and
hoops to jump through?

--=20
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy


From MAILER-DAEMON Sun Jul 23 07:49:48 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dZFOO-00066k-HG
	for mharc-groff@gnu.org; Sun, 23 Jul 2017 07:49:48 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:45025)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <ralph@inputplus.co.uk>) id 1dZFOL-00066V-RS
	for groff@gnu.org; Sun, 23 Jul 2017 07:49:46 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <ralph@inputplus.co.uk>) id 1dZFOI-0003Ye-PL
	for groff@gnu.org; Sun, 23 Jul 2017 07:49:45 -0400
Received: from relay01.pair.com ([209.68.5.15]:32861)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)
	(Exim 4.71) (envelope-from <ralph@inputplus.co.uk>)
	id 1dZFOI-0003YU-LK
	for groff@gnu.org; Sun, 23 Jul 2017 07:49:42 -0400
Received: from orac.inputplus.co.uk (213.175.115.87.dyn.plus.net
	[87.115.175.213])
	by relay01.pair.com (Postfix) with ESMTP id BCB4DD0077E
	for <groff@gnu.org>; Sun, 23 Jul 2017 07:49:41 -0400 (EDT)
Received: from orac.inputplus.co.uk (orac.inputplus.co.uk [IPv6:::1])
	by orac.inputplus.co.uk (Postfix) with ESMTP id EB2522014E
	for <groff@gnu.org>; Sun, 23 Jul 2017 12:49:40 +0100 (BST)
To: groff@gnu.org
From: Ralph Corderoy <ralph@inputplus.co.uk>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
In-reply-to: <5973B9F6.7080708@ntlworld.com>
References: <20170720215301.7b111801@e.h>
	<20170721103000.889B01FA63@orac.inputplus.co.uk>
	<20170721155028.GA14220@augustus>
	<20170721163238.E54F8219C2@orac.inputplus.co.uk>
	<20170721203036.PeGAm%steffen@sdaoden.eu>
	<20170722135611.GC67926@athene.usta.de>
	<CAGcdajcrzUZ96Uh37MY2jbb+kotfU+-aY0r67WwRFxjeaOaN3w@mail.gmail.com>
	<59738921.1010004@ntlworld.com>
	<20170722193249.GA3619@foveal5.foveal.com>
	<1500754642.3787.5.camel@deb2.fort.knox.uk>
	<5973B9F6.7080708@ntlworld.com>
Date: Sun, 23 Jul 2017 12:49:40 +0100
Message-Id: <20170723114940.EB2522014E@orac.inputplus.co.uk>
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy]
X-Received-From: 209.68.5.15
Subject: Re: [Groff] mom : unicode in .INCLUDE'd files
X-BeenThere: groff@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: <groff.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/groff>,
	<mailto:groff-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/groff/>
List-Post: <mailto:groff@gnu.org>
List-Help: <mailto:groff-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/groff>,
	<mailto:groff-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 23 Jul 2017 11:49:46 -0000

Hi Keith,

> The simple reality is that, if we wish to preserve groff's current
> utility on MS-Windows, insistence on UTF-8 only as an input encoding
> is not a viable option.

I understand the Windows API has moved to UTF-16, but does that mean
text file on disk are stored in that format?  I wouldn't have thought so
else we'd hear of more problems with these files being unreadable by
many programs that expect one byte per rune, or UTF-8.

If groff read its input a byte at a time from a "binary" file, parsing
the multi-byte sequences that can occur in UTF-8, would that not work on
Windows for its plain text files?

--=20
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy


From MAILER-DAEMON Sun Jul 23 07:58:23 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dZFWh-0007QI-Cw
	for mharc-groff@gnu.org; Sun, 23 Jul 2017 07:58:23 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:46048)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <ralph@inputplus.co.uk>) id 1dZFWe-0007QB-GP
	for groff@gnu.org; Sun, 23 Jul 2017 07:58:21 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <ralph@inputplus.co.uk>) id 1dZFWb-0006li-Dp
	for groff@gnu.org; Sun, 23 Jul 2017 07:58:20 -0400
Received: from relay01.pair.com ([209.68.5.15]:32905)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)
	(Exim 4.71) (envelope-from <ralph@inputplus.co.uk>)
	id 1dZFWb-0006lc-9y
	for groff@gnu.org; Sun, 23 Jul 2017 07:58:17 -0400
Received: from orac.inputplus.co.uk (213.175.115.87.dyn.plus.net
	[87.115.175.213])
	by relay01.pair.com (Postfix) with ESMTP id 57E6FD004D8
	for <groff@gnu.org>; Sun, 23 Jul 2017 07:58:16 -0400 (EDT)
Received: from orac.inputplus.co.uk (orac.inputplus.co.uk [IPv6:::1])
	by orac.inputplus.co.uk (Postfix) with ESMTP id 9B9202014E
	for <groff@gnu.org>; Sun, 23 Jul 2017 12:58:15 +0100 (BST)
To: groff@gnu.org
From: Ralph Corderoy <ralph@inputplus.co.uk>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
In-reply-to: <5973BCE0.1090500@ntlworld.com>
References: <20170720215301.7b111801@e.h>
	<20170721103000.889B01FA63@orac.inputplus.co.uk>
	<20170721155028.GA14220@augustus>
	<20170721163238.E54F8219C2@orac.inputplus.co.uk>
	<20170721202804.GA20000@augustus>
	<20170722104409.E0B55215AA@orac.inputplus.co.uk>
	<5973BCE0.1090500@ntlworld.com>
Date: Sun, 23 Jul 2017 12:58:15 +0100
Message-Id: <20170723115815.9B9202014E@orac.inputplus.co.uk>
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy]
X-Received-From: 209.68.5.15
Subject: Re: [Groff] mom : unicode in .INCLUDE'd files
X-BeenThere: groff@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: <groff.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/groff>,
	<mailto:groff-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/groff/>
List-Post: <mailto:groff@gnu.org>
List-Help: <mailto:groff-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/groff>,
	<mailto:groff-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 23 Jul 2017 11:58:21 -0000

Hi Keith,

> >     https://lists.gnu.org/archive/html/groff/2006-01/msg00022.html
>
> I guess Werner's original proposal, that soelim and preconv should be
> folded into a single preprocessor, was never implemented; maybe that
> idea should be revisited?

I can't think of problem with input working like that if sticking with
the preconv approach.  Presumably, each new source file would have the
encoding determined afresh.

--=20
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy


From MAILER-DAEMON Sun Jul 23 08:04:58 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dZFd4-00006O-T3
	for mharc-groff@gnu.org; Sun, 23 Jul 2017 08:04:58 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:47780)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <ralph@inputplus.co.uk>) id 1dZFd2-00006I-HR
	for groff@gnu.org; Sun, 23 Jul 2017 08:04:57 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <ralph@inputplus.co.uk>) id 1dZFcz-00015a-9X
	for groff@gnu.org; Sun, 23 Jul 2017 08:04:56 -0400
Received: from relay01.pair.com ([209.68.5.15]:32963)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)
	(Exim 4.71) (envelope-from <ralph@inputplus.co.uk>)
	id 1dZFcz-00015M-4C
	for groff@gnu.org; Sun, 23 Jul 2017 08:04:53 -0400
Received: from orac.inputplus.co.uk (213.175.115.87.dyn.plus.net
	[87.115.175.213])
	by relay01.pair.com (Postfix) with ESMTP id DB862D007E4;
	Sun, 23 Jul 2017 08:04:50 -0400 (EDT)
Received: from orac.inputplus.co.uk (orac.inputplus.co.uk [IPv6:::1])
	by orac.inputplus.co.uk (Postfix) with ESMTP id 387C3218CD;
	Sun, 23 Jul 2017 13:04:50 +0100 (BST)
To: "E. Hoffmann" <erichhoffmann@gmx.de>
cc: groff@gnu.org
mail-followup-to: groff@gnu.org
From: Ralph Corderoy <ralph@inputplus.co.uk>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
In-reply-to: <20170722170037.2f615a5f@e.h>
References: <20170720215301.7b111801@e.h>
	<20170721103000.889B01FA63@orac.inputplus.co.uk>
	<20170721155028.GA14220@augustus>
	<20170721163238.E54F8219C2@orac.inputplus.co.uk>
	<20170721202804.GA20000@augustus> <20170722170037.2f615a5f@e.h>
Date: Sun, 23 Jul 2017 13:04:50 +0100
Message-Id: <20170723120450.387C3218CD@orac.inputplus.co.uk>
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy]
X-Received-From: 209.68.5.15
Subject: Re: [Groff] mom : unicode in .INCLUDE'd files
X-BeenThere: groff@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: <groff.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/groff>,
	<mailto:groff-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/groff/>
List-Post: <mailto:groff@gnu.org>
List-Help: <mailto:groff-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/groff>,
	<mailto:groff-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 23 Jul 2017 12:04:57 -0000

Hi Erich,

> > >     $ soelim foo | preconv | groff -Tutf8 | grep .
...
> > >     $ groff -V -Tutf8 -sk foo
> > >     preconv foo | soelim | troff -Tutf8 | grotty
...
>     >preconv -eutf8 example.mom | soelim | troff -Tutf8 | grotty
>
> Output:
>
>     >        Abschnitt.DOCTYPE     DEFAULT UNICODE: =C3=84=E2=80=90=C3=96=
=E2=80=90=C3=9C =C3=92=E2=80=90=C3=93=E2=80=90=C3=94=E2=80=90=C3=95=E2=80=
=90=C5=8E=E2=80=90=C5=8C=E2=80=90
>     > =C3=87 =C3=A4=E2=80=90=C3=B6=E2=80=90=C3=BC=E2=80=90 =C3=9F =C3=B2=
=E2=80=90=C3=B3=E2=80=90=C3=B4=E2=80=90=C3=B5=E2=80=90=C5=8F=E2=80=90=C5=8D=
=E2=80=90 =C3=A7 so=E2=80=99eD FILE BEGINS: UNICODE: =C3=83=E2=80=90=C3=83=
=E2=80=90=C3=83  =C3=83=E2=80=90=C3=83=E2=80=90
>     > =C3=83=E2=80=90=C3=83=E2=80=90=C3=85=E2=80=90=C3=85=E2=80=90 =C3=83=
 =C3=83=C2=A4=E2=80=90=C3=83=C2=B6=E2=80=90=C3=83=C2=BC=E2=80=90 =C3=83 =C3=
=83=C2=B2=E2=80=90=C3=83=C2=B3=E2=80=90=C3=83=C2=B4=E2=80=90=C3=83=C2=B5=E2=
=80=90=C3=85=E2=80=90=C3=85=E2=80=90 =C3=83=C2=A7

And that's faulty output since you didn't request `=C2=B2', for example.

> But I want pdfmom and change that to
>
> preconv -eutf8 example.mom | soelim | pdfmom > example.preconv.pdf

And you're carrying the fault over.  My email that Peter replied to
above shows how groff gets the pipeline order wrong for your needs, and
thus -V's output is wrong too.

> As it seems, the preconv-pipe manages latin1, but not the "true"
> utf-8.

preconv is only processing example.mom, converting it to input groff can
read.  When groff does that, it sees the .so and goes off and reads a
second file, one that preconv hasn't converted.  That's you problem.

--=20
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy


From MAILER-DAEMON Sun Jul 23 08:23:59 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dZFvT-0003TB-8X
	for mharc-groff@gnu.org; Sun, 23 Jul 2017 08:23:59 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:50237)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <mbianchi@foveal.com>) id 1dZFvR-0003T4-16
	for groff@gnu.org; Sun, 23 Jul 2017 08:23:58 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <mbianchi@foveal.com>) id 1dZFvN-0008Gp-Ur
	for groff@gnu.org; Sun, 23 Jul 2017 08:23:57 -0400
Received: from relay01.pair.com ([209.68.5.15]:33063)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)
	(Exim 4.71) (envelope-from <mbianchi@foveal.com>) id 1dZFvN-0008Gc-Pe
	for groff@gnu.org; Sun, 23 Jul 2017 08:23:53 -0400
Received: from foveal8 (pool-74-102-212-139.nwrknj.fios.verizon.net
	[74.102.212.139])
	by relay01.pair.com (Postfix) with ESMTP id 5D094D00802;
	Sun, 23 Jul 2017 08:23:51 -0400 (EDT)
Received: by foveal8 (Postfix, from userid 617)
	id 1E5DD11E0042; Sun, 23 Jul 2017 08:23:51 -0400 (EDT)
Date: Sun, 23 Jul 2017 08:23:51 -0400
From: Mike Bianchi <MBianchi@Foveal.com>
To: Ralph Corderoy <ralph@inputplus.co.uk>
Cc: groff@gnu.org
Message-ID: <20170723122350.GA3446@foveal5.foveal.com>
References: <20170720215301.7b111801@e.h>
	<20170721103000.889B01FA63@orac.inputplus.co.uk>
	<20170721155028.GA14220@augustus>
	<20170721163238.E54F8219C2@orac.inputplus.co.uk>
	<20170721202804.GA20000@augustus>
	<20170722104409.E0B55215AA@orac.inputplus.co.uk>
	<5973BCE0.1090500@ntlworld.com>
	<20170723115815.9B9202014E@orac.inputplus.co.uk>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20170723115815.9B9202014E@orac.inputplus.co.uk>
User-Agent: Mutt/1.5.23 (2014-03-12)
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy]
X-Received-From: 209.68.5.15
Subject: Re: [Groff] mom : unicode in .INCLUDE'd files
X-BeenThere: groff@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: <groff.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/groff>,
	<mailto:groff-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/groff/>
List-Post: <mailto:groff@gnu.org>
List-Help: <mailto:groff-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/groff>,
	<mailto:groff-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 23 Jul 2017 12:23:58 -0000

This library purports to be a way to approach the problem ...

  https://www.autoitconsulting.com/site/development/utf-8-utf-16-text-encoding-detection-library/ 

        UTF-8 and UTF-16 Text Encoding Detection Library
        by Jonathan Bennett | Aug 23, 2014 | Development |

This post shows how to detect UTF-8 and UTF-16 text and presents a fully
functional C++ and C# library that can be used to help with the detection.

I recently had to upgrade the text file handling feature of AutoIt to better
handle text files where no byte order mark (BOM) was present.  The older
version of code I was using worked fine for UTF-8 files (with or without BOM)
but it wasn't able to detect UTF-16 files without a BOM. I tried to the the
IsTextUnicode Win32 API function but this seemed extremely unreliable and
wouldn't detect UTF-16 Big-Endian text in my tests.

Note, especially for UTF-16 detection, there is always an element of ambiguity.
This post by Raymond shows that however you try and detect encoding there will
always be some sequence of bytes that will make your guesses look stupid.

Here are the detection methods I'm currently using for the various types of
text file.  The order of the checks I perform are:

    BOM
    UTF-8
    UTF-16 (newline)
    UTF-16 (null distribution)
	:
	:

--
 Mike Bianchi


From MAILER-DAEMON Sun Jul 23 08:29:24 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dZG0h-0004BM-W0
	for mharc-groff@gnu.org; Sun, 23 Jul 2017 08:29:24 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:50756)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <gardnerjohng@gmail.com>) id 1dZG0f-0004BE-6k
	for groff@gnu.org; Sun, 23 Jul 2017 08:29:22 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <gardnerjohng@gmail.com>) id 1dZG0e-00029r-12
	for groff@gnu.org; Sun, 23 Jul 2017 08:29:21 -0400
Received: from mail-oi0-x234.google.com ([2607:f8b0:4003:c06::234]:33832)
	by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)
	(Exim 4.71) (envelope-from <gardnerjohng@gmail.com>)
	id 1dZG0d-00029E-QF
	for groff@gnu.org; Sun, 23 Jul 2017 08:29:19 -0400
Received: by mail-oi0-x234.google.com with SMTP id x3so2774871oia.1
	for <groff@gnu.org>; Sun, 23 Jul 2017 05:29:19 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
	h=mime-version:in-reply-to:references:from:date:message-id:subject:to
	:cc; bh=7vPHnULpND8AD2zWWvcpbY9kpNHv0QKfgzTcUB/ucNk=;
	b=LKqHgLexF8YWKocZpERGDJMh3KO0ObtY4FlvOC8AHW5WKmpXbRBfga8a3UeWCfLIau
	9XYXri271+haqj23Pu/n/yYY9UtA4OZ5XGRb/+51NkceLZiOB8AmsoK5Ddy7c9RV9O+8
	bg9sGzClSExJMc2kzPW6bALlVxD1rAeKlxlkokHs5ivLBdUVXJRHk/VG0UDkM8VmQt9B
	7lKCBwA+ysWgKu0bq0Wnr07Kt2vnzMZOKlGivtqx4gQpW+wsgSE5b6TO4dDnUUQJ57VN
	fVVMrlqioaTbMsQa3niCbFHQavzOWuBKrYonrBxXvb2h07jVmKxhQanI1nNMTdFekVzL
	vuZg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
	d=1e100.net; s=20161025;
	h=x-gm-message-state:mime-version:in-reply-to:references:from:date
	:message-id:subject:to:cc;
	bh=7vPHnULpND8AD2zWWvcpbY9kpNHv0QKfgzTcUB/ucNk=;
	b=HRxiQW66iemyMeC1ufqSesX5+8kEXO3Mhc87sWYjr+ztEBgtw72ktZaZ0327AI2KlP
	LDCMglian7x3q7VnDFZBlYvjrWvpKvqZiVj1BPhoVIeJH0jWgJU3MA8NHZUlSAHIJZlH
	vi5zeV7CLg3LguhwzlsYwPoRIwBqUy48IGHdC81n0FgclIM1RTUYTrkOv5jnQBhiePUA
	d1u8CR4KheWdlENsqLN6ZMT+BycIITTSe0DhRrgTS2IgWvpP7EjD9wSLcQIp332J1YWJ
	HsoKOPqn9J4ANTKlk5YUS3rGmFeGOFdolHfWPj8RhpFBDcAvcw7uG7R9T70AcNPlGdjs
	+eYg==
X-Gm-Message-State: AIVw1105nMzbfH5CEICK0KlnWzWyStEquvzZ7ej646j+G74/nFpM+jzg
	BSxOZ5WyOxBvFPQxSlt7nEhR2gvvQjIRaoE=
X-Received: by 10.202.188.10 with SMTP id m10mr6151176oif.21.1500812959055;
	Sun, 23 Jul 2017 05:29:19 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.74.43.89 with HTTP; Sun, 23 Jul 2017 05:29:03 -0700 (PDT)
In-Reply-To: <20170723122350.GA3446@foveal5.foveal.com>
References: <20170720215301.7b111801@e.h>
	<20170721103000.889B01FA63@orac.inputplus.co.uk>
	<20170721155028.GA14220@augustus>
	<20170721163238.E54F8219C2@orac.inputplus.co.uk>
	<20170721202804.GA20000@augustus>
	<20170722104409.E0B55215AA@orac.inputplus.co.uk>
	<5973BCE0.1090500@ntlworld.com>
	<20170723115815.9B9202014E@orac.inputplus.co.uk>
	<20170723122350.GA3446@foveal5.foveal.com>
From: John Gardner <gardnerjohng@gmail.com>
Date: Sun, 23 Jul 2017 22:29:03 +1000
Message-ID: <CAGcdajcJW=hCeOWKPTVNxKNvrZfRkVvJr0Hs=r+2d6FurGMMAw@mail.gmail.com>
To: Mike Bianchi <MBianchi@foveal.com>
Cc: Ralph Corderoy <ralph@inputplus.co.uk>, groff <groff@gnu.org>
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 2607:f8b0:4003:c06::234
Content-Type: text/plain; charset="UTF-8"
X-Content-Filtered-By: Mailman/MimeDel 2.1.21
Subject: Re: [Groff] mom : unicode in .INCLUDE'd files
X-BeenThere: groff@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: <groff.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/groff>,
	<mailto:groff-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/groff/>
List-Post: <mailto:groff@gnu.org>
List-Help: <mailto:groff-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/groff>,
	<mailto:groff-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 23 Jul 2017 12:29:22 -0000

>
> UTF-8 and UTF-16 Text Encoding Detection Library


That was posted in *2014?? *Suddenly I've forgotten if time's flowing
backwards or forwards...

What's the rationale for choosing UTF-16 in the first place? It offers
nothing that UTF-8 can't already handle... (to my flimsy understanding)

On 23 July 2017 at 22:23, Mike Bianchi <MBianchi@foveal.com> wrote:

> This library purports to be a way to approach the problem ...
>
>   https://www.autoitconsulting.com/site/development/utf-8-
> utf-16-text-encoding-detection-library/
>
>         UTF-8 and UTF-16 Text Encoding Detection Library
>         by Jonathan Bennett | Aug 23, 2014 | Development |
>
> This post shows how to detect UTF-8 and UTF-16 text and presents a fully
> functional C++ and C# library that can be used to help with the detection.
>
> I recently had to upgrade the text file handling feature of AutoIt to
> better
> handle text files where no byte order mark (BOM) was present.  The older
> version of code I was using worked fine for UTF-8 files (with or without
> BOM)
> but it wasn't able to detect UTF-16 files without a BOM. I tried to the the
> IsTextUnicode Win32 API function but this seemed extremely unreliable and
> wouldn't detect UTF-16 Big-Endian text in my tests.
>
> Note, especially for UTF-16 detection, there is always an element of
> ambiguity.
> This post by Raymond shows that however you try and detect encoding there
> will
> always be some sequence of bytes that will make your guesses look stupid.
>
> Here are the detection methods I'm currently using for the various types of
> text file.  The order of the checks I perform are:
>
>     BOM
>     UTF-8
>     UTF-16 (newline)
>     UTF-16 (null distribution)
>         :
>         :
>
> --
>  Mike Bianchi
>
>

From MAILER-DAEMON Sun Jul 23 08:40:07 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dZGB5-0005Iu-5c
	for mharc-groff@gnu.org; Sun, 23 Jul 2017 08:40:07 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:51700)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <ralph@inputplus.co.uk>) id 1dZGB1-0005Ie-Rn
	for groff@gnu.org; Sun, 23 Jul 2017 08:40:04 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <ralph@inputplus.co.uk>) id 1dZGAy-0007Fx-R4
	for groff@gnu.org; Sun, 23 Jul 2017 08:40:03 -0400
Received: from relay01.pair.com ([209.68.5.15]:33161)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)
	(Exim 4.71) (envelope-from <ralph@inputplus.co.uk>)
	id 1dZGAy-0007FT-Mz
	for groff@gnu.org; Sun, 23 Jul 2017 08:40:00 -0400
Received: from orac.inputplus.co.uk (213.175.115.87.dyn.plus.net
	[87.115.175.213])
	by relay01.pair.com (Postfix) with ESMTP id 468FFD00802
	for <groff@gnu.org>; Sun, 23 Jul 2017 08:39:58 -0400 (EDT)
Received: from orac.inputplus.co.uk (orac.inputplus.co.uk [IPv6:::1])
	by orac.inputplus.co.uk (Postfix) with ESMTP id 87E982014E
	for <groff@gnu.org>; Sun, 23 Jul 2017 13:39:57 +0100 (BST)
To: groff <groff@gnu.org>
From: Ralph Corderoy <ralph@inputplus.co.uk>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
In-reply-to: <CAGcdajcJW=hCeOWKPTVNxKNvrZfRkVvJr0Hs=r+2d6FurGMMAw@mail.gmail.com>
References: <20170720215301.7b111801@e.h>
	<20170721103000.889B01FA63@orac.inputplus.co.uk>
	<20170721155028.GA14220@augustus>
	<20170721163238.E54F8219C2@orac.inputplus.co.uk>
	<20170721202804.GA20000@augustus>
	<20170722104409.E0B55215AA@orac.inputplus.co.uk>
	<5973BCE0.1090500@ntlworld.com>
	<20170723115815.9B9202014E@orac.inputplus.co.uk>
	<20170723122350.GA3446@foveal5.foveal.com>
	<CAGcdajcJW=hCeOWKPTVNxKNvrZfRkVvJr0Hs=r+2d6FurGMMAw@mail.gmail.com>
Date: Sun, 23 Jul 2017 13:39:57 +0100
Message-Id: <20170723123957.87E982014E@orac.inputplus.co.uk>
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy]
X-Received-From: 209.68.5.15
Subject: Re: [Groff] mom : unicode in .INCLUDE'd files
X-BeenThere: groff@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: <groff.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/groff>,
	<mailto:groff-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/groff/>
List-Post: <mailto:groff@gnu.org>
List-Help: <mailto:groff-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/groff>,
	<mailto:groff-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 23 Jul 2017 12:40:04 -0000

Hi John,

> What's the rationale for choosing UTF-16 in the first place?

History.  Microsoft plumped for UCS-2, both UCS-2BE and UCS-2LE I think.
That's a fixed width;  two bytes per rune.  When that became
insufficient, UTF-16 was a backwards-compatible upgrade AIUI.

--=20
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy


From MAILER-DAEMON Sun Jul 23 08:45:44 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dZGGV-00066O-PV
	for mharc-groff@gnu.org; Sun, 23 Jul 2017 08:45:43 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:52544)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <gardnerjohng@gmail.com>) id 1dZGGT-00066F-FX
	for groff@gnu.org; Sun, 23 Jul 2017 08:45:42 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <gardnerjohng@gmail.com>) id 1dZGGS-0004ez-Cr
	for groff@gnu.org; Sun, 23 Jul 2017 08:45:41 -0400
Received: from mail-oi0-x230.google.com ([2607:f8b0:4003:c06::230]:32895)
	by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)
	(Exim 4.71) (envelope-from <gardnerjohng@gmail.com>)
	id 1dZGGR-0004ee-W1
	for groff@gnu.org; Sun, 23 Jul 2017 08:45:40 -0400
Received: by mail-oi0-x230.google.com with SMTP id a9so6244969oih.0
	for <groff@gnu.org>; Sun, 23 Jul 2017 05:45:39 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
	h=mime-version:in-reply-to:references:from:date:message-id:subject:to
	:cc; bh=0p5ZKIr8NOGecupmTme0CUpCnvzIlZMGlvOlGJY+JXc=;
	b=WGL/jwWUeKk+muXZ51T20BrD8rSZza80hUYe3SQoiudyfkhUPiuulJigTSRGLoRTWF
	ME0O6oaWtgOAXNtZGgPw5JYPGYZi4Oypt/m7wM4Id4mWVsWrEqJbjn3FIVx+Q8qWzpYU
	/LFiVNmJCzvEt3PFFHoeMd0WXJ7/testicuLTcu2tq8pXuxf6UZTMOFJE4QuQonsA4EE
	S6yy75s5cfBCKXQWTQn9QFL3QQlOT97OVDM+WxAg54Qk+byo3CjIZ841tgw8guE+k+Nf
	xxGQQe4gg1ABrF3rTFcR5xC919qK17y3l/a8g5SkQE1MP0uumFBRfPn3EayjbwX3R7mP
	G+cA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
	d=1e100.net; s=20161025;
	h=x-gm-message-state:mime-version:in-reply-to:references:from:date
	:message-id:subject:to:cc;
	bh=0p5ZKIr8NOGecupmTme0CUpCnvzIlZMGlvOlGJY+JXc=;
	b=lv6RQTH+E03RERI3OUCn/zXtPDGr/PrXjk2z+CcPqFFjQAc8kuBTrAyYfAXGEzfyii
	G7H0s6DnqsSRhsRkG7rSEzCk0kJ4futnvQ9CBKPJuw3ghFSs4wHucE2X+3iJf4mmzj8O
	1uhlHy2AktuNqEDfq0Ialpj5AIT6aRBv6PL7sOx96rETMBQlg9jKb2UaeAru7CL1MAZv
	5DFOp0JL60L+2w4L9k5/2li1KOM72bAr2FyXEuH+fAjKtUclhKvdJXz2/VKGRqzZ1kTu
	Ava+kPx6/KDByp3xIxn7CZMYiNbcwOq6GQeAXDkZWbmclbw2s4ISucmIcMjoNooKa4RJ
	94ig==
X-Gm-Message-State: AIVw113h/uEgkA4n9BzSq7dLBQzIYBulXvfQ7DNz1hAYILt+ViD+Q3pA
	49HKqwpcTPgKNTytDy1Hw7snNkEmPA==
X-Received: by 10.202.216.4 with SMTP id p4mr4861875oig.91.1500813939005; Sun,
	23 Jul 2017 05:45:39 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.74.43.89 with HTTP; Sun, 23 Jul 2017 05:45:23 -0700 (PDT)
In-Reply-To: <20170723123957.87E982014E@orac.inputplus.co.uk>
References: <20170720215301.7b111801@e.h>
	<20170721103000.889B01FA63@orac.inputplus.co.uk>
	<20170721155028.GA14220@augustus>
	<20170721163238.E54F8219C2@orac.inputplus.co.uk>
	<20170721202804.GA20000@augustus>
	<20170722104409.E0B55215AA@orac.inputplus.co.uk>
	<5973BCE0.1090500@ntlworld.com>
	<20170723115815.9B9202014E@orac.inputplus.co.uk>
	<20170723122350.GA3446@foveal5.foveal.com>
	<CAGcdajcJW=hCeOWKPTVNxKNvrZfRkVvJr0Hs=r+2d6FurGMMAw@mail.gmail.com>
	<20170723123957.87E982014E@orac.inputplus.co.uk>
From: John Gardner <gardnerjohng@gmail.com>
Date: Sun, 23 Jul 2017 22:45:23 +1000
Message-ID: <CAGcdajciOWV1fvLzY0cuNYSeMN1Dz5jSqpwxr8eSva6yYH-CKA@mail.gmail.com>
To: Ralph Corderoy <ralph@inputplus.co.uk>
Cc: groff <groff@gnu.org>
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 2607:f8b0:4003:c06::230
Content-Type: text/plain; charset="UTF-8"
X-Content-Filtered-By: Mailman/MimeDel 2.1.21
Subject: Re: [Groff] mom : unicode in .INCLUDE'd files
X-BeenThere: groff@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: <groff.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/groff>,
	<mailto:groff-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/groff/>
List-Post: <mailto:groff@gnu.org>
List-Help: <mailto:groff-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/groff>,
	<mailto:groff-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 23 Jul 2017 12:45:42 -0000

Strange, I remember reading that the extra storage space required for
regular 7-bit characters is what crippled the adoption of Unicode. Prior to
UTF-8, I mean...

I wonder if the serviette (napkin) which Ken Thompson plotted UTF-8 on has
a home in the MIT museum somewhere... =)

On 23 July 2017 at 22:39, Ralph Corderoy <ralph@inputplus.co.uk> wrote:

> Hi John,
>
> > What's the rationale for choosing UTF-16 in the first place?
>
> History.  Microsoft plumped for UCS-2, both UCS-2BE and UCS-2LE I think.
> That's a fixed width;  two bytes per rune.  When that became
> insufficient, UTF-16 was a backwards-compatible upgrade AIUI.
>
> --
> Cheers, Ralph.
> https://plus.google.com/+RalphCorderoy
>
>

From MAILER-DAEMON Sun Jul 23 09:02:23 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dZGWd-0000Rx-Na
	for mharc-groff@gnu.org; Sun, 23 Jul 2017 09:02:23 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:56488)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <schwarze@usta.de>) id 1dZGWa-0000RY-Qq
	for groff@gnu.org; Sun, 23 Jul 2017 09:02:21 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <schwarze@usta.de>) id 1dZGWW-0007hV-Tp
	for groff@gnu.org; Sun, 23 Jul 2017 09:02:20 -0400
Received: from scc-mailout-kit-02.scc.kit.edu ([129.13.231.82]:45391)
	by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)
	(Exim 4.71) (envelope-from <schwarze@usta.de>) id 1dZGWW-0007eg-NA
	for groff@gnu.org; Sun, 23 Jul 2017 09:02:16 -0400
Received: from asta-nat.asta.uni-karlsruhe.de ([172.22.63.82]
	helo=hekate.usta.de) by scc-mailout-kit-02.scc.kit.edu with esmtps
	(TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
	(envelope-from <schwarze@usta.de>)
	id 1dZGWR-0004YL-TG; Sun, 23 Jul 2017 15:02:13 +0200
Received: from donnerwolke.usta.de ([172.24.96.3])
	by hekate.usta.de with esmtp (Exim 4.77)
	(envelope-from <schwarze@usta.de>)
	id 1dZGWQ-0005q7-Lj; Sun, 23 Jul 2017 15:02:10 +0200
Received: from athene.usta.de ([172.24.96.10])
	by donnerwolke.usta.de with esmtp (Exim 4.84_2)
	(envelope-from <schwarze@usta.de>)
	id 1dZGWQ-00028S-9h; Sun, 23 Jul 2017 15:02:10 +0200
Received: from localhost (athene.usta.de [local])
	by athene.usta.de (OpenSMTPD) with ESMTPA id 07f766ef;
	Sun, 23 Jul 2017 15:02:10 +0200 (CEST)
Date: Sun, 23 Jul 2017 15:02:10 +0200
From: Ingo Schwarze <schwarze@usta.de>
To: Ralph Corderoy <ralph@inputplus.co.uk>
Cc: groff@gnu.org
Message-ID: <20170723130210.GA10902@athene.usta.de>
References: <20170721103000.889B01FA63@orac.inputplus.co.uk>
	<20170721155028.GA14220@augustus>
	<20170721163238.E54F8219C2@orac.inputplus.co.uk>
	<20170721203036.PeGAm%steffen@sdaoden.eu>
	<20170722135611.GC67926@athene.usta.de>
	<CAGcdajcrzUZ96Uh37MY2jbb+kotfU+-aY0r67WwRFxjeaOaN3w@mail.gmail.com>
	<59738921.1010004@ntlworld.com>
	<20170722193249.GA3619@foveal5.foveal.com>
	<1500754642.3787.5.camel@deb2.fort.knox.uk>
	<20170723113816.0F5962014E@orac.inputplus.co.uk>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20170723113816.0F5962014E@orac.inputplus.co.uk>
User-Agent: Mutt/1.6.2 (2016-07-01)
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
	[fuzzy]
X-Received-From: 129.13.231.82
Subject: Re: [Groff] mom : unicode in .INCLUDE'd files
X-BeenThere: groff@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: <groff.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/groff>,
	<mailto:groff-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/groff/>
List-Post: <mailto:groff@gnu.org>
List-Help: <mailto:groff-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/groff>,
	<mailto:groff-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 23 Jul 2017 13:02:21 -0000

Hi Ralph,

Ralph Corderoy wrote on Sun, Jul 23, 2017 at 12:38:16PM +0100:

> UTF-8 comes along and groff can't adopt it because it's already
> taken an incompatible fork.

In theory, that's true.

If you see a two-, three-, or four-byte sequence that forms a
valid UTF-8 character, it could theoretically be a sequence of
two, three, or four ISO-LATIN-1 characters.

In practice, these combinations of ISO-LATIN-1 characters are
nonsensical and simply do not occur in real-world files.

So, you can simply read the file up to the first byte with the
high bit set.  If that starts are valid UTF-8 sequence, the file
is UTF-8.  Otherwise, it is ISO-LATIN-1.  I'm doing exactly that
in mandoc, and i have never seen a misclassification in practice.
Groff could do the same and remain backward-compatible.

That doesn't even require a heavy, sophisticated library like
uchardet.

If somebody insists on processing a maliciously crafted ISO-LATIN-1
file where the first non-ASCII byte sequence looks like UTF-8, they
will have to put a charset annotation into the file or use a -K
option.  But that won't get into the way of processing historical
files because those just won't contain such nonsense.

Of course, to process native UTF-16 on Windows, preconv will be
needed just like now.  No change there.  Oh, maybe things get easier
even on Windows, because you gain the additional option to use
"iconv -t UTF-8" just like for any other text file and don't
necessarily need the special preconv(1) tool any longer.

Yours,
  Ingo


From MAILER-DAEMON Sun Jul 23 10:58:06 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dZIKc-0003B9-Gm
	for mharc-groff@gnu.org; Sun, 23 Jul 2017 10:58:06 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:40964)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <erichhoffmann@gmx.de>) id 1dZIKZ-0003B2-RE
	for groff@gnu.org; Sun, 23 Jul 2017 10:58:04 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <erichhoffmann@gmx.de>) id 1dZIKW-0002PG-Pw
	for groff@gnu.org; Sun, 23 Jul 2017 10:58:03 -0400
Received: from mout.gmx.net ([212.227.15.18]:56203)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16)
	(Exim 4.71) (envelope-from <erichhoffmann@gmx.de>)
	id 1dZIKW-0002MC-Eg
	for groff@gnu.org; Sun, 23 Jul 2017 10:58:00 -0400
Received: from e.h ([46.81.254.171]) by mail.gmx.com (mrgmx003
	[212.227.17.190]) with ESMTPSA (Nemesis) id 0LvEZe-1diXju1v4E-010PPm;
	Sun, 23 Jul 2017 16:57:52 +0200
Date: Sun, 23 Jul 2017 17:01:01 +0200
From: "E. Hoffmann" <erichhoffmann@gmx.de>
To: Ralph Corderoy <ralph@inputplus.co.uk>
Cc: groff@gnu.org
Message-ID: <20170723170101.589b486e@e.h>
In-Reply-To: <20170723120450.387C3218CD@orac.inputplus.co.uk>
References: <20170720215301.7b111801@e.h>
	<20170721103000.889B01FA63@orac.inputplus.co.uk>
	<20170721155028.GA14220@augustus>
	<20170721163238.E54F8219C2@orac.inputplus.co.uk>
	<20170721202804.GA20000@augustus> <20170722170037.2f615a5f@e.h>
	<20170723120450.387C3218CD@orac.inputplus.co.uk>
X-Mailer: Claws Mail 3.15.0-dirty (GTK+ 2.24.31; x86_64-slackware-linux-gnu)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
X-Provags-ID: V03:K0:0N6M2W5vh4G9v4IxWdOOKFCTMLSsBg+WrP3JESrW+SYV74ZCKw2
	2J3VmBSDMxPcQduGf30YEhy+XIFREuMZGIRY3MVdX49uKyvPs/GGCKikAQq8FjxCDgCs1/t
	a2Eb0SFOsdxx3m0VbN03/TQ3qSoqqNWtROvihHWw1xySDSoXdYv0UPumu/PO5nk5lxGqtHn
	rxA7YGSJZofAn2YQcaaRw==
X-UI-Out-Filterresults: notjunk:1;V01:K0:XIX88ErtAig=:/XTVoaBlGoeDHrxhSJ+JUt
	3ZfauY4j3ibLWLpXEVEBpTutJzvfGce58DWZoxXB1fXqFQ3xGi2VEydWKmhUVml0BPpDh250K
	wrtDoweQZhVU3s7Si75H7SOg1dSIgeJKxfPwc9kjOoTEVKPT56J3L85FPinvYGlUOyULuK2Kd
	kZs4lGZBH7KTX7a3VbTjhJuGm+45btyHHjZNqRBgqjqHurRMKhV3oa2bkBXPxPFP5u8DWecLu
	W/x7q4I4IjVLnOEGEz+HVNEVYlUAIafQ88IMcxkQic/FeF0akHs42pDvceHjXRFfvi1AA8XNQ
	vjJQ/Dq9eYWLkUugL1nDBt2/6UpF1EMMCBl+PFK2Gz5h+LSCEpa0MnpPd4CUtoRG0olv/pZuK
	oUTG6aNeXY1S7xAdG3fDOeIn4Kp6cTl5S7bbm1yPJ8Rhmrgbf/ZtNgltarRHDMS+kBeU07rZe
	+d7i1pozJ/NLV+dg1GB7ZAwNMSvEsLvRegtQ8EIeeUHXTSqTHm2tpXZQ5vnHzrh8QGS30+1sV
	f8gLV/Kfa+7RzLVCR3LuDTF134fJFappVOFtyUD921SO2BhwiqH1RGyYKzJn77drv00AGp+3X
	teg9uj1eUw8t93P6J/VMVr+2n0YqoDvU7BaPAh4aFTcWKZ3AtD23QVquc3SfEHhXoKyE/wTS0
	cHH60PDf9XHKKlFM9BBvASTlpl2WzSPo5TiWZQ3tMcqyECyhDIHoA5Re76PkGj3vZbL1NDbMd
	q+1Qh/4p3wwcrLEyh1URxGd/ENJX91Kp5yyCp0l80s01i14e/mtismoCAO2amaxPTaicbgA/o
	iBa7+Xlbyxbb1LJYV9/ZHff1hnYP139uMwqr60PKvj5O+MW0X4=
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
	[fuzzy]
X-Received-From: 212.227.15.18
Subject: Re: [Groff] mom : unicode in .INCLUDE'd files
X-BeenThere: groff@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: <groff.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/groff>,
	<mailto:groff-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/groff/>
List-Post: <mailto:groff@gnu.org>
List-Help: <mailto:groff-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/groff>,
	<mailto:groff-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 23 Jul 2017 14:58:04 -0000

Am Sun, 23 Jul 2example17 13:example4:5example +example1exampleexample
schrieb Ralph Corderoy <ralph@inputplus.co.uk>:

> Hi Erich,

Hi Ralph.  Thank you for looking into it.

[...]
> preconv is only processing example.mom, converting it to input groff
> can read.  When groff does that, it sees the .so and goes off and
> reads a second file, one that preconv hasn't converted.  That's you
> problem.

I wrote the following script:


----------------------------------------------------------------------\

for IN in $(ls *.mom); do
    OU=3D${IN%%.*}
            echo -n =C2=B7
                  preconv -eutf8 $IN > $OU
              done
              echo ';'

              cat example

              soelim example |  pdfmom  > example.pdf

----------------------------------------------------------------------/

The concerned lines in example.mom are:

----------------------------------------------------------------------\

.pp
latin1: =C3=84-=C3=96-=C3=9C =C3=92-=C3=93-=C3=94-=C3=95- =C3=87 =C3=A4-=C3=
=B6-=C3=BC =C3=9F =C3=B2-=C3=B3-=C3=B4-=C3=B5 =C3=A7
.PP
UNICODE:  -=C5=8E-=C5=8C- and -=C5=8F-=C5=8D-
.PP
so'eD FILE BEGINS:
.PP
.so identical_string

----------------------------------------------------------------------/

So I hoped that each momfile is preconv'ed, and yes it is, compare
identical_string:

----------------------------------------------------------------------\

.PP
latin1: \[u00C4]-\[u00D6]-\[u00DC] \[u00D2]-\[u00D3]-\[u00D4]-\[u00D5]-
\[u00C7] \[u00E4]-\[u00F6]-\[u00FC] \[u00DF]
\[u00F2]-\[u00F3]-\[u00F4]-\[u00F5] \[u00E7] .PP \# UNICODE:
-\[u014E]-\[u014C]- and -\[u014F]-\[u014D]- .PP

----------------------------------------------------------------------/

But again, only the latin1 chars show up, in both the example and the
identical_string files.  Thus, the .so-handling is _not_ the problem.

   I keep getting the error messages:

example.mom:12: warning: can't find special character `u004F_0306'
example.mom:12: warning: can't find special character `u004F_0304'
example.mom:12: warning: can't find special character `u006F_0306'
example.mom:12: warning: can't find special character `u006F_0304'

Which means, =C5=8E =C5=8C  and  =C5=8F =C5=8D are nowhere recognized.

   OK I know that I'm not the sharpest knife in the drawer, perhaps
this is beyond my level, and I admit that I can't really follow the
'-V output' discussion.  My fault.

Cheers, Erich



From MAILER-DAEMON Sun Jul 23 13:10:43 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dZKOx-0004T9-7Y
	for mharc-groff@gnu.org; Sun, 23 Jul 2017 13:10:43 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:32810)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <kdmarsha111953@gmail.com>) id 1dZKOv-0004T1-6b
	for groff@gnu.org; Sun, 23 Jul 2017 13:10:42 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <kdmarsha111953@gmail.com>) id 1dZKOr-000583-Pr
	for groff@gnu.org; Sun, 23 Jul 2017 13:10:41 -0400
Received: from mail-wr0-f180.google.com ([209.85.128.180]:34312)
	by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)
	(Exim 4.71) (envelope-from <kdmarsha111953@gmail.com>)
	id 1dZKOr-00057Z-G3
	for groff@gnu.org; Sun, 23 Jul 2017 13:10:37 -0400
Received: by mail-wr0-f180.google.com with SMTP id 12so98138148wrb.1
	for <groff@gnu.org>; Sun, 23 Jul 2017 10:10:37 -0700 (PDT)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
	d=1e100.net; s=20161025;
	h=x-gm-message-state:subject:to:references:from:message-id:date
	:user-agent:mime-version:in-reply-to:content-transfer-encoding;
	bh=2bdeDqYAMzqILpgzP184hoJFu345IVVTjrdqrUcYIDI=;
	b=OT5ym4YFXna6VUC3iaGXlC+N721Bpgd8VNwNQPyQ0w0FfuGRNJ1nIyhjcIQQV1EsrV
	p+mFi7Auw7AWHCK6BUBrxTgOP6LFNBilCDRM2UObciOk+hIo/rv/phCZ8ftAAxhpwzgX
	m0nDmb2RgbX1XR3tvaufROn9SZUj2PORtoMmuVR+SSN8NY9ehsFgB6YNbNA16mWGrcr2
	EXCcqAFnMXCaLc78qCri82fEqdO0M04mFzGhtFFlWtIYgYfgi4OsuHb1/DLVPLPHZxqX
	mP2LI9Qm21ZVDHc7g1HUbNc7fenbncTgwinUzJ4jtVjELF2Y81TaJI1TULXHel2q7buF
	0HZQ==
X-Gm-Message-State: AIVw113VS37NoJPzrRinOVQxY9kLMIQYYhxz9IQyPDl8gggIpTuO+qfx
	Y0F8a1S3qSZ6jpq5
X-Received: by 10.223.139.154 with SMTP id o26mr9348572wra.178.1500829836100; 
	Sun, 23 Jul 2017 10:10:36 -0700 (PDT)
Received: from [192.168.0.10]
	(cpc80379-grim18-2-0-cust356.12-3.cable.virginm.net. [86.17.75.101])
	by smtp.googlemail.com with ESMTPSA id
	q27sm9248939wrc.94.2017.07.23.10.10.33 for <groff@gnu.org>
	(version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
	Sun, 23 Jul 2017 10:10:33 -0700 (PDT)
To: groff@gnu.org
References: <20170720215301.7b111801@e.h>
	<20170721103000.889B01FA63@orac.inputplus.co.uk>
	<20170721155028.GA14220@augustus>
	<20170721163238.E54F8219C2@orac.inputplus.co.uk>
	<20170721203036.PeGAm%steffen@sdaoden.eu>
	<20170722135611.GC67926@athene.usta.de>
	<CAGcdajcrzUZ96Uh37MY2jbb+kotfU+-aY0r67WwRFxjeaOaN3w@mail.gmail.com>
	<59738921.1010004@ntlworld.com>
	<20170722193249.GA3619@foveal5.foveal.com>
	<1500754642.3787.5.camel@deb2.fort.knox.uk>
	<5973B9F6.7080708@ntlworld.com>
	<20170723114940.EB2522014E@orac.inputplus.co.uk>
From: Keith Marshall <keith.d.marshall@ntlworld.com>
Message-ID: <5974D889.9050807@ntlworld.com>
Date: Sun, 23 Jul 2017 18:10:33 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101
	Thunderbird/38.4.0
MIME-Version: 1.0
In-Reply-To: <20170723114940.EB2522014E@orac.inputplus.co.uk>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
	[fuzzy]
X-Received-From: 209.85.128.180
Subject: Re: [Groff] mom : unicode in .INCLUDE'd files
X-BeenThere: groff@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: <groff.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/groff>,
	<mailto:groff-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/groff/>
List-Post: <mailto:groff@gnu.org>
List-Help: <mailto:groff-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/groff>,
	<mailto:groff-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 23 Jul 2017 17:10:42 -0000

On 23/07/17 12:49, Ralph Corderoy wrote:
>> The simple reality is that, if we wish to preserve groff's current
>> utility on MS-Windows, insistence on UTF-8 only as an input encoding
>> is not a viable option.
> 
> I understand the Windows API has moved to UTF-16, but does that mean
> text file on disk are stored in that format?

Many of their tools do save in that format, yes.  Some also support 
saving with a single or DBCS encoding, compatible with current code 
page, but...

> I wouldn't have thought so else we'd hear of more problems with these
> files being unreadable by many programs that expect one byte per
> rune, or UTF-8.

...not UTF-8, (which is up to four bytes per "rune", so doesn't fit 
the single byte encoding model, the DBCS model, or the UTF-16LE model.

> If groff read its input a byte at a time from a "binary" file,
> parsing the multi-byte sequences that can occur in UTF-8, would that
> not work on Windows for its plain text files?

That could work, but with caveats.  For example, if it's a "binary" 
file, (opened in O_BINARY mode), then the C/C++ API will not swallow 
the '\r' bytes of CRLF line endings, so groff would need to handle 
them.  With a "text" file, (opened in O_TEXT mode), then those '\r' 
bytes are swallowed by the C/C++ API, so groff would never see them.  
OTOH, UTF-8 (beyond its pure ASCII subset) is not a plain text format 
on Windows, (since it conforms to neither DBCS nor UTF-16LE model, nor 
to any of the single byte encodings), so even creating such a file is 
a hurdle which not many Windows users will be willing to jump; (sure, 
it's not even difficult, e.g. to convert from UTF-16LE with iconv; 
it's just another obstacle to be overcome).

-- 
Regards,
Keith.


From MAILER-DAEMON Sun Jul 23 13:17:01 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dZKV3-0005Wi-2r
	for mharc-groff@gnu.org; Sun, 23 Jul 2017 13:17:01 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:34248)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <ddsnell@frontier.com>) id 1dZKV0-0005Wb-QG
	for groff@gnu.org; Sun, 23 Jul 2017 13:16:59 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <ddsnell@frontier.com>) id 1dZKUx-00006O-KQ
	for groff@gnu.org; Sun, 23 Jul 2017 13:16:58 -0400
Received: from filter04.dlls.pa.frontiernet.net ([199.224.80.234]:46358)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <ddsnell@frontier.com>) id 1dZKUx-00005s-GK
	for groff@gnu.org; Sun, 23 Jul 2017 13:16:55 -0400
Received: from localhost (localhost [127.0.0.1])
	by filter04.dlls.pa.frontiernet.net (Postfix) with ESMTP id D57BFC99C9
	for <groff@gnu.org>; Sun, 23 Jul 2017 17:16:53 +0000 (UTC)
Received: from relay02.dlls.pa.frontiernet.net ([199.224.80.245])
	by localhost (filter04.dlls.pa.frontiernet.net [199.224.80.234])
	(amavisd-new, port 10024)
	with LMTP id TMRSXAFESaTi for <groff@gnu.org>;
	Sun, 23 Jul 2017 17:16:40 +0000 (UTC)
X-Originating-IP: [50.109.244.145]
X-Previous-IP: 50.109.244.145
Received: from zothique.localdomain (unknown [50.109.244.145])
	by relay02.dlls.pa.frontiernet.net (Postfix) with ESMTPA id AEB007FDA
	for <groff@gnu.org>; Sun, 23 Jul 2017 17:16:39 +0000 (UTC)
Date: Sun, 23 Jul 2017 10:16:39 -0700
From: Dale Snell <ddsnell@frontier.com>
To: groff@gnu.org
Message-ID: <20170723101639.32fd43c0@zothique.localdomain>
In-Reply-To: <20170723170101.589b486e@e.h>
References: <20170720215301.7b111801@e.h>
	<20170721103000.889B01FA63@orac.inputplus.co.uk>
	<20170721155028.GA14220@augustus>
	<20170721163238.E54F8219C2@orac.inputplus.co.uk>
	<20170721202804.GA20000@augustus> <20170722170037.2f615a5f@e.h>
	<20170723120450.387C3218CD@orac.inputplus.co.uk>
	<20170723170101.589b486e@e.h>
Reply-To: ddsnell@frontier.com
Organization: Organization?  Me?  ROFL!!
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
	[fuzzy]
X-Received-From: 199.224.80.234
Subject: Re: [Groff] mom : unicode in .INCLUDE'd files
X-BeenThere: groff@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: <groff.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/groff>,
	<mailto:groff-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/groff/>
List-Post: <mailto:groff@gnu.org>
List-Help: <mailto:groff-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/groff>,
	<mailto:groff-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 23 Jul 2017 17:16:59 -0000

On Sun, 23 Jul 2017 17:01:01 +0200, in message
20170723170101.589b486e@e.h, E. Hoffmann wrote:

>    I keep getting the error messages:
>=20
> example.mom:12: warning: can't find special character `u004F_0306'
> example.mom:12: warning: can't find special character `u004F_0304'
> example.mom:12: warning: can't find special character `u006F_0306'
> example.mom:12: warning: can't find special character `u006F_0304'
>=20
> Which means, =C5=8E =C5=8C  and  =C5=8F =C5=8D are nowhere recognized.

Hi Erich,

I suspect that the typeface you're using simply doesn't have the
"char-with-breve" and "char-with-macron" characters.  That
happened to me a while back.  I needed vowels with macrons to
print out some Romaji (romanized Japanese) words.  I kept getting
errors like yours.  I did a bit of digging, and found that the
standard Palatino Roman (P) font didn't have those characters.
However, the U-P font did.  So I switched to that font and haven't
had any trouble since.  The U-<whatever> fonts should be part of
the standard 1.22.3 version of Groff.

Just a thought.

--Dale

--=20
=E2=80=9CBuild a man a fire, and he=E2=80=99ll be warm for a day.  Set a ma=
n on
fire, and he=E2=80=99ll be warm for the rest of his life.=E2=80=9D
    -- Sir Terry Pratchett.


From MAILER-DAEMON Sun Jul 23 13:50:34 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dZL1T-0001En-O0
	for mharc-groff@gnu.org; Sun, 23 Jul 2017 13:50:33 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:37026)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <erichhoffmann@gmx.de>) id 1dZL1R-0001EQ-Eb
	for groff@gnu.org; Sun, 23 Jul 2017 13:50:30 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <erichhoffmann@gmx.de>) id 1dZL1O-0006yd-Bj
	for groff@gnu.org; Sun, 23 Jul 2017 13:50:29 -0400
Received: from mout.gmx.net ([212.227.15.18]:57517)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16)
	(Exim 4.71) (envelope-from <erichhoffmann@gmx.de>)
	id 1dZL1O-0006y9-18
	for groff@gnu.org; Sun, 23 Jul 2017 13:50:26 -0400
Received: from e.h ([46.81.254.171]) by mail.gmx.com (mrgmx001
	[212.227.17.190]) with ESMTPSA (Nemesis) id 0Lev1D-1dx7WE2XHj-00qhLv;
	Sun, 23 Jul 2017 19:50:21 +0200
Date: Sun, 23 Jul 2017 19:53:31 +0200
From: "E. Hoffmann" <erichhoffmann@gmx.de>
To: ddsnell@frontier.com
Cc: groff@gnu.org
Message-Id: <20170723195331.e142bd644d56abfc8b87fb5c@gmx.de>
In-Reply-To: <20170723101639.32fd43c0@zothique.localdomain>
References: <20170720215301.7b111801@e.h>
	<20170721103000.889B01FA63@orac.inputplus.co.uk>
	<20170721155028.GA14220@augustus>
	<20170721163238.E54F8219C2@orac.inputplus.co.uk>
	<20170721202804.GA20000@augustus> <20170722170037.2f615a5f@e.h>
	<20170723120450.387C3218CD@orac.inputplus.co.uk>
	<20170723170101.589b486e@e.h>
	<20170723101639.32fd43c0@zothique.localdomain>
X-Mailer: Sylpheed 3.5.0 (GTK+ 2.24.31; x86_64-slackware-linux-gnu)
Mime-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
X-Provags-ID: V03:K0:aVopOAqw4obWRG+tsioPhvtR1Txoo/dFtvT/tTzVz98rL5Zsp+w
	6h4hz03LPPwN7DppFyPlSKczvdXzf1xN6f5T6iY/xY9n4PTzJOM2HkD8p2xKUPp5SxGBSEp
	nRucREhDvQIoMrv6C7uPJNNt7Wu7gtHtGqm4CHbfjVYOlhqMAcwAHwteWCmMCCL+ZTEUEnZ
	7Iue887T9XBMoiNznEMPA==
X-UI-Out-Filterresults: notjunk:1;V01:K0:HNzm2nr9h0g=:MFgaVZQcvpXYitSV+qtYHM
	pfeKjjCJ3Lja42N7LVxjM6Psp/hawZ9nFAnqN/ciPkQkXanxnn16n3EoiQ8kSLUbhZYcYbvTD
	dlYzJlHqHPFIQVYbqAsYoRevPVfIbxWhXBDe3mPXY/hQFrvxMgvd25W+otv5wVWsJmTzBD/9m
	HnUxpNbvQhW235UGZDQl0oBDagZP46k8gmNiSX9XVPpQ75oM8sSlDdvz3YmqqlXrjjBdZqR6d
	A0TWncSgQAW7sfw94Gt+nCoIxwXeCvxickS6Mz0UlcBNKh4WEQgUQyNDr0hkJOMCGSREQxuQV
	eKy1xuAd731jbD64tbi0/jMjY0I14pmIjoi6P2iDkrOWwF4pI5J5koa+h2m+M0Slvbr3jEDDy
	e7otMCJLI/HuwvyJAae3U+pLXIvPn8SiGrkp1J0b4H8nQOfL15OpYa5MY5qX1yo0gWZNB8Aiv
	BltfxgIrV65v7fVmSLTZS5yb2dNGQ9/GKtX+hcep/+iOz7kcXvvsVSjzDpuVC1z5gLBkUYlBp
	sqPU1QUGO7Pr5Bv9jVn4uxoUnmDLEcL30Ftj/C2yLjeWHbJ+xdKlSOZTWU3JMfHwWeZ7GfYHu
	NAQmNiobgwcKr5bu0H4YAm7iepRi/VsYMtwyLD2ws17n1M53mKEvypbOFT0y15wub/xWl7VlQ
	O/T1dJJ2f5pOH5CENIDO6GCZJNZf+WrKH0Ph0ArCEd3h/AM38Y7+Jroo+71DFDd77iVWpptGz
	X747Kj52/0Jc/rGwU25z2U4HXCfAGkkSPZghFEtduECbqhuiH3XuCN5uY+YQ6tR5yaIrPQVSU
	Ydk0dTsfYra0B9Z7O7UrdCY9STc0StG748yZZuH6kfnC3SUCNk=
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
	[fuzzy]
X-Received-From: 212.227.15.18
Subject: Re: [Groff] mom : unicode in .INCLUDE'd files SOLVED
X-BeenThere: groff@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: <groff.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/groff>,
	<mailto:groff-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/groff/>
List-Post: <mailto:groff@gnu.org>
List-Help: <mailto:groff-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/groff>,
	<mailto:groff-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 23 Jul 2017 17:50:30 -0000

On Sun, 23 Jul 2017 10:16:39 -0700
Dale Snell <ddsnell@frontier.com> wrote:

[...]
> Hi Erich,

Hi Dale.  Thank you, I think you are right.

> I suspect that the typeface you're using simply doesn't have the
> "char-with-breve" and "char-with-macron" characters.  That
> happened to me a while back.  I needed vowels with macrons to
> print out some Romaji (romanized Japanese) words.

Similiar with me, I need them to print Pinyin (romanized Chinese).

> I kept getting
> errors like yours.  I did a bit of digging, and found that the
> standard Palatino Roman (P) font didn't have those characters.
> However, the U-P font did.  So I switched to that font and haven't
> had any trouble since.  The U-<whatever> fonts should be part of
> the standard 1.22.3 version of Groff.

The U-<whatever> do the trick.

Now I only have to figure out how mom sets the basic font.
the .FAMILY macro doesn't accept seem to accept U-***, I have to insert
the .FT U-TR &c. after each .PP macro.

Thanks!

erich


--
"Alles wird gut."  (Eine Hoffnung)


From MAILER-DAEMON Sun Jul 23 13:57:33 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dZL8H-0002GF-Pc
	for mharc-groff@gnu.org; Sun, 23 Jul 2017 13:57:33 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:37746)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <kdmarsha111953@gmail.com>) id 1dZL8F-0002G8-Ng
	for groff@gnu.org; Sun, 23 Jul 2017 13:57:32 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <kdmarsha111953@gmail.com>) id 1dZL8A-0001jy-Rm
	for groff@gnu.org; Sun, 23 Jul 2017 13:57:31 -0400
Received: from mail-wm0-f42.google.com ([74.125.82.42]:35337)
	by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)
	(Exim 4.71) (envelope-from <kdmarsha111953@gmail.com>)
	id 1dZL8A-0001f7-KH
	for groff@gnu.org; Sun, 23 Jul 2017 13:57:26 -0400
Received: by mail-wm0-f42.google.com with SMTP id c184so8293261wmd.0
	for <groff@gnu.org>; Sun, 23 Jul 2017 10:57:24 -0700 (PDT)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
	d=1e100.net; s=20161025;
	h=x-gm-message-state:subject:to:references:from:message-id:date
	:user-agent:mime-version:in-reply-to:content-transfer-encoding;
	bh=Ol7qIOCnETPU8Uvf6/ac1P64pJy+Xu7GgVD//81Jew4=;
	b=rSC7/v6wdody8SYq6OUoCkT0oLLPdni9XkalTQysKUVwDfWs4/OKFusGxtWdetXIfo
	+VNG59i594YSjoqbRG12XhqOtX//HYMdXyBbVk5yNVzIcmGRD7BdHXmEZ+JevbshzKb9
	p0cV2DLI1a7RoQgrUkLLaVQqEWi9ypcumYz/FarZDoJjUtvmY7pVf6R5/qnF/F35eavo
	1r4abEjgis/Ae4/EU8cYdpBlmbczIEu/D1/+Npi4qibG4B3w6U52PeWuTTifTk+Aqq3+
	Y/wYstf3Fyd6It9EugfRwLYP7jKMQ+OfJeK5GOoTyMKOZVmKt62roM+I+w1WKHoirDGw
	NuzA==
X-Gm-Message-State: AIVw110+cqNvVrSZAfopP1v6bTwa+EcsME+/IDp/kh0cZKuB0NmpUpeK
	oHNa7nRDk7wps51F
X-Received: by 10.28.183.196 with SMTP id h187mr3147396wmf.134.1500832643481; 
	Sun, 23 Jul 2017 10:57:23 -0700 (PDT)
Received: from [192.168.0.10]
	(cpc80379-grim18-2-0-cust356.12-3.cable.virginm.net. [86.17.75.101])
	by smtp.googlemail.com with ESMTPSA id
	i7sm9334544wrc.54.2017.07.23.10.57.22 for <groff@gnu.org>
	(version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
	Sun, 23 Jul 2017 10:57:22 -0700 (PDT)
To: groff@gnu.org
References: <20170720215301.7b111801@e.h>
	<20170721103000.889B01FA63@orac.inputplus.co.uk>
	<20170721155028.GA14220@augustus>
	<20170721163238.E54F8219C2@orac.inputplus.co.uk>
	<20170721202804.GA20000@augustus>
	<20170722104409.E0B55215AA@orac.inputplus.co.uk>
	<5973BCE0.1090500@ntlworld.com>
	<20170723115815.9B9202014E@orac.inputplus.co.uk>
	<20170723122350.GA3446@foveal5.foveal.com>
	<CAGcdajcJW=hCeOWKPTVNxKNvrZfRkVvJr0Hs=r+2d6FurGMMAw@mail.gmail.com>
	<20170723123957.87E982014E@orac.inputplus.co.uk>
From: Keith Marshall <keith.d.marshall@ntlworld.com>
Message-ID: <5974E381.6040904@ntlworld.com>
Date: Sun, 23 Jul 2017 18:57:21 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101
	Thunderbird/38.4.0
MIME-Version: 1.0
In-Reply-To: <20170723123957.87E982014E@orac.inputplus.co.uk>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
	[fuzzy]
X-Received-From: 74.125.82.42
Subject: Re: [Groff] mom : unicode in .INCLUDE'd files
X-BeenThere: groff@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: <groff.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/groff>,
	<mailto:groff-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/groff/>
List-Post: <mailto:groff@gnu.org>
List-Help: <mailto:groff-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/groff>,
	<mailto:groff-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 23 Jul 2017 17:57:33 -0000

On 23/07/17 13:39, Ralph Corderoy wrote:
>> What's the rationale for choosing UTF-16 in the first place?
> 
> History.  Microsoft plumped for UCS-2, both UCS-2BE and UCS-2LE
> I think.

That's not as I recall it: UCS-2, yes, but always UCS-2LE, (since their 
focus was on Intel x86 -- a little-endian memory organization, so only 
little-endian UCS-2 would be useful for representation of wchar_t).

> That's a fixed width;  two bytes per rune.  When that became
> insufficient, UTF-16 was a backwards-compatible upgrade AIUI.

Yep.  Yet another of Bill's "no one will ever need more than 640kB of 
memory" moments, IIRC: "16-bits should be sufficient to represent any 
character which anyone will ever want to display".  Of course, he was 
wrong on both counts, and when they realized that 16-bits wasn't going 
to be enough, they changed their definition of Unicode[*] to represent 
UTF-16LE, and added support for surrogate pairs to the APIs.

[*]: When Microsoft documentation refers to "Unicode", they invariably 
mean UTF-16LE; they seem reluctant to as much as acknowledge that any 
other variant exists; (there are a few rare, hard to find, instances 
where UTF-7 or UTF-8 are mentioned ... and then, usually to caution 
against using them).

-- 
Regards,
Keith.


From MAILER-DAEMON Sun Jul 23 14:23:21 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dZLXF-00064X-9J
	for mharc-groff@gnu.org; Sun, 23 Jul 2017 14:23:21 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:40390)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <ddsnell@frontier.com>) id 1dZLXD-00064R-Kn
	for groff@gnu.org; Sun, 23 Jul 2017 14:23:20 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <ddsnell@frontier.com>) id 1dZLXA-0004cU-HY
	for groff@gnu.org; Sun, 23 Jul 2017 14:23:19 -0400
Received: from filter04.dlls.pa.frontiernet.net ([199.224.80.234]:53382)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <ddsnell@frontier.com>) id 1dZLXA-0004cI-Ck
	for groff@gnu.org; Sun, 23 Jul 2017 14:23:16 -0400
Received: from localhost (localhost [127.0.0.1])
	by filter04.dlls.pa.frontiernet.net (Postfix) with ESMTP id CD03A4B149; 
	Sun, 23 Jul 2017 18:23:13 +0000 (UTC)
Received: from relay02.dlls.pa.frontiernet.net ([199.224.80.245])
	by localhost (filter04.dlls.pa.frontiernet.net [199.224.80.234])
	(amavisd-new, port 10024)
	with LMTP id HsINJLX4hWuo; Sun, 23 Jul 2017 18:23:02 +0000 (UTC)
X-Originating-IP: [50.109.244.145]
X-Previous-IP: 50.109.244.145
Received: from zothique.localdomain (unknown [50.109.244.145])
	by relay02.dlls.pa.frontiernet.net (Postfix) with ESMTPA id 236A47F42; 
	Sun, 23 Jul 2017 18:23:01 +0000 (UTC)
Date: Sun, 23 Jul 2017 11:21:13 -0700
From: Dale Snell <ddsnell@frontier.com>
To: groff@gnu.org
Message-ID: <20170723112113.5242674d@zothique.localdomain>
In-Reply-To: <20170723195331.e142bd644d56abfc8b87fb5c@gmx.de>
References: <20170720215301.7b111801@e.h>
	<20170721103000.889B01FA63@orac.inputplus.co.uk>
	<20170721155028.GA14220@augustus>
	<20170721163238.E54F8219C2@orac.inputplus.co.uk>
	<20170721202804.GA20000@augustus> <20170722170037.2f615a5f@e.h>
	<20170723120450.387C3218CD@orac.inputplus.co.uk>
	<20170723170101.589b486e@e.h>
	<20170723101639.32fd43c0@zothique.localdomain>
	<20170723195331.e142bd644d56abfc8b87fb5c@gmx.de>
Reply-To: ddsnell@frontier.com
Organization: Department of redundant redundancies and tautological
	pleonasms department
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha256;
	boundary="Sig_/1dFyYolCIZ.6Xm/78+4Z8MT";
	protocol="application/pgp-signature"
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
	[fuzzy]
X-Received-From: 199.224.80.234
Subject: Re: [Groff] mom : unicode in .INCLUDE'd files SOLVED
X-BeenThere: groff@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: <groff.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/groff>,
	<mailto:groff-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/groff/>
List-Post: <mailto:groff@gnu.org>
List-Help: <mailto:groff-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/groff>,
	<mailto:groff-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 23 Jul 2017 18:23:20 -0000

--Sig_/1dFyYolCIZ.6Xm/78+4Z8MT
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

On Sun, 23 Jul 2017 19:53:31 +0200, in message
20170723195331.e142bd644d56abfc8b87fb5c@gmx.de, E. Hoffmann wrote:

> The U-<whatever> do the trick.
>=20
> Now I only have to figure out how mom sets the basic font.
> the .FAMILY macro doesn't accept seem to accept U-***, I have to
> insert the .FT U-TR &c. after each .PP macro.

That's odd, it works fine for me.  I've got

    .FAMILY U-P
    .FT R

in various .mom files, and I've not had any trouble with the fonts
loading.  Actually, I've put those lines in a "style-sheet" file
that gets loaded into my regular .mom files.  I don't know what
the problem could be.  What error messages are you seeing?

--Dale

--=20
=E2=80=9CThe clean sleek looks of Star Trek is where we will aim and
Babylon 5 is where we=E2=80=99ll end up.=E2=80=9D
   -- Dwight Shultz

--Sig_/1dFyYolCIZ.6Xm/78+4Z8MT
Content-Type: application/pgp-signature
Content-Description: OpenPGP digital signature

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJZdOkZAAoJEB7OMKhvWRr8ZdQH/0lhKsIkjIs+wTQdzavT5L2E
Cn8+MS6X3Ybgbq7qNPoQzXBmM4Zc9eAvb1EKkItuvnNGmzqbejO4dcBu5hPyOk8w
BHuTGLgIl1iUTdMvfM9wQj8fVN+1fgUh2Mq3OjzhxpG8Hbdi6set8oz60kiBJk+o
AmqRWWUxoiAe+jegylRlpu2EoOh36z8jDsM4a/ogn0jAJMCkCqUhJ3HkJ9OGAkMl
NZ8Mwk3kdvM4E3Safpa923Al6Rk6hs94+P3Dektc7ahr6d1lwVBBOa5TqewuQKHH
e90jl/TVHSaXoFGcTSt7dnZzDj9Yl8a+TSEwo5jb+KrlWX9AdAe2ZbG4/F+VGeg=
=Fv2w
-----END PGP SIGNATURE-----

--Sig_/1dFyYolCIZ.6Xm/78+4Z8MT--


From MAILER-DAEMON Sun Jul 23 15:01:18 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dZM7y-0003VR-3u
	for mharc-groff@gnu.org; Sun, 23 Jul 2017 15:01:18 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:47532)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <schwarze@usta.de>) id 1dZM7u-0003Tx-Fw
	for groff@gnu.org; Sun, 23 Jul 2017 15:01:16 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <schwarze@usta.de>) id 1dZM7s-0000v0-VZ
	for groff@gnu.org; Sun, 23 Jul 2017 15:01:14 -0400
Received: from scc-mailout-kit-01.scc.kit.edu
	([2a00:1398:9:f712::810d:e751]:33471)
	by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)
	(Exim 4.71) (envelope-from <schwarze@usta.de>) id 1dZM7s-0000rE-Lp
	for groff@gnu.org; Sun, 23 Jul 2017 15:01:12 -0400
Received: from asta-nat.asta.uni-karlsruhe.de ([172.22.63.82]
	helo=hekate.usta.de) by scc-mailout-kit-01.scc.kit.edu with esmtps
	(TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
	(envelope-from <schwarze@usta.de>) id 1dZM7n-0008Kj-GP
	for groff@gnu.org; Sun, 23 Jul 2017 21:01:08 +0200
Received: from donnerwolke.usta.de ([172.24.96.3])
	by hekate.usta.de with esmtp (Exim 4.77)
	(envelope-from <schwarze@usta.de>) id 1dZM7n-0003aj-8d
	for groff@gnu.org; Sun, 23 Jul 2017 21:01:07 +0200
Received: from athene.usta.de ([172.24.96.10])
	by donnerwolke.usta.de with esmtp (Exim 4.84_2)
	(envelope-from <schwarze@usta.de>) id 1dZM7m-00068p-VF
	for groff@gnu.org; Sun, 23 Jul 2017 21:01:07 +0200
Received: from localhost (athene.usta.de [local])
	by athene.usta.de (OpenSMTPD) with ESMTPA id c83dada5
	for <groff@gnu.org>; Sun, 23 Jul 2017 21:01:06 +0200 (CEST)
Date: Sun, 23 Jul 2017 21:01:06 +0200
From: Ingo Schwarze <schwarze@usta.de>
To: groff@gnu.org
Message-ID: <20170723190106.GA88489@athene.usta.de>
References: <20170620160800.GE64805@athene.usta.de>
	<20170620.181627.1649857539413198133.wl@gnu.org>
	<20170624003920.GA69848@athene.usta.de>
	<20170624.094200.1665329325929415906.wl@gnu.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20170624.094200.1665329325929415906.wl@gnu.org>
User-Agent: Mutt/1.6.2 (2016-07-01)
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
X-Received-From: 2a00:1398:9:f712::810d:e751
Subject: Re: [Groff] Why does tty-char.tmac not represent meaning?
X-BeenThere: groff@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: <groff.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/groff>,
	<mailto:groff-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/groff/>
List-Post: <mailto:groff@gnu.org>
List-Help: <mailto:groff-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/groff>,
	<mailto:groff-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 23 Jul 2017 19:01:16 -0000

Hi,

Werner LEMBERG wrote on Sat, Jun 24, 2017 at 09:42:00AM +0200:

> After a quick glance: LGTM.  Thanks for working on this.

I just completed this work, handling national language characters
in the same way.  Unless there are objections, it would be nice
if somebody with commit access could put it in.

After that, i will change mandoc to match.

For ease of reference, i'm appending the full rationale and the
complete patch to this mail.

Yours,
  Ingo


 ----- 8< ----- schnipp ----- >8 ----- 8< ----- schnapp ----- >8 -----

Since there was positive feedback and no concerns raised, here is
an improved and completed version of the patch.

tty-char_symbols.patch
  is a modified version of the original patch with exactly one
  change: incorporating Doug McIlroy's suggestion to print "<nabla>"
  for \(gr, to avoid possible confusion over the wording "gradient"
  when it is used in a scalar product (div) or vector product (rot).

tty-char_letters.patch
  addresses national language characters, using the same principle.
  Some notes:

  1. The bulk of the patch changes the representation of Greek
     letters.  Neither the old nor the new representation is adequate
     for reading Greek text.  You *really* have to use -Tutf8 for
     Greek text.  But the new representation makes mathematical
     formulae intelligible.

  2. \(ss is represented as simply "ss".  As a native German speaker,
     i can confirm that this is definitely the right thing to do,
     and there is no reasonable alternative.

  3. The Spanish inverted exclamation and question marks are
     represented by the same characters in upright position.
     While that will probably look strange to Spanish readers,
     i can think of no better way, and i think everybody will
     understand it.  The current "i" and "c" is unintelligible.

  4. Icelandic Eth is represented as "Dh", Thorn as "Th".  If you
     have a better idea, that's fine with me; i don't speak Icelandic,
     so i may well be wrong in this respect.

tty-char_combined.patch
is a combination of both patches (appended below).

 ----- 8< ----- schnipp ----- >8 ----- 8< ----- schnapp ----- >8 -----

diff --git a/tmac/tty-char.tmac b/tmac/tty-char.tmac
index 93ca2b7c..943e454f 100644
--- a/tmac/tty-char.tmac
+++ b/tmac/tty-char.tmac
@@ -9,7 +9,8 @@
 .\" - work with all of -Tascii, -Tlatin1, -Tutf8, and -Tcp1047.
 .\" - work on devices that display only the last overstruck character
 .\"   as well as on devices that support overstriking
-.\" - represent the character's graphical shape (not its meaning)
+.\" - help understanding the character's meaning, only aiming to imitate
+.\"   a particular graphical shape when that doesn't hinder understanding
 .\"
 .\" Note that the optical appearance of the definitions contained in this
 .\" file is inferior compared to that of the replacement characters defined
@@ -30,70 +31,68 @@
 .tty-char \[rn] \*[tty-rn]
 .tty-char \[ua] \z|^
 .tty-char \[da] \z|v
-.tty-char \[sc] S
+.tty-char \[sc] <section>
 .tty-char \[ct] \z/c
-.tty-char \[dg] \z|-
-.tty-char \[dd] \z|=
-.tty-char \[ib] (\z=_
-.tty-char \[ip] \z=_)
-.tty-char \[sb] (=
-.tty-char \[sp] =)
-.tty-char \[if] oo
-.tty-char \[pt] oc
+.tty-char \[dg] <*>
+.tty-char \[dd] <**>
+.tty-char \[ib] <subset\~or\~equal>
+.tty-char \[ip] <superset\~or\~equal>
+.tty-char \[sb] <proper\~subset>
+.tty-char \[sp] <proper\~superset>
+.tty-char \[if] <infinity>
+.tty-char \[pt] <proportional\~to>
 .tty-char \[es] {}
-.tty-char \[ca] (^)
-.tty-char \[cu] U
-.tty-char \[de] o
-.tty-char \[di] -:-
-.tty-char \[tdi] -:-
+.tty-char \[ca] <intersection>
+.tty-char \[cu] <union>
+.tty-char \[de] <degree>
+.tty-char \[di] /
+.tty-char \[tdi] /
 .tty-char \[no] ~
 .tty-char \[tno] ~
-.tty-char \[gr] \Z'\*[tty-rn]'V
-.tty-char \[is] \z'\z,I
-.tty-char \[mo] E
-.tty-char \[pd] a
-.tty-char \[sr] \e/
-.tty-char \[*C] \z_H
-.tty-char \[*D] \z_/\z_\e
-.tty-char \[*F] \zIO
-.tty-char \[*G] |\*[tty-rn]
-.tty-char \[*H] \z-O
-.tty-char \[*L] /\e
-.tty-char \[*P] TT
-.tty-char \[*Q] \zIY
-.tty-char \[*S] \z_\Z'\*[tty-rn]'>
-.tty-char \[*W] \z_O
-.if c\[ss] .tty-char \[*b] \[ss]
-.tty-char \[*b] B
-.tty-char \[*a] a
-.tty-char \[*c] \z,E
-.tty-char \[*d] d
-.tty-char \[*e] e
-.tty-char \[+e] e
-.tty-char \[*f] \z|o
-.tty-char \[+f] \z|o
-.tty-char \[*g] y
-.tty-char \[*h] \z-0
-.tty-char \[+h] \z-0
-.tty-char \[*i] i
-.tty-char \[*k] k
-.tty-char \[*l] \z>\e
-.tty-char \[*m] \z,u
-.tty-char \[*n] v
-.tty-char \[*p] \z-n
-.tty-char \[+p] \z-w
-.tty-char \[*q] \z|u
-.tty-char \[*r] p
-.tty-char \[*s] \z-o
-.tty-char \[*t] \z~t
-.tty-char \[*u] u
-.tty-char \[*w] w
-.tty-char \[*x] x
-.tty-char \[*y] n
-.tty-char \[*z] \z,C
-.tty-char \[ts] s
-.\" Definition of \[ss] should follow that of \[*b].
-.tty-char \[ss] B
+.tty-char \[gr] <nabla>
+.tty-char \[is] <integral>
+.tty-char \[mo] <element\~of>
+.tty-char \[pd] <del>
+.tty-char \[sr] <sqrt>
+.tty-char \[*C] <Xi>
+.tty-char \[*D] <Delta>
+.tty-char \[*F] <Phi>
+.tty-char \[*G] <Gamma>
+.tty-char \[*H] <Theta>
+.tty-char \[*L] <Lambda>
+.tty-char \[*P] <Pi>
+.tty-char \[*Q] <Psi>
+.tty-char \[*S] <Sigma>
+.tty-char \[*W] <Omega>
+.tty-char \[*b] <beta>
+.tty-char \[*a] <alpha>
+.tty-char \[*c] <xi>
+.tty-char \[*d] <delta>
+.tty-char \[*e] <epsilon>
+.tty-char \[+e] <epsilon>
+.tty-char \[*f] <phi>
+.tty-char \[+f] <phi>
+.tty-char \[*g] <gamma>
+.tty-char \[*h] <theta>
+.tty-char \[+h] <theta>
+.tty-char \[*i] <iota>
+.tty-char \[*k] <kappa>
+.tty-char \[*l] <lambda>
+.tty-char \[*m] <mu>
+.tty-char \[*n] <nu>
+.tty-char \[*p] <pi>
+.tty-char \[+p] <pi>
+.tty-char \[*q] <psi>
+.tty-char \[*r] <rho>
+.tty-char \[*s] <sigma>
+.tty-char \[*t] <tau>
+.tty-char \[*u] <upsilon>
+.tty-char \[*w] <omega>
+.tty-char \[*x] <chi>
+.tty-char \[*y] <eta>
+.tty-char \[*z] <zeta>
+.tty-char \[ts] <sigma>
+.tty-char \[ss] ss
 .tty-char \[c*] \zO\[mu]
 .tty-char \[c+] \zO+
 .tty-char \[AN] ^
@@ -103,31 +102,31 @@
 .if c\[md] .tty-char \[pc] \[md]
 .if c\[pc] .tty-char \[md] \[pc]
 .if c\[pc] .tty-char \[a.] \[pc]
-.tty-char \[Im] I
-.tty-char \[Re] R
+.tty-char \[Im] <Im>
+.tty-char \[Re] <Re>
 .tty-char \[/L] \z/L
 .tty-char \[/l] \z/l
-.tty-char \[%0] %o
+.tty-char \[%0] <permille>
 .tty-char \[ao] o
 .tty-char \[a"] """"
 .tty-char \[ab] \z'`
 .tty-char \[ah] v
 .tty-char \[ho] \[ac]
-.tty-char \[/_] \z_/
+.tty-char \[/_] <angle>
 .tty-char \[=~] =~
 .tty-char \[|=] -~
-.tty-char \[Ah] N
-.tty-char \[CR] _|
-.tty-char \[fa] \z-V
-.tty-char \[nm] \z/E
-.tty-char \[pp] \z_|
+.tty-char \[Ah] <Aleph>
+.tty-char \[CR] <cr>
+.tty-char \[fa] <for\~all>
+.tty-char \[nm] <not\~element\~of>
+.tty-char \[pp] <perpendicular>
 .tty-char \[sd] ''
-.tty-char \[st] -)
-.tty-char \[te] 3
+.tty-char \[st] <such\~that>
+.tty-char \[te] <there\~exists>
 .if c\[md] .tty-char \[tf] .\[md].
-.tty-char \[tf] .:.
+.tty-char \[tf] <therefore>
 .if c\[md] .tty-char \[3d] .\[md].
-.tty-char \[3d] .:.
+.tty-char \[3d] <therefore>
 .tty-char \[wp] p
 .tty-char \[~~] ~~
 .tty-char \[Fn] \z,f
@@ -144,7 +143,7 @@
 .tty-char \[lt] ,-
 .tty-char \[rt] -.
 .\" Latin characters
-.tty-char \[r!] \z,i
+.tty-char \[r!] !
 .tty-char \[Po] \z-L
 .tty-char \[Cs] \zox
 .tty-char \[Ye] \z=Y
@@ -153,16 +152,16 @@
 .tty-char \[Of] \z_a
 .tty-char \[Fo] <<
 .tty-char \[a-] \*[tty-rn]
-.tty-char \[S2] 2
-.tty-char \[S3] 3
-.tty-char \[ps] 9|
+.tty-char \[S2] ^2
+.tty-char \[S3] ^3
+.tty-char \[ps] <paragraph>
 .tty-char \[md] .
 .tty-char \[pc] .
 .tty-char \[ac] ,
-.tty-char \[S1] 1
+.tty-char \[S1] ^1
 .tty-char \[Om] \z_o
 .tty-char \[Fc] >>
-.tty-char \[r?] \z'c
+.tty-char \[r?] ?
 .tty-char \[`A] \z`A
 .tty-char \['A] \z'A
 .tty-char \[^A] \z^A
@@ -179,7 +178,7 @@
 .tty-char \['I] \z'I
 .tty-char \[^I] \z^I
 .tty-char \[:I] \z"I
-.tty-char \[-D] \z-D
+.tty-char \[-D] Dh
 .tty-char \[~N] \z~N
 .tty-char \[`O] \z`O
 .tty-char \['O] \z'O
@@ -192,7 +191,7 @@
 .tty-char \[^U] \z^U
 .tty-char \[:U] \z"U
 .tty-char \['Y] \z'Y
-.tty-char \[TP] \zIb
+.tty-char \[TP] Th
 .tty-char \[`a] \z`a
 .tty-char \['a] \z'a
 .tty-char \[^a] \z^a
@@ -209,7 +208,7 @@
 .tty-char \['i] \z'i
 .tty-char \[^i] \z^i
 .tty-char \[:i] \z"i
-.tty-char \[Sd] \z`\z'o
+.tty-char \[Sd] dh
 .tty-char \[~n] \z~n
 .tty-char \[`o] \z`o
 .tty-char \['o] \z'o
@@ -222,7 +221,7 @@
 .tty-char \[^u] \z^u
 .tty-char \[:u] \z"u
 .tty-char \['y] \z'y
-.tty-char \[Tp] \zpb
+.tty-char \[Tp] th
 .tty-char \[:y] \z"y
 .\" for Turkish
 .tty-char "\[G ab]" G
@@ -230,7 +229,7 @@
 .tty-char "\[I .]" I
 .\"tty-char \[:y] \ij
 .tty-char \[arrowvertex] |
-.tty-char \[mc] \z,u
+.tty-char \[mc] <micro>
 .cp \n[_C]
 .
 .ie '\*(.T'cp1047' \


From MAILER-DAEMON Sun Jul 23 16:49:31 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dZNoh-0008Q9-9I
	for mharc-groff@gnu.org; Sun, 23 Jul 2017 16:49:31 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:34032)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <deri@chuzzlewit.myzen.co.uk>) id 1dZNoe-0008Q2-QV
	for groff@gnu.org; Sun, 23 Jul 2017 16:49:29 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <deri@chuzzlewit.myzen.co.uk>) id 1dZNob-0001So-Mg
	for groff@gnu.org; Sun, 23 Jul 2017 16:49:28 -0400
Received: from smarthost01d.mail.zen.net.uk ([212.23.1.7]:51729)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <deri@chuzzlewit.myzen.co.uk>) id 1dZNob-0001Py-BX
	for groff@gnu.org; Sun, 23 Jul 2017 16:49:25 -0400
Received: from [82.71.22.80] (helo=pip.chuzzlewit.co.uk)
	by smarthost01d.mail.zen.net.uk with esmtpsa
	(TLS1.0:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.80)
	(envelope-from <deri@chuzzlewit.myzen.co.uk>)
	id 1dZNoK-0004Sd-Lo; Sun, 23 Jul 2017 20:49:08 +0000
From: Deri James <deri@chuzzlewit.myzen.co.uk>
To: groff@gnu.org, ddsnell@frontier.com
Date: Sun, 23 Jul 2017 21:49:08 +0100
Message-ID: <10148543.2B2auInLYV@pip.chuzzlewit.co.uk>
User-Agent: KMail/4.14.5 (Linux/4.4.74-desktop-1.mga5; KDE/4.14.30; x86_64; ; )
In-Reply-To: <20170723101639.32fd43c0@zothique.localdomain>
References: <20170720215301.7b111801@e.h> <20170723170101.589b486e@e.h>
	<20170723101639.32fd43c0@zothique.localdomain>
MIME-Version: 1.0
X-Originating-smarthost01d-IP: [82.71.22.80]
Feedback-ID: 82.71.22.80
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 212.23.1.7
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: 7Bit
X-Content-Filtered-By: Mailman/MimeDel 2.1.21
Subject: Re: [Groff] mom : unicode in .INCLUDE'd files
X-BeenThere: groff@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: <groff.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/groff>,
	<mailto:groff-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/groff/>
List-Post: <mailto:groff@gnu.org>
List-Help: <mailto:groff-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/groff>,
	<mailto:groff-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 23 Jul 2017 20:49:29 -0000

On Sun 23 Jul 2017 10:16:39 Dale Snell wrote:
> Hi Erich,
> 
> I suspect that the typeface you're using simply doesn't have the
> "char-with-breve" and "char-with-macron" characters.  That
> happened to me a while back.  I needed vowels with macrons to
> print out some Romaji (romanized Japanese) words.  I kept getting
> errors like yours.  I did a bit of digging, and found that the
> standard Palatino Roman (P) font didn't have those characters.
> However, the U-P font did.  So I switched to that font and haven't
> had any trouble since.  The U-<whatever> fonts should be part of
> the standard 1.22.3 version of Groff.
> 
> Just a thought.
> 
> --Dale
> 

Hi Dale,

It ought to work if you include:-

-P-yU

On the pdfmom command line. This should tell gropdf to use the U- foundry, so you don't have to alter the 
actual mom document.

Cheers 

Deri


From MAILER-DAEMON Sun Jul 23 17:22:08 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dZOKG-0006ds-Lu
	for mharc-groff@gnu.org; Sun, 23 Jul 2017 17:22:08 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:37595)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <ddsnell@frontier.com>) id 1dZOKE-0006dk-T0
	for groff@gnu.org; Sun, 23 Jul 2017 17:22:07 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <ddsnell@frontier.com>) id 1dZOKB-0005K6-Nu
	for groff@gnu.org; Sun, 23 Jul 2017 17:22:06 -0400
Received: from filter01.dlls.pa.frontiernet.net ([199.224.80.228]:41143)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <ddsnell@frontier.com>) id 1dZOKB-0005Jl-K2
	for groff@gnu.org; Sun, 23 Jul 2017 17:22:03 -0400
Received: from localhost (localhost [127.0.0.1])
	by filter01.dlls.pa.frontiernet.net (Postfix) with ESMTP id 9A51117578A
	for <groff@gnu.org>; Sun, 23 Jul 2017 21:22:01 +0000 (UTC)
Received: from relay01.dlls.pa.frontiernet.net ([199.224.80.244])
	by localhost (filter01.dlls.pa.frontiernet.net [199.224.80.228])
	(amavisd-new, port 10024)
	with LMTP id bC7HWvMP7n96 for <groff@gnu.org>;
	Sun, 23 Jul 2017 21:21:49 +0000 (UTC)
X-Originating-IP: [50.109.244.145]
X-Previous-IP: 50.109.244.145
Received: from zothique.localdomain (unknown [50.109.244.145])
	by relay01.dlls.pa.frontiernet.net (Postfix) with ESMTPA id 10B3139975
	for <groff@gnu.org>; Sun, 23 Jul 2017 21:21:48 +0000 (UTC)
Date: Sun, 23 Jul 2017 14:21:47 -0700
From: Dale Snell <ddsnell@frontier.com>
To: groff@gnu.org
Message-ID: <20170723142147.201c7015@zothique.localdomain>
In-Reply-To: <10148543.2B2auInLYV@pip.chuzzlewit.co.uk>
References: <20170720215301.7b111801@e.h> <20170723170101.589b486e@e.h>
	<20170723101639.32fd43c0@zothique.localdomain>
	<10148543.2B2auInLYV@pip.chuzzlewit.co.uk>
Reply-To: ddsnell@frontier.com
Organization: Organization?  Me?  ROFL!!
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
	[fuzzy]
X-Received-From: 199.224.80.228
Subject: Re: [Groff] mom : unicode in .INCLUDE'd files
X-BeenThere: groff@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: <groff.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/groff>,
	<mailto:groff-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/groff/>
List-Post: <mailto:groff@gnu.org>
List-Help: <mailto:groff-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/groff>,
	<mailto:groff-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 23 Jul 2017 21:22:08 -0000

On Sun, 23 Jul 2017 21:49:08 +0100, in message
10148543.2B2auInLYV@pip.chuzzlewit.co.uk, Deri James wrote:

> Hi Dale,
> 
> It ought to work if you include:-
> 
> -P-yU
> 
> On the pdfmom command line. This should tell gropdf to use the U-
> foundry, so you don't have to alter the actual mom document.

Thanks, Deri, that's something I didn't know.  I must have missed
it when I read the docs for gropdf.  That or I simply forgot about
that option, since I normally use pdfmom to build my PDFs.

I suppose should re-read the docs.  :-)

--Dale

-- 
When the fog came in on little cat feet last night, it left these
little muddy paw prints on the hood of my car.


From MAILER-DAEMON Sun Jul 23 17:26:35 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dZOOZ-0007Iq-28
	for mharc-groff@gnu.org; Sun, 23 Jul 2017 17:26:35 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:38416)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <palopezv@gmail.com>) id 1dZOOV-0007Ia-FJ
	for groff@gnu.org; Sun, 23 Jul 2017 17:26:33 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <palopezv@gmail.com>) id 1dZOOS-0007Ro-8g
	for groff@gnu.org; Sun, 23 Jul 2017 17:26:31 -0400
Received: from mail-ua0-f176.google.com ([209.85.217.176]:34878)
	by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)
	(Exim 4.71) (envelope-from <palopezv@gmail.com>) id 1dZOOS-0007QL-4I
	for groff@gnu.org; Sun, 23 Jul 2017 17:26:28 -0400
Received: by mail-ua0-f176.google.com with SMTP id d29so56610346uai.2
	for <groff@gnu.org>; Sun, 23 Jul 2017 14:26:26 -0700 (PDT)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
	d=1e100.net; s=20161025;
	h=x-gm-message-state:message-id:mime-version:to:from:subject:date
	:importance:in-reply-to:references;
	bh=lavUKUxJ7+a/m+UyowNinsHkJgUoAft4L7YBLdUva+8=;
	b=VvibODN0+QpKTnAf11P9yRV2jc4edxMqK1Ke4iHNEqy8xaNfLteKU1b+0cZXw8Na72
	zIF2q8NJ6V3KfaQDr7wNJ9jlNtg455adUBG5m8wWvRh4TMQ5mapQcf7HUOcoV3sogiEt
	zSIoV42u6ivPvDBDmlnxpH5/5bolfUO/9g9d+yKGdZ/MUw0VDsmdNBJCvMgBIAGH4h18
	Mh3/lqF7pbsJBTMWLYujxRT4xoo9hAxdq6a7v4xxUlC4QkFvq9gYsErlN3H6QjJwV/tD
	TxRy356AOmxctwKyTsfl1h/GTvz5tueXoFfLguNb4+l7cVIv6npA58tG0zAF+CYxoc2m
	w5AA==
X-Gm-Message-State: AIVw111fSo9CFcKddFNfh4v4ypR26NaqUeZiuJPcUow+PT7ZIu7Zxm+E
	foCkC3dsx3fgL7KH6FM=
X-Received: by 10.176.95.131 with SMTP id b3mr7185216uaj.99.1500845185592;
	Sun, 23 Jul 2017 14:26:25 -0700 (PDT)
Received: from ?IPv6:::ffff:192.168.0.2? ([200.118.223.109])
	by smtp.gmail.com with ESMTPSA id 61sm2329117uao.14.2017.07.23.14.26.24
	for <groff@gnu.org>
	(version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
	Sun, 23 Jul 2017 14:26:25 -0700 (PDT)
Message-ID: <59751481.c3239f0a.394bd.28c6@mx.google.com>
MIME-Version: 1.0
To: "groff@gnu.org" <groff@gnu.org>
From: =?utf-8?Q?Pedro_A._L=C3=B3pez-Valencia?= <palopezv@gmail.com>
Date: Sun, 23 Jul 2017 16:26:24 -0500
Importance: normal
X-Priority: 3
In-Reply-To: <20170723190106.GA88489@athene.usta.de>
References: <20170620160800.GE64805@athene.usta.de>
	<20170620.181627.1649857539413198133.wl@gnu.org>
	<20170624003920.GA69848@athene.usta.de>
	<20170624.094200.1665329325929415906.wl@gnu.org>
	<20170723190106.GA88489@athene.usta.de>
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
	[fuzzy]
X-Received-From: 209.85.217.176
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
X-Content-Filtered-By: Mailman/MimeDel 2.1.21
Subject: Re: [Groff] Why does tty-char.tmac not represent meaning?
X-BeenThere: groff@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: <groff.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/groff>,
	<mailto:groff-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/groff/>
List-Post: <mailto:groff@gnu.org>
List-Help: <mailto:groff-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/groff>,
	<mailto:groff-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 23 Jul 2017 21:26:33 -0000


  3. The Spanish inverted exclamation and question marks are
     represented by the same characters in upright position.
     While that will probably look strange to Spanish readers,
     i can think of no better way, and i think everybody will
     understand it.  The current "i" and "c" is unintelligible.

Yes, as a native Spanish speaker, I agree. Nice application of the principl=
e of least surprise, I should add.
=20

From MAILER-DAEMON Mon Jul 24 02:38:29 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dZX0f-0002qN-5q
	for mharc-groff@gnu.org; Mon, 24 Jul 2017 02:38:29 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:40117)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <hohe72@arcor.de>) id 1dZX0d-0002q3-Qx
	for groff@gnu.org; Mon, 24 Jul 2017 02:38:28 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <hohe72@arcor.de>) id 1dZX0a-0007Yz-Mv
	for groff@gnu.org; Mon, 24 Jul 2017 02:38:27 -0400
Received: from vsmx012.vodafonemail.xion.oxcs.net ([153.92.174.90]:21792)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)
	(Exim 4.71) (envelope-from <hohe72@arcor.de>) id 1dZX0a-0007Sz-FY
	for groff@gnu.org; Mon, 24 Jul 2017 02:38:24 -0400
Received: from vsmx004.vodafonemail.xion.oxcs.net (unknown [192.168.75.198])
	by mta-8-out.mta.xion.oxcs.net (Postfix) with ESMTP id 829578CDF6F
	for <groff@gnu.org>; Mon, 24 Jul 2017 06:38:10 +0000 (UTC)
Received: from bivalve.home (unknown [89.244.101.141])
	by mta-8-out.mta.xion.oxcs.net (Postfix) with ESMTPA id 2C312CDF7E
	for <groff@gnu.org>; Mon, 24 Jul 2017 06:38:07 +0000 (UTC)
Date: Mon, 24 Jul 2017 08:38:05 +0200
From: <hohe72@arcor.de>
To: groff@gnu.org
Message-ID: <20170724083805.3b404a71@bivalve.home>
In-Reply-To: <20170723170101.589b486e@e.h>
References: <20170720215301.7b111801@e.h>
	<20170721103000.889B01FA63@orac.inputplus.co.uk>
	<20170721155028.GA14220@augustus>
	<20170721163238.E54F8219C2@orac.inputplus.co.uk>
	<20170721202804.GA20000@augustus> <20170722170037.2f615a5f@e.h>
	<20170723120450.387C3218CD@orac.inputplus.co.uk>
	<20170723170101.589b486e@e.h>
X-Operating-System: MSDOS 4.5
Mime-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
X-VADE-STATUS: LEGIT
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
	[fuzzy]
X-Received-From: 153.92.174.90
Subject: Re: [Groff] mom : unicode in .INCLUDE'd files
X-BeenThere: groff@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: <groff.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/groff>,
	<mailto:groff-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/groff/>
List-Post: <mailto:groff@gnu.org>
List-Help: <mailto:groff-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/groff>,
	<mailto:groff-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 24 Jul 2017 06:38:28 -0000


For the protocol, write a macro:

.de SYS-include    \" .SYS-include  filename  extension
.ds SYS-filename \\$1
.sy preconv -e utf8 \\*[SYS-filename]  \
   | grap | pic | tbl | eqn -T\\*(.T > \\*[SYS-filename].intermediate
.so \\*[SYS-filename].intermediate
.sy rm \\*[SYS-filename].intermediate
..

Note that for instance, picture processing inside included files will
not be handled as well, because the .so request is processed by groff
(or troff), whereas the picture requests (.PS,.PE) are processed by
pic beforehand. So at the call to pic the file isn't included.

This is similar to a call to preconv.

Please note, that I don't know the various command lines that
(partially) work around that, nor the popular macros.

> example.mom:12: warning: can't find special character `u004F_0306'

If I guess right, this happens because the deployed font doesn't have a
glyph at the designated unicode location/ number.



From MAILER-DAEMON Tue Jul 25 01:42:21 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dZsbt-0007AP-B4
	for mharc-groff@gnu.org; Tue, 25 Jul 2017 01:42:21 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:51092)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <erichhoffmann@gmx.de>) id 1dZsbq-000791-UW
	for groff@gnu.org; Tue, 25 Jul 2017 01:42:19 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <erichhoffmann@gmx.de>) id 1dZsbn-0004ac-MT
	for groff@gnu.org; Tue, 25 Jul 2017 01:42:18 -0400
Received: from mout.gmx.net ([212.227.17.21]:55828)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16)
	(Exim 4.71) (envelope-from <erichhoffmann@gmx.de>)
	id 1dZsbn-0004YZ-CW
	for groff@gnu.org; Tue, 25 Jul 2017 01:42:15 -0400
Received: from e.h ([46.81.254.171]) by mail.gmx.com (mrgmx102
	[212.227.17.168]) with ESMTPSA (Nemesis) id 0MGXV6-1dMhJS0aEy-00DJE5;
	Tue, 25 Jul 2017 07:42:10 +0200
Date: Tue, 25 Jul 2017 07:45:24 +0200
From: "E. Hoffmann" <erichhoffmann@gmx.de>
To: groff@gnu.org
Cc: ddsnell@frontier.com
Message-Id: <20170725074524.c65952e276b29dcd10bcfaaa@gmx.de>
In-Reply-To: <20170723112113.5242674d@zothique.localdomain>
References: <20170720215301.7b111801@e.h>
	<20170721103000.889B01FA63@orac.inputplus.co.uk>
	<20170721155028.GA14220@augustus>
	<20170721163238.E54F8219C2@orac.inputplus.co.uk>
	<20170721202804.GA20000@augustus> <20170722170037.2f615a5f@e.h>
	<20170723120450.387C3218CD@orac.inputplus.co.uk>
	<20170723170101.589b486e@e.h>
	<20170723101639.32fd43c0@zothique.localdomain>
	<20170723195331.e142bd644d56abfc8b87fb5c@gmx.de>
	<20170723112113.5242674d@zothique.localdomain>
X-Mailer: Sylpheed 3.5.0 (GTK+ 2.24.31; x86_64-slackware-linux-gnu)
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-Provags-ID: V03:K0:292OJwnoAwz4VeZAPQkwBseOvxDR+tXf3q318a4w36KnVlNDxGZ
	IsfY9+gcXlU/YiajQQItbsQfhXaYayevBsXrKu8clh9wmPKJTzzTNJWjCGi9ybpgMMLulK5
	aco+Eu3Q8lE6+V3R5JMD2yWDkHWPd6jNCXK4bE3535/fVN0nIuOlp28+Ls0aIDvfkPnetU1
	I/KGYA4RGGcQfUQYBA4TA==
X-UI-Out-Filterresults: notjunk:1;V01:K0:1BWZcoq6R8w=:BtA5OnaaDiMpaSvSgZebsL
	+73bNWJCA+F1zpPXaKBvjDCXNlj99Rn99PhlhFAlDO+nIZuYPPS1q8wcQmOBQP0VtNVdS0VkN
	4dqWkXAGrb2YndCp37tRMVqbm9DRMKI5Xbx1YX8aZwc2g1THrXre4h0aLAA5RLQk8jIThigSf
	OBTcu6XdvpMlBT82xa9X1iXfGS7n7xMNJN/fGuIF6Pc6iCkSxpbPilxfdJDoLJqtWeAhkX2O8
	8A6Q5WEs8L6OPjyCJ8H8kcNb2HKpKL3vq9WpouHruDhJiAA+CDIdoWdgu7oHklC3DnCc6MAW1
	n0m2o9LFH+ZUqi0q/nLswBUCk961MIUbK6IsBBH3wQfUHJ7EAt+U1muwAHy/6a03b7UXUzWHa
	rc2kp61ohKojhw9xgJLBBgNh4PK49vxkA6O7rVV3jZG71QQZCQU8JoHMTL5doxntXVeHTbewZ
	OFJktjjxINRq4s8mvgAtGLJzIawlmOpwkurPAggU6a1QHdno85j/0JBSqtqUtYO2Ys35VFLOQ
	AbwTBZl7VrWZeubBe2VIXgrAltm+qiMzzHP5NAHpeIdz2Q6yJkUpcwTyQy5JCaCJEZeYEsPYi
	xQ9sciU0fx5X7Q2MDvlr3gRr3DzOaDwUZqoBCRI6XWwCS4jJ7Cs13cApP/D5RZGo6y3GFBer2
	6sJGQPGMbYL9rlYzO66jgCv/17nKHGhhNNQxiFkv2TxPgJK3YCvjJzdf+tLDRR8nIWyjC2R3t
	xBJe1aFkGGv2gEih4ofz+7a5LAYpNW3r0i9alYChBkl+L1MpW+FhodcwHTh+pTmSqDTbz2Wrb
	UXpFHBgjAb5faJtnF9kzlUB1EkvJPOU3LGI26CGBVA2Kry1aiM=
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
	[fuzzy]
X-Received-From: 212.227.17.21
Subject: Re: [Groff] mom : unicode in .INCLUDE'd files SOLVED
X-BeenThere: groff@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: <groff.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/groff>,
	<mailto:groff-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/groff/>
List-Post: <mailto:groff@gnu.org>
List-Help: <mailto:groff-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/groff>,
	<mailto:groff-request@gnu.org?subject=subscribe>
X-List-Received-Date: Tue, 25 Jul 2017 05:42:20 -0000

On Sun, 23 Jul 2017 11:21:13 -0700
Dale Snell <ddsnell@frontier.com> wrote:

[...]
> That's odd, it works fine for me.  I've got
>
>     .FAMILY U-P
>     .FT R

Dale, again you are right.  I think I just was tired.  Thanks, problem
solved. Erich

>
> --Dale
>
> --
> “The clean sleek looks of Star Trek is where we will aim and
> Babylon 5 is where we’ll end up.”
>    -- Dwight Shultz


--
"Alles wird gut."  (Eine Hoffnung)


From MAILER-DAEMON Tue Jul 25 03:36:55 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dZuOk-0000Cy-Tr
	for mharc-groff@gnu.org; Tue, 25 Jul 2017 03:36:54 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:42488)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <erichhoffmann@gmx.de>) id 1dZuOh-0000CG-TA
	for groff@gnu.org; Tue, 25 Jul 2017 03:36:53 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <erichhoffmann@gmx.de>) id 1dZuOd-0002e2-SA
	for groff@gnu.org; Tue, 25 Jul 2017 03:36:51 -0400
Received: from mout.gmx.net ([212.227.17.21]:54912)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16)
	(Exim 4.71) (envelope-from <erichhoffmann@gmx.de>)
	id 1dZuOd-0002dQ-FX
	for groff@gnu.org; Tue, 25 Jul 2017 03:36:47 -0400
Received: from e.h ([46.81.254.171]) by mail.gmx.com (mrgmx101
	[212.227.17.168]) with ESMTPSA (Nemesis) id 0M3eDF-1drZlr1dwX-00rDAe;
	Tue, 25 Jul 2017 09:36:42 +0200
Date: Tue, 25 Jul 2017 09:39:57 +0200
From: "E. Hoffmann" <erichhoffmann@gmx.de>
To: Deri James <deri@chuzzlewit.myzen.co.uk>
Cc: groff@gnu.org, ddsnell@frontier.com
Message-Id: <20170725093957.63be7d8f1d1b3caea76aacb7@gmx.de>
In-Reply-To: <10148543.2B2auInLYV@pip.chuzzlewit.co.uk>
References: <20170720215301.7b111801@e.h> <20170723170101.589b486e@e.h>
	<20170723101639.32fd43c0@zothique.localdomain>
	<10148543.2B2auInLYV@pip.chuzzlewit.co.uk>
X-Mailer: Sylpheed 3.5.0 (GTK+ 2.24.31; x86_64-slackware-linux-gnu)
Mime-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
X-Provags-ID: V03:K0:T+5+6JRTie1U2bt4+911DnLQw8UzVAlgg8/IJCWA7DOZC6fGzvW
	MrPHVwLLyyvz4XxIFliVawrP3vpTWKIV28tgTuV8TNn0bWLHccOr4fXQUE3khONHzxbsfMc
	T9iY7N4Tene06TASlnT6gELr7dt6c67h31fHnbasAaHu3l58pqxvCOYA3iCsEB939RgdjuV
	AzhjBQMcHs/87J8dkQcMA==
X-UI-Out-Filterresults: notjunk:1;V01:K0:PtwZEEbJZuA=:bPhK2O6Itd1a6qPCmrO4jS
	RyIfpa3s5uAnMmOhri1+RIeUMLSpMLMROPph94HwitP/vYw26BhuSyv96QLNBPq5gxrEfxru7
	o9DbjVqRkZNIcn70Vt/pU2amaJeKC74Aibftf/VKH5A3JDUVL4C/m9uOkI5iepjhX84D/WGzU
	irmxl7RA3Mwyf5LPyhXJJWYRB96QHurFbu1CRSPInhQ7yogR2ZvFAjQIyqERDoqES5KSBFD9s
	7aI43JgxtoSEu1WO2DAgYAuSIcGt7snc3bf3cK97N5SzolCIKIePTie1XOpsNglCJOJ5qOXoc
	DPDpbScZiMwXRvkKxq+PkZ9PGlW1Kh4KmurFF8UhWIgD9PfTppSZSJuiCgiiG1JiVpblAi8bj
	dqWWj+H3oeRVIiznJf6IgXcnRbjJQabZm5tb8eAu3BY9XKlSM1GMq5eLE3vtvVaPKqqdlHlcW
	2nHyTk8fH8szjK8YmSejRKT6eUdy7NTnR8pISDH7KpIBwVVjujzcsULoBi/QVJCkeYYQYqd/d
	sPi+T6QMPyKLm5YoXJq4thbBfpbnXyjyBMrb1mMwQ/BD+OhakTggkjwBP7PCy4OQ8Xhh5eFSd
	0WMBeKATlY0RC7k8iEtjXS9Bh1JHENvzkZx8DzcnarbbhIWrBFaRQl1gcrTgAa8tAUbfKmcSG
	8Jv0Z2GxpHIkezpkgu0wy5DwgxWsg0JVa7b4Rgrzt3T7tJVlZzUmT4I4kDV9ViDDhDekRhtfB
	87rYWbvK+/KHT271+q69AjAWDquJ+WV6DtTkZs0hJ/mTW9yOsUKYLOCpoDRnuRAIGduIHd8nB
	PAL1J0pMrORzzgv5rbRq1YM9LJZTRxm6y8mGMaDTfNUFp7ceHs=
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
	[fuzzy]
X-Received-From: 212.227.17.21
Subject: Re: [Groff] mom : unicode in .INCLUDE'd files
X-BeenThere: groff@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: <groff.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/groff>,
	<mailto:groff-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/groff/>
List-Post: <mailto:groff@gnu.org>
List-Help: <mailto:groff-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/groff>,
	<mailto:groff-request@gnu.org?subject=subscribe>
X-List-Received-Date: Tue, 25 Jul 2017 07:36:53 -0000

On Sun, 23 Jul 2017 21:49:08 +0100
Deri James <deri@chuzzlewit.myzen.co.uk> wrote:

[...]

> It ought to work if you include:-

> -P-yU

> On the pdfmom command line. This should tell gropdf to use the U- foundry, so you don't have to alter the
> actual mom document.

Works perfectly for me.  There is always more than one way to skin a
cat...

   So definitely my problem is solved.

Thanks! Erich

ps. OT

Let me add a word to the MS Windows discussion.

   At work, I have to use that OS and the MS WORD nightmare, ok I'm not
a WORD wizard, but when my colleagues have a problem with WORD (who
doesn't?), they ask me.  I really are not ranting about a program
that I don't know.

   I got the impression that no one of those Word Processor Users
will ever use groff or tex.  This is not a question of "who's
got the better arguments".  I fear it is their basic assumption
"it has to work by itself".  Just like someone who sits in a car,
expecting that this car will transport him to any location he wants
to, streets or not, without any interference from him as the driver.
Heavens how often did I have this conversation:

- Hey Erich, where is my document, did _you_ delete it?
- What document?
- That protocol thing I wrote last week.
- Where did you save it?
- What do you mean 'where did I save it'?  In WORD of course!
- What is the name of your protocol?
  (...)

   WORD is very easy in it's first steps, as long as your Documents
have only two or three pages of length, as long as you have only
very simple paragraphs and simplistic tables and as long as you don't
want to do some black magic as add or remove or change the numbering
of headings...it invites its users "do everything in one place"
and "create fancy word art in 5 minutes".  Most of the WORD people
I know don't even learn to use Styles and automatic formatting.
Thus, when they want a greater distance between paragraphs, they
just hit the enter button twice, and when they want to write a list,
they hit the tab button and enter the item numbering by hand, and go
on entering the indentation with the tab button, if not with simple
Spaces, each single line, after which they complain how complicated
this is.  Even the WORD nightmare can do better, you just have to
learn it which isn't so hard after all.  But it is a reality.  Not even
WORD works by itself alone.

   Now I imagine one of those WORD users looking at - a command line,
or, Lord Jesus save us all, - a pipe, or, now the world is going
to end, - follow this thread on sourcing and utf'es and processing
and preconv'ing...

   No chance, and not even because WORD has flaws, but because of
a wholistic mentality against IT on any level.  Imagine, some weeks ago
someone told me: "You know, I have my problems with these new
technologies." He was talking about email.  He was actually calling
email a "new technology"!  So what do we expect?

No offence meant.






From MAILER-DAEMON Wed Jul 26 20:18:10 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1daWVG-0005GD-7v
	for mharc-groff@gnu.org; Wed, 26 Jul 2017 20:18:10 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:55416)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <bjarniig@hi.is>) id 1daWVD-0005G7-S4
	for groff@gnu.org; Wed, 26 Jul 2017 20:18:08 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <bjarniig@hi.is>) id 1daWVD-0000tN-3c
	for groff@gnu.org; Wed, 26 Jul 2017 20:18:07 -0400
Received: from mx2.hi.is ([2a00:c88:4000:1650::165:104]:44358)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)
	(Exim 4.71) (envelope-from <bjarniig@hi.is>) id 1daWVC-0000qA-Ne
	for groff@gnu.org; Wed, 26 Jul 2017 20:18:07 -0400
Received: from hekla.rhi.hi.is (hekla.rhi.hi.is
	[IPv6:2a00:c88:4000:1650::165:2])
	by mx2.hi.is (8.14.7/8.14.7) with ESMTP id v6QNtOW0001185
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256
	verify=NO) for <groff@gnu.org>; Wed, 26 Jul 2017 23:55:27 GMT
Received: from hekla.rhi.hi.is (localhost [127.0.0.1])
	by hekla.rhi.hi.is (8.14.4/8.14.4) with ESMTP id v6QNtOrm022137
	for <groff@gnu.org>; Wed, 26 Jul 2017 23:55:24 GMT
Received: (from bjarniig@localhost)
	by hekla.rhi.hi.is (8.14.4/8.14.4/Submit) id v6QNtObx022136
	for groff@gnu.org; Wed, 26 Jul 2017 23:55:24 GMT
Date: Wed, 26 Jul 2017 23:55:24 +0000
From: Bjarni Ingi Gislason <bjarniig@rhi.hi.is>
To: groff@gnu.org
Message-ID: <20170726235524.GA22133@rhi.hi.is>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.5.20 (2009-12-10)
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
X-Received-From: 2a00:c88:4000:1650::165:104
Subject: [Groff] git-repository: Adjustments not yet completed after transfer
X-BeenThere: groff@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: <groff.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/groff>,
	<mailto:groff-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/groff/>
List-Post: <mailto:groff@gnu.org>
List-Help: <mailto:groff-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/groff>,
	<mailto:groff-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 27 Jul 2017 00:18:08 -0000

  The git repository is not yet fully functional as the bug #51415 has yet to
be committed.

  My master branch shows me in read the names of manual pages, that are
created and should therefore not be tracked.


