From MAILER-DAEMON Mon Jul 03 12:26:22 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dS4B4-0004oy-2K
	for mharc-coreutils@gnu.org; Mon, 03 Jul 2017 12:26:22 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:47123)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <for.ad@free.fr>) id 1dS419-0000Vv-Rd
	for coreutils@gnu.org; Mon, 03 Jul 2017 12:16:08 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <for.ad@free.fr>) id 1dS415-0005dj-Tr
	for coreutils@gnu.org; Mon, 03 Jul 2017 12:16:07 -0400
Received: from smtp1-g21.free.fr ([2a01:e0c:1:1599::10]:64671)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)
	(Exim 4.71) (envelope-from <for.ad@free.fr>) id 1dS415-0005bh-OV
	for coreutils@gnu.org; Mon, 03 Jul 2017 12:16:03 -0400
Received: from webmail.free.fr (unknown [172.20.243.59])
	by smtp1-g21.free.fr (Postfix) with ESMTP id CCBDDB0053E
	for <coreutils@gnu.org>; Mon,  3 Jul 2017 18:16:00 +0200 (CEST)
Received: from APuteaux-656-1-115-21.w86-249.abo.wanadoo.fr ([86.249.230.21])
	by webmail.free.fr
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII;
 format=flowed
Content-Transfer-Encoding: 7bit
Date: Mon, 03 Jul 2017 18:16:00 +0200
From: BaBa <for.ad@free.fr>
To: coreutils@gnu.org
Subject: How the ls command interpret this?
Message-ID: <ccb12f08dc137b606f3961a766ec17da@free.fr>
X-Sender: for.ad@free.fr
User-Agent: Webmail Free/1.2.3
X-detected-operating-system: by eggs.gnu.org: Windows NT kernel [generic]
	[fuzzy]
X-Received-From: 2a01:e0c:1:1599::10
X-Mailman-Approved-At: Mon, 03 Jul 2017 12:26:20 -0400
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 03 Jul 2017 16:16:08 -0000

Hello,

touch a[b]
ls ?[b]


From MAILER-DAEMON Mon Jul 03 13:17:38 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dS4yg-0001Ez-Ex
	for mharc-coreutils@gnu.org; Mon, 03 Jul 2017 13:17:38 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:34807)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1dS4yd-0001EZ-JC
	for coreutils@gnu.org; Mon, 03 Jul 2017 13:17:36 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1dS4ya-0001be-FJ
	for coreutils@gnu.org; Mon, 03 Jul 2017 13:17:35 -0400
Received: from mx1.redhat.com ([209.132.183.28]:53902)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)
	(Exim 4.71) (envelope-from <eblake@redhat.com>) id 1dS4ya-0001ZX-4y
	for coreutils@gnu.org; Mon, 03 Jul 2017 13:17:32 -0400
Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com
	[10.5.11.12])
	(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
	(No client certificate requested)
	by mx1.redhat.com (Postfix) with ESMTPS id 3252679710;
	Mon,  3 Jul 2017 17:17:30 +0000 (UTC)
DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 3252679710
Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com;
	dmarc=none (p=none dis=none) header.from=redhat.com
Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com;
	spf=pass smtp.mailfrom=eblake@redhat.com
DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 3252679710
Received: from [10.10.120.36] (ovpn-120-36.rdu2.redhat.com [10.10.120.36])
	by smtp.corp.redhat.com (Postfix) with ESMTP id AAB6C900CD;
	Mon,  3 Jul 2017 17:17:29 +0000 (UTC)
Subject: Re: How the ls command interpret this?
To: BaBa <for.ad@free.fr>, coreutils@gnu.org
References: <ccb12f08dc137b606f3961a766ec17da@free.fr>
From: Eric Blake <eblake@redhat.com>
Openpgp: url=http://people.redhat.com/eblake/eblake.gpg
Organization: Red Hat, Inc.
Message-ID: <c7db82c1-b9b5-5772-b091-36c60bd90d7b@redhat.com>
Date: Mon, 3 Jul 2017 12:17:28 -0500
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
	Thunderbird/52.2.0
MIME-Version: 1.0
In-Reply-To: <ccb12f08dc137b606f3961a766ec17da@free.fr>
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature";
	boundary="vTOeQsUGFvqNHJwLTQNfG6NBDEbQakUjK"
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12
X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16
	(mx1.redhat.com [10.5.110.25]);
	Mon, 03 Jul 2017 17:17:30 +0000 (UTC)
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
	[fuzzy]
X-Received-From: 209.132.183.28
X-Content-Filtered-By: Mailman/MimeDel 2.1.21
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 03 Jul 2017 17:17:36 -0000

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--vTOeQsUGFvqNHJwLTQNfG6NBDEbQakUjK
From: Eric Blake <eblake@redhat.com>
To: BaBa <for.ad@free.fr>, coreutils@gnu.org
Message-ID: <c7db82c1-b9b5-5772-b091-36c60bd90d7b@redhat.com>
Subject: Re: How the ls command interpret this?
References: <ccb12f08dc137b606f3961a766ec17da@free.fr>
In-Reply-To: <ccb12f08dc137b606f3961a766ec17da@free.fr>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

On 07/03/2017 11:16 AM, BaBa wrote:
> Hello,
>=20
> touch a[b]
> ls ?[b]

Well, the correct answer is: it depends on what is in your current
directory before you start things.  Here's some examples, all of which
produce different ls output, even though all of them include your two
commands verbatim in the middle:

Case 1:
$ mkdir foo
$ cd foo
$ touch ab
$ touch a[b]    # which re-touches 'ab' thanks to globbing
$ ls ?[b]       # which matches 'ab' thanks to globbing
ab
$ cd ..
$ rm -rf foo

Case 2:
$ mkdir foo
$ cd foo
$ touch bb
$ touch cb
$ touch a[b]    # the glob doesn't match, so it is passed unchanged
$ ls ?[b]       # the glob matches two out of three files
bb  cb
$ cd ..
$ rm -rf foo

Case 3:
$ mkdir foo
$ cd foo
$ touch a[b]    # the glob doesn't match, so it is passed unchanged
$ ls ?[b]       # the glob doesn't match, so it is passed unchanged
ls: cannot access '?[b]': No such file or directory
$ cd ..
$ rm -rf foo

Case 4:
$ mkdir foo
$ cd foo
$ touch '?[b]'  # no globbing involved, touches the file ?[b]
$ touch a[b]    # the glob doesn't match, so it is passed unchanged
$ ls ?[b]       # the glob doesn't match, so it is passed unchanged
'?[b]'
$ cd ..
$ rm -rf foo

In case 4, note that I'm using a new-enough version of ls that
automatically quotes its output so that it is suitable for reinput to
the shell (the '' are added to show that I want quoting in place, to
avoid the shell interpreting my filename as a glob).

There's also possibilities if you have a shell that supports options on
how globbing behaves (here, I'm using bash):

Case 5:
$ mkdir foo
$ cd foo
$ shopt -s failglob
$ touch a[b]
bash: no match: a[b]
$ shopt -u failglob
$ shopt -s nullglob
$ touch a[b]
touch: missing file operand
Try 'touch --help' for more information.
$ cd ..
$ rm -rf foo

But by now, hopefully I've answered your question.  If not, write 'echo'
instead of 'ls' on every command line that you are unsure of how the
shell will glob things, and thus on what arguments are being handed to ls=
=2E

--=20
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org


--vTOeQsUGFvqNHJwLTQNfG6NBDEbQakUjK
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
Comment: Public key at http://people.redhat.com/eblake/eblake.gpg
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBCAAGBQJZWnwoAAoJEKeha0olJ0Nqr8oH/ipwkH7q4qX2xP3E0n2nn03s
ebKu0nKcJzNSen3qHvTU452vxWquT8XaS2jZxR06YbfD6UCq4meSbTRMt1K4zx/x
FjSS0L41DmQ/YpkyJASPJEbVekIGjktpdNgTQj5HbKZGRMzQ5krysvEUN/XyaiHQ
O3YSMNePznxGKoyBSbQKcaC4ifSWbEzO3yEUUKRepzddaKoRevOgdmWZPLaCq8pb
458xqGypWj5Pnlr+GD6VENEI/de9PQ3OCYtTC6PbpdiTlUum6suWbAg8F0cFFCFh
LvMPsmySv/VmPKytx1IzfqL+HIoFZXgv9A3dLkh4prLbGhCXAyVltjbC8mNxCF0=
=lvRs
-----END PGP SIGNATURE-----

--vTOeQsUGFvqNHJwLTQNfG6NBDEbQakUjK--


From MAILER-DAEMON Mon Jul 03 14:30:35 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dS67H-00061T-QA
	for mharc-coreutils@gnu.org; Mon, 03 Jul 2017 14:30:35 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:51208)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <for.ad@free.fr>) id 1dS67F-00060D-9H
	for coreutils@gnu.org; Mon, 03 Jul 2017 14:30:33 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <for.ad@free.fr>) id 1dS67B-0003xY-8C
	for coreutils@gnu.org; Mon, 03 Jul 2017 14:30:33 -0400
Received: from smtp6-g21.free.fr ([212.27.42.6]:45092)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)
	(Exim 4.71) (envelope-from <for.ad@free.fr>) id 1dS67B-0003vf-2E
	for coreutils@gnu.org; Mon, 03 Jul 2017 14:30:29 -0400
Received: from webmail.free.fr (unknown [172.20.243.6])
	by smtp6-g21.free.fr (Postfix) with ESMTP id 1D1C37801C3
	for <coreutils@gnu.org>; Mon,  3 Jul 2017 20:30:26 +0200 (CEST)
Received: from APuteaux-656-1-115-21.w86-249.abo.wanadoo.fr ([86.249.230.21])
	by webmail.free.fr
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8;
 format=flowed
Date: Mon, 03 Jul 2017 20:30:26 +0200
From: BaBa <for.ad@free.fr>
To: coreutils@gnu.org
Subject: Re: How the ls command interpret this?
In-Reply-To: <c7db82c1-b9b5-5772-b091-36c60bd90d7b@redhat.com>
References: <ccb12f08dc137b606f3961a766ec17da@free.fr>
	<c7db82c1-b9b5-5772-b091-36c60bd90d7b@redhat.com>
Message-ID: <8ad70c715e714ae4d1cf445fb39b803f@free.fr>
X-Sender: for.ad@free.fr
User-Agent: Webmail Free/1.2.3
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: Windows NT kernel [generic]
	[fuzzy]
X-Received-From: 212.27.42.6
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 03 Jul 2017 18:30:34 -0000

Le 2017-07-03 19:17, Eric Blake a =C3=A9crit :
> Case 3:
> $ mkdir foo
> $ cd foo
> $ touch a[b]    # the glob doesn't match, so it is passed unchanged
> $ ls ?[b]       # the glob doesn't match, so it is passed unchanged
> ls: cannot access '?[b]': No such file or directory
> $ cd ..
> $ rm -rf foo

Yes understood, the glob didn't match.

Thanks.


From MAILER-DAEMON Sat Jul 08 12:44:39 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dTsqV-0006bU-6J
	for mharc-coreutils@gnu.org; Sat, 08 Jul 2017 12:44:39 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:56949)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <meyering@gmail.com>) id 1dTsqQ-0006Xh-KX
	for coreutils@gnu.org; Sat, 08 Jul 2017 12:44:38 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <meyering@gmail.com>) id 1dTsqN-00059T-0t
	for coreutils@gnu.org; Sat, 08 Jul 2017 12:44:34 -0400
Received: from mail-pg0-f47.google.com ([74.125.83.47]:33004)
	by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)
	(Exim 4.71) (envelope-from <meyering@gmail.com>) id 1dTsqM-00052S-N5
	for coreutils@gnu.org; Sat, 08 Jul 2017 12:44:30 -0400
Received: by mail-pg0-f47.google.com with SMTP id k14so30680873pgr.0
	for <coreutils@gnu.org>; Sat, 08 Jul 2017 09:44:08 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
	h=sender:from:to:subject:date:message-id:mime-version
	:content-transfer-encoding;
	bh=DgeATjFhDOg932fA7om8pbMeTg8p+PjkjZhVtCscZNg=;
	b=QecjEffovEBDwsiggBc9i6+wJGs6ISblW2T9UlgM4T98eGEPGDkeC9hn7S1/uULD22
	4JLHzOx+e0caKkPsCbZDz32LspJkLDc8d93Qivcku3WdqFvC0Xiwc7k/tDO1qKGQk+B7
	63793t+Xl/2RsjonAIiT+zcp1NeIMbOqGLN2NDotU9DIM0stPbIGGodJd6LrjcVEoGod
	09ZOXHg/B2oIeHLCtSSap53wF7Xu3aH6yGlrqhTiTIlsx0UIJESzg1EJTQndPkngHish
	lTubz2lZT2vMHV6m4MvoCYi9dckHrXfWSa+TJIkMQ0YwAQXAFvGcf8mZ0YvcEbeb5xLb
	v7ZA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
	d=1e100.net; s=20161025;
	h=x-gm-message-state:sender:from:to:subject:date:message-id
	:mime-version:content-transfer-encoding;
	bh=DgeATjFhDOg932fA7om8pbMeTg8p+PjkjZhVtCscZNg=;
	b=mVXjuSfwC3Leyh+XGunGN8stzDTbPHvzdM9zP8WMBcfxvx0DvlXVDVs0J57VTG4r2/
	lp/3PFtcCwwBrnGWaWu2jFkccpKtKMYDJBl483kpMWIvA/qMt6BIWXaNoF8KlnFaYWVi
	bXas+0sUpBcvaMvrdnz1irxgLWfCsOzhbDe6FLntwn0+QqWzX5bmCvZAAGdkt0dCr2Zg
	L95+8vv/rSeSRmrY13cfMWeXiqpZbyeqQm4vrYLil2CxKE6S3TypxEIDvbezKVJSMxos
	qigc3pr8F2Fvvt9dc4wQj8IzVHi2T/tpBsUqgf2vAhtVoJ6ronRQshCf1YzsiJzl2bTj
	h3uQ==
X-Gm-Message-State: AIVw112FaRYQ3JMq3Q2gqEpWc1n+S4nI6Wn57JyDY88kgBKFArygDQut
	xJyQ2KAK8cSBNNAM
X-Received: by 10.98.63.220 with SMTP id z89mr36548672pfj.84.1499532186632;
	Sat, 08 Jul 2017 09:43:06 -0700 (PDT)
Received: from dhcp-172-20-189-76.dhcp.thefacebook.com
	([2620:10d:c090:180::1:427e]) by smtp.gmail.com with ESMTPSA id
	y1sm15744430pff.13.2017.07.08.09.43.04 for <coreutils@gnu.org>
	(version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256);
	Sat, 08 Jul 2017 09:43:05 -0700 (PDT)
Sender: Jim Meyering <meyering@gmail.com>
From: Jim Meyering <jim@meyering.net>
To: coreutils@gnu.org
Subject: groups bug fix, groups test fix and NEWS tweak
Date: Sat, 08 Jul 2017 18:42:55 +0200
Message-ID: <lubpf3eftq98xc.fsf@meyering.net>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
	[fuzzy]
X-Received-From: 74.125.83.47
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sat, 08 Jul 2017 16:44:38 -0000

Here are three proposed patches:

  groups: do not exit early
  tests: groups-dash.sh: avoid false failure
  doc: tweak wording

>From df2c30dca6bb9c07be9b6e5b12ff6d1634e6c417 Mon Sep 17 00:00:00 2001
From: Jim Meyering <jim@meyering.net>
Date: Fri, 2 Jun 2017 21:50:04 -0700
Subject: [PATCH 1/3] doc: tweak wording

* NEWS (Bug fixes): Tweak wording of the mv/cp-vs-symlink-ownership
entry and the one about df.
---
 NEWS | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/NEWS b/NEWS
index b834fa16c..35ba31197 100644
--- a/NEWS
+++ b/NEWS
@@ -4,9 +4,9 @@ GNU coreutils NEWS                                    -*- o=
utline -*-

 ** Bug fixes

-  cp and mv now only warn about failure to preserve ownership of symlinks.
-  cp (without -p) will no longer exit with a failure status, and mv will
-  also no longer leave such symlinks remaining in the source file system.
+  cp and mv now merely warn about any failure to preserve symlink ownershi=
p.
+  Before, cp (without -p) would exit with a failure status, and a cross-de=
vice
+  mv would leave such symlinks behind in the source file system.
   [the bug dates back to the initial implementation]

   date and touch no longer overwrite the heap with large
@@ -18,7 +18,7 @@ GNU coreutils NEWS                                    -*-=
 outline -*-
   [bug introduced in coreutils-8.24]

   df no longer interacts with excluded file system types, so for example
-  specifying -x nfs will no longer hang with problematic nfs mounts.
+  specifying -x nfs no longer hangs with problematic nfs mounts.
   [bug introduced in coreutils-8.21]

   split no longer exits when invocations of a --filter return EPIPE.
--=20
2.13.0


>From ed57568ea5fc9dcca8fd4dce00bb3085727e119a Mon Sep 17 00:00:00 2001
From: Jim Meyering <jim@meyering.net>
Date: Sat, 8 Jul 2017 12:01:34 +0200
Subject: [PATCH 2/3] tests: groups-dash.sh: avoid false failure
MIME-Version: 1.0
Content-Type: text/plain; charset=3DUTF-8
Content-Transfer-Encoding: 8bit

* tests/misc/groups-dash.sh: Avoid false failure on a system for which
"none" is a valid user name.  The first invocation would succeed, and
the second would fail with "groups: =E2=80=98--=E2=80=99: no such user".
Use a user name that cannot exist.
---
 tests/misc/groups-dash.sh | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/tests/misc/groups-dash.sh b/tests/misc/groups-dash.sh
index ca81cb40b..37675b4db 100755
--- a/tests/misc/groups-dash.sh
+++ b/tests/misc/groups-dash.sh
@@ -19,12 +19,14 @@
 . "${srcdir=3D.}/tests/init.sh"; path_prepend_ ./src
 print_ver_ groups

+# An invalid user name
+user=3D:invalid
+
+printf '%s\n' "groups: ':invalid': no such user" > exp || framework_failur=
e_
+
 # Coreutils 6.9 and earlier failed to display information on first argument
 # if later argument was --.
-groups none -- > out 2>&1 && fail=3D1
-echo $? >> out
-groups -- none -- > exp 2>&1 && fail=3D1
-echo $? >> exp
+returns_ 1 groups $user -- > out 2>&1 || fail=3D1

 compare exp out || fail=3D1

--=20
2.13.0


>From 3542aedddb1e90e75a48d3a7ad97f179db783072 Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering@fb.com>
Date: Sat, 8 Jul 2017 13:01:55 +0200
Subject: [PATCH 3/3] groups: do not exit early

Most programs take care to operate on all command-line-specified
operands before exiting.  That is an important feature that allows
to identify all problems with the first run.  However, groups would
exit upon the first problematic user name.
* src/groups.c (main): Do not exit immediately upon error.
* tests/misc/groups-process-all.sh: New file. Test for this.
* tests/local.mk (all_tests): Add it.
* NEWS (Bug fixes): Mention this.
---
 NEWS                             |  3 +++
 src/groups.c                     | 11 +++++++----
 tests/local.mk                   |  1 +
 tests/misc/groups-process-all.sh | 26 ++++++++++++++++++++++++++
 4 files changed, 37 insertions(+), 4 deletions(-)
 create mode 100755 tests/misc/groups-process-all.sh

diff --git a/NEWS b/NEWS
index 35ba31197..c8cb5cb5a 100644
--- a/NEWS
+++ b/NEWS
@@ -21,6 +21,9 @@ GNU coreutils NEWS                                    -*-=
 outline -*-
   specifying -x nfs no longer hangs with problematic nfs mounts.
   [bug introduced in coreutils-8.21]

+  `groups inva:lid root` no longer exits immediately upon failure.
+  Now, it prints a diagnostic or a line to stdout for each argument.
+
   split no longer exits when invocations of a --filter return EPIPE.
   [bug introduced in coreutils-8.26]

diff --git a/src/groups.c b/src/groups.c
index 0a5c5be88..ccfe8a2d5 100644
--- a/src/groups.c
+++ b/src/groups.c
@@ -122,17 +122,20 @@ main (int argc, char **argv)
   else
     {
       /* At least one argument.  Divulge the details of the specified user=
s.  */
-      while (optind < argc)
+      for ( ; optind < argc; optind++)
         {
           struct passwd *pwd =3D getpwnam (argv[optind]);
           if (pwd =3D=3D NULL)
-            die (EXIT_FAILURE, 0, _("%s: no such user"),
-                 quote (argv[optind]));
+            {
+              error (0, 0, _("%s: no such user"), quote (argv[optind]));
+              ok =3D false;
+              continue;
+            }
           ruid =3D pwd->pw_uid;
           rgid =3D egid =3D pwd->pw_gid;

           printf ("%s : ", argv[optind]);
-          if (!print_group_list (argv[optind++], ruid, rgid, egid, true, '=
 '))
+          if (!print_group_list (argv[optind], ruid, rgid, egid, true, ' '=
))
             ok =3D false;
           putchar ('\n');
         }
diff --git a/tests/local.mk b/tests/local.mk
index 235bcbe65..8fc48c489 100644
--- a/tests/local.mk
+++ b/tests/local.mk
@@ -296,6 +296,7 @@ all_tests =3D					\
   tests/misc/false-status.sh			\
   tests/misc/fold.pl				\
   tests/misc/groups-dash.sh			\
+  tests/misc/groups-process-all.sh		\
   tests/misc/groups-version.sh			\
   tests/misc/head-c.sh				\
   tests/misc/head-pos.sh			\
diff --git a/tests/misc/groups-process-all.sh b/tests/misc/groups-process-a=
ll.sh
new file mode 100755
index 000000000..a352ecc10
--- /dev/null
+++ b/tests/misc/groups-process-all.sh
@@ -0,0 +1,26 @@
+#!/bin/sh
+# Ensure groups processes all arguments before exiting.
+# With coreutils-2.27 and prior, it would exit upon first failure.
+
+# Copyright (C) 2017 Free Software Foundation, Inc.
+
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+. "${srcdir=3D.}/tests/init.sh"; path_prepend_ ./src
+print_ver_ groups
+
+returns_ 1 groups :1 :2 :3 2> err || fail=3D1
+test $(wc -l < err) =3D 3 || fail=3D1
+
+Exit $fail
--=20
2.13.0



From MAILER-DAEMON Sun Jul 09 03:17:44 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dU6TQ-0000A8-Go
	for mharc-coreutils@gnu.org; Sun, 09 Jul 2017 03:17:44 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:35146)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <whitebox@nefkom.net>) id 1dU6TO-00009r-8V
	for coreutils@gnu.org; Sun, 09 Jul 2017 03:17:43 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <whitebox@nefkom.net>) id 1dU6TL-0007Y8-3X
	for coreutils@gnu.org; Sun, 09 Jul 2017 03:17:42 -0400
Received: from mail-out.m-online.net ([212.18.0.10]:43849)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)
	(Exim 4.71) (envelope-from <whitebox@nefkom.net>) id 1dU6TK-0007GL-T3
	for coreutils@gnu.org; Sun, 09 Jul 2017 03:17:39 -0400
Received: from frontend01.mail.m-online.net (unknown [192.168.8.182])
	by mail-out.m-online.net (Postfix) with ESMTP id 3x506258gBz1qrfR
	for <coreutils@gnu.org>; Sun,  9 Jul 2017 09:17:14 +0200 (CEST)
Received: from localhost (dynscan1.mnet-online.de [192.168.6.70])
	by mail.m-online.net (Postfix) with ESMTP id 3x50624Tqcz3jgY4
	for <coreutils@gnu.org>; Sun,  9 Jul 2017 09:17:14 +0200 (CEST)
X-Virus-Scanned: amavisd-new at mnet-online.de
Received: from mail.mnet-online.de ([192.168.8.182])
	by localhost (dynscan1.mail.m-online.net [192.168.6.70]) (amavisd-new,
	port 10024) with ESMTP id bEdIyigbvUwJ for <coreutils@gnu.org>;
	Sun,  9 Jul 2017 09:17:14 +0200 (CEST)
X-Auth-Info: gGkyQcs6Eb8nNYuHoLYN+yFRqCqoCEDN1BGjdVMCUYp5nRRXVZTPFYboJyrqnqVA
Received: from igel.home (ppp-88-217-2-240.dynamic.mnet-online.de
	[88.217.2.240])
	(using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits))
	(No client certificate requested)
	by mail.mnet-online.de (Postfix) with ESMTPSA
	for <coreutils@gnu.org>; Sun,  9 Jul 2017 09:17:13 +0200 (CEST)
Received: by igel.home (Postfix, from userid 1000)
	id 559A72C01CA; Sun,  9 Jul 2017 09:17:13 +0200 (CEST)
From: Andreas Schwab <schwab@linux-m68k.org>
To: coreutils@gnu.org
Subject: [PATCH] nproc: fix indentation of usage output
X-Yow: Can I have an IMPULSE ITEM instead?
Date: Sun, 09 Jul 2017 09:17:13 +0200
Message-ID: <87van2oz9i.fsf@linux-m68k.org>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.2 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
	[fuzzy]
X-Received-From: 212.18.0.10
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 09 Jul 2017 07:17:43 -0000

* src/nproc.c (usage): Align output.
---
 src/nproc.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/src/nproc.c b/src/nproc.c
index fdcce591f4..d7455db70f 100644
--- a/src/nproc.c
+++ b/src/nproc.c
@@ -61,8 +61,8 @@ which may be less than the number of online processors\n\
 \n\
 "), stdout);
       fputs (_("\
-     --all       print the number of installed processors\n\
-     --ignore=N  if possible, exclude N processing units\n\
+      --all      print the number of installed processors\n\
+      --ignore=N if possible, exclude N processing units\n\
 "), stdout);
 
       fputs (HELP_OPTION_DESCRIPTION, stdout);
-- 
2.13.2

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."


From MAILER-DAEMON Mon Jul 10 02:17:25 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dUS0b-0008GI-KX
	for mharc-coreutils@gnu.org; Mon, 10 Jul 2017 02:17:25 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:49922)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1dUS0Z-0008G0-GQ
	for coreutils@gnu.org; Mon, 10 Jul 2017 02:17:24 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1dUS0U-0003La-G6
	for coreutils@gnu.org; Mon, 10 Jul 2017 02:17:23 -0400
Received: from mout.kundenserver.de ([212.227.126.131]:51345)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16)
	(Exim 4.71) (envelope-from <mail@bernhard-voelker.de>)
	id 1dUS0U-0003LF-1n
	for coreutils@gnu.org; Mon, 10 Jul 2017 02:17:18 -0400
Received: from [192.168.101.10] ([91.12.166.234]) by mrelayeu.kundenserver.de
	(mreue003 [212.227.15.167]) with ESMTPSA (Nemesis) id
	0Mgrgq-1dHIqF2azA-00M37O; Mon, 10 Jul 2017 08:17:11 +0200
Subject: Re: groups bug fix, groups test fix and NEWS tweak
To: Jim Meyering <jim@meyering.net>, coreutils@gnu.org
References: <lubpf3eftq98xc.fsf@meyering.net>
From: Bernhard Voelker <mail@bernhard-voelker.de>
Message-ID: <368e77e4-49dc-4c52-a341-b5fd783d0bda@bernhard-voelker.de>
Date: Mon, 10 Jul 2017 08:17:08 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101
	Thunderbird/45.6.0
MIME-Version: 1.0
In-Reply-To: <lubpf3eftq98xc.fsf@meyering.net>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Provags-ID: V03:K0:q6uXqECp9fSxm3j1LYbhnFKESmktxHkHxFoX7BL2Sv/z/ndmxb1
	xibUo2iKnprc0vr+3NZJrpfMEPS+FO0eg662Srna0IcI2F8fCIx5tAPsIIKUGXNceznly2l
	emWup1mxGJSrvQsmX/rWwDjS0nyCcdfOVZba0ULqsipX7nAhpDntQlylnhmq8VSpqVWTajv
	td7iAxZfZQFcG57XeHSSA==
X-UI-Out-Filterresults: notjunk:1;V01:K0:edytHI5QWEE=:bDrPp6FtcX0/A0I6PycFxO
	Ov9q/LabwVrqVsFURqQTfE0m6IRRAeyWbJ/E16dkZcRU+hM93Odfx4v+8q54eJgtjecY/AvNw
	HYxCI28tTuqRpa0e9JQ4mOfmUuvk0tYxhOCyy5wXtPNfaujxe5nkiP14Hy0RkwTCbVXhtn8J7
	S/MT5G/WUx+7CU+ZUrl+wB48HZ2FxtNRfysTVzdXxkaJ2VkwXvDOf5VLQCU6OH3vboyeq+mB5
	hrh3hUXCyU5KG8OLAWBcRO/TYUek9ANo3GrjM0tU7apbGWi6u1dpP/59bk6qYAip3BxpMYOOp
	dXiy2klY81eOlRyLB86OI1quKLKL19dBm+ZhwC3GzCxdH1jw88SHsBymVjfntlK74GtD8u6tp
	FOUKFqgxbxQg8GaMOsZWWYChXJgyJRKgnxWctMDY6BKPGzpVAlMBEpEf7ncfV6w84oed5Gvih
	sZNsfYNMxzbz+oyojcSp2OYNzdltDtSUujEZjHEhF7W0VUIknnZKj9p+zMo2BSTj5wQP5vaby
	ZMsiytZS7m3Ykx7bY2AqjneRSM8mgkn523OToKdfNjZU3YnODUwIDHKNzdxuMOBonDctaMKgv
	IFgsFJNre9fq4cys+VAMPWGMu9OlBMb0hTAiKCt6+qugCuTePtJPpnu0TmqlXjCUXchHP27aa
	+/etn7QBzhXTgIVnx2cYnBGDZQfd0x/QeBJQJiXNl9dF+sN1cZNPNWsRpbIRrcLxczrlAk/xK
	RscaCcLxa5GYaWlW
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
	[fuzzy]
X-Received-From: 212.227.126.131
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 10 Jul 2017 06:17:24 -0000

On 07/08/2017 06:42 PM, Jim Meyering wrote:
> Here are three proposed patches:
> 
>   groups: do not exit early

Nice catch!

FWIW the bug seems to have been introduced when replacing the
old groups.sh by the groups.c file with commit
  167b8025aca487de001da2448c1aebc2747bc1d3 in v6.11
and some additional checks have been added with immediate exit()
as well with
  71b7ddcdd5c473f9eaf6035b96acc68c14cb379e in v8.15

>   tests: groups-dash.sh: avoid false failure
>   doc: tweak wording

+1 thanks

Have a nice day,
Berny

> 
> From df2c30dca6bb9c07be9b6e5b12ff6d1634e6c417 Mon Sep 17 00:00:00 2001
> From: Jim Meyering <jim@meyering.net>
> Date: Fri, 2 Jun 2017 21:50:04 -0700
> Subject: [PATCH 1/3] doc: tweak wording
> 
> * NEWS (Bug fixes): Tweak wording of the mv/cp-vs-symlink-ownership
> entry and the one about df.
> ---
>  NEWS | 8 ++++----
>  1 file changed, 4 insertions(+), 4 deletions(-)
> 
> diff --git a/NEWS b/NEWS
> index b834fa16c..35ba31197 100644
> --- a/NEWS
> +++ b/NEWS
> @@ -4,9 +4,9 @@ GNU coreutils NEWS                                    -*- outline -*-
> 
>  ** Bug fixes
> 
> -  cp and mv now only warn about failure to preserve ownership of symlinks.
> -  cp (without -p) will no longer exit with a failure status, and mv will
> -  also no longer leave such symlinks remaining in the source file system.
> +  cp and mv now merely warn about any failure to preserve symlink ownership.
> +  Before, cp (without -p) would exit with a failure status, and a cross-device
> +  mv would leave such symlinks behind in the source file system.
>    [the bug dates back to the initial implementation]
> 
>    date and touch no longer overwrite the heap with large
> @@ -18,7 +18,7 @@ GNU coreutils NEWS                                    -*- outline -*-
>    [bug introduced in coreutils-8.24]
> 
>    df no longer interacts with excluded file system types, so for example
> -  specifying -x nfs will no longer hang with problematic nfs mounts.
> +  specifying -x nfs no longer hangs with problematic nfs mounts.
>    [bug introduced in coreutils-8.21]
> 
>    split no longer exits when invocations of a --filter return EPIPE.
> 


From MAILER-DAEMON Mon Jul 10 04:22:03 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dUTxD-0008BG-Ez
	for mharc-coreutils@gnu.org; Mon, 10 Jul 2017 04:22:03 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:48069)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <meyering@gmail.com>) id 1dUTxB-00089q-Ej
	for coreutils@gnu.org; Mon, 10 Jul 2017 04:22:02 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <meyering@gmail.com>) id 1dUTxA-0005As-Ed
	for coreutils@gnu.org; Mon, 10 Jul 2017 04:22:01 -0400
Received: from mail-vk0-x233.google.com ([2607:f8b0:400c:c05::233]:36426)
	by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)
	(Exim 4.71) (envelope-from <meyering@gmail.com>) id 1dUTxA-0004wu-8k
	for coreutils@gnu.org; Mon, 10 Jul 2017 04:22:00 -0400
Received: by mail-vk0-x233.google.com with SMTP id y70so42840674vky.3
	for <coreutils@gnu.org>; Mon, 10 Jul 2017 01:21:37 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
	h=mime-version:sender:in-reply-to:references:from:date:message-id
	:subject:to:cc;
	bh=VIP84n4a13cY6roKFGXKtOc7xNQZhXgzFXMdI21PyJM=;
	b=B5c3N2Au4nFdhQnL218qQocP130PjZ92+AdOAebIiy1Ws3q2BYFDiup4qsU5491Ao9
	ob3pZzzRucEnf+zGc9iEAMOo4kaLILo3P06zW+4db9mH4Qd4cwusfZif5jaZilRRiAG8
	o+L1cmnZK6XyFnlRe4Cj4ikfgGmSO9PGc1v0Xox2q42FnDQptjIlz00zTMBfbsM1pVEx
	/ep96Knx8gi2/IxcYS8r8YhL3p6CxSXnK29NGVWpzuJEVKYi5mgOoQrv3tLxIDAn6UxH
	PjKVHE7TlCac8zqbGozsCBbMBOToB0+G9suWHzlxjD+KX3wyeX0ajWlF4Zo7r+z6dzZB
	y8LQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
	d=1e100.net; s=20161025;
	h=x-gm-message-state:mime-version:sender:in-reply-to:references:from
	:date:message-id:subject:to:cc;
	bh=VIP84n4a13cY6roKFGXKtOc7xNQZhXgzFXMdI21PyJM=;
	b=dEaJJv9EPDUkGUUrgt1emw4jK/grRaMD4SwVHNe8nRgSdLYzBM35rTNibMK4yqbJGO
	1+e9RJrgDdnTYhSlORnHffim9/yJalffIp4CjlfCA1Vz2z6icEM6PP122/uzX4lPPkic
	vWkw//qVkbj1uLmYlfe/WoPPm2F2I2lG6nxL+FtUb6jMKfxkLXQTXp7uiYAomfHAbo2U
	YPU2XvynP2Xgs1JsfP+3RJWgv36oLUE6Zss5+AicOBSen9RslC7G8uA3ptz2Hzu9U6Xh
	bYHoeKN/wmR0tSHbJhIhSKz3rz0s8IL4InakZMEylgYtRNMSvb52HdVoCQP2SHGRWA+n
	+orA==
X-Gm-Message-State: AIVw112ceizBWhL7ER8z7zYU07ukc2X6l55ATAMAC+KFByHNOT4m1K31
	DDalZKm7uLKQjvKT0ZuZBSfHH964vg==
X-Received: by 10.31.154.150 with SMTP id c144mr4117629vke.125.1499674896939; 
	Mon, 10 Jul 2017 01:21:36 -0700 (PDT)
MIME-Version: 1.0
Sender: meyering@gmail.com
Received: by 10.176.66.68 with HTTP; Mon, 10 Jul 2017 01:21:16 -0700 (PDT)
In-Reply-To: <368e77e4-49dc-4c52-a341-b5fd783d0bda@bernhard-voelker.de>
References: <lubpf3eftq98xc.fsf@meyering.net>
	<368e77e4-49dc-4c52-a341-b5fd783d0bda@bernhard-voelker.de>
From: Jim Meyering <jim@meyering.net>
Date: Mon, 10 Jul 2017 10:21:16 +0200
X-Google-Sender-Auth: DMcMrbxmfkzDdPAhIij2lHZTdBk
Message-ID: <CA+8g5KFnJ0BRHP4apb45MeBXt+Rd1raShTEoMgOfZ_dQYCKGxw@mail.gmail.com>
Subject: Re: groups bug fix, groups test fix and NEWS tweak
To: Bernhard Voelker <mail@bernhard-voelker.de>
Cc: Coreutils <coreutils@gnu.org>
Content-Type: text/plain; charset="UTF-8"
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 2607:f8b0:400c:c05::233
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 10 Jul 2017 08:22:02 -0000

On Mon, Jul 10, 2017 at 8:17 AM, Bernhard Voelker
<mail@bernhard-voelker.de> wrote:
> On 07/08/2017 06:42 PM, Jim Meyering wrote:
>> Here are three proposed patches:
>>
>>   groups: do not exit early
>
> Nice catch!
>
> FWIW the bug seems to have been introduced when replacing the
> old groups.sh by the groups.c file with commit
>   167b8025aca487de001da2448c1aebc2747bc1d3 in v6.11

Hi Bernie,
Thank you for the review. Good catch.
I'd forgotten to attribute that.

I've added this line to NEWS:
+  [bug introduced in the bourne-shell-to-C rewrite for coreutils-6.11]

and this one to the commit log:
    Bug introduced via commit v6.10-56-g167b8025ac.


From MAILER-DAEMON Mon Jul 10 12:41:59 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dUbl1-0001TJ-Hk
	for mharc-coreutils@gnu.org; Mon, 10 Jul 2017 12:41:59 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:51498)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1dUbkz-0001RT-CR
	for coreutils@gnu.org; Mon, 10 Jul 2017 12:41:58 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1dUbkw-0000O9-9W
	for coreutils@gnu.org; Mon, 10 Jul 2017 12:41:57 -0400
Received: from mail.magicbluesmoke.com ([82.195.144.49]:44188)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)
	(Exim 4.71) (envelope-from <P@draigBrady.com>) id 1dUbkw-0000Mg-1z
	for coreutils@gnu.org; Mon, 10 Jul 2017 12:41:54 -0400
Received: from localhost.localdomain (unknown [73.93.153.18])
	(using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))
	(No client certificate requested)
	by mail.magicbluesmoke.com (Postfix) with ESMTPSA id CD51794FA;
	Mon, 10 Jul 2017 17:41:50 +0100 (IST)
Subject: Re: [PATCH] nproc: fix indentation of usage output
To: Andreas Schwab <schwab@linux-m68k.org>, coreutils@gnu.org
References: <87van2oz9i.fsf@linux-m68k.org>
From: =?UTF-8?Q?P=c3=a1draig_Brady?= <P@draigBrady.com>
Message-ID: <cf99fe3c-5859-d5f4-20ce-d29fd00f5592@draigBrady.com>
Date: Mon, 10 Jul 2017 09:41:43 -0700
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101
	Thunderbird/45.8.0
MIME-Version: 1.0
In-Reply-To: <87van2oz9i.fsf@linux-m68k.org>
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: 7bit
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
	[fuzzy]
X-Received-From: 82.195.144.49
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 10 Jul 2017 16:41:58 -0000

On 09/07/17 00:17, Andreas Schwab wrote:
> * src/nproc.c (usage): Align output.
> ---
>  src/nproc.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/src/nproc.c b/src/nproc.c
> index fdcce591f4..d7455db70f 100644
> --- a/src/nproc.c
> +++ b/src/nproc.c
> @@ -61,8 +61,8 @@ which may be less than the number of online processors\n\
>  \n\
>  "), stdout);
>        fputs (_("\
> -     --all       print the number of installed processors\n\
> -     --ignore=N  if possible, exclude N processing units\n\
> +      --all      print the number of installed processors\n\
> +      --ignore=N if possible, exclude N processing units\n\
>  "), stdout);
>  
>        fputs (HELP_OPTION_DESCRIPTION, stdout);
> 

Applied.

Thanks


From MAILER-DAEMON Mon Jul 17 13:32:55 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dX9t9-0001Xl-QQ
	for mharc-coreutils@gnu.org; Mon, 17 Jul 2017 13:32:55 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:39683)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <elfring@users.sourceforge.net>) id 1dX9m4-0005Zz-48
	for coreutils@gnu.org; Mon, 17 Jul 2017 13:25:37 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <elfring@users.sourceforge.net>) id 1dX9m0-0004zV-Nt
	for coreutils@gnu.org; Mon, 17 Jul 2017 13:25:36 -0400
Received: from mout.web.de ([212.227.15.4]:58663)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16)
	(Exim 4.71) (envelope-from <elfring@users.sourceforge.net>)
	id 1dX9m0-0004yi-Cb
	for coreutils@gnu.org; Mon, 17 Jul 2017 13:25:32 -0400
Received: from [192.168.1.3] ([77.181.229.206]) by smtp.web.de (mrweb003
	[213.165.67.108]) with ESMTPSA (Nemesis) id 0M2MZY-1dnSEo1u7w-00s5eu
	for <coreutils@gnu.org>; Mon, 17 Jul 2017 19:25:30 +0200
To: coreutils@gnu.org
From: SF Markus Elfring <elfring@users.sourceforge.net>
Subject: Determination of file lists from selected folders without returning
	directory names
Message-ID: <af44fed2-a5a2-21a5-a3fa-6982a5d37e1d@users.sourceforge.net>
Date: Mon, 17 Jul 2017 19:25:27 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
	Thunderbird/52.2.1
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Language: en-GB
Content-Transfer-Encoding: 8bit
X-Provags-ID: V03:K0:sKkau0mW3R3VydWtrmOHS/Pth57bFvtIP7pTEpPT/VR418Va9zL
	l4MQEU9Wrj55X1fH5rCYr2m4UR7eb4mROiWwzM2DMkxl4fPdTIDV0/4PA3GpPwfgOsvJ0Ip
	T1QWiohFqz1HLoMcBxkxMiAU5TcnCeQ5JRNOM/nTl1SZUEx4D55cNeq/Dz3h+uTQSVmlgYb
	Sweird5hOCZbbWlv5jSrQ==
X-UI-Out-Filterresults: notjunk:1;V01:K0:LPII3C/GhQ0=:+4Zp5OzrlGDNsLavxx7S+A
	XAjHyvO1Ho45eZFbQg/oiW0Gz3CiBXW9XuDosow5TcG1XauP6JuU7A3hlrIy7/OkAEz1XostE
	F7XPIMOX+hLtFnF8J4GHmlBIOLKtIqgXgB7af6bLzpLw8WvfuXFmsSZj6sjFCFNevI7t0rQL4
	R7r4jKb9wfiM2D6KE3heVwo98eY8+eEB1irJ+DedyTntvfokWrQ7JHkd16AjPMNsqh/08h4A3
	W3vGC9cwaNGbVvZosUWkj34kcNeusGaqslch13wFwjB8JWiZtLbtPxKm1Cto48CylW5tAMBH2
	McJOLau3A3aV3cUgQilcEgT9RzWXabIEpL8Ir3VUFLcum1uoAhQW8KvAjakq6F5QKCVIEr7tv
	RPLO4/UEm9LuzscuwUZmiRPWE0+3Jxc2nnuge2egcbr7kgvTvCOPCPaZzOryeQtVNM2TV3dWK
	f2212TelQCOVKemLDugE5J26Uy9qmA8w510Lgll0NcGMYgcL3njDhsD1qf2b0645qq9kn91RA
	IvR/DMngqcigt2Xtlv5QBskdMmJ98ab3p57iCMY+CMxxc2zYqWS58Z19qS8A30Stkur5F9eTu
	lj+oDrf+83g2cQRUVfw8/2UMbbRlXqGclreHV2RtlcTROABCXhwKAuKEd4C2fQqHLUM/h9TVi
	KONuifcb+J6opXmd4RrbumzRQ72k0dAxmFrOyi79A6yA+NqgQkn7vl/7+i2ioopwQSfYrgVYP
	PSJDbexjsPlpr6LTmCDL+33OeHk5J77ZHMGiOv6b2Gbac5lw7nh2uqppO2wCOVtcMkE3nlp2b
	FKGDI+7aLOpSgyJLY9oYIPEHGs24KlMB7CJllkSTHN3dh/uRK8=
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
	[fuzzy]
X-Received-From: 212.227.15.4
X-Mailman-Approved-At: Mon, 17 Jul 2017 13:32:54 -0400
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 17 Jul 2017 17:25:37 -0000

Hello,

The tool “ls” supports the filtering of directory contents. I am looking for
possibilities for performing such data processing a bit faster when directory
names can (and should) be excluded.

A corresponding result could be achieved by using a subshell (which I would
like to avoid for this use case) for a command like “(cd ${my_dir} && ls *txt)”.

Can such functionality become interesting for any more software tools?

Regards,
Markus


From MAILER-DAEMON Mon Jul 17 14:02:54 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dXAMA-0005ZK-CS
	for mharc-coreutils@gnu.org; Mon, 17 Jul 2017 14:02:54 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:48574)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1dXAM8-0005Y5-6s
	for coreutils@gnu.org; Mon, 17 Jul 2017 14:02:53 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1dXAM4-0007I9-5T
	for coreutils@gnu.org; Mon, 17 Jul 2017 14:02:52 -0400
Received: from mx1.redhat.com ([209.132.183.28]:37144)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)
	(Exim 4.71) (envelope-from <eblake@redhat.com>) id 1dXAM3-0007Hv-SJ
	for coreutils@gnu.org; Mon, 17 Jul 2017 14:02:48 -0400
Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com
	[10.5.11.15])
	(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
	(No client certificate requested)
	by mx1.redhat.com (Postfix) with ESMTPS id E0FBCC0587E2;
	Mon, 17 Jul 2017 18:02:45 +0000 (UTC)
DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com E0FBCC0587E2
Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com;
	dmarc=none (p=none dis=none) header.from=redhat.com
Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com;
	spf=pass smtp.mailfrom=eblake@redhat.com
DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com E0FBCC0587E2
Received: from [10.10.120.160] (ovpn-120-160.rdu2.redhat.com [10.10.120.160])
	by smtp.corp.redhat.com (Postfix) with ESMTP id 7EB506292E;
	Mon, 17 Jul 2017 18:02:45 +0000 (UTC)
Subject: Re: Determination of file lists from selected folders without
	returning directory names
To: SF Markus Elfring <elfring@users.sourceforge.net>, coreutils@gnu.org
References: <af44fed2-a5a2-21a5-a3fa-6982a5d37e1d@users.sourceforge.net>
From: Eric Blake <eblake@redhat.com>
Openpgp: url=http://people.redhat.com/eblake/eblake.gpg
Organization: Red Hat, Inc.
Message-ID: <262b412b-9de8-3dc8-d634-408843db51b2@redhat.com>
Date: Mon, 17 Jul 2017 13:02:44 -0500
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
	Thunderbird/52.2.1
MIME-Version: 1.0
In-Reply-To: <af44fed2-a5a2-21a5-a3fa-6982a5d37e1d@users.sourceforge.net>
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature";
	boundary="eFIewkUqEPT2PthJS6gm9PDXSrwD0t2G6"
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15
X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16
	(mx1.redhat.com [10.5.110.32]);
	Mon, 17 Jul 2017 18:02:46 +0000 (UTC)
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
	[fuzzy]
X-Received-From: 209.132.183.28
X-Content-Filtered-By: Mailman/MimeDel 2.1.21
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 17 Jul 2017 18:02:53 -0000

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--eFIewkUqEPT2PthJS6gm9PDXSrwD0t2G6
From: Eric Blake <eblake@redhat.com>
To: SF Markus Elfring <elfring@users.sourceforge.net>, coreutils@gnu.org
Message-ID: <262b412b-9de8-3dc8-d634-408843db51b2@redhat.com>
Subject: Re: Determination of file lists from selected folders without
 returning directory names
References: <af44fed2-a5a2-21a5-a3fa-6982a5d37e1d@users.sourceforge.net>
In-Reply-To: <af44fed2-a5a2-21a5-a3fa-6982a5d37e1d@users.sourceforge.net>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

On 07/17/2017 12:25 PM, SF Markus Elfring wrote:
> Hello,
>=20
> The tool =E2=80=9Cls=E2=80=9D supports the filtering of directory conte=
nts. I am looking for
> possibilities for performing such data processing a bit faster when dir=
ectory
> names can (and should) be excluded.

If you use GNU findutils:

find -maxdepth 1 \! -type d

if you want to stick to POSIX find, you'll have to be a bit more verbose
and use -prune.

--=20
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org


--eFIewkUqEPT2PthJS6gm9PDXSrwD0t2G6
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Comment: Public key at http://people.redhat.com/eblake/eblake.gpg
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEzBAEBCAAdFiEEccLMIrHEYCkn0vOqp6FrSiUnQ2oFAlls+8QACgkQp6FrSiUn
Q2ofhQgApvGpZlyH0eGNV+392pVegfU7leofzCEnZ61ZYmlaeSCXXaueh23N8xzu
nT606nFLQcZzzjTt/6gb+VKEgTAKRYeBJvzakA9wbZjoCvjlv5VzHtPphuH2+Ni8
rIak36KIrWPOdYbtw+NL8tsp/XtZP+cKpsNjmphAODe7AvZ+6RKPS18zHkDrnpIM
P4UoCftXKFmfr7sw19CyBLICOMD6vdxfJTZ8g5rTY1Fr9rtdhWCx3JRXmr5Xj0pA
LeTqKwA2cmSYza+VA7vKtn2hDSE37hNAHz9OGgTcudTAAQ+vVLeW0/eYFL+6kj6X
8uIchpI3tEGhvzC6fIv14PbHkjzTYg==
=hsEv
-----END PGP SIGNATURE-----

--eFIewkUqEPT2PthJS6gm9PDXSrwD0t2G6--


From MAILER-DAEMON Mon Jul 17 14:10:02 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dXAT3-00008N-Uy
	for mharc-coreutils@gnu.org; Mon, 17 Jul 2017 14:10:01 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:50490)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <962-396-1872@kylheku.com>) id 1dXAT1-00006t-8s
	for coreutils@gnu.org; Mon, 17 Jul 2017 14:10:00 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <962-396-1872@kylheku.com>) id 1dXASx-0001Wk-Sn
	for coreutils@gnu.org; Mon, 17 Jul 2017 14:09:59 -0400
Received: from smtp-out-so.shaw.ca ([64.59.136.138]:35735)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)
	(Exim 4.71) (envelope-from <962-396-1872@kylheku.com>)
	id 1dXASx-0001MH-ON
	for coreutils@gnu.org; Mon, 17 Jul 2017 14:09:55 -0400
Received: from kylheku.com ([70.79.163.252]) by shaw.ca with SMTP
	id XASidazMSDJTWXASjdnL3g; Mon, 17 Jul 2017 12:09:43 -0600
X-Authority-Analysis: v=2.2 cv=B4DJ6KlM c=1 sm=1 tr=0
	a=95A0EdhkF1LMGt25d7h1IQ==:117 a=95A0EdhkF1LMGt25d7h1IQ==:17
	a=IkcTkHD0fZMA:10 a=SMorJkV_YP8A:10 a=G3gG6ho9WtcA:10
	a=L3-nv-tgjbbja19k-cMA:9 a=QEXdDO2ut3YA:10
Received: from www-data by kylheku.com with local (Exim 4.72)
	(envelope-from <962-396-1872@kylheku.com>)
	id 1dXASi-0008FM-Hx; Mon, 17 Jul 2017 11:09:40 -0700
To: SF Markus Elfring <elfring@users.sourceforge.net>
Subject: Re: Determination of file lists from selected folders without
	returning directory names
X-PHP-Originating-Script: 501:rcmail.php
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8;
 format=flowed
Content-Transfer-Encoding: 8bit
Date: Mon, 17 Jul 2017 11:09:40 -0700
From: "Kaz Kylheku (Coreutils)" <962-396-1872@kylheku.com>
Cc: coreutils@gnu.org
In-Reply-To: <af44fed2-a5a2-21a5-a3fa-6982a5d37e1d@users.sourceforge.net>
References: <af44fed2-a5a2-21a5-a3fa-6982a5d37e1d@users.sourceforge.net>
Message-ID: <c97927cd182f34e3527fe741cce0b726@mail.kylheku.com>
X-Sender: 962-396-1872@kylheku.com
User-Agent: Roundcube Webmail/0.9.2
X-CMAE-Envelope: MS4wfDSWtSDcWpkhyu/aasgZE5kXGGG34GjpytlZiRqj/03iav5uaiZ623rcTu3O4V5MXUIfAbLTsNQVyNlmb9bQahp9bK3Xf+H8dnXgtVAGz99k74qXsVfE
	I21SUZthqjeKRwkxqC8kIw96bVHlCWYdVVgzKy2LFGRXx95RtBPhVoqmkXBbq/pYL5RP4wEYikgVQ32zDIlOSWm1fyk15yXNlNk=
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy]
X-Received-From: 64.59.136.138
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 17 Jul 2017 18:10:00 -0000

On 17.07.2017 10:25, SF Markus Elfring wrote:
> Hello,
> 
> The tool “ls” supports the filtering of directory contents.

No, it actually doesn't. In a shell command like

   ls *.txt

it is actually the shell which performs the *.txt filename expansion,
before the ls program is executed. That program receives all of the
file names as individual arguments, rather than the original pattern.

If you just want the names themselves, you can simply do:

   echo *.txt

One advantage of this is that it avoids the limitations on the
size of the argument vector that can be passed to a child process,
because echo is built-in (if we're talking about GNU Bash, which
is reasonable, given that we're in the GNU Coreutils list.)

Iterating over the matching names is possible using the for syntax:

   for x in *.txt ; do
     commands ...
   done

That also avoids limitations on argument passing since it is all
built-in syntax.


> A corresponding result could be achieved by using a subshell (which I 
> would
> like to avoid for this use case) for a command like “(cd ${my_dir} && 
> ls *txt)”.

If you want to capture these strings into a variable, you can't really
avoid a sub-process. For instance simply doing:

    names=$(echo *txt)

involves a sub-process for the command substitution. Since it is echo,
it could be optimized. I just checked though; bash 4.3.48 is forking
a child process for this. So there is hardly much additional 
disadvantage
from doing:

    names=$(cd; echo *txt)

that just adds a chdir() system call to the child script.

If the goal is to just dump the names on standard output without the
directory prefix, it can be done like this:

    for name in dir/*txt ; do
      echo ${name#dir/}
    done

this involves no forking of a child process.

To get them on the same line as with echo:

    for name in dir/*txt ; do
      printf "%s " "$name"
    done
    echo # emit final newline



From MAILER-DAEMON Mon Jul 17 15:03:56 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dXBJE-0006W8-I8
	for mharc-coreutils@gnu.org; Mon, 17 Jul 2017 15:03:56 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:38845)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <elfring@users.sourceforge.net>) id 1dXBJC-0006T0-FX
	for coreutils@gnu.org; Mon, 17 Jul 2017 15:03:55 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <elfring@users.sourceforge.net>) id 1dXBJ9-0007la-3Y
	for coreutils@gnu.org; Mon, 17 Jul 2017 15:03:54 -0400
Received: from mout.web.de ([212.227.15.3]:65140)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16)
	(Exim 4.71) (envelope-from <elfring@users.sourceforge.net>)
	id 1dXBJ8-0007eu-Pz
	for coreutils@gnu.org; Mon, 17 Jul 2017 15:03:51 -0400
Received: from [192.168.1.3] ([77.181.229.206]) by smtp.web.de (mrweb002
	[213.165.67.108]) with ESMTPSA (Nemesis) id 0MPH2u-1dbb9m45Gy-004VzV;
	Mon, 17 Jul 2017 21:03:39 +0200
Subject: Re: Determination of file lists from selected folders without
	returning directory names
To: Eric Blake <eblake@redhat.com>
References: <af44fed2-a5a2-21a5-a3fa-6982a5d37e1d@users.sourceforge.net>
	<262b412b-9de8-3dc8-d634-408843db51b2@redhat.com>
Cc: coreutils@gnu.org
From: SF Markus Elfring <elfring@users.sourceforge.net>
Message-ID: <4fc39cfa-1573-2723-d133-d9a71a426546@users.sourceforge.net>
Date: Mon, 17 Jul 2017 21:03:35 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
	Thunderbird/52.2.1
MIME-Version: 1.0
In-Reply-To: <262b412b-9de8-3dc8-d634-408843db51b2@redhat.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-GB
Content-Transfer-Encoding: 8bit
X-Provags-ID: V03:K0:7HnyR7yA/l6TFMchqj4Lewhf4dWo+1X5W9VOqgyGy/i4cbXLjqM
	knjY3uP4zGJWFsoVLxDi6+z0ABBCtwK+5l2ImSCo0yNrksXpjaGjaL0PDC1rigRFlvpLd2B
	+urUh68vHDJvkmOzK34cOFcOWHdP67N/EqnLO2G1pcAjEvjOOZy/Y8GsfS5GyxAx0ZuW4wT
	QsOx3QuMqF2DzGMMhNqYw==
X-UI-Out-Filterresults: notjunk:1;V01:K0:3WR6sVbwni0=:lRlZ3Z+WH1cCvu0Ii7y7Dg
	CpSi3BqpirNnw6tBP2Wis5qGyGX/appFtAKbT4O0/3pdG+jBlDfGXWAw9o4YCd5aOga9gcIeC
	1DBbJO2kVxfaTpFZS4UdQIkvRWXFC+pCda/cRITHwUO2x9QoKMyLBLGOawBnWDTleu9lE4rR8
	odPTiCre49IBxU5nS/4fl3TNB6Wm30k6fejB1QQmXPB32KuG2/iEy20r20w+sbhtfHmMjQETf
	MKZwlgZjjFxPUhTIXa/K3DSUE1ICBDVS6kJyjYDz7ofoT4/IUvdu1yXnME/KJfirTDHjVBiZK
	eP2Qq/dh1Ds8QwQq7S+J9EdgcAwz9A7DK+f9vpq0/zRfsgcZO1DWcCqFahsaIUPwGV7H60bQZ
	IyWWstdpBGGbBDvRDW6bCeISLoyqTMmRiHLOhIZzVaOeCpRRXdvCQ5H2k3va64IdnkjomrmEc
	nTRQC4xYkbRuJSKV/f9IcwnwIvYbulrLBwh8tkt35v1+jeHb0DBgigT7mhQch6ShONSvr7Lof
	6rmvCOOtPg2HsTwuhQKcM30HcJjbeDhI0xU70fDvocoFlRakCX2W24TEp/TsiEp7q84rb/G3+
	Eb2qKHUwMnjuipTubJ5+loovMyJly50JnAuNJ+XDU9Zm46KnWSjHa7OM/ylJTE4JhgLyy+Bcg
	aa6S2mE1iQouLHCYItu3WeiSG3B43YbrOaGcYd/0QSQMiWBy0ubCgJUM25pnjWYQGBY6jbsYi
	zgD9F9S2lEIBEJY6IESDgWhj8DC2mMWUPsNQrdAd10sh3d2EgAQ+QKgJc/PT1d6tpvjboH0BD
	mel0uLOEHGp2kHP5CIeyFmbOY27ddV+aqqZ2TWa/GpkwcEgZv0=
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
	[fuzzy]
X-Received-From: 212.227.15.3
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 17 Jul 2017 19:03:55 -0000

>> The tool “ls” supports the filtering of directory contents. I am looking for
>> possibilities for performing such data processing a bit faster when directory
>> names can (and should) be excluded.
> 
> If you use GNU findutils:
> 
> find -maxdepth 1 \! -type d
> 
> if you want to stick to POSIX find, you'll have to be a bit more verbose
> and use -prune.

The result does not look in the way I was looking for when I try the command
“find ~/Projekte/Bau -name '*txt' -prune” or
“find ~/Projekte/Bau -name '*txt' \! -type d” out.


Does any programming interface exist already which will provide only
basenames for the found files directly?

Regards,
Markus


From MAILER-DAEMON Mon Jul 17 15:38:01 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dXBqD-0001uI-Kl
	for mharc-coreutils@gnu.org; Mon, 17 Jul 2017 15:38:01 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:47744)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <elfring@users.sourceforge.net>) id 1dXBqB-0001tD-Lw
	for coreutils@gnu.org; Mon, 17 Jul 2017 15:38:00 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <elfring@users.sourceforge.net>) id 1dXBq8-0002Gw-7W
	for coreutils@gnu.org; Mon, 17 Jul 2017 15:37:59 -0400
Received: from mout.web.de ([212.227.15.4]:57285)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16)
	(Exim 4.71) (envelope-from <elfring@users.sourceforge.net>)
	id 1dXBq7-0002GH-SW
	for coreutils@gnu.org; Mon, 17 Jul 2017 15:37:56 -0400
Received: from [192.168.1.3] ([77.181.229.206]) by smtp.web.de (mrweb004
	[213.165.67.108]) with ESMTPSA (Nemesis) id 0MZims-1dCvA632UQ-00LTWc;
	Mon, 17 Jul 2017 21:37:44 +0200
Subject: Re: Determination of file lists from selected folders without
	returning directory names
To: Kaz Kylheku <962-396-1872@kylheku.com>
Cc: coreutils@gnu.org
References: <af44fed2-a5a2-21a5-a3fa-6982a5d37e1d@users.sourceforge.net>
	<c97927cd182f34e3527fe741cce0b726@mail.kylheku.com>
From: SF Markus Elfring <elfring@users.sourceforge.net>
Message-ID: <7c44c5c2-e5cd-b368-a8bd-d7dc6dc3de3f@users.sourceforge.net>
Date: Mon, 17 Jul 2017 21:37:41 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
	Thunderbird/52.2.1
MIME-Version: 1.0
In-Reply-To: <c97927cd182f34e3527fe741cce0b726@mail.kylheku.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-GB
Content-Transfer-Encoding: 8bit
X-Provags-ID: V03:K0:aOrr4BmJnjVz7aoyRzHMYhH71KRHK+FbdwoOQpOj9Odv8OTh0mu
	pbeRAOfh27ZVEaIgmhNER//cJDIn71VdNReIeIeGxK5NMIMhjOuHZFZ3pbto7WjUnjP21so
	GUujOTFShpUHignVV34eUqVpRPnIjA7eMeToAZq/V8ZbM5aADM/HtWKrpVQo1pf/pEA1+kT
	GC6RxKh4flI1X46sfGcaQ==
X-UI-Out-Filterresults: notjunk:1;V01:K0:bzT7MfNskd4=:dZnJCHAYb2/7ITo/ZnXtoy
	l9wB9ZXFCu90EygixNAj/fi3ipZ7NSNYrdohl8J2yS31VDxEUhV2FgcO4s9k9oESwDrwLwFje
	bvT0HGqTDyqGK2/ZlMt45rFOkHfshIi96LnX9TTDcZ+Ohz4rSmV2YUJqvlxt1iQ5Pg0vR3ePY
	f5IFxg3dNUUIswwVZab8Mmyz+iUIQir4vg65Iha6VPuhglazb99uxNw9VUdfKwzcOrTcKiuzV
	ojVXSi8MWzv9Wv5MnyzKp75/0lNeWpqmfxSHjBU8n+/V4lF8ZYrveDLQQcyoZ5bzF+QttLVzI
	hIm56In9dZ1OayVTzHa+wqhus4IyiC5g6czWDTn9ru8V4oNAHlIbY2hucc3V80lkchCUO+/7V
	GVXfphFvpGP4VxD4Lcc+VboIEZquh8gyFWXU4IY8eNjo7xSHJGZb916uSQ9juAvO/CSiv3zxu
	qyHa3YFdkpAwgBeCWJvEdTOCNEUSwPCid1NpEolupMUXu7s3uVtfD4deupYggsV/Siq6zlotY
	dmUuSqaE5cfw755qG1u66dO/+cnTQqVw7O8Htv9RXZVLfbytHZbWsOqzoC/tzt29Qq5h6gIFh
	PoZvUSN4E+0xSWc/gfEyDjwpDcTkWCwYBOJ2OJNZ/oTmFgr37LR/Q01mzgqBfv3XgnEwxbILl
	A2ODEjTA52ozyZ/2Q9YGCOTyt6fPCiWcn+hHE7bl+UC92v5qZSdAQF+unYU6/VK5HjKQLRLvx
	ojKjSLQPTyKplTmVBzfubYiViSVl326oGr6pHsvI7FoYimn40x9pU0sLcix1yHXkiK1tGmyjf
	WE/IuhyuJpgD0mkvKtD32SBDrG6/28O9NOV4Ll3WKUx7kU5pDw=
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
	[fuzzy]
X-Received-From: 212.227.15.4
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 17 Jul 2017 19:38:00 -0000

>> A corresponding result could be achieved by using a subshell (which I would
>> like to avoid for this use case) for a command like “(cd ${my_dir} && ls *txt)”.
> 
> If you want to capture these strings into a variable, you can't really
> avoid a sub-process.

I looked at a programming interface like the function “opendir”.
I imagine that there are more advanced possibilities to improve the software
run time characteristics for this use case.


> For instance simply doing:
> 
>    names=$(echo *txt)
> 
> involves a sub-process for the command substitution. Since it is echo,
> it could be optimized.

I hope so.


> I just checked though; …

Thanks for your interest.


> If the goal is to just dump the names on standard output without the
> directory prefix,

Something like that.


> it can be done like this:
> 
>    for name in dir/*txt ; do
>      echo ${name#dir/}
>    done

I would like to avoid such an operation “Remove matching prefix pattern” generally.
If the desired file lists contain only basenames, extra prefixes do not need
to be deleted.

Regards,
Markus


From MAILER-DAEMON Mon Jul 17 15:42:13 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dXBuH-0003V3-Gb
	for mharc-coreutils@gnu.org; Mon, 17 Jul 2017 15:42:13 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:49037)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1dXBuE-0003TK-3W
	for coreutils@gnu.org; Mon, 17 Jul 2017 15:42:11 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1dXBuA-0004wy-LR
	for coreutils@gnu.org; Mon, 17 Jul 2017 15:42:10 -0400
Received: from mx1.redhat.com ([209.132.183.28]:56086)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)
	(Exim 4.71) (envelope-from <eblake@redhat.com>) id 1dXBuA-0004wj-Bb
	for coreutils@gnu.org; Mon, 17 Jul 2017 15:42:06 -0400
Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com
	[10.5.11.14])
	(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
	(No client certificate requested)
	by mx1.redhat.com (Postfix) with ESMTPS id DFEBE7F6A2;
	Mon, 17 Jul 2017 19:42:04 +0000 (UTC)
DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com DFEBE7F6A2
Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com;
	dmarc=none (p=none dis=none) header.from=redhat.com
Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com;
	spf=pass smtp.mailfrom=eblake@redhat.com
DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com DFEBE7F6A2
Received: from [10.10.120.160] (ovpn-120-160.rdu2.redhat.com [10.10.120.160])
	by smtp.corp.redhat.com (Postfix) with ESMTP id 65B025D9CB;
	Mon, 17 Jul 2017 19:42:04 +0000 (UTC)
Subject: Re: Determination of file lists from selected folders without
	returning directory names
To: SF Markus Elfring <elfring@users.sourceforge.net>
References: <af44fed2-a5a2-21a5-a3fa-6982a5d37e1d@users.sourceforge.net>
	<262b412b-9de8-3dc8-d634-408843db51b2@redhat.com>
	<4fc39cfa-1573-2723-d133-d9a71a426546@users.sourceforge.net>
Cc: coreutils@gnu.org
From: Eric Blake <eblake@redhat.com>
Openpgp: url=http://people.redhat.com/eblake/eblake.gpg
Organization: Red Hat, Inc.
Message-ID: <1cde9c3b-1c00-03fa-7ebd-d51cea12bf2b@redhat.com>
Date: Mon, 17 Jul 2017 14:42:03 -0500
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
	Thunderbird/52.2.1
MIME-Version: 1.0
In-Reply-To: <4fc39cfa-1573-2723-d133-d9a71a426546@users.sourceforge.net>
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature";
	boundary="8KhVgPWk2mR7GvuX8EkQa74GMklcuKi3L"
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14
X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16
	(mx1.redhat.com [10.5.110.25]);
	Mon, 17 Jul 2017 19:42:05 +0000 (UTC)
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
	[fuzzy]
X-Received-From: 209.132.183.28
X-Content-Filtered-By: Mailman/MimeDel 2.1.21
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 17 Jul 2017 19:42:11 -0000

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--8KhVgPWk2mR7GvuX8EkQa74GMklcuKi3L
From: Eric Blake <eblake@redhat.com>
To: SF Markus Elfring <elfring@users.sourceforge.net>
Cc: coreutils@gnu.org
Message-ID: <1cde9c3b-1c00-03fa-7ebd-d51cea12bf2b@redhat.com>
Subject: Re: Determination of file lists from selected folders without
 returning directory names
References: <af44fed2-a5a2-21a5-a3fa-6982a5d37e1d@users.sourceforge.net>
 <262b412b-9de8-3dc8-d634-408843db51b2@redhat.com>
 <4fc39cfa-1573-2723-d133-d9a71a426546@users.sourceforge.net>
In-Reply-To: <4fc39cfa-1573-2723-d133-d9a71a426546@users.sourceforge.net>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

On 07/17/2017 02:03 PM, SF Markus Elfring wrote:
>>> The tool =E2=80=9Cls=E2=80=9D supports the filtering of directory con=
tents. I am looking for
>>> possibilities for performing such data processing a bit faster when d=
irectory
>>> names can (and should) be excluded.
>>
>> If you use GNU findutils:
>>
>> find -maxdepth 1 \! -type d
>>
>> if you want to stick to POSIX find, you'll have to be a bit more verbo=
se
>> and use -prune.
>=20
> The result does not look in the way I was looking for when I try the co=
mmand
> =E2=80=9Cfind ~/Projekte/Bau -name '*txt' -prune=E2=80=9D or

Are you sure that does what you want?  It prunes only directories named
'*txt' (if there are any).

> =E2=80=9Cfind ~/Projekte/Bau -name '*txt' \! -type d=E2=80=9D out.
>=20
>=20
> Does any programming interface exist already which will provide only
> basenames for the found files directly?

Again, if you are using GNU find, then the -printf directive is your
friend.  If you are trying to use just POSIX tools, then you have to
pipeline together multiple tools to get the same effect.

Please, help us help you.  Give us a sample directory layout, along with
sample output that you are hoping to produce from it.  Chances are,
existing tools can do what you want, but having a concrete example in
front of us instead of a vague request will make it easier (and
hopefully you aren't trying to use us to answer a homework problem for yo=
u).

For example, I can do:

$ mkdir -p a/b
$ touch a/f1 a/f2.txt a/b/f3 a/b/f4.txt
$ find a -maxdepth 1 \! -type d -printf %f\\n
f1
f2.txt

--=20
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org


--8KhVgPWk2mR7GvuX8EkQa74GMklcuKi3L
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Comment: Public key at http://people.redhat.com/eblake/eblake.gpg
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEzBAEBCAAdFiEEccLMIrHEYCkn0vOqp6FrSiUnQ2oFAlltEwsACgkQp6FrSiUn
Q2qTOggAgQFr+jVsjT5but1c2vwiue/idG2kKZ0TzjSf65hm/jzZic4fPzJvsR90
29F4boNPnfwC1TQtMrsB7uEIL0k7P+eMJOqvJyMex0QTvYvnSNLLf+GJKTPbCon4
d/UHC8QqD4ZZPYj+5Qt0CCr/B3h6JDtQ7YVk7SYI70X7S+/wuka9MneQmSH8xc/+
CR9SMk7E+z3u0lNzIA1d8gyKnZcbtDRfmICPB6wq59KySLUJQDxJOOnFxIw901nb
0UnRn7v3CoW5g0LlIdH12y+kt5J/dAdbcCPvHZylnc9VRtQWUkrdUQg3y9d1HOjh
63Rc/64QgAy+E32A1g7vEoP5JK22KA==
=urbD
-----END PGP SIGNATURE-----

--8KhVgPWk2mR7GvuX8EkQa74GMklcuKi3L--


From MAILER-DAEMON Mon Jul 17 16:25:17 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dXCZx-0002LX-FR
	for mharc-coreutils@gnu.org; Mon, 17 Jul 2017 16:25:17 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:33857)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <elfring@users.sourceforge.net>) id 1dXCZu-0002LO-Qg
	for coreutils@gnu.org; Mon, 17 Jul 2017 16:25:15 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <elfring@users.sourceforge.net>) id 1dXCZr-0007px-Ai
	for coreutils@gnu.org; Mon, 17 Jul 2017 16:25:14 -0400
Received: from mout.web.de ([212.227.15.3]:52062)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16)
	(Exim 4.71) (envelope-from <elfring@users.sourceforge.net>)
	id 1dXCZq-0007oG-W5
	for coreutils@gnu.org; Mon, 17 Jul 2017 16:25:11 -0400
Received: from [192.168.1.3] ([77.181.229.206]) by smtp.web.de (mrweb002
	[213.165.67.108]) with ESMTPSA (Nemesis) id 0LbZlL-1dvq9542Rv-00lFgy;
	Mon, 17 Jul 2017 22:25:05 +0200
Subject: Re: Determination of file lists from selected folders without
	returning directory names
To: Eric Blake <eblake@redhat.com>
Cc: coreutils@gnu.org
References: <af44fed2-a5a2-21a5-a3fa-6982a5d37e1d@users.sourceforge.net>
	<262b412b-9de8-3dc8-d634-408843db51b2@redhat.com>
	<4fc39cfa-1573-2723-d133-d9a71a426546@users.sourceforge.net>
	<1cde9c3b-1c00-03fa-7ebd-d51cea12bf2b@redhat.com>
From: SF Markus Elfring <elfring@users.sourceforge.net>
Message-ID: <157f0302-f99f-605c-a9fa-ae5312e82a52@users.sourceforge.net>
Date: Mon, 17 Jul 2017 22:25:02 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
	Thunderbird/52.2.1
MIME-Version: 1.0
In-Reply-To: <1cde9c3b-1c00-03fa-7ebd-d51cea12bf2b@redhat.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-GB
Content-Transfer-Encoding: 8bit
X-Provags-ID: V03:K0:I9Roq9RPmraZXlujzO/0yUf/dHQhHvWHXPVtvQIwYZAEb10XuJ9
	FOo3hOgLmHwXllr8tAvemo48VYHpOGZY8gnB1Z50JaN3YOXpZy5iIIP5Czg57Ckmoczj3W0
	CDuXLW5JIuK+KL/F2VUb1KRaEKTWyWDFKy5nIcKuIW1Uc5CEM6s9qSCYG+rDn4MLRxzEydy
	xxwTZVf+BV/h7Fp20b0wA==
X-UI-Out-Filterresults: notjunk:1;V01:K0:JV4JnUav6ro=:85GQaaHCQAm0EBfBtAmKLY
	scNO5mjhE2m9CvBZ7h8XhbckL2CLFICe9LVQQ3D40FYKu53dGh8Xys4u7sGrH/UjYnOe1eld3
	SF5SZy9SZmmIsYVUmKyi+h+BXEaHxwFtp+IEoQ5vD5B58RGw9/96XlNsn76LMvyzv7oxHbfAo
	U1avYTLJTTGqhKKBUVovThFCRRbufOPQC2bYoMrh02OUH6lYKJk7EwsVnYuJPQHxzMJLbd+4z
	PNZKq5g0jNLv0CqYIxoONJ4Y+n7r8ZoacBLjw51mHjIrRE3QmedZv7hCsyb+BGb2+GYsGEFYi
	cVR2oTuOG+3Y+pew3qI9eFh5+m1UHNkToCyuQA/HQ3srMz1njlB1k2AQf8HFRVeBgVWq+Auau
	a8LP/7F9cOuIgfswj8ao4DpjciO/qR9n6W/2e/zCVYMFoP29gZx4+asAivFswaosRXCmi+Lca
	P9gwDCboIGOsAgmkWifrin9L8+CC+tvP36jkbsx5wM5XQowNJ1OSTEPdHaIjTFUDgAD5bKhfw
	AiihOKZP0Ujb7ktSSmLoZX046K8nr3KdcAo1rBoM6NyLcvVkNjOYlDbzKzL67vYoqGZaASUUa
	KcoBoPkKwP1vDjj68562ZpcXauD2nI41YY9OJy77d3nbdEBu6eiJDbUTeRDIcyF06YCno69Gx
	bGend5bdnj7gzRpgJz7/Lhe+8OJp+U8gvauzNMg3PBs2ntod2Kgs1/t8D5QP15YXoS1H7z1zN
	F20mxn9AfcCQHr9Is96jLxu1C2/e6yMWmo7KstGdwDNR3Xwyuy8ZzoNJmBdV01riy+ffcA1JX
	y//KJeQ7IIFRgtfhHbwvGyiQXUdCHTgP/z5V/m24JbGY4EwxgA=
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
	[fuzzy]
X-Received-From: 212.227.15.3
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 17 Jul 2017 20:25:15 -0000

>> The result does not look in the way I was looking for when I try the command
>> “find ~/Projekte/Bau -name '*txt' -prune” or
> 
> Are you sure that does what you want?

No. - My imaginations can not be mapped to this command interface directly
so far.


>> Does any programming interface exist already which will provide only
>> basenames for the found files directly?
> 
> Again, if you are using GNU find, then the -printf directive is your friend.

This information can be appropriate for some use cases.


> If you are trying to use just POSIX tools, then you have to pipeline together
> multiple tools to get the same effect.

I know that this data processing style is usual.


Does a mapping of the function “opendir” (or “fdopendir”) exist for a command?


> Please, help us help you.

Your feedback is useful already.


> Chances are, existing tools can do what you want, but having a concrete example
> in front of us instead of a vague request will make it easier

Can a similar feature request “Determination of a file list from a single folder
without changing the working directory” (for the tool “GNU Make”) help for
a better understanding?
https://savannah.gnu.org/bugs/index.php?51309


> $ find a -maxdepth 1 \! -type d -printf %f\\n

Can it be that this command retrieved more data than necessary just for the
display of the basename?

Regards,
Markus


From MAILER-DAEMON Mon Jul 17 16:40:40 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dXCoq-0000q1-Oo
	for mharc-coreutils@gnu.org; Mon, 17 Jul 2017 16:40:40 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:39665)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1dXCoo-0000oO-Hi
	for coreutils@gnu.org; Mon, 17 Jul 2017 16:40:39 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1dXCol-0007va-J1
	for coreutils@gnu.org; Mon, 17 Jul 2017 16:40:38 -0400
Received: from mx1.redhat.com ([209.132.183.28]:42042)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)
	(Exim 4.71) (envelope-from <eblake@redhat.com>) id 1dXCol-0007ut-5A
	for coreutils@gnu.org; Mon, 17 Jul 2017 16:40:35 -0400
Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com
	[10.5.11.11])
	(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
	(No client certificate requested)
	by mx1.redhat.com (Postfix) with ESMTPS id 1C695C057EC9;
	Mon, 17 Jul 2017 20:40:34 +0000 (UTC)
DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 1C695C057EC9
Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com;
	dmarc=none (p=none dis=none) header.from=redhat.com
Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com;
	spf=pass smtp.mailfrom=eblake@redhat.com
DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 1C695C057EC9
Received: from [10.10.120.160] (ovpn-120-160.rdu2.redhat.com [10.10.120.160])
	by smtp.corp.redhat.com (Postfix) with ESMTP id 99EAF60464;
	Mon, 17 Jul 2017 20:40:33 +0000 (UTC)
Subject: Re: Determination of file lists from selected folders without
	returning directory names
To: SF Markus Elfring <elfring@users.sourceforge.net>
References: <af44fed2-a5a2-21a5-a3fa-6982a5d37e1d@users.sourceforge.net>
	<262b412b-9de8-3dc8-d634-408843db51b2@redhat.com>
	<4fc39cfa-1573-2723-d133-d9a71a426546@users.sourceforge.net>
	<1cde9c3b-1c00-03fa-7ebd-d51cea12bf2b@redhat.com>
	<157f0302-f99f-605c-a9fa-ae5312e82a52@users.sourceforge.net>
Cc: coreutils@gnu.org
From: Eric Blake <eblake@redhat.com>
Openpgp: url=http://people.redhat.com/eblake/eblake.gpg
Organization: Red Hat, Inc.
Message-ID: <813649f5-891d-54b4-f339-75b1f6e29cc4@redhat.com>
Date: Mon, 17 Jul 2017 15:40:32 -0500
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
	Thunderbird/52.2.1
MIME-Version: 1.0
In-Reply-To: <157f0302-f99f-605c-a9fa-ae5312e82a52@users.sourceforge.net>
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature";
	boundary="fnqVAK5QHprtBq4ul54OAqI09uCw50BGa"
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11
X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16
	(mx1.redhat.com [10.5.110.32]);
	Mon, 17 Jul 2017 20:40:34 +0000 (UTC)
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
	[fuzzy]
X-Received-From: 209.132.183.28
X-Content-Filtered-By: Mailman/MimeDel 2.1.21
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 17 Jul 2017 20:40:39 -0000

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--fnqVAK5QHprtBq4ul54OAqI09uCw50BGa
From: Eric Blake <eblake@redhat.com>
To: SF Markus Elfring <elfring@users.sourceforge.net>
Cc: coreutils@gnu.org
Message-ID: <813649f5-891d-54b4-f339-75b1f6e29cc4@redhat.com>
Subject: Re: Determination of file lists from selected folders without
 returning directory names
References: <af44fed2-a5a2-21a5-a3fa-6982a5d37e1d@users.sourceforge.net>
 <262b412b-9de8-3dc8-d634-408843db51b2@redhat.com>
 <4fc39cfa-1573-2723-d133-d9a71a426546@users.sourceforge.net>
 <1cde9c3b-1c00-03fa-7ebd-d51cea12bf2b@redhat.com>
 <157f0302-f99f-605c-a9fa-ae5312e82a52@users.sourceforge.net>
In-Reply-To: <157f0302-f99f-605c-a9fa-ae5312e82a52@users.sourceforge.net>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

On 07/17/2017 03:25 PM, SF Markus Elfring wrote:
> Does a mapping of the function =E2=80=9Copendir=E2=80=9D (or =E2=80=9Cf=
dopendir=E2=80=9D) exist for a command?
>=20

Only indirectly, and typically via programs that (can) do recursion.
'find' is your best bet (it does not sort; while ls does, so find is
inherently going to visit files faster than ls); couple find with a
-maxdepth 1 so that it doesn't recurse by default, and the only thing
faster is then shell expansion of globs (as in 'echo *').

> Can a similar feature request =E2=80=9CDetermination of a file list fro=
m a single folder
> without changing the working directory=E2=80=9D (for the tool =E2=80=9C=
GNU Make=E2=80=9D) help for
> a better understanding?
> https://savannah.gnu.org/bugs/index.php?51309

Yes, that helps immensely for framing the problem you are trying to
solve.  'echo "echo $(ls *txt)"' is inherently slower (and at more risk
of bugs due to \n in file names, etc) than 'echo *.txt'.

>=20
>=20
>> $ find a -maxdepth 1 \! -type d -printf %f\\n
>=20
> Can it be that this command retrieved more data than necessary just for=
 the
> display of the basename?

strace it and find out, or read the source (this is open source, after
all).  But my gut feel is that no, it's pretty well optimized already
for a minimum of syscalls, particularly for filesystems that report file
types through readdir() (thus avoiding the need for secondary stat()
calls).  Optimizing find may be better discussed on the findutils list,
though.  For a directory with enough contents, it might even be that the
GNU find invocation outperforms a comparable $(cd dir && echo *txt),
because it avoids userspace globbing overhead, although I did not try to
benchmark that to confirm things.

--=20
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org


--fnqVAK5QHprtBq4ul54OAqI09uCw50BGa
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Comment: Public key at http://people.redhat.com/eblake/eblake.gpg
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEzBAEBCAAdFiEEccLMIrHEYCkn0vOqp6FrSiUnQ2oFAlltIMAACgkQp6FrSiUn
Q2rhgQgAkKGJJ7+biJUFe4MjicqTLdIE2U0P3HSACGDWHFIygXKSGVjNP1Bz6xvc
p/JPpT5JVYNXTjFbKHUiKOxnPxNw/i1uvli6ms7Vg6AZ2Yzm2/xSaVak1Y4ye49g
zwZ5uFKGo2OH/k64yAPAs2LWcH9SAVE7pLFwIvSdxlDSKKKP4L5IigoUm1re1l7O
GyDqOmqInJQaKzQ3AsqtKRvjLpfQBFSYhY+Ay9OEJUbSeJcmig5OswNIBu2AmKia
jYa7Cs5IM333V0Gwf8yOHupaR+rjLxCWx7wrjuKzTL4ebXUSky4IExUiZjaIAot7
eF8et2xy9ppaCJZbhCelZL0y4Fa70g==
=DQCG
-----END PGP SIGNATURE-----

--fnqVAK5QHprtBq4ul54OAqI09uCw50BGa--


From MAILER-DAEMON Mon Jul 17 17:00:41 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dXD8C-0000Xf-Sm
	for mharc-coreutils@gnu.org; Mon, 17 Jul 2017 17:00:40 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:47529)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <elfring@users.sourceforge.net>) id 1dXD8A-0000XJ-RN
	for coreutils@gnu.org; Mon, 17 Jul 2017 17:00:39 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <elfring@users.sourceforge.net>) id 1dXD86-00041W-Bv
	for coreutils@gnu.org; Mon, 17 Jul 2017 17:00:38 -0400
Received: from mout.web.de ([212.227.15.4]:58781)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16)
	(Exim 4.71) (envelope-from <elfring@users.sourceforge.net>)
	id 1dXD86-00040u-0j
	for coreutils@gnu.org; Mon, 17 Jul 2017 17:00:34 -0400
Received: from [192.168.1.3] ([77.181.229.206]) by smtp.web.de (mrweb001
	[213.165.67.108]) with ESMTPSA (Nemesis) id 0Mb8h9-1dDlUu2999-00KdFC;
	Mon, 17 Jul 2017 23:00:27 +0200
Subject: Re: Determination of file lists from selected folders without
	returning directory names
To: Eric Blake <eblake@redhat.com>
Cc: coreutils@gnu.org
References: <af44fed2-a5a2-21a5-a3fa-6982a5d37e1d@users.sourceforge.net>
	<262b412b-9de8-3dc8-d634-408843db51b2@redhat.com>
	<4fc39cfa-1573-2723-d133-d9a71a426546@users.sourceforge.net>
	<1cde9c3b-1c00-03fa-7ebd-d51cea12bf2b@redhat.com>
	<157f0302-f99f-605c-a9fa-ae5312e82a52@users.sourceforge.net>
	<813649f5-891d-54b4-f339-75b1f6e29cc4@redhat.com>
From: SF Markus Elfring <elfring@users.sourceforge.net>
Message-ID: <1ef28e81-1c34-0b36-7d21-01eb46a662be@users.sourceforge.net>
Date: Mon, 17 Jul 2017 23:00:24 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
	Thunderbird/52.2.1
MIME-Version: 1.0
In-Reply-To: <813649f5-891d-54b4-f339-75b1f6e29cc4@redhat.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-GB
Content-Transfer-Encoding: 8bit
X-Provags-ID: V03:K0:83wggomT4uwagLIqGEq6uDD26dQ1ETXg/B9kNnZv8EiGh8+f0yl
	DwtDcJSKmvCFiaotx+Cm+B84X0HmajmH7Xm+ybk3m/EkIyB4bD9hL31GpewMxosOk0urOU1
	mQ1oAJmjXmjR3HDSX805RDO+s3OoTKkmk4X+vaBhjQzhVtCGOZqTXmAVQkU0Pa2LpZ6p+rJ
	uVqoiDxfM4LoVEG19Ep4g==
X-UI-Out-Filterresults: notjunk:1;V01:K0:Segswul0FyE=:futWRq+7YiJEY+rgfOegHi
	AmHp/xuBf9K/MpNIaahUva0/PZwcvnB3Y8Mnr3f+j2X6C12/YmEEib8/3nONaU7xuolrjBXbl
	Qhe7E7fW1wBMDMNdn73ToONyu8xK+Ge8D1YsKupxkXU4dAFdAXcB61vO07dQizAuyZKLcGNyf
	loS89lkQqk0ddypRuBC14jf1Mhs/7krc9N3VthkO5AeZfzRNna1hrAQ0BtvVJMpxIzGVbMwIj
	+DlVzdt0Brz+Zie8noPLryHpZYuTCpfTSjW/8pRfIGvqiJYib0xEv8TQ0GlkN5qawg7aOF+px
	i0VcMBQwHuFDWujtrsvmpyzFPpjZ+fK24K66u3M5VhyhYdJtOzN0ZzzKVh6sUI3EpLPSFNcVH
	dlZbdfCtYRhfERW6ISEr0wEgIFUjsk5zK+qHcAJOBVQkAmY81+9lPlfAmoLTh7jaaCqjI71rD
	n5M2b3OadNHAd43FPXy5x1JSpK3DiZWJuPIKUlLG1pLoxdBDtJ3/XuM0VhqsDyCLte3eqvLcA
	3tjUyxH237EnTpz2d7orxHU4Pi14Vh8sxaQpIY52VJxjfpdqkwqIYxOXgwQt5ICzr9CeIrRyq
	lXwX7cCfaBfnmOmIkp5e+S8p6aevhMN0zDY+omp8dk/yyrZ3uILgCOuuXr17kFJLRRdUIARzD
	fs/ZPsmCTrVM3NAb7Gb8Z9RmPtam97UDANPQzDYMOQWL8rK3nZN0r2d5oa2SlSdKQ7EjbJEnx
	rYyhRn9SxYD2sUkE6SZdt80Bq/qjAYV/G20Ofi0QHhVaghHxcMhkdOZyNgZlB4juL5n7JvQXi
	IaDON7q5bdqagc5J4GIru4thSFzrKzGRhiePYilWdXlnxyutAY=
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
	[fuzzy]
X-Received-From: 212.227.15.4
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 17 Jul 2017 21:00:40 -0000

>> https://savannah.gnu.org/bugs/index.php?51309
> 
> Yes, that helps immensely for framing the problem you are trying to solve.

Would you like to look into any more scripts where I got a need
for special handling of file lists?


> 'echo "echo $(ls *txt)"' is inherently slower (and at more risk
> of bugs due to \n in file names, etc) than 'echo *.txt'.

It seems that I constructed a questionable example there.


>>> $ find a -maxdepth 1 \! -type d -printf %f\\n
>>
>> Can it be that this command retrieved more data than necessary just for the
>> display of the basename?
> 
> strace it and find out, or read the source (this is open source, after all).

Which source file would be relevant in this case?

I might get also an inappropriate impression from the wording in the manual.

“…
              %f     File's name with any leading directories removed (only the last element).
…”


The result of this functionality could fit to my imaginations.


> GNU find invocation outperforms a comparable $(cd dir && echo *txt),
> because it avoids userspace globbing overhead, although I did not try to
> benchmark that to confirm things.

I am curious to know a bit more about such a technical comparison.

Regards,
Markus


From MAILER-DAEMON Mon Jul 17 17:23:23 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dXDUB-0006iJ-Cg
	for mharc-coreutils@gnu.org; Mon, 17 Jul 2017 17:23:23 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:60022)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <962-396-1872@kylheku.com>) id 1dXDU9-0006i1-Gh
	for coreutils@gnu.org; Mon, 17 Jul 2017 17:23:22 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <962-396-1872@kylheku.com>) id 1dXDU6-0001i0-IB
	for coreutils@gnu.org; Mon, 17 Jul 2017 17:23:21 -0400
Received: from smtp-out-so.shaw.ca ([64.59.136.138]:53807)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)
	(Exim 4.71) (envelope-from <962-396-1872@kylheku.com>)
	id 1dXDU6-0001h0-DB
	for coreutils@gnu.org; Mon, 17 Jul 2017 17:23:18 -0400
Received: from kylheku.com ([70.79.163.252]) by shaw.ca with SMTP
	id XDU1d0SWaMaqMXDU2dadPz; Mon, 17 Jul 2017 15:23:16 -0600
X-Authority-Analysis: v=2.2 cv=Qc8WhoTv c=1 sm=1 tr=0
	a=95A0EdhkF1LMGt25d7h1IQ==:117 a=95A0EdhkF1LMGt25d7h1IQ==:17
	a=IkcTkHD0fZMA:10 a=SMorJkV_YP8A:10 a=G3gG6ho9WtcA:10
	a=GCvUPEvY9NzQk5FnvhcA:9 a=QEXdDO2ut3YA:10
Received: from www-data by kylheku.com with local (Exim 4.72)
	(envelope-from <962-396-1872@kylheku.com>)
	id 1dXDU1-0003pO-8a; Mon, 17 Jul 2017 14:23:13 -0700
To: SF Markus Elfring <elfring@users.sourceforge.net>
Subject: Re: Determination of file lists from selected folders without
	returning directory names
X-PHP-Originating-Script: 501:rcmail.php
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8;
 format=flowed
Content-Transfer-Encoding: 8bit
Date: Mon, 17 Jul 2017 14:23:13 -0700
From: "Kaz Kylheku (Coreutils)" <962-396-1872@kylheku.com>
Cc: coreutils@gnu.org
In-Reply-To: <7c44c5c2-e5cd-b368-a8bd-d7dc6dc3de3f@users.sourceforge.net>
References: <af44fed2-a5a2-21a5-a3fa-6982a5d37e1d@users.sourceforge.net>
	<c97927cd182f34e3527fe741cce0b726@mail.kylheku.com>
	<7c44c5c2-e5cd-b368-a8bd-d7dc6dc3de3f@users.sourceforge.net>
Message-ID: <b6745e3004da59da1e859847ed671d21@mail.kylheku.com>
X-Sender: 962-396-1872@kylheku.com
User-Agent: Roundcube Webmail/0.9.2
X-CMAE-Envelope: MS4wfANXjZBi8vZgVbtiTQ05DuaF95Kco+zYspcQHD8lv/W9OSBCecDSlMf7ym8GQrYIJjPq6MJF21wgWShk/THGa0AQ76qGoS64NibxJISUZIl8ZQoCeQJF
	Rwoa0J28Qr/aJwLk3grQwmFFSbyW7gNP8OYgNT1wuQg9+8RBiKvB3AoZHIkrHIz+xH3W+1FlaT6DkWOykCZLrAgKpKd9CStO8ks=
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy]
X-Received-From: 64.59.136.138
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 17 Jul 2017 21:23:22 -0000

On 17.07.2017 12:37, SF Markus Elfring wrote:
>>> A corresponding result could be achieved by using a subshell (which I 
>>> would
>>> like to avoid for this use case) for a command like “(cd ${my_dir} && 
>>> ls *txt)”.
>> 
>> If you want to capture these strings into a variable, you can't really
>> avoid a sub-process.
> 
> I looked at a programming interface like the function “opendir”.
> I imagine that there are more advanced possibilities to improve the 
> software
> run time characteristics for this use case.

Well, if it has to be fast, perhaps don't write the code in the shell 
language.

Even an interpreted scripting language that can do string handling 
without
resorting to fork()-based command substitution will beat the shell at 
many
tasks.

>> it can be done like this:
>> 
>>    for name in dir/*txt ; do
>>      echo ${name#dir/}
>>    done
> 
> I would like to avoid such an operation “Remove matching prefix
> pattern” generally.
> If the desired file lists contain only basenames, extra prefixes do not 
> need
> to be deleted.

I.e. we can use the basename function:

   for name in dir/*txt; do
     basename "$name"
   done

prints the basenames of the matching files, one per line.


From MAILER-DAEMON Mon Jul 17 18:03:31 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dXE71-0004IG-RN
	for mharc-coreutils@gnu.org; Mon, 17 Jul 2017 18:03:31 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:52922)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1dXE6z-0004I0-Q7
	for coreutils@gnu.org; Mon, 17 Jul 2017 18:03:31 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1dXE6w-0002PL-MQ
	for coreutils@gnu.org; Mon, 17 Jul 2017 18:03:29 -0400
Received: from mx1.redhat.com ([209.132.183.28]:33274)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)
	(Exim 4.71) (envelope-from <eblake@redhat.com>) id 1dXE6w-0002Os-Ce
	for coreutils@gnu.org; Mon, 17 Jul 2017 18:03:26 -0400
Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com
	[10.5.11.12])
	(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
	(No client certificate requested)
	by mx1.redhat.com (Postfix) with ESMTPS id 158867CB8D;
	Mon, 17 Jul 2017 22:03:25 +0000 (UTC)
DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 158867CB8D
Authentication-Results: ext-mx02.extmail.prod.ext.phx2.redhat.com;
	dmarc=none (p=none dis=none) header.from=redhat.com
Authentication-Results: ext-mx02.extmail.prod.ext.phx2.redhat.com;
	spf=pass smtp.mailfrom=eblake@redhat.com
DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 158867CB8D
Received: from [10.10.120.160] (ovpn-120-160.rdu2.redhat.com [10.10.120.160])
	by smtp.corp.redhat.com (Postfix) with ESMTP id 6BE4777DC6;
	Mon, 17 Jul 2017 22:03:24 +0000 (UTC)
Subject: Re: Determination of file lists from selected folders without
	returning directory names
To: "Kaz Kylheku (Coreutils)" <962-396-1872@kylheku.com>,
	SF Markus Elfring <elfring@users.sourceforge.net>
References: <af44fed2-a5a2-21a5-a3fa-6982a5d37e1d@users.sourceforge.net>
	<c97927cd182f34e3527fe741cce0b726@mail.kylheku.com>
	<7c44c5c2-e5cd-b368-a8bd-d7dc6dc3de3f@users.sourceforge.net>
	<b6745e3004da59da1e859847ed671d21@mail.kylheku.com>
Cc: coreutils@gnu.org
From: Eric Blake <eblake@redhat.com>
Openpgp: url=http://people.redhat.com/eblake/eblake.gpg
Organization: Red Hat, Inc.
Message-ID: <3fbf68d1-a05c-dafd-4206-ab608137fdbd@redhat.com>
Date: Mon, 17 Jul 2017 17:03:23 -0500
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
	Thunderbird/52.2.1
MIME-Version: 1.0
In-Reply-To: <b6745e3004da59da1e859847ed671d21@mail.kylheku.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature";
	boundary="wVVSs76mwoc6I3QTIuN9bvVDw29smxA9F"
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12
X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16
	(mx1.redhat.com [10.5.110.26]);
	Mon, 17 Jul 2017 22:03:25 +0000 (UTC)
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
	[fuzzy]
X-Received-From: 209.132.183.28
X-Content-Filtered-By: Mailman/MimeDel 2.1.21
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 17 Jul 2017 22:03:31 -0000

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--wVVSs76mwoc6I3QTIuN9bvVDw29smxA9F
From: Eric Blake <eblake@redhat.com>
To: "Kaz Kylheku (Coreutils)" <962-396-1872@kylheku.com>,
 SF Markus Elfring <elfring@users.sourceforge.net>
Cc: coreutils@gnu.org
Message-ID: <3fbf68d1-a05c-dafd-4206-ab608137fdbd@redhat.com>
Subject: Re: Determination of file lists from selected folders without
 returning directory names
References: <af44fed2-a5a2-21a5-a3fa-6982a5d37e1d@users.sourceforge.net>
 <c97927cd182f34e3527fe741cce0b726@mail.kylheku.com>
 <7c44c5c2-e5cd-b368-a8bd-d7dc6dc3de3f@users.sourceforge.net>
 <b6745e3004da59da1e859847ed671d21@mail.kylheku.com>
In-Reply-To: <b6745e3004da59da1e859847ed671d21@mail.kylheku.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

On 07/17/2017 04:23 PM, Kaz Kylheku (Coreutils) wrote:
> Well, if it has to be fast, perhaps don't write the code in the shell
> language.

Too true.  But that's starting to get off-topic for coreutils (and more
appropriate for the bash list).

>=20
> Even an interpreted scripting language that can do string handling with=
out
> resorting to fork()-based command substitution will beat the shell at m=
any
> tasks.
>=20
>>> it can be done like this:
>>>
>>>    for name in dir/*txt ; do
>>>      echo ${name#dir/}
>>>    done
>>
>> I would like to avoid such an operation =E2=80=9CRemove matching prefi=
x
>> pattern=E2=80=9D generally.
>> If the desired file lists contain only basenames, extra prefixes do
>> not need
>> to be deleted.
>=20
> I.e. we can use the basename function:
>=20
>   for name in dir/*txt; do
>     basename "$name"
>   done

basename is not (generally) a shell builtin.  You are fork()ing out to
one basename process per filename with this code, UNLESS you've defined
basename as a shell function - but the definition of said shell function
would resemble the 'echo ${name#dir/}' line from above.  And writing a
POSIX-compliant basename shell function is not trivial (a number of
attempts on the web get it horribly wrong on the corner cases).

GNU coreutils' basename _does_ have an extension beyond POSIX that lets
you get by with fewer processes (one process for the entire list, rather
than one per member of the list):

basename -a dir/*txt

insofar as you don't run into command-line length limits (and you may
want to mix with -z to avoid ambiguities with newlines embedded in those
filenames).

--=20
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org


--wVVSs76mwoc6I3QTIuN9bvVDw29smxA9F
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Comment: Public key at http://people.redhat.com/eblake/eblake.gpg
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEzBAEBCAAdFiEEccLMIrHEYCkn0vOqp6FrSiUnQ2oFAlltNCsACgkQp6FrSiUn
Q2qHGAf+Ny1FSU0QraNWtetJGgvBf+QjBmJPdIdP614Z2asD4cgUMVwjozQisnYY
H7jKC3fSxN2wMHvnB6uUmzj/DyBJgLYN5YUKJjWuX6vObwu/KY8QaMc3Jy5wT9iG
TrRwgGdXsiloDT+0W7E6wOhqgg1O0yaESa8I+GOz+fKSOe6ukUjOHQoJiGq7ah/C
khOSPE9HGuie8x7EtyR04En41FS5x+2e0/RuyZo6EayW+bkbZegN2FW1brhCpMSD
L5F8MnuSuLyTFtxbcaoCQZ6akL0zrPA0NcMd+WWe9M/6uqm5tf8Ke2SxcRjDUA3Y
4YQ4NXXq0cx2kgjz2lHOcrPOjFKbtw==
=O8KX
-----END PGP SIGNATURE-----

--wVVSs76mwoc6I3QTIuN9bvVDw29smxA9F--


From MAILER-DAEMON Tue Jul 18 04:18:09 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dXNhp-0000zy-Fj
	for mharc-coreutils@gnu.org; Tue, 18 Jul 2017 04:18:09 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:34987)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <elfring@users.sourceforge.net>) id 1dXNhn-0000zs-G3
	for coreutils@gnu.org; Tue, 18 Jul 2017 04:18:08 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <elfring@users.sourceforge.net>) id 1dXNhk-0002Ba-DX
	for coreutils@gnu.org; Tue, 18 Jul 2017 04:18:07 -0400
Received: from mout.web.de ([212.227.17.12]:65044)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16)
	(Exim 4.71) (envelope-from <elfring@users.sourceforge.net>)
	id 1dXNhk-0002B5-2S
	for coreutils@gnu.org; Tue, 18 Jul 2017 04:18:04 -0400
Received: from [192.168.1.3] ([77.182.168.23]) by smtp.web.de (mrweb103
	[213.165.67.124]) with ESMTPSA (Nemesis) id 0LlWCR-1e5cG72nFn-00bHIo;
	Tue, 18 Jul 2017 10:17:52 +0200
Subject: Re: Determination of file lists from selected folders without
	returning directory names
To: Kaz Kylheku <962-396-1872@kylheku.com>
Cc: coreutils@gnu.org
References: <af44fed2-a5a2-21a5-a3fa-6982a5d37e1d@users.sourceforge.net>
	<c97927cd182f34e3527fe741cce0b726@mail.kylheku.com>
	<7c44c5c2-e5cd-b368-a8bd-d7dc6dc3de3f@users.sourceforge.net>
	<b6745e3004da59da1e859847ed671d21@mail.kylheku.com>
From: SF Markus Elfring <elfring@users.sourceforge.net>
Message-ID: <12c091e8-380d-b9c4-5d95-78d2d1f02041@users.sourceforge.net>
Date: Tue, 18 Jul 2017 10:17:38 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
	Thunderbird/52.2.1
MIME-Version: 1.0
In-Reply-To: <b6745e3004da59da1e859847ed671d21@mail.kylheku.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-GB
Content-Transfer-Encoding: 8bit
X-Provags-ID: V03:K0:SzheOdpFsP+OM6TDIJ2POPT7e1RF+ubi4V8iCu6FCottz3av04S
	k2p4EjuDLfbgwihjNTVdPaOrOmiFe9Fb/N7C5Qw7aNKF9HMxY8Cq33wUOmC8s3r+RvRsg+X
	86dJZncjyTgVBwgTG+dedRqXh2nW8AIdLU7cAFKJIMWys2NoZSx1CFnusByT2hh3E+EVx4p
	tMtZl0Kj6fcHb6ysN5+LA==
X-UI-Out-Filterresults: notjunk:1;V01:K0:5jdBZMLDpyM=:39+UJlFVaUIeZugzm0Oj7X
	KhcZe4wmvMr7Qz9V3xOwM+9SsHm1cs41Vuf1NBIg8HOMQS3U/XeYvNHsHn0Eb4TeYaLSL/xxr
	nRs8bjyeT2o9kLrqTXoq+z0i24pBEO5KkknoQ57+hoe33y4jZn4nxFS0lQgm3UlX1q6Xlgjfd
	V6ycYLdIYggmrfCOEpQhHOdWlwlhSVPApfdD9mZ8UeD6MW/S9sJIqSbxTPay67RvDkQd9/eVW
	0K9Zip08mCug5TuI2x8hTtuEoHN6o0hSSyA3NDGGaBYy23y9w4x08lIas6WTiwK8Tj2ScsluF
	uznEPfYgh23ANkOm3uUZOtmpv4JeDTYZgu5MdYs/+n2Drc1i30FB3FmvhSccAcVBZ2B837oQ6
	9TbzKCzGKkO7JLAubpjUVNp01OM03O7GDpfk67y2Fms1mREnafNzpfC+A5VmoitGV6h0tSPSz
	enz+s2Vcj4lhvRfWjO7aHQD3NSHijAyIJTEGgtbwhPwKDcYoWRiawxexqxWxV8KuoZwREXvdp
	nE7BjUmwCly1xXsuNE0RTf/zg5jGGpaX7m07lqEG/FmxaRdHAAFjBxWBt47nIW3PDE3oWF/bQ
	2cFQRiH0ZARoSTbenjn9CuYr3MCPVPciAg1/xDCbqUs8lPOggyXzBeNjvUR4W61vf8Kcqc6+I
	AjrQHp6TSmJgFgzi1hRVidxP5NWBAJcvZqu0h3DJz1NOs6ku5yXHzktHzsIqrOq3u38sfgP6P
	wdZT3n0Fmx4qKhEvwnSXRKr9nMcHOF2cPDmnqTl9VwSZXSI5/oZZsjF+He2LTtePQ2kwv7tcw
	bOtgCXsN/pCx7ZO8UJe0uJqknzeQgQyHLE1eEEZj9LxtiI7rr0=
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
	[fuzzy]
X-Received-From: 212.227.17.12
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Tue, 18 Jul 2017 08:18:08 -0000

>> I imagine that there are more advanced possibilities to improve the software
>> run time characteristics for this use case.
> 
> Well, if it has to be fast, perhaps don't write the code in the shell language.

To which “shell” would you like to refer to?


> Even an interpreted scripting language that can do string handling without
> resorting to fork()-based command substitution will beat the shell at many tasks.

How do you think about additional approaches to reduce the forking of special processes?


>> I would like to avoid such an operation “Remove matching prefix pattern” generally.
>> If the desired file lists contain only basenames, extra prefixes do not need
>> to be deleted.
> 
> I.e. we can use the basename function:
> 
>   for name in dir/*txt; do
>     basename "$name"
>   done
> 
> prints the basenames of the matching files, one per line.

There is also the GNU variant available for such a command.

   for X in $(basename --suffix=.txt dir/*txt); do my_work $X; done


But how often can it be avoided to delete extra data like prefixes (and suffixes)?

Can it occasionally be a bit more efficient to provide only the essential
values at the beginning of an algorithm so that so they will be extended on demand?

Regards,
Markus


From MAILER-DAEMON Tue Jul 18 10:37:57 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dXTdN-0001gr-Bq
	for mharc-coreutils@gnu.org; Tue, 18 Jul 2017 10:37:57 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:35321)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <962-396-1872@kylheku.com>) id 1dXTdK-0001d1-EN
	for coreutils@gnu.org; Tue, 18 Jul 2017 10:37:55 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <962-396-1872@kylheku.com>) id 1dXTdH-000497-Q1
	for coreutils@gnu.org; Tue, 18 Jul 2017 10:37:54 -0400
Received: from smtp-out-no.shaw.ca ([64.59.134.13]:50369)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)
	(Exim 4.71) (envelope-from <962-396-1872@kylheku.com>)
	id 1dXTdH-000481-Ji
	for coreutils@gnu.org; Tue, 18 Jul 2017 10:37:51 -0400
Received: from kylheku.com ([70.79.163.252]) by shaw.ca with SMTP
	id XTdDdGRetM9gtXTdFd81j2; Tue, 18 Jul 2017 08:37:49 -0600
X-Authority-Analysis: v=2.2 cv=a+JAzQaF c=1 sm=1 tr=0
	a=95A0EdhkF1LMGt25d7h1IQ==:117 a=95A0EdhkF1LMGt25d7h1IQ==:17
	a=IkcTkHD0fZMA:10 a=SMorJkV_YP8A:10 a=G3gG6ho9WtcA:10
	a=09C2tZXrmmfWu1f1dvYA:9 a=QEXdDO2ut3YA:10
Received: from www-data by kylheku.com with local (Exim 4.72)
	(envelope-from <962-396-1872@kylheku.com>)
	id 1dXTdD-0000ou-NU; Tue, 18 Jul 2017 07:37:47 -0700
To: SF Markus Elfring <elfring@users.sourceforge.net>
Subject: Re: Determination of file lists from selected folders without
	returning directory names
X-PHP-Originating-Script: 501:rcmail.php
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8;
 format=flowed
Content-Transfer-Encoding: 8bit
Date: Tue, 18 Jul 2017 07:37:47 -0700
From: "Kaz Kylheku (Coreutils)" <962-396-1872@kylheku.com>
Cc: coreutils@gnu.org
In-Reply-To: <12c091e8-380d-b9c4-5d95-78d2d1f02041@users.sourceforge.net>
References: <af44fed2-a5a2-21a5-a3fa-6982a5d37e1d@users.sourceforge.net>
	<c97927cd182f34e3527fe741cce0b726@mail.kylheku.com>
	<7c44c5c2-e5cd-b368-a8bd-d7dc6dc3de3f@users.sourceforge.net>
	<b6745e3004da59da1e859847ed671d21@mail.kylheku.com>
	<12c091e8-380d-b9c4-5d95-78d2d1f02041@users.sourceforge.net>
Message-ID: <a7afb68cad095d83c90450700f168aa5@mail.kylheku.com>
X-Sender: 962-396-1872@kylheku.com
User-Agent: Roundcube Webmail/0.9.2
X-CMAE-Envelope: MS4wfAw0CtfYQjVgK4T2vrFlbuRNcXifsP3iSqRANYhtZIPC3lRdRhq0DIR2VDJdpuTNq+3XlO00ov9FDcmCsbf1JCHe5kJkG09Ldao6dVLL1eRgXSooGfoc
	Ids0P5Hdl1Qg+Yb4+nlWZGzekmS0G5E9nZUeylkw/2prdVttVctrBgz8par2aJzSTWeAv512NTihTVWzV5qN3Q0vMG8/+EGdDAA=
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy]
X-Received-From: 64.59.134.13
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Tue, 18 Jul 2017 14:37:56 -0000

On 18.07.2017 01:17, SF Markus Elfring wrote:
>>> I imagine that there are more advanced possibilities to improve the 
>>> software
>>> run time characteristics for this use case.
>> 
>> Well, if it has to be fast, perhaps don't write the code in the shell 
>> language.
> 
> To which “shell” would you like to refer to?

The "Shell Command Language", called by that name in POSIX, and to its
dialects.

>> Even an interpreted scripting language that can do string handling 
>> without
>> resorting to fork()-based command substitution will beat the shell at 
>> many tasks.
> 
> How do you think about additional approaches to reduce the forking of
> special processes?

I think: don't do text processing whose speed matters in a language 
where
you have to even think about the issue "how do I reduce fork() 
occurrences
in string processing code" and in which you don't even know whether
some command involves a fork or is a built-in primitive.

If you've resigned to developing something in the shell, and that 
something
has to process many items of data, try not to write a shell loop for the
task, and try to avoid idioms which run a process for each item.
Rather, coordinate commands which do the heavy lifting.

If I had to strip a large number of paths to their basenames, and it had
to be done in portable shell code, I would filter those names through 
sed:
one process invocations and some pipe inter-process I/O.

>> I.e. we can use the basename function:
>> 
>>   for name in dir/*txt; do
>>     basename "$name"
>>   done
>> 
>> prints the basenames of the matching files, one per line.
> 
> There is also the GNU variant available for such a command.
> 
>    for X in $(basename --suffix=.txt dir/*txt); do my_work $X; done

What you're doing here is destroying the validity of these expanded
paths; the "my_work" command or function cannot access things
through these paths, unless it restores the "dir/" prefix,
which it has not been given as an input.

When you expand dir/*txt, each one of the expansions is a correct
relative path to an object. The stripped basenames aren't.

Whatever "my_work" is doing, if it involves accessing the files,
you're probably making its job more difficult.

> But how often can it be avoided to delete extra data like prefixes
> (and suffixes)?

Pretty much all of the time.

> Can it occasionally be a bit more efficient to provide only the 
> essential
> values at the beginning of an algorithm so that so they will be
> extended on demand?

That sounds like a generic description of the whole body of "lazy"
or "late binding" techniques; but it's unclear how it is supposed
to apply here.

Maybe "my_work" could be given relative paths that resolve; if it needs
shortened names for some reason, let it compute them.

Or "my_work" could be given a quoted pattern:

    my_work '*.txt'

then it can expand it as needed, in whatever directory it wants.




From MAILER-DAEMON Tue Jul 18 11:38:09 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dXUZc-0005rU-Tz
	for mharc-coreutils@gnu.org; Tue, 18 Jul 2017 11:38:08 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:58013)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <elfring@users.sourceforge.net>) id 1dXUZZ-0005qx-SM
	for coreutils@gnu.org; Tue, 18 Jul 2017 11:38:06 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <elfring@users.sourceforge.net>) id 1dXUZV-0003rQ-18
	for coreutils@gnu.org; Tue, 18 Jul 2017 11:38:05 -0400
Received: from mout.web.de ([212.227.17.11]:61820)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16)
	(Exim 4.71) (envelope-from <elfring@users.sourceforge.net>)
	id 1dXUZU-0003nv-Lf
	for coreutils@gnu.org; Tue, 18 Jul 2017 11:38:00 -0400
Received: from [192.168.1.3] ([77.182.168.23]) by smtp.web.de (mrweb103
	[213.165.67.124]) with ESMTPSA (Nemesis) id 0MHp4r-1dUSh61G3R-003bSs;
	Tue, 18 Jul 2017 17:37:49 +0200
Subject: Re: Determination of file lists from selected folders without
	returning directory names
To: Kaz Kylheku <962-396-1872@kylheku.com>
Cc: coreutils@gnu.org
References: <af44fed2-a5a2-21a5-a3fa-6982a5d37e1d@users.sourceforge.net>
	<c97927cd182f34e3527fe741cce0b726@mail.kylheku.com>
	<7c44c5c2-e5cd-b368-a8bd-d7dc6dc3de3f@users.sourceforge.net>
	<b6745e3004da59da1e859847ed671d21@mail.kylheku.com>
	<12c091e8-380d-b9c4-5d95-78d2d1f02041@users.sourceforge.net>
	<a7afb68cad095d83c90450700f168aa5@mail.kylheku.com>
From: SF Markus Elfring <elfring@users.sourceforge.net>
Message-ID: <1cc2666c-2fbb-20ca-f427-e2caf4a5c661@users.sourceforge.net>
Date: Tue, 18 Jul 2017 17:37:46 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
	Thunderbird/52.2.1
MIME-Version: 1.0
In-Reply-To: <a7afb68cad095d83c90450700f168aa5@mail.kylheku.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-GB
Content-Transfer-Encoding: 8bit
X-Provags-ID: V03:K0:TCZjeBO88hN/RFXaqpQFS0SgdKSVEaPx2pxBd52gh2WlVMWgq68
	dyr6Dy3STbebYKFEKIiKwkUObv/FO7gG46t4deumvD/1znNB5dtzbPDNivX3Xc6FDPeiw/Z
	DJVLjXoYrez9otJQgrI0g+cRDrktpv6x2r3WN1VI0Fsu57/jjBbsvDWkUCzlWE0IbbcqCpS
	u+w1zEuOAPC1hFP2j9SxA==
X-UI-Out-Filterresults: notjunk:1;V01:K0:XngIeksVw7M=:0eeGaCpcWsIK5KQlVI32Ql
	4HACOZAfKFCsCiRrwXAtj2z89yNIq/fhbd7K+Susq0QFDs6sRyxWW59kXbzS1pe57Xu8FpIpZ
	BauQjWjE2qlPRRmUsWg+i7o+wLloD1ORrqvmVkGPMxfoXQ9LttPtN2NBusi42+wHj0RsC77Ur
	k6o+tlRMwLu8nhOZskjCGB70YLdVs49vLObv6KkdQYxtQ/i6CLBFqUcGcbIkKMoQO+IcgCvA6
	wfgxChnsShl8/rfxHa0OthVhgxKpAt8fMqcsRjdmBQP7SzSUdP6bWpiUpddoP08FOgHV8w8P1
	7AvXf5JBHoe8s2Kuh1SurPrczaWezw/X71viEF364mjqQK2bG9a/n/qQRk6r94ZmPLJCufvzB
	/5g+jzES6bGJWoMrgpHDQOnwYneCslIe8w2e6f/QgcNdpP6xlbcprO/860cVsjZS9wyIOU3vE
	JiWU1azyuf/mhq8L/LJnevHzzU1vf//vobDoxloVdle5w3Yg1i2sLv7XlY1eyEy96BvmOTczq
	X/YKqTgxuiQs8PuWAhtoemuHvUjMLMgyZCDw6+ZyLeub2tgGEt6YApo3riafY7AhwKLXiG742
	4WWLpgbP8wLp0iIy0/Eb6iORuEJ8sVqYjB2QKKO7U/UPfpxrtbZR9g0bppSb0hA/oKaHXJtqR
	NEsA/i3N8Et5WAdp2T2ygepGt1fO3CXKLZ3+kjr6hqqloa0JNxSG8K0exTlAlS9Bfa9aIAlRJ
	CRn/PH0JC8ewDarPBeexwiP4CiJgZ3B4h/qO8emHEtTq7ekiBIikjNuYCWxtHS1RrCAmcagFZ
	/r2TqEIlbPUjgaCktS2NGg/JcvOOuGxo67gd91mSYeLYefNLjg=
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
	[fuzzy]
X-Received-From: 212.227.17.11
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Tue, 18 Jul 2017 15:38:06 -0000

> When you expand dir/*txt, each one of the expansions is a correct
> relative path to an object.

I agree.


> The stripped basenames aren't.

They can be used for special data processing tasks.


> Whatever "my_work" is doing, if it involves accessing the files,
> you're probably making its job more difficult.

Such a view depends on the desired implementation details.

Use cases:
* The reduced names can belong to well-known directories.

* File name usage can be analysed across the directory hierarchy.


> Or "my_work" could be given a quoted pattern:
> 
>    my_work '*.txt'
> 
> then it can expand it as needed, in whatever directory it wants.

Are there other programs (besides the tool “find”) which support
such a data processing style?

Regards,
Markus


From MAILER-DAEMON Tue Jul 18 15:24:11 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dXY6N-0000PA-OR
	for mharc-coreutils@gnu.org; Tue, 18 Jul 2017 15:24:11 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:41186)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1dXY6L-0000Ox-Kv
	for coreutils@gnu.org; Tue, 18 Jul 2017 15:24:10 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1dXY6H-0006R1-Lo
	for coreutils@gnu.org; Tue, 18 Jul 2017 15:24:09 -0400
Received: from mout.kundenserver.de ([212.227.126.135]:56392)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16)
	(Exim 4.71) (envelope-from <mail@bernhard-voelker.de>)
	id 1dXY6H-0006Pe-BX
	for coreutils@gnu.org; Tue, 18 Jul 2017 15:24:05 -0400
Received: from [192.168.101.10] ([91.12.166.234]) by mrelayeu.kundenserver.de
	(mreue002 [212.227.15.167]) with ESMTPSA (Nemesis) id
	0MVVCo-1d7d6L0sdq-00YiPA; Tue, 18 Jul 2017 21:23:47 +0200
Subject: Re: Determination of file lists from selected folders without
	returning directory names
To: SF Markus Elfring <elfring@users.sourceforge.net>,
	Kaz Kylheku <962-396-1872@kylheku.com>
References: <af44fed2-a5a2-21a5-a3fa-6982a5d37e1d@users.sourceforge.net>
	<c97927cd182f34e3527fe741cce0b726@mail.kylheku.com>
	<7c44c5c2-e5cd-b368-a8bd-d7dc6dc3de3f@users.sourceforge.net>
	<b6745e3004da59da1e859847ed671d21@mail.kylheku.com>
	<12c091e8-380d-b9c4-5d95-78d2d1f02041@users.sourceforge.net>
	<a7afb68cad095d83c90450700f168aa5@mail.kylheku.com>
	<1cc2666c-2fbb-20ca-f427-e2caf4a5c661@users.sourceforge.net>
Cc: coreutils@gnu.org
From: Bernhard Voelker <mail@bernhard-voelker.de>
Message-ID: <5442056c-efe3-4198-9f11-b2e0fd63e90c@bernhard-voelker.de>
Date: Tue, 18 Jul 2017 21:23:45 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101
	Thunderbird/45.6.0
MIME-Version: 1.0
In-Reply-To: <1cc2666c-2fbb-20ca-f427-e2caf4a5c661@users.sourceforge.net>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
X-Provags-ID: V03:K0:M5I4dNzfxsVDPU0B9qGFdlbqGWDoe9AlggMdoNr50IJzWKc5Gga
	++pRDlgnXq9hz023T7G2ZEXolJtpRbnTowBqxjVT7HaDDL20vJ3sDdpgxgporEAazDBB7sg
	GFVLQRhWyimW3SoGZaV/YOtrfLrMZrEr+4C5agXQ6lbPYYbrUCPQKh3XE85hypbF4yPpx2O
	s19eVUFXjRT+l80bs9cFQ==
X-UI-Out-Filterresults: notjunk:1;V01:K0:2pn2wiwb/pM=:1p8IkhzwHcQ+zdRK1RPPXV
	CbTpys6Hf+ydK2TcHvXS9+4bgmGKcc4pZFPxENPJHBXOoYvlRcU2rGrOLVTWe8RKG3aiPtNYa
	PWWAcXHF1I47LXqp/s+REp6lm57Yg7TuURgC9Mb13jZZfm3gMmVQyWUjdp/CGMIVa2A7NQJE5
	76KRbdXRuig83QL2XXEbdKPzP/6bWr5PYbQgEmJghgY2BMpHwrz5i+R9HeLz73v8Vh/yIbcBm
	qc7L/w9ODNmWwroJ43VpsSMN4ACPzkfvNVaaDnFIE0hBFXO4j5HVUlxUTMfrk453DQWs1gp32
	YWIh2QPzoqtW3UR9YsoUkETKNjGxXfr7Y9DZP+Cjt0YQRIW8M2b43Rkxe4YylaqliXPwyrkft
	qRKHC3lBhauy3nL1JhSNhpppNtfelDsazXG+lOwjAWt50KuLZwrBQR+teKj3a+ks0aoU2xNRO
	T0XNIvB9SFUVrppSBKgvjkpZf833z+NYjnX7SzbKBkjMNmWzLXm4XLMbmG+ff9JyWPK9NBh2U
	WgtJqzoczQ/vBK1ccdOiac4wqN3y4JnbbtBmjs2yX7jgxwuU0K7VT3kd6V/MS3uf3eBoJlT3v
	stbPDGQy98OU9fywjQYCSXgchKbYPFs+0pw4jc8xcQLiZUPJ3IlAkm8kAua7OEMze0cXDQ1ak
	h9On2WzeLdPq/K8Z9Elk+KNma9nFibDZFU57DiNKtlrVuHjY6hwU1qvqUdU+ZpeRR5QJKfy3S
	4aVbtkMd0VETgCBPEw8978HU1sHGa9k8bXvXTGzWoXeACTNuXWf6b22xS94=
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
	[fuzzy]
X-Received-From: 212.227.126.135
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Tue, 18 Jul 2017 19:24:10 -0000

On 07/18/2017 05:37 PM, SF Markus Elfring wrote:
> Are there other programs (besides the tool “find”) which support
> such a data processing style?

Sorry you lost me.  What are we talking about?

This discussion mentions constructs with globbing *.txt and passing
their basenames as parameter to another script ... while worrying
about the performance of stripping off the directories part of each
item.

When globbing in such shell, you easily hit the maximum command line
length of the shell, here:

  $ xargs -r --show-limits </dev/null 2>&1 | grep '^Maximum .* actually use:'
  Maximum length of command we could actually use: 2089058

Furthermore, many file systems do not scale very
well with several hundred thousand of files inside a directory.

So how many files do you have: thousands, millions?
And if the files are furtherly processed by an external script, why should
a simple `basename` or ${name##*/} hurt the performance?

AFAICS your best bet is still based on find(1).

Have a nice day,
Berny


From MAILER-DAEMON Tue Jul 18 15:54:03 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dXYZH-0002Qs-EI
	for mharc-coreutils@gnu.org; Tue, 18 Jul 2017 15:54:03 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:47301)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <elfring@users.sourceforge.net>) id 1dXYZE-0002QL-FB
	for coreutils@gnu.org; Tue, 18 Jul 2017 15:54:01 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <elfring@users.sourceforge.net>) id 1dXYZB-0007wJ-Cf
	for coreutils@gnu.org; Tue, 18 Jul 2017 15:54:00 -0400
Received: from mout.web.de ([212.227.17.11]:62855)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16)
	(Exim 4.71) (envelope-from <elfring@users.sourceforge.net>)
	id 1dXYZB-0007oL-1G
	for coreutils@gnu.org; Tue, 18 Jul 2017 15:53:57 -0400
Received: from [192.168.1.3] ([77.182.168.23]) by smtp.web.de (mrweb101
	[213.165.67.124]) with ESMTPSA (Nemesis) id 0MIeQO-1dVMho0Fmi-002FAa;
	Tue, 18 Jul 2017 21:53:40 +0200
Subject: Re: Determination of file lists from selected folders without
	returning directory names
To: =?UTF-8?Q?Bernhard_V=c3=b6lker?= <mail@bernhard-voelker.de>
References: <af44fed2-a5a2-21a5-a3fa-6982a5d37e1d@users.sourceforge.net>
	<c97927cd182f34e3527fe741cce0b726@mail.kylheku.com>
	<7c44c5c2-e5cd-b368-a8bd-d7dc6dc3de3f@users.sourceforge.net>
	<b6745e3004da59da1e859847ed671d21@mail.kylheku.com>
	<12c091e8-380d-b9c4-5d95-78d2d1f02041@users.sourceforge.net>
	<a7afb68cad095d83c90450700f168aa5@mail.kylheku.com>
	<1cc2666c-2fbb-20ca-f427-e2caf4a5c661@users.sourceforge.net>
	<5442056c-efe3-4198-9f11-b2e0fd63e90c@bernhard-voelker.de>
From: SF Markus Elfring <elfring@users.sourceforge.net>
Cc: Kaz Kylheku <962-396-1872@kylheku.com>, coreutils@gnu.org
Message-ID: <fc6b1b8c-bb26-314d-5160-d5c4c4e7b36f@users.sourceforge.net>
Date: Tue, 18 Jul 2017 21:53:36 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
	Thunderbird/52.2.1
MIME-Version: 1.0
In-Reply-To: <5442056c-efe3-4198-9f11-b2e0fd63e90c@bernhard-voelker.de>
Content-Type: text/plain; charset=utf-8
Content-Language: en-GB
Content-Transfer-Encoding: 8bit
X-Provags-ID: V03:K0:dEJYDCzAZIa6DflLFj50qHenvL89QkUd5ifnFgqrivFeEado0HG
	C1aAptpnkX8CEFberMnAwWmSJg50/CTIttcyKfr+iHcuFpruBy+PG+/mE1gghMNbaJAZ/Lr
	lX4iHgxcP2XQmcIXv3sayRTTerC65+hjL0Xl1mWjNnZ0HjcCkU+h4wGFGNWRtNwcnkjLpXc
	XXW+cMsbKWMBsQoqM1qfg==
X-UI-Out-Filterresults: notjunk:1;V01:K0:tknC8nfAV6A=:DLThMCbxvtMZ6KRnnaP+Hh
	39Pef0td4WTNGKmsepEssMne8HCF8r32Vudq5e1mqEODUZRh7RKj9YKQWFjPojlaDnosDp70I
	SpPU0qCxAJdpe+dRejAAqifD+GgY2Jj/mYsYzd/WZPQ9VOP36tODaXsCtjsQptWSS8yUaLLJG
	93kMThy+16doA7hq+2oZSFa0YJYLN1pcRQf/mKPbvmeRga9vmTiemGR9yO14cE8JS2Ub2THjI
	dsZ6pTuG26WTmE5J6kKgKmMWuSMXgq3NVNS6XkywI2UvGf/+EqwzVtRobSRicl9cvl658kpPX
	zOd5IC6QB1ppvI1sQA9+H5druFsx+5+YwgIUws73Bb3j9Q4UZljRz6twT9tgVj9KZubvxKQwe
	ovmjG+aOI5cXa+H7dpf+neR/aXeck+GOUYErMzdPT5DAc6d80Pq3hcFvCmNpW0mmtC70jj77/
	j0M+7jznHGjhC/of4evzFnmQzyf253EFuE/Z1fGiY6cgmNaP6h0pnZNXkY+aVl6noO4z1wVnj
	h8O+g6MfNtBhTM0tjH55+GGmSK/gEDTZa9vqgeNZ6idwRgPlQyp28sCAkvnBBrXUzK7utLCBM
	yb+T4juKPlOWwItxKj/m8hLymzbZ2scf9VKZE/IF4auG1ofHRlr1L4PrDfOKq03gvVGR6KgsM
	xwQUA2tDoot4rIs1TZGfFCsgiZTFRWUeDm8SpHHzCCX0avCHzewdujFct3Ne0OfS9HF5tWA7a
	a6ZrgKwECMtwPmqDenQlkRDdg3Et7zSsHGNzCaNXwvxTqi/I8u4Ja3gAgzsD4qnlZo0S7rMzT
	Q7v6XTmtnCqU1WUYEg3LRJFVLHTHt5EYU4p3duiwMU4MpkwVu0=
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
	[fuzzy]
X-Received-From: 212.227.17.11
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Tue, 18 Jul 2017 19:54:01 -0000

>> Are there other programs (besides the tool “find”) which support
>> such a data processing style?
> 
> Sorry you lost me.  What are we talking about?

It is generally possible to achieve something with a command like
the following.

   :> find Test_dir -name '*.txt' -printf %f\\n


> This discussion mentions constructs with globbing *.txt and passing
> their basenames as parameter to another script ...

Yes …


> while worrying about the performance of stripping off the directories part
> of each item.

There are corresponding software development concerns.


> When globbing in such shell, you easily hit the maximum command line
> length of the shell, …

This limitation can occasionally trigger further development challenges.


> Furthermore, many file systems do not scale very
> well with several hundred thousand of files inside a directory.

Scalability issues can also become interesting.


> And if the files are furtherly processed by an external script,

I imagine that there are a few software improvement opportunities
to consider.


> why should a simple `basename` or ${name##*/} hurt the performance?

Such data deletion causes also corresponding processing costs, doesn't it?
Would it be nicer to avoid them if the initial input data could be provided
as “basenames” already?

Regards,
Markus


From MAILER-DAEMON Tue Jul 18 16:50:10 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dXZRa-00037S-Tr
	for mharc-coreutils@gnu.org; Tue, 18 Jul 2017 16:50:10 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:36228)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1dXZRZ-00036I-1X
	for coreutils@gnu.org; Tue, 18 Jul 2017 16:50:10 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1dXZRV-0007Na-Vf
	for coreutils@gnu.org; Tue, 18 Jul 2017 16:50:09 -0400
Received: from mout.kundenserver.de ([217.72.192.75]:55163)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16)
	(Exim 4.71) (envelope-from <mail@bernhard-voelker.de>)
	id 1dXZRV-0007Jq-KJ
	for coreutils@gnu.org; Tue, 18 Jul 2017 16:50:05 -0400
Received: from [192.168.101.10] ([91.12.166.234]) by mrelayeu.kundenserver.de
	(mreue102 [212.227.15.183]) with ESMTPSA (Nemesis) id
	0MgOZS-1dIdK61n6c-00Njve; Tue, 18 Jul 2017 22:49:48 +0200
Subject: Re: Determination of file lists from selected folders without
	returning directory names
To: SF Markus Elfring <elfring@users.sourceforge.net>
References: <af44fed2-a5a2-21a5-a3fa-6982a5d37e1d@users.sourceforge.net>
	<c97927cd182f34e3527fe741cce0b726@mail.kylheku.com>
	<7c44c5c2-e5cd-b368-a8bd-d7dc6dc3de3f@users.sourceforge.net>
	<b6745e3004da59da1e859847ed671d21@mail.kylheku.com>
	<12c091e8-380d-b9c4-5d95-78d2d1f02041@users.sourceforge.net>
	<a7afb68cad095d83c90450700f168aa5@mail.kylheku.com>
	<1cc2666c-2fbb-20ca-f427-e2caf4a5c661@users.sourceforge.net>
	<5442056c-efe3-4198-9f11-b2e0fd63e90c@bernhard-voelker.de>
	<fc6b1b8c-bb26-314d-5160-d5c4c4e7b36f@users.sourceforge.net>
Cc: Kaz Kylheku <962-396-1872@kylheku.com>, coreutils@gnu.org
From: Bernhard Voelker <mail@bernhard-voelker.de>
Message-ID: <28d7c18d-8291-c6a7-d138-958ae6358b50@bernhard-voelker.de>
Date: Tue, 18 Jul 2017 22:49:47 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101
	Thunderbird/45.6.0
MIME-Version: 1.0
In-Reply-To: <fc6b1b8c-bb26-314d-5160-d5c4c4e7b36f@users.sourceforge.net>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
X-Provags-ID: V03:K0:z1R8yvkBR++9R53R+ipbq49UIewnqNBsTme54feMZSmJ8jb1qE+
	T3WgKUFiGiY1mh1QC41PwRcW6YCoxktv498Zb4ohu6NyAnKIVg3MTIBSDQIHaKKuHyLZx7x
	53yTwrg3Vc91BZ/hPimZpWL9kXFtvZe1oBPZSzfY1F4zLMXmGDJ+zN9qKUjRV8+FEQ+7BJ+
	itVoUebKMXvraegYn8hVQ==
X-UI-Out-Filterresults: notjunk:1;V01:K0:RUCVh5nDMlo=:7wVCJj1aMfCsVAT3V1F/p3
	pdkam843m/KM6jyumnYIse3oTFihr8D826esuqEdVPz+KteRdWKC7DnSaY9zKbd5Q8CO7YRNY
	8+zbX4GW/RUR3xc2fId3BtHn9i2eUJ2WR4JHgQ/+REFHNcPSp4um/lemxokTtdyouyggLlfba
	6PqK5O7V5QPnPSCVFes/zDT9Vx+09M8r4995RT6dVFGnmyXdt/cDAn1rQcUdhq6buvXGjK+pj
	/o10mG0BkoHabReqBr2zxODnIWrrGRjt7N1WfVSZt8VzcOZNAzJDD3CnI4gpZQP5GWWrRoVP4
	vNKJUBdMiINodAvmHMR4OnpfoO7z9T/m3wk1FyHR+AYJn6yLB0TZNqYJUDR2Ndpq32y9gdLey
	bi81vTa1EEe/PgWiAnBdgmMr5WvnnQzUgPo1gr7uwZbUNlFI1Vzs1ZgnpLSBFosOlwsDj8kLg
	zEA9ZzuAREVoeeiODVwfX0gUq745VhkJeMJKVYvYCIYkqIzk4YYNAuiB9dguSG8wVUsWX96+z
	c/BYVajXcaUb0nuIxBgvASOR7rl+i4AZBxOSt2e5xCVyNrUxEDvn7fjqbLm+ncL1vgiSMa+up
	2kdHHPshdDFeeUa3h1sdBPotlbiwIeq2yVn/TZc13BtwaisApZEMhqif7gGkSaKWJFAa3zzG7
	DuNpDc3KMVQmq6V9meOw71d6JMsNLXgBipVL7/qyaV8Ff9/xr+YK5Jj9BCnqmJ/EcHuf2JokJ
	jSYVfHxi1Hc3tWrWTY1M+gnpULuksn4srNGJYxVvVp1bqVgCIAZUnib9y70=
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
	[fuzzy]
X-Received-From: 217.72.192.75
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Tue, 18 Jul 2017 20:50:10 -0000

On 07/18/2017 09:53 PM, SF Markus Elfring wrote:
>> why should a simple `basename` or ${name##*/} hurt the performance?
> 
> Such data deletion causes also corresponding processing costs, doesn't it?
> Would it be nicer to avoid them if the initial input data could be provided
> as “basenames” already?

Sorry, it seems I missed the most important question:
What is your exact use case?
We need numbers and commands to reproduce it.

Otherwise, this is all just guesswork, and we could spend years to discuss
what could maybe be considered, etc.  You already got the ideal answer for
probably 99.9% of the cases - "use find(1)" - so unless you describe your
outstanding edge case in more detail, I'm afraid there's nothing more we
could help you with.

Have a nice day,
Berny


From MAILER-DAEMON Tue Jul 18 20:37:59 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dXd02-0004uy-Ts
	for mharc-coreutils@gnu.org; Tue, 18 Jul 2017 20:37:58 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:35428)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <sloanlance@gmail.com>) id 1dXbEj-00019w-Mg
	for coreutils@gnu.org; Tue, 18 Jul 2017 18:45:05 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <sloanlance@gmail.com>) id 1dXbEi-0002ym-UR
	for coreutils@gnu.org; Tue, 18 Jul 2017 18:45:01 -0400
Received: from mail-it0-x235.google.com ([2607:f8b0:4001:c0b::235]:34739)
	by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)
	(Exim 4.71) (envelope-from <sloanlance@gmail.com>)
	id 1dXbEi-0002y2-PY
	for coreutils@gnu.org; Tue, 18 Jul 2017 18:45:00 -0400
Received: by mail-it0-x235.google.com with SMTP id v205so3104188itf.1
	for <coreutils@gnu.org>; Tue, 18 Jul 2017 15:44:58 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
	h=mime-version:sender:from:date:message-id:subject:to;
	bh=KTqCSagx1GEwB1mxSQv578z6OIGaF0rFdcXCv5SIrM8=;
	b=oslodezvYaf6C2tU0IIEw150CaRtKG1WW6zQyp0E9pCAIoq49eVSwkAg6UJk2CQSOM
	E/IPybW6QkUv3TbLFdWCWV1nYGfDJaJP6N6m+8EKTQuG5isvACs4nsE7AX3SetwPUYqy
	17JUhXuucYq+KTUae4WZF7TcsgHj9EzjJ52+q6MBMOyn7jDJSZcs43Z3VQyCIz+K7I8q
	/wsQKMqNUb6ZwvADR3fMhJa91UmKBnV+xxQ7AoWClUk5PSVspaaUa1OMbCW11JdATIi/
	x8zLBgK70VXmWEl21v4tovmnJ+a0hKHhTOhVvt+U7Gggwy4XkuR1rkJLYEk+zCeeoh/Y
	gWbw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
	d=1e100.net; s=20161025;
	h=x-gm-message-state:mime-version:sender:from:date:message-id:subject
	:to; bh=KTqCSagx1GEwB1mxSQv578z6OIGaF0rFdcXCv5SIrM8=;
	b=cowCXxjd8hZm81pko1ZbTZZRt2bX+xto5hlhP6xfDGguYt+MKmbPHjgSnohPq0uq5X
	QR8PiOh/z8YuJxomB7b9Jby1lLvOQIaXHGEXvzgG++seA522KjA+ImRgCYSQgGGe6hrx
	r5bymdXoe88l4vW7/IH6Cex6a5wk/VYTDQmYkM2yudqY+n88tSAMvfPCLsvNIUR+kt1u
	PT1zQhBptohlB6JIgE356yv3vRMJSspmiy7Ol2RMrjIrgUmL8N3xgYmPSv7RNrv0GMR8
	GKaQ1+cqkHy+MHzA1uXuu7ttyP2LkX95RjGgPt32Wom9KeIBvt26bLJUGH62GviR6AEJ
	vITA==
X-Gm-Message-State: AIVw110J3BPe+N3fyeU3PDS5kiYRhwZr7aDOtWJgvlEWbYV+PoDnrlXi
	GvYRdCva/b3U49aFHDW0xfg3qASpAg==
X-Received: by 10.36.95.65 with SMTP id r62mr4707885itb.40.1500417897566; Tue,
	18 Jul 2017 15:44:57 -0700 (PDT)
MIME-Version: 1.0
Sender: sloanlance@gmail.com
Received: by 10.107.35.201 with HTTP; Tue, 18 Jul 2017 15:44:16 -0700 (PDT)
From: Lance E Sloan <sloanlance+coreutils_gnu.org@gmail.com>
Date: Tue, 18 Jul 2017 18:44:16 -0400
X-Google-Sender-Auth: Of552mOt9-k0w-PVnya7yH8uj_c
Message-ID: <CAJecg5pGpQU_CZ-AdMtA88fzGs7RPd+dHG__AeYTvergm=iKFA@mail.gmail.com>
Subject: coreutils feature requests?
To: coreutils@gnu.org
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 2607:f8b0:4001:c0b::235
X-Mailman-Approved-At: Tue, 18 Jul 2017 20:37:57 -0400
Content-Type: text/plain; charset="UTF-8"
X-Content-Filtered-By: Mailman/MimeDel 2.1.21
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Tue, 18 Jul 2017 22:45:05 -0000

Hi, all.

Aside from a bug report, is there a way to submit a feature request for
coreutils?  I have a couple of requests in mind, which I don't think
qualify as bugs:

1. Add a feature to "cut" that allows selected fields to be output in a
different order.  That is "cut -f4,1,8-6" would cause it to output fields
in the order of 4, 1, 8, 7, and 6.

2. Add a feature to "comm" to output trailing column delimiters even if a
line has only one or two columns.

If I should file these as bug reports, tell me and I'll do so.

Also, is there a list of open issues, feature requests, and bug reports?

Thanks in advance.

From MAILER-DAEMON Tue Jul 18 21:07:05 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dXdSD-0003rE-Be
	for mharc-coreutils@gnu.org; Tue, 18 Jul 2017 21:07:05 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:58989)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <962-396-1872@kylheku.com>) id 1dXdSB-0003q2-5Q
	for coreutils@gnu.org; Tue, 18 Jul 2017 21:07:04 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <962-396-1872@kylheku.com>) id 1dXdS6-0000CF-5w
	for coreutils@gnu.org; Tue, 18 Jul 2017 21:07:03 -0400
Received: from smtp-out-no.shaw.ca ([64.59.134.12]:58597)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)
	(Exim 4.71) (envelope-from <962-396-1872@kylheku.com>)
	id 1dXdS6-00009Z-0A
	for coreutils@gnu.org; Tue, 18 Jul 2017 21:06:58 -0400
Received: from kylheku.com ([70.79.163.252]) by shaw.ca with SMTP
	id XdRsdtemyI8mCXdRtdVlfK; Tue, 18 Jul 2017 19:06:46 -0600
X-Authority-Analysis: v=2.2 cv=HahkdmM8 c=1 sm=1 tr=0
	a=95A0EdhkF1LMGt25d7h1IQ==:117 a=95A0EdhkF1LMGt25d7h1IQ==:17
	a=IkcTkHD0fZMA:10 a=SMorJkV_YP8A:10 a=G3gG6ho9WtcA:10
	a=QYL17zd3Ilrz0viMC68A:9 a=7Zwj6sZBwVKJAoWSPKxL6X1jA+E=:19
	a=QEXdDO2ut3YA:10
Received: from www-data by kylheku.com with local (Exim 4.72)
	(envelope-from <962-396-1872@kylheku.com>)
	id 1dXdRr-000696-TS; Tue, 18 Jul 2017 18:06:43 -0700
To: Lance E Sloan <sloanlance+coreutils_gnu.org@gmail.com>
Subject: Re: coreutils feature =?UTF-8?Q?requests=3F?=
X-PHP-Originating-Script: 501:rcmail.php
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8;
 format=flowed
Content-Transfer-Encoding: quoted-printable
Date: Tue, 18 Jul 2017 18:06:43 -0700
From: "Kaz Kylheku (Coreutils)" <962-396-1872@kylheku.com>
Cc: coreutils@gnu.org
In-Reply-To: <CAJecg5pGpQU_CZ-AdMtA88fzGs7RPd+dHG__AeYTvergm=iKFA@mail.gmail.com>
References: <CAJecg5pGpQU_CZ-AdMtA88fzGs7RPd+dHG__AeYTvergm=iKFA@mail.gmail.com>
Message-ID: <ed6ef41b902c4a37f5c78e8f791d23f6@mail.kylheku.com>
X-Sender: 962-396-1872@kylheku.com
User-Agent: Roundcube Webmail/0.9.2
X-CMAE-Envelope: MS4wfMof5oI2nN1dLX2xlX13gYYD6k4g6zUQTNQcMVvLa/m7SQesElRD73Twp4zZx2bDG6opQTFYOUISN+iH1QAJu+Pbx0avEfg/4J8QJGPVnL+0z3RNJ+Ij
	Q0z4JQFi8cXGhRvKoFx13a1fuoAK11ck5jMEIDm02Vm9Fx1RaJfU5iu06SK9kHAieEMP2fIOZmUUj8snQWtJ0KdGwyEiDZuSbMulado7cKMaVMtVKtzFQ7kh
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy]
X-Received-From: 64.59.134.12
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Wed, 19 Jul 2017 01:07:04 -0000

On 18.07.2017 15:44, Lance E Sloan wrote:
> Hi, all.
>=20
> Aside from a bug report, is there a way to submit a feature request for
> coreutils?  I have a couple of requests in mind, which I don't think
> qualify as bugs:
>=20
> 1. Add a feature to "cut" that allows selected fields to be output in a
> different order.  That is "cut -f4,1,8-6" would cause it to output=20
> fields
> in the order of 4, 1, 8, 7, and 6.

I'm amazed that it doesn't work this way; the utility of implicitly
sorting the fields appears low compared to the damage that it does
to the flexibility of cut. (What little it has!)

If POSIX specifies it, I have to say that its requirements are
suboptimal (as is often the case in diverse areas).

Indeed, the requirement is sadly given as:

    The elements in list can be repeated, can overlap, and can be=20
specified
    in any order, but the bytes, characters, or fields selected shall be
    written in the order of the input data. If an element appears in the
    selection list more than once, it shall be written exactly once.

Do people often write cut specifications in ad hoc orders, with
repetitions and then rely on the sorting behavior?

The GNU implementation of cut could lead standardization in this
area to improve things.

All the possible solutions to make cut not sort the fields or
de-duplicate them are ugly. Either you  need a global option to
opt out of that behavior, like "-o" (preserve (o)rder) and
keep remembering to use it, or else provide order-preserving
versions of the various options, like perhaps through capitalization:
-F, -C, -B.

An attractive alternative is to have a whole new command which
mirrors cut, like say "clip", which is exactly like cut, but
order and repetition preserving.

Note that the --complement option is semantically incompatible with
order-preserving mode; the complement concept follows from the
selected elements being regarded as a set rather than ordered
sequence. If -F, -C or -B is used with --complement, it has to be
diagnosed.  Or if there is a "clip" command, then that simply
doesn't support --complement.


From MAILER-DAEMON Tue Jul 18 23:25:02 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dXfbi-0000b3-EG
	for mharc-coreutils@gnu.org; Tue, 18 Jul 2017 23:25:02 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:53421)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <sloanlance@gmail.com>) id 1dXfbf-0000an-OV
	for coreutils@gnu.org; Tue, 18 Jul 2017 23:25:01 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <sloanlance@gmail.com>) id 1dXfbe-0006vm-4i
	for coreutils@gnu.org; Tue, 18 Jul 2017 23:24:59 -0400
Received: from mail-it0-x229.google.com ([2607:f8b0:4001:c0b::229]:38477)
	by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)
	(Exim 4.71) (envelope-from <sloanlance@gmail.com>)
	id 1dXfbd-0006vV-Um
	for coreutils@gnu.org; Tue, 18 Jul 2017 23:24:58 -0400
Received: by mail-it0-x229.google.com with SMTP id h199so23210367ith.1
	for <coreutils@gnu.org>; Tue, 18 Jul 2017 20:24:57 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
	h=mime-version:sender:in-reply-to:references:from:date:message-id
	:subject:to:cc;
	bh=q2u+JJcXBlU90QN+D4mbzET8/a/xB/vIqzDPsvixigQ=;
	b=cZeL6BZxGcF4rblSwOfY6xyx/PWUKv0btvtI2q3YtmHu8R9I9k0pXqG8/bKI3jR4qR
	o9hUUR9w6AV0e5JnypEM9AjpoxowGFwo6fOzF/k7UjR7p35F+LfvRBse9eGyqSMTtnMS
	G7u3wrJbr5za0268LFSG4doKgq2FRSlFkcqKWmWj1GWhtWEBtSRZBlN04vctylLuwjY3
	OjDWqpyOhhmtfAbPIeJVOV0ib2VSQBQf2pTEZq0vzwdz6vomb/Of3j85JcggF8MnOErP
	8q/D9uitLJtHQ44U4RJFJnfzV0EOJWTK+Gq3aFEqj3ysJt5XjAVx9fp36/OzDz418Pes
	A15A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
	d=1e100.net; s=20161025;
	h=x-gm-message-state:mime-version:sender:in-reply-to:references:from
	:date:message-id:subject:to:cc;
	bh=q2u+JJcXBlU90QN+D4mbzET8/a/xB/vIqzDPsvixigQ=;
	b=carR8eQVh8MaPeSmVyiNZnuS+tztSJYTNl5hSWGOy8t9v/P4CuIp7lso3TTZrXgZIk
	yaR76UL7foP8AIGVV/AhAA/z/JxwCRvVDYz+NtWnm3s1QwizzEcbiGs1lBeNUl48BjmP
	pCyveKkB7/plShMAHwSYbmjDvUtaHSHjDivUNh0AQgBDgQeeMBqjZrDx9t+fakC7Ytv4
	5IGmlbsJImNNOHH1oRPvsunczwSTn9htXtO92jr7cZqKRQYWq+4SvlMoMaWBGKT3yU7V
	f3mL2cSs7f2Fh18xPTaiPMS7bh8ZN+tnVAbLJXA57uGPXcus9mTtNmQ8SCrjf+HH4Fvb
	UAdg==
X-Gm-Message-State: AIVw110EjIUYZED37uwWvfCsMCdzvqWOFaHp48VzqBhZSEHsMaZ2XO67
	G0OucBbi+YEM3tZ2sOrl+VOZILrKMQ==
X-Received: by 10.36.193.130 with SMTP id e124mr363735itg.115.1500434696927;
	Tue, 18 Jul 2017 20:24:56 -0700 (PDT)
MIME-Version: 1.0
Sender: sloanlance@gmail.com
Received: by 10.107.35.201 with HTTP; Tue, 18 Jul 2017 20:24:16 -0700 (PDT)
In-Reply-To: <ed6ef41b902c4a37f5c78e8f791d23f6@mail.kylheku.com>
References: <CAJecg5pGpQU_CZ-AdMtA88fzGs7RPd+dHG__AeYTvergm=iKFA@mail.gmail.com>
	<ed6ef41b902c4a37f5c78e8f791d23f6@mail.kylheku.com>
From: Lance E Sloan <sloanlance+coreutils_gnu.org@gmail.com>
Date: Tue, 18 Jul 2017 23:24:16 -0400
X-Google-Sender-Auth: DF0zA1Ua8a_lkOQPNNLKS0WVyEA
Message-ID: <CAJecg5r6U4X-O2J5x_HSExSEy19Skksyw7jrEdwASabjaind_w@mail.gmail.com>
Subject: Re: coreutils feature requests?
To: "Kaz Kylheku (Coreutils)" <962-396-1872@kylheku.com>
Cc: coreutils@gnu.org
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 2607:f8b0:4001:c0b::229
Content-Type: text/plain; charset="UTF-8"
X-Content-Filtered-By: Mailman/MimeDel 2.1.21
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Wed, 19 Jul 2017 03:25:01 -0000

Hi, Kaz.

Thanks for the response.  I'm happy that you've come to the same conclusion
that I'd reached. I only have a moment for a quick reply now, but my
inclination was to add a -o / --ordered option to make -f (and other
options) work the way I'd expected.

I'd be happy to implement the feature myself and contribute it back to the
community. I'd just like to know what are the procedures. Should I wait for
the feature request to be approved by someone?  I didn't want to spend time
implementing it only to see out rejected soon afterwards.

I've seen the hacking readme and I understand what to do technically.
However, I couldn't find any documentation about what happens
procedurally.  If you or others can share some insight, I'd appreciate it.

Thanks in advance.

On Tue, Jul 18, 2017 at 21:06 Kaz Kylheku (Coreutils) <
962-396-1872@kylheku.com> wrote:

> On 18.07.2017 15:44, Lance E Sloan wrote:
> > Hi, all.
> >
> > Aside from a bug report, is there a way to submit a feature request for
> > coreutils?  I have a couple of requests in mind, which I don't think
> > qualify as bugs:
> >
> > 1. Add a feature to "cut" that allows selected fields to be output in a
> > different order.  That is "cut -f4,1,8-6" would cause it to output
> > fields
> > in the order of 4, 1, 8, 7, and 6.
>
> I'm amazed that it doesn't work this way; the utility of implicitly
> sorting the fields appears low compared to the damage that it does
> to the flexibility of cut. (What little it has!)
>
> If POSIX specifies it, I have to say that its requirements are
> suboptimal (as is often the case in diverse areas).
>
> Indeed, the requirement is sadly given as:
>
>     The elements in list can be repeated, can overlap, and can be
> specified
>     in any order, but the bytes, characters, or fields selected shall be
>     written in the order of the input data. If an element appears in the
>     selection list more than once, it shall be written exactly once.
>
> Do people often write cut specifications in ad hoc orders, with
> repetitions and then rely on the sorting behavior?
>
> The GNU implementation of cut could lead standardization in this
> area to improve things.
>
> All the possible solutions to make cut not sort the fields or
> de-duplicate them are ugly. Either you  need a global option to
> opt out of that behavior, like "-o" (preserve (o)rder) and
> keep remembering to use it, or else provide order-preserving
> versions of the various options, like perhaps through capitalization:
> -F, -C, -B.
>
> An attractive alternative is to have a whole new command which
> mirrors cut, like say "clip", which is exactly like cut, but
> order and repetition preserving.
>
> Note that the --complement option is semantically incompatible with
> order-preserving mode; the complement concept follows from the
> selected elements being regarded as a set rather than ordered
> sequence. If -F, -C or -B is used with --complement, it has to be
> diagnosed.  Or if there is a "clip" command, then that simply
> doesn't support --complement.
>

From MAILER-DAEMON Wed Jul 19 04:44:10 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dXkaY-0006J3-DV
	for mharc-coreutils@gnu.org; Wed, 19 Jul 2017 04:44:10 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:59877)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <elfring@users.sourceforge.net>) id 1dXkaV-0006FY-Ba
	for coreutils@gnu.org; Wed, 19 Jul 2017 04:44:08 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <elfring@users.sourceforge.net>) id 1dXkaS-0006An-1M
	for coreutils@gnu.org; Wed, 19 Jul 2017 04:44:07 -0400
Received: from mout.web.de ([212.227.17.12]:62631)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16)
	(Exim 4.71) (envelope-from <elfring@users.sourceforge.net>)
	id 1dXkaR-00069a-Mf
	for coreutils@gnu.org; Wed, 19 Jul 2017 04:44:03 -0400
Received: from [192.168.1.3] ([78.49.196.208]) by smtp.web.de (mrweb101
	[213.165.67.124]) with ESMTPSA (Nemesis) id 0M3jwL-1dpHKr2QxV-00rJJ4;
	Wed, 19 Jul 2017 10:43:52 +0200
Subject: Re: Determination of file lists from selected folders without
	returning directory names
To: =?UTF-8?Q?Bernhard_V=c3=b6lker?= <mail@bernhard-voelker.de>
Cc: Kaz Kylheku <962-396-1872@kylheku.com>, coreutils@gnu.org
References: <af44fed2-a5a2-21a5-a3fa-6982a5d37e1d@users.sourceforge.net>
	<c97927cd182f34e3527fe741cce0b726@mail.kylheku.com>
	<7c44c5c2-e5cd-b368-a8bd-d7dc6dc3de3f@users.sourceforge.net>
	<b6745e3004da59da1e859847ed671d21@mail.kylheku.com>
	<12c091e8-380d-b9c4-5d95-78d2d1f02041@users.sourceforge.net>
	<a7afb68cad095d83c90450700f168aa5@mail.kylheku.com>
	<1cc2666c-2fbb-20ca-f427-e2caf4a5c661@users.sourceforge.net>
	<5442056c-efe3-4198-9f11-b2e0fd63e90c@bernhard-voelker.de>
	<fc6b1b8c-bb26-314d-5160-d5c4c4e7b36f@users.sourceforge.net>
	<28d7c18d-8291-c6a7-d138-958ae6358b50@bernhard-voelker.de>
From: SF Markus Elfring <elfring@users.sourceforge.net>
Message-ID: <8658ef8b-da51-179e-3897-de60298e957f@users.sourceforge.net>
Date: Wed, 19 Jul 2017 10:43:40 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
	Thunderbird/52.2.1
MIME-Version: 1.0
In-Reply-To: <28d7c18d-8291-c6a7-d138-958ae6358b50@bernhard-voelker.de>
Content-Type: text/plain; charset=utf-8
Content-Language: en-GB
Content-Transfer-Encoding: 8bit
X-Provags-ID: V03:K0:eOlzI/smmHsqujahe55RokRuhAcK2zPsCjPW1QPyztIgGga1b+S
	VpYS9l+Zw/JM7cFtmSQBiuKWvDLw9bweSJ7ezExm1IE+KavTpJWUPU57IrgNrzRWt2UpwBj
	7pExI58Xzbm9xjkcEujy1LYVHTDhv/jF+yj2fmo1oM0IGcVwwX+u8pWuQ5ZWwCIp6XWvD+L
	dbsh/Ak0QduTYiCaAby4Q==
X-UI-Out-Filterresults: notjunk:1;V01:K0:s6jfAOt1MGs=:BXBjr6Ho2ICPsLz8TthMZN
	5AUXkLu5faGvel2Kw4VpkVqfZ1s+EE7IuCjbH+++wwENcNya00AOWOmnBVZVmsGVNGgunxbMI
	a/qNxP+3Mta+gft3Zv62wBsXxR2oD+CMA3pO/oRJ5turxe4kkpYv7N8OW7AyMBNykVBkzhtMj
	KayAca4AZi327ZfZHrLZn5RaQwuU0Zhkb2a+rtNMS7kHatBmlkFTfBcAAHDo8U7WBi83qw/Vt
	+EYOW9pFNfS7+whWBwJ+37ZlePIYdKUhueA8FKU1JN6rFPk4z6FY8F+kLLr1W9QVH3/sO6Nps
	ivoudUoD3tbD+JwT/VYpHg+8Aker8pnOgX2DLQL9LnEvzgRxL6EuzcAj+yxwm4sYXtInQ7sXy
	bvKlRnrWQUeQdPOZXHuVmRGbvJ+ifihbeVhANr3dnBA3OmImtgzugrEEg1ZSGB5k7+mLjie/m
	DOg+jm+0DGs1RQZ+vmv/s9GcXgxVylTTQqrcKbr/v8p2rK96EOlJ6kP01sKi2A2Kh9rVRF//1
	GyANIc4Po5CejSsgekCHZ/+2CfrT8tX2MxtCC/CqQ4S5C4XHm43yEG+AqARkwQZvym4XLf0Nv
	z3vSRPD0dwREtuHYInfn8uB2KnYEn8BGILx9spaJZvGPaXhc0sPHgL5PEF1Dy/4D2VKzX4KyV
	cIUyQHsKe5oYUiEKeWsAded18WyFg4iHYk3q990QaaFWS422bRLN0uJZeuWQ+czvq78lFQDY6
	XJpROSDZN86HeZBd068L27upkdlYTN3kQk6N5gm9L0vc8B37/MtrueSyC7Q85CKSo1w1mkTf1
	3X/SNRxdPZeAcjELJsNN7MnA+HijN6T2Tbd1LDaVN1G05DeOak=
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
	[fuzzy]
X-Received-From: 212.227.17.12
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Wed, 19 Jul 2017 08:44:08 -0000

>>> why should a simple `basename` or ${name##*/} hurt the performance?
>>
>> Such data deletion causes also corresponding processing costs, doesn't it?
>> Would it be nicer to avoid them if the initial input data could be provided
>> as “basenames” already?
> 
> Sorry, it seems I missed the most important question:
> What is your exact use case?

You might be looking for more information than I suggest so far.

Possible use cases:
* The reduced names can belong to known directories.

* File name usage can be analysed across the folder hierarchy.



> We need numbers and commands to reproduce it.

I propose to take another look at the applied data processing style.

* When is it more efficient to work only with the required data at
  the beginning of an algorithm?

* Under which circumstances will you tolerate that more data can be
  provided than are needed for an action?


> Otherwise, this is all just guesswork, and we could spend years to discuss
> what could maybe be considered, etc.

How much will it matter to compare consequences from eager and lazy evaluation?


> You already got the ideal answer for probably 99.9% of the cases - "use find(1)"

This tool is already usable for various file searches.


> - so unless you describe your outstanding edge case in more detail,
> I'm afraid there's nothing more we could help you with.

I imagine that there can be a target conflict between the flexibility of
a known program and a search task with special constraints.

How much will data processing for the parameter “-printf” influence
rum time characteristics in undesired ways when the output function
could be a fixed one like “basename()”?

Regards,
Markus


From MAILER-DAEMON Wed Jul 19 06:52:31 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dXmal-0000RE-Np
	for mharc-coreutils@gnu.org; Wed, 19 Jul 2017 06:52:31 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:34548)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1dXmaj-0000R4-9z
	for coreutils@gnu.org; Wed, 19 Jul 2017 06:52:30 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1dXmag-0006Ys-8P
	for coreutils@gnu.org; Wed, 19 Jul 2017 06:52:29 -0400
Received: from mx1.redhat.com ([209.132.183.28]:40872)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)
	(Exim 4.71) (envelope-from <eblake@redhat.com>) id 1dXmaf-0006Ye-VN
	for coreutils@gnu.org; Wed, 19 Jul 2017 06:52:26 -0400
Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com
	[10.5.11.15])
	(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
	(No client certificate requested)
	by mx1.redhat.com (Postfix) with ESMTPS id 4BFF52FFC47;
	Wed, 19 Jul 2017 10:52:24 +0000 (UTC)
DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 4BFF52FFC47
Authentication-Results: ext-mx05.extmail.prod.ext.phx2.redhat.com;
	dmarc=none (p=none dis=none) header.from=redhat.com
Authentication-Results: ext-mx05.extmail.prod.ext.phx2.redhat.com;
	spf=pass smtp.mailfrom=eblake@redhat.com
DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 4BFF52FFC47
Received: from [10.10.121.28] (ovpn-121-28.rdu2.redhat.com [10.10.121.28])
	by smtp.corp.redhat.com (Postfix) with ESMTP id DA74B7A439;
	Wed, 19 Jul 2017 10:52:22 +0000 (UTC)
Subject: Re: Determination of file lists from selected folders without
	returning directory names
To: SF Markus Elfring <elfring@users.sourceforge.net>,
	=?UTF-8?Q?Bernhard_V=c3=b6lker?= <mail@bernhard-voelker.de>
References: <af44fed2-a5a2-21a5-a3fa-6982a5d37e1d@users.sourceforge.net>
	<c97927cd182f34e3527fe741cce0b726@mail.kylheku.com>
	<7c44c5c2-e5cd-b368-a8bd-d7dc6dc3de3f@users.sourceforge.net>
	<b6745e3004da59da1e859847ed671d21@mail.kylheku.com>
	<12c091e8-380d-b9c4-5d95-78d2d1f02041@users.sourceforge.net>
	<a7afb68cad095d83c90450700f168aa5@mail.kylheku.com>
	<1cc2666c-2fbb-20ca-f427-e2caf4a5c661@users.sourceforge.net>
	<5442056c-efe3-4198-9f11-b2e0fd63e90c@bernhard-voelker.de>
	<fc6b1b8c-bb26-314d-5160-d5c4c4e7b36f@users.sourceforge.net>
	<28d7c18d-8291-c6a7-d138-958ae6358b50@bernhard-voelker.de>
	<8658ef8b-da51-179e-3897-de60298e957f@users.sourceforge.net>
Cc: Kaz Kylheku <962-396-1872@kylheku.com>, coreutils@gnu.org
From: Eric Blake <eblake@redhat.com>
Openpgp: url=http://people.redhat.com/eblake/eblake.gpg
Organization: Red Hat, Inc.
Message-ID: <ffda35d0-eca9-beb8-846a-f7f00aace11f@redhat.com>
Date: Wed, 19 Jul 2017 05:52:22 -0500
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
	Thunderbird/52.2.1
MIME-Version: 1.0
In-Reply-To: <8658ef8b-da51-179e-3897-de60298e957f@users.sourceforge.net>
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature";
	boundary="sTPs752cDPmJPcnsunvvOqxxDBSsi6xn8"
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15
X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16
	(mx1.redhat.com [10.5.110.29]);
	Wed, 19 Jul 2017 10:52:24 +0000 (UTC)
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
	[fuzzy]
X-Received-From: 209.132.183.28
X-Content-Filtered-By: Mailman/MimeDel 2.1.21
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Wed, 19 Jul 2017 10:52:30 -0000

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--sTPs752cDPmJPcnsunvvOqxxDBSsi6xn8
From: Eric Blake <eblake@redhat.com>
To: SF Markus Elfring <elfring@users.sourceforge.net>,
 =?UTF-8?Q?Bernhard_V=c3=b6lker?= <mail@bernhard-voelker.de>
Cc: Kaz Kylheku <962-396-1872@kylheku.com>, coreutils@gnu.org
Message-ID: <ffda35d0-eca9-beb8-846a-f7f00aace11f@redhat.com>
Subject: Re: Determination of file lists from selected folders without
 returning directory names
References: <af44fed2-a5a2-21a5-a3fa-6982a5d37e1d@users.sourceforge.net>
 <c97927cd182f34e3527fe741cce0b726@mail.kylheku.com>
 <7c44c5c2-e5cd-b368-a8bd-d7dc6dc3de3f@users.sourceforge.net>
 <b6745e3004da59da1e859847ed671d21@mail.kylheku.com>
 <12c091e8-380d-b9c4-5d95-78d2d1f02041@users.sourceforge.net>
 <a7afb68cad095d83c90450700f168aa5@mail.kylheku.com>
 <1cc2666c-2fbb-20ca-f427-e2caf4a5c661@users.sourceforge.net>
 <5442056c-efe3-4198-9f11-b2e0fd63e90c@bernhard-voelker.de>
 <fc6b1b8c-bb26-314d-5160-d5c4c4e7b36f@users.sourceforge.net>
 <28d7c18d-8291-c6a7-d138-958ae6358b50@bernhard-voelker.de>
 <8658ef8b-da51-179e-3897-de60298e957f@users.sourceforge.net>
In-Reply-To: <8658ef8b-da51-179e-3897-de60298e957f@users.sourceforge.net>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

On 07/19/2017 03:43 AM, SF Markus Elfring wrote:
>>>> why should a simple `basename` or ${name##*/} hurt the performance?
>>>
>>> Such data deletion causes also corresponding processing costs, doesn'=
t it?

Have you heard of premature optimization?

If you have a program that spends 1% of its time generating then
stripping text, and 99% of its time doing much lengthier tasks on the
resulting text (and pretty much any syscall to open() or otherwise
manipulate files in the file system generally is MUCH slower than simple
text manipulation), then optimizing the text manipulation portion CANNOT
give you more than a 1% speedup in your overall program.

Instead of wasting our time arguing that providing the full relative
name to a file (which is a sane default for most uses, as acting on the
file itself from a different directory needs that full name), where
stripping the extra information for the rare cases (what cases? you
haven't even given a concrete demonstration with numbers of how often
you are actually encountering situations where having just the basename
to begin with lets you still do the right thing), the RIGHT thing to do
is benchmark it yourself.

At this point, patches speak louder than words.  If you can write a
benchmark where you write a simple .c program that can access just
basenames of files from an arbitrary directory (or an enhancement to the
find program), and show that your program outperforms baseline find for
your given use case, and where the outperformance actually makes a
difference to the overall usage pattern (that is, your benchmark also
shows that it was much more than 1% of the overall time spent on
producing then stripping the prefix data), then it is worth patching
find to provide that mode of operation (and such proof belongs best on
the findutils list).

> I propose to take another look at the applied data processing style.

I propose that you quit trying to micro-optimize something without a
benchmark case that we can reproduce showing that the amount of time
spent producing then stripping the data even makes a difference.

>=20
> How much will data processing for the parameter =E2=80=9C-printf=E2=80=9D=
 influence
> rum time characteristics in undesired ways when the output function
> could be a fixed one like =E2=80=9Cbasename()=E2=80=9D?

Benchmark it and see for yourself.  And the answer is probably not
enough to be worth changing things.

--=20
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org


--sTPs752cDPmJPcnsunvvOqxxDBSsi6xn8
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Comment: Public key at http://people.redhat.com/eblake/eblake.gpg
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEzBAEBCAAdFiEEccLMIrHEYCkn0vOqp6FrSiUnQ2oFAllvOeYACgkQp6FrSiUn
Q2pOtQf9H8ciec6Is6Ucd2QlSWEQRCv5/MGjzNqzFXRVMkmYKBHJ9qqVOzmG/4Kp
o5CaB10iG30PRYVTjCMPnCG8fOmOCG5GnrZ5I1YcxE8xwRXS1JubteEFrj6MMglo
CZmSwrLZHKg24COoep8PVpHUaLxLZa6Q5ReQ2d6u/lGTNaLWfxrWyykePsZELEDb
FXqqD239fzPPIc+WRPj9IzCgB0NE7T9thJu/oA31Mq+VRCvGgZznYrK/W7y9j6dQ
whU+9jjBcrxITdZd5PGShanhfTLkAF/7+UGP2eZHDP7O6ap5GeyKgDRmiCMqxsUU
WbpQ862b9eHXCTdXos0w16Mw6LGAeg==
=HHHO
-----END PGP SIGNATURE-----

--sTPs752cDPmJPcnsunvvOqxxDBSsi6xn8--


From MAILER-DAEMON Wed Jul 19 08:55:02 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dXoVK-0005cK-9U
	for mharc-coreutils@gnu.org; Wed, 19 Jul 2017 08:55:02 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:60323)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <Kenneth.Nellis@conduent.com>) id 1dXoVH-0005Y9-4B
	for coreutils@gnu.org; Wed, 19 Jul 2017 08:55:00 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <Kenneth.Nellis@conduent.com>) id 1dXoVC-00049r-7j
	for coreutils@gnu.org; Wed, 19 Jul 2017 08:54:59 -0400
Received: from esa6.hc188666.iphmx.com ([68.232.145.192]:6248)
	by eggs.gnu.org with esmtps (TLS1.0:RSA_ARCFOUR_SHA1:16) (Exim 4.71)
	(envelope-from <Kenneth.Nellis@conduent.com>) id 1dXoVB-00048J-Sn
	for coreutils@gnu.org; Wed, 19 Jul 2017 08:54:54 -0400
X-IronPort-AV: E=Sophos;i="5.40,381,1496120400"; 
   d="scan'208";a="330774"
Received: from mail-cys01nam02lp0051.outbound.protection.outlook.com (HELO
	NAM02-CY1-obe.outbound.protection.outlook.com) ([207.46.163.51])
	by ob1.hc188666.iphmx.com with ESMTP/TLS/AES256-SHA256;
	19 Jul 2017 07:54:49 -0500
Received: from BN6PR2001MB1074.namprd20.prod.outlook.com (10.172.110.16) by
	BN6PR2001MB1075.namprd20.prod.outlook.com (10.172.110.17) with
	Microsoft SMTP Server (version=TLS1_2,
	cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id
	15.1.1261.13; Wed, 19 Jul 2017 12:54:47 +0000
Received: from BN6PR2001MB1074.namprd20.prod.outlook.com ([10.172.110.16]) by
	BN6PR2001MB1074.namprd20.prod.outlook.com ([10.172.110.16]) with
	mapi id 15.01.1261.024; Wed, 19 Jul 2017 12:54:47 +0000
From: "Nellis, Kenneth" <Kenneth.Nellis@conduent.com>
To: Lance E Sloan <sloanlance+coreutils_gnu.org@gmail.com>, "Kaz Kylheku
	(Coreutils)" <962-396-1872@kylheku.com>
CC: "coreutils@gnu.org" <coreutils@gnu.org>
Subject: RE: coreutils feature requests?
Thread-Topic: coreutils feature requests?
Thread-Index: AQHTACdmlzu/0FH+rkik7mR8BJ2DCaJaVkKAgAAmbwCAAJ2P0A==
Date: Wed, 19 Jul 2017 12:54:47 +0000
Message-ID: <BN6PR2001MB107464DC3D9DF5CB83103EA7F1A60@BN6PR2001MB1074.namprd20.prod.outlook.com>
References: <CAJecg5pGpQU_CZ-AdMtA88fzGs7RPd+dHG__AeYTvergm=iKFA@mail.gmail.com>
	<ed6ef41b902c4a37f5c78e8f791d23f6@mail.kylheku.com>
	<CAJecg5r6U4X-O2J5x_HSExSEy19Skksyw7jrEdwASabjaind_w@mail.gmail.com>
In-Reply-To: <CAJecg5r6U4X-O2J5x_HSExSEy19Skksyw7jrEdwASabjaind_w@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: gmail.com; dkim=none (message not signed)
	header.d=none; gmail.com;
	dmarc=none action=none header.from=conduent.com; 
x-originating-ip: [192.203.187.55]
x-ms-publictraffictype: Email
x-microsoft-exchange-diagnostics: 1; BN6PR2001MB1075;
	20:47NU7o6yDJuaujO5PPm9EUAu0E25gEGof2IC4T+lmQisyBGU/ZF7PsT+/lVgKTOkMGSOal+IPYxphfvAKTmhLlO1mS56+dIDMf1426ti+U4+r23JwHGWtISjuYkbMhTgt15pY2bHK5zZUJC5JJeyjcqQrzUbfOR5mp12GHEd7m1XsbMrt1qH6tX5mKYOPWg5ER9uaxYga+E2Kp9TF1qATSphsDtJZm8731QB/lzZWCXpoJsTSDMdBvKw4aStI8fO
x-ms-office365-filtering-correlation-id: 272f2cce-2758-4345-fec5-08d4cea55632
x-microsoft-antispam: UriScan:; BCL:0; PCL:0;
	RULEID:(300000500095)(300135000095)(300000501095)(300135300095)(22001)(300000502095)(300135100095)(2017030254075)(300000503095)(300135400095)(2017052603031)(201703131423075)(201703031133081)(201702281549075)(300000504095)(300135200095)(300000505095)(300135600095)(300000506095)(300135500095);
	SRVR:BN6PR2001MB1075; 
x-ms-traffictypediagnostic: BN6PR2001MB1075:
x-exchange-antispam-report-test: UriScan:(133145235818549)(236129657087228)(131327999870524)(148574349560750)(155532106045638);
x-microsoft-antispam-prvs: <BN6PR2001MB1075C50598C9ABB2D1BC1B68F1A60@BN6PR2001MB1075.namprd20.prod.outlook.com>
x-exchange-antispam-report-cfa-test: BCL:0; PCL:0;
	RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(601004)(2401047)(5005006)(8121501046)(2017060910075)(10201501046)(93006095)(93001095)(3002001)(100000703101)(100105400095)(6041248)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123558100)(20161123562025)(20161123564025)(20161123555025)(20161123560025)(6072148)(100000704101)(100105200095)(100000705101)(100105500095);
	SRVR:BN6PR2001MB1075; BCL:0; PCL:0;
	RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095);
	SRVR:BN6PR2001MB1075; 
x-forefront-prvs: 0373D94D15
x-forefront-antispam-report: SFV:NSPM;
	SFS:(10009020)(6009001)(39410400002)(39400400002)(39450400003)(39860400002)(39840400002)(39850400002)(51874003)(51914003)(24454002)(7736002)(6246003)(81166006)(3280700002)(53936002)(66066001)(86362001)(2900100001)(25786009)(6506006)(50986999)(54356999)(77096006)(4326008)(38730400002)(76176999)(14454004)(3660700001)(189998001)(229853002)(9686003)(2950100002)(305945005)(2906002)(3480700004)(5660300001)(102836003)(8936002)(39060400002)(7696004)(7116003)(55016002)(53546010)(3846002)(74316002)(6116002)(72206003)(6436002)(99286003)(8676002)(33656002)(478600001)(369524004);
	DIR:OUT; SFP:1101; SCL:1; SRVR:BN6PR2001MB1075;
	H:BN6PR2001MB1074.namprd20.prod.outlook.com; FPR:; SPF:None;
	MLV:ovrnspm; PTR:InfoNoRecords; LANG:en; 
spamdiagnosticoutput: 1:99
spamdiagnosticmetadata: NSPM
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: conduent.com
X-MS-Exchange-CrossTenant-originalarrivaltime: 19 Jul 2017 12:54:47.5315 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 1aed4588-b8ce-43a8-a775-989538fd30d8
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR2001MB1075
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 68.232.145.192
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Wed, 19 Jul 2017 12:55:00 -0000

RnJvbTogY29yZXV0aWxzID4gSGksIEthei4NCj4gDQo+IFRoYW5rcyBmb3IgdGhlIHJlc3BvbnNl
LiAgSSdtIGhhcHB5IHRoYXQgeW91J3ZlIGNvbWUgdG8gdGhlIHNhbWUNCj4gY29uY2x1c2lvbg0K
PiB0aGF0IEknZCByZWFjaGVkLiBJIG9ubHkgaGF2ZSBhIG1vbWVudCBmb3IgYSBxdWljayByZXBs
eSBub3csIGJ1dCBteQ0KPiBpbmNsaW5hdGlvbiB3YXMgdG8gYWRkIGEgLW8gLyAtLW9yZGVyZWQg
b3B0aW9uIHRvIG1ha2UgLWYgKGFuZCBvdGhlcg0KPiBvcHRpb25zKSB3b3JrIHRoZSB3YXkgSSdk
IGV4cGVjdGVkLg0KPiANCj4gSSdkIGJlIGhhcHB5IHRvIGltcGxlbWVudCB0aGUgZmVhdHVyZSBt
eXNlbGYgYW5kIGNvbnRyaWJ1dGUgaXQgYmFjayB0byB0aGUNCj4gY29tbXVuaXR5LiBJJ2QganVz
dCBsaWtlIHRvIGtub3cgd2hhdCBhcmUgdGhlIHByb2NlZHVyZXMuIFNob3VsZCBJIHdhaXQNCj4g
Zm9yDQo+IHRoZSBmZWF0dXJlIHJlcXVlc3QgdG8gYmUgYXBwcm92ZWQgYnkgc29tZW9uZT8gIEkg
ZGlkbid0IHdhbnQgdG8gc3BlbmQNCj4gdGltZQ0KPiBpbXBsZW1lbnRpbmcgaXQgb25seSB0byBz
ZWUgb3V0IHJlamVjdGVkIHNvb24gYWZ0ZXJ3YXJkcy4NCj4gDQo+IEkndmUgc2VlbiB0aGUgaGFj
a2luZyByZWFkbWUgYW5kIEkgdW5kZXJzdGFuZCB3aGF0IHRvIGRvIHRlY2huaWNhbGx5Lg0KPiBI
b3dldmVyLCBJIGNvdWxkbid0IGZpbmQgYW55IGRvY3VtZW50YXRpb24gYWJvdXQgd2hhdCBoYXBw
ZW5zDQo+IHByb2NlZHVyYWxseS4gIElmIHlvdSBvciBvdGhlcnMgY2FuIHNoYXJlIHNvbWUgaW5z
aWdodCwgSSdkIGFwcHJlY2lhdGUgaXQuDQo+IA0KPiBUaGFua3MgaW4gYWR2YW5jZS4NCj4gDQo+
IE9uIFR1ZSwgSnVsIDE4LCAyMDE3IGF0IDIxOjA2IEtheiBLeWxoZWt1IChDb3JldXRpbHMpIDwN
Cj4gOTYyLTM5Ni0xODcyQGt5bGhla3UuY29tPiB3cm90ZToNCj4gDQo+ID4gT24gMTguMDcuMjAx
NyAxNTo0NCwgTGFuY2UgRSBTbG9hbiB3cm90ZToNCj4gPiA+IEhpLCBhbGwuDQo+ID4gPg0KPiA+
ID4gQXNpZGUgZnJvbSBhIGJ1ZyByZXBvcnQsIGlzIHRoZXJlIGEgd2F5IHRvIHN1Ym1pdCBhIGZl
YXR1cmUgcmVxdWVzdA0KPiBmb3INCj4gPiA+IGNvcmV1dGlscz8gIEkgaGF2ZSBhIGNvdXBsZSBv
ZiByZXF1ZXN0cyBpbiBtaW5kLCB3aGljaCBJIGRvbid0IHRoaW5rDQo+ID4gPiBxdWFsaWZ5IGFz
IGJ1Z3M6DQo+ID4gPg0KPiA+ID4gMS4gQWRkIGEgZmVhdHVyZSB0byAiY3V0IiB0aGF0IGFsbG93
cyBzZWxlY3RlZCBmaWVsZHMgdG8gYmUgb3V0cHV0IGluDQo+IGENCj4gPiA+IGRpZmZlcmVudCBv
cmRlci4gIFRoYXQgaXMgImN1dCAtZjQsMSw4LTYiIHdvdWxkIGNhdXNlIGl0IHRvIG91dHB1dA0K
PiA+ID4gZmllbGRzDQo+ID4gPiBpbiB0aGUgb3JkZXIgb2YgNCwgMSwgOCwgNywgYW5kIDYuDQo+
ID4NCj4gPiBJJ20gYW1hemVkIHRoYXQgaXQgZG9lc24ndCB3b3JrIHRoaXMgd2F5OyB0aGUgdXRp
bGl0eSBvZiBpbXBsaWNpdGx5DQo+ID4gc29ydGluZyB0aGUgZmllbGRzIGFwcGVhcnMgbG93IGNv
bXBhcmVkIHRvIHRoZSBkYW1hZ2UgdGhhdCBpdCBkb2VzDQo+ID4gdG8gdGhlIGZsZXhpYmlsaXR5
IG9mIGN1dC4gKFdoYXQgbGl0dGxlIGl0IGhhcyEpDQo+ID4NCj4gPiBJZiBQT1NJWCBzcGVjaWZp
ZXMgaXQsIEkgaGF2ZSB0byBzYXkgdGhhdCBpdHMgcmVxdWlyZW1lbnRzIGFyZQ0KPiA+IHN1Ym9w
dGltYWwgKGFzIGlzIG9mdGVuIHRoZSBjYXNlIGluIGRpdmVyc2UgYXJlYXMpLg0KPiA+DQo+ID4g
SW5kZWVkLCB0aGUgcmVxdWlyZW1lbnQgaXMgc2FkbHkgZ2l2ZW4gYXM6DQo+ID4NCj4gPiAgICAg
VGhlIGVsZW1lbnRzIGluIGxpc3QgY2FuIGJlIHJlcGVhdGVkLCBjYW4gb3ZlcmxhcCwgYW5kIGNh
biBiZQ0KPiA+IHNwZWNpZmllZA0KPiA+ICAgICBpbiBhbnkgb3JkZXIsIGJ1dCB0aGUgYnl0ZXMs
IGNoYXJhY3RlcnMsIG9yIGZpZWxkcyBzZWxlY3RlZCBzaGFsbCBiZQ0KPiA+ICAgICB3cml0dGVu
IGluIHRoZSBvcmRlciBvZiB0aGUgaW5wdXQgZGF0YS4gSWYgYW4gZWxlbWVudCBhcHBlYXJzIGlu
IHRoZQ0KPiA+ICAgICBzZWxlY3Rpb24gbGlzdCBtb3JlIHRoYW4gb25jZSwgaXQgc2hhbGwgYmUg
d3JpdHRlbiBleGFjdGx5IG9uY2UuDQo+ID4NCj4gPiBEbyBwZW9wbGUgb2Z0ZW4gd3JpdGUgY3V0
IHNwZWNpZmljYXRpb25zIGluIGFkIGhvYyBvcmRlcnMsIHdpdGgNCj4gPiByZXBldGl0aW9ucyBh
bmQgdGhlbiByZWx5IG9uIHRoZSBzb3J0aW5nIGJlaGF2aW9yPw0KPiA+DQo+ID4gVGhlIEdOVSBp
bXBsZW1lbnRhdGlvbiBvZiBjdXQgY291bGQgbGVhZCBzdGFuZGFyZGl6YXRpb24gaW4gdGhpcw0K
PiA+IGFyZWEgdG8gaW1wcm92ZSB0aGluZ3MuDQo+ID4NCj4gPiBBbGwgdGhlIHBvc3NpYmxlIHNv
bHV0aW9ucyB0byBtYWtlIGN1dCBub3Qgc29ydCB0aGUgZmllbGRzIG9yDQo+ID4gZGUtZHVwbGlj
YXRlIHRoZW0gYXJlIHVnbHkuIEVpdGhlciB5b3UgIG5lZWQgYSBnbG9iYWwgb3B0aW9uIHRvDQo+
ID4gb3B0IG91dCBvZiB0aGF0IGJlaGF2aW9yLCBsaWtlICItbyIgKHByZXNlcnZlIChvKXJkZXIp
IGFuZA0KPiA+IGtlZXAgcmVtZW1iZXJpbmcgdG8gdXNlIGl0LCBvciBlbHNlIHByb3ZpZGUgb3Jk
ZXItcHJlc2VydmluZw0KPiA+IHZlcnNpb25zIG9mIHRoZSB2YXJpb3VzIG9wdGlvbnMsIGxpa2Ug
cGVyaGFwcyB0aHJvdWdoIGNhcGl0YWxpemF0aW9uOg0KPiA+IC1GLCAtQywgLUIuDQo+ID4NCj4g
PiBBbiBhdHRyYWN0aXZlIGFsdGVybmF0aXZlIGlzIHRvIGhhdmUgYSB3aG9sZSBuZXcgY29tbWFu
ZCB3aGljaA0KPiA+IG1pcnJvcnMgY3V0LCBsaWtlIHNheSAiY2xpcCIsIHdoaWNoIGlzIGV4YWN0
bHkgbGlrZSBjdXQsIGJ1dA0KPiA+IG9yZGVyIGFuZCByZXBldGl0aW9uIHByZXNlcnZpbmcuDQo+
ID4NCj4gPiBOb3RlIHRoYXQgdGhlIC0tY29tcGxlbWVudCBvcHRpb24gaXMgc2VtYW50aWNhbGx5
IGluY29tcGF0aWJsZSB3aXRoDQo+ID4gb3JkZXItcHJlc2VydmluZyBtb2RlOyB0aGUgY29tcGxl
bWVudCBjb25jZXB0IGZvbGxvd3MgZnJvbSB0aGUNCj4gPiBzZWxlY3RlZCBlbGVtZW50cyBiZWlu
ZyByZWdhcmRlZCBhcyBhIHNldCByYXRoZXIgdGhhbiBvcmRlcmVkDQo+ID4gc2VxdWVuY2UuIElm
IC1GLCAtQyBvciAtQiBpcyB1c2VkIHdpdGggLS1jb21wbGVtZW50LCBpdCBoYXMgdG8gYmUNCj4g
PiBkaWFnbm9zZWQuICBPciBpZiB0aGVyZSBpcyBhICJjbGlwIiBjb21tYW5kLCB0aGVuIHRoYXQg
c2ltcGx5DQo+ID4gZG9lc24ndCBzdXBwb3J0IC0tY29tcGxlbWVudC4NCg0KSW5kZWVkLCBJJ3Zl
IHdhbnRlZCB0aGlzIGZlYXR1cmUgZm9yIGEgd2hpbGUuIFdlIHVzZSBRTlg0IGluIG91ciANCmxl
Z2FjeSBlbWJlZGRlZCBwcm9kdWN0LCBhbmQgaXRzIGN1dCBpbmRlZWQgb3V0cHV0cyBmaWVsZHMg
aW4gdGhlIA0Kb3JkZXIgc3BlY2lmaWVkLiBJIHdhcyBxdWl0ZSBzdXJwcmlzZWQgd2hlbiB0aGUg
c2NyaXB0IEkgd3JvdGUgZm9yIA0KUU5YNCBkaWRuJ3Qgd29yayB3aXRoIEdudS4NCg0KcW54ND4g
ZWNobyAxLDIsMyw0LDUsNiB8IGN1dCAtZCwgLWYzLDUsMg0KMyw1LDINCnFueDQ+IHVzZSBjdXQN
CmN1dCAtIGN1dCBvdXQgc2VsZWN0ZWQgZmllbGRzIG9mIGVhY2ggbGluZSBvZiBhIGZpbGUgKFBP
U0lYKQ0KDQpjdXQgLWMgbGlzdCBbZmlsZSAuLi5dDQpjdXQgLWYgbGlzdCBbLWQgZGVsaW1dIFst
c10gW2ZpbGUgLi4uXQ0KT3B0aW9uczoNCiAtYyAgICAgIEN1dCBvdXQgdGhlIDxsaXN0PiBvZiBm
aWVsZHMgYmFzZWQgb24gY2hhcmFjdGVyIHBvc2l0aW9uLg0KIC1mICAgICAgQ3V0IG91dCB0aGUg
PGxpc3Q+IG9mIGZpZWxkcyBiYXNlZCBvbiBmaWVsZCBwb3NpdGlvbi4NCiAtZCAgICAgIFNwZWNp
ZnkgdGhlIGZpZWxkIGRlbGltaXRlciBjaGFyYWN0ZXIuDQogLXMgICAgICBTdXBwcmVzcyB0aGUg
b3V0cHV0IG9mIGxpbmVzIHdoaWNoIGNvbnRhaW4gbm8gZmllbGQgZGVsaW1pdGVyLg0KV2hlcmU6
DQogbGlzdCAgICBJcyBhIGxpc3Qgb2YgY2hhcmFjdGVyIG9yIGZpZWxkIHBvc2l0aW9ucyBzZXBh
cmF0ZWQgYnkgYSA8c3BhY2U+DQogICAgICAgICBvciBhIDxjb21tYT4uIEEgcmFuZ2Ugb2YgcG9z
aXRpb25zIG1heSBiZSBzcGVjaWZpZWQgdXNpbmcgdGhlDQogICAgICAgICA8aHlwaGVuPi4NCkV4
YW1wbGVzOg0KIGN1dCAtYyAyMC0yOSw0MC00OSBteWZpbGUgICAgT3V0cHV0IGNoYXJhY3RlciBj
b2x1bW5zIDIwIHRvIDI5IGFuZCA0MCB0byA0OSANCiAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIG9mIHRoZSBmaWxlICdteWZpbGUnDQogY3V0IC1mIDMsNSw3LTExIC1kfCBteWZpbGUgICBP
dXRwdXQgZmllbGRzIG51bWJlcmVkIDMsIDUsIGFuZCA3IHRvIDExIG9mIHRoZQ0KICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgZmlsZSAnbXlmaWxlJy4gIFRoZSBmaWVsZHMgaW4gZWFjaCBp
bnB1dCBsaW5lDQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICBhcmUgc2VwYXJhdGVkIGJ5
IHRoZSAnfCcgY2hhcmFjdGVyLg0KcW54ND4gDQoNCkkgc3VwcG9ydCB0aGUgY2hhbmdlLg0KDQot
LUtlbiBOZWxsaXMNCg==


From MAILER-DAEMON Wed Jul 19 09:18:48 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dXosK-00066w-PA
	for mharc-coreutils@gnu.org; Wed, 19 Jul 2017 09:18:48 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:38351)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <steeve.mccauley@gmail.com>) id 1dXosI-00066k-14
	for coreutils@gnu.org; Wed, 19 Jul 2017 09:18:47 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <steeve.mccauley@gmail.com>) id 1dXosG-0004yZ-7p
	for coreutils@gnu.org; Wed, 19 Jul 2017 09:18:46 -0400
Received: from mail-vk0-x232.google.com ([2607:f8b0:400c:c05::232]:32982)
	by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)
	(Exim 4.71) (envelope-from <steeve.mccauley@gmail.com>)
	id 1dXosG-0004yA-2G
	for coreutils@gnu.org; Wed, 19 Jul 2017 09:18:44 -0400
Received: by mail-vk0-x232.google.com with SMTP id p193so269251vkd.0
	for <coreutils@gnu.org>; Wed, 19 Jul 2017 06:18:42 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
	h=mime-version:in-reply-to:references:from:date:message-id:subject:to
	:cc; bh=BoKE+Y8o5XKcTABuvY7e7SWokB14kuIm/ELRInaZid4=;
	b=l9i2CVeOAnDgHmvVXe9ueENa7tDzCis4VZaXIclrzvuLaJl+AORddXjMLrPUxLG0/a
	yZbB727Kf1OXERRvx/NaSuja8nL4L++gcwE5k3ZZheeqFPzcG+tBcmVPkTUxxjFZLPKt
	91/cd3mKaWYE2LXw72xvSJaNOI3h4s5mQzGtAGzI9jQchnQtV4nGZSjpZ0JrFEQOA1pW
	XjV/UC7e3txKIVuWEjhQt5zrbl9opa9E4RXDLvinxEp5J/KvjG2WZOWVfLEra9952gVD
	uF1E86XeQG+oN458Mv7GfTd5anTfCBigkI+xw5KVOiAST9dev3oMMkAgKnF5qsZlL8ai
	kQQQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
	d=1e100.net; s=20161025;
	h=x-gm-message-state:mime-version:in-reply-to:references:from:date
	:message-id:subject:to:cc;
	bh=BoKE+Y8o5XKcTABuvY7e7SWokB14kuIm/ELRInaZid4=;
	b=SfZjiAhMcG2/FEFKzkdtnlGYKTH78AOt1MkojhbgrJNydfDgxX1HjHFsA2fGDovBwm
	LMyaXw/WNxFi3ycu2MrqStbj2fhQ+O9SlMqZRkOzIriMQYNHFCkoCpCQYcfJUENgHRbT
	3ZXHKL7yDKh5ymAIS77s4kmL0bhcnwiQ5JHMjGVyjLkjhpbfzqv0yUPoyBrsPsp/S4SD
	vRbz+rgCjndLwSSqOneKTH64bOfmN+lTmz5uKMsv1RgRy3dfY1E6SNhGfT7IMbJk8G+0
	626jsRYd2eM7oFkDJ3I0+JNcmhzOzfr3ckcG9YMYgpO9EY0l8rPBd239uQp1sr12qUka
	U3sA==
X-Gm-Message-State: AIVw110f47bdKbPocrmD5dbCTP1WZX/LnZwADiBVJ6Averu0Hz8JOVZT
	F6xthO8QtxW6ZqhjVu/j53746vF/Jw==
X-Received: by 10.31.194.210 with SMTP id s201mr45404vkf.2.1500470321912; Wed,
	19 Jul 2017 06:18:41 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.176.28.15 with HTTP; Wed, 19 Jul 2017 06:18:01 -0700 (PDT)
In-Reply-To: <BN6PR2001MB107464DC3D9DF5CB83103EA7F1A60@BN6PR2001MB1074.namprd20.prod.outlook.com>
References: <CAJecg5pGpQU_CZ-AdMtA88fzGs7RPd+dHG__AeYTvergm=iKFA@mail.gmail.com>
	<ed6ef41b902c4a37f5c78e8f791d23f6@mail.kylheku.com>
	<CAJecg5r6U4X-O2J5x_HSExSEy19Skksyw7jrEdwASabjaind_w@mail.gmail.com>
	<BN6PR2001MB107464DC3D9DF5CB83103EA7F1A60@BN6PR2001MB1074.namprd20.prod.outlook.com>
From: Steeve McCauley <steeve.mccauley@gmail.com>
Date: Wed, 19 Jul 2017 09:18:01 -0400
Message-ID: <CA+MRLqnRXjrOmqObVMObBhDUo5uRZAUMo=vOuE=tKOWSZMk2ZQ@mail.gmail.com>
Subject: Re: coreutils feature requests?
To: "Nellis, Kenneth" <Kenneth.Nellis@conduent.com>
Cc: Lance E Sloan <sloanlance+coreutils_gnu.org@gmail.com>, 
	"Kaz Kylheku (Coreutils)" <962-396-1872@kylheku.com>,
	"coreutils@gnu.org" <coreutils@gnu.org>
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 2607:f8b0:400c:c05::232
Content-Type: text/plain; charset="UTF-8"
X-Content-Filtered-By: Mailman/MimeDel 2.1.21
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Wed, 19 Jul 2017 13:18:47 -0000

I can't believe I'd never thought of reordering output columns like this.
FWIW, I agree that another option should be used to prevent issues with
backward compatibility.

$ echo 1,2,3,4,5,6 | cut -d, -f3,5,2
2,3,5

$ echo 1,2,3,4,5,6 | cut -d, -f3,5,2 -o
3,5,2

Should this be extended to character output as well?

echo output | cut -c6,4,2 -o
tpu


On Wed, Jul 19, 2017 at 8:54 AM, Nellis, Kenneth <
Kenneth.Nellis@conduent.com> wrote:

> From: coreutils > Hi, Kaz.
> >
> > Thanks for the response.  I'm happy that you've come to the same
> > conclusion
> > that I'd reached. I only have a moment for a quick reply now, but my
> > inclination was to add a -o / --ordered option to make -f (and other
> > options) work the way I'd expected.
> >
> > I'd be happy to implement the feature myself and contribute it back to
> the
> > community. I'd just like to know what are the procedures. Should I wait
> > for
> > the feature request to be approved by someone?  I didn't want to spend
> > time
> > implementing it only to see out rejected soon afterwards.
> >
> > I've seen the hacking readme and I understand what to do technically.
> > However, I couldn't find any documentation about what happens
> > procedurally.  If you or others can share some insight, I'd appreciate
> it.
> >
> > Thanks in advance.
> >
> > On Tue, Jul 18, 2017 at 21:06 Kaz Kylheku (Coreutils) <
> > 962-396-1872@kylheku.com> wrote:
> >
> > > On 18.07.2017 15:44, Lance E Sloan wrote:
> > > > Hi, all.
> > > >
> > > > Aside from a bug report, is there a way to submit a feature request
> > for
> > > > coreutils?  I have a couple of requests in mind, which I don't think
> > > > qualify as bugs:
> > > >
> > > > 1. Add a feature to "cut" that allows selected fields to be output in
> > a
> > > > different order.  That is "cut -f4,1,8-6" would cause it to output
> > > > fields
> > > > in the order of 4, 1, 8, 7, and 6.
> > >
> > > I'm amazed that it doesn't work this way; the utility of implicitly
> > > sorting the fields appears low compared to the damage that it does
> > > to the flexibility of cut. (What little it has!)
> > >
> > > If POSIX specifies it, I have to say that its requirements are
> > > suboptimal (as is often the case in diverse areas).
> > >
> > > Indeed, the requirement is sadly given as:
> > >
> > >     The elements in list can be repeated, can overlap, and can be
> > > specified
> > >     in any order, but the bytes, characters, or fields selected shall
> be
> > >     written in the order of the input data. If an element appears in
> the
> > >     selection list more than once, it shall be written exactly once.
> > >
> > > Do people often write cut specifications in ad hoc orders, with
> > > repetitions and then rely on the sorting behavior?
> > >
> > > The GNU implementation of cut could lead standardization in this
> > > area to improve things.
> > >
> > > All the possible solutions to make cut not sort the fields or
> > > de-duplicate them are ugly. Either you  need a global option to
> > > opt out of that behavior, like "-o" (preserve (o)rder) and
> > > keep remembering to use it, or else provide order-preserving
> > > versions of the various options, like perhaps through capitalization:
> > > -F, -C, -B.
> > >
> > > An attractive alternative is to have a whole new command which
> > > mirrors cut, like say "clip", which is exactly like cut, but
> > > order and repetition preserving.
> > >
> > > Note that the --complement option is semantically incompatible with
> > > order-preserving mode; the complement concept follows from the
> > > selected elements being regarded as a set rather than ordered
> > > sequence. If -F, -C or -B is used with --complement, it has to be
> > > diagnosed.  Or if there is a "clip" command, then that simply
> > > doesn't support --complement.
>
> Indeed, I've wanted this feature for a while. We use QNX4 in our
> legacy embedded product, and its cut indeed outputs fields in the
> order specified. I was quite surprised when the script I wrote for
> QNX4 didn't work with Gnu.
>
> qnx4> echo 1,2,3,4,5,6 | cut -d, -f3,5,2
> 3,5,2
> qnx4> use cut
> cut - cut out selected fields of each line of a file (POSIX)
>
> cut -c list [file ...]
> cut -f list [-d delim] [-s] [file ...]
> Options:
>  -c      Cut out the <list> of fields based on character position.
>  -f      Cut out the <list> of fields based on field position.
>  -d      Specify the field delimiter character.
>  -s      Suppress the output of lines which contain no field delimiter.
> Where:
>  list    Is a list of character or field positions separated by a <space>
>          or a <comma>. A range of positions may be specified using the
>          <hyphen>.
> Examples:
>  cut -c 20-29,40-49 myfile    Output character columns 20 to 29 and 40 to
> 49
>                               of the file 'myfile'
>  cut -f 3,5,7-11 -d| myfile   Output fields numbered 3, 5, and 7 to 11 of
> the
>                               file 'myfile'.  The fields in each input line
>                               are separated by the '|' character.
> qnx4>
>
> I support the change.
>
> --Ken Nellis
>



-- 
:wq

From MAILER-DAEMON Wed Jul 19 09:29:25 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dXp2b-000252-9M
	for mharc-coreutils@gnu.org; Wed, 19 Jul 2017 09:29:25 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:40493)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <Kenneth.Nellis@conduent.com>) id 1dXp2Y-000249-Ib
	for coreutils@gnu.org; Wed, 19 Jul 2017 09:29:23 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <Kenneth.Nellis@conduent.com>) id 1dXp2Q-0000hR-Cm
	for coreutils@gnu.org; Wed, 19 Jul 2017 09:29:22 -0400
Received: from esa9.hc188666.iphmx.com ([216.71.154.48]:46731)
	by eggs.gnu.org with esmtps (TLS1.0:RSA_ARCFOUR_SHA1:16) (Exim 4.71)
	(envelope-from <Kenneth.Nellis@conduent.com>) id 1dXp2Q-0000gw-2E
	for coreutils@gnu.org; Wed, 19 Jul 2017 09:29:14 -0400
X-IronPort-AV: E=Sophos;i="5.40,381,1496120400"; 
   d="scan'208";a="343980"
Received: from mail-sn1nam01lp0120.outbound.protection.outlook.com (HELO
	NAM01-SN1-obe.outbound.protection.outlook.com) ([207.46.163.120])
	by ob1.hc188666.iphmx.com with ESMTP/TLS/AES256-SHA256;
	19 Jul 2017 08:29:12 -0500
Received: from BN6PR2001MB1074.namprd20.prod.outlook.com (10.172.110.16) by
	BN6PR2001MB1076.namprd20.prod.outlook.com (10.172.110.18) with
	Microsoft SMTP Server (version=TLS1_2,
	cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id
	15.1.1261.13; Wed, 19 Jul 2017 13:29:10 +0000
Received: from BN6PR2001MB1074.namprd20.prod.outlook.com ([10.172.110.16]) by
	BN6PR2001MB1074.namprd20.prod.outlook.com ([10.172.110.16]) with
	mapi id 15.01.1261.024; Wed, 19 Jul 2017 13:29:10 +0000
From: "Nellis, Kenneth" <Kenneth.Nellis@conduent.com>
To: Steeve McCauley <steeve.mccauley@gmail.com>
CC: Lance E Sloan <sloanlance+coreutils_gnu.org@gmail.com>, "Kaz Kylheku
	(Coreutils)" <962-396-1872@kylheku.com>, "coreutils@gnu.org"
	<coreutils@gnu.org>
Subject: RE: coreutils feature requests?
Thread-Topic: coreutils feature requests?
Thread-Index: AQHTACdmlzu/0FH+rkik7mR8BJ2DCaJaVkKAgAAmbwCAAJ2P0IAACFWAgAABhxA=
Date: Wed, 19 Jul 2017 13:29:10 +0000
Message-ID: <BN6PR2001MB1074D25959ADEB88B732CBA9F1A60@BN6PR2001MB1074.namprd20.prod.outlook.com>
References: <CAJecg5pGpQU_CZ-AdMtA88fzGs7RPd+dHG__AeYTvergm=iKFA@mail.gmail.com>
	<ed6ef41b902c4a37f5c78e8f791d23f6@mail.kylheku.com>
	<CAJecg5r6U4X-O2J5x_HSExSEy19Skksyw7jrEdwASabjaind_w@mail.gmail.com>
	<BN6PR2001MB107464DC3D9DF5CB83103EA7F1A60@BN6PR2001MB1074.namprd20.prod.outlook.com>
	<CA+MRLqnRXjrOmqObVMObBhDUo5uRZAUMo=vOuE=tKOWSZMk2ZQ@mail.gmail.com>
In-Reply-To: <CA+MRLqnRXjrOmqObVMObBhDUo5uRZAUMo=vOuE=tKOWSZMk2ZQ@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: gmail.com; dkim=none (message not signed)
	header.d=none; gmail.com;
	dmarc=none action=none header.from=conduent.com; 
x-originating-ip: [192.203.187.55]
x-ms-publictraffictype: Email
x-microsoft-exchange-diagnostics: 1; BN6PR2001MB1076;
	20:i+P1Ok+g7LJUd1ZVFUSy3JeClWbSXoFpOZlvmMisG/mwGExc7w8iGH/hC1ETSRqTZoRLhy7qtSM7mSqehnj/VgTFp4z92QQkt/q9erf+ZXVu7CsFetohx98sqZL42dcR+B53US/dkS7aXsJ7k2SWDmn12PSC0pvI6ujN47NQAoPFwDnk8yjoY5YnfzF53BJfK6CzTKs1vAG3psLlMScbs1r8cyxP962mSHGyQD1VWhn4zC2Z2deNhJNlxWLFhHqW
x-ms-office365-filtering-correlation-id: 9eb5b667-191c-4a7d-4d21-08d4ceaa23dc
x-microsoft-antispam: UriScan:; BCL:0; PCL:0;
	RULEID:(300000500095)(300135000095)(300000501095)(300135300095)(22001)(300000502095)(300135100095)(2017030254075)(300000503095)(300135400095)(2017052603031)(201703131423075)(201703031133081)(300000504095)(300135200095)(300000505095)(300135600095)(300000506095)(300135500095);
	SRVR:BN6PR2001MB1076; 
x-ms-traffictypediagnostic: BN6PR2001MB1076:
x-exchange-antispam-report-test: UriScan:(236129657087228)(247924648384137);
x-microsoft-antispam-prvs: <BN6PR2001MB10766417C7F4E8A119F9E17BF1A60@BN6PR2001MB1076.namprd20.prod.outlook.com>
x-exchange-antispam-report-cfa-test: BCL:0; PCL:0;
	RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(601004)(2401047)(2017060910075)(5005006)(8121501046)(3002001)(10201501046)(93006095)(93001095)(100000703101)(100105400095)(6041248)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123560025)(20161123562025)(20161123558100)(20161123555025)(20161123564025)(6072148)(100000704101)(100105200095)(100000705101)(100105500095);
	SRVR:BN6PR2001MB1076; BCL:0; PCL:0;
	RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095);
	SRVR:BN6PR2001MB1076; 
x-forefront-prvs: 0373D94D15
x-forefront-antispam-report: SFV:NSPM;
	SFS:(10009020)(6009001)(39410400002)(39450400003)(39400400002)(39850400002)(39860400002)(39840400002)(7696004)(102836003)(3846002)(2900100001)(6116002)(74316002)(7116003)(76176999)(5660300001)(305945005)(54356999)(50986999)(6506006)(39060400002)(3280700002)(53936002)(93886004)(3660700001)(189998001)(478600001)(72206003)(77096006)(2906002)(6436002)(14454004)(25786009)(55016002)(8676002)(6916009)(8936002)(66066001)(54906002)(2950100002)(6246003)(7736002)(110136004)(3480700004)(33656002)(9686003)(81166006)(229853002)(99286003)(38730400002)(4326008)(86362001);
	DIR:OUT; SFP:1101; SCL:1; SRVR:BN6PR2001MB1076;
	H:BN6PR2001MB1074.namprd20.prod.outlook.com; FPR:; SPF:None;
	MLV:ovrnspm; PTR:InfoNoRecords; LANG:en; 
spamdiagnosticoutput: 1:99
spamdiagnosticmetadata: NSPM
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: conduent.com
X-MS-Exchange-CrossTenant-originalarrivaltime: 19 Jul 2017 13:29:10.5453 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 1aed4588-b8ce-43a8-a775-989538fd30d8
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR2001MB1076
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 216.71.154.48
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Wed, 19 Jul 2017 13:29:23 -0000

RnJvbTogU3RlZXZlIE1jQ2F1bGV5DQo+IEkgY2FuJ3QgYmVsaWV2ZSBJJ2QgbmV2ZXIgdGhvdWdo
dCBvZiByZW9yZGVyaW5nIG91dHB1dCBjb2x1bW5zIGxpa2UgdGhpcy7CoCANCj4gRldJVywgSSBh
Z3JlZSB0aGF0IGFub3RoZXIgb3B0aW9uIHNob3VsZCBiZSB1c2VkIHRvIHByZXZlbnQgaXNzdWVz
IHdpdGggYmFja3dhcmQgY29tcGF0aWJpbGl0eS4NCj4gDQo+ICQgZWNobyAxLDIsMyw0LDUsNiB8
IGN1dCAtZCwgLWYzLDUsMg0KPiAyLDMsNQ0KPiANCj4gJCBlY2hvIDEsMiwzLDQsNSw2IHwgY3V0
IC1kLCAtZjMsNSwyIC1vDQo+IDMsNSwyDQo+IFNob3VsZCB0aGlzIGJlIGV4dGVuZGVkIHRvIGNo
YXJhY3RlciBvdXRwdXQgYXMgd2VsbD8NCj4gZWNobyBvdXRwdXQgfCBjdXQgLWM2LDQsMiAtbw0K
PiB0cHUNCj4gDQoNCkFic29sdXRlbHkhIEl0IHdvdWxkIGJlIGV4cGVjdGVkIGJlaGF2aW9yIChJ
TUhPKS4gSSBzZWUgbm8gcmVhc29uIG5vdCB0by4NCg0KSW4gYWRkaXRpb24sIHNvIHRoYXQgc2Ny
aXB0cyBjYW4gd29yayBhY3Jvc3MgcGxhdGZvcm1zLCBJIChzdHJvbmdseSkgcmVjb21tZW5kIA0K
dGhhdCBhIGN1dC1zcGVjaWZpYyBlbnZpcm9ubWVudCB2YXJpYWJsZSBiZSBkZWZpbmVkIHRvIGFs
bG93IHNwZWNpZnlpbmcgdGhlDQpmaWVsZCBvcmRlcmluZyBiZWhhdmlvci4gSW4gdGhhdCB3YXkg
bXkgUU5YIDQgc2NyaXB0ICh3aG9zZSBjdXQgd291bGQgYmFsayBhdA0KdGhlIC1vIG9wdGlvbikg
d291bGQgd29yayB3aXRoIEdudS4gT25lIHBvc3NpYmlsaXR5Og0KDQpDVVRfT1BUSU9OUz0tbw0K
DQotLUtlbiBOZWxsaXMNCg==


From MAILER-DAEMON Wed Jul 19 10:15:53 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dXplZ-00086m-T7
	for mharc-coreutils@gnu.org; Wed, 19 Jul 2017 10:15:53 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:52607)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1dXplX-00086b-1z
	for coreutils@gnu.org; Wed, 19 Jul 2017 10:15:52 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1dXplU-0002ll-Dc
	for coreutils@gnu.org; Wed, 19 Jul 2017 10:15:51 -0400
Received: from mx1.redhat.com ([209.132.183.28]:52286)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)
	(Exim 4.71) (envelope-from <eblake@redhat.com>) id 1dXplU-0002lA-48
	for coreutils@gnu.org; Wed, 19 Jul 2017 10:15:48 -0400
Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com
	[10.5.11.14])
	(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
	(No client certificate requested)
	by mx1.redhat.com (Postfix) with ESMTPS id D6AC1FED4;
	Wed, 19 Jul 2017 14:15:46 +0000 (UTC)
DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com D6AC1FED4
Authentication-Results: ext-mx05.extmail.prod.ext.phx2.redhat.com;
	dmarc=none (p=none dis=none) header.from=redhat.com
Authentication-Results: ext-mx05.extmail.prod.ext.phx2.redhat.com;
	spf=pass smtp.mailfrom=eblake@redhat.com
DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com D6AC1FED4
Received: from [10.10.121.28] (ovpn-121-28.rdu2.redhat.com [10.10.121.28])
	by smtp.corp.redhat.com (Postfix) with ESMTP id 291B96C51E;
	Wed, 19 Jul 2017 14:15:44 +0000 (UTC)
Subject: Re: coreutils feature requests?
To: "Nellis, Kenneth" <Kenneth.Nellis@conduent.com>,
	Steeve McCauley <steeve.mccauley@gmail.com>
References: <CAJecg5pGpQU_CZ-AdMtA88fzGs7RPd+dHG__AeYTvergm=iKFA@mail.gmail.com>
	<ed6ef41b902c4a37f5c78e8f791d23f6@mail.kylheku.com>
	<CAJecg5r6U4X-O2J5x_HSExSEy19Skksyw7jrEdwASabjaind_w@mail.gmail.com>
	<BN6PR2001MB107464DC3D9DF5CB83103EA7F1A60@BN6PR2001MB1074.namprd20.prod.outlook.com>
	<CA+MRLqnRXjrOmqObVMObBhDUo5uRZAUMo=vOuE=tKOWSZMk2ZQ@mail.gmail.com>
	<BN6PR2001MB1074D25959ADEB88B732CBA9F1A60@BN6PR2001MB1074.namprd20.prod.outlook.com>
Cc: "coreutils@gnu.org" <coreutils@gnu.org>,
	"Kaz Kylheku (Coreutils)" <962-396-1872@kylheku.com>,
	Lance E Sloan <sloanlance+coreutils_gnu.org@gmail.com>
From: Eric Blake <eblake@redhat.com>
Openpgp: url=http://people.redhat.com/eblake/eblake.gpg
Organization: Red Hat, Inc.
Message-ID: <2fa5a4be-7ba9-3257-713f-b1e907e03489@redhat.com>
Date: Wed, 19 Jul 2017 09:15:43 -0500
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
	Thunderbird/52.2.1
MIME-Version: 1.0
In-Reply-To: <BN6PR2001MB1074D25959ADEB88B732CBA9F1A60@BN6PR2001MB1074.namprd20.prod.outlook.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature";
	boundary="KaQrrbc5J1SA3IH2r729TJPTtbrF9v4g3"
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14
X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16
	(mx1.redhat.com [10.5.110.29]);
	Wed, 19 Jul 2017 14:15:47 +0000 (UTC)
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
	[fuzzy]
X-Received-From: 209.132.183.28
X-Content-Filtered-By: Mailman/MimeDel 2.1.21
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Wed, 19 Jul 2017 14:15:52 -0000

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--KaQrrbc5J1SA3IH2r729TJPTtbrF9v4g3
From: Eric Blake <eblake@redhat.com>
To: "Nellis, Kenneth" <Kenneth.Nellis@conduent.com>,
 Steeve McCauley <steeve.mccauley@gmail.com>
Cc: "coreutils@gnu.org" <coreutils@gnu.org>,
 "Kaz Kylheku (Coreutils)" <962-396-1872@kylheku.com>,
 Lance E Sloan <sloanlance+coreutils_gnu.org@gmail.com>
Message-ID: <2fa5a4be-7ba9-3257-713f-b1e907e03489@redhat.com>
Subject: Re: coreutils feature requests?
References: <CAJecg5pGpQU_CZ-AdMtA88fzGs7RPd+dHG__AeYTvergm=iKFA@mail.gmail.com>
 <ed6ef41b902c4a37f5c78e8f791d23f6@mail.kylheku.com>
 <CAJecg5r6U4X-O2J5x_HSExSEy19Skksyw7jrEdwASabjaind_w@mail.gmail.com>
 <BN6PR2001MB107464DC3D9DF5CB83103EA7F1A60@BN6PR2001MB1074.namprd20.prod.outlook.com>
 <CA+MRLqnRXjrOmqObVMObBhDUo5uRZAUMo=vOuE=tKOWSZMk2ZQ@mail.gmail.com>
 <BN6PR2001MB1074D25959ADEB88B732CBA9F1A60@BN6PR2001MB1074.namprd20.prod.outlook.com>
In-Reply-To: <BN6PR2001MB1074D25959ADEB88B732CBA9F1A60@BN6PR2001MB1074.namprd20.prod.outlook.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

On 07/19/2017 08:29 AM, Nellis, Kenneth wrote:
> From: Steeve McCauley
>> I can't believe I'd never thought of reordering output columns like th=
is. =20
>> FWIW, I agree that another option should be used to prevent issues wit=
h backward compatibility.
>>
>> $ echo 1,2,3,4,5,6 | cut -d, -f3,5,2
>> 2,3,5
>>
>> $ echo 1,2,3,4,5,6 | cut -d, -f3,5,2 -o
>> 3,5,2

Unfortunately, it takes time to add an option, and then for that
addition to percolate into the pre-built binaries of all the distros
that you use.  In the meantime, you can already use awk to get the
behavior you want today.  And since awk is already portable and required
by POSIX to be able to reorder output fields, it's that much more of a
burden to justify adding a feature to cut (adding a feature is easy if
it is easy to show that no other existing tools can fill the gap, but a
one-line awk program doesn't feel like no other tools being able to fill
the gap).  Adding a new feature to GNU Coreutils is also easier to do if
you can find someone else (like BSD) that already has the feature - but
to my knowledge, no common 'cut -o' exists in other major distributions.

> In addition, so that scripts can work across platforms, I (strongly) re=
commend=20
> that a cut-specific environment variable be defined to allow specifying=
 the
> field ordering behavior. In that way my QNX 4 script (whose cut would b=
alk at
> the -o option) would work with Gnu. One possibility:
>=20
> CUT_OPTIONS=3D-o

Absolutely not.  We have already learned from experience that
environment variables that modify core program functionality, especially
NEW environment variables that are not already accounted for in old
scripts, tend to break old scripts.  If you want a different default,
our recommended approach is to write a shell script wrapper installed
earlier on your $PATH, rather than abusing environment variables for
everyone.

--=20
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org


--KaQrrbc5J1SA3IH2r729TJPTtbrF9v4g3
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Comment: Public key at http://people.redhat.com/eblake/eblake.gpg
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEzBAEBCAAdFiEEccLMIrHEYCkn0vOqp6FrSiUnQ2oFAllvaY8ACgkQp6FrSiUn
Q2pLVgf/Zk0tCSzlsc2qp2USUrAOmeYF/g8Yj3slVsuBb1tDJwHGmPgho4Il//CX
bcYcC8njAtqnYUE4wBkzUJaWRdQAaigXqEgS5TmaEwaM57QKjloimgqCckffYsNz
PmmxAFQjxHcofiRNC8rT+p68fca/uPkrPm/ii30HTep8+TAYTZuIIRfeCz19heJN
U0UmWLraGZMGsIReDKvxpDUOyHeqwSlPXUOOvcWB/i7yuDhAE3VtPdaRyUm9YWfV
CJuth46/CcQqaGEmcUROFbTEpKIv3x51zF8bnaaZuCrmQTpetQyBOetKflA9V7Z0
8dXJc94Kj4ou5uhJcXUtAKEq7yISow==
=iUhN
-----END PGP SIGNATURE-----

--KaQrrbc5J1SA3IH2r729TJPTtbrF9v4g3--


From MAILER-DAEMON Wed Jul 19 10:21:48 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dXprI-0000gY-Ha
	for mharc-coreutils@gnu.org; Wed, 19 Jul 2017 10:21:48 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:53983)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <962-396-1872@kylheku.com>) id 1dXprF-0000gS-VE
	for coreutils@gnu.org; Wed, 19 Jul 2017 10:21:46 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <962-396-1872@kylheku.com>) id 1dXprD-0005Ua-BK
	for coreutils@gnu.org; Wed, 19 Jul 2017 10:21:45 -0400
Received: from smtp-out-so.shaw.ca ([64.59.136.139]:38144)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)
	(Exim 4.71) (envelope-from <962-396-1872@kylheku.com>)
	id 1dXprD-0005U2-5q
	for coreutils@gnu.org; Wed, 19 Jul 2017 10:21:43 -0400
Received: from kylheku.com ([70.79.163.252]) by shaw.ca with SMTP
	id Xpr4dmo6gDJTWXpr5duc05; Wed, 19 Jul 2017 08:21:40 -0600
X-Authority-Analysis: v=2.2 cv=B4DJ6KlM c=1 sm=1 tr=0
	a=95A0EdhkF1LMGt25d7h1IQ==:117 a=95A0EdhkF1LMGt25d7h1IQ==:17
	a=IkcTkHD0fZMA:10 a=SMorJkV_YP8A:10 a=G3gG6ho9WtcA:10
	a=73h8QIhlec70nnu_8hQA:9 a=QEXdDO2ut3YA:10
Received: from www-data by kylheku.com with local (Exim 4.72)
	(envelope-from <962-396-1872@kylheku.com>)
	id 1dXpr4-0006r8-1c; Wed, 19 Jul 2017 07:21:34 -0700
To: "Nellis, Kenneth" <Kenneth.Nellis@conduent.com>
Subject: RE: coreutils feature =?UTF-8?Q?requests=3F?=
X-PHP-Originating-Script: 501:rcmail.php
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8;
 format=flowed
Content-Transfer-Encoding: 8bit
Date: Wed, 19 Jul 2017 07:21:33 -0700
From: "Kaz Kylheku (Coreutils)" <962-396-1872@kylheku.com>
Cc: Steeve McCauley <steeve.mccauley@gmail.com>, coreutils@gnu.org, Lance E
	Sloan <sloanlance+coreutils_gnu.org@gmail.com>
In-Reply-To: <BN6PR2001MB1074D25959ADEB88B732CBA9F1A60@BN6PR2001MB1074.namprd20.prod.outlook.com>
References: <CAJecg5pGpQU_CZ-AdMtA88fzGs7RPd+dHG__AeYTvergm=iKFA@mail.gmail.com>
	<ed6ef41b902c4a37f5c78e8f791d23f6@mail.kylheku.com>
	<CAJecg5r6U4X-O2J5x_HSExSEy19Skksyw7jrEdwASabjaind_w@mail.gmail.com>
	<BN6PR2001MB107464DC3D9DF5CB83103EA7F1A60@BN6PR2001MB1074.namprd20.prod.outlook.com>
	<CA+MRLqnRXjrOmqObVMObBhDUo5uRZAUMo=vOuE=tKOWSZMk2ZQ@mail.gmail.com>
	<BN6PR2001MB1074D25959ADEB88B732CBA9F1A60@BN6PR2001MB1074.namprd20.prod.outlook.com>
Message-ID: <4f80047842e78bd4a41ab906396d678e@mail.kylheku.com>
X-Sender: 962-396-1872@kylheku.com
User-Agent: Roundcube Webmail/0.9.2
X-CMAE-Envelope: MS4wfHHB/lrNhAWoRdMJOG+EG1IPZuIDXH/c3mKSdbELvNJvGhF+6WhjnvmG9tP7PyfzJj7yks6tzGXjgo0pjKVsAkja/tyxRB1FWAUv+hHhU8yK1pgH1Uv4
	SAtzzFHYpN24dOmCNQ5FHiTJCYliq2Y4+1GnKUBt9xnMAn4RIwKA79VDoaOsswI0+Xa2ZbY+CBA/14nLJwo9UDvnsDhVCa4oBQRlBzP8+M+0AFXiuB9qIOZR
	uu5sDs2MwtNKq12iL1lMt2WZNwgH3xiJTrmhbbNbotuQyHLTYb/WAACEkcsOkkLWgSEkeo/1vYACTk0R6WKeIQ==
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy]
X-Received-From: 64.59.136.139
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Wed, 19 Jul 2017 14:21:47 -0000

On 19.07.2017 06:29, Nellis, Kenneth wrote:
> From: Steeve McCauley
>> I can't believe I'd never thought of reordering output columns like 
>> this. 
>> FWIW, I agree that another option should be used to prevent issues 
>> with backward compatibility.
>> 
>> $ echo 1,2,3,4,5,6 | cut -d, -f3,5,2
>> 2,3,5
>> 
>> $ echo 1,2,3,4,5,6 | cut -d, -f3,5,2 -o
>> 3,5,2
>> Should this be extended to character output as well?
>> echo output | cut -c6,4,2 -o
>> tpu
>> 
> 
> Absolutely! It would be expected behavior (IMHO). I see no reason not 
> to.

POSIX expends considerable text in requiring that the fields constitute
a set, and are de-duplicated and put into order. (Quoted in earlier
message.) So indeed, the behavior cannot just be changed to match the
QNX "cut", not just because of backward compatibility (always the 
primary
concern) but standard conformance also (close second).

QNX has a conformance bug here. I wouldn't continue to rely on it.

> In addition, so that scripts can work across platforms, I (strongly) 
> recommend
> that a cut-specific environment variable be defined to allow specifying 
> the
> field ordering behavior. In that way my QNX 4 script (whose cut would 
> balk at
> the -o option) would work with Gnu. One possibility:

On the other hand, I hope I'm not alone in behind opposed to introducing
new global variables which alter language or tool semantics.

Scripts can target multiple implementations of a command by defining a
wrapping function rather than a magic global variable.

A portable script cannot rely on cut having order-preserving fields 
anyway.

Everyone in POSIX-land implementing different features and then using
a zoo of environment variables to emulate each other's features and
quirks would be unmanageable.



From MAILER-DAEMON Wed Jul 19 12:45:51 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dXs6h-0005CV-OZ
	for mharc-coreutils@gnu.org; Wed, 19 Jul 2017 12:45:51 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:60604)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <elfring@users.sourceforge.net>) id 1dXs6f-0005CO-U2
	for coreutils@gnu.org; Wed, 19 Jul 2017 12:45:50 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <elfring@users.sourceforge.net>) id 1dXs6c-00006H-Q0
	for coreutils@gnu.org; Wed, 19 Jul 2017 12:45:49 -0400
Received: from mout.web.de ([212.227.17.11]:63955)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16)
	(Exim 4.71) (envelope-from <elfring@users.sourceforge.net>)
	id 1dXs6c-00005n-EZ
	for coreutils@gnu.org; Wed, 19 Jul 2017 12:45:46 -0400
Received: from [192.168.1.3] ([78.49.196.208]) by smtp.web.de (mrweb101
	[213.165.67.124]) with ESMTPSA (Nemesis) id 0MSaZk-1d6TLa0JL2-00Rb1E;
	Wed, 19 Jul 2017 18:45:30 +0200
Subject: Re: Determination of file lists from selected folders without
	returning directory names
To: Eric Blake <eblake@redhat.com>
Cc: =?UTF-8?Q?Bernhard_V=c3=b6lker?= <mail@bernhard-voelker.de>,
	Kaz Kylheku <962-396-1872@kylheku.com>, coreutils@gnu.org
References: <af44fed2-a5a2-21a5-a3fa-6982a5d37e1d@users.sourceforge.net>
	<c97927cd182f34e3527fe741cce0b726@mail.kylheku.com>
	<7c44c5c2-e5cd-b368-a8bd-d7dc6dc3de3f@users.sourceforge.net>
	<b6745e3004da59da1e859847ed671d21@mail.kylheku.com>
	<12c091e8-380d-b9c4-5d95-78d2d1f02041@users.sourceforge.net>
	<a7afb68cad095d83c90450700f168aa5@mail.kylheku.com>
	<1cc2666c-2fbb-20ca-f427-e2caf4a5c661@users.sourceforge.net>
	<5442056c-efe3-4198-9f11-b2e0fd63e90c@bernhard-voelker.de>
	<fc6b1b8c-bb26-314d-5160-d5c4c4e7b36f@users.sourceforge.net>
	<28d7c18d-8291-c6a7-d138-958ae6358b50@bernhard-voelker.de>
	<8658ef8b-da51-179e-3897-de60298e957f@users.sourceforge.net>
	<ffda35d0-eca9-beb8-846a-f7f00aace11f@redhat.com>
From: SF Markus Elfring <elfring@users.sourceforge.net>
Message-ID: <78f6f80a-bf64-3972-9b25-1cc62ad67aa3@users.sourceforge.net>
Date: Wed, 19 Jul 2017 18:45:25 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
	Thunderbird/52.2.1
MIME-Version: 1.0
In-Reply-To: <ffda35d0-eca9-beb8-846a-f7f00aace11f@redhat.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-GB
Content-Transfer-Encoding: 8bit
X-Provags-ID: V03:K0:G4MESLOySNrtb5Lwd9O/IOk9kEdw4anWSVfSB/7/dbSsxv9IZSy
	ObvyOi+VWhAaxpY7K/3OOA0p5hnUzLnZiDUiLXLCKcUh+NkPiKvXpHjXr6q0oxp7DoVvwsH
	O6CQu5ZFkWfIyrcv7j81EZZMc659nUGGsnpBb9ns7yRT54aLNqMQqn3c/+/ExYV0rFzatuT
	7w2QmN0G4RLdriWCM4DJw==
X-UI-Out-Filterresults: notjunk:1;V01:K0:UQ/uyCYzSC0=:OclWHrEPDoy9i4KqRNRhj4
	xoyuOCPXMdVy3FZ6F7L6wd8JgdBQfz3BUxAoPE86HTglhvpF31IMPECJyx6H793Nig18eA+5s
	1BpOJR1EXUSNCYbr0xzDN2I7NT/qZ01IUIWan1gVwSdOn/IoOqi9iYGojEg34/Pe5ocqkc6lE
	pNhDMJUBGc1hMt7+/Y9D/WgEmaVxh+AwYsp0yxxtGprxahGzRUum8QjtfISo7FFT8uuOixT4t
	iPlUpLbW3fYfPdYcUxjZ8mNu/g+cnDWtO7Zq5QQwyO550nTJvkaP0V8n8XZ/87UjJdqvZEt3W
	cQ8PdncBEmUJO6StYDph7gQP8Lozm2bJwCL/fixTL0PLPI3hC0dj44Zp+NGHtFpglWqgqY06v
	K+1+LTDaV9yZuaOtwmdiTxcIeS2s/Yq6CmGIJY5U7NeBwMYFsPaO7uDZ/6zFQsa0cZ9a9IVma
	Pw/+FJe10zyk6yW+PUryR5GqIbO19K7VKETC0cI0uqWvnJtJHARDoEkBimpIlCBJWOEBPPV/Q
	fVWTqYv5sZK4gmHNRF+LmhzVVx79vP/qbMz+8eokaGjBSHUT8ic36as6A9yrp+CFrLyXOAaIF
	LP8XxBr0bwneon1ZOqf7Cer+JKGM5Bit6nN9Ot/EcrqoyhANd+VePpnwLCggkYTimKS4uc85p
	t/Tk3F9n5ft/A9OaeHzCvjhowy8InE8DUY1KKppKjzDUIiAbgGKXwthbpQIVWwMakue9m7P9V
	bUX3yzS14vkFPssNvOFFxFx4hM0FnurQEUmQoqe9P+zlwByR3T66t3Av0qsmWuCCFzuN0glnf
	6GkFHw8/I1N6aqq7FMnvxYWC3/+3e5o3pVBn3DXpk7AWGwVXYI=
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
	[fuzzy]
X-Received-From: 212.227.17.11
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Wed, 19 Jul 2017 16:45:51 -0000

> If you can write a benchmark where you write a simple .c program
> that can access just basenames of files from an arbitrary directory
> (or an enhancement to the find program), and show that your program
> outperforms baseline find for your given use case, and where the
> outperformance actually makes a difference to the overall usage pattern
> (that is, your benchmark also shows that it was much more than 1% of

I guess that I can not really achieve the kind of software improvement
which you would prefer.


> the overall time spent on producing then stripping the prefix data),

I am proposing only to reduce the influence of such a data transformation.


> then it is worth patching find to provide that mode of operation
> (and such proof belongs best on the findutils list).

I am curious if any other developers (than me) or system testers care
for this implementation detail around the topic “Better support for
data processing with basenames”.
https://savannah.gnu.org/bugs/index.php?51506

Regards,
Markus


From MAILER-DAEMON Wed Jul 19 13:04:47 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dXsP1-0003io-Ll
	for mharc-coreutils@gnu.org; Wed, 19 Jul 2017 13:04:47 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:36410)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <sloanlance@gmail.com>) id 1dXsOw-0003hw-Vt
	for coreutils@gnu.org; Wed, 19 Jul 2017 13:04:44 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <sloanlance@gmail.com>) id 1dXsOv-0001DZ-2U
	for coreutils@gnu.org; Wed, 19 Jul 2017 13:04:42 -0400
Received: from mail-it0-x235.google.com ([2607:f8b0:4001:c0b::235]:38280)
	by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)
	(Exim 4.71) (envelope-from <sloanlance@gmail.com>)
	id 1dXsOu-0001DP-Re
	for coreutils@gnu.org; Wed, 19 Jul 2017 13:04:41 -0400
Received: by mail-it0-x235.google.com with SMTP id h199so3814614ith.1
	for <coreutils@gnu.org>; Wed, 19 Jul 2017 10:04:40 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
	h=mime-version:sender:in-reply-to:references:from:date:message-id
	:subject:to:cc;
	bh=uyc4V74FPcFlgfGfYloAtz2Iatbj6/eARfyOM4yqPiI=;
	b=HQRBo+qF4OLH+jMNjsG+ZXHiXeVCcJVM8HNCe2ks2CZNN9Oq0mSBcW8w40pj+Ajmhp
	usIbF5fVNXU69sbtm3mpnsFzBBlN6byBa7ImGWXBXFJRk59Lbk/pMwDijp577Q8YnIau
	wkLiw8+3W9Ys9CgjjDYDrovbA3M/NJDATfTIcXnu3c/f+OuShYVo9E4HTFAES2fHMxrM
	EsFXLJKOy8aEaI9dUV4o/gW2HqmySc5W7lLcGrC5bWpVagllHgONg1ptMra1GsIVs9k+
	YQGkRDhoihIX/RXBzRhTIfm/jAwWYkUZB7U/AvCOOjneKRx98Yimw+1p3DY7sPqtS1a6
	CCGQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
	d=1e100.net; s=20161025;
	h=x-gm-message-state:mime-version:sender:in-reply-to:references:from
	:date:message-id:subject:to:cc;
	bh=uyc4V74FPcFlgfGfYloAtz2Iatbj6/eARfyOM4yqPiI=;
	b=tZg1rLy2oDB91cenYapIR4UDOC7JrRWE8YCTo5gHWnqaa19v6wvrHDlSks7EA9QpvI
	ZGnJLvup7dQqhsj7D/e8jfA3lQjz88IeTtX3HAyKcOoc1hxnRCFdf9Z5r2GxSBA/BoL0
	Dt8B1z+TwKtcDCBFSH1HKFQiUzWyg7L6N5ivKHLFN1TDBifILEImmXHQiO54jsu7B+S1
	FfBnIpUjdzPNeW/qqxVGfdbFit+7Z6gB7UEBMGaA3dPCz4MKQ69kkuVQWDuWGcCt7gSs
	sZ5++lYjPRKV+fEq5LZQzChNUSuN7PZ1APTDEJDshKsYLeHjODIl3q1p8iO6SaMijLln
	1dvw==
X-Gm-Message-State: AIVw112bVdCNLwBTrFbw1JTef82wXXNn/A7t12Rd1doMj8zi5cxmFEyw
	kPJ1+uHxuIWqqeysXzwSzLCWSDJ/NTli
X-Received: by 10.36.65.18 with SMTP id x18mr575965ita.104.1500483879762; Wed,
	19 Jul 2017 10:04:39 -0700 (PDT)
MIME-Version: 1.0
Sender: sloanlance@gmail.com
Received: by 10.107.35.201 with HTTP; Wed, 19 Jul 2017 10:03:59 -0700 (PDT)
In-Reply-To: <2fa5a4be-7ba9-3257-713f-b1e907e03489@redhat.com>
References: <CAJecg5pGpQU_CZ-AdMtA88fzGs7RPd+dHG__AeYTvergm=iKFA@mail.gmail.com>
	<ed6ef41b902c4a37f5c78e8f791d23f6@mail.kylheku.com>
	<CAJecg5r6U4X-O2J5x_HSExSEy19Skksyw7jrEdwASabjaind_w@mail.gmail.com>
	<BN6PR2001MB107464DC3D9DF5CB83103EA7F1A60@BN6PR2001MB1074.namprd20.prod.outlook.com>
	<CA+MRLqnRXjrOmqObVMObBhDUo5uRZAUMo=vOuE=tKOWSZMk2ZQ@mail.gmail.com>
	<BN6PR2001MB1074D25959ADEB88B732CBA9F1A60@BN6PR2001MB1074.namprd20.prod.outlook.com>
	<2fa5a4be-7ba9-3257-713f-b1e907e03489@redhat.com>
From: Lance E Sloan <sloanlance+coreutils_gnu.org@gmail.com>
Date: Wed, 19 Jul 2017 13:03:59 -0400
X-Google-Sender-Auth: VrbfjdCG9ndpuwzVcC6RmWbYaa0
Message-ID: <CAJecg5qzYtikTFuJbk6V02nqWdyjsu_4zwj65MWQMtnO_Cfszg@mail.gmail.com>
Subject: Re: coreutils feature requests?
To: Eric Blake <eblake@redhat.com>
Cc: "coreutils@gnu.org" <coreutils@gnu.org>
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 2607:f8b0:4001:c0b::235
Content-Type: text/plain; charset="UTF-8"
X-Content-Filtered-By: Mailman/MimeDel 2.1.21
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Wed, 19 Jul 2017 17:04:44 -0000

Hi, Eric.

Thank you for the thoughtful response.  I regret that I have trouble
understanding your point of view, though.  Please know that I do not mean
any disrespect.  I'd appreciate it if you could explain why you're opposed
to adding new features to cut (or to comm).

It may help if I explain my point of view.  I think that if the changes
I've described to cut and comm are considered useful by enough people, then
incorporating them into GNU coreutils is justified.  You're right, most
users won't see the new features overnight.  It takes time for the features
to reach their systems through updates, new OS installations, etc.
However, their delayed gratification doesn't mean we shouldn't attempt to
distribute these useful features.  If the lack of instantaneous utilization
were a true barrier, most software would never get (or never would have
gotten) any new features.

I agree, that for the moment, I may need to use another solution to get the
features I've suggested.  To accomplish the field reordering that cut
lacks, I'm trying a variety of other solutions.  I can use awk, as you
suggest.  I could also do it with bash, perl, python, php, sed, javascript,
java, C, etc.  Each solution has its benefits and drawbacks.

My considerations for a solution:

1.  I need this feature to process several files that have millions of
lines each.  I need to do this on an ongoing, periodic basis.  I can't
afford for the process to be slow.
2.  Since I have a large amount of data, I'm avoiding regular expressions
and interpreted languages, which take longer to complete the job.  That
eliminates awk and several other possible solutions.  A compiled C
application would be best.
3.  I don't need to write a special program for this purpose if one already
exists.  The cut utility already does most of the job.
4. Part of my data processing uses jq.  I've figured out how to do this
field reordering with it, but it makes my jq filter more complex and more
difficult for my successors to maintain.  As written on
https://stedolan.github.io/jq/ , "jq is like sed for JSON data".  I don't
consider sed to be a good solution for a problem of this size, so jq
probably isn't ideal, either.

Since a C implementation should run the fastest and cut from GNU's
coreutils is written in C and presumably doesn't need much work to support
this, it seems like the best solution.

Even if this feature suggestion isn't approved by the GNU community, I will
implement it for my own use anyway.  I can enjoy the new functionality
(which I think should have been added to cut long ago) and keep it to
myself or I can contribute it back to the online community.  I could
distribute it as my own fork of GNU coreutils or as a patch to it.
However, if it were merged into GNU's coreutils, it would get the most
exposure and be helpful to more people.

While I'm glad that several people on this mailing list like my suggestion,
I still don't know much about the procedure for getting it approved for
inclusion in GNU's coreutils.  I'd really like to know what needs to be
done.

With regard to your objection to a special environment variable: I agree.
I didn't feel strongly about it at first, but I was leaning towards not
implementing env. var. support for this.  It just didn't feel right.  I
have written programs that use env. var. to specify or override default
options.  However, your point about adding this to an existing, established
program where it could possibly cause variable conflicts is enough to set
my option a little more strongly against an env. var. for this purpose.

On 19 July 2017 at 10:15, Eric Blake <eblake@redhat.com> wrote:

> On 07/19/2017 08:29 AM, Nellis, Kenneth wrote:
> > From: Steeve McCauley
> >> I can't believe I'd never thought of reordering output columns like
> this.
> >> FWIW, I agree that another option should be used to prevent issues with
> backward compatibility.
>
> Unfortunately, it takes time to add an option, and then for that
> addition to percolate into the pre-built binaries of all the distros
> that you use.  In the meantime, you can already use awk to get the
> behavior you want today.  And since awk is already portable and required
> by POSIX to be able to reorder output fields, it's that much more of a
> burden to justify adding a feature to cut (adding a feature is easy if
> it is easy to show that no other existing tools can fill the gap, but a
> one-line awk program doesn't feel like no other tools being able to fill
> the gap).  Adding a new feature to GNU Coreutils is also easier to do if
> you can find someone else (like BSD) that already has the feature - but
> to my knowledge, no common 'cut -o' exists in other major distributions.
>

From MAILER-DAEMON Wed Jul 19 13:30:20 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dXsnk-0003qd-28
	for mharc-coreutils@gnu.org; Wed, 19 Jul 2017 13:30:20 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:41103)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1dXsnh-0003qQ-8z
	for coreutils@gnu.org; Wed, 19 Jul 2017 13:30:18 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1dXsne-0006XK-3x
	for coreutils@gnu.org; Wed, 19 Jul 2017 13:30:17 -0400
Received: from mx1.redhat.com ([209.132.183.28]:37364)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)
	(Exim 4.71) (envelope-from <eblake@redhat.com>) id 1dXsnd-0006Wo-R2
	for coreutils@gnu.org; Wed, 19 Jul 2017 13:30:14 -0400
Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com
	[10.5.11.11])
	(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
	(No client certificate requested)
	by mx1.redhat.com (Postfix) with ESMTPS id 896627C840;
	Wed, 19 Jul 2017 17:30:12 +0000 (UTC)
DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 896627C840
Authentication-Results: ext-mx03.extmail.prod.ext.phx2.redhat.com;
	dmarc=none (p=none dis=none) header.from=redhat.com
Authentication-Results: ext-mx03.extmail.prod.ext.phx2.redhat.com;
	spf=pass smtp.mailfrom=eblake@redhat.com
DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 896627C840
Received: from [10.10.121.28] (ovpn-121-28.rdu2.redhat.com [10.10.121.28])
	by smtp.corp.redhat.com (Postfix) with ESMTP id 0E6287F483;
	Wed, 19 Jul 2017 17:30:11 +0000 (UTC)
Subject: Re: coreutils feature requests?
To: Lance E Sloan <sloanlance+coreutils_gnu.org@gmail.com>
References: <CAJecg5pGpQU_CZ-AdMtA88fzGs7RPd+dHG__AeYTvergm=iKFA@mail.gmail.com>
	<ed6ef41b902c4a37f5c78e8f791d23f6@mail.kylheku.com>
	<CAJecg5r6U4X-O2J5x_HSExSEy19Skksyw7jrEdwASabjaind_w@mail.gmail.com>
	<BN6PR2001MB107464DC3D9DF5CB83103EA7F1A60@BN6PR2001MB1074.namprd20.prod.outlook.com>
	<CA+MRLqnRXjrOmqObVMObBhDUo5uRZAUMo=vOuE=tKOWSZMk2ZQ@mail.gmail.com>
	<BN6PR2001MB1074D25959ADEB88B732CBA9F1A60@BN6PR2001MB1074.namprd20.prod.outlook.com>
	<2fa5a4be-7ba9-3257-713f-b1e907e03489@redhat.com>
	<CAJecg5qzYtikTFuJbk6V02nqWdyjsu_4zwj65MWQMtnO_Cfszg@mail.gmail.com>
Cc: "coreutils@gnu.org" <coreutils@gnu.org>
From: Eric Blake <eblake@redhat.com>
Openpgp: url=http://people.redhat.com/eblake/eblake.gpg
Organization: Red Hat, Inc.
Message-ID: <cdfebfa6-7475-1bda-6c85-2267b775f381@redhat.com>
Date: Wed, 19 Jul 2017 12:30:11 -0500
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
	Thunderbird/52.2.1
MIME-Version: 1.0
In-Reply-To: <CAJecg5qzYtikTFuJbk6V02nqWdyjsu_4zwj65MWQMtnO_Cfszg@mail.gmail.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature";
	boundary="hH7WrnXaA5Hrtm4J28v5oS6hN0Aq89qhE"
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11
X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16
	(mx1.redhat.com [10.5.110.27]);
	Wed, 19 Jul 2017 17:30:12 +0000 (UTC)
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
	[fuzzy]
X-Received-From: 209.132.183.28
X-Content-Filtered-By: Mailman/MimeDel 2.1.21
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Wed, 19 Jul 2017 17:30:18 -0000

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--hH7WrnXaA5Hrtm4J28v5oS6hN0Aq89qhE
From: Eric Blake <eblake@redhat.com>
To: Lance E Sloan <sloanlance+coreutils_gnu.org@gmail.com>
Cc: "coreutils@gnu.org" <coreutils@gnu.org>
Message-ID: <cdfebfa6-7475-1bda-6c85-2267b775f381@redhat.com>
Subject: Re: coreutils feature requests?
References: <CAJecg5pGpQU_CZ-AdMtA88fzGs7RPd+dHG__AeYTvergm=iKFA@mail.gmail.com>
 <ed6ef41b902c4a37f5c78e8f791d23f6@mail.kylheku.com>
 <CAJecg5r6U4X-O2J5x_HSExSEy19Skksyw7jrEdwASabjaind_w@mail.gmail.com>
 <BN6PR2001MB107464DC3D9DF5CB83103EA7F1A60@BN6PR2001MB1074.namprd20.prod.outlook.com>
 <CA+MRLqnRXjrOmqObVMObBhDUo5uRZAUMo=vOuE=tKOWSZMk2ZQ@mail.gmail.com>
 <BN6PR2001MB1074D25959ADEB88B732CBA9F1A60@BN6PR2001MB1074.namprd20.prod.outlook.com>
 <2fa5a4be-7ba9-3257-713f-b1e907e03489@redhat.com>
 <CAJecg5qzYtikTFuJbk6V02nqWdyjsu_4zwj65MWQMtnO_Cfszg@mail.gmail.com>
In-Reply-To: <CAJecg5qzYtikTFuJbk6V02nqWdyjsu_4zwj65MWQMtnO_Cfszg@mail.gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

On 07/19/2017 12:03 PM, Lance E Sloan wrote:
> Hi, Eric.
>=20
> Thank you for the thoughtful response.  I regret that I have trouble
> understanding your point of view, though.  Please know that I do not me=
an
> any disrespect.  I'd appreciate it if you could explain why you're oppo=
sed
> to adding new features to cut (or to comm).

I'm not opposed to well-justified new features.  It's just that the bar
for justifying new features is rather high (it's a lot of code to add to
get field reordering, and that code has to be tested; it is also a
question of how many users will rely on that extension.  A testsuite
addition is mandatory as part of adding the new feature, if the new
feature is worth adding).

Furthermore, on the topic of cut, we've already had the discussion
multiple times over multiple years:

https://www.gnu.org/software/coreutils/rejected_requests.html
which links to
http://lists.gnu.org/archive/html/bug-coreutils/2012-12/msg00115.html
which links to
http://lists.gnu.org/archive/html/bug-coreutils/2005-06/msg00125.html

Reading some of those threads show that there's not outright opposition,
so much as "why isn't awk good enough, since it already is standardized
and does what you want" - mixed with quotes like "In spite of all of
that, the existing behavior (of not honoring the user-specified
field/column-number ordering) is non-intuitive enough that I'd consider
a patch adding an option to make cut provide the more sensible
behavior."  And remember that patches DO speak louder than words.
Repeating a mantra of "this new feature would be great, can someone else
implement it for me" is a lot easier to ignore than "this new feature
helped me, and here's the patch for your consideration".  So I'm not
outright rejecting the idea, so much as stating that you are not the
first to propose it, and none of the previous proposals got it incorporat=
ed.

I haven't made any decisions about the benefit of a change to comm, and
will leave that to other maintainers to chime in.

> Even if this feature suggestion isn't approved by the GNU community, I =
will
> implement it for my own use anyway.

Absolutely!  That's the beauty of free software, that you ARE permitted
to improve it and share your changes, whether or not other people pick
up on your changes.

--=20
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org


--hH7WrnXaA5Hrtm4J28v5oS6hN0Aq89qhE
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Comment: Public key at http://people.redhat.com/eblake/eblake.gpg
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEzBAEBCAAdFiEEccLMIrHEYCkn0vOqp6FrSiUnQ2oFAllvlyMACgkQp6FrSiUn
Q2q1FQf/YiJaLVre+l9n45EZXXATa9GBhpkETVitsmvWqXWHgHebNhYK2oJgLMcy
4qR3NGOqMZH0/7HhOz3jxshNdMkfrVPomXmXL9ldy0NB2e1mxanB3xtfile3Ng/Z
MiBbIzIgc311JM1y+T/WoGtF7WxCFbu1TACKckAXqRdfRYr8tWttsHUAxm1PjU9B
IfLpoXAQZ2Qnugr3QXY13BuoWJ0k1g2hNDLOkvSzpRV/dm4Mbf34vx6kpVapNaZk
aEP05+QorlXAqES/lur8bx8yAa+aRk4XrNvJ+gv/hdQ8y/wI6XZ/0SoVoKr8VmQn
LyNNvz1FQX526mDuB8AvQUY9EGKTmA==
=dc2I
-----END PGP SIGNATURE-----

--hH7WrnXaA5Hrtm4J28v5oS6hN0Aq89qhE--


From MAILER-DAEMON Wed Jul 19 13:32:47 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dXsq7-0004DQ-CO
	for mharc-coreutils@gnu.org; Wed, 19 Jul 2017 13:32:47 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:41586)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <962-396-1872@kylheku.com>) id 1dXsq4-0004C2-WE
	for coreutils@gnu.org; Wed, 19 Jul 2017 13:32:45 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <962-396-1872@kylheku.com>) id 1dXsq1-0007VZ-S6
	for coreutils@gnu.org; Wed, 19 Jul 2017 13:32:44 -0400
Received: from smtp-out-so.shaw.ca ([64.59.136.139]:59449)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)
	(Exim 4.71) (envelope-from <962-396-1872@kylheku.com>)
	id 1dXsq1-0007VO-MW
	for coreutils@gnu.org; Wed, 19 Jul 2017 13:32:41 -0400
Received: from kylheku.com ([70.79.163.252]) by shaw.ca with SMTP
	id XspxdnuaUDJTWXspydvNz8; Wed, 19 Jul 2017 11:32:39 -0600
X-Authority-Analysis: v=2.2 cv=B4DJ6KlM c=1 sm=1 tr=0
	a=95A0EdhkF1LMGt25d7h1IQ==:117 a=95A0EdhkF1LMGt25d7h1IQ==:17
	a=IkcTkHD0fZMA:10 a=SMorJkV_YP8A:10 a=G3gG6ho9WtcA:10
	a=dE9UyeI9sgApHK3XWr8A:9 a=QEXdDO2ut3YA:10
Received: from www-data by kylheku.com with local (Exim 4.72)
	(envelope-from <962-396-1872@kylheku.com>)
	id 1dXspx-0002ij-Bw; Wed, 19 Jul 2017 10:32:37 -0700
To: Lance E Sloan <sloanlance+coreutils_gnu.org@gmail.com>
Subject: Re: coreutils feature =?UTF-8?Q?requests=3F?=
X-PHP-Originating-Script: 501:rcmail.php
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8;
 format=flowed
Content-Transfer-Encoding: quoted-printable
Date: Wed, 19 Jul 2017 10:32:37 -0700
From: "Kaz Kylheku (Coreutils)" <962-396-1872@kylheku.com>
Cc: coreutils@gnu.org
In-Reply-To: <CAJecg5qzYtikTFuJbk6V02nqWdyjsu_4zwj65MWQMtnO_Cfszg@mail.gmail.com>
References: <CAJecg5pGpQU_CZ-AdMtA88fzGs7RPd+dHG__AeYTvergm=iKFA@mail.gmail.com>
	<ed6ef41b902c4a37f5c78e8f791d23f6@mail.kylheku.com>
	<CAJecg5r6U4X-O2J5x_HSExSEy19Skksyw7jrEdwASabjaind_w@mail.gmail.com>
	<BN6PR2001MB107464DC3D9DF5CB83103EA7F1A60@BN6PR2001MB1074.namprd20.prod.outlook.com>
	<CA+MRLqnRXjrOmqObVMObBhDUo5uRZAUMo=vOuE=tKOWSZMk2ZQ@mail.gmail.com>
	<BN6PR2001MB1074D25959ADEB88B732CBA9F1A60@BN6PR2001MB1074.namprd20.prod.outlook.com>
	<2fa5a4be-7ba9-3257-713f-b1e907e03489@redhat.com>
	<CAJecg5qzYtikTFuJbk6V02nqWdyjsu_4zwj65MWQMtnO_Cfszg@mail.gmail.com>
Message-ID: <e4af79035112adaf5ee5b0c38ab91d85@mail.kylheku.com>
X-Sender: 962-396-1872@kylheku.com
User-Agent: Roundcube Webmail/0.9.2
X-CMAE-Envelope: MS4wfMV1z010exLznQORpNokH4vjoaK76dd5oazDTE0iQxPMPivsHJ3WEyCA3XzGiO0VJ26d1hHPjwu0O4HYvJtpcHq88TfN5YOtQyXYbWrhKVPxp+kJJpVk
	Kqr/Chiw04ZyTFoVq9mkzrEF2HalMXnTHqAgsmMyUkcAR/ew2bo9XigCcBNha2W3nXeUFlPHzVOzyZMUn2dJYL4qoavajCy3gRfNzP2iFtpqhL/Tm0Non6Qj
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy]
X-Received-From: 64.59.136.139
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Wed, 19 Jul 2017 17:32:46 -0000

On 19.07.2017 10:03, Lance E Sloan wrote:
> With regard to your objection to a special environment variable: I=20
> agree.
> I didn't feel strongly about it at first, but I was leaning towards not
> implementing env. var. support for this.  It just didn't feel right.  I
> have written programs that use env. var. to specify or override default
> options.  However, your point about adding this to an existing,=20
> established
> program where it could possibly cause variable conflicts is enough to=20
> set
> my option a little more strongly against an env. var. for this purpose.

My own objection to such influencing env-vars is rooted in
"global variables are bad"; i.e. the inherited wisdom from decades
of computer science and software engineering practice.

Say you want to combine code from two or more scripts, all of which
want to configure the same tools in different ways via globals. Now
you have a mess of saving and restoring the global so that it
has the value that each bit of code expects when that code is running.

With magic globals we can't look at a line of code and know exactly
what it will do, without reasoning about what are the current values
of the globals. That reasoning may be intractable: i.e. situations in
which it simply cannot be known. Then the code has to ensure that the
variables have certain values by assigning them. Except, it can't
just assign because that will influence some other code; it has to
save the prior values, then assign, then restore when done.

Of course environment variables can be scoped:

    OPT=3Dvalue command

but that's local syntax now, which defeats the purpose of the
"action at a distance" effect of the variable; it might as well be
transliterated to:

    command --opt=3Dvalue

and there we are.





From MAILER-DAEMON Wed Jul 19 13:43:59 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dXt0x-0006YB-GM
	for mharc-coreutils@gnu.org; Wed, 19 Jul 2017 13:43:59 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:43512)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <962-396-1872@kylheku.com>) id 1dXt0v-0006Y4-Q1
	for coreutils@gnu.org; Wed, 19 Jul 2017 13:43:58 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <962-396-1872@kylheku.com>) id 1dXt0s-0000Vi-Mc
	for coreutils@gnu.org; Wed, 19 Jul 2017 13:43:57 -0400
Received: from smtp-out-no.shaw.ca ([64.59.134.13]:40559)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)
	(Exim 4.71) (envelope-from <962-396-1872@kylheku.com>)
	id 1dXt0s-0000V3-Gg
	for coreutils@gnu.org; Wed, 19 Jul 2017 13:43:54 -0400
Received: from kylheku.com ([70.79.163.252]) by shaw.ca with SMTP
	id Xt0ndRTHoM9gtXt0odCBBT; Wed, 19 Jul 2017 11:43:52 -0600
X-Authority-Analysis: v=2.2 cv=a+JAzQaF c=1 sm=1 tr=0
	a=95A0EdhkF1LMGt25d7h1IQ==:117 a=95A0EdhkF1LMGt25d7h1IQ==:17
	a=IkcTkHD0fZMA:10 a=SMorJkV_YP8A:10 a=G3gG6ho9WtcA:10
	a=gqlWSQy-Crgl6Mjm7NUA:9 a=QEXdDO2ut3YA:10
Received: from www-data by kylheku.com with local (Exim 4.72)
	(envelope-from <962-396-1872@kylheku.com>)
	id 1dXt0m-0002xi-Vx; Wed, 19 Jul 2017 10:43:49 -0700
To: Eric Blake <eblake@redhat.com>
Subject: Re: coreutils feature =?UTF-8?Q?requests=3F?=
X-PHP-Originating-Script: 501:rcmail.php
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8;
 format=flowed
Content-Transfer-Encoding: quoted-printable
Date: Wed, 19 Jul 2017 10:43:48 -0700
From: "Kaz Kylheku (Coreutils)" <962-396-1872@kylheku.com>
Cc: Lance E Sloan <sloanlance+coreutils_gnu.org@gmail.com>, coreutils@gnu.org
In-Reply-To: <cdfebfa6-7475-1bda-6c85-2267b775f381@redhat.com>
References: <CAJecg5pGpQU_CZ-AdMtA88fzGs7RPd+dHG__AeYTvergm=iKFA@mail.gmail.com>
	<ed6ef41b902c4a37f5c78e8f791d23f6@mail.kylheku.com>
	<CAJecg5r6U4X-O2J5x_HSExSEy19Skksyw7jrEdwASabjaind_w@mail.gmail.com>
	<BN6PR2001MB107464DC3D9DF5CB83103EA7F1A60@BN6PR2001MB1074.namprd20.prod.outlook.com>
	<CA+MRLqnRXjrOmqObVMObBhDUo5uRZAUMo=vOuE=tKOWSZMk2ZQ@mail.gmail.com>
	<BN6PR2001MB1074D25959ADEB88B732CBA9F1A60@BN6PR2001MB1074.namprd20.prod.outlook.com>
	<2fa5a4be-7ba9-3257-713f-b1e907e03489@redhat.com>
	<CAJecg5qzYtikTFuJbk6V02nqWdyjsu_4zwj65MWQMtnO_Cfszg@mail.gmail.com>
	<cdfebfa6-7475-1bda-6c85-2267b775f381@redhat.com>
Message-ID: <96813fa891560b117b94e4bdcbef8c07@mail.kylheku.com>
X-Sender: 962-396-1872@kylheku.com
User-Agent: Roundcube Webmail/0.9.2
X-CMAE-Envelope: MS4wfBgyFu/vTHLfuNIm2FJctrh+hdoGepwwf+zdfSEsusSuMErt3ydhA3QtP2w2vl1qGW7kmQzN6kUC71/K5hf3LG/3A2rjW8rwdVYKwUia9M42C+FmVriu
	QWNjuCE/hm6wa2C4/HcbdI0wWjE+9FXzofez/oJwd6n5szZ3K7hxnK1w4/PYiPDo+KAX64VicVwmSMpn10dPPHFJnglbimVdxGfqEJX3mMLS24ahjpqatSAn
	ofprlmhglugud5uDBffzCw==
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy]
X-Received-From: 64.59.134.13
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Wed, 19 Jul 2017 17:43:58 -0000

On 19.07.2017 10:30, Eric Blake wrote:
> On 07/19/2017 12:03 PM, Lance E Sloan wrote:
>> Hi, Eric.
>>=20
>> Thank you for the thoughtful response.  I regret that I have trouble
>> understanding your point of view, though.  Please know that I do not=20
>> mean
>> any disrespect.  I'd appreciate it if you could explain why you're=20
>> opposed
>> to adding new features to cut (or to comm).
>=20
> I'm not opposed to well-justified new features.  It's just that the bar
> for justifying new features is rather high (it's a lot of code to add=20
> to
> get field reordering, and that code has to be tested; it is also a
> question of how many users will rely on that extension.  A testsuite
> addition is mandatory as part of adding the new feature, if the new
> feature is worth adding).

It is nontrivial code. For instance if we look at how the function
cut_bytes works in the implementation, what it's doing is simply
doing a getchar() from the stream, and querying a data structure
to determine whether the byte should be printed or not.
(That data structure consists of a pointer which marches through
field range descriptors in parallel with going through the data.)

cut_fields is more complicated due to the delimiting of fields,
but essentially the same overall approach.

Basically, printing of fields that isn't sorted and de-duplicated
is a rewrite of all parts of the utility other than command
line processing and the printing of usage help text.

It's like two different programs in one, sharing a minimal
skeleton.



From MAILER-DAEMON Wed Jul 19 13:53:50 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dXtAU-0000TV-0E
	for mharc-coreutils@gnu.org; Wed, 19 Jul 2017 13:53:50 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:45464)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <assafgordon@gmail.com>) id 1dXtAQ-0000EL-In
	for coreutils@gnu.org; Wed, 19 Jul 2017 13:53:48 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <assafgordon@gmail.com>) id 1dXtAO-0005rk-LO
	for coreutils@gnu.org; Wed, 19 Jul 2017 13:53:46 -0400
Received: from mail-io0-x22d.google.com ([2607:f8b0:4001:c06::22d]:34220)
	by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)
	(Exim 4.71) (envelope-from <assafgordon@gmail.com>)
	id 1dXtAO-0005re-EL
	for coreutils@gnu.org; Wed, 19 Jul 2017 13:53:44 -0400
Received: by mail-io0-x22d.google.com with SMTP id l7so3300885iof.1
	for <coreutils@gnu.org>; Wed, 19 Jul 2017 10:53:44 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
	h=mime-version:in-reply-to:references:from:date:message-id:subject:to
	:cc; bh=FoEh7zmnWyGTHRppMgJAoPF3Lzq9TA6JmDjuel48ltU=;
	b=hu54vdsPg5MY9SYlheTl/Kk7k1CP6T8UASg5CtiFilv1ZVAm2JnbXKZBMlzgcLn6J9
	RJ+93XCZeVm8e26apUgbjRptA3+U3CR26715oZetz5br/wClOx5qIC/mXKldzf4OTCEr
	sFrqgzCM1NyQ2wEtPPHzca3y3v/iEhgGq0sZDRsV7T546lyLZ2SzHIbWIaNcAe7P4Lks
	1/h4gw0Kui6HENP2h+Xjq6j+jx53hLZTZnQuPH//KD6VcSvKMKOYOCAQ8fdZCFRm9kJV
	GPrLQ5PrmoJboit2Llj33JGrwWyTisyRb0GuZ6975/sAEovFW1zY6kovXiKT4QmIfmvG
	KMZw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
	d=1e100.net; s=20161025;
	h=x-gm-message-state:mime-version:in-reply-to:references:from:date
	:message-id:subject:to:cc;
	bh=FoEh7zmnWyGTHRppMgJAoPF3Lzq9TA6JmDjuel48ltU=;
	b=IzKphMEGhBnHUJ+vbumf2WYLkQFh0vkLUnQ+G0PEPNA1Mw85eB9hyGe13uABZNwgNw
	evIBsp1Lb+f+hm4LSEvJ8c0WigIGCZMC54QpKdhdQl8eeESgwgMIs/dDr6ZB5Gr31l9N
	ihEQ8QlZm7cMX00jC3vfj+5q8eLcuaKD//R/3Yha8G2284cC8NhWDyCB6gIq2QcGck0X
	5SAZwwtqVK887XsGBIJb0B56QK3ZGZh3bmg7J1dZ0Ua9VxVSlPHboOIWG9JxOi5OuDeB
	GnhkYliFQYnDc9eSpYDbNdK4l7qG7il9PKaUmetQwfY7w5KggRjncogvwm5t/agcFmJ4
	Ylwg==
X-Gm-Message-State: AIVw1131xkg20gF724jgVNl8Mk7j1+c8RrqIXH7HhI9J60RvHO6Tqzbm
	xu30hJMd9OaooVz6AY16T3jgbskpCQ==
X-Received: by 10.107.156.142 with SMTP id f136mr1031570ioe.240.1500486823360; 
	Wed, 19 Jul 2017 10:53:43 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.79.37.13 with HTTP; Wed, 19 Jul 2017 10:53:42 -0700 (PDT)
In-Reply-To: <CAJecg5qzYtikTFuJbk6V02nqWdyjsu_4zwj65MWQMtnO_Cfszg@mail.gmail.com>
References: <CAJecg5pGpQU_CZ-AdMtA88fzGs7RPd+dHG__AeYTvergm=iKFA@mail.gmail.com>
	<ed6ef41b902c4a37f5c78e8f791d23f6@mail.kylheku.com>
	<CAJecg5r6U4X-O2J5x_HSExSEy19Skksyw7jrEdwASabjaind_w@mail.gmail.com>
	<BN6PR2001MB107464DC3D9DF5CB83103EA7F1A60@BN6PR2001MB1074.namprd20.prod.outlook.com>
	<CA+MRLqnRXjrOmqObVMObBhDUo5uRZAUMo=vOuE=tKOWSZMk2ZQ@mail.gmail.com>
	<BN6PR2001MB1074D25959ADEB88B732CBA9F1A60@BN6PR2001MB1074.namprd20.prod.outlook.com>
	<2fa5a4be-7ba9-3257-713f-b1e907e03489@redhat.com>
	<CAJecg5qzYtikTFuJbk6V02nqWdyjsu_4zwj65MWQMtnO_Cfszg@mail.gmail.com>
From: Assaf Gordon <assafgordon@gmail.com>
Date: Wed, 19 Jul 2017 13:53:42 -0400
Message-ID: <CAHN-GmwuLLHAE9yVKoruMPeQ2zbSwsWm4MAqcqnHr5Qwx+Nmkw@mail.gmail.com>
Subject: Re: coreutils feature requests?
To: Lance E Sloan <sloanlance+coreutils_gnu.org@gmail.com>
Cc: Eric Blake <eblake@redhat.com>, "coreutils@gnu.org" <coreutils@gnu.org>
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 2607:f8b0:4001:c06::22d
Content-Type: text/plain; charset="UTF-8"
X-Content-Filtered-By: Mailman/MimeDel 2.1.21
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Wed, 19 Jul 2017 17:53:48 -0000

Hello Lance and all,

On Jul 19, 2017, at 13:03, Lance E Sloan <sloanlance+coreutils_gnu.org@
gmail.com> wrote:

I'd appreciate it if you could explain why you're opposed
to adding new features to cut (or to comm).


If I may chime in about this:

There is also a delicate balance between adding more features and leading
to bloated software, and keeping the program lean but providing less
functionality.

Sometimes, it boils down to a judgement call of the maintainers.

And then there's also the unix philosophy of having a tool "do one thing
and do it well".

It may help if I explain my point of view.


I think that what would help the most is if you can share
an actual problem that you have (i.e. the input and desired output),
and perhaps we can find a good solution using existing tools.

My considerations for a solution:

1.  I need this feature to process several files that have millions of
lines each.  I need to do this on an ongoing, periodic basis.  I can't
afford for the process to be slow.


Here's a concrete example:

===

$ time wc -l 1.txt
18833902 1.sql

real    0m0.442s
user    0m0.232s
sys     0m0.208s

$ time cut -f1,3,5 1.txt > /dev/null

real    0m2.923s
user    0m2.736s
sys     0m0.188s

$ time mawk '{print $3,$1,$5}' 1.txt > /dev/null

real    0m4.903s
user    0m4.680s
sys     0m0.224s

====

Using existing tools ('mawk' in this case)
gives you all of awk's flexibility, at a slight increase of cost.
The example file had 18M lines - and we're still talking about just 4
seconds
of user time.


2.  Since I have a large amount of data, I'm avoiding regular expressions
and interpreted languages, which take longer to complete the job.  That
eliminates awk and several other possible solutions.  A compiled C
application would be best.


I agree that regex on every line is slow, but awk just for the sake
of reordering lines will not require any regex.

4. Part of my data processing uses jq.  I've figured out how to do this
field reordering with it, but it makes my jq filter more complex and more
difficult for my successors to maintain.  As written on
https://stedolan.github.io/jq/ , "jq is like sed for JSON data".  I don't
consider sed to be a good solution for a problem of this size, so jq
probably isn't ideal, either.


(disclaimer: I'm a maintainer of GNU sed, and I've also contributed code to
jq).

This point confuses me a bit:
cut,awk,sed are all line-based tools: meaning your logical "records"
are expected to be on one line (at least - that is the most common usage of
the tools).

jq is JSON based - it doesn't at all need records to be contained in a
single line
(though it can do it with optional arguments).

Is your input file "one JSON record per line" ?
Or are you using 'jq' to read non-JSON input and treat it as an array?

In any case: 'jq' implements a small virtual machine to execute your script,
I'm not sure it would be the fastest tool for the job (or much faster
than sed/awk's implantation). It is certainly an "interpreted language"
which you wrote above you are trying to avoid.

Similarly, saying "I don't consider sed to be a good solution" -
you haven't yet told us what your actual need is.
So we can't tell if sed is good or bad...

Since a C implementation should run the fastest and cut from GNU's
coreutils is written in C and presumably doesn't need much work to support
this, it seems like the best solution.

Even if this feature suggestion isn't approved by the GNU community, I will
implement it for my own use anyway.  I can enjoy the new functionality
(which I think should have been added to cut long ago) and keep it to
myself or I can contribute it back to the online community.  I could
distribute it as my own fork of GNU coreutils or as a patch to it.
However, if it were merged into GNU's coreutils, it would get the most
exposure and be helpful to more people.


cut's implementation is optimized for cutting columns and
not for reordering them.
I think that if you try to add code to 'cut' that allows reordering
of output fields, you'll discover that while it's very doable,
it also significantly complicates the code.

A previous message on this thread stipulated that it takes
extra effort to 'sort the columns' - that is incorrect for the current
implementation.

Regardless,
If you actually implement it - please do send the patch.
To be considered for inclusion, it will need to be efficient (i.e. not make
'cut' slower
than it is now), be correctly implemented for all sorts of edge-cases,
and have good tests that cover the new functionality.
Updating manual pages and the documentation is needed as well.
You'll also need to assign copyright of the patch to the FSF.

Good places to start:
http://git.savannah.gnu.org/cgit/coreutils.git/tree/HACKING
http://git.savannah.gnu.org/cgit/coreutils.git/tree/README-hacking

Here's an example of a patch that added a new feature to 'comm':
https://git.savannah.gnu.org/cgit/coreutils.git/commit/?id=
b50a151346c42816034b5c26266eb753b7dbe737
You can see that the kind of changes at accompany a new feature.




Then again,
Given that the canonical way to reorder columns for many decades has been:
awk '{print $9,$2,$6}'
and that this canonical way would 'just works'
on *any* existing posix system (think: every BSD, Solaris, AIX,
and systems such as AlpineLinux which use BusyBox instead of GNU coreutils)
-
there is a very high barrier to adding such non-standard feature.

regards,
- assaf

From MAILER-DAEMON Wed Jul 19 13:56:17 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dXtCr-0001l3-I2
	for mharc-coreutils@gnu.org; Wed, 19 Jul 2017 13:56:17 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:46085)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <dualbus@gmail.com>) id 1dXtCo-0001ku-TT
	for coreutils@gnu.org; Wed, 19 Jul 2017 13:56:15 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <dualbus@gmail.com>) id 1dXtCo-0007Dt-2y
	for coreutils@gnu.org; Wed, 19 Jul 2017 13:56:14 -0400
Received: from mail-qk0-x234.google.com ([2607:f8b0:400d:c09::234]:33676)
	by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)
	(Exim 4.71) (envelope-from <dualbus@gmail.com>) id 1dXtCn-0007C3-VM
	for coreutils@gnu.org; Wed, 19 Jul 2017 13:56:14 -0400
Received: by mail-qk0-x234.google.com with SMTP id p126so2189661qkf.0
	for <coreutils@gnu.org>; Wed, 19 Jul 2017 10:56:12 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
	h=mime-version:in-reply-to:references:from:date:message-id:subject:to
	:cc:content-transfer-encoding;
	bh=mV5rfbjEZy19pXQX1jKJ2bbCOIwOs3q+Y8T6UfHAlOw=;
	b=k9MjHVd7gI88T9yyjZ/xOU6XWCzusHSCVXDbUs3AhqAVIIXui15hnjtbQ3ZYp9/tOu
	Nr1xqqc4fac70e14GUf3Fiyp9ci7sdyp6oB+f6iZRjWgA/Ng4p1gke/KX36C7Qk/55a7
	fj2TPyaZbo+hJ8D8pp+McGpOlUJLrqNFwor+2ix/qn+cW53HGZ12qB96g5+y+dbqvDRB
	Bmcuf7GdtS1ygZBA943iMCxETDYQCJnZtm0ytvZb8IOE9gmwkeAQD/G0GwODrqMzqNdU
	cKYL1Jhu7+hMnuvsQ6AEgmXdOPhM+IenNxiL+pHyeQrsiQJOk+QffTrpvu+cGZAffGCO
	sZ/A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
	d=1e100.net; s=20161025;
	h=x-gm-message-state:mime-version:in-reply-to:references:from:date
	:message-id:subject:to:cc:content-transfer-encoding;
	bh=mV5rfbjEZy19pXQX1jKJ2bbCOIwOs3q+Y8T6UfHAlOw=;
	b=Pt60sDEZAfyxcVX980WXUGlq+1Rz5FC2KCd6PvFcYPTpNTZX5qLjVsGvR0Ay5qNatj
	WEKDia5dHbnMUgPkVJe3aBx86L+DrGk6ScAUpd5zrBsOSAz10PHtZcNcV6L/Qhx6Ryzi
	S+gzpeiPlUmQ5DYkZH87TnHJ2smF+UrVVov1sXRHzNmS8VJ7d6fJdsGjWYxjygAlvpAt
	DkOBCXE3VDoge6J9/AeeYpqHcNglKy81+4v1xM6iFOu/GKRTV0rC8ydj+ZH8S0H4hXUW
	CHWq4Ev6Yp+vCr/NOXkCKS7LlpXTclI8GFbbL/eG/DyMIa+czjt4fTZ2xI9dTTT+U7E7
	uVyg==
X-Gm-Message-State: AIVw113MeCfwYIZBVpPNzx2so0c24hcDd+gsrwMPVbJKojJdtdMl73fe
	9/dWdZzXdDdwju9ObOhrPhAZAldWEA==
X-Received: by 10.55.3.130 with SMTP id 124mr1226353qkd.31.1500486971761; Wed,
	19 Jul 2017 10:56:11 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.12.141.204 with HTTP; Wed, 19 Jul 2017 10:55:51 -0700 (PDT)
In-Reply-To: <78f6f80a-bf64-3972-9b25-1cc62ad67aa3@users.sourceforge.net>
References: <af44fed2-a5a2-21a5-a3fa-6982a5d37e1d@users.sourceforge.net>
	<c97927cd182f34e3527fe741cce0b726@mail.kylheku.com>
	<7c44c5c2-e5cd-b368-a8bd-d7dc6dc3de3f@users.sourceforge.net>
	<b6745e3004da59da1e859847ed671d21@mail.kylheku.com>
	<12c091e8-380d-b9c4-5d95-78d2d1f02041@users.sourceforge.net>
	<a7afb68cad095d83c90450700f168aa5@mail.kylheku.com>
	<1cc2666c-2fbb-20ca-f427-e2caf4a5c661@users.sourceforge.net>
	<5442056c-efe3-4198-9f11-b2e0fd63e90c@bernhard-voelker.de>
	<fc6b1b8c-bb26-314d-5160-d5c4c4e7b36f@users.sourceforge.net>
	<28d7c18d-8291-c6a7-d138-958ae6358b50@bernhard-voelker.de>
	<8658ef8b-da51-179e-3897-de60298e957f@users.sourceforge.net>
	<ffda35d0-eca9-beb8-846a-f7f00aace11f@redhat.com>
	<78f6f80a-bf64-3972-9b25-1cc62ad67aa3@users.sourceforge.net>
From: Eduardo Bustamante <dualbus@gmail.com>
Date: Wed, 19 Jul 2017 12:55:51 -0500
Message-ID: <CAOSMAuvN-N0HTRFJdBf_CpzEujN8Cdu15VmctQzirxPfCr8BdA@mail.gmail.com>
Subject: Re: Determination of file lists from selected folders without
	returning directory names
To: SF Markus Elfring <elfring@users.sourceforge.net>
Cc: Eric Blake <eblake@redhat.com>,
	=?UTF-8?Q?Bernhard_V=C3=B6lker?= <mail@bernhard-voelker.de>, 
	Kaz Kylheku <962-396-1872@kylheku.com>, Coreutils <coreutils@gnu.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 2607:f8b0:400d:c09::234
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Wed, 19 Jul 2017 17:56:16 -0000

On Wed, Jul 19, 2017 at 11:45 AM, SF Markus Elfring
<elfring@users.sourceforge.net> wrote:
[...]
> I am curious if any other developers (than me) or system testers care
> for this implementation detail around the topic =E2=80=9CBetter support f=
or
> data processing with basenames=E2=80=9D.
> https://savannah.gnu.org/bugs/index.php?51506

In my opinion, it's counterproductive to debate these kinds of
proposals unless there are *concrete* use cases. So far, this seems to
be "what if someone some day wants to do FOO for whatever reason and
needs it to run super fast, so let's do a bunch of optimizations to
support FOO".

Can you provide a concrete and real-world example of why this is needed?


From MAILER-DAEMON Wed Jul 19 15:03:49 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dXuGD-00072k-2v
	for mharc-coreutils@gnu.org; Wed, 19 Jul 2017 15:03:49 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:59287)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <sloanlance@gmail.com>) id 1dXuGA-000718-RI
	for coreutils@gnu.org; Wed, 19 Jul 2017 15:03:48 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <sloanlance@gmail.com>) id 1dXuG9-0000I3-IJ
	for coreutils@gnu.org; Wed, 19 Jul 2017 15:03:46 -0400
Received: from mail-it0-x229.google.com ([2607:f8b0:4001:c0b::229]:38468)
	by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)
	(Exim 4.71) (envelope-from <sloanlance@gmail.com>)
	id 1dXuG9-0000Hs-Au
	for coreutils@gnu.org; Wed, 19 Jul 2017 15:03:45 -0400
Received: by mail-it0-x229.google.com with SMTP id h199so5418842ith.1
	for <coreutils@gnu.org>; Wed, 19 Jul 2017 12:03:45 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
	h=mime-version:sender:in-reply-to:references:from:date:message-id
	:subject:to:cc;
	bh=0/0zcwY03WP988xl3ouzJ16uqrTrbk5eELeu1+hVSkI=;
	b=lgMLHPJ+najhNIGa6UDJKG8HAIWN0DDMIuAsGDgWhdHRbVolOMRf/aQgiothKzc4I3
	yOBJIj+MfNtBSmvBKsDsErTYqFZx8et2I7IWYaKdGdksovmWxxOvD7U2TlA5AF+YThTD
	BD5Hk5Ho4eg9PDE/62JhBW1lQoYdzfrbps95gH/IOxt6sVfWGZEUgryZ8fyV/FfXW5SJ
	zYYOmHMcDCXj1Kkuv/yyhgKH2o3zwafkfnlNmekB/QAo7KHL8164EWVe6cxwdlhDXTyi
	4ueIGPpFGeXcY7oz2wyvR9RZzFaCnJFKvlxKRhgjnmd7vFRtaMv6XgPEfP6rmfkqysQu
	lVSg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
	d=1e100.net; s=20161025;
	h=x-gm-message-state:mime-version:sender:in-reply-to:references:from
	:date:message-id:subject:to:cc;
	bh=0/0zcwY03WP988xl3ouzJ16uqrTrbk5eELeu1+hVSkI=;
	b=dABSGAFI0BIGjqEovpMy9fhiv7xaCbxrbAQV+8cpWig3qIC4YdcVm4XfVUzVIYGO5C
	8TE113R69iI2Y0pRZv6YjvcyJNIryXsnKX9Agft6K1RujFutdbDtUvC75xhxnoSEhj51
	BtKDadnZdX1szm8xjIUG8a7ojhILMMnkgztOhL2VLYzJKJujG1/qhVKhwcTiC4TkNd5z
	OfkRwtckVeZP872RWuercHERHWVC6twHGdAIXl8vk3FBrkF11K3pcF85tpzapNl5GiAh
	nJOooQAj9c8f2+Rvp4k1F9dZZ7roOhPwXRrAACNyeNNqWkYWX+XbDRJjQTcHofZi7u0J
	m8CA==
X-Gm-Message-State: AIVw111WmDvc7WYV78XbWIhN5ch3/q7YmzT1kuE5RDQsC4yuSJSP+9Lb
	CBbGA2MNrJLNvArA6GrDLdcae27CKw==
X-Received: by 10.36.44.77 with SMTP id i74mr1028519iti.42.1500491024434; Wed,
	19 Jul 2017 12:03:44 -0700 (PDT)
MIME-Version: 1.0
Sender: sloanlance@gmail.com
Received: by 10.107.35.201 with HTTP; Wed, 19 Jul 2017 12:03:03 -0700 (PDT)
In-Reply-To: <BN6PR2001MB1074D25959ADEB88B732CBA9F1A60@BN6PR2001MB1074.namprd20.prod.outlook.com>
References: <CAJecg5pGpQU_CZ-AdMtA88fzGs7RPd+dHG__AeYTvergm=iKFA@mail.gmail.com>
	<ed6ef41b902c4a37f5c78e8f791d23f6@mail.kylheku.com>
	<CAJecg5r6U4X-O2J5x_HSExSEy19Skksyw7jrEdwASabjaind_w@mail.gmail.com>
	<BN6PR2001MB107464DC3D9DF5CB83103EA7F1A60@BN6PR2001MB1074.namprd20.prod.outlook.com>
	<CA+MRLqnRXjrOmqObVMObBhDUo5uRZAUMo=vOuE=tKOWSZMk2ZQ@mail.gmail.com>
	<BN6PR2001MB1074D25959ADEB88B732CBA9F1A60@BN6PR2001MB1074.namprd20.prod.outlook.com>
From: Lance E Sloan <sloanlance+coreutils_gnu.org@gmail.com>
Date: Wed, 19 Jul 2017 15:03:03 -0400
X-Google-Sender-Auth: Sfb7v3bjIN-Okv9zhCUYrrE_LzM
Message-ID: <CAJecg5rBSy2BtQC46+SuhtfNzot2wydh4RVTMyg9pG-34768Xw@mail.gmail.com>
Subject: Re: coreutils feature requests?
To: "Nellis, Kenneth" <Kenneth.Nellis@conduent.com>
Cc: "coreutils@gnu.org" <coreutils@gnu.org>
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 2607:f8b0:4001:c0b::229
Content-Type: text/plain; charset="UTF-8"
X-Content-Filtered-By: Mailman/MimeDel 2.1.21
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Wed, 19 Jul 2017 19:03:48 -0000

Hi, Ken.

I hadn't considered extending the cut reordering feature to bytes and
characters, too.  I'm not opposed to it.  My interest is only in delimited
fields.  If it turns out to be trivial to apply reordering to the other two
selector types, then I guess it should be done.

I don't like the environment variable suggestion, though.  It doesn't feel
like a good idea, but I can't articulate why.  I guess it's mostly because
cut doesn't already use environment variables in any way.  The
implementation would be more complex if env. var. support were a
requirement, whereas I believe the reordering feature is a (hopefully
somewhat simple) modification of cut's current functionality.  Eric Blake's
suggestion to avoid variable conflicts also seems like a good reason not to
support env. var.

When I was first thinking about this feature, I was trying to decide
whether I'd like to add a new "-F" field selector option or a new "-o"
option to specify that the order of the field specification is
significant.  I decided that "-F" wouldn't be good, because existing usage
of cut would need to put it in place of the "-f" option.  However, "-o"
would be an additional option that could be specified programmatically.
Maybe like:

orderedOption=$(cut -f1 -o 2> /dev/null <<< '-o') # set "-o" option only if
it works
cut -d' ' -f4,1,3,2 ${orderedOption} <<< 'another hacker GNU just' # "just
another GNU hacker" if "-o" available

I'd also thought that by adding the "-o" option, in addition to working
with the "-f" option, it could work with "-b" and "-c" as well.


On 19 July 2017 at 09:29, Nellis, Kenneth <Kenneth.Nellis@conduent.com>
wrote:

> From: Steeve McCauley
> > I can't believe I'd never thought of reordering output columns like
> this.
> > FWIW, I agree that another option should be used to prevent issues with
> backward compatibility.
> >
> > $ echo 1,2,3,4,5,6 | cut -d, -f3,5,2
> > 2,3,5
> >
> > $ echo 1,2,3,4,5,6 | cut -d, -f3,5,2 -o
> > 3,5,2
> > Should this be extended to character output as well?
> > echo output | cut -c6,4,2 -o
> > tpu
> >
>
> Absolutely! It would be expected behavior (IMHO). I see no reason not to.
>
> In addition, so that scripts can work across platforms, I (strongly)
> recommend
> that a cut-specific environment variable be defined to allow specifying the
> field ordering behavior. In that way my QNX 4 script (whose cut would balk
> at
> the -o option) would work with Gnu. One possibility:
>
> CUT_OPTIONS=-o
>
> --Ken Nellis
>

From MAILER-DAEMON Wed Jul 19 17:08:26 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dXwCo-000054-On
	for mharc-coreutils@gnu.org; Wed, 19 Jul 2017 17:08:26 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:55786)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1dXwCm-00004x-7O
	for coreutils@gnu.org; Wed, 19 Jul 2017 17:08:25 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1dXwCi-0007t5-Uu
	for coreutils@gnu.org; Wed, 19 Jul 2017 17:08:24 -0400
Received: from mout.kundenserver.de ([217.72.192.75]:55645)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16)
	(Exim 4.71) (envelope-from <mail@bernhard-voelker.de>)
	id 1dXwCi-0007sn-Jv
	for coreutils@gnu.org; Wed, 19 Jul 2017 17:08:20 -0400
Received: from [192.168.101.10] ([91.12.166.234]) by mrelayeu.kundenserver.de
	(mreue102 [212.227.15.183]) with ESMTPSA (Nemesis) id
	0MhlTv-1dJrSD3VGF-00MuUn; Wed, 19 Jul 2017 23:08:02 +0200
Subject: Re: coreutils feature requests?
To: "Kaz Kylheku (Coreutils)" <962-396-1872@kylheku.com>,
	Eric Blake <eblake@redhat.com>
References: <CAJecg5pGpQU_CZ-AdMtA88fzGs7RPd+dHG__AeYTvergm=iKFA@mail.gmail.com>
	<ed6ef41b902c4a37f5c78e8f791d23f6@mail.kylheku.com>
	<CAJecg5r6U4X-O2J5x_HSExSEy19Skksyw7jrEdwASabjaind_w@mail.gmail.com>
	<BN6PR2001MB107464DC3D9DF5CB83103EA7F1A60@BN6PR2001MB1074.namprd20.prod.outlook.com>
	<CA+MRLqnRXjrOmqObVMObBhDUo5uRZAUMo=vOuE=tKOWSZMk2ZQ@mail.gmail.com>
	<BN6PR2001MB1074D25959ADEB88B732CBA9F1A60@BN6PR2001MB1074.namprd20.prod.outlook.com>
	<2fa5a4be-7ba9-3257-713f-b1e907e03489@redhat.com>
	<CAJecg5qzYtikTFuJbk6V02nqWdyjsu_4zwj65MWQMtnO_Cfszg@mail.gmail.com>
	<cdfebfa6-7475-1bda-6c85-2267b775f381@redhat.com>
	<96813fa891560b117b94e4bdcbef8c07@mail.kylheku.com>
Cc: coreutils@gnu.org, Lance E Sloan <sloanlance+coreutils_gnu.org@gmail.com>
From: Bernhard Voelker <mail@bernhard-voelker.de>
Message-ID: <d3ea3008-e547-57ce-8dce-b3c1004cf7c7@bernhard-voelker.de>
Date: Wed, 19 Jul 2017 23:08:00 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101
	Thunderbird/45.6.0
MIME-Version: 1.0
In-Reply-To: <96813fa891560b117b94e4bdcbef8c07@mail.kylheku.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Provags-ID: V03:K0:wMIHbyTiWG06mOAzSofdqhDUHJvBbB0LhUsqFy/hxtZ0z+uyogo
	lO3ewO15tKGRWniF5eG+tcf+wVv42d+JWOKYwCTAWeVw3osgd42xEQxJL5HkZgEdhrr3FRz
	/IfOdpjLhX7ASXlaXCll74nEgg9HeqjLTjVHhVmIb2StgQunH2jAr1+JYxWTqEZPl+Baq6p
	cG+CbwPFOqUpEUF/GG9Hw==
X-UI-Out-Filterresults: notjunk:1;V01:K0:CI3/UY9J5uY=:Xv7GRj2bAKkjmJClJyYQa0
	tdYkg80/OAm58kVD4A1x+ZOOnUB7Ev3iGNLQ1slerDgAxWSnjIt7BrXGjXIG3xLZn9hi0ipUu
	0O3rQssuasPFBcBmZoPSJ9vfRT9763Cpaa7XlkdKHy7dQzQHlo+MEHeD98BvybBNkyS7d39n+
	ZTMug+3Jol1mijJCArmjgUqRwHYDqorJ/M4AfjLP6BHmZs1ttn+fdhJ8Mtt3qzIGo7KeHRGqe
	wUyiexNgrRUl9ApigK2DYT8tXZhKBnsU0o7tgoG8wNYishfC249S7WdVw79gfKGG53b0B9xvX
	ZdqXsdjmmWH//VcSw7vP4pOz6jPdSxGiyeNgEvkT3MW/WGL6Ar/l6j1NO5bN1Nx2bKaezIgWj
	HjxQD6oI6F9oNTb2xTBE0yiUjsyWGfCy6eduwKXDGR/hZvNQXcrjf+pegfliyDM1aDPa4TMF3
	9rrsmmGC9l8BHe7xIR/azwiRh1oVgnUVNz1vUV35029W7dIck0DcD3Vfys5rkEoDwl2HP6tMj
	xcN0BhN25tp9X+84RqR5lVshhq4fD9pKfC/Uxv3RRdJYDfu7rWFDJfDQ2zh9DSO2nivTL3GA0
	mle23VVqZN9cl2yrTaoaHZE1wyA0QljNaHxI7EQU2sYYXiNjOhq6yZzeaj9bsAUR/ia8/DFJd
	FQ6PsC9ZF/n3O3vgNoB5c+wseOv6XTZgTeTmhSq3+ruR4SN8UCArl7F2gs+KZg9kErNxEfZPq
	9nVGIjEIao+6FwrJ608g4cAWVo5sTrEis0EI97JbyQc6xf3KiJIJgz3/nzU=
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
	[fuzzy]
X-Received-From: 217.72.192.75
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Wed, 19 Jul 2017 21:08:25 -0000

On 07/19/2017 07:43 PM, Kaz Kylheku (Coreutils) wrote:
> It is nontrivial code. For instance if we look at how the function
> cut_bytes works in the implementation, what it's doing is simply
> doing a getchar() from the stream, and querying a data structure
> to determine whether the byte should be printed or not.
> (That data structure consists of a pointer which marches through
> field range descriptors in parallel with going through the data.)
> 
> cut_fields is more complicated due to the delimiting of fields,
> but essentially the same overall approach.
> 
> Basically, printing of fields that isn't sorted and de-duplicated
> is a rewrite of all parts of the utility other than command
> line processing and the printing of usage help text.
> 
> It's like two different programs in one, sharing a minimal
> skeleton.

+1

Another point: it is already documented that cut(1) output is
never good for reordering:

http://git.sv.gnu.org/cgit/coreutils.git/tree/doc/coreutils.texi?id=545f181f4e#n5938

  Note @command{awk} supports more sophisticated field processing,
  like reordering fields, and handling fields aligned with blank characters.
  By default @command{awk} uses (and discards) runs of blank characters
  to separate fields, and ignores leading and trailing blanks.
  @example
  @verbatim
  awk '{print $2}'      # print the second field
  awk '{print $(NF-1)}' # print the penultimate field
  awk '{print $2,$1}'   # reorder the first two fields
  @end verbatim
  @end example
  Note while @command{cut} accepts field specifications in
  arbitrary order, output is always in the order encountered in the file.

and even more: it suggests to use join:

  In the unlikely event that @command{awk} is unavailable,
  one can use the @command{join} command, to process blank
  characters as @command{awk} does above.
  @example
  @verbatim
  join -a1 -o 1.2     - /dev/null # print the second field
  join -a1 -o 1.2,1.1 - /dev/null # reorder the first two fields
  @end verbatim
  @end example

Is this sufficient?

Have a nice day,
Berny


From MAILER-DAEMON Wed Jul 19 19:42:37 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dXyc1-0005mu-ON
	for mharc-coreutils@gnu.org; Wed, 19 Jul 2017 19:42:37 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:35134)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <coreutils@tlinx.org>) id 1dXybz-0005ks-D3
	for coreutils@gnu.org; Wed, 19 Jul 2017 19:42:36 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <coreutils@tlinx.org>) id 1dXybw-0002Bt-8w
	for coreutils@gnu.org; Wed, 19 Jul 2017 19:42:35 -0400
Received: from ishtar.tlinx.org ([173.164.175.65]:33642
	helo=Ishtar.sc.tlinx.org)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)
	(Exim 4.71) (envelope-from <coreutils@tlinx.org>) id 1dXybv-00029y-PI
	for coreutils@gnu.org; Wed, 19 Jul 2017 19:42:32 -0400
Received: from [192.168.3.12] (Athenae [192.168.3.12])
	by Ishtar.sc.tlinx.org (8.14.7/8.14.4/SuSE Linux 0.8) with ESMTP id
	v6JNgKLR038766; Wed, 19 Jul 2017 16:42:22 -0700
Message-ID: <596FEE5C.1020909@tlinx.org>
Date: Wed, 19 Jul 2017 16:42:20 -0700
From: L A Walsh <coreutils@tlinx.org>
User-Agent: Thunderbird
MIME-Version: 1.0
To: "Kaz Kylheku (Coreutils)" <962-396-1872@kylheku.com>
CC: Lance E Sloan <sloanlance+coreutils_gnu.org@gmail.com>, coreutils@gnu.org
Subject: ENVVars Removal and functional replacements (was Re: coreutils feature
	requests?)
References: <CAJecg5pGpQU_CZ-AdMtA88fzGs7RPd+dHG__AeYTvergm=iKFA@mail.gmail.com>	<ed6ef41b902c4a37f5c78e8f791d23f6@mail.kylheku.com>	<CAJecg5r6U4X-O2J5x_HSExSEy19Skksyw7jrEdwASabjaind_w@mail.gmail.com>	<BN6PR2001MB107464DC3D9DF5CB83103EA7F1A60@BN6PR2001MB1074.namprd20.prod.outlook.com>	<CA+MRLqnRXjrOmqObVMObBhDUo5uRZAUMo=vOuE=tKOWSZMk2ZQ@mail.gmail.com>	<BN6PR2001MB1074D25959ADEB88B732CBA9F1A60@BN6PR2001MB1074.namprd20.prod.outlook.com>	<2fa5a4be-7ba9-3257-713f-b1e907e03489@redhat.com>	<CAJecg5qzYtikTFuJbk6V02nqWdyjsu_4zwj65MWQMtnO_Cfszg@mail.gmail.com>
	<e4af79035112adaf5ee5b0c38ab91d85@mail.kylheku.com>
In-Reply-To: <e4af79035112adaf5ee5b0c38ab91d85@mail.kylheku.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x (no
	timestamps) [generic] [fuzzy]
X-Received-From: 173.164.175.65
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Wed, 19 Jul 2017 23:42:36 -0000

Kaz Kylheku (Coreutils) wrote:
> On 19.07.2017 10:03, Lance E Sloan wrote:
>> With regard to your objection to a special environment variable: ..
> My own objection to such influencing env-vars is rooted in
> "global variables are bad"; i.e. the inherited wisdom from decades
> of computer science and software engineering practice.
----
    Sorry to interject here, but I have to point out the evils
"that everyone knows" about getting rid of ENV vars.
I don't believe you should put comments about ENV vars in the same
sentence as "inherited Wisdom" or global variables (that are
hidden in binaries).  I would say that most of the universe
would be very inconvenienced were ENV vars to be removed, and that
doing so, tool-by-tool is a way to generate workarounds that
are far worse. 

    Look at use of env vars for HOME, LOCALE support, COLOR preferences,
flags (CFLAGS, CFLAGS) various tools default options... the list is
endless.  99% of the problems of ENV vars are caused by bad design, not
the Env vars.  Several examples of that are applying ENV vars that are
really meant for *interactive use*, to run-time use all the time.

    Any script that people think has a problem w/ENV vars can use
"env -i SCRIPT".  That clears all Env vars.  If you want to
remove all ENV vars starting with LESS, you can just as easily
do that:
env -u ${!LESS*} SCRIPT

The problem comes with those who don't know how to use env
vars and those too lazy to be troubled to clean the env to
what they want.

As soon as you show replacements for things like PATH, HOME, etc,
that work as defaults for all utils, I might agree, but the gnu
utils are becoming more broken over time as they disallow setting
up defaults for a given-machine's interactive use that don't
just work in some shell.

Alias and shell funcs aren't the answer, as they are not referenced
when you do something as as simple as "sudo xxyz", or similar. 
You *can* configure sudo to _allow_ ENV vars to be propagated,
but you can't do that with aliases.  Nether can you reliably
have functions work with sudo -- early version, yes, and late versions,
yes, but that's not exactly reliable -- and they don't work
at all unless the command goes through a function-compatible
shell.


FWIW, you remove env vars, and you end up with people doing things
like:

mv /usr/bin/CMD /usr/bin/CMD.bin
echo -e "#!/bin/sh\n[set ENVVARS & CMD flags]\n
        exec -a CMD CMD.bin -newflags\n" >CMD
+x CMD

I count 9 of those usages in my /usr/bin dir.  Only 3 of them
are ones I added, the rest are added by distro's to re-invent
Env vars.

Those are a sample of a new nightmare being created by the
ENV-var/config-file naysayers -- a far more toxic solution IMO,
as they aren't centralized.

Perhaps centralizing, at least for linux, would be using
an overlay file system on top of existing binary directories
that overlay shell scripts on top of binaries that don't
allow defaults to be configured with them calling the
real binaries via a hidden path (/usr/.bin/...).  That might
almost be reasonable, but certainly a pain, since there's
no way to reliable get at the real binary.

But such is the legacy of removing ENV vars to allow defaults --
even if those are limited to interactive execution.

Linda










From MAILER-DAEMON Thu Jul 20 03:28:04 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dY5sR-0002HS-Ph
	for mharc-coreutils@gnu.org; Thu, 20 Jul 2017 03:28:03 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:47699)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <auerswal@unix-ag.uni-kl.de>) id 1dY5sP-0002Fa-5P
	for coreutils@gnu.org; Thu, 20 Jul 2017 03:28:01 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <auerswal@unix-ag.uni-kl.de>) id 1dY5sO-0006Su-DD
	for coreutils@gnu.org; Thu, 20 Jul 2017 03:28:01 -0400
Received: from mailgw1.uni-kl.de ([2001:638:208:120::220]:44609)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)
	(Exim 4.71) (envelope-from <auerswal@unix-ag.uni-kl.de>)
	id 1dY5sO-0006QP-6s
	for coreutils@gnu.org; Thu, 20 Jul 2017 03:28:00 -0400
Received: from sushi.unix-ag.uni-kl.de (sushi.unix-ag.uni-kl.de
	[IPv6:2001:638:208:ef34:0:ff:fe00:65])
	by mailgw1.uni-kl.de (8.14.4/8.14.4/Debian-8+deb8u2) with ESMTP id
	v6K7CGTP028456
	(version=TLSv1/SSLv3 cipher=ECDHE-RSA-AES256-SHA bits=256 verify=NOT); 
	Thu, 20 Jul 2017 09:12:16 +0200
Received: from sushi.unix-ag.uni-kl.de (ip6-localhost [IPv6:::1])
	by sushi.unix-ag.uni-kl.de (8.14.4/8.14.4/Debian-4+deb7u1) with ESMTP
	id v6K7CFkQ021976
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256
	verify=NO); Thu, 20 Jul 2017 09:12:15 +0200
Received: (from auerswal@localhost)
	by sushi.unix-ag.uni-kl.de (8.14.4/8.14.4/Submit) id v6K7CFFX021975;
	Thu, 20 Jul 2017 09:12:15 +0200
Date: Thu, 20 Jul 2017 09:12:15 +0200
From: Erik Auerswald <auerswal@unix-ag.uni-kl.de>
To: Lance E Sloan <sloanlance+coreutils_gnu.org@gmail.com>
Cc: Eric Blake <eblake@redhat.com>, "coreutils@gnu.org" <coreutils@gnu.org>
Subject: Re: coreutils feature requests?
Message-ID: <20170720071215.GB21350@unix-ag.uni-kl.de>
References: <CAJecg5pGpQU_CZ-AdMtA88fzGs7RPd+dHG__AeYTvergm=iKFA@mail.gmail.com>
	<ed6ef41b902c4a37f5c78e8f791d23f6@mail.kylheku.com>
	<CAJecg5r6U4X-O2J5x_HSExSEy19Skksyw7jrEdwASabjaind_w@mail.gmail.com>
	<BN6PR2001MB107464DC3D9DF5CB83103EA7F1A60@BN6PR2001MB1074.namprd20.prod.outlook.com>
	<CA+MRLqnRXjrOmqObVMObBhDUo5uRZAUMo=vOuE=tKOWSZMk2ZQ@mail.gmail.com>
	<BN6PR2001MB1074D25959ADEB88B732CBA9F1A60@BN6PR2001MB1074.namprd20.prod.outlook.com>
	<2fa5a4be-7ba9-3257-713f-b1e907e03489@redhat.com>
	<CAJecg5qzYtikTFuJbk6V02nqWdyjsu_4zwj65MWQMtnO_Cfszg@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <CAJecg5qzYtikTFuJbk6V02nqWdyjsu_4zwj65MWQMtnO_Cfszg@mail.gmail.com>
User-Agent: Mutt/1.5.21 (2010-09-15)
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
X-Received-From: 2001:638:208:120::220
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 20 Jul 2017 07:28:02 -0000

Hi Lance,

On Wed, Jul 19, 2017 at 01:03:59PM -0400, Lance E Sloan wrote:
> [...]
> Even if this feature suggestion isn't approved by the GNU community, I will
> implement it for my own use anyway.  I can enjoy the new functionality
> (which I think should have been added to cut long ago) and keep it to
> myself or I can contribute it back to the online community.  I could
> distribute it as my own fork of GNU coreutils or as a patch to it.

If I may suggest you post a patch to this list and/or a GNU Coreutils bug
report, so that others can pick it up if they would like to. This does not
preclude you from any of the other options you mentioned.

> However, if it were merged into GNU's coreutils, it would get the most
> exposure and be helpful to more people.

One needs to look beyond the current quarter's earnings to see this. ;-)

Thanks,
Erik
-- 
Trying to understand [the GNU GPL] in terms of the goals and values
of open source is like trying [to] understand a CD drive's retractable
drawer as a cupholder.
                        -- Richard Stallman


From MAILER-DAEMON Thu Jul 20 03:28:12 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dY5sa-0002N0-19
	for mharc-coreutils@gnu.org; Thu, 20 Jul 2017 03:28:12 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:47746)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <auerswal@unix-ag.uni-kl.de>) id 1dY5sW-0002LV-Me
	for coreutils@gnu.org; Thu, 20 Jul 2017 03:28:10 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <auerswal@unix-ag.uni-kl.de>) id 1dY5sV-0006VS-R6
	for coreutils@gnu.org; Thu, 20 Jul 2017 03:28:08 -0400
Received: from mailgw1.uni-kl.de ([2001:638:208:120::220]:44634)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)
	(Exim 4.71) (envelope-from <auerswal@unix-ag.uni-kl.de>)
	id 1dY5sV-0006V7-JL
	for coreutils@gnu.org; Thu, 20 Jul 2017 03:28:07 -0400
Received: from sushi.unix-ag.uni-kl.de (sushi.unix-ag.uni-kl.de
	[IPv6:2001:638:208:ef34:0:ff:fe00:65])
	by mailgw1.uni-kl.de (8.14.4/8.14.4/Debian-8+deb8u2) with ESMTP id
	v6K71ZVM023562
	(version=TLSv1/SSLv3 cipher=ECDHE-RSA-AES256-SHA bits=256 verify=NOT)
	for <coreutils@gnu.org>; Thu, 20 Jul 2017 09:01:35 +0200
Received: from sushi.unix-ag.uni-kl.de (ip6-localhost [IPv6:::1])
	by sushi.unix-ag.uni-kl.de (8.14.4/8.14.4/Debian-4+deb7u1) with ESMTP
	id v6K71YcW021807
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256
	verify=NO) for <coreutils@gnu.org>; Thu, 20 Jul 2017 09:01:34 +0200
Received: (from auerswal@localhost)
	by sushi.unix-ag.uni-kl.de (8.14.4/8.14.4/Submit) id v6K71YoN021806
	for coreutils@gnu.org; Thu, 20 Jul 2017 09:01:34 +0200
Date: Thu, 20 Jul 2017 09:01:34 +0200
From: Erik Auerswald <auerswal@unix-ag.uni-kl.de>
To: coreutils@gnu.org
Subject: Re: coreutils feature requests?
Message-ID: <20170720070134.GA21350@unix-ag.uni-kl.de>
References: <CAJecg5pGpQU_CZ-AdMtA88fzGs7RPd+dHG__AeYTvergm=iKFA@mail.gmail.com>
	<ed6ef41b902c4a37f5c78e8f791d23f6@mail.kylheku.com>
	<CAJecg5r6U4X-O2J5x_HSExSEy19Skksyw7jrEdwASabjaind_w@mail.gmail.com>
	<BN6PR2001MB107464DC3D9DF5CB83103EA7F1A60@BN6PR2001MB1074.namprd20.prod.outlook.com>
	<CA+MRLqnRXjrOmqObVMObBhDUo5uRZAUMo=vOuE=tKOWSZMk2ZQ@mail.gmail.com>
	<BN6PR2001MB1074D25959ADEB88B732CBA9F1A60@BN6PR2001MB1074.namprd20.prod.outlook.com>
	<2fa5a4be-7ba9-3257-713f-b1e907e03489@redhat.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <2fa5a4be-7ba9-3257-713f-b1e907e03489@redhat.com>
User-Agent: Mutt/1.5.21 (2010-09-15)
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
X-Received-From: 2001:638:208:120::220
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 20 Jul 2017 07:28:10 -0000

Hi,

On Wed, Jul 19, 2017 at 09:15:43AM -0500, Eric Blake wrote:
> On 07/19/2017 08:29 AM, Nellis, Kenneth wrote:
> > From: Steeve McCauley
> >> I can't believe I'd never thought of reordering output columns like this.  
> >> FWIW, I agree that another option should be used to prevent issues with backward compatibility.
> >>
> >> $ echo 1,2,3,4,5,6 | cut -d, -f3,5,2
> >> 2,3,5
> >>
> >> $ echo 1,2,3,4,5,6 | cut -d, -f3,5,2 -o
> >> 3,5,2
> 
> Unfortunately, it takes time to add an option, and then for that
> addition to percolate into the pre-built binaries of all the distros
> that you use.
> [...]
> the gap).
> Adding a new feature to GNU Coreutils is also easier to do if
> you can find someone else (like BSD) that already has the feature - but
> to my knowledge, no common 'cut -o' exists in other major distributions.

I have read this reasoning quite often on this list. Does it mean that GNU
does not innovate, but it might follow the lead of innovators? I thought
one of the goals of GNU was to improve on Unix
(https://www.gnu.org/gnu/initial-announcement.html).
The announcements states "We will make all improvements that are
convenient[...]." Are those times over?

Sorry for the flame bait, but I needed to get this out.

Thanks,
Erik
-- 
Design your product to please the users.
                        -- Paul Graham


From MAILER-DAEMON Thu Jul 20 07:09:12 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dY9KS-00062j-4X
	for mharc-coreutils@gnu.org; Thu, 20 Jul 2017 07:09:12 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:48672)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <sloanlance@gmail.com>) id 1dY9KQ-00062a-3V
	for coreutils@gnu.org; Thu, 20 Jul 2017 07:09:11 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <sloanlance@gmail.com>) id 1dY9KO-0000uk-F0
	for coreutils@gnu.org; Thu, 20 Jul 2017 07:09:10 -0400
Received: from mail-io0-x229.google.com ([2607:f8b0:4001:c06::229]:33359)
	by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)
	(Exim 4.71) (envelope-from <sloanlance@gmail.com>)
	id 1dY9KO-0000tR-8B
	for coreutils@gnu.org; Thu, 20 Jul 2017 07:09:08 -0400
Received: by mail-io0-x229.google.com with SMTP id 5so10466748iow.0
	for <coreutils@gnu.org>; Thu, 20 Jul 2017 04:09:06 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
	h=mime-version:references:in-reply-to:from:date:message-id:subject:to
	:cc; bh=tkvxdloOAIlPQ6zDoEbnxSwPz5YRNUfCNv7sD+8bh5U=;
	b=Ekpu4qtfs4+xuxV4iUoTrd3Z0HEtCZSjeFsBJvuA37k7AlgAf3hCvNjw10ZnMe4IgD
	77bkClliI9iEjZX79myq2y4ZkyGTLkoPcBVSvscmZpdKPeE+/TezDgiRK640m2v8ncVs
	UhkuaWCkqDjtf7R1uEw+m0uNBjauEJ51MYpBYPYWeBjlcX5c7f1MJuGwxxrZwftPQVSF
	Uv45FFZOusymALN0xUPEH7yJtaHgCw9b+MPZ5cjLJypaAYKxw2ceeVUNaDvbPvQ2sbQq
	plxNuXM+Gr28q0nIvxTVswLBzWzpLua6iR92iU0pyBYQ0AJhzyDrFlMJcpSbgiF1CCk4
	7X5Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
	d=1e100.net; s=20161025;
	h=x-gm-message-state:mime-version:references:in-reply-to:from:date
	:message-id:subject:to:cc;
	bh=tkvxdloOAIlPQ6zDoEbnxSwPz5YRNUfCNv7sD+8bh5U=;
	b=eeWRn+GezHE9UF3brQ9ljHwfNXIHjn/Mcex6FFkzGRExaEjtoqlJzDCTueuaqElGnP
	cW/CZSGySuLz6rf/YZplYIHIKVPUwmCei8OQQC3oeGDACIjnJgr8P/6lky+aLdAxCdtj
	Y+etFY4BUA3apX5YG6cFTeNNlhfxovRjqPbC4ve5YogNaMkUIsPFPDRrcCn/et/TxYKj
	yFt9sxMI7PU2v8xPpytkpTPJ1GYNxzn+mIcmW6vjNeSbzt4qM/UsqsoF2wRhvXAjl3UW
	g/W7i4TFzF46/eesQCZQL7//CZR+bnWrcuuiA+zIOfAe1/NkOg0RGDkg+iNzz8cv+JyT
	/nBQ==
X-Gm-Message-State: AIVw1109+Eli8PoqZX8DsXRTn1CPUUUCAI13spOkNfUN7+TTDfDY+tYQ
	WeilKz33TIbKZasLHTChWozazzl1xg==
X-Received: by 10.107.129.9 with SMTP id c9mr2960795iod.17.1500548945944; Thu,
	20 Jul 2017 04:09:05 -0700 (PDT)
MIME-Version: 1.0
References: <CAJecg5pGpQU_CZ-AdMtA88fzGs7RPd+dHG__AeYTvergm=iKFA@mail.gmail.com>
	<ed6ef41b902c4a37f5c78e8f791d23f6@mail.kylheku.com>
	<CAJecg5r6U4X-O2J5x_HSExSEy19Skksyw7jrEdwASabjaind_w@mail.gmail.com>
	<BN6PR2001MB107464DC3D9DF5CB83103EA7F1A60@BN6PR2001MB1074.namprd20.prod.outlook.com>
	<CA+MRLqnRXjrOmqObVMObBhDUo5uRZAUMo=vOuE=tKOWSZMk2ZQ@mail.gmail.com>
	<BN6PR2001MB1074D25959ADEB88B732CBA9F1A60@BN6PR2001MB1074.namprd20.prod.outlook.com>
	<2fa5a4be-7ba9-3257-713f-b1e907e03489@redhat.com>
	<CAJecg5qzYtikTFuJbk6V02nqWdyjsu_4zwj65MWQMtnO_Cfszg@mail.gmail.com>
	<cdfebfa6-7475-1bda-6c85-2267b775f381@redhat.com>
	<96813fa891560b117b94e4bdcbef8c07@mail.kylheku.com>
	<d3ea3008-e547-57ce-8dce-b3c1004cf7c7@bernhard-voelker.de>
In-Reply-To: <d3ea3008-e547-57ce-8dce-b3c1004cf7c7@bernhard-voelker.de>
From: Lance E Sloan <sloanlance+coreutils_gnu.org@gmail.com>
Date: Thu, 20 Jul 2017 11:08:55 +0000
Message-ID: <CAJecg5oR0FCqhH=8D-PBk3N8JLmq-S5LiQFXUc3SnXSAnUuq6w@mail.gmail.com>
Subject: Re: coreutils feature requests?
To: Bernhard Voelker <mail@bernhard-voelker.de>
Cc: coreutils@gnu.org
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 2607:f8b0:4001:c06::229
Content-Type: text/plain; charset="UTF-8"
X-Content-Filtered-By: Mailman/MimeDel 2.1.21
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 20 Jul 2017 11:09:11 -0000

Hi, Berny.

You've beaten me to the punch. I have another message draft in progress
about join. I'll send it later.

I haven't compared join with cut and awk, but it does the job.

On Wed, Jul 19, 2017 at 17:08 Bernhard Voelker <mail@bernhard-voelker.de>
wrote:

> On 07/19/2017 07:43 PM, Kaz Kylheku (Coreutils) wrote:
> > It is nontrivial code. For instance if we look at how the function
> > cut_bytes works in the implementation, what it's doing is simply
> > doing a getchar() from the stream, and querying a data structure
> > to determine whether the byte should be printed or not.
> > (That data structure consists of a pointer which marches through
> > field range descriptors in parallel with going through the data.)
> >
> > cut_fields is more complicated due to the delimiting of fields,
> > but essentially the same overall approach.
> >
> > Basically, printing of fields that isn't sorted and de-duplicated
> > is a rewrite of all parts of the utility other than command
> > line processing and the printing of usage help text.
> >
> > It's like two different programs in one, sharing a minimal
> > skeleton.
>
> +1
>
> Another point: it is already documented that cut(1) output is
> never good for reordering:
>
>
> http://git.sv.gnu.org/cgit/coreutils.git/tree/doc/coreutils.texi?id=545f181f4e#n5938
>
>   Note @command{awk} supports more sophisticated field processing,
>   like reordering fields, and handling fields aligned with blank
> characters.
>   By default @command{awk} uses (and discards) runs of blank characters
>   to separate fields, and ignores leading and trailing blanks.
>   @example
>   @verbatim
>   awk '{print $2}'      # print the second field
>   awk '{print $(NF-1)}' # print the penultimate field
>   awk '{print $2,$1}'   # reorder the first two fields
>   @end verbatim
>   @end example
>   Note while @command{cut} accepts field specifications in
>   arbitrary order, output is always in the order encountered in the file.
>
> and even more: it suggests to use join:
>
>   In the unlikely event that @command{awk} is unavailable,
>   one can use the @command{join} command, to process blank
>   characters as @command{awk} does above.
>   @example
>   @verbatim
>   join -a1 -o 1.2     - /dev/null # print the second field
>   join -a1 -o 1.2,1.1 - /dev/null # reorder the first two fields
>   @end verbatim
>   @end example
>
> Is this sufficient?
>
> Have a nice day,
> Berny
>

From MAILER-DAEMON Thu Jul 20 08:55:02 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dYAys-0007sg-4b
	for mharc-coreutils@gnu.org; Thu, 20 Jul 2017 08:55:02 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:48474)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1dYAyq-0007sQ-44
	for coreutils@gnu.org; Thu, 20 Jul 2017 08:55:01 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1dYAym-0008Eh-Se
	for coreutils@gnu.org; Thu, 20 Jul 2017 08:55:00 -0400
Received: from mx1.redhat.com ([209.132.183.28]:35210)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)
	(Exim 4.71) (envelope-from <eblake@redhat.com>) id 1dYAym-00089v-Jh
	for coreutils@gnu.org; Thu, 20 Jul 2017 08:54:56 -0400
Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com
	[10.5.11.12])
	(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
	(No client certificate requested)
	by mx1.redhat.com (Postfix) with ESMTPS id 82E34C058ECF;
	Thu, 20 Jul 2017 12:54:53 +0000 (UTC)
DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 82E34C058ECF
Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com;
	dmarc=none (p=none dis=none) header.from=redhat.com
Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com;
	spf=pass smtp.mailfrom=eblake@redhat.com
DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 82E34C058ECF
Received: from [10.10.120.37] (ovpn-120-37.rdu2.redhat.com [10.10.120.37])
	by smtp.corp.redhat.com (Postfix) with ESMTP id CCB6582701;
	Thu, 20 Jul 2017 12:54:52 +0000 (UTC)
Subject: Re: coreutils feature requests?
To: Erik Auerswald <auerswal@unix-ag.uni-kl.de>, coreutils@gnu.org
References: <CAJecg5pGpQU_CZ-AdMtA88fzGs7RPd+dHG__AeYTvergm=iKFA@mail.gmail.com>
	<ed6ef41b902c4a37f5c78e8f791d23f6@mail.kylheku.com>
	<CAJecg5r6U4X-O2J5x_HSExSEy19Skksyw7jrEdwASabjaind_w@mail.gmail.com>
	<BN6PR2001MB107464DC3D9DF5CB83103EA7F1A60@BN6PR2001MB1074.namprd20.prod.outlook.com>
	<CA+MRLqnRXjrOmqObVMObBhDUo5uRZAUMo=vOuE=tKOWSZMk2ZQ@mail.gmail.com>
	<BN6PR2001MB1074D25959ADEB88B732CBA9F1A60@BN6PR2001MB1074.namprd20.prod.outlook.com>
	<2fa5a4be-7ba9-3257-713f-b1e907e03489@redhat.com>
	<20170720070134.GA21350@unix-ag.uni-kl.de>
From: Eric Blake <eblake@redhat.com>
Openpgp: url=http://people.redhat.com/eblake/eblake.gpg
Organization: Red Hat, Inc.
Message-ID: <6c94691a-2a4e-0e3f-3351-a815fb82b860@redhat.com>
Date: Thu, 20 Jul 2017 07:54:52 -0500
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
	Thunderbird/52.2.1
MIME-Version: 1.0
In-Reply-To: <20170720070134.GA21350@unix-ag.uni-kl.de>
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature";
	boundary="9VbX4hxFs93MQBOeOkgkgEMTVfvRCr2ex"
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12
X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16
	(mx1.redhat.com [10.5.110.32]);
	Thu, 20 Jul 2017 12:54:53 +0000 (UTC)
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
	[fuzzy]
X-Received-From: 209.132.183.28
X-Content-Filtered-By: Mailman/MimeDel 2.1.21
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 20 Jul 2017 12:55:01 -0000

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--9VbX4hxFs93MQBOeOkgkgEMTVfvRCr2ex
From: Eric Blake <eblake@redhat.com>
To: Erik Auerswald <auerswal@unix-ag.uni-kl.de>, coreutils@gnu.org
Message-ID: <6c94691a-2a4e-0e3f-3351-a815fb82b860@redhat.com>
Subject: Re: coreutils feature requests?
References: <CAJecg5pGpQU_CZ-AdMtA88fzGs7RPd+dHG__AeYTvergm=iKFA@mail.gmail.com>
 <ed6ef41b902c4a37f5c78e8f791d23f6@mail.kylheku.com>
 <CAJecg5r6U4X-O2J5x_HSExSEy19Skksyw7jrEdwASabjaind_w@mail.gmail.com>
 <BN6PR2001MB107464DC3D9DF5CB83103EA7F1A60@BN6PR2001MB1074.namprd20.prod.outlook.com>
 <CA+MRLqnRXjrOmqObVMObBhDUo5uRZAUMo=vOuE=tKOWSZMk2ZQ@mail.gmail.com>
 <BN6PR2001MB1074D25959ADEB88B732CBA9F1A60@BN6PR2001MB1074.namprd20.prod.outlook.com>
 <2fa5a4be-7ba9-3257-713f-b1e907e03489@redhat.com>
 <20170720070134.GA21350@unix-ag.uni-kl.de>
In-Reply-To: <20170720070134.GA21350@unix-ag.uni-kl.de>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

On 07/20/2017 02:01 AM, Erik Auerswald wrote:
>> Adding a new feature to GNU Coreutils is also easier to do if
>> you can find someone else (like BSD) that already has the feature - bu=
t
>> to my knowledge, no common 'cut -o' exists in other major distribution=
s.
>=20
> I have read this reasoning quite often on this list. Does it mean that =
GNU
> does not innovate, but it might follow the lead of innovators? I though=
t
> one of the goals of GNU was to improve on Unix
> (https://www.gnu.org/gnu/initial-announcement.html).
> The announcements states "We will make all improvements that are
> convenient[...]." Are those times over?

No. We still welcome useful contributions, even if there is no other
precedence.  It's just that having precedence makes it easier to get new
features added, but that is not a prerequisite.

Let me reword my thoughts on the matter:

I personally don't need 'cut' to reorder fields, because I already know
how to use other standard utilities to do the same.  So I'm not going to
spend MY time implementing an enhancement to cut.  And someone else
asking ME to do the work won't get very far.  (So far, over the years,
most of the requests have been "will YOU implement this feature for me?"
- so of course, the answer is "why, when you already have the feature
via other means?").

But if someone is determined enough that it is a useful feature (and it
probably is, I have NOT ruled it out, and I even went so far as to
provide quotes from earlier discussions saying that having the feature
to reorder fields in cut via a new option WOULD make cut more intuitive)
- then PROVE that it is useful, by providing the patch (including
documentation and testsuite changes).  And in all likelihood, once such
a patch is posted, I will GLADLY help review and incorporate the feature
(because _I_ did not have to do the heavy lifting for something I would
not personally use).

--=20
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org


--9VbX4hxFs93MQBOeOkgkgEMTVfvRCr2ex
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Comment: Public key at http://people.redhat.com/eblake/eblake.gpg
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEzBAEBCAAdFiEEccLMIrHEYCkn0vOqp6FrSiUnQ2oFAllwqBwACgkQp6FrSiUn
Q2qFWQgAnQc9ovdbw+Jyvgxstae5VAedRKhapA0cBYmZMtQ0r76wi6rqIQSruQiI
Onv+sGjMVOgKSp6tgIbssLMH0DRaHTFdNsfG+CUkoL97dHkZotFRF9gJu1vCyf6g
mZSFq3yyteNpQgQNf5HmltR0FRNeoz98mW8jTuXpls3TzFLr46X02CLkyjDOpGve
t1ziMQ9tt8Lr2fYW9Eq53F0XE0xsTFLcWZh8Gu13OeOraztUW+GnQqwwAqDGPStk
02SNZKpYElObtRv0BfhnfG71q4yU9k7OFaS+jo3xTE9ReLPXIMm+KJV39wRLV2YE
xMbtVigBIoZbUn4IqKeovFtXaP4/qA==
=F0DS
-----END PGP SIGNATURE-----

--9VbX4hxFs93MQBOeOkgkgEMTVfvRCr2ex--


From MAILER-DAEMON Thu Jul 20 17:03:28 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dYIbX-0007g1-Uq
	for mharc-coreutils@gnu.org; Thu, 20 Jul 2017 17:03:27 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:47316)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eggert@cs.ucla.edu>) id 1dYIbV-0007fc-Th
	for coreutils@gnu.org; Thu, 20 Jul 2017 17:03:26 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eggert@cs.ucla.edu>) id 1dYIbS-0003RC-RG
	for coreutils@gnu.org; Thu, 20 Jul 2017 17:03:25 -0400
Received: from zimbra.cs.ucla.edu ([131.179.128.68]:40556)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)
	(Exim 4.71) (envelope-from <eggert@cs.ucla.edu>) id 1dYIbS-0003Qr-LP
	for coreutils@gnu.org; Thu, 20 Jul 2017 17:03:22 -0400
Received: from localhost (localhost [127.0.0.1])
	by zimbra.cs.ucla.edu (Postfix) with ESMTP id C39F6160230
	for <coreutils@gnu.org>; Thu, 20 Jul 2017 14:03:19 -0700 (PDT)
Received: from zimbra.cs.ucla.edu ([127.0.0.1])
	by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032)
	with ESMTP id t6eANs6v1OU3; Thu, 20 Jul 2017 14:03:19 -0700 (PDT)
Received: from localhost (localhost [127.0.0.1])
	by zimbra.cs.ucla.edu (Postfix) with ESMTP id 16A8B160193;
	Thu, 20 Jul 2017 14:03:19 -0700 (PDT)
X-Virus-Scanned: amavisd-new at zimbra.cs.ucla.edu
Received: from zimbra.cs.ucla.edu ([127.0.0.1])
	by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026)
	with ESMTP id iSnpifBMIzti; Thu, 20 Jul 2017 14:03:19 -0700 (PDT)
Received: from Penguin.CS.UCLA.EDU (Penguin.CS.UCLA.EDU [131.179.64.200])
	by zimbra.cs.ucla.edu (Postfix) with ESMTPSA id EC02916017A;
	Thu, 20 Jul 2017 14:03:18 -0700 (PDT)
From: Paul Eggert <eggert@cs.ucla.edu>
To: coreutils@gnu.org
Cc: Paul Eggert <eggert@cs.ucla.edu>
Subject: [PATCH 1/2] build: update gnulib submodule to latest
Date: Thu, 20 Jul 2017 14:03:12 -0700
Message-Id: <20170720210313.26813-1-eggert@cs.ucla.edu>
X-Mailer: git-send-email 2.13.3
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy]
X-Received-From: 131.179.128.68
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 20 Jul 2017 21:03:26 -0000

---
 gnulib | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/gnulib b/gnulib
index efb84214a..f014f540f 160000
--- a/gnulib
+++ b/gnulib
@@ -1 +1 @@
-Subproject commit efb84214ac14749188ab8294a52b4e91475c13b6
+Subproject commit f014f540f1b7165444ca56a4a35850689ad383a5
-- 
2.13.3



From MAILER-DAEMON Thu Jul 20 17:03:29 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dYIbZ-0007gz-38
	for mharc-coreutils@gnu.org; Thu, 20 Jul 2017 17:03:29 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:47317)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eggert@cs.ucla.edu>) id 1dYIbV-0007fd-U7
	for coreutils@gnu.org; Thu, 20 Jul 2017 17:03:26 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eggert@cs.ucla.edu>) id 1dYIbS-0003RK-Ua
	for coreutils@gnu.org; Thu, 20 Jul 2017 17:03:25 -0400
Received: from zimbra.cs.ucla.edu ([131.179.128.68]:40554)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)
	(Exim 4.71) (envelope-from <eggert@cs.ucla.edu>) id 1dYIbS-0003Qq-LZ
	for coreutils@gnu.org; Thu, 20 Jul 2017 17:03:22 -0400
Received: from localhost (localhost [127.0.0.1])
	by zimbra.cs.ucla.edu (Postfix) with ESMTP id D1A6316023A
	for <coreutils@gnu.org>; Thu, 20 Jul 2017 14:03:20 -0700 (PDT)
Received: from zimbra.cs.ucla.edu ([127.0.0.1])
	by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032)
	with ESMTP id Z-6R5IPJlQRC; Thu, 20 Jul 2017 14:03:20 -0700 (PDT)
Received: from localhost (localhost [127.0.0.1])
	by zimbra.cs.ucla.edu (Postfix) with ESMTP id 06316160239;
	Thu, 20 Jul 2017 14:03:20 -0700 (PDT)
X-Virus-Scanned: amavisd-new at zimbra.cs.ucla.edu
Received: from zimbra.cs.ucla.edu ([127.0.0.1])
	by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026)
	with ESMTP id jnScxo72Zfcp; Thu, 20 Jul 2017 14:03:19 -0700 (PDT)
Received: from Penguin.CS.UCLA.EDU (Penguin.CS.UCLA.EDU [131.179.64.200])
	by zimbra.cs.ucla.edu (Postfix) with ESMTPSA id E1E57160193;
	Thu, 20 Jul 2017 14:03:19 -0700 (PDT)
From: Paul Eggert <eggert@cs.ucla.edu>
To: coreutils@gnu.org
Cc: Paul Eggert <eggert@cs.ucla.edu>
Subject: [PATCH 2/2] shred: use explicit_bzero
Date: Thu, 20 Jul 2017 14:03:13 -0700
Message-Id: <20170720210313.26813-2-eggert@cs.ucla.edu>
X-Mailer: git-send-email 2.13.3
In-Reply-To: <20170720210313.26813-1-eggert@cs.ucla.edu>
References: <20170720210313.26813-1-eggert@cs.ucla.edu>
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy]
X-Received-From: 131.179.128.68
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 20 Jul 2017 21:03:27 -0000

* NEWS: Document this.
* bootstrap.conf (gnulib_modules): Add explicit_bzero.
* gl/lib/randint.c (randint_free):
* gl/lib/randread.c (randread_free):
* src/blake2/blake2-impl.h (secure_zero_memory):
* src/shred.c (dopass, do_wipefd):
Prefer explicit_bzero to memset when erasing secrets.
---
 NEWS                     | 4 ++++
 bootstrap.conf           | 1 +
 gl/lib/randint.c         | 2 +-
 gl/lib/randread.c        | 2 +-
 src/blake2/blake2-impl.h | 3 +--
 src/shred.c              | 4 ++--
 6 files changed, 10 insertions(+), 6 deletions(-)

diff --git a/NEWS b/NEWS
index 110229bd8..dfd2837a0 100644
--- a/NEWS
+++ b/NEWS
@@ -25,6 +25,10 @@ GNU coreutils NEWS                                    -*- outline -*-
   Now, it prints a diagnostic or a line to stdout for each argument.
   [bug introduced in the bourne-shell-to-C rewrite for coreutils-6.11]
 
+  shred now erases buffers containing secrets via the explicit_bzero
+  function, which should be more reliable.
+  [potential bug has always been present in 'shred']
+
   split no longer exits when invocations of a --filter return EPIPE.
   [bug introduced in coreutils-8.26]
 
diff --git a/bootstrap.conf b/bootstrap.conf
index 4db77a3d7..9064a94bb 100644
--- a/bootstrap.conf
+++ b/bootstrap.conf
@@ -69,6 +69,7 @@ gnulib_modules="
   euidaccess
   exclude
   exitfail
+  explicit_bzero
   faccessat
   fadvise
   fchdir
diff --git a/gl/lib/randint.c b/gl/lib/randint.c
index 4561067e8..b15982f46 100644
--- a/gl/lib/randint.c
+++ b/gl/lib/randint.c
@@ -198,7 +198,7 @@ randint_genmax (struct randint_source *s, randint genmax)
 void
 randint_free (struct randint_source *s)
 {
-  memset (s, 0, sizeof *s);
+  explicit_bzero (s, sizeof *s);
   free (s);
 }
 
diff --git a/gl/lib/randread.c b/gl/lib/randread.c
index 9c70a1875..834f8446a 100644
--- a/gl/lib/randread.c
+++ b/gl/lib/randread.c
@@ -341,7 +341,7 @@ int
 randread_free (struct randread_source *s)
 {
   FILE *source = s->source;
-  memset (s, 0, sizeof *s);
+  explicit_bzero (s, sizeof *s);
   free (s);
   return (source ? fclose (source) : 0);
 }
diff --git a/src/blake2/blake2-impl.h b/src/blake2/blake2-impl.h
index 5dff7fc7a..241e5abf5 100644
--- a/src/blake2/blake2-impl.h
+++ b/src/blake2/blake2-impl.h
@@ -153,8 +153,7 @@ static BLAKE2_INLINE uint64_t rotr64( const uint64_t w, const unsigned c )
 /* prevents compiler optimizing out memset() */
 static BLAKE2_INLINE void secure_zero_memory(void *v, size_t n)
 {
-  static void *(*const volatile memset_v)(void *, int, size_t) = &memset;
-  memset_v(v, 0, n);
+  explicit_bzero (v, n);
 }
 
 #endif
diff --git a/src/shred.c b/src/shred.c
index 7926e7aa4..c95546cb7 100644
--- a/src/shred.c
+++ b/src/shred.c
@@ -653,7 +653,7 @@ dopass (int fd, struct stat const *st, char const *qname, off_t *sizep,
     }
 
 free_pattern_mem:
-  memset (pbuf, 0, FILLPATTERN_SIZE);
+  explicit_bzero (pbuf, FILLPATTERN_SIZE);
   free (fill_pattern_mem);
 
   return other_error ? -1 : write_error;
@@ -987,7 +987,7 @@ do_wipefd (int fd, char const *qname, struct randint_source *s,
     }
 
 wipefd_out:
-  memset (passarray, 0, flags->n_iterations * sizeof (int));
+  explicit_bzero (passarray, flags->n_iterations * sizeof (int));
   free (passarray);
   return ok;
 }
-- 
2.13.3



From MAILER-DAEMON Sun Jul 23 04:30:38 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dZCHe-0004XG-M3
	for mharc-coreutils@gnu.org; Sun, 23 Jul 2017 04:30:38 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:54147)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1dZCHc-0004X8-DA
	for coreutils@gnu.org; Sun, 23 Jul 2017 04:30:37 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1dZCHZ-0005MU-9U
	for coreutils@gnu.org; Sun, 23 Jul 2017 04:30:36 -0400
Received: from mail.magicbluesmoke.com ([82.195.144.49]:44804)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)
	(Exim 4.71) (envelope-from <P@draigBrady.com>) id 1dZCHY-0005LT-US
	for coreutils@gnu.org; Sun, 23 Jul 2017 04:30:33 -0400
Received: from localhost.localdomain (unknown [73.93.152.135])
	(using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))
	(No client certificate requested)
	by mail.magicbluesmoke.com (Postfix) with ESMTPSA id BEF3D9AE7;
	Sun, 23 Jul 2017 09:30:25 +0100 (IST)
Subject: Re: [PATCH 2/2] shred: use explicit_bzero
To: Paul Eggert <eggert@cs.ucla.edu>, coreutils@gnu.org
References: <20170720210313.26813-1-eggert@cs.ucla.edu>
	<20170720210313.26813-2-eggert@cs.ucla.edu>
From: =?UTF-8?Q?P=c3=a1draig_Brady?= <P@draigBrady.com>
Message-ID: <cbceb699-d295-cd32-7d52-fc0527ae249d@draigBrady.com>
Date: Sun, 23 Jul 2017 01:30:22 -0700
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101
	Thunderbird/45.8.0
MIME-Version: 1.0
In-Reply-To: <20170720210313.26813-2-eggert@cs.ucla.edu>
Content-Type: multipart/mixed; boundary="------------17E908B774E98A133A6C1478"
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
	[fuzzy]
X-Received-From: 82.195.144.49
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 23 Jul 2017 08:30:37 -0000

This is a multi-part message in MIME format.
--------------17E908B774E98A133A6C1478
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: quoted-printable

On 20/07/17 14:03, Paul Eggert wrote:
> * NEWS: Document this.
> * bootstrap.conf (gnulib_modules): Add explicit_bzero.
> * gl/lib/randint.c (randint_free):
> * gl/lib/randread.c (randread_free):
> * src/blake2/blake2-impl.h (secure_zero_memory):
> * src/shred.c (dopass, do_wipefd):
> Prefer explicit_bzero to memset when erasing secrets.

> +  shred now erases buffers containing secrets via the explicit_bzero
> +  function, which should be more reliable.
> +  [potential bug has always been present in 'shred']

shred doesn't have any secrets though right?
Ah I see, it used to have secrets since it originally
read the input file to seed the random number generator.
How about the attached to remove this redundant zeroing entirely?

Also I think I'll remove the explicit_bzero() from blake2
as src/blake2 is copied directly from the reference implementation.
So it would be best to change there and have a one way sync.

cheers,
P=E1draig


--------------17E908B774E98A133A6C1478
Content-Type: text/x-patch;
 name="shred-avoid-bzero.patch"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: attachment;
 filename="shred-avoid-bzero.patch"

=46rom b9626205effc42753d3be3923ca85c73024bbcd7 Mon Sep 17 00:00:00 2001
From: =3D?UTF-8?q?P=3DC3=3DA1draig=3D20Brady?=3D <P@draigBrady.com>
Date: Sun, 23 Jul 2017 01:06:33 -0700
Subject: [PATCH] shred: remove redundant zeroing of freed memory

* src/shred.c (dopass): shred used to read the input file,
and so needed to ensure internal memory was cleared.
This is no longer the case since SH-UTILS-1_16f-260-gf381610
so avoid this redundant clearing.
(do_wipefd): Likewise.
* NEWS: Remove the recent mention of this issue.
---
 NEWS        | 4 ----
 src/shred.c | 2 --
 2 files changed, 6 deletions(-)

diff --git a/NEWS b/NEWS
index dfd2837..110229b 100644
--- a/NEWS
+++ b/NEWS
@@ -25,10 +25,6 @@ GNU coreutils NEWS                                    =
-*- outline -*-
   Now, it prints a diagnostic or a line to stdout for each argument.
   [bug introduced in the bourne-shell-to-C rewrite for coreutils-6.11]
=20
-  shred now erases buffers containing secrets via the explicit_bzero
-  function, which should be more reliable.
-  [potential bug has always been present in 'shred']
-
   split no longer exits when invocations of a --filter return EPIPE.
   [bug introduced in coreutils-8.26]
=20
diff --git a/src/shred.c b/src/shred.c
index c95546c..e6df409 100644
--- a/src/shred.c
+++ b/src/shred.c
@@ -653,7 +653,6 @@ dopass (int fd, struct stat const *st, char const *qn=
ame, off_t *sizep,
     }
=20
 free_pattern_mem:
-  explicit_bzero (pbuf, FILLPATTERN_SIZE);
   free (fill_pattern_mem);
=20
   return other_error ? -1 : write_error;
@@ -987,7 +986,6 @@ do_wipefd (int fd, char const *qname, struct randint_=
source *s,
     }
=20
 wipefd_out:
-  explicit_bzero (passarray, flags->n_iterations * sizeof (int));
   free (passarray);
   return ok;
 }
--=20
2.9.3


--------------17E908B774E98A133A6C1478--


From MAILER-DAEMON Sun Jul 23 06:01:20 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dZDhQ-0000W4-QG
	for mharc-coreutils@gnu.org; Sun, 23 Jul 2017 06:01:20 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:34236)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1dZDhP-0000Um-4G
	for coreutils@gnu.org; Sun, 23 Jul 2017 06:01:20 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1dZDhM-0001XA-2a
	for coreutils@gnu.org; Sun, 23 Jul 2017 06:01:19 -0400
Received: from mail.magicbluesmoke.com ([82.195.144.49]:34468)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)
	(Exim 4.71) (envelope-from <P@draigBrady.com>) id 1dZDhL-0001W9-Rt
	for coreutils@gnu.org; Sun, 23 Jul 2017 06:01:16 -0400
Received: from localhost.localdomain (unknown [73.93.152.135])
	(using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))
	(No client certificate requested)
	by mail.magicbluesmoke.com (Postfix) with ESMTPSA id E19CE9F46;
	Sun, 23 Jul 2017 11:01:10 +0100 (IST)
Subject: [PATCH] tests: avoid a false failure on AIX
To: Michael Felt <michael@felt.demon.nl>, Coreutils <coreutils@gnu.org>
References: <d6eba7e5-2d3c-0e22-ac30-84a24004bf4a@felt.demon.nl>
	<0557f29d-82ed-17ba-47fd-23c6bd3b299c@felt.demon.nl>
From: =?UTF-8?Q?P=c3=a1draig_Brady?= <P@draigBrady.com>
Message-ID: <ff8a68dd-0ab8-728e-a1b9-daa95cc6fb02@draigBrady.com>
Date: Sun, 23 Jul 2017 03:01:08 -0700
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101
	Thunderbird/45.8.0
MIME-Version: 1.0
In-Reply-To: <0557f29d-82ed-17ba-47fd-23c6bd3b299c@felt.demon.nl>
Content-Type: multipart/mixed; boundary="------------35C02F0ABD27617EFCC672EB"
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
	[fuzzy]
X-Received-From: 82.195.144.49
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 23 Jul 2017 10:01:20 -0000

This is a multi-part message in MIME format.
--------------35C02F0ABD27617EFCC672EB
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

On 29/06/17 05:22, Michael Felt wrote:
> FAIL: tests/misc/sync
> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>=20

> + diff -u exp err
> --- exp	2017-06-26 19:40:59 +0000
> +++ err	2017-06-26 19:40:59 +0000
> @@ -1 +1 @@
> -sync: error opening 'norw': Permission denied
> +sync: error opening 'norw': Is a directory
> + fail=3D1

The attached should fix this.

thanks,
P=C3=A1draig

--------------35C02F0ABD27617EFCC672EB
Content-Type: text/x-patch;
 name="aix-sync-test.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename="aix-sync-test.patch"

>From 79d20021adce10ff0ae5b553087675f803c776a4 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?P=C3=A1draig=20Brady?= <P@draigBrady.com>
Date: Sun, 23 Jul 2017 01:54:03 -0700
Subject: [PATCH] tests: avoid a false failure on AIX

* tests/misc/sync.sh: Normalize the error messages
when syncing a non read/write directory, as AIX
gives the "Is a directory" error.
---
 tests/misc/sync.sh | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/tests/misc/sync.sh b/tests/misc/sync.sh
index 423c606..8adcb3a 100755
--- a/tests/misc/sync.sh
+++ b/tests/misc/sync.sh
@@ -37,7 +37,9 @@ returns_ 1 sync file nofile || fail=1
 mkdir norw || framework_failure_
 chmod 0 norw || framework_failure_
 if ! test -r norw; then
-  sync norw 2>err
+  sync norw 2>errt
+  # AIX gives "Is a directory"
+  sed 's/Is a directory/Permission denied/' < errt >err || framework_failure_
   printf "sync: error opening 'norw': Permission denied\n" >exp
   compare exp err || fail=1
 fi
-- 
2.9.3


--------------35C02F0ABD27617EFCC672EB--


From MAILER-DAEMON Sun Jul 23 06:07:50 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dZDni-0002DL-6d
	for mharc-coreutils@gnu.org; Sun, 23 Jul 2017 06:07:50 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:35077)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1dZDng-0002DE-Eo
	for coreutils@gnu.org; Sun, 23 Jul 2017 06:07:49 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1dZDnd-00035X-Cn
	for coreutils@gnu.org; Sun, 23 Jul 2017 06:07:48 -0400
Received: from mout.kundenserver.de ([212.227.126.134]:58294)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16)
	(Exim 4.71) (envelope-from <mail@bernhard-voelker.de>)
	id 1dZDnd-000357-1Q
	for coreutils@gnu.org; Sun, 23 Jul 2017 06:07:45 -0400
Received: from [192.168.101.10] ([91.12.166.234]) by mrelayeu.kundenserver.de
	(mreue003 [212.227.15.167]) with ESMTPSA (Nemesis) id
	0Lx3Yj-1dg7Yz0S6L-016iG7; Sun, 23 Jul 2017 12:07:40 +0200
Subject: Re: [PATCH] tests: avoid a false failure on AIX
To: =?UTF-8?Q?P=c3=a1draig_Brady?= <P@draigBrady.com>,
	Michael Felt <michael@felt.demon.nl>, Coreutils <coreutils@gnu.org>
References: <d6eba7e5-2d3c-0e22-ac30-84a24004bf4a@felt.demon.nl>
	<0557f29d-82ed-17ba-47fd-23c6bd3b299c@felt.demon.nl>
	<ff8a68dd-0ab8-728e-a1b9-daa95cc6fb02@draigBrady.com>
From: Bernhard Voelker <mail@bernhard-voelker.de>
Message-ID: <a32ed816-971d-1cde-6374-d2e50ee06352@bernhard-voelker.de>
Date: Sun, 23 Jul 2017 12:07:39 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101
	Thunderbird/45.6.0
MIME-Version: 1.0
In-Reply-To: <ff8a68dd-0ab8-728e-a1b9-daa95cc6fb02@draigBrady.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
X-Provags-ID: V03:K0:XXH7GID+kI1YdRbv5Qy2hczhPR+LZat8g+M5SOxQpfzEz2F8KLN
	+NzPCzP6l2qMgi8Mw9taT8Wub/gnfhZ5LZ/2O+BIkNs5Mc70xcfRh2EQGzUoGKPoFDb9I/i
	IAZwVKt97FPxI9Q4Aqkf6PVsa0Ys1DmzpNQuEt6CLv0ykygpwT6b9CDLhW3dhXdvJCDe3WD
	TcsdzFKJy047XwU+fr8ug==
X-UI-Out-Filterresults: notjunk:1;V01:K0:Zq995W4HQqE=:JTH5VQgHDhov7wpypeGfOb
	bjhMcf648IEIutVKxB+tjPRu+1w3shWnCGCU8KSnMm8aOBQpmFOUy594mdmFIrH1o24pFQE2P
	ZdOy7+VPpnuVSmP65ge7Nfa2zf76aIKp9KhhZNUSI7lDZVHhJthnP0osJi+2PvgDn+2THrdrC
	5PmVD0GgDuWprpLLLONVy6e3qSihNJWvNk6QcDB+q26kNwwXI5xyby0bbc40DDeE70ZbXmSIX
	c1/NsUjpW5WRGs6bYazbiv0MhnodrAZo7yMZSD7RmIDpiLKk0RLGWLJ6IYluFkvN75MXR2em6
	wudEJZfRZRkTkXj6M670F0873YO8cQfSZdum6ck3mXCRi0BH0ssDtikzdbzN8z7u23odQnAGH
	BjV3MViZIHjCvicI18eEE96LoSI7V3vFWxU45DztR+yb6l37VkMTWhP87S55b4ZAjjO0l2wol
	6tyqYojDksMiLjlKldNQxjOuvTrWXWTxmWIadNMB/LWDzCODjxPFq/oaVL2CP/41wxjUZc0cP
	xwE+ibiuKnxlj8L/CIIn4nVoC2h6mH7uF98fAaj4sbct7vPyR+OqjaeZ26ZbzqKbHcdkgp8Tj
	SC7Qc1A/bGU5KTtcOCNTp2ikFSuEBlqXWnHJ8ZrWWZfQLVW6PhfVLZQQhb9Oq+fz6G8szbqCG
	NXfjGl1Y8kuMEV/0TmYM0GlUGxT8edHBImifNYg9UsXNo7fzv8p9xsA1ZfahmDg2ejt+NlwFG
	p79BlNnRm98O/xHtS08tEtHkxCHHJmQ4fA4B3MNgdpQJ5cObKfMvoEv76+Q=
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
	[fuzzy]
X-Received-From: 212.227.126.134
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 23 Jul 2017 10:07:49 -0000

On 07/23/2017 12:01 PM, Pádraig Brady wrote:
> -  sync norw 2>err
> +  sync norw 2>errt

While at it, I think we check the exit code as well.  WDYT?

Have a nice day,
Berny




From MAILER-DAEMON Sun Jul 23 06:37:30 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dZEGQ-00063a-8S
	for mharc-coreutils@gnu.org; Sun, 23 Jul 2017 06:37:30 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:38387)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1dZEGN-00063R-Gu
	for coreutils@gnu.org; Sun, 23 Jul 2017 06:37:28 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1dZEGK-0001CX-Eo
	for coreutils@gnu.org; Sun, 23 Jul 2017 06:37:27 -0400
Received: from mail.magicbluesmoke.com ([82.195.144.49]:37476)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)
	(Exim 4.71) (envelope-from <P@draigBrady.com>) id 1dZEGK-0001Bz-7W
	for coreutils@gnu.org; Sun, 23 Jul 2017 06:37:24 -0400
Received: from localhost.localdomain (unknown [73.93.152.135])
	(using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))
	(No client certificate requested)
	by mail.magicbluesmoke.com (Postfix) with ESMTPSA id 88777A1BA;
	Sun, 23 Jul 2017 11:37:21 +0100 (IST)
Subject: [PATCH] kill: fix signal number to name lookup on AIX
To: Michael Felt <michael@felt.demon.nl>, Coreutils <coreutils@gnu.org>
References: <d6eba7e5-2d3c-0e22-ac30-84a24004bf4a@felt.demon.nl>
	<0557f29d-82ed-17ba-47fd-23c6bd3b299c@felt.demon.nl>
From: =?UTF-8?Q?P=c3=a1draig_Brady?= <P@draigBrady.com>
Message-ID: <16f50a12-4a01-6fc4-bcc3-e2d37e6f0702@draigBrady.com>
Date: Sun, 23 Jul 2017 03:37:18 -0700
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101
	Thunderbird/45.8.0
MIME-Version: 1.0
In-Reply-To: <0557f29d-82ed-17ba-47fd-23c6bd3b299c@felt.demon.nl>
Content-Type: multipart/mixed; boundary="------------1495093CFCE92EBD97973D1C"
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
	[fuzzy]
X-Received-From: 82.195.144.49
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 23 Jul 2017 10:37:28 -0000

This is a multi-part message in MIME format.
--------------1495093CFCE92EBD97973D1C
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

On 29/06/17 05:22, Michael Felt wrote:
> FAIL: tests/misc/kill

> ++ env kill -l HUP TERM
> ++ tail -n1
> + SIGTERM=3D15
> ++ env kill -l 15
> + test EXIT =3D TERM
> + fail=3D1

I think our hack of using the wait() status access macros for
manipulating the exit status from shells is failing for AIX.
The attached avoids the use of WTERMSIG() on AIX.
Perhaps we should do this for all platforms?

cheers,
P=C3=A1draig

--------------1495093CFCE92EBD97973D1C
Content-Type: text/x-patch;
 name="aix-signal-lookup.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename="aix-signal-lookup.patch"

>From 35eb8c25f59c6bd81949a4daf4339f1eac30d15b Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?P=C3=A1draig=20Brady?= <P@draigBrady.com>
Date: Sun, 23 Jul 2017 03:25:32 -0700
Subject: [PATCH] kill: fix signal number to name lookup on AIX

* src/operand2sig.c (operand2sig): AIX uses a different bit pattern
in the returned status from the wait() functions and from shells.
Therefore hardcode the selection of the lower bits of the provided
number when on AIX.
* NEWS: Mention the fix.
---
 NEWS              |  4 ++++
 src/operand2sig.c | 14 ++++++++++++--
 2 files changed, 16 insertions(+), 2 deletions(-)

diff --git a/NEWS b/NEWS
index dfd2837..22bef54 100644
--- a/NEWS
+++ b/NEWS
@@ -25,6 +25,10 @@ GNU coreutils NEWS                                    -*- outline -*-
   Now, it prints a diagnostic or a line to stdout for each argument.
   [bug introduced in the bourne-shell-to-C rewrite for coreutils-6.11]
 
+  kill now converts from number to signal name correctly on AIX.
+  Previously it would have always returned the 'EXIT' name.
+  [bug introduced in fileutils-4.1.9]
+
   shred now erases buffers containing secrets via the explicit_bzero
   function, which should be more reliable.
   [potential bug has always been present in 'shred']
diff --git a/src/operand2sig.c b/src/operand2sig.c
index d59ccb9..3b57b21 100644
--- a/src/operand2sig.c
+++ b/src/operand2sig.c
@@ -53,8 +53,18 @@ operand2sig (char const *operand, char *signame)
       char *endp;
       long int l = (errno = 0, strtol (operand, &endp, 10));
       int i = l;
-      signum = (operand == endp || *endp || errno || i != l ? -1
-                : WIFSIGNALED (i) ? WTERMSIG (i) : i);
+      signum = (operand == endp || *endp || errno || i != l ? -1 : i);
+
+      if (signum != -1)
+        {
+#ifdef _AIX
+          /* AIX uses a different bit pattern for status returned
+             from shell and wait(), therefore just consider the lower bits. */
+          signum &= signum >= 0xFF ? 0xFF : 0x7F;
+#else
+          signum = WIFSIGNALED (signum) ? WTERMSIG (signum) : signum;
+#endif
+        }
     }
   else
     {
-- 
2.9.3


--------------1495093CFCE92EBD97973D1C--


From MAILER-DAEMON Sun Jul 23 06:39:29 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dZEIL-0006RC-Ha
	for mharc-coreutils@gnu.org; Sun, 23 Jul 2017 06:39:29 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:38556)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1dZEII-0006Qv-Rn
	for coreutils@gnu.org; Sun, 23 Jul 2017 06:39:27 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1dZEIF-0001lN-8n
	for coreutils@gnu.org; Sun, 23 Jul 2017 06:39:26 -0400
Received: from mail.magicbluesmoke.com ([82.195.144.49]:37490)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)
	(Exim 4.71) (envelope-from <P@draigBrady.com>) id 1dZEIF-0001l9-23
	for coreutils@gnu.org; Sun, 23 Jul 2017 06:39:23 -0400
Received: from localhost.localdomain (unknown [73.93.152.135])
	(using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))
	(No client certificate requested)
	by mail.magicbluesmoke.com (Postfix) with ESMTPSA id 20BFCA1C6;
	Sun, 23 Jul 2017 11:39:20 +0100 (IST)
Subject: Re: [PATCH] tests: avoid a false failure on AIX
To: Bernhard Voelker <mail@bernhard-voelker.de>,
	Michael Felt <michael@felt.demon.nl>, Coreutils <coreutils@gnu.org>
References: <d6eba7e5-2d3c-0e22-ac30-84a24004bf4a@felt.demon.nl>
	<0557f29d-82ed-17ba-47fd-23c6bd3b299c@felt.demon.nl>
	<ff8a68dd-0ab8-728e-a1b9-daa95cc6fb02@draigBrady.com>
	<a32ed816-971d-1cde-6374-d2e50ee06352@bernhard-voelker.de>
From: =?UTF-8?Q?P=c3=a1draig_Brady?= <P@draigBrady.com>
Message-ID: <4a039689-a156-3461-0d21-71d370375aa4@draigBrady.com>
Date: Sun, 23 Jul 2017 03:39:19 -0700
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101
	Thunderbird/45.8.0
MIME-Version: 1.0
In-Reply-To: <a32ed816-971d-1cde-6374-d2e50ee06352@bernhard-voelker.de>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
	[fuzzy]
X-Received-From: 82.195.144.49
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 23 Jul 2017 10:39:27 -0000

On 23/07/17 03:07, Bernhard Voelker wrote:
> On 07/23/2017 12:01 PM, P=C3=A1draig Brady wrote:
>> -  sync norw 2>err
>> +  sync norw 2>errt
>=20
> While at it, I think we check the exit code as well.  WDYT?

Good point!
I've also done this:

-  sync norw 2>errt
+  returns_ 1 sync norw 2>errt

cheers,
P=C3=A1draig


From MAILER-DAEMON Sun Jul 23 15:11:48 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dZMI8-0004Zy-1k
	for mharc-coreutils@gnu.org; Sun, 23 Jul 2017 15:11:48 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:49476)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <meyering@gmail.com>) id 1dZMI5-0004ZY-QV
	for coreutils@gnu.org; Sun, 23 Jul 2017 15:11:46 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <meyering@gmail.com>) id 1dZMI4-0004T0-K3
	for coreutils@gnu.org; Sun, 23 Jul 2017 15:11:45 -0400
Received: from mail-ua0-x236.google.com ([2607:f8b0:400c:c08::236]:37982)
	by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)
	(Exim 4.71) (envelope-from <meyering@gmail.com>) id 1dZMI4-0004Rj-Fg
	for coreutils@gnu.org; Sun, 23 Jul 2017 15:11:44 -0400
Received: by mail-ua0-x236.google.com with SMTP id w45so65224250uac.5
	for <coreutils@gnu.org>; Sun, 23 Jul 2017 12:11:42 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
	h=mime-version:sender:in-reply-to:references:from:date:message-id
	:subject:to:cc:content-transfer-encoding;
	bh=XYuH52DF4++ADawzdMSALlbyKRWvG8v4GVgiEe6aEgQ=;
	b=kHSx3JlXx4DKdysmWvD3VB1z4oZM2qr3yL8PjhmKVwLRgIUIXOJhL0G8WaxBIY+fMn
	TKGEnKgIy/ldaLx3oH0cszHxfhS/aT8W0SFnVzvfjsY3t+nROOE1o755HNCcq1dMp2ty
	i22AV0ApQKAyVoktO43u9NXb8U+OWLtTrlpkAO1/ZXF6fXqPMjbk/ZWNOvnp36KQ65cC
	/WL9N3Xnbu5lHrbUm4vQWh5/kcX74otLseZ3z0IzsvWdaET+Jtez5fiDYaav7ufDNplt
	lqbg3EvK4BlzFNpMgK/S65dWrPepHlBHIRQbpox90WZwwZU8yobuMoCSmE+ZfD30/se7
	5Usg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
	d=1e100.net; s=20161025;
	h=x-gm-message-state:mime-version:sender:in-reply-to:references:from
	:date:message-id:subject:to:cc:content-transfer-encoding;
	bh=XYuH52DF4++ADawzdMSALlbyKRWvG8v4GVgiEe6aEgQ=;
	b=UF5IIXMuqO1MASkGuUqFlwc8UHFeI7lDZnpGZflwi8a1KJh2wNNWNrSV3k5AaXkZIi
	VH7OwSCBNwlwAscHAkmYjc6dL0R1W9COkr3g/0CSuyR8Tz69b9UGOVHhz4ZiohR5zSha
	iWCnQPWVk4kKVht8gnzEqwaxK2eYk0BP8y1tEls6sz+3Vcwq06PKuS4ZvWTIe2GD8o2q
	chU2OgKwN4ywO0vE00Bmdqrwq0Gk7oVIpx7rejOYdToSMu+nsOq4nfP+6qZANcfetZZT
	mOu1A7D89E5JVsanmFRMH7O7Orbo05xlb+ufuyhMCP3qmWynhQ3DO5xetFhchyam9DhQ
	+vwQ==
X-Gm-Message-State: AIVw113gDtFkHKZ21c2kXKcizAYIMYEgPQoHVFk8ZQRfuRl3HbThcW5M
	/vHp7R3eX5F4elsqBft5AAxIKoemlg==
X-Received: by 10.176.23.204 with SMTP id p12mr9025225uaf.156.1500837102315;
	Sun, 23 Jul 2017 12:11:42 -0700 (PDT)
MIME-Version: 1.0
Sender: meyering@gmail.com
Received: by 10.176.75.161 with HTTP; Sun, 23 Jul 2017 12:11:21 -0700 (PDT)
In-Reply-To: <cbceb699-d295-cd32-7d52-fc0527ae249d@draigBrady.com>
References: <20170720210313.26813-1-eggert@cs.ucla.edu>
	<20170720210313.26813-2-eggert@cs.ucla.edu>
	<cbceb699-d295-cd32-7d52-fc0527ae249d@draigBrady.com>
From: Jim Meyering <jim@meyering.net>
Date: Sun, 23 Jul 2017 21:11:21 +0200
X-Google-Sender-Auth: oDMyfvSbL3mmhEvsiRubu6YLx7M
Message-ID: <CA+8g5KFeUErLLmpWKcZr=Gr8PyesAm21MBvvJj9dZrtBP5OBZA@mail.gmail.com>
Subject: Re: [PATCH 2/2] shred: use explicit_bzero
To: =?UTF-8?Q?P=C3=A1draig_Brady?= <P@draigbrady.com>
Cc: Paul Eggert <eggert@cs.ucla.edu>, Coreutils <coreutils@gnu.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 2607:f8b0:400c:c08::236
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 23 Jul 2017 19:11:46 -0000

On Sun, Jul 23, 2017 at 10:30 AM, P=C3=A1draig Brady <P@draigbrady.com> wro=
te:
> On 20/07/17 14:03, Paul Eggert wrote:
>> * NEWS: Document this.
>> * bootstrap.conf (gnulib_modules): Add explicit_bzero.
>> * gl/lib/randint.c (randint_free):
>> * gl/lib/randread.c (randread_free):
>> * src/blake2/blake2-impl.h (secure_zero_memory):
>> * src/shred.c (dopass, do_wipefd):
>> Prefer explicit_bzero to memset when erasing secrets.
>
>> +  shred now erases buffers containing secrets via the explicit_bzero
>> +  function, which should be more reliable.
>> +  [potential bug has always been present in 'shred']
>
> shred doesn't have any secrets though right?
> Ah I see, it used to have secrets since it originally
> read the input file to seed the random number generator.
> How about the attached to remove this redundant zeroing entirely?
>
> Also I think I'll remove the explicit_bzero() from blake2
> as src/blake2 is copied directly from the reference implementation.
> So it would be best to change there and have a one way sync.

The patch looks fine.
Plan for blake2 makes sense.


From MAILER-DAEMON Sun Jul 23 15:21:10 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dZMRC-0006YW-3j
	for mharc-coreutils@gnu.org; Sun, 23 Jul 2017 15:21:10 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:51901)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eggert@cs.ucla.edu>) id 1dZMR9-0006Y0-Sd
	for coreutils@gnu.org; Sun, 23 Jul 2017 15:21:08 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eggert@cs.ucla.edu>) id 1dZMR6-0007xs-Nl
	for coreutils@gnu.org; Sun, 23 Jul 2017 15:21:07 -0400
Received: from zimbra.cs.ucla.edu ([131.179.128.68]:58608)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)
	(Exim 4.71) (envelope-from <eggert@cs.ucla.edu>) id 1dZMR6-0007xW-HF
	for coreutils@gnu.org; Sun, 23 Jul 2017 15:21:04 -0400
Received: from localhost (localhost [127.0.0.1])
	by zimbra.cs.ucla.edu (Postfix) with ESMTP id 63BC0160061;
	Sun, 23 Jul 2017 12:21:02 -0700 (PDT)
Received: from zimbra.cs.ucla.edu ([127.0.0.1])
	by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032)
	with ESMTP id NJB0kbaC0kRm; Sun, 23 Jul 2017 12:21:01 -0700 (PDT)
Received: from localhost (localhost [127.0.0.1])
	by zimbra.cs.ucla.edu (Postfix) with ESMTP id 9F84D160064;
	Sun, 23 Jul 2017 12:21:01 -0700 (PDT)
X-Virus-Scanned: amavisd-new at zimbra.cs.ucla.edu
Received: from zimbra.cs.ucla.edu ([127.0.0.1])
	by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026)
	with ESMTP id tbowZUZHR6OS; Sun, 23 Jul 2017 12:21:01 -0700 (PDT)
Received: from [192.168.1.9] (unknown [47.153.184.153])
	by zimbra.cs.ucla.edu (Postfix) with ESMTPSA id 82BAD160061;
	Sun, 23 Jul 2017 12:21:01 -0700 (PDT)
Subject: Re: [PATCH 2/2] shred: use explicit_bzero
To: =?UTF-8?Q?P=c3=a1draig_Brady?= <P@draigBrady.com>, coreutils@gnu.org
References: <20170720210313.26813-1-eggert@cs.ucla.edu>
	<20170720210313.26813-2-eggert@cs.ucla.edu>
	<cbceb699-d295-cd32-7d52-fc0527ae249d@draigBrady.com>
From: Paul Eggert <eggert@cs.ucla.edu>
Organization: UCLA Computer Science Department
Message-ID: <bc968d99-2fe5-e8bc-ca0b-4da8d3bf7cc8@cs.ucla.edu>
Date: Sun, 23 Jul 2017 12:20:55 -0700
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
	Thunderbird/52.2.1
MIME-Version: 1.0
In-Reply-To: <cbceb699-d295-cd32-7d52-fc0527ae249d@draigBrady.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy]
X-Received-From: 131.179.128.68
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 23 Jul 2017 19:21:08 -0000

P=C3=A1draig Brady wrote:
> How about the attached to remove this redundant zeroing entirely?
>=20
> Also I think I'll remove the explicit_bzero() from blake2
> as src/blake2 is copied directly from the reference implementation.
> So it would be best to change there and have a one way sync.

These both sound good to me too, and thanks.


From MAILER-DAEMON Sun Jul 23 17:41:35 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dZOd4-0000rU-Si
	for mharc-coreutils@gnu.org; Sun, 23 Jul 2017 17:41:34 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:39956)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <michael@felt.demon.nl>) id 1dZOd2-0000r4-Sw
	for coreutils@gnu.org; Sun, 23 Jul 2017 17:41:33 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <michael@felt.demon.nl>) id 1dZOcy-0004dO-0c
	for coreutils@gnu.org; Sun, 23 Jul 2017 17:41:32 -0400
Received: from felt-1.demon.nl ([80.101.98.107]:48210 helo=felt.demon.nl)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <michael@felt.demon.nl>) id 1dZOcx-0004aN-K6
	for coreutils@gnu.org; Sun, 23 Jul 2017 17:41:27 -0400
Received: from [192.168.129.4] (x004.home.local [192.168.129.4])
	by felt.demon.nl (AIX7.1/8.14.4/8.14.4) with ESMTP id v6NLfJZc11468912
	for <coreutils@gnu.org>; Sun, 23 Jul 2017 21:41:20 GMT
Subject: Re: [PATCH] tests: avoid a false failure on AIX
To: coreutils@gnu.org
References: <d6eba7e5-2d3c-0e22-ac30-84a24004bf4a@felt.demon.nl>
	<0557f29d-82ed-17ba-47fd-23c6bd3b299c@felt.demon.nl>
	<ff8a68dd-0ab8-728e-a1b9-daa95cc6fb02@draigBrady.com>
	<a32ed816-971d-1cde-6374-d2e50ee06352@bernhard-voelker.de>
	<4a039689-a156-3461-0d21-71d370375aa4@draigBrady.com>
From: Michael Felt <michael@felt.demon.nl>
Message-ID: <b6f8af7e-54a0-c65f-d001-063984aa2dfa@felt.demon.nl>
Date: Sun, 23 Jul 2017 23:41:20 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:50.0) Gecko/20100101
	Thunderbird/50.0
MIME-Version: 1.0
In-Reply-To: <4a039689-a156-3461-0d21-71d370375aa4@draigBrady.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-MIME-Autoconverted: from 8bit to quoted-printable by felt.demon.nl id
	v6NLfJZc11468912
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 80.101.98.107
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 23 Jul 2017 21:41:33 -0000

On 23/07/2017 12:39, P=C3=A1draig Brady wrote:
> On 23/07/17 03:07, Bernhard Voelker wrote:
>> On 07/23/2017 12:01 PM, P=C3=A1draig Brady wrote:
>>> -  sync norw 2>err
>>> +  sync norw 2>errt
>> While at it, I think we check the exit code as well.  WDYT?
> Good point!
> I've also done this:
>
> -  sync norw 2>errt
> +  returns_ 1 sync norw 2>errt
>
> cheers,
> P=C3=A1draig
>
>
I'll test the patch sometime tomorrow. Thanks for the patch!



From MAILER-DAEMON Tue Jul 25 05:16:42 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dZvxK-0002eN-EV
	for mharc-coreutils@gnu.org; Tue, 25 Jul 2017 05:16:42 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:36847)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eggert@cs.ucla.edu>) id 1dZvxH-0002cr-OI
	for coreutils@gnu.org; Tue, 25 Jul 2017 05:16:40 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eggert@cs.ucla.edu>) id 1dZvxG-0007Th-OQ
	for coreutils@gnu.org; Tue, 25 Jul 2017 05:16:39 -0400
Received: from zimbra.cs.ucla.edu ([131.179.128.68]:45806)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)
	(Exim 4.71) (envelope-from <eggert@cs.ucla.edu>)
	id 1dZvxC-0007Pm-4D; Tue, 25 Jul 2017 05:16:34 -0400
Received: from localhost (localhost [127.0.0.1])
	by zimbra.cs.ucla.edu (Postfix) with ESMTP id B2015160502;
	Tue, 25 Jul 2017 02:16:31 -0700 (PDT)
Received: from zimbra.cs.ucla.edu ([127.0.0.1])
	by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032)
	with ESMTP id gavlZgq2g3aL; Tue, 25 Jul 2017 02:16:30 -0700 (PDT)
Received: from localhost (localhost [127.0.0.1])
	by zimbra.cs.ucla.edu (Postfix) with ESMTP id C896416051F;
	Tue, 25 Jul 2017 02:16:30 -0700 (PDT)
X-Virus-Scanned: amavisd-new at zimbra.cs.ucla.edu
Received: from zimbra.cs.ucla.edu ([127.0.0.1])
	by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026)
	with ESMTP id v0uEqxmqf1_7; Tue, 25 Jul 2017 02:16:30 -0700 (PDT)
Received: from [192.168.1.9] (unknown [47.153.184.153])
	by zimbra.cs.ucla.edu (Postfix) with ESMTPSA id A970B160502;
	Tue, 25 Jul 2017 02:16:30 -0700 (PDT)
To: bug-findutils@gnu.org, Coreutils <coreutils@gnu.org>
From: Paul Eggert <eggert@cs.ucla.edu>
Subject: Several changes made to fts.c in Gnulib
Organization: UCLA Computer Science Department
Message-ID: <7f13dda4-c3ff-7a6e-defe-952a2b357d1e@cs.ucla.edu>
Date: Tue, 25 Jul 2017 02:16:30 -0700
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
	Thunderbird/52.2.1
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy]
X-Received-From: 131.179.128.68
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Tue, 25 Jul 2017 09:16:40 -0000

I found some subtle bugs in Gnulib fts.c, and created a test case that re=
liably=20
fails in reiserfs due to confusion with st_nlink. I added this test case =
to=20
Gnulib and fixed the bugs I found. As far as I can see, the only problems=
 that=20
should affect findutils and coreutils are minor performance problems. How=
ever, I=20
thought I'd send out some email as a heads-up, because the code is reason=
ably=20
subtle and I may have introduced some bugs.

Here are the patches I installed:

http://git.savannah.gnu.org/cgit/gnulib.git/commit/?id=3Db7363b496caf6060=
0eefa0a5f2d37f77a5dee606

http://git.savannah.gnu.org/cgit/gnulib.git/commit/?id=3D9c7ef4eef9540a0a=
762c4781d7ed5a1b8623e59b

http://git.savannah.gnu.org/cgit/gnulib.git/commit/?id=3Df7eb1b99e30517fc=
50c130cdecec24059a1b7c4f

http://git.savannah.gnu.org/cgit/gnulib.git/commit/?id=3D2649851d0409c3fa=
fee7cf396d11c10892ac0e53

http://git.savannah.gnu.org/cgit/gnulib.git/commit/?id=3Db446700e06b71f81=
70c46371261e8334888df7f8

http://git.savannah.gnu.org/cgit/gnulib.git/commit/?id=3De66b0b94e7239fb4=
7c9ccc2c704bb1149164d6ed


From MAILER-DAEMON Wed Jul 26 14:41:31 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1daRFT-0007em-NK
	for mharc-coreutils@gnu.org; Wed, 26 Jul 2017 14:41:31 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:34002)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <michael@felt.demon.nl>) id 1daRFS-0007eS-0T
	for coreutils@gnu.org; Wed, 26 Jul 2017 14:41:30 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <michael@felt.demon.nl>) id 1daRFO-0007Cy-UE
	for coreutils@gnu.org; Wed, 26 Jul 2017 14:41:30 -0400
Received: from felt-1.demon.nl ([80.101.98.107]:57795 helo=felt.demon.nl)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <michael@felt.demon.nl>) id 1daRFO-00077t-HQ
	for coreutils@gnu.org; Wed, 26 Jul 2017 14:41:26 -0400
Received: from [192.168.129.200] (x053 [192.168.129.53])
	by felt.demon.nl (AIX7.1/8.14.4/8.14.4) with ESMTP id v6QIfEMS10223728; 
	Wed, 26 Jul 2017 18:41:16 GMT
Subject: Re: [PATCH] tests: avoid a false failure on AIX
To: =?UTF-8?Q?P=c3=a1draig_Brady?= <P@draigBrady.com>,
	Coreutils <coreutils@gnu.org>
References: <d6eba7e5-2d3c-0e22-ac30-84a24004bf4a@felt.demon.nl>
	<0557f29d-82ed-17ba-47fd-23c6bd3b299c@felt.demon.nl>
	<ff8a68dd-0ab8-728e-a1b9-daa95cc6fb02@draigBrady.com>
From: Michael Felt <michael@felt.demon.nl>
Message-ID: <c4bc5ffa-6444-8833-9bd2-c0e150bd99b2@felt.demon.nl>
Date: Wed, 26 Jul 2017 20:41:15 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101
	Thunderbird/52.2.1
MIME-Version: 1.0
In-Reply-To: <ff8a68dd-0ab8-728e-a1b9-daa95cc6fb02@draigBrady.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-MIME-Autoconverted: from 8bit to quoted-printable by felt.demon.nl id
	v6QIfEMS10223728
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 80.101.98.107
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Wed, 26 Jul 2017 18:41:30 -0000



On 7/23/2017 12:01 PM, P=C3=A1draig Brady wrote:
> On 29/06/17 05:22, Michael Felt wrote:
>> FAIL: tests/misc/sync
>> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>>
>> + diff -u exp err
>> --- exp	2017-06-26 19:40:59 +0000
>> +++ err	2017-06-26 19:40:59 +0000
>> @@ -1 +1 @@
>> -sync: error opening 'norw': Permission denied
>> +sync: error opening 'norw': Is a directory
>> + fail=3D1
> The attached should fix this.
Finally got around to checking this, but ran into this (again I think).

ld: 0711-317 ERROR: Undefined symbol: .renameat

Will try again tomorrow.
>
> thanks,
> P=C3=A1draig



From MAILER-DAEMON Thu Jul 27 04:22:51 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dae4J-0002ef-BY
	for mharc-coreutils@gnu.org; Thu, 27 Jul 2017 04:22:51 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:38709)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <michael@felt.demon.nl>) id 1dae4E-0002cs-PY
	for coreutils@gnu.org; Thu, 27 Jul 2017 04:22:48 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <michael@felt.demon.nl>) id 1dae4C-00034k-1u
	for coreutils@gnu.org; Thu, 27 Jul 2017 04:22:46 -0400
Received: from felt-1.demon.nl ([80.101.98.107]:59659 helo=felt.demon.nl)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <michael@felt.demon.nl>) id 1dae4B-00033Z-Kx
	for coreutils@gnu.org; Thu, 27 Jul 2017 04:22:44 -0400
Received: from [192.168.129.200] (x053 [192.168.129.53])
	by felt.demon.nl (AIX7.1/8.14.4/8.14.4) with ESMTP id v6R8Mfls15007914
	for <coreutils@gnu.org>; Thu, 27 Jul 2017 08:22:42 GMT
Subject: Re: [PATCH] kill: fix signal number to name lookup on AIX
To: coreutils@gnu.org
References: <d6eba7e5-2d3c-0e22-ac30-84a24004bf4a@felt.demon.nl>
	<0557f29d-82ed-17ba-47fd-23c6bd3b299c@felt.demon.nl>
	<16f50a12-4a01-6fc4-bcc3-e2d37e6f0702@draigBrady.com>
From: Michael Felt <michael@felt.demon.nl>
Message-ID: <a04c541f-aa3d-86a0-a31b-8aeb5e665698@felt.demon.nl>
Date: Thu, 27 Jul 2017 10:22:43 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101
	Thunderbird/52.2.1
MIME-Version: 1.0
In-Reply-To: <16f50a12-4a01-6fc4-bcc3-e2d37e6f0702@draigBrady.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-MIME-Autoconverted: from 8bit to quoted-printable by felt.demon.nl id
	v6R8Mfls15007914
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 80.101.98.107
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 27 Jul 2017 08:22:48 -0000

I think both patches are working as expected. Had to find the "patched"=20
8.27 tarball you sent previously to get rid of the missing function.

Test performed on AIX 6.1 (6100-09-08-1642), on NFS filesystem (trying=20
on local filesystem on AIX 5.3, but will probably fail (initially)=20
because of /bin/sh glob issues. Will retry using SHELL=3D/bin/ksh and "ba=
sh-4"

Hint: I expect the issue here is that the ln is done on a NFS server=20
rather than "local" filesystem.

Also, not run (here) as root, so those tests were skipped here.

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D
    GNU coreutils 8.27.4-3c9d7: ./tests/test-suite.log
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D

# TOTAL: 592
# PASS:  414
# SKIP:  177
# XFAIL: 0
# FAIL:  1
# XPASS: 0
# ERROR: 0

from ./tests/test-suite.log

FAIL: tests/ln/sf-1
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D

++ initial_cwd_=3D/data/prj/gnu/coreutils/coreutils-8.27.4-3c9d7
+++ testdir_prefix_
+++ printf gt
++ pfx_=3Dgt
+++ mktempd_ /data/prj/gnu/coreutils/coreutils-8.27.4-3c9d7 gt-sf-1.sh.XX=
XX
+++ case $# in
+++ destdir_=3D/data/prj/gnu/coreutils/coreutils-8.27.4-3c9d7
+++ template_=3Dgt-sf-1.sh.XXXX
+++ MAX_TRIES_=3D4
+++ case $destdir_ in
+++ destdir_slash_=3D/data/prj/gnu/coreutils/coreutils-8.27.4-3c9d7/
+++ case $template_ in
++++ unset TMPDIR
+++ d=3D/data/prj/gnu/coreutils/coreutils-8.27.4-3c9d7/gt-sf-1.sh.tVrQ
+++ case $d in
+++ :
+++ test -d /data/prj/gnu/coreutils/coreutils-8.27.4-3c9d7/gt-sf-1.sh.tVr=
Q
++++ ls -dgo /data/prj/gnu/coreutils/coreutils-8.27.4-3c9d7/gt-sf-1.sh.tV=
rQ
+++ perms=3D'drwx--S--- 2 4096 Jul 27 08:08=20
/data/prj/gnu/coreutils/coreutils-8.27.4-3c9d7/gt-sf-1.sh.tVrQ'
+++ case $perms in
+++ :
+++ echo /data/prj/gnu/coreutils/coreutils-8.27.4-3c9d7/gt-sf-1.sh.tVrQ
+++ return
++ test_dir_=3D/data/prj/gnu/coreutils/coreutils-8.27.4-3c9d7/gt-sf-1.sh.=
tVrQ
++ cd /data/prj/gnu/coreutils/coreutils-8.27.4-3c9d7/gt-sf-1.sh.tVrQ
++ gl_init_sh_nl_=3D'
'
++ IFS=3D'
'
++ for sig_ in 1 2 3 13 15
+++ expr 1 + 128
++ eval 'trap '\''Exit 129'\'' 1'
+++ trap 'Exit 129' 1
++ for sig_ in 1 2 3 13 15
+++ expr 2 + 128
++ eval 'trap '\''Exit 130'\'' 2'
+++ trap 'Exit 130' 2
++ for sig_ in 1 2 3 13 15
+++ expr 3 + 128
++ eval 'trap '\''Exit 131'\'' 3'
+++ trap 'Exit 131' 3
++ for sig_ in 1 2 3 13 15
+++ expr 13 + 128
++ eval 'trap '\''Exit 141'\'' 13'
+++ trap 'Exit 141' 13
++ for sig_ in 1 2 3 13 15
+++ expr 15 + 128
++ eval 'trap '\''Exit 143'\'' 15'
+++ trap 'Exit 143' 15
++ trap remove_tmp_ 0
+ path_prepend_ ./src
+ test 1 '!=3D' 0
+ path_dir_=3D./src
+ case $path_dir_ in
+ abs_path_dir_=3D/data/prj/gnu/coreutils/coreutils-8.27.4-3c9d7/./src
+ case $abs_path_dir_ in
+=20
PATH=3D/data/prj/gnu/coreutils/coreutils-8.27.4-3c9d7/./src:/data/prj/gnu=
/coreutils/coreutils-8.27.4-3c9d7/src:/opt/bin:/
usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/usr/java6/jre/bin:/us=
r/java6/bin:/usr/vacpp/bin:/home/michael/bin:/op
t/svn/bin
+ create_exe_shims_ /data/prj/gnu/coreutils/coreutils-8.27.4-3c9d7/./src
+ case $EXEEXT in
+ return 0
+ shift
+ test 0 '!=3D' 0
+ export PATH
+ print_ver_ ln
+ require_built_ ln
+ skip_=3Dno
+ for i in '"$@"'
+ case " $built_programs " in
+ test no =3D yes
+ test yes =3D yes
+ local i
+ for i in '$*'
+ env ln --version
ln (GNU coreutils) 8.27.4-3c9d7
Copyright (C) 2017 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later=20
<http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by Mike Parker and David MacKenzie.
+ echo foo
+ ln -s . b
+ ln -sf a b
+ case $(cat err) in
++ cat err
+++ stat -f -c %l .
++ expr 4294967295 + 1
+ name_max_plus1=3D4294967296
+ test 4294967296 -gt 1
++ printf '%0*d' 4294967296 0
./tests/ln/sf-1.sh: line 38: printf: warning: 0: Result too large
./tests/ln/sf-1.sh: xrealloc: cannot allocate 133652480 bytes (134088304=20
bytes allocated)
FAIL tests/ln/sf-1.sh (exit status: 2)



On 7/23/2017 12:37 PM, P=C3=A1draig Brady wrote:
> On 29/06/17 05:22, Michael Felt wrote:
>> FAIL: tests/misc/kill
>> ++ env kill -l HUP TERM
>> ++ tail -n1
>> + SIGTERM=3D15
>> ++ env kill -l 15
>> + test EXIT =3D TERM
>> + fail=3D1
> I think our hack of using the wait() status access macros for
> manipulating the exit status from shells is failing for AIX.
> The attached avoids the use of WTERMSIG() on AIX.
> Perhaps we should do this for all platforms?
>
> cheers,
> P=C3=A1draig



From MAILER-DAEMON Thu Jul 27 04:24:53 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dae6H-0003Pr-Nc
	for mharc-coreutils@gnu.org; Thu, 27 Jul 2017 04:24:53 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:39756)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <michael@felt.demon.nl>) id 1dae6G-0003Pl-9D
	for coreutils@gnu.org; Thu, 27 Jul 2017 04:24:53 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <michael@felt.demon.nl>) id 1dae6D-0004Sf-JB
	for coreutils@gnu.org; Thu, 27 Jul 2017 04:24:52 -0400
Received: from felt-1.demon.nl ([80.101.98.107]:59672 helo=felt.demon.nl)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <michael@felt.demon.nl>) id 1dae6D-0004Qu-4t
	for coreutils@gnu.org; Thu, 27 Jul 2017 04:24:49 -0400
Received: from [192.168.129.200] (x053 [192.168.129.53])
	by felt.demon.nl (AIX7.1/8.14.4/8.14.4) with ESMTP id v6R8OjHE9437400
	for <coreutils@gnu.org>; Thu, 27 Jul 2017 08:24:45 GMT
Subject: Re: [PATCH] kill: fix signal number to name lookup on AIX
To: coreutils@gnu.org
References: <d6eba7e5-2d3c-0e22-ac30-84a24004bf4a@felt.demon.nl>
	<0557f29d-82ed-17ba-47fd-23c6bd3b299c@felt.demon.nl>
	<16f50a12-4a01-6fc4-bcc3-e2d37e6f0702@draigBrady.com>
	<a04c541f-aa3d-86a0-a31b-8aeb5e665698@felt.demon.nl>
From: Michael Felt <michael@felt.demon.nl>
Message-ID: <4225863e-5ed8-7cca-a4c0-dc5acd95d4b0@felt.demon.nl>
Date: Thu, 27 Jul 2017 10:24:48 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101
	Thunderbird/52.2.1
MIME-Version: 1.0
In-Reply-To: <a04c541f-aa3d-86a0-a31b-8aeb5e665698@felt.demon.nl>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-MIME-Autoconverted: from 8bit to quoted-printable by felt.demon.nl id
	v6R8OjHE9437400
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 80.101.98.107
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 27 Jul 2017 08:24:53 -0000



On 7/27/2017 10:22 AM, Michael Felt wrote:
> I think both patches are working as expected. Had to find the=20
> "patched" 8.27 tarball you sent previously to get rid of the missing=20
> function.
>
> Test performed on AIX 6.1 (6100-09-08-1642), on NFS filesystem (trying=20
> on local filesystem on AIX 5.3, but will probably fail (initially)=20
> because of /bin/sh glob issues. Will retry using SHELL=3D/bin/ksh and=20
> "bash-4"
>
On AIX 5.3 /bin/sh cannot run the tests...
make[4]: Entering directory '/tmp/coreutils-8.27.4-3c9d7'
make[4]: execvp: /bin/sh: Arg list too long

>
>
> On 7/23/2017 12:37 PM, P=C3=A1draig Brady wrote:
>> On 29/06/17 05:22, Michael Felt wrote:
>>> FAIL: tests/misc/kill
>>> ++ env kill -l HUP TERM
>>> ++ tail -n1
>>> + SIGTERM=3D15
>>> ++ env kill -l 15
>>> + test EXIT =3D TERM
>>> + fail=3D1
>> I think our hack of using the wait() status access macros for
>> manipulating the exit status from shells is failing for AIX.
>> The attached avoids the use of WTERMSIG() on AIX.
>> Perhaps we should do this for all platforms?
>>
>> cheers,
>> P=C3=A1draig
>
>
>



From MAILER-DAEMON Thu Jul 27 05:58:55 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dafZH-0003gY-Ai
	for mharc-coreutils@gnu.org; Thu, 27 Jul 2017 05:58:55 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:60389)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <aixtools@gmail.com>) id 1dafZD-0003ae-Et
	for coreutils@gnu.org; Thu, 27 Jul 2017 05:58:52 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <aixtools@gmail.com>) id 1dafZ9-0000VM-RD
	for coreutils@gnu.org; Thu, 27 Jul 2017 05:58:51 -0400
Received: from mail-oi0-x241.google.com ([2607:f8b0:4003:c06::241]:37169)
	by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)
	(Exim 4.71) (envelope-from <aixtools@gmail.com>) id 1dafZ9-0000V3-JP
	for coreutils@gnu.org; Thu, 27 Jul 2017 05:58:47 -0400
Received: by mail-oi0-x241.google.com with SMTP id j194so9390583oib.4
	for <coreutils@gnu.org>; Thu, 27 Jul 2017 02:58:47 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
	h=mime-version:in-reply-to:references:from:date:message-id:subject:to
	:content-transfer-encoding;
	bh=/p0bS96mDdfumLzZVNF4xH0j8ewuDVlTYN4WhSycAJs=;
	b=AErogaV1O8ipvWL5aZMyl+uZj7mK6rj6Fl8QgU+3MX/u1kt09M0GR92Db86qS7P3MV
	NPoxDj6TuA7LaaM9KXU27KXxsVw3zCAeKNsUI8KqrFUNldRVNfycb0jUYMpiHQRQpdjs
	ATmoZOw3gaFvMHvWXLkuOkQ2aMUNUca79ZZw/Z5v/nkYOscWTLlUSQZRL551J4KUL69A
	jwaZll0Glh0sMt7JErgRNuqxndbL4F9NE0AVPbJSQeocz2/rDLEgZM9EFLfo8b3MjiUC
	/taz/ZLYpDvYkFyrksfyJbIM/KdelfMBH1f/Ph5oVRiimYxM6Ibi3p9x7taqeoZeeWQM
	4ccw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
	d=1e100.net; s=20161025;
	h=x-gm-message-state:mime-version:in-reply-to:references:from:date
	:message-id:subject:to:content-transfer-encoding;
	bh=/p0bS96mDdfumLzZVNF4xH0j8ewuDVlTYN4WhSycAJs=;
	b=shP5bxqajM10jTVF2j5yfuQp2fTKUNL0AtUoudOn+TZQKi/YlGHPrD+a8KglJIsrre
	JKMPVbpsY1+hXYusL0h6jU41NOSzhJ009dCFRiJ7rqc5wW3gYl2b1b9oG0XAhekoaN4g
	cGW3HgyxwuGkcjL01dktT4USlD6sWUck1FzIrkU5r1OJxFV4qdl12bkuNxDO3dTsQejA
	I+b3X3fTJcMMrVQVIWq0fQWzqB9KPW7Eo6CyT6/B7nFVFtFTDK/MXhEq8ex6a/ZtK7aa
	aQhpy4vI+QxlEceQsgEMCVNIBITer5fGZmN6xEHDnhuSJyLBxswWgo57dHcNT+gKX0gs
	Kqmw==
X-Gm-Message-State: AIVw110oplPhGqxOldFbpirxVTOwZaMnNRJrK8L5TrE0t2PuUq0ysxt9
	XISTAt8gqbz8dOLjbpsSfjfIRsSt2w==
X-Received: by 10.202.215.130 with SMTP id o124mr3185534oig.204.1501149526532; 
	Thu, 27 Jul 2017 02:58:46 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.182.241.138 with HTTP; Thu, 27 Jul 2017 02:58:46 -0700 (PDT)
In-Reply-To: <CANvxniXPVXJw4YhXhQHTNFZqDUU7fx=zxcsJF=fdj5ZuDAPenQ@mail.gmail.com>
References: <8e8eaed6-2358-e2fe-272d-689101d2b30f@draigBrady.com>
	<CANvxniXzbZoQBDUpmhuT7zsK9d18DnZKCCYRiNDtQabv-E6tWQ@mail.gmail.com>
	<973883a1-e589-13fc-9e26-bd2b6e883e6d@draigBrady.com>
	<e683bdef-4976-615f-4318-636aa9051a0e@felt.demon.nl>
	<7dee834b-b26a-02d8-c07b-0871c8a98dad@draigBrady.com>
	<b26d7bc9-b1c1-3e14-1682-558a2d276866@felt.demon.nl>
	<fee5ef63-2fdd-ce4c-62ae-72fa9efe3f8f@draigBrady.com>
	<CANvxniV_dU5QjHAX=pW5r_nbDd7Xi=7FrA709y5LLAqtwi+0Jw@mail.gmail.com>
	<b6c6b80c-69a7-44cf-8703-c1703ae95682@draigBrady.com>
	<CANvxniXPVXJw4YhXhQHTNFZqDUU7fx=zxcsJF=fdj5ZuDAPenQ@mail.gmail.com>
From: Michael Felt <aixtools@gmail.com>
Date: Thu, 27 Jul 2017 11:58:46 +0200
Message-ID: <CANvxniXyEZcfXu1Zk8Dsc-awQmkxqLA+uxjZbE==ZybZn3X9Yg@mail.gmail.com>
Subject: Re: coreutils-8.27 released [stable]
To: =?UTF-8?Q?P=C3=A1draig_Brady?= <P@draigbrady.com>, 
	Coreutils <coreutils@gnu.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 2607:f8b0:4003:c06::241
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 27 Jul 2017 09:58:53 -0000

On AIX - I expect the reason this test fails:

FAIL: tests/id/setgid
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D


comes from the logic here:
...
+ local rm_version=3D8.27.4-3c9d7
+ case ":$rm_version:" in
+ gp1=3D4294967294
++ expr 4294967294 + 1
+ gp1=3D4294967295
+ echo 4294967295
+ chroot --skip-chdir --user=3Dnobody:+4294967295 --groups=3D / env
PATH=3D/home/michael/coreutils-8.27.4-3c9d7/./src:/data/prj
/gnu/coreutils/coreutils-8.27.4-3c9d7/src:/opt/bin:/usr/bin:/etc:/usr/sbin:=
/usr/ucb:/usr/bin/X11:/sbin:/usr/java6/jre/bin
:/usr/java6/bin:/usr/vacpp/bin:/home/michael/bin:/opt/svn/bin id -G
chroot: invalid group

...
It has been a long long time, but from memory the userid and groupid
-1 (aka +4294967295) are reserved on AIX since forever. The accepted
values for nobody:nobody are -2:-2 aka 4294967294:4294967294

michael@x071:[/home/michael/coreutils-8.27.4-3c9d7]grep nobody
/etc/passwd /etc/group
/etc/passwd:nobody:!:4294967294:4294967294::/:
/etc/group:nobody:!:4294967294:nobody,lpd

>From the command line - some feedback -

michael@x071:[/home/michael/coreutils-8.27.4-3c9d7]touch xx
michael@x071:[/home/michael/coreutils-8.27.4-3c9d7]ls -l xx
-rw------- 1 root felt 0 Jul 27 09:43 xx


michael@x071:[/home/michael/coreutils-8.27.4-3c9d7]src/chgrp --version
chgrp (GNU coreutils) 8.27.4-3c9d7
Copyright (C) 2017 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.htm=
l>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by David MacKenzie and Jim Meyering.
michael@x071:[/home/michael/coreutils-8.27.4-3c9d7]src/chgrp 4294967294 xx
michael@x071:[/home/michael/coreutils-8.27.4-3c9d7]ls -l xx
-rw------- 1 root nobody 0 Jul 27 09:43 xx
michael@x071:[/home/michael/coreutils-8.27.4-3c9d7]src/chgrp 4294967295 xx
michael@x071:[/home/michael/coreutils-8.27.4-3c9d7]ls -l xx
-rw------- 1 root nobody 0 Jul 27 09:43 xx
michael@x071:[/home/michael/coreutils-8.27.4-3c9d7]src/chgrp felt xx
michael@x071:[/home/michael/coreutils-8.27.4-3c9d7]src/chgrp 4294967295 xx
michael@x071:[/home/michael/coreutils-8.27.4-3c9d7]ls -l xx
-rw------- 1 root felt 0 Jul 27 09:43 xx
michael@x071:[/home/michael/coreutils-8.27.4-3c9d7]src/chgrp 4294967293 xx
michael@x071:[/home/michael/coreutils-8.27.4-3c9d7]ls -l xx
-rw------- 1 root 4294967293 0 Jul 27 09:43 xx
michael@x071:[/home/michael/coreutils-8.27.4-3c9d7]id
uid=3D0(root) gid=3D0(system)
groups=3D0(system),2(bin),3(sys),7(security),8(cron),10(audit),11(lp),13(db=
sysadm),207(idsldap)

Note: src/chmod does not give an error status!!

michael@x071:[/home/michael/coreutils-8.27.4-3c9d7]src/chgrp 4294967295 xx
michael@x071:[/home/michael/coreutils-8.27.4-3c9d7]echo $?
0
michael@x071:[/home/michael/coreutils-8.27.4-3c9d7]ls -l xx
-rw------- 1 root 4294967293 0 Jul 27 09:43 xx


On Thu, Jul 27, 2017 at 11:35 AM, Michael Felt <aixtools@gmail.com> wrote:
> Seems I was sending mails to myself before my vacation. As I am behind
> a firewall that won't let me use my VPN - back to gmail :)
>
> Anyway, after applying your two patches - this is the result on
> http://download.aixtools.net/test/test-suite-ksh-AIX6109-root-jfs2.log
>
> I'll forward the "expensive and VERBOSE" tests I ran last June. Sigh.
>
> Michael
>
> On Wed, Jun 21, 2017 at 6:33 PM, P=C3=A1draig Brady <P@draigbrady.com> wr=
ote:
>> On 21/06/17 03:39, Michael Felt wrote:
>>> So, after downloading the tarball, and updating local.mk I was able to
>>> build from the sources.
>>>
>>> Will provide the details later, but summary of "make check" is:
>>>
>>> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D
>>> Testsuite summary for GNU coreutils 8.27.4-3c9d7
>>> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D
>>> # TOTAL: 592
>>> # PASS:  412
>>> # SKIP:  177
>>> # XFAIL: 0
>>> # FAIL:  3
>>> # XPASS: 0
>>> # ERROR: 0
>>> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D
>>> See ./tests/test-suite.log
>>> Please report to bug-coreutils@gnu.org
>>> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D
>>
>> Add VERBOSE=3Dyes to the make check to get more details in tests/test-su=
ite.log
>> (and you might attached that?)


From MAILER-DAEMON Thu Jul 27 11:28:53 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dakib-00034R-Ro
	for mharc-coreutils@gnu.org; Thu, 27 Jul 2017 11:28:53 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:38756)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <michael@felt.demon.nl>) id 1dakiY-0002uO-TA
	for coreutils@gnu.org; Thu, 27 Jul 2017 11:28:51 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <michael@felt.demon.nl>) id 1dakiU-0005Yl-Se
	for coreutils@gnu.org; Thu, 27 Jul 2017 11:28:50 -0400
Received: from felt-1.demon.nl ([80.101.98.107]:60656 helo=felt.demon.nl)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <michael@felt.demon.nl>) id 1dakiU-0005Y0-Gj
	for coreutils@gnu.org; Thu, 27 Jul 2017 11:28:46 -0400
Received: from [192.168.129.4] (x004.home.local [192.168.129.4])
	by felt.demon.nl (AIX7.1/8.14.4/8.14.4) with ESMTP id v6RFSiQW8257736
	for <coreutils@gnu.org>; Thu, 27 Jul 2017 15:28:45 GMT
Subject: Re: [PATCH] kill: fix signal number to name lookup on AIX
From: Michael Felt <michael@felt.demon.nl>
To: coreutils@gnu.org
References: <d6eba7e5-2d3c-0e22-ac30-84a24004bf4a@felt.demon.nl>
	<0557f29d-82ed-17ba-47fd-23c6bd3b299c@felt.demon.nl>
	<16f50a12-4a01-6fc4-bcc3-e2d37e6f0702@draigBrady.com>
	<a04c541f-aa3d-86a0-a31b-8aeb5e665698@felt.demon.nl>
	<4225863e-5ed8-7cca-a4c0-dc5acd95d4b0@felt.demon.nl>
Message-ID: <56d0bba1-a5f4-431b-8677-0db3db013349@felt.demon.nl>
Date: Thu, 27 Jul 2017 17:28:44 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:50.0) Gecko/20100101
	Thunderbird/50.0
MIME-Version: 1.0
In-Reply-To: <4225863e-5ed8-7cca-a4c0-dc5acd95d4b0@felt.demon.nl>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Content-Language: en-US
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 80.101.98.107
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 27 Jul 2017 15:28:51 -0000

On 27/07/2017 10:24, Michael Felt wrote:
> On AIX 5.3 /bin/sh cannot run the tests...
> make[4]: Entering directory '/tmp/coreutils-8.27.4-3c9d7'
> make[4]: execvp: /bin/sh: Arg list too long 

Seems, as I feared, /bin/sh is hard-coded in make check.

At least, export SHELL=/different/shell; ./configure ...; make; make 
check - does not have any impact.



From MAILER-DAEMON Thu Jul 27 13:11:04 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1damJU-00049E-46
	for mharc-coreutils@gnu.org; Thu, 27 Jul 2017 13:11:04 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:35903)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <michael@felt.demon.nl>) id 1damJR-00047j-C2
	for coreutils@gnu.org; Thu, 27 Jul 2017 13:11:02 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <michael@felt.demon.nl>) id 1damJN-0001AT-4N
	for coreutils@gnu.org; Thu, 27 Jul 2017 13:11:01 -0400
Received: from felt-1.demon.nl ([80.101.98.107]:60890 helo=felt.demon.nl)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <michael@felt.demon.nl>) id 1damJM-00019W-N6
	for coreutils@gnu.org; Thu, 27 Jul 2017 13:10:57 -0400
Received: from [192.168.129.4] (x004.home.local [192.168.129.4])
	by felt.demon.nl (AIX7.1/8.14.4/8.14.4) with ESMTP id v6RHAqkV8257776; 
	Thu, 27 Jul 2017 17:10:52 GMT
Subject: Re: [PATCH] kill: fix signal number to name lookup on AIX
To: coreutils@gnu.org, =?UTF-8?Q?P=c3=a1draig_Brady?= <P@draigBrady.com>
References: <d6eba7e5-2d3c-0e22-ac30-84a24004bf4a@felt.demon.nl>
	<0557f29d-82ed-17ba-47fd-23c6bd3b299c@felt.demon.nl>
	<16f50a12-4a01-6fc4-bcc3-e2d37e6f0702@draigBrady.com>
	<a04c541f-aa3d-86a0-a31b-8aeb5e665698@felt.demon.nl>
From: Michael Felt <michael@felt.demon.nl>
Message-ID: <4bef75c7-5e7d-f4d5-4acf-da4719d04433@felt.demon.nl>
Date: Thu, 27 Jul 2017 19:10:52 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:50.0) Gecko/20100101
	Thunderbird/50.0
MIME-Version: 1.0
In-Reply-To: <a04c541f-aa3d-86a0-a31b-8aeb5e665698@felt.demon.nl>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Content-Language: en-US
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 80.101.98.107
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 27 Jul 2017 17:11:02 -0000

On 27/07/2017 10:22, Michael Felt wrote:
> Hint: I expect the issue here is that the ln is done on a NFS server 
> rather than "local" filesystem.
>
> Also, not run (here) as root, so those tests were skipped here.
>
> ========================================================
>    GNU coreutils 8.27.4-3c9d7: ./tests/test-suite.log
> ========================================================
>
> # TOTAL: 592
> # PASS:  414
> # SKIP:  177
> # XFAIL: 0
> # FAIL:  1
> # XPASS: 0
> # ERROR: 0
>
> from ./tests/test-suite.log
>
> FAIL: tests/ln/sf-1
> =================== 

Ok, it looks like the test above above succeeds when on jfs2.

Running as root - a test previously skipped now fails. And it is a real 
bug, it seems, as the AIX cp command (/usr/bin/cp) works as expected.

Look at the commands used to create the c4 and c8 files (the directory 
was given mode 777 to permit the writing regardless of userid.


michael@x071:[/home/michael/coreutils-8.27.4-3c9d7]touch c
michael@x071:[/home/michael/coreutils-8.27.4-3c9d7]chmod a=r,ug+sx c
michael@x071:[/home/michael/coreutils-8.27.4-3c9d7]ls -l c
-r-sr-sr-- 1 michael felt 0 Jul 27 16:53 c
michael@x071:[/home/michael/coreutils-8.27.4-3c9d7]su
root's Password:
michael@x071:[/home/michael/coreutils-8.27.4-3c9d7]cp -p c c1
michael@x071:[/home/michael/coreutils-8.27.4-3c9d7]chroot --skip-chdir 
--user=nobody src/cp -p c c2
chroot: option --skip-chdir only permitted if NEWROOT is old '/'
Try 'chroot --help' for more information.
michael@x071:[/home/michael/coreutils-8.27.4-3c9d7]chroot --skip-chdir 
--user=nobody / src/cp -p c c2
cp: cannot create regular file 'c2': Permission denied
michael@x071:[/home/michael/coreutils-8.27.4-3c9d7]ls -l c c??
ls: cannot access 'c??': No such file or directory
-r-sr-sr-- 1 michael felt 0 Jul 27 16:53 c
michael@x071:[/home/michael/coreutils-8.27.4-3c9d7]ls -l c c?
-r-sr-sr-- 1 michael felt 0 Jul 27 16:53 c
-r-sr-sr-- 1 michael felt 0 Jul 27 16:53 c1
michael@x071:[/home/michael/coreutils-8.27.4-3c9d7]chmod 777 .
michael@x071:[/home/michael/coreutils-8.27.4-3c9d7]chroot --skip-chdir 
--user=nobody / src/cp -p c c2
michael@x071:[/home/michael/coreutils-8.27.4-3c9d7]ls -l c c?
-r-sr-sr-- 1 michael felt 0 Jul 27 16:53 c
-r-sr-sr-- 1 michael felt 0 Jul 27 16:53 c1
-r-sr-sr-- 1 nobody  felt 0 Jul 27 16:53 c2
michael@x071:[/home/michael/coreutils-8.27.4-3c9d7]su nobody -c src/cp 
-p c c3
michael@x071:[/home/michael/coreutils-8.27.4-3c9d7]ls -l c c?
-r-sr-sr-- 1 michael felt 0 Jul 27 16:53 c
-r-sr-sr-- 1 michael felt 0 Jul 27 16:53 c1
-r-sr-sr-- 1 nobody  felt 0 Jul 27 16:53 c2
-r-sr-sr-- 1 nobody  felt 0 Jul 27 16:53 c3
michael@x071:[/home/michael/coreutils-8.27.4-3c9d7]su nobody -c 
/usr/bin/cp -p c c4
michael@x071:[/home/michael/coreutils-8.27.4-3c9d7]ls -l c c?
-r-sr-sr-- 1 michael felt 0 Jul 27 16:53 c
-r-sr-sr-- 1 michael felt 0 Jul 27 16:53 c1
-r-sr-sr-- 1 nobody  felt 0 Jul 27 16:53 c2
-r-sr-sr-- 1 nobody  felt 0 Jul 27 16:53 c3
-r-xr-xr-- 1 nobody  felt 0 Jul 27 16:53 c4
michael@x071:[/home/michael/coreutils-8.27.4-3c9d7]

Here I exit from SU mode.

As the file c is owned by michael - not surprising that c5 has 
permission bits. But c2 is owned by nobody, and I would not expect those 
special bits to copy as michael

michael@x071:[/home/michael/coreutils-8.27.4-3c9d7]id
uid=199(michael) gid=1954(felt) 
groups=1954(felt),1(staff),33(httpd),208(xdata)
michael@x071:[/home/michael/coreutils-8.27.4-3c9d7]cp -p c c5
michael@x071:[/home/michael/coreutils-8.27.4-3c9d7]src/cp -p c c6
michael@x071:[/home/michael/coreutils-8.27.4-3c9d7]ls -l c c?
-r-sr-sr-- 1 michael felt 0 Jul 27 16:53 c
-r-sr-sr-- 1 michael felt 0 Jul 27 16:53 c1
-r-sr-sr-- 1 nobody  felt 0 Jul 27 16:53 c2
-r-sr-sr-- 1 nobody  felt 0 Jul 27 16:53 c3
-r-xr-xr-- 1 nobody  felt 0 Jul 27 16:53 c4
-r-sr-sr-- 1 michael felt 0 Jul 27 16:53 c5
-r-sr-sr-- 1 michael felt 0 Jul 27 16:53 c6
michael@x071:[/home/michael/coreutils-8.27.4-3c9d7]src/cp -p c2 c6
cp: cannot create regular file 'c6': Permission denied
michael@x071:[/home/michael/coreutils-8.27.4-3c9d7]src/cp -p c2 c7
michael@x071:[/home/michael/coreutils-8.27.4-3c9d7]ls -l c c?
-r-sr-sr-- 1 michael felt 0 Jul 27 16:53 c
-r-sr-sr-- 1 michael felt 0 Jul 27 16:53 c1
-r-sr-sr-- 1 nobody  felt 0 Jul 27 16:53 c2
-r-sr-sr-- 1 nobody  felt 0 Jul 27 16:53 c3
-r-xr-xr-- 1 nobody  felt 0 Jul 27 16:53 c4
-r-sr-sr-- 1 michael felt 0 Jul 27 16:53 c5
-r-sr-sr-- 1 michael felt 0 Jul 27 16:53 c6
-r-sr-sr-- 1 michael felt 0 Jul 27 16:53 c7
michael@x071:[/home/michael/coreutils-8.27.4-3c9d7]/usr/bin/cp -p c2 c8
michael@x071:[/home/michael/coreutils-8.27.4-3c9d7]ls -l c c?
-r-sr-sr-- 1 michael felt 0 Jul 27 16:53 c
-r-sr-sr-- 1 michael felt 0 Jul 27 16:53 c1
-r-sr-sr-- 1 nobody  felt 0 Jul 27 16:53 c2
-r-sr-sr-- 1 nobody  felt 0 Jul 27 16:53 c3
-r-xr-xr-- 1 nobody  felt 0 Jul 27 16:53 c4
-r-sr-sr-- 1 michael felt 0 Jul 27 16:53 c5
-r-sr-sr-- 1 michael felt 0 Jul 27 16:53 c6
-r-sr-sr-- 1 michael felt 0 Jul 27 16:53 c7
-r-xr-xr-- 1 michael felt 0 Jul 27 16:53 c8
michael@x071:[/home/michael/coreutils-8.27.4-3c9d7]

So, it seems that src/cp is still copying the file mode, regardless.



From MAILER-DAEMON Sun Jul 30 16:46:53 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dbv6z-0007yT-RY
	for mharc-coreutils@gnu.org; Sun, 30 Jul 2017 16:46:53 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:37841)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <reuti@staff.uni-marburg.de>) id 1dbv6x-0007yM-Fh
	for coreutils@gnu.org; Sun, 30 Jul 2017 16:46:52 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <reuti@staff.uni-marburg.de>) id 1dbv6t-000828-Hq
	for coreutils@gnu.org; Sun, 30 Jul 2017 16:46:51 -0400
Received: from vhrz24.hrz.uni-marburg.de ([137.248.1.34]:58534)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)
	(Exim 4.71) (envelope-from <reuti@staff.uni-marburg.de>)
	id 1dbv6t-0007zZ-7N
	for coreutils@gnu.org; Sun, 30 Jul 2017 16:46:47 -0400
Received: from [10.0.1.5] (pD9E3CA99.dip0.t-ipconnect.de [217.227.202.153])
	(authenticated bits=0)
	by vhrz24.HRZ.Uni-Marburg.DE (8.14.4/8.14.4/Debian-8+deb8u2) with ESMTP
	id v6UKkU5c056389
	(version=TLSv1/SSLv3 cipher=ECDHE-RSA-AES128-SHA bits=128 verify=NOT); 
	Sun, 30 Jul 2017 22:46:32 +0200
Subject: Re: How the ls command interpret this?
Mime-Version: 1.0 (Apple Message framework v1085)
Content-Type: text/plain; charset=iso-8859-1
From: Reuti <reuti@staff.uni-marburg.de>
In-Reply-To: <8ad70c715e714ae4d1cf445fb39b803f@free.fr>
Date: Sun, 30 Jul 2017 22:46:29 +0200
Cc: coreutils@gnu.org
Content-Transfer-Encoding: quoted-printable
Message-Id: <187E547F-042F-421F-B239-247DFC112D62@staff.uni-marburg.de>
References: <ccb12f08dc137b606f3961a766ec17da@free.fr>
	<c7db82c1-b9b5-5772-b091-36c60bd90d7b@redhat.com>
	<8ad70c715e714ae4d1cf445fb39b803f@free.fr>
To: BaBa <for.ad@free.fr>
X-Pgp-Agent: GPGMail 1.4.1
X-Mailer: Apple Mail (2.1085)
X-Null-Tag: d0cdf18d2def7aeb155c9b6eedf9a105
X-UniMR-MailScanner-Information: see
	http://www.uni-marburg.de/hrz/internet/mail/spam/
X-UniMR-MailScanner-ID: v6UKkU5c056389
X-UniMR-MailScanner: Found to be clean
X-UniMR-MailScanner-From: reuti@staff.uni-marburg.de
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
	[fuzzy]
X-Received-From: 137.248.1.34
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 30 Jul 2017 20:46:52 -0000

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


Am 03.07.2017 um 20:30 schrieb BaBa:

> Le 2017-07-03 19:17, Eric Blake a =E9crit :
>> Case 3:
>> $ mkdir foo
>> $ cd foo
>> $ touch a[b]    # the glob doesn't match, so it is passed unchanged
>> $ ls ?[b]       # the glob doesn't match, so it is passed unchanged
>> ls: cannot access '?[b]': No such file or directory
>> $ cd ..
>> $ rm -rf foo
>=20
> Yes understood, the glob didn't match.

And if globbing fails, it won't try it without. I mean, it could use =
?'[b]' and try again which would succeed.

$ ls ?'[b]'
a[b]
-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - https://gpgtools.org

iEYEARECAAYFAll+RaYACgkQo/GbGkBRnRo8CACfTVsYbnmUok1uTuNauBgJSB0f
tiEAmQEymx4yw0B4Nu9GvSvo3DEHJ4Y8
=3DNGjE
-----END PGP SIGNATURE-----


From MAILER-DAEMON Sun Jul 30 20:19:24 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dbyQe-00070J-3s
	for mharc-coreutils@gnu.org; Sun, 30 Jul 2017 20:19:24 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:40092)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eggert@cs.ucla.edu>) id 1dbyQc-000706-9t
	for coreutils@gnu.org; Sun, 30 Jul 2017 20:19:23 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eggert@cs.ucla.edu>) id 1dbyQb-0006S3-6J
	for coreutils@gnu.org; Sun, 30 Jul 2017 20:19:22 -0400
Received: from zimbra.cs.ucla.edu ([131.179.128.68]:54518)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)
	(Exim 4.71) (envelope-from <eggert@cs.ucla.edu>) id 1dbyQa-0006Rq-TG
	for coreutils@gnu.org; Sun, 30 Jul 2017 20:19:21 -0400
Received: from localhost (localhost [127.0.0.1])
	by zimbra.cs.ucla.edu (Postfix) with ESMTP id 149F61606F0
	for <coreutils@gnu.org>; Sun, 30 Jul 2017 17:19:20 -0700 (PDT)
Received: from zimbra.cs.ucla.edu ([127.0.0.1])
	by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032)
	with ESMTP id dITddDXfcigk; Sun, 30 Jul 2017 17:19:19 -0700 (PDT)
Received: from localhost (localhost [127.0.0.1])
	by zimbra.cs.ucla.edu (Postfix) with ESMTP id 3FD5A1606F7;
	Sun, 30 Jul 2017 17:19:19 -0700 (PDT)
X-Virus-Scanned: amavisd-new at zimbra.cs.ucla.edu
Received: from zimbra.cs.ucla.edu ([127.0.0.1])
	by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026)
	with ESMTP id tQIgSQgPdgmH; Sun, 30 Jul 2017 17:19:19 -0700 (PDT)
Received: from day.fios-router.home (unknown [47.153.184.153])
	by zimbra.cs.ucla.edu (Postfix) with ESMTPSA id 1F8CD1606F0;
	Sun, 30 Jul 2017 17:19:19 -0700 (PDT)
From: Paul Eggert <eggert@cs.ucla.edu>
To: coreutils@gnu.org
Cc: Paul Eggert <eggert@cs.ucla.edu>
Subject: [PATCH 2/4] shred: avoid rename race
Date: Sun, 30 Jul 2017 17:19:05 -0700
Message-Id: <1501460347-19275-2-git-send-email-eggert@cs.ucla.edu>
X-Mailer: git-send-email 2.7.4
In-Reply-To: <1501460347-19275-1-git-send-email-eggert@cs.ucla.edu>
References: <1501460347-19275-1-git-send-email-eggert@cs.ucla.edu>
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy]
X-Received-From: 131.179.128.68
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 31 Jul 2017 00:19:23 -0000

Use renameat2 to avoid a rename race condition, on recent-enough
GNU/Linux.
* bootstrap.conf (gnulib_modules): Add renameat2.
* src/shred.c: Include renameat2.h.
(wipename): Use renameat2 instead of rename.
---
 bootstrap.conf |  1 +
 src/shred.c    | 75 ++++++++++++++++++++++++----------------------------------
 2 files changed, 32 insertions(+), 44 deletions(-)

diff --git a/bootstrap.conf b/bootstrap.conf
index 9064a94..188b1ef 100644
--- a/bootstrap.conf
+++ b/bootstrap.conf
@@ -206,6 +206,7 @@ gnulib_modules="
   remove
   rename
   renameat
+  renameat2
   rmdir
   root-dev-ino
   rpmatch
diff --git a/src/shred.c b/src/shred.c
index e8fe307..47cfc26 100644
--- a/src/shred.c
+++ b/src/shred.c
@@ -93,6 +93,7 @@
 #include "human.h"
 #include "randint.h"
 #include "randread.h"
+#include "renameat2.h"
 #include "stat-size.h"
 
 /* Default number of times to overwrite.  */
@@ -1077,7 +1078,6 @@ wipename (char *oldname, char const *qoldname, struct Options const *flags)
 {
   char *newname = xstrdup (oldname);
   char *base = last_component (newname);
-  size_t len = base_len (base);
   char *dir = dir_name (newname);
   char *qdir = xstrdup (quotef (dir));
   bool first = true;
@@ -1090,49 +1090,36 @@ wipename (char *oldname, char const *qoldname, struct Options const *flags)
   if (flags->verbose)
     error (0, 0, _("%s: removing"), qoldname);
 
-  while ((flags->remove_file != remove_unlink) && len)
-    {
-      memset (base, nameset[0], len);
-      base[len] = 0;
-      do
-        {
-          struct stat st;
-          if (lstat (newname, &st) < 0)
-            {
-              if (rename (oldname, newname) == 0)
-                {
-                  if (0 <= dir_fd && dosync (dir_fd, qdir) != 0)
-                    ok = false;
-                  if (flags->verbose)
-                    {
-                      /*
-                       * People seem to understand this better than talking
-                       * about renaming oldname.  newname doesn't need
-                       * quoting because we picked it.  oldname needs to
-                       * be quoted only the first time.
-                       */
-                      char const *old = (first ? qoldname : oldname);
-                      error (0, 0, _("%s: renamed to %s"),
-                             old, newname);
-                      first = false;
-                    }
-                  memcpy (oldname + (base - newname), base, len + 1);
-                  break;
-                }
-              else
-                {
-                  /* The rename failed: give up on this length.  */
-                  break;
-                }
-            }
-          else
-            {
-              /* newname exists, so increment BASE so we use another */
-            }
-        }
-      while (incname (base, len));
-      len--;
-    }
+  if (flags->remove_file != remove_unlink)
+    for (size_t len = base_len (base); len != 0; len--)
+      {
+        memset (base, nameset[0], len);
+        base[len] = 0;
+        bool rename_ok;
+        while (! (rename_ok = (renameat2 (AT_FDCWD, oldname, AT_FDCWD, newname,
+                                          RENAME_NOREPLACE)
+                               == 0))
+               && errno == EEXIST && incname (base, len))
+          continue;
+        if (rename_ok)
+          {
+            if (0 <= dir_fd && dosync (dir_fd, qdir) != 0)
+              ok = false;
+            if (flags->verbose)
+              {
+                /* People seem to understand this better than talking
+                   about renaming OLDNAME.  NEWNAME doesn't need
+                   quoting because we picked it.  OLDNAME needs to be
+                   quoted only the first time.  */
+                char const *old = first ? qoldname : oldname;
+                error (0, 0, _("%s: renamed to %s"), old, newname);
+                first = false;
+              }
+            memcpy (oldname + (base - newname), base, len + 1);
+            break;
+          }
+      }
+
   if (unlink (oldname) != 0)
     {
       error (0, errno, _("%s: failed to remove"), qoldname);
-- 
2.7.4



From MAILER-DAEMON Sun Jul 30 20:19:24 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dbyQe-00070m-LB
	for mharc-coreutils@gnu.org; Sun, 30 Jul 2017 20:19:24 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:40090)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eggert@cs.ucla.edu>) id 1dbyQc-000705-4A
	for coreutils@gnu.org; Sun, 30 Jul 2017 20:19:22 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eggert@cs.ucla.edu>) id 1dbyQX-0006QE-7Z
	for coreutils@gnu.org; Sun, 30 Jul 2017 20:19:22 -0400
Received: from zimbra.cs.ucla.edu ([131.179.128.68]:54494)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)
	(Exim 4.71) (envelope-from <eggert@cs.ucla.edu>) id 1dbyQX-0006Q4-0q
	for coreutils@gnu.org; Sun, 30 Jul 2017 20:19:17 -0400
Received: from localhost (localhost [127.0.0.1])
	by zimbra.cs.ucla.edu (Postfix) with ESMTP id 03B1C1606F9
	for <coreutils@gnu.org>; Sun, 30 Jul 2017 17:19:16 -0700 (PDT)
Received: from zimbra.cs.ucla.edu ([127.0.0.1])
	by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032)
	with ESMTP id mDzGA2YOVYOv; Sun, 30 Jul 2017 17:19:15 -0700 (PDT)
Received: from localhost (localhost [127.0.0.1])
	by zimbra.cs.ucla.edu (Postfix) with ESMTP id 5FD1D1606F8;
	Sun, 30 Jul 2017 17:19:15 -0700 (PDT)
X-Virus-Scanned: amavisd-new at zimbra.cs.ucla.edu
Received: from zimbra.cs.ucla.edu ([127.0.0.1])
	by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026)
	with ESMTP id xNZl8rgdQsiI; Sun, 30 Jul 2017 17:19:15 -0700 (PDT)
Received: from day.fios-router.home (unknown [47.153.184.153])
	by zimbra.cs.ucla.edu (Postfix) with ESMTPSA id 208741606F7;
	Sun, 30 Jul 2017 17:19:15 -0700 (PDT)
From: Paul Eggert <eggert@cs.ucla.edu>
To: coreutils@gnu.org
Cc: Paul Eggert <eggert@cs.ucla.edu>
Subject: [PATCH 1/4] build: update gnulib submodule to latest
Date: Sun, 30 Jul 2017 17:19:04 -0700
Message-Id: <1501460347-19275-1-git-send-email-eggert@cs.ucla.edu>
X-Mailer: git-send-email 2.7.4
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy]
X-Received-From: 131.179.128.68
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 31 Jul 2017 00:19:23 -0000

---
 gnulib | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/gnulib b/gnulib
index f014f54..324b778 160000
--- a/gnulib
+++ b/gnulib
@@ -1 +1 @@
-Subproject commit f014f540f1b7165444ca56a4a35850689ad383a5
+Subproject commit 324b77834b094bbda8abb3352465776a54b23549
-- 
2.7.4



From MAILER-DAEMON Sun Jul 30 20:19:34 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dbyQo-00078W-Q8
	for mharc-coreutils@gnu.org; Sun, 30 Jul 2017 20:19:34 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:40140)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eggert@cs.ucla.edu>) id 1dbyQh-00073K-Ty
	for coreutils@gnu.org; Sun, 30 Jul 2017 20:19:33 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eggert@cs.ucla.edu>) id 1dbyQg-0006TJ-5p
	for coreutils@gnu.org; Sun, 30 Jul 2017 20:19:27 -0400
Received: from zimbra.cs.ucla.edu ([131.179.128.68]:54536)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)
	(Exim 4.71) (envelope-from <eggert@cs.ucla.edu>) id 1dbyQf-0006T0-T4
	for coreutils@gnu.org; Sun, 30 Jul 2017 20:19:26 -0400
Received: from localhost (localhost [127.0.0.1])
	by zimbra.cs.ucla.edu (Postfix) with ESMTP id 0F6DF1606F7
	for <coreutils@gnu.org>; Sun, 30 Jul 2017 17:19:25 -0700 (PDT)
Received: from zimbra.cs.ucla.edu ([127.0.0.1])
	by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032)
	with ESMTP id NZO6nFLYHGxq; Sun, 30 Jul 2017 17:19:22 -0700 (PDT)
Received: from localhost (localhost [127.0.0.1])
	by zimbra.cs.ucla.edu (Postfix) with ESMTP id C5B1D1606F0;
	Sun, 30 Jul 2017 17:19:22 -0700 (PDT)
X-Virus-Scanned: amavisd-new at zimbra.cs.ucla.edu
Received: from zimbra.cs.ucla.edu ([127.0.0.1])
	by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026)
	with ESMTP id oyEG2cauDX2f; Sun, 30 Jul 2017 17:19:22 -0700 (PDT)
Received: from day.fios-router.home (unknown [47.153.184.153])
	by zimbra.cs.ucla.edu (Postfix) with ESMTPSA id A23C11606DC;
	Sun, 30 Jul 2017 17:19:22 -0700 (PDT)
From: Paul Eggert <eggert@cs.ucla.edu>
To: coreutils@gnu.org
Cc: Paul Eggert <eggert@cs.ucla.edu>
Subject: [PATCH 3/4] copy: make backup files more reliably
Date: Sun, 30 Jul 2017 17:19:06 -0700
Message-Id: <1501460347-19275-3-git-send-email-eggert@cs.ucla.edu>
X-Mailer: git-send-email 2.7.4
In-Reply-To: <1501460347-19275-1-git-send-email-eggert@cs.ucla.edu>
References: <1501460347-19275-1-git-send-email-eggert@cs.ucla.edu>
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy]
X-Received-From: 131.179.128.68
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 31 Jul 2017 00:19:33 -0000

* NEWS, doc/coreutils.texi (Backup options): Document the change.
* bootstrap.conf (gnulib_modules): Add backup-rename.
* src/copy.c (copy_internal): Silently switch to numbered backups
if a simple backup might lose data.  Use backup_file_rename
to avoid races with numbered backups.
* tests/cp/backup-is-src.sh, tests/mv/backup-is-src.sh:
Adjust to match new behavior.
---
 NEWS                      | 12 +++++++++
 bootstrap.conf            |  1 +
 doc/coreutils.texi        | 18 +++++++------
 src/copy.c                | 67 +++++++++++++++++------------------------------
 tests/cp/backup-is-src.sh | 16 +++++------
 tests/mv/backup-is-src.sh | 23 ++++++----------
 6 files changed, 62 insertions(+), 75 deletions(-)

diff --git a/NEWS b/NEWS
index 110229b..7c22ebb 100644
--- a/NEWS
+++ b/NEWS
@@ -9,6 +9,18 @@ GNU coreutils NEWS                                    -*- outline -*-
   mv would leave such symlinks behind in the source file system.
   [the bug dates back to the initial implementation]
 
+  When creating numbered backups, cp, install, ln, and mv now avoid
+  races that could lose backup data in unlikely circumstances.  Since
+  the fix relies on the renameat2 system call of Linux kernel 3.15 and
+  later, the races are still present on other platforms.
+  [the bug dates back to the initial implementation]
+
+  cp, install, ln, and mv now use a numbered instead of a simple
+  backup if copying a backup file to what might be its original.
+  E.g., 'rm -f a a~; : > a; echo data > a~; cp --backup=simple a~ ./a'
+  now makes a numbered backup file instead of losing the data.
+  [the bug dates back to the initial implementation]
+
   date and touch no longer overwrite the heap with large
   user specified TZ values (CVE-2017-7476).
   [bug introduced in coreutils-8.27]
diff --git a/bootstrap.conf b/bootstrap.conf
index 188b1ef..f8f65f7 100644
--- a/bootstrap.conf
+++ b/bootstrap.conf
@@ -35,6 +35,7 @@ gnulib_modules="
   assert
   autobuild
   backupfile
+  backup-rename
   base32
   base64
   buffer-lcm
diff --git a/doc/coreutils.texi b/doc/coreutils.texi
index 77e993e..8c4746e 100644
--- a/doc/coreutils.texi
+++ b/doc/coreutils.texi
@@ -865,17 +865,19 @@ Never make backups.
 @opindex numbered @r{backup method}
 Always make numbered backups.
 
-@item existing
-@itemx nil
-@opindex existing @r{backup method}
-Make numbered backups of files that already have them, simple backups
-of the others.
-
 @item simple
 @itemx never
 @opindex simple @r{backup method}
-Always make simple backups.  Please note @samp{never} is not to be
-confused with @samp{none}.
+Make simple backups, except make a numbered backup if the source might
+be a simple backup of the destination; this avoids losing source data.
+Please note @samp{never} is not to be confused with @samp{none}.
+
+@item existing
+@itemx nil
+@opindex existing @r{backup method}
+Make numbered backups of files that already have them, or if the source
+might be a simple backup of the destination.
+Otherwise, make simple backups.
 
 @end table
 
diff --git a/src/copy.c b/src/copy.c
index a2eee1b..263abc1 100644
--- a/src/copy.c
+++ b/src/copy.c
@@ -1837,7 +1837,6 @@ copy_internal (char const *src_name, char const *dst_name,
   bool restore_dst_mode = false;
   char *earlier_file = NULL;
   char *dst_backup = NULL;
-  bool backup_succeeded = false;
   bool delayed_ok;
   bool copied_as_regular = false;
   bool dest_is_symlink = false;
@@ -2070,10 +2069,11 @@ copy_internal (char const *src_name, char const *dst_name,
                 }
             }
 
+          char const *srcbase;
           if (x->backup_type != no_backups
               /* Don't try to back up a destination if the last
                  component of src_name is "." or "..".  */
-              && ! dot_or_dotdot (last_component (src_name))
+              && ! dot_or_dotdot (srcbase = last_component (src_name))
               /* Create a backup of each destination directory in move mode,
                  but not in copy mode.  FIXME: it might make sense to add an
                  option to suppress backup creation also for move mode.
@@ -2081,55 +2081,38 @@ copy_internal (char const *src_name, char const *dst_name,
                  existing hierarchy.  */
               && (x->move_mode || ! S_ISDIR (dst_sb.st_mode)))
             {
-              char *tmp_backup = find_backup_file_name (dst_name,
-                                                        x->backup_type);
-
-              /* Detect (and fail) when creating the backup file would
-                 destroy the source file.  Before, running the commands
+              /* Silently use numbered backups if creating the backup
+                 file might destroy the source file.  Otherwise, the commands:
                  cd /tmp; rm -f a a~; : > a; echo A > a~; cp --b=simple a~ a
                  would leave two zero-length files: a and a~.  */
-              /* FIXME: but simply change e.g., the final a~ to './a~'
-                 and the source will still be destroyed.  */
-              if (STREQ (tmp_backup, src_name))
+              enum backup_type backup_type = x->backup_type;
+              if (backup_type != numbered_backups)
                 {
-                  const char *fmt;
-                  fmt = (x->move_mode
-                 ? _("backing up %s would destroy source;  %s not moved")
-                 : _("backing up %s would destroy source;  %s not copied"));
-                  error (0, 0, fmt,
-                         quoteaf_n (0, dst_name),
-                         quoteaf_n (1, src_name));
-                  free (tmp_backup);
-                  return false;
+                  size_t srcbaselen = strlen (srcbase);
+                  char const *dstbase = last_component (dst_name);
+                  size_t dstbaselen = strlen (dstbase);
+                  if (srcbaselen == dstbaselen + strlen (simple_backup_suffix)
+                      && memcmp (srcbase, dstbase, dstbaselen) == 0
+                      && STREQ (srcbase + dstbaselen, simple_backup_suffix))
+                    backup_type = numbered_backups;
                 }
 
+              char *tmp_backup = backup_file_rename (dst_name, backup_type);
+
               /* FIXME: use fts:
                  Using alloca for a file name that may be arbitrarily
                  long is not recommended.  In fact, even forming such a name
                  should be discouraged.  Eventually, this code will be rewritten
                  to use fts, so using alloca here will be less of a problem.  */
-              ASSIGN_STRDUPA (dst_backup, tmp_backup);
-              free (tmp_backup);
-              /* In move mode, when src_name and dst_name are on the
-                 same partition (FIXME, and when they are non-directories),
-                 make the operation atomic: link dest
-                 to backup, then rename src to dest.  */
-              if (rename (dst_name, dst_backup) != 0)
+              if (tmp_backup)
                 {
-                  if (errno != ENOENT)
-                    {
-                      error (0, errno, _("cannot backup %s"),
-                             quoteaf (dst_name));
-                      return false;
-                    }
-                  else
-                    {
-                      dst_backup = NULL;
-                    }
+                  ASSIGN_STRDUPA (dst_backup, tmp_backup);
+                  free (tmp_backup);
                 }
-              else
+              else if (errno != ENOENT)
                 {
-                  backup_succeeded = true;
+                  error (0, errno, _("cannot backup %s"), quoteaf (dst_name));
+                  return false;
                 }
               new_dst = true;
             }
@@ -2194,7 +2177,7 @@ copy_internal (char const *src_name, char const *dst_name,
      sure we'll create a directory.  Also don't announce yet when moving
      so we can distinguish renames versus copies.  */
   if (x->verbose && !x->move_mode && !S_ISDIR (src_mode))
-    emit_verbose (src_name, dst_name, backup_succeeded ? dst_backup : NULL);
+    emit_verbose (src_name, dst_name, dst_backup);
 
   /* Associate the destination file name with the source device and inode
      so that if we encounter a matching dev/ino pair in the source tree
@@ -2317,8 +2300,7 @@ copy_internal (char const *src_name, char const *dst_name,
           if (x->verbose)
             {
               printf (_("renamed "));
-              emit_verbose (src_name, dst_name,
-                            backup_succeeded ? dst_backup : NULL);
+              emit_verbose (src_name, dst_name, dst_backup);
             }
 
           if (x->set_security_context)
@@ -2423,8 +2405,7 @@ copy_internal (char const *src_name, char const *dst_name,
       if (x->verbose && !S_ISDIR (src_mode))
         {
           printf (_("copied "));
-          emit_verbose (src_name, dst_name,
-                        backup_succeeded ? dst_backup : NULL);
+          emit_verbose (src_name, dst_name, dst_backup);
         }
       new_dst = true;
     }
diff --git a/tests/cp/backup-is-src.sh b/tests/cp/backup-is-src.sh
index 3e4a79f..8077106 100755
--- a/tests/cp/backup-is-src.sh
+++ b/tests/cp/backup-is-src.sh
@@ -20,17 +20,15 @@
 print_ver_ cp
 
 echo a > a || framework_failure_
+cp a a0 || framework_failure_
 echo a-tilde > a~ || framework_failure_
+cp a~ a~0 || framework_failure_
 
-# This cp command should exit nonzero.
-cp --b=simple a~ a > out 2>&1 && fail=1
+# This cp command should not trash the source.
+cp --b=simple a~ ./a > out 2>&1 || fail=1
 
-sed "s,cp:,XXX:," out > out2
-
-cat > exp <<\EOF
-XXX: backing up 'a' would destroy source;  'a~' not copied
-EOF
-
-compare exp out2 || fail=1
+compare a~0 a || fail=1
+compare a~0 a~ || fail=1
+compare a0 a.~1~ || fail=1
 
 Exit $fail
diff --git a/tests/mv/backup-is-src.sh b/tests/mv/backup-is-src.sh
index 04e9f7c..c1310de 100755
--- a/tests/mv/backup-is-src.sh
+++ b/tests/mv/backup-is-src.sh
@@ -22,25 +22,18 @@ cleanup_() { rm -rf "$other_partition_tmpdir"; }
 . "$abs_srcdir/tests/other-fs-tmpdir"
 
 a="$other_partition_tmpdir/a"
-a2="$other_partition_tmpdir/a~"
+a2="$other_partition_tmpdir/./a~"
 
-rm -f "$a" "$a2" || framework_failure_
+rm -f "$a" "$a2" a20 || framework_failure_
 echo a > "$a" || framework_failure_
+cp "$a" a0 || framework_failure_
 echo a2 > "$a2" || framework_failure_
+cp "$a2" a20 || framework_failure_
 
-# This mv command should exit nonzero.
-mv --b=simple "$a2" "$a" > out 2>&1 && fail=1
+# This mv command should not trash the source.
+mv --b=simple "$a2" "$a" > out 2>&1 || fail=1
 
-sed \
-   -e "s,mv:,XXX:," \
-   -e "s,$a,YYY," \
-   -e "s,$a2,ZZZ," \
-  out > out2
-
-cat > exp <<\EOF
-XXX: backing up 'YYY' would destroy source;  'ZZZ' not moved
-EOF
-
-compare exp out2 || fail=1
+compare a20 "$a" || fail=1
+compare a0 "$a.~1~" || fail=1
 
 Exit $fail
-- 
2.7.4



From MAILER-DAEMON Sun Jul 30 20:19:37 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dbyQo-00078d-UR
	for mharc-coreutils@gnu.org; Sun, 30 Jul 2017 20:19:34 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:40180)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eggert@cs.ucla.edu>) id 1dbyQn-00077J-5t
	for coreutils@gnu.org; Sun, 30 Jul 2017 20:19:34 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eggert@cs.ucla.edu>) id 1dbyQl-0006VV-3N
	for coreutils@gnu.org; Sun, 30 Jul 2017 20:19:33 -0400
Received: from zimbra.cs.ucla.edu ([131.179.128.68]:54556)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)
	(Exim 4.71) (envelope-from <eggert@cs.ucla.edu>) id 1dbyQk-0006V0-QB
	for coreutils@gnu.org; Sun, 30 Jul 2017 20:19:31 -0400
Received: from localhost (localhost [127.0.0.1])
	by zimbra.cs.ucla.edu (Postfix) with ESMTP id E886F1606F6
	for <coreutils@gnu.org>; Sun, 30 Jul 2017 17:19:29 -0700 (PDT)
Received: from zimbra.cs.ucla.edu ([127.0.0.1])
	by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032)
	with ESMTP id V1DSuyWp68TA; Sun, 30 Jul 2017 17:19:29 -0700 (PDT)
Received: from localhost (localhost [127.0.0.1])
	by zimbra.cs.ucla.edu (Postfix) with ESMTP id 0F43F1606F0;
	Sun, 30 Jul 2017 17:19:29 -0700 (PDT)
X-Virus-Scanned: amavisd-new at zimbra.cs.ucla.edu
Received: from zimbra.cs.ucla.edu ([127.0.0.1])
	by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026)
	with ESMTP id pzQLzf4eoJa9; Sun, 30 Jul 2017 17:19:28 -0700 (PDT)
Received: from day.fios-router.home (unknown [47.153.184.153])
	by zimbra.cs.ucla.edu (Postfix) with ESMTPSA id E2ED11606DC;
	Sun, 30 Jul 2017 17:19:28 -0700 (PDT)
From: Paul Eggert <eggert@cs.ucla.edu>
To: coreutils@gnu.org
Cc: Paul Eggert <eggert@cs.ucla.edu>
Subject: [PATCH 4/4] copy: sanity-check --suffix
Date: Sun, 30 Jul 2017 17:19:07 -0700
Message-Id: <1501460347-19275-4-git-send-email-eggert@cs.ucla.edu>
X-Mailer: git-send-email 2.7.4
In-Reply-To: <1501460347-19275-1-git-send-email-eggert@cs.ucla.edu>
References: <1501460347-19275-1-git-send-email-eggert@cs.ucla.edu>
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy]
X-Received-From: 131.179.128.68
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 31 Jul 2017 00:19:34 -0000

* src/cp.c, src/install.c, src/ln.c, src/mv.c (main):
Use set_simple_backup_suffix, to sanity-check the user-supplied
backup suffix.
---
 NEWS          | 4 ++++
 src/cp.c      | 4 +++-
 src/install.c | 4 +++-
 src/ln.c      | 4 +++-
 src/mv.c      | 4 +++-
 5 files changed, 16 insertions(+), 4 deletions(-)

diff --git a/NEWS b/NEWS
index 7c22ebb..13bbc96 100644
--- a/NEWS
+++ b/NEWS
@@ -21,6 +21,10 @@ GNU coreutils NEWS                                    -*- outline -*-
   now makes a numbered backup file instead of losing the data.
   [the bug dates back to the initial implementation]
 
+  cp, install, ln, and mv now ignore nonsensical backup suffixes.
+  For example, --suffix='/' and --suffix='' are now no-ops.
+  [the bug dates back to the initial implementation]
+
   date and touch no longer overwrite the heap with large
   user specified TZ values (CVE-2017-7476).
   [bug introduced in coreutils-8.27]
diff --git a/src/cp.c b/src/cp.c
index a6f0c64..6949a67 100644
--- a/src/cp.c
+++ b/src/cp.c
@@ -930,6 +930,7 @@ main (int argc, char **argv)
   int c;
   bool ok;
   bool make_backups = false;
+  char const *backup_suffix = NULL;
   char *version_control_string = NULL;
   struct cp_options x;
   bool copy_contents = false;
@@ -1126,7 +1127,7 @@ main (int argc, char **argv)
 
         case 'S':
           make_backups = true;
-          simple_backup_suffix = optarg;
+          backup_suffix = optarg;
           break;
 
         case_GETOPT_HELP_CHAR;
@@ -1161,6 +1162,7 @@ main (int argc, char **argv)
                    ? xget_version (_("backup type"),
                                    version_control_string)
                    : no_backups);
+  set_simple_backup_suffix (backup_suffix);
 
   if (x.dereference == DEREF_UNDEFINED)
     {
diff --git a/src/install.c b/src/install.c
index 592c345..5b68261 100644
--- a/src/install.c
+++ b/src/install.c
@@ -796,6 +796,7 @@ main (int argc, char **argv)
   int exit_status = EXIT_SUCCESS;
   const char *specified_mode = NULL;
   bool make_backups = false;
+  char const *backup_suffix = NULL;
   char *version_control_string = NULL;
   bool mkdir_and_install = false;
   struct cp_options x;
@@ -873,7 +874,7 @@ main (int argc, char **argv)
           break;
         case 'S':
           make_backups = true;
-          simple_backup_suffix = optarg;
+          backup_suffix = optarg;
           break;
         case 't':
           if (target_directory)
@@ -949,6 +950,7 @@ main (int argc, char **argv)
                    ? xget_version (_("backup type"),
                                    version_control_string)
                    : no_backups);
+  set_simple_backup_suffix (backup_suffix);
 
   if (x.preserve_security_context && (x.set_security_context || scontext))
     die (EXIT_FAILURE, 0,
diff --git a/src/ln.c b/src/ln.c
index a70b7d5..e86f581 100644
--- a/src/ln.c
+++ b/src/ln.c
@@ -438,6 +438,7 @@ main (int argc, char **argv)
   int c;
   bool ok;
   bool make_backups = false;
+  char const *backup_suffix = NULL;
   char *version_control_string = NULL;
   char const *target_directory = NULL;
   bool no_target_directory = false;
@@ -515,7 +516,7 @@ main (int argc, char **argv)
           break;
         case 'S':
           make_backups = true;
-          simple_backup_suffix = optarg;
+          backup_suffix = optarg;
           break;
         case_GETOPT_HELP_CHAR;
         case_GETOPT_VERSION_CHAR (PROGRAM_NAME, AUTHORS);
@@ -565,6 +566,7 @@ main (int argc, char **argv)
   backup_type = (make_backups
                  ? xget_version (_("backup type"), version_control_string)
                  : no_backups);
+  set_simple_backup_suffix (backup_suffix);
 
   if (relative && !symbolic_link)
     {
diff --git a/src/mv.c b/src/mv.c
index a6c6e39..fc1fca4 100644
--- a/src/mv.c
+++ b/src/mv.c
@@ -336,6 +336,7 @@ main (int argc, char **argv)
   int c;
   bool ok;
   bool make_backups = false;
+  char const *backup_suffix = NULL;
   char *version_control_string = NULL;
   struct cp_options x;
   char *target_directory = NULL;
@@ -405,7 +406,7 @@ main (int argc, char **argv)
           break;
         case 'S':
           make_backups = true;
-          simple_backup_suffix = optarg;
+          backup_suffix = optarg;
           break;
         case 'Z':
           /* As a performance enhancement, don't even bother trying
@@ -469,6 +470,7 @@ main (int argc, char **argv)
                    ? xget_version (_("backup type"),
                                    version_control_string)
                    : no_backups);
+  set_simple_backup_suffix (backup_suffix);
 
   hash_init ();
 
-- 
2.7.4



From MAILER-DAEMON Sun Jul 30 23:54:40 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dc1my-0002QM-Pa
	for mharc-coreutils@gnu.org; Sun, 30 Jul 2017 23:54:40 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:51060)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <962-396-1872@kylheku.com>) id 1dc1mw-0002Pu-AD
	for coreutils@gnu.org; Sun, 30 Jul 2017 23:54:39 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <962-396-1872@kylheku.com>) id 1dc1mt-0002QI-8h
	for coreutils@gnu.org; Sun, 30 Jul 2017 23:54:38 -0400
Received: from smtp-out-no.shaw.ca ([64.59.134.13]:45138)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)
	(Exim 4.71) (envelope-from <962-396-1872@kylheku.com>)
	id 1dc1mt-0002JT-20
	for coreutils@gnu.org; Sun, 30 Jul 2017 23:54:35 -0400
Received: from kylheku.com ([70.79.163.252]) by shaw.ca with SMTP
	id c1medFifEI8mCc1mfd5fxb; Sun, 30 Jul 2017 21:54:22 -0600
X-Authority-Analysis: v=2.2 cv=HahkdmM8 c=1 sm=1 tr=0
	a=95A0EdhkF1LMGt25d7h1IQ==:117 a=95A0EdhkF1LMGt25d7h1IQ==:17
	a=IkcTkHD0fZMA:10 a=SMorJkV_YP8A:10 a=G3gG6ho9WtcA:10
	a=dKjrhbEiOUsBAIHe49kA:9 a=QEXdDO2ut3YA:10
Received: from www-data by kylheku.com with local (Exim 4.72)
	(envelope-from <962-396-1872@kylheku.com>)
	id 1dc1me-00058f-BG; Sun, 30 Jul 2017 20:54:20 -0700
To: Reuti <reuti@staff.uni-marburg.de>
Subject: Re: How the ls command interpret =?UTF-8?Q?this=3F?=
X-PHP-Originating-Script: 501:rcmail.php
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8;
 format=flowed
Content-Transfer-Encoding: 8bit
Date: Sun, 30 Jul 2017 20:54:20 -0700
From: "Kaz Kylheku (Coreutils)" <962-396-1872@kylheku.com>
Cc: BaBa <for.ad@free.fr>, coreutils@gnu.org
In-Reply-To: <187E547F-042F-421F-B239-247DFC112D62@staff.uni-marburg.de>
References: <ccb12f08dc137b606f3961a766ec17da@free.fr>
	<c7db82c1-b9b5-5772-b091-36c60bd90d7b@redhat.com>
	<8ad70c715e714ae4d1cf445fb39b803f@free.fr>
	<187E547F-042F-421F-B239-247DFC112D62@staff.uni-marburg.de>
Message-ID: <effcebbfe575f4dc8c84cb82bd2ff1dc@mail.kylheku.com>
X-Sender: 962-396-1872@kylheku.com
User-Agent: Roundcube Webmail/0.9.2
X-CMAE-Envelope: MS4wfFayGs2vagEa4EG4I7OOnHwEUKwM7kASXUKww+nkZwrUD3n6bwF6QfMVgxgyHEhY/3RttBJAr0oamIq3AILTw/SfVJbuRQxgpZQsbZ51ELrgux1k7e++
	DCMWExApzcO/idIHh0tTsw/bjRP/pr6W8Z6ceWfoTIybmUG9f+0v7qrEyb0YuAphN5a2dFs5AmEz0CEhhKDiBfQ/7EXoG66FgGymMaRzK2vDdntMwNJ1vO3I
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy]
X-Received-From: 64.59.134.13
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 31 Jul 2017 03:54:39 -0000

On 30.07.2017 13:46, Reuti wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1

Is that necessary?

What's the use of reducing your plausible deniability of rubbish 
postings?

> Am 03.07.2017 um 20:30 schrieb BaBa:
> 
>> Le 2017-07-03 19:17, Eric Blake a écrit :
>>> Case 3:
>>> $ mkdir foo
>>> $ cd foo
>>> $ touch a[b]    # the glob doesn't match, so it is passed unchanged
>>> $ ls ?[b]       # the glob doesn't match, so it is passed unchanged
>>> ls: cannot access '?[b]': No such file or directory
>>> $ cd ..
>>> $ rm -rf foo
>> 
>> Yes understood, the glob didn't match.
> 
> And if globbing fails, it won't try it without.

Try what without what?

If globbing fails, then what happens is that the
unexpanded glob pattern remains. (That's the POSIX shell behavior;
GNU Bash has a "nullglob" flag which causes non-matching globs to
expand to nothing, and the GNU C Library's glob() function has
a similar flag.)

> I mean, it could use
> ?'[b]' and try again which would succeed.

Why would globbing re-try with random permutations of the pattern
syntax?

(Why not also '?'[b]? Why not ?'['b]?)

Such a ridiculous complication of its specification of behavior
wouldn't help anyone; only lay traps for the unwary script writer.



From MAILER-DAEMON Mon Jul 31 04:46:37 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dc6LV-00057O-6G
	for mharc-coreutils@gnu.org; Mon, 31 Jul 2017 04:46:37 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:52254)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <kdudka@redhat.com>) id 1dc6LT-00056r-1j
	for coreutils@gnu.org; Mon, 31 Jul 2017 04:46:35 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <kdudka@redhat.com>) id 1dc6LP-0005eF-68
	for coreutils@gnu.org; Mon, 31 Jul 2017 04:46:35 -0400
Received: from mx1.redhat.com ([209.132.183.28]:50942)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)
	(Exim 4.71) (envelope-from <kdudka@redhat.com>) id 1dc6LO-0005c8-Ux
	for coreutils@gnu.org; Mon, 31 Jul 2017 04:46:31 -0400
Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com
	[10.5.11.16])
	(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
	(No client certificate requested)
	by mx1.redhat.com (Postfix) with ESMTPS id AF461FF326;
	Mon, 31 Jul 2017 08:46:27 +0000 (UTC)
DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com AF461FF326
Authentication-Results: ext-mx02.extmail.prod.ext.phx2.redhat.com;
	dmarc=none (p=none dis=none) header.from=redhat.com
Authentication-Results: ext-mx02.extmail.prod.ext.phx2.redhat.com;
	spf=fail smtp.mailfrom=kdudka@redhat.com
Received: from kdudka-nb.localnet (unknown [10.43.2.37])
	by smtp.corp.redhat.com (Postfix) with ESMTPS id 64AE161781;
	Mon, 31 Jul 2017 08:46:27 +0000 (UTC)
From: Kamil Dudka <kdudka@redhat.com>
To: Paul Eggert <eggert@cs.ucla.edu>
Cc: coreutils@gnu.org
Subject: Re: [PATCH 3/4] copy: make backup files more reliably
Date: Mon, 31 Jul 2017 10:46:34 +0200
Message-ID: <1552189.DO5DapkadM@kdudka-nb>
User-Agent: KMail/4.14.10 (Linux/4.9.34-gentoo; KDE/4.14.32; x86_64; ; )
In-Reply-To: <1501460347-19275-3-git-send-email-eggert@cs.ucla.edu>
References: <1501460347-19275-1-git-send-email-eggert@cs.ucla.edu>
	<1501460347-19275-3-git-send-email-eggert@cs.ucla.edu>
MIME-Version: 1.0
Content-Transfer-Encoding: 7Bit
Content-Type: text/plain; charset="us-ascii"
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16
X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16
	(mx1.redhat.com [10.5.110.26]);
	Mon, 31 Jul 2017 08:46:27 +0000 (UTC)
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
	[fuzzy]
X-Received-From: 209.132.183.28
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 31 Jul 2017 08:46:35 -0000

On Sunday, July 30, 2017 17:19:06 Paul Eggert wrote:
> +  cp, install, ln, and mv now use a numbered instead of a simple
> +  backup if copying a backup file to what might be its original.
> +  E.g., 'rm -f a a~; : > a; echo data > a~; cp --backup=simple a~ ./a'
> +  now makes a numbered backup file instead of losing the data.

I do not think that this behavior is expected at all.  If the user
explicitly asks for a simple backup, cp should either make a single
backup or fail honestly if the simple backup cannot be made.  That
is how --backup=simple has always been documented (and it is still
documented such in the cp(1) man page):

      simple, never
              always make simple backups

Kamil


From MAILER-DAEMON Mon Jul 31 13:30:12 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dcEWC-0008Kc-2f
	for mharc-coreutils@gnu.org; Mon, 31 Jul 2017 13:30:12 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:39819)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <sven.c.dack@sky.com>) id 1dcDTZ-0005VK-Ib
	for coreutils@gnu.org; Mon, 31 Jul 2017 12:23:26 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <sven.c.dack@sky.com>) id 1dcDTU-0005nh-5m
	for coreutils@gnu.org; Mon, 31 Jul 2017 12:23:23 -0400
Received: from nm16-vm1.bullet.mail.ir2.yahoo.com ([212.82.96.207]:60224)
	by eggs.gnu.org with esmtps (TLS1.0:RSA_ARCFOUR_SHA1:16) (Exim 4.71)
	(envelope-from <sven.c.dack@sky.com>) id 1dcDTT-0005mM-Pf
	for coreutils@gnu.org; Mon, 31 Jul 2017 12:23:20 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sky.com; s=s1024;
	t=1501518197; bh=j3G3IHi2y5u3qnj2bdQgsEVTenUFrNzMm1XjS5qs7to=;
	h=To:From:Subject:Date:From:Subject;
	b=FkuGzUZGYHFytf47/LFlnGWlCdp4rQ89QtQw2Gg+WELg0kxlLdmlFq8MLrA3CraJd/Nx5GyS2DBR8fSA3Otd3NKG6R14Tmaiv5+9PKSgstmYZWNwiZQOdUThNh3p8FEjIHKCEioWDVSlM7DB87vfe7rHkosBk1uWFyAJnrv157c=
Received: from [212.82.98.57] by nm16.bullet.mail.ir2.yahoo.com with NNFMP;
	31 Jul 2017 16:23:17 -0000
Received: from [46.228.39.86] by tm10.bullet.mail.ir2.yahoo.com with NNFMP;
	31 Jul 2017 16:23:17 -0000
Received: from [127.0.0.1] by smtp123.mail.ir2.yahoo.com with NNFMP;
	31 Jul 2017 16:23:17 -0000
X-Yahoo-Newman-Id: 687023.36304.bm@smtp123.mail.ir2.yahoo.com
X-Yahoo-Newman-Property: ymail-3
X-YMail-OSG: gQJ8FWUVM1k1CUD6qNvKOd3JoPdit4XZDRf2ySPeL0yqlxH
	e_b2kQbn8tLSHIKpU.Gi0_t.Jcy01HgWo_5OtCkxkP8TB5OratoEDrOJrRPq
	wvRW5bBLJy7eFKMxQOcDX_crsel1hQIwrQWKzxSeAaKBqVPrRS0nnJ4nEwQU
	zBllrBe3hqhJsaEP5OLkJopXDt6y.s4e8AsZ9buWZSn0C2VItg2UKv7wqfR1
	LJPf6xVI6BtKEanhrOY7FH9Wab6rvObpa2lMrfhOSfWJ1ael3og.L2HNPsFF
	h4FFq.buCBQIhCWpIo2cB3kquWzP4HxF3sA1L7avJC42aD10ahA23_nnKDm0
	WwrfAYKvRlS0pMitVViFFCZ5oBmKWfR7z.X_v8976.fjpWhVA_1kPcbYzuM_
	mpC9m_HLajPjaMmkd0xxvjOoIBe18Y7tQglO8ekrcbKRo1awFwxBxq7nGk1m
	g6S.h7sr4Ouq.enrX4PXH_jJ0d3m8ryRU3nOY9cNaBYUY44CaoHMYmpPECkm
	1HIqrpBoO2BEWijpLgzxwVS96aDJ51M33GtkYHgsW6UdKssbyCv4-
X-Yahoo-SMTP: vPZU.OKswBC.RMkOW9onWi6Zk8BriyC1yj0kMdBuqpLuLjitZVXhzK.Ofd5b
To: coreutils@gnu.org
From: "Sven C. Dack" <sven.c.dack@sky.com>
Subject: sort -V behaviour
Message-ID: <a64be8af-fb17-874a-0928-540616555fec@sky.com>
Date: Mon, 31 Jul 2017 17:23:16 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
	Thunderbird/52.2.1
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Content-Language: en-GB
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy]
X-Received-From: 212.82.96.207
X-Mailman-Approved-At: Mon, 31 Jul 2017 13:30:10 -0400
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 31 Jul 2017 16:23:26 -0000

Hello,

I have a question about the -V option to sort, but first some examples:

$ echo -e "1\n1.2\n1.2.3\n1.2.3.4"|sort -V
1
1.2
1.2.3
1.2.3.4

$ echo -e "f1\nf1.2\nf1.2.3\nf1.2.3.4"|sort -V
f1
f1.2
f1.2.3
f1.2.3.4

$ echo -e "/1\n/1.2\n/1.2.3\n/1.2.3.4"|sort -V
/1
/1.2
/1.2.3
/1.2.3.4

$ echo -e "1f\n1.2f\n1.2.3f\n1.2.3.4f"|sort -V
1f
1.2f
1.2.3f
1.2.3.4f

$ echo -e "1/\n1.2/\n1.2.3/\n1.2.3.4/"|sort -V
1.2.3.4/
1.2.3/
1.2/
1/

My question is, why does the -V option reverse the order in the last case?

This behaviour is unintuitive and seems wrong to me. When -V sorts 
version numbers of files/lines/etc. and assumes a version number of i.e. 
"1.2.3" to be higher than "1.2" then it shouldn't matter if the version 
number is part of a file or a directory name.

Can somebody please explain this behaviour?

Sven



From MAILER-DAEMON Mon Jul 31 14:41:23 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dcFd4-0001eo-Uv
	for mharc-coreutils@gnu.org; Mon, 31 Jul 2017 14:41:23 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:56465)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <962-396-1872@kylheku.com>) id 1dcFd1-0001dZ-H9
	for coreutils@gnu.org; Mon, 31 Jul 2017 14:41:20 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <962-396-1872@kylheku.com>) id 1dcFcy-0002vs-BY
	for coreutils@gnu.org; Mon, 31 Jul 2017 14:41:19 -0400
Received: from smtp-out-so.shaw.ca ([64.59.136.139]:48298)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)
	(Exim 4.71) (envelope-from <962-396-1872@kylheku.com>)
	id 1dcFcy-0002tF-5h
	for coreutils@gnu.org; Mon, 31 Jul 2017 14:41:16 -0400
Received: from kylheku.com ([70.79.163.252]) by shaw.ca with SMTP
	id cFcpdlDOBMaqMcFcrdKrsD; Mon, 31 Jul 2017 12:41:12 -0600
X-Authority-Analysis: v=2.2 cv=Qc8WhoTv c=1 sm=1 tr=0
	a=95A0EdhkF1LMGt25d7h1IQ==:117 a=95A0EdhkF1LMGt25d7h1IQ==:17
	a=IkcTkHD0fZMA:10 a=SMorJkV_YP8A:10 a=G3gG6ho9WtcA:10
	a=Ly9OxVHz69su1wT8K4kA:9 a=QEXdDO2ut3YA:10
Received: from www-data by kylheku.com with local (Exim 4.72)
	(envelope-from <962-396-1872@kylheku.com>)
	id 1dcFcp-0007Xx-Ms; Mon, 31 Jul 2017 11:41:07 -0700
To: "Sven C. Dack" <sven.c.dack@sky.com>
Subject: Re: sort -V behaviour
X-PHP-Originating-Script: 501:rcmail.php
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8;
 format=flowed
Content-Transfer-Encoding: quoted-printable
Date: Mon, 31 Jul 2017 11:41:07 -0700
From: "Kaz Kylheku (Coreutils)" <962-396-1872@kylheku.com>
Cc: coreutils@gnu.org
In-Reply-To: <a64be8af-fb17-874a-0928-540616555fec@sky.com>
References: <a64be8af-fb17-874a-0928-540616555fec@sky.com>
Message-ID: <de9a13b758df1b0834f8d0ffb3baf80e@mail.kylheku.com>
X-Sender: 962-396-1872@kylheku.com
User-Agent: Roundcube Webmail/0.9.2
X-CMAE-Envelope: MS4wfFoU65vy7nFwXbzRvFBFksanuhBVQOiT6cE1v1yWrTBKt26D5CGbtGolTTthxtI9zggFuqtI0T0IkhpgGoKKJ327biIPfGxORQjAY2WNkrvcWP3AyCMc
	h4PpJOuF3a179Viz6cQw6s4/cRjvVGSoE6wS828igOaSz7t6OCFhmV7MWN4Qdb5VU7sucqvJkNNORnEWQBSD8D16nnOstUh2y+M=
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy]
X-Received-From: 64.59.136.139
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 31 Jul 2017 18:41:20 -0000

On 31.07.2017 09:23, Sven C. Dack wrote:
> Hello,
>=20
> I have a question about the -V option to sort, but first some examples:
>=20
> $ echo -e "1\n1.2\n1.2.3\n1.2.3.4"|sort -V
> 1
> 1.2
> 1.2.3
> 1.2.3.4
>=20
> $ echo -e "f1\nf1.2\nf1.2.3\nf1.2.3.4"|sort -V
> f1
> f1.2
> f1.2.3
> f1.2.3.4
>=20
> $ echo -e "/1\n/1.2\n/1.2.3\n/1.2.3.4"|sort -V
> /1
> /1.2
> /1.2.3
> /1.2.3.4
>=20
> $ echo -e "1f\n1.2f\n1.2.3f\n1.2.3.4f"|sort -V
> 1f
> 1.2f
> 1.2.3f
> 1.2.3.4f

Note that this also has a problem, though the behavior is what you
expect, so you don't notice.

Here, only the last three lines of input contain version numbers.

In each one, the last dot and everything after it is considered
a suffix; the versions being sorted are "1", "1.2" and "1.2.3".


> $ echo -e "1/\n1.2/\n1.2.3/\n1.2.3.4/"|sort -V
> 1.2.3.4/
> 1.2.3/
> 1.2/
> 1/
>=20
> My question is, why does the -V option reverse the order in the last=20
> case?


 From the info documentation:

      Version-sorted strings are compared such that if VER1 and VER2 are
   version numbers and PREFIX and SUFFIX (SUFFIX matching the regular
   expression `(\.[A-Za-z~][A-Za-z0-9~]*)*') are strings then VER1 < VER2
   implies that the name composed of "PREFIX VER1 SUFFIX" sorts before
   "PREFIX VER2 SUFFIX".

Looks the SUFFIX regex doesn't match, so these names are not treated
as version names. It doesn't match because of the trailing slash.

If the trailing slash were included in the suffix match, there would
still be the problem that .4/, .3/ and ./2 are a the suffix,
and the version numbers are "1.2.3", "1.2", and "1", with the last
"1" being a non-version-number input.

Also this is noted:

      This functionality is implemented using gnulib's `filevercmp'
   function, which has some caveats worth noting.
   [...]
      * Some suffixes will not be matched by the regular expression
        mentioned above.  Consequently these examples may not sort as you
        expect:

             abc-1.2.3.4.7z
             abc-1.2.3.7z

             abc-1.2.3.4.x86_64.rpm
             abc-1.2.3.x86_64.rpm

Oops! And as you can see from these examples it is tricky.
Sometimes suffixes contain numeric stuff, which is why it's
specified that way.

Here the .7z files don't match the requirement for treatment as
version numbers, because the suffix, following the required period,
must begin with a letter or tilde.

> This behaviour is unintuitive and seems wrong to me.

I agree that the specification is not ideal, but it's not easy to
see how it can be improved given the threat of numeric junk like 7z
which cannot be treated as part of the version.

Consider that 1.7z looks like a bigger version than 1.2.7z,
if the 7 is wrongly treated as part of the version!!!

The designers who specified the filevercmp function were clearly
sober to these cases.


From MAILER-DAEMON Mon Jul 31 16:10:58 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dcH1m-0001Fe-8e
	for mharc-coreutils@gnu.org; Mon, 31 Jul 2017 16:10:58 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:58018)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <kdudka@redhat.com>) id 1dcH1j-0001FL-0w
	for coreutils@gnu.org; Mon, 31 Jul 2017 16:10:56 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <kdudka@redhat.com>) id 1dcH1e-0003cO-47
	for coreutils@gnu.org; Mon, 31 Jul 2017 16:10:55 -0400
Received: from mx1.redhat.com ([209.132.183.28]:42290)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)
	(Exim 4.71) (envelope-from <kdudka@redhat.com>) id 1dcH1d-0003bn-TX
	for coreutils@gnu.org; Mon, 31 Jul 2017 16:10:50 -0400
Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com
	[10.5.11.14])
	(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
	(No client certificate requested)
	by mx1.redhat.com (Postfix) with ESMTPS id 5CEC92DBB45;
	Mon, 31 Jul 2017 20:10:48 +0000 (UTC)
DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 5CEC92DBB45
Authentication-Results: ext-mx09.extmail.prod.ext.phx2.redhat.com;
	dmarc=none (p=none dis=none) header.from=redhat.com
Authentication-Results: ext-mx09.extmail.prod.ext.phx2.redhat.com;
	spf=fail smtp.mailfrom=kdudka@redhat.com
Received: from kdudka-nb.localnet (ovpn-204-94.brq.redhat.com [10.40.204.94])
	by smtp.corp.redhat.com (Postfix) with ESMTPS id 4935567C7D;
	Mon, 31 Jul 2017 20:10:47 +0000 (UTC)
From: Kamil Dudka <kdudka@redhat.com>
To: "Sven C. Dack" <sven.c.dack@sky.com>
Cc: coreutils@gnu.org
Subject: Re: sort -V behaviour
Date: Mon, 31 Jul 2017 22:10:54 +0200
Message-ID: <2113821.xzn2RFEMiG@kdudka-nb>
User-Agent: KMail/4.14.10 (Linux/4.9.34-gentoo; KDE/4.14.32; x86_64; ; )
In-Reply-To: <a64be8af-fb17-874a-0928-540616555fec@sky.com>
References: <a64be8af-fb17-874a-0928-540616555fec@sky.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 7Bit
Content-Type: text/plain; charset="us-ascii"
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14
X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16
	(mx1.redhat.com [10.5.110.38]);
	Mon, 31 Jul 2017 20:10:48 +0000 (UTC)
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
	[fuzzy]
X-Received-From: 209.132.183.28
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 31 Jul 2017 20:10:57 -0000

On Monday, July 31, 2017 17:23:16 Sven C. Dack wrote:
> Hello,
> 
> I have a question about the -V option to sort, but first some examples:
> 
> $ echo -e "1\n1.2\n1.2.3\n1.2.3.4"|sort -V
> 1
> 1.2
> 1.2.3
> 1.2.3.4
> 
> $ echo -e "f1\nf1.2\nf1.2.3\nf1.2.3.4"|sort -V
> f1
> f1.2
> f1.2.3
> f1.2.3.4
> 
> $ echo -e "/1\n/1.2\n/1.2.3\n/1.2.3.4"|sort -V
> /1
> /1.2
> /1.2.3
> /1.2.3.4
> 
> $ echo -e "1f\n1.2f\n1.2.3f\n1.2.3.4f"|sort -V
> 1f
> 1.2f
> 1.2.3f
> 1.2.3.4f
> 
> $ echo -e "1/\n1.2/\n1.2.3/\n1.2.3.4/"|sort -V
> 1.2.3.4/
> 1.2.3/
> 1.2/
> 1/
> 
> My question is, why does the -V option reverse the order in the last case?

The version compare predicate used in 'sort -V' was originally designed for 
sorting (base) file names in the output of 'ls -v' and the '/' character is 
not allowed in (base) file names.

The gnulib function implementing the predicate is called filevercmp(), which 
makes it obvious what it does.  But I guess that we should say it explicitly 
in the documentation of 'sort -V' to avoid false expectations like this...

Kamil

> This behaviour is unintuitive and seems wrong to me. When -V sorts
> version numbers of files/lines/etc. and assumes a version number of i.e.
> "1.2.3" to be higher than "1.2" then it shouldn't matter if the version
> number is part of a file or a directory name.
> 
> Can somebody please explain this behaviour?
> 
> Sven


From MAILER-DAEMON Mon Jul 31 16:13:13 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dcH3w-0001Rx-TK
	for mharc-coreutils@gnu.org; Mon, 31 Jul 2017 16:13:12 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:59049)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <sven.c.dack@sky.com>) id 1dcH3u-0001Ri-KE
	for coreutils@gnu.org; Mon, 31 Jul 2017 16:13:12 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <sven.c.dack@sky.com>) id 1dcH3p-0005BJ-Lr
	for coreutils@gnu.org; Mon, 31 Jul 2017 16:13:10 -0400
Received: from nm35-vm6.bullet.mail.ir2.yahoo.com ([212.82.97.129]:53347)
	by eggs.gnu.org with esmtps (TLS1.0:RSA_ARCFOUR_SHA1:16) (Exim 4.71)
	(envelope-from <sven.c.dack@sky.com>) id 1dcH3p-00059d-AA
	for coreutils@gnu.org; Mon, 31 Jul 2017 16:13:05 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sky.com; s=s1024;
	t=1501531984; bh=cU8JWQjew2iAFBc3PJZ+u78JPqcdt+mGT0cWDCDe2p4=;
	h=Subject:To:References:From:Cc:Date:In-Reply-To:From:Subject;
	b=c4wVkljQ0nTsO7WdXHeUdw0tG/tCIsterlGz5mjUSP1WeGCpqzLXf90IYbpGhz17hxqv01nDMYbpTNOdF5YpLde9Q2rdDDf2ywGSy3z9exk3bWBW35ibOnKOP6JtI9E1ha8CCbU3C6xn+XTVG9cViEeyzCxduYXL7ehMnJr9opk=
Received: from [212.82.98.62] by nm35.bullet.mail.ir2.yahoo.com with NNFMP;
	31 Jul 2017 20:13:04 -0000
Received: from [46.228.39.93] by tm15.bullet.mail.ir2.yahoo.com with NNFMP;
	31 Jul 2017 20:13:04 -0000
Received: from [127.0.0.1] by smtp130.mail.ir2.yahoo.com with NNFMP;
	31 Jul 2017 20:13:04 -0000
X-Yahoo-Newman-Id: 126660.65789.bm@smtp130.mail.ir2.yahoo.com
X-Yahoo-Newman-Property: ymail-3
X-YMail-OSG: 9svx1sAVM1nEfb1TKEReICzanBy1yzjtLmzOb_gE6ng3bna
	edSBsDxdrG7uYPde2xZS_tWYUwSFMmXknqr.FEKTdCNZURtzOX0ETZ4zijU2
	BzqCsu_WodxyIrppvgYcYzon6bnX2I1rBbgxcM_j5UGfoSa.MXmJO8WIhvVw
	5461WGOQth0aHaxR0FmabRZiNqn.mAi3XyTnoIRQwA0kfvu5LAnO.MUX2ABl
	B7A05ZtB_xNwQayS4utlbg99PFjiyA0QF5Z.ojPmYhjAk9BR_nXAmOfaHdOT
	.wrA4C1B.GtQKZHBuqW_B9CcQ2JD7ofpvFY67mrr0z39FGD3YNT48N5RsnQ6
	g03Yc_10CykiwJzFu4l9h4ho6pePQbt0Qcywo_pa9Aw8XFVhoxZeBsFeYkW4
	90VdEaFe1DxHm8OmjbzkaVsUb6ZDXLEHgG1q2sufXHVey0paCsFQH0LqKiHJ
	oJf5TkSGXKrL16WAtPeEPwUtWbk3TFCVrVGvdYN1cQeoWBwKINfY7Ov0QWYx
	PA_8mFTA5KY5qE2Z3CDBdoiOgj4UBnzFJ.3TC8tAST8S5drCGWvVqvDQ4_EB
	OBq1EQzqYWxx4ObF1dTqwvQbhBnOInzw.n7unIc0Cozx4NBt8ihg-
X-Yahoo-SMTP: vPZU.OKswBC.RMkOW9onWi6Zk8BriyC1yj0kMdBuqpLuLjitZVXhzK.Ofd5b
Subject: Re: sort -V behaviour
To: "Kaz Kylheku (Coreutils)" <962-396-1872@kylheku.com>
References: <a64be8af-fb17-874a-0928-540616555fec@sky.com>
	<de9a13b758df1b0834f8d0ffb3baf80e@mail.kylheku.com>
From: "Sven C. Dack" <sven.c.dack@sky.com>
Cc: coreutils@gnu.org
Message-ID: <166b0dcf-21d0-9ae0-e94c-b40c2da8709a@sky.com>
Date: Mon, 31 Jul 2017 21:13:03 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
	Thunderbird/52.2.1
MIME-Version: 1.0
In-Reply-To: <de9a13b758df1b0834f8d0ffb3baf80e@mail.kylheku.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Content-Language: en-GB
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy]
X-Received-From: 212.82.97.129
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 31 Jul 2017 20:13:12 -0000

Hello Kaz,

thank you for your explanation.

Looks like I'll have to work around it.

It would be nice if one could somehow specify a flag to disable this 
behaviour and to make 'sort' less clever about file extensions, because 
with 'sort' is the context not always file names. It's not the 'ls' 
command, but it's a universal sorting command for all kinds of data. At 
least this is how I have always treated it.


On 31/07/17 19:41, Kaz Kylheku (Coreutils) wrote:
...
>> This behaviour is unintuitive and seems wrong to me.
>
> I agree that the specification is not ideal, but it's not easy to
> see how it can be improved given the threat of numeric junk like 7z
> which cannot be treated as part of the version.
>
> Consider that 1.7z looks like a bigger version than 1.2.7z,
> if the 7 is wrongly treated as part of the version!!!
>
> The designers who specified the filevercmp function were clearly
> sober to these cases.

I understand your reasoning, but I cannot agree with it. As you yourself 
have realized does 1.7z indeed look like it's a higher version than 
1.2.7z (that's because to many people it just is).

Only because of you recognizing 7z as a file extension (used by 7-zip) 
should the 'sort' command not automatically recognize file extensions, 
too. If you look at 
https://www.file-extensions.org/extensions/begin-with-number then you'll 
see that there are far too many extensions including many, which are 
numbers. Strangely enough could I not find an extension ending in a "/" 
for that matter...

I can certainly see how within the context of file names the behaviour 
makes a variety of use cases easier, but the input to the 'sort' command 
generally makes no assumptions about the context. It doesn't assume all 
strings to be surnames, or that all numbers are currencies, or taxable, 
or that these can be rounded the nearest penny or cent for example.

So I'm going blame it on the use of the filevercmp() function. Obviously 
was it designed specifically for file names and not as a universal 
version number comparison function.

Cheers,
Sven



From MAILER-DAEMON Mon Jul 31 19:55:15 2017
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1dcKWp-00064O-Bu
	for mharc-coreutils@gnu.org; Mon, 31 Jul 2017 19:55:15 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:58042)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1dcKWm-000644-26
	for coreutils@gnu.org; Mon, 31 Jul 2017 19:55:13 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1dcKWi-0005CH-VE
	for coreutils@gnu.org; Mon, 31 Jul 2017 19:55:12 -0400
Received: from mout.kundenserver.de ([212.227.126.134]:53976)
	by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16)
	(Exim 4.71) (envelope-from <mail@bernhard-voelker.de>)
	id 1dcKWi-00055h-J5
	for coreutils@gnu.org; Mon, 31 Jul 2017 19:55:08 -0400
Received: from [192.168.101.10] ([91.12.166.234]) by mrelayeu.kundenserver.de
	(mreue001 [212.227.15.167]) with ESMTPSA (Nemesis) id
	0MH974-1dOd972UFe-00DsN1; Tue, 01 Aug 2017 01:54:59 +0200
To: Coreutils <coreutils@gnu.org>, Josef Cejka <jcejka@suse.com>
Cc: Philipp Thomas <pth@suse.de>
From: Bernhard Voelker <mail@bernhard-voelker.de>
Subject: [PATCH] df: avoid stat() for dummy file systems with -l
Message-ID: <c4db6b63-e210-2634-a79a-e93733ed0a57@bernhard-voelker.de>
Date: Tue, 1 Aug 2017 01:54:56 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101
	Thunderbird/45.6.0
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="------------034680A623B7AE0C2F75313D"
X-Provags-ID: V03:K0:lAQHAc1jyJ7pnm21VugL02man0ke/nVx6XGICjsyXnoII4lDUgQ
	h+b7oaqhWUgGp2mPkqNwqnFXRJ1U4yWGre2djeIvfeyUEj2GmXm/Qr1y3ma/xW5qSJizhqj
	wTGvIKquW6CeSBH9kQhuKZOG9+VqIYnNgHK1FJYReghafYXRm4c/3G5wMMELk6wWlTWff5F
	Frk0Tr2ZQ7YOnxHe5w+rg==
X-UI-Out-Filterresults: notjunk:1;V01:K0:YkCrWcfyyFA=:pI9r5IusxpcKogFjvwmF+O
	Pbtg34C9dSj3VHNdG3MrVkRNoyD/dQa758QKnxZJZgkf7geFK+ZBb4hyzKDkaDz3I6oq/0pi4
	43ee2nEOoxiWp0TNni+FUwRCkThsJjnW2YxQ8vpn65yAupknFR0KJiiMrBgi64rrv/sDZosYy
	ERlogbn8TwwR0nMdFZ9V9D0gU+tQYblKLGhiXFebeAt72FXe1fiImhsv8QsfsDWMIOmU/lq5v
	SLY9IF/kus13T0G7+W9fbJTwtn72bacc+dIgmnM9ER7bdpmUh5yc2G/m1d1pVs0Dck3N+AqFr
	tuieiO9GdXwSrhHbAlN0fTWqPLdEgrMvp8bMwLqdcvMZIikfSFJOv4ImEPQAgdckJnziflyrl
	6hg1WDkPzE6VumqwfFSPbTfQrORkToPfJQSRTsy9WNnUA4io2lRZ65chjSbKj5/sFZPJqHzGi
	e357T4fZB3OOUw52LUdafH+OXk0s/x1K0Z3rl+njW3TriolDvivU0FqeGTLbTCSmk2yJuA79E
	dXGbbxSyuUcH9AKRo60JCS6VYfAXv+8GOe6V2yjanAdrMOQc241kg1W3e0Iw+cHCCRMxY7aIW
	D8H5cAp/gNm78wfpJyebgiIiJ5Dx8GY111MPzxMk3WTVQZXqqbfbQ8X1PdmxvcHuflJ1uprME
	42iBed4h1SrJMfRqVvNvdGvtrpWopYa7744+dba6YrDgamP9i+KY4cgSezhEbuj/x6EtUrVHC
	f2F3SmadQ5fiGO2vLa5N0BVFWqOKLYQLgnTSIQ==
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
	[fuzzy]
X-Received-From: 212.227.126.134
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils/>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 31 Jul 2017 23:55:13 -0000

This is a multi-part message in MIME format.
--------------034680A623B7AE0C2F75313D
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

The attached fixes a hang for unaccessible NFS mounts which
are automounted by systemd.

See e.g.
  http://blog.tomecek.net/post/automount-with-systemd/
for how to configure such a systemd unit to reproduce.

Do we need a test case for this?

Have a nice day,
Berny

--------------034680A623B7AE0C2F75313D
Content-Type: text/x-patch;
 name="0001-df-avoid-stat-for-dummy-file-systems-with-l.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename="0001-df-avoid-stat-for-dummy-file-systems-with-l.patch"

>From 12c1918e836711d1a25dcb936dbc65c1e31d0813 Mon Sep 17 00:00:00 2001
From: Josef Cejka <jcejka@suse.com>
Date: Tue, 1 Aug 2017 01:50:34 +0200
Subject: [PATCH] df: avoid stat() for dummy file systems with -l

When systemd is configured to automount a remote file system - see
'man systemd.automount(5)', then the mount point is initially
mounted by systemd with the file system type "autofs".
When the resource is used later on, then the wanted file system is
mounted over that mount point on demand.
'df -l' triggered systemd to mount the file system because it called
stat() on the mount point.
Instead of single-casing "autofs" targets, we can avoid stat()ing
all dummy file systems (which includes "autofs"), because those are
skipped later on in get_dev() anyway.

*src/df.c (filter_mount_list): Also skip dummy file systems unless
the -a option or a specific target are given.
* NEWS: Mention the fix.

Co-authored-by: Bernhard Voelker <mail@bernhard-voelker.de>

Fixes http://bugzilla.suse.com/show_bug.cgi?id=1043059
---
 NEWS     | 4 ++++
 src/df.c | 1 +
 2 files changed, 5 insertions(+)

diff --git a/NEWS b/NEWS
index 13bbc96..7ea70af 100644
--- a/NEWS
+++ b/NEWS
@@ -37,6 +37,10 @@ GNU coreutils NEWS                                    -*- outline -*-
   specifying -x nfs no longer hangs with problematic nfs mounts.
   [bug introduced in coreutils-8.21]
 
+  df -l no longer interacts with dummy file system types, so for example
+  no longer hangs with problematic nfs mounted via system.automount(5).
+  [bug introduced in coreutils-8.21]
+
   `groups inva:lid root` no longer exits immediately upon failure.
   Now, it prints a diagnostic or a line to stdout for each argument.
   [bug introduced in the bourne-shell-to-C rewrite for coreutils-6.11]
diff --git a/src/df.c b/src/df.c
index 678aa47..bd4cc8b 100644
--- a/src/df.c
+++ b/src/df.c
@@ -674,6 +674,7 @@ filter_mount_list (bool devices_only)
          On Linux we probably have me_dev populated from /proc/self/mountinfo,
          however we still stat() in case another device was mounted later.  */
       if ((me->me_remote && show_local_fs)
+          || (me->me_dummy && !show_all_fs && !show_listed_fs)
           || (!selected_fstype (me->me_type) || excluded_fstype (me->me_type))
           || -1 == stat (me->me_mountdir, &buf))
         {
-- 
2.1.4


--------------034680A623B7AE0C2F75313D--


