From MAILER-DAEMON Fri Jun 01 03:19:36 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1SaM92-0003GW-Kn
	for mharc-coreutils@gnu.org; Fri, 01 Jun 2012 03:19:36 -0400
Received: from eggs.gnu.org ([208.118.235.92]:60967)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1SaM8v-0003Fw-5q
	for coreutils@gnu.org; Fri, 01 Jun 2012 03:19:34 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1SaM8s-0002Yu-Q9
	for coreutils@gnu.org; Fri, 01 Jun 2012 03:19:28 -0400
Received: from mx.meyering.net ([88.168.87.75]:43202)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1SaM8s-0002YP-IT
	for coreutils@gnu.org; Fri, 01 Jun 2012 03:19:26 -0400
Received: from rho.meyering.net (localhost.localdomain [127.0.0.1])
	by rho.meyering.net (Acme Bit-Twister) with ESMTP id 1646560053;
	Fri,  1 Jun 2012 09:19:25 +0200 (CEST)
From: Jim Meyering <jim@meyering.net>
To: coreutils@gnu.org
Subject: [PATCH] stat,tail: recognize new file system type: panfs
Date: Fri, 01 Jun 2012 09:19:25 +0200
Message-ID: <87lik74hz6.fsf@rho.meyering.net>
Lines: 64
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 88.168.87.75
Cc: tgummels@redhat.com
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 01 Jun 2012 07:19:35 -0000

FYI,

>From 5fdd5c6310098c5ce3cbc608450d5c4104f18c2f Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering@redhat.com>
Date: Fri, 1 Jun 2012 09:14:25 +0200
Subject: [PATCH] stat,tail: recognize new file system type: panfs

* src/stat.c (human_fstype) [__linux__]: Add a 'case' for the new
remote file system type: panfs (0xAAD7AAEA).
* NEWS (New features): Mention stat -f.
(Bug fixes): Mention it for tail -f.
Reported by Travis Gummels in http://bugzilla.redhat.com/827199
---
 NEWS       | 8 ++++++++
 THANKS.in  | 1 +
 src/stat.c | 2 ++
 3 files changed, 11 insertions(+)

diff --git a/NEWS b/NEWS
index daea549..7c7c2c3 100644
--- a/NEWS
+++ b/NEWS
@@ -15,6 +15,14 @@ GNU coreutils NEWS                                    -*=
- outline -*-
   in all situations on GNU/Linux, by correctly determining the block size.
   [df bug since coreutils-5.0.91, stat bug since the initial implementatio=
n]

+  tail -f no longer tries to use inotify on PanFS file systems
+  [you might say this was introduced in coreutils-7.5, along with inotify
+   support, but even now, its magic number isn't in the usual place.]
+
+** New features
+
+  stat -f recognizes the new remote file system type, panfs.
+

 * Noteworthy changes in release 8.17 (2012-05-10) [stable]

diff --git a/THANKS.in b/THANKS.in
index 43c9d6b..b9a6c64 100644
--- a/THANKS.in
+++ b/THANKS.in
@@ -603,6 +603,7 @@ Toralf F=F6rster                      toralf.foerster@g=
mx.de
 Torbjorn Granlund                   tege@nada.kth.se
 Torbjorn Lindgren                   tl@funcom.no
 Torsten Landschoff                  torsten@pclab.ifg.uni-kiel.de
+Travis Gummels                      tgummels@redhat.com
 Tristan Miller                      psychonaut@nothingisreal.com
 Tzvi Rotshtein                      tzvi.ro@gmail.com
 Ulrich Drepper                      drepper@gnu.org
diff --git a/src/stat.c b/src/stat.c
index d80c624..94b63f6 100644
--- a/src/stat.c
+++ b/src/stat.c
@@ -357,6 +357,8 @@ human_fstype (STRUCT_STATVFS const *statfsbuf)
       return "openprom";
     case S_MAGIC_OCFS2: /* 0x7461636f remote */
       return "ocfs2";
+    case S_MAGIC_PANFS: /* 0xAAD7AAEA remote */
+      return "panfs";
     case S_MAGIC_PIPEFS: /* 0x50495045 remote */
       /* FIXME: change syntax or add an optional attribute like "inotify:n=
o".
          The above is labeled as "remote" so that tail always uses polling,
--
1.7.10.2.605.gbefc5ed


From MAILER-DAEMON Fri Jun 01 09:05:53 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1SaRY9-0007dq-7f
	for mharc-coreutils@gnu.org; Fri, 01 Jun 2012 09:05:53 -0400
Received: from eggs.gnu.org ([208.118.235.92]:56064)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1SaRY2-0007dg-CL
	for coreutils@gnu.org; Fri, 01 Jun 2012 09:05:52 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1SaRXs-0003xx-Vp
	for coreutils@gnu.org; Fri, 01 Jun 2012 09:05:45 -0400
Received: from mx1.redhat.com ([209.132.183.28]:48890)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1SaRXs-0003xh-Nq
	for coreutils@gnu.org; Fri, 01 Jun 2012 09:05:36 -0400
Received: from int-mx09.intmail.prod.int.phx2.redhat.com
	(int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id q51D5XFH023282
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK)
	for <coreutils@gnu.org>; Fri, 1 Jun 2012 09:05:34 -0400
Received: from [10.36.116.80] (ovpn-116-80.ams2.redhat.com [10.36.116.80])
	by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP
	id q51D5Vhr004427
	(version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO)
	for <coreutils@gnu.org>; Fri, 1 Jun 2012 09:05:33 -0400
Message-ID: <4FC8BE1B.9080905@draigBrady.com>
Date: Fri, 01 Jun 2012 14:05:31 +0100
From: =?UTF-8?B?UMOhZHJhaWcgQnJhZHk=?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:6.0) Gecko/20110816 Thunderbird/6.0
MIME-Version: 1.0
To: Coreutils <coreutils@gnu.org>
Subject: Fwd: [HEADS-UP] Rawhide: /tmp is now on tmpfs
References: <4FC74BD0.7030800@draigBrady.com>
In-Reply-To: <4FC74BD0.7030800@draigBrady.com>
X-Enigmail-Version: 1.3.2
X-Forwarded-Message-Id: <4FC74BD0.7030800@draigBrady.com>
Content-Type: text/plain; charset=UTF-8
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22
Content-Transfer-Encoding: quoted-printable
X-MIME-Autoconverted: from 8bit to quoted-printable by mx1.redhat.com id
	q51D5XFH023282
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.132.183.28
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 01 Jun 2012 13:05:52 -0000

It seems like we should default to /var/tmp in sort(1) and tac(1),
as debian and fedora at least are probably moving to tmpfs for /tmp.
I'll monitor the situation a bit before making the change.

cheers,
P=C3=A1draig.

-------- Original Message --------
Subject: Re: [HEADS-UP] Rawhide: /tmp is now on tmpfs
Date: Thu, 31 May 2012 11:45:36 +0100
From: P=C3=A1draig Brady <P@draigBrady.com>
Reply-To: Development discussions related to Fedora <devel@lists.fedorapr=
oject.org>
To: Development discussions related to Fedora <devel@lists.fedoraproject.=
org>
CC: Roberto Ragusa <mail@robertoragusa.it>

On 05/31/2012 08:14 AM, Roberto Ragusa wrote:
> On 05/31/2012 02:40 AM, Lennart Poettering wrote:
>> Heya!
>>
>> Please be aware that since the most recent systemd uploads /tmp is now
>> in tmpfs by default in Rawhide/F18.
> [...]
>> This will most likely lead to a problem or two with software that isn'=
t
>> happy about /tmp being small.
>=20
> For example "sort".

This is a good example because `sort` algorithmically needs
something below RAM in the memory hierarchy (i.e. bigger),
but with the same persistence characteristics of /tmp.

Currently `sort` defaults to $TMPDIR or if not set '/tmp'.

Now /var/tmp should be "more persistent" which we don't need,
but shouldn't be an issue, but should also not be in RAM
and so is more appropriate.

So I'll patch sort to default to /var/tmp rather than /tmp.

I'm a little worried about the general availability of /var/tmp.
I know I've created distros without it at least.

For my own reference, sort does support a list of tmp dirs,
but it'll need to be tweaked to support non existent dirs:

  $ seq 100000 | sort -T /foo -T /tmp -S1M
  sort: cannot create temporary file in `/foo': No such file or directory

`tac` from coreutils also needs a similar patch.

cheers,
P=C3=A1draig.


From MAILER-DAEMON Fri Jun 01 09:13:57 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1SaRfx-0000W8-LU
	for mharc-coreutils@gnu.org; Fri, 01 Jun 2012 09:13:57 -0400
Received: from eggs.gnu.org ([208.118.235.92]:42866)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1SaRfm-0000U1-90
	for coreutils@gnu.org; Fri, 01 Jun 2012 09:13:55 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1SaRfg-0005m0-5b
	for coreutils@gnu.org; Fri, 01 Jun 2012 09:13:45 -0400
Received: from mx.meyering.net ([88.168.87.75]:44221)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1SaRff-0005lr-Um
	for coreutils@gnu.org; Fri, 01 Jun 2012 09:13:40 -0400
Received: from rho.meyering.net (localhost.localdomain [127.0.0.1])
	by rho.meyering.net (Acme Bit-Twister) with ESMTP id 6C907600BC;
	Fri,  1 Jun 2012 15:13:38 +0200 (CEST)
From: Jim Meyering <jim@meyering.net>
To: =?iso-8859-1?Q?P=E1draig?= Brady <P@draigBrady.com>
Subject: Re: Fwd: [HEADS-UP] Rawhide: /tmp is now on tmpfs
In-Reply-To: <4FC8BE1B.9080905@draigBrady.com> (=?iso-8859-1?Q?=22P=E1drai?=
	=?iso-8859-1?Q?g?= Brady"'s message of
	"Fri, 01 Jun 2012 14:05:31 +0100")
References: <4FC74BD0.7030800@draigBrady.com> <4FC8BE1B.9080905@draigBrady.com>
Date: Fri, 01 Jun 2012 15:13:38 +0200
Message-ID: <87k3zr6upp.fsf@rho.meyering.net>
Lines: 15
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 88.168.87.75
Cc: Coreutils <coreutils@gnu.org>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 01 Jun 2012 13:13:55 -0000

P=E1draig Brady wrote:
> It seems like we should default to /var/tmp in sort(1) and tac(1),
> as debian and fedora at least are probably moving to tmpfs for /tmp.
> I'll monitor the situation a bit before making the change.

Thanks.

Just to be clear, sort would continue to honor $TMPDIR, and
your suggested change would affect only those rare uses of sort
for which $TMPDIR is not already set?

i.e., in sort.c,

  -# define DEFAULT_TMPDIR "/tmp"
  +# define DEFAULT_TMPDIR "/var/tmp"


From MAILER-DAEMON Fri Jun 01 09:21:02 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1SaRmo-00019e-Qq
	for mharc-coreutils@gnu.org; Fri, 01 Jun 2012 09:21:02 -0400
Received: from eggs.gnu.org ([208.118.235.92]:48070)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1SaRmi-00016o-2b
	for coreutils@gnu.org; Fri, 01 Jun 2012 09:21:01 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1SaRmb-0007I2-Lh
	for coreutils@gnu.org; Fri, 01 Jun 2012 09:20:55 -0400
Received: from mx1.redhat.com ([209.132.183.28]:14011)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1SaRmb-0007Hi-Bj
	for coreutils@gnu.org; Fri, 01 Jun 2012 09:20:49 -0400
Received: from int-mx10.intmail.prod.int.phx2.redhat.com
	(int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id q51DKdp6024750
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK);
	Fri, 1 Jun 2012 09:20:39 -0400
Received: from [10.36.116.80] (ovpn-116-80.ams2.redhat.com [10.36.116.80])
	by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP
	id q51DKbB0014082
	(version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO);
	Fri, 1 Jun 2012 09:20:38 -0400
Message-ID: <4FC8C1A4.9020106@draigBrady.com>
Date: Fri, 01 Jun 2012 14:20:36 +0100
From: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:6.0) Gecko/20110816 Thunderbird/6.0
MIME-Version: 1.0
To: Jim Meyering <jim@meyering.net>
Subject: Re: [HEADS-UP] Rawhide: /tmp is now on tmpfs
References: <4FC74BD0.7030800@draigBrady.com> <4FC8BE1B.9080905@draigBrady.com>
	<87k3zr6upp.fsf@rho.meyering.net>
In-Reply-To: <87k3zr6upp.fsf@rho.meyering.net>
X-Enigmail-Version: 1.3.2
Content-Type: text/plain; charset=ISO-8859-1
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23
Content-Transfer-Encoding: quoted-printable
X-MIME-Autoconverted: from 8bit to quoted-printable by mx1.redhat.com id
	q51DKdp6024750
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.132.183.28
Cc: Coreutils <coreutils@gnu.org>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 01 Jun 2012 13:21:01 -0000

On 06/01/2012 02:13 PM, Jim Meyering wrote:
> P=E1draig Brady wrote:
>> It seems like we should default to /var/tmp in sort(1) and tac(1),
>> as debian and fedora at least are probably moving to tmpfs for /tmp.
>> I'll monitor the situation a bit before making the change.
>=20
> Thanks.
>=20
> Just to be clear, sort would continue to honor $TMPDIR, and
> your suggested change would affect only those rare uses of sort
> for which $TMPDIR is not already set?

TMPDIR is not normally set in my experience.
It can be set to override the default.

If it was normally set to point to tmpfs,
then we couldn't really use it, but I
don't think this is the case.

> i.e., in sort.c,
>=20
>   -# define DEFAULT_TMPDIR "/tmp"
>   +# define DEFAULT_TMPDIR "/var/tmp"

Right.

cheers,
P=E1draig.


From MAILER-DAEMON Fri Jun 01 09:21:55 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1SaRnf-0001Gy-13
	for mharc-coreutils@gnu.org; Fri, 01 Jun 2012 09:21:55 -0400
Received: from eggs.gnu.org ([208.118.235.92]:55825)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <basavaraja.b@gmail.com>) id 1SaRjU-0000nO-Gl
	for coreutils@gnu.org; Fri, 01 Jun 2012 09:17:41 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <basavaraja.b@gmail.com>) id 1SaRjG-0006kt-DM
	for coreutils@gnu.org; Fri, 01 Jun 2012 09:17:36 -0400
Received: from mail-gg0-f169.google.com ([209.85.161.169]:62100)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <basavaraja.b@gmail.com>) id 1SaRjG-0006kV-61
	for coreutils@gnu.org; Fri, 01 Jun 2012 09:17:22 -0400
Received: by ggm4 with SMTP id 4so2013953ggm.0
	for <coreutils@gnu.org>; Fri, 01 Jun 2012 06:17:19 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
	h=mime-version:date:message-id:subject:from:to:content-type;
	bh=FLamdIUQgmbJAuqjAmSv0YTrm4/kVygEjiUknGDtRLo=;
	b=Beq5e+bluNudBddwaExuxnG99rKZn5o7NUGeEc4yQ9u2/i3AFtRmzsl66dYZo2yyBa
	/3BQ0hb0g0LmhJHPNP/7WNIY6h6tBdk5ergzQKmGF3BRJp1qUaaxHfzrW7097ErMmTLE
	CJDCKzfglg2UI1/X82IT3RxKHcPjdMNHsL498PLdAnoGlhvU8Zq2cWxVKz2xNK5k3fNA
	GIWrNMMku7t+pBORxmhKs4obFrOz9miQS+xFSbu1TZ6vlc6zLqHxkTA8NOaYznylg62m
	gGJmayYlduMWBKxuM93ss4HMMq9lLviZggxWbqGLlLbSJPn/H/L5tzbr80FyklcsWx4h
	RLWw==
MIME-Version: 1.0
Received: by 10.236.152.97 with SMTP id c61mr2526497yhk.130.1338556639696;
	Fri, 01 Jun 2012 06:17:19 -0700 (PDT)
Received: by 10.236.73.133 with HTTP; Fri, 1 Jun 2012 06:17:19 -0700 (PDT)
Date: Fri, 1 Jun 2012 18:47:19 +0530
Message-ID: <CADE=9AQr7d7pQUZKtpG-TC298AV5+o4waKr6REnY+ikVa3K9xg@mail.gmail.com>
Subject: Problem while compiling libstdbuf
From: Basavaraj B <basavaraja.b@gmail.com>
To: coreutils@gnu.org
Content-Type: multipart/alternative; boundary=20cf303f669e05576504c16901ab
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.85.161.169
X-Mailman-Approved-At: Fri, 01 Jun 2012 09:21:54 -0400
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 01 Jun 2012 13:17:41 -0000

--20cf303f669e05576504c16901ab
Content-Type: text/plain; charset=ISO-8859-1

Hi Team,

I am compiling coreutils-8.14 on NonStop (TANDEM) platform.
Currently I need to compile stdbuf which has libstdbuf.so dependency.

I modified configure and configure.ac to avoid check of $elf_sys and $GCC
so that I could generate stdbuf and libstdbuf.so successfully.
Only stdbuf binary is generated but no libstdbuf.so.

we use c99 compiler.
Please help me solving this issue.

Regards,
Basavaraj

--20cf303f669e05576504c16901ab
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Hi Team,<div><br></div><div>I am compiling coreutils-8.14 on NonStop (TANDE=
M) platform.</div><div>Currently I need to compile stdbuf which has libstdb=
uf.so=A0dependency.</div><div><br></div><div>I modified configure and <a hr=
ef=3D"http://configure.ac">configure.ac</a> to avoid check of=A0$elf_sys an=
d=A0$GCC so that I could generate stdbuf and libstdbuf.so successfully.</di=
v>
<div>Only stdbuf binary is generated but no libstdbuf.so.</div><div><br></d=
iv><div>we use c99 compiler.=A0</div><div>Please help me solving this issue=
.</div><div><br></div><div>Regards,</div><div>Basavaraj=A0</div><div><br></=
div>

--20cf303f669e05576504c16901ab--


From MAILER-DAEMON Fri Jun 01 09:31:57 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1SaRxN-00051n-Pz
	for mharc-coreutils@gnu.org; Fri, 01 Jun 2012 09:31:57 -0400
Received: from eggs.gnu.org ([208.118.235.92]:40701)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1SaRxK-00051T-Q2
	for coreutils@gnu.org; Fri, 01 Jun 2012 09:31:56 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1SaRxE-0001nX-FG
	for coreutils@gnu.org; Fri, 01 Jun 2012 09:31:54 -0400
Received: from mx1.redhat.com ([209.132.183.28]:62726)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1SaRxE-0001nM-8F
	for coreutils@gnu.org; Fri, 01 Jun 2012 09:31:48 -0400
Received: from int-mx02.intmail.prod.int.phx2.redhat.com
	(int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id q51DVjYl011006
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK);
	Fri, 1 Jun 2012 09:31:45 -0400
Received: from [10.36.116.80] (ovpn-116-80.ams2.redhat.com [10.36.116.80])
	by int-mx02.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP
	id q51DVhgK007148
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO);
	Fri, 1 Jun 2012 09:31:45 -0400
Message-ID: <4FC8C43F.4060309@draigBrady.com>
Date: Fri, 01 Jun 2012 14:31:43 +0100
From: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:6.0) Gecko/20110816 Thunderbird/6.0
MIME-Version: 1.0
To: Basavaraj B <basavaraja.b@gmail.com>
Subject: Re: Problem while compiling libstdbuf
References: <CADE=9AQr7d7pQUZKtpG-TC298AV5+o4waKr6REnY+ikVa3K9xg@mail.gmail.com>
In-Reply-To: <CADE=9AQr7d7pQUZKtpG-TC298AV5+o4waKr6REnY+ikVa3K9xg@mail.gmail.com>
X-Enigmail-Version: 1.3.2
Content-Type: text/plain; charset=ISO-8859-1
X-Scanned-By: MIMEDefang 2.67 on 10.5.11.12
Content-Transfer-Encoding: quoted-printable
X-MIME-Autoconverted: from 8bit to quoted-printable by mx1.redhat.com id
	q51DVjYl011006
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.132.183.28
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 01 Jun 2012 13:31:56 -0000

On 06/01/2012 02:17 PM, Basavaraj B wrote:
> Hi Team,
>=20
> I am compiling coreutils-8.14 on NonStop (TANDEM) platform.
> Currently I need to compile stdbuf which has libstdbuf.so dependency.
>=20
> I modified configure and configure.ac <http://configure.ac> to avoid ch=
eck of $elf_sys and $GCC so that I could generate stdbuf and libstdbuf.so=
 successfully.
> Only stdbuf binary is generated but no libstdbuf.so.
>=20
> we use c99 compiler.=20
> Please help me solving this issue.

The libstdbuf shared lib was restricted to gcc elf systems,
due to the amazing variance for shared lib support on various systems.
I didn't want to introduce libtool either due to the added complexity.

So...

If you look in src/Makefile.am you can see:

  # Note libstdbuf is only compiled if GCC is available
  # (as per the check in configure.ac), so these flags should be availabl=
e.
  # libtool is probably required to relax this dependency.
  libstdbuf_so_LDFLAGS =3D -shared
  libstdbuf_so_CFLAGS =3D -fPIC $(AM_CFLAGS)

So I guess you might need to tweak those options?
If you get it working it would be great if you
could report back so that we might support something
more generic in the build.

cheers,
P=E1draig.


From MAILER-DAEMON Fri Jun 01 09:39:23 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1SaS4Z-0006Aa-DO
	for mharc-coreutils@gnu.org; Fri, 01 Jun 2012 09:39:23 -0400
Received: from eggs.gnu.org ([208.118.235.92]:47851)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1SaS4S-0006AO-8P
	for coreutils@gnu.org; Fri, 01 Jun 2012 09:39:21 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1SaS4M-0003r4-L2
	for coreutils@gnu.org; Fri, 01 Jun 2012 09:39:15 -0400
Received: from mx.meyering.net ([88.168.87.75]:44314)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1SaS4M-0003qf-E3
	for coreutils@gnu.org; Fri, 01 Jun 2012 09:39:10 -0400
Received: from rho.meyering.net (localhost.localdomain [127.0.0.1])
	by rho.meyering.net (Acme Bit-Twister) with ESMTP id E3CD9600BC;
	Fri,  1 Jun 2012 15:39:06 +0200 (CEST)
From: Jim Meyering <jim@meyering.net>
To: =?iso-8859-1?Q?P=E1draig?= Brady <P@draigBrady.com>
Subject: Re: [HEADS-UP] Rawhide: /tmp is now on tmpfs
In-Reply-To: <4FC8C1A4.9020106@draigBrady.com> (=?iso-8859-1?Q?=22P=E1drai?=
	=?iso-8859-1?Q?g?= Brady"'s message of
	"Fri, 01 Jun 2012 14:20:36 +0100")
References: <4FC74BD0.7030800@draigBrady.com>
	<4FC8BE1B.9080905@draigBrady.com> <87k3zr6upp.fsf@rho.meyering.net>
	<4FC8C1A4.9020106@draigBrady.com>
Date: Fri, 01 Jun 2012 15:39:06 +0200
Message-ID: <87ehpz6tj9.fsf@rho.meyering.net>
Lines: 39
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 88.168.87.75
Cc: Coreutils <coreutils@gnu.org>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 01 Jun 2012 13:39:22 -0000

P=E1draig Brady wrote:
> On 06/01/2012 02:13 PM, Jim Meyering wrote:
>> P=E1draig Brady wrote:
>>> It seems like we should default to /var/tmp in sort(1) and tac(1),
>>> as debian and fedora at least are probably moving to tmpfs for /tmp.
>>> I'll monitor the situation a bit before making the change.
>>
>> Thanks.
>>
>> Just to be clear, sort would continue to honor $TMPDIR, and
>> your suggested change would affect only those rare uses of sort
>> for which $TMPDIR is not already set?
>
> TMPDIR is not normally set in my experience.
> It can be set to override the default.

Oh, you're right: unset TMPDIR is probably not rare at all.
I've been setting it myself for so long...

> If it was normally set to point to tmpfs,
> then we couldn't really use it, but I
> don't think this is the case.
>
>> i.e., in sort.c,
>>
>>   -# define DEFAULT_TMPDIR "/tmp"
>>   +# define DEFAULT_TMPDIR "/var/tmp"
>
> Right.

Good.  That should be fine.
Anyone who is using sort on very large data sets must already deal
with disk space via manually-set TMPDIR or an explicit -T option.

However, I'm sure some people have /tmp deliberately large
(and much larger than /var/tmp) for precisely this use case, and
would not be pleased.  But if those same people upgrade to Fedora 18
they will adapt (simple, manual config) so that /tmp remains disk-backed
with whatever size they require.


From MAILER-DAEMON Tue Jun 05 16:40:49 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1Sc0Yb-0007AP-QL
	for mharc-coreutils@gnu.org; Tue, 05 Jun 2012 16:40:49 -0400
Received: from eggs.gnu.org ([208.118.235.92]:58366)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Sc0YX-0006zK-I6
	for coreutils@gnu.org; Tue, 05 Jun 2012 16:40:48 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Sc0YU-0006sA-W1
	for coreutils@gnu.org; Tue, 05 Jun 2012 16:40:45 -0400
Received: from mx.meyering.net ([88.168.87.75]:60187)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Sc0YT-0006rk-NQ
	for coreutils@gnu.org; Tue, 05 Jun 2012 16:40:42 -0400
Received: from rho.meyering.net (localhost.localdomain [127.0.0.1])
	by rho.meyering.net (Acme Bit-Twister) with ESMTP id D84646184D
	for <coreutils@gnu.org>; Tue,  5 Jun 2012 22:40:38 +0200 (CEST)
From: Jim Meyering <jim@meyering.net>
To: coreutils@gnu.org
Subject: oops: unmarked diagnostic
Date: Tue, 05 Jun 2012 22:40:38 +0200
Message-ID: <87bokx5w6x.fsf@rho.meyering.net>
Lines: 28
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 88.168.87.75
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Tue, 05 Jun 2012 20:40:48 -0000

I noticed this a minute too late.

>From b9f84267338ad8f1a88f235f9e32b236cc82b864 Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering@redhat.com>
Date: Tue, 5 Jun 2012 22:38:26 +0200
Subject: [PATCH] maint: mark new diagnostic for translation

* src/head.c (elide_tail_lines_seekable): Mark new diagnostic.
I will look at the results of "make syntax-check", I will look...
---
 src/head.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/head.c b/src/head.c
index 0d5e1b2..c13c064 100644
--- a/src/head.c
+++ b/src/head.c
@@ -671,7 +671,7 @@ elide_tail_lines_seekable (const char *pretty_filename, int fd,
               /* Set file pointer to the byte after what we've output.  */
               if (lseek (fd, pos + n + 1, SEEK_SET) < 0)
                 {
-                  error (0, errno, "%s: failed to reset file pointer",
+                  error (0, errno, _("%s: failed to reset file pointer"),
                          quote (pretty_filename));
                   return false;
                 }
--
1.7.11.rc1


From MAILER-DAEMON Tue Jun 05 16:42:50 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1Sc0aY-0000F4-V0
	for mharc-coreutils@gnu.org; Tue, 05 Jun 2012 16:42:50 -0400
Received: from eggs.gnu.org ([208.118.235.92]:58704)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Sc0aT-0000DC-MJ
	for coreutils@gnu.org; Tue, 05 Jun 2012 16:42:49 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Sc0aO-0007TH-BT
	for coreutils@gnu.org; Tue, 05 Jun 2012 16:42:45 -0400
Received: from mx.meyering.net ([88.168.87.75]:60194)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Sc0aN-0007TA-IU
	for coreutils@gnu.org; Tue, 05 Jun 2012 16:42:40 -0400
Received: from rho.meyering.net (localhost.localdomain [127.0.0.1])
	by rho.meyering.net (Acme Bit-Twister) with ESMTP id EE00F600BA
	for <coreutils@gnu.org>; Tue,  5 Jun 2012 22:42:37 +0200 (CEST)
From: Jim Meyering <jim@meyering.net>
To: coreutils@gnu.org
Subject: su, going, going, ...
Date: Tue, 05 Jun 2012 22:42:37 +0200
Message-ID: <8762b55w3m.fsf@rho.meyering.net>
Lines: 1220
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 88.168.87.75
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Tue, 05 Jun 2012 20:42:49 -0000

No one objected, so I'm about to remove su, with this:
(I'll wait 10-12 hours, just in case ;-)

>From db2c12039e5b008245b33778d28e6d9ab1156844 Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering@redhat.com>
Date: Fri, 25 May 2012 18:10:25 +0200
Subject: [PATCH] su: remove program (util-linux is now the best source for
 it)

* README: Omit "su" from list of programs.
* src/su.c: Remove file.
* src/Makefile.am: Remove su-related rules and variables.
* tests/misc/su-fail: Remove test.
* tests/Makefile.am (TESTS): Remove misc/su-fail.
* tests/misc/invalid-opt: Remove su-related code.
* src/.gitignore: Remove su.
* man/su.x: Remove file.
* man/Makefile.am (su.1): Remove rule.
* po/POTFILES.in: Remove su.c from the list.
* TODO: Remove ancient entry.
* NEWS (Changes in behavior): Mention it.
* doc/coreutils.texi: Remove su-related description.
* AUTHORS: Remove su.
* m4/lib-check.m4 (cu_LIB_CHECK): Remove file/macro.
* configure.ac: Remove su-related code and sole use of cu_LIB_CHECK.
* scripts/git-hooks/commit-msg: Remove su from this list, too.
---
 AUTHORS                      |   1 -
 NEWS                         |   8 +
 README                       |   2 +-
 TODO                         |   3 -
 configure.ac                 |  13 +-
 doc/coreutils.texi           | 164 +-------------
 m4/lib-check.m4              |  60 -----
 man/.gitignore               |   1 -
 man/Makefile.am              |   3 +-
 man/su.x                     |   4 -
 po/POTFILES.in               |   1 -
 scripts/git-hooks/commit-msg |   2 +-
 src/.gitignore               |   1 -
 src/Makefile.am              |  63 +-----
 src/su.c                     | 520 ---------------------------------------=
----
 tests/Makefile.am            |   1 -
 tests/misc/help-version      |   1 -
 tests/misc/invalid-opt       |   1 -
 tests/misc/su-fail           |  30 ---
 19 files changed, 17 insertions(+), 862 deletions(-)
 delete mode 100644 m4/lib-check.m4
 delete mode 100644 man/su.x
 delete mode 100644 src/su.c
 delete mode 100755 tests/misc/su-fail

diff --git a/AUTHORS b/AUTHORS
index d1024a5..9984a2e 100644
--- a/AUTHORS
+++ b/AUTHORS
@@ -80,7 +80,6 @@ split: Torbj=F6rn Granlund, Richard M. Stallman
 stat: Michael Meskes
 stdbuf: P=E1draig Brady
 stty: David MacKenzie
-su: David MacKenzie
 sum: Kayvan Aghaiepour, David MacKenzie
 sync: Jim Meyering
 tac: Jay Lepreau, David MacKenzie
diff --git a/NEWS b/NEWS
index c0e5fdb..54d24c3 100644
--- a/NEWS
+++ b/NEWS
@@ -30,6 +30,14 @@ GNU coreutils NEWS                                    -*=
- outline -*-

   stat -f recognizes the new remote file system type, panfs.

+** Changes in behavior
+
+  su: this program has been removed.  We stopped installing "su" by
+  default with the release of coreutils-6.9.90 on 2007-12-01.  Now,
+  that the util-linux package has the union of the Suse and Fedora
+  patches as well as enough support to build on the Hurd, we no longer
+  have any reason to include it here.
+

 * Noteworthy changes in release 8.17 (2012-05-10) [stable]

diff --git a/README b/README
index 3cc476a..09961a0 100644
--- a/README
+++ b/README
@@ -13,7 +13,7 @@ The programs that can be built with this package are:
   link ln logname ls md5sum mkdir mkfifo mknod mktemp mv nice nl nohup
   nproc od paste pathchk pinky pr printenv printf ptx pwd readlink realpath
   rm rmdir runcon seq sha1sum sha224sum sha256sum sha384sum sha512sum shred
-  shuf sleep sort split stat stdbuf stty su sum sync tac tail tee test
+  shuf sleep sort split stat stdbuf stty sum sync tac tail tee test
   timeout touch tr true truncate tsort tty uname unexpand uniq unlink
   uptime users vdir wc who whoami yes

diff --git a/TODO b/TODO
index 8a8cf43..cd434e5 100644
--- a/TODO
+++ b/TODO
@@ -78,9 +78,6 @@ unexpand: [http://www.opengroup.org/onlinepubs/007908799/=
xcu/unexpand.html]
   printf 'x\t \t y\n'|unexpand -t 8,9 should print its input, unmodified.
   printf 'x\t \t y\n'|unexpand -t 5,8 should print "x\ty\n"

-Let GNU su use the 'wheel' group if appropriate.
-  (there are a couple patches, already)
-
 sort: Investigate better sorting algorithms; see Knuth vol. 3.

   We tried list merge sort, but it was about 50% slower than the
diff --git a/configure.ac b/configure.ac
index 5a4860e..5651fd1 100644
--- a/configure.ac
+++ b/configure.ac
@@ -380,7 +380,6 @@ AC_CHECK_DECLS([strsignal, sys_siglist, _sys_siglist, _=
_sys_siglist], , ,
   [AC_INCLUDES_DEFAULT
 #include <signal.h>])

-cu_LIB_CHECK
 cu_GMP

 # Build df only if there's a point to it.
@@ -421,7 +420,7 @@ t=3D`sed -n '/^'$v' =3D/,/[[^\]]$/p' $mk \
 no_install_progs_default=3D`echo "$t"|sed 's/ $//'`

 # Unfortunately, due to the way autoconf's AS_HELP_STRING works, the list
-# of default-not-installed programs, "arch hostname su", must appear in two
+# of default-not-installed programs, "arch hostname", must appear in two
 # places: in this file below, and in $mk.  Using "$no_install_progs_defaul=
t"
 # below cannot work.  And we can't substitute the names into $mk because
 # automake needs the literals, too.
@@ -448,7 +447,7 @@ esac
 # Note how the second argument below is a literal, with "," separators.
 # That is required due to the way the macro works, and since the
 # corresponding ./configure option argument is comma-separated on input.
-gl_INCLUDE_EXCLUDE_PROG([optional_bin_progs], [arch,hostname,su])
+gl_INCLUDE_EXCLUDE_PROG([optional_bin_progs], [arch,hostname])

 # Now that we know which programs will actually be built up, figure out
 # which optional helper progs should be compiled.
@@ -457,14 +456,6 @@ case " $optional_bin_progs " in
   *' stdbuf '*) gl_ADD_PROG([optional_pkglib_progs], [libstdbuf.so]) ;;
 esac

-# Set INSTALL_SU if su installation has been requested via
-# --enable-install-program=3Dsu.
-AC_SUBST([INSTALL_SU])
-case " $optional_bin_progs " in
-  *' su '*) INSTALL_SU=3Dyes ;;
-  *)        INSTALL_SU=3Dno ;;
-esac
-
 MAN=3D`echo "$optional_bin_progs "|sed 's/ /.1 /g;s/ $//'|tr -d '\\015\\01=
2'`

 # Change "ginstall.1" to "install.1" in $MAN.
diff --git a/doc/coreutils.texi b/doc/coreutils.texi
index 767267b..f7251b2 100644
--- a/doc/coreutils.texi
+++ b/doc/coreutils.texi
@@ -109,7 +109,6 @@
 * stat: (coreutils)stat invocation.             Report file(system) status.
 * stdbuf: (coreutils)stdbuf invocation.         Modify stdio buffering.
 * stty: (coreutils)stty invocation.             Print/change terminal sett=
ings.
-* su: (coreutils)su invocation.                 Modify user and group ID.
 * sum: (coreutils)sum invocation.               Print traditional checksum.
 * sync: (coreutils)sync invocation.             Synchronize memory and dis=
k.
 * tac: (coreutils)tac invocation.               Reverse files.
@@ -200,7 +199,7 @@ Top
 * User information::             id logname whoami groups users who
 * System context::               date arch nproc uname hostname hostid upt=
ime
 * SELinux context::              chcon runcon
-* Modified command invocation::  chroot env nice nohup stdbuf su timeout
+* Modified command invocation::  chroot env nice nohup stdbuf timeout
 * Process control::              kill
 * Delaying::                     sleep
 * Numeric operations::           factor seq
@@ -443,7 +442,6 @@ Top
 * nice invocation::              Run a command with modified niceness
 * nohup invocation::             Run a command immune to hangups
 * stdbuf invocation::            Run a command with modified I/O buffering
-* su invocation::                Run a command with substitute user and gr=
oup ID
 * timeout invocation::           Run a command with a time limit

 Process control
@@ -766,7 +764,7 @@ Exit status
 Here are some of the exceptions:
 @command{chroot}, @command{env}, @command{expr}, @command{nice},
 @command{nohup}, @command{printenv}, @command{sort}, @command{stdbuf},
-@command{su}, @command{test}, @command{timeout}, @command{tty}.
+@command{test}, @command{timeout}, @command{tty}.


 @node Backup options
@@ -15266,7 +15264,6 @@ Modified command invocation
 * nice invocation::             Modify niceness.
 * nohup invocation::            Immunize to hangups.
 * stdbuf invocation::           Modify buffering of standard streams.
-* su invocation::               Modify user and group ID.
 * timeout invocation::          Run with time limit.
 @end menu

@@ -15794,163 +15791,6 @@ stdbuf invocation
 @end display


-@node su invocation
-@section @command{su}: Run a command with substitute user and group ID
-
-@pindex su
-@cindex substitute user and group IDs
-@cindex user ID, switching
-@cindex super-user, becoming
-@cindex root, becoming
-
-@command{su} allows one user to temporarily become another user.  It runs a
-command (often an interactive shell) with the real and effective user
-ID, group ID, and supplemental groups of a given @var{user}.  Synopsis:
-
-@example
-su [@var{option}]@dots{} [@var{user} [@var{arg}]@dots{}]
-@end example
-
-@cindex passwd entry, and @command{su} shell
-@flindex /bin/sh
-@flindex /etc/passwd
-If no @var{user} is given, the default is @code{root}, the super-user.
-The shell to use is taken from @var{user}'s @code{passwd} entry, or
-@file{/bin/sh} if none is specified there.  If @var{user} has a
-password, @command{su} prompts for the password unless run by a user with
-effective user ID of zero (the super-user).
-
-@vindex HOME
-@vindex SHELL
-@vindex USER
-@vindex LOGNAME
-@cindex login shell
-By default, @command{su} does not change the current directory.
-It sets the environment variables @env{HOME} and @env{SHELL}
-from the password entry for @var{user}, and if @var{user} is not
-the super-user, sets @env{USER} and @env{LOGNAME} to @var{user}.
-By default, the shell is not a login shell.
-
-Any additional @var{arg}s are passed as additional arguments to the
-shell.
-
-@cindex @option{-su}
-GNU @command{su} does not treat @file{/bin/sh} or any other shells special=
ly
-(e.g., by setting @code{argv[0]} to @option{-su}, passing @option{-c} only
-to certain shells, etc.).
-
-@findex syslog
-@command{su} can optionally be compiled to use @code{syslog} to report
-failed, and optionally successful, @command{su} attempts.  (If the system
-supports @code{syslog}.)  However, GNU @command{su} does not check if the
-user is a member of the @code{wheel} group; see below.
-
-The program accepts the following options.  Also see @ref{Common options}.
-
-@table @samp
-@item -c @var{command}
-@itemx --command=3D@var{command}
-@opindex -c
-@opindex --command
-Pass @var{command}, a single command line to run, to the shell with
-a @option{-c} option instead of starting an interactive shell.
-
-@item -f
-@itemx --fast
-@opindex -f
-@opindex --fast
-@flindex .cshrc
-@cindex file name pattern expansion, disabled
-@cindex globbing, disabled
-Pass the @option{-f} option to the shell.  This probably only makes sense
-if the shell run is @command{csh} or @command{tcsh}, for which the @option=
{-f}
-option prevents reading the startup file (@file{.cshrc}).  With
-Bourne-like shells, the @option{-f} option disables file name pattern
-expansion (globbing), which is not likely to be useful.
-
-@item -
-@itemx -l
-@itemx --login
-@opindex -
-@opindex -l
-@opindex --login
-@c other variables already indexed above
-@vindex TERM
-@vindex PATH
-@cindex login shell, creating
-Make the shell a login shell.  This means the following.  Unset all
-environment variables except @env{TERM}, @env{HOME}, and @env{SHELL}
-(which are set as described above), and @env{USER} and @env{LOGNAME}
-(which are set, even for the super-user, as described above), and set
-@env{PATH} to a compiled-in default value.  Change to @var{user}'s home
-directory.  Prepend @samp{-} to the shell's name, intended to make it
-read its login startup file(s).
-
-@item -m
-@itemx -p
-@itemx --preserve-environment
-@opindex -m
-@opindex -p
-@opindex --preserve-environment
-@cindex environment, preserving
-@flindex /etc/shells
-@cindex restricted shell
-Do not change the environment variables @env{HOME}, @env{USER},
-@env{LOGNAME}, or @env{SHELL}.  Run the shell given in the environment
-variable @env{SHELL} instead of the shell from @var{user}'s passwd
-entry, unless the user running @command{su} is not the super-user and
-@var{user}'s shell is restricted.  A @dfn{restricted shell} is one that
-is not listed in the file @file{/etc/shells}, or in a compiled-in list
-if that file does not exist.  Parts of what this option does can be
-overridden by @option{--login} and @option{--shell}.
-
-@item -s @var{shell}
-@itemx --shell=3D@var{shell}
-@opindex -s
-@opindex --shell
-Run @var{shell} instead of the shell from @var{user}'s passwd entry,
-unless the user running @command{su} is not the super-user and @var{user}'s
-shell is restricted (see @option{-m} just above).
-
-@end table
-
-@cindex exit status of @command{su}
-Exit status:
-
-@display
-125 if @command{su} itself fails
-126 if subshell is found but cannot be invoked
-127 if subshell cannot be found
-the exit status of the subshell otherwise
-@end display
-
-@cindex wheel group, not supported
-@cindex group wheel, not supported
-@cindex fascism
-@subsection Why GNU @command{su} does not support the @samp{wheel} group
-
-(This section is by Richard Stallman.)
-
-@cindex Twenex
-@cindex MIT AI lab
-Sometimes a few of the users try to hold total power over all the
-rest.  For example, in 1984, a few users at the MIT AI lab decided to
-seize power by changing the operator password on the Twenex system and
-keeping it secret from everyone else.  (I was able to thwart this coup
-and give power back to the users by patching the kernel, but I
-wouldn't know how to do that in Unix.)
-
-However, occasionally the rulers do tell someone.  Under the usual
-@command{su} mechanism, once someone learns the root password who
-sympathizes with the ordinary users, he or she can tell the rest.  The
-``wheel group'' feature would make this impossible, and thus cement the
-power of the rulers.
-
-I'm on the side of the masses, not that of the rulers.  If you are
-used to supporting the bosses and sysadmins in whatever they do, you
-might find this idea strange at first.
-
-
 @node timeout invocation
 @section @command{timeout}: Run a command with a time limit

diff --git a/m4/lib-check.m4 b/m4/lib-check.m4
deleted file mode 100644
index a33c623..0000000
--- a/m4/lib-check.m4
+++ /dev/null
@@ -1,60 +0,0 @@
-#serial 11
-
-dnl Misc lib-related macros for coreutils.
-
-# Copyright (C) 1993-2012 Free Software Foundation, Inc.
-
-# This program is free software: you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation, either version 3 of the License, or
-# (at your option) any later version.
-
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU General Public License for more details.
-
-# You should have received a copy of the GNU General Public License
-# along with this program.  If not, see <http://www.gnu.org/licenses/>.
-
-# Written by Jim Meyering.
-
-AC_DEFUN([cu_LIB_CHECK],
-[
-
-  # Check for libypsec.a on Dolphin M88K machines.
-  AC_CHECK_LIB([ypsec], [main])
-
-  # m88k running dgux 5.4 needs this
-  AC_CHECK_LIB([ldgc], [main])
-
-  # The -lsun library is required for YP support on Irix-4.0.5 systems.
-  # m88k/svr3 DolphinOS systems using YP need -lypsec for id.
-  AC_SEARCH_LIBS([yp_match], [sun ypsec])
-
-  # SysV needs -lsec, older versions of Linux need -lshadow for
-  # shadow passwords.  UnixWare 7 needs -lgen.
-  AC_SEARCH_LIBS([getspnam], [shadow sec gen])
-
-  AC_CHECK_HEADERS([shadow.h])
-
-  # Requirements for su.c.
-  shadow_includes=3D"\
-$ac_includes_default
-#if HAVE_SHADOW_H
-# include <shadow.h>
-#endif
-"
-  AC_CHECK_MEMBERS([struct spwd.sp_pwdp],,,[$shadow_includes])
-  AC_CHECK_FUNCS([getspnam])
-
-  # SCO-ODT-3.0 is reported to need -lufc for crypt.
-  # NetBSD needs -lcrypt for crypt.
-  LIB_CRYPT=3D
-  cu_saved_libs=3D"$LIBS"
-  AC_SEARCH_LIBS([crypt], [ufc crypt],
-                 [test "$ac_cv_search_crypt" =3D "none required" ||
-                  LIB_CRYPT=3D"$ac_cv_search_crypt"])
-  LIBS=3D"$cu_saved_libs"
-  AC_SUBST([LIB_CRYPT])
-])
diff --git a/man/.gitignore b/man/.gitignore
index 9ae1183..aa6fa51 100644
--- a/man/.gitignore
+++ b/man/.gitignore
@@ -76,7 +76,6 @@ split.1
 stat.1
 stdbuf.1
 stty.1
-su.1
 sum.1
 sync.1
 tac.1
diff --git a/man/Makefile.am b/man/Makefile.am
index d869f9b..cc93a42 100644
--- a/man/Makefile.am
+++ b/man/Makefile.am
@@ -110,7 +110,6 @@ split.1:	$(common_dep)	$(srcdir)/split.x	../src/split.c
 stat.1:		$(common_dep)	$(srcdir)/stat.x	../src/stat.c
 stdbuf.1:	$(common_dep)	$(srcdir)/stdbuf.x	../src/stdbuf.c
 stty.1:		$(common_dep)	$(srcdir)/stty.x	../src/stty.c
-su.1:		$(common_dep)	$(srcdir)/su.x		../src/su.c
 sum.1:		$(common_dep)	$(srcdir)/sum.x		../src/sum.c
 sync.1:		$(common_dep)	$(srcdir)/sync.x	../src/sync.c
 tac.1:		$(common_dep)	$(srcdir)/tac.x		../src/tac.c
@@ -192,7 +191,7 @@ ASSORT =3D LC_ALL=3DC sort

 # Ensure that for each .x file in this directory, there is a
 # corresponding .1 file in the definition of $(dist_man1_MANS) above.
-# But since that expansion usually lacks programs like su and arch,
+# But since that expansion usually lacks programs like arch and hostname,
 # add them here manually.
 .PHONY: check-x-vs-1
 check-x-vs-1:
diff --git a/man/su.x b/man/su.x
deleted file mode 100644
index b368dd6..0000000
--- a/man/su.x
+++ /dev/null
@@ -1,4 +0,0 @@
-[NAME]
-su \- run a shell with substitute user and group IDs
-[DESCRIPTION]
-.\" Add any additional description here
diff --git a/po/POTFILES.in b/po/POTFILES.in
index 7b3de0b..b938dfc 100644
--- a/po/POTFILES.in
+++ b/po/POTFILES.in
@@ -118,7 +118,6 @@ src/split.c
 src/stat.c
 src/stdbuf.c
 src/stty.c
-src/su.c
 src/sum.c
 src/sync.c
 src/system.h
diff --git a/scripts/git-hooks/commit-msg b/scripts/git-hooks/commit-msg
index 3c1c63d..a5e4090 100755
--- a/scripts/git-hooks/commit-msg
+++ b/scripts/git-hooks/commit-msg
@@ -19,7 +19,7 @@ my @valid =3D qw(
     mv nice nl nohup nproc od paste pathchk pinky pr printenv printf
     ptx pwd readlink realpath rm rmdir runcon seq sha1sum sha224sum sha256=
sum
     sha384sum sha512sum shred shuf sleep sort split stat stdbuf stty
-    su sum sync tac tail tee test timeout touch tr true truncate tsort
+    sum sync tac tail tee test timeout touch tr true truncate tsort
     tty uname unexpand uniq unlink uptime users vdir wc who whoami yes

     copy gnulib tests maint doc build scripts
diff --git a/src/.gitignore b/src/.gitignore
index 9c4c9b7..d78178c 100644
--- a/src/.gitignore
+++ b/src/.gitignore
@@ -87,7 +87,6 @@ split
 stat
 stdbuf
 stty
-su
 sum
 sync
 tac
diff --git a/src/Makefile.am b/src/Makefile.am
index 06ab615..3099505 100644
--- a/src/Makefile.am
+++ b/src/Makefile.am
@@ -21,7 +21,7 @@
 # Hence, if you want to install programs from this list anyway, say A and =
B,
 # use --enable-install-program=3DA,B
 no_install__progs =3D \
-  arch hostname su
+  arch hostname

 build_if_possible__progs =3D \
   chroot	\
@@ -32,7 +32,6 @@ build_if_possible__progs =3D \
   pinky		\
   stdbuf	\
   stty		\
-  su		\
   uptime	\
   users		\
   who
@@ -164,7 +163,7 @@ noinst_HEADERS =3D	\
 EXTRA_DIST =3D dcgen dircolors.hin tac-pipe.c \
   wheel-gen.pl extract-magic c99-to-c89.diff
 BUILT_SOURCES =3D
-CLEANFILES =3D $(SCRIPTS) su
+CLEANFILES =3D $(SCRIPTS)

 # Also remove these sometimes-built programs.
 # For example, even when excluded, they're built via _sc_check-AUTHORS.
@@ -268,7 +267,6 @@ split_LDADD =3D $(LDADD)
 stat_LDADD =3D $(LDADD)
 stdbuf_LDADD =3D $(LDADD)
 stty_LDADD =3D $(LDADD)
-su_LDADD =3D $(LDADD)
 sum_LDADD =3D $(LDADD)
 sync_LDADD =3D $(LDADD)
 tac_LDADD =3D $(LDADD)
@@ -357,9 +355,6 @@ factor_LDADD +=3D $(LIB_GMP)
 # for getloadavg
 uptime_LDADD +=3D $(GETLOADAVG_LIBS)

-# for crypt
-su_LDADD +=3D $(LIB_CRYPT)
-
 # for various ACL functions
 copy_LDADD +=3D $(LIB_ACL)
 ls_LDADD +=3D $(LIB_ACL)
@@ -402,60 +397,6 @@ RELEASE_YEAR =3D \
   `sed -n '/.*COPYRIGHT_YEAR =3D \([0-9][0-9][0-9][0-9]\) };/s//\1/p' \
     $(top_srcdir)/lib/version-etc.c`

-all-local: su$(EXEEXT)
-
-installed_su =3D $(DESTDIR)$(bindir)/`echo su|sed '$(transform)'`
-
-setuid_root_mode =3D a=3Drx,u+s
-
-install_su =3D						\
-  if test "$(INSTALL_SU)" =3D yes; then			\
-    p=3Dsu;						\
-    echo " $(INSTALL_PROGRAM) $$p $(installed_su)";	\
-    $(INSTALL_PROGRAM) $$p $(installed_su);		\
-    echo " chown root $(installed_su)";			\
-    chown root $(installed_su);				\
-    echo " chmod $(setuid_root_mode) $(installed_su)";	\
-    chmod $(setuid_root_mode) $(installed_su);		\
-  else							\
-    :;							\
-  fi
-
-install-root: su$(EXEEXT)
-	@$(install_su)
-
-install-exec-hook: su$(EXEEXT)
-	@if test "$(INSTALL_SU)" =3D yes; then			\
-	  TMPFILE=3D$(DESTDIR)$(bindir)/.su-$$$$;			\
-	  rm -f $$TMPFILE;					\
-	  echo > $$TMPFILE;					\
-## See if we can create a setuid root executable in $(bindir).
-## If not, then don't even try to install su.
-	  can_create_suid_root_executable=3Dno;			\
-	  chown root $$TMPFILE > /dev/null 2>&1			\
-	    && chmod $(setuid_root_mode) $$TMPFILE > /dev/null 2>&1 \
-	    && can_create_suid_root_executable=3Dyes;		\
-	  rm -f $$TMPFILE;					\
-	  if test $$can_create_suid_root_executable =3D yes; then \
-	    $(install_su);					\
-	  else							\
-	    echo "WARNING: insufficient access; not installing su"; \
-	    echo "NOTE: to install su, run 'make install-root' as root"; \
-	    rm -f $(installed_su);				\
-	  fi;							\
-	else :;							\
-	fi
-
-uninstall-local:
-# Remove su only if it's one we installed.
-	@if test "$(INSTALL_SU)" =3D yes; then			\
-	  if grep '$(PACKAGE_NAME)' $(installed_su) > /dev/null 2>&1; then \
-	    echo "  rm -f $(installed_su)"; \
-	    rm -f $(installed_su); \
-	  else :; \
-	  fi; \
-	fi
-
 copy_sources =3D copy.c cp-hash.c extent-scan.c extent-scan.h

 # Use 'ginstall' in the definition of PROGRAMS and in dependencies to avoid
diff --git a/src/su.c b/src/su.c
deleted file mode 100644
index bb54cc3..0000000
--- a/src/su.c
+++ /dev/null
@@ -1,520 +0,0 @@
-/* su for GNU.  Run a shell with substitute user and group IDs.
-   Copyright (C) 1992-2012 Free Software Foundation, Inc.
-
-   This program is free software: you can redistribute it and/or modify
-   it under the terms of the GNU General Public License as published by
-   the Free Software Foundation, either version 3 of the License, or
-   (at your option) any later version.
-
-   This program is distributed in the hope that it will be useful,
-   but WITHOUT ANY WARRANTY; without even the implied warranty of
-   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-   GNU General Public License for more details.
-
-   You should have received a copy of the GNU General Public License
-   along with this program.  If not, see <http://www.gnu.org/licenses/>.  =
*/
-
-/* Run a shell with the real and effective UID and GID and groups
-   of USER, default 'root'.
-
-   The shell run is taken from USER's password entry, /bin/sh if
-   none is specified there.  If the account has a password, su
-   prompts for a password unless run by a user with real UID 0.
-
-   Does not change the current directory.
-   Sets 'HOME' and 'SHELL' from the password entry for USER, and if
-   USER is not root, sets 'USER' and 'LOGNAME' to USER.
-   The subshell is not a login shell.
-
-   If one or more ARGs are given, they are passed as additional
-   arguments to the subshell.
-
-   Does not handle /bin/sh or other shells specially
-   (setting argv[0] to "-su", passing -c only to certain shells, etc.).
-   I don't see the point in doing that, and it's ugly.
-
-   This program intentionally does not support a "wheel group" that
-   restricts who can su to UID 0 accounts.  RMS considers that to
-   be fascist.
-
-   Compile-time options:
-   -DSYSLOG_SUCCESS	Log successful su's (by default, to root) with syslog.
-   -DSYSLOG_FAILURE	Log failed su's (by default, to root) with syslog.
-
-   -DSYSLOG_NON_ROOT	Log all su's, not just those to root (UID 0).
-   Never logs attempted su's to nonexistent accounts.
-
-   Written by David MacKenzie <djm@gnu.ai.mit.edu>.  */
-
-#include <config.h>
-#include <stdio.h>
-#include <getopt.h>
-#include <sys/types.h>
-#include <pwd.h>
-#include <grp.h>
-
-#include "system.h"
-#include "getpass.h"
-
-#if HAVE_SYSLOG_H && HAVE_SYSLOG
-# include <syslog.h>
-#else
-# undef SYSLOG_SUCCESS
-# undef SYSLOG_FAILURE
-# undef SYSLOG_NON_ROOT
-#endif
-
-#if HAVE_SYS_PARAM_H
-# include <sys/param.h>
-#endif
-
-#ifndef HAVE_ENDGRENT
-# define endgrent() ((void) 0)
-#endif
-
-#ifndef HAVE_ENDPWENT
-# define endpwent() ((void) 0)
-#endif
-
-#if HAVE_SHADOW_H
-# include <shadow.h>
-#endif
-
-#include "error.h"
-
-/* The official name of this program (e.g., no 'g' prefix).  */
-#define PROGRAM_NAME "su"
-
-#define AUTHORS proper_name ("David MacKenzie")
-
-#if HAVE_PATHS_H
-# include <paths.h>
-#endif
-
-/* The default PATH for simulated logins to non-superuser accounts.  */
-#ifdef _PATH_DEFPATH
-# define DEFAULT_LOGIN_PATH _PATH_DEFPATH
-#else
-# define DEFAULT_LOGIN_PATH ":/usr/ucb:/bin:/usr/bin"
-#endif
-
-/* The default PATH for simulated logins to superuser accounts.  */
-#ifdef _PATH_DEFPATH_ROOT
-# define DEFAULT_ROOT_LOGIN_PATH _PATH_DEFPATH_ROOT
-#else
-# define DEFAULT_ROOT_LOGIN_PATH "/usr/ucb:/bin:/usr/bin:/etc"
-#endif
-
-/* The shell to run if none is given in the user's passwd entry.  */
-#define DEFAULT_SHELL "/bin/sh"
-
-/* The user to become if none is specified.  */
-#define DEFAULT_USER "root"
-
-char *crypt (char const *key, char const *salt);
-
-static void run_shell (char const *, char const *, char **, size_t)
-     ATTRIBUTE_NORETURN;
-
-/* If true, pass the '-f' option to the subshell.  */
-static bool fast_startup;
-
-/* If true, simulate a login instead of just starting a shell.  */
-static bool simulate_login;
-
-/* If true, change some environment vars to indicate the user su'd to.  */
-static bool change_environment;
-
-static struct option const longopts[] =3D
-{
-  {"command", required_argument, NULL, 'c'},
-  {"fast", no_argument, NULL, 'f'},
-  {"login", no_argument, NULL, 'l'},
-  {"preserve-environment", no_argument, NULL, 'p'},
-  {"shell", required_argument, NULL, 's'},
-  {GETOPT_HELP_OPTION_DECL},
-  {GETOPT_VERSION_OPTION_DECL},
-  {NULL, 0, NULL, 0}
-};
-
-/* Add NAME=3DVAL to the environment, checking for out of memory errors.  =
*/
-
-static void
-xsetenv (char const *name, char const *val)
-{
-  size_t namelen =3D strlen (name);
-  size_t vallen =3D strlen (val);
-  char *string =3D xmalloc (namelen + 1 + vallen + 1);
-  strcpy (string, name);
-  string[namelen] =3D '=3D';
-  strcpy (string + namelen + 1, val);
-  if (putenv (string) !=3D 0)
-    xalloc_die ();
-}
-
-#if defined SYSLOG_SUCCESS || defined SYSLOG_FAILURE
-/* Log the fact that someone has run su to the user given by PW;
-   if SUCCESSFUL is true, they gave the correct password, etc.  */
-
-static void
-log_su (struct passwd const *pw, bool successful)
-{
-  const char *new_user, *old_user, *tty;
-
-# ifndef SYSLOG_NON_ROOT
-  if (pw->pw_uid)
-    return;
-# endif
-  new_user =3D pw->pw_name;
-  /* The utmp entry (via getlogin) is probably the best way to identify
-     the user, especially if someone su's from a su-shell.  */
-  old_user =3D getlogin ();
-  if (!old_user)
-    {
-      /* getlogin can fail -- usually due to lack of utmp entry.
-         Resort to getpwuid.  */
-      errno =3D 0;
-      uid_t ruid =3D getuid ();
-      uid_t NO_UID =3D -1;
-      struct passwd *pwd =3D (ruid =3D=3D NO_UID && errno ? NULL : getpwui=
d (ruid));
-      old_user =3D (pwd ? pwd->pw_name : "");
-    }
-  tty =3D ttyname (STDERR_FILENO);
-  if (!tty)
-    tty =3D "none";
-  /* 4.2BSD openlog doesn't have the third parameter.  */
-  openlog (last_component (program_name), 0
-# ifdef LOG_AUTH
-           , LOG_AUTH
-# endif
-           );
-  syslog (LOG_NOTICE,
-# ifdef SYSLOG_NON_ROOT
-          "%s(to %s) %s on %s",
-# else
-          "%s%s on %s",
-# endif
-          successful ? "" : "FAILED SU ",
-# ifdef SYSLOG_NON_ROOT
-          new_user,
-# endif
-          old_user, tty);
-  closelog ();
-}
-#endif
-
-/* Ask the user for a password.
-   Return true if the user gives the correct password for entry PW,
-   false if not.  Return true without asking for a password if run by UID 0
-   or if PW has an empty password.  */
-
-static bool
-correct_password (const struct passwd *pw)
-{
-  char *unencrypted, *encrypted, *correct;
-#if HAVE_GETSPNAM && HAVE_STRUCT_SPWD_SP_PWDP
-  /* Shadow passwd stuff for SVR3 and maybe other systems.  */
-  struct spwd *sp =3D getspnam (pw->pw_name);
-
-  endspent ();
-  if (sp)
-    correct =3D sp->sp_pwdp;
-  else
-#endif
-    correct =3D pw->pw_passwd;
-
-  if (getuid () =3D=3D 0 || !correct || correct[0] =3D=3D '\0')
-    return true;
-
-  unencrypted =3D getpass (_("Password:"));
-  if (!unencrypted)
-    {
-      error (0, 0, _("getpass: cannot open /dev/tty"));
-      return false;
-    }
-  encrypted =3D crypt (unencrypted, correct);
-  memset (unencrypted, 0, strlen (unencrypted));
-  return STREQ (encrypted, correct);
-}
-
-/* Update 'environ' for the new shell based on PW, with SHELL being
-   the value for the SHELL environment variable.  */
-
-static void
-modify_environment (const struct passwd *pw, const char *shell)
-{
-  if (simulate_login)
-    {
-      /* Leave TERM unchanged.  Set HOME, SHELL, USER, LOGNAME, PATH.
-         Unset all other environment variables.  */
-      char const *term =3D getenv ("TERM");
-      if (term)
-        term =3D xstrdup (term);
-      environ =3D xmalloc ((6 + !!term) * sizeof (char *));
-      environ[0] =3D NULL;
-      if (term)
-        xsetenv ("TERM", term);
-      xsetenv ("HOME", pw->pw_dir);
-      xsetenv ("SHELL", shell);
-      xsetenv ("USER", pw->pw_name);
-      xsetenv ("LOGNAME", pw->pw_name);
-      xsetenv ("PATH", (pw->pw_uid
-                        ? DEFAULT_LOGIN_PATH
-                        : DEFAULT_ROOT_LOGIN_PATH));
-    }
-  else
-    {
-      /* Set HOME, SHELL, and if not becoming a super-user,
-         USER and LOGNAME.  */
-      if (change_environment)
-        {
-          xsetenv ("HOME", pw->pw_dir);
-          xsetenv ("SHELL", shell);
-          if (pw->pw_uid)
-            {
-              xsetenv ("USER", pw->pw_name);
-              xsetenv ("LOGNAME", pw->pw_name);
-            }
-        }
-    }
-}
-
-/* Become the user and group(s) specified by PW.  */
-
-static void
-change_identity (const struct passwd *pw)
-{
-#ifdef HAVE_INITGROUPS
-  errno =3D 0;
-  if (initgroups (pw->pw_name, pw->pw_gid) =3D=3D -1)
-    error (EXIT_CANCELED, errno, _("cannot set groups"));
-  endgrent ();
-#endif
-  if (setgid (pw->pw_gid))
-    error (EXIT_CANCELED, errno, _("cannot set group id"));
-  if (setuid (pw->pw_uid))
-    error (EXIT_CANCELED, errno, _("cannot set user id"));
-}
-
-/* Run SHELL, or DEFAULT_SHELL if SHELL is empty.
-   If COMMAND is nonzero, pass it to the shell with the -c option.
-   Pass ADDITIONAL_ARGS to the shell as more arguments; there
-   are N_ADDITIONAL_ARGS extra arguments.  */
-
-static void
-run_shell (char const *shell, char const *command, char **additional_args,
-           size_t n_additional_args)
-{
-  size_t n_args =3D 1 + fast_startup + 2 * !!command + n_additional_args +=
 1;
-  char const **args =3D xnmalloc (n_args, sizeof *args);
-  size_t argno =3D 1;
-
-  if (simulate_login)
-    {
-      char *arg0;
-      char *shell_basename;
-
-      shell_basename =3D last_component (shell);
-      arg0 =3D xmalloc (strlen (shell_basename) + 2);
-      arg0[0] =3D '-';
-      strcpy (arg0 + 1, shell_basename);
-      args[0] =3D arg0;
-    }
-  else
-    args[0] =3D last_component (shell);
-  if (fast_startup)
-    args[argno++] =3D "-f";
-  if (command)
-    {
-      args[argno++] =3D "-c";
-      args[argno++] =3D command;
-    }
-  memcpy (args + argno, additional_args, n_additional_args * sizeof *args);
-  args[argno + n_additional_args] =3D NULL;
-  execv (shell, (char **) args);
-
-  {
-    int exit_status =3D (errno =3D=3D ENOENT ? EXIT_ENOENT : EXIT_CANNOT_I=
NVOKE);
-    error (0, errno, "%s", shell);
-    exit (exit_status);
-  }
-}
-
-/* Return true if SHELL is a restricted shell (one not returned by
-   getusershell), else false, meaning it is a standard shell.  */
-
-static bool
-restricted_shell (const char *shell)
-{
-  char *line;
-
-  setusershell ();
-  while ((line =3D getusershell ()) !=3D NULL)
-    {
-      if (*line !=3D '#' && STREQ (line, shell))
-        {
-          endusershell ();
-          return false;
-        }
-    }
-  endusershell ();
-  return true;
-}
-
-void
-usage (int status)
-{
-  if (status !=3D EXIT_SUCCESS)
-    emit_try_help ();
-  else
-    {
-      printf (_("Usage: %s [OPTION]... [-] [USER [ARG]...]\n"), program_na=
me);
-      fputs (_("\
-Change the effective user id and group id to that of USER.\n\
-\n\
-  -, -l, --login               make the shell a login shell\n\
-  -c, --command=3DCOMMAND        pass a single COMMAND to the shell with -=
c\n\
-  -f, --fast                   pass -f to the shell (for csh or tcsh)\n\
-  -m, --preserve-environment   do not reset environment variables\n\
-  -p                           same as -m\n\
-  -s, --shell=3DSHELL            run SHELL if /etc/shells allows it\n\
-"), stdout);
-      fputs (HELP_OPTION_DESCRIPTION, stdout);
-      fputs (VERSION_OPTION_DESCRIPTION, stdout);
-      fputs (_("\
-\n\
-A mere - implies -l.   If USER not given, assume root.\n\
-"), stdout);
-      emit_ancillary_info ();
-    }
-  exit (status);
-}
-
-int
-main (int argc, char **argv)
-{
-  int optc;
-  const char *new_user =3D DEFAULT_USER;
-  char *command =3D NULL;
-  char *shell =3D NULL;
-  struct passwd *pw;
-  struct passwd pw_copy;
-
-  initialize_main (&argc, &argv);
-  set_program_name (argv[0]);
-  setlocale (LC_ALL, "");
-  bindtextdomain (PACKAGE, LOCALEDIR);
-  textdomain (PACKAGE);
-
-  initialize_exit_failure (EXIT_CANCELED);
-  atexit (close_stdout);
-
-  fast_startup =3D false;
-  simulate_login =3D false;
-  change_environment =3D true;
-
-  while ((optc =3D getopt_long (argc, argv, "c:flmps:", longopts, NULL)) !=
=3D -1)
-    {
-      switch (optc)
-        {
-        case 'c':
-          command =3D optarg;
-          break;
-
-        case 'f':
-          fast_startup =3D true;
-          break;
-
-        case 'l':
-          simulate_login =3D true;
-          break;
-
-        case 'm':
-        case 'p':
-          change_environment =3D false;
-          break;
-
-        case 's':
-          shell =3D optarg;
-          break;
-
-        case_GETOPT_HELP_CHAR;
-
-        case_GETOPT_VERSION_CHAR (PROGRAM_NAME, AUTHORS);
-
-        default:
-          usage (EXIT_CANCELED);
-        }
-    }
-
-  if (optind < argc && STREQ (argv[optind], "-"))
-    {
-      simulate_login =3D true;
-      ++optind;
-    }
-  if (optind < argc)
-    new_user =3D argv[optind++];
-
-  pw =3D getpwnam (new_user);
-  if (! (pw && pw->pw_name && pw->pw_name[0] && pw->pw_dir && pw->pw_dir[0]
-         && pw->pw_passwd))
-    error (EXIT_CANCELED, 0, _("user %s does not exist"), new_user);
-
-  /* Make a copy of the password information and point pw at the local
-     copy instead.  Otherwise, some systems (e.g. GNU/Linux) would clobber
-     the static data through the getlogin call from log_su.
-     Also, make sure pw->pw_shell is a nonempty string.
-     It may be NULL when NEW_USER is a username that is retrieved via NIS =
(YP),
-     but that doesn't have a default shell listed.  */
-  pw_copy =3D *pw;
-  pw =3D &pw_copy;
-  pw->pw_name =3D xstrdup (pw->pw_name);
-  pw->pw_passwd =3D xstrdup (pw->pw_passwd);
-  pw->pw_dir =3D xstrdup (pw->pw_dir);
-  pw->pw_shell =3D xstrdup (pw->pw_shell && pw->pw_shell[0]
-                          ? pw->pw_shell
-                          : DEFAULT_SHELL);
-  endpwent ();
-
-  if (!correct_password (pw))
-    {
-#ifdef SYSLOG_FAILURE
-      log_su (pw, false);
-#endif
-      error (EXIT_CANCELED, 0, _("incorrect password"));
-    }
-#ifdef SYSLOG_SUCCESS
-  else
-    {
-      log_su (pw, true);
-    }
-#endif
-
-  if (!shell && !change_environment)
-    shell =3D getenv ("SHELL");
-  if (shell && getuid () !=3D 0 && restricted_shell (pw->pw_shell))
-    {
-      /* The user being su'd to has a nonstandard shell, and so is
-         probably a uucp account or has restricted access.  Don't
-         compromise the account by allowing access with a standard
-         shell.  */
-      error (0, 0, _("using restricted shell %s"), pw->pw_shell);
-      shell =3D NULL;
-    }
-  shell =3D xstrdup (shell ? shell : pw->pw_shell);
-  modify_environment (pw, shell);
-
-  change_identity (pw);
-  if (simulate_login && chdir (pw->pw_dir) !=3D 0)
-    error (0, errno, _("warning: cannot change directory to %s"), pw->pw_d=
ir);
-
-  /* error() flushes stderr, but does not check for write failure.
-     Normally, we would catch this via our atexit() hook of
-     close_stdout, but execv() gets in the way.  If stderr
-     encountered a write failure, there is no need to try calling
-     error() again.  */
-  if (ferror (stderr))
-    exit (EXIT_CANCELED);
-
-  run_shell (shell, command, argv + optind, MAX (0, argc - optind));
-}
diff --git a/tests/Makefile.am b/tests/Makefile.am
index 0bafc5f..d8bc930 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -280,7 +280,6 @@ TESTS =3D						\
   misc/stty-invalid				\
   misc/stty-pairs				\
   misc/stty-row-col				\
-  misc/su-fail					\
   misc/sum					\
   misc/sum-sysv					\
   misc/tac					\
diff --git a/tests/misc/help-version b/tests/misc/help-version
index 7416062..e19f5b1 100755
--- a/tests/misc/help-version
+++ b/tests/misc/help-version
@@ -208,7 +208,6 @@ nohup_setup () { args=3D--version; }
 printf_setup () { args=3Dfoo; }
 seq_setup () { args=3D10; }
 sleep_setup () { args=3D0; }
-su_setup () { args=3D--version; }
 stdbuf_setup () { args=3D"-oL true"; }
 timeout_setup () { args=3D--version; }

diff --git a/tests/misc/invalid-opt b/tests/misc/invalid-opt
index 8eae3c2..8f7024c 100755
--- a/tests/misc/invalid-opt
+++ b/tests/misc/invalid-opt
@@ -36,7 +36,6 @@ my %exit_status =3D
     nohup =3D> 125,
     sort =3D> 2,
     stdbuf =3D> 125,
-    su =3D> 125,
     test =3D> 0,
     timeout =3D> 125,
     true =3D> 0,
diff --git a/tests/misc/su-fail b/tests/misc/su-fail
deleted file mode 100755
index c2e0475..0000000
--- a/tests/misc/su-fail
+++ /dev/null
@@ -1,30 +0,0 @@
-#!/bin/sh
-# Test su failure cases
-
-# Copyright (C) 2009-2012 Free Software Foundation, Inc.
-
-# This program is free software: you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation, either version 3 of the License, or
-# (at your option) any later version.
-
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU General Public License for more details.
-
-# You should have received a copy of the GNU General Public License
-# along with this program.  If not, see <http://www.gnu.org/licenses/>.
-
-. "${srcdir=3D.}/init.sh"; path_prepend_ ../src
-print_ver_ su
-
-require_built_ su
-
-# Very little that we can test without a root password
-su --- / true # unknown option
-test $? =3D 125 || fail=3D1
-su no_such_user
-test $? =3D 125 || fail=3D1
-
-Exit $fail
--
1.7.11.rc1


From MAILER-DAEMON Wed Jun 06 16:34:24 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1ScMvw-0004CQ-BT
	for mharc-coreutils@gnu.org; Wed, 06 Jun 2012 16:34:24 -0400
Received: from eggs.gnu.org ([208.118.235.92]:55561)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1ScMvt-0004Ak-Og
	for coreutils@gnu.org; Wed, 06 Jun 2012 16:34:22 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1ScMvr-0005iQ-TT
	for coreutils@gnu.org; Wed, 06 Jun 2012 16:34:21 -0400
Received: from mx.meyering.net ([88.168.87.75]:35875)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1ScMvr-0005iD-LN
	for coreutils@gnu.org; Wed, 06 Jun 2012 16:34:19 -0400
Received: from rho.meyering.net (localhost.localdomain [127.0.0.1])
	by rho.meyering.net (Acme Bit-Twister) with ESMTP id 184E660051
	for <coreutils@gnu.org>; Wed,  6 Jun 2012 22:34:17 +0200 (CEST)
From: Jim Meyering <jim@meyering.net>
To: coreutils@gnu.org
Subject: [PATCH] maint: detect a new type of duplicate in THANKS.in
Date: Wed, 06 Jun 2012 22:34:17 +0200
Message-ID: <878vg018om.fsf@rho.meyering.net>
Lines: 38
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 88.168.87.75
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Wed, 06 Jun 2012 20:34:23 -0000

FYI,

>From 7b6ce030cc54b34c7fc8a87a0c89137fb5c78a50 Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering@redhat.com>
Date: Wed, 6 Jun 2012 12:46:27 +0200
Subject: [PATCH] maint: detect a new type of duplicate in THANKS.in

* cfg.mk (sc_THANKS_in_duplicates): New rule.
---
 cfg.mk | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/cfg.mk b/cfg.mk
index 7a86c2a..040d7e1 100644
--- a/cfg.mk
+++ b/cfg.mk
@@ -413,6 +413,19 @@ sc_preprocessor_indentation:
 	  echo '$(ME): skipping test $@: cppi not installed' 1>&2;	\
 	fi

+# THANKS.in is a list of name/email pairs for people who are mentioned in
+# commit logs (and generated ChangeLog), but who are not also listed as an
+# author of a commit.  Name/email pairs of commit authors are automatically
+# extracted from the repository.  As a very minor factorization, when
+# someone who was initially listed only in THANKS.in later authors a commit,
+# this rule detects that their pair may now be removed from THANKS.in.
+sc_THANKS_in_duplicates:
+	{ git log --pretty=format:%aN | sort -u;			\
+	    cut -b-36 THANKS.in | sed '/^$$/d;s/  *$$//'; }		\
+	  | sort | uniq -d | grep .					\
+	    && { echo '$(ME): remove the above names from THANKS.in'	\
+		  1>&2; exit 1; } || :
+
 # Override the default Cc: used in generating an announcement.
 announcement_Cc_ = $(translation_project_), \
   coreutils@gnu.org, coreutils-announce@gnu.org
--
1.7.11.rc1


From MAILER-DAEMON Wed Jun 06 16:37:26 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1ScMys-0004Q1-M4
	for mharc-coreutils@gnu.org; Wed, 06 Jun 2012 16:37:26 -0400
Received: from eggs.gnu.org ([208.118.235.92]:33537)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1ScMyl-0004Nb-KK
	for coreutils@gnu.org; Wed, 06 Jun 2012 16:37:25 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1ScMyj-0006X9-AZ
	for coreutils@gnu.org; Wed, 06 Jun 2012 16:37:19 -0400
Received: from mx.meyering.net ([88.168.87.75]:35886)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1ScMyj-0006Wl-3V
	for coreutils@gnu.org; Wed, 06 Jun 2012 16:37:17 -0400
Received: from rho.meyering.net (localhost.localdomain [127.0.0.1])
	by rho.meyering.net (Acme Bit-Twister) with ESMTP id 998B2602D3
	for <coreutils@gnu.org>; Wed,  6 Jun 2012 22:37:15 +0200 (CEST)
From: Jim Meyering <jim@meyering.net>
To: coreutils@gnu.org
Subject: Re: su, going, going, ...
In-Reply-To: <8762b55w3m.fsf@rho.meyering.net> (Jim Meyering's message of
	"Tue, 05 Jun 2012 22:42:37 +0200")
References: <8762b55w3m.fsf@rho.meyering.net>
Date: Wed, 06 Jun 2012 22:37:15 +0200
Message-ID: <87396818jo.fsf@rho.meyering.net>
Lines: 29
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 88.168.87.75
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Wed, 06 Jun 2012 20:37:25 -0000

Jim Meyering wrote:
> No one objected, so I'm about to remove su, with this:
> (I'll wait 10-12 hours, just in case ;-)
>
>>From db2c12039e5b008245b33778d28e6d9ab1156844 Mon Sep 17 00:00:00 2001
> From: Jim Meyering <meyering@redhat.com>
> Date: Fri, 25 May 2012 18:10:25 +0200
> Subject: [PATCH] su: remove program (util-linux is now the best source for
>  it)
>
> * README: Omit "su" from list of programs.
> * src/su.c: Remove file.
> * src/Makefile.am: Remove su-related rules and variables.
> * tests/misc/su-fail: Remove test.
> * tests/Makefile.am (TESTS): Remove misc/su-fail.
> * tests/misc/invalid-opt: Remove su-related code.
> * src/.gitignore: Remove su.
> * man/su.x: Remove file.
> * man/Makefile.am (su.1): Remove rule.
> * po/POTFILES.in: Remove su.c from the list.
> * TODO: Remove ancient entry.
> * NEWS (Changes in behavior): Mention it.
> * doc/coreutils.texi: Remove su-related description.
> * AUTHORS: Remove su.
> * m4/lib-check.m4 (cu_LIB_CHECK): Remove file/macro.
> * configure.ac: Remove su-related code and sole use of cu_LIB_CHECK.
> * scripts/git-hooks/commit-msg: Remove su from this list, too.

Pushed, finally.


From MAILER-DAEMON Wed Jun 06 18:34:45 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1ScOoP-00038D-NC
	for mharc-coreutils@gnu.org; Wed, 06 Jun 2012 18:34:45 -0400
Received: from eggs.gnu.org ([208.118.235.92]:50276)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <allan@archlinux.org>) id 1ScOoM-000381-JC
	for coreutils@gnu.org; Wed, 06 Jun 2012 18:34:43 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <allan@archlinux.org>) id 1ScOoK-0005X5-Mz
	for coreutils@gnu.org; Wed, 06 Jun 2012 18:34:42 -0400
Received: from gerolde.archlinux.org ([66.211.214.132]:60246)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <allan@archlinux.org>) id 1ScOoK-0005Wo-In
	for coreutils@gnu.org; Wed, 06 Jun 2012 18:34:40 -0400
Received: from [10.1.1.2] (122-148-115-77.static.dsl.dodo.com.au
	[122.148.115.77])
	by gerolde.archlinux.org (Postfix) with ESMTPSA id 01D699018A;
	Wed,  6 Jun 2012 18:34:02 -0400 (EDT)
Message-ID: <4FCFDAF8.6020508@archlinux.org>
Date: Thu, 07 Jun 2012 08:34:32 +1000
From: Allan McRae <allan@archlinux.org>
User-Agent: Mozilla/5.0 (X11; Linux i686;
	rv:13.0) Gecko/20120605 Thunderbird/13.0
MIME-Version: 1.0
To: Jim Meyering <jim@meyering.net>
Subject: Re: su, going, going, ...
References: <8762b55w3m.fsf@rho.meyering.net> <87396818jo.fsf@rho.meyering.net>
In-Reply-To: <87396818jo.fsf@rho.meyering.net>
X-Enigmail-Version: 1.4.2
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 66.211.214.132
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Wed, 06 Jun 2012 22:34:44 -0000

On 07/06/12 06:37, Jim Meyering wrote:
> Jim Meyering wrote:
>> No one objected, so I'm about to remove su, with this:
>> (I'll wait 10-12 hours, just in case ;-)
>>
>> >From db2c12039e5b008245b33778d28e6d9ab1156844 Mon Sep 17 00:00:00 2001
>> From: Jim Meyering <meyering@redhat.com>
>> Date: Fri, 25 May 2012 18:10:25 +0200
>> Subject: [PATCH] su: remove program (util-linux is now the best source for
>>  it)
>>
>> * README: Omit "su" from list of programs.
>> * src/su.c: Remove file.
>> * src/Makefile.am: Remove su-related rules and variables.
>> * tests/misc/su-fail: Remove test.
>> * tests/Makefile.am (TESTS): Remove misc/su-fail.
>> * tests/misc/invalid-opt: Remove su-related code.
>> * src/.gitignore: Remove su.
>> * man/su.x: Remove file.
>> * man/Makefile.am (su.1): Remove rule.
>> * po/POTFILES.in: Remove su.c from the list.
>> * TODO: Remove ancient entry.
>> * NEWS (Changes in behavior): Mention it.
>> * doc/coreutils.texi: Remove su-related description.
>> * AUTHORS: Remove su.
>> * m4/lib-check.m4 (cu_LIB_CHECK): Remove file/macro.
>> * configure.ac: Remove su-related code and sole use of cu_LIB_CHECK.
>> * scripts/git-hooks/commit-msg: Remove su from this list, too.
> 
> Pushed, finally.
> 

Good to see, but....  It is my understanding that there is not a
released version of util-linux with su in it, and I do not see details
of a release on the horizon.

Is there going to be a hold on coreutils releases until an appropriate
version of util-linux is released, or could we end up with the latest
release of both packages not providing su?

Allan



From MAILER-DAEMON Thu Jun 07 05:48:28 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1ScZKO-0005Qk-UM
	for mharc-coreutils@gnu.org; Thu, 07 Jun 2012 05:48:28 -0400
Received: from eggs.gnu.org ([208.118.235.92]:56153)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1ScZKL-0005QW-W1
	for coreutils@gnu.org; Thu, 07 Jun 2012 05:48:27 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1ScZKK-0007PN-30
	for coreutils@gnu.org; Thu, 07 Jun 2012 05:48:25 -0400
Received: from mx.meyering.net ([88.168.87.75]:37993)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1ScZKJ-0007PG-RI
	for coreutils@gnu.org; Thu, 07 Jun 2012 05:48:24 -0400
Received: from rho.meyering.net (localhost.localdomain [127.0.0.1])
	by rho.meyering.net (Acme Bit-Twister) with ESMTP id EC510600B1;
	Thu,  7 Jun 2012 11:48:21 +0200 (CEST)
From: Jim Meyering <jim@meyering.net>
To: Allan McRae <allan@archlinux.org>
Subject: Re: su, going, going, ...
In-Reply-To: <4FCFDAF8.6020508@archlinux.org> (Allan McRae's message of "Thu, 
	07 Jun 2012 08:34:32 +1000")
References: <8762b55w3m.fsf@rho.meyering.net>
	<87396818jo.fsf@rho.meyering.net> <4FCFDAF8.6020508@archlinux.org>
Date: Thu, 07 Jun 2012 11:48:21 +0200
Message-ID: <87vcj3wize.fsf@rho.meyering.net>
Lines: 44
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 88.168.87.75
Cc: List util-linux-ng <util-linux-ng@vger.kernel.org>, coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 07 Jun 2012 09:48:27 -0000

Allan McRae wrote:
> On 07/06/12 06:37, Jim Meyering wrote:
>> Jim Meyering wrote:
>>> No one objected, so I'm about to remove su, with this:
>>> (I'll wait 10-12 hours, just in case ;-)
>>>
>>> >From db2c12039e5b008245b33778d28e6d9ab1156844 Mon Sep 17 00:00:00 2001
>>> From: Jim Meyering <meyering@redhat.com>
>>> Date: Fri, 25 May 2012 18:10:25 +0200
>>> Subject: [PATCH] su: remove program (util-linux is now the best source for
>>>  it)
>>>
>>> * README: Omit "su" from list of programs.
>>> * src/su.c: Remove file.
>>> * src/Makefile.am: Remove su-related rules and variables.
>>> * tests/misc/su-fail: Remove test.
>>> * tests/Makefile.am (TESTS): Remove misc/su-fail.
>>> * tests/misc/invalid-opt: Remove su-related code.
>>> * src/.gitignore: Remove su.
>>> * man/su.x: Remove file.
>>> * man/Makefile.am (su.1): Remove rule.
>>> * po/POTFILES.in: Remove su.c from the list.
>>> * TODO: Remove ancient entry.
>>> * NEWS (Changes in behavior): Mention it.
>>> * doc/coreutils.texi: Remove su-related description.
>>> * AUTHORS: Remove su.
>>> * m4/lib-check.m4 (cu_LIB_CHECK): Remove file/macro.
>>> * configure.ac: Remove su-related code and sole use of cu_LIB_CHECK.
>>> * scripts/git-hooks/commit-msg: Remove su from this list, too.
>>
>> Pushed, finally.
>>
>
> Good to see, but....  It is my understanding that there is not a
> released version of util-linux with su in it, and I do not see details
> of a release on the horizon.
>
> Is there going to be a hold on coreutils releases until an appropriate
> version of util-linux is released, or could we end up with the latest
> release of both packages not providing su?

I hadn't realized that -- nor that it might cause a problem,
as your message implies.  I've Cc'd the util-linux list.
Let's see if/when the next release is planned.


From MAILER-DAEMON Thu Jun 07 10:48:10 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1Sce0Q-0002C5-8q
	for mharc-coreutils@gnu.org; Thu, 07 Jun 2012 10:48:10 -0400
Received: from eggs.gnu.org ([208.118.235.92]:51265)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <kzak@redhat.com>) id 1Sce0I-0002Bt-EN
	for coreutils@gnu.org; Thu, 07 Jun 2012 10:48:08 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <kzak@redhat.com>) id 1Sce0C-0002zi-9i
	for coreutils@gnu.org; Thu, 07 Jun 2012 10:48:02 -0400
Received: from mx1.redhat.com ([209.132.183.28]:49048)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <kzak@redhat.com>) id 1Sce0C-0002zW-1Y
	for coreutils@gnu.org; Thu, 07 Jun 2012 10:47:56 -0400
Received: from int-mx01.intmail.prod.int.phx2.redhat.com
	(int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id q57Elo4Z004575
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK);
	Thu, 7 Jun 2012 10:47:50 -0400
Received: from x2 (dhcp-25-158.brq.redhat.com [10.34.25.158])
	by int-mx01.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP
	id q57Ellwt025648; Thu, 7 Jun 2012 10:47:49 -0400
Date: Thu, 7 Jun 2012 16:47:47 +0200
From: Karel Zak <kzak@redhat.com>
To: Allan McRae <allan@archlinux.org>
Subject: Re: su, going, going, ...
Message-ID: <20120607144747.GC28227@x2.net.home>
References: <8762b55w3m.fsf@rho.meyering.net> <87396818jo.fsf@rho.meyering.net>
	<4FCFDAF8.6020508@archlinux.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <4FCFDAF8.6020508@archlinux.org>
User-Agent: Mutt/1.5.21 (2011-07-01)
X-Scanned-By: MIMEDefang 2.67 on 10.5.11.11
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.132.183.28
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 07 Jun 2012 14:48:08 -0000

On Thu, Jun 07, 2012 at 08:34:32AM +1000, Allan McRae wrote:
> Good to see, but....  It is my understanding that there is not a
> released version of util-linux with su in it, and I do not see details
> of a release on the horizon.

 We usually release util-linux twice per year, the latest release
 2.21 was in February, and I'd like to have release 2.22 in July.

 If there will be a real problem with this schedule than we can
 release util-linux 2.21.3 bugfix release with backported su(1).

    Karel

-- 
 Karel Zak  <kzak@redhat.com>
 http://karelzak.blogspot.com


From MAILER-DAEMON Thu Jun 07 10:54:20 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1Sce6O-0005x0-Uy
	for mharc-coreutils@gnu.org; Thu, 07 Jun 2012 10:54:20 -0400
Received: from eggs.gnu.org ([208.118.235.92]:38951)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Sce6H-0005vo-8G
	for coreutils@gnu.org; Thu, 07 Jun 2012 10:54:19 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Sce6C-0004OH-Cv
	for coreutils@gnu.org; Thu, 07 Jun 2012 10:54:12 -0400
Received: from mx.meyering.net ([88.168.87.75]:38803)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Sce6C-0004O0-49
	for coreutils@gnu.org; Thu, 07 Jun 2012 10:54:08 -0400
Received: from rho.meyering.net (localhost.localdomain [127.0.0.1])
	by rho.meyering.net (Acme Bit-Twister) with ESMTP id 8B8AA601C9;
	Thu,  7 Jun 2012 16:54:06 +0200 (CEST)
From: Jim Meyering <jim@meyering.net>
To: Karel Zak <kzak@redhat.com>
Subject: Re: su, going, going, ...
In-Reply-To: <20120607144747.GC28227@x2.net.home> (Karel Zak's message of
	"Thu, 7 Jun 2012 16:47:47 +0200")
References: <8762b55w3m.fsf@rho.meyering.net>
	<87396818jo.fsf@rho.meyering.net> <4FCFDAF8.6020508@archlinux.org>
	<20120607144747.GC28227@x2.net.home>
Date: Thu, 07 Jun 2012 16:54:06 +0200
Message-ID: <87pq9buq9d.fsf@rho.meyering.net>
Lines: 20
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 88.168.87.75
Cc: Allan McRae <allan@archlinux.org>, coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 07 Jun 2012 14:54:19 -0000

Karel Zak wrote:
> On Thu, Jun 07, 2012 at 08:34:32AM +1000, Allan McRae wrote:
>> Good to see, but....  It is my understanding that there is not a
>> released version of util-linux with su in it, and I do not see details
>> of a release on the horizon.
>
>  We usually release util-linux twice per year, the latest release
>  2.21 was in February, and I'd like to have release 2.22 in July.
>
>  If there will be a real problem with this schedule than we can
>  release util-linux 2.21.3 bugfix release with backported su(1).

Thanks, Karel.
That sounds perfect.
While coreutils' "Bug fixes" list is longer than I'd like,

    http://git.sv.gnu.org/cgit/coreutils.git/tree/NEWS

currently I see no pressing need to make a release,
so we should be comfortable releasing after util-linux-2.22.


From MAILER-DAEMON Thu Jun 07 11:01:49 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1SceDd-0000tK-OP
	for mharc-coreutils@gnu.org; Thu, 07 Jun 2012 11:01:49 -0400
Received: from eggs.gnu.org ([208.118.235.92]:54225)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <kzak@redhat.com>) id 1SceDX-0000sS-OL
	for coreutils@gnu.org; Thu, 07 Jun 2012 11:01:48 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <kzak@redhat.com>) id 1SceDS-0006Og-QQ
	for coreutils@gnu.org; Thu, 07 Jun 2012 11:01:43 -0400
Received: from mx1.redhat.com ([209.132.183.28]:27338)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <kzak@redhat.com>) id 1SceDS-0006MW-IX
	for coreutils@gnu.org; Thu, 07 Jun 2012 11:01:38 -0400
Received: from int-mx12.intmail.prod.int.phx2.redhat.com
	(int-mx12.intmail.prod.int.phx2.redhat.com [10.5.11.25])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id q57F1YZw009857
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK);
	Thu, 7 Jun 2012 11:01:34 -0400
Received: from x2 (dhcp-25-158.brq.redhat.com [10.34.25.158])
	by int-mx12.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP
	id q57F1XL0008176; Thu, 7 Jun 2012 11:01:34 -0400
Date: Thu, 7 Jun 2012 17:01:33 +0200
From: Karel Zak <kzak@redhat.com>
To: Jim Meyering <jim@meyering.net>
Subject: Re: su, going, going, ...
Message-ID: <20120607150133.GD28227@x2.net.home>
References: <8762b55w3m.fsf@rho.meyering.net> <87396818jo.fsf@rho.meyering.net>
	<4FCFDAF8.6020508@archlinux.org> <87vcj3wize.fsf@rho.meyering.net>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <87vcj3wize.fsf@rho.meyering.net>
User-Agent: Mutt/1.5.21 (2011-07-01)
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.25
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.132.183.28
Cc: Allan McRae <allan@archlinux.org>,
	List util-linux-ng <util-linux-ng@vger.kernel.org>, coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 07 Jun 2012 15:01:48 -0000

On Thu, Jun 07, 2012 at 11:48:21AM +0200, Jim Meyering wrote:
> > Is there going to be a hold on coreutils releases until an appropriate
> > version of util-linux is released, or could we end up with the latest
> > release of both packages not providing su?
> 
> I hadn't realized that -- nor that it might cause a problem,
> as your message implies.  I've Cc'd the util-linux list.

Ah, I'm subscribed to the coreutils list where is my reply :-)
http://thread.gmane.org/gmane.comp.gnu.coreutils.general/2757/focus=2762

-- 
 Karel Zak  <kzak@redhat.com>
 http://karelzak.blogspot.com


From MAILER-DAEMON Thu Jun 07 11:56:14 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1Scf4I-0000yd-OZ
	for mharc-coreutils@gnu.org; Thu, 07 Jun 2012 11:56:14 -0400
Received: from eggs.gnu.org ([208.118.235.92]:60264)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jannhorn@googlemail.com>) id 1Scf4G-0000yT-Hv
	for coreutils@gnu.org; Thu, 07 Jun 2012 11:56:13 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jannhorn@googlemail.com>) id 1Scf4C-0001aa-SY
	for coreutils@gnu.org; Thu, 07 Jun 2012 11:56:12 -0400
Received: from mail-bk0-f41.google.com ([209.85.214.41]:56816)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jannhorn@googlemail.com>) id 1Scf4C-0001Wa-IS
	for coreutils@gnu.org; Thu, 07 Jun 2012 11:56:08 -0400
Received: by bkcjm19 with SMTP id jm19so835564bkc.0
	for <coreutils@gnu.org>; Thu, 07 Jun 2012 08:56:05 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
	d=googlemail.com; s=20120113;
	h=date:from:to:subject:message-id:references:mime-version
	:content-type:content-disposition:in-reply-to:user-agent;
	bh=AYb/oiCdUPXxCUh4R+28/W1GysTFafSm1SDp38Lji3g=;
	b=blOmwh9zXiXMywEisSAgtlnYReX+AcH5d3BJX5In5YUfXv77kBWH9YIoYyXgCMIS0n
	eQOFxrQoCkIELMl1tJfqp9h5rvQqiIg0sU+nY5JujO+IUwFJbK2VeFvW6ECAkxwyqGLs
	8pUpyMras8LKLHe4nWsrRLZfIFASBtr1g9SwKoYthY9n3VT19LazOw55QrhylLQc4IJK
	r+0D9XgCLw+6so2khgCBK3Z0E8OhEZKrqRZ8LI2aU+dgtEzOm4QZTpTSB53UevyBmbkT
	UqmZSX/1ZbzyI0G1PmoUMaHEaAkvR8/v1Vh5uHpDY9zXohMaHmJqZyWgVgih4v2vW7/w
	8nBQ==
Received: by 10.204.152.199 with SMTP id h7mr2103545bkw.39.1339084565327;
	Thu, 07 Jun 2012 08:56:05 -0700 (PDT)
Received: from Jann-PC.fritz.box (77-23-225-249-dynip.superkabel.de.
	[77.23.225.249])
	by mx.google.com with ESMTPS id x23sm4402375bkw.12.2012.06.07.08.56.03
	(version=SSLv3 cipher=OTHER); Thu, 07 Jun 2012 08:56:04 -0700 (PDT)
Date: Thu, 7 Jun 2012 17:56:01 +0200
From: Jann Horn <jannhorn@googlemail.com>
To: coreutils@gnu.org
Subject: Re: feature proposal for tee: exit immediately when the output pipe
	closes
Message-ID: <20120607155601.GA32204@Jann-PC.fritz.box>
References: <20120518145838.GA31109@Jann-PC.fritz.box>
	<20120518190018.GB31109@Jann-PC.fritz.box>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha1;
	protocol="application/pgp-signature"; boundary="OgqxwSJOaUobr8KG"
Content-Disposition: inline
In-Reply-To: <20120518190018.GB31109@Jann-PC.fritz.box>
User-Agent: Mutt/1.5.21 (2010-09-15)
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.85.214.41
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 07 Jun 2012 15:56:13 -0000


--OgqxwSJOaUobr8KG
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Fri, May 18, 2012 at 09:00:18PM +0200, Jann Horn wrote:
> On Fri, May 18, 2012 at 04:58:38PM +0200, Jann Horn wrote:
> > Hello,
> > what would you think about letting "tee" exit immediately when the outp=
ut pipe was closed? It
> > should be pretty straightforward to implement using select/epoll, right?
> >=20
> >  -- Jann
>=20
> Here's a patch that works for me.

Any opinion on this? Maybe I just didn't see it, but I think I received no =
responses about this yet. Personally,
I see no reason to keep the current behavior.

 -- Jann

--OgqxwSJOaUobr8KG
Content-Type: application/pgp-signature

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iQIcBAEBAgAGBQJP0M8RAAoJEGhmizV0f2d1c/0P/3Ll850LSEy3iexG5eBd2EbJ
DFlQm7/YjWlNwIUCt3D3oS/94t8yfKGagFLpdN46W6RJxeVXxkzjkq74FJL85AQj
FQ59gELMoiG8UzEQQMZnYBqsfjFDI4HieQpKB/J0lN+FSLFdRT4L9UeyFLIelBFC
jnds5G+6DlABgxx2X/xphaLLGOYRc1UI3IQfEHTXR8hIeRjIu5VHUq/TPBVMCKv0
cvXuLXUnANwXX07LNiVkXABLsInNeisYpWssD9P0YW5SmUtzqAIaTx251sB1hgoQ
hS0LmG9czdInxobzyxf4l/HL4aO/c13YsV309TlDn7txdEXBeUyBJmiHyvpLOUGr
lYBKRBHSV5W5VFAXqEpMoaxyBqF2ZHAOgsZTjYhOQi18rxTxtv0VmAQFV9seYptu
PVJX3xc6wuccSNJxreI9JdYflYPzCR9TVB8bf/7fnmwVHOUu2jmjtma75ew63+7M
80bSvTZOXXI+NU/4XB/zkIBLhPN3OnlppXts9DHqJUJG14Gb0XWdfneh9ADXIcUh
CMPsEZcQhQlW4i92SKUGRQH2hVueGkhUs9oUPlKL6/vgH/kp/i1JPQ6OA3DyU63s
gK/fC0Jqb9Nv5wwKPMDNR68veCUMnBnhHP04K2flNqBD0nC894xex7FU51PXUKx5
sEsCGOlpfW80jsEDSxzL
=ZTJl
-----END PGP SIGNATURE-----

--OgqxwSJOaUobr8KG--


From MAILER-DAEMON Thu Jun 07 12:10:27 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1ScfI3-00076k-BF
	for mharc-coreutils@gnu.org; Thu, 07 Jun 2012 12:10:27 -0400
Received: from eggs.gnu.org ([208.118.235.92]:55331)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1ScfI0-0006ye-0V
	for coreutils@gnu.org; Thu, 07 Jun 2012 12:10:26 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1ScfHy-0004dC-5m
	for coreutils@gnu.org; Thu, 07 Jun 2012 12:10:23 -0400
Received: from mail3.vodafone.ie ([213.233.128.45]:43112)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1ScfHx-0004cw-W2
	for coreutils@gnu.org; Thu, 07 Jun 2012 12:10:22 -0400
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: ApMBAFDR0E9tTEvz/2dsb2JhbAANOLdAAQEBBDIBRhALDQsJFg8JAwIBAgFFBg0BBwEBwQqLHYYMA5YvhGOFDIda
Received: from unknown (HELO [192.168.1.79]) ([109.76.75.243])
	by mail3.vodafone.ie with ESMTP; 07 Jun 2012 17:10:19 +0100
Message-ID: <4FD0D26B.2080608@draigBrady.com>
Date: Thu, 07 Jun 2012 17:10:19 +0100
From: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:6.0) Gecko/20110816 Thunderbird/6.0
MIME-Version: 1.0
To: Jann Horn <jannhorn@googlemail.com>
Subject: Re: feature proposal for tee: exit immediately when the output pipe
	closes
References: <20120518145838.GA31109@Jann-PC.fritz.box>
	<20120518190018.GB31109@Jann-PC.fritz.box>
	<20120607155601.GA32204@Jann-PC.fritz.box>
In-Reply-To: <20120607155601.GA32204@Jann-PC.fritz.box>
X-Enigmail-Version: 1.3.2
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 213.233.128.45
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 07 Jun 2012 16:10:26 -0000

On 06/07/2012 04:56 PM, Jann Horn wrote:
> On Fri, May 18, 2012 at 09:00:18PM +0200, Jann Horn wrote:
>> On Fri, May 18, 2012 at 04:58:38PM +0200, Jann Horn wrote:
>>> Hello,
>>> what would you think about letting "tee" exit immediately when the output pipe was closed? It
>>> should be pretty straightforward to implement using select/epoll, right?
>>>
>>>  -- Jann
>>
>> Here's a patch that works for me.
> 
> Any opinion on this? Maybe I just didn't see it, but I think I received no responses about this yet. Personally,
> I see no reason to keep the current behavior.
> 
>  -- Jann

I CC'd you on http://bugs.gnu.org/11540
Hopefully that proposal suffices.
I'll look at merging something soon.

cheers,
Pádraig.


From MAILER-DAEMON Thu Jun 07 16:27:39 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1ScjIx-00054y-In
	for mharc-coreutils@gnu.org; Thu, 07 Jun 2012 16:27:39 -0400
Received: from eggs.gnu.org ([208.118.235.92]:53707)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeff.janes@gmail.com>) id 1Scj4F-0005GX-MB
	for coreutils@gnu.org; Thu, 07 Jun 2012 16:12:28 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jeff.janes@gmail.com>) id 1Scj4D-0004YS-T0
	for coreutils@gnu.org; Thu, 07 Jun 2012 16:12:27 -0400
Received: from mail-yw0-f41.google.com ([209.85.213.41]:50761)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeff.janes@gmail.com>) id 1Scj4D-0004WZ-NM
	for coreutils@gnu.org; Thu, 07 Jun 2012 16:12:25 -0400
Received: by yhr47 with SMTP id 47so887901yhr.0
	for <coreutils@gnu.org>; Thu, 07 Jun 2012 13:12:22 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
	h=mime-version:date:message-id:subject:from:to:content-type;
	bh=KbHhu/jsg6yBfe0RJQvzpwYD5vaoqsTcLlcgN6RdoL4=;
	b=K4w8rlPex9O31P0UTev9GYoZOivkV7v2wJbDJQ19u3lxEfQhgcmE9VFklecavRHi7L
	+f43Gc13FqKsoXjoKgv5NGZ/jf43uh5x+t3XnXu8xrS+AvBLGwXbU25PKZlzbaDCJHHX
	JgLBQjs4umXdMxWkwXKj3Kgq0FCWFq3vPmatMgi4E00Eo8iCAB2OxG4J8MOQOWO/7rGb
	AXJ78jKq6GmG8PbmDBYDasmlMH3gjiOfUH40nMmYWRkEWPpIcbl+xieeqjOTNiu/tmDE
	ZB7dNmSs621W5TJYuawLMoGXoLhqNWR3c7xs8JgM0id6ldzOlGZQFC+Q+wYrzq8w3jtP
	uSug==
MIME-Version: 1.0
Received: by 10.50.156.234 with SMTP id wh10mr2097239igb.20.1339099942103;
	Thu, 07 Jun 2012 13:12:22 -0700 (PDT)
Received: by 10.42.80.65 with HTTP; Thu, 7 Jun 2012 13:12:22 -0700 (PDT)
Date: Thu, 7 Jun 2012 13:12:22 -0700
Message-ID: <CAMkU=1zDA=RFzxb-O_x2TG+EumrdUebZJb4UOJAM_xretGpCCA@mail.gmail.com>
Subject: overly aggressive memory usage by sort.c
From: Jeff Janes <jeff.janes@gmail.com>
To: coreutils@gnu.org
Content-Type: text/plain; charset=ISO-8859-1
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.85.213.41
X-Mailman-Approved-At: Thu, 07 Jun 2012 16:27:38 -0400
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 07 Jun 2012 20:12:28 -0000

In commit a507ed6ede5064b8f15c979e54e6de3bb478d73e, first appearing in
v8.16, the default memory usage was changed to take all of available
memory, rather than half of it.  I think this is too aggressive.

If I start a very large file sort on a previously idle server, it will
report almost all physical memory as being available, and so sort will
take all of it.  But as soon as the heavy IO (reading the data to be
sorted, writing temp files) starts up, the kernel needs more memory
for buffering in order to make the IO efficient.  The kernel and the
sort start competing for memory, a little bit of paging/swapping
starts, time in iowait increases, and the overall sort performance
drops by roughly a factor of 2.

I don't know what the correct proportion of available memory to take
would be, but I think it is >0.5 and <1.0.  Maybe 0.75.  But I think
that just going back to 0.5 would be better than the status quo.  Or
perhaps the upper limit clamp could be based on physical memory
instead of available, so rather than:

mem = MAX (avail, total / 8);

maybe:

mem = MIN(total/4*3, MAX (avail, total / 8));


Cheers,

Jeff


From MAILER-DAEMON Thu Jun 07 20:11:18 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1ScmnO-0000ef-6j
	for mharc-coreutils@gnu.org; Thu, 07 Jun 2012 20:11:18 -0400
Received: from eggs.gnu.org ([208.118.235.92]:44563)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1ScmnK-0000cE-OZ
	for coreutils@gnu.org; Thu, 07 Jun 2012 20:11:16 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1ScmnH-0003yh-PM
	for coreutils@gnu.org; Thu, 07 Jun 2012 20:11:14 -0400
Received: from mail3.vodafone.ie ([213.233.128.45]:52651)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1ScmnH-0003xj-JX
	for coreutils@gnu.org; Thu, 07 Jun 2012 20:11:11 -0400
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: ApMBADVC0U9tTEvz/2dsb2JhbAANOLdsAQEBBDIBRhALDQsJFg8JAwIBAgFFBg0BBwEBwSSLHRWFawOWL4RjhQyHWoFW
Received: from unknown (HELO [192.168.1.79]) ([109.76.75.243])
	by mail3.vodafone.ie with ESMTP; 08 Jun 2012 01:11:09 +0100
Message-ID: <4FD1431C.5010500@draigBrady.com>
Date: Fri, 08 Jun 2012 01:11:08 +0100
From: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:6.0) Gecko/20110816 Thunderbird/6.0
MIME-Version: 1.0
To: Jeff Janes <jeff.janes@gmail.com>
Subject: Re: overly aggressive memory usage by sort.c
References: <CAMkU=1zDA=RFzxb-O_x2TG+EumrdUebZJb4UOJAM_xretGpCCA@mail.gmail.com>
In-Reply-To: <CAMkU=1zDA=RFzxb-O_x2TG+EumrdUebZJb4UOJAM_xretGpCCA@mail.gmail.com>
X-Enigmail-Version: 1.3.2
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 213.233.128.45
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 08 Jun 2012 00:11:16 -0000

On 06/07/2012 09:12 PM, Jeff Janes wrote:
> In commit a507ed6ede5064b8f15c979e54e6de3bb478d73e, first appearing in
> v8.16, the default memory usage was changed to take all of available
> memory, rather than half of it.  I think this is too aggressive.
> 
> If I start a very large file sort on a previously idle server, it will
> report almost all physical memory as being available, and so sort will
> take all of it.  But as soon as the heavy IO (reading the data to be
> sorted, writing temp files) starts up, the kernel needs more memory
> for buffering in order to make the IO efficient.  The kernel and the
> sort start competing for memory, a little bit of paging/swapping
> starts, time in iowait increases, and the overall sort performance
> drops by roughly a factor of 2.
> 
> I don't know what the correct proportion of available memory to take
> would be, but I think it is >0.5 and <1.0.  Maybe 0.75.  But I think
> that just going back to 0.5 would be better than the status quo.  Or
> perhaps the upper limit clamp could be based on physical memory
> instead of available, so rather than:
> 
> mem = MAX (avail, total / 8);
> 
> maybe:
> 
> mem = MIN(total/4*3, MAX (avail, total / 8));

I have to agree. In general patches like this shouldn't
go in without extensive performance testing.

The thread discussing the patch is here:
http://bugs.gnu.org/10877

There are other things we might consider with external files.

- ensure they're written to disk rather than ram.
I.E. avoid /tmp if it's tmpfs as is becoming more common on systems

- use posix_fadvise as is done in dd, to mark the external files
as non cachable, as the only reason you'd be using them is when
you don't have enough RAM anyway.

The above considers a 2 level memory hierarchy.
Increasingly though the "memory wall" is an issue,
so a 3 level hierarchy involving increasingly large CPU caches
should be considered.

cheers,
Pádraig.


From MAILER-DAEMON Thu Jun 07 20:14:32 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1ScmqW-00013i-Im
	for mharc-coreutils@gnu.org; Thu, 07 Jun 2012 20:14:32 -0400
Received: from eggs.gnu.org ([208.118.235.92]:45175)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1ScmqT-00013c-1g
	for coreutils@gnu.org; Thu, 07 Jun 2012 20:14:31 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1ScmqQ-0004UU-IO
	for coreutils@gnu.org; Thu, 07 Jun 2012 20:14:28 -0400
Received: from mail3.vodafone.ie ([213.233.128.45]:3114)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1ScmqQ-0004UP-6O
	for coreutils@gnu.org; Thu, 07 Jun 2012 20:14:26 -0400
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: ArABAH1D0U9tTEvz/2dsb2JhbAANOLIphUMBAQEEAQIvATkNEAsNCwkWDwkDAgECARYvBg0BBwEBiBK5EIsyBYVmA5J3gziEY4xmgVYI
Received: from unknown (HELO [192.168.1.79]) ([109.76.75.243])
	by mail3.vodafone.ie with ESMTP; 08 Jun 2012 01:14:24 +0100
Message-ID: <4FD143E0.8000306@draigBrady.com>
Date: Fri, 08 Jun 2012 01:14:24 +0100
From: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:6.0) Gecko/20110816 Thunderbird/6.0
MIME-Version: 1.0
To: Chen Guo <chenguo4@yahoo.com>
Subject: Re: Taking advantage of L1 and L2 cache in sort
References: <4B8CE672.5030606@draigBrady.com>
	<657794.87632.qm@web180001.mail.gq1.yahoo.com>
	<4B8DB5FE.6010006@draigBrady.com>
In-Reply-To: <4B8DB5FE.6010006@draigBrady.com>
X-Enigmail-Version: 1.3.2
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 213.233.128.45
Cc: Joey Degges <jdegges@gmail.com>, coreutils@gnu.org,
	Shaun Jackman <sjackman@bcgsc.ca>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 08 Jun 2012 00:14:31 -0000

On 03/03/2010 01:06 AM, Pádraig Brady wrote:
> On 02/03/10 18:20, Chen Guo wrote:
>> This is exactly what that guy Shaun Jackman was talking about earlier.
>> I'm actually really surprised this is faster, if I can dig up his e-mail I'll
>> forward him this, I remember him saying something about experimenting
>> with exactly this.
> 
> I missed that thread but yes he pretty much had the
> same idea as I stumbled on when trying to perturb
> the posix_fadvise() testing by changing the buffer size.
> http://lists.gnu.org/archive/html/bug-coreutils/2010-02/msg00151.html
> Spooky :)
> 
> Shaun, you can use `taskset` to set process affinity BTW.
> 
>> Can you profile the difference in the number of I/O system calls?
> 
> $ TMPDIR=/ram LANG=C /usr/bin/time -v strace -c ./src/sort sort.t/sort.1.test > /dev/null
> % time     seconds  usecs/call     calls    errors syscall
> ------ ----------- ----------- --------- --------- ----------------
>  70.70    0.283077       21775        13           read
>  28.97    0.115983       19331         6           munmap
>   0.32    0.001268           0     21609           write
>   0.01    0.000054           8         7           open
>   0.00    0.000000           0         9           close
>   0.00    0.000000           0         1           execve
>   0.00    0.000000           0         1         1 access
>   0.00    0.000000           0         3           brk
>   0.00    0.000000           0         1         1 ioctl
>   0.00    0.000000           0         1           uname
>   0.00    0.000000           0         5           mprotect
>   0.00    0.000000           0        25           rt_sigaction
>   0.00    0.000000           0         1           rt_sigprocmask
>   0.00    0.000000           0         4           getrlimit
>   0.00    0.000000           0        16           mmap2
>   0.00    0.000000           0         9           fstat64
>   0.00    0.000000           0         2         1 futex
>   0.00    0.000000           0         1           set_thread_area
>   0.00    0.000000           0         1           set_tid_address
>   0.00    0.000000           0         1           fadvise64_64
>   0.00    0.000000           0         1           set_robust_list
> ------ ----------- ----------- --------- --------- ----------------
> 100.00    0.400382                 21717         3 total
>     Command being timed: "strace -c ./src/sort sort.t/sort.1.test"
>     User time (seconds): 26.91
>     System time (seconds): 2.01
>     Percent of CPU this job got: 90%
>     Elapsed (wall clock) time (h:mm:ss or m:ss): 0:32.02
>     Average shared text size (kbytes): 0
>     Average unshared data size (kbytes): 0
>     Average stack size (kbytes): 0
>     Average total size (kbytes): 0
>     Maximum resident set size (kbytes): 0
>     Average resident set size (kbytes): 0
>     Major (requiring I/O) page faults: 3
>     Minor (reclaiming a frame) page faults: 181060
>     Voluntary context switches: 87362
>     Involuntary context switches: 2526
>     Swaps: 0
>     File system inputs: 173504
>     File system outputs: 0
>     Socket messages sent: 0
>     Socket messages received: 0
>     Signals delivered: 0
>     Page size (bytes): 4096
>     Exit status: 0
> 
> $ TMPDIR=/ram LANG=C /usr/bin/time -v strace -c ./src/sort -S1M sort.t/sort.1.test > /dev/null
> % time     seconds  usecs/call     calls    errors syscall
> ------ ----------- ----------- --------- --------- ----------------
>  38.95    0.035011           1     60991           read
>  33.47    0.030081          90       334           unlink
>  24.17    0.021721           0     81864           write
>   2.07    0.001862           2      1006           munmap
>   0.75    0.000670           1       673           open
>   0.23    0.000209           0      1016           mmap2
>   0.19    0.000167           0       675           fstat64
>   0.09    0.000085           0       675           close
>   0.07    0.000062           0       334           fcntl64
>   0.02    0.000018           0      1337           rt_sigprocmask
>   0.00    0.000000           0         1           execve
>   0.00    0.000000           0         1         1 access
>   0.00    0.000000           0         3           brk
>   0.00    0.000000           0         1         1 ioctl
>   0.00    0.000000           0         1           gettimeofday
>   0.00    0.000000           0         1           uname
>   0.00    0.000000           0         5           mprotect
>   0.00    0.000000           0       334           _llseek
>   0.00    0.000000           0        25           rt_sigaction
>   0.00    0.000000           0         1           getrlimit
>   0.00    0.000000           0         2         1 futex
>   0.00    0.000000           0         1           set_thread_area
>   0.00    0.000000           0         1           set_tid_address
>   0.00    0.000000           0       335           fadvise64_64
>   0.00    0.000000           0         1           set_robust_list
> ------ ----------- ----------- --------- --------- ----------------
> 100.00    0.089886                149618         3 total
>     Command being timed: "strace -c ./src/sort -S1M sort.t/sort.1.test"
>     User time (seconds): 21.76
>     System time (seconds): 4.51
>     Percent of CPU this job got: 98%
>     Elapsed (wall clock) time (h:mm:ss or m:ss): 0:26.79
>     Average shared text size (kbytes): 0
>     Average unshared data size (kbytes): 0
>     Average stack size (kbytes): 0
>     Average total size (kbytes): 0
>     Maximum resident set size (kbytes): 0
>     Average resident set size (kbytes): 0
>     Major (requiring I/O) page faults: 3
>     Minor (reclaiming a frame) page faults: 23038
>     Voluntary context switches: 598317
>     Involuntary context switches: 2316
>     Swaps: 0
>     File system inputs: 173504
>     File system outputs: 0
>     Socket messages sent: 0
>     Socket messages received: 0
>     Signals delivered: 0
>     Page size (bytes): 4096
>     Exit status: 0

Hmm I think the results above are invalid
due to MALLOC_PERTURB_ biasing against large buffers.
In the test below the extra external file process dominates
any gains from CPU cache usage:

$ # Test setup
$ cache_size() { lscpu | grep cache | tail -n1 | tr -s ' ' | cut -d' ' -f3; }
$ export TMPDIR=/dev/shm # RAM
$ export OMP_NUM_THREADS=1 # limit to a single CPU to simplify
$ unset MALLOC_PERTURB_ # negatively impacts performance tests
$ shuf -i 1-10000000 > sort.test # 78MB

$ LANG=C /usr/bin/time -v strace -c sort -S$(cache_size) sort.test > /dev/null
% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
 66.33    0.034878         185       189           unlink
 17.89    0.009408           0     45766           read
  8.73    0.004590           0     57711           write
  4.53    0.002381           4       571           munmap
  1.28    0.000671           2       382           open
  0.46    0.000242           1       384           fstat
  0.35    0.000183           0       580           mmap
  0.19    0.000100           0       384           close
  0.14    0.000074           0       757           rt_sigprocmask
  0.11    0.000058           0       189           fcntl
  0.00    0.000000           0       189           lseek
  0.00    0.000000           0         5           mprotect
  0.00    0.000000           0         4           brk
  0.00    0.000000           0        25           rt_sigaction
  0.00    0.000000           0         1         1 ioctl
  0.00    0.000000           0         1         1 access
  0.00    0.000000           0         1           execve
  0.00    0.000000           0         1           getrlimit
  0.00    0.000000           0         1           arch_prctl
  0.00    0.000000           0         1           set_tid_address
  0.00    0.000000           0       190           fadvise64
  0.00    0.000000           0         1           set_robust_list
------ ----------- ----------- --------- --------- ----------------
100.00    0.052585                107333         2 total
	Command being timed: "strace -c sort --parallel=1 -T/dev/shm -S3072K sort.test"
	User time (seconds): 25.88
	System time (seconds): 5.81
	Percent of CPU this job got: 94%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 0:33.59
	Average shared text size (kbytes): 0
	Average unshared data size (kbytes): 0
	Average stack size (kbytes): 0
	Average total size (kbytes): 0
	Maximum resident set size (kbytes): 4968
	Average resident set size (kbytes): 0
	Major (requiring I/O) page faults: 0
	Minor (reclaiming a frame) page faults: 13900
	Voluntary context switches: 429178
	Involuntary context switches: 4882
	Swaps: 0
	File system inputs: 0
	File system outputs: 0
	Socket messages sent: 0
	Socket messages received: 0
	Signals delivered: 0
	Page size (bytes): 4096
	Exit status: 0


$ LANG=C /usr/bin/time -v strace -c sort -S200M sort.test > /dev/null
% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
 80.68    0.348335         565       617           read
 11.58    0.049994        3846        13           munmap
  5.33    0.022995        7665         3           unlink
  2.38    0.010281           0     38521           write
  0.01    0.000052          52         1         1 access
  0.01    0.000042           8         5           mprotect
  0.01    0.000032           1        25           rt_sigaction
  0.00    0.000000           0        10           open
  0.00    0.000000           0        12           close
  0.00    0.000000           0        12           fstat
  0.00    0.000000           0         3           lseek
  0.00    0.000000           0        22           mmap
  0.00    0.000000           0         4           brk
  0.00    0.000000           0        13           rt_sigprocmask
  0.00    0.000000           0         1         1 ioctl
  0.00    0.000000           0         1           execve
  0.00    0.000000           0         3           fcntl
  0.00    0.000000           0         1           getrlimit
  0.00    0.000000           0         1           arch_prctl
  0.00    0.000000           0         1           set_tid_address
  0.00    0.000000           0         4           fadvise64
  0.00    0.000000           0         1           set_robust_list
------ ----------- ----------- --------- --------- ----------------
100.00    0.431731                 39274         2 total
	Command being timed: "strace -c sort -S200M sort.test"
	User time (seconds): 19.82
	System time (seconds): 4.33
	Percent of CPU this job got: 94%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 0:25.57
	Average shared text size (kbytes): 0
	Average unshared data size (kbytes): 0
	Average stack size (kbytes): 0
	Average total size (kbytes): 0
	Maximum resident set size (kbytes): 205532
	Average resident set size (kbytes): 0
	Major (requiring I/O) page faults: 355
	Minor (reclaiming a frame) page faults: 98057
	Voluntary context switches: 159217
	Involuntary context switches: 4681
	Swaps: 0
	File system inputs: 140344
	File system outputs: 154168
	Socket messages sent: 0
	Socket messages received: 0
	Signals delivered: 0
	Page size (bytes): 4096
	Exit status: 0

I notice also that performance gets worse for the above input
for the small buffer size when I enable multithreading on my 4 core i3.
	User time (seconds): 35.40
	System time (seconds): 9.20
	Percent of CPU this job got: 120%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 0:36.98

cheers,
Pádraig.


From MAILER-DAEMON Sat Jun 09 16:19:37 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1SdS8H-0004NC-5e
	for mharc-coreutils@gnu.org; Sat, 09 Jun 2012 16:19:37 -0400
Received: from eggs.gnu.org ([208.118.235.92]:54470)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <kerolasa@gmail.com>) id 1SdS8B-0004MX-Cj
	for coreutils@gnu.org; Sat, 09 Jun 2012 16:19:34 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <kerolasa@gmail.com>) id 1SdS86-0001un-SJ
	for coreutils@gnu.org; Sat, 09 Jun 2012 16:19:30 -0400
Received: from mail-wi0-f177.google.com ([209.85.212.177]:56856)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <kerolasa@gmail.com>) id 1SdS86-0001ui-C4
	for coreutils@gnu.org; Sat, 09 Jun 2012 16:19:26 -0400
Received: by wibhm14 with SMTP id hm14so1245979wib.12
	for <coreutils@gnu.org>; Sat, 09 Jun 2012 13:19:23 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
	h=mime-version:reply-to:sender:date:x-google-sender-auth:message-id
	:subject:from:to:content-type:content-transfer-encoding;
	bh=Qc2Or8NwB3lAxL3dQZ83eKn/z9bE2q8mzT3V8ZDflLc=;
	b=RxGwvM3KB7XR8atv2mWDxY07aWN9mXHtUGAfRCxFwgkdQw1+iCo6ANmxkk+OmS22nH
	lqMwLGzpgCLIjF/KRZnVZ4fW/ncP4qXWMgTkppyLUbicJK0iqyc9EnL7F9qMQdJkO7B5
	X6unm9eZk/DuukSv+u1tFbh2/kmyMowGVAXHsVBzLVvjQbHpabPoda6cBwZEItrOYXmO
	55+IPXAUFVXujoIqZ6oYpN7Z+AhwSguQtVVgufjk6tQLiHytOFEB7FryU3lchG8VwBEO
	u0qDjHbgjMLs34zzn0dZwwbuQ7oA91ctu8ThDAV8aPas8jCcaOCbRm2oHI2OJds2yk1C
	qqkg==
MIME-Version: 1.0
Received: by 10.216.207.67 with SMTP id m45mr2156653weo.175.1339273163524;
	Sat, 09 Jun 2012 13:19:23 -0700 (PDT)
Sender: kerolasa@gmail.com
Received: by 10.194.56.10 with HTTP; Sat, 9 Jun 2012 13:19:23 -0700 (PDT)
Date: Sat, 9 Jun 2012 22:19:23 +0200
X-Google-Sender-Auth: lrDErkvoK8B7iATx6EALsIf9Ja4
Message-ID: <CAG27Bk0dwWn9L8MJ8x5TBchNvZ5CY2y+ATeAmypDqZi1Y8esZw@mail.gmail.com>
Subject: rename: move command from util-linux to coreutils
From: Sami Kerola <kerolasa@iki.fi>
To: coreutils@gnu.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.85.212.177
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
Reply-To: kerolasa@gmail.com
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sat, 09 Jun 2012 20:19:35 -0000

Hello,

The next coreutils and util-linux releases are going to be made close
to each others to avoid 'su' coming from multiple source, or being
missing.  At same go moving 'rename' could make sense.

I completely rewrote the command as the util-linux way doing things
would not be a strong match how code is crafted in coreutils.  Primary
difference is, as well known, usage of gnulib and a little bit
different coding style.

Please find my proposal how the new rename might be implemented.  I am
almost sure the merge should not happen as is, as there are some cases
which might not work correctly; for example directories are not
renamed, nor files moved out of directories.  Perhaps that is right, or
wrong.  Comments are definitely welcome.

Another decision which might be wrong is stdin file list handling.
Command, by default, expects null terminated strings and nothing else
will work.  One might argue the command should, as xargs, allow white
space terminated input.  IMHO allowing white space termination
encourages users to use error prone input style, and therefore it is
not justified to be default nor supported in modern commands.  But I
could overlook something, so maybe this needs to be discussed as well.

Finally, without any doubt I wrote half of English somehow wrong so human
language review is highly recommendable.


The following changes since commit 928dd73762e69cfeaab4a7ec9dd8f30f86a45ed4=
:

  su: remove program (util-linux is now the best source for it)
(2012-06-06 22:33:08 +0200)

are available in the git repository at:

  git://github.com/kerolasa/coreutils.git rename

for you to fetch changes up to 62ccdd1435210cb8f30ba50f297f57759f264d50:

  rename: move command from util-linux to coreutils (2012-06-09 21:57:03 +0=
200)

----------------------------------------------------------------
Sami Kerola (1):
      rename: move command from util-linux to coreutils

 AUTHORS                      |    1 +
 NEWS                         |    2 +
 README                       |    8 +-
 doc/coreutils.texi           |  117 +++++++-
 man/.gitignore               |    1 +
 man/Makefile.am              |    1 +
 man/rename.x                 |   10 +
 po/POTFILES.in               |    1 +
 scripts/git-hooks/commit-msg |   13 +-
 src/.gitignore               |    1 +
 src/Makefile.am              |    4 +
 src/rename.c                 |  646 ++++++++++++++++++++++++++++++++++++++=
++++
 tests/Makefile.am            |    1 +
 tests/misc/help-version      |    1 +
 tests/misc/invalid-opt       |    1 +
 tests/misc/rename-foo2bar    |   30 ++
 16 files changed, 827 insertions(+), 11 deletions(-)
 create mode 100644 man/rename.x
 create mode 100644 src/rename.c
 create mode 100755 tests/misc/rename-foo2bar

diff --git a/AUTHORS b/AUTHORS
index 9984a2e..f2737d2 100644
--- a/AUTHORS
+++ b/AUTHORS
@@ -63,6 +63,7 @@ ptx: Fran=E7ois Pinard
 pwd: Jim Meyering
 readlink: Dmitry V. Levin
 realpath: P=E1draig Brady
+rename: Sami Kerola
 rm: Paul Rubin, David MacKenzie, Richard M. Stallman, Jim Meyering
 rmdir: David MacKenzie
 runcon: Russell Coker
diff --git a/NEWS b/NEWS
index 54d24c3..306f07a 100644
--- a/NEWS
+++ b/NEWS
@@ -38,6 +38,8 @@ GNU coreutils NEWS
 -*- outline -*-
   patches as well as enough support to build on the Hurd, we no longer
   have any reason to include it here.

+  rename: Rename multiple files using a pattern.  This command will
+  replace an utility with same name in util-linux package.

 * Noteworthy changes in release 8.17 (2012-05-10) [stable]

diff --git a/README b/README
index 09961a0..5f21924 100644
--- a/README
+++ b/README
@@ -12,10 +12,10 @@ The programs that can be built with this package are:
   factor false fmt fold groups head hostid hostname id install join kill
   link ln logname ls md5sum mkdir mkfifo mknod mktemp mv nice nl nohup
   nproc od paste pathchk pinky pr printenv printf ptx pwd readlink realpat=
h
-  rm rmdir runcon seq sha1sum sha224sum sha256sum sha384sum sha512sum shre=
d
-  shuf sleep sort split stat stdbuf stty sum sync tac tail tee test
-  timeout touch tr true truncate tsort tty uname unexpand uniq unlink
-  uptime users vdir wc who whoami yes
+  rename rm rmdir runcon seq sha1sum sha224sum sha256sum sha384sum
+  sha512sum shred shuf sleep sort split stat stdbuf stty sum sync tac tail
+  tee test timeout touch tr true truncate tsort tty uname unexpand uniq
+  unlink uptime users vdir wc who whoami yes

 See the file NEWS for a list of major changes in the current release.

diff --git a/doc/coreutils.texi b/doc/coreutils.texi
index f7251b2..ef90459 100644
--- a/doc/coreutils.texi
+++ b/doc/coreutils.texi
@@ -95,6 +95,7 @@
 * pwd: (coreutils)pwd invocation.               Print working directory.
 * readlink: (coreutils)readlink invocation.     Print referent of a symlin=
k.
 * realpath: (coreutils)readpath invocation.     Print resolved file names.
+* rename: (coreutils)rename invocation.         Rename multiple files.
 * rm: (coreutils)rm invocation.                 Remove files.
 * rmdir: (coreutils)rmdir invocation.           Remove empty directories.
 * runcon: (coreutils)runcon invocation.         Run in specified SELinux C=
TX.
@@ -194,7 +195,7 @@ Free Documentation License''.
 * Printing text::                echo printf yes
 * Conditions::                   false true test expr
 * Redirection::                  tee
-* File name manipulation::       dirname basename pathchk mktemp realpath
+* File name manipulation::       dirname basename pathchk rename
mktemp realpath
 * Working context::              pwd stty printenv tty
 * User information::             id logname whoami groups users who
 * System context::               date arch nproc uname hostname hostid upt=
ime
@@ -380,6 +381,7 @@ File name manipulation
 * basename invocation::          Strip directory and suffix from a file na=
me
 * dirname invocation::           Strip last file name component
 * pathchk invocation::           Check file name validity and portability
+* rename invocation::            Rename multiple files using a pattern
 * mktemp invocation::            Create temporary file or directory
 * realpath invocation::          Print resolved file names

@@ -12439,6 +12441,7 @@ This section describes commands that
manipulate file names.
 * basename invocation::         Strip directory and suffix from a file nam=
e.
 * dirname invocation::          Strip last file name component.
 * pathchk invocation::          Check file name validity and portability.
+* rename invocation::           Rename multiple files using a pattern.
 * mktemp invocation::           Create temporary file or directory.
 * realpath invocation::         Print resolved file names.
 @end menu
@@ -12663,6 +12666,118 @@ Exit status:
 1 otherwise.
 @end display

+
+@node rename invocation
+@section @command{rename}: Rename multiple files using a pattern
+
+@pindex rename
+@cindex rename multiple files
+
+@command{rename} moves multiple files using a pattern. Synopsis:
+
+@example
+rename [@var{option}] @var{expression} @var{replacement} @var{file...}
+rename [@var{option}] --files0-from=3DFILE  @var{--exec} @var{command}
+find . -print0 \
+  | rename [@var{option}] @var{expression} @var{replacement}
+find . -print0 \
+  | rename [@var{option}] @var{--exec} @var{command}
+@end example
+
+The @command{rename} command does multiple @var{file} renames same way as
+@command{mv} would do individually. The @command{rename} will not rename
+directory names, nor move files out of directory in which they are.
+
+The program accepts the following options.  Also see @ref{Common options}.
+
+@table @samp
+
+@filesZeroFromOption{rename}
+
+
+@item -e @var{command -o --option}
+@itemx --exec @var{command -o --option}
+@opindex -e
+@opindex --exec
+@cindex external execution
+Use @command{command -o --option} to manipulate file name string.
+@smallexample
+# Show how ba would be removed from bash
+find /bin/ba* -print0 | rename -t -e sed 's/bash/sh/'
+@end smallexample
+
+
+@item -R
+@itemx -r
+@itemx --recursive
+@opindex -R
+@opindex -r
+@opindex --recursive
+@cindex recursive renaming
+Rename files recursively.
+
+@item -x
+@itemx --one-file-system
+@opindex -x
+@opindex --one-file-system
+@cindex file systems, omitting renaming to different
+Skip subdirectories that are on different file systems from the one that
+the rename started on.
+
+@optBackup
+
+@optBackupSuffix
+
+@item -f
+@itemx --force
+@opindex -f
+@opindex --force
+@cindex prompts, omitting
+Do not prompt the user before removing a destination file.
+@mvOptsIfn
+
+@item -i
+@itemx --interactive
+@opindex -i
+@opindex --interactive
+@cindex prompts, forcing
+Prompt whether to overwrite each existing destination file, regardless
+of its permissions.
+If the response is not affirmative, the file is skipped.
+@mvOptsIfn
+
+@item -n
+@itemx --no-clobber
+@opindex -n
+@opindex --no-clobber
+@cindex prompts, omitting
+Do not overwrite an existing file.
+@mvOptsIfn
+This option is mutually exclusive with @option{-b} or @option{--backup} op=
tion.
+
+@item -t
+@itemx --test
+@opindex -t
+@opindex --test
+Show printout how @command{rename} would affect without performing any cha=
nges.
+
+@item -v
+@itemx --verbose
+@opindex -v
+@opindex --verbose
+Print the name of each file before moving it.
+
+@end table
+
+@cindex exit status of @command{rename}
+Exit status:
+
+@display
+0 all files where renamed succesfully
+1 otherwise.
+@end display
+
+
 @node mktemp invocation
 @section @command{mktemp}: Create temporary file or directory

diff --git a/man/.gitignore b/man/.gitignore
index aa6fa51..22640ea 100644
--- a/man/.gitignore
+++ b/man/.gitignore
@@ -60,6 +60,7 @@ ptx.1
 pwd.1
 readlink.1
 realpath.1
+rename.1
 rm.1
 rmdir.1
 seq.1
diff --git a/man/Makefile.am b/man/Makefile.am
index cc93a42..177eec7 100644
--- a/man/Makefile.am
+++ b/man/Makefile.am
@@ -93,6 +93,7 @@ ptx.1:		$(common_dep)	$(srcdir)/ptx.x		../src/ptx.c
 pwd.1:		$(common_dep)	$(srcdir)/pwd.x		../src/pwd.c
 readlink.1:	$(common_dep)	$(srcdir)/readlink.x	../src/readlink.c
 realpath.1:	$(common_dep)	$(srcdir)/realpath.x	../src/realpath.c
+rename.1:	$(common_dep)	$(srcdir)/rename.x	../src/rename.c
 rm.1:		$(common_dep)	$(srcdir)/rm.x		../src/rm.c
 rmdir.1:	$(common_dep)	$(srcdir)/rmdir.x	../src/rmdir.c
 runcon.1:	$(common_dep)	$(srcdir)/runcon.x	../src/runcon.c
diff --git a/man/rename.x b/man/rename.x
new file mode 100644
index 0000000..f44af45
--- /dev/null
+++ b/man/rename.x
@@ -0,0 +1,10 @@
+[NAME]
+rename \- rename multiple files using a pattern
+[DESCRIPTION]
+.\" Add any additional description here
+[EXAMPLES]
+.B rename .htm .html *.htm
+.br
+This will move all files ending .htm to end with .html in current director=
y.
+[SEE ALSO]
+rename(2)
diff --git a/po/POTFILES.in b/po/POTFILES.in
index b938dfc..70e097e 100644
--- a/po/POTFILES.in
+++ b/po/POTFILES.in
@@ -105,6 +105,7 @@ src/readlink.c
 src/realpath.c
 src/relpath.c
 src/remove.c
+src/rename.c
 src/rm.c
 src/rmdir.c
 src/runcon.c
diff --git a/scripts/git-hooks/commit-msg b/scripts/git-hooks/commit-msg
index a5e4090..4b609a7 100755
--- a/scripts/git-hooks/commit-msg
+++ b/scripts/git-hooks/commit-msg
@@ -15,12 +15,13 @@ my @valid =3D qw(
     arch base64 basename cat chcon chgrp chmod chown chroot cksum comm
     cp csplit cut date dd df dir dircolors dirname du echo env expand
     expr factor false fmt fold groups head hostid hostname id install
-    join kill link ln logname ls md5sum mkdir mkfifo mknod mktemp
-    mv nice nl nohup nproc od paste pathchk pinky pr printenv printf
-    ptx pwd readlink realpath rm rmdir runcon seq sha1sum sha224sum sha256=
sum
-    sha384sum sha512sum shred shuf sleep sort split stat stdbuf stty
-    sum sync tac tail tee test timeout touch tr true truncate tsort
-    tty uname unexpand uniq unlink uptime users vdir wc who whoami yes
+    join kill link ln logname ls md5sum mkdir mkfifo mknod mktemp mv
+    nice nl nohup nproc od paste pathchk pinky pr printenv printf ptx
+    pwd readlink realpath rename rm rmdir runcon seq sha1sum sha224sum
+    sha256sum sha384sum sha512sum shred shuf sleep sort split stat
+    stdbuf stty sum sync tac tail tee test timeout touch tr true
+    truncate tsort tty uname unexpand uniq unlink uptime users vdir wc
+    who whoami yes

     copy gnulib tests maint doc build scripts
     );
diff --git a/src/.gitignore b/src/.gitignore
index d78178c..a6e092f 100644
--- a/src/.gitignore
+++ b/src/.gitignore
@@ -69,6 +69,7 @@ ptx
 pwd
 readlink
 realpath
+rename
 rm
 rmdir
 runcon
diff --git a/src/Makefile.am b/src/Makefile.am
index 3099505..47bb59e 100644
--- a/src/Makefile.am
+++ b/src/Makefile.am
@@ -97,6 +97,7 @@ EXTRA_PROGRAMS =3D \
   pwd		\
   readlink	\
   realpath	\
+  rename	\
   rm		\
   rmdir		\
   runcon	\
@@ -249,6 +250,7 @@ ptx_LDADD =3D $(LDADD)
 pwd_LDADD =3D $(LDADD)
 readlink_LDADD =3D $(LDADD)
 realpath_LDADD =3D $(LDADD)
+rename_LDADD =3D $(LDADD)
 rm_LDADD =3D $(LDADD)
 rmdir_LDADD =3D $(LDADD)
 runcon_LDADD =3D $(LDADD)
@@ -302,6 +304,7 @@ copy_LDADD =3D
 cp_LDADD +=3D $(copy_LDADD)
 ginstall_LDADD +=3D $(copy_LDADD)
 mv_LDADD +=3D $(copy_LDADD)
+rename_LDADD +=3D $(copy_LDADD)

 remove_LDADD =3D
 mv_LDADD +=3D $(remove_LDADD)
@@ -424,6 +427,7 @@ timeout_SOURCES =3D timeout.c operand2sig.c

 mv_SOURCES =3D mv.c remove.c $(copy_sources)
 rm_SOURCES =3D rm.c remove.c
+rename_SOURCES =3D rename.c $(copy_sources)

 mkdir_SOURCES =3D mkdir.c prog-fprintf.c
 rmdir_SOURCES =3D rmdir.c prog-fprintf.c
diff --git a/src/rename.c b/src/rename.c
new file mode 100644
index 0000000..e97ddac
--- /dev/null
+++ b/src/rename.c
@@ -0,0 +1,646 @@
+/* rename -- rename multiple files using a pattern
+   Copyright (C) 2012- Free Software Foundation, Inc.
+
+   This program is free software: you can redistribute it and/or modify
+   it under the terms of the GNU General Public License as published by
+   the Free Software Foundation, either version 3 of the License, or
+   (at your option) any later version.
+
+   This program is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+   GNU General Public License for more details.
+
+   You should have received a copy of the GNU General Public License
+   along with this program.  If not, see <http://www.gnu.org/licenses/>.  =
*/
+
+/* This command supports the same syntax as util-linux rename, which was
+   written by Andries Brouwer at Jan 1st, 2000.  The command appeared to
+   coreutils as the util-linux package is less obvious package to ship a
+   command such as this.
+
+   The coreutils implementation was done by Sami Kerola.  */
+
+#include <config.h>
+
+#include <assert.h>		/* for assert */
+#include <errno.h>		/* for errno */
+#include <getopt.h>		/* for no_argument, optind, optarg, etc */
+#include <libintl.h>		/* for bindtextdomain, textdomain */
+#include <limits.h>		/* for CHAR_MAX */
+#include <locale.h>		/* for setlocale, LC_ALL */
+#include <selinux/selinux.h>	/* for is_selinux_enabled */
+#include <stdbool.h>		/* for false, true, bool */
+#include <stdio.h>		/* for NULL, stdout, fprintf, etc */
+#include <stdlib.h>		/* for EXIT_FAILURE, exit, etc */
+#include <sys/param.h>		/* for MIN */
+#include <sys/stat.h>		/* for stat, fstat, S_ISREG */
+#include <sys/wait.h>		/* for waitpid */
+#include <unistd.h>		/* for STDIN_FILENO, close, isatty */
+
+#include "argv-iter.h"		/* for argv_iter_init_argv, etc */
+#include "backupfile.h"		/* for xget_version, etc */
+#include "closein.h"		/* for close_stdin */
+#include "configmake.h"		/* for LOCALEDIR */
+#include "copy.h"		/* for cp_options, etc */
+#include "cp-hash.h"		/* for hash_init */
+#include "error.h"		/* for error */
+#include "fts_.h"		/* for FTS, FTSENT, FTS_PHYSICAL, etc */
+#include "physmem.h"		/* for physmem_available */
+#include "progname.h"		/* for program_name, etc */
+#include "quotearg.h"		/* for quotearg_colon */
+#include "quote.h"		/* for quote */
+#include "readtokens0.h"	/* for Tokens, readtokens0, etc */
+#include "system.h"		/* for _, STREQ, etc */
+#include "unlocked-io.h"	/* for fputs */
+#include "xalloc.h"		/* for xalloc_die, xstrdup */
+#include "xfts.h"		/* for xfts_open */
+
+/* The official name of this program (e.g., no `g' prefix).  */
+#define PROGRAM_NAME "rename"
+
+#define AUTHORS \
+  proper_name ("Sami Kerola")
+
+#if RENAME_DEBUG
+#define FTS_CROSS_CHECK(Fts) fts_cross_check (Fts)
+#else
+#define FTS_CROSS_CHECK(Fts)
+#endif
+
+/* True if we have ever read the standard input. */
+static bool have_read_stdin;
+
+/* If true change the modes of directories recursively. */
+static bool recurse;
+
+/* If true only print out what a rename would do.  */
+static bool test_only =3D false;
+
+/* If set tradional string + replacement is wanted */
+static char *original_str;
+static char *replacement;
+
+/* The exec is using these.  */
+static char *cmd_to_run =3D NULL;
+static char **cmd_args =3D NULL;
+
+static void
+cp_option_init (struct cp_options *x)
+{
+  bool selinux_enabled =3D (0 < is_selinux_enabled ());
+
+  cp_options_default (x);
+  x->copy_as_regular =3D false;
+  x->reflink_mode =3D REFLINK_NEVER;
+  x->dereference =3D DEREF_NEVER;
+  x->unlink_dest_before_opening =3D false;
+  x->unlink_dest_after_failed_open =3D false;
+  x->hard_link =3D false;
+  x->interactive =3D I_UNSPECIFIED;
+  x->move_mode =3D true;
+  x->one_file_system =3D false;
+  x->preserve_ownership =3D true;
+  x->preserve_links =3D true;
+  x->preserve_mode =3D true;
+  x->preserve_timestamps =3D true;
+  x->preserve_security_context =3D selinux_enabled;
+  x->reduce_diagnostics =3D false;
+  x->data_copy_required =3D true;
+  x->require_preserve =3D false;
+  x->require_preserve_context =3D false;
+  x->preserve_xattr =3D true;
+  x->require_preserve_xattr =3D false;
+  x->recursive =3D false;
+  x->sparse_mode =3D SPARSE_AUTO;
+  x->symbolic_link =3D false;
+  x->set_mode =3D false;
+  x->mode =3D 0;
+  x->stdin_tty =3D isatty (STDIN_FILENO);
+
+  x->open_dangling_dest_symlink =3D false;
+  x->update =3D false;
+  x->verbose =3D false;
+  x->dest_info =3D NULL;
+  x->src_info =3D NULL;
+}
+
+/* Change first occurence of 'substr' to be 'replacement' in
+   'filename'.  This function is used when classical rename is been
+   queried.  */
+static char *
+construct_dest_name (const char *filename, const char *substr,
+		     const char *replacement)
+{
+  char *tok =3D NULL;
+  char *newstr =3D NULL;
+  char *basename;
+
+  /* Search if file name has path.  */
+  basename =3D strrchr (filename, '/');
+  if (basename =3D=3D NULL)
+    tok =3D strstr (filename, substr);
+  else
+    tok =3D strstr (basename, substr);
+  if (tok =3D=3D NULL)
+    return xstrdup (filename);
+
+  newstr =3D
+    xmalloc (strlen (filename) - strlen (substr) + strlen (replacement) + =
1);
+
+  /* Construct new file name.  */
+  if (newstr =3D=3D NULL)
+    return NULL;
+  memcpy (newstr, filename, tok - filename);
+  memcpy (newstr + (tok - filename), replacement, strlen (replacement));
+  memcpy (newstr + (tok - filename) + strlen (replacement),
+	  tok + strlen (substr),
+	  strlen (filename) - strlen (substr) - (tok - filename));
+  memset (newstr + strlen (filename) - strlen (substr) + strlen (replaceme=
nt),
+	  0, 1);
+  return newstr;
+}
+
+/* Child will perform filename altering.  */
+char *
+request_child (const char *filename)
+{
+  pid_t exec_pid;
+  FILE *mother_file =3D NULL;
+  int child2mother_pipefd[2];
+  int mother2child_pipefd[2];
+  int status =3D EXIT_SUCCESS;
+  char *ret =3D NULL;
+  char *basename;
+  size_t skip =3D 0;
+
+  signal (SIGPIPE, SIG_IGN);
+  pipe (mother2child_pipefd);
+  pipe (child2mother_pipefd);
+
+  exec_pid =3D fork ();
+  if (0 < exec_pid)
+    {
+      /* Mother.  */
+      close (mother2child_pipefd[0]);
+      close (child2mother_pipefd[1]);
+      mother_file =3D fdopen (child2mother_pipefd[0], "r");
+    }
+  if (0 =3D=3D exec_pid)
+    {
+      /* Child.  */
+      close (STDIN_FILENO);
+      close (STDOUT_FILENO);
+      close (mother2child_pipefd[1]);
+      close (child2mother_pipefd[0]);
+
+      /* Associate input and output with a pipe.  */
+      dup2 (mother2child_pipefd[0], STDIN_FILENO);
+      dup2 (child2mother_pipefd[1], STDOUT_FILENO);
+
+      execvp (cmd_to_run, cmd_args);
+      /* Never reached after successful execvp.  */
+      error (0, errno, _("executing file %s failed"), quote (cmd_to_run));
+      exit (EXIT_FAILURE);
+    }
+
+  /* Search if file name has path.  */
+  basename =3D strrchr (filename, '/');
+  if (basename !=3D NULL)
+    skip =3D strlen (filename) - strlen (basename);
+  /* Communicate with the child.  */
+  if (0 < skip)
+    write (mother2child_pipefd[1], basename, strlen (basename));
+  else
+    write (mother2child_pipefd[1], filename, strlen (filename));
+  close (mother2child_pipefd[1]);
+  /* Child working, prepare memory for new file name.  */
+  ret =3D xmalloc ((size_t) PATH_MAX);
+  memcpy (ret, filename, skip);
+  /* Read what child had to tell.  */
+  fgets (ret + skip, PATH_MAX - skip, mother_file);
+  waitpid (exec_pid, &status, 0);
+  /* Child is terminated.  */
+  if (mother_file !=3D NULL)
+    fclose (mother_file);
+  if (WEXITSTATUS (status) !=3D EXIT_SUCCESS)
+    {
+      exit (EXIT_FAILURE);
+    }
+  return ret;
+}
+
+/* Request subroutine about how file name should be affected, and
+   perform rename file operation.  */
+static bool
+rename_entity (FTSENT * ent, struct cp_options *x)
+{
+  char *destination;
+  bool copy_into_self;
+  bool rename_succeeded;
+  bool ok =3D true;
+
+  /* Ask for new name. */
+  if (cmd_to_run !=3D NULL)
+    {
+      destination =3D request_child (ent->fts_path);
+    }
+  else
+    {
+      destination =3D
+	construct_dest_name (ent->fts_path, original_str, replacement);
+    }
+
+  /* Perform move operation. */
+  if (test_only =3D=3D true)
+    printf ("%s -> %s\n", ent->fts_path, destination);
+  else
+    ok &=3D
+      copy (ent->fts_path, destination, false, x, &copy_into_self,
+	    &rename_succeeded);
+
+  if (ok)
+    {
+      if (copy_into_self)
+	{
+	  ok =3D false;
+	}
+    }
+  free (destination);
+  return ok;
+}
+
+/* Read entities from directory, and request renaming.  */
+static bool
+prepare_rename (char **files, int bit_flags, struct cp_options *x)
+{
+  bool ok =3D true;
+
+  if (*files !=3D NULL)
+    {
+      FTS *fts =3D xfts_open (files, bit_flags, NULL);
+
+      while (1)
+	{
+	  FTSENT *ent;
+
+	  ent =3D fts_read (fts);
+	  if (ent =3D=3D NULL)
+	    {
+	      if (errno !=3D 0)
+		{
+		  error (0, errno,
+			 _("fts_read failed: %s"),
+			 quotearg_colon (fts->fts_path));
+		  ok =3D false;
+		}
+	      break;
+	    }
+	  FTS_CROSS_CHECK (fts);
+	  if (!recurse)
+	    fts_set (fts, ent, FTS_SKIP);
+	  ok &=3D rename_entity (ent, x);
+	}
+
+      if (fts_close (fts) !=3D 0)
+	{
+	  error (0, errno, _("fts_close failed"));
+	  ok =3D false;
+	}
+    }
+
+  return ok;
+}
+
+void
+usage (int status)
+{
+  if (status !=3D EXIT_SUCCESS)
+    emit_try_help ();
+  else
+    {
+      printf (_("\
+Usage: %s [OPTION]... EXPRESSION REPLACEMENT FILE...\n\
+  or:  %s [OPTION] -0 FILE -e COMMAND [OPTION]\n\
+"), program_name, program_name);
+      fputs (_("\
+Rename lots of files in directory.\n\
+\n\
+"), stdout);
+      fputs (_("\
+  -0, --files0-from=3DFILE  read rename entries from NUL-terminated file\n=
\
+  -e, --exec=3DCOMMAND      manipulate file names by using a command\n\
+  -r, -R, --recursive     rename in directories recursively\n\
+  -x, --one-file-system   skip directories on different file systems\n\
+      --backup[=3DCONTROL]  make a backup of each existing destination fil=
e\n\
+  -b                      like --backup but does not accept an argument\n\
+  -S, --suffix=3DSUFFIX     override the usual backup suffix\n\
+  -f, --force             do not prompt before overwriting\n\
+  -i, --interactive       prompt before overwrite\n\
+  -n, --no-clobber        do not overwrite an existing file\n\
+If you specify more than one of -f, -i, -n, only the final one takes effec=
t.\n\
+  -t, --test              show a dry-run print out\n\
+  -v, --verbose           explain what is being done\n\
+"), stdout);
+      fputs (HELP_OPTION_DESCRIPTION, stdout);
+      fputs (VERSION_OPTION_DESCRIPTION, stdout);
+      /* FIXME: IMHO the messges bellow should be BACKUP_OPTION_DESCRIPTIO=
N
+         similar to help and version.  */
+      fputs (_("\
+\n\
+The backup suffix is `~', unless set with --suffix or SIMPLE_BACKUP_SUFFIX=
.\n\
+The version control method may be selected via the --backup option or
through\n\
+the VERSION_CONTROL environment variable.  Here are the values:\n\
+\n\
+"), stdout);
+      fputs (_("\
+  none, off       never make backups (even if --backup is given)\n\
+  numbered, t     make numbered backups\n\
+  existing, nil   numbered if numbered backups exist, simple otherwise\n\
+  simple, never   always make simple backups\n\
+"), stdout);
+      emit_ancillary_info ();
+    }
+  exit (status);
+}
+
+int
+main (int argc, char **argv)
+{
+  bool make_backups =3D false;
+  int i, c;
+  char *backup_suffix_string;
+  char *version_control_string =3D NULL;
+  struct cp_options x;
+  char **files;
+  char *files_from =3D NULL;
+  bool ok;
+  struct Tokens tok;
+
+  /* Bit flags that control how fts works.  */
+  int bit_flags =3D FTS_NOSTAT;
+  bit_flags |=3D FTS_PHYSICAL;
+
+  static const struct option longopts[] =3D {
+    {"files0-from", optional_argument, NULL, '0'},
+    {"backup", optional_argument, NULL, 'b'},
+    {"dereference-args", no_argument, NULL, 'D'},
+    {"exec", required_argument, NULL, 'e'},
+    {"exec+", required_argument, NULL, 'E'},
+    {"force", no_argument, NULL, 'f'},
+    {"interactive", no_argument, NULL, 'i'},
+    {"dereference", no_argument, NULL, 'L'},
+    {"no-clobber", no_argument, NULL, 'n'},
+    {"recursive", no_argument, NULL, 'r'},
+    {"suffix", required_argument, NULL, 'S'},
+    {"test", no_argument, NULL, 't'},
+    {"verbose", no_argument, NULL, 'v'},
+    {GETOPT_HELP_OPTION_DECL},
+    {GETOPT_VERSION_OPTION_DECL},
+    {NULL, 0, NULL, 0}
+  };
+
+  initialize_main (&argc, &argv);
+  set_program_name (argv[0]);
+  setlocale (LC_ALL, "");
+  bindtextdomain (PACKAGE, LOCALEDIR);
+  textdomain (PACKAGE);
+
+  atexit (close_stdin);
+
+  cp_option_init (&x);
+  recurse =3D false;
+
+  /* FIXME: consider not calling getenv for SIMPLE_BACKUP_SUFFIX unless
+     we'll actually use backup_suffix_string.  */
+  backup_suffix_string =3D getenv ("SIMPLE_BACKUP_SUFFIX");
+
+  while ((c =3D
+	  getopt_long (argc, argv, "0::b::e:finrS:tv", longopts,
+		       NULL)) !=3D -1)
+    {
+      switch (c)
+	{
+	case '0':
+	  if (files_from !=3D NULL)
+	    error (0, 0,
+		   _("warning: input file %s will not be read"),
+		   quote (files_from));
+	  if (optarg)
+	    files_from =3D optarg;
+	  else
+	    files_from =3D "-";
+	  break;
+	case 'b':
+	  make_backups =3D true;
+	  if (optarg)
+	    version_control_string =3D optarg;
+	  break;
+	case 'e':
+	  cmd_to_run =3D optarg;
+	  cmd_args =3D (argv + optind) - 1;
+	  optind =3D argc;
+	  if (files_from =3D=3D NULL)
+	    files_from =3D "-";
+	  goto options_done;
+	case 'f':
+	  x.interactive =3D I_ALWAYS_YES;
+	  break;
+	case 'i':
+	  x.interactive =3D I_ASK_USER;
+	  break;
+	case 'n':
+	  x.interactive =3D I_ALWAYS_NO;
+	  break;
+	case 'r':
+	case 'R':
+	  recurse =3D true;
+	  break;
+	case 'v':
+	  x.verbose =3D true;
+	  break;
+	case 'S':
+	  make_backups =3D true;
+	  backup_suffix_string =3D optarg;
+	  break;
+	case 't':
+	  test_only =3D true;
+	  break;
+
+	  case_GETOPT_HELP_CHAR;
+	  case_GETOPT_VERSION_CHAR (PROGRAM_NAME, AUTHORS);
+	default:
+	  usage (EXIT_FAILURE);
+	}
+    }
+options_done:
+
+  /* Check usage of backup options */
+  if (make_backups && x.interactive =3D=3D I_ALWAYS_NO)
+    {
+      error (0, 0,
+	     _("options --backup and --no-clobber are mutually exclusive"));
+      usage (EXIT_FAILURE);
+    }
+
+  if (backup_suffix_string)
+    simple_backup_suffix =3D xstrdup (backup_suffix_string);
+
+  x.backup_type =3D (make_backups
+		   ? xget_version (_("backup type"),
+				   version_control_string) : no_backups);
+
+  /* Command expects
+     ++ match and replacement arguments
+     ++ or exec option
+     accoppanied with
+     ++ file list as command line argument
+     ++ or read from file option argument
+     ++ or read stdin */
+  if (optind < argc - 1)
+    {
+      original_str =3D argv[optind];
+      replacement =3D argv[optind + 1];
+      optind +=3D 2;
+    }
+  if (original_str =3D=3D NULL && cmd_to_run =3D=3D NULL)
+    {
+      error (0, 0, _("not enough arguments"));
+      usage (EXIT_FAILURE);
+    }
+  if (optind =3D=3D argc)
+    {
+      files_from =3D "-";
+    }
+
+  hash_init ();
+  bool read_tokens =3D false;
+  struct argv_iterator *ai;
+  if (files_from)
+    {
+      FILE *stream;
+      /* When using --files0-from=3DF, you may not specify any files
+         on the command-line.  */
+      if (optind < argc)
+	{
+	  error (0, 0, _("extra operand %s"), quote (argv[optind]));
+	  fprintf (stderr, "%s\n",
+		   _("file operands cannot be combined with --files0-from"));
+	  usage (EXIT_FAILURE);
+	}
+
+      if (STREQ (files_from, "-"))
+	stream =3D stdin;
+      else
+	{
+	  stream =3D fopen (files_from, "r");
+	  if (stream =3D=3D NULL)
+	    error (EXIT_FAILURE, errno, _("cannot open %s for reading"),
+		   quote (files_from));
+	}
+
+      /* Read the file list into RAM if we can detect its size and that
+         size is reasonable.  Otherwise, we'll read a name at a time.  */
+      struct stat st;
+      if (fstat (fileno (stream), &st) =3D=3D 0
+	  && S_ISREG (st.st_mode)
+	  && st.st_size <=3D MIN (10 * 1024 * 1024, physmem_available () / 2))
+	{
+	  read_tokens =3D true;
+	  readtokens0_init (&tok);
+	  if (!readtokens0 (stream, &tok) || fclose (stream) !=3D 0)
+	    error (EXIT_FAILURE, 0, _("cannot read file names from %s"),
+		   quote (files_from));
+	  files =3D tok.tok;
+	  ai =3D argv_iter_init_argv (files);
+
+	}
+      else
+	{
+	  files =3D NULL;
+	  ai =3D argv_iter_init_stream (stream);
+	}
+    }
+  else
+    {
+      static char *stdin_only[] =3D { NULL };
+      files =3D (optind < argc ? argv + optind : stdin_only);
+      ai =3D argv_iter_init_argv (files);
+    }
+
+  if (!ai)
+    xalloc_die ();
+
+  /* prepare_rename will not find cycles on its own, so ask fts_read
+     to check for them accurately.  */
+  bit_flags |=3D FTS_TIGHT_CYCLE_CHECK;
+
+  ok =3D true;
+  for (i =3D 0; /* empty */ ; i++)
+    {
+      bool skip_file =3D false;
+      enum argv_iter_err ai_err;
+      char *input_path =3D argv_iter (ai, &ai_err);
+      if (!input_path)
+	{
+	  switch (ai_err)
+	    {
+	    case AI_ERR_EOF:
+	      goto argv_iter_done;
+	    case AI_ERR_READ:
+	      error (0, errno, _("%s: read error"),
+		     quotearg_colon (files_from));
+	      ok =3D false;
+	      goto argv_iter_done;
+	    case AI_ERR_MEM:
+	      xalloc_die ();
+	    case AI_ERR_OK:
+	    default:
+	      assert (!"unexpected error code from argv_iter");
+	    }
+	}
+
+      if (!input_path[0])
+	{
+	  /* Diagnose a zero-length file name.  When it's one
+	     among many, knowing the record number may help.  */
+	  if (files_from =3D=3D NULL)
+	    error (0, 0, "%s", _("invalid zero-length file name"));
+	  else
+	    {
+	      /* Using the standard `filename:line-number:' prefix here is
+	         not totally appropriate, since NUL is the separator, not NL,
+	         but it might be better than nothing.  */
+	      unsigned long int file_number =3D argv_iter_n_args (ai);
+	      error (0, 0, "%s:%lu: %s", quotearg_colon (files_from),
+		     file_number, _("invalid zero-length file name"));
+	    }
+	  skip_file =3D true;
+	}
+
+      if (skip_file)
+	ok =3D false;
+      else
+	{
+	  static char *temp_argv[] =3D { NULL, NULL };
+	  temp_argv[0] =3D input_path;
+	  ok &=3D prepare_rename (temp_argv, bit_flags, &x);
+	}
+    }
+argv_iter_done:
+
+  /* No command line arguments.  Read from stdin.  However, no
+     arguments with --files0-from is reason to skip this. */
+  if (ok && !files_from && argv_iter_n_args (ai) =3D=3D 0)
+    ok &=3D prepare_rename (NULL, bit_flags, &x);
+
+  if (read_tokens)
+    readtokens0_free (&tok);
+
+  argv_iter_free (ai);
+
+  if (have_read_stdin && close (STDIN_FILENO) !=3D 0)
+    error (EXIT_FAILURE, errno, "-");
+
+  exit (ok ? EXIT_SUCCESS : EXIT_FAILURE);
+}
diff --git a/tests/Makefile.am b/tests/Makefile.am
index d8bc930..d620dc8 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -226,6 +226,7 @@ TESTS =3D						\
   misc/readlink-fp-loop				\
   misc/readlink-root				\
   misc/realpath					\
+  misc/rename-foo2bar				\
   misc/runcon-no-reorder			\
   misc/sha1sum					\
   misc/sha1sum-vec				\
diff --git a/tests/misc/help-version b/tests/misc/help-version
index e19f5b1..40ee81c 100755
--- a/tests/misc/help-version
+++ b/tests/misc/help-version
@@ -178,6 +178,7 @@ ginstall_setup () { args=3D"$tmp_in $tmp_in2"; }
 mv_setup () { args=3D"$tmp_in $tmp_in2"; }
 mkdir_setup () { args=3D$tmp_dir/subdir; }
 realpath_setup () { args=3D$tmp_in; }
+rename_setup () { args=3D--version; }
 rmdir_setup () { args=3D$tmp_dir; }
 rm_setup () { args=3D$tmp_in; }
 shred_setup () { args=3D$tmp_in; }
diff --git a/tests/misc/invalid-opt b/tests/misc/invalid-opt
index 8f7024c..6e6f059 100755
--- a/tests/misc/invalid-opt
+++ b/tests/misc/invalid-opt
@@ -42,6 +42,7 @@ my %exit_status =3D
     tty =3D> 2,
     printf =3D> 0,
     printenv =3D> 2,
+    rename =3D> 1,
   );

 my %expected_out =3D
diff --git a/tests/misc/rename-foo2bar b/tests/misc/rename-foo2bar
new file mode 100755
index 0000000..0760f6f
--- /dev/null
+++ b/tests/misc/rename-foo2bar
@@ -0,0 +1,30 @@
+#!/bin/sh
+# test rename basic functionality
+
+# Copyright (C) 2012 Free Software Foundation, Inc.
+
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+. "${srcdir=3D.}/init.sh"; path_prepend_ ../src
+print_ver_ rename
+
+for i in foo1 foo2 ; do
+  echo a > $i || framework_failure
+done
+
+rename foo bar foo? 2>/dev/null || fail=3D1
+test -f bar1 || fail=3D1
+test -f bar2 || fail=3D1
+
+Exit $fail

--=20
=A0=A0 Sami Kerola
=A0=A0 http://www.iki.fi/kerolasa/


From MAILER-DAEMON Sat Jun 09 16:36:05 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1SdSOD-0006Ks-HT
	for mharc-coreutils@gnu.org; Sat, 09 Jun 2012 16:36:05 -0400
Received: from eggs.gnu.org ([208.118.235.92]:33833)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <kerolasa@gmail.com>) id 1SdSOB-0006Jj-4K
	for coreutils@gnu.org; Sat, 09 Jun 2012 16:36:04 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <kerolasa@gmail.com>) id 1SdSO9-00053M-Hy
	for coreutils@gnu.org; Sat, 09 Jun 2012 16:36:02 -0400
Received: from mail-wg0-f49.google.com ([74.125.82.49]:45473)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <kerolasa@gmail.com>) id 1SdSO9-00053H-9A
	for coreutils@gnu.org; Sat, 09 Jun 2012 16:36:01 -0400
Received: by wgbds1 with SMTP id ds1so1396100wgb.30
	for <coreutils@gnu.org>; Sat, 09 Jun 2012 13:35:59 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
	h=sender:from:to:cc:subject:date:message-id:x-mailer:in-reply-to
	:references; bh=MVEvFjlB0DSd33r1JHtx6cxXG9Hx9xe8uuKTv+guaQM=;
	b=sVVmDEb/k05oGZetx366qGnBTOJNNt7t9YfOxYoDqryi4zATJAkUh6S0Hv95xsbHBQ
	kP5k7I9eG7Iqa/JjAO+d7GVoXaKrfM/wxPxdV6+jcvhUKKU4EB9k6cJTjb8MKnF1+R6u
	yrTOPF4zL+OIsJ+nxG0o3Y72CPGa1nhHis1Lg9NojblopT7hUdQGTgASZsJ0C7KOd9IZ
	2Wb6YxJ5fWgR9sJbSC6TcTaDoJt9PZyMBFFyRhte+BMOdd+4z3ivl/XBU763RpFhYM8h
	sNfbdw8UImuL1Gzs307ERJ/8KeUiSF3CGEx63psJtbETQRAvXkr7tJITt78vyX2e9bnA
	oBjA==
Received: by 10.216.226.78 with SMTP id a56mr3630106weq.133.1339274158216;
	Sat, 09 Jun 2012 13:35:58 -0700 (PDT)
Received: from localhost (i53087.upc-i.chello.nl. [62.195.53.87])
	by mx.google.com with ESMTPS id ch9sm18547985wib.8.2012.06.09.13.35.57
	(version=TLSv1/SSLv3 cipher=OTHER);
	Sat, 09 Jun 2012 13:35:57 -0700 (PDT)
Sender: Sami Kerola <kerolasa@gmail.com>
From: Sami Kerola <kerolasa@iki.fi>
To: coreutils@gnu.org
Subject: [PATCH] maint: additional su removal
Date: Sat,  9 Jun 2012 22:32:17 +0200
Message-Id: <1339273937-15171-1-git-send-email-kerolasa@iki.fi>
X-Mailer: git-send-email 1.7.10.4
In-Reply-To: <8762b55w3m.fsf@rho.meyering.net>
	<87396818jo.fsf@rho.meyering.net>
References: <87396818jo.fsf@rho.meyering.net>
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 74.125.82.49
Cc: kerolasa@iki.fi
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sat, 09 Jun 2012 20:36:04 -0000


* tests/misc/help-version: Remove expected su exit code.
---
 tests/misc/help-version |    1 -
 1 file changed, 1 deletion(-)

diff --git a/tests/misc/help-version b/tests/misc/help-version
index e19f5b1..4563625 100755
--- a/tests/misc/help-version
+++ b/tests/misc/help-version
@@ -30,7 +30,6 @@ expected_failure_status_env=125
 expected_failure_status_nice=125
 expected_failure_status_nohup=125
 expected_failure_status_stdbuf=125
-expected_failure_status_su=125
 expected_failure_status_timeout=125
 expected_failure_status_printenv=2
 expected_failure_status_tty=3
-- 
1.7.10.4



From MAILER-DAEMON Sun Jun 10 16:17:05 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1SdoZN-0003oh-4f
	for mharc-coreutils@gnu.org; Sun, 10 Jun 2012 16:17:05 -0400
Received: from eggs.gnu.org ([208.118.235.92]:48169)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1SdoZK-0003oa-O0
	for coreutils@gnu.org; Sun, 10 Jun 2012 16:17:03 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1SdoZI-00020K-Sj
	for coreutils@gnu.org; Sun, 10 Jun 2012 16:17:02 -0400
Received: from mx.meyering.net ([88.168.87.75]:50269)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1SdoZI-00020D-Kq
	for coreutils@gnu.org; Sun, 10 Jun 2012 16:17:00 -0400
Received: from rho.meyering.net (localhost.localdomain [127.0.0.1])
	by rho.meyering.net (Acme Bit-Twister) with ESMTP id B56EA600BD
	for <coreutils@gnu.org>; Sun, 10 Jun 2012 22:16:58 +0200 (CEST)
From: Jim Meyering <jim@meyering.net>
To: coreutils@gnu.org
Subject: [PATCH] build: update gnulib to latest; correct comment grammar
Date: Sun, 10 Jun 2012 22:16:58 +0200
Message-ID: <87d356c479.fsf@rho.meyering.net>
Lines: 52
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 88.168.87.75
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 10 Jun 2012 20:17:04 -0000

FYI,
(plus, I've pushed the two strncpy-related patches I posted before
and updated the gnulib-updating one into a newer version that also
corrects for the new grammar "all these" syntax-check rule)

>From f378f1c51cf4784ec0b0f0e7233319924e1283b2 Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering@redhat.com>
Date: Thu, 24 May 2012 12:32:41 +0200
Subject: [PATCH] build: update gnulib to latest; correct comment grammar

* tests/misc/help-version: Fix comment grammar:
s/all these/all of these/
* gl/lib/tempname.c.diff: Likewise.
---
 gl/lib/tempname.c.diff  | 2 +-
 gnulib                  | 2 +-
 tests/misc/help-version | 2 +-
 3 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/gl/lib/tempname.c.diff b/gl/lib/tempname.c.diff
index 3e30c97..b1c2236 100644
--- a/gl/lib/tempname.c.diff
+++ b/gl/lib/tempname.c.diff
@@ -64,7 +64,7 @@ index 2da5afe..562955a 100644
    /* A lower bound on the number of temporary files to attempt to
       generate.  The maximum total number of temporary file names that
       can exist for a given template is 62**6.  It should never be
-      necessary to try all these combinations.  Instead if a reasonable
+      necessary to try all of these combinations.  Instead if a reasonable
       number of names is tried (we define reasonable as 62**3) fail to
 -     give the system administrator the chance to remove the problems.  */
 +     give the system administrator the chance to remove the problems.
diff --git a/gnulib b/gnulib
index a3a0496..70a9bd6 160000
--- a/gnulib
+++ b/gnulib
@@ -1 +1 @@
-Subproject commit a3a049672d410b1deba8f1537016f2ece42b4e88
+Subproject commit 70a9bd65b77ec4e516f500e7348bc1c332d91ea5
diff --git a/tests/misc/help-version b/tests/misc/help-version
index e19f5b1..4a82b04 100755
--- a/tests/misc/help-version
+++ b/tests/misc/help-version
@@ -1,5 +1,5 @@
 #! /bin/sh
-# Make sure all these programs work properly
+# Make sure all of these programs work properly
 # when invoked with --help or --version.

 # Copyright (C) 2000-2012 Free Software Foundation, Inc.
--
1.7.11.rc2


From MAILER-DAEMON Sun Jun 10 16:24:11 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1SdogF-0004QU-6o
	for mharc-coreutils@gnu.org; Sun, 10 Jun 2012 16:24:11 -0400
Received: from eggs.gnu.org ([208.118.235.92]:53547)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1SdogC-0004Q0-Ls
	for coreutils@gnu.org; Sun, 10 Jun 2012 16:24:09 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1SdogA-0003Ok-Ue
	for coreutils@gnu.org; Sun, 10 Jun 2012 16:24:08 -0400
Received: from mx.meyering.net ([88.168.87.75]:50290)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1SdogA-0003O9-Cr
	for coreutils@gnu.org; Sun, 10 Jun 2012 16:24:06 -0400
Received: from rho.meyering.net (localhost.localdomain [127.0.0.1])
	by rho.meyering.net (Acme Bit-Twister) with ESMTP id 6061A60156;
	Sun, 10 Jun 2012 22:24:03 +0200 (CEST)
From: Jim Meyering <jim@meyering.net>
To: Sami Kerola <kerolasa@iki.fi>
Subject: Re: [PATCH] maint: additional su removal
In-Reply-To: <1339273937-15171-1-git-send-email-kerolasa@iki.fi> (Sami
	Kerola's message of "Sat, 9 Jun 2012 22:32:17 +0200")
References: <87396818jo.fsf@rho.meyering.net>
	<1339273937-15171-1-git-send-email-kerolasa@iki.fi>
Date: Sun, 10 Jun 2012 22:24:03 +0200
Message-ID: <877gvec3vg.fsf@rho.meyering.net>
Lines: 20
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 88.168.87.75
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 10 Jun 2012 20:24:09 -0000

Sami Kerola wrote:
> * tests/misc/help-version: Remove expected su exit code.
> ---
>  tests/misc/help-version |    1 -
>  1 file changed, 1 deletion(-)
>
> diff --git a/tests/misc/help-version b/tests/misc/help-version
> index e19f5b1..4563625 100755
> --- a/tests/misc/help-version
> +++ b/tests/misc/help-version
> @@ -30,7 +30,6 @@ expected_failure_status_env=125
>  expected_failure_status_nice=125
>  expected_failure_status_nohup=125
>  expected_failure_status_stdbuf=125
> -expected_failure_status_su=125

Thanks.
I've pushed that with a tweaked subject:

    maint: remove su testing artifact


From MAILER-DAEMON Sun Jun 10 16:36:07 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1Sdorn-0008N1-5i
	for mharc-coreutils@gnu.org; Sun, 10 Jun 2012 16:36:07 -0400
Received: from eggs.gnu.org ([208.118.235.92]:41103)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Sdork-0008Mt-RA
	for coreutils@gnu.org; Sun, 10 Jun 2012 16:36:06 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Sdorj-0006Ki-4o
	for coreutils@gnu.org; Sun, 10 Jun 2012 16:36:04 -0400
Received: from mx.meyering.net ([88.168.87.75]:50318)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Sdori-0006Ka-TQ
	for coreutils@gnu.org; Sun, 10 Jun 2012 16:36:03 -0400
Received: from rho.meyering.net (localhost.localdomain [127.0.0.1])
	by rho.meyering.net (Acme Bit-Twister) with ESMTP id DA739600BD;
	Sun, 10 Jun 2012 22:36:00 +0200 (CEST)
From: Jim Meyering <jim@meyering.net>
To: kerolasa@gmail.com
Subject: Re: rename: move command from util-linux to coreutils
In-Reply-To: <CAG27Bk0dwWn9L8MJ8x5TBchNvZ5CY2y+ATeAmypDqZi1Y8esZw@mail.gmail.com>
	(Sami Kerola's message of "Sat, 9 Jun 2012 22:19:23 +0200")
References: <CAG27Bk0dwWn9L8MJ8x5TBchNvZ5CY2y+ATeAmypDqZi1Y8esZw@mail.gmail.com>
Date: Sun, 10 Jun 2012 22:36:00 +0200
Message-ID: <871ulmc3bj.fsf@rho.meyering.net>
Lines: 42
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 88.168.87.75
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 10 Jun 2012 20:36:06 -0000

Sami Kerola wrote:
> The next coreutils and util-linux releases are going to be made close
> to each others to avoid 'su' coming from multiple source, or being
> missing.  At same go moving 'rename' could make sense.
>
> I completely rewrote the command as the util-linux way doing things
> would not be a strong match how code is crafted in coreutils.  Primary
> difference is, as well known, usage of gnulib and a little bit
> different coding style.
>
> Please find my proposal how the new rename might be implemented.  I am
> almost sure the merge should not happen as is, as there are some cases
> which might not work correctly; for example directories are not
> renamed, nor files moved out of directories.  Perhaps that is right, or
> wrong.  Comments are definitely welcome.
>
> Another decision which might be wrong is stdin file list handling.
> Command, by default, expects null terminated strings and nothing else
> will work.  One might argue the command should, as xargs, allow white
> space terminated input.  IMHO allowing white space termination
> encourages users to use error prone input style, and therefore it is
> not justified to be default nor supported in modern commands.  But I
> could overlook something, so maybe this needs to be discussed as well.
>
> Finally, without any doubt I wrote half of English somehow wrong so human
> language review is highly recommendable.

Thanks for the suggestion.
Have you already discussed this on the util-linux list and/or
with Karel Zak?

Here's some quick feedback after a very cursory look:
I see unchecked syscalls, from write and waitpid to close and dup2.

Style nits: I saw "TYPE * var_name" in at least one declaration.
I think it was a parameter list.
It should be "TYPE *var_name".

I also saw a TAB or two used in indentation.  Use only spaces.
If you run "make syntax-check", it will show you where.

Providing more test coverage would be most useful/welcome.


From MAILER-DAEMON Sun Jun 10 17:25:36 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1Sdpdg-00084P-G4
	for mharc-coreutils@gnu.org; Sun, 10 Jun 2012 17:25:36 -0400
Received: from eggs.gnu.org ([208.118.235.92]:32996)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <kerolasa@gmail.com>) id 1Sdpdd-00083Z-Kk
	for coreutils@gnu.org; Sun, 10 Jun 2012 17:25:34 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <kerolasa@gmail.com>) id 1Sdpdb-00011I-NR
	for coreutils@gnu.org; Sun, 10 Jun 2012 17:25:33 -0400
Received: from mail-we0-f169.google.com ([74.125.82.169]:62254)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <kerolasa@gmail.com>) id 1Sdpdb-00010q-Dp
	for coreutils@gnu.org; Sun, 10 Jun 2012 17:25:31 -0400
Received: by wefh52 with SMTP id h52so2363276wef.0
	for <coreutils@gnu.org>; Sun, 10 Jun 2012 14:25:28 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
	h=mime-version:reply-to:sender:in-reply-to:references:date
	:x-google-sender-auth:message-id:subject:from:to:cc:content-type
	:content-transfer-encoding;
	bh=f7HS5J6/Hq8YlasiioOxRnBdI0Ly3NSqyxsE0Pn/2YQ=;
	b=KjRN5896qw2hAZng0IIbwj1Ws9gV7Xxxm6PyBYzJ71KPVmUAjaaocXKjdcArl5JJnG
	78mY18U3aO0GwOFVEtIPVKDRv6yAr+ku2ho5CT7qDdPRyeqetUGNHnTg3Op/CEV2pai9
	5AD9f2TpmAhu8TlDKbdmzeD6N1BkzuaWgWmBz7Fvp1WBAiJQoITQ7XxuKFDnl2U/maER
	rsKmu+zYxdbOe72qC6dnqcsJ9Pm8j4MAPeuNhDFa+H3DN9kKpDOeF3Db+tlz3x4NW6CA
	hVSBUUvcjK1PPENHAe7CN8wHlhFJdxrSP49sMcJfOLUdvGHGg3dkoHmtngmejlML/qVs
	oGJg==
MIME-Version: 1.0
Received: by 10.216.207.67 with SMTP id m45mr3706782weo.175.1339363528695;
	Sun, 10 Jun 2012 14:25:28 -0700 (PDT)
Sender: kerolasa@gmail.com
Received: by 10.194.56.10 with HTTP; Sun, 10 Jun 2012 14:25:28 -0700 (PDT)
In-Reply-To: <871ulmc3bj.fsf@rho.meyering.net>
References: <CAG27Bk0dwWn9L8MJ8x5TBchNvZ5CY2y+ATeAmypDqZi1Y8esZw@mail.gmail.com>
	<871ulmc3bj.fsf@rho.meyering.net>
Date: Sun, 10 Jun 2012 23:25:28 +0200
X-Google-Sender-Auth: DbiMZLQAUcw6sXrg5VRFDvxBSE0
Message-ID: <CAG27Bk0VuuGbe2nC9-gW+bEoGjA3v6Lw30V-enUSNAHT5cTbfw@mail.gmail.com>
Subject: Re: rename: move command from util-linux to coreutils
From: Sami Kerola <kerolasa@iki.fi>
To: Jim Meyering <jim@meyering.net>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 74.125.82.169
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
Reply-To: kerolasa@gmail.com
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 10 Jun 2012 21:25:35 -0000

On Sun, Jun 10, 2012 at 10:36 PM, Jim Meyering <jim@meyering.net> wrote:

Hi Jim et.al.

> Thanks for the suggestion.
> Have you already discussed this on the util-linux list and/or
> with Karel Zak?

Yes I did. As a matter of fact the proposal is rather old. First time
thought moving the command out of util-linux to coreutils was at
summer 2011, when few mails where sent back and forth.

https://lists.gnu.org/archive/html/coreutils/2011-06/msg00080.html

Now when su moved I thought it's good moment to try doing this as well.

> Here's some quick feedback after a very cursory look:
> I see unchecked syscalls, from write and waitpid to close and dup2.

I'll correct that to my git tomorrow.

> Style nits: I saw "TYPE * var_name" in at least one declaration.
> I think it was a parameter list.
> It should be "TYPE *var_name".

Uh, oh. I cannot trust 'indent -gnu rename.c' fixing everything. I
will fix this at same go with unchecked syscalls.

> I also saw a TAB or two used in indentation. =A0Use only spaces.
> If you run "make syntax-check", it will show you where.

Same for this.

> Providing more test coverage would be most useful/welcome.

I will write at least --exec test.

--=20
=A0=A0 Sami Kerola
=A0=A0 http://www.iki.fi/kerolasa/


From MAILER-DAEMON Mon Jun 11 02:26:22 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1Sdy50-0006h7-20
	for mharc-coreutils@gnu.org; Mon, 11 Jun 2012 02:26:22 -0400
Received: from eggs.gnu.org ([208.118.235.92]:39653)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Sdy4x-0006fz-F9
	for coreutils@gnu.org; Mon, 11 Jun 2012 02:26:20 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Sdy4v-0005gW-2p
	for coreutils@gnu.org; Mon, 11 Jun 2012 02:26:19 -0400
Received: from mx.meyering.net ([88.168.87.75]:51752)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Sdy4u-0005gG-RF
	for coreutils@gnu.org; Mon, 11 Jun 2012 02:26:17 -0400
Received: from rho.meyering.net (localhost.localdomain [127.0.0.1])
	by rho.meyering.net (Acme Bit-Twister) with ESMTP id B797E6004B;
	Mon, 11 Jun 2012 08:26:14 +0200 (CEST)
From: Jim Meyering <jim@meyering.net>
To: kerolasa@gmail.com
Subject: Re: rename: move command from util-linux to coreutils
In-Reply-To: <CAG27Bk0VuuGbe2nC9-gW+bEoGjA3v6Lw30V-enUSNAHT5cTbfw@mail.gmail.com>
	(Sami Kerola's message of "Sun, 10 Jun 2012 23:25:28 +0200")
References: <CAG27Bk0dwWn9L8MJ8x5TBchNvZ5CY2y+ATeAmypDqZi1Y8esZw@mail.gmail.com>
	<871ulmc3bj.fsf@rho.meyering.net>
	<CAG27Bk0VuuGbe2nC9-gW+bEoGjA3v6Lw30V-enUSNAHT5cTbfw@mail.gmail.com>
Date: Mon, 11 Jun 2012 08:26:14 +0200
Message-ID: <87k3ze9xfd.fsf@rho.meyering.net>
Lines: 45
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 88.168.87.75
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 11 Jun 2012 06:26:21 -0000

Sami Kerola wrote:

> On Sun, Jun 10, 2012 at 10:36 PM, Jim Meyering <jim@meyering.net> wrote:
>
> Hi Jim et.al.
>
>> Thanks for the suggestion.
>> Have you already discussed this on the util-linux list and/or
>> with Karel Zak?
>
> Yes I did. As a matter of fact the proposal is rather old. First time
> thought moving the command out of util-linux to coreutils was at
> summer 2011, when few mails where sent back and forth.
>
> https://lists.gnu.org/archive/html/coreutils/2011-06/msg00080.html
>
> Now when su moved I thought it's good moment to try doing this as well.
>
>> Here's some quick feedback after a very cursory look:
>> I see unchecked syscalls, from write and waitpid to close and dup2.
>
> I'll correct that to my git tomorrow.
>
>> Style nits: I saw "TYPE * var_name" in at least one declaration.
>> I think it was a parameter list.
>> It should be "TYPE *var_name".
>
> Uh, oh. I cannot trust 'indent -gnu rename.c' fixing everything. I
> will fix this at same go with unchecked syscalls.

With GNU indent, you have to specify types that are unknown
to it via e.g., -Tuintmax_t

>> I also saw a TAB or two used in indentation. =A0Use only spaces.
>> If you run "make syntax-check", it will show you where.
>
> Same for this.

Use indent --no-tabs

>> Providing more test coverage would be most useful/welcome.
>
> I will write at least --exec test.

Good.  Thanks!


From MAILER-DAEMON Mon Jun 11 02:30:57 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1Sdy9R-0007ay-Uo
	for mharc-coreutils@gnu.org; Mon, 11 Jun 2012 02:30:57 -0400
Received: from eggs.gnu.org ([208.118.235.92]:47715)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Sdy9P-0007an-5L
	for coreutils@gnu.org; Mon, 11 Jun 2012 02:30:56 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Sdy9N-0006Xd-9L
	for coreutils@gnu.org; Mon, 11 Jun 2012 02:30:54 -0400
Received: from mx.meyering.net ([88.168.87.75]:51765)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Sdy9N-0006XZ-2N
	for coreutils@gnu.org; Mon, 11 Jun 2012 02:30:53 -0400
Received: from rho.meyering.net (localhost.localdomain [127.0.0.1])
	by rho.meyering.net (Acme Bit-Twister) with ESMTP id C70BD6004B;
	Mon, 11 Jun 2012 08:30:51 +0200 (CEST)
From: Jim Meyering <jim@meyering.net>
To: Paul Eggert <eggert@cs.ucla.edu>
Subject: Re: overly aggressive memory usage by sort.c
In-Reply-To: <4FD1431C.5010500@draigBrady.com> (=?iso-8859-1?Q?=22P=E1drai?=
	=?iso-8859-1?Q?g?= Brady"'s message of
	"Fri, 08 Jun 2012 01:11:08 +0100")
References: <CAMkU=1zDA=RFzxb-O_x2TG+EumrdUebZJb4UOJAM_xretGpCCA@mail.gmail.com>
	<4FD1431C.5010500@draigBrady.com>
Date: Mon, 11 Jun 2012 08:30:51 +0200
Message-ID: <878vfu9x7o.fsf@rho.meyering.net>
Lines: 50
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 88.168.87.75
Cc: Jeff Janes <jeff.janes@gmail.com>, coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 11 Jun 2012 06:30:56 -0000

P=E1draig Brady wrote:
> On 06/07/2012 09:12 PM, Jeff Janes wrote:
>> In commit a507ed6ede5064b8f15c979e54e6de3bb478d73e, first appearing in
>> v8.16, the default memory usage was changed to take all of available
>> memory, rather than half of it.  I think this is too aggressive.
>>
>> If I start a very large file sort on a previously idle server, it will
>> report almost all physical memory as being available, and so sort will
>> take all of it.  But as soon as the heavy IO (reading the data to be
>> sorted, writing temp files) starts up, the kernel needs more memory
>> for buffering in order to make the IO efficient.  The kernel and the
>> sort start competing for memory, a little bit of paging/swapping
>> starts, time in iowait increases, and the overall sort performance
>> drops by roughly a factor of 2.
>>
>> I don't know what the correct proportion of available memory to take
>> would be, but I think it is >0.5 and <1.0.  Maybe 0.75.  But I think
>> that just going back to 0.5 would be better than the status quo.  Or
>> perhaps the upper limit clamp could be based on physical memory
>> instead of available, so rather than:
>>
>> mem =3D MAX (avail, total / 8);
>>
>> maybe:
>>
>> mem =3D MIN(total/4*3, MAX (avail, total / 8));
>
> I have to agree. In general patches like this shouldn't
> go in without extensive performance testing.
>
> The thread discussing the patch is here:
> http://bugs.gnu.org/10877
>
> There are other things we might consider with external files.
>
> - ensure they're written to disk rather than ram.
> I.E. avoid /tmp if it's tmpfs as is becoming more common on systems
>
> - use posix_fadvise as is done in dd, to mark the external files
> as non cachable, as the only reason you'd be using them is when
> you don't have enough RAM anyway.
>
> The above considers a 2 level memory hierarchy.
> Increasingly though the "memory wall" is an issue,
> so a 3 level hierarchy involving increasingly large CPU caches
> should be considered.

Hi Paul,

Any objection to reverting that and adjusting the comment to match?


From MAILER-DAEMON Mon Jun 11 06:02:57 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1Se1Sb-0004my-4v
	for mharc-coreutils@gnu.org; Mon, 11 Jun 2012 06:02:57 -0400
Received: from eggs.gnu.org ([208.118.235.92]:36188)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eggert@cs.ucla.edu>) id 1Se1SY-0004ma-Pu
	for coreutils@gnu.org; Mon, 11 Jun 2012 06:02:56 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eggert@cs.ucla.edu>) id 1Se1ST-0002v6-Nd
	for coreutils@gnu.org; Mon, 11 Jun 2012 06:02:54 -0400
Received: from smtp.cs.ucla.edu ([131.179.128.62]:56662)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eggert@cs.ucla.edu>) id 1Se1ST-0002uo-Fv
	for coreutils@gnu.org; Mon, 11 Jun 2012 06:02:49 -0400
Received: from localhost (localhost.localdomain [127.0.0.1])
	by smtp.cs.ucla.edu (Postfix) with ESMTP id 44210A60002;
	Mon, 11 Jun 2012 03:02:47 -0700 (PDT)
X-Virus-Scanned: amavisd-new at smtp.cs.ucla.edu
Received: from smtp.cs.ucla.edu ([127.0.0.1])
	by localhost (smtp.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10024)
	with ESMTP id CS+bVrCAUDy6; Mon, 11 Jun 2012 03:02:46 -0700 (PDT)
Received: from [192.168.1.10] (pool-71-189-109-235.lsanca.fios.verizon.net
	[71.189.109.235])
	by smtp.cs.ucla.edu (Postfix) with ESMTPSA id 8A81A39E8008;
	Mon, 11 Jun 2012 03:02:46 -0700 (PDT)
Message-ID: <4FD5C249.7090907@cs.ucla.edu>
Date: Mon, 11 Jun 2012 03:02:49 -0700
From: Paul Eggert <eggert@cs.ucla.edu>
Organization: UCLA Computer Science Department
User-Agent: Mozilla/5.0 (X11; Linux i686;
	rv:12.0) Gecko/20120430 Thunderbird/12.0.1
MIME-Version: 1.0
To: Jim Meyering <jim@meyering.net>
Subject: Re: overly aggressive memory usage by sort.c
References: <CAMkU=1zDA=RFzxb-O_x2TG+EumrdUebZJb4UOJAM_xretGpCCA@mail.gmail.com>
	<4FD1431C.5010500@draigBrady.com> <878vfu9x7o.fsf@rho.meyering.net>
In-Reply-To: <878vfu9x7o.fsf@rho.meyering.net>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
X-Received-From: 131.179.128.62
Cc: Jeff Janes <jeff.janes@gmail.com>, coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 11 Jun 2012 10:02:56 -0000

On 06/10/2012 11:30 PM, Jim Meyering wrote:
> Any objection to reverting that and adjusting the comment to match?

Yes, as this would run afoul of bug 10877
<http://bugs.gnu.org/10877>.  Instead,
I suggest following Jeff Janes's advice
and installing the following patch:

>From c498519fcf4e4a2cc93c1ca75d0b263e215fc01a Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert@cs.ucla.edu>
Date: Mon, 11 Jun 2012 02:47:05 -0700
Subject: [PATCH] sort: by default, do not exceed 3/4 of physical memory

* src/sort.c (default_sort_size): Do not exceed 3/4 of total memory.
See Jeff Janes's bug report in
<http://lists.gnu.org/archive/html/coreutils/2012-06/msg00018.html>.
---
 src/sort.c |    9 ++++++---
 1 files changed, 6 insertions(+), 3 deletions(-)

diff --git a/src/sort.c b/src/sort.c
index 2593a2a..a517f4e 100644
--- a/src/sort.c
+++ b/src/sort.c
@@ -1412,10 +1412,13 @@ default_sort_size (void)
   double mem = MAX (avail, total / 8);
   struct rlimit rlimit;
 
-  /* Let SIZE be MEM, but no more than the maximum object size or
-     system resource limits.  Don't bother to check for values like
-     RLIM_INFINITY since in practice they are not much less than SIZE_MAX.  */
+  /* Let SIZE be MEM, but no more than the maximum object size, 3/4 of
+     total memory, or system resource limits.  Don't bother to check
+     for values like RLIM_INFINITY since in practice they are not much
+     less than SIZE_MAX.  */
   size_t size = SIZE_MAX;
+  if (total * 0.75 < size)
+    size = total * 0.75;
   if (getrlimit (RLIMIT_DATA, &rlimit) == 0 && rlimit.rlim_cur < size)
     size = rlimit.rlim_cur;
 #ifdef RLIMIT_AS
-- 
1.7.6.5



From MAILER-DAEMON Mon Jun 11 06:58:53 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1Se2Kj-0000uc-Dh
	for mharc-coreutils@gnu.org; Mon, 11 Jun 2012 06:58:53 -0400
Received: from eggs.gnu.org ([208.118.235.92]:59861)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eggert@cs.ucla.edu>) id 1Se2Kg-0000uL-3C
	for coreutils@gnu.org; Mon, 11 Jun 2012 06:58:51 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eggert@cs.ucla.edu>) id 1Se2Kb-0008GO-1P
	for coreutils@gnu.org; Mon, 11 Jun 2012 06:58:49 -0400
Received: from smtp.cs.ucla.edu ([131.179.128.62]:51436)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eggert@cs.ucla.edu>) id 1Se2Ka-0008GK-PR
	for coreutils@gnu.org; Mon, 11 Jun 2012 06:58:44 -0400
Received: from localhost (localhost.localdomain [127.0.0.1])
	by smtp.cs.ucla.edu (Postfix) with ESMTP id 48227A60004;
	Mon, 11 Jun 2012 03:58:43 -0700 (PDT)
X-Virus-Scanned: amavisd-new at smtp.cs.ucla.edu
Received: from smtp.cs.ucla.edu ([127.0.0.1])
	by localhost (smtp.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10024)
	with ESMTP id 6hnL7fN9MUiQ; Mon, 11 Jun 2012 03:58:42 -0700 (PDT)
Received: from [192.168.1.10] (pool-71-189-109-235.lsanca.fios.verizon.net
	[71.189.109.235])
	by smtp.cs.ucla.edu (Postfix) with ESMTPSA id 6BE3CA60002;
	Mon, 11 Jun 2012 03:58:42 -0700 (PDT)
Message-ID: <4FD5CF65.7000609@cs.ucla.edu>
Date: Mon, 11 Jun 2012 03:58:45 -0700
From: Paul Eggert <eggert@cs.ucla.edu>
Organization: UCLA Computer Science Department
User-Agent: Mozilla/5.0 (X11; Linux i686;
	rv:12.0) Gecko/20120430 Thunderbird/12.0.1
MIME-Version: 1.0
To: Jim Meyering <jim@meyering.net>
Subject: Re: overly aggressive memory usage by sort.c
References: <CAMkU=1zDA=RFzxb-O_x2TG+EumrdUebZJb4UOJAM_xretGpCCA@mail.gmail.com>
	<4FD1431C.5010500@draigBrady.com> <878vfu9x7o.fsf@rho.meyering.net>
	<4FD5C249.7090907@cs.ucla.edu>
In-Reply-To: <4FD5C249.7090907@cs.ucla.edu>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
X-Received-From: 131.179.128.62
Cc: Jeff Janes <jeff.janes@gmail.com>, coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 11 Jun 2012 10:58:51 -0000

On 06/11/2012 03:02 AM, Paul Eggert wrote:
> * src/sort.c (default_sort_size): Do not exceed 3/4 of total memory

Come to think of it, that patch is too conservative, as it
limits things to at most 3/8 of total memory.  The 3/4 limit
should be applied after 'size' is halved, not before.  Here's
a better patch:

>From dcfc5f8e6373dc7afe8e75c837dd81bbf724bb55 Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert@cs.ucla.edu>
Date: Mon, 11 Jun 2012 02:47:05 -0700
Subject: [PATCH] sort: by default, do not exceed 3/4 of physical memory

* src/sort.c (default_sort_size): Do not exceed 3/4 of total memory.
See Jeff Janes's bug report in
<http://lists.gnu.org/archive/html/coreutils/2012-06/msg00018.html>.
---
 src/sort.c |   11 ++++++++---
 1 files changed, 8 insertions(+), 3 deletions(-)

diff --git a/src/sort.c b/src/sort.c
index 2593a2a..f21e092 100644
--- a/src/sort.c
+++ b/src/sort.c
@@ -1412,9 +1412,10 @@ default_sort_size (void)
   double mem = MAX (avail, total / 8);
   struct rlimit rlimit;
 
-  /* Let SIZE be MEM, but no more than the maximum object size or
-     system resource limits.  Don't bother to check for values like
-     RLIM_INFINITY since in practice they are not much less than SIZE_MAX.  */
+  /* Let SIZE be MEM, but no more than the maximum object size,
+     total memory, or system resource limits.  Don't bother to check
+     for values like RLIM_INFINITY since in practice they are not much
+     less than SIZE_MAX.  */
   size_t size = SIZE_MAX;
   if (getrlimit (RLIMIT_DATA, &rlimit) == 0 && rlimit.rlim_cur < size)
     size = rlimit.rlim_cur;
@@ -1434,6 +1435,10 @@ default_sort_size (void)
     size = rlimit.rlim_cur / 16 * 15;
 #endif
 
+  /* Similarly, leave a 1/4 margin for physical memory.  */
+  if (total * 0.75 < size)
+    size = total * 0.75;
+
   /* Return the minimum of MEM and SIZE, but no less than
      MIN_SORT_SIZE.  Avoid the MIN macro here, as it is not quite
      right when only one argument is floating point.  */
-- 
1.7.6.5



From MAILER-DAEMON Mon Jun 11 07:47:34 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1Se35q-0007qK-VN
	for mharc-coreutils@gnu.org; Mon, 11 Jun 2012 07:47:34 -0400
Received: from eggs.gnu.org ([208.118.235.92]:33403)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <bernhard.voelker@siemens-enterprise.com>)
	id 1Se35o-0007pn-CO
	for coreutils@gnu.org; Mon, 11 Jun 2012 07:47:33 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <bernhard.voelker@siemens-enterprise.com>)
	id 1Se35m-00020a-BM
	for coreutils@gnu.org; Mon, 11 Jun 2012 07:47:31 -0400
Received: from senmx11-mx.siemens-enterprise.com ([62.134.46.9]:49015)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <bernhard.voelker@siemens-enterprise.com>)
	id 1Se35m-000205-4F
	for coreutils@gnu.org; Mon, 11 Jun 2012 07:47:30 -0400
Received: from MCHP01HTC.global-ad.net (unknown [172.29.42.234])
	by senmx11-mx.siemens-enterprise.com (Server) with ESMTP id
	CCC5F1EB841B; Mon, 11 Jun 2012 13:47:27 +0200 (CEST)
Received: from MCHP02MSX.global-ad.net ([169.254.4.138]) by
	MCHP01HTC.global-ad.net ([172.29.42.234]) with mapi id 14.01.0339.001;
	Mon, 11 Jun 2012 13:47:27 +0200
From: "Voelker, Bernhard" <bernhard.voelker@siemens-enterprise.com>
To: Paul Eggert <eggert@cs.ucla.edu>, Jim Meyering <jim@meyering.net>
Subject: RE: overly aggressive memory usage by sort.c
Thread-Topic: overly aggressive memory usage by sort.c
Thread-Index: AQHNR8EyhIj6QL2KxEuV3UZms5Xgypb0/igg
Date: Mon, 11 Jun 2012 11:47:26 +0000
Message-ID: <A75C39900FCF0B42A2CE8B26F837D2C3010923@MCHP02MSX.global-ad.net>
References: <CAMkU=1zDA=RFzxb-O_x2TG+EumrdUebZJb4UOJAM_xretGpCCA@mail.gmail.com>
	<4FD1431C.5010500@draigBrady.com> <878vfu9x7o.fsf@rho.meyering.net>
	<4FD5C249.7090907@cs.ucla.edu> <4FD5CF65.7000609@cs.ucla.edu>
In-Reply-To: <4FD5CF65.7000609@cs.ucla.edu>
Accept-Language: de-DE, en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [172.28.132.35]
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
X-Received-From: 62.134.46.9
Cc: Jeff Janes <jeff.janes@gmail.com>, "coreutils@gnu.org" <coreutils@gnu.org>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 11 Jun 2012 11:47:33 -0000

Paul Eggert wrote:
> On 06/11/2012 03:02 AM, Paul Eggert wrote:
> > * src/sort.c (default_sort_size): Do not exceed 3/4 of total memory
>=20
> Come to think of it, that patch is too conservative, as it
> limits things to at most 3/8 of total memory.  The 3/4 limit
> should be applied after 'size' is halved, not before.  Here's
> a better patch:

Then shouldn't the  * 0.75 be _instead_ of the /2 then?

>    size_t size =3D SIZE_MAX;
>    if (getrlimit (RLIMIT_DATA, &rlimit) =3D=3D 0 && rlimit.rlim_cur < siz=
e)
>      size =3D rlimit.rlim_cur;
> @@ -1434,6 +1435,10 @@ default_sort_size (void)
>      size =3D rlimit.rlim_cur / 16 * 15;
>  #endif
> =20
> +  /* Similarly, leave a 1/4 margin for physical memory.  */
> +  if (total * 0.75 < size)
> +    size =3D total * 0.75;
> +

and I'd put it before the rlimit calculation:
e.g. a system has 8GB of RAM, but the admin want sort to
only take 4GB. He'll use ulimit or plimit to 4GB.
With this patch, sort will only take 3GB.
Am I wrong?

Have a nice day,
Berny




From MAILER-DAEMON Mon Jun 11 13:24:35 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1Se8Lz-0001ba-Sn
	for mharc-coreutils@gnu.org; Mon, 11 Jun 2012 13:24:35 -0400
Received: from eggs.gnu.org ([208.118.235.92]:37336)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eggert@cs.ucla.edu>) id 1Se8Lw-0001bO-RZ
	for coreutils@gnu.org; Mon, 11 Jun 2012 13:24:34 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eggert@cs.ucla.edu>) id 1Se8Lv-0007t1-2k
	for coreutils@gnu.org; Mon, 11 Jun 2012 13:24:32 -0400
Received: from smtp.cs.ucla.edu ([131.179.128.62]:44345)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eggert@cs.ucla.edu>) id 1Se8Lu-0007sK-Sz
	for coreutils@gnu.org; Mon, 11 Jun 2012 13:24:30 -0400
Received: from localhost (localhost.localdomain [127.0.0.1])
	by smtp.cs.ucla.edu (Postfix) with ESMTP id 288C8A60005;
	Mon, 11 Jun 2012 10:24:29 -0700 (PDT)
X-Virus-Scanned: amavisd-new at smtp.cs.ucla.edu
Received: from smtp.cs.ucla.edu ([127.0.0.1])
	by localhost (smtp.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10024)
	with ESMTP id Z3vP3gg01UyD; Mon, 11 Jun 2012 10:24:28 -0700 (PDT)
Received: from penguin.cs.ucla.edu (Penguin.CS.UCLA.EDU [131.179.64.200])
	by smtp.cs.ucla.edu (Postfix) with ESMTPSA id B5437A60002;
	Mon, 11 Jun 2012 10:24:28 -0700 (PDT)
Message-ID: <4FD629CC.4030908@cs.ucla.edu>
Date: Mon, 11 Jun 2012 10:24:28 -0700
From: Paul Eggert <eggert@cs.ucla.edu>
Organization: UCLA Computer Science Department
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:12.0) Gecko/20120430 Thunderbird/12.0.1
MIME-Version: 1.0
To: "Voelker, Bernhard" <bernhard.voelker@siemens-enterprise.com>
Subject: Re: overly aggressive memory usage by sort.c
References: <CAMkU=1zDA=RFzxb-O_x2TG+EumrdUebZJb4UOJAM_xretGpCCA@mail.gmail.com>
	<4FD1431C.5010500@draigBrady.com> <878vfu9x7o.fsf@rho.meyering.net>
	<4FD5C249.7090907@cs.ucla.edu> <4FD5CF65.7000609@cs.ucla.edu>
	<A75C39900FCF0B42A2CE8B26F837D2C3010923@MCHP02MSX.global-ad.net>
In-Reply-To: <A75C39900FCF0B42A2CE8B26F837D2C3010923@MCHP02MSX.global-ad.net>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
X-Received-From: 131.179.128.62
Cc: "coreutils@gnu.org" <coreutils@gnu.org>, Jeff Janes <jeff.janes@gmail.com>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 11 Jun 2012 17:24:34 -0000

On 06/11/2012 04:47 AM, Voelker, Bernhard wrote:
> shouldn't the  * 0.75 be _instead_ of the /2 then?

In some sense it is "instead" already.
The /2 is for hard limits, i.e.,
the program will crash if you exceed them.
The 0.75 is for a soft limit, i.e., the program will
run (though perhaps slowly) if you exceed it.
The idea is to be more-conservative about hard limits
than about soft ones.

> and I'd put it before the rlimit calculation:
> e.g. a system has 8GB of RAM, but the admin want sort to
> only take 4GB. He'll use ulimit or plimit to 4GB.
> With this patch, sort will only take 3GB.
> Am I wrong?

I think so.  The newly inserted code
"if (total * 0.75 < size) size = total * 0.75;"
is independent of the limits affected by
the ulimit and plimit commands.  In your example
'sort' will consume 4 * 15/16 GB if physmem_available
reports 4 GB available.


From MAILER-DAEMON Tue Jun 12 07:37:49 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1SePPw-0001Fr-VV
	for mharc-coreutils@gnu.org; Tue, 12 Jun 2012 07:37:48 -0400
Received: from eggs.gnu.org ([208.118.235.92]:56102)
	by lists.gnu.org with esmtp (Exim 4.71) (envelope-from <pth@suse.de>)
	id 1SePPu-0001FH-Lo
	for coreutils@gnu.org; Tue, 12 Jun 2012 07:37:48 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <pth@suse.de>) id 1SePPp-0005Mf-Ox
	for coreutils@gnu.org; Tue, 12 Jun 2012 07:37:46 -0400
Received: from cantor2.suse.de ([195.135.220.15]:44060 helo=mx2.suse.de)
	by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <pth@suse.de>)
	id 1SePPp-0005Lx-IO
	for coreutils@gnu.org; Tue, 12 Jun 2012 07:37:41 -0400
Received: from relay1.suse.de (unknown [195.135.220.254])
	(using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
	(No client certificate requested)
	by mx2.suse.de (Postfix) with ESMTP id 7F5D68FC93
	for <coreutils@gnu.org>; Tue, 12 Jun 2012 13:37:38 +0200 (CEST)
Date: Tue, 12 Jun 2012 13:37:38 +0200
From: Philipp Thomas <pth@suse.de>
To: coreutils@gnu.org
Subject: Re: [RFC/PATCH] cp: Add option to pre-allocate space for files
Message-ID: <20120612113738.GM6237@paradies.suse.de>
References: <90cdbf76a2ca009ab20d719e55385197.squirrel@ssl-webmail-vh.clara.net>
	<4FAD342A.3050503@draigBrady.com>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="4Ckj6UjgE2iN1+kY"
Content-Disposition: inline
In-Reply-To: <4FAD342A.3050503@draigBrady.com>
X-Operating-System: openSUSE - Kernel Linux 3.3.0-2-desktop x86_64
Organization: SUSE LINUX Products GmbH, =?iso-8859-1?Q?G?=
	=?iso-8859-1?Q?F=3A_Jeff_Hawn=2C_Jennifer_Guild=2C_Felix_Imend=F6rffer=2C?=
	=?iso-8859-1?Q?_HRB_21284_=28AG_N=FCrnberg=29?=
User-Agent: Mutt/1.5.21 (2010-09-15)
Content-Transfer-Encoding: 7bit
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.4-2.6
X-Received-From: 195.135.220.15
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Tue, 12 Jun 2012 11:37:48 -0000


--4Ckj6UjgE2iN1+kY
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

* P=E1draig Brady (P@draigBrady.com) [20120511 17:46]:

> This feature is already under consideration.
> See the comments at: http://bugs.gnu.org/9500

FWIW, SUSE patches coreutils (patch attached) to use fallocate when
sparse=3Dnever or non-sparse files with sparse=3Dauto (!make_holes in
copy.c(copy_reg).  But as Eric wrote in the above thread, it would need a
gnulib module for a correct implementation.

BTW, biggest win is for ocfs2 as it must take cluster wide locks on
fragmentation, which was the reason we added the patch.

Philipp

--4Ckj6UjgE2iN1+kY
Content-Type: text/x-patch; charset=us-ascii
Content-Disposition: attachment; filename="coreutils-fallocate-1.patch"

--- src/copy.c
+++ src/copy.c
@@ -1044,6 +1044,8 @@ copy_reg (char const *src_name, char const *dst_name,
           buf_size -= buf_size % blcm;
           if (buf_size == 0 || blcm_max < buf_size)
             buf_size = blcm;
+
+          fallocate(dest_desc, FALLOC_FL_KEEP_SIZE, 0, src_sb->st_size);
         }
 
       /* Make a buffer with space for a sentinel at the end.  */

--4Ckj6UjgE2iN1+kY--


From MAILER-DAEMON Tue Jun 12 11:10:33 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1SeSjp-0004b0-IC
	for mharc-coreutils@gnu.org; Tue, 12 Jun 2012 11:10:33 -0400
Received: from eggs.gnu.org ([208.118.235.92]:50049)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1SeSjA-0004Lm-Lv
	for coreutils@gnu.org; Tue, 12 Jun 2012 11:10:31 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1SeSiB-00082N-E9
	for coreutils@gnu.org; Tue, 12 Jun 2012 11:09:52 -0400
Received: from mx1.redhat.com ([209.132.183.28]:44659)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>)
	id 1SeSgd-0006zS-2j; Tue, 12 Jun 2012 11:07:15 -0400
Received: from int-mx01.intmail.prod.int.phx2.redhat.com
	(int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id q5CF78sn006526
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK);
	Tue, 12 Jun 2012 11:07:08 -0400
Received: from [10.3.113.88] (ovpn-113-88.phx2.redhat.com [10.3.113.88])
	by int-mx01.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP
	id q5CF76jK002216; Tue, 12 Jun 2012 11:07:06 -0400
Message-ID: <4FD75B19.6060100@redhat.com>
Date: Tue, 12 Jun 2012 09:07:05 -0600
From: Eric Blake <eblake@redhat.com>
Organization: Red Hat
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:12.0) Gecko/20120430 Thunderbird/12.0.1
MIME-Version: 1.0
To: Rich Felker <dalias@aerifal.cx>
Subject: Re: gnulib portability issues
References: <20120610003909.GA14918@brightrain.aerifal.cx>
	<4FD73444.9000402@redhat.com>
	<20120612134650.GU163@brightrain.aerifal.cx>
	<4FD74C12.9020202@redhat.com>
	<20120612142053.GW163@brightrain.aerifal.cx>
In-Reply-To: <20120612142053.GW163@brightrain.aerifal.cx>
X-Enigmail-Version: 1.4.2
OpenPGP: url=http://people.redhat.com/eblake/eblake.gpg
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature";
	boundary="------------enig1CE0ECD6989431E805CE718D"
X-Scanned-By: MIMEDefang 2.67 on 10.5.11.11
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.132.183.28
Cc: Bruno Haible <bruno@clisp.org>, bug-gnulib@gnu.org,
	Coreutils <coreutils@gnu.org>, Reuben Thomas <rrt@sc3d.org>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Tue, 12 Jun 2012 15:10:31 -0000

This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--------------enig1CE0ECD6989431E805CE718D
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

On 06/12/2012 08:20 AM, Rich Felker wrote:

> To me, the only difference is the pain of replacing them. You cannot
> have these bit patterns in an LD80 without your program having invoked
> undefined behavior (accessing an object as a type other than its
> effective type or one of the allowed exceptions), so there's no
> fundamental reason either should handle it. In fact it's conceivable
> that machines could exist where even just loading the bit pattern into
> a floating point register would generate a signal,

But isnanl() is required to operate in spite of signaling
representations (that is, isnanl() need NOT load a bit pattern into a
floating point register).

> so really od should
> be performing the test on the unsigned char[] array *before*
> reinterpreting it to LD80 rather than calling a function that takes an
> LD80 argument to make the test..

But then you are forcing programs to add #ifdefs and sizeof(long
double)=3D=3D12 checks, instead of letting libc do it for them.  That is,=

testing for an invalid LD80 bit representation is
implementation-specific (it depends on the specific hardware and
compiler ABI before you even have a long double that can have the
invalid representation), but isnanl() is generic - we'd much rather have
the generic code do the implementation-specific filtering rather than
having to make every client learn the #ifdefs necessary for
implementation-specific bit pattern filtering.

That said, practicality may win on this one, and you may indeed convince
Jim that adding an #ifdef into coreutils' od.c is better than bloating
isnanl() for the majority of applications that never convert raw bit
patterns into long doubles.

--=20
Eric Blake   eblake@redhat.com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


--------------enig1CE0ECD6989431E805CE718D
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)
Comment: Public key at http://people.redhat.com/eblake/eblake.gpg
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQEcBAEBCAAGBQJP11sZAAoJEKeha0olJ0NqeDAH/1BqDMriy+ttDTFR7LVIg46m
2myZHBcsL46ho1Wx5ta+JBJoFXGrBU7ZEQTWH/s9yizZn6ozImbrslUYLiLqvJv3
j2hrky58hfp6NGNM3Axll6Hy0SsZk4o4C20GTOUnr9zgvTub99tRhEhKEtyN89Xp
q2Ad9bimzpSZJwnU6PTUDYCCvsHIKAKJP4ddV3pTft3C68DfnmUzLEoMce/CoVQi
P15kSS94q3+Ozf+O6V/cyITpwDRLcnXPrNd4ax/x6ffeUqFG74B+CMqiN4tV7nKc
6/60KUhf6LLu2AutJUdtc305RxXDCIpiMPeZSC4QrDCP2r1xNiMec9UeCsJreuc=
=+F25
-----END PGP SIGNATURE-----

--------------enig1CE0ECD6989431E805CE718D--


From MAILER-DAEMON Tue Jun 12 15:03:28 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1SeWNE-0008On-4d
	for mharc-coreutils@gnu.org; Tue, 12 Jun 2012 15:03:28 -0400
Received: from eggs.gnu.org ([208.118.235.92]:59363)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeff.janes@gmail.com>) id 1SeWN7-0008M2-1y
	for coreutils@gnu.org; Tue, 12 Jun 2012 15:03:26 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jeff.janes@gmail.com>) id 1SeWN3-0001Vp-Nu
	for coreutils@gnu.org; Tue, 12 Jun 2012 15:03:20 -0400
Received: from mail-yx0-f169.google.com ([209.85.213.169]:48070)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeff.janes@gmail.com>) id 1SeWN3-0001TV-GN
	for coreutils@gnu.org; Tue, 12 Jun 2012 15:03:17 -0400
Received: by yenm7 with SMTP id m7so4489176yen.0
	for <coreutils@gnu.org>; Tue, 12 Jun 2012 12:03:14 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
	h=mime-version:in-reply-to:references:date:message-id:subject:from:to
	:cc:content-type;
	bh=BPPSniGZy21iymbJKPowmTnU4SHraANdNcZRrjc/JLQ=;
	b=MWJKY+pkSEA2EavApJhaqWbJSL/nhLWNiqYQkObhFBei2UKpd1R1ZD383M6LPaYjo8
	8cq35pDyXYRWjY3oazZYWi89USkkaVMDf0qqlIZt4XIyPMYS+eI3PnaijEvX2vI/j4M9
	UTkZsRPqK3Nb+OzKuMHZ759Toi+o19LhP6I/vhhXb/wRjGVpLXJm4WU8uNDhi6zYLyJ+
	2aJASkhzBcRo1kmQc2JWGMTMkVJxtpD8jUhQ5WixxU/fS4MYIKQjOPduI/W2hdwvU4ER
	5+ID3LPKwbl+kiucLmVX3gh6Obwx1dOgfNBBSYfdYbO7ugSYkfkFh2C/xNxE2kyJO+ND
	zsXQ==
MIME-Version: 1.0
Received: by 10.50.46.230 with SMTP id y6mr9206597igm.20.1339527794192; Tue,
	12 Jun 2012 12:03:14 -0700 (PDT)
Received: by 10.42.80.65 with HTTP; Tue, 12 Jun 2012 12:03:14 -0700 (PDT)
In-Reply-To: <4FD5C249.7090907@cs.ucla.edu>
References: <CAMkU=1zDA=RFzxb-O_x2TG+EumrdUebZJb4UOJAM_xretGpCCA@mail.gmail.com>
	<4FD1431C.5010500@draigBrady.com> <878vfu9x7o.fsf@rho.meyering.net>
	<4FD5C249.7090907@cs.ucla.edu>
Date: Tue, 12 Jun 2012 12:03:14 -0700
Message-ID: <CAMkU=1zqrpOjM_2+86SwbhtAqLiKikWsQhr4skdm_s8yzu9biw@mail.gmail.com>
Subject: Re: overly aggressive memory usage by sort.c
From: Jeff Janes <jeff.janes@gmail.com>
To: Paul Eggert <eggert@cs.ucla.edu>
Content-Type: text/plain; charset=ISO-8859-1
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.85.213.169
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Tue, 12 Jun 2012 19:03:26 -0000

On Mon, Jun 11, 2012 at 3:02 AM, Paul Eggert <eggert@cs.ucla.edu> wrote:
> On 06/10/2012 11:30 PM, Jim Meyering wrote:
>> Any objection to reverting that and adjusting the comment to match?
>
> Yes, as this would run afoul of bug 10877
> <http://bugs.gnu.org/10877>.

My reading of bug 10877 was as a complaint about using too small of an
assumption for piped input, with the rest of the issue (using 1/16
rather than 1/8 of physmem, and coincidentally using avail/2 rather
than avail) just being a side-track.  So the patch applied there did
not fix a verified problem.  It changed a long-standing behavior in
order to make a long-erroneous comment become accurate--but since
there is no reason that I know of to think that the new behavior is
better than the old one, I think changing the comment would have been
better than changing the behavior.  On the other hand the "new"
behavior is already two releases old, so you could argue that
reverting it entirely at this point is not particularly conservative.
I do have some evidence that changing to use all of avail_mem was too
aggressive, but do we have evidence one way or the other about
physmem/8 versus physmem/16?

Cheers,

Jeff


From MAILER-DAEMON Tue Jun 12 18:43:29 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1SeZo9-0006Ic-6p
	for mharc-coreutils@gnu.org; Tue, 12 Jun 2012 18:43:29 -0400
Received: from eggs.gnu.org ([208.118.235.92]:39726)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <dalias@aerifal.cx>) id 1SeTsf-0004FO-VJ
	for coreutils@gnu.org; Tue, 12 Jun 2012 12:23:51 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <dalias@aerifal.cx>) id 1SeTsZ-0005T4-ML
	for coreutils@gnu.org; Tue, 12 Jun 2012 12:23:45 -0400
Received: from 216-12-86-13.cv.mvl.ntelos.net ([216.12.86.13]:38698
	helo=brightrain.aerifal.cx) by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <dalias@aerifal.cx>) id 1SeTsZ-0005RY-Fb
	for coreutils@gnu.org; Tue, 12 Jun 2012 12:23:39 -0400
Received: from dalias by brightrain.aerifal.cx with local (Exim 3.15 #2)
	id 1SeTPk-0007QE-00; Tue, 12 Jun 2012 15:53:52 +0000
Date: Tue, 12 Jun 2012 11:53:52 -0400
To: Eric Blake <eblake@redhat.com>
Subject: Re: gnulib portability issues
Message-ID: <20120612155352.GZ163@brightrain.aerifal.cx>
References: <20120610003909.GA14918@brightrain.aerifal.cx>
	<4FD73444.9000402@redhat.com>
	<20120612134650.GU163@brightrain.aerifal.cx>
	<4FD74C12.9020202@redhat.com>
	<20120612142053.GW163@brightrain.aerifal.cx>
	<4FD75B19.6060100@redhat.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <4FD75B19.6060100@redhat.com>
User-Agent: Mutt/1.5.21 (2010-09-15)
From: Rich Felker <dalias@aerifal.cx>
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6, seldom 2.4 (older,
	4)
X-Received-From: 216.12.86.13
X-Mailman-Approved-At: Tue, 12 Jun 2012 18:43:28 -0400
Cc: Bruno Haible <bruno@clisp.org>, bug-gnulib@gnu.org,
	Coreutils <coreutils@gnu.org>, Reuben Thomas <rrt@sc3d.org>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Tue, 12 Jun 2012 16:23:51 -0000

On Tue, Jun 12, 2012 at 09:07:05AM -0600, Eric Blake wrote:
> On 06/12/2012 08:20 AM, Rich Felker wrote:
> 
> > To me, the only difference is the pain of replacing them. You cannot
> > have these bit patterns in an LD80 without your program having invoked
> > undefined behavior (accessing an object as a type other than its
> > effective type or one of the allowed exceptions), so there's no
> > fundamental reason either should handle it. In fact it's conceivable
> > that machines could exist where even just loading the bit pattern into
> > a floating point register would generate a signal,
> 
> But isnanl() is required to operate in spite of signaling
> representations (that is, isnanl() need NOT load a bit pattern into a
> floating point register).

Signaling nans and trap representations are different things per the
standard. A signaling nan is a legitimate value that raises a floating
point exception flag (not a signal) when it's used in arithmetic, and
which is created in an implementation-defined way, if at all. A trap
representation (for any type, not just floating point) is a pattern of
bits that results in undefined behavior if it is read by an lvalue
expression that does not have character type.

> > so really od should
> > be performing the test on the unsigned char[] array *before*
> > reinterpreting it to LD80 rather than calling a function that takes an
> > LD80 argument to make the test..
> 
> But then you are forcing programs to add #ifdefs and sizeof(long
> double)==12 checks, instead of letting libc do it for them.  That is,
> testing for an invalid LD80 bit representation is
> implementation-specific (it depends on the specific hardware and
> compiler ABI before you even have a long double that can have the
> invalid representation), but isnanl() is generic - we'd much rather have
> the generic code do the implementation-specific filtering rather than
> having to make every client learn the #ifdefs necessary for
> implementation-specific bit pattern filtering.

I agree this is ugly. One possible solution would be a function (in
gnulib) to inspect a byte array and report whether it's a valid long
double representation.

> That said, practicality may win on this one, and you may indeed convince
> Jim that adding an #ifdef into coreutils' od.c is better than bloating
> isnanl() for the majority of applications that never convert raw bit
> patterns into long doubles.

I don't really care if the fix is an ugly bit test in in od.c or a
call to isnanl in od.c before the call to printf and letting gnulib's
isnanl handle the case; avoiding the printf replacement is enough for
me.

I do think only approaches that avoid reading the bits as LD80 before
they're known to be a non-trap representation can be 100% safe against
obscure machines, but on the other hand, avoiding machine-specific
code in od.c will probably make the situation better for practical
portability.

Rich


From MAILER-DAEMON Wed Jun 13 05:32:49 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1SejwX-00016C-UZ
	for mharc-coreutils@gnu.org; Wed, 13 Jun 2012 05:32:49 -0400
Received: from eggs.gnu.org ([208.118.235.92]:38480)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <kzak@redhat.com>) id 1SejwR-00013y-1V
	for coreutils@gnu.org; Wed, 13 Jun 2012 05:32:48 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <kzak@redhat.com>) id 1SejwK-0003r5-Rb
	for coreutils@gnu.org; Wed, 13 Jun 2012 05:32:42 -0400
Received: from mx1.redhat.com ([209.132.183.28]:58490)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <kzak@redhat.com>) id 1SejwK-0003r0-Jl
	for coreutils@gnu.org; Wed, 13 Jun 2012 05:32:36 -0400
Received: from int-mx12.intmail.prod.int.phx2.redhat.com
	(int-mx12.intmail.prod.int.phx2.redhat.com [10.5.11.25])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id q5D9WZkP008688
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK)
	for <coreutils@gnu.org>; Wed, 13 Jun 2012 05:32:35 -0400
Received: from x2 (ovpn-116-57.ams2.redhat.com [10.36.116.57])
	by int-mx12.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP
	id q5D9WXb0026593
	for <coreutils@gnu.org>; Wed, 13 Jun 2012 05:32:34 -0400
Date: Wed, 13 Jun 2012 11:32:33 +0200
From: Karel Zak <kzak@redhat.com>
To: coreutils@gnu.org
Subject: ifdef HAVE_DECL_
Message-ID: <20120613093233.GA10561@x2.net.home>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
User-Agent: Mutt/1.5.21 (2011-07-01)
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.25
Content-Transfer-Encoding: quoted-printable
X-MIME-Autoconverted: from 8bit to quoted-printable by mx1.redhat.com id
	q5D9WZkP008688
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.132.183.28
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Wed, 13 Jun 2012 09:32:49 -0000


 Hi,

 autoconf docs about AC_CHECK_DECLS:

   Unlike the other =E2=80=98AC_CHECK_*S=E2=80=99 macros, when a symbol i=
s not
   declared, HAVE_DECL_symbol is defined to =E2=80=980=E2=80=99 instead o=
f leaving
   HAVE_DECL_symbol undeclared. When you are sure that the check was
   performed, use HAVE_DECL_symbol in #if:

        #if !HAVE_DECL_SYMBOL
           extern char *symbol;
        #endif

 This is also valid for AC_CHECK_DECLS_ONCE.

 It means that use #ifdef or #ifndef for HAVE_DECL_* is very probably bug.

 Let's quickly check our code for #ifdef:

 $ grep -r  '[[:blank:]]*#[[:blank:]]*ifdef[[:blank:]]*HAVE_DECL_' *
 gnulib/lib/argp-parse.c:#ifdef HAVE_DECL_PROGRAM_INVOCATION_NAME
 gnulib/lib/argp-parse.c:#ifdef HAVE_DECL_PROGRAM_INVOCATION_SHORT_NAME


 More complex (and I hope without false positives) solution:

 for decl in $(awk '/define HAVE_DECL_.*ac_have_decl/ { print $2 }' confi=
gure); do=20
    grep -rnE '[[:blank:]]*#[[:blank:]]*if(ndef|def)[[:blank:]]*'$decl *;=
 done | sort -u

 configure:45306:#ifndef HAVE_DECL_STRNDUP
 gnulib/lib/argp-parse.c:882:#ifdef HAVE_DECL_PROGRAM_INVOCATION_NAME
 gnulib/lib/argp-parse.c:886:#ifdef HAVE_DECL_PROGRAM_INVOCATION_SHORT_NA=
ME
 gnulib/lib/error.c:103:#  ifndef HAVE_DECL_STRERROR_R
 gnulib/lib/fpending.h:24:#ifndef HAVE_DECL___FPENDING
 gnulib/lib/strtoimax.c:31:# ifndef HAVE_DECL_STRTOULL
 gnulib/lib/strtoimax.c:40:# ifndef HAVE_DECL_STRTOLL
 gnulib/m4/strndup.m4:27:#ifndef HAVE_DECL_STRNDUP
 lib/error.c:103:#  ifndef HAVE_DECL_STRERROR_R
 lib/fpending.h:24:#ifndef HAVE_DECL___FPENDING
 lib/strtoimax.c:31:# ifndef HAVE_DECL_STRTOULL
 lib/strtoimax.c:40:# ifndef HAVE_DECL_STRTOLL
 m4/strndup.m4:27:#ifndef HAVE_DECL_STRNDUP


 Bugs?

    Karel

--=20
 Karel Zak  <kzak@redhat.com>
 http://karelzak.blogspot.com


From MAILER-DAEMON Wed Jun 13 05:45:52 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1Sek9A-0003pM-QB
	for mharc-coreutils@gnu.org; Wed, 13 Jun 2012 05:45:52 -0400
Received: from eggs.gnu.org ([208.118.235.92]:35144)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <kzak@redhat.com>) id 1Sek93-0003nI-Sb
	for coreutils@gnu.org; Wed, 13 Jun 2012 05:45:51 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <kzak@redhat.com>) id 1Sek8x-0006Gi-14
	for coreutils@gnu.org; Wed, 13 Jun 2012 05:45:45 -0400
Received: from mx1.redhat.com ([209.132.183.28]:22591)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <kzak@redhat.com>) id 1Sek8w-0006G1-P2
	for coreutils@gnu.org; Wed, 13 Jun 2012 05:45:38 -0400
Received: from int-mx11.intmail.prod.int.phx2.redhat.com
	(int-mx11.intmail.prod.int.phx2.redhat.com [10.5.11.24])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id q5D9jZrp003457
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK)
	for <coreutils@gnu.org>; Wed, 13 Jun 2012 05:45:37 -0400
Received: from x2 (ovpn-116-57.ams2.redhat.com [10.36.116.57])
	by int-mx11.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP
	id q5D9jY7b017836
	for <coreutils@gnu.org>; Wed, 13 Jun 2012 05:45:35 -0400
Date: Wed, 13 Jun 2012 11:45:33 +0200
From: Karel Zak <kzak@redhat.com>
To: coreutils@gnu.org
Subject: Re: ifdef HAVE_DECL_
Message-ID: <20120613094533.GB10561@x2.net.home>
References: <20120613093233.GA10561@x2.net.home>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20120613093233.GA10561@x2.net.home>
User-Agent: Mutt/1.5.21 (2011-07-01)
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.24
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.132.183.28
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Wed, 13 Jun 2012 09:45:51 -0000

On Wed, Jun 13, 2012 at 11:32:33AM +0200, Karel Zak wrote:
 
>  More complex (and I hope without false positives) solution:
 
 well... ;-)

> 
>  for decl in $(awk '/define HAVE_DECL_.*ac_have_decl/ { print $2 }' configure); do 
>     grep -rnE '[[:blank:]]*#[[:blank:]]*if(ndef|def)[[:blank:]]*'$decl *; done | sort -u
> 
>  configure:45306:#ifndef HAVE_DECL_STRNDUP
>  gnulib/lib/argp-parse.c:882:#ifdef HAVE_DECL_PROGRAM_INVOCATION_NAME
>  gnulib/lib/argp-parse.c:886:#ifdef HAVE_DECL_PROGRAM_INVOCATION_SHORT_NAME

 this seems like a bug

>  gnulib/lib/error.c:103:#  ifndef HAVE_DECL_STRERROR_R
>  gnulib/lib/fpending.h:24:#ifndef HAVE_DECL___FPENDING
>  gnulib/lib/strtoimax.c:31:# ifndef HAVE_DECL_STRTOULL
>  gnulib/lib/strtoimax.c:40:# ifndef HAVE_DECL_STRTOLL
>  gnulib/m4/strndup.m4:27:#ifndef HAVE_DECL_STRNDUP
>  lib/error.c:103:#  ifndef HAVE_DECL_STRERROR_R
>  lib/fpending.h:24:#ifndef HAVE_DECL___FPENDING
>  lib/strtoimax.c:31:# ifndef HAVE_DECL_STRTOULL
>  lib/strtoimax.c:40:# ifndef HAVE_DECL_STRTOLL

 and this like false positives (there is #if before #ifndef)

>  m4/strndup.m4:27:#ifndef HAVE_DECL_STRNDUP

 not sure

    Karel

-- 
 Karel Zak  <kzak@redhat.com>
 http://karelzak.blogspot.com


From MAILER-DAEMON Wed Jun 13 09:29:16 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1SendM-0006yp-Jd
	for mharc-coreutils@gnu.org; Wed, 13 Jun 2012 09:29:16 -0400
Received: from eggs.gnu.org ([208.118.235.92]:47548)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <inpost@gmail.com>) id 1SendF-0006yA-A3
	for coreutils@gnu.org; Wed, 13 Jun 2012 09:29:14 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <inpost@gmail.com>) id 1Send7-0008HT-2B
	for coreutils@gnu.org; Wed, 13 Jun 2012 09:29:08 -0400
Received: from mail-pb0-f41.google.com ([209.85.160.41]:41042)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <inpost@gmail.com>) id 1Send6-0008Gn-PY
	for coreutils@gnu.org; Wed, 13 Jun 2012 09:29:00 -0400
Received: by pbbrp2 with SMTP id rp2so2544133pbb.0
	for <coreutils@gnu.org>; Wed, 13 Jun 2012 06:28:57 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
	h=mime-version:date:message-id:subject:from:to:content-type;
	bh=irQ1RlKDwAFlebCifrNLpW+WL9xNbbK2Ya2Ywk+7jjU=;
	b=aKYMcrRd99ALwX5Uc0BaImvcht4IpqJsenJLvDAmSEwmeICRpZUaeqOB4Md77NS/iF
	dwTUDD7+QOOVkPnLWRWqWewwKxRGc+1US7BRK/KdF7R54wUDT9Yq/bnigX5vAEPtw419
	78D1mU2Lkcg7Tch7uXgfIUMXdUfEr2lBQRNitLaBvm3ja8Ylc+2LWgxtf7DXx1nfunxg
	/MeXpISKpl+CsZ7hN1bA5mkkGcDYbAC4N7qm1nx4Cdh/phzgZe5di3VS0nX+Kzy3IvIU
	+ilhOYnMnwlhNQh6oOFM8sXFfWe/zzUW1yftTDcyadm4yd5cTLLzN7hJ54GQbRqZ5DRI
	7STQ==
MIME-Version: 1.0
Received: by 10.68.134.201 with SMTP id pm9mr52312840pbb.49.1339594137305;
	Wed, 13 Jun 2012 06:28:57 -0700 (PDT)
Received: by 10.68.135.233 with HTTP; Wed, 13 Jun 2012 06:28:57 -0700 (PDT)
Date: Wed, 13 Jun 2012 14:28:57 +0100
Message-ID: <CAET1fe5Cw-LBoDse0hU51nBc84wKqA7Hgx_kMVSteY_Mc=dBDw@mail.gmail.com>
Subject: redirect command with tar
From: e-letter <inpost@gmail.com>
To: coreutils@gnu.org
Content-Type: text/plain; charset=UTF-8
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.85.160.41
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Wed, 13 Jun 2012 13:29:15 -0000

Readers,

The command 'tar' is used to update a tar file stored on a portable
hard disk drive, of files stored on a local hard disk:

tar -uvf /path/to/portable/harddisk/ *

In the command terminal, the list of files added to the tar file is
shown. In future, can the redirect output command be applied, e.g.:

tar -uvf /path/to/portable/harddisk/ * >
~/tmp/list/of/files/added/to/harddisk.txt

Thanks in advance.


From MAILER-DAEMON Wed Jun 13 09:33:48 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1Senhk-0001Ui-Ms
	for mharc-coreutils@gnu.org; Wed, 13 Jun 2012 09:33:48 -0400
Received: from eggs.gnu.org ([208.118.235.92]:36775)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1Senhe-0001Tc-Of
	for coreutils@gnu.org; Wed, 13 Jun 2012 09:33:47 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1SenhZ-0001Jw-Di
	for coreutils@gnu.org; Wed, 13 Jun 2012 09:33:42 -0400
Received: from mx1.redhat.com ([209.132.183.28]:29222)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1SenhZ-0001JK-5s
	for coreutils@gnu.org; Wed, 13 Jun 2012 09:33:37 -0400
Received: from int-mx12.intmail.prod.int.phx2.redhat.com
	(int-mx12.intmail.prod.int.phx2.redhat.com [10.5.11.25])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id q5DDXYx3016448
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK);
	Wed, 13 Jun 2012 09:33:34 -0400
Received: from [10.3.113.3] ([10.3.113.3])
	by int-mx12.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP
	id q5DDXYCw014119; Wed, 13 Jun 2012 09:33:34 -0400
Message-ID: <4FD896AC.3020600@redhat.com>
Date: Wed, 13 Jun 2012 07:33:32 -0600
From: Eric Blake <eblake@redhat.com>
Organization: Red Hat
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:12.0) Gecko/20120430 Thunderbird/12.0.1
MIME-Version: 1.0
To: e-letter <inpost@gmail.com>
Subject: Re: redirect command with tar
References: <CAET1fe5Cw-LBoDse0hU51nBc84wKqA7Hgx_kMVSteY_Mc=dBDw@mail.gmail.com>
In-Reply-To: <CAET1fe5Cw-LBoDse0hU51nBc84wKqA7Hgx_kMVSteY_Mc=dBDw@mail.gmail.com>
X-Enigmail-Version: 1.4.2
OpenPGP: url=http://people.redhat.com/eblake/eblake.gpg
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature";
	boundary="------------enig6245311F0FB783F11734E145"
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.25
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.132.183.28
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Wed, 13 Jun 2012 13:33:47 -0000

This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--------------enig6245311F0FB783F11734E145
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

On 06/13/2012 07:28 AM, e-letter wrote:
> Readers,
>=20
> The command 'tar' is used to update a tar file stored on a portable
> hard disk drive, of files stored on a local hard disk:
>=20
> tar -uvf /path/to/portable/harddisk/ *
>=20
> In the command terminal, the list of files added to the tar file is
> shown. In future, can the redirect output command be applied, e.g.:
>=20
> tar -uvf /path/to/portable/harddisk/ * >
> ~/tmp/list/of/files/added/to/harddisk.txt

Wrong list.  This is the GNU Coreutils list, but you are asking
questions about tar and the shell.  You will get a better response by
asking on bug-tar@gnu.org.

--=20
Eric Blake   eblake@redhat.com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


--------------enig6245311F0FB783F11734E145
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)
Comment: Public key at http://people.redhat.com/eblake/eblake.gpg
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQEcBAEBCAAGBQJP2JasAAoJEKeha0olJ0NqcI0H/1slsFF8jJf2z7OWrFbZRLxY
JtJGxGgVQyOH4JElBAJJ5CDFPbUOlcBjh76gsf4JkMTxOcIXEI0vupWMmQ13wV+U
s4l+YrBuHqwbRzZl2CcWPdIYqwpcHfLRlHFhDJgquE1jR9x7mneqTZuRlYD2jlUp
upIoCnfTRsnVnyxj5Dtq325oUoVZiBILvysd12rTDw+Q8PXWskXd2v8GTT+xP1yj
61ehZEBwItzKXlPnoASlu4Jj3+ai4DhvH4bGnDDOJFiRNePex6dU9bqCjv2j/ysk
ZEFpbMPrsVhhdEwYOnQ+VOLaEkZQpHbPyqidHPA7d1j0XL/inRS71IyNMHstFY0=
=T4/N
-----END PGP SIGNATURE-----

--------------enig6245311F0FB783F11734E145--


From MAILER-DAEMON Wed Jun 13 14:39:28 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1SesTY-0001w6-7T
	for mharc-coreutils@gnu.org; Wed, 13 Jun 2012 14:39:28 -0400
Received: from eggs.gnu.org ([208.118.235.92]:56971)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jannhorn@googlemail.com>) id 1SesTV-0001v8-OC
	for coreutils@gnu.org; Wed, 13 Jun 2012 14:39:26 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jannhorn@googlemail.com>) id 1SesTU-0003YX-47
	for coreutils@gnu.org; Wed, 13 Jun 2012 14:39:25 -0400
Received: from mail-bk0-f41.google.com ([209.85.214.41]:48241)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jannhorn@googlemail.com>) id 1SesTT-0003YF-Pz
	for coreutils@gnu.org; Wed, 13 Jun 2012 14:39:24 -0400
Received: by bkcjm19 with SMTP id jm19so917285bkc.0
	for <coreutils@gnu.org>; Wed, 13 Jun 2012 11:39:20 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
	d=googlemail.com; s=20120113;
	h=date:from:to:cc:subject:message-id:references:mime-version
	:content-type:content-disposition:in-reply-to:user-agent;
	bh=AT82tF3AFQDxAvs821btZOw83smiK5MOjQzpurB9Agg=;
	b=vKiaaobkgTabWrEOJbRJr2ThEO37vWobShBf0eelNp8p7Ii0KEfAv1pSry9CsazlXy
	S2BIJ9nvdfQIjblVM0q5ZvSW3STDdsPd8TZTdoetlG0g2hFd8vUf4aEHNVkMnUZIexlL
	7zAcKMzbKa1LKu/Jp46KPVpkY8Hbe0st7RGXBF9lB2fetJI9uT5FEEqQxB5ZEQ/OKBZD
	zNKrN5EwpX5WVxGdX9bbGw4OM4qE1QNfUR9GnLx6vDqOOeJyDpkmTVKPVDvp/UaygLRO
	eYvLM3Tbtfm+5sfjjkXg3H3l5BxZ93d9r9CQZk2GwqRFrZoyG1U44rE4w+I7CDCIj5B7
	hUeg==
Received: by 10.204.152.6 with SMTP id e6mr13742865bkw.18.1339612760849;
	Wed, 13 Jun 2012 11:39:20 -0700 (PDT)
Received: from Jann-PC.fritz.box (77-23-225-249-dynip.superkabel.de.
	[77.23.225.249])
	by mx.google.com with ESMTPS id gw6sm3976356bkc.16.2012.06.13.11.39.19
	(version=SSLv3 cipher=OTHER); Wed, 13 Jun 2012 11:39:20 -0700 (PDT)
Date: Wed, 13 Jun 2012 20:39:18 +0200
From: Jann Horn <jannhorn@googlemail.com>
To: e-letter <inpost@gmail.com>
Subject: Re: redirect command with tar
Message-ID: <20120613183918.GF32204@Jann-PC.fritz.box>
References: <CAET1fe5Cw-LBoDse0hU51nBc84wKqA7Hgx_kMVSteY_Mc=dBDw@mail.gmail.com>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha1;
	protocol="application/pgp-signature"; boundary="tmoQ0UElFV5VgXgH"
Content-Disposition: inline
In-Reply-To: <CAET1fe5Cw-LBoDse0hU51nBc84wKqA7Hgx_kMVSteY_Mc=dBDw@mail.gmail.com>
User-Agent: Mutt/1.5.21 (2010-09-15)
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.85.214.41
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Wed, 13 Jun 2012 18:39:26 -0000


--tmoQ0UElFV5VgXgH
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Wed, Jun 13, 2012 at 02:28:57PM +0100, e-letter wrote:
> Readers,
>=20
> The command 'tar' is used to update a tar file stored on a portable
> hard disk drive, of files stored on a local hard disk:
>=20
> tar -uvf /path/to/portable/harddisk/ *
>=20
> In the command terminal, the list of files added to the tar file is
> shown. In future, can the redirect output command be applied, e.g.:
>=20
> tar -uvf /path/to/portable/harddisk/ * >
> ~/tmp/list/of/files/added/to/harddisk.txt

Uh, did you actually try it? Works for me...

--tmoQ0UElFV5VgXgH
Content-Type: application/pgp-signature

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iQIcBAEBAgAGBQJP2N5WAAoJEGhmizV0f2d1sqMQAJFHUV42rR1WGJTrv0kGrvSS
vqlxJeTxYWJFn2NYpAtwjPoUO4nStZGZbfTA+TlsaBoeRLPxXooXIOpMSpLRZpOY
J3uCwK5+PN5zUHk/qI8wtzGnkd0swUthH2++vvw8FC5ziJXHuHESo1+xBtSiE05g
ZHQKfAvoMTZhqPlXc4808T2Vfpeqd4mnRTwX+usXMGnvh+GafA44h/7aO+f+RvdX
meVHTMLIP10zeJ3FfxZDcrD+vXlF/oQGOxBYKXuqM3GYZQM5gg4aq4PZicHbLmnJ
7505Dlxt7M0Vhi1Y/hAXhY+bZu1hQhc8u+rQu6Wue1prpHbFH8OwmrdJUhSuuZhZ
zRJyJnYV5Dg3ks0JFiyGIX9Mai77SGF7Oumk3iT5p962ZzIzPcHa0mOlINKFPzWJ
C4y8kQnBgv4dEQ2bJwr6/try1NkWgz2sekiYlhi1cCMjUWQHX4oaCFc/YF+YhH/I
MSy1/tUZZc58fmxQ7S4TQIY4pH96S4mY/MgxCR5rGUF5llVgqXHhAJs1c17/KwNg
dkC8gpnPQbwlSXAPPNcB6Sn34ADtkVnZNF7jEvV1nWwe94tCdRJkgmf1/bPMJNJQ
hK7HRedAXh1hmZLVWhDF1jd8sAnSu9CKQSfXtqzFX29hSxFLsdt+gUqCq/0OcaXD
596IpNo2VJnbuUWnzZKa
=Es+A
-----END PGP SIGNATURE-----

--tmoQ0UElFV5VgXgH--


From MAILER-DAEMON Thu Jun 14 02:02:15 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1Sf38J-0001gT-Gh
	for mharc-coreutils@gnu.org; Thu, 14 Jun 2012 02:02:15 -0400
Received: from eggs.gnu.org ([208.118.235.92]:47946)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1Sf38G-0001dB-2T
	for coreutils@gnu.org; Thu, 14 Jun 2012 02:02:13 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1Sf38D-0001mr-UU
	for coreutils@gnu.org; Thu, 14 Jun 2012 02:02:11 -0400
Received: from moutng.kundenserver.de ([212.227.126.187]:59554)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1Sf38D-0001k4-LH
	for coreutils@gnu.org; Thu, 14 Jun 2012 02:02:09 -0400
Received: from [192.168.2.108] (p4FF767F9.dip.t-dialin.net [79.247.103.249])
	by mrelayeu.kundenserver.de (node=mrbap3) with ESMTP (Nemesis)
	id 0M1XmT-1Rlq700x3Z-00tVho; Thu, 14 Jun 2012 08:01:55 +0200
Message-ID: <4FD97E52.6070202@bernhard-voelker.de>
Date: Thu, 14 Jun 2012 08:01:54 +0200
From: Bernhard Voelker <mail@bernhard-voelker.de>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:12.0) Gecko/20120421 Thunderbird/12.0
MIME-Version: 1.0
To: Paul Eggert <eggert@cs.ucla.edu>
Subject: [PATCH] maint: cleanup size determination in sort [was: overly
	aggressive memory usage by sort.c]
References: <CAMkU=1zDA=RFzxb-O_x2TG+EumrdUebZJb4UOJAM_xretGpCCA@mail.gmail.com>
	<4FD1431C.5010500@draigBrady.com> <878vfu9x7o.fsf@rho.meyering.net>
	<4FD5C249.7090907@cs.ucla.edu> <4FD5CF65.7000609@cs.ucla.edu>
	<A75C39900FCF0B42A2CE8B26F837D2C3010923@MCHP02MSX.global-ad.net>
	<4FD629CC.4030908@cs.ucla.edu>
In-Reply-To: <4FD629CC.4030908@cs.ucla.edu>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
X-Provags-ID: V02:K0:gPUfkHd2RuzOpOTb+QMYWyyPUoqTnB4JxaFEvwYm/U8
	iATF5FLTeoziTDRNr+veSaX1OxHh9bCe67RSaZa42y4CNngd/L
	s49JJMKkcyuW06VtbtrWukGGrQdy29OAfr/Q7wj0/qtoAFCJ7A
	SfkV5Sghf7WTim4F2/GvQFlgJESFdUCh6eB549hawNeAMbGAQk
	BnDK7Z5V428ck0dXF6DMrdSKcWdtA1BQQg8u875D8AIQy4JAhD
	T8hYBnmH7QHvnpG1P9neMAj3u3bAWCVthkRGZlehd37skgRwl5
	QO9CAIqIuljHz2Hxfc/bwRVi7QT0Zb8THx6zTMSAs8IEwibwrf
	t6hLY9vQ+OmaXyvIDRmNmNeym5IdkSMF6nBraT0Bs
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 212.227.126.187
Cc: "coreutils@gnu.org" <coreutils@gnu.org>, "Voelker,
	Bernhard" <bernhard.voelker@siemens-enterprise.com>,
	Jeff Janes <jeff.janes@gmail.com>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 14 Jun 2012 06:02:13 -0000

On 06/11/2012 07:24 PM, Paul Eggert wrote:
> On 06/11/2012 01:47 PM, Voelker, Bernhard wrote:
>> Am I wrong?
> 
> I think so.  The newly inserted code
> "if (total * 0.75 < size) size = total * 0.75;"
> is independent of the limits affected by
> the ulimit and plimit commands.  In your example
> 'sort' will consume 4 * 15/16 GB if physmem_available
> reports 4 GB available.

Thanks for the clarification. I was confused.

Here's a patch to move the code regarding physical/available/total
memory down to where it's used (on top of your patch).

Have a nice day,
Berny


>From 1957c9a46e0427b2bbb346c2cb29710e7cbd6182 Mon Sep 17 00:00:00 2001
From: Bernhard Voelker <mail@bernhard-voelker.de>
Date: Thu, 14 Jun 2012 07:54:34 +0200
Subject: [PATCH] maint: cleanup size determination in sort

* src/sort.c (default_sort_size): Move physmem code "down" to first use.
---
 src/sort.c |   11 ++++++-----
 1 files changed, 6 insertions(+), 5 deletions(-)

diff --git a/src/sort.c b/src/sort.c
index f21e092..c0d0c84 100644
--- a/src/sort.c
+++ b/src/sort.c
@@ -1405,11 +1405,6 @@ specify_nthreads (int oi, char c, char const *s)
 static size_t
 default_sort_size (void)
 {
-  /* Let MEM be available memory or 1/8 of total memory, whichever
-     is greater.  */
-  double avail = physmem_available ();
-  double total = physmem_total ();
-  double mem = MAX (avail, total / 8);
   struct rlimit rlimit;

   /* Let SIZE be MEM, but no more than the maximum object size,
@@ -1435,6 +1430,12 @@ default_sort_size (void)
     size = rlimit.rlim_cur / 16 * 15;
 #endif

+  /* Let MEM be available memory or 1/8 of total memory, whichever
+     is greater.  */
+  double avail = physmem_available ();
+  double total = physmem_total ();
+  double mem = MAX (avail, total / 8);
+
   /* Similarly, leave a 1/4 margin for physical memory.  */
   if (total * 0.75 < size)
     size = total * 0.75;
-- 
1.7.7



From MAILER-DAEMON Thu Jun 14 12:55:44 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1SfDKi-0004pQ-Or
	for mharc-coreutils@gnu.org; Thu, 14 Jun 2012 12:55:44 -0400
Received: from eggs.gnu.org ([208.118.235.92]:55350)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1SfDKf-0004oT-DJ
	for coreutils@gnu.org; Thu, 14 Jun 2012 12:55:42 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1SfDKZ-0000uF-8W
	for coreutils@gnu.org; Thu, 14 Jun 2012 12:55:40 -0400
Received: from mx1.redhat.com ([209.132.183.28]:44569)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>)
	id 1SfDKN-0000if-UG; Thu, 14 Jun 2012 12:55:24 -0400
Received: from int-mx10.intmail.prod.int.phx2.redhat.com
	(int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id q5EGtLpS002009
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK);
	Thu, 14 Jun 2012 12:55:22 -0400
Received: from [10.3.113.3] ([10.3.113.3])
	by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP
	id q5EGtLcY009687; Thu, 14 Jun 2012 12:55:21 -0400
Message-ID: <4FDA1778.3030908@redhat.com>
Date: Thu, 14 Jun 2012 10:55:20 -0600
From: Eric Blake <eblake@redhat.com>
Organization: Red Hat
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:12.0) Gecko/20120430 Thunderbird/12.0.1
MIME-Version: 1.0
To: Akim Demaille <akim@lrde.epita.fr>
Subject: Re: new module: relpath
References: <1CFC4E7F-BFCD-4055-9217-97C6A79975D3@lrde.epita.fr>
In-Reply-To: <1CFC4E7F-BFCD-4055-9217-97C6A79975D3@lrde.epita.fr>
X-Enigmail-Version: 1.4.2
OpenPGP: url=http://people.redhat.com/eblake/eblake.gpg
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature";
	boundary="------------enigBB95F88B67E3CFD6C73C8D43"
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.132.183.28
Cc: Bug-gnulib <bug-gnulib@gnu.org>, Coreutils <coreutils@gnu.org>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 14 Jun 2012 16:55:43 -0000

This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--------------enigBB95F88B67E3CFD6C73C8D43
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

On 06/14/2012 10:02 AM, Akim Demaille wrote:
> Hi all,
>=20
> In Bison, I consider computing relative paths between files
> (one concrete use is when the user creates the parser implementation
> file in --output=3Dsub1/sub2/parser.c and the header file in
> --defines=3Dsub3/sub4/parser.h, in which case parser.c should include
> "../../sub3/sub4/parser.h").

I like the idea, but...

> +/* Return FROM represented as relative to the dir of TARGET.
> +   The result is malloced.  */
> +
> +char *
> +convert_abs_rel (const char *from, const char *target)
> +{
> +  char *realtarget =3D canonicalize_filename_mode (target, CAN_MISSING=
);
> +  char *realfrom =3D canonicalize_filename_mode (from, CAN_MISSING);
> +
> +  /* Write to a PATH_MAX buffer.  */
> +  char *relative_from =3D xmalloc (PATH_MAX);

We really shouldn't be using malloc(PATH_MAX); there are platforms like
Hurd where it is undefined or extremely large (coreutils caps things to
avoid this issue), and you are wasting memory if it is even possible to
allocate this much.  Furthermore, it is possible for the computed
relative path to be longer than PATH_MAX, but still valid (especially
true when we cap PATH_MAX lower than what the system really supports).
This is a bug in coreutils, but we should fix it as part of migrating to
gnulib.

> +
> +  /* Get dirname to generate paths relative to.  */
> +  realtarget[dir_len (realtarget)] =3D '\0';
> +
> +  if (!relpath (realfrom, realtarget, relative_from, PATH_MAX))

Since limiting to PATH_MAX is a bad idea, it seems like a better
interface would be one that returns a char*, already malloc'd, of the
appropriate length, regardless of what PATH_MAX may (or may not) be.
Also, we can compute a runtime bound on the string length - if I'm
reasoning about it correctly, the output should never exceed the maximum
of the canonical target name, or of the original target name plus a
multiple of '../' patterns corresponding to the number of '/' found in
the canonical from name.

--=20
Eric Blake   eblake@redhat.com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


--------------enigBB95F88B67E3CFD6C73C8D43
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)
Comment: Public key at http://people.redhat.com/eblake/eblake.gpg
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQEcBAEBCAAGBQJP2hd5AAoJEKeha0olJ0Nq9xoIAILuvc4ctShLSvGo+PNErX+i
PvUi+ymMEgym4Xe4m4nkHL18ssyJe7ytu2fe3QGf6NQ+qjuxOPFIzzSCRpXqp8F8
VbBrofJli2USDy1CJAZEvi2r3hTJyr0GK/QUDHUAwdbBZ5Cz0RLFfPUtfu6tp8qO
nPiYS9CFX9eYepLxVUGWI4xEvb8CqBOP80YjQSTRsgkmK5SKLX2J4etLRVsbAiKN
AqFIQPpDbqcGBaAx56BXRaLTY+XeEQiwjo/VxCW7bdzVIEDVKaP9z7M0GVmHyLQ9
FxZNxMsIYnrpxb1U4YVS00/irYwQGwCfP9oNDebS8kB5N3ln+zv6HCe2Qp03KSQ=
=mUAI
-----END PGP SIGNATURE-----

--------------enigBB95F88B67E3CFD6C73C8D43--


From MAILER-DAEMON Thu Jun 14 16:22:32 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1SfGYq-00012H-FK
	for mharc-coreutils@gnu.org; Thu, 14 Jun 2012 16:22:32 -0400
Received: from eggs.gnu.org ([208.118.235.92]:45755)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <inpost@gmail.com>) id 1SfGYo-000124-Pn
	for coreutils@gnu.org; Thu, 14 Jun 2012 16:22:31 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <inpost@gmail.com>) id 1SfGYn-0005of-5l
	for coreutils@gnu.org; Thu, 14 Jun 2012 16:22:30 -0400
Received: from mail-pb0-f41.google.com ([209.85.160.41]:50228)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <inpost@gmail.com>) id 1SfGYm-0005nh-T2
	for coreutils@gnu.org; Thu, 14 Jun 2012 16:22:29 -0400
Received: by pbbrp2 with SMTP id rp2so4710318pbb.0
	for <coreutils@gnu.org>; Thu, 14 Jun 2012 13:22:26 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
	h=mime-version:date:message-id:subject:from:to:content-type;
	bh=qHDibJiazXhdBQ4cuaALGqMLViI+J2pZ/eGtdJDpZHk=;
	b=WWsYTpctEXph3bcMC2u8Kod3MRKFztWVGF3p8fYHWAPUQryIKDIunrv/Q2oRCd5sok
	+ThxyGaxJvlF+1gWwZLq6lnShx0g7A1YpE0J8XPICD7m90K8+8QaLDRDcbbKoWYYL3BK
	jnGm26o8r06IEJkaz9sxmmJvfNKJVx2uFvLdB3O7yn84f96N0vXjxqGrwHXNoM9z+fcn
	CQRusndHTrOpBJgu2Q/54WYmqJg8deGZJFo3BdUuOpUkoWQVRWAUr4GAFiuTytsD7rjc
	tOjfDaU2r37le5mimB0INce4DnOfMew0ordDeI59ncQ65sWnPJ5Myu2SfohhtzIb25Oq
	IRXw==
MIME-Version: 1.0
Received: by 10.68.134.161 with SMTP id pl1mr12411526pbb.29.1339705345853;
	Thu, 14 Jun 2012 13:22:25 -0700 (PDT)
Received: by 10.68.135.233 with HTTP; Thu, 14 Jun 2012 13:22:25 -0700 (PDT)
Date: Thu, 14 Jun 2012 21:22:25 +0100
Message-ID: <CAET1fe46FgJ8sYtBZpkYGdH3jXiUqS9HiODCQhax01jaFKqnfQ@mail.gmail.com>
Subject: use comm command with regular expression
From: e-letter <inpost@gmail.com>
To: coreutils@gnu.org
Content-Type: text/plain; charset=UTF-8
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.85.160.41
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 14 Jun 2012 20:22:31 -0000

Readers,

File 1 contains data:

/some/text/abcd.xyz

File 2:

abcd.xyz

The manual does not seem to indicate that regular expressions can be
used with 'comm'. The task is to be able to compare file1 to file2
using a regular expression as a criterion for comparison, such as:

*cd.xyz

Then create a new file 'file3' that contains only those lines that
satisfy the regular expression, but must contain the same format style
as in file1.

Any help please?

Thanks in advance.


From MAILER-DAEMON Fri Jun 15 16:02:15 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1Sfcil-0002Oa-AM
	for mharc-coreutils@gnu.org; Fri, 15 Jun 2012 16:02:15 -0400
Received: from eggs.gnu.org ([208.118.235.92]:56832)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <bob@proulx.com>) id 1Sfcii-0002Na-0X
	for coreutils@gnu.org; Fri, 15 Jun 2012 16:02:13 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <bob@proulx.com>) id 1Sfcig-0006gK-Co
	for coreutils@gnu.org; Fri, 15 Jun 2012 16:02:11 -0400
Received: from joseki.proulx.com ([216.17.153.58]:34459)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <bob@proulx.com>) id 1Sfcig-0006fg-5c
	for coreutils@gnu.org; Fri, 15 Jun 2012 16:02:10 -0400
Received: from hysteria.proulx.com (hysteria.proulx.com [192.168.230.119])
	by joseki.proulx.com (Postfix) with ESMTP id 5BBFA211F1;
	Fri, 15 Jun 2012 14:02:07 -0600 (MDT)
Received: by hysteria.proulx.com (Postfix, from userid 1000)
	id 46B9E2DC61; Fri, 15 Jun 2012 14:02:07 -0600 (MDT)
Date: Fri, 15 Jun 2012 14:02:07 -0600
From: Bob Proulx <bob@proulx.com>
To: e-letter <inpost@gmail.com>
Subject: Re: use comm command with regular expression
Message-ID: <20120615200207.GA19244@hysteria.proulx.com>
Mail-Followup-To: e-letter <inpost@gmail.com>, coreutils@gnu.org
References: <CAET1fe46FgJ8sYtBZpkYGdH3jXiUqS9HiODCQhax01jaFKqnfQ@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <CAET1fe46FgJ8sYtBZpkYGdH3jXiUqS9HiODCQhax01jaFKqnfQ@mail.gmail.com>
User-Agent: Mutt/1.5.21 (2010-09-15)
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 2)
X-Received-From: 216.17.153.58
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 15 Jun 2012 20:02:13 -0000

e-letter wrote:
> File 1 contains data:
> /some/text/abcd.xyz
> 
> File 2:
> abcd.xyz
> 
> The manual does not seem to indicate that regular expressions can be
> used with 'comm'.

They can't be.  It doesn't make sense for comm.  Perhaps you should be
using 'grep' or 'sed'?

> The task is to be able to compare file1 to file2 using a regular
> expression as a criterion for comparison, such as:
> 
> *cd.xyz
> 
> Then create a new file 'file3' that contains only those lines that
> satisfy the regular expression, but must contain the same format style
> as in file1.

Sounds like a homework assignment.

> Any help please?

  $ grep -F cd.xyz file1 > file3

Or perhaps:

  $ grep -f file2 file1 > file3

But note that "*cd.xyz" doesn't make sense as a regular expression.
It looks like a file glob match instead.  Watch that string being used
for a regular expression carefully.  Perhaps "cd\.xyz" instead.

Good luck on working through your homework! :-)

Bob


From MAILER-DAEMON Sat Jun 16 03:02:56 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1Sfn28-0004p3-2L
	for mharc-coreutils@gnu.org; Sat, 16 Jun 2012 03:02:56 -0400
Received: from eggs.gnu.org ([208.118.235.92]:49835)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <inpost@gmail.com>) id 1Sfn25-0004mi-F3
	for coreutils@gnu.org; Sat, 16 Jun 2012 03:02:54 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <inpost@gmail.com>) id 1Sfn23-0003gf-K8
	for coreutils@gnu.org; Sat, 16 Jun 2012 03:02:53 -0400
Received: from mail-pz0-f41.google.com ([209.85.210.41]:34047)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <inpost@gmail.com>) id 1Sfn23-0003gF-Bi
	for coreutils@gnu.org; Sat, 16 Jun 2012 03:02:51 -0400
Received: by dakp5 with SMTP id p5so5230131dak.0
	for <coreutils@gnu.org>; Sat, 16 Jun 2012 00:02:48 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
	h=mime-version:in-reply-to:references:date:message-id:subject:from:to
	:content-type; bh=uDvPqQJ985E/qaNul+vMSEpXk3aKr23DCl7mwsmGFGo=;
	b=LhOmQj6/XirUMR7635NZpArjNEbF+1ytfaPzQQaaHRieL2+mcWOtxUiW/f8IUM0Foj
	Ir4kZY9mwxl271kybrNX/4SshQhv26jN6iJYf27joqJ4IUCytJ/ZjKAq8moMFd3Bt7zC
	tD4Z8+R4EZWEillJlo4XXacuGCkdN3MwscpfBEnYuMu95GxLjpn0JK+ncS1hR/UC52sr
	y4YALYKCJsAjYb5c+guu2GT1rHCzWNYt9lEYC0KvHfGPcbqm0nuyK+GpI8qJFHISkKWo
	TXQcqQFE43r7xXPvoJTjCgubbs3ykKFRE1J6uB3+0aqjATlnALrvPAqsqJy5cqMvVGec
	fbpw==
MIME-Version: 1.0
Received: by 10.68.132.166 with SMTP id ov6mr28693854pbb.24.1339830168052;
	Sat, 16 Jun 2012 00:02:48 -0700 (PDT)
Received: by 10.68.130.228 with HTTP; Sat, 16 Jun 2012 00:02:48 -0700 (PDT)
In-Reply-To: <20120615200207.GA19244@hysteria.proulx.com>
References: <CAET1fe46FgJ8sYtBZpkYGdH3jXiUqS9HiODCQhax01jaFKqnfQ@mail.gmail.com>
	<20120615200207.GA19244@hysteria.proulx.com>
Date: Sat, 16 Jun 2012 08:02:48 +0100
Message-ID: <CAET1fe7VKLkNOavVWUeSgnpEoow61wXVtY5SyNExKJJOu7oUbg@mail.gmail.com>
Subject: Re: use comm command with regular expression
From: e-letter <inpost@gmail.com>
To: coreutils@gnu.org
Content-Type: text/plain; charset=UTF-8
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.85.210.41
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sat, 16 Jun 2012 07:02:54 -0000

On 15/06/2012, Bob Proulx <bob@proulx.com> wrote:
>
> They can't be.  It doesn't make sense for comm.  Perhaps you should be
> using 'grep' or 'sed'?
>

Had a brief look and the manual for 'bash' which makes reference to
grep, but so far awk and perl and supposed to be options. Too many
choices!

>
> Sounds like a homework assignment.
>

Well, my own assignment in my own home (managing files)!


From MAILER-DAEMON Sun Jun 17 20:37:35 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1SgPyJ-0002ux-SF
	for mharc-coreutils@gnu.org; Sun, 17 Jun 2012 20:37:35 -0400
Received: from eggs.gnu.org ([208.118.235.92]:38047)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <bob@proulx.com>) id 1SgPyH-0002un-4A
	for coreutils@gnu.org; Sun, 17 Jun 2012 20:37:34 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <bob@proulx.com>) id 1SgPyF-00089z-5k
	for coreutils@gnu.org; Sun, 17 Jun 2012 20:37:32 -0400
Received: from joseki.proulx.com ([216.17.153.58]:44184)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <bob@proulx.com>) id 1SgPyE-00089m-Qa
	for coreutils@gnu.org; Sun, 17 Jun 2012 20:37:31 -0400
Received: from hysteria.proulx.com (hysteria.proulx.com [192.168.230.119])
	by joseki.proulx.com (Postfix) with ESMTP id BBA11211D8;
	Sun, 17 Jun 2012 18:37:26 -0600 (MDT)
Received: by hysteria.proulx.com (Postfix, from userid 1000)
	id 6EEF62DCC1; Sun, 17 Jun 2012 18:37:26 -0600 (MDT)
Date: Sun, 17 Jun 2012 18:37:26 -0600
From: Bob Proulx <bob@proulx.com>
To: e-letter <inpost@gmail.com>
Subject: Re: use comm command with regular expression
Message-ID: <20120618003726.GA21663@hysteria.proulx.com>
Mail-Followup-To: e-letter <inpost@gmail.com>, coreutils@gnu.org
References: <CAET1fe46FgJ8sYtBZpkYGdH3jXiUqS9HiODCQhax01jaFKqnfQ@mail.gmail.com>
	<20120615200207.GA19244@hysteria.proulx.com>
	<CAET1fe7VKLkNOavVWUeSgnpEoow61wXVtY5SyNExKJJOu7oUbg@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <CAET1fe7VKLkNOavVWUeSgnpEoow61wXVtY5SyNExKJJOu7oUbg@mail.gmail.com>
User-Agent: Mutt/1.5.21 (2010-09-15)
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 2)
X-Received-From: 216.17.153.58
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 18 Jun 2012 00:37:34 -0000

e-letter wrote:
> Bob Proulx wrote:
> > Sounds like a homework assignment.
> 
> Well, my own assignment in my own home (managing files)!

Okay then.  You can imagine that a lot of students try to short
circuit things!

> > They can't be.  It doesn't make sense for comm.  Perhaps you should be
> > using 'grep' or 'sed'?
> 
> Had a brief look and the manual for 'bash' which makes reference to
> grep, but so far awk and perl and supposed to be options. Too many
> choices!

Perhaps if you describe your problem in more detail we on the mailing
list could help better?  So far you have said:

> File 1 contains data:
> /some/text/abcd.xyz
> 
> File 2:
> abcd.xyz

The first contains fully qualified paths to files and the second one
contains only basenames.  Okay.

> The task is to be able to compare file1 to file2 using a regular
> expression as a criterion for comparison, such as:
> 
> *cd.xyz

That looks like a file glob.  It is called globbing because the '*'
matches a glob of characters.  You can get more documentation on that
style of pattern with 'man 7 glob'.  Most importantly globbing is not
a regular expression.  They have different syntaxes from each other.
The command line uses file globbing.  This would match the file in the
current directory from the command line.

  $ ls *cd.xyz

> Then create a new file 'file3' that contains only those lines that
> satisfy the regular expression, but must contain the same format style
> as in file1.

I suggested this:

  $ grep -F cd.xyz file1 > file3

In this above I suggested -F to turn off regular expressions and to
use the string literally.  Otherwise the '.' matches any character and
should be escaped.  Escaping the dot would be "cd\.xyz".  However that
may be difficult if you have a file of partial filenames.  You would
need to preprocess it first.  Easier if you can take them as literal
strings.

Or perhaps:

  $ grep -F -f file2 file1 > file3

Wasn't sure what you were asking for.  In the above it uses the file2
as a list of strings to select from file1.  The -f takes a file
argument.  They will be full regular expressions unless -F is given
too.  It seems like that will give you exactly what you want.

> Had a brief look and the manual for 'bash' which makes reference to
> grep, but so far awk and perl and supposed to be options. Too many
> choices!

Bash is a command line shell that also has features that make it good
for use in controling other programs.  It is an expansion from the
/bin/sh shell.  But it really only has simple strings for data
structures.  For simple things it is great.  For more complex things
then languages like Perl (which includes Ruby, Python and Awk) are
better since they have complex data structures and can do virtually
anything you can think of doing without language limitations.

Awk is the original little scripting language from way back and it is
therefore the most standard and portable.  If I am doing something
relatively simple then I use awk since it can be written to run
everywhere.  But it does have some quirks that make it difficult to
use for larger programs.

Perl then also appeared on the scene some time later.  It combined a
lot of syntax from the shell, sed, grep, awk, and other utilities.
It's popularity has probably peaked.  It was more popular a few years
ago.  The syntax isn't clean and it has a lot of special cases.  This
has caused many people to like the even newer languages of Python and
Ruby which have more recently appeared on the scene.

How to choose?  If 'grep -F -f file2 file1 > file3' does what you want
then I would definitely use the shell and stop there.  If not then I
would move on to Perl.  Or others would suggest either Python or Ruby.

Bob


From MAILER-DAEMON Mon Jun 18 08:50:17 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1SgbPN-0002vU-FY
	for mharc-coreutils@gnu.org; Mon, 18 Jun 2012 08:50:17 -0400
Received: from eggs.gnu.org ([208.118.235.92]:55039)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1SgbPG-0002vL-B7
	for coreutils@gnu.org; Mon, 18 Jun 2012 08:50:16 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1SgbP6-0001lC-HM
	for coreutils@gnu.org; Mon, 18 Jun 2012 08:50:09 -0400
Received: from mx1.redhat.com ([209.132.183.28]:41410)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>)
	id 1SgbOy-0001j2-Ch; Mon, 18 Jun 2012 08:49:52 -0400
Received: from int-mx09.intmail.prod.int.phx2.redhat.com
	(int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id q5ICnnk8029267
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK);
	Mon, 18 Jun 2012 08:49:50 -0400
Received: from [10.3.113.98] (ovpn-113-98.phx2.redhat.com [10.3.113.98])
	by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP
	id q5ICnmeg012489; Mon, 18 Jun 2012 08:49:49 -0400
Message-ID: <4FDF23EC.9040202@redhat.com>
Date: Mon, 18 Jun 2012 06:49:48 -0600
From: Eric Blake <eblake@redhat.com>
Organization: Red Hat
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:13.0) Gecko/20120605 Thunderbird/13.0
MIME-Version: 1.0
To: Akim Demaille <akim@lrde.epita.fr>
Subject: Re: new module: relpath
References: <1CFC4E7F-BFCD-4055-9217-97C6A79975D3@lrde.epita.fr>
	<4FDA1778.3030908@redhat.com>
	<83AA7816-FC0F-486E-B590-9E67D61AA501@lrde.epita.fr>
In-Reply-To: <83AA7816-FC0F-486E-B590-9E67D61AA501@lrde.epita.fr>
X-Enigmail-Version: 1.4.2
OpenPGP: url=http://people.redhat.com/eblake/eblake.gpg
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature";
	boundary="------------enig9CBDCC797329AA023D344301"
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.132.183.28
Cc: Bug-gnulib <bug-gnulib@gnu.org>, Coreutils <coreutils@gnu.org>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 18 Jun 2012 12:50:16 -0000

This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--------------enig9CBDCC797329AA023D344301
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

On 06/18/2012 04:02 AM, Akim Demaille wrote:

>> We really shouldn't be using malloc(PATH_MAX); there are platforms lik=
e
>> Hurd where it is undefined or extremely large (coreutils caps things t=
o
>> avoid this issue), and you are wasting memory if it is even possible t=
o
>> allocate this much.  Furthermore, it is possible for the computed
>> relative path to be longer than PATH_MAX, but still valid (especially
>> true when we cap PATH_MAX lower than what the system really supports).=

>> This is a bug in coreutils, but we should fix it as part of migrating =
to
>> gnulib.
>=20
> Actually relpath uses realpath, and realpath features
> (canonicalize-lgpl):
>=20
> #ifdef PATH_MAX
>   path_max =3D PATH_MAX;
> #else
>   path_max =3D pathconf (name, _PC_PATH_MAX);
>   if (path_max <=3D 0)
>     path_max =3D 8192;
> #endif
>=20
>   if (resolved =3D=3D NULL)
>     {
>       rpath =3D malloc (path_max);
>       if (rpath =3D=3D NULL)
>         {
>           /* It's easier to set errno to ENOMEM than to rely on the
>              'malloc-posix' gnulib module.  */
>           errno =3D ENOMEM;
>           return NULL;
>         }
>     }
>=20
> so I'm not sure fighting that limitations is worth it.

But in the context of canonicalize-lgpl, we are using path_max as an
initial allocation hint, and can allocate larger than that (that is, the
8k-constrained PATH_MAX is not an absolute limit - see the later
realloc() of rpath), whereas the interface of relpath as first proposed
IS constrained by the limit (it does not reallocate larger).  It is
still worth fixing our public interface to not have a needless
constraint; it's okay to start with a conservative guess to minimize
allocation churn in the common case, but only if you can go beyond that
guess.

--=20
Eric Blake   eblake@redhat.com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org




--------------enig9CBDCC797329AA023D344301
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)
Comment: Public key at http://people.redhat.com/eblake/eblake.gpg
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQEcBAEBCAAGBQJP3yPsAAoJEKeha0olJ0NqjUsIAJ6SZc+01EzO4t2jJLylTObo
oZDV8eqwvY2ftje3GmVd6WbB0EOuXDHCyCJ4jJL8Zf5LESsTbCJffBQupYCrphmB
jdU8R7nMwmko7R0kgFjTWAmNBIniagMAntUftizDPX4alFlB+KWqjXEt5yw1PBzO
RcM/DDGrJqiA77XAIiQX/s/mru1VxSSYAYIOk/Jfu5gvhXpwI1RYZWyhedY2QCq6
Yqg/1SZwvvSWWiifXzsBd+lfZa/Xv4w401iRnLlNSXTME6lYhXSjZfy7Y8t5cEHE
Es/JfnwA0wqXtCYmx4xnbJnD+cDsDUwRsz6dPssr5wCK6v6oCNXgLVGGfOM5jQ8=
=NIFv
-----END PGP SIGNATURE-----

--------------enig9CBDCC797329AA023D344301--


From MAILER-DAEMON Mon Jun 18 09:51:06 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1SgcMD-0004eq-VC
	for mharc-coreutils@gnu.org; Mon, 18 Jun 2012 09:51:05 -0400
Received: from eggs.gnu.org ([208.118.235.92]:32891)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <akim@lrde.epita.fr>) id 1SgYnW-0005ab-QD
	for coreutils@gnu.org; Mon, 18 Jun 2012 06:03:08 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <akim@lrde.epita.fr>) id 1SgYnV-0001BF-0Y
	for coreutils@gnu.org; Mon, 18 Jun 2012 06:03:02 -0400
Received: from sao-paulo.lrde.epita.fr ([163.5.55.1]:60039
	helo=kualalumpur.lrde.epita.fr)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <akim@lrde.epita.fr>)
	id 1SgYnR-00019x-41; Mon, 18 Jun 2012 06:02:57 -0400
Received: from erebus.lrde.epita.fr ([192.168.101.165])
	by kualalumpur.lrde.epita.fr with esmtpsa
	(TLS-1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.63)
	(envelope-from <akim@lrde.epita.fr>)
	id 1SgYnO-0003U7-LS; Mon, 18 Jun 2012 12:02:54 +0200
Subject: Re: new module: relpath
Mime-Version: 1.0 (Apple Message framework v1278)
Content-Type: text/plain; charset=iso-8859-1
From: Akim Demaille <akim@lrde.epita.fr>
In-Reply-To: <4FDA1778.3030908@redhat.com>
Date: Mon, 18 Jun 2012 12:02:54 +0200
Content-Transfer-Encoding: quoted-printable
Message-Id: <83AA7816-FC0F-486E-B590-9E67D61AA501@lrde.epita.fr>
References: <1CFC4E7F-BFCD-4055-9217-97C6A79975D3@lrde.epita.fr>
	<4FDA1778.3030908@redhat.com>
To: Eric Blake <eblake@redhat.com>
X-Mailer: Apple Mail (2.1278)
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
X-Received-From: 163.5.55.1
X-Mailman-Approved-At: Mon, 18 Jun 2012 09:51:04 -0400
Cc: Bug-gnulib <bug-gnulib@gnu.org>, Coreutils <coreutils@gnu.org>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 18 Jun 2012 10:03:08 -0000


Le 14 juin 2012 =E0 18:55, Eric Blake a =E9crit :

>> +/* Return FROM represented as relative to the dir of TARGET.
>> +   The result is malloced.  */
>> +
>> +char *
>> +convert_abs_rel (const char *from, const char *target)
>> +{
>> +  char *realtarget =3D canonicalize_filename_mode (target, =
CAN_MISSING);
>> +  char *realfrom =3D canonicalize_filename_mode (from, CAN_MISSING);
>> +
>> +  /* Write to a PATH_MAX buffer.  */
>> +  char *relative_from =3D xmalloc (PATH_MAX);
>=20
> We really shouldn't be using malloc(PATH_MAX); there are platforms =
like
> Hurd where it is undefined or extremely large (coreutils caps things =
to
> avoid this issue), and you are wasting memory if it is even possible =
to
> allocate this much.  Furthermore, it is possible for the computed
> relative path to be longer than PATH_MAX, but still valid (especially
> true when we cap PATH_MAX lower than what the system really supports).
> This is a bug in coreutils, but we should fix it as part of migrating =
to
> gnulib.

Actually relpath uses realpath, and realpath features
(canonicalize-lgpl):

#ifdef PATH_MAX
  path_max =3D PATH_MAX;
#else
  path_max =3D pathconf (name, _PC_PATH_MAX);
  if (path_max <=3D 0)
    path_max =3D 8192;
#endif

  if (resolved =3D=3D NULL)
    {
      rpath =3D malloc (path_max);
      if (rpath =3D=3D NULL)
        {
          /* It's easier to set errno to ENOMEM than to rely on the
             'malloc-posix' gnulib module.  */
          errno =3D ENOMEM;
          return NULL;
        }
    }

so I'm not sure fighting that limitations is worth it.



From MAILER-DAEMON Mon Jun 18 15:24:39 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1SghZ1-0000H1-9B
	for mharc-coreutils@gnu.org; Mon, 18 Jun 2012 15:24:39 -0400
Received: from eggs.gnu.org ([208.118.235.92]:35489)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <chen.guo.0625@gmail.com>) id 1SghYw-0000Gv-QF
	for coreutils@gnu.org; Mon, 18 Jun 2012 15:24:37 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <chen.guo.0625@gmail.com>) id 1SghYt-00005K-87
	for coreutils@gnu.org; Mon, 18 Jun 2012 15:24:34 -0400
Received: from mail-yx0-f169.google.com ([209.85.213.169]:37377)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <chen.guo.0625@gmail.com>) id 1SghYs-0008WS-UZ
	for coreutils@gnu.org; Mon, 18 Jun 2012 15:24:31 -0400
Received: by yenr5 with SMTP id r5so4063568yen.0
	for <coreutils@gnu.org>; Mon, 18 Jun 2012 12:24:28 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
	h=mime-version:in-reply-to:references:date:message-id:subject:from:to
	:cc:content-type;
	bh=FtjVyjZO/zaHftuI3sdzSmm81uJ7V8EFbB3j3TwSxdE=;
	b=fk2KIG9SDP5jSVwKe6FFrTj07HiWppvsrodp/siw3zPaDNxDUanJ4mdZxEWT2JDRvY
	ZqikY4a3gsvPeAQl2dwywwKse18KMxeVV/m4ib7jDayOyqRBPh060WbkYXAGTKMl08Qj
	zLVtP7vHLXf6jZVzwvmnlB3AB37NXmNhY5PjU1b0ohHfsVFwmdqgYWJPbdiptgJ+APtH
	bzSMMgqUraf7BLl0ZtJDQE69JXywb6tj6ZIDjsQiOGYDWkB0TNd92vcue7Ru3M1zp0Go
	EmszKPPix+a5+h3K3oYaUsxRY2Y1XDEJJWF6HqybsTbAJDaojAN9//CDUNbUNcB2VkUs
	8mhg==
MIME-Version: 1.0
Received: by 10.50.171.98 with SMTP id at2mr9512397igc.26.1340047467591; Mon,
	18 Jun 2012 12:24:27 -0700 (PDT)
Received: by 10.50.104.9 with HTTP; Mon, 18 Jun 2012 12:24:27 -0700 (PDT)
In-Reply-To: <4FD143E0.8000306@draigBrady.com>
References: <4B8CE672.5030606@draigBrady.com>
	<657794.87632.qm@web180001.mail.gq1.yahoo.com>
	<4B8DB5FE.6010006@draigBrady.com> <4FD143E0.8000306@draigBrady.com>
Date: Mon, 18 Jun 2012 12:24:27 -0700
Message-ID: <CANxwL_BFM7VtQX-544MztPhJ+j5N8hdrTdE8=iOp3DXwgw9VcA@mail.gmail.com>
Subject: Re: Taking advantage of L1 and L2 cache in sort
From: Chen Guo <chen.guo.0625@gmail.com>
To: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigbrady.com>
Content-Type: multipart/alternative; boundary=e89a8f234b3349b9c004c2c41de4
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.85.213.169
Cc: Joey Degges <jdegges@gmail.com>, coreutils@gnu.org,
	Shaun Jackman <sjackman@bcgsc.ca>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 18 Jun 2012 19:24:38 -0000

--e89a8f234b3349b9c004c2c41de4
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Fascinating, Padraig. I wonder, did this email just get to my inbox really,
really late, or what prompted you to dig up a 2 year old thread?

Anyhow, it seems you had to specifically unset MALLOC_PERTURB_,
so is it by default set? I forget if we ever made the change to default to
a small buffer, but either way if desired I can make the change to read
MALLOC_PERTURB_ from env[] and set the buffer size accordingly
based on that.

On Thu, Jun 7, 2012 at 5:14 PM, P=E1draig Brady <P@draigbrady.com> wrote:

> On 03/03/2010 01:06 AM, P=E1draig Brady wrote:
> > On 02/03/10 18:20, Chen Guo wrote:
> >> This is exactly what that guy Shaun Jackman was talking about earlier.
> >> I'm actually really surprised this is faster, if I can dig up his
> e-mail I'll
> >> forward him this, I remember him saying something about experimenting
> >> with exactly this.
> >
> > I missed that thread but yes he pretty much had the
> > same idea as I stumbled on when trying to perturb
> > the posix_fadvise() testing by changing the buffer size.
> > http://lists.gnu.org/archive/html/bug-coreutils/2010-02/msg00151.html
> > Spooky :)
> >
> > Shaun, you can use `taskset` to set process affinity BTW.
> >
> >> Can you profile the difference in the number of I/O system calls?
> >
> > $ TMPDIR=3D/ram LANG=3DC /usr/bin/time -v strace -c ./src/sort
> sort.t/sort.1.test > /dev/null
> > % time     seconds  usecs/call     calls    errors syscall
> > ------ ----------- ----------- --------- --------- ----------------
> >  70.70    0.283077       21775        13           read
> >  28.97    0.115983       19331         6           munmap
> >   0.32    0.001268           0     21609           write
> >   0.01    0.000054           8         7           open
> >   0.00    0.000000           0         9           close
> >   0.00    0.000000           0         1           execve
> >   0.00    0.000000           0         1         1 access
> >   0.00    0.000000           0         3           brk
> >   0.00    0.000000           0         1         1 ioctl
> >   0.00    0.000000           0         1           uname
> >   0.00    0.000000           0         5           mprotect
> >   0.00    0.000000           0        25           rt_sigaction
> >   0.00    0.000000           0         1           rt_sigprocmask
> >   0.00    0.000000           0         4           getrlimit
> >   0.00    0.000000           0        16           mmap2
> >   0.00    0.000000           0         9           fstat64
> >   0.00    0.000000           0         2         1 futex
> >   0.00    0.000000           0         1           set_thread_area
> >   0.00    0.000000           0         1           set_tid_address
> >   0.00    0.000000           0         1           fadvise64_64
> >   0.00    0.000000           0         1           set_robust_list
> > ------ ----------- ----------- --------- --------- ----------------
> > 100.00    0.400382                 21717         3 total
> >     Command being timed: "strace -c ./src/sort sort.t/sort.1.test"
> >     User time (seconds): 26.91
> >     System time (seconds): 2.01
> >     Percent of CPU this job got: 90%
> >     Elapsed (wall clock) time (h:mm:ss or m:ss): 0:32.02
> >     Average shared text size (kbytes): 0
> >     Average unshared data size (kbytes): 0
> >     Average stack size (kbytes): 0
> >     Average total size (kbytes): 0
> >     Maximum resident set size (kbytes): 0
> >     Average resident set size (kbytes): 0
> >     Major (requiring I/O) page faults: 3
> >     Minor (reclaiming a frame) page faults: 181060
> >     Voluntary context switches: 87362
> >     Involuntary context switches: 2526
> >     Swaps: 0
> >     File system inputs: 173504
> >     File system outputs: 0
> >     Socket messages sent: 0
> >     Socket messages received: 0
> >     Signals delivered: 0
> >     Page size (bytes): 4096
> >     Exit status: 0
> >
> > $ TMPDIR=3D/ram LANG=3DC /usr/bin/time -v strace -c ./src/sort -S1M
> sort.t/sort.1.test > /dev/null
> > % time     seconds  usecs/call     calls    errors syscall
> > ------ ----------- ----------- --------- --------- ----------------
> >  38.95    0.035011           1     60991           read
> >  33.47    0.030081          90       334           unlink
> >  24.17    0.021721           0     81864           write
> >   2.07    0.001862           2      1006           munmap
> >   0.75    0.000670           1       673           open
> >   0.23    0.000209           0      1016           mmap2
> >   0.19    0.000167           0       675           fstat64
> >   0.09    0.000085           0       675           close
> >   0.07    0.000062           0       334           fcntl64
> >   0.02    0.000018           0      1337           rt_sigprocmask
> >   0.00    0.000000           0         1           execve
> >   0.00    0.000000           0         1         1 access
> >   0.00    0.000000           0         3           brk
> >   0.00    0.000000           0         1         1 ioctl
> >   0.00    0.000000           0         1           gettimeofday
> >   0.00    0.000000           0         1           uname
> >   0.00    0.000000           0         5           mprotect
> >   0.00    0.000000           0       334           _llseek
> >   0.00    0.000000           0        25           rt_sigaction
> >   0.00    0.000000           0         1           getrlimit
> >   0.00    0.000000           0         2         1 futex
> >   0.00    0.000000           0         1           set_thread_area
> >   0.00    0.000000           0         1           set_tid_address
> >   0.00    0.000000           0       335           fadvise64_64
> >   0.00    0.000000           0         1           set_robust_list
> > ------ ----------- ----------- --------- --------- ----------------
> > 100.00    0.089886                149618         3 total
> >     Command being timed: "strace -c ./src/sort -S1M sort.t/sort.1.test"
> >     User time (seconds): 21.76
> >     System time (seconds): 4.51
> >     Percent of CPU this job got: 98%
> >     Elapsed (wall clock) time (h:mm:ss or m:ss): 0:26.79
> >     Average shared text size (kbytes): 0
> >     Average unshared data size (kbytes): 0
> >     Average stack size (kbytes): 0
> >     Average total size (kbytes): 0
> >     Maximum resident set size (kbytes): 0
> >     Average resident set size (kbytes): 0
> >     Major (requiring I/O) page faults: 3
> >     Minor (reclaiming a frame) page faults: 23038
> >     Voluntary context switches: 598317
> >     Involuntary context switches: 2316
> >     Swaps: 0
> >     File system inputs: 173504
> >     File system outputs: 0
> >     Socket messages sent: 0
> >     Socket messages received: 0
> >     Signals delivered: 0
> >     Page size (bytes): 4096
> >     Exit status: 0
>
> Hmm I think the results above are invalid
> due to MALLOC_PERTURB_ biasing against large buffers.
> In the test below the extra external file process dominates
> any gains from CPU cache usage:
>
> $ # Test setup
> $ cache_size() { lscpu | grep cache | tail -n1 | tr -s ' ' | cut -d' '
> -f3; }
> $ export TMPDIR=3D/dev/shm # RAM
> $ export OMP_NUM_THREADS=3D1 # limit to a single CPU to simplify
> $ unset MALLOC_PERTURB_ # negatively impacts performance tests
> $ shuf -i 1-10000000 > sort.test # 78MB
>
> $ LANG=3DC /usr/bin/time -v strace -c sort -S$(cache_size) sort.test >
> /dev/null
> % time     seconds  usecs/call     calls    errors syscall
> ------ ----------- ----------- --------- --------- ----------------
>  66.33    0.034878         185       189           unlink
>  17.89    0.009408           0     45766           read
>  8.73    0.004590           0     57711           write
>  4.53    0.002381           4       571           munmap
>  1.28    0.000671           2       382           open
>  0.46    0.000242           1       384           fstat
>  0.35    0.000183           0       580           mmap
>  0.19    0.000100           0       384           close
>  0.14    0.000074           0       757           rt_sigprocmask
>  0.11    0.000058           0       189           fcntl
>  0.00    0.000000           0       189           lseek
>  0.00    0.000000           0         5           mprotect
>  0.00    0.000000           0         4           brk
>  0.00    0.000000           0        25           rt_sigaction
>  0.00    0.000000           0         1         1 ioctl
>  0.00    0.000000           0         1         1 access
>  0.00    0.000000           0         1           execve
>  0.00    0.000000           0         1           getrlimit
>  0.00    0.000000           0         1           arch_prctl
>  0.00    0.000000           0         1           set_tid_address
>  0.00    0.000000           0       190           fadvise64
>  0.00    0.000000           0         1           set_robust_list
> ------ ----------- ----------- --------- --------- ----------------
> 100.00    0.052585                107333         2 total
>        Command being timed: "strace -c sort --parallel=3D1 -T/dev/shm
> -S3072K sort.test"
>        User time (seconds): 25.88
>        System time (seconds): 5.81
>        Percent of CPU this job got: 94%
>        Elapsed (wall clock) time (h:mm:ss or m:ss): 0:33.59
>        Average shared text size (kbytes): 0
>        Average unshared data size (kbytes): 0
>        Average stack size (kbytes): 0
>        Average total size (kbytes): 0
>        Maximum resident set size (kbytes): 4968
>        Average resident set size (kbytes): 0
>        Major (requiring I/O) page faults: 0
>        Minor (reclaiming a frame) page faults: 13900
>        Voluntary context switches: 429178
>        Involuntary context switches: 4882
>        Swaps: 0
>        File system inputs: 0
>        File system outputs: 0
>        Socket messages sent: 0
>        Socket messages received: 0
>        Signals delivered: 0
>        Page size (bytes): 4096
>        Exit status: 0
>
>
> $ LANG=3DC /usr/bin/time -v strace -c sort -S200M sort.test > /dev/null
> % time     seconds  usecs/call     calls    errors syscall
> ------ ----------- ----------- --------- --------- ----------------
>  80.68    0.348335         565       617           read
>  11.58    0.049994        3846        13           munmap
>  5.33    0.022995        7665         3           unlink
>  2.38    0.010281           0     38521           write
>  0.01    0.000052          52         1         1 access
>  0.01    0.000042           8         5           mprotect
>  0.01    0.000032           1        25           rt_sigaction
>  0.00    0.000000           0        10           open
>  0.00    0.000000           0        12           close
>  0.00    0.000000           0        12           fstat
>  0.00    0.000000           0         3           lseek
>  0.00    0.000000           0        22           mmap
>  0.00    0.000000           0         4           brk
>  0.00    0.000000           0        13           rt_sigprocmask
>  0.00    0.000000           0         1         1 ioctl
>  0.00    0.000000           0         1           execve
>  0.00    0.000000           0         3           fcntl
>  0.00    0.000000           0         1           getrlimit
>  0.00    0.000000           0         1           arch_prctl
>  0.00    0.000000           0         1           set_tid_address
>  0.00    0.000000           0         4           fadvise64
>  0.00    0.000000           0         1           set_robust_list
> ------ ----------- ----------- --------- --------- ----------------
> 100.00    0.431731                 39274         2 total
>        Command being timed: "strace -c sort -S200M sort.test"
>        User time (seconds): 19.82
>        System time (seconds): 4.33
>        Percent of CPU this job got: 94%
>        Elapsed (wall clock) time (h:mm:ss or m:ss): 0:25.57
>        Average shared text size (kbytes): 0
>        Average unshared data size (kbytes): 0
>        Average stack size (kbytes): 0
>        Average total size (kbytes): 0
>        Maximum resident set size (kbytes): 205532
>        Average resident set size (kbytes): 0
>        Major (requiring I/O) page faults: 355
>        Minor (reclaiming a frame) page faults: 98057
>        Voluntary context switches: 159217
>        Involuntary context switches: 4681
>        Swaps: 0
>        File system inputs: 140344
>        File system outputs: 154168
>        Socket messages sent: 0
>        Socket messages received: 0
>        Signals delivered: 0
>        Page size (bytes): 4096
>        Exit status: 0
>
> I notice also that performance gets worse for the above input
> for the small buffer size when I enable multithreading on my 4 core i3.
>        User time (seconds): 35.40
>        System time (seconds): 9.20
>        Percent of CPU this job got: 120%
>        Elapsed (wall clock) time (h:mm:ss or m:ss): 0:36.98
>
> cheers,
> P=E1draig.
>

--e89a8f234b3349b9c004c2c41de4
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Fascinating, Padraig. I wonder, did this email just get to my inbox really,=
<br>really late, or what prompted you to dig up a 2 year old thread?<br><br=
>Anyhow, it seems you had to specifically unset MALLOC_PERTURB_,<br>so is i=
t by default set? I forget if we ever made the change to default to<br>
a small buffer, but either way if desired I can make the change to read<br>=
MALLOC_PERTURB_ from env[] and set the buffer size accordingly<br>based on =
that.<br><br><div class=3D"gmail_quote">On Thu, Jun 7, 2012 at 5:14 PM, P=
=E1draig Brady <span dir=3D"ltr">&lt;<a href=3D"mailto:P@draigbrady.com" ta=
rget=3D"_blank">P@draigbrady.com</a>&gt;</span> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">On 03/03/2010 01:06 AM, P=E1draig Brady wrot=
e:<br>
&gt; On 02/03/10 18:20, Chen Guo wrote:<br>
&gt;&gt; This is exactly what that guy Shaun Jackman was talking about earl=
ier.<br>
&gt;&gt; I&#39;m actually really surprised this is faster, if I can dig up =
his e-mail I&#39;ll<br>
&gt;&gt; forward him this, I remember him saying something about experiment=
ing<br>
&gt;&gt; with exactly this.<br>
&gt;<br>
&gt; I missed that thread but yes he pretty much had the<br>
&gt; same idea as I stumbled on when trying to perturb<br>
&gt; the posix_fadvise() testing by changing the buffer size.<br>
&gt; <a href=3D"http://lists.gnu.org/archive/html/bug-coreutils/2010-02/msg=
00151.html" target=3D"_blank">http://lists.gnu.org/archive/html/bug-coreuti=
ls/2010-02/msg00151.html</a><br>
&gt; Spooky :)<br>
&gt;<br>
&gt; Shaun, you can use `taskset` to set process affinity BTW.<br>
&gt;<br>
&gt;&gt; Can you profile the difference in the number of I/O system calls?<=
br>
&gt;<br>
&gt; $ TMPDIR=3D/ram LANG=3DC /usr/bin/time -v strace -c ./src/sort sort.t/=
sort.1.test &gt; /dev/null<br>
&gt; % time =A0 =A0 seconds =A0usecs/call =A0 =A0 calls =A0 =A0errors sysca=
ll<br>
&gt; ------ ----------- ----------- --------- --------- ----------------<br=
>
&gt; =A070.70 =A0 =A00.283077 =A0 =A0 =A0 21775 =A0 =A0 =A0 =A013 =A0 =A0 =
=A0 =A0 =A0 read<br>
&gt; =A028.97 =A0 =A00.115983 =A0 =A0 =A0 19331 =A0 =A0 =A0 =A0 6 =A0 =A0 =
=A0 =A0 =A0 munmap<br>
&gt; =A0 0.32 =A0 =A00.001268 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 21609 =A0 =A0 =
=A0 =A0 =A0 write<br>
&gt; =A0 0.01 =A0 =A00.000054 =A0 =A0 =A0 =A0 =A0 8 =A0 =A0 =A0 =A0 7 =A0 =
=A0 =A0 =A0 =A0 open<br>
&gt; =A0 0.00 =A0 =A00.000000 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 =A0 9 =A0 =
=A0 =A0 =A0 =A0 close<br>
&gt; =A0 0.00 =A0 =A00.000000 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 =A0 1 =A0 =
=A0 =A0 =A0 =A0 execve<br>
&gt; =A0 0.00 =A0 =A00.000000 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 =A0 1 =A0 =
=A0 =A0 =A0 1 access<br>
&gt; =A0 0.00 =A0 =A00.000000 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 =A0 3 =A0 =
=A0 =A0 =A0 =A0 brk<br>
&gt; =A0 0.00 =A0 =A00.000000 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 =A0 1 =A0 =
=A0 =A0 =A0 1 ioctl<br>
&gt; =A0 0.00 =A0 =A00.000000 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 =A0 1 =A0 =
=A0 =A0 =A0 =A0 uname<br>
&gt; =A0 0.00 =A0 =A00.000000 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 =A0 5 =A0 =
=A0 =A0 =A0 =A0 mprotect<br>
&gt; =A0 0.00 =A0 =A00.000000 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 =A025 =A0 =
=A0 =A0 =A0 =A0 rt_sigaction<br>
&gt; =A0 0.00 =A0 =A00.000000 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 =A0 1 =A0 =
=A0 =A0 =A0 =A0 rt_sigprocmask<br>
&gt; =A0 0.00 =A0 =A00.000000 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 =A0 4 =A0 =
=A0 =A0 =A0 =A0 getrlimit<br>
&gt; =A0 0.00 =A0 =A00.000000 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 =A016 =A0 =
=A0 =A0 =A0 =A0 mmap2<br>
&gt; =A0 0.00 =A0 =A00.000000 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 =A0 9 =A0 =
=A0 =A0 =A0 =A0 fstat64<br>
&gt; =A0 0.00 =A0 =A00.000000 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 =A0 2 =A0 =
=A0 =A0 =A0 1 futex<br>
&gt; =A0 0.00 =A0 =A00.000000 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 =A0 1 =A0 =
=A0 =A0 =A0 =A0 set_thread_area<br>
&gt; =A0 0.00 =A0 =A00.000000 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 =A0 1 =A0 =
=A0 =A0 =A0 =A0 set_tid_address<br>
&gt; =A0 0.00 =A0 =A00.000000 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 =A0 1 =A0 =
=A0 =A0 =A0 =A0 fadvise64_64<br>
&gt; =A0 0.00 =A0 =A00.000000 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 =A0 1 =A0 =
=A0 =A0 =A0 =A0 set_robust_list<br>
&gt; ------ ----------- ----------- --------- --------- ----------------<br=
>
&gt; 100.00 =A0 =A00.400382 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 21717 =A0 =A0 =
=A0 =A0 3 total<br>
&gt; =A0 =A0 Command being timed: &quot;strace -c ./src/sort sort.t/sort.1.=
test&quot;<br>
&gt; =A0 =A0 User time (seconds): 26.91<br>
&gt; =A0 =A0 System time (seconds): 2.01<br>
&gt; =A0 =A0 Percent of CPU this job got: 90%<br>
&gt; =A0 =A0 Elapsed (wall clock) time (h:mm:ss or m:ss): 0:32.02<br>
&gt; =A0 =A0 Average shared text size (kbytes): 0<br>
&gt; =A0 =A0 Average unshared data size (kbytes): 0<br>
&gt; =A0 =A0 Average stack size (kbytes): 0<br>
&gt; =A0 =A0 Average total size (kbytes): 0<br>
&gt; =A0 =A0 Maximum resident set size (kbytes): 0<br>
&gt; =A0 =A0 Average resident set size (kbytes): 0<br>
&gt; =A0 =A0 Major (requiring I/O) page faults: 3<br>
&gt; =A0 =A0 Minor (reclaiming a frame) page faults: 181060<br>
&gt; =A0 =A0 Voluntary context switches: 87362<br>
&gt; =A0 =A0 Involuntary context switches: 2526<br>
&gt; =A0 =A0 Swaps: 0<br>
&gt; =A0 =A0 File system inputs: 173504<br>
&gt; =A0 =A0 File system outputs: 0<br>
&gt; =A0 =A0 Socket messages sent: 0<br>
&gt; =A0 =A0 Socket messages received: 0<br>
&gt; =A0 =A0 Signals delivered: 0<br>
&gt; =A0 =A0 Page size (bytes): 4096<br>
&gt; =A0 =A0 Exit status: 0<br>
&gt;<br>
&gt; $ TMPDIR=3D/ram LANG=3DC /usr/bin/time -v strace -c ./src/sort -S1M so=
rt.t/sort.1.test &gt; /dev/null<br>
&gt; % time =A0 =A0 seconds =A0usecs/call =A0 =A0 calls =A0 =A0errors sysca=
ll<br>
&gt; ------ ----------- ----------- --------- --------- ----------------<br=
>
&gt; =A038.95 =A0 =A00.035011 =A0 =A0 =A0 =A0 =A0 1 =A0 =A0 60991 =A0 =A0 =
=A0 =A0 =A0 read<br>
&gt; =A033.47 =A0 =A00.030081 =A0 =A0 =A0 =A0 =A090 =A0 =A0 =A0 334 =A0 =A0=
 =A0 =A0 =A0 unlink<br>
&gt; =A024.17 =A0 =A00.021721 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 81864 =A0 =A0 =
=A0 =A0 =A0 write<br>
&gt; =A0 2.07 =A0 =A00.001862 =A0 =A0 =A0 =A0 =A0 2 =A0 =A0 =A01006 =A0 =A0=
 =A0 =A0 =A0 munmap<br>
&gt; =A0 0.75 =A0 =A00.000670 =A0 =A0 =A0 =A0 =A0 1 =A0 =A0 =A0 673 =A0 =A0=
 =A0 =A0 =A0 open<br>
&gt; =A0 0.23 =A0 =A00.000209 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A01016 =A0 =A0=
 =A0 =A0 =A0 mmap2<br>
&gt; =A0 0.19 =A0 =A00.000167 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 675 =A0 =A0=
 =A0 =A0 =A0 fstat64<br>
&gt; =A0 0.09 =A0 =A00.000085 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 675 =A0 =A0=
 =A0 =A0 =A0 close<br>
&gt; =A0 0.07 =A0 =A00.000062 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 334 =A0 =A0=
 =A0 =A0 =A0 fcntl64<br>
&gt; =A0 0.02 =A0 =A00.000018 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A01337 =A0 =A0=
 =A0 =A0 =A0 rt_sigprocmask<br>
&gt; =A0 0.00 =A0 =A00.000000 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 =A0 1 =A0 =
=A0 =A0 =A0 =A0 execve<br>
&gt; =A0 0.00 =A0 =A00.000000 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 =A0 1 =A0 =
=A0 =A0 =A0 1 access<br>
&gt; =A0 0.00 =A0 =A00.000000 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 =A0 3 =A0 =
=A0 =A0 =A0 =A0 brk<br>
&gt; =A0 0.00 =A0 =A00.000000 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 =A0 1 =A0 =
=A0 =A0 =A0 1 ioctl<br>
&gt; =A0 0.00 =A0 =A00.000000 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 =A0 1 =A0 =
=A0 =A0 =A0 =A0 gettimeofday<br>
&gt; =A0 0.00 =A0 =A00.000000 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 =A0 1 =A0 =
=A0 =A0 =A0 =A0 uname<br>
&gt; =A0 0.00 =A0 =A00.000000 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 =A0 5 =A0 =
=A0 =A0 =A0 =A0 mprotect<br>
&gt; =A0 0.00 =A0 =A00.000000 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 334 =A0 =A0=
 =A0 =A0 =A0 _llseek<br>
&gt; =A0 0.00 =A0 =A00.000000 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 =A025 =A0 =
=A0 =A0 =A0 =A0 rt_sigaction<br>
&gt; =A0 0.00 =A0 =A00.000000 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 =A0 1 =A0 =
=A0 =A0 =A0 =A0 getrlimit<br>
&gt; =A0 0.00 =A0 =A00.000000 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 =A0 2 =A0 =
=A0 =A0 =A0 1 futex<br>
&gt; =A0 0.00 =A0 =A00.000000 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 =A0 1 =A0 =
=A0 =A0 =A0 =A0 set_thread_area<br>
&gt; =A0 0.00 =A0 =A00.000000 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 =A0 1 =A0 =
=A0 =A0 =A0 =A0 set_tid_address<br>
&gt; =A0 0.00 =A0 =A00.000000 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 335 =A0 =A0=
 =A0 =A0 =A0 fadvise64_64<br>
&gt; =A0 0.00 =A0 =A00.000000 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 =A0 1 =A0 =
=A0 =A0 =A0 =A0 set_robust_list<br>
&gt; ------ ----------- ----------- --------- --------- ----------------<br=
>
&gt; 100.00 =A0 =A00.089886 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0149618 =A0 =A0 =
=A0 =A0 3 total<br>
&gt; =A0 =A0 Command being timed: &quot;strace -c ./src/sort -S1M sort.t/so=
rt.1.test&quot;<br>
&gt; =A0 =A0 User time (seconds): 21.76<br>
&gt; =A0 =A0 System time (seconds): 4.51<br>
&gt; =A0 =A0 Percent of CPU this job got: 98%<br>
&gt; =A0 =A0 Elapsed (wall clock) time (h:mm:ss or m:ss): 0:26.79<br>
&gt; =A0 =A0 Average shared text size (kbytes): 0<br>
&gt; =A0 =A0 Average unshared data size (kbytes): 0<br>
&gt; =A0 =A0 Average stack size (kbytes): 0<br>
&gt; =A0 =A0 Average total size (kbytes): 0<br>
&gt; =A0 =A0 Maximum resident set size (kbytes): 0<br>
&gt; =A0 =A0 Average resident set size (kbytes): 0<br>
&gt; =A0 =A0 Major (requiring I/O) page faults: 3<br>
&gt; =A0 =A0 Minor (reclaiming a frame) page faults: 23038<br>
&gt; =A0 =A0 Voluntary context switches: 598317<br>
&gt; =A0 =A0 Involuntary context switches: 2316<br>
&gt; =A0 =A0 Swaps: 0<br>
&gt; =A0 =A0 File system inputs: 173504<br>
&gt; =A0 =A0 File system outputs: 0<br>
&gt; =A0 =A0 Socket messages sent: 0<br>
&gt; =A0 =A0 Socket messages received: 0<br>
&gt; =A0 =A0 Signals delivered: 0<br>
&gt; =A0 =A0 Page size (bytes): 4096<br>
&gt; =A0 =A0 Exit status: 0<br>
<br>
Hmm I think the results above are invalid<br>
due to MALLOC_PERTURB_ biasing against large buffers.<br>
In the test below the extra external file process dominates<br>
any gains from CPU cache usage:<br>
<br>
$ # Test setup<br>
$ cache_size() { lscpu | grep cache | tail -n1 | tr -s &#39; &#39; | cut -d=
&#39; &#39; -f3; }<br>
$ export TMPDIR=3D/dev/shm # RAM<br>
$ export OMP_NUM_THREADS=3D1 # limit to a single CPU to simplify<br>
$ unset MALLOC_PERTURB_ # negatively impacts performance tests<br>
$ shuf -i 1-10000000 &gt; sort.test # 78MB<br>
<br>
$ LANG=3DC /usr/bin/time -v strace -c sort -S$(cache_size) sort.test &gt; /=
dev/null<br>
% time =A0 =A0 seconds =A0usecs/call =A0 =A0 calls =A0 =A0errors syscall<br=
>
------ ----------- ----------- --------- --------- ----------------<br>
=A066.33 =A0 =A00.034878 =A0 =A0 =A0 =A0 185 =A0 =A0 =A0 189 =A0 =A0 =A0 =
=A0 =A0 unlink<br>
=A017.89 =A0 =A00.009408 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 45766 =A0 =A0 =A0 =
=A0 =A0 read<br>
 =A08.73 =A0 =A00.004590 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 57711 =A0 =A0 =A0 =
=A0 =A0 write<br>
 =A04.53 =A0 =A00.002381 =A0 =A0 =A0 =A0 =A0 4 =A0 =A0 =A0 571 =A0 =A0 =A0 =
=A0 =A0 munmap<br>
 =A01.28 =A0 =A00.000671 =A0 =A0 =A0 =A0 =A0 2 =A0 =A0 =A0 382 =A0 =A0 =A0 =
=A0 =A0 open<br>
 =A00.46 =A0 =A00.000242 =A0 =A0 =A0 =A0 =A0 1 =A0 =A0 =A0 384 =A0 =A0 =A0 =
=A0 =A0 fstat<br>
 =A00.35 =A0 =A00.000183 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 580 =A0 =A0 =A0 =
=A0 =A0 mmap<br>
 =A00.19 =A0 =A00.000100 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 384 =A0 =A0 =A0 =
=A0 =A0 close<br>
 =A00.14 =A0 =A00.000074 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 757 =A0 =A0 =A0 =
=A0 =A0 rt_sigprocmask<br>
 =A00.11 =A0 =A00.000058 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 189 =A0 =A0 =A0 =
=A0 =A0 fcntl<br>
 =A00.00 =A0 =A00.000000 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 189 =A0 =A0 =A0 =
=A0 =A0 lseek<br>
 =A00.00 =A0 =A00.000000 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 =A0 5 =A0 =A0 =
=A0 =A0 =A0 mprotect<br>
 =A00.00 =A0 =A00.000000 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 =A0 4 =A0 =A0 =
=A0 =A0 =A0 brk<br>
 =A00.00 =A0 =A00.000000 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 =A025 =A0 =A0 =
=A0 =A0 =A0 rt_sigaction<br>
 =A00.00 =A0 =A00.000000 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 =A0 1 =A0 =A0 =
=A0 =A0 1 ioctl<br>
 =A00.00 =A0 =A00.000000 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 =A0 1 =A0 =A0 =
=A0 =A0 1 access<br>
 =A00.00 =A0 =A00.000000 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 =A0 1 =A0 =A0 =
=A0 =A0 =A0 execve<br>
 =A00.00 =A0 =A00.000000 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 =A0 1 =A0 =A0 =
=A0 =A0 =A0 getrlimit<br>
 =A00.00 =A0 =A00.000000 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 =A0 1 =A0 =A0 =
=A0 =A0 =A0 arch_prctl<br>
 =A00.00 =A0 =A00.000000 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 =A0 1 =A0 =A0 =
=A0 =A0 =A0 set_tid_address<br>
 =A00.00 =A0 =A00.000000 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 190 =A0 =A0 =A0 =
=A0 =A0 fadvise64<br>
 =A00.00 =A0 =A00.000000 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 =A0 1 =A0 =A0 =
=A0 =A0 =A0 set_robust_list<br>
------ ----------- ----------- --------- --------- ----------------<br>
100.00 =A0 =A00.052585 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0107333 =A0 =A0 =A0 =
=A0 2 total<br>
 =A0 =A0 =A0 =A0Command being timed: &quot;strace -c sort --parallel=3D1 -T=
/dev/shm -S3072K sort.test&quot;<br>
 =A0 =A0 =A0 =A0User time (seconds): 25.88<br>
 =A0 =A0 =A0 =A0System time (seconds): 5.81<br>
 =A0 =A0 =A0 =A0Percent of CPU this job got: 94%<br>
 =A0 =A0 =A0 =A0Elapsed (wall clock) time (h:mm:ss or m:ss): 0:33.59<br>
 =A0 =A0 =A0 =A0Average shared text size (kbytes): 0<br>
 =A0 =A0 =A0 =A0Average unshared data size (kbytes): 0<br>
 =A0 =A0 =A0 =A0Average stack size (kbytes): 0<br>
 =A0 =A0 =A0 =A0Average total size (kbytes): 0<br>
 =A0 =A0 =A0 =A0Maximum resident set size (kbytes): 4968<br>
 =A0 =A0 =A0 =A0Average resident set size (kbytes): 0<br>
 =A0 =A0 =A0 =A0Major (requiring I/O) page faults: 0<br>
 =A0 =A0 =A0 =A0Minor (reclaiming a frame) page faults: 13900<br>
 =A0 =A0 =A0 =A0Voluntary context switches: 429178<br>
 =A0 =A0 =A0 =A0Involuntary context switches: 4882<br>
 =A0 =A0 =A0 =A0Swaps: 0<br>
 =A0 =A0 =A0 =A0File system inputs: 0<br>
 =A0 =A0 =A0 =A0File system outputs: 0<br>
 =A0 =A0 =A0 =A0Socket messages sent: 0<br>
 =A0 =A0 =A0 =A0Socket messages received: 0<br>
 =A0 =A0 =A0 =A0Signals delivered: 0<br>
 =A0 =A0 =A0 =A0Page size (bytes): 4096<br>
 =A0 =A0 =A0 =A0Exit status: 0<br>
<br>
<br>
$ LANG=3DC /usr/bin/time -v strace -c sort -S200M sort.test &gt; /dev/null<=
br>
% time =A0 =A0 seconds =A0usecs/call =A0 =A0 calls =A0 =A0errors syscall<br=
>
------ ----------- ----------- --------- --------- ----------------<br>
=A080.68 =A0 =A00.348335 =A0 =A0 =A0 =A0 565 =A0 =A0 =A0 617 =A0 =A0 =A0 =
=A0 =A0 read<br>
=A011.58 =A0 =A00.049994 =A0 =A0 =A0 =A03846 =A0 =A0 =A0 =A013 =A0 =A0 =A0 =
=A0 =A0 munmap<br>
 =A05.33 =A0 =A00.022995 =A0 =A0 =A0 =A07665 =A0 =A0 =A0 =A0 3 =A0 =A0 =A0 =
=A0 =A0 unlink<br>
 =A02.38 =A0 =A00.010281 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 38521 =A0 =A0 =A0 =
=A0 =A0 write<br>
 =A00.01 =A0 =A00.000052 =A0 =A0 =A0 =A0 =A052 =A0 =A0 =A0 =A0 1 =A0 =A0 =
=A0 =A0 1 access<br>
 =A00.01 =A0 =A00.000042 =A0 =A0 =A0 =A0 =A0 8 =A0 =A0 =A0 =A0 5 =A0 =A0 =
=A0 =A0 =A0 mprotect<br>
 =A00.01 =A0 =A00.000032 =A0 =A0 =A0 =A0 =A0 1 =A0 =A0 =A0 =A025 =A0 =A0 =
=A0 =A0 =A0 rt_sigaction<br>
 =A00.00 =A0 =A00.000000 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 =A010 =A0 =A0 =
=A0 =A0 =A0 open<br>
 =A00.00 =A0 =A00.000000 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 =A012 =A0 =A0 =
=A0 =A0 =A0 close<br>
 =A00.00 =A0 =A00.000000 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 =A012 =A0 =A0 =
=A0 =A0 =A0 fstat<br>
 =A00.00 =A0 =A00.000000 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 =A0 3 =A0 =A0 =
=A0 =A0 =A0 lseek<br>
 =A00.00 =A0 =A00.000000 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 =A022 =A0 =A0 =
=A0 =A0 =A0 mmap<br>
 =A00.00 =A0 =A00.000000 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 =A0 4 =A0 =A0 =
=A0 =A0 =A0 brk<br>
 =A00.00 =A0 =A00.000000 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 =A013 =A0 =A0 =
=A0 =A0 =A0 rt_sigprocmask<br>
 =A00.00 =A0 =A00.000000 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 =A0 1 =A0 =A0 =
=A0 =A0 1 ioctl<br>
 =A00.00 =A0 =A00.000000 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 =A0 1 =A0 =A0 =
=A0 =A0 =A0 execve<br>
 =A00.00 =A0 =A00.000000 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 =A0 3 =A0 =A0 =
=A0 =A0 =A0 fcntl<br>
 =A00.00 =A0 =A00.000000 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 =A0 1 =A0 =A0 =
=A0 =A0 =A0 getrlimit<br>
 =A00.00 =A0 =A00.000000 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 =A0 1 =A0 =A0 =
=A0 =A0 =A0 arch_prctl<br>
 =A00.00 =A0 =A00.000000 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 =A0 1 =A0 =A0 =
=A0 =A0 =A0 set_tid_address<br>
 =A00.00 =A0 =A00.000000 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 =A0 4 =A0 =A0 =
=A0 =A0 =A0 fadvise64<br>
 =A00.00 =A0 =A00.000000 =A0 =A0 =A0 =A0 =A0 0 =A0 =A0 =A0 =A0 1 =A0 =A0 =
=A0 =A0 =A0 set_robust_list<br>
------ ----------- ----------- --------- --------- ----------------<br>
100.00 =A0 =A00.431731 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 39274 =A0 =A0 =A0 =
=A0 2 total<br>
 =A0 =A0 =A0 =A0Command being timed: &quot;strace -c sort -S200M sort.test&=
quot;<br>
 =A0 =A0 =A0 =A0User time (seconds): 19.82<br>
 =A0 =A0 =A0 =A0System time (seconds): 4.33<br>
 =A0 =A0 =A0 =A0Percent of CPU this job got: 94%<br>
 =A0 =A0 =A0 =A0Elapsed (wall clock) time (h:mm:ss or m:ss): 0:25.57<br>
 =A0 =A0 =A0 =A0Average shared text size (kbytes): 0<br>
 =A0 =A0 =A0 =A0Average unshared data size (kbytes): 0<br>
 =A0 =A0 =A0 =A0Average stack size (kbytes): 0<br>
 =A0 =A0 =A0 =A0Average total size (kbytes): 0<br>
 =A0 =A0 =A0 =A0Maximum resident set size (kbytes): 205532<br>
 =A0 =A0 =A0 =A0Average resident set size (kbytes): 0<br>
 =A0 =A0 =A0 =A0Major (requiring I/O) page faults: 355<br>
 =A0 =A0 =A0 =A0Minor (reclaiming a frame) page faults: 98057<br>
 =A0 =A0 =A0 =A0Voluntary context switches: 159217<br>
 =A0 =A0 =A0 =A0Involuntary context switches: 4681<br>
 =A0 =A0 =A0 =A0Swaps: 0<br>
 =A0 =A0 =A0 =A0File system inputs: 140344<br>
 =A0 =A0 =A0 =A0File system outputs: 154168<br>
 =A0 =A0 =A0 =A0Socket messages sent: 0<br>
 =A0 =A0 =A0 =A0Socket messages received: 0<br>
 =A0 =A0 =A0 =A0Signals delivered: 0<br>
 =A0 =A0 =A0 =A0Page size (bytes): 4096<br>
 =A0 =A0 =A0 =A0Exit status: 0<br>
<br>
I notice also that performance gets worse for the above input<br>
for the small buffer size when I enable multithreading on my 4 core i3.<br>
 =A0 =A0 =A0 =A0User time (seconds): 35.40<br>
 =A0 =A0 =A0 =A0System time (seconds): 9.20<br>
 =A0 =A0 =A0 =A0Percent of CPU this job got: 120%<br>
 =A0 =A0 =A0 =A0Elapsed (wall clock) time (h:mm:ss or m:ss): 0:36.98<br>
<br>
cheers,<br>
P=E1draig.<br>
</blockquote></div><br>

--e89a8f234b3349b9c004c2c41de4--


From MAILER-DAEMON Mon Jun 18 15:33:25 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1SghhV-00039A-Sx
	for mharc-coreutils@gnu.org; Mon, 18 Jun 2012 15:33:25 -0400
Received: from eggs.gnu.org ([208.118.235.92]:53553)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1SghhS-00038q-L4
	for coreutils@gnu.org; Mon, 18 Jun 2012 15:33:24 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1SghhP-0002hu-5R
	for coreutils@gnu.org; Mon, 18 Jun 2012 15:33:22 -0400
Received: from mx.meyering.net ([88.168.87.75]:46596)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1SghhO-0002h7-UB
	for coreutils@gnu.org; Mon, 18 Jun 2012 15:33:19 -0400
Received: from rho.meyering.net (rho.meyering.net [127.0.0.1])
	by rho.meyering.net (Acme Bit-Twister) with ESMTP id 9D10060061;
	Mon, 18 Jun 2012 21:33:14 +0200 (CEST)
From: Jim Meyering <jim@meyering.net>
To: Chen Guo <chen.guo.0625@gmail.com>
Subject: Re: Taking advantage of L1 and L2 cache in sort
In-Reply-To: <CANxwL_BFM7VtQX-544MztPhJ+j5N8hdrTdE8=iOp3DXwgw9VcA@mail.gmail.com>
	(Chen Guo's message of "Mon, 18 Jun 2012 12:24:27 -0700")
References: <4B8CE672.5030606@draigBrady.com>
	<657794.87632.qm@web180001.mail.gq1.yahoo.com>
	<4B8DB5FE.6010006@draigBrady.com> <4FD143E0.8000306@draigBrady.com>
	<CANxwL_BFM7VtQX-544MztPhJ+j5N8hdrTdE8=iOp3DXwgw9VcA@mail.gmail.com>
Date: Mon, 18 Jun 2012 21:33:14 +0200
Message-ID: <87395s5sat.fsf@rho.meyering.net>
Lines: 16
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 88.168.87.75
Cc: Shaun Jackman <sjackman@bcgsc.ca>, coreutils@gnu.org,
	Joey Degges <jdegges@gmail.com>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 18 Jun 2012 19:33:24 -0000

Chen Guo wrote:
> Fascinating, Padraig. I wonder, did this email just get to my inbox really,
> really late, or what prompted you to dig up a 2 year old thread?
>
> Anyhow, it seems you had to specifically unset MALLOC_PERTURB_,
> so is it by default set?

MALLOC_PERTURB_ should be set for most people.
The idea is to detect problems earlier rather than later,
and essentially for free.  (except when profiling/timing
and with certain work loads or tight mem/CPU constraints)

I've written about this a few times.
Here's the most recent appeal:

    http://thread.gmane.org/gmane.linux.redhat.fedora.devel/165080


From MAILER-DAEMON Wed Jun 20 03:16:57 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1ShF9t-00007b-16
	for mharc-coreutils@gnu.org; Wed, 20 Jun 2012 03:16:57 -0400
Received: from eggs.gnu.org ([208.118.235.92]:44451)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1ShF9l-00006u-G9
	for coreutils@gnu.org; Wed, 20 Jun 2012 03:16:55 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1ShF9e-00084Q-Bc
	for coreutils@gnu.org; Wed, 20 Jun 2012 03:16:49 -0400
Received: from mx.meyering.net ([88.168.87.75]:52165)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1ShF9e-00084E-3z
	for coreutils@gnu.org; Wed, 20 Jun 2012 03:16:42 -0400
Received: from rho.meyering.net (rho.meyering.net [127.0.0.1])
	by rho.meyering.net (Acme Bit-Twister) with ESMTP id 712D160249;
	Wed, 20 Jun 2012 09:16:40 +0200 (CEST)
From: Jim Meyering <jim@meyering.net>
To: Bernhard Voelker <mail@bernhard-voelker.de>
Subject: Re: [PATCH] maint: cleanup size determination in sort [was: overly
	aggressive memory usage by sort.c]
In-Reply-To: <4FD97E52.6070202@bernhard-voelker.de> (Bernhard Voelker's
	message of "Thu, 14 Jun 2012 08:01:54 +0200")
References: <CAMkU=1zDA=RFzxb-O_x2TG+EumrdUebZJb4UOJAM_xretGpCCA@mail.gmail.com>
	<4FD1431C.5010500@draigBrady.com> <878vfu9x7o.fsf@rho.meyering.net>
	<4FD5C249.7090907@cs.ucla.edu> <4FD5CF65.7000609@cs.ucla.edu>
	<A75C39900FCF0B42A2CE8B26F837D2C3010923@MCHP02MSX.global-ad.net>
	<4FD629CC.4030908@cs.ucla.edu> <4FD97E52.6070202@bernhard-voelker.de>
Date: Wed, 20 Jun 2012 09:16:40 +0200
Message-ID: <87r4tawizr.fsf@rho.meyering.net>
Lines: 59
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 88.168.87.75
Cc: Paul Eggert <eggert@cs.ucla.edu>, "coreutils@gnu.org" <coreutils@gnu.org>,
	"Voelker, Bernhard" <bernhard.voelker@siemens-enterprise.com>,
	Jeff Janes <jeff.janes@gmail.com>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Wed, 20 Jun 2012 07:16:55 -0000

Bernhard Voelker wrote:
> Here's a patch to move the code regarding physical/available/total
> memory down to where it's used (on top of your patch).

Thanks.  I've pushed that -- tweaked to move the rlimit declaration down, too:

>From daab10d9f58d62af896d6b33d06e0d4998f2f114 Mon Sep 17 00:00:00 2001
From: Bernhard Voelker <mail@bernhard-voelker.de>
Date: Wed, 20 Jun 2012 07:57:31 +0200
Subject: [PATCH] maint: sort: style adjustment to help clarify size determination

* src/sort.c (default_sort_size): Move physmem code "down" to first use.
---
 src/sort.c | 15 +++++++--------
 1 file changed, 7 insertions(+), 8 deletions(-)

diff --git a/src/sort.c b/src/sort.c
index 2593a2a..5a48ce6 100644
--- a/src/sort.c
+++ b/src/sort.c
@@ -1400,22 +1400,15 @@ specify_nthreads (int oi, char c, char const *s)
   return nthreads;
 }

-
 /* Return the default sort size.  */
 static size_t
 default_sort_size (void)
 {
-  /* Let MEM be available memory or 1/8 of total memory, whichever
-     is greater.  */
-  double avail = physmem_available ();
-  double total = physmem_total ();
-  double mem = MAX (avail, total / 8);
-  struct rlimit rlimit;
-
   /* Let SIZE be MEM, but no more than the maximum object size or
      system resource limits.  Don't bother to check for values like
      RLIM_INFINITY since in practice they are not much less than SIZE_MAX.  */
   size_t size = SIZE_MAX;
+  struct rlimit rlimit;
   if (getrlimit (RLIMIT_DATA, &rlimit) == 0 && rlimit.rlim_cur < size)
     size = rlimit.rlim_cur;
 #ifdef RLIMIT_AS
@@ -1434,6 +1427,12 @@ default_sort_size (void)
     size = rlimit.rlim_cur / 16 * 15;
 #endif

+  /* Let MEM be available memory or 1/8 of total memory, whichever
+     is greater.  */
+  double avail = physmem_available ();
+  double total = physmem_total ();
+  double mem = MAX (avail, total / 8);
+
   /* Return the minimum of MEM and SIZE, but no less than
      MIN_SORT_SIZE.  Avoid the MIN macro here, as it is not quite
      right when only one argument is floating point.  */
--
1.7.11


From MAILER-DAEMON Wed Jun 20 08:38:16 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1ShKAq-0006lP-HQ
	for mharc-coreutils@gnu.org; Wed, 20 Jun 2012 08:38:16 -0400
Received: from eggs.gnu.org ([208.118.235.92]:37492)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeff.janes@gmail.com>) id 1ShKAj-0006kP-Sz
	for coreutils@gnu.org; Wed, 20 Jun 2012 08:38:15 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jeff.janes@gmail.com>) id 1ShKAe-0004WC-UP
	for coreutils@gnu.org; Wed, 20 Jun 2012 08:38:09 -0400
Received: from mail-yw0-f41.google.com ([209.85.213.41]:35611)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeff.janes@gmail.com>) id 1ShKAe-0004V7-Gp
	for coreutils@gnu.org; Wed, 20 Jun 2012 08:38:04 -0400
Received: by yhr47 with SMTP id 47so6542181yhr.0
	for <coreutils@gnu.org>; Wed, 20 Jun 2012 05:38:01 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
	h=mime-version:in-reply-to:references:date:message-id:subject:from:to
	:cc:content-type:content-transfer-encoding;
	bh=5bmCww13oQx/EHlmVqpJXLFdrP0X1CoNUvIDtcWezHE=;
	b=adr/hU9RrsycSffEeZRrh1GVQ8v5LraEZlGrUevFrYvP8Pe4wuxC4hy/TMw0QO/OB6
	WkeB9aWF9NNY7+NlwkCW80CDnp528d3IUP2nTixjfrMXeHg1VQeEn4vc6QhlVC0tcNN6
	tMKqfeuhDXS9viwE8INluqfmNGzhelEjMJglqaeXfQcN3iUypn0179pAh9uhXqcUIQ5u
	JkTG4crtcwp+oAJqAsMiSApsJtNDHiok5mthfnmvGvW0y6MYpx8OLPWv409rXtLlWoLy
	6fKMRvCQOYDeCNHKLp4Xpg97Q/OxOy8nQPf0qkJqHflI8NmaXo2rOj2oOyGWgJF3AZa9
	13iw==
MIME-Version: 1.0
Received: by 10.50.190.234 with SMTP id gt10mr3803558igc.20.1340195881418;
	Wed, 20 Jun 2012 05:38:01 -0700 (PDT)
Received: by 10.42.80.65 with HTTP; Wed, 20 Jun 2012 05:38:01 -0700 (PDT)
In-Reply-To: <87r4tawizr.fsf@rho.meyering.net>
References: <CAMkU=1zDA=RFzxb-O_x2TG+EumrdUebZJb4UOJAM_xretGpCCA@mail.gmail.com>
	<4FD1431C.5010500@draigBrady.com> <878vfu9x7o.fsf@rho.meyering.net>
	<4FD5C249.7090907@cs.ucla.edu> <4FD5CF65.7000609@cs.ucla.edu>
	<A75C39900FCF0B42A2CE8B26F837D2C3010923@MCHP02MSX.global-ad.net>
	<4FD629CC.4030908@cs.ucla.edu>
	<4FD97E52.6070202@bernhard-voelker.de>
	<87r4tawizr.fsf@rho.meyering.net>
Date: Wed, 20 Jun 2012 05:38:01 -0700
Message-ID: <CAMkU=1xRUkL=ZEFhtJ5vU4h4dWXm=XdHR7Kc8gTux1vozUYRUg@mail.gmail.com>
Subject: Re: [PATCH] maint: cleanup size determination in sort [was: overly
	aggressive memory usage by sort.c]
From: Jeff Janes <jeff.janes@gmail.com>
To: Jim Meyering <jim@meyering.net>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.85.213.41
Cc: Bernhard Voelker <mail@bernhard-voelker.de>,
	"coreutils@gnu.org" <coreutils@gnu.org>, "Voelker,
	Bernhard" <bernhard.voelker@siemens-enterprise.com>,
	Paul Eggert <eggert@cs.ucla.edu>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Wed, 20 Jun 2012 12:38:15 -0000

On Wed, Jun 20, 2012 at 12:16 AM, Jim Meyering <jim@meyering.net> wrote:
> Bernhard Voelker wrote:
>> Here's a patch to move the code regarding physical/available/total
>> memory down to where it's used (on top of your patch).
>
> Thanks. =A0I've pushed that -- tweaked to move the rlimit declaration dow=
n, too:

The committed version does not include the limit to no more than 75%
of physical memory.  Was that intentional or an oversight?

Cheers,

Jeff


From MAILER-DAEMON Wed Jun 20 08:45:20 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1ShKHg-00023K-OD
	for mharc-coreutils@gnu.org; Wed, 20 Jun 2012 08:45:20 -0400
Received: from eggs.gnu.org ([208.118.235.92]:34974)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1ShKHa-000232-FH
	for coreutils@gnu.org; Wed, 20 Jun 2012 08:45:20 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1ShKHU-0007jC-2a
	for coreutils@gnu.org; Wed, 20 Jun 2012 08:45:14 -0400
Received: from mx.meyering.net ([88.168.87.75]:53089)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1ShKHT-0007fV-Qv
	for coreutils@gnu.org; Wed, 20 Jun 2012 08:45:07 -0400
Received: from rho.meyering.net (rho.meyering.net [127.0.0.1])
	by rho.meyering.net (Acme Bit-Twister) with ESMTP id AA447600BC;
	Wed, 20 Jun 2012 14:45:05 +0200 (CEST)
From: Jim Meyering <jim@meyering.net>
To: Jeff Janes <jeff.janes@gmail.com>
Subject: Re: [PATCH] maint: cleanup size determination in sort [was: overly
	aggressive memory usage by sort.c]
In-Reply-To: <CAMkU=1xRUkL=ZEFhtJ5vU4h4dWXm=XdHR7Kc8gTux1vozUYRUg@mail.gmail.com>
	(Jeff Janes's message of "Wed, 20 Jun 2012 05:38:01 -0700")
References: <CAMkU=1zDA=RFzxb-O_x2TG+EumrdUebZJb4UOJAM_xretGpCCA@mail.gmail.com>
	<4FD1431C.5010500@draigBrady.com> <878vfu9x7o.fsf@rho.meyering.net>
	<4FD5C249.7090907@cs.ucla.edu> <4FD5CF65.7000609@cs.ucla.edu>
	<A75C39900FCF0B42A2CE8B26F837D2C3010923@MCHP02MSX.global-ad.net>
	<4FD629CC.4030908@cs.ucla.edu> <4FD97E52.6070202@bernhard-voelker.de>
	<87r4tawizr.fsf@rho.meyering.net>
	<CAMkU=1xRUkL=ZEFhtJ5vU4h4dWXm=XdHR7Kc8gTux1vozUYRUg@mail.gmail.com>
Date: Wed, 20 Jun 2012 14:45:05 +0200
Message-ID: <87wr32up7y.fsf@rho.meyering.net>
Lines: 13
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 88.168.87.75
Cc: Bernhard Voelker <mail@bernhard-voelker.de>,
	"coreutils@gnu.org" <coreutils@gnu.org>, "Voelker,
	Bernhard" <bernhard.voelker@siemens-enterprise.com>,
	Paul Eggert <eggert@cs.ucla.edu>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Wed, 20 Jun 2012 12:45:20 -0000

Jeff Janes wrote:
> On Wed, Jun 20, 2012 at 12:16 AM, Jim Meyering <jim@meyering.net> wrote:
>> Bernhard Voelker wrote:
>>> Here's a patch to move the code regarding physical/available/total
>>> memory down to where it's used (on top of your patch).
>>
>> Thanks. =A0I've pushed that -- tweaked to move the rlimit declaration do=
wn, too:
>
> The committed version does not include the limit to no more than 75%
> of physical memory.  Was that intentional or an oversight?

Bernhard's patch was quite deliberately (as the commit log says)
intended to induce no semantic change.


From MAILER-DAEMON Wed Jun 20 09:52:15 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1ShLKR-0006M5-MZ
	for mharc-coreutils@gnu.org; Wed, 20 Jun 2012 09:52:15 -0400
Received: from eggs.gnu.org ([208.118.235.92]:36984)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eggert@cs.ucla.edu>) id 1ShLKL-0006Jd-FY
	for coreutils@gnu.org; Wed, 20 Jun 2012 09:52:13 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eggert@cs.ucla.edu>) id 1ShLKD-0004rG-H2
	for coreutils@gnu.org; Wed, 20 Jun 2012 09:52:09 -0400
Received: from smtp.cs.ucla.edu ([131.179.128.62]:56108)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eggert@cs.ucla.edu>) id 1ShLKD-0004qT-8V
	for coreutils@gnu.org; Wed, 20 Jun 2012 09:52:01 -0400
Received: from localhost (localhost.localdomain [127.0.0.1])
	by smtp.cs.ucla.edu (Postfix) with ESMTP id C0439A6000B;
	Wed, 20 Jun 2012 06:51:57 -0700 (PDT)
X-Virus-Scanned: amavisd-new at smtp.cs.ucla.edu
Received: from smtp.cs.ucla.edu ([127.0.0.1])
	by localhost (smtp.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10024)
	with ESMTP id EyyQA+gsg2YL; Wed, 20 Jun 2012 06:51:57 -0700 (PDT)
Received: from [192.168.1.10] (pool-108-23-119-2.lsanca.fios.verizon.net
	[108.23.119.2])
	by smtp.cs.ucla.edu (Postfix) with ESMTPSA id B3E3339E8006;
	Wed, 20 Jun 2012 06:51:56 -0700 (PDT)
Message-ID: <4FE1D578.5020109@cs.ucla.edu>
Date: Wed, 20 Jun 2012 06:51:52 -0700
From: Paul Eggert <eggert@cs.ucla.edu>
Organization: UCLA Computer Science Department
User-Agent: Mozilla/5.0 (X11; Linux i686;
	rv:12.0) Gecko/20120430 Thunderbird/12.0.1
MIME-Version: 1.0
To: Jim Meyering <jim@meyering.net>
Subject: Re: [PATCH] maint: cleanup size determination in sort [was: overly
	aggressive memory usage by sort.c]
References: <CAMkU=1zDA=RFzxb-O_x2TG+EumrdUebZJb4UOJAM_xretGpCCA@mail.gmail.com>
	<4FD1431C.5010500@draigBrady.com> <878vfu9x7o.fsf@rho.meyering.net>
	<4FD5C249.7090907@cs.ucla.edu> <4FD5CF65.7000609@cs.ucla.edu>
	<A75C39900FCF0B42A2CE8B26F837D2C3010923@MCHP02MSX.global-ad.net>
	<4FD629CC.4030908@cs.ucla.edu>
	<4FD97E52.6070202@bernhard-voelker.de>
	<87r4tawizr.fsf@rho.meyering.net>
	<CAMkU=1xRUkL=ZEFhtJ5vU4h4dWXm=XdHR7Kc8gTux1vozUYRUg@mail.gmail.com>
	<87wr32up7y.fsf@rho.meyering.net>
In-Reply-To: <87wr32up7y.fsf@rho.meyering.net>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
X-Received-From: 131.179.128.62
Cc: Bernhard Voelker <mail@bernhard-voelker.de>,
	Jeff Janes <jeff.janes@gmail.com>, "coreutils@gnu.org" <coreutils@gnu.org>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Wed, 20 Jun 2012 13:52:14 -0000

On 06/20/2012 05:45 AM, Jim Meyering wrote:
> Bernhard's patch was quite deliberately (as the commit log says)
> intended to induce no semantic change.

Following up on this, here's an updated version of a patch that
should fix the problem that Jeff Janes reported in
<http://lists.gnu.org/archive/html/coreutils/2012-06/msg00018.html>.,
while not reintroducing bug 10877.  Jeff, can you please try this out in
your environment?  It'd be nicer if it could be verified to work
there.  Thanks.

>From ad47c76273b84d64d1c612b2cc49e9679b187004 Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert@cs.ucla.edu>
Date: Mon, 11 Jun 2012 02:47:05 -0700
Subject: [PATCH] sort: by default, do not exceed 3/4 of physical memory

* src/sort.c (default_sort_size): Do not exceed 3/4 of total memory.
See Jeff Janes's bug report in
<http://lists.gnu.org/archive/html/coreutils/2012-06/msg00018.html>.
---
 src/sort.c |   11 ++++++++---
 1 files changed, 8 insertions(+), 3 deletions(-)

diff --git a/src/sort.c b/src/sort.c
index 5a48ce6..4036059 100644
--- a/src/sort.c
+++ b/src/sort.c
@@ -1404,9 +1404,10 @@ specify_nthreads (int oi, char c, char const *s)
 static size_t
 default_sort_size (void)
 {
-  /* Let SIZE be MEM, but no more than the maximum object size or
-     system resource limits.  Don't bother to check for values like
-     RLIM_INFINITY since in practice they are not much less than SIZE_MAX.  */
+  /* Let SIZE be MEM, but no more than the maximum object size,
+     total memory, or system resource limits.  Don't bother to check
+     for values like RLIM_INFINITY since in practice they are not much
+     less than SIZE_MAX.  */
   size_t size = SIZE_MAX;
   struct rlimit rlimit;
   if (getrlimit (RLIMIT_DATA, &rlimit) == 0 && rlimit.rlim_cur < size)
@@ -1433,6 +1434,10 @@ default_sort_size (void)
   double total = physmem_total ();
   double mem = MAX (avail, total / 8);
 
+  /* Leave a 1/4 margin for physical memory.  */
+  if (total * 0.75 < size)
+    size = total * 0.75;
+
   /* Return the minimum of MEM and SIZE, but no less than
      MIN_SORT_SIZE.  Avoid the MIN macro here, as it is not quite
      right when only one argument is floating point.  */
-- 
1.7.6.5



From MAILER-DAEMON Thu Jun 21 07:23:43 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1ShfUF-0000fh-Ib
	for mharc-coreutils@gnu.org; Thu, 21 Jun 2012 07:23:43 -0400
Received: from eggs.gnu.org ([208.118.235.92]:44088)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1ShfUD-0000fB-7k
	for coreutils@gnu.org; Thu, 21 Jun 2012 07:23:42 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1ShfUB-0001Pw-H5
	for coreutils@gnu.org; Thu, 21 Jun 2012 07:23:40 -0400
Received: from moutng.kundenserver.de ([212.227.17.10]:56726)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1ShfUB-0001Pn-6w
	for coreutils@gnu.org; Thu, 21 Jun 2012 07:23:39 -0400
Received: from [192.168.2.108] (p4FF74356.dip.t-dialin.net [79.247.67.86])
	by mrelayeu.kundenserver.de (node=mreu2) with ESMTP (Nemesis)
	id 0LfWo9-1S6C7l2uC1-00owkL; Thu, 21 Jun 2012 13:23:36 +0200
Message-ID: <4FE30437.9050609@bernhard-voelker.de>
Date: Thu, 21 Jun 2012 13:23:35 +0200
From: Bernhard Voelker <mail@bernhard-voelker.de>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:13.0) Gecko/20120601 Thunderbird/13.0
MIME-Version: 1.0
To: Coreutils <coreutils@gnu.org>
Subject: [PATCH] dd: document multiplier for number options like count=N
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
X-Provags-ID: V02:K0:BAWXOF3l068YbgtoGy9/XD7lhDpoM6c3XyeGg+2YVX5
	dvFFERVe1Pl45I5lXcNGi6nMy0vC9E2CXz4IjdLmUfbcg4iGp+
	2zVsQYNmihycekz0CL6ZAItZ/V3t/OzbQNGf42DDdmfhpOfZur
	CsPoOMBhcWGODpyfMqQtJXS6R/JpF8bu09DqUgJokVkz335B1X
	MlmjZ0L125pMV+poaCivn8jn4NtGfR/CGIT2kusLK73/w7X0EZ
	pLxFuCKMVOcCCYtD19tu14BENm2wipxhMkodkU3EV02093vRKO
	JI2t71aZf8UBRJJEYoSSE1oMYx5fir78nvs0SwuYocIYcbFsls
	GYsvL1pxDcn7yr+zLYg7ygtkMjJbZmSG80L3/844L
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 212.227.17.10
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 21 Jun 2012 11:23:42 -0000

I was surprised that the "N" options of dd also accept a multiplier
suffix, e.g. "count=10M". The following patch documents it.

Have a nice day,
Berny

>From 6e683c1273c2f6da2cdb74e3d6e8ee3cbd2236ed Mon Sep 17 00:00:00 2001
From: Bernhard Voelker <mail@bernhard-voelker.de>
Date: Thu, 21 Jun 2012 13:03:41 +0200
Subject: [PATCH] dd: document multiplier for number options like count=N

* src/dd.c (usage): Add "N" to the description of multipliers.
* doc/coreutils.texi (dd invocation): Likewise.
---
 doc/coreutils.texi |    4 ++--
 src/dd.c           |    2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/doc/coreutils.texi b/doc/coreutils.texi
index f7251b2..08ef2d8 100644
--- a/doc/coreutils.texi
+++ b/doc/coreutils.texi
@@ -8399,8 +8399,8 @@ affected file descriptors, even after @command{dd} exits.
 @end table

 @cindex multipliers after numbers
-The numeric-valued strings above (@var{bytes} and @var{blocks}) can be
-followed by a multiplier: @samp{b}=512, @samp{c}=1,
+The numeric-valued strings above (@var{n}, @var{bytes} and @var{blocks})
+can be followed by a multiplier: @samp{b}=512, @samp{c}=1,
 @samp{w}=2, @samp{x@var{m}}=@var{m}, or any of the
 standard block size suffixes like @samp{k}=1024 (@pxref{Block size}).

diff --git a/src/dd.c b/src/dd.c
index 163d514..f86deb5 100644
--- a/src/dd.c
+++ b/src/dd.c
@@ -540,7 +540,7 @@ Copy a file, converting and formatting according to the operands.\n\
 "), stdout);
       fputs (_("\
 \n\
-BLOCKS and BYTES may be followed by the following multiplicative suffixes:\n\
+N, BLOCKS and BYTES may be followed by the following multiplicative suffixes:\n\
 c =1, w =2, b =512, kB =1000, K =1024, MB =1000*1000, M =1024*1024, xM =M\n\
 GB =1000*1000*1000, G =1024*1024*1024, and so on for T, P, E, Z, Y.\n\
 \n\
-- 
1.7.7




From MAILER-DAEMON Fri Jun 22 06:30:13 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1Si181-0006b0-8S
	for mharc-coreutils@gnu.org; Fri, 22 Jun 2012 06:30:13 -0400
Received: from eggs.gnu.org ([208.118.235.92]:41132)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Si17w-0006Xj-Dj
	for coreutils@gnu.org; Fri, 22 Jun 2012 06:30:12 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Si17r-0001vX-Mo
	for coreutils@gnu.org; Fri, 22 Jun 2012 06:30:07 -0400
Received: from mx1.redhat.com ([209.132.183.28]:51567)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Si17r-0001qL-Fe
	for coreutils@gnu.org; Fri, 22 Jun 2012 06:30:03 -0400
Received: from int-mx02.intmail.prod.int.phx2.redhat.com
	(int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id q5MAU02q002031
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK)
	for <coreutils@gnu.org>; Fri, 22 Jun 2012 06:30:01 -0400
Received: from pb-n5110.redhat.com (ovpn-116-30.ams2.redhat.com [10.36.116.30])
	by int-mx02.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP
	id q5MATxh6022265
	for <coreutils@gnu.org>; Fri, 22 Jun 2012 06:30:00 -0400
From: =?UTF-8?q?P=C3=A1draig=20Brady?= <P@draigBrady.com>
To: coreutils@gnu.org
Subject: 
Date: Fri, 22 Jun 2012 11:29:53 +0100
Message-Id: <1340360995-6769-1-git-send-email-P@draigBrady.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
X-Scanned-By: MIMEDefang 2.67 on 10.5.11.12
Content-Transfer-Encoding: quoted-printable
X-MIME-Autoconverted: from 8bit to quoted-printable by mx1.redhat.com id
	q5MAU02q002031
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.132.183.28
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 22 Jun 2012 10:30:12 -0000

FYI I've just pushed these cleanups.

cheers,
P=C3=A1draig.



From MAILER-DAEMON Fri Jun 22 06:30:24 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1Si18C-0006gp-CX
	for mharc-coreutils@gnu.org; Fri, 22 Jun 2012 06:30:24 -0400
Received: from eggs.gnu.org ([208.118.235.92]:41165)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Si18A-0006g5-EQ
	for coreutils@gnu.org; Fri, 22 Jun 2012 06:30:23 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Si183-000256-Kl
	for coreutils@gnu.org; Fri, 22 Jun 2012 06:30:22 -0400
Received: from mx1.redhat.com ([209.132.183.28]:44140)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Si183-00024b-Cr
	for coreutils@gnu.org; Fri, 22 Jun 2012 06:30:15 -0400
Received: from int-mx02.intmail.prod.int.phx2.redhat.com
	(int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id q5MAUB2J028819
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK);
	Fri, 22 Jun 2012 06:30:11 -0400
Received: from pb-n5110.redhat.com (ovpn-116-30.ams2.redhat.com [10.36.116.30])
	by int-mx02.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP
	id q5MATxh7022265; Fri, 22 Jun 2012 06:30:10 -0400
From: =?UTF-8?q?P=C3=A1draig=20Brady?= <P@draigBrady.com>
To: coreutils@gnu.org
Subject: [PATCH 1/2] maint: suppress echoing of sc_THANKS_in_duplicates
	implementation
Date: Fri, 22 Jun 2012 11:29:54 +0100
Message-Id: <1340360995-6769-2-git-send-email-P@draigBrady.com>
In-Reply-To: <1340360995-6769-1-git-send-email-P@draigBrady.com>
References: <1340360995-6769-1-git-send-email-P@draigBrady.com>
X-Scanned-By: MIMEDefang 2.67 on 10.5.11.12
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.132.183.28
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 22 Jun 2012 10:30:23 -0000

* cfg.mk (sc_THANKS_in_duplicates): Suppress the echoing of these
commands run during `make syntax-check`
---
 cfg.mk |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/cfg.mk b/cfg.mk
index d14db04..de05c4b 100644
--- a/cfg.mk
+++ b/cfg.mk
@@ -420,7 +420,7 @@ sc_preprocessor_indentation:
 # someone who was initially listed only in THANKS.in later authors a commit,
 # this rule detects that their pair may now be removed from THANKS.in.
 sc_THANKS_in_duplicates:
-	{ git log --pretty=format:%aN | sort -u;			\
+	@{ git log --pretty=format:%aN | sort -u;			\
 	    cut -b-36 THANKS.in | sed '/^$$/d;s/  *$$//'; }		\
 	  | sort | uniq -d | grep .					\
 	    && { echo '$(ME): remove the above names from THANKS.in'	\
-- 
1.7.6.4



From MAILER-DAEMON Fri Jun 22 06:30:28 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1Si18G-0006kr-Gp
	for mharc-coreutils@gnu.org; Fri, 22 Jun 2012 06:30:28 -0400
Received: from eggs.gnu.org ([208.118.235.92]:41180)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Si18D-0006jl-Ou
	for coreutils@gnu.org; Fri, 22 Jun 2012 06:30:27 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Si187-00025z-BL
	for coreutils@gnu.org; Fri, 22 Jun 2012 06:30:25 -0400
Received: from mx1.redhat.com ([209.132.183.28]:63943)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Si187-00025f-3F
	for coreutils@gnu.org; Fri, 22 Jun 2012 06:30:19 -0400
Received: from int-mx02.intmail.prod.int.phx2.redhat.com
	(int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id q5MAUGgU028829
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK);
	Fri, 22 Jun 2012 06:30:16 -0400
Received: from pb-n5110.redhat.com (ovpn-116-30.ams2.redhat.com [10.36.116.30])
	by int-mx02.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP
	id q5MATxh8022265; Fri, 22 Jun 2012 06:30:15 -0400
From: =?UTF-8?q?P=C3=A1draig=20Brady?= <P@draigBrady.com>
To: coreutils@gnu.org
Subject: [PATCH 2/2] maint: refactor common mode bits used to create files
Date: Fri, 22 Jun 2012 11:29:55 +0100
Message-Id: <1340360995-6769-3-git-send-email-P@draigBrady.com>
In-Reply-To: <1340360995-6769-1-git-send-email-P@draigBrady.com>
References: <1340360995-6769-1-git-send-email-P@draigBrady.com>
X-Scanned-By: MIMEDefang 2.67 on 10.5.11.12
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.132.183.28
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 22 Jun 2012 10:30:27 -0000

* src/system.h (MODE_RW_UGO): The new refactored define (666).
* src/mkfifo.c: Use the new define.
* src/mknod.c: Likewise.
* src/split.c: Likewise.
* src/system.h: Likewise.
* src/touch.c: Likewise.
* src/truncate.c: Likewise.

Suggested-by: Jim Meyering
---
 src/dd.c       |    2 +-
 src/mkfifo.c   |    2 +-
 src/mknod.c    |    2 +-
 src/split.c    |    3 +--
 src/system.h   |    3 +++
 src/touch.c    |    5 +----
 src/truncate.c |    4 +---
 7 files changed, 9 insertions(+), 12 deletions(-)

diff --git a/src/dd.c b/src/dd.c
index 163d514..d937336 100644
--- a/src/dd.c
+++ b/src/dd.c
@@ -2211,7 +2211,7 @@ main (int argc, char **argv)
     }
   else
     {
-      mode_t perms = S_IRUSR | S_IWUSR | S_IRGRP | S_IWGRP | S_IROTH | S_IWOTH;
+      mode_t perms = MODE_RW_UGO;
       int opts
         = (output_flags
            | (conversions_mask & C_NOCREAT ? 0 : O_CREAT)
diff --git a/src/mkfifo.c b/src/mkfifo.c
index e5c871d..e524c44 100644
--- a/src/mkfifo.c
+++ b/src/mkfifo.c
@@ -114,7 +114,7 @@ main (int argc, char **argv)
            _("failed to set default file creation context to %s"),
            quote (scontext));
 
-  newmode = (S_IRUSR | S_IWUSR | S_IRGRP | S_IWGRP | S_IROTH | S_IWOTH);
+  newmode = MODE_RW_UGO;
   if (specified_mode)
     {
       struct mode_change *change = mode_compile (specified_mode);
diff --git a/src/mknod.c b/src/mknod.c
index 3a6d695..dc158b4 100644
--- a/src/mknod.c
+++ b/src/mknod.c
@@ -120,7 +120,7 @@ main (int argc, char **argv)
         }
     }
 
-  newmode = (S_IRUSR | S_IWUSR | S_IRGRP | S_IWGRP | S_IROTH | S_IWOTH);
+  newmode = MODE_RW_UGO;
   if (specified_mode)
     {
       struct mode_change *change = mode_compile (specified_mode);
diff --git a/src/split.c b/src/split.c
index 53ee271..46d2511 100644
--- a/src/split.c
+++ b/src/split.c
@@ -362,8 +362,7 @@ create (const char *name)
     {
       if (verbose)
         fprintf (stdout, _("creating file %s\n"), quote (name));
-      return open (name, O_WRONLY | O_CREAT | O_TRUNC | O_BINARY,
-                   (S_IRUSR | S_IWUSR | S_IRGRP | S_IWGRP | S_IROTH | S_IWOTH));
+      return open (name, O_WRONLY | O_CREAT | O_TRUNC | O_BINARY, MODE_RW_UGO);
     }
   else
     {
diff --git a/src/system.h b/src/system.h
index 06f09cb..5e3b3cb 100644
--- a/src/system.h
+++ b/src/system.h
@@ -30,6 +30,9 @@ you must include <sys/types.h> before including this file
 
 #include <sys/stat.h>
 
+/* Commonly used file permission combination.  */
+#define MODE_RW_UGO (S_IRUSR | S_IWUSR | S_IRGRP | S_IWGRP | S_IROTH | S_IWOTH)
+
 #if !defined HAVE_MKFIFO
 # define mkfifo(name, mode) mknod (name, (mode) | S_IFIFO, 0)
 #endif
diff --git a/src/touch.c b/src/touch.c
index 368516e..5976a34 100644
--- a/src/touch.c
+++ b/src/touch.c
@@ -131,11 +131,8 @@ touch (const char *file)
   else if (! (no_create || no_dereference))
     {
       /* Try to open FILE, creating it if necessary.  */
-      int default_permissions =
-        S_IRUSR | S_IWUSR | S_IRGRP | S_IWGRP | S_IROTH | S_IWOTH;
       fd = fd_reopen (STDIN_FILENO, file,
-                      O_WRONLY | O_CREAT | O_NONBLOCK | O_NOCTTY,
-                      default_permissions);
+                      O_WRONLY | O_CREAT | O_NONBLOCK | O_NOCTTY, MODE_RW_UGO);
 
       /* Don't save a copy of errno if it's EISDIR, since that would lead
          touch to give a bogus diagnostic for e.g., 'touch /' (assuming
diff --git a/src/truncate.c b/src/truncate.c
index e37ab38..c1e9666 100644
--- a/src/truncate.c
+++ b/src/truncate.c
@@ -244,7 +244,6 @@ main (int argc, char **argv)
   off_t size IF_LINT ( = 0);
   off_t rsize = -1;
   rel_mode_t rel_mode = rm_abs;
-  mode_t omode;
   int c, fd = -1, oflags;
   char const *fname;
 
@@ -385,11 +384,10 @@ main (int argc, char **argv)
     }
 
   oflags = O_WRONLY | (no_create ? 0 : O_CREAT) | O_NONBLOCK;
-  omode = S_IRUSR | S_IWUSR | S_IRGRP | S_IWGRP | S_IROTH | S_IWOTH;
 
   while ((fname = *argv++) != NULL)
     {
-      if ((fd = open (fname, oflags, omode)) == -1)
+      if ((fd = open (fname, oflags, MODE_RW_UGO)) == -1)
         {
           /* 'truncate -s0 -c no-such-file'  shouldn't gen error
              'truncate -s0 no-such-dir/file' should gen ENOENT error
-- 
1.7.6.4



From MAILER-DAEMON Fri Jun 22 07:21:10 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1Si1vK-00024B-GG
	for mharc-coreutils@gnu.org; Fri, 22 Jun 2012 07:21:10 -0400
Received: from eggs.gnu.org ([208.118.235.92]:47329)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Si1vI-000244-Kn
	for coreutils@gnu.org; Fri, 22 Jun 2012 07:21:09 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Si1vD-0002IE-DD
	for coreutils@gnu.org; Fri, 22 Jun 2012 07:21:08 -0400
Received: from mx1.redhat.com ([209.132.183.28]:35111)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Si1vD-0002HR-5B
	for coreutils@gnu.org; Fri, 22 Jun 2012 07:21:03 -0400
Received: from int-mx12.intmail.prod.int.phx2.redhat.com
	(int-mx12.intmail.prod.int.phx2.redhat.com [10.5.11.25])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id q5MBKuKL028680
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK);
	Fri, 22 Jun 2012 07:20:57 -0400
Received: from [10.36.116.30] (ovpn-116-30.ams2.redhat.com [10.36.116.30])
	by int-mx12.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP
	id q5MBKsL2027214
	(version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO);
	Fri, 22 Jun 2012 07:20:55 -0400
Message-ID: <4FE45515.4050702@draigBrady.com>
Date: Fri, 22 Jun 2012 12:20:53 +0100
From: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:6.0) Gecko/20110816 Thunderbird/6.0
MIME-Version: 1.0
To: Bernhard Voelker <mail@bernhard-voelker.de>
Subject: Re: [PATCH] dd: document multiplier for number options like count=N
References: <4FE30437.9050609@bernhard-voelker.de>
In-Reply-To: <4FE30437.9050609@bernhard-voelker.de>
X-Enigmail-Version: 1.3.2
Content-Type: text/plain; charset=ISO-8859-1
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.25
Content-Transfer-Encoding: quoted-printable
X-MIME-Autoconverted: from 8bit to quoted-printable by mx1.redhat.com id
	q5MBKuKL028680
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.132.183.28
Cc: Coreutils <coreutils@gnu.org>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 22 Jun 2012 11:21:09 -0000

On 06/21/2012 12:23 PM, Bernhard Voelker wrote:
> I was surprised that the "N" options of dd also accept a multiplier
> suffix, e.g. "count=3D10M". The following patch documents it.

pushed.

cheers,
P=E1draig.


From MAILER-DAEMON Thu Jun 28 18:51:49 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1SkNYz-0002kZ-Jq
	for mharc-coreutils@gnu.org; Thu, 28 Jun 2012 18:51:49 -0400
Received: from eggs.gnu.org ([208.118.235.92]:34671)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <stevenlowson@gmail.com>) id 1SkNWP-0002am-3V
	for coreutils@gnu.org; Thu, 28 Jun 2012 18:49:10 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <stevenlowson@gmail.com>) id 1SkNWM-0005AU-Mz
	for coreutils@gnu.org; Thu, 28 Jun 2012 18:49:08 -0400
Received: from mail-we0-f169.google.com ([74.125.82.169]:43524)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <stevenlowson@gmail.com>) id 1SkNWM-00059a-EW
	for coreutils@gnu.org; Thu, 28 Jun 2012 18:49:06 -0400
Received: by wefh52 with SMTP id h52so2338498wef.0
	for <coreutils@gnu.org>; Thu, 28 Jun 2012 15:49:03 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
	h=message-id:date:from:user-agent:mime-version:to:subject
	:content-type:content-transfer-encoding;
	bh=pHXlN6SPrMgpBeSJa/a5jo3/Aaraieplwvb0j+fODbE=;
	b=XKIAAmOVLHji6Ucs/UZqDIU5XdW62M/Xry/Z0xLlq4AaFIy3hz2BeHddMXOB1XkGYj
	ZwoXo6/h0GoT3eE+lcbZMUraTGZgyiwWc3/R7Ox3tJ8EyEf0azBru7iAjoIQIZYSXTcd
	UKyAo33LnJJfV2+Ui833/ZzoPlmWJXYENxljPXZC5voOndThMn1rN9xCVBiLW1TmVOWS
	SnlNTifklrZ7w5iqVl+CNkOA2tEeo9RvHg0/HvaoAJMhqrBWWEV5ZXLGz0vQVmRrfdfW
	vjtrkByRIj/otjUQaW5j5dxJyNa0rNsirgYAajaLp7VaRWhYfglTVMlg0cqV/yIJwlJf
	FQPw==
Received: by 10.180.98.39 with SMTP id ef7mr149841wib.21.1340923743596;
	Thu, 28 Jun 2012 15:49:03 -0700 (PDT)
Received: from [192.168.0.2]
	(cpc6-dund13-2-0-cust209.sgyl.cable.virginmedia.com.
	[94.174.148.210])
	by mx.google.com with ESMTPS id m4sm2864128wie.1.2012.06.28.15.49.01
	(version=SSLv3 cipher=OTHER); Thu, 28 Jun 2012 15:49:02 -0700 (PDT)
Message-ID: <4FECDF51.7060908@gmail.com>
Date: Thu, 28 Jun 2012 23:48:49 +0100
From: Steven Lowson <stevenlowson@gmail.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:13.0) Gecko/20120616 Thunderbird/13.0.1
MIME-Version: 1.0
To: coreutils@gnu.org
Subject: Change to rmdir flag?
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 74.125.82.169
X-Mailman-Approved-At: Thu, 28 Jun 2012 18:51:48 -0400
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 28 Jun 2012 22:49:10 -0000

Hi Guys,

I would like to propose a change to one of the rmdir flags specifically 
--ignore-fail-on-non-empty to something a bit more succinct. A more 
suitable name would be -force or -f.

Regards

Steve


From MAILER-DAEMON Thu Jun 28 19:16:32 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1SkNwu-0005cs-4M
	for mharc-coreutils@gnu.org; Thu, 28 Jun 2012 19:16:32 -0400
Received: from eggs.gnu.org ([208.118.235.92]:41588)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1SkNws-0005ce-5P
	for coreutils@gnu.org; Thu, 28 Jun 2012 19:16:31 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1SkNwq-0004wx-Bs
	for coreutils@gnu.org; Thu, 28 Jun 2012 19:16:29 -0400
Received: from mx1.redhat.com ([209.132.183.28]:2419)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1SkNwq-0004wd-3s
	for coreutils@gnu.org; Thu, 28 Jun 2012 19:16:28 -0400
Received: from int-mx11.intmail.prod.int.phx2.redhat.com
	(int-mx11.intmail.prod.int.phx2.redhat.com [10.5.11.24])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id q5SNGPt9020532
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK);
	Thu, 28 Jun 2012 19:16:25 -0400
Received: from [10.3.113.134] (ovpn-113-134.phx2.redhat.com [10.3.113.134])
	by int-mx11.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP
	id q5SNGOHx031422; Thu, 28 Jun 2012 19:16:24 -0400
Message-ID: <4FECE5C1.9010905@redhat.com>
Date: Thu, 28 Jun 2012 17:16:17 -0600
From: Eric Blake <eblake@redhat.com>
Organization: Red Hat
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:13.0) Gecko/20120615 Thunderbird/13.0.1
MIME-Version: 1.0
To: Steven Lowson <stevenlowson@gmail.com>
Subject: Re: Change to rmdir flag?
References: <4FECDF51.7060908@gmail.com>
In-Reply-To: <4FECDF51.7060908@gmail.com>
X-Enigmail-Version: 1.4.2
OpenPGP: url=http://people.redhat.com/eblake/eblake.gpg
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature";
	boundary="------------enigFF129FE8FF47CE8DD2A8BC80"
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.24
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.132.183.28
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 28 Jun 2012 23:16:31 -0000

This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--------------enigFF129FE8FF47CE8DD2A8BC80
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

On 06/28/2012 04:48 PM, Steven Lowson wrote:
> Hi Guys,
>=20
> I would like to propose a change to one of the rmdir flags specifically=

> --ignore-fail-on-non-empty to something a bit more succinct. A more
> suitable name would be -force or -f.

Thanks for the suggestion.  However, we're unlikely to burn a short
letter without existing precedence in some other implementation, since
that risks collisions with future standardization.  But do you realize
that long options can be abbreviated if the abbreviation is unambiguous?
 Therefore, 'rmdir --i' is equivalent to 'rmdir
--ignore-fail-on-non-empty', which is almost as good as your proposed
'rmdir -f' and shorter than your proposed 'rmdir --force'.

--=20
Eric Blake   eblake@redhat.com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org




--------------enigFF129FE8FF47CE8DD2A8BC80
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)
Comment: Public key at http://people.redhat.com/eblake/eblake.gpg
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQEcBAEBCAAGBQJP7OXEAAoJEKeha0olJ0NqYx0H/iThLqXO73oQG3AZ22JX1NvO
oggx6VnvuTMEn3RrBH7I17RYAyaiTy8RuUP/Xnda5jIFGP9QcgDGU08ZR6dcLI45
n2ZY0dev4goSCzv8o+ILu0L+lVBvL+FTKtDH9dFvSNOVWA53pGcnby9Ujr7xBt1g
HAqxEv/uQE6BYSo0WPM/poD2nN0b1dOGzZxcXszKehKcOU1XAQixMUvJ/DsjzuLp
guwLYYSmbniB8j4O4/SMd8tMT8eSM64VXmetFOQfDdvw6ZFXjHd6OUDynWwOemee
yJKDzrF3P0gKoXRb5ACSPE3extouXHizDyOTkKE8MXywSoTHUHJePN1akJHRFoQ=
=r725
-----END PGP SIGNATURE-----

--------------enigFF129FE8FF47CE8DD2A8BC80--


From MAILER-DAEMON Sat Jun 30 19:15:08 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1Sl6se-0006pT-PN
	for mharc-coreutils@gnu.org; Sat, 30 Jun 2012 19:15:08 -0400
Received: from eggs.gnu.org ([208.118.235.92]:55540)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <bob@proulx.com>) id 1Sl6sb-0006mS-Sv
	for coreutils@gnu.org; Sat, 30 Jun 2012 19:15:07 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <bob@proulx.com>) id 1Sl6sa-0008CG-28
	for coreutils@gnu.org; Sat, 30 Jun 2012 19:15:05 -0400
Received: from joseki.proulx.com ([216.17.153.58]:52401)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <bob@proulx.com>) id 1Sl6sZ-000859-Qq
	for coreutils@gnu.org; Sat, 30 Jun 2012 19:15:03 -0400
Received: from hysteria.proulx.com (hysteria.proulx.com [192.168.230.119])
	by joseki.proulx.com (Postfix) with ESMTP id EDED2211EE;
	Sat, 30 Jun 2012 17:15:00 -0600 (MDT)
Received: by hysteria.proulx.com (Postfix, from userid 1000)
	id 85EB12DC59; Sat, 30 Jun 2012 17:15:00 -0600 (MDT)
Date: Sat, 30 Jun 2012 17:15:00 -0600
From: Bob Proulx <bob@proulx.com>
To: Stefano Lattarini <stefano.lattarini@gmail.com>,
	Karl Berry <karl@freefriends.org>
Subject: Re: bug#11828: Problems building the coreutils info documentation
	with the development version of makeinfo
Message-ID: <20120630231500.GA22255@hysteria.proulx.com>
Mail-Followup-To: Stefano Lattarini <stefano.lattarini@gmail.com>,
	Karl Berry <karl@freefriends.org>, coreutils@gnu.org
References: <201206302229.q5UMTpBx019247@freefriends.org>
	<4FEF7FEB.3010304@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <4FEF7FEB.3010304@gmail.com>
User-Agent: Mutt/1.5.21 (2010-09-15)
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 2)
X-Received-From: 216.17.153.58
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sat, 30 Jun 2012 23:15:07 -0000

Sending the discussion to the discussion side of things instead of the
bug ticket.

Stefano Lattarini wrote:
> Karl Berry wrote:
> > P.S. I took bug-coreutils out of cc simply because I didn't want to deal
> > with "opening" a bug report (doesn't it require the bug number somewhere
> > to append)?

That was certainly the safe thing to do without having any other
information about it. :-)

> I don't think so; the debbugs software should be smart enough to
> realize that you're answering to a massage that has already opened a
> bug report, and it should thus avoid to open a new bug report.  I
> might be wrong on this, though, and I don't feel inclined to
> experiment right now.

Correct.  With the exception that the debbugs robot needs to have had
time to have received and processed the original message so that it
can recognize the second message as a reply to it.  This usually isn't
a problem with replies that a human receives and responds to.

But for example they can be sync'd up in time if two messages are in
the hold queue waiting for approval in Mailman.  If both are approved
at the same time then it is a race condition as to which message
Mailman will process first.  Murphy says that case always generates a
new bug ticket.

In that case it is good to approve only the first message first, wait
a few minutes or observe the passage of the original email, then
approve the second one afterward.  Then the debbugs robot can
recognize the second as a reply to the first and do the right thing.
That will avoid creating a second bug ticket.

Bob


