From MAILER-DAEMON Sat Jan 01 16:32:30 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PZ93u-0001Zg-0K
	for mharc-coreutils@gnu.org; Sat, 01 Jan 2011 16:32:30 -0500
Received: from [140.186.70.92] (port=38249 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PZ93r-0001Yg-Je
	for coreutils@gnu.org; Sat, 01 Jan 2011 16:32:28 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PZ93q-0001UW-Fi
	for coreutils@gnu.org; Sat, 01 Jan 2011 16:32:27 -0500
Received: from mx.meyering.net ([82.230.74.64]:53041)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PZ93q-0001UR-4l
	for coreutils@gnu.org; Sat, 01 Jan 2011 16:32:26 -0500
Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000)
	id 9874E6018B; Sat,  1 Jan 2011 22:32:23 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: coreutils@gnu.org
Date: Sat, 01 Jan 2011 22:32:23 +0100
Message-ID: <87sjxcb994.fsf@meyering.net>
Lines: 39
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
Subject: [coreutils] THANKS-generating and copyright-updating changes pushed
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sat, 01 Jan 2011 21:32:28 -0000

FYI, I've just pushed these administrative changes:

commit 9d6231ef2a74671fa08bc263519edfe46e8d1805
Author: Jim Meyering <meyering@redhat.com>
Date:   Sat Jan 1 11:37:32 2011 +0100

    maint: update all copyright year number ranges

    Run "make update-copyright".

commit 257909013ef559418f612e8592f55b29dafda154
Author: Jim Meyering <meyering@redhat.com>
Date:   Sat Jan 1 11:10:00 2011 +0100

    build: update gnulib for version-etc copyright year update

    * tests/sample-test: Update copyright to 2011, to appease syntax-check.

commit 9a008a9e24166375cf512457155d38c760c89258
Author: Jim Meyering <meyering@redhat.com>
Date:   Mon Nov 17 12:05:27 2008 +0100

    maint: generate much of the THANKS file

    Before this change, we had a tendency to manually list each
    contributor's name in THANKS.  Now, each commit "Author" is
    included in the generated THANKS file automatically, and most
    of the old THANKS file is now a template, THANKS.in.
    We'll still have to manually list the names of people who report
    problems without a usable patch.

    * THANKS.in: New file, derived from THANKS, but removing names of
    those who are listed as git log 'Author:'s.
    * THANKS: Remove file.
    * thanks-gen: New file.
    * Makefile.am (THANKS): New rule.
    (EXTRA_DIST): Add .mailmap, THANKS.in and thanks-gen.
    * .gitignore: Add THANKS and THANKS-to-translators.
    * .mailmap: Unify on single address and name-spelling per contributor.


From MAILER-DAEMON Mon Jan 03 07:33:09 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PZjb2-0004YK-TZ
	for mharc-coreutils@gnu.org; Mon, 03 Jan 2011 07:33:08 -0500
Received: from [140.186.70.92] (port=60369 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PZjaz-0004W0-9n
	for coreutils@gnu.org; Mon, 03 Jan 2011 07:33:07 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PZjay-0001Tx-AE
	for coreutils@gnu.org; Mon, 03 Jan 2011 07:33:05 -0500
Received: from mx.meyering.net ([82.230.74.64]:54303)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PZjay-0001Tc-3P
	for coreutils@gnu.org; Mon, 03 Jan 2011 07:33:04 -0500
Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000)
	id C445560190; Mon,  3 Jan 2011 13:33:01 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: coreutils@gnu.org
Subject: Re: [coreutils] coreutils-8.9 next week
In-Reply-To: <87r5czdxqg.fsf@meyering.net> (Jim Meyering's message of "Thu, 30
	Dec 2010 17:36:07 +0100")
References: <87r5czdxqg.fsf@meyering.net>
Date: Mon, 03 Jan 2011 13:33:01 +0100
Message-ID: <87mxnii2v6.fsf@meyering.net>
Lines: 11
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 03 Jan 2011 12:33:07 -0000

Jim Meyering wrote:
> Given today's bug fix, I expect to release coreutils-8.9 next
> week, so if you know of any other changes that would be appropriate
> for a bug-fix-only release, please tell us now.

Unless I hear otherwise, I'll release coreutils-8.9 tomorrow
with the current changes:

    http://git.savannah.gnu.org/cgit/coreutils.git/

plus an update to the latest from gnulib.


From MAILER-DAEMON Tue Jan 04 12:34:47 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PaAmV-0002ME-Jw
	for mharc-coreutils@gnu.org; Tue, 04 Jan 2011 12:34:47 -0500
Received: from [140.186.70.92] (port=59889 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PaAmT-0002Kw-EW
	for coreutils@gnu.org; Tue, 04 Jan 2011 12:34:46 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PaAmS-0008RK-5n
	for coreutils@gnu.org; Tue, 04 Jan 2011 12:34:45 -0500
Received: from mx.meyering.net ([82.230.74.64]:43649)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>)
	id 1PaAmN-0008Pn-8l; Tue, 04 Jan 2011 12:34:39 -0500
Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000)
	id E6D85601ED; Tue,  4 Jan 2011 18:34:37 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: info-gnu@gnu.org
Mail-Followup-To: bug-coreutils@gnu.org
Date: Tue, 04 Jan 2011 18:34:31 +0100
Message-ID: <87tyhobmjc.fsf@meyering.net>
Lines: 56
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="=-=-=";
	micalg=pgp-sha256; protocol="application/pgp-signature"
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
Cc: coordinator@translationproject.org, coreutils@gnu.org,
	coreutils-announce@gnu.org
Subject: [coreutils] coreutils-8.9 released [stable]
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Tue, 04 Jan 2011 17:34:46 -0000

--=-=-=
Content-Type: text/plain

This is to announce coreutils-8.9, a stable, bug-fix release.
The sole bug was in split, introduced in 8.8.

See NEWS below for a summary.

Here's the GNU Coreutils home page:
    http://www.gnu.org/software/coreutils/

Jim [on behalf of the coreutils maintainers]

Here are the compressed sources:
  http://ftp.gnu.org/gnu/coreutils/coreutils-8.9.tar.xz    (4.6 MB)
  http://ftp.gnu.org/gnu/coreutils/coreutils-8.9.tar.gz     (11 MB)

Here are the GPG detached signatures[*]:
  http://ftp.gnu.org/gnu/coreutils/coreutils-8.9.tar.xz.sig
  http://ftp.gnu.org/gnu/coreutils/coreutils-8.9.tar.gz.sig

If you prefer a mirror, you can use links like these:
  http://ftpmirror.gnu.org/coreutils/coreutils-8.9.tar.xz
  http://ftpmirror.gnu.org/coreutils/coreutils-8.9.tar.gz

[*] You can use either of the above signature files to verify that
the corresponding file (without the .sig suffix) is intact.  First,
be sure to download both the .sig file and the corresponding tarball.
Then, run a command like this:

  gpg --verify coreutils-8.9.tar.gz.sig

If that command fails because you don't have the required public key,
then run this command to import it:

  gpg --keyserver keys.gnupg.net --recv-keys 000BEEEE

and rerun the `gpg --verify' command.

This release was bootstrapped with the following tools:
  Autoconf 2.68.11-45b92
  Automake 1.11a
  Gnulib v0.0-4673-gfa6be5b
  Bison 2.4.3

=========================================================
NEWS

* Noteworthy changes in release 8.9 (2011-01-04) [stable]

** Bug fixes

  split no longer creates files with a suffix length that
  is dependent on the number of bytes or lines per file.
  [bug introduced in coreutils-8.8]

----
Also announced at http://savannah.gnu.org/forum/forum.php?forum_id=6679

--=-=-=
Content-Type: application/pgp-signature

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.10-svn4867 (GNU/Linux)

iQIcBAEBCAAGBQJNI1otAAoJEH/Z/MsAC+7uY7QP/1WTZTQeymeyc7ATnIqeD7xG
aqqvAbTEHp1ZH3ks2XAL0s62+fVwKX6/jxx+XKMN3feppY9yCKtuOwJOjEj+dUS8
3eUz3qgVURGrNIYcd0/r67PMpn6O48v049ts+K0eqmm9LbTh+McvQy2y9eoSPD9x
iOqk2mWaM6uzSdbfJFHARc2JD2GqdmeR8ASWj8K9vKGJDK87aQfQv6YsWTySOIrq
AMmR8VNuDtpsnKq6E27gxmcdFUBXw6nV5SqXmFru4aZDFE7RdXzrwIPy6iCz97xM
zF3UKYNKzAJIxqd7PrDmnJt9FVRbrhjZc9o+2Dr1+N4DdCRNgJMZ+x3znmPfcEtn
w6xQa3emmiCsfA+BOTn2Ey4pzvUZo8HGRBzQnnxtmjuvDo2f+/lCzNYhm/iLzySw
ls/Tk5NQxilPoV1StWDxdv1jUPF0XJR85V+4QAhMVoBSkF57c5+xq5dvuAk0QALH
amaHgn4Vqe2qUZ1U8YaYkOu2R29udHdJx5Ao59ceFYJNiHz1sMDG7zUy8TC3Xp34
gXIEriyvEybu8Y6t8nKX3ZsoyGquCzyA2kJfGRjvqhxI8uNb8Rj2I34bvpePrvRs
sVyrHiuTFM1s4kRnAr7ZgED3YCwGlTCfYHkZ0fl8V7cjrZwdm3SdFpXy7038QWYB
HGDSNMzxvggeO4p8AibX
=I7ob
-----END PGP SIGNATURE-----
--=-=-=--


From MAILER-DAEMON Tue Jan 04 13:14:57 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PaBPN-0005ux-J0
	for mharc-coreutils@gnu.org; Tue, 04 Jan 2011 13:14:57 -0500
Received: from [140.186.70.92] (port=56626 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PaBPL-0005tP-FU
	for coreutils@gnu.org; Tue, 04 Jan 2011 13:14:56 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PaBPK-0007hX-JC
	for coreutils@gnu.org; Tue, 04 Jan 2011 13:14:55 -0500
Received: from mx.meyering.net ([82.230.74.64]:36087)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>)
	id 1PaBPI-0007gu-LK; Tue, 04 Jan 2011 13:14:52 -0500
Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000)
	id 2007260190; Tue,  4 Jan 2011 19:14:49 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: coreutils@gnu.org, bug-gnulib@gnu.org
Date: Tue, 04 Jan 2011 19:14:49 +0100
Message-ID: <87ipy4bko6.fsf@meyering.net>
Lines: 10
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
Cc: 
Subject: [coreutils] new branch in gnulib: coreutils-8.9
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Tue, 04 Jan 2011 18:14:56 -0000

In releasing coreutils-8.9, I left its gnulib submodule pointing at a
private commit that was going to be pushed, pending an ACK...  Since a
slightly different commit was pushed, coreutils' submodule SHA1 was
invalid for a short interval.  However, I have pushed my local commit
to the new gnulib branch named "coreutils-8.9".  With that, anyone
cloning coreutils and trying to build from the 8.9 tag *will* get a
usable gnulib hierarchy.

Thanks to Eric Blake for noticing so quickly and for the suggestion
to push my commit to a branch in gnulib.


From MAILER-DAEMON Tue Jan 04 14:08:43 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PaCFP-0005a3-C1
	for mharc-coreutils@gnu.org; Tue, 04 Jan 2011 14:08:43 -0500
Received: from [140.186.70.92] (port=44448 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PaCEW-0003gL-9y
	for coreutils@gnu.org; Tue, 04 Jan 2011 14:08:41 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <Ralf.Wildenhues@gmx.de>) id 1PaCES-0005z0-O9
	for coreutils@gnu.org; Tue, 04 Jan 2011 14:07:46 -0500
Received: from mailout-de.gmx.net ([213.165.64.22]:57432 helo=mail.gmx.net)
	by eggs.gnu.org with smtp (Exim 4.71)
	(envelope-from <Ralf.Wildenhues@gmx.de>) id 1PaCES-0005yT-8K
	for coreutils@gnu.org; Tue, 04 Jan 2011 14:07:44 -0500
Received: (qmail invoked by alias); 04 Jan 2011 19:07:42 -0000
Received: from xdsl-78-34-209-126.netcologne.de (EHLO localhost.localdomain)
	[78.34.209.126]
	by mail.gmx.net (mp026) with SMTP; 04 Jan 2011 20:07:42 +0100
X-Authenticated: #13673931
X-Provags-ID: V01U2FsdGVkX19W/q0Kdd5W5t38NOdfQM5bU+ZNsZkRRRbul4c7MK
	ee5Aa95QO53yUB
Received: from ralf by localhost.localdomain with local (Exim 4.69)
	(envelope-from <Ralf.Wildenhues@gmx.de>)
	id 1PaCEO-0001g0-Td; Tue, 04 Jan 2011 20:07:40 +0100
Date: Tue, 4 Jan 2011 20:07:40 +0100
From: Ralf Wildenhues <Ralf.Wildenhues@gmx.de>
To: Jim Meyering <jim@meyering.net>
Message-ID: <20110104190740.GA6420@gmx.de>
Mail-Followup-To: Jim Meyering <jim@meyering.net>, coreutils@gnu.org,
	bug-gnulib@gnu.org
References: <87ipy4bko6.fsf@meyering.net>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <87ipy4bko6.fsf@meyering.net>
Organization: Institute for Numerical Simulation, University of Bonn
User-Agent: Mutt/1.5.20 (2010-08-04)
X-Y-GMX-Trusted: 0
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
Cc: bug-gnulib@gnu.org, coreutils@gnu.org
Subject: [coreutils] Re: new branch in gnulib: coreutils-8.9
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Tue, 04 Jan 2011 19:08:41 -0000

Hi Jim,

* Jim Meyering wrote on Tue, Jan 04, 2011 at 07:14:49PM CET:
> In releasing coreutils-8.9, I left its gnulib submodule pointing at a
> private commit that was going to be pushed, pending an ACK...  Since a
> slightly different commit was pushed, coreutils' submodule SHA1 was
> invalid for a short interval.  However, I have pushed my local commit
> to the new gnulib branch named "coreutils-8.9".  With that, anyone
> cloning coreutils and trying to build from the 8.9 tag *will* get a
> usable gnulib hierarchy.

FYI, this would have also been a good point at which you could have
merged the commit used for coreutils into the gnulib master branch.
Even a fake merge 'git merge -s ours coreutils-8.9' would be
appropriate, if the end result is semantically that the effects of your
patch are incorporated in the tree.

You could still do this.

That way, the next update of gnulib as coreutils submodule could still
be a fast-forward.

Cheers,
Ralf


From MAILER-DAEMON Tue Jan 04 15:06:35 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PaD9P-0004hS-LD
	for mharc-coreutils@gnu.org; Tue, 04 Jan 2011 15:06:35 -0500
Received: from [140.186.70.92] (port=50142 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PaD9L-0003Pp-MV
	for coreutils@gnu.org; Tue, 04 Jan 2011 15:06:34 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PaD92-0003DX-IK
	for coreutils@gnu.org; Tue, 04 Jan 2011 15:06:14 -0500
Received: from mx.meyering.net ([82.230.74.64]:56722)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>)
	id 1PaD8r-0003A4-6y; Tue, 04 Jan 2011 15:06:01 -0500
Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000)
	id 6294B601C8; Tue,  4 Jan 2011 21:05:59 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: coreutils@gnu.org
In-Reply-To: <20110104190740.GA6420@gmx.de> (Ralf Wildenhues's message of
	"Tue, 4 Jan 2011 20:07:40 +0100")
References: <87ipy4bko6.fsf@meyering.net> <20110104190740.GA6420@gmx.de>
Date: Tue, 04 Jan 2011 21:05:59 +0100
Message-ID: <87vd24a0yg.fsf@meyering.net>
Lines: 32
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
Cc: bug-gnulib@gnu.org
Subject: [coreutils] Re: new branch in gnulib: coreutils-8.9
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Tue, 04 Jan 2011 20:06:34 -0000

Ralf Wildenhues wrote:
> Hi Jim,
>
> * Jim Meyering wrote on Tue, Jan 04, 2011 at 07:14:49PM CET:
>> In releasing coreutils-8.9, I left its gnulib submodule pointing at a
>> private commit that was going to be pushed, pending an ACK...  Since a
>> slightly different commit was pushed, coreutils' submodule SHA1 was
>> invalid for a short interval.  However, I have pushed my local commit
>> to the new gnulib branch named "coreutils-8.9".  With that, anyone
>> cloning coreutils and trying to build from the 8.9 tag *will* get a
>> usable gnulib hierarchy.
>
> FYI, this would have also been a good point at which you could have
> merged the commit used for coreutils into the gnulib master branch.
> Even a fake merge 'git merge -s ours coreutils-8.9' would be
> appropriate, if the end result is semantically that the effects of your
> patch are incorporated in the tree.
>
> You could still do this.
>
> That way, the next update of gnulib as coreutils submodule could still
> be a fast-forward.

Hi Ralf,

I've done as you suggest (note that it required temporarily
disabling the hook that prohibits pushing merge commits to master).
That has the added benefit that the branch is no longer needed.

Thanks for the suggestion,

Jim


From MAILER-DAEMON Tue Jan 04 15:09:57 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PaDCf-0006IP-7I
	for mharc-coreutils@gnu.org; Tue, 04 Jan 2011 15:09:57 -0500
Received: from [140.186.70.92] (port=32939 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PaDCY-0006EI-Rn
	for coreutils@gnu.org; Tue, 04 Jan 2011 15:09:55 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1PaDCV-0004Ry-Pi
	for coreutils@gnu.org; Tue, 04 Jan 2011 15:09:48 -0500
Received: from mx1.redhat.com ([209.132.183.28]:40506)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>)
	id 1PaDCV-0004Rp-Gv; Tue, 04 Jan 2011 15:09:47 -0500
Received: from int-mx09.intmail.prod.int.phx2.redhat.com
	(int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22])
	by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id p04K9kOQ007574
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK);
	Tue, 4 Jan 2011 15:09:46 -0500
Received: from [10.3.113.114] (ovpn-113-114.phx2.redhat.com [10.3.113.114])
	by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP
	id p04K9juK019011; Tue, 4 Jan 2011 15:09:45 -0500
Message-ID: <4D237E89.9080906@redhat.com>
Date: Tue, 04 Jan 2011 13:09:45 -0700
From: Eric Blake <eblake@redhat.com>
Organization: Red Hat
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US;
	rv:1.9.2.13) Gecko/20101209 Fedora/3.1.7-0.35.b3pre.fc14
	Lightning/1.0b3pre Mnenhy/0.8.3 Thunderbird/3.1.7
MIME-Version: 1.0
To: Jim Meyering <jim@meyering.net>
References: <87ipy4bko6.fsf@meyering.net> <20110104190740.GA6420@gmx.de>
	<87vd24a0yg.fsf@meyering.net>
In-Reply-To: <87vd24a0yg.fsf@meyering.net>
X-Enigmail-Version: 1.1.2
OpenPGP: url=http://people.redhat.com/eblake/eblake.gpg
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature";
	boundary="------------enig00AF936DECFFB269A8D4D90F"
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
Cc: bug-gnulib@gnu.org, coreutils@gnu.org
Subject: [coreutils] Re: new branch in gnulib: coreutils-8.9
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Tue, 04 Jan 2011 20:09:55 -0000

This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--------------enig00AF936DECFFB269A8D4D90F
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

On 01/04/2011 01:05 PM, Jim Meyering wrote:
>> That way, the next update of gnulib as coreutils submodule could still=

>> be a fast-forward.
>=20
> Hi Ralf,
>=20
> I've done as you suggest (note that it required temporarily
> disabling the hook that prohibits pushing merge commits to master).
> That has the added benefit that the branch is no longer needed.

Except that you --amend'ed the commit before merging, so that the
gnulib.git master branch refers to commit a77937c rather than the
intended fa6be5b that is current on the gnulib.git coreutils-8.9 branch,
so right now, one _still_ needs the coreutils-8.9 branch to exist.

--=20
Eric Blake   eblake@redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org


--------------enig00AF936DECFFB269A8D4D90F
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Public key at http://people.redhat.com/eblake/eblake.gpg
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/

iQEcBAEBCAAGBQJNI36JAAoJEKeha0olJ0NqnZoH/A+7EhVXlgh0oThnfRtS3vY1
wbpILWVHxsFSsg60kcGXxuIuRyZxj0/r1eOXCLGA07wtUWvBpw4pOZbOYN0MUK7o
YqAD8thBET6sgcqSmP3TbQrEzs6VbI84uTSPnAqzF50I85mMPJYEwwUmOALm8Lwu
eoWyGjad8sAuJiB+UKxqmxlCnFi4M2Ncsa09mvhjS8S7ukCGzu8sOMr/EO4jQUx8
2dvh6sltYBGjUZrOWTsgWqDcY+wv2kEtXO/WuDO2NhRg9Hpj1+z7OkoXVaARF60s
JO2COT/puEKsEQxOmzHLkwifBdST9cFZT9R4SIadf1GA3wjXkah/gT8lNBvUgU4=
=PcQW
-----END PGP SIGNATURE-----

--------------enig00AF936DECFFB269A8D4D90F--


From MAILER-DAEMON Tue Jan 04 15:21:20 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PaDNg-0000M7-0Y
	for mharc-coreutils@gnu.org; Tue, 04 Jan 2011 15:21:20 -0500
Received: from [140.186.70.92] (port=58773 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PaDNb-0000Id-FN
	for coreutils@gnu.org; Tue, 04 Jan 2011 15:21:17 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PaDNY-0000L4-Gg
	for coreutils@gnu.org; Tue, 04 Jan 2011 15:21:13 -0500
Received: from mx.meyering.net ([82.230.74.64]:44249)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>)
	id 1PaDNK-0000Gr-U8; Tue, 04 Jan 2011 15:20:59 -0500
Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000)
	id 4EF9C601C8; Tue,  4 Jan 2011 21:20:57 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: Eric Blake <eblake@redhat.com>
In-Reply-To: <4D237E89.9080906@redhat.com> (Eric Blake's message of "Tue, 04
	Jan 2011 13:09:45 -0700")
References: <87ipy4bko6.fsf@meyering.net> <20110104190740.GA6420@gmx.de>
	<87vd24a0yg.fsf@meyering.net> <4D237E89.9080906@redhat.com>
Date: Tue, 04 Jan 2011 21:20:57 +0100
Message-ID: <87pqsca09i.fsf@meyering.net>
Lines: 23
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
Cc: bug-gnulib@gnu.org, coreutils@gnu.org
Subject: [coreutils] Re: new branch in gnulib: coreutils-8.9
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Tue, 04 Jan 2011 20:21:17 -0000

Eric Blake wrote:
> On 01/04/2011 01:05 PM, Jim Meyering wrote:
>>> That way, the next update of gnulib as coreutils submodule could still
>>> be a fast-forward.
>>
>> Hi Ralf,
>>
>> I've done as you suggest (note that it required temporarily
>> disabling the hook that prohibits pushing merge commits to master).
>> That has the added benefit that the branch is no longer needed.
>
> Except that you --amend'ed the commit before merging, so that the
> gnulib.git master branch refers to commit a77937c rather than the
> intended fa6be5b that is current on the gnulib.git coreutils-8.9 branch,
> so right now, one _still_ needs the coreutils-8.9 branch to exist.

sigh.  Thanks, Eric.
My first merge used an old, invalid branch.
This one does the right thing:

  git merge -s ours origin/coreutils-8.9

coming up...


From MAILER-DAEMON Tue Jan 04 16:16:00 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PaEEa-0008Vp-Ix
	for mharc-coreutils@gnu.org; Tue, 04 Jan 2011 16:16:00 -0500
Received: from [140.186.70.92] (port=40044 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PaEEZ-0008VO-4W
	for coreutils@gnu.org; Tue, 04 Jan 2011 16:16:00 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PaEEY-0004H8-Bi
	for coreutils@gnu.org; Tue, 04 Jan 2011 16:15:59 -0500
Received: from mx.meyering.net ([82.230.74.64]:35346)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>)
	id 1PaEEV-0004GU-MW; Tue, 04 Jan 2011 16:15:55 -0500
Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000)
	id 12E76603DD; Tue,  4 Jan 2011 22:15:54 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: coreutils@gnu.org
In-Reply-To: <87ipy4bko6.fsf@meyering.net> (Jim Meyering's message of "Tue, 04
	Jan 2011 19:14:49 +0100")
References: <87ipy4bko6.fsf@meyering.net>
Date: Tue, 04 Jan 2011 22:15:53 +0100
Message-ID: <87k4ik9xpy.fsf@meyering.net>
Lines: 42
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
Cc: bug-gnulib@gnu.org
Subject: [coreutils] Re: new branch in gnulib: coreutils-8.9
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Tue, 04 Jan 2011 21:16:00 -0000

Jim Meyering wrote:
> In releasing coreutils-8.9, I left its gnulib submodule pointing at a
> private commit that was going to be pushed, pending an ACK...  Since a
> slightly different commit was pushed, coreutils' submodule SHA1 was
> invalid for a short interval.  However, I have pushed my local commit
> to the new gnulib branch named "coreutils-8.9".  With that, anyone
> cloning coreutils and trying to build from the 8.9 tag *will* get a
> usable gnulib hierarchy.
>
> Thanks to Eric Blake for noticing so quickly and for the suggestion
> to push my commit to a branch in gnulib.

I've included a patch, below, that should prevent recurrence.
I've made the new rule a dependent of "alpha beta stable",
so that it's only run at release time.

Obviously it'd be nice to do this for each and every submodule,
but for that, you'd need a URL for each of them, so I'm
keeping it simple for now.

Eventually I'll move it into maint.mk, where it will perform the test
only when gnulib is one of the submodules.

diff --git a/cfg.mk b/cfg.mk
index 4df74f0..0517201 100644
--- a/cfg.mk
+++ b/cfg.mk
@@ -337,3 +337,14 @@ include $(srcdir)/dist-check.mk
 update-copyright-env = \
   UPDATE_COPYRIGHT_USE_INTERVALS=1 \
   UPDATE_COPYRIGHT_MAX_LINE_LENGTH=79
+
+# Just before tagging a release, ensure that the gnulib submodule
+# commit we're using is public.
+alpha beta stable: gnulib-commit-check
+.PHONY: gnulib-commit-check
+gnulib-commit-check:
+	submod=gnulib; \
+	st=$$(git submodule status $$submod) || exit 1; \
+	commit=$$(echo $$st | awk '{ print $$1 }') || exit 1; \
+	wget -q -O /dev/null \
+	  "http://git.savannah.gnu.org/gitweb/?p=$$submod.git;a=tree;h=$$commit"


From MAILER-DAEMON Tue Jan 04 16:29:44 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PaERs-0004G4-Kz
	for mharc-coreutils@gnu.org; Tue, 04 Jan 2011 16:29:44 -0500
Received: from [140.186.70.92] (port=54792 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PaERr-0004Fe-0g
	for coreutils@gnu.org; Tue, 04 Jan 2011 16:29:44 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1PaERp-0007pD-Kg
	for coreutils@gnu.org; Tue, 04 Jan 2011 16:29:42 -0500
Received: from mx1.redhat.com ([209.132.183.28]:21800)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>)
	id 1PaERp-0007p8-7k; Tue, 04 Jan 2011 16:29:41 -0500
Received: from int-mx10.intmail.prod.int.phx2.redhat.com
	(int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23])
	by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id p04LTdvX012822
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK);
	Tue, 4 Jan 2011 16:29:40 -0500
Received: from [10.3.113.114] (ovpn-113-114.phx2.redhat.com [10.3.113.114])
	by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP
	id p04LTdgR017137; Tue, 4 Jan 2011 16:29:39 -0500
Message-ID: <4D239142.9060901@redhat.com>
Date: Tue, 04 Jan 2011 14:29:38 -0700
From: Eric Blake <eblake@redhat.com>
Organization: Red Hat
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US;
	rv:1.9.2.13) Gecko/20101209 Fedora/3.1.7-0.35.b3pre.fc14
	Lightning/1.0b3pre Mnenhy/0.8.3 Thunderbird/3.1.7
MIME-Version: 1.0
To: Jim Meyering <jim@meyering.net>
Subject: Re: [coreutils] Re: new branch in gnulib: coreutils-8.9
References: <87ipy4bko6.fsf@meyering.net> <87k4ik9xpy.fsf@meyering.net>
In-Reply-To: <87k4ik9xpy.fsf@meyering.net>
X-Enigmail-Version: 1.1.2
OpenPGP: url=http://people.redhat.com/eblake/eblake.gpg
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature";
	boundary="------------enig98486BCA93CA732F81007402"
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
Cc: bug-gnulib@gnu.org, coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Tue, 04 Jan 2011 21:29:44 -0000

This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--------------enig98486BCA93CA732F81007402
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

On 01/04/2011 02:15 PM, Jim Meyering wrote:
>    UPDATE_COPYRIGHT_MAX_LINE_LENGTH=3D79
> +
> +# Just before tagging a release, ensure that the gnulib submodule
> +# commit we're using is public.
> +alpha beta stable: gnulib-commit-check
> +.PHONY: gnulib-commit-check
> +gnulib-commit-check:
> +	submod=3Dgnulib; \
> +	st=3D$$(git submodule status $$submod) || exit 1; \
> +	commit=3D$$(echo $$st | awk '{ print $$1 }') || exit 1; \
> +	wget -q -O /dev/null \
> +	  "http://git.savannah.gnu.org/gitweb/?p=3D$$submod.git;a=3Dtree;h=3D=
$$commit"

I'm trying to come up with some
git submodule foreach 'command $sha1'
formula that will allow you to make the same check without having to
resort to wget or even to hardcoding the URL of each upstream repository.=


--=20
Eric Blake   eblake@redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org


--------------enig98486BCA93CA732F81007402
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Public key at http://people.redhat.com/eblake/eblake.gpg
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/

iQEcBAEBCAAGBQJNI5FCAAoJEKeha0olJ0NqfZoH/2iVN0eUSXrpujxlWhDt8czp
XsGiLs1rclTEFhtNP9H9/J1OLJZ+otilao0Afm16vzvmYBcRcxEew/KMzkr4qieL
JkRx8jwgkXxyJ3sdWiGJkX5tZH/Ia0W/pZPd8U5tv7j+sXfsgKN01v1kYT/B5y9f
6rcWxbZMulljzng7jXZ/+9wbKIB5iY9ea5aEbCBuLQcA5ROW81yYpovcgMcw+OSe
tWFAQW/NAqOxb66CdGOVPoxkNxSezDK6sHwdEdlHBB+GJu46dNNT54a8f93jUXby
px2mxOt2mjhMajkytPCeLVQfUXzYwR1CdazVsEfHtlPfmoQQTcXpG2tBtdPn9kg=
=iSl0
-----END PGP SIGNATURE-----

--------------enig98486BCA93CA732F81007402--


From MAILER-DAEMON Tue Jan 04 17:01:34 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PaEwg-0007Mv-Hu
	for mharc-coreutils@gnu.org; Tue, 04 Jan 2011 17:01:34 -0500
Received: from [140.186.70.92] (port=48503 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PaEwe-0007Lo-4O
	for coreutils@gnu.org; Tue, 04 Jan 2011 17:01:33 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1PaEwc-00063L-VU
	for coreutils@gnu.org; Tue, 04 Jan 2011 17:01:31 -0500
Received: from mx1.redhat.com ([209.132.183.28]:53286)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>)
	id 1PaEwZ-00062R-V5; Tue, 04 Jan 2011 17:01:28 -0500
Received: from int-mx12.intmail.prod.int.phx2.redhat.com
	(int-mx12.intmail.prod.int.phx2.redhat.com [10.5.11.25])
	by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id p04M1QaP025485
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK);
	Tue, 4 Jan 2011 17:01:26 -0500
Received: from [10.3.113.114] (ovpn-113-114.phx2.redhat.com [10.3.113.114])
	by int-mx12.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP
	id p04M1POl001631; Tue, 4 Jan 2011 17:01:26 -0500
Message-ID: <4D2398B5.4000902@redhat.com>
Date: Tue, 04 Jan 2011 15:01:25 -0700
From: Eric Blake <eblake@redhat.com>
Organization: Red Hat
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US;
	rv:1.9.2.13) Gecko/20101209 Fedora/3.1.7-0.35.b3pre.fc14
	Lightning/1.0b3pre Mnenhy/0.8.3 Thunderbird/3.1.7
MIME-Version: 1.0
Subject: Re: [coreutils] Re: new branch in gnulib: coreutils-8.9
References: <87ipy4bko6.fsf@meyering.net> <87k4ik9xpy.fsf@meyering.net>
	<4D239142.9060901@redhat.com>
In-Reply-To: <4D239142.9060901@redhat.com>
X-Enigmail-Version: 1.1.2
OpenPGP: url=http://people.redhat.com/eblake/eblake.gpg
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature";
	boundary="------------enig9AA7356C222832F54B1B2B7A"
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.25
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
Cc: bug-gnulib@gnu.org, coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Tue, 04 Jan 2011 22:01:33 -0000

This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--------------enig9AA7356C222832F54B1B2B7A
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

On 01/04/2011 02:29 PM, Eric Blake wrote:
>> +# Just before tagging a release, ensure that the gnulib submodule
>> +# commit we're using is public.
>> +alpha beta stable: gnulib-commit-check
>> +.PHONY: gnulib-commit-check
>> +gnulib-commit-check:
>> +	submod=3Dgnulib; \
>> +	st=3D$$(git submodule status $$submod) || exit 1; \
>> +	commit=3D$$(echo $$st | awk '{ print $$1 }') || exit 1; \
>> +	wget -q -O /dev/null \
>> +	  "http://git.savannah.gnu.org/gitweb/?p=3D$$submod.git;a=3Dtree;h=3D=
$$commit"
>=20
> I'm trying to come up with some
> git submodule foreach 'command $sha1'
> formula that will allow you to make the same check without having to
> resort to wget or even to hardcoding the URL of each upstream repositor=
y.

Found one, and it even works on libvirt (where the gnulib submodule is
named .gnulib instead of gnulib).  It should also work on a repository
with multiple submodules, although I have not yet tested it on bison.git.=


gnulib-commit-check:
	git submodule foreach test '$$(git merge-base --independent \
	  origin $$sha1 | wc -w)' =3D 1

This effectively runs the shell command:

git merge-base --independent origin $sha1

with $sha1 set to the value recorded in the superproject (identical to
your computation of $commit), and outputs the minimal set of revisions
not reachable from any other in the set of arguments.  If origin
contains $sha1, the output is a single hash (it happens to be that of
origin); if $sha1 is a local-only revision, the output is two hashes, so
the test fails, which in turn lets 'make alpha' fail.

No need for wget, awk, or a hardcoded URL :)

--=20
Eric Blake   eblake@redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org


--------------enig9AA7356C222832F54B1B2B7A
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Public key at http://people.redhat.com/eblake/eblake.gpg
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/

iQEcBAEBCAAGBQJNI5i1AAoJEKeha0olJ0NqJkUH/2m9aWTkNdmzNnVNeeUDQeDd
dpJfQnm4ZIMcLWV9X02AtyQqi91ixhAtB7LEkagH4iTgG+O9rqmW7NFmNYhC2qQe
CZWbWjPImfIblZwBoa9x1GLmaPvpK9v61ogewm4eaGDqywaHyICoXiHZy3J6tjz1
Gi47B5mlqkhTLtttv8B4L4fi3747AX+R3CTiJ2vp/Sr/PXWHNFDFRIEDgVEkqy6u
l9/uiP7am4cBsKvaZHkOfbOkslCOj+6Al79QGt+baj6PLsk5xdJ9+T/dsLLPMNLX
IvAT8fzKA34gWgM+Tf6OTGbUFjfY3X/F8vW1dpEpix+JEhkeGUV1Y0opp/xaxKY=
=0Lsn
-----END PGP SIGNATURE-----

--------------enig9AA7356C222832F54B1B2B7A--


From MAILER-DAEMON Tue Jan 04 17:10:31 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PaF5L-0006Hm-4Z
	for mharc-coreutils@gnu.org; Tue, 04 Jan 2011 17:10:31 -0500
Received: from [140.186.70.92] (port=58865 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PaErr-0005b4-SO
	for coreutils@gnu.org; Tue, 04 Jan 2011 16:56:36 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <public@plane.gmane.org>) id 1PaErq-00052o-Az
	for coreutils@gnu.org; Tue, 04 Jan 2011 16:56:35 -0500
Received: from plane.gmane.org ([80.91.229.3]:42212)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <public@plane.gmane.org>) id 1PaErq-000523-36
	for coreutils@gnu.org; Tue, 04 Jan 2011 16:56:34 -0500
Received: from public by plane.gmane.org with local (Exim 4.69)
	(envelope-from <public@plane.gmane.org>) id 1PaErl-0004g6-66
	for coreutils@gnu.org; Tue, 04 Jan 2011 22:56:29 +0100
Received: from mail-out.m-online.net ([212.18.0.9])
	by plane.gmane.org with esmtp (Exim 4.69)
	(envelope-from <whitebox@nefkom.net>)
	id 1PaErZ-0004Zf-D8; Tue, 04 Jan 2011 22:56:17 +0100
Received: from frontend1.mail.m-online.net (unknown [192.168.8.180])
	by mail-out.m-online.net (Postfix) with ESMTP id DF58B1C1DA1F;
	Tue,  4 Jan 2011 22:56:16 +0100 (CET)
Received: from localhost (dynscan1.mnet-online.de [192.168.8.164])
	by mail.m-online.net (Postfix) with ESMTP id C5D3A1C00192;
	Tue,  4 Jan 2011 22:56:16 +0100 (CET)
X-Virus-Scanned: amavisd-new at mnet-online.de
Received: from mail.mnet-online.de ([192.168.8.180])
	by localhost (dynscan1.mail.m-online.net [192.168.8.164]) (amavisd-new,
	port 10024)
	with ESMTP id iao+ANWyYkL7; Tue,  4 Jan 2011 22:56:15 +0100 (CET)
Received: from igel.home (ppp-88-217-104-20.dynamic.mnet-online.de
	[88.217.104.20]) by mail.mnet-online.de (Postfix) with ESMTP;
	Tue,  4 Jan 2011 22:56:15 +0100 (CET)
Received: by igel.home (Postfix, from userid 501)
	id 82A77CA2A0; Tue,  4 Jan 2011 22:56:15 +0100 (CET)
From: Andreas Schwab <schwab@linux-m68k.org>
To: Eric Blake <public-eblake-H+wXaHxf7aLQT0dZR+AlfA@plane.gmane.org>
References: <87ipy4bko6.fsf@meyering.net> <87k4ik9xpy.fsf@meyering.net>
	<4D239142.9060901@redhat.com>
X-Yow: Somewhere in suburban Honolulu, an unemployed bellhop is whipping up
	a batch of illegal psilocybin chop suey!!
Date: Tue, 04 Jan 2011 22:56:15 +0100
In-Reply-To: <4D239142.9060901@redhat.com> (Eric Blake's message of "Tue, 04
	Jan 2011 14:29:38 -0700")
Message-ID: <m2wrmkgwow.fsf@igel.home>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.2.91 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Mailman-Approved-At: Tue, 04 Jan 2011 17:10:29 -0500
Cc: public-coreutils-mXXj517/zsQ@plane.gmane.org,
	Jim Meyering <public-jim-Oxw1nKZkIVjk1uMJSBkQmQ@plane.gmane.org>,
	public-bug-gnulib-mXXj517/zsQ@plane.gmane.org
Subject: [coreutils] Re: new branch in gnulib: coreutils-8.9
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Tue, 04 Jan 2011 21:56:36 -0000



Eric Blake <eblake-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> writes:

> I'm trying to come up with some
> git submodule foreach 'command $sha1'
> formula that will allow you to make the same check without having to
> resort to wget or even to hardcoding the URL of each upstream repository.

Like git ls-remote?

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."



From MAILER-DAEMON Tue Jan 04 17:10:58 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PaF5m-0006Jq-6H
	for mharc-coreutils@gnu.org; Tue, 04 Jan 2011 17:10:58 -0500
Received: from [140.186.70.92] (port=41222 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PaF4F-0003E4-Fu
	for coreutils@gnu.org; Tue, 04 Jan 2011 17:10:57 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1PaF3U-0006sr-L6
	for coreutils@gnu.org; Tue, 04 Jan 2011 17:08:37 -0500
Received: from mx1.redhat.com ([209.132.183.28]:14695)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>)
	id 1PaF3T-0006sY-CI; Tue, 04 Jan 2011 17:08:35 -0500
Received: from int-mx12.intmail.prod.int.phx2.redhat.com
	(int-mx12.intmail.prod.int.phx2.redhat.com [10.5.11.25])
	by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id p04M8YWR025678
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK);
	Tue, 4 Jan 2011 17:08:34 -0500
Received: from [10.3.113.114] (ovpn-113-114.phx2.redhat.com [10.3.113.114])
	by int-mx12.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP
	id p04M8Xrm003509; Tue, 4 Jan 2011 17:08:33 -0500
Message-ID: <4D239A5B.9010005@redhat.com>
Date: Tue, 04 Jan 2011 15:08:27 -0700
From: Eric Blake <eblake@redhat.com>
Organization: Red Hat
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US;
	rv:1.9.2.13) Gecko/20101209 Fedora/3.1.7-0.35.b3pre.fc14
	Lightning/1.0b3pre Mnenhy/0.8.3 Thunderbird/3.1.7
MIME-Version: 1.0
Subject: Re: [coreutils] Re: new branch in gnulib: coreutils-8.9
References: <87ipy4bko6.fsf@meyering.net>
	<87k4ik9xpy.fsf@meyering.net>	<4D239142.9060901@redhat.com>
	<4D2398B5.4000902@redhat.com>
In-Reply-To: <4D2398B5.4000902@redhat.com>
X-Enigmail-Version: 1.1.2
OpenPGP: url=http://people.redhat.com/eblake/eblake.gpg
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature";
	boundary="------------enig19B35F3BF5D4A1A0DC2B67A7"
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.25
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
Cc: bug-gnulib@gnu.org, coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Tue, 04 Jan 2011 22:10:57 -0000

This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--------------enig19B35F3BF5D4A1A0DC2B67A7
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

On 01/04/2011 03:01 PM, Eric Blake wrote:
> Found one, and it even works on libvirt (where the gnulib submodule is
> named .gnulib instead of gnulib).  It should also work on a repository
> with multiple submodules, although I have not yet tested it on bison.gi=
t.
>=20
> gnulib-commit-check:
> 	git submodule foreach test '$$(git merge-base --independent \
> 	  origin $$sha1 | wc -w)' =3D 1
>=20
> This effectively runs the shell command:
>=20
> git merge-base --independent origin $sha1
>=20
> with $sha1 set to the value recorded in the superproject (identical to
> your computation of $commit), and outputs the minimal set of revisions
> not reachable from any other in the set of arguments.  If origin
> contains $sha1, the output is a single hash (it happens to be that of
> origin); if $sha1 is a local-only revision, the output is two hashes, s=
o
> the test fails, which in turn lets 'make alpha' fail.

Not quite robust enough - if your superproject contains commits not
pushed upstream yet, then merge-base would still output a single
reference.  This should close that gap, by requiring that the least
reachable commit be exactly the upstream origin.

gnulib-commit-check:
	git submodule foreach test '$$(git rev-parse origin)' \
	  =3D '"$$(git merge-base --independent origin $$sha1)"'

--=20
Eric Blake   eblake@redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org


--------------enig19B35F3BF5D4A1A0DC2B67A7
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Public key at http://people.redhat.com/eblake/eblake.gpg
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/

iQEcBAEBCAAGBQJNI5pbAAoJEKeha0olJ0NqerAH/0AtwLsuY8OI7WOSRnwKcfx3
wZWJFlABqWk1KeHMCZpps9DOQPrSmj0TJGHqC39qbW40KLs/j0ZdEjDRr4UofYYq
mxR1nTY0zLGwuWoneUy+qdUgyKyFhvHNL58XRn1VFh4XhwuRG/nmOLHa7vSpaBl1
Ve5eyghuyLB7I1WPG/IFaM6KTUe+peaa+wPwSPNk2OyTKvuYqFNaCAAsMpDXkdaQ
PQ5zABDbVGptx/49N8A3KJ47YH5psBx8LjWklSG9v6Dk8nytaMxOEq6ppDLBLsC9
HJqfI7CfN5f+XLrqgW8j8wkUGFC4q/pQr/678Z+U2yEyeUx8LthZ9E2xgq3VcDI=
=E0Mv
-----END PGP SIGNATURE-----

--------------enig19B35F3BF5D4A1A0DC2B67A7--


From MAILER-DAEMON Wed Jan 05 10:23:09 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PaVCf-0003W3-JQ
	for mharc-coreutils@gnu.org; Wed, 05 Jan 2011 10:23:09 -0500
Received: from [140.186.70.92] (port=59885 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PaVCd-0003Us-IJ
	for coreutils@gnu.org; Wed, 05 Jan 2011 10:23:08 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PaVCc-0003iV-JF
	for coreutils@gnu.org; Wed, 05 Jan 2011 10:23:07 -0500
Received: from mx.meyering.net ([82.230.74.64]:40059)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>)
	id 1PaVCZ-0003hO-A3; Wed, 05 Jan 2011 10:23:03 -0500
Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000)
	id 880EB600E7; Wed,  5 Jan 2011 16:23:02 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: Eric Blake <eblake@redhat.com>
Subject: Re: [coreutils] Re: new branch in gnulib: coreutils-8.9
In-Reply-To: <4D239A5B.9010005@redhat.com> (Eric Blake's message of "Tue, 04
	Jan 2011 15:08:27 -0700")
References: <87ipy4bko6.fsf@meyering.net> <87k4ik9xpy.fsf@meyering.net>
	<4D239142.9060901@redhat.com> <4D2398B5.4000902@redhat.com>
	<4D239A5B.9010005@redhat.com>
Date: Wed, 05 Jan 2011 16:23:02 +0100
Message-ID: <87y66z74tl.fsf@meyering.net>
Lines: 33
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
Cc: bug-gnulib@gnu.org, coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Wed, 05 Jan 2011 15:23:08 -0000

Eric Blake wrote:

> On 01/04/2011 03:01 PM, Eric Blake wrote:
>> Found one, and it even works on libvirt (where the gnulib submodule is
>> named .gnulib instead of gnulib).  It should also work on a repository
>> with multiple submodules, although I have not yet tested it on bison.git.
>>
>> gnulib-commit-check:
>> 	git submodule foreach test '$$(git merge-base --independent \
>> 	  origin $$sha1 | wc -w)' = 1
>>
>> This effectively runs the shell command:
>>
>> git merge-base --independent origin $sha1
>>
>> with $sha1 set to the value recorded in the superproject (identical to
>> your computation of $commit), and outputs the minimal set of revisions
>> not reachable from any other in the set of arguments.  If origin
>> contains $sha1, the output is a single hash (it happens to be that of
>> origin); if $sha1 is a local-only revision, the output is two hashes, so
>> the test fails, which in turn lets 'make alpha' fail.
>
> Not quite robust enough - if your superproject contains commits not
> pushed upstream yet, then merge-base would still output a single
> reference.  This should close that gap, by requiring that the least
> reachable commit be exactly the upstream origin.
>
> gnulib-commit-check:
> 	git submodule foreach test '$$(git rev-parse origin)' \
> 	  = '"$$(git merge-base --independent origin $$sha1)"'

Sounds nice.  Thanks!
I'll take a look this evening.


From MAILER-DAEMON Thu Jan 06 04:44:47 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PamOl-0002Z8-5E
	for mharc-coreutils@gnu.org; Thu, 06 Jan 2011 04:44:47 -0500
Received: from [140.186.70.92] (port=51575 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PamOi-0002YR-KP
	for coreutils@gnu.org; Thu, 06 Jan 2011 04:44:45 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PamOh-0001ut-HD
	for coreutils@gnu.org; Thu, 06 Jan 2011 04:44:44 -0500
Received: from mx.meyering.net ([82.230.74.64]:34770)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>)
	id 1PamOe-0001uG-H5; Thu, 06 Jan 2011 04:44:40 -0500
Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000)
	id 0E80660179; Thu,  6 Jan 2011 10:44:39 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: Eric Blake <eblake@redhat.com>
Subject: Re: [coreutils] Re: new branch in gnulib: coreutils-8.9
In-Reply-To: <4D239A5B.9010005@redhat.com> (Eric Blake's message of "Tue, 04
	Jan 2011 15:08:27 -0700")
References: <87ipy4bko6.fsf@meyering.net> <87k4ik9xpy.fsf@meyering.net>
	<4D239142.9060901@redhat.com> <4D2398B5.4000902@redhat.com>
	<4D239A5B.9010005@redhat.com>
Date: Thu, 06 Jan 2011 10:44:38 +0100
Message-ID: <87k4ii4b95.fsf@meyering.net>
Lines: 88
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
Cc: bug-gnulib@gnu.org, coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 06 Jan 2011 09:44:45 -0000

Eric Blake wrote:
> On 01/04/2011 03:01 PM, Eric Blake wrote:
>> Found one, and it even works on libvirt (where the gnulib submodule is
>> named .gnulib instead of gnulib).  It should also work on a repository
>> with multiple submodules, although I have not yet tested it on bison.git.
>>
>> gnulib-commit-check:
>> 	git submodule foreach test '$$(git merge-base --independent \
>> 	  origin $$sha1 | wc -w)' =3D 1
>>
>> This effectively runs the shell command:
>>
>> git merge-base --independent origin $sha1
>>
>> with $sha1 set to the value recorded in the superproject (identical to
>> your computation of $commit), and outputs the minimal set of revisions
>> not reachable from any other in the set of arguments.  If origin
>> contains $sha1, the output is a single hash (it happens to be that of
>> origin); if $sha1 is a local-only revision, the output is two hashes, so
>> the test fails, which in turn lets 'make alpha' fail.
>
> Not quite robust enough - if your superproject contains commits not
> pushed upstream yet, then merge-base would still output a single
> reference.  This should close that gap, by requiring that the least
> reachable commit be exactly the upstream origin.
>
> gnulib-commit-check:
> 	git submodule foreach test '$$(git rev-parse origin)' \
> 	  =3D '"$$(git merge-base --independent origin $$sha1)"'

Thanks again for the fine test.
It passed my tests, so I propose this in your name:

>From 8ba2dc9163f753c4953e8686f2b611d4e2a3ae84 Mon Sep 17 00:00:00 2001
From: Eric Blake <eblake@redhat.com>
Date: Thu, 6 Jan 2011 10:35:18 +0100
Subject: [PATCH] maint.mk: add pre-release check to ensure submodule commit=
s are public

* top/maint.mk (public-submodule-commit): New rule.
(submodule-checks): New variable.
(alpha beta stable): Depend on the variable.
---
 ChangeLog    |    7 +++++++
 top/maint.mk |   12 +++++++++++-
 2 files changed, 18 insertions(+), 1 deletions(-)

diff --git a/ChangeLog b/ChangeLog
index 6b99e92..5e319f9 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,10 @@
+2011-01-06  Eric Blake  <eblake@redhat.com>
+
+	maint.mk: add pre-release check to ensure submodule commits are public
+	* top/maint.mk (public-submodule-commit): New rule.
+	(submodule-checks): New variable.
+	(alpha beta stable): Depend on the variable.
+
 2011-01-05  P=C3=A1draig Brady <P@draigBrady.com>
 	and Jim Meyering  <meyering@redhat.com>

diff --git a/top/maint.mk b/top/maint.mk
index 5545e69..3a3ede6 100644
--- a/top/maint.mk
+++ b/top/maint.mk
@@ -1118,9 +1118,19 @@ no-submodule-changes:
 	  : ;								\
 	fi

+submodule-checks =3D no-submodule-changes public-submodule-commit
+
+# Ensure that each sub-module commit we're using is public.
+# Without this, it is too easy to tag and release code that
+# cannot be built from a fresh clone.
+.PHONY: public-submodule-commit
+public-submodule-commit:
+	git submodule foreach 'test $$(git rev-parse origin)' \
+	  =3D '"$$(git merge-base --independent origin $$sha1)"'
+
 .PHONY: alpha beta stable
 ALL_RECURSIVE_TARGETS +=3D alpha beta stable
-alpha beta stable: $(local-check) writable-files no-submodule-changes
+alpha beta stable: $(local-check) writable-files $(submodule-checks)
 	test $@ =3D stable						\
 	  && { echo $(VERSION) | grep -E '^[0-9]+(\.[0-9]+)+$$'		\
 	       || { echo "invalid version string: $(VERSION)" 1>&2; exit 1;};}\
--
1.7.3.4


From MAILER-DAEMON Thu Jan 06 07:06:14 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1Paobd-0004EK-SC
	for mharc-coreutils@gnu.org; Thu, 06 Jan 2011 07:06:13 -0500
Received: from [140.186.70.92] (port=38672 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1Paobb-0004DM-DO
	for coreutils@gnu.org; Thu, 06 Jan 2011 07:06:12 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1PaobY-0002GU-P3
	for coreutils@gnu.org; Thu, 06 Jan 2011 07:06:11 -0500
Received: from mail1.slb.deg.dub.stisp.net ([84.203.253.98]:18607)
	by eggs.gnu.org with smtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1PaobY-0002G2-EI
	for coreutils@gnu.org; Thu, 06 Jan 2011 07:06:08 -0500
Received: (qmail 99383 invoked from network); 6 Jan 2011 12:06:05 -0000
Received: from unknown (HELO ?192.168.2.25?) (84.203.137.218)
	by mail1.slb.deg.dub.stisp.net with SMTP; 6 Jan 2011 12:06:05 -0000
Message-ID: <4D25AFED.1090902@draigBrady.com>
Date: Thu, 06 Jan 2011 12:05:01 +0000
From: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US;
	rv:1.9.1.8) Gecko/20100227 Thunderbird/3.0.3
MIME-Version: 1.0
To: Assaf Gordon <assafgordon@gmail.com>
Subject: Re: [coreutils] join feature: auto-format
References: <4CACDEE5.6050908@gmail.com>
	<4CAD0E47.7070005@draigBrady.com>	<4CAD9F55.6000001@draigBrady.com>
	<4CAE06BA.2080402@gmail.com> <4CAE1099.4010702@draigBrady.com>
In-Reply-To: <4CAE1099.4010702@draigBrady.com>
X-Enigmail-Version: 1.0.1
Content-Type: multipart/mixed; boundary="------------010307090902090400050007"
X-detected-operating-system: by eggs.gnu.org: FreeBSD 4.6-4.9
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 06 Jan 2011 12:06:12 -0000

This is a multi-part message in MIME format.
--------------010307090902090400050007
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit

On 07/10/10 19:25, Pádraig Brady wrote:
> On 07/10/10 18:43, Assaf Gordon wrote:
>> Pádraig Brady wrote, On 10/07/2010 06:22 AM:
>>> On 07/10/10 01:03, Pádraig Brady wrote:
>>>> On 06/10/10 21:41, Assaf Gordon wrote:
>>>>>
>>>>> The "--auto-format" feature simply builds the "-o" format line automatically, based on the number of columns from both input files.
>>>>
>>>> Thanks for persisting with this and presenting a concise example.
>>>> I agree that this is useful and can't think of a simple workaround.
>>>> Perhaps the interface would be better as:
>>>>
>>>> -o {all (default), padded, FORMAT}
>>>>
>>>> where padded is the functionality you're suggesting?
>>>
>>> Thinking more about it, we mightn't need any new options at all.
>>> Currently -e is redundant if -o is not specified.
>>> So how about changing that so that if -e is specified
>>> we operate as above by auto inserting empty fields?
>>> Also I wouldn't base on the number of fields in the first line,
>>> instead auto padding to the biggest number of fields
>>> on the current lines under consideration.
>>
>> My concern is the principle of "least surprise" - if there are existing scripts/programs that specify "-e" without "-o" (doesn't make sense, but still possible) - this change will alter their behavior.
>>
>> Also, implying/forcing 'auto-format' when "-e" is used without "-o" might be a bit confusing.
> 
> Well seeing as -e without -o currently does nothing,
> I don't think we need to worry too much about changing that behavior.
> Also to me, specifying -e EMPTY implicitly means I want
> fields missing from one of the files replaced with EMPTY.
> 
> Note POSIX is more explicit, and describes our current operation:
> 
> -e EMPTY
>   Replace empty output fields in the list selected by -o with EMPTY
> 
> So changing that would be an extension to POSIX.
> But I still think it makes sense.
> I'll prepare a patch soon, to do as I describe above,
> unless there are objections.

The attached changes `join` (from what's done on other platforms) so that...

`join -e` will automatically pad missing fields from one file
so that the same number of fields are output from each file.
Previously -e was only used for missing fields specified with -o or -j.

With this change join now does:

$ cat file1
a 1 2
b 1
d 1 2

$ cat file2
a 3 4
b 3 4
c 3 4

$ join -a1 -a2 -1 1 -2 1 -e. file1 file2
a 1 2 3 4
b 1 . 3 4
c . . 3 4
d 1 2 . .

$ join -a1 -a2 -1 1 -2 4 -e. file1 file2
. . . . a 3 4
. . . . b 3 4
. . . . c 3 4
a 1 2 . .
b 1 .
d 1 2 . .

$ join -a1 -a2 -1 4 -2 1 -e. file1 file2
. a 1 2 . . .
. b 1 . .
. d 1 2 . . .
a . . 3 4
b . . 3 4
c . . 3 4

$ join -a1 -a2 -1 4 -2 4 -e. file1 file2
. a 1 2 a 3 4
. a 1 2 b 3 4
. a 1 2 c 3 4
. b 1 . a 3 4
. b 1 . b 3 4
. b 1 . c 3 4
. d 1 2 a 3 4
. d 1 2 b 3 4
. d 1 2 c 3 4

While -e without -o was previously a noop, and so could safely be extended IMHO,
this will also change the behavior when with -e and -j are specified.
Previously if -j > 1 was specified, and that field was missing,
then -e would be used in its place, rather than the empty string.
This still does that, but also does the padding.
Without the -j issue I'd be 80:20 for just extending -e to auto pad,
but given -j I'm 50:50.  The alternative it to select this with
say '-o padded', but that's less discoverable, and complicates
the interface somewhat.

cheers,
Pádraig.

--------------010307090902090400050007
Content-Type: text/x-patch;
 name="join-auto-format.diff"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename="join-auto-format.diff"

>From 3ea251d770fd79c37ee5195480574ccd6bfa156c Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?P=C3=A1draig=20Brady?= <P@draigBrady.com>
Date: Wed, 5 Jan 2011 11:52:54 +0000
Subject: [PATCH] join: auto pad missing fields when -e specified

* src/join.c (prfields): A new function refactored from prjoin(),
to output all but the join field. This function also applies
the extra padding when appropriate.
(prjoin): Don't swap line1 and line2 when line1 is blank
so that the padding is applied to the right place.
* tests/misc/join: Remove existing uses of -e without -o,
and add 2 new cases to test the new auto padding behavior.
* NEWS: Mention the change in behavior.
Suggestion from Assaf Gordon
---
 NEWS            |    6 ++++
 src/join.c      |   81 ++++++++++++++++++++++++++++++++++++-------------------
 tests/misc/join |   16 ++++++++---
 3 files changed, 71 insertions(+), 32 deletions(-)

diff --git a/NEWS b/NEWS
index 2a71ca6..f02e8a8 100644
--- a/NEWS
+++ b/NEWS
@@ -2,6 +2,12 @@ GNU coreutils NEWS                                    -*- outline -*-
 
 * Noteworthy changes in release ?.? (????-??-??) [?]
 
+** Changes in behavior
+
+  join -e will automatically pad missing fields from one file
+  so that the same number of fields are output from each file.
+  Previously -e was only used for missing fields specified with -o or -j.
+
 
 * Noteworthy changes in release 8.9 (2011-01-04) [stable]
 
diff --git a/src/join.c b/src/join.c
index afda5a1..1df24a9 100644
--- a/src/join.c
+++ b/src/join.c
@@ -527,6 +527,45 @@ prfield (size_t n, struct line const *line)
     fputs (empty_filler, stdout);
 }
 
+/* Output all the fields in line, other than the join field.  */
+
+static void
+prfields (struct line const *line, size_t join_field,
+          struct line const *oline, size_t ojoin_field)
+{
+  size_t i;
+  char output_separator = tab < 0 ? ' ' : tab;
+
+  for (i = 0; i < join_field && i < line->nfields; ++i)
+    {
+      putchar (output_separator);
+      prfield (i, line);
+    }
+  for (i = join_field + 1; i < line->nfields; ++i)
+    {
+      putchar (output_separator);
+      prfield (i, line);
+    }
+  if (empty_filler && oline->nfields > line->nfields)
+    {
+      bool extra_filler = false;
+
+      /* Increase padding by 1 to account for the implicit
+         join field outputted for a blank line.  */
+      if (line != &uni_blank)
+        extra_filler = true;
+      /* Or if we didn't print a join field, increase padding.  */
+      else if (oline->nfields <= ojoin_field || !oline->fields[ojoin_field].len)
+        extra_filler = true;
+
+      for (i = !extra_filler; i < oline->nfields - line->nfields; ++i)
+        {
+          putchar (output_separator);
+          fputs (empty_filler, stdout);
+        }
+    }
+}
+
 /* Print the join of LINE1 and LINE2.  */
 
 static void
@@ -534,6 +573,8 @@ prjoin (struct line const *line1, struct line const *line2)
 {
   const struct outlist *outlist;
   char output_separator = tab < 0 ? ' ' : tab;
+  size_t field;
+  struct line const *line;
 
   outlist = outlist_head.next;
   if (outlist)
@@ -543,9 +584,6 @@ prjoin (struct line const *line1, struct line const *line2)
       o = outlist;
       while (1)
         {
-          size_t field;
-          struct line const *line;
-
           if (o->file == 0)
             {
               if (line1 == &uni_blank)
@@ -574,37 +612,24 @@ prjoin (struct line const *line1, struct line const *line2)
     }
   else
     {
-      size_t i;
-
       if (line1 == &uni_blank)
         {
-          struct line const *t;
-          t = line1;
-          line1 = line2;
-          line2 = t;
+          line = line2;
+          field = join_field_2;
         }
-      prfield (join_field_1, line1);
-      for (i = 0; i < join_field_1 && i < line1->nfields; ++i)
-        {
-          putchar (output_separator);
-          prfield (i, line1);
-        }
-      for (i = join_field_1 + 1; i < line1->nfields; ++i)
+      else
         {
-          putchar (output_separator);
-          prfield (i, line1);
+          line = line1;
+          field = join_field_1;
         }
 
-      for (i = 0; i < join_field_2 && i < line2->nfields; ++i)
-        {
-          putchar (output_separator);
-          prfield (i, line2);
-        }
-      for (i = join_field_2 + 1; i < line2->nfields; ++i)
-        {
-          putchar (output_separator);
-          prfield (i, line2);
-        }
+      /* Output the join field.  */
+      prfield (field, line);
+
+      /* Output other fields.  */
+      prfields (line1, join_field_1, line2, join_field_2);
+      prfields (line2, join_field_2, line1, join_field_1);
+
       putchar ('\n');
     }
 }
diff --git a/tests/misc/join b/tests/misc/join
index 3696a03..7ca628b 100755
--- a/tests/misc/join
+++ b/tests/misc/join
@@ -43,7 +43,7 @@ my @tv = (
 ['1e', '-a2',      ["a 1\nb\n", "b\n"],   "b\n",         0],
 ['1f', '-a2',      ["b\n", "a\nb\n"], "a\nb\n",          0],
 
-['2a', '-a1 -e .',  ["a\nb\nc\n", "a x y\nb\nc\n"], "a x y\nb\nc\n", 0],
+['2a', '-a1',  ["a\nb\nc\n", "a x y\nb\nc\n"], "a x y\nb\nc\n", 0],
 ['2b', '-a1 -e . -o 2.1,2.2,2.3',  ["a\nb\nc\n", "a x y\nb\nc\n"],
  "a x y\nb . .\nc . .\n", 0],
 ['2c', '-a1 -e . -o 2.1,2.2,2.3',  ["a\nb\nc\nd\n", "a x y\nb\nc\n"],
@@ -104,13 +104,13 @@ my @tv = (
  ["apr 15\naug 20\ndec 18\n", "apr 06\naug 14\ndate\nfeb 15\n"],
   "06 apr\n14 aug\n- -\n15 -\n", 0],
 
-['6a', '-e -',
+['6a', '',
  ["a 1\nb 2\nd 4\n", "a 21\nb 22\nc 23\nf 26\n"],
  "a 1 21\nb 2 22\n", 0],
-['6b', '-a1 -e -',
+['6b', '-a1',
  ["a 1\nb 2\nd 4\n", "a 21\nb 22\nc 23\nf 26\n"],
  "a 1 21\nb 2 22\nd 4\n", 0],
-['6c', '-a1 -e -',
+['6c', '-a1',
  ["a 21\nb 22\nc 23\nf 26\n", "a 1\nb 2\nd 4\n"],
  "a 21 1\nb 22 2\nc 23\nf 26\n", 0],
 
@@ -127,6 +127,14 @@ my @tv = (
 # From David Dyck
 ['9a', '', [" a 1\n b 2\n", " a Y\n b Z\n"], "a 1 Y\nb 2 Z\n", 0],
 
+# Test -e without -o (auto pad)
+['10a', '-a1 -a2 -e .',
+ ["a 1 2\nb 1\nd 1 2\n", "a 3 4\nb 3 4\nc 3 4\n"],
+ "a 1 2 3 4\nb 1 . 3 4\nc . . 3 4\nd 1 2 . .\n", 0],
+['10b', '-a1 -a2 -j3 -e .',
+ ["a 1 2\nb 1\nd 1 2\n", "a 3 4\nb 3 4\nc 3 4\n"],
+ "2 a 1 . .\n. b 1 . .\n2 d 1 . .\n4 . . a 3\n4 . . b 3\n4 . . c 3\n"],
+
 # From Tim Smithers: fixed in 1.22l
 ['trailing-sp', '-t: -1 1 -2 1', ["a:x \n", "a:y \n"], "a:x :y \n", 0],
 
-- 
1.7.3.4


--------------010307090902090400050007--


From MAILER-DAEMON Thu Jan 06 09:52:37 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1ParCe-0005Sk-PV
	for mharc-coreutils@gnu.org; Thu, 06 Jan 2011 09:52:36 -0500
Received: from [140.186.70.92] (port=57573 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1ParCa-0004vW-Fo
	for coreutils@gnu.org; Thu, 06 Jan 2011 09:52:35 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1ParBn-0007I8-W8
	for coreutils@gnu.org; Thu, 06 Jan 2011 09:51:45 -0500
Received: from mx1.redhat.com ([209.132.183.28]:12027)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>)
	id 1ParB9-0007Aa-KW; Thu, 06 Jan 2011 09:51:03 -0500
Received: from int-mx02.intmail.prod.int.phx2.redhat.com
	(int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12])
	by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id p06Ep1OK020234
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK);
	Thu, 6 Jan 2011 09:51:02 -0500
Received: from [10.3.113.114] (ovpn-113-114.phx2.redhat.com [10.3.113.114])
	by int-mx02.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP
	id p06Ep18h027649; Thu, 6 Jan 2011 09:51:01 -0500
Message-ID: <4D25D6D4.5040401@redhat.com>
Date: Thu, 06 Jan 2011 07:51:00 -0700
From: Eric Blake <eblake@redhat.com>
Organization: Red Hat
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US;
	rv:1.9.2.13) Gecko/20101209 Fedora/3.1.7-0.35.b3pre.fc14
	Lightning/1.0b3pre Mnenhy/0.8.3 Thunderbird/3.1.7
MIME-Version: 1.0
To: Jim Meyering <jim@meyering.net>
Subject: Re: [coreutils] Re: new branch in gnulib: coreutils-8.9
References: <87ipy4bko6.fsf@meyering.net>
	<87k4ik9xpy.fsf@meyering.net>	<4D239142.9060901@redhat.com>
	<4D2398B5.4000902@redhat.com>	<4D239A5B.9010005@redhat.com>
	<87k4ii4b95.fsf@meyering.net>
In-Reply-To: <87k4ii4b95.fsf@meyering.net>
X-Enigmail-Version: 1.1.2
OpenPGP: url=http://people.redhat.com/eblake/eblake.gpg
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature";
	boundary="------------enig2E3FAA186A7E4579FE6AA502"
X-Scanned-By: MIMEDefang 2.67 on 10.5.11.12
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
Cc: bug-gnulib@gnu.org, coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 06 Jan 2011 14:52:35 -0000

This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--------------enig2E3FAA186A7E4579FE6AA502
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

On 01/06/2011 02:44 AM, Jim Meyering wrote:
>> gnulib-commit-check:
>> 	git submodule foreach test '$$(git rev-parse origin)' \
>> 	  =3D '"$$(git merge-base --independent origin $$sha1)"'
>=20
> Thanks again for the fine test.
> It passed my tests, so I propose this in your name:
>=20
>>From 8ba2dc9163f753c4953e8686f2b611d4e2a3ae84 Mon Sep 17 00:00:00 2001
> From: Eric Blake <eblake@redhat.com>
> Date: Thu, 6 Jan 2011 10:35:18 +0100
> Subject: [PATCH] maint.mk: add pre-release check to ensure submodule co=
mmits are public

Yes, feel free to push that in my name to gnulib.

> +submodule-checks =3D no-submodule-changes public-submodule-commit

If someone complains about these two tests being git-centric, a solution
for a project using maint.mk with a different VCS would be to make
submodule-checks defined via ?=3D, so that it can be overridden in cfg.mk=

to an empty macro to skip git submodule consistency checks.  But I'm
okay using =3D instead of ?=3D and waiting until someone requests that ex=
tra
flexibility, if you don't want to change it now.

> +# Ensure that each sub-module commit we're using is public.
> +# Without this, it is too easy to tag and release code that
> +# cannot be built from a fresh clone.
> +.PHONY: public-submodule-commit
> +public-submodule-commit:

And public-submodule-commit is definitely a better name than
gnulib-commit-check.

--=20
Eric Blake   eblake@redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org


--------------enig2E3FAA186A7E4579FE6AA502
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Public key at http://people.redhat.com/eblake/eblake.gpg
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/

iQEcBAEBCAAGBQJNJdbUAAoJEKeha0olJ0NqWgsH/1Fuj5cb5xvqJp64Q+elkQar
wnHj2kc+8K8nd5ZEePpOnLVOfLcCHnpmch7yXMMku0DhAUcnMmmgWem3/Qc4kjE0
DbHXJZ9K9+5ZXZQAvijUxCHnTttvIV/HOxmVmjGTl/kVKIjgxBNdBxl1sWOHJchE
EKkgPz1rgCo5m3Wbh+Qh+dGRC3fyQHt4SUIKjcJgH8AkhDZM+P7HhPNzaGz0bOf8
Pb7VTxO7NulceT493/lT3NPafAMAUU0AYU4E091Iw8tmumHNLXv6Zbuzkj2k6izY
XIBVz7/IkpCELg1hwO501rcNwtBR1epQJPs7BsIzb1L4st0qAQrUmFIob0wJq4w=
=Vppw
-----END PGP SIGNATURE-----

--------------enig2E3FAA186A7E4579FE6AA502--


From MAILER-DAEMON Thu Jan 06 10:03:30 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1ParNC-0001IW-EQ
	for mharc-coreutils@gnu.org; Thu, 06 Jan 2011 10:03:30 -0500
Received: from [140.186.70.92] (port=48426 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1ParN8-0001H8-Ei
	for coreutils@gnu.org; Thu, 06 Jan 2011 10:03:28 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1ParN4-0001Ti-QS
	for coreutils@gnu.org; Thu, 06 Jan 2011 10:03:24 -0500
Received: from mx.meyering.net ([82.230.74.64]:51409)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>)
	id 1ParMw-0001QZ-N9; Thu, 06 Jan 2011 10:03:14 -0500
Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000)
	id 4242C60179; Thu,  6 Jan 2011 16:03:13 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: Eric Blake <eblake@redhat.com>
Subject: Re: [coreutils] Re: new branch in gnulib: coreutils-8.9
In-Reply-To: <4D25D6D4.5040401@redhat.com> (Eric Blake's message of "Thu, 06
	Jan 2011 07:51:00 -0700")
References: <87ipy4bko6.fsf@meyering.net> <87k4ik9xpy.fsf@meyering.net>
	<4D239142.9060901@redhat.com> <4D2398B5.4000902@redhat.com>
	<4D239A5B.9010005@redhat.com> <87k4ii4b95.fsf@meyering.net>
	<4D25D6D4.5040401@redhat.com>
Date: Thu, 06 Jan 2011 16:03:13 +0100
Message-ID: <87sjx6yszy.fsf@meyering.net>
Lines: 94
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
Cc: bug-gnulib@gnu.org, coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 06 Jan 2011 15:03:28 -0000

Eric Blake wrote:

> On 01/06/2011 02:44 AM, Jim Meyering wrote:
>>> gnulib-commit-check:
>>> 	git submodule foreach test '$$(git rev-parse origin)' \
>>> 	  =3D '"$$(git merge-base --independent origin $$sha1)"'
>>
>> Thanks again for the fine test.
>> It passed my tests, so I propose this in your name:
>>
>>>From 8ba2dc9163f753c4953e8686f2b611d4e2a3ae84 Mon Sep 17 00:00:00 2001
>> From: Eric Blake <eblake@redhat.com>
>> Date: Thu, 6 Jan 2011 10:35:18 +0100
>> Subject: [PATCH] maint.mk: add pre-release check to ensure submodule
>> commits are public
>
> Yes, feel free to push that in my name to gnulib.
>
>> +submodule-checks =3D no-submodule-changes public-submodule-commit
>
> If someone complains about these two tests being git-centric, a solution
> for a project using maint.mk with a different VCS would be to make
> submodule-checks defined via ?=3D, so that it can be overridden in cfg.mk
> to an empty macro to skip git submodule consistency checks.  But I'm
> okay using =3D instead of ?=3D and waiting until someone requests that ex=
tra
> flexibility, if you don't want to change it now.

Good idea.
I've also made the test silently skip when there is no .git/ directory,
just like no-submodule-changes rule does.

One more iteration:

>From 807533b8c9cec5320db9cf7e0746d229bd0700a6 Mon Sep 17 00:00:00 2001
From: Eric Blake <eblake@redhat.com>
Date: Thu, 6 Jan 2011 10:35:18 +0100
Subject: [PATCH] maint.mk: add pre-release check to ensure submodule commit=
s are public

* top/maint.mk (public-submodule-commit): New rule.
(submodule-checks): New variable.
(alpha beta stable): Depend on the variable.
---
 ChangeLog    |    7 +++++++
 top/maint.mk |   18 +++++++++++++++++-
 2 files changed, 24 insertions(+), 1 deletions(-)

diff --git a/ChangeLog b/ChangeLog
index 6b99e92..5e319f9 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,10 @@
+2011-01-06  Eric Blake  <eblake@redhat.com>
+
+	maint.mk: add pre-release check to ensure submodule commits are public
+	* top/maint.mk (public-submodule-commit): New rule.
+	(submodule-checks): New variable.
+	(alpha beta stable): Depend on the variable.
+
 2011-01-05  P=C3=A1draig Brady <P@draigBrady.com>
 	and Jim Meyering  <meyering@redhat.com>

diff --git a/top/maint.mk b/top/maint.mk
index 5545e69..f892304 100644
--- a/top/maint.mk
+++ b/top/maint.mk
@@ -1118,9 +1118,25 @@ no-submodule-changes:
 	  : ;								\
 	fi

+submodule-checks ?=3D no-submodule-changes public-submodule-commit
+
+# Ensure that each sub-module commit we're using is public.
+# Without this, it is too easy to tag and release code that
+# cannot be built from a fresh clone.
+.PHONY: public-submodule-commit
+public-submodule-commit:
+	if test -d $(srcdir)/.git; then					\
+	  git submodule foreach 'test $$(git rev-parse origin)'		\
+	      =3D '"$$(git merge-base --independent origin $$sha1)"'	\
+	    || { echo '$(ME): found non-public submodule commit' >&2;	\
+		 exit 1; };						\
+	else								\
+	  : ;								\
+	fi
+
 .PHONY: alpha beta stable
 ALL_RECURSIVE_TARGETS +=3D alpha beta stable
-alpha beta stable: $(local-check) writable-files no-submodule-changes
+alpha beta stable: $(local-check) writable-files $(submodule-checks)
 	test $@ =3D stable						\
 	  && { echo $(VERSION) | grep -E '^[0-9]+(\.[0-9]+)+$$'		\
 	       || { echo "invalid version string: $(VERSION)" 1>&2; exit 1;};}\
--
1.7.3.4


From MAILER-DAEMON Thu Jan 06 10:20:10 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PardJ-0008AU-R7
	for mharc-coreutils@gnu.org; Thu, 06 Jan 2011 10:20:09 -0500
Received: from [140.186.70.92] (port=60837 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PardH-00088M-0D
	for coreutils@gnu.org; Thu, 06 Jan 2011 10:20:08 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1PardE-00066o-Mp
	for coreutils@gnu.org; Thu, 06 Jan 2011 10:20:05 -0500
Received: from mx1.redhat.com ([209.132.183.28]:20540)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>)
	id 1Pard3-00060F-6b; Thu, 06 Jan 2011 10:19:53 -0500
Received: from int-mx02.intmail.prod.int.phx2.redhat.com
	(int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12])
	by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id p06FJqWO032411
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK);
	Thu, 6 Jan 2011 10:19:52 -0500
Received: from [10.3.113.114] (ovpn-113-114.phx2.redhat.com [10.3.113.114])
	by int-mx02.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP
	id p06FJpG7004819; Thu, 6 Jan 2011 10:19:51 -0500
Message-ID: <4D25DD97.5040506@redhat.com>
Date: Thu, 06 Jan 2011 08:19:51 -0700
From: Eric Blake <eblake@redhat.com>
Organization: Red Hat
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US;
	rv:1.9.2.13) Gecko/20101209 Fedora/3.1.7-0.35.b3pre.fc14
	Lightning/1.0b3pre Mnenhy/0.8.3 Thunderbird/3.1.7
MIME-Version: 1.0
To: Jim Meyering <jim@meyering.net>
Subject: Re: [coreutils] Re: new branch in gnulib: coreutils-8.9
References: <87ipy4bko6.fsf@meyering.net>
	<87k4ik9xpy.fsf@meyering.net>	<4D239142.9060901@redhat.com>
	<4D2398B5.4000902@redhat.com>	<4D239A5B.9010005@redhat.com>
	<87k4ii4b95.fsf@meyering.net>	<4D25D6D4.5040401@redhat.com>
	<87sjx6yszy.fsf@meyering.net>
In-Reply-To: <87sjx6yszy.fsf@meyering.net>
X-Enigmail-Version: 1.1.2
OpenPGP: url=http://people.redhat.com/eblake/eblake.gpg
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature";
	boundary="------------enigFC4B50616271C44E485B0001"
X-Scanned-By: MIMEDefang 2.67 on 10.5.11.12
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
Cc: bug-gnulib@gnu.org, coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 06 Jan 2011 15:20:08 -0000

This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--------------enigFC4B50616271C44E485B0001
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

On 01/06/2011 08:03 AM, Jim Meyering wrote:
>>> +submodule-checks =3D no-submodule-changes public-submodule-commit
>>
>> If someone complains about these two tests being git-centric, a soluti=
on
>> for a project using maint.mk with a different VCS would be to make
>> submodule-checks defined via ?=3D, so that it can be overridden in cfg=
=2Emk
>> to an empty macro to skip git submodule consistency checks.  But I'm
>> okay using =3D instead of ?=3D and waiting until someone requests that=
 extra
>> flexibility, if you don't want to change it now.
>=20
> Good idea.
> I've also made the test silently skip when there is no .git/ directory,=

> just like no-submodule-changes rule does.
>=20
> One more iteration:

ACK.

--=20
Eric Blake   eblake@redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org


--------------enigFC4B50616271C44E485B0001
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Public key at http://people.redhat.com/eblake/eblake.gpg
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/

iQEcBAEBCAAGBQJNJd2XAAoJEKeha0olJ0NqaLQIAJvxVRYKExwakdRBxV/e2HGA
rz+CvUHpeK53E9OhkLbe4TU1CkNRlrwe0tSIYNs537EqewhJ6tjdEHIWbsJl8VC+
vnmBLJotPC0n+qo/KMo/DbSNO2Pn2MjBhuRqGmnjOeYawvL6sxpBy1VP8HOn0Oma
yrUqDac5ZwA+4tOyI70XOqvikU2ZsO1uMR1v5G5ZbFSWBLOXyn1bh2l5FXWVj770
r14PO1YoafjS0C3ikgAw+hRbBEoE19Y9GaC+iECtlCnNa1qIYLRW6HjYb7/v+eMg
N1a5puKtVfwrqPb/VSFBOastGfLtonJTv5OVsanpLQeUJlIi0h/mwS2J+QYm9Hw=
=S8S1
-----END PGP SIGNATURE-----

--------------enigFC4B50616271C44E485B0001--


From MAILER-DAEMON Thu Jan 06 17:06:24 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PaxyS-0002Tr-0G
	for mharc-coreutils@gnu.org; Thu, 06 Jan 2011 17:06:24 -0500
Received: from [140.186.70.92] (port=60849 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PaxyP-0002SJ-JP
	for coreutils@gnu.org; Thu, 06 Jan 2011 17:06:22 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1PaxyO-0001wI-8j
	for coreutils@gnu.org; Thu, 06 Jan 2011 17:06:21 -0500
Received: from mx1.redhat.com ([209.132.183.28]:16161)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>)
	id 1PaxyN-0001wC-U7; Thu, 06 Jan 2011 17:06:20 -0500
Received: from int-mx01.intmail.prod.int.phx2.redhat.com
	(int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11])
	by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id p06M6F39028489
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK);
	Thu, 6 Jan 2011 17:06:15 -0500
Received: from [10.3.113.114] (ovpn-113-114.phx2.redhat.com [10.3.113.114])
	by int-mx01.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP
	id p06M6FkS014501; Thu, 6 Jan 2011 17:06:15 -0500
Message-ID: <4D263CD6.3010201@redhat.com>
Date: Thu, 06 Jan 2011 15:06:14 -0700
From: Eric Blake <eblake@redhat.com>
Organization: Red Hat
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US;
	rv:1.9.2.13) Gecko/20101209 Fedora/3.1.7-0.35.b3pre.fc14
	Lightning/1.0b3pre Mnenhy/0.8.3 Thunderbird/3.1.7
MIME-Version: 1.0
To: =?UTF-8?B?UMOhZHJhaWcgQnJhZHk=?= <P@draigBrady.com>
References: <AANLkTimypg15p_04D+q_phU6mLsFdOUbk_V_c1Uka3NC@mail.gmail.com>	<20110101225618.GD6935@nb.net.home>	<4D230A31.1090404@draigBrady.com>	<20110104143740.GA2933@nb.net.home>	<4D2336EE.9090603@draigBrady.com>	<20110104155211.GB2933@nb.net.home>	<4D2347DD.5020605@draigBrady.com>	<87ei8rafjj.fsf@meyering.net>	<4D244707.8000109@draigBrady.com>
	<8739p7a935.fsf_-_@meyering.net> <4D245EB0.9040104@draigBrady.com>
In-Reply-To: <4D245EB0.9040104@draigBrady.com>
X-Enigmail-Version: 1.1.2
OpenPGP: url=http://people.redhat.com/eblake/eblake.gpg
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature";
	boundary="------------enigA0C50019AB4DCAA82431C005"
X-Scanned-By: MIMEDefang 2.67 on 10.5.11.11
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
Cc: bug-gnulib <bug-gnulib@gnu.org>, Coreutils <coreutils@gnu.org>
Subject: [coreutils] 
	Re: make ignore_value more generic; deprecate ignore_ptr
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 06 Jan 2011 22:06:22 -0000

This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--------------enigA0C50019AB4DCAA82431C005
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

On 01/05/2011 05:06 AM, P=C3=A1draig Brady wrote:
> Looks good. I'll apply the following to coreutils,
> along with a gnulib update soon.
>=20
> thanks,
> P=C3=A1draig.
>=20
> commit 79138dbb615644c1b2d9ec05afa4e028e8b0ef06
> Author: P=C3=A1draig Brady <P@draigBrady.com>
> Date:   Wed Jan 5 12:01:14 2011 +0000
>=20
>     maint: replace uses of ignore_ptr with ignore_value

This isn't in yet, and it bit me while playing with my fixes to
sc_prohibit_strcmp.  Without it, I get build failures (configuring with
--enable-gcc-warnings) about deprecated ignore_ptr, but with it, I still
get build failures:

chcon.c: In function 'process_file':
chcon.c:229:30: error: cast from function call of type 'struct FTSENT *'
to non-matching type 'long int' [-Wbad-function-cast]

This is on Fedora 14, with gcc 4.5.1.

--=20
Eric Blake   eblake@redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org


--------------enigA0C50019AB4DCAA82431C005
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Public key at http://people.redhat.com/eblake/eblake.gpg
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/

iQEcBAEBCAAGBQJNJjzWAAoJEKeha0olJ0NqsfEH/1oLqD9bRg2s2Z+t0e8T5CNL
CJIwrCPQ6+WdvkX425S6C96YlfqBKn6NdRO0R7O4Zjaoqtin75hHAZLcR8JPzIaI
JpGOYrjiuDwujymDlhjrqowQP4X8wxSGD+1A9BkrSD7YX3qMm6zPdKrWEc509c9S
Dmk1a4Wq/Tlt8XMIPoJFVfJVYY2o5vz47tzU0oRXu6+BdTkj6srCBNg03YYvG5YK
Npbch+YguTcC7BGrK87IeV9F2QpGPIeFwV/el5ficT9Lq167DwH4feGIt9XNYVfZ
JWb5OZIbBI3rTNKXml1gJFQF+m26ULPZpLotNHZ8ODLQQ/v6ltHYmDH11eVvUiw=
=mn8N
-----END PGP SIGNATURE-----

--------------enigA0C50019AB4DCAA82431C005--


From MAILER-DAEMON Thu Jan 06 17:16:08 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1Pay7s-0006rU-BY
	for mharc-coreutils@gnu.org; Thu, 06 Jan 2011 17:16:08 -0500
Received: from [140.186.70.92] (port=40291 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1Pay7q-0006p9-8W
	for coreutils@gnu.org; Thu, 06 Jan 2011 17:16:07 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1Pay7p-0003bI-AC
	for coreutils@gnu.org; Thu, 06 Jan 2011 17:16:06 -0500
Received: from mx1.redhat.com ([209.132.183.28]:32999)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>)
	id 1Pay7m-0003ay-Bo; Thu, 06 Jan 2011 17:16:02 -0500
Received: from int-mx10.intmail.prod.int.phx2.redhat.com
	(int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23])
	by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id p06MFxHO031158
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK);
	Thu, 6 Jan 2011 17:15:59 -0500
Received: from [10.3.113.114] (ovpn-113-114.phx2.redhat.com [10.3.113.114])
	by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP
	id p06MFwA8027908; Thu, 6 Jan 2011 17:15:58 -0500
Message-ID: <4D263F1D.6050505@redhat.com>
Date: Thu, 06 Jan 2011 15:15:57 -0700
From: Eric Blake <eblake@redhat.com>
Organization: Red Hat
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US;
	rv:1.9.2.13) Gecko/20101209 Fedora/3.1.7-0.35.b3pre.fc14
	Lightning/1.0b3pre Mnenhy/0.8.3 Thunderbird/3.1.7
MIME-Version: 1.0
References: <AANLkTimypg15p_04D+q_phU6mLsFdOUbk_V_c1Uka3NC@mail.gmail.com>	<20110101225618.GD6935@nb.net.home>	<4D230A31.1090404@draigBrady.com>	<20110104143740.GA2933@nb.net.home>	<4D2336EE.9090603@draigBrady.com>	<20110104155211.GB2933@nb.net.home>	<4D2347DD.5020605@draigBrady.com>	<87ei8rafjj.fsf@meyering.net>	<4D244707.8000109@draigBrady.com>	<8739p7a935.fsf_-_@meyering.net>
	<4D245EB0.9040104@draigBrady.com> <4D263CD6.3010201@redhat.com>
In-Reply-To: <4D263CD6.3010201@redhat.com>
X-Enigmail-Version: 1.1.2
OpenPGP: url=http://people.redhat.com/eblake/eblake.gpg
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature";
	boundary="------------enig3C699E7A1192577D1D910041"
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
Cc: bug-gnulib <bug-gnulib@gnu.org>,
	=?UTF-8?B?UMOhZHJhaWcgQnJhZHk=?= <P@draigBrady.com>,
	Coreutils <coreutils@gnu.org>
Subject: [coreutils] 
	Re: make ignore_value more generic; deprecate ignore_ptr
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 06 Jan 2011 22:16:07 -0000

This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--------------enig3C699E7A1192577D1D910041
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

On 01/06/2011 03:06 PM, Eric Blake wrote:
> On 01/05/2011 05:06 AM, P=C3=A1draig Brady wrote:
>> Looks good. I'll apply the following to coreutils,
>> along with a gnulib update soon.
>>
>> thanks,
>> P=C3=A1draig.
>>
>> commit 79138dbb615644c1b2d9ec05afa4e028e8b0ef06
>> Author: P=C3=A1draig Brady <P@draigBrady.com>
>> Date:   Wed Jan 5 12:01:14 2011 +0000
>>
>>     maint: replace uses of ignore_ptr with ignore_value
>=20
> This isn't in yet, and it bit me while playing with my fixes to
> sc_prohibit_strcmp.  Without it, I get build failures (configuring with=

> --enable-gcc-warnings) about deprecated ignore_ptr, but with it, I stil=
l
> get build failures:
>=20
> chcon.c: In function 'process_file':
> chcon.c:229:30: error: cast from function call of type 'struct FTSENT *=
'
> to non-matching type 'long int' [-Wbad-function-cast]

Hmm, since the whole point of ignore_value is to work around a gcc
warning, we can use gcc extensions to avoid the warning.  What do you
think about this patch, which once again makes coreutils happy?

diff --git i/ChangeLog w/ChangeLog
index 228b4e8..2db0f0b 100644
--- i/ChangeLog
+++ w/ChangeLog
@@ -1,5 +1,9 @@
 2011-01-06  Eric Blake  <eblake@redhat.com>

+	ignore_value: work around gcc -Wbad-function-cast
+	* lib/ignore-value.h (ignore_value): Provide separate gcc
+	definition.
+
 	maint.mk: improve sc_prohibit_strcmp regex
 	* top/maint.mk (sc_prohibit_strcmp): Detect strcmp()!=3D0, as
 	documented.  Also, detect strcmp((expr),expr) =3D=3D 0.
diff --git i/lib/ignore-value.h w/lib/ignore-value.h
index 8f60b0e..9b73793 100644
--- i/lib/ignore-value.h
+++ w/lib/ignore-value.h
@@ -47,8 +47,13 @@
 #  endif
 # endif

+# if __GNUC__ < 3 || (__GNUC__ =3D=3D 3 && __GNUC_MINOR__ < 1)
+#  define ignore_value(x) ((void) (x))
+# else
 static inline void _ignore_value (intptr_t p) { (void) p; }
-# define ignore_value(x) _ignore_value ((intptr_t) x)
+#  define ignore_value(x) \
+     _ignore_value (({__typeof__(x) __x =3D (x); (intptr_t) __x;}))
+# endif

 /* ignore_value works for both scalars and pointers; deprecate
ignore_ptr.  */
 static inline void ATTRIBUTE_DEPRECATED

--=20
Eric Blake   eblake@redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org


--------------enig3C699E7A1192577D1D910041
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Public key at http://people.redhat.com/eblake/eblake.gpg
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/

iQEcBAEBCAAGBQJNJj8dAAoJEKeha0olJ0NqlZYIAJy7dgg/WbOmj162Msk01vbJ
SO8pFTBLX6HUOVu6sWmBzbZNIVcqBO6UCcnPmFTSDDpQBtdbRktl+gj1KaFF3ADA
KzbHxP/+zsA20TSX9MrV2Ymxj24S9OYPkSbZ1ML/ZhFmgb40ysIsQpf2GOTC0tfY
S1mUrPLlwh8zLnSFl5ScSYFJxlkrrisoBOfxdcfqxw7VXW03nYIDSx3O3kzEm/+8
4obIajByT8Enx3k/51tCqKJE3YyLa0JEw5vzoU0A+Uyv/mLk2IxseOS9JMOIamSu
1VxkkuXRNIYRlp4pFuPEqsh1+M7gvigou3iawqiT6/wu2dhsjFj3ppZBji1tNzY=
=mdBO
-----END PGP SIGNATURE-----

--------------enig3C699E7A1192577D1D910041--


From MAILER-DAEMON Thu Jan 06 17:19:48 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PayBP-0005Gi-TK
	for mharc-coreutils@gnu.org; Thu, 06 Jan 2011 17:19:48 -0500
Received: from [140.186.70.92] (port=41359 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PayBM-0005ED-Eo
	for coreutils@gnu.org; Thu, 06 Jan 2011 17:19:45 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1PayBL-00047E-6G
	for coreutils@gnu.org; Thu, 06 Jan 2011 17:19:44 -0500
Received: from mx1.redhat.com ([209.132.183.28]:36746)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>)
	id 1PayBI-00044H-Tj; Thu, 06 Jan 2011 17:19:41 -0500
Received: from int-mx09.intmail.prod.int.phx2.redhat.com
	(int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22])
	by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id p06MJccq011633
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK);
	Thu, 6 Jan 2011 17:19:38 -0500
Received: from [10.3.113.114] (ovpn-113-114.phx2.redhat.com [10.3.113.114])
	by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP
	id p06MJbWQ003582; Thu, 6 Jan 2011 17:19:37 -0500
Message-ID: <4D263FF9.9070809@redhat.com>
Date: Thu, 06 Jan 2011 15:19:37 -0700
From: Eric Blake <eblake@redhat.com>
Organization: Red Hat
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US;
	rv:1.9.2.13) Gecko/20101209 Fedora/3.1.7-0.35.b3pre.fc14
	Lightning/1.0b3pre Mnenhy/0.8.3 Thunderbird/3.1.7
MIME-Version: 1.0
References: <AANLkTimypg15p_04D+q_phU6mLsFdOUbk_V_c1Uka3NC@mail.gmail.com>	<20110101225618.GD6935@nb.net.home>	<4D230A31.1090404@draigBrady.com>	<20110104143740.GA2933@nb.net.home>	<4D2336EE.9090603@draigBrady.com>	<20110104155211.GB2933@nb.net.home>	<4D2347DD.5020605@draigBrady.com>	<87ei8rafjj.fsf@meyering.net>	<4D244707.8000109@draigBrady.com>	<8739p7a935.fsf_-_@meyering.net>	<4D245EB0.9040104@draigBrady.com>
	<4D263CD6.3010201@redhat.com> <4D263F1D.6050505@redhat.com>
In-Reply-To: <4D263F1D.6050505@redhat.com>
X-Enigmail-Version: 1.1.2
OpenPGP: url=http://people.redhat.com/eblake/eblake.gpg
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature";
	boundary="------------enig460963B86D5A1BF50642F622"
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
Cc: =?UTF-8?B?UMOhZHJhaWcgQnJhZHk=?= <P@draigBrady.com>,
	bug-gnulib <bug-gnulib@gnu.org>, Coreutils <coreutils@gnu.org>
Subject: [coreutils] 
	Re: make ignore_value more generic; deprecate ignore_ptr
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 06 Jan 2011 22:19:45 -0000

This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--------------enig460963B86D5A1BF50642F622
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

On 01/06/2011 03:15 PM, Eric Blake wrote:
>=20
> +# if __GNUC__ < 3 || (__GNUC__ =3D=3D 3 && __GNUC_MINOR__ < 1)
> +#  define ignore_value(x) ((void) (x))
> +# else
>  static inline void _ignore_value (intptr_t p) { (void) p; }
> -# define ignore_value(x) _ignore_value ((intptr_t) x)
> +#  define ignore_value(x) \
> +     _ignore_value (({__typeof__(x) __x =3D (x); (intptr_t) __x;}))
> +# endif

In fact, the simpler:

#  define ignore_value(x) (({__typeof__(x) __x =3D (x);(void)__x;}))

works on coreutils, and has the added perk of silencing _wur on aggregate=
s!

--=20
Eric Blake   eblake@redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org


--------------enig460963B86D5A1BF50642F622
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Public key at http://people.redhat.com/eblake/eblake.gpg
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/

iQEcBAEBCAAGBQJNJj/5AAoJEKeha0olJ0Nq1l0H/0zNDxaR83b0fz9Z3G2Hdzre
/XhqXERxr9G1+n/RgahCVVivc/eONSsxV9bxvtTUxLAyAmX4JVyvdYrVw/p6++8d
Z6SSlFOygEjTO1Orez3pFU7cgWSUZFrUDn5bTu9ohkIYrVKuCuCiqeAaWf28Wnfn
bf+r8Tv+641hL1A+78bmyHjkkU8PWi0asumahKEN0BgRl4di86/J6cBuc5djZzTz
usVgVV1YiY/lIk0yumkKyOR6QQm0/BT8Fdn0mzS9z2FzxqeWbPjsOkh2NAdbYFVQ
hWwW5466XTxQrKgUFOoP2v/ndq+SjaVmhGW++Faz7kxUjGaoLkCJWyX8YK8fPrY=
=Opcs
-----END PGP SIGNATURE-----

--------------enig460963B86D5A1BF50642F622--


From MAILER-DAEMON Thu Jan 06 21:23:57 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1Pb1zh-0000Bg-Oo
	for mharc-coreutils@gnu.org; Thu, 06 Jan 2011 21:23:57 -0500
Received: from [140.186.70.92] (port=58604 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1Pb1zf-0000AV-74
	for coreutils@gnu.org; Thu, 06 Jan 2011 21:23:56 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Pb1ze-0000OQ-5n
	for coreutils@gnu.org; Thu, 06 Jan 2011 21:23:55 -0500
Received: from mail1.slb.deg.dub.stisp.net ([84.203.253.98]:11107)
	by eggs.gnu.org with smtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Pb1zd-0000O6-Rm
	for coreutils@gnu.org; Thu, 06 Jan 2011 21:23:54 -0500
Received: (qmail 30312 invoked from network); 7 Jan 2011 02:23:51 -0000
Received: from unknown (HELO ?192.168.2.25?) (84.203.137.218)
	by mail1.slb.deg.dub.stisp.net with SMTP; 7 Jan 2011 02:23:51 -0000
Message-ID: <4D267935.9000008@draigBrady.com>
Date: Fri, 07 Jan 2011 02:23:49 +0000
From: =?UTF-8?B?UMOhZHJhaWcgQnJhZHk=?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US;
	rv:1.9.1.8) Gecko/20100227 Thunderbird/3.0.3
MIME-Version: 1.0
To: Coreutils <coreutils@gnu.org>
X-Enigmail-Version: 1.0.1
Content-Type: multipart/mixed; boundary="------------020900010504030107020205"
X-detected-operating-system: by eggs.gnu.org: FreeBSD 4.6-4.9
Subject: [coreutils] [PATCH] maint: suppress some clang scan-build warnings
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 07 Jan 2011 02:23:56 -0000

This is a multi-part message in MIME format.
--------------020900010504030107020205
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit



--------------020900010504030107020205
Content-Type: text/x-patch;
 name="clang-warnings.diff"
Content-Transfer-Encoding: 8bit
Content-Disposition: attachment;
 filename="clang-warnings.diff"

commit 2865537cb7fe1f7df64681ad4a0b0f00a2cbf004
Author: PĂˇdraig Brady <P@draigBrady.com>
Date:   Sat Dec 18 23:06:10 2010 +0000

    maint: suppress some clang scan-build warnings
    
    * src/pr.c (char_to_clump): Remove a dead store.
    * src/remove.c (fts_skip_tree): Likewise.
    * src/sort.c (key_warnings): Likewise.
    (sort): Suppress an uninitialized pointer warning.

diff --git a/src/pr.c b/src/pr.c
index 4748604..7e6b13c 100644
--- a/src/pr.c
+++ b/src/pr.c
@@ -2695,7 +2695,7 @@ char_to_clump (char c)
               width = 2;
               chars = 2;
               *s++ = '^';
-              *s++ = c ^ 0100;
+              *s = c ^ 0100;
             }
           else
             {
diff --git a/src/remove.c b/src/remove.c
index a473dab..9d5d730 100644
--- a/src/remove.c
+++ b/src/remove.c
@@ -402,7 +402,7 @@ fts_skip_tree (FTS *fts, FTSENT *ent)
 {
   fts_set (fts, ent, FTS_SKIP);
   /* Ensure that we do not process ENT a second time.  */
-  ent = fts_read (fts);
+  ignore_value (fts_read (fts));
 }
 
 /* Upon unlink failure, or when the user declines to remove ENT, mark
diff --git a/src/sort.c b/src/sort.c
index cbc0c38..fa9ba7e 100644
--- a/src/sort.c
+++ b/src/sort.c
@@ -2341,10 +2341,10 @@ key_warnings (struct keyfield const *gkey, bool gkey_only)
           pn = stpcpy (stpcpy (pn, "-k "), umaxtostr (sword + 1, tmp));
           if (key->eword != SIZE_MAX)
             {
-              po = stpcpy (stpcpy (po, " -"), umaxtostr (eword + 1, tmp));
-              pn = stpcpy (stpcpy (pn, ","),
-                           umaxtostr (eword + 1
-                                      + (key->echar == SIZE_MAX), tmp));
+              stpcpy (stpcpy (po, " -"), umaxtostr (eword + 1, tmp));
+              stpcpy (stpcpy (pn, ","),
+                      umaxtostr (eword + 1
+                                 + (key->echar == SIZE_MAX), tmp));
             }
           error (0, 0, _("obsolescent key `%s' used; consider `%s' instead"),
                  obuf, nbuf);
@@ -3770,6 +3770,7 @@ sort (char *const *files, size_t nfiles, char const *output_file,
       size_t nthreads)
 {
   struct buffer buf;
+  IF_LINT (buf.buf == NULL);
   size_t ntemps = 0;
   bool output_file_created = false;
 

--------------020900010504030107020205--


From MAILER-DAEMON Fri Jan 07 01:29:21 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1Pb5pB-00022Z-6I
	for mharc-coreutils@gnu.org; Fri, 07 Jan 2011 01:29:21 -0500
Received: from [140.186.70.92] (port=54659 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1Pb1mM-0006h5-TY
	for coreutils@gnu.org; Thu, 06 Jan 2011 21:10:12 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <kwzh@gnu.org>) id 1Pb1mK-0006M6-LL
	for coreutils@gnu.org; Thu, 06 Jan 2011 21:10:10 -0500
Received: from fencepost.gnu.org ([140.186.70.10]:52106)
	by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <kwzh@gnu.org>)
	id 1Pb1mK-0006M2-GY
	for coreutils@gnu.org; Thu, 06 Jan 2011 21:10:08 -0500
Received: from kwzh by fencepost.gnu.org with local (Exim 4.69)
	(envelope-from <kwzh@gnu.org>)
	id 1Pb1mG-0004rA-2d; Thu, 06 Jan 2011 21:10:04 -0500
From: kwzh@gnu.org (Karl Heuer)
To: coreutils@gnu.org
Content-Type: multipart/mixed; boundary="nextpart-29834"
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
Message-Id: <E1Pb1mG-0004rA-2d@fencepost.gnu.org>
Date: Thu, 06 Jan 2011 21:10:04 -0500
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
X-Mailman-Approved-At: Fri, 07 Jan 2011 01:29:19 -0500
Subject: [coreutils] [PATCH] coreutils new feature: split --filter
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 07 Jan 2011 02:10:12 -0000

This is a multi-part message in MIME format.

--nextpart-29834
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

For some years now I've been bemoaning the lack of a
`split --filter=COMMAND' feature, which would write to
pipes instead of files, instead of having to split to
a set of files and then run the filter on each of them.
The environment variable FILE is set to the name of the
file that would have been written; it's the command's
responsibility to use that string appropriately.
Sample usage (note proper quoting of $FILE):
$ split --filter 'gzip >$FILE.gz'

The details turned out to be rather trickier than I'd
expected, but I think it's ready to be shared with the
folks in charge of coreutils.  Source patch attached.
I haven't touched the documentation or tests.

--nextpart-29834
Content-Type: text/x-diff; name="split.patch"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="split.patch"

KioqIGNvcmV1dGlscy04Ljkvc3JjL3NwbGl0LmN+CVNhdCBKYW4gIDEgMTM6MTk6MjMgMjAx
MQotLS0gY29yZXV0aWxzLTguOS9zcmMvc3BsaXQuYwlUaHUgSmFuICA2IDE3OjI2OjMwIDIw
MTEKKioqKioqKioqKioqKioqCioqKiAyNSwzMSAqKioqCi0tLSAyNSwzMyAtLS0tCiAgI2lu
Y2x1ZGUgPGFzc2VydC5oPgogICNpbmNsdWRlIDxzdGRpby5oPgogICNpbmNsdWRlIDxnZXRv
cHQuaD4KKyAjaW5jbHVkZSA8c2lnbmFsLmg+CiAgI2luY2x1ZGUgPHN5cy90eXBlcy5oPgor
ICNpbmNsdWRlIDxzeXMvd2FpdC5oPgogIAogICNpbmNsdWRlICJzeXN0ZW0uaCIKICAjaW5j
bHVkZSAiZXJyb3IuaCIKKioqKioqKioqKioqKioqCioqKiAzNSw0MCAqKioqCi0tLSAzNyw0
MyAtLS0tCiAgI2luY2x1ZGUgImZ1bGwtd3JpdGUuaCIKICAjaW5jbHVkZSAicXVvdGUuaCIK
ICAjaW5jbHVkZSAic2FmZS1yZWFkLmgiCisgI2luY2x1ZGUgInNpZzJzdHIuaCIKICAjaW5j
bHVkZSAieGZyZW9wZW4uaCIKICAjaW5jbHVkZSAieHN0cnRvbC5oIgogIAoqKioqKioqKioq
KioqKioKKioqIDQ1LDUwICoqKioKLS0tIDQ4LDY4IC0tLS0KICAgIHByb3Blcl9uYW1lX3V0
ZjggKCJUb3Jiam9ybiBHcmFubHVuZCIsICJUb3JialwzMDNcMjY2cm4gR3Jhbmx1bmQiKSwg
XAogICAgcHJvcGVyX25hbWUgKCJSaWNoYXJkIE0uIFN0YWxsbWFuIikKICAKKyAvKiBTaGVs
bCBjb21tYW5kIHRvIGZpbHRlciB0aHJvdWdoLCBpbnN0ZWFkIG9mIGNyZWF0aW5nIGZpbGVz
LiAgKi8KKyBzdGF0aWMgY2hhciBjb25zdCAqZmlsdGVyX2NvbW1hbmQ7CisgCisgLyogUHJv
Y2VzcyBJRCBvZiB0aGUgZmlsdGVyLiAgKi8KKyBzdGF0aWMgaW50IGZpbHRlcl9waWQ7Cisg
CisgLyogQXJyYXkgb2Ygb3BlbiBwaXBlcy4gICovCisgc3RhdGljIGludCAqb3Blbl9waXBl
czsKKyBzdGF0aWMgaW50IG9wZW5fcGlwZXNfc2l6ZTsKKyBzdGF0aWMgaW50IG9wZW5fcGlw
ZXNfbGVuOworIAorIC8qIEJsb2NrZWQgc2lnbmFscy4gICovCisgc3RhdGljIHNpZ3NldF90
IG9sZGJsb2NrZWQ7Cisgc3RhdGljIHNpZ3NldF90IG5ld2Jsb2NrZWQ7CisgCiAgLyogQmFz
ZSBuYW1lIG9mIG91dHB1dCBmaWxlcy4gICovCiAgc3RhdGljIGNoYXIgY29uc3QgKm91dGJh
c2U7CiAgCioqKioqKioqKioqKioqKgoqKiogMTAzLDEwOCAqKioqCi0tLSAxMjEsMTI3IC0t
LS0KICAgIHsidW5idWZmZXJlZCIsIG5vX2FyZ3VtZW50LCBOVUxMLCAndSd9LAogICAgeyJz
dWZmaXgtbGVuZ3RoIiwgcmVxdWlyZWRfYXJndW1lbnQsIE5VTEwsICdhJ30sCiAgICB7Im51
bWVyaWMtc3VmZml4ZXMiLCBub19hcmd1bWVudCwgTlVMTCwgJ2QnfSwKKyAgIHsiZmlsdGVy
IiwgcmVxdWlyZWRfYXJndW1lbnQsIE5VTEwsICdmJ30sCiAgICB7InZlcmJvc2UiLCBub19h
cmd1bWVudCwgTlVMTCwgVkVSQk9TRV9PUFRJT059LAogICAgeyItaW8tYmxrc2l6ZSIsIHJl
cXVpcmVkX2FyZ3VtZW50LCBOVUxMLAogICAgICBJT19CTEtTSVpFX09QVElPTn0sIC8qIGRv
IG5vdCBkb2N1bWVudCAqLwoqKioqKioqKioqKioqKioKKioqIDE3MCwxNzUgKioqKgotLS0g
MTg5LDE5NSAtLS0tCiAgICAtQywgLS1saW5lLWJ5dGVzPVNJWkUgICBwdXQgYXQgbW9zdCBT
SVpFIGJ5dGVzIG9mIGxpbmVzIHBlciBvdXRwdXQgZmlsZVxuXAogICAgLWQsIC0tbnVtZXJp
Yy1zdWZmaXhlcyAgdXNlIG51bWVyaWMgc3VmZml4ZXMgaW5zdGVhZCBvZiBhbHBoYWJldGlj
XG5cCiAgICAtZSwgLS1lbGlkZS1lbXB0eS1maWxlcyAgZG8gbm90IGdlbmVyYXRlIGVtcHR5
IG91dHB1dCBmaWxlcyB3aXRoIGAtbidcblwKKyAgIC1mLCAtLWZpbHRlcj1DT01NQU5EICAg
IHdyaXRlIHRvIHNoZWxsIENPTU1BTkQ7IGZpbGUgbmFtZSBpcyAkRklMRVxuXAogICAgLWws
IC0tbGluZXM9TlVNQkVSICAgICAgcHV0IE5VTUJFUiBsaW5lcyBwZXIgb3V0cHV0IGZpbGVc
blwKICAgIC1uLCAtLW51bWJlcj1DSFVOS1MgICAgIGdlbmVyYXRlIENIVU5LUyBvdXRwdXQg
ZmlsZXMuICBTZWUgYmVsb3dcblwKICAgIC11LCAtLXVuYnVmZmVyZWQgICAgICAgIGltbWVk
aWF0ZWx5IGNvcHkgaW5wdXQgdG8gb3V0cHV0IHdpdGggYC1uIHIvLi4uJ1xuXAoqKioqKioq
KioqKioqKioKKioqIDI1NiwyNjUgKioqKgogIHN0YXRpYyBpbnQKICBjcmVhdGUgKGNvbnN0
IGNoYXIqIG5hbWUpCiAgewohICAgaWYgKHZlcmJvc2UpCiEgICAgIGZwcmludGYgKHN0ZG91
dCwgXygiY3JlYXRpbmcgZmlsZSAlc1xuIiksIHF1b3RlIChuYW1lKSk7CiEgICByZXR1cm4g
b3BlbiAobmFtZSwgT19XUk9OTFkgfCBPX0NSRUFUIHwgT19UUlVOQyB8IE9fQklOQVJZLAoh
ICAgICAgICAgICAgICAgIChTX0lSVVNSIHwgU19JV1VTUiB8IFNfSVJHUlAgfCBTX0lXR1JQ
IHwgU19JUk9USCB8IFNfSVdPVEgpKTsKICB9CiAgCiAgLyogV3JpdGUgQllURVMgYnl0ZXMg
YXQgQlAgdG8gYW4gb3V0cHV0IGZpbGUuCi0tLSAyNzYsMzkxIC0tLS0KICBzdGF0aWMgaW50
CiAgY3JlYXRlIChjb25zdCBjaGFyKiBuYW1lKQogIHsKISAgIGlmICghZmlsdGVyX2NvbW1h
bmQpCiEgICAgIHsKISAgICAgICBpZiAodmVyYm9zZSkKISAgICAgICAgIGZwcmludGYgKHN0
ZG91dCwgXygiY3JlYXRpbmcgZmlsZSAlc1xuIiksIHF1b3RlIChuYW1lKSk7CiEgICAgICAg
cmV0dXJuIG9wZW4gKG5hbWUsIE9fV1JPTkxZIHwgT19DUkVBVCB8IE9fVFJVTkMgfCBPX0JJ
TkFSWSwKISAgICAgICAgICAgICAgICAgICAgKFNfSVJVU1IgfCBTX0lXVVNSIHwgU19JUkdS
UCB8IFNfSVdHUlAgfCBTX0lST1RIIHwgU19JV09USCkpOwohICAgICB9CiEgICBlbHNlCiEg
ICAgIHsKISAgICAgICBpbnQgZmRfcGFpclsyXTsKISAgICAgICBwaWRfdCBjaGlsZF9waWQ7
CiEgICAgICAgY2hhciBjb25zdCAqc2hlbGxfcHJvZyA9IGdldGVudiAoIlNIRUxMIik7CiEg
ICAgICAgaWYgKHNoZWxsX3Byb2cgPT0gTlVMTCkKISAgICAgICAgIHNoZWxsX3Byb2cgPSAi
L2Jpbi9zaCI7CiEgICAgICAgaWYgKHNldGVudiAoIkZJTEUiLCBuYW1lLCAxKSAhPSAwKQoh
ICAgICAgICAgZXJyb3IgKEVYSVRfRkFJTFVSRSwgZXJybm8sICJjYW5ub3Qgc2V0IGVudmly
b25tZW50IHZhcmlhYmxlIik7CiEgICAgICAgaWYgKHZlcmJvc2UpCiEgICAgICAgICBmcHJp
bnRmIChzdGRvdXQsIF8oImV4ZWN1dGluZyB3aXRoIEZJTEU9JXNcbiIpLCBxdW90ZSAobmFt
ZSkpOwohICAgICAgIGlmIChwaXBlIChmZF9wYWlyKSAhPSAwKQohICAgICAgICAgZXJyb3Ig
KEVYSVRfRkFJTFVSRSwgZXJybm8sIF8oImNhbm5vdCBjcmVhdGUgcGlwZSIpKTsKISAgICAg
ICBjaGlsZF9waWQgPSBmb3JrICgpOwohICAgICAgIGlmIChjaGlsZF9waWQgPT0gMCkKISAg
ICAgICAgIHsKISAgICAgICAgICAgLyogVGhpcyBpcyB0aGUgY2hpbGQgcHJvY2Vzcy4gIElm
IGFuIGVycm9yIG9jY3VycyBoZXJlLCB0aGUKISAgICAgICAgICAgICAgcGFyZW50IHdpbGwg
ZXZlbnR1YWxseSBsZWFybiBhYm91dCBpdCBhZnRlciBkb2luZyBhIHdhaXQsCiEgICAgICAg
ICAgICAgIGF0IHdoaWNoIHRpbWUgaXQgd2lsbCBlbWl0IGl0cyBvd24gZXJyb3IgbWVzc2Fn
ZS4gICovCiEgICAgICAgICAgIGludCBqOwohICAgICAgICAgICAvKiBXZSBoYXZlIHRvIGNs
b3NlIGFueSBwaXBlcyB0aGF0IHdlcmUgb3BlbmVkIGR1cmluZyBhbgohICAgICAgICAgICAg
ICBlYXJsaWVyIGNhbGwsIG90aGVyd2lzZSB0aGlzIHByb2Nlc3Mgd2lsbCBiZSBob2xkaW5n
IGEKISAgICAgICAgICAgICAgd3JpdGUtcGlwZSB0aGF0IHdpbGwgcHJldmVudCB0aGUgZWFy
bGllciBwcm9jZXNzIGZyb20KISAgICAgICAgICAgICAgcmVhZGluZyBhbiBFT0Ygb24gdGhl
IGNvcnJlc3BvbmRpbmcgcmVhZC1waXBlLiAgKi8KISAgICAgICAgICAgZm9yIChqID0gMDsg
aiA8IG9wZW5fcGlwZXNfbGVuOyArK2opCiEgICAgICAgICAgICAgaWYgKGNsb3NlIChvcGVu
X3BpcGVzW2pdKSAhPSAwKQohICAgICAgICAgICAgICAgZXJyb3IgKEVYSVRfRkFJTFVSRSwg
ZXJybm8sIF8oImNsb3NpbmcgcHJpb3IgcGlwZSIpKTsKISAgICAgICAgICAgaWYgKGNsb3Nl
IChmZF9wYWlyWzFdKSkKISAgICAgICAgICAgICBlcnJvciAoRVhJVF9GQUlMVVJFLCBlcnJu
bywgXygiY2xvc2luZyBvdXRwdXQgcGlwZSIpKTsKISAgICAgICAgICAgaWYgKGZkX3BhaXJb
MF0gIT0gU1RESU5fRklMRU5PKQohICAgICAgICAgICAgIHsKISAgICAgICAgICAgICAgIGlm
IChkdXAyIChmZF9wYWlyWzBdLCBTVERJTl9GSUxFTk8pICE9IFNURElOX0ZJTEVOTykKISAg
ICAgICAgICAgICAgICAgZXJyb3IgKEVYSVRfRkFJTFVSRSwgZXJybm8sIF8oIm1vdmluZyBp
bnB1dCBwaXBlIikpOwohICAgICAgICAgICAgICAgaWYgKGNsb3NlIChmZF9wYWlyWzBdKSAh
PSAwKQohICAgICAgICAgICAgICAgICBlcnJvciAoRVhJVF9GQUlMVVJFLCBlcnJubywgXygi
Y2xvc2luZyBpbnB1dCBwaXBlIikpOwohICAgICAgICAgICAgIH0KISAgICAgICAgICAgc2ln
cHJvY21hc2sgKFNJR19TRVRNQVNLLCAmb2xkYmxvY2tlZCwgTlVMTCk7CiEgICAgICAgICAg
IGV4ZWNsIChzaGVsbF9wcm9nLCBsYXN0X2NvbXBvbmVudCAoc2hlbGxfcHJvZyksICItYyIs
CiEgICAgICAgICAgICAgICAgICBmaWx0ZXJfY29tbWFuZCwgKGNoYXIgKikgTlVMTCk7CiEg
ICAgICAgICAgIGVycm9yIChFWElUX0ZBSUxVUkUsIGVycm5vLCAiJXMiLCBzaGVsbF9wcm9n
KTsKISAgICAgICAgIH0KISAgICAgICBpZiAoY2hpbGRfcGlkID09IC0xKQohICAgICAgICAg
ZXJyb3IgKEVYSVRfRkFJTFVSRSwgZXJybm8sIF8oImNhbm5vdCBzcGF3biBuZXcgcHJvY2Vz
cyIpKTsKISAgICAgICBpZiAoY2xvc2UgKGZkX3BhaXJbMF0pICE9IDApCiEgICAgICAgICBl
cnJvciAoRVhJVF9GQUlMVVJFLCBlcnJubywgXygiY2xvc2luZyBpbnB1dCBwaXBlIikpOwoh
ICAgICAgIGZpbHRlcl9waWQgPSBjaGlsZF9waWQ7CiEgICAgICAgaWYgKG9wZW5fcGlwZXNf
bGVuID09IG9wZW5fcGlwZXNfc2l6ZSkKISAgICAgICAgIG9wZW5fcGlwZXMgPSAoaW50ICop
IHgybnJlYWxsb2MgKG9wZW5fcGlwZXMsICZvcGVuX3BpcGVzX3NpemUsCiEgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBzaXplb2YgKm9wZW5fcGlwZXMpOwoh
ICAgICAgIG9wZW5fcGlwZXNbb3Blbl9waXBlc19sZW4rK10gPSBmZF9wYWlyWzFdOwohICAg
ICAgIHJldHVybiBmZF9wYWlyWzFdOwohICAgICB9CiEgfQohIAohIC8qIENsb3NlIHRoZSBv
dXRwdXQgZmlsZSwgYW5kIGRvIGFueSBhc3NvY2lhdGVkIGNsZWFudXAuCiEgICAgSWYgRlAg
YW5kIEZEIGFyZSBib3RoIHNwZWNpZmllZCwgdGhleSByZWZlciB0byB0aGUgc2FtZSBvcGVu
IGZpbGU7CiEgICAgaW4gdGhpcyBjYXNlIEZQIGlzIGNsb3NlZCwgYnV0IEZEIGlzIHN0aWxs
IHVzZWQgaW4gY2xlYW51cC4gICovCiEgc3RhdGljIHZvaWQKISBjbG9zZW91dCAoRklMRSAq
ZnAsIGludCBmZCwgcGlkX3QgcGlkLCBjaGFyIGNvbnN0ICpuYW1lKQohIHsKISAgIGlmIChm
cCAhPSBOVUxMICYmIGZjbG9zZSAoZnApICE9IDAgJiYgZXJybm8gIT0gRVBJUEUpCiEgICAg
IGVycm9yIChFWElUX0ZBSUxVUkUsIGVycm5vLCAiJXMiLCBuYW1lKTsKISAgIGlmIChmZCA+
PSAwKQohICAgICB7CiEgICAgICAgaWYgKGZwID09IE5VTEwgJiYgY2xvc2UgKGZkKSA8IDAp
CiEgICAgICAgICBlcnJvciAoRVhJVF9GQUlMVVJFLCBlcnJubywgIiVzIiwgbmFtZSk7CiEg
ICAgICAgaW50IGo7CiEgICAgICAgZm9yIChqID0gMDsgaiA8IG9wZW5fcGlwZXNfbGVuOyAr
K2opCiEgICAgICAgICBpZiAob3Blbl9waXBlc1tqXSA9PSBmZCkgewohICAgICAgICAgICBv
cGVuX3BpcGVzW2pdID0gb3Blbl9waXBlc1stLW9wZW5fcGlwZXNfbGVuXTsKISAgICAgICAg
ICAgYnJlYWs7CiEgICAgICAgICB9CiEgICAgIH0KISAgIGlmIChwaWQgPiAwKQohICAgICB7
CiEgICAgICAgaW50IHdzdGF0dXMgPSAwOwohICAgICAgIGlmICh3YWl0cGlkIChwaWQsICZ3
c3RhdHVzLCAwKSA9PSAtMSAmJiBlcnJubyAhPSBFQ0hJTEQpCiEgICAgICAgICBlcnJvciAo
RVhJVF9GQUlMVVJFLCBlcnJubywgXygid2FpdGluZyBmb3IgY2hpbGQgcHJvY2VzcyIpKTsK
ISAgICAgICBpZiAoV0lGU0lHTkFMRUQgKHdzdGF0dXMpKQohICAgICAgICAgewohICAgICAg
ICAgICBpbnQgc2lnID0gV1RFUk1TSUcgKHdzdGF0dXMpOwohICAgICAgICAgICBpZiAoc2ln
ICE9IFNJR1BJUEUpCiEgICAgICAgICAgICAgewohICAgICAgICAgICAgICAgY2hhciBzaWdu
YW1lW1NJRzJTVFJfTUFYKzFdOwohICAgICAgICAgICAgICAgaWYgKHNpZzJzdHIgKHNpZywg
c2lnbmFtZSkgIT0gMCkKISAgICAgICAgICAgICAgICAgc3ByaW50ZihzaWduYW1lLCAiJWQi
LCBzaWcpOwohICAgICAgICAgICAgICAgZXJyb3IgKHNpZyArIDEyOCwgMCwKISAgICAgICAg
ICAgICAgICAgICAgICBfKCJ3aXRoIEZJTEU9JXMsIHNpZ25hbCAlcyAoJXMpIGZyb20gY29t
bWFuZDogJXMiKSwKISAgICAgICAgICAgICAgICAgICAgICBuYW1lLCBzaWduYW1lLCBzdHJz
aWduYWwoc2lnKSwgZmlsdGVyX2NvbW1hbmQpOwohICAgICAgICAgICAgIH0KISAgICAgICAg
IH0KISAgICAgICBlbHNlIGlmIChXSUZFWElURUQgKHdzdGF0dXMpKQohICAgICAgICAgewoh
ICAgICAgICAgICBpbnQgZXggPSBXRVhJVFNUQVRVUyAod3N0YXR1cyk7CiEgICAgICAgICAg
IGlmIChleCAhPSAwKQohICAgICAgICAgICAgIGVycm9yIChleCwgMCwgXygid2l0aCBGSUxF
PSVzLCBleGl0ICVkIGZyb20gY29tbWFuZDogJXMiKSwKISAgICAgICAgICAgICAgICAgICAg
bmFtZSwgZXgsIGZpbHRlcl9jb21tYW5kKTsKISAgICAgICAgIH0KISAgICAgICBlbHNlCiEg
ICAgICAgICAvKiBzaG91bGRuJ3QgaGFwcGVuLiAgKi8KISAgICAgICAgIGVycm9yIChFWElU
X0ZBSUxVUkUsIDAsCiEgICAgICAgICAgICAgICAgXygidW5rbm93biBzdGF0dXMgZnJvbSBj
b21tYW5kICgweCVYKSIpLCB3c3RhdHVzKTsKISAgICAgfQogIH0KICAKICAvKiBXcml0ZSBC
WVRFUyBieXRlcyBhdCBCUCB0byBhbiBvdXRwdXQgZmlsZS4KKioqKioqKioqKioqKioqCioq
KiAyNzMsMjg1ICoqKioKICAgICAgewogICAgICAgIGlmICghYnAgJiYgYnl0ZXMgPT0gMCAm
JiBlbGlkZV9lbXB0eV9maWxlcykKICAgICAgICAgIHJldHVybjsKISAgICAgICBpZiAob3V0
cHV0X2Rlc2MgPj0gMCAmJiBjbG9zZSAob3V0cHV0X2Rlc2MpIDwgMCkKISAgICAgICAgIGVy
cm9yIChFWElUX0ZBSUxVUkUsIGVycm5vLCAiJXMiLCBvdXRmaWxlKTsKICAgICAgICBuZXh0
X2ZpbGVfbmFtZSAoKTsKICAgICAgICBpZiAoKG91dHB1dF9kZXNjID0gY3JlYXRlIChvdXRm
aWxlKSkgPCAwKQogICAgICAgICAgZXJyb3IgKEVYSVRfRkFJTFVSRSwgZXJybm8sICIlcyIs
IG91dGZpbGUpOwogICAgICB9CiEgICBpZiAoZnVsbF93cml0ZSAob3V0cHV0X2Rlc2MsIGJw
LCBieXRlcykgIT0gYnl0ZXMpCiAgICAgIGVycm9yIChFWElUX0ZBSUxVUkUsIGVycm5vLCAi
JXMiLCBvdXRmaWxlKTsKICB9CiAgCi0tLSAzOTksNDEwIC0tLS0KICAgICAgewogICAgICAg
IGlmICghYnAgJiYgYnl0ZXMgPT0gMCAmJiBlbGlkZV9lbXB0eV9maWxlcykKICAgICAgICAg
IHJldHVybjsKISAgICAgICBjbG9zZW91dCAoTlVMTCwgb3V0cHV0X2Rlc2MsIGZpbHRlcl9w
aWQsIG91dGZpbGUpOwogICAgICAgIG5leHRfZmlsZV9uYW1lICgpOwogICAgICAgIGlmICgo
b3V0cHV0X2Rlc2MgPSBjcmVhdGUgKG91dGZpbGUpKSA8IDApCiAgICAgICAgICBlcnJvciAo
RVhJVF9GQUlMVVJFLCBlcnJubywgIiVzIiwgb3V0ZmlsZSk7CiAgICAgIH0KISAgIGlmIChm
dWxsX3dyaXRlIChvdXRwdXRfZGVzYywgYnAsIGJ5dGVzKSAhPSBieXRlcyAmJiBlcnJubyAh
PSBFUElQRSkKICAgICAgZXJyb3IgKEVYSVRfRkFJTFVSRSwgZXJybm8sICIlcyIsIG91dGZp
bGUpOwogIH0KICAKKioqKioqKioqKioqKioqCioqKiA1MDEsNTA3ICoqKioKICAgICAgICAg
ICAgICAgIC8qIFdlIGRvbid0IHVzZSB0aGUgc3Rkb3V0IGJ1ZmZlciBoZXJlIHNpbmNlIHdl
J3JlIHdyaXRpbmcKICAgICAgICAgICAgICAgICAgIGxhcmdlIGNodW5rcyBmcm9tIGFuIGV4
aXN0aW5nIGZpbGUsIHNvIGl0J3MgbW9yZSBlZmZpY2llbnQKICAgICAgICAgICAgICAgICAg
IHRvIHdyaXRlIG91dCBkaXJlY3RseS4gICovCiEgICAgICAgICAgICAgICBpZiAoZnVsbF93
cml0ZSAoU1RET1VUX0ZJTEVOTywgYnAsIHRvX3dyaXRlKSAhPSB0b193cml0ZSkKICAgICAg
ICAgICAgICAgICAgZXJyb3IgKEVYSVRfRkFJTFVSRSwgZXJybm8sICIlcyIsIF8oIndyaXRl
IGVycm9yIikpOwogICAgICAgICAgICAgIH0KICAgICAgICAgICAgZWxzZQotLS0gNjI2LDYz
MyAtLS0tCiAgICAgICAgICAgICAgICAvKiBXZSBkb24ndCB1c2UgdGhlIHN0ZG91dCBidWZm
ZXIgaGVyZSBzaW5jZSB3ZSdyZSB3cml0aW5nCiAgICAgICAgICAgICAgICAgICBsYXJnZSBj
aHVua3MgZnJvbSBhbiBleGlzdGluZyBmaWxlLCBzbyBpdCdzIG1vcmUgZWZmaWNpZW50CiAg
ICAgICAgICAgICAgICAgICB0byB3cml0ZSBvdXQgZGlyZWN0bHkuICAqLwohICAgICAgICAg
ICAgICAgaWYgKGZ1bGxfd3JpdGUgKFNURE9VVF9GSUxFTk8sIGJwLCB0b193cml0ZSkgIT0g
dG9fd3JpdGUKISAgICAgICAgICAgICAgICAgICAmJiBlcnJubyAhPSBFUElQRSkKICAgICAg
ICAgICAgICAgICAgZXJyb3IgKEVYSVRfRkFJTFVSRSwgZXJybm8sICIlcyIsIF8oIndyaXRl
IGVycm9yIikpOwogICAgICAgICAgICAgIH0KICAgICAgICAgICAgZWxzZQoqKioqKioqKioq
KioqKioKKioqIDU2NCw1NzAgKioqKgogICAgICAgICAgZXJyb3IgKEVYSVRfRkFJTFVSRSwg
ZXJybm8sICIlcyIsIGluZmlsZSk7CiAgICAgICAgZWxzZSBpZiAobl9yZWFkID09IDApCiAg
ICAgICAgICBicmVhazsgLyogZW9mLiAgKi8KISAgICAgICBpZiAoZnVsbF93cml0ZSAoU1RE
T1VUX0ZJTEVOTywgYnVmLCBuX3JlYWQpICE9IG5fcmVhZCkKICAgICAgICAgIGVycm9yIChF
WElUX0ZBSUxVUkUsIGVycm5vLCAiJXMiLCBxdW90ZSAoIi0iKSk7CiAgICAgICAgc3RhcnQg
Kz0gbl9yZWFkOwogICAgICB9Ci0tLSA2OTAsNjk2IC0tLS0KICAgICAgICAgIGVycm9yIChF
WElUX0ZBSUxVUkUsIGVycm5vLCAiJXMiLCBpbmZpbGUpOwogICAgICAgIGVsc2UgaWYgKG5f
cmVhZCA9PSAwKQogICAgICAgICAgYnJlYWs7IC8qIGVvZi4gICovCiEgICAgICAgaWYgKGZ1
bGxfd3JpdGUgKFNURE9VVF9GSUxFTk8sIGJ1Ziwgbl9yZWFkKSAhPSBuX3JlYWQgJiYgZXJy
bm8gIT0gRVBJUEUpCiAgICAgICAgICBlcnJvciAoRVhJVF9GQUlMVVJFLCBlcnJubywgIiVz
IiwgcXVvdGUgKCItIikpOwogICAgICAgIHN0YXJ0ICs9IG5fcmVhZDsKICAgICAgfQoqKioq
KioqKioqKioqKioKKioqIDU3NSw1ODAgKioqKgotLS0gNzAxLDcwNyAtLS0tCiAgICBjaGFy
ICpvZl9uYW1lOwogICAgaW50IG9mZDsKICAgIEZJTEUqIG9maWxlOworICAgaW50IG9waWQ7
CiAgfSBvZl90OwogIAogIGVudW0KKioqKioqKioqKioqKioqCioqKiA2MzcsNjUwICoqKioK
ICAgICAgICAgICAgICAgICAgZXJyb3IgKEVYSVRfRkFJTFVSRSwgZXJybm8sICIlcyIsIGZp
bGVzW2lfY2hlY2tdLm9mX25hbWUpOwogICAgICAgICAgICAgIH0KICAKISAgICAgICAgICAg
aWYgKGZjbG9zZSAoZmlsZXNbaV9yZW9wZW5dLm9maWxlKSAhPSAwKQogICAgICAgICAgICAg
IGVycm9yIChFWElUX0ZBSUxVUkUsIGVycm5vLCAiJXMiLCBmaWxlc1tpX3Jlb3Blbl0ub2Zf
bmFtZSk7CiAgICAgICAgICAgIGZpbGVzW2lfcmVvcGVuXS5vZmQgPSBPRkRfQVBQRU5EOwog
ICAgICAgICAgfQogIAogICAgICAgIGZpbGVzW2lfY2hlY2tdLm9mZCA9IGZkOwogICAgICAg
IGlmICghKGZpbGVzW2lfY2hlY2tdLm9maWxlID0gZmRvcGVuIChmZCwgImEiKSkpCiAgICAg
ICAgICBlcnJvciAoRVhJVF9GQUlMVVJFLCBlcnJubywgIiVzIiwgZmlsZXNbaV9jaGVja10u
b2ZfbmFtZSk7CiAgICAgIH0KICAKICAgIHJldHVybiBmaWxlX2xpbWl0OwotLS0gNzY0LDc4
MCAtLS0tCiAgICAgICAgICAgICAgICAgIGVycm9yIChFWElUX0ZBSUxVUkUsIGVycm5vLCAi
JXMiLCBmaWxlc1tpX2NoZWNrXS5vZl9uYW1lKTsKICAgICAgICAgICAgICB9CiAgCiEgICAg
ICAgICAgIGlmIChmY2xvc2UgKGZpbGVzW2lfcmVvcGVuXS5vZmlsZSkgIT0gMCAmJiBlcnJu
byAhPSBFUElQRSkKICAgICAgICAgICAgICBlcnJvciAoRVhJVF9GQUlMVVJFLCBlcnJubywg
IiVzIiwgZmlsZXNbaV9yZW9wZW5dLm9mX25hbWUpOworICAgICAgICAgICBmaWxlc1tpX3Jl
b3Blbl0ub2ZpbGUgPSBOVUxMOwogICAgICAgICAgICBmaWxlc1tpX3Jlb3Blbl0ub2ZkID0g
T0ZEX0FQUEVORDsKICAgICAgICAgIH0KICAKICAgICAgICBmaWxlc1tpX2NoZWNrXS5vZmQg
PSBmZDsKICAgICAgICBpZiAoIShmaWxlc1tpX2NoZWNrXS5vZmlsZSA9IGZkb3BlbiAoZmQs
ICJhIikpKQogICAgICAgICAgZXJyb3IgKEVYSVRfRkFJTFVSRSwgZXJybm8sICIlcyIsIGZp
bGVzW2lfY2hlY2tdLm9mX25hbWUpOworICAgICAgIGZpbGVzW2lfY2hlY2tdLm9waWQgPSBm
aWx0ZXJfcGlkOworICAgICAgIGZpbHRlcl9waWQgPSAwOwogICAgICB9CiAgCiAgICByZXR1
cm4gZmlsZV9saW1pdDsKKioqKioqKioqKioqKioqCioqKiA2NTgsNjYzICoqKioKLS0tIDc4
OCw3OTQgLS0tLQogIHN0YXRpYyB2b2lkCiAgbGluZXNfcnIgKHVpbnRtYXhfdCBrLCB1aW50
bWF4X3QgbiwgY2hhciAqYnVmLCBzaXplX3QgYnVmc2l6ZSkKICB7CisgICBib29sIHdyYXBw
ZWQgPSBmYWxzZTsKICAgIGJvb2wgZmlsZV9saW1pdDsKICAgIHNpemVfdCBpX2ZpbGU7CiAg
ICBvZl90ICpmaWxlczsKKioqKioqKioqKioqKioqCioqKiA2NzgsNjgzICoqKioKLS0tIDgw
OSw4MTUgLS0tLQogICAgICAgICAgICBmaWxlc1tpX2ZpbGVdLm9mX25hbWUgPSB4c3RyZHVw
IChvdXRmaWxlKTsKICAgICAgICAgICAgZmlsZXNbaV9maWxlXS5vZmQgPSBPRkRfTkVXOwog
ICAgICAgICAgICBmaWxlc1tpX2ZpbGVdLm9maWxlID0gTlVMTDsKKyAgICAgICAgICAgZmls
ZXNbaV9maWxlXS5vcGlkID0gMDsKICAgICAgICAgIH0KICAgICAgICBpX2ZpbGUgPSAwOwog
ICAgICAgIGZpbGVfbGltaXQgPSBmYWxzZTsKKioqKioqKioqKioqKioqCioqKiA3MTUsNzI0
ICoqKioKICAgICAgICAgICAgICB7CiAgICAgICAgICAgICAgICBpZiAobGluZV9ubyA9PSBr
ICYmIHVuYnVmZmVyZWQpCiAgICAgICAgICAgICAgICAgIHsKISAgICAgICAgICAgICAgICAg
ICBpZiAoZnVsbF93cml0ZSAoU1RET1VUX0ZJTEVOTywgYnAsIHRvX3dyaXRlKSAhPSB0b193
cml0ZSkKICAgICAgICAgICAgICAgICAgICAgIGVycm9yIChFWElUX0ZBSUxVUkUsIGVycm5v
LCAiJXMiLCBfKCJ3cml0ZSBlcnJvciIpKTsKICAgICAgICAgICAgICAgICAgfQohICAgICAg
ICAgICAgICAgZWxzZSBpZiAobGluZV9ubyA9PSBrICYmIGZ3cml0ZSAoYnAsIHRvX3dyaXRl
LCAxLCBzdGRvdXQpICE9IDEpCiAgICAgICAgICAgICAgICAgIHsKICAgICAgICAgICAgICAg
ICAgICBjbGVhcmVyciAoc3Rkb3V0KTsgLyogVG8gc2lsZW5jZSBjbG9zZV9zdGRvdXQoKS4g
ICovCiAgICAgICAgICAgICAgICAgICAgZXJyb3IgKEVYSVRfRkFJTFVSRSwgZXJybm8sICIl
cyIsIF8oIndyaXRlIGVycm9yIikpOwotLS0gODQ3LDg1OCAtLS0tCiAgICAgICAgICAgICAg
ewogICAgICAgICAgICAgICAgaWYgKGxpbmVfbm8gPT0gayAmJiB1bmJ1ZmZlcmVkKQogICAg
ICAgICAgICAgICAgICB7CiEgICAgICAgICAgICAgICAgICAgaWYgKGZ1bGxfd3JpdGUgKFNU
RE9VVF9GSUxFTk8sIGJwLCB0b193cml0ZSkgIT0gdG9fd3JpdGUKISAgICAgICAgICAgICAg
ICAgICAgICAgJiYgZXJybm8gIT0gRVBJUEUpCiAgICAgICAgICAgICAgICAgICAgICBlcnJv
ciAoRVhJVF9GQUlMVVJFLCBlcnJubywgIiVzIiwgXygid3JpdGUgZXJyb3IiKSk7CiAgICAg
ICAgICAgICAgICAgIH0KISAgICAgICAgICAgICAgIGVsc2UgaWYgKGxpbmVfbm8gPT0gayAm
JiBmd3JpdGUgKGJwLCB0b193cml0ZSwgMSwgc3Rkb3V0KSAhPSAxCiEgICAgICAgICAgICAg
ICAgICAgICAgICAmJiBlcnJubyAhPSBFUElQRSkKICAgICAgICAgICAgICAgICAgewogICAg
ICAgICAgICAgICAgICAgIGNsZWFyZXJyIChzdGRvdXQpOyAvKiBUbyBzaWxlbmNlIGNsb3Nl
X3N0ZG91dCgpLiAgKi8KICAgICAgICAgICAgICAgICAgICBlcnJvciAoRVhJVF9GQUlMVVJF
LCBlcnJubywgIiVzIiwgXygid3JpdGUgZXJyb3IiKSk7CioqKioqKioqKioqKioqKgoqKiog
NzM0LDc1MiAqKioqCiAgICAgICAgICAgICAgICAgIHsKICAgICAgICAgICAgICAgICAgICAv
KiBOb3RlIHdyaXRpbmcgdG8gZmQsIHJhdGhlciB0aGFuIGZsdXNoaW5nIHRoZSBGSUxFIGdp
dmVzCiAgICAgICAgICAgICAgICAgICAgICAgYW4gOCUgcGVyZm9ybWFuY2UgYmVuZWZpdCwg
ZHVlIHRvIHJlZHVjZWQgZGF0YSBjb3B5aW5nLiAgKi8KISAgICAgICAgICAgICAgICAgICBp
ZiAoZnVsbF93cml0ZSAoZmlsZXNbaV9maWxlXS5vZmQsIGJwLCB0b193cml0ZSkgIT0gdG9f
d3JpdGUpCiAgICAgICAgICAgICAgICAgICAgICBlcnJvciAoRVhJVF9GQUlMVVJFLCBlcnJu
bywgIiVzIiwgZmlsZXNbaV9maWxlXS5vZl9uYW1lKTsKICAgICAgICAgICAgICAgICAgfQoh
ICAgICAgICAgICAgICAgZWxzZSBpZiAoZndyaXRlIChicCwgdG9fd3JpdGUsIDEsIGZpbGVz
W2lfZmlsZV0ub2ZpbGUpICE9IDEpCiAgICAgICAgICAgICAgICAgIGVycm9yIChFWElUX0ZB
SUxVUkUsIGVycm5vLCAiJXMiLCBmaWxlc1tpX2ZpbGVdLm9mX25hbWUpOwogICAgICAgICAg
ICAgICAgaWYgKGZpbGVfbGltaXQpCiAgICAgICAgICAgICAgICAgIHsKISAgICAgICAgICAg
ICAgICAgICBpZiAoZmNsb3NlIChmaWxlc1tpX2ZpbGVdLm9maWxlKSAhPSAwKQogICAgICAg
ICAgICAgICAgICAgICAgZXJyb3IgKEVYSVRfRkFJTFVSRSwgZXJybm8sICIlcyIsIGZpbGVz
W2lfZmlsZV0ub2ZfbmFtZSk7CiAgICAgICAgICAgICAgICAgICAgZmlsZXNbaV9maWxlXS5v
ZmQgPSBPRkRfQVBQRU5EOwogICAgICAgICAgICAgICAgICB9CiAgICAgICAgICAgICAgICBp
ZiAobmV4dCAmJiArK2lfZmlsZSA9PSBuKQohICAgICAgICAgICAgICAgICBpX2ZpbGUgPSAw
OwogICAgICAgICAgICAgIH0KICAKICAgICAgICAgICAgYnAgPSBicF9vdXQ7Ci0tLSA4Njgs
ODkyIC0tLS0KICAgICAgICAgICAgICAgICAgewogICAgICAgICAgICAgICAgICAgIC8qIE5v
dGUgd3JpdGluZyB0byBmZCwgcmF0aGVyIHRoYW4gZmx1c2hpbmcgdGhlIEZJTEUgZ2l2ZXMK
ICAgICAgICAgICAgICAgICAgICAgICBhbiA4JSBwZXJmb3JtYW5jZSBiZW5lZml0LCBkdWUg
dG8gcmVkdWNlZCBkYXRhIGNvcHlpbmcuICAqLwohICAgICAgICAgICAgICAgICAgIGlmIChm
dWxsX3dyaXRlIChmaWxlc1tpX2ZpbGVdLm9mZCwgYnAsIHRvX3dyaXRlKSAhPSB0b193cml0
ZQohICAgICAgICAgICAgICAgICAgICAgICAmJiBlcnJubyAhPSBFUElQRSkKICAgICAgICAg
ICAgICAgICAgICAgIGVycm9yIChFWElUX0ZBSUxVUkUsIGVycm5vLCAiJXMiLCBmaWxlc1tp
X2ZpbGVdLm9mX25hbWUpOwogICAgICAgICAgICAgICAgICB9CiEgICAgICAgICAgICAgICBl
bHNlIGlmIChmd3JpdGUgKGJwLCB0b193cml0ZSwgMSwgZmlsZXNbaV9maWxlXS5vZmlsZSkg
IT0gMQohICAgICAgICAgICAgICAgICAgICAgICAgJiYgZXJybm8gIT0gRVBJUEUpCiAgICAg
ICAgICAgICAgICAgIGVycm9yIChFWElUX0ZBSUxVUkUsIGVycm5vLCAiJXMiLCBmaWxlc1tp
X2ZpbGVdLm9mX25hbWUpOwogICAgICAgICAgICAgICAgaWYgKGZpbGVfbGltaXQpCiAgICAg
ICAgICAgICAgICAgIHsKISAgICAgICAgICAgICAgICAgICBpZiAoZmNsb3NlIChmaWxlc1tp
X2ZpbGVdLm9maWxlKSAhPSAwICYmIGVycm5vICE9IEVQSVBFKQogICAgICAgICAgICAgICAg
ICAgICAgZXJyb3IgKEVYSVRfRkFJTFVSRSwgZXJybm8sICIlcyIsIGZpbGVzW2lfZmlsZV0u
b2ZfbmFtZSk7CisgICAgICAgICAgICAgICAgICAgZmlsZXNbaV9maWxlXS5vZmlsZSA9IE5V
TEw7CiAgICAgICAgICAgICAgICAgICAgZmlsZXNbaV9maWxlXS5vZmQgPSBPRkRfQVBQRU5E
OwogICAgICAgICAgICAgICAgICB9CiAgICAgICAgICAgICAgICBpZiAobmV4dCAmJiArK2lf
ZmlsZSA9PSBuKQohICAgICAgICAgICAgICAgICB7CiEgICAgICAgICAgICAgICAgICAgd3Jh
cHBlZCA9IHRydWU7CiEgICAgICAgICAgICAgICAgICAgaV9maWxlID0gMDsKISAgICAgICAg
ICAgICAgICAgfQogICAgICAgICAgICAgIH0KICAKICAgICAgICAgICAgYnAgPSBicF9vdXQ7
CioqKioqKioqKioqKioqKgoqKiogNzU3LDc2NyAqKioqCiAgICAgICBhbmQgdG8gc2lnbmFs
IGFueSB3YWl0aW5nIGZpZm8gY29uc3VtZXJzLgogICAgICAgQWxzbywgY2xvc2UgYW55IG9w
ZW4gZmlsZSBkZXNjcmlwdG9ycy4KICAgICAgIEZJWE1FOiBTaG91bGQgd2UgZG8gdGhpcyBi
ZWZvcmUgRVhJVF9GQUlMVVJFPyAgKi8KISAgIGZvciAoaV9maWxlID0gMDsgIWsgJiYgIWVs
aWRlX2VtcHR5X2ZpbGVzICYmIGlfZmlsZSA8IG47IGlfZmlsZSsrKQogICAgICB7CiEgICAg
ICAgZmlsZV9saW1pdCB8PSBvZmlsZV9vcGVuIChmaWxlcywgaV9maWxlLCBuKTsKISAgICAg
ICBpZiAoZmNsb3NlIChmaWxlc1tpX2ZpbGVdLm9maWxlKSAhPSAwKQohICAgICAgICAgZXJy
b3IgKEVYSVRfRkFJTFVSRSwgZXJybm8sICIlcyIsIGZpbGVzW2lfZmlsZV0ub2ZfbmFtZSk7
CiAgICAgIH0KICB9CiAgCi0tLSA4OTcsOTE0IC0tLS0KICAgICAgIGFuZCB0byBzaWduYWwg
YW55IHdhaXRpbmcgZmlmbyBjb25zdW1lcnMuCiAgICAgICBBbHNvLCBjbG9zZSBhbnkgb3Bl
biBmaWxlIGRlc2NyaXB0b3JzLgogICAgICAgRklYTUU6IFNob3VsZCB3ZSBkbyB0aGlzIGJl
Zm9yZSBFWElUX0ZBSUxVUkU/ICAqLwohICAgaWYgKCFrKQogICAgICB7CiEgICAgICAgaW50
IGNlaWxpbmcgPSAod3JhcHBlZCA/IG4gOiBpX2ZpbGUpOwohICAgICAgIGZvciAoaV9maWxl
ID0gMDsgaV9maWxlIDwgbjsgaV9maWxlKyspCiEgICAgICAgICB7CiEgICAgICAgICAgIGlm
IChpX2ZpbGUgPj0gY2VpbGluZyAmJiAhZWxpZGVfZW1wdHlfZmlsZXMpCiEgICAgICAgICAg
ICAgZmlsZV9saW1pdCB8PSBvZmlsZV9vcGVuIChmaWxlcywgaV9maWxlLCBuKTsKISAgICAg
ICAgICAgaWYgKGZpbGVzW2lfZmlsZV0ub2ZkID49IDApCiEgICAgICAgICAgICAgY2xvc2Vv
dXQgKGZpbGVzW2lfZmlsZV0ub2ZpbGUsIGZpbGVzW2lfZmlsZV0ub2ZkLAohICAgICAgICAg
ICAgICAgICAgICAgICBmaWxlc1tpX2ZpbGVdLm9waWQsIGZpbGVzW2lfZmlsZV0ub2ZfbmFt
ZSk7CiEgICAgICAgICAgIGZpbGVzW2lfZmlsZV0ub2ZkID0gT0ZEX0FQUEVORDsKISAgICAg
ICAgIH0KICAgICAgfQogIH0KICAKKioqKioqKioqKioqKioqCioqKiA4MjQsODMwICoqKioK
ICAgICAgICBpbnQgdGhpc19vcHRpbmQgPSBvcHRpbmQgPyBvcHRpbmQgOiAxOwogICAgICAg
IGNoYXIgKnNsYXNoOwogIAohICAgICAgIGMgPSBnZXRvcHRfbG9uZyAoYXJnYywgYXJndiwg
IjAxMjM0NTY3ODlDOmE6YjpkZWw6bjp1IiwgbG9uZ29wdHMsIE5VTEwpOwogICAgICAgIGlm
IChjID09IC0xKQogICAgICAgICAgYnJlYWs7CiAgCi0tLSA5NzEsOTc4IC0tLS0KICAgICAg
ICBpbnQgdGhpc19vcHRpbmQgPSBvcHRpbmQgPyBvcHRpbmQgOiAxOwogICAgICAgIGNoYXIg
KnNsYXNoOwogIAohICAgICAgIGMgPSBnZXRvcHRfbG9uZyAoYXJnYywgYXJndiwgIjAxMjM0
NTY3ODlDOmE6YjpkZWY6bDpuOnUiLAohICAgICAgICAgICAgICAgICAgICAgICAgbG9uZ29w
dHMsIE5VTEwpOwogICAgICAgIGlmIChjID09IC0xKQogICAgICAgICAgYnJlYWs7CiAgCioq
KioqKioqKioqKioqKgoqKiogOTU1LDk2MCAqKioqCi0tLSAxMTAzLDExMTIgLS0tLQogICAg
ICAgICAgICBlbGlkZV9lbXB0eV9maWxlcyA9IHRydWU7CiAgICAgICAgICAgIGJyZWFrOwog
IAorICAgICAgICAgY2FzZSAnZic6CisgICAgICAgICAgIGZpbHRlcl9jb21tYW5kID0gb3B0
YXJnOworICAgICAgICAgICBicmVhazsKKyAKICAgICAgICAgIGNhc2UgSU9fQkxLU0laRV9P
UFRJT046CiAgICAgICAgICAgIHsKICAgICAgICAgICAgICB1aW50bWF4X3QgdG1wX2Jsa19z
aXplOwoqKioqKioqKioqKioqKioKKioqIDEwNDgsMTA1MyAqKioqCi0tLSAxMjAwLDEyMTcg
LS0tLQogIAogICAgYnVmID0gcHRyX2FsaWduICh4bWFsbG9jIChpbl9ibGtfc2l6ZSArIDEg
KyBwYWdlX3NpemUgLSAxKSwgcGFnZV9zaXplKTsKICAKKyAgIC8qIFdoZW4gZmlsdGVyaW5n
LCBjbG9zdXJlIG9mIG9uZSBwaXBlIG11c3Qgbm90IHRlcm1pbmF0ZSB0aGUgcHJvY2VzcywK
KyAgICAgIGFzIHRoZXJlIG1heSBzdGlsbCBiZSBvdGhlciBzdHJlYW1zIGV4cGVjdGluZyBp
bnB1dCBmcm9tIHVzLiAgKi8KKyAgIHNpZ2VtcHR5c2V0ICgmbmV3YmxvY2tlZCk7CisgICBp
ZiAoZmlsdGVyX2NvbW1hbmQpCisgICAgIHsKKyAgICAgICBzdHJ1Y3Qgc2lnYWN0aW9uIGFj
dDsKKyAgICAgICBzaWdhY3Rpb24gKFNJR1BJUEUsIE5VTEwsICZhY3QpOworICAgICAgIGlm
IChhY3Quc2FfaGFuZGxlciAhPSBTSUdfSUdOKQorICAgICAgICAgc2lnYWRkc2V0ICgmbmV3
YmxvY2tlZCwgU0lHUElQRSk7CisgICAgIH0KKyAgIHNpZ3Byb2NtYXNrIChTSUdfQkxPQ0ss
ICZuZXdibG9ja2VkLCAmb2xkYmxvY2tlZCk7CisgCiAgICBzd2l0Y2ggKHNwbGl0X3R5cGUp
CiAgICAgIHsKICAgICAgY2FzZSB0eXBlX2RpZ2l0czoKKioqKioqKioqKioqKioqCioqKiAx
MDg0LDEwOTMgKioqKgogICAgICAgIGFib3J0ICgpOwogICAgICB9CiAgCiAgICBpZiAoY2xv
c2UgKFNURElOX0ZJTEVOTykgIT0gMCkKICAgICAgZXJyb3IgKEVYSVRfRkFJTFVSRSwgZXJy
bm8sICIlcyIsIGluZmlsZSk7CiEgICBpZiAob3V0cHV0X2Rlc2MgPj0gMCAmJiBjbG9zZSAo
b3V0cHV0X2Rlc2MpIDwgMCkKISAgICAgZXJyb3IgKEVYSVRfRkFJTFVSRSwgZXJybm8sICIl
cyIsIG91dGZpbGUpOwogIAogICAgZXhpdCAoRVhJVF9TVUNDRVNTKTsKICB9Ci0tLSAxMjQ4
LDEyNTggLS0tLQogICAgICAgIGFib3J0ICgpOwogICAgICB9CiAgCisgICBzaWdwcm9jbWFz
ayAoU0lHX1NFVE1BU0ssICZvbGRibG9ja2VkLCBOVUxMKTsKKyAKICAgIGlmIChjbG9zZSAo
U1RESU5fRklMRU5PKSAhPSAwKQogICAgICBlcnJvciAoRVhJVF9GQUlMVVJFLCBlcnJubywg
IiVzIiwgaW5maWxlKTsKISAgIGNsb3Nlb3V0IChOVUxMLCBvdXRwdXRfZGVzYywgZmlsdGVy
X3BpZCwgb3V0ZmlsZSk7CiAgCiAgICBleGl0IChFWElUX1NVQ0NFU1MpOwogIH0K

--nextpart-29834--


From MAILER-DAEMON Fri Jan 07 02:38:54 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1Pb6uU-00029c-Gz
	for mharc-coreutils@gnu.org; Fri, 07 Jan 2011 02:38:54 -0500
Received: from [140.186.70.92] (port=45169 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1Pb6uS-00029W-II
	for coreutils@gnu.org; Fri, 07 Jan 2011 02:38:53 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Pb6uR-0001nB-Gk
	for coreutils@gnu.org; Fri, 07 Jan 2011 02:38:52 -0500
Received: from mx.meyering.net ([82.230.74.64]:34000)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Pb6uR-0001n3-9U
	for coreutils@gnu.org; Fri, 07 Jan 2011 02:38:51 -0500
Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000)
	id D053A600D2; Fri,  7 Jan 2011 08:38:48 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: =?utf-8?Q?P=C3=A1draig?= Brady <P@draigBrady.com>
Subject: Re: [coreutils] [PATCH] maint: suppress some clang scan-build warnings
In-Reply-To: <4D267935.9000008@draigBrady.com> (=?utf-8?Q?=22P=C3=A1draig?=
	Brady"'s message of "Fri, 07 Jan 2011 02:23:49 +0000")
References: <4D267935.9000008@draigBrady.com>
Date: Fri, 07 Jan 2011 08:38:48 +0100
Message-ID: <87lj2xxiwn.fsf@meyering.net>
Lines: 43
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
Cc: Coreutils <coreutils@gnu.org>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 07 Jan 2011 07:38:53 -0000

P=C3=A1draig Brady wrote:
>
>     maint: suppress some clang scan-build warnings
>
>     * src/pr.c (char_to_clump): Remove a dead store.
>     * src/remove.c (fts_skip_tree): Likewise.
>     * src/sort.c (key_warnings): Likewise.
>     (sort): Suppress an uninitialized pointer warning.

Thanks!  That is fine, with one correction below.

I hope clang's analysis improves to the point
that we can make coreutils warning-free without too much effort.
I noticed that this warning about ln.c is bogus:

    203 && (backup_type =3D=3D no_backups || !symbolic_link)
    204 && (!symbolic_link || stat (source, &source_stats) =3D=3D 0)
    205 && SAME_INODE (source_stats, dest_stats)

      Within the expansion of the macro 'SAME_INODE':
      The left operand of '=3D=3D' is a garbage value

Here's the expansion:
         ((source_stats).st_ino =3D=3D (dest_stats).st_ino &&
          (source_stats).st_dev =3D=3D (dest_stats).st_dev)

The problem is that clang made the assumption, a few lines above,
that "symbolic_link" was true, yet doesn't deduce the consequence:
whenever SAME_INODE is reached, the just-prior call to stat has
succeeded, and hence has defined "source_stats".

...
> diff --git a/src/sort.c b/src/sort.c
...
> @@ -3770,6 +3770,7 @@ sort (char *const *files, size_t nfiles, char const=
 *output_file,
>        size_t nthreads)
>  {
>    struct buffer buf;
> +  IF_LINT (buf.buf =3D=3D NULL);

ETOOLATE ;-)  You must mean this:

    IF_LINT (buf.buf =3D NULL);


From MAILER-DAEMON Fri Jan 07 07:31:04 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PbBTD-0005lM-Px
	for mharc-coreutils@gnu.org; Fri, 07 Jan 2011 07:31:03 -0500
Received: from [140.186.70.92] (port=43481 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1Pb8Cp-0007oX-Lj
	for coreutils@gnu.org; Fri, 07 Jan 2011 04:01:57 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <rakib.mullick@gmail.com>) id 1Pb8Co-0000bV-1L
	for coreutils@gnu.org; Fri, 07 Jan 2011 04:01:55 -0500
Received: from mail-wy0-f169.google.com ([74.125.82.169]:33958)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <rakib.mullick@gmail.com>) id 1Pb8Cn-0000b3-TB
	for coreutils@gnu.org; Fri, 07 Jan 2011 04:01:53 -0500
Received: by wyj26 with SMTP id 26so17969538wyj.0
	for <coreutils@gnu.org>; Fri, 07 Jan 2011 01:01:52 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma;
	h=domainkey-signature:mime-version:received:received:date:message-id
	:subject:from:to:content-type;
	bh=ogfnES245wdLiSyvdS8UBltYPFJ9BELPtgVgOfyOrSk=;
	b=lfYYXBZe+QvF9H95HoA0woT7clkOfiTkP5YFlRF0XoJYlV37Jtk3w1UOl9N/9JbS/a
	7/jLV3E7FjxMC0oEmUimdwolXyjdxqWPKSbOmdd8VOccgDNHy4znm48m7f8k7wrWFmXi
	46MY/C8k8xjGdyqMVnY6A0CB9mviDwktsoAJI=
DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma;
	h=mime-version:date:message-id:subject:from:to:content-type;
	b=Wla6jZoaW5RYMxJZH45T8lYN65IBTdZhPfhjPKwpM2TiV/I4ZjO3uh1tk0blU+jRXu
	Nx4PEztrW2kgGCl1NoYCmn966OxdxCP3Xra2HTS7FxkMe+8Krpz23F7LTrYbOaFgaznn
	6zY11svJimCmwYzNBSfjo3LvxEfpUdPH3GAY8=
MIME-Version: 1.0
Received: by 10.216.144.205 with SMTP id n55mr1329842wej.5.1294390911962; Fri,
	07 Jan 2011 01:01:51 -0800 (PST)
Received: by 10.216.72.143 with HTTP; Fri, 7 Jan 2011 01:01:51 -0800 (PST)
Date: Fri, 7 Jan 2011 04:01:51 -0500
Message-ID: <AANLkTinGw+2sqPfZsV=CNRiW5xBRVjXA6BYUFHNrBKpR@mail.gmail.com>
From: Rakib Mullick <rakib.mullick@gmail.com>
To: coreutils@gnu.org
Content-Type: text/plain; charset=ISO-8859-1
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 2)
X-Mailman-Approved-At: Fri, 07 Jan 2011 07:31:02 -0500
Subject: [coreutils] RFC for adding file creation mode feature into touch
	utility.
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 07 Jan 2011 09:01:57 -0000

Hello list,

This is a Request For Comment for adding a new feature into touch
utility. touch is basically an utility to modify a file's timestamps
and it is also used for file creation vastly. We can create a file by
just doing 'touch filename'. There are lot of cases where we need to
change file's permission after creation. To do that we need to
separately do 'chmod -/+ mode filename'. Now, my proposal is to adding
mode changing feature to the touch utility, it will help us to create
a file by just doing 'touch --mode xxx filename'. It will create a
file with specified mode.

Hope to get some response.

thanks in advance!
rakib


From MAILER-DAEMON Fri Jan 07 08:03:26 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PbByY-0000TH-1Z
	for mharc-coreutils@gnu.org; Fri, 07 Jan 2011 08:03:26 -0500
Received: from [140.186.70.92] (port=50926 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PbByV-0000T8-Bk
	for coreutils@gnu.org; Fri, 07 Jan 2011 08:03:24 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1PbByT-00087t-0r
	for coreutils@gnu.org; Fri, 07 Jan 2011 08:03:23 -0500
Received: from mail1.slb.deg.dub.stisp.net ([84.203.253.98]:8921)
	by eggs.gnu.org with smtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1PbByS-00087h-M4
	for coreutils@gnu.org; Fri, 07 Jan 2011 08:03:20 -0500
Received: (qmail 10823 invoked from network); 7 Jan 2011 13:03:18 -0000
Received: from unknown (HELO ?192.168.2.25?) (84.203.137.218)
	by mail1.slb.deg.dub.stisp.net with SMTP; 7 Jan 2011 13:03:18 -0000
Message-ID: <4D270F11.2040309@draigBrady.com>
Date: Fri, 07 Jan 2011 13:03:13 +0000
From: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US;
	rv:1.9.1.8) Gecko/20100227 Thunderbird/3.0.3
MIME-Version: 1.0
To: Assaf Gordon <assafgordon@gmail.com>
Subject: Re: [coreutils] join feature: auto-format
References: <4CACDEE5.6050908@gmail.com>	<4CAD0E47.7070005@draigBrady.com>	<4CAD9F55.6000001@draigBrady.com>	<4CAE06BA.2080402@gmail.com>
	<4CAE1099.4010702@draigBrady.com> <4D25AFED.1090902@draigBrady.com>
In-Reply-To: <4D25AFED.1090902@draigBrady.com>
X-Enigmail-Version: 1.0.1
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
X-detected-operating-system: by eggs.gnu.org: FreeBSD 4.6-4.9
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 07 Jan 2011 13:03:24 -0000

On 06/01/11 12:05, Pádraig Brady wrote:
> On 07/10/10 19:25, Pádraig Brady wrote:
>> On 07/10/10 18:43, Assaf Gordon wrote:
>>> Pádraig Brady wrote, On 10/07/2010 06:22 AM:
>>>> On 07/10/10 01:03, Pádraig Brady wrote:
>>>>> On 06/10/10 21:41, Assaf Gordon wrote:
>>>>>>
>>>>>> The "--auto-format" feature simply builds the "-o" format line automatically, based on the number of columns from both input files.
>>>>>
>>>>> Thanks for persisting with this and presenting a concise example.
>>>>> I agree that this is useful and can't think of a simple workaround.
>>>>> Perhaps the interface would be better as:
>>>>>
>>>>> -o {all (default), padded, FORMAT}
>>>>>
>>>>> where padded is the functionality you're suggesting?
>>>>
>>>> Thinking more about it, we mightn't need any new options at all.
>>>> Currently -e is redundant if -o is not specified.
>>>> So how about changing that so that if -e is specified
>>>> we operate as above by auto inserting empty fields?
>>>> Also I wouldn't base on the number of fields in the first line,
>>>> instead auto padding to the biggest number of fields
>>>> on the current lines under consideration.
>>>
>>> My concern is the principle of "least surprise" - if there are existing scripts/programs that specify "-e" without "-o" (doesn't make sense, but still possible) - this change will alter their behavior.
>>>
>>> Also, implying/forcing 'auto-format' when "-e" is used without "-o" might be a bit confusing.
>>
>> Well seeing as -e without -o currently does nothing,
>> I don't think we need to worry too much about changing that behavior.
>> Also to me, specifying -e EMPTY implicitly means I want
>> fields missing from one of the files replaced with EMPTY.
>>
>> Note POSIX is more explicit, and describes our current operation:
>>
>> -e EMPTY
>>   Replace empty output fields in the list selected by -o with EMPTY
>>
>> So changing that would be an extension to POSIX.
>> But I still think it makes sense.
>> I'll prepare a patch soon, to do as I describe above,
>> unless there are objections.
> 
> The attached changes `join` (from what's done on other platforms) so that...
> 
> `join -e` will automatically pad missing fields from one file
> so that the same number of fields are output from each file.
> Previously -e was only used for missing fields specified with -o or -j.
> 
> With this change join now does:
> 
> $ cat file1
> a 1 2
> b 1
> d 1 2
> 
> $ cat file2
> a 3 4
> b 3 4
> c 3 4
> 
> $ join -a1 -a2 -1 1 -2 1 -e. file1 file2
> a 1 2 3 4
> b 1 . 3 4
> c . . 3 4
> d 1 2 . .
> 
> $ join -a1 -a2 -1 1 -2 4 -e. file1 file2
> . . . . a 3 4
> . . . . b 3 4
> . . . . c 3 4
> a 1 2 . .
> b 1 .
> d 1 2 . .
> 
> $ join -a1 -a2 -1 4 -2 1 -e. file1 file2
> . a 1 2 . . .
> . b 1 . .
> . d 1 2 . . .
> a . . 3 4
> b . . 3 4
> c . . 3 4
> 
> $ join -a1 -a2 -1 4 -2 4 -e. file1 file2
> . a 1 2 a 3 4
> . a 1 2 b 3 4
> . a 1 2 c 3 4
> . b 1 . a 3 4
> . b 1 . b 3 4
> . b 1 . c 3 4
> . d 1 2 a 3 4
> . d 1 2 b 3 4
> . d 1 2 c 3 4
> 
> While -e without -o was previously a noop, and so could safely be extended IMHO,
> this will also change the behavior when with -e and -j are specified.
> Previously if -j > 1 was specified, and that field was missing,
> then -e would be used in its place, rather than the empty string.
> This still does that, but also does the padding.
> Without the -j issue I'd be 80:20 for just extending -e to auto pad,
> but given -j I'm 50:50.  The alternative it to select this with
> say '-o padded', but that's less discoverable, and complicates
> the interface somewhat.

Considering this more, I think it's safer to auto pad only
when '-o padded' is specified. I notice the plan9 `join` man page
has an example that uses -e '' to explicitly specify the NUL string as filler,
which would have triggered our auto pad if we left it as above.

cheers,
Pádraig.


From MAILER-DAEMON Fri Jan 07 09:55:48 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PbDjI-00080e-JN
	for mharc-coreutils@gnu.org; Fri, 07 Jan 2011 09:55:48 -0500
Received: from [140.186.70.92] (port=53439 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PbDjG-0007zo-7c
	for coreutils@gnu.org; Fri, 07 Jan 2011 09:55:47 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1PbDjE-0000Zw-10
	for coreutils@gnu.org; Fri, 07 Jan 2011 09:55:46 -0500
Received: from mx1.redhat.com ([209.132.183.28]:23872)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>)
	id 1PbDjC-0000ZU-7m; Fri, 07 Jan 2011 09:55:42 -0500
Received: from int-mx01.intmail.prod.int.phx2.redhat.com
	(int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11])
	by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id p07EtfrD002106
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK);
	Fri, 7 Jan 2011 09:55:41 -0500
Received: from [10.3.113.114] (ovpn-113-114.phx2.redhat.com [10.3.113.114])
	by int-mx01.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP
	id p07EteM3015898; Fri, 7 Jan 2011 09:55:41 -0500
Message-ID: <4D27296C.80203@redhat.com>
Date: Fri, 07 Jan 2011 07:55:40 -0700
From: Eric Blake <eblake@redhat.com>
Organization: Red Hat
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US;
	rv:1.9.2.13) Gecko/20101209 Fedora/3.1.7-0.35.b3pre.fc14
	Lightning/1.0b3pre Mnenhy/0.8.3 Thunderbird/3.1.7
MIME-Version: 1.0
To: Karl Heuer <kwzh@gnu.org>
Subject: Re: [coreutils] [PATCH] coreutils new feature: split --filter
References: <E1Pb1mG-0004rA-2d@fencepost.gnu.org>
In-Reply-To: <E1Pb1mG-0004rA-2d@fencepost.gnu.org>
X-Enigmail-Version: 1.1.2
OpenPGP: url=http://people.redhat.com/eblake/eblake.gpg
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature";
	boundary="------------enigBD2B52B5260B39966FD927B1"
X-Scanned-By: MIMEDefang 2.67 on 10.5.11.11
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 07 Jan 2011 14:55:47 -0000

This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--------------enigBD2B52B5260B39966FD927B1
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

On 01/06/2011 07:10 PM, Karl Heuer wrote:
> For some years now I've been bemoaning the lack of a
> `split --filter=3DCOMMAND' feature, which would write to
> pipes instead of files, instead of having to split to
> a set of files and then run the filter on each of them.
> The environment variable FILE is set to the name of the
> file that would have been written; it's the command's
> responsibility to use that string appropriately.
> Sample usage (note proper quoting of $FILE):
> $ split --filter 'gzip >$FILE.gz'
>=20
> The details turned out to be rather trickier than I'd
> expected, but I think it's ready to be shared with the
> folks in charge of coreutils.  Source patch attached.
> I haven't touched the documentation or tests.

Thanks for the contribution.  The idea sounds like it might be useful,
but before we can review and incorporate it, we'd need to clear up
copyright status.  Are you willing to assign copyright to the FSF?  If
so, we can send you the details off-list on how to start that paperwork
process.

--=20
Eric Blake   eblake@redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org


--------------enigBD2B52B5260B39966FD927B1
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Public key at http://people.redhat.com/eblake/eblake.gpg
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/

iQEcBAEBCAAGBQJNJylsAAoJEKeha0olJ0NqyKAH/RFxAQmowsjMdScYc4QP1lI6
4fbjVqEuxnqpB8apkhw4Xnqogpd6mpkWZmGnHgR/Z1+KX+Rc6I6ZVQV/CI0CnYdM
GtF4LLjqTkK4MgLCWDYdu+8uW7Nq9GeTy9WTismqEmYqsHXkbO9byHpzlGMOujRo
cgFhzfXEwpILz9VnFMKk9lpGv3R/do8E1eFzE98OWyxsM9/rJw450nG4gIZ4lTzM
HEjysoA9HapYg0bY3SUrR18kvFH5xrR6u+7gsfsD3bBFf35ER5SUD7B4/oA/alfL
uwPRLG0lDHMaPmlzcyGTXELDMHpuVqJvqy2UZLzgJFHZ7xje8r28WlWM8UgcKYQ=
=Byjr
-----END PGP SIGNATURE-----

--------------enigBD2B52B5260B39966FD927B1--


From MAILER-DAEMON Fri Jan 07 10:05:37 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PbDsn-00017z-CH
	for mharc-coreutils@gnu.org; Fri, 07 Jan 2011 10:05:37 -0500
Received: from [140.186.70.92] (port=60164 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PbDsl-00017l-4u
	for coreutils@gnu.org; Fri, 07 Jan 2011 10:05:35 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PbDsj-0003Zv-W0
	for coreutils@gnu.org; Fri, 07 Jan 2011 10:05:34 -0500
Received: from mx.meyering.net ([82.230.74.64]:34989)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>)
	id 1PbDsj-0003Zf-NZ; Fri, 07 Jan 2011 10:05:33 -0500
Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000)
	id 5C42B60155; Fri,  7 Jan 2011 16:05:32 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: kwzh@gnu.org (Karl Heuer)
Subject: Re: [coreutils] [PATCH] coreutils new feature: split --filter
In-Reply-To: <E1Pb1mG-0004rA-2d@fencepost.gnu.org> (Karl Heuer's message of
	"Thu, 06 Jan 2011 21:10:04 -0500")
References: <E1Pb1mG-0004rA-2d@fencepost.gnu.org>
Date: Fri, 07 Jan 2011 16:05:32 +0100
Message-ID: <878vywwy83.fsf@meyering.net>
Lines: 31
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 07 Jan 2011 15:05:36 -0000

Karl Heuer wrote:
> For some years now I've been bemoaning the lack of a
> `split --filter=COMMAND' feature, which would write to
> pipes instead of files, instead of having to split to
> a set of files and then run the filter on each of them.
> The environment variable FILE is set to the name of the
> file that would have been written; it's the command's
> responsibility to use that string appropriately.
> Sample usage (note proper quoting of $FILE):
> $ split --filter 'gzip >$FILE.gz'
>
> The details turned out to be rather trickier than I'd
> expected, but I think it's ready to be shared with the
> folks in charge of coreutils.  Source patch attached.
> I haven't touched the documentation or tests.

Hi Karl,
It's been a long time.  Hope you're doing well.

This sounds like a feature I'd be happy to use.
I think we're ok on the copyright front, since split
used to be part of the textutils and you have an assignment
on file for that project.

Do you feel like updating NEWS and doc/coreutils.texi and/or
writing a test or two?

Regardless, someone should be able to review your patch
in the next few days.

Thanks!


From MAILER-DAEMON Fri Jan 07 10:09:23 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PbDwR-0003TB-9O
	for mharc-coreutils@gnu.org; Fri, 07 Jan 2011 10:09:23 -0500
Received: from [140.186.70.92] (port=33546 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PbDwO-0003RP-RG
	for coreutils@gnu.org; Fri, 07 Jan 2011 10:09:21 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PbDwN-0004MY-K5
	for coreutils@gnu.org; Fri, 07 Jan 2011 10:09:20 -0500
Received: from mx.meyering.net ([82.230.74.64]:39663)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PbDwN-0004MO-AE
	for coreutils@gnu.org; Fri, 07 Jan 2011 10:09:19 -0500
Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000)
	id AAF5E60155; Fri,  7 Jan 2011 16:09:18 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: =?utf-8?Q?P=C3=A1draig?= Brady <P@draigBrady.com>
Subject: Re: [coreutils] join feature: auto-format
In-Reply-To: <4D270F11.2040309@draigBrady.com> (=?utf-8?Q?=22P=C3=A1draig?=
	Brady"'s message of "Fri, 07 Jan 2011 13:03:13 +0000")
References: <4CACDEE5.6050908@gmail.com> <4CAD0E47.7070005@draigBrady.com>
	<4CAD9F55.6000001@draigBrady.com> <4CAE06BA.2080402@gmail.com>
	<4CAE1099.4010702@draigBrady.com> <4D25AFED.1090902@draigBrady.com>
	<4D270F11.2040309@draigBrady.com>
Date: Fri, 07 Jan 2011 16:09:18 +0100
Message-ID: <8739p4wy1t.fsf@meyering.net>
Lines: 19
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
Cc: Assaf Gordon <assafgordon@gmail.com>, coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 07 Jan 2011 15:09:22 -0000

P=C3=A1draig Brady wrote:
...
>> While -e without -o was previously a noop, and so could safely be extend=
ed IMHO,
>> this will also change the behavior when with -e and -j are specified.
>> Previously if -j > 1 was specified, and that field was missing,
>> then -e would be used in its place, rather than the empty string.
>> This still does that, but also does the padding.
>> Without the -j issue I'd be 80:20 for just extending -e to auto pad,
>> but given -j I'm 50:50.  The alternative it to select this with
>> say '-o padded', but that's less discoverable, and complicates
>> the interface somewhat.
>
> Considering this more, I think it's safer to auto pad only
> when '-o padded' is specified. I notice the plan9 `join` man page
> has an example that uses -e '' to explicitly specify the NUL string as fi=
ller,
> which would have triggered our auto pad if we left it as above.

I'm glad you found that, confirming that
the conservative approach is better.


From MAILER-DAEMON Fri Jan 07 10:23:53 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PbEAT-0003gm-C5
	for mharc-coreutils@gnu.org; Fri, 07 Jan 2011 10:23:53 -0500
Received: from [140.186.70.92] (port=51674 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PbEAQ-0003eo-Mf
	for coreutils@gnu.org; Fri, 07 Jan 2011 10:23:51 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1PbEAO-0007Ti-W7
	for coreutils@gnu.org; Fri, 07 Jan 2011 10:23:50 -0500
Received: from mx1.redhat.com ([209.132.183.28]:32248)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1PbEAO-0007TX-9V
	for coreutils@gnu.org; Fri, 07 Jan 2011 10:23:48 -0500
Received: from int-mx01.intmail.prod.int.phx2.redhat.com
	(int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11])
	by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id p07FNkYl006092
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK);
	Fri, 7 Jan 2011 10:23:46 -0500
Received: from [10.3.113.114] (ovpn-113-114.phx2.redhat.com [10.3.113.114])
	by int-mx01.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP
	id p07FNjqH024402; Fri, 7 Jan 2011 10:23:45 -0500
Message-ID: <4D273001.6000807@redhat.com>
Date: Fri, 07 Jan 2011 08:23:45 -0700
From: Eric Blake <eblake@redhat.com>
Organization: Red Hat
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US;
	rv:1.9.2.13) Gecko/20101209 Fedora/3.1.7-0.35.b3pre.fc14
	Lightning/1.0b3pre Mnenhy/0.8.3 Thunderbird/3.1.7
MIME-Version: 1.0
To: Rakib Mullick <rakib.mullick@gmail.com>
Subject: Re: [coreutils] RFC for adding file creation mode feature into touch
	utility.
References: <AANLkTinGw+2sqPfZsV=CNRiW5xBRVjXA6BYUFHNrBKpR@mail.gmail.com>
In-Reply-To: <AANLkTinGw+2sqPfZsV=CNRiW5xBRVjXA6BYUFHNrBKpR@mail.gmail.com>
X-Enigmail-Version: 1.1.2
OpenPGP: url=http://people.redhat.com/eblake/eblake.gpg
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature";
	boundary="------------enig315C805C6F0F5B4ECEE76900"
X-Scanned-By: MIMEDefang 2.67 on 10.5.11.11
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 07 Jan 2011 15:23:52 -0000

This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--------------enig315C805C6F0F5B4ECEE76900
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

On 01/07/2011 02:01 AM, Rakib Mullick wrote:
> Hello list,
>=20
> This is a Request For Comment for adding a new feature into touch
> utility. touch is basically an utility to modify a file's timestamps
> and it is also used for file creation vastly. We can create a file by
> just doing 'touch filename'. There are lot of cases where we need to
> change file's permission after creation. To do that we need to
> separately do 'chmod -/+ mode filename'. Now, my proposal is to adding
> mode changing feature to the touch utility, it will help us to create
> a file by just doing 'touch --mode xxx filename'. It will create a
> file with specified mode.
>=20
> Hope to get some response.

Hmm - mkdir already supports -m/--mode, so it does sound like making
touch support the same option might be a useful enhancement.  Of course,
if the file exists, the --mode option would be ignored.  And POSIX
requires that if the --mode option is not provided but a file is
created, that the file have mode 0666 as modified by umask.  We can't
use -m, though (POSIX already requires it for mtime).

Oops - that means I found a (slightly-related) existing bug - touch's
-a, -m, and -t have no long-option counterpart, which is a violation of
GNU Coding Standards.

I'm 70:30 for adding this idea, and it's rather simple to do, as well
(would still need doc, NEWS, and test additions).

Note that 'touch -r a b' must still use mode 0666-umask for b, rather
than copying a's permissions, because -r is mandated by POSIX; but
'touch -r a -M + b' is a great way to create b with the same permissions
as a (assuming b didn't exist).

diff --git i/src/touch.c w/src/touch.c
index 49be961..c327f53 100644
--- i/src/touch.c
+++ w/src/touch.c
@@ -28,6 +28,7 @@
 #include "argmatch.h"
 #include "error.h"
 #include "fd-reopen.h"
+#include "modechange.h"
 #include "parse-datetime.h"
 #include "posixtm.h"
 #include "posixver.h"
@@ -86,8 +87,9 @@ static struct option const longopts[] =3D
   {"time", required_argument, NULL, TIME_OPTION},
   {"no-create", no_argument, NULL, 'c'},
   {"date", required_argument, NULL, 'd'},
-  {"reference", required_argument, NULL, 'r'},
   {"no-dereference", no_argument, NULL, 'h'},
+  {"mode", required_argument, NULL, 'M'},
+  {"reference", required_argument, NULL, 'r'},
   {GETOPT_HELP_OPTION_DECL},
   {GETOPT_VERSION_OPTION_DECL},
   {NULL, 0, NULL, 0}
@@ -120,7 +122,7 @@ get_reldate (struct timespec *result,
    Return true if successful.  */

 static bool
-touch (const char *file)
+touch (const char *file, mode_t mode)
 {
   bool ok;
   int fd =3D -1;
@@ -132,11 +134,9 @@ touch (const char *file)
   else if (! (no_create || no_dereference))
     {
       /* Try to open FILE, creating it if necessary.  */
-      int default_permissions =3D
-        S_IRUSR | S_IWUSR | S_IRGRP | S_IWGRP | S_IROTH | S_IWOTH;
       fd =3D fd_reopen (STDIN_FILENO, file,
                       O_WRONLY | O_CREAT | O_NONBLOCK | O_NOCTTY,
-                      default_permissions);
+                      mode);

       /* Don't save a copy of errno if it's EISDIR, since that would lea=
d
          touch to give a bogus diagnostic for e.g., `touch /' (assuming
@@ -239,6 +239,7 @@ Mandatory arguments to long options are mandatory
for short options too.\n\
                          file (useful only on systems that can change
the\n\
                          timestamps of a symlink)\n\
   -m                     change only the modification time\n\
+  -M, --mode=3DMODE        set file mode (as in chmod), not a=3Drw - uma=
sk\n\
 "), stdout);
       fputs (_("\
   -r, --reference=3DFILE   use this file's times instead of current time=
\n\
@@ -265,6 +266,8 @@ main (int argc, char **argv)
   bool date_set =3D false;
   bool ok =3D true;
   char const *flex_date =3D NULL;
+  const char *specified_mode =3D NULL;
+  mode_t mode =3D S_IRUSR | S_IWUSR | S_IRGRP | S_IWGRP | S_IROTH | S_IW=
OTH;

   initialize_main (&argc, &argv);
   set_program_name (argv[0]);
@@ -277,7 +280,7 @@ main (int argc, char **argv)
   change_times =3D 0;
   no_create =3D use_ref =3D false;

-  while ((c =3D getopt_long (argc, argv, "acd:fhmr:t:", longopts, NULL))=

!=3D -1)
+  while ((c =3D getopt_long (argc, argv, "acd:fhmM:r:t:", longopts,
NULL)) !=3D -1)
     {
       switch (c)
         {
@@ -304,6 +307,10 @@ main (int argc, char **argv)
           change_times |=3D CH_MTIME;
           break;

+        case 'M':
+          specified_mode =3D optarg;
+          break;
+
         case 'r':
           use_ref =3D true;
           ref_file =3D optarg;
@@ -361,6 +368,8 @@ main (int argc, char **argv)
           if (change_times & CH_MTIME)
             get_reldate (&newtime[1], flex_date, &newtime[1]);
         }
+      if (specified_mode)
+        mode =3D ref_stats.st_mode;
     }
   else
     {
@@ -424,6 +433,17 @@ main (int argc, char **argv)
         newtime[1].tv_nsec =3D newtime[0].tv_nsec =3D UTIME_NOW;
     }

+  if (specified_mode)
+    {
+      mode_t umask_value =3D umask (0);
+      struct mode_change *change =3D mode_compile (specified_mode);
+      if (!change)
+        error (EXIT_FAILURE, 0, _("invalid mode %s"),
+               quote (specified_mode));
+      mode =3D mode_adjust (mode, false, umask_value, change, NULL);
+      free (change);
+    }
+
   if (optind =3D=3D argc)
     {
       error (0, 0, _("missing file operand"));
@@ -431,7 +451,7 @@ main (int argc, char **argv)
     }

   for (; optind < argc; ++optind)
-    ok &=3D touch (argv[optind]);
+    ok &=3D touch (argv[optind], mode);

   exit (ok ? EXIT_SUCCESS : EXIT_FAILURE);
 }


--=20
Eric Blake   eblake@redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org


--------------enig315C805C6F0F5B4ECEE76900
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Public key at http://people.redhat.com/eblake/eblake.gpg
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/

iQEcBAEBCAAGBQJNJzABAAoJEKeha0olJ0Nq+vUH/AlnyXLbuuBIgolrK9291IQI
rYM5umnTXtaK8j/iwkq8IwPpG2wofRDSm5aEQbbGDHVIiwOElUqIt+1pzYxrhoaM
yF15aFjPhq8hE6t9HoD0np7AW5rlwwIjmxKhyVD9AqJsMYInvGUcstRaotKEiwut
qJaStfzOegaE+iZkzU+ZCcP4qgANHr+pSMV65SXI7PdP3r4elYb24qtyO8aB5C/G
mIfoGSa5tbCErTzg3mFWjxBt6fJtkTwkukTM0zxbI5vbqpzeV2fMsW83/UYDFkYM
rLpnyX2H5271qzx2LCU1+xa9IDH7L3tYLTRM1Iegob44e1YPVST4WmALbEW4Gkk=
=noot
-----END PGP SIGNATURE-----

--------------enig315C805C6F0F5B4ECEE76900--


From MAILER-DAEMON Fri Jan 07 11:23:59 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PbF6d-0001GF-Le
	for mharc-coreutils@gnu.org; Fri, 07 Jan 2011 11:23:59 -0500
Received: from [140.186.70.92] (port=53936 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PbF6Y-0001Cq-Nl
	for coreutils@gnu.org; Fri, 07 Jan 2011 11:23:58 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1PbF6Q-0002fQ-ID
	for coreutils@gnu.org; Fri, 07 Jan 2011 11:23:48 -0500
Received: from mail1.slb.deg.dub.stisp.net ([84.203.253.98]:9661)
	by eggs.gnu.org with smtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1PbF6Q-0002eJ-AA
	for coreutils@gnu.org; Fri, 07 Jan 2011 11:23:46 -0500
Received: (qmail 46247 invoked from network); 7 Jan 2011 16:23:44 -0000
Received: from unknown (HELO ?192.168.2.25?) (84.203.137.218)
	by mail1.slb.deg.dub.stisp.net with SMTP; 7 Jan 2011 16:23:44 -0000
Message-ID: <4D273E0B.6060203@draigBrady.com>
Date: Fri, 07 Jan 2011 16:23:39 +0000
From: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US;
	rv:1.9.1.8) Gecko/20100227 Thunderbird/3.0.3
MIME-Version: 1.0
To: coreutils@gnu.org, rakib.mullick@gmail.com
Subject: Re: [coreutils] RFC for adding file creation mode feature into touch
	utility.
References: <AANLkTinGw+2sqPfZsV=CNRiW5xBRVjXA6BYUFHNrBKpR@mail.gmail.com>
	<4D273001.6000807@redhat.com>
In-Reply-To: <4D273001.6000807@redhat.com>
X-Enigmail-Version: 1.0.1
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
X-detected-operating-system: by eggs.gnu.org: FreeBSD 4.6-4.9
Cc: 
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 07 Jan 2011 16:23:58 -0000

On 07/01/11 15:23, Eric Blake wrote:
> On 01/07/2011 02:01 AM, Rakib Mullick wrote:
>> Hello list,
>>
>> This is a Request For Comment for adding a new feature into touch
>> utility. touch is basically an utility to modify a file's timestamps
>> and it is also used for file creation vastly. We can create a file by
>> just doing 'touch filename'. There are lot of cases where we need to
>> change file's permission after creation. To do that we need to
>> separately do 'chmod -/+ mode filename'. Now, my proposal is to adding
>> mode changing feature to the touch utility, it will help us to create
>> a file by just doing 'touch --mode xxx filename'. It will create a
>> file with specified mode.
>>
>> Hope to get some response.
> 
> Hmm - mkdir already supports -m/--mode, so it does sound like making
> touch support the same option might be a useful enhancement.  Of course,
> if the file exists, the --mode option would be ignored.  And POSIX
> requires that if the --mode option is not provided but a file is
> created, that the file have mode 0666 as modified by umask.  We can't
> use -m, though (POSIX already requires it for mtime).

Why does mkdir have this option rather than relying on umask?
My guess is to close security races with sticky bits etc.
and that creating a file will not have the same issues?

...
> I'm 70:30 for adding this idea, and it's rather simple to do, as well
> (would still need doc, NEWS, and test additions).

I'm a little less enthused to diverge from POSIX for this.

> Note that 'touch -r a b' must still use mode 0666-umask for b, rather
> than copying a's permissions, because -r is mandated by POSIX; but
> 'touch -r a -M + b' is a great way to create b with the same permissions
> as a (assuming b didn't exist).

Note also the related `cp --attr old new` which will create a file
with the perms of 'old'. Also it's more general in that it will
copy any extended attributes. Timestamps are not copied since
that's not usually wanted, and one can always do a subsequent `touch -r`

cheers,
Pádraig.


From MAILER-DAEMON Fri Jan 07 11:39:13 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PbFLN-0004Cq-EW
	for mharc-coreutils@gnu.org; Fri, 07 Jan 2011 11:39:13 -0500
Received: from [140.186.70.92] (port=55548 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PbFLK-0004By-Nq
	for coreutils@gnu.org; Fri, 07 Jan 2011 11:39:11 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1PbFLJ-0006UD-0K
	for coreutils@gnu.org; Fri, 07 Jan 2011 11:39:10 -0500
Received: from mx1.redhat.com ([209.132.183.28]:9583)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1PbFLI-0006Tw-Lz
	for coreutils@gnu.org; Fri, 07 Jan 2011 11:39:08 -0500
Received: from int-mx02.intmail.prod.int.phx2.redhat.com
	(int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12])
	by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id p07Gd3ms021121
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK);
	Fri, 7 Jan 2011 11:39:04 -0500
Received: from [10.3.113.114] (ovpn-113-114.phx2.redhat.com [10.3.113.114])
	by int-mx02.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP
	id p07Gd3Hx024316; Fri, 7 Jan 2011 11:39:03 -0500
Message-ID: <4D2741A6.8060307@redhat.com>
Date: Fri, 07 Jan 2011 09:39:02 -0700
From: Eric Blake <eblake@redhat.com>
Organization: Red Hat
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US;
	rv:1.9.2.13) Gecko/20101209 Fedora/3.1.7-0.35.b3pre.fc14
	Lightning/1.0b3pre Mnenhy/0.8.3 Thunderbird/3.1.7
MIME-Version: 1.0
To: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
Subject: Re: [coreutils] RFC for adding file creation mode feature into touch
	utility.
References: <AANLkTinGw+2sqPfZsV=CNRiW5xBRVjXA6BYUFHNrBKpR@mail.gmail.com>	<4D273001.6000807@redhat.com>
	<4D273E0B.6060203@draigBrady.com>
In-Reply-To: <4D273E0B.6060203@draigBrady.com>
X-Enigmail-Version: 1.1.2
OpenPGP: url=http://people.redhat.com/eblake/eblake.gpg
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature";
	boundary="------------enigBAAD9C5413CD72B75B906F4E"
X-Scanned-By: MIMEDefang 2.67 on 10.5.11.12
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
Cc: rakib.mullick@gmail.com, coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 07 Jan 2011 16:39:11 -0000

This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--------------enigBAAD9C5413CD72B75B906F4E
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

On 01/07/2011 09:23 AM, P=E1draig Brady wrote:
>> Hmm - mkdir already supports -m/--mode, so it does sound like making
>> touch support the same option might be a useful enhancement.  Of cours=
e,
>> if the file exists, the --mode option would be ignored.  And POSIX
>> requires that if the --mode option is not provided but a file is
>> created, that the file have mode 0666 as modified by umask.  We can't
>> use -m, though (POSIX already requires it for mtime).
>=20
> Why does mkdir have this option rather than relying on umask?
> My guess is to close security races with sticky bits etc.
> and that creating a file will not have the same issues?

mkdir has -m because POSIX requires it; also, you are right that the
setup of 'mkdir -p -m' creates a hierarchy where you want the
permissions to be set correctly for the entire set for race avoidance.
But touch does not create hierarchies.

>=20
> ...
>> I'm 70:30 for adding this idea, and it's rather simple to do, as well
>> (would still need doc, NEWS, and test additions).
>=20
> I'm a little less enthused to diverge from POSIX for this.

It's not diverging from POSIX, so much as adding an extension.  It all
boils down to whether the extension would prove useful, and whether it
can optimize work flows faster than any existing alternative that gets
the same end result but with more steps.  But thinking about it more,
I'd rather use just the long-option --mode rather than burn a short
option -M (or any other).

As for creating a file with +x or with 07000 permissions already added,
why?  Creating an empty file as already executable or setuid doesn't
make much sense (the empty file will be a nop); why not wait until the
file has final executable contents before adding +x or special
permissions later?  So, that leaves only permissions matching of a
reference file (and recent GNU cp already supports that), or restricting
permissions less than touch's default of 0666 (but touch honors umask,
so just modify your umask before touch), neither of which need a chmod
after the fact, nor a --mode argument to touch.

What would really clinch the deal is if any other touch implementation
out there already has something like this.  But off the top of my head,
I don't know of any (neither FreeBSD nor Solaris touch provides anything
like this).  So I guess I'm now 50:50 (undecided on whether the
extension makes sense without more evidence of its usefulness).

--=20
Eric Blake   eblake@redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org


--------------enigBAAD9C5413CD72B75B906F4E
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Public key at http://people.redhat.com/eblake/eblake.gpg
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/

iQEcBAEBCAAGBQJNJ0GmAAoJEKeha0olJ0Nq7TcH/2aCVpX486FSN62te5Lgny4t
Cy02J5qZgF3ecFg8rh4rTdKeqggJmmsVKQqAfiPQ/jvKBx5vpsz/LBGEU6jdlMhp
LSaM/R59a7nU9rxcHj+z7v9FKujfkgRIiQuSNNpopbVtMlDLZEgAW83uZ+o9fauI
afd8vzxM37g20Jc4DAWKNC0aunLNSr0WsyYESFWY2Fx6dt3z7ZpS4LDDvKymz26/
D0Y3Spz3TAqRA7X3jVij4npzkHYCWrHdpbS92f5XNZcDwl9we3MmK46Xk8KRfTct
rqVom/kWloBNy1/Ise4oX80SXsGqdMfdffLR7bZ7WHhFpPrmwIJ1tHfWZFmFTQI=
=3pbZ
-----END PGP SIGNATURE-----

--------------enigBAAD9C5413CD72B75B906F4E--


From MAILER-DAEMON Fri Jan 07 12:06:47 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PbFm3-00017U-6a
	for mharc-coreutils@gnu.org; Fri, 07 Jan 2011 12:06:47 -0500
Received: from [140.186.70.92] (port=56634 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PbFlz-000122-H8
	for coreutils@gnu.org; Fri, 07 Jan 2011 12:06:45 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PbFlr-0003J3-0c
	for coreutils@gnu.org; Fri, 07 Jan 2011 12:06:36 -0500
Received: from mx.meyering.net ([82.230.74.64]:49340)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PbFlq-0003Ip-LM
	for coreutils@gnu.org; Fri, 07 Jan 2011 12:06:34 -0500
Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000)
	id E09126006A; Fri,  7 Jan 2011 18:06:32 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: Eric Blake <eblake@redhat.com>
Subject: Re: [coreutils] RFC for adding file creation mode feature into touch
	utility.
In-Reply-To: <4D2741A6.8060307@redhat.com> (Eric Blake's message of "Fri, 07
	Jan 2011 09:39:02 -0700")
References: <AANLkTinGw+2sqPfZsV=CNRiW5xBRVjXA6BYUFHNrBKpR@mail.gmail.com>
	<4D273001.6000807@redhat.com> <4D273E0B.6060203@draigBrady.com>
	<4D2741A6.8060307@redhat.com>
Date: Fri, 07 Jan 2011 18:06:32 +0100
Message-ID: <87wrmgve1z.fsf@meyering.net>
Lines: 55
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
Cc: rakib.mullick@gmail.com, =?utf-8?Q?P=C3=A1draig?= Brady <P@draigBrady.com>,
	coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 07 Jan 2011 17:06:46 -0000

Eric Blake wrote:
> On 01/07/2011 09:23 AM, P=C3=A1draig Brady wrote:
>>> Hmm - mkdir already supports -m/--mode, so it does sound like making
>>> touch support the same option might be a useful enhancement.  Of course,
>>> if the file exists, the --mode option would be ignored.  And POSIX
>>> requires that if the --mode option is not provided but a file is
>>> created, that the file have mode 0666 as modified by umask.  We can't
>>> use -m, though (POSIX already requires it for mtime).
>>
>> Why does mkdir have this option rather than relying on umask?
>> My guess is to close security races with sticky bits etc.
>> and that creating a file will not have the same issues?
>
> mkdir has -m because POSIX requires it; also, you are right that the
> setup of 'mkdir -p -m' creates a hierarchy where you want the
> permissions to be set correctly for the entire set for race avoidance.
> But touch does not create hierarchies.
>
>>
>> ...
>>> I'm 70:30 for adding this idea, and it's rather simple to do, as well
>>> (would still need doc, NEWS, and test additions).
>>
>> I'm a little less enthused to diverge from POSIX for this.
>
> It's not diverging from POSIX, so much as adding an extension.  It all
> boils down to whether the extension would prove useful, and whether it
> can optimize work flows faster than any existing alternative that gets
> the same end result but with more steps.  But thinking about it more,
> I'd rather use just the long-option --mode rather than burn a short
> option -M (or any other).
>
> As for creating a file with +x or with 07000 permissions already added,
> why?  Creating an empty file as already executable or setuid doesn't
> make much sense (the empty file will be a nop); why not wait until the
> file has final executable contents before adding +x or special
> permissions later?  So, that leaves only permissions matching of a
> reference file (and recent GNU cp already supports that), or restricting
> permissions less than touch's default of 0666 (but touch honors umask,
> so just modify your umask before touch), neither of which need a chmod
> after the fact, nor a --mode argument to touch.
>
> What would really clinch the deal is if any other touch implementation
> out there already has something like this.  But off the top of my head,
> I don't know of any (neither FreeBSD nor Solaris touch provides anything
> like this).  So I guess I'm now 50:50 (undecided on whether the
> extension makes sense without more evidence of its usefulness).

I confess that my first reaction was "why bother?"
Touch is nominally a very simple tool, and adding --mode support
feels like adding unwarranted bloat -- and not in line with
the one-job-one-tool philosophy.

Why would one prefer the GNU-specific touch --mode ...
over the portable combination of touch and chmod?


From MAILER-DAEMON Fri Jan 07 13:34:01 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PbH8T-00045z-Fn
	for mharc-coreutils@gnu.org; Fri, 07 Jan 2011 13:34:01 -0500
Received: from [140.186.70.92] (port=39655 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PbH8R-00044X-By
	for coreutils@gnu.org; Fri, 07 Jan 2011 13:34:00 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <kwzh@gnu.org>) id 1PbH8Q-0002wE-Ds
	for coreutils@gnu.org; Fri, 07 Jan 2011 13:33:59 -0500
Received: from fencepost.gnu.org ([140.186.70.10]:54814)
	by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <kwzh@gnu.org>)
	id 1PbH8Q-0002wA-B4
	for coreutils@gnu.org; Fri, 07 Jan 2011 13:33:58 -0500
Received: from kwzh by fencepost.gnu.org with local (Exim 4.69)
	(envelope-from <kwzh@gnu.org>)
	id 1PbH8M-0003PM-4L; Fri, 07 Jan 2011 13:33:54 -0500
From: kwzh@gnu.org (Karl Heuer)
To: Jim Meyering <jim@meyering.net>
Subject: Re: [coreutils] [PATCH] coreutils new feature: split --filter
Message-Id: <E1PbH8M-0003PM-4L@fencepost.gnu.org>
Date: Fri, 07 Jan 2011 13:33:54 -0500
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 07 Jan 2011 18:34:00 -0000

>I think we're ok on the copyright front, since split
>used to be part of the textutils and you have an assignment
>on file for that project.

Great.  I am, of course, OK with assigning copyright in any case.

>Do you feel like updating NEWS and doc/coreutils.texi and/or
>writing a test or two?

Sure.  I haven't looked at the format for new tests; it might save
some time if I just say that input FOO should produce output BAR,
and let someone else translate that into what's needed.  Meanwhile,
if someone else thinks of tests that should be included (or finds
a problem with my code as written), keep me in the loop to avoid
duplication of effort.

--Karl


From MAILER-DAEMON Fri Jan 07 14:31:55 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PbI2V-0006Lt-GQ
	for mharc-coreutils@gnu.org; Fri, 07 Jan 2011 14:31:55 -0500
Received: from [140.186.70.92] (port=55187 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PbI2S-0006KU-S8
	for coreutils@gnu.org; Fri, 07 Jan 2011 14:31:53 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PbI2R-0000dZ-RT
	for coreutils@gnu.org; Fri, 07 Jan 2011 14:31:52 -0500
Received: from mx.meyering.net ([82.230.74.64]:32886)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>)
	id 1PbI2R-0000dG-Lr; Fri, 07 Jan 2011 14:31:51 -0500
Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000)
	id 5A89560157; Fri,  7 Jan 2011 20:31:50 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: kwzh@gnu.org (Karl Heuer)
Subject: Re: [coreutils] [PATCH] coreutils new feature: split --filter
In-Reply-To: <E1PbH8M-0003PM-4L@fencepost.gnu.org> (Karl Heuer's message of
	"Fri, 07 Jan 2011 13:33:54 -0500")
References: <E1PbH8M-0003PM-4L@fencepost.gnu.org>
Date: Fri, 07 Jan 2011 20:31:50 +0100
Message-ID: <87fwt4v7bt.fsf@meyering.net>
Lines: 19
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 07 Jan 2011 19:31:54 -0000

Karl Heuer wrote:
>>I think we're ok on the copyright front, since split
>>used to be part of the textutils and you have an assignment
>>on file for that project.
>
> Great.  I am, of course, OK with assigning copyright in any case.
>
>>Do you feel like updating NEWS and doc/coreutils.texi and/or
>>writing a test or two?
>
> Sure.  I haven't looked at the format for new tests; it might save
> some time if I just say that input FOO should produce output BAR,
> and let someone else translate that into what's needed.  Meanwhile,
> if someone else thinks of tests that should be included (or finds
> a problem with my code as written), keep me in the loop to avoid
> duplication of effort.

You may want to use tests/sample-test as a starting point
and to take inspiration from the others in tests/misc/split-*.


From MAILER-DAEMON Sat Jan 08 00:11:05 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PbR4z-00041f-9q
	for mharc-coreutils@gnu.org; Sat, 08 Jan 2011 00:11:05 -0500
Received: from [140.186.70.92] (port=59950 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PbR4x-00041Y-PZ
	for coreutils@gnu.org; Sat, 08 Jan 2011 00:11:04 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <rakib.mullick@gmail.com>) id 1PbR4w-0000CK-3t
	for coreutils@gnu.org; Sat, 08 Jan 2011 00:11:03 -0500
Received: from mail-ww0-f49.google.com ([74.125.82.49]:57586)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <rakib.mullick@gmail.com>) id 1PbR4v-0000C3-Tg
	for coreutils@gnu.org; Sat, 08 Jan 2011 00:11:02 -0500
Received: by wwb17 with SMTP id 17so18603651wwb.30
	for <coreutils@gnu.org>; Fri, 07 Jan 2011 21:11:01 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma;
	h=domainkey-signature:mime-version:received:received:in-reply-to
	:references:date:message-id:subject:from:to:cc:content-type
	:content-transfer-encoding;
	bh=RxLsW+agApvyrnm0ANwAezdP6UvKvo0ytSItqeicdeY=;
	b=WVYO7iflWizOk183ZdPU5P9mfqX5sORuXBVra3ygsbKBBoF5pvHIa1h3XaCsv13Ztp
	MlurYTjI5QlXmLUwcCNBfa9jjcDlPFSjwrLUoY29kU21BckDngDdabPBSeUzcYjU8Gxt
	qFsJ5MsjG9O6OSMmzdHkHIwlzVKyQF1aL71cU=
DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma;
	h=mime-version:in-reply-to:references:date:message-id:subject:from:to
	:cc:content-type:content-transfer-encoding;
	b=hOBLnwBttUNL71xVecIM3sM4hmrEqXj6VNx8tWVpV9ZBg9+pVUVd0RMNCdNMwybRQ2
	X9VXDdDif1NWfNWCPbzSYNdCDBUHmLmM6+xcdLpFSR5k1j8AACmxioh7LDySKXzQHiNm
	T3Av9kJRGsmbNnY9aqPkGuiyIhlUWhzjgIin8=
MIME-Version: 1.0
Received: by 10.216.82.9 with SMTP id n9mr25575188wee.35.1294463461303; Fri,
	07 Jan 2011 21:11:01 -0800 (PST)
Received: by 10.216.72.143 with HTTP; Fri, 7 Jan 2011 21:11:01 -0800 (PST)
In-Reply-To: <87wrmgve1z.fsf@meyering.net>
References: <AANLkTinGw+2sqPfZsV=CNRiW5xBRVjXA6BYUFHNrBKpR@mail.gmail.com>
	<4D273001.6000807@redhat.com> <4D273E0B.6060203@draigBrady.com>
	<4D2741A6.8060307@redhat.com> <87wrmgve1z.fsf@meyering.net>
Date: Sat, 8 Jan 2011 11:11:01 +0600
Message-ID: <AANLkTini04wQ7g8sAexc9Kv886NHbz3NBF2+BAXNR-pv@mail.gmail.com>
Subject: Re: [coreutils] RFC for adding file creation mode feature into touch
	utility.
From: Rakib Mullick <rakib.mullick@gmail.com>
To: Jim Meyering <jim@meyering.net>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 2)
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sat, 08 Jan 2011 05:11:04 -0000

On Fri, Jan 7, 2011 at 11:06 PM, Jim Meyering <jim@meyering.net> wrote:
> Eric Blake wrote:
>> On 01/07/2011 09:23 AM, P=E1draig Brady wrote:
>>>> Hmm - mkdir already supports -m/--mode, so it does sound like making
>>>> touch support the same option might be a useful enhancement. =A0Of cou=
rse,
>>>> if the file exists, the --mode option would be ignored. =A0And POSIX
>>>> requires that if the --mode option is not provided but a file is
>>>> created, that the file have mode 0666 as modified by umask. =A0We can'=
t
>>>> use -m, though (POSIX already requires it for mtime).
>>>
>>> Why does mkdir have this option rather than relying on umask?
>>> My guess is to close security races with sticky bits etc.
>>> and that creating a file will not have the same issues?
>>
>> mkdir has -m because POSIX requires it; also, you are right that the
>> setup of 'mkdir -p -m' creates a hierarchy where you want the
>> permissions to be set correctly for the entire set for race avoidance.
>> But touch does not create hierarchies.
>>
>>>
>>> ...
>>>> I'm 70:30 for adding this idea, and it's rather simple to do, as well
>>>> (would still need doc, NEWS, and test additions).
>>>
>>> I'm a little less enthused to diverge from POSIX for this.
>>
>> It's not diverging from POSIX, so much as adding an extension. =A0It all
>> boils down to whether the extension would prove useful, and whether it
>> can optimize work flows faster than any existing alternative that gets
>> the same end result but with more steps. =A0But thinking about it more,
>> I'd rather use just the long-option --mode rather than burn a short
>> option -M (or any other).
>>
>> As for creating a file with +x or with 07000 permissions already added,
>> why? =A0Creating an empty file as already executable or setuid doesn't
>> make much sense (the empty file will be a nop); why not wait until the
>> file has final executable contents before adding +x or special
>> permissions later? =A0So, that leaves only permissions matching of a
>> reference file (and recent GNU cp already supports that), or restricting
>> permissions less than touch's default of 0666 (but touch honors umask,
>> so just modify your umask before touch), neither of which need a chmod
>> after the fact, nor a --mode argument to touch.
>>
>> What would really clinch the deal is if any other touch implementation
>> out there already has something like this. =A0But off the top of my head=
,
>> I don't know of any (neither FreeBSD nor Solaris touch provides anything
>> like this). =A0So I guess I'm now 50:50 (undecided on whether the
>> extension makes sense without more evidence of its usefulness).
>
> I confess that my first reaction was "why bother?"
> Touch is nominally a very simple tool, and adding --mode support
> feels like adding unwarranted bloat -- and not in line with
> the one-job-one-tool philosophy.

Yes, right. Then, touch only supposed to change file's timestamp. No?
But, touch also creates a file for us. Not only it creates a file for
us, when it creates a file it takes reference from other files. So,
althrough we're in philosophy of one-tool-do-one-job-well, there are
something else that we can get.

And if adding --mode option to touch gets really out of philosophy,
then I think we need a new tool to for creating file with all the
options that we might have want. That will truly help us to keep the
philosophy intact.

>
> Why would one prefer the GNU-specific touch --mode ...
> over the portable combination of touch and chmod?
>
Cause its easier to do 'touch --mode xxx filename' than 'touch
filename' and 'chmod xxx'. And Eric also pointed out that we will be
able to do 'touch -r a -M b' to create a referenced file with
different mode.

thanks,
rakib


From MAILER-DAEMON Sat Jan 08 01:06:39 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PbRwl-0005Q0-Bv
	for mharc-coreutils@gnu.org; Sat, 08 Jan 2011 01:06:39 -0500
Received: from [140.186.70.92] (port=43283 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PbRwh-0005LU-DZ
	for coreutils@gnu.org; Sat, 08 Jan 2011 01:06:36 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <bob@proulx.com>) id 1PbRwg-00083q-34
	for coreutils@gnu.org; Sat, 08 Jan 2011 01:06:35 -0500
Received: from joseki.proulx.com ([216.17.153.58]:35507)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <bob@proulx.com>) id 1PbRwf-00083U-Tz
	for coreutils@gnu.org; Sat, 08 Jan 2011 01:06:34 -0500
Received: from hysteria.proulx.com (hysteria.proulx.com [192.168.230.119])
	by joseki.proulx.com (Postfix) with ESMTP id 22BE521363;
	Fri,  7 Jan 2011 23:06:31 -0700 (MST)
Received: by hysteria.proulx.com (Postfix, from userid 1000)
	id 0D1202DD73; Fri,  7 Jan 2011 23:06:31 -0700 (MST)
Date: Fri, 7 Jan 2011 23:06:31 -0700
From: Bob Proulx <bob@proulx.com>
To: Rakib Mullick <rakib.mullick@gmail.com>
Subject: Re: [coreutils] RFC for adding file creation mode feature into
	touch utility.
Message-ID: <20110108060631.GA17899@hysteria.proulx.com>
Mail-Followup-To: Rakib Mullick <rakib.mullick@gmail.com>,
	coreutils@gnu.org
References: <AANLkTinGw+2sqPfZsV=CNRiW5xBRVjXA6BYUFHNrBKpR@mail.gmail.com>
	<4D273001.6000807@redhat.com> <4D273E0B.6060203@draigBrady.com>
	<4D2741A6.8060307@redhat.com> <87wrmgve1z.fsf@meyering.net>
	<AANLkTini04wQ7g8sAexc9Kv886NHbz3NBF2+BAXNR-pv@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <AANLkTini04wQ7g8sAexc9Kv886NHbz3NBF2+BAXNR-pv@mail.gmail.com>
User-Agent: Mutt/1.5.20 (2009-06-14)
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 1)
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sat, 08 Jan 2011 06:06:37 -0000

Rakib Mullick wrote:
> Yes, right. Then, touch only supposed to change file's timestamp. No?
> But, touch also creates a file for us. Not only it creates a file for
> us, when it creates a file it takes reference from other files.

Yes.  That is what touch does.

> So, althrough we're in philosophy of one-tool-do-one-job-well, there
> are something else that we can get.

What did you mean to say there?  That didn't parse for me.

It is also to not duplicate functionality in every tool.  Otherwise
every tool that creates files would need a --mode override.  That
would definitely be too much.

> And if adding --mode option to touch gets really out of philosophy,
> then I think we need a new tool to for creating file with all the
> options that we might have want. That will truly help us to keep the
> philosophy intact.

No.  That doesn't follow at all.  The Unix philosophy isn't just that
tools should do one thing and do it well but also that functionality
shouldn't be duplicated all over the place.  Creating yet another tool
to do the task doesn't make sense when you can touch and chmod
already.  That other tool you propose would be a duplication of
chmod.  And that is the problem here too.  Should every tool that
creates files have a --mode override?

And for that matter I have yet to see a rationale for using --mode
here in a real example that can't be satisified simply by using an
appropriate umask.

> > Why would one prefer the GNU-specific touch --mode ...
> > over the portable combination of touch and chmod?
>
> Cause its easier to do 'touch --mode xxx filename' than 'touch
> filename' and 'chmod xxx'.

I disagree.  Doing the touch followed by chmod is the more obvious
way.  And for many decades the standard way.  And therefore easier to
maintain.

> And Eric also pointed out that we will be able to do 'touch -r a -M
> b' to create a referenced file with different mode.

I am not strongly opposed to having this added to the code.  But to me
this looks like a solution looking for a problem to solve.  It is the
type of thing that if I ever saw actually used in a script that I
would immediately edit into their standard touch and chmod versions
without giving it a second thought.

Bob


From MAILER-DAEMON Sat Jan 08 01:44:28 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PbSXM-0000nf-GY
	for mharc-coreutils@gnu.org; Sat, 08 Jan 2011 01:44:28 -0500
Received: from [140.186.70.92] (port=51992 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PbSXK-0000nU-9s
	for coreutils@gnu.org; Sat, 08 Jan 2011 01:44:27 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <rakib.mullick@gmail.com>) id 1PbSXI-0004EU-BP
	for coreutils@gnu.org; Sat, 08 Jan 2011 01:44:25 -0500
Received: from mail-ww0-f49.google.com ([74.125.82.49]:46109)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <rakib.mullick@gmail.com>) id 1PbSXI-0004EO-5Y
	for coreutils@gnu.org; Sat, 08 Jan 2011 01:44:24 -0500
Received: by wwb17 with SMTP id 17so18634303wwb.30
	for <coreutils@gnu.org>; Fri, 07 Jan 2011 22:44:23 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma;
	h=domainkey-signature:mime-version:received:received:in-reply-to
	:references:date:message-id:subject:from:to:content-type
	:content-transfer-encoding;
	bh=ap5BZWx9uIoNRAed6LHxHmFYJxenFmH72xWV0lvjgII=;
	b=FSs03u+EXF9l6fcwbvol0gFh8cRrTbOf3EJvVKl0ulhu4u5OVkuu0d8yPXaB/Z+QEg
	Jt6cWH7+YT2854Iq7jFM7IQwQx87rCtTjI1wxICKxFGwtGvgy/6YzAVs26vSXOW7Crv/
	AvfsNo/RHogLoEt/VqEBNlAkX+x3Ouq8r9zAk=
DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma;
	h=mime-version:in-reply-to:references:date:message-id:subject:from:to
	:content-type:content-transfer-encoding;
	b=xbTj+YMezvgeRssUtd0sRJAbS/bund7yiIE351i8Zc/kJhYDJxHAOGFS+Lwz2kDEBE
	ISX9Wt2q7fch88PTyjchkYenXg6od335ejtGf8SCmKqir04/ts07TPRX+VHuXNoOMtql
	6vjUDlpewTzVpI4P3IzsrE5SQyfrLO2C7m2MI=
MIME-Version: 1.0
Received: by 10.216.173.7 with SMTP id u7mr64324wel.50.1294469063243; Fri, 07
	Jan 2011 22:44:23 -0800 (PST)
Received: by 10.216.72.143 with HTTP; Fri, 7 Jan 2011 22:44:23 -0800 (PST)
In-Reply-To: <20110108060631.GA17899@hysteria.proulx.com>
References: <AANLkTinGw+2sqPfZsV=CNRiW5xBRVjXA6BYUFHNrBKpR@mail.gmail.com>
	<4D273001.6000807@redhat.com> <4D273E0B.6060203@draigBrady.com>
	<4D2741A6.8060307@redhat.com> <87wrmgve1z.fsf@meyering.net>
	<AANLkTini04wQ7g8sAexc9Kv886NHbz3NBF2+BAXNR-pv@mail.gmail.com>
	<20110108060631.GA17899@hysteria.proulx.com>
Date: Sat, 8 Jan 2011 12:44:23 +0600
Message-ID: <AANLkTimNdmj4ZbWYiEPxrU=K3OBXMQ7TtMvhhJ7Vygxy@mail.gmail.com>
Subject: Re: [coreutils] RFC for adding file creation mode feature into touch
	utility.
From: Rakib Mullick <rakib.mullick@gmail.com>
To: bob@proulx.com, coreutils@gnu.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 2)
Cc: 
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sat, 08 Jan 2011 06:44:27 -0000

On Sat, Jan 8, 2011 at 12:06 PM, Bob Proulx <bob@proulx.com> wrote:
> Rakib Mullick wrote:
>> Yes, right. Then, touch only supposed to change file's timestamp. No?
>> But, touch also creates a file for us. Not only it creates a file for
>> us, when it creates a file it takes reference from other files.
>
> Yes. =A0That is what touch does.
>
>> So, althrough we're in philosophy of one-tool-do-one-job-well, there
>> are something else that we can get.
>
> What did you mean to say there? =A0That didn't parse for me.

Don't get me wrong. I do have full respect for this philosophy. I
wanted to mean that, there are some extra functionality we often
export for our good.

>
> It is also to not duplicate functionality in every tool. =A0Otherwise
> every tool that creates files would need a --mode override. =A0That
> would definitely be too much.
>
I'm not sure how many tools out there for creating file (except those
for creating special files, tmp files etc.). I think touch is mostly
used for creating general purpose file and that is why I proposed to
have touch this kind of option.

>> And if adding --mode option to touch gets really out of philosophy,
>> then I think we need a new tool to for creating file with all the
>> options that we might have want. That will truly help us to keep the
>> philosophy intact.
>
> No. =A0That doesn't follow at all. =A0The Unix philosophy isn't just that
> tools should do one thing and do it well but also that functionality
> shouldn't be duplicated all over the place. =A0Creating yet another tool
> to do the task doesn't make sense when you can touch and chmod
> already. =A0That other tool you propose would be a duplication of
> chmod. =A0And that is the problem here too. =A0Should every tool that
> creates files have a --mode override?
>
I'm not sure which other tools you are reffering. mkdir, mknod already
have --mode option and in my subject I have clearly specify about
touch utility.

> And for that matter I have yet to see a rationale for using --mode
> here in a real example that can't be satisified simply by using an
> appropriate umask.
>
>> > Why would one prefer the GNU-specific touch --mode ...
>> > over the portable combination of touch and chmod?
>>
>> Cause its easier to do 'touch --mode xxx filename' than 'touch
>> filename' and 'chmod xxx'.
>
> I disagree. =A0Doing the touch followed by chmod is the more obvious
> way. =A0And for many decades the standard way. =A0And therefore easier to
> maintain.
>
If you disagree, I really don't have to say anything. And I also
disagree with your obvious way. Utility should something that reflects
user needs, offcourse maintaining all the standards. Programmers does
the tough job to make user's life easier, not how a programmer can
make a life harder for user.

>> And Eric also pointed out that we will be able to do 'touch -r a -M
>> b' to create a referenced file with different mode.
>
> I am not strongly opposed to having this added to the code. =A0But to me
> this looks like a solution looking for a problem to solve. =A0It is the
> type of thing that if I ever saw actually used in a script that I
> would immediately edit into their standard touch and chmod versions
> without giving it a second thought.
>
I was pointing to the advantage of adding this feature. Maybe we'll
find more when it will be in action. :)


thanks,
rakib
> Bob
>


From MAILER-DAEMON Sat Jan 08 04:02:20 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PbUgl-0007Gt-Uo
	for mharc-coreutils@gnu.org; Sat, 08 Jan 2011 04:02:19 -0500
Received: from [140.186.70.92] (port=51297 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PbUgi-0007GZ-NY
	for coreutils@gnu.org; Sat, 08 Jan 2011 04:02:17 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PbUgh-00084W-1R
	for coreutils@gnu.org; Sat, 08 Jan 2011 04:02:16 -0500
Received: from mx.meyering.net ([82.230.74.64]:39925)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PbUgg-000846-Rp
	for coreutils@gnu.org; Sat, 08 Jan 2011 04:02:15 -0500
Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000)
	id 1169760109; Sat,  8 Jan 2011 10:02:12 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: Rakib Mullick <rakib.mullick@gmail.com>
Subject: Re: [coreutils] RFC for adding file creation mode feature into touch
	utility.
In-Reply-To: <AANLkTini04wQ7g8sAexc9Kv886NHbz3NBF2+BAXNR-pv@mail.gmail.com>
	(Rakib Mullick's message of "Sat, 8 Jan 2011 11:11:01 +0600")
References: <AANLkTinGw+2sqPfZsV=CNRiW5xBRVjXA6BYUFHNrBKpR@mail.gmail.com>
	<4D273001.6000807@redhat.com> <4D273E0B.6060203@draigBrady.com>
	<4D2741A6.8060307@redhat.com> <87wrmgve1z.fsf@meyering.net>
	<AANLkTini04wQ7g8sAexc9Kv886NHbz3NBF2+BAXNR-pv@mail.gmail.com>
Date: Sat, 08 Jan 2011 10:02:11 +0100
Message-ID: <874o9jvkdo.fsf@meyering.net>
Lines: 61
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sat, 08 Jan 2011 09:02:18 -0000

Rakib Mullick wrote:
>> I confess that my first reaction was "why bother?"
>> Touch is nominally a very simple tool, and adding --mode support
>> feels like adding unwarranted bloat -- and not in line with
>> the one-job-one-tool philosophy.
>
> Yes, right. Then, touch only supposed to change file's timestamp. No?
> But, touch also creates a file for us. Not only it creates a file for
> us, when it creates a file it takes reference from other files. So,
> althrough we're in philosophy of one-tool-do-one-job-well, there are
> something else that we can get.
>
> And if adding --mode option to touch gets really out of philosophy,
> then I think we need a new tool to for creating file with all the
> options that we might have want. That will truly help us to keep the
> philosophy intact.

?

When the utility of a new feature is not obvious, the proposal for
that new feature must be accompanied by significant justification,
and so far, the argument that "adding this option to touch saves us the
trouble of invoking chmod" is insufficient.

Typical "good" justification usually goes like this:

    I want to do X using POSIX tools, and the portable way to do it is
    too cumbersome, but if I add this option to one of the coreutils,
    it becomes much easier.  Here is how to do X using current tools:

        ...list of actual commands (inefficient, cumbersome, etc)...

    and now, do X using the program with the proposed change:

        ...list of commands demonstrating benefits of proposed change...

Then, if there is no reasonably simple/concise way to solve the problem
with some other tool, we evaluate whether some coreutils program
should be extended to do the job.

>> Why would one prefer the GNU-specific touch --mode ...
>> over the portable combination of touch and chmod?
>>
> Cause its easier to do 'touch --mode xxx filename' than 'touch
> filename' and 'chmod xxx'.

As mentioned above, that is not enough justification.

> And Eric also pointed out that we will be
> able to do 'touch -r a -M b' to create a referenced file with
> different mode.

Let's back up a step...
Why do you want an option for this when you can already use touch
to create a file with selected permissions via your umask?
If you say that you require an execute bit or some special bit to
be set, please explain why.  I do not see how having such bits set
on an empty file would be useful.  If you're about to add content,
then the time stamps set by touch will be overwritten, and you might as
well have created the file via the shell, (if there was only one) say
with ": > FILE".


From MAILER-DAEMON Sat Jan 08 11:30:45 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1Pbbgj-0000vY-FU
	for mharc-coreutils@gnu.org; Sat, 08 Jan 2011 11:30:45 -0500
Received: from [140.186.70.92] (port=46548 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1Pbbgh-0000uk-Ud
	for coreutils@gnu.org; Sat, 08 Jan 2011 11:30:45 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <rakib.mullick@gmail.com>) id 1Pbbgg-0003Sn-K2
	for coreutils@gnu.org; Sat, 08 Jan 2011 11:30:43 -0500
Received: from mail-wy0-f169.google.com ([74.125.82.169]:59714)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <rakib.mullick@gmail.com>) id 1Pbbgg-0003Sg-Fo
	for coreutils@gnu.org; Sat, 08 Jan 2011 11:30:42 -0500
Received: by wyj26 with SMTP id 26so19112079wyj.0
	for <coreutils@gnu.org>; Sat, 08 Jan 2011 08:30:41 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma;
	h=domainkey-signature:mime-version:received:received:in-reply-to
	:references:date:message-id:subject:from:to:cc:content-type
	:content-transfer-encoding;
	bh=Z+VlCNxtVyChSbpGpyiMMQENfOcgJHaazTpPKIFc4sc=;
	b=CpMIMNfQ8otcxtVMeI3lN2AGgomnmkZv++/ifyN2jnfWy58YW51BzODlkO7aogGwyN
	ic+fLxtRK1HaDeiXqImQyzoGYGrDRYKTQu85TWRX+TWIxRnNI64NlYYJeESoe+C3C3bg
	i4S6eQvqr+wDj2AWexbyGW5uhG7Ms6ohqMaUc=
DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma;
	h=mime-version:in-reply-to:references:date:message-id:subject:from:to
	:cc:content-type:content-transfer-encoding;
	b=AFBhj1uyCt/avTPz++JIs2bcwMdujJidnxU/UfBalLySf8/z7dMU6n8cq5C2O0zigw
	MIminW1I8AFtrgR8mhXQajQjlRaCADixek+NNTOGr74rSkXiW0HqE8Tx/FeHIjTMpvY3
	wmHCOMNUN/OlWbWFO34pHfPTQveQg2uy6KQ8Y=
MIME-Version: 1.0
Received: by 10.216.139.224 with SMTP id c74mr16754091wej.50.1294504241045;
	Sat, 08 Jan 2011 08:30:41 -0800 (PST)
Received: by 10.216.72.143 with HTTP; Sat, 8 Jan 2011 08:30:40 -0800 (PST)
In-Reply-To: <874o9jvkdo.fsf@meyering.net>
References: <AANLkTinGw+2sqPfZsV=CNRiW5xBRVjXA6BYUFHNrBKpR@mail.gmail.com>
	<4D273001.6000807@redhat.com> <4D273E0B.6060203@draigBrady.com>
	<4D2741A6.8060307@redhat.com> <87wrmgve1z.fsf@meyering.net>
	<AANLkTini04wQ7g8sAexc9Kv886NHbz3NBF2+BAXNR-pv@mail.gmail.com>
	<874o9jvkdo.fsf@meyering.net>
Date: Sat, 8 Jan 2011 22:30:40 +0600
Message-ID: <AANLkTimkonRUED8c_bFvzaLdPDjJjChJ6CSw736At5wq@mail.gmail.com>
Subject: Re: [coreutils] RFC for adding file creation mode feature into touch
	utility.
From: Rakib Mullick <rakib.mullick@gmail.com>
To: Jim Meyering <jim@meyering.net>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 2)
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sat, 08 Jan 2011 16:30:45 -0000

On Sat, Jan 8, 2011 at 3:02 PM, Jim Meyering <jim@meyering.net> wrote:
> Rakib Mullick wrote:
>>> I confess that my first reaction was "why bother?"
>>> Touch is nominally a very simple tool, and adding --mode support
>>> feels like adding unwarranted bloat -- and not in line with
>>> the one-job-one-tool philosophy.
>>
>> Yes, right. Then, touch only supposed to change file's timestamp. No?
>> But, touch also creates a file for us. Not only it creates a file for
>> us, when it creates a file it takes reference from other files. So,
>> althrough we're in philosophy of one-tool-do-one-job-well, there are
>> something else that we can get.
>>
>> And if adding --mode option to touch gets really out of philosophy,
>> then I think we need a new tool to for creating file with all the
>> options that we might have want. That will truly help us to keep the
>> philosophy intact.
>
> ?
>
> When the utility of a new feature is not obvious, the proposal for
> that new feature must be accompanied by significant justification,
> and so far, the argument that "adding this option to touch saves us the
> trouble of invoking chmod" is insufficient.
>
> Typical "good" justification usually goes like this:
>
> =A0 =A0I want to do X using POSIX tools, and the portable way to do it is
> =A0 =A0too cumbersome, but if I add this option to one of the coreutils,
> =A0 =A0it becomes much easier. =A0Here is how to do X using current tools=
:
>
> =A0 =A0 =A0 =A0...list of actual commands (inefficient, cumbersome, etc).=
..
>
> =A0 =A0and now, do X using the program with the proposed change:
>
> =A0 =A0 =A0 =A0...list of commands demonstrating benefits of proposed cha=
nge...
>
> Then, if there is no reasonably simple/concise way to solve the problem
> with some other tool, we evaluate whether some coreutils program
> should be extended to do the job.
>
>>> Why would one prefer the GNU-specific touch --mode ...
>>> over the portable combination of touch and chmod?
>>>
>> Cause its easier to do 'touch --mode xxx filename' than 'touch
>> filename' and 'chmod xxx'.
>
> As mentioned above, that is not enough justification.
>
>> And Eric also pointed out that we will be
>> able to do 'touch -r a -M b' to create a referenced file with
>> different mode.
>
> Let's back up a step...
> Why do you want an option for this when you can already use touch
> to create a file with selected permissions via your umask?

I think umask change the the file creation mask system wide. I might
not want it for every file I create.

> If you say that you require an execute bit or some special bit to
> be set, please explain why. =A0I do not see how having such bits set
> on an empty file would be useful. =A0If you're about to add content,

No. It's not just about setting an execute bit, say I might have
create a file with 'sudo touch filename', in that case I just can see
the file by doing 'cat filename' or 'vim filename'. So, again I need
to do, 'sudo chmod xxx filename' to make it available to its user. So,
it can be summed up as:

(Assume, to create a file under /usr/src/work)
             sudo touch /usr/src/work/filename (to create a file)
             sudo chmod xxx /usr/src/work/filename (to give a user
proper permission)

After adding this feature we can do this:
            sudo touch --mode xxx /usr/src/work/filename

thats it.

> then the time stamps set by touch will be overwritten, and you might as
> well have created the file via the shell, (if there was only one) say
> with ": > FILE".
>


From MAILER-DAEMON Sat Jan 08 12:22:19 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PbcUd-0004a3-0H
	for mharc-coreutils@gnu.org; Sat, 08 Jan 2011 12:22:19 -0500
Received: from [140.186.70.92] (port=60854 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PbcUJ-0004Wc-Gz
	for coreutils@gnu.org; Sat, 08 Jan 2011 12:22:17 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PbcU1-0004DH-OD
	for coreutils@gnu.org; Sat, 08 Jan 2011 12:21:59 -0500
Received: from mx.meyering.net ([82.230.74.64]:33951)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PbcU1-0004D7-7y
	for coreutils@gnu.org; Sat, 08 Jan 2011 12:21:41 -0500
Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000)
	id 5CAC9600B5; Sat,  8 Jan 2011 18:21:39 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: Jonathan Nieder <jrnieder@gmail.com>
In-Reply-To: <20110105193634.GA22486@burratino> (Jonathan Nieder's message of
	"Wed, 5 Jan 2011 13:36:34 -0600")
References: <20110105193634.GA22486@burratino>
Date: Sat, 08 Jan 2011 18:21:39 +0100
Message-ID: <87lj2vtios.fsf@meyering.net>
Lines: 119
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
Cc: 609049@bugs.debian.org, coreutils@gnu.org
Subject: [coreutils] Re: Bug#609049: du: process_file: Assertion `level ==
	prev_level - 1' failed
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sat, 08 Jan 2011 17:22:17 -0000

Jonathan Nieder wrote:
> Package: coreutils
> Version: 8.5-1
>
> Happy new year.  Today I ran
>
>  $ du -sk ~/*
>
> to see where all the space is going.  The traversal almost certainly
> was in ~/src, and being bored, I suspended it to examine that
> directory:
>
>  ^Z
>  [1]+  Stopped                 du -sk ~/*
>
> In another terminal I ran "cd ~/src && du -sk *".  ~/src/netbsd was
> very big, so I moved it (using chmod +w first) to ~/.cache on the same
> filesystem for future reference.  And now time to resume the toplevel
> search:
>
>  $ fg
>  du -sk ~/*
>  du: fts_read failed: No such file or directory
>  du: du.c:583: process_file: Assertion `level == prev_level - 1' failed.
>  Aborted (core dumped)
>
> Lacking debugging symbols, the core dump does not seem so useful, but
> I still have it.  libc6 is 2.11.2-7, filesystem is ext4, and the
> kernel is 2.6.37-rc7-686.  If there is additional information I could
> provide, just ask.

Thank you for the fine bug report.
Here's a fix:

While I do have a reproducer that will become a test suite addition
(coming soon), it relies on python (a first) and the python-inotify
package, so I'll have to be careful to skip the test when those
prerequisites aren't installed.  Also, there's an inherent race condition,
so I'll have to find the right compromise between absolute test-robustness
(too expensive in time and inodes) and reasonableness.

>From 92bbec0f697575a8bcbbe3fe0cb54b468c1022a4 Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering@redhat.com>
Date: Sat, 8 Jan 2011 17:44:55 +0100
Subject: [PATCH] du: don't abort when a subdir is renamed out from under us

* NEWS (Bug fixes): Mention it.
* src/du.c (prev_level): Move declaration "up" to file-scope global.
(du_files): Reset prev_level to 0 upon abnormal fts_read termination.
Also improve a diagnostic.
Reported by Johathan Nieder in http://bugs.debian.org/609049
---
 NEWS     |    7 +++++++
 src/du.c |   15 +++++++++++----
 2 files changed, 18 insertions(+), 4 deletions(-)

diff --git a/NEWS b/NEWS
index 2a71ca6..793a8e6 100644
--- a/NEWS
+++ b/NEWS
@@ -2,6 +2,13 @@ GNU coreutils NEWS                                    -*- outline -*-

 * Noteworthy changes in release ?.? (????-??-??) [?]

+** Bug fixes
+
+  du could abort with a failed assertion when part of the hierarchy being
+  traversed was moved to a higher level in the directory tree, and when
+  there was at least one more command line directory argument following
+  the one containing the moved sub-tree.
+

 * Noteworthy changes in release 8.9 (2011-01-04) [stable]

diff --git a/src/du.c b/src/du.c
index 77deb0c..4dc5456 100644
--- a/src/du.c
+++ b/src/du.c
@@ -63,8 +63,11 @@ extern bool fts_debug;
 /* A set of dev/ino pairs.  */
 static struct di_set *di_set;

-/* Define a class for collecting directory information. */
+/* Keep track of the preceding "level" (depth in hierarchy)
+   from one call of process_file to the next.  */
+static size_t prev_level;

+/* Define a class for collecting directory information. */
 struct duinfo
 {
   /* Size of files in directory.  */
@@ -399,7 +402,6 @@ process_file (FTS *fts, FTSENT *ent)
   struct duinfo dui;
   struct duinfo dui_to_print;
   size_t level;
-  static size_t prev_level;
   static size_t n_alloc;
   /* First element of the structure contains:
      The sum of the st_size values of all entries in the single directory
@@ -582,10 +584,15 @@ du_files (char **files, int bit_flags)
             {
               if (errno != 0)
                 {
-                  /* FIXME: try to give a better message  */
-                  error (0, errno, _("fts_read failed"));
+                  error (0, errno, _("%s: fts_read failed"),
+                         quotearg_colon (fts->fts_path));
                   ok = false;
                 }
+
+              /* When exiting this loop early, be careful to reset the
+                 global, prev_level, used in process_file.  Otherwise, its
+                 (level == prev_level - 1) assertion could fail.  */
+              prev_level = 0;
               break;
             }
           FTS_CROSS_CHECK (fts);
--
1.7.3.5


From MAILER-DAEMON Sat Jan 08 20:51:08 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PbkR2-0002gR-KR
	for mharc-coreutils@gnu.org; Sat, 08 Jan 2011 20:51:08 -0500
Received: from [140.186.70.92] (port=39520 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PbkR0-0002fi-CL
	for coreutils@gnu.org; Sat, 08 Jan 2011 20:51:07 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <kwzh@gnu.org>) id 1PbkQz-0003IC-8H
	for coreutils@gnu.org; Sat, 08 Jan 2011 20:51:06 -0500
Received: from fencepost.gnu.org ([140.186.70.10]:49452)
	by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <kwzh@gnu.org>)
	id 1PbkQz-0003I8-1J
	for coreutils@gnu.org; Sat, 08 Jan 2011 20:51:05 -0500
Received: from kwzh by fencepost.gnu.org with local (Exim 4.69)
	(envelope-from <kwzh@gnu.org>)
	id 1PbkQu-0001e9-UU; Sat, 08 Jan 2011 20:51:01 -0500
From: kwzh@gnu.org (Karl Heuer)
To: coreutils@gnu.org
Content-Type: multipart/mixed; boundary="nextpart-32127"
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
Message-Id: <E1PbkQu-0001e9-UU@fencepost.gnu.org>
Date: Sat, 08 Jan 2011 20:51:01 -0500
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
Subject: [coreutils] [PATCH] coreutils buglets, useless-if-before-free
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 09 Jan 2011 01:51:07 -0000

This is a multi-part message in MIME format.

--nextpart-32127
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

`useless-if-before-free --help' prints a literal
NUL character instead of the backslash-zero that
was intended.  Also, the "must lie within the
first 8 lines" line is on line 9, and hence not
getting automatically updated.  This patch fixes
the former by adding a backslash, and the latter
by condensing the three lines of what-it-does to
a single line, leaving one line of slack for the
future.  (An alternative would be to move the
what-it-does text to appear below the VERSION.)

--nextpart-32127
Content-Type: text/x-diff; name="useless-if.patch"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="useless-if.patch"

KioqIGNvcmV1dGlscy04LjkvYnVpbGQtYXV4L3VzZWxlc3MtaWYtYmVmb3JlLWZyZWV+CVNh
dCBKYW4gIDEgMTM6MTk6MjcgMjAxMQotLS0gY29yZXV0aWxzLTguOS9idWlsZC1hdXgvdXNl
bGVzcy1pZi1iZWZvcmUtZnJlZQlTYXQgSmFuICA4IDE3OjM5OjU3IDIwMTEKKioqKioqKioq
KioqKioqCioqKiAyLDEyICoqKioKICAgICYgZXZhbCAnZXhlYyBwZXJsIC13U1QgIiQwIiAk
YXJndjpxJwogICAgICBpZiAwOwogICMgRGV0ZWN0IGluc3RhbmNlcyBvZiAiaWYgKHApIGZy
ZWUgKHApOyIuCiEgIyBMaWtld2lzZSBmb3IgImlmIChwICE9IE5VTEwpIGZyZWUgKHApOyIu
ICBBbmQgd2l0aCBicmFjZXMuCiEgIyBBbHNvIGRldGVjdCAiaWYgKE5VTEwgIT0gcCkgZnJl
ZSAocCk7Ii4KISAjIEFuZCB3aXRoIDAgaW4gcGxhY2Ugb2YgTlVMTC4KICAKISBteSAkVkVS
U0lPTiA9ICcyMDA5LTA0LTE2IDE1OjU3JzsgIyBVVEMKICAjIFRoZSBkZWZpbml0aW9uIGFi
b3ZlIG11c3QgbGllIHdpdGhpbiB0aGUgZmlyc3QgOCBsaW5lcyBpbiBvcmRlcgogICMgZm9y
IHRoZSBFbWFjcyB0aW1lLXN0YW1wIHdyaXRlIGhvb2sgKGF0IGVuZCkgdG8gdXBkYXRlIGl0
LgogICMgSWYgeW91IGNoYW5nZSB0aGlzIGZpbGUgd2l0aCBFbWFjcywgcGxlYXNlIGxldCB0
aGUgd3JpdGUgaG9vawotLS0gMiwxMCAtLS0tCiAgICAmIGV2YWwgJ2V4ZWMgcGVybCAtd1NU
ICIkMCIgJGFyZ3Y6cScKICAgICAgaWYgMDsKICAjIERldGVjdCBpbnN0YW5jZXMgb2YgImlm
IChwKSBmcmVlIChwKTsiLgohICMgTGlrZXdpc2UgImlmIChwICE9IDApIiwgImlmICgwICE9
IHApIiwgb3Igd2l0aCBOVUxMOyBhbmQgd2l0aCBicmFjZXMuCiAgCiEgbXkgJFZFUlNJT04g
PSAnMjAxMS0wMS0wOSAwMTozOSc7ICMgVVRDCiAgIyBUaGUgZGVmaW5pdGlvbiBhYm92ZSBt
dXN0IGxpZSB3aXRoaW4gdGhlIGZpcnN0IDggbGluZXMgaW4gb3JkZXIKICAjIGZvciB0aGUg
RW1hY3MgdGltZS1zdGFtcCB3cml0ZSBob29rIChhdCBlbmQpIHRvIHVwZGF0ZSBpdC4KICAj
IElmIHlvdSBjaGFuZ2UgdGhpcyBmaWxlIHdpdGggRW1hY3MsIHBsZWFzZSBsZXQgdGhlIHdy
aXRlIGhvb2sKKioqKioqKioqKioqKioqCioqKiA2Myw2OSAqKioqCiAgCiAgT1BUSU9OUzoK
ICAKISAgICAtLWxpc3QgICAgICAgcHJpbnQgb25seSB0aGUgbmFtZSBvZiBlYWNoIG1hdGNo
aW5nIEZJTEUgKFwwLXRlcm1pbmF0ZWQpCiAgICAgLS1uYW1lPU4gICAgIGFkZCBuYW1lIE4g
dG8gdGhlIGxpc3Qgb2YgXGBmcmVlXCctbGlrZSBmdW5jdGlvbnMgdG8gZGV0ZWN0OwogICAg
ICAgICAgICAgICAgICAgIG1heSBiZSByZXBlYXRlZAogIAotLS0gNjEsNjcgLS0tLQogIAog
IE9QVElPTlM6CiAgCiEgICAgLS1saXN0ICAgICAgIHByaW50IG9ubHkgdGhlIG5hbWUgb2Yg
ZWFjaCBtYXRjaGluZyBGSUxFIChcXDAtdGVybWluYXRlZCkKICAgICAtLW5hbWU9TiAgICAg
YWRkIG5hbWUgTiB0byB0aGUgbGlzdCBvZiBcYGZyZWVcJy1saWtlIGZ1bmN0aW9ucyB0byBk
ZXRlY3Q7CiAgICAgICAgICAgICAgICAgICAgbWF5IGJlIHJlcGVhdGVkCiAgCg==

--nextpart-32127--


From MAILER-DAEMON Sun Jan 09 04:28:22 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PbrZV-0008RZ-TH
	for mharc-coreutils@gnu.org; Sun, 09 Jan 2011 04:28:22 -0500
Received: from [140.186.70.92] (port=48032 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PbrZT-0008Qp-28
	for coreutils@gnu.org; Sun, 09 Jan 2011 04:28:20 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PbrZR-0001It-Vs
	for coreutils@gnu.org; Sun, 09 Jan 2011 04:28:18 -0500
Received: from mx.meyering.net ([82.230.74.64]:44422)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>)
	id 1PbrZO-0001IK-Io; Sun, 09 Jan 2011 04:28:15 -0500
Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000)
	id 46B5B60261; Sun,  9 Jan 2011 10:28:12 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: kwzh@gnu.org (Karl Heuer)
Subject: Re: [coreutils] [PATCH] coreutils buglets, useless-if-before-free
In-Reply-To: <E1PbkQu-0001e9-UU@fencepost.gnu.org> (Karl Heuer's message of
	"Sat, 08 Jan 2011 20:51:01 -0500")
References: <E1PbkQu-0001e9-UU@fencepost.gnu.org>
Date: Sun, 09 Jan 2011 10:28:12 +0100
Message-ID: <87fwt2toib.fsf@meyering.net>
Lines: 109
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
Cc: bug-gnulib@gnu.org, coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 09 Jan 2011 09:28:20 -0000

Karl Heuer wrote:
> `useless-if-before-free --help' prints a literal
> NUL character instead of the backslash-zero that
> was intended.  Also, the "must lie within the
> first 8 lines" line is on line 9, and hence not
> getting automatically updated.  This patch fixes
> the former by adding a backslash, and the latter
> by condensing the three lines of what-it-does to
> a single line, leaving one line of slack for the
> future.  (An alternative would be to move the
> what-it-does text to appear below the VERSION.)

Thanks, Karl!
FYI, I've pushed that to the gnulib.git repository,
since that's the canonical source for that particular file.
I massaged your summary into a ChangeLog entry and
"git commit"ted it.  Once I'd done that, I ran:

  git format-patch --stdout -1 > FILE

to produce the patch you see below, which includes authorship
information (your name and email) and a commit log message.

We prefer to receive patches of the form below because then
we don't have to write as much and can simply run

  git am FILE

to commit the patch to our local repository.

If you're new to git, here's a quick intro:
  http://git.sv.gnu.org/cgit/coreutils.git/tree/README-hacking

Your patch counts as trivial, so I don't need a gnulib copyright
assignment just yet, but if you think you will be submitting other
changes to gnulib, please fill out the copyright assignment form for
gnulib, and you might as well add coreutils (you can do 4 at once):

  git.sv.gnu.org/gitweb/?p=gnulib.git;f=doc/Copyright/request-assign.future;hb=HEAD
  http://www.gnu.org/prep/maintain/html_node/Copyright-Papers.html


>From 1b58d734924189cf963936e202f361dd19bd109c Mon Sep 17 00:00:00 2001
From: Karl Heuer <kwzh@gnu.org>
Date: Sun, 9 Jan 2011 10:16:08 +0100
Subject: [PATCH] useless-if-before-free: fix typo in --help and make the internal,

automatic version date update process work once again.
--help output contained a NUL character instead of the
backslash-zero that was intended.  Also, the "must lie within
the first 8 lines" line is on line 9, and hence not getting
automatically updated.
* build-aux/useless-if-before-free: Fix the former by adding a
backslash, and the latter by condensing the three lines of what-it-does
to a single line, leaving one line of slack for the future.
---
 ChangeLog                        |   12 ++++++++++++
 build-aux/useless-if-before-free |    8 +++-----
 2 files changed, 15 insertions(+), 5 deletions(-)

diff --git a/ChangeLog b/ChangeLog
index eb0d8bd..a2070af 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,15 @@
+2011-01-09  Karl Heuer  <kwzh@gnu.org>
+
+	useless-if-before-free: fix typo in --help and make the internal,
+	automatic version date update process work once again.
+	--help output contained a NUL character instead of the
+	backslash-zero that was intended.  Also, the "must lie within
+	the first 8 lines" line is on line 9, and hence not getting
+	automatically updated.
+	* build-aux/useless-if-before-free: Fix the former by adding a
+	backslash, and the latter by condensing the three lines of what-it-does
+	to a single line, leaving one line of slack for the future.
+
 2011-01-08  Bruno Haible  <bruno@clisp.org>

 	uninorm tests: Preserve copyright of Unicode data file.
diff --git a/build-aux/useless-if-before-free b/build-aux/useless-if-before-free
index 01a6d3d..a6c228b 100755
--- a/build-aux/useless-if-before-free
+++ b/build-aux/useless-if-before-free
@@ -2,11 +2,9 @@ eval '(exit $?0)' && eval 'exec perl -wST "$0" ${1+"$@"}'
   & eval 'exec perl -wST "$0" $argv:q'
     if 0;
 # Detect instances of "if (p) free (p);".
-# Likewise for "if (p != NULL) free (p);".  And with braces.
-# Also detect "if (NULL != p) free (p);".
-# And with 0 in place of NULL.
+# Likewise "if (p != 0)", "if (0 != p)", or with NULL; and with braces.

-my $VERSION = '2009-04-16 15:57'; # UTC
+my $VERSION = '2011-01-09 01:39'; # UTC
 # The definition above must lie within the first 8 lines in order
 # for the Emacs time-stamp write hook (at end) to update it.
 # If you change this file with Emacs, please let the write hook
@@ -63,7 +61,7 @@ detect free-like functions named FOO and BAR.

 OPTIONS:

-   --list       print only the name of each matching FILE (\0-terminated)
+   --list       print only the name of each matching FILE (\\0-terminated)
    --name=N     add name N to the list of \`free\'-like functions to detect;
                   may be repeated

--
1.7.3.5


From MAILER-DAEMON Sun Jan 09 14:03:22 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1Pc0Xy-00066h-Rn
	for mharc-coreutils@gnu.org; Sun, 09 Jan 2011 14:03:22 -0500
Received: from [140.186.70.92] (port=54437 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1Pc0Xx-00066Z-3c
	for coreutils@gnu.org; Sun, 09 Jan 2011 14:03:22 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <bob@proulx.com>) id 1Pc0Xv-0000Ct-FV
	for coreutils@gnu.org; Sun, 09 Jan 2011 14:03:20 -0500
Received: from joseki.proulx.com ([216.17.153.58]:53326)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <bob@proulx.com>) id 1Pc0Xu-0000Cb-WB
	for coreutils@gnu.org; Sun, 09 Jan 2011 14:03:19 -0500
Received: from hysteria.proulx.com (hysteria.proulx.com [192.168.230.119])
	by joseki.proulx.com (Postfix) with ESMTP id 018F821311;
	Sun,  9 Jan 2011 12:03:16 -0700 (MST)
Received: by hysteria.proulx.com (Postfix, from userid 1000)
	id E3B532DD80; Sun,  9 Jan 2011 12:03:15 -0700 (MST)
Date: Sun, 9 Jan 2011 12:03:15 -0700
From: Bob Proulx <bob@proulx.com>
To: Rakib Mullick <rakib.mullick@gmail.com>
Subject: Re: [coreutils] RFC for adding file creation mode feature into
	touch utility.
Message-ID: <20110109190315.GA15320@hysteria.proulx.com>
Mail-Followup-To: Rakib Mullick <rakib.mullick@gmail.com>,
	coreutils@gnu.org
References: <AANLkTinGw+2sqPfZsV=CNRiW5xBRVjXA6BYUFHNrBKpR@mail.gmail.com>
	<4D273001.6000807@redhat.com> <4D273E0B.6060203@draigBrady.com>
	<4D2741A6.8060307@redhat.com> <87wrmgve1z.fsf@meyering.net>
	<AANLkTini04wQ7g8sAexc9Kv886NHbz3NBF2+BAXNR-pv@mail.gmail.com>
	<874o9jvkdo.fsf@meyering.net>
	<AANLkTimkonRUED8c_bFvzaLdPDjJjChJ6CSw736At5wq@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <AANLkTimkonRUED8c_bFvzaLdPDjJjChJ6CSw736At5wq@mail.gmail.com>
User-Agent: Mutt/1.5.20 (2009-06-14)
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 1)
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 09 Jan 2011 19:03:22 -0000

Rakib Mullick wrote:
> Jim Meyering wrote:
> > Why do you want an option for this when you can already use touch
> > to create a file with selected permissions via your umask?
> 
> I think umask change the the file creation mask system wide. I might
> not want it for every file I create.

No.  The umask only affects the current process.  It doesn't affect
other processes system wide.  It will affect files created by the
current process, but that is the point of it.

> > If you say that you require an execute bit or some special bit to
> > be set, please explain why.  I do not see how having such bits set
> > on an empty file would be useful.  If you're about to add content,
> 
> No. It's not just about setting an execute bit, say I might have
> create a file with 'sudo touch filename', in that case I just can see
> the file by doing 'cat filename' or 'vim filename'.

In which case the file is zero sized.  Emitting the file with 'cat'
would read a zero sized file.  But you wouldn't want to edit the file
with 'editor filename'.  Since you specifically created it as a root
owned file then the user would not have permissions to edit the file.
And you wouldn't want to 'sudo editor filename' either because of the
security implications.  You /may/ want to use 'sudoedit'.

> So, again I need to do, 'sudo chmod xxx filename' to make it
> available to its user. So, it can be summed up as:

Please say exactly what mode bits you want to set.  That was the
question asked in the last email.  Please don't say mode xxx but
instead say exactly what mode bits you are talking about.

Because otherwise we have to imagine and guess.  I can only guess that
you mean to make it unavailable, right?

  $ sudo touch /tmp/afile
  $ ls -l /tmp/afile
  -rw-r--r-- 1 root root 0 Jan  9 11:48 /tmp/afile

If you wanted to block it then you should need to chmod it.

  $ sudo chmod go-rwx /tmp/afile
  $ ls -l /tmp/afile
  -rw------- 1 root root 0 Jan  9 11:48 /tmp/afile

> (Assume, to create a file under /usr/src/work)
>              sudo touch /usr/src/work/filename (to create a file)
>              sudo chmod xxx /usr/src/work/filename (to give a user proper permission)

Please say exactly what mode bits you want to set.

Because with the above touch you would have a root owned file.  The
only way you could give a user permission to write to the file would
be to open it up for global writing by any user.

  $ sudo chmod ugo=rw /tmp/afile
  $ ls -l /tmp/afile
  -rw-rw-rw- 1 root root 0 Jan  9 11:48 /tmp/afile

That is bad.  You do not want to do this.

> After adding this feature we can do this:
>             sudo touch --mode xxx /usr/src/work/filename
> thats it.

As I read your message you are saying you want to do:

  sudo touch --mode ugo=rw /usr/src/work/filename

Instead of doing:

  sudo touch /usr/src/work/filename
  sudo chmod ugo=rw /usr/src/work/filename

Just wishing to reduce two commands down to one command is not a good
reason for doing this.

Secondly you don't want to make files writable by other just to share
them.  That is bad.

If you want to provide for a group of users to be able to share access
to a file then you should use group permissions.  Create a work group
for use by the group.  Make the directory owned and writable by the
group.  Then when new files are created it will be owned by the group.
There is no need to use sudo.  Just operate as a normal user.  For
each user that is to have access to this group add that user to the
group.

This is a very short summary of the process.  Search the web for User
Private Group and read about how Unix file group permissions enable
this behavior natively.  I believe this is the concept that you should
be using.

Bob


From MAILER-DAEMON Sun Jan 09 17:06:03 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1Pc3Ol-00079L-Fo
	for mharc-coreutils@gnu.org; Sun, 09 Jan 2011 17:06:03 -0500
Received: from [140.186.70.92] (port=43763 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1Pc3Oi-000784-Vo
	for coreutils@gnu.org; Sun, 09 Jan 2011 17:06:02 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Pc3Oh-0000es-4u
	for coreutils@gnu.org; Sun, 09 Jan 2011 17:06:00 -0500
Received: from mx.meyering.net ([82.230.74.64]:56716)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Pc3Og-0000dz-Mk
	for coreutils@gnu.org; Sun, 09 Jan 2011 17:05:59 -0500
Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000)
	id BFC246031E; Sun,  9 Jan 2011 23:05:56 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: Jonathan Nieder <jrnieder@gmail.com>
Subject: Re: [coreutils] Re: Bug#609049: du: process_file: Assertion `level ==
	prev_level - 1' failed
In-Reply-To: <87lj2vtios.fsf@meyering.net> (Jim Meyering's message of "Sat, 08
	Jan 2011 18:21:39 +0100")
References: <20110105193634.GA22486@burratino> <87lj2vtios.fsf@meyering.net>
Date: Sun, 09 Jan 2011 23:05:56 +0100
Message-ID: <87hbdhspff.fsf@meyering.net>
Lines: 200
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
Cc: 609049@bugs.debian.org, coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 09 Jan 2011 22:06:02 -0000

Jim Meyering wrote:
...
> While I do have a reproducer that will become a test suite addition
> (coming soon), it relies on python (a first) and the python-inotify
> package, so I'll have to be careful to skip the test when those
> prerequisites aren't installed.  Also, there's an inherent race condition,
> so I'll have to find the right compromise between absolute test-robustness
> (too expensive in time and inodes) and reasonableness.

Here's a complete patch, including the test.
I would have preferred to avoid the one-second sleep.
Maybe someone here can see a better way.

>From 9bf47055f64efb56d022feca01ad901e85e21bc1 Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering@redhat.com>
Date: Sat, 8 Jan 2011 17:44:55 +0100
Subject: [PATCH] du: don't abort when a subdir is renamed during traversal

* NEWS (Bug fixes): Mention it.
* src/du.c (prev_level): Move declaration "up" to file-scope global.
(du_files): Reset prev_level to 0 upon abnormal fts_read termination.
Reported by Johathan Nieder in http://bugs.debian.org/609049
Also, improve a diagnostic.
* tests/du/move-dir-while-traversing: Test for the above.
* tests/Makefile.am (TESTS): Add it.
---
 NEWS                               |    8 +++
 src/du.c                           |   15 +++++--
 tests/Makefile.am                  |    1 +
 tests/du/move-dir-while-traversing |   85 ++++++++++++++++++++++++++++++++++++
 4 files changed, 105 insertions(+), 4 deletions(-)
 create mode 100755 tests/du/move-dir-while-traversing

diff --git a/NEWS b/NEWS
index 2a71ca6..5a70243 100644
--- a/NEWS
+++ b/NEWS
@@ -2,6 +2,14 @@ GNU coreutils NEWS                                    -*- outline -*-

 * Noteworthy changes in release ?.? (????-??-??) [?]

+** Bug fixes
+
+  du would abort with a failed assertion when two conditions are met:
+  part of the hierarchy being traversed is moved to a higher level in the
+  directory tree, and there is at least one more command line directory
+  argument following the one containing the moved sub-tree.
+  [bug introduced in coreutils-5.1.0]
+

 * Noteworthy changes in release 8.9 (2011-01-04) [stable]

diff --git a/src/du.c b/src/du.c
index 77deb0c..671cac7 100644
--- a/src/du.c
+++ b/src/du.c
@@ -63,8 +63,11 @@ extern bool fts_debug;
 /* A set of dev/ino pairs.  */
 static struct di_set *di_set;

-/* Define a class for collecting directory information. */
+/* Keep track of the preceding "level" (depth in hierarchy)
+   from one call of process_file to the next.  */
+static size_t prev_level;

+/* Define a class for collecting directory information. */
 struct duinfo
 {
   /* Size of files in directory.  */
@@ -399,7 +402,6 @@ process_file (FTS *fts, FTSENT *ent)
   struct duinfo dui;
   struct duinfo dui_to_print;
   size_t level;
-  static size_t prev_level;
   static size_t n_alloc;
   /* First element of the structure contains:
      The sum of the st_size values of all entries in the single directory
@@ -582,10 +584,15 @@ du_files (char **files, int bit_flags)
             {
               if (errno != 0)
                 {
-                  /* FIXME: try to give a better message  */
-                  error (0, errno, _("fts_read failed"));
+                  error (0, errno, _("fts_read failed: %s"),
+                         quotearg_colon (fts->fts_path));
                   ok = false;
                 }
+
+              /* When exiting this loop early, be careful to reset the
+                 global, prev_level, used in process_file.  Otherwise, its
+                 (level == prev_level - 1) assertion could fail.  */
+              prev_level = 0;
               break;
             }
           FTS_CROSS_CHECK (fts);
diff --git a/tests/Makefile.am b/tests/Makefile.am
index 06d81f0..a5dbd3e 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -370,6 +370,7 @@ TESTS =						\
   du/long-from-unreadable			\
   du/long-sloop					\
   du/max-depth					\
+  du/move-dir-while-traversing			\
   du/no-deref					\
   du/no-x					\
   du/one-file-system				\
diff --git a/tests/du/move-dir-while-traversing b/tests/du/move-dir-while-traversing
new file mode 100755
index 0000000..e42bc93
--- /dev/null
+++ b/tests/du/move-dir-while-traversing
@@ -0,0 +1,85 @@
+#!/bin/sh
+# Trigger a failed assertion in coreutils-8.9 and earlier.
+
+# Copyright (C) 2011 Free Software Foundation, Inc.
+
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+. "${srcdir=.}/init.sh"; path_prepend_ ../src
+print_ver_ du
+
+# We use a python-inotify script, so...
+python -m pyinotify -h > /dev/null \
+  || skip_ 'python-inotify package not installed'
+
+# Move a directory "up" while du is processing its sub-directories.
+# While du is processing a hierarchy .../B/C/D/... this script
+# detects when du opens D/, and then moves C/ "up" one level
+# so that it is a sibling of B/.
+# Given the inherent race condition, we have to add enough "weight"
+# under D/ so that in most cases, the monitor performs the single
+# rename syscall before du finishes processing the subtree under D/.
+
+cat <<'EOF' > inotify-watch-for-dir-access.py
+#!/usr/bin/env python
+from pyinotify import *
+dir = sys.argv[1]
+dest_parent = os.path.dirname(os.path.dirname(dir))
+dest = os.path.join(dest_parent, os.path.basename(dir))
+
+class ProcessDir(ProcessEvent):
+
+    def process_IN_OPEN(self, event):
+        os.rename(dir, dest)
+        sys.exit(0)
+
+    def process_default(self, event):
+        pass
+
+wm = WatchManager()
+notifier = Notifier(wm)
+wm.watch_transient_file(dir, IN_OPEN, ProcessDir)
+print 'started'
+notifier.loop()
+EOF
+chmod a+x inotify-watch-for-dir-access.py
+
+long=d/e/f/g/h/i/j/k/l/m/n/o/p/q/r/s/t/u/v/w/x/y/z
+t=T/U
+mkdir -p $t/3/a/b/c/$long d2 || framework_failure
+timeout 6 ./inotify-watch-for-dir-access.py $t/3/a/b > start-msg &
+
+# Wait for the watcher to start...
+nonempty() { test -s start-msg && return 0; sleep $1; }
+retry_delay_ nonempty .1 5
+
+# The above delay is insufficient in ~50% of my trials.
+# Sometimes, when under heavy load, a parallel "make check" would
+# fail this test when sleeping just 0.1 seconds here.
+sleep 1
+
+# The above watches for an IN_OPEN event on $t/3/a/b,
+# and when it triggers, moves the parent, $t/3/a, up one level
+# so it's directly under $t.
+
+du -a $t d2 2> err
+# Before coreutils-8.10, du would abort.
+test $? = 1 || fail=1
+
+# check for the new diagnostic
+printf "du: fts_read failed: $t/3/a/b: No such file or directory\n" > exp \
+  || fail=1
+compare err exp || fail=1
+
+Exit $fail
--
1.7.3.5


From MAILER-DAEMON Sun Jan 09 19:00:07 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1Pc5B9-0007ai-KU
	for mharc-coreutils@gnu.org; Sun, 09 Jan 2011 19:00:07 -0500
Received: from [140.186.70.92] (port=49572 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1Pc5B7-0007YI-Ao
	for coreutils@gnu.org; Sun, 09 Jan 2011 19:00:06 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Pc5B6-0001kT-5f
	for coreutils@gnu.org; Sun, 09 Jan 2011 19:00:05 -0500
Received: from mail1.slb.deg.dub.stisp.net ([84.203.253.98]:17306)
	by eggs.gnu.org with smtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Pc5B5-0001i5-QF
	for coreutils@gnu.org; Sun, 09 Jan 2011 19:00:04 -0500
Received: (qmail 18322 invoked from network); 10 Jan 2011 00:00:00 -0000
Received: from unknown (HELO ?192.168.2.25?) (84.203.137.218)
	by mail1.slb.deg.dub.stisp.net with SMTP; 10 Jan 2011 00:00:00 -0000
Message-ID: <4D2A4BF0.1020303@draigBrady.com>
Date: Sun, 09 Jan 2011 23:59:44 +0000
From: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US;
	rv:1.9.1.8) Gecko/20100227 Thunderbird/3.0.3
MIME-Version: 1.0
To: Jim Meyering <jim@meyering.net>
Subject: Re: [coreutils] Re: Bug#609049: du: process_file: Assertion `level
	==	prev_level - 1' failed
References: <20110105193634.GA22486@burratino> <87lj2vtios.fsf@meyering.net>
	<87hbdhspff.fsf@meyering.net>
In-Reply-To: <87hbdhspff.fsf@meyering.net>
X-Enigmail-Version: 1.0.1
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
X-detected-operating-system: by eggs.gnu.org: FreeBSD 4.6-4.9
Cc: Jonathan Nieder <jrnieder@gmail.com>, 609049@bugs.debian.org,
	coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 10 Jan 2011 00:00:06 -0000

On 09/01/11 22:05, Jim Meyering wrote:
> diff --git a/tests/du/move-dir-while-traversing b/tests/du/move-dir-while-traversing

> +# We use a python-inotify script, so...
> +python -m pyinotify -h > /dev/null \
> +  || skip_ 'python-inotify package not installed'

A small point is that error is a bit fedora specific.
The package is python-pyinotify on debian/ubuntu for example.

> +
> +# Move a directory "up" while du is processing its sub-directories.
> +# While du is processing a hierarchy .../B/C/D/... this script
> +# detects when du opens D/, and then moves C/ "up" one level
> +# so that it is a sibling of B/.
> +# Given the inherent race condition, we have to add enough "weight"
> +# under D/ so that in most cases, the monitor performs the single
> +# rename syscall before du finishes processing the subtree under D/.
> +
> +cat <<'EOF' > inotify-watch-for-dir-access.py
> +#!/usr/bin/env python
> +from pyinotify import *

I generally don't include everything from a module,
to keep the namespace clean.
I'd do instead:

import pyinotify as pn
import os,sys
...


> +dir = sys.argv[1]
> +dest_parent = os.path.dirname(os.path.dirname(dir))
> +dest = os.path.join(dest_parent, os.path.basename(dir))
> +
> +class ProcessDir(ProcessEvent):
> +
> +    def process_IN_OPEN(self, event):
> +        os.rename(dir, dest)
> +        sys.exit(0)
> +
> +    def process_default(self, event):
> +        pass
> +
> +wm = WatchManager()
> +notifier = Notifier(wm)
> +wm.watch_transient_file(dir, IN_OPEN, ProcessDir)
> +print 'started'

The above print is buffered by default.
As we're using it for synchronization I'd

sys.stdout.write('started\n')
sys.stdout.flush()

> +notifier.loop()
> +EOF
> +chmod a+x inotify-watch-for-dir-access.py
> +
> +long=d/e/f/g/h/i/j/k/l/m/n/o/p/q/r/s/t/u/v/w/x/y/z
> +t=T/U
> +mkdir -p $t/3/a/b/c/$long d2 || framework_failure
> +timeout 6 ./inotify-watch-for-dir-access.py $t/3/a/b > start-msg &
> +
> +# Wait for the watcher to start...
> +nonempty() { test -s start-msg && return 0; sleep $1; }
> +retry_delay_ nonempty .1 5

I think the above may iterate only once?
How about:

nonempty() {
  test -s start-msg || { sleep $1; return 1; }
}
retry_delay_ nonempty .1 5 || framework_failure

> +
> +# The above delay is insufficient in ~50% of my trials.
> +# Sometimes, when under heavy load, a parallel "make check" would
> +# fail this test when sleeping just 0.1 seconds here.
> +sleep 1

Hopefully this extra sleep is not required now

Nice fix BTW!

cheers,
Pádraig.


From MAILER-DAEMON Mon Jan 10 02:01:31 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PcBkx-0004va-IN
	for mharc-coreutils@gnu.org; Mon, 10 Jan 2011 02:01:31 -0500
Received: from [140.186.70.92] (port=51183 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PcBku-0004vK-PL
	for coreutils@gnu.org; Mon, 10 Jan 2011 02:01:30 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PcBkt-0000vH-8s
	for coreutils@gnu.org; Mon, 10 Jan 2011 02:01:28 -0500
Received: from mx.meyering.net ([82.230.74.64]:60074)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PcBks-0000ud-TN
	for coreutils@gnu.org; Mon, 10 Jan 2011 02:01:27 -0500
Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000)
	id E93EB60155; Mon, 10 Jan 2011 08:01:24 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: =?utf-8?Q?P=C3=A1draig?= Brady <P@draigBrady.com>
Subject: Re: [coreutils] Re: Bug#609049: du: process_file: Assertion `level
	==	prev_level - 1' failed
In-Reply-To: <4D2A4BF0.1020303@draigBrady.com> (=?utf-8?Q?=22P=C3=A1draig?=
	Brady"'s message of "Sun, 09 Jan 2011 23:59:44 +0000")
References: <20110105193634.GA22486@burratino> <87lj2vtios.fsf@meyering.net>
	<87hbdhspff.fsf@meyering.net> <4D2A4BF0.1020303@draigBrady.com>
Date: Mon, 10 Jan 2011 08:01:24 +0100
Message-ID: <87bp3ps0mz.fsf@meyering.net>
Lines: 128
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
Cc: Jonathan Nieder <jrnieder@gmail.com>, 609049@bugs.debian.org,
	coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 10 Jan 2011 07:01:30 -0000

P=C3=A1draig Brady wrote:

> On 09/01/11 22:05, Jim Meyering wrote:
>> diff --git a/tests/du/move-dir-while-traversing
>> b/tests/du/move-dir-while-traversing
>
>> +# We use a python-inotify script, so...
>> +python -m pyinotify -h > /dev/null \
>> +  || skip_ 'python-inotify package not installed'
>
> A small point is that error is a bit fedora specific.
> The package is python-pyinotify on debian/ubuntu for example.

Good point.  I've dropped the "-":

  || skip_ 'python inotify package not installed'

>> +from pyinotify import *
>
> I generally don't include everything from a module,
> to keep the namespace clean.
> I'd do instead:
>
> import pyinotify as pn
> import os,sys
> ...

I prefer that, too.

>> +dir =3D sys.argv[1]
...
>> +print 'started'
>
> The above print is buffered by default.
> As we're using it for synchronization I'd
>
> sys.stdout.write('started\n')
> sys.stdout.flush()

Good catch!

...
>> +nonempty() { test -s start-msg && return 0; sleep $1; }
>> +retry_delay_ nonempty .1 5
>
> I think the above may iterate only once?
> How about:
>
> nonempty() {
>   test -s start-msg || { sleep $1; return 1; }
> }
> retry_delay_ nonempty .1 5 || framework_failure

Yes, that fixes another bug.

>> +# The above delay is insufficient in ~50% of my trials.
>> +# Sometimes, when under heavy load, a parallel "make check" would
>> +# fail this test when sleeping just 0.1 seconds here.
>> +sleep 1
>
> Hopefully this extra sleep is not required now

Bingo.  It does, so far.
Thanks for the advice and fixes.
Here's the incremental I'm testing with:

diff --git a/tests/du/move-dir-while-traversing b/tests/du/move-dir-while-t=
raversing
index e42bc93..fe1615c 100755
--- a/tests/du/move-dir-while-traversing
+++ b/tests/du/move-dir-while-traversing
@@ -21,7 +21,7 @@ print_ver_ du

 # We use a python-inotify script, so...
 python -m pyinotify -h > /dev/null \
-  || skip_ 'python-inotify package not installed'
+  || skip_ 'python inotify package not installed'

 # Move a directory "up" while du is processing its sub-directories.
 # While du is processing a hierarchy .../B/C/D/... this script
@@ -33,12 +33,14 @@ python -m pyinotify -h > /dev/null \

 cat <<'EOF' > inotify-watch-for-dir-access.py
 #!/usr/bin/env python
-from pyinotify import *
+import pyinotify as pn
+import os,sys
+
 dir =3D sys.argv[1]
 dest_parent =3D os.path.dirname(os.path.dirname(dir))
 dest =3D os.path.join(dest_parent, os.path.basename(dir))

-class ProcessDir(ProcessEvent):
+class ProcessDir(pn.ProcessEvent):

     def process_IN_OPEN(self, event):
         os.rename(dir, dest)
@@ -47,10 +49,11 @@ class ProcessDir(ProcessEvent):
     def process_default(self, event):
         pass

-wm =3D WatchManager()
-notifier =3D Notifier(wm)
-wm.watch_transient_file(dir, IN_OPEN, ProcessDir)
-print 'started'
+wm =3D pn.WatchManager()
+notifier =3D pn.Notifier(wm)
+wm.watch_transient_file(dir, pn.IN_OPEN, ProcessDir)
+sys.stdout.write('started\n')
+sys.stdout.flush()
 notifier.loop()
 EOF
 chmod a+x inotify-watch-for-dir-access.py
@@ -61,14 +64,9 @@ mkdir -p $t/3/a/b/c/$long d2 || framework_failure
 timeout 6 ./inotify-watch-for-dir-access.py $t/3/a/b > start-msg &

 # Wait for the watcher to start...
-nonempty() { test -s start-msg && return 0; sleep $1; }
+nonempty() { test -s start-msg || { sleep $1; return 1; }; }
 retry_delay_ nonempty .1 5

-# The above delay is insufficient in ~50% of my trials.
-# Sometimes, when under heavy load, a parallel "make check" would
-# fail this test when sleeping just 0.1 seconds here.
-sleep 1
-
 # The above watches for an IN_OPEN event on $t/3/a/b,
 # and when it triggers, moves the parent, $t/3/a, up one level
 # so it's directly under $t.


From MAILER-DAEMON Mon Jan 10 04:37:06 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PcEBW-00026D-Ho
	for mharc-coreutils@gnu.org; Mon, 10 Jan 2011 04:37:06 -0500
Received: from [140.186.70.92] (port=51511 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PcEBT-0001zu-Ui
	for coreutils@gnu.org; Mon, 10 Jan 2011 04:37:04 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PcEBS-0008Rq-KP
	for coreutils@gnu.org; Mon, 10 Jan 2011 04:37:03 -0500
Received: from mx.meyering.net ([82.230.74.64]:34276)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PcEBS-0008Rk-E6
	for coreutils@gnu.org; Mon, 10 Jan 2011 04:37:02 -0500
Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000)
	id 7B3CA60296; Mon, 10 Jan 2011 10:37:01 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: =?utf-8?Q?P=C3=A1draig?= Brady <P@draigBrady.com>
Subject: Re: [coreutils] Re: Bug#609049: du: process_file: Assertion `level
	==	prev_level - 1' failed
In-Reply-To: <87bp3ps0mz.fsf@meyering.net> (Jim Meyering's message of "Mon, 10
	Jan 2011 08:01:24 +0100")
References: <20110105193634.GA22486@burratino> <87lj2vtios.fsf@meyering.net>
	<87hbdhspff.fsf@meyering.net> <4D2A4BF0.1020303@draigBrady.com>
	<87bp3ps0mz.fsf@meyering.net>
Date: Mon, 10 Jan 2011 10:37:01 +0100
Message-ID: <87mxn9qev6.fsf@meyering.net>
Lines: 11
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
Cc: Jonathan Nieder <jrnieder@gmail.com>, 609049@bugs.debian.org,
	coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 10 Jan 2011 09:37:04 -0000

Jim Meyering wrote:
...
> Here's the incremental I'm testing with:
>
> diff --git a/tests/du/move-dir-while-traversing b/tests/du/move-dir-while-traversing
> index e42bc93..fe1615c 100755

Thanks again for the quick feedback.
I squashed in the changes above and pushed the result.
I also updated to the latest gnulib in a separate commit,
since it passed the tests I ran.


From MAILER-DAEMON Mon Jan 10 13:18:55 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PcMKV-0005Lq-9j
	for mharc-coreutils@gnu.org; Mon, 10 Jan 2011 13:18:55 -0500
Received: from [140.186.70.92] (port=57970 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PcMKS-0005K2-MY
	for coreutils@gnu.org; Mon, 10 Jan 2011 13:18:53 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <rakib.mullick@gmail.com>) id 1PcMKR-0004Db-6r
	for coreutils@gnu.org; Mon, 10 Jan 2011 13:18:52 -0500
Received: from mail-ew0-f41.google.com ([209.85.215.41]:50185)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <rakib.mullick@gmail.com>) id 1PcLzQ-0007FI-86
	for coreutils@gnu.org; Mon, 10 Jan 2011 12:57:08 -0500
Received: by mail-ew0-f41.google.com with SMTP id 27so9504987ewy.0
	for <coreutils@gnu.org>; Mon, 10 Jan 2011 09:57:06 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma;
	h=domainkey-signature:mime-version:received:received:in-reply-to
	:references:date:message-id:subject:from:to:content-type
	:content-transfer-encoding;
	bh=cf2ZhEAxYNEsT3N9y2u5te/kzKfnF1ErnFBuMRkkAGY=;
	b=IoD1uF2vPe7vZdxvVhs+OPANdWk22K+jIlT4S4mWXiDN3wItuxfpMTPIMQmZj0y5++
	ymQiw2gEE2/y/J0IDaImM4BLfio0YTgmxNwrm9cEyylpjPPnuIc0Emjp/Zy8Lvgw6qgo
	ZKAx+tCe01O1s6O9SyVjL1yITOWO5zTYW6sgY=
DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma;
	h=mime-version:in-reply-to:references:date:message-id:subject:from:to
	:content-type:content-transfer-encoding;
	b=P4C+PpCNk5qZnsh9gWKPsEDx+27xmaeZOol0MoRJm4SLLh5Rwced9ZySyDyDCTSVy9
	L66+K7vtedLjHL3v61tsn/KdrQlNK749dUbYqsT4mMTlrqz+PHEgh53ms359pwDljnOE
	xyo0JKL/Uz+7w/6pOUmi+tBc6trjtERVl8vgE=
MIME-Version: 1.0
Received: by 10.216.82.9 with SMTP id n9mr27649431wee.35.1294682226618; Mon,
	10 Jan 2011 09:57:06 -0800 (PST)
Received: by 10.216.72.143 with HTTP; Mon, 10 Jan 2011 09:57:06 -0800 (PST)
In-Reply-To: <20110109190315.GA15320@hysteria.proulx.com>
References: <AANLkTinGw+2sqPfZsV=CNRiW5xBRVjXA6BYUFHNrBKpR@mail.gmail.com>
	<4D273001.6000807@redhat.com> <4D273E0B.6060203@draigBrady.com>
	<4D2741A6.8060307@redhat.com> <87wrmgve1z.fsf@meyering.net>
	<AANLkTini04wQ7g8sAexc9Kv886NHbz3NBF2+BAXNR-pv@mail.gmail.com>
	<874o9jvkdo.fsf@meyering.net>
	<AANLkTimkonRUED8c_bFvzaLdPDjJjChJ6CSw736At5wq@mail.gmail.com>
	<20110109190315.GA15320@hysteria.proulx.com>
Date: Mon, 10 Jan 2011 23:57:06 +0600
Message-ID: <AANLkTin5U34eTtZG2G1Wk=SL53-9voAXgVK6vwxrQkkG@mail.gmail.com>
Subject: Re: [coreutils] RFC for adding file creation mode feature into touch
	utility.
From: Rakib Mullick <rakib.mullick@gmail.com>
To: bob@proulx.com, coreutils@gnu.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 2)
Cc: 
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 10 Jan 2011 18:18:53 -0000

On Mon, Jan 10, 2011 at 1:03 AM, Bob Proulx <bob@proulx.com> wrote:
> Rakib Mullick wrote:
>> Jim Meyering wrote:
>> > Why do you want an option for this when you can already use touch
>> > to create a file with selected permissions via your umask?
>>
>> I think umask change the the file creation mask system wide. I might
>> not want it for every file I create.
>
> No. =A0The umask only affects the current process. =A0It doesn't affect
> other processes system wide. =A0It will affect files created by the
> current process, but that is the point of it.
>
>> > If you say that you require an execute bit or some special bit to
>> > be set, please explain why. =A0I do not see how having such bits set
>> > on an empty file would be useful. =A0If you're about to add content,
>>
>> No. It's not just about setting an execute bit, say I might have
>> create a file with 'sudo touch filename', in that case I just can see
>> the file by doing 'cat filename' or 'vim filename'.
>
> In which case the file is zero sized. =A0Emitting the file with 'cat'
> would read a zero sized file. =A0But you wouldn't want to edit the file
> with 'editor filename'. =A0Since you specifically created it as a root
> owned file then the user would not have permissions to edit the file.
> And you wouldn't want to 'sudo editor filename' either because of the
> security implications. =A0You /may/ want to use 'sudoedit'.
>
Yes. To be more precise I want to give user the write permission, as
if he/she can do 'editor filename' and can edit and save.

>> So, again I need to do, 'sudo chmod xxx filename' to make it
>> available to its user. So, it can be summed up as:
>
> Please say exactly what mode bits you want to set. =A0That was the
> question asked in the last email. =A0Please don't say mode xxx but
> instead say exactly what mode bits you are talking about.
>
In my case, I needed to set write permission for the user. I tried to
tell it generically, that's why I used xxx.

> Because otherwise we have to imagine and guess. =A0I can only guess that
> you mean to make it unavailable, right?
>
> =A0$ sudo touch /tmp/afile
> =A0$ ls -l /tmp/afile
> =A0-rw-r--r-- 1 root root 0 Jan =A09 11:48 /tmp/afile
>
> If you wanted to block it then you should need to chmod it.
>
> =A0$ sudo chmod go-rwx /tmp/afile
> =A0$ ls -l /tmp/afile
> =A0-rw------- 1 root root 0 Jan =A09 11:48 /tmp/afile
>
>> (Assume, to create a file under /usr/src/work)
>> =A0 =A0 =A0 =A0 =A0 =A0 =A0sudo touch /usr/src/work/filename (to create =
a file)
>> =A0 =A0 =A0 =A0 =A0 =A0 =A0sudo chmod xxx /usr/src/work/filename (to giv=
e a user proper permission)
>
> Please say exactly what mode bits you want to set.
>
In my case, I wanted to set write mode. But, anyone might have wanted
to set other bit's.

> Because with the above touch you would have a root owned file. =A0The
> only way you could give a user permission to write to the file would
> be to open it up for global writing by any user.
>
> =A0$ sudo chmod ugo=3Drw /tmp/afile
> =A0$ ls -l /tmp/afile
> =A0-rw-rw-rw- 1 root root 0 Jan =A09 11:48 /tmp/afile
>
> That is bad. =A0You do not want to do this.
>
Yes. I didn't want it.

>> After adding this feature we can do this:
>> =A0 =A0 =A0 =A0 =A0 =A0 sudo touch --mode xxx /usr/src/work/filename
>> thats it.
>
> As I read your message you are saying you want to do:
>
> =A0sudo touch --mode ugo=3Drw /usr/src/work/filename
>
Maybe, yes. But, I might have want to set different permission in
different context.

> Instead of doing:
>
> =A0sudo touch /usr/src/work/filename
> =A0sudo chmod ugo=3Drw /usr/src/work/filename
>
> Just wishing to reduce two commands down to one command is not a good
> reason for doing this.
>
Yes. When other file creation utility could have one, then why not in touch=
?

> Secondly you don't want to make files writable by other just to share
> them. =A0That is bad.
>
> If you want to provide for a group of users to be able to share access
> to a file then you should use group permissions. =A0Create a work group
> for use by the group. =A0Make the directory owned and writable by the
> group. =A0Then when new files are created it will be owned by the group.
> There is no need to use sudo. =A0Just operate as a normal user. =A0For
> each user that is to have access to this group add that user to the
> group.
>
> This is a very short summary of the process. =A0Search the web for User
> Private Group and read about how Unix file group permissions enable
> this behavior natively. =A0I believe this is the concept that you should
> be using.
>
Thanks for your help. When things get changed, the way of doing things
also get changed. Since its not going to be changed, so I have to use
this concept. :)


thanks,
rakib


From MAILER-DAEMON Mon Jan 10 15:37:13 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PcOUL-00067g-6q
	for mharc-coreutils@gnu.org; Mon, 10 Jan 2011 15:37:13 -0500
Received: from [140.186.70.92] (port=37725 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PcOOH-00021K-3i
	for coreutils@gnu.org; Mon, 10 Jan 2011 15:30:57 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <crockabiscuit@yahoo.com>) id 1PcOOG-0006Zf-0z
	for coreutils@gnu.org; Mon, 10 Jan 2011 15:30:56 -0500
Received: from nm30.bullet.mail.ne1.yahoo.com ([98.138.90.93]:23386)
	by eggs.gnu.org with smtp (Exim 4.71)
	(envelope-from <crockabiscuit@yahoo.com>) id 1PcOOF-0006ZJ-SV
	for coreutils@gnu.org; Mon, 10 Jan 2011 15:30:55 -0500
Received: from [98.138.90.53] by nm30.bullet.mail.ne1.yahoo.com with NNFMP;
	10 Jan 2011 20:30:55 -0000
Received: from [98.138.88.232] by tm6.bullet.mail.ne1.yahoo.com with NNFMP;
	10 Jan 2011 20:30:54 -0000
Received: from [127.0.0.1] by omp1032.mail.ne1.yahoo.com with NNFMP;
	10 Jan 2011 20:30:54 -0000
X-Yahoo-Newman-Property: ymail-3
X-Yahoo-Newman-Id: 961745.32146.bm@omp1032.mail.ne1.yahoo.com
Received: (qmail 60280 invoked by uid 60001); 10 Jan 2011 20:30:54 -0000
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024;
	t=1294691454; bh=ooG+UzpeVQxqlfSmW/HowlmLFMkUFUryZrt5tLz3uRE=;
	h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:MIME-Version:Content-Type;
	b=OL7S5Gsiwzrcb/tTTe8YlN6EpG/zyiOI3Li0lL1o96/zFSkbjQvURhK3+RQrLtTzzXKOuJUlLDXtood0Cvlg/5xxo2V+55ylQM6afvzMnnRClfehh5gHeMPdT1eIAqlysec0jGz9FgsrdVpH65mCrPgfMfJG2N2dYJ/0EBvjr9U=
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com;
	h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:MIME-Version:Content-Type;
	b=1MJAZEo9Wcw6OAlCRp4U5tcUp+cENnxKRIabNTDphNzI4LCyyCYfP+UGGXgtk5KCO3GIAPMpjL0Rt1twqt4fUh/stQcTNB+BIQLQ2ejo8dc0fi/QZHxwpT2kvl/VP1KQj5/OS1W2xsIeCGCcStGL9mq+q92nddoNl7PY/uFDPLg=;
Message-ID: <766487.59657.qm@web120308.mail.ne1.yahoo.com>
X-YMail-OSG: DRCiNhIVM1lNprjjRXOvF1EVGzzVoI9ZLXsIwhKwO1gE7ba
	JJHKZetUdIf5FYApbv.FIKOWp2X7HmYO052J9OyXVCRf28.ZJhjFWchJYDV5
	F2f1WuszwbQZ3SjPRz8vs6Ga2VNHf8TIUSVmmkhVvjsqzo5_YX3qZhmIO9XH
	_PDxY_oTovahTo9dD2nRUO75eWlHBTs1VhSJl
Received: from [180.68.85.212] by web120308.mail.ne1.yahoo.com via HTTP;
	Mon, 10 Jan 2011 12:30:54 PST
X-Mailer: YahooMailRC/553 YahooMailWebService/0.8.107.285259
Date: Mon, 10 Jan 2011 12:30:54 -0800 (PST)
From: crocket <crockabiscuit@yahoo.com>
To: coreutils@gnu.org
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Mailman-Approved-At: Mon, 10 Jan 2011 15:37:12 -0500
Subject: [coreutils] "ls -lh" rounds up the size of files instead of
	rounding half up/down.
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 10 Jan 2011 20:30:58 -0000

There is a file whose size is 42949837312bytes.
It's almost exactly 40.000153064727783GB.
"ls -lh" displays it as 41GB.

It is resulted due to rounding up.

However, rounding half down/up is much more common than rounding up in the real 
world.
When I expect moderate file size approximation, this 1GB difference is not 
ignorable.

Is there any reason for this behavior?
Or is it just a mistake?



      


From MAILER-DAEMON Mon Jan 10 15:43:42 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PcOab-0002MQ-Uv
	for mharc-coreutils@gnu.org; Mon, 10 Jan 2011 15:43:41 -0500
Received: from [140.186.70.92] (port=57344 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PcOaa-0002KY-3u
	for coreutils@gnu.org; Mon, 10 Jan 2011 15:43:41 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1PcOaY-0001BG-ND
	for coreutils@gnu.org; Mon, 10 Jan 2011 15:43:39 -0500
Received: from mx1.redhat.com ([209.132.183.28]:3728)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1PcOaY-0001Ad-Fh
	for coreutils@gnu.org; Mon, 10 Jan 2011 15:43:38 -0500
Received: from int-mx12.intmail.prod.int.phx2.redhat.com
	(int-mx12.intmail.prod.int.phx2.redhat.com [10.5.11.25])
	by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id p0AKhati019736
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK);
	Mon, 10 Jan 2011 15:43:36 -0500
Received: from [10.3.113.124] (ovpn-113-124.phx2.redhat.com [10.3.113.124])
	by int-mx12.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP
	id p0AKhab0027108; Mon, 10 Jan 2011 15:43:36 -0500
Message-ID: <4D2B6F77.9030805@redhat.com>
Date: Mon, 10 Jan 2011 13:43:35 -0700
From: Eric Blake <eblake@redhat.com>
Organization: Red Hat
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US;
	rv:1.9.2.13) Gecko/20101209 Fedora/3.1.7-0.35.b3pre.fc14
	Lightning/1.0b3pre Mnenhy/0.8.3 Thunderbird/3.1.7
MIME-Version: 1.0
To: crocket <crockabiscuit@yahoo.com>
Subject: Re: [coreutils] "ls -lh" rounds up the size of files instead
	of	rounding half up/down.
References: <766487.59657.qm@web120308.mail.ne1.yahoo.com>
In-Reply-To: <766487.59657.qm@web120308.mail.ne1.yahoo.com>
X-Enigmail-Version: 1.1.2
OpenPGP: url=http://people.redhat.com/eblake/eblake.gpg
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature";
	boundary="------------enig3DD1BD332924F76C5B14A216"
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.25
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 10 Jan 2011 20:43:41 -0000

This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--------------enig3DD1BD332924F76C5B14A216
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

On 01/10/2011 01:30 PM, crocket wrote:
> There is a file whose size is 42949837312bytes.
> It's almost exactly 40.000153064727783GB.
> "ls -lh" displays it as 41GB.
>=20
> It is resulted due to rounding up.
>=20
> However, rounding half down/up is much more common than rounding up in =
the real=20
> world.
> When I expect moderate file size approximation, this 1GB difference is =
not=20
> ignorable.
>=20
> Is there any reason for this behavior?

Yes.  -h is a GNU extension, but the comparable -k and -s options have
behavior specified by POSIX.  And POSIX requires that these options
round up to the next integral number of units.  Therefore, it is as if
once you cross a unit boundary, ls is treating that unit as the block
size and rounding up to the effective number of blocks of that unit
size, since you've lost the fractional portion (that is, it's better to
overestimate than underestimate when it comes to disk usage).

http://pubs.opengroup.org/onlinepubs/9699919799/utilities/ls.html

under STDOUT: "If any of the -l, -n, -s, [XSI] [Option Start] -g, or -o
[Option End] options is specified, each list of files within the
directory shall be preceded by a status line indicating the number of
file system blocks occupied by files in the directory in 512-byte units
if the -k option is not specified, or 1024-byte units if the -k option
is specified, rounded up to the next integral number of units".

> Or is it just a mistake?

No, it is intentional.

--=20
Eric Blake   eblake@redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org


--------------enig3DD1BD332924F76C5B14A216
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Public key at http://people.redhat.com/eblake/eblake.gpg
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/

iQEcBAEBCAAGBQJNK293AAoJEKeha0olJ0NqW8YH/1Hvu5YCwbQ7art0yhXrYseo
QR4TReJ5Zh9oX3BM1aJFz5DkbSGF1NIiW8Yjj08pHdtvsCpzPr44f65AwFa3RGv6
wIFTD8Drw/1SbOUKgOnDrrxpcMquGa/ubrKsYUmbOsKijsbtV5A8TXW8LMFzOZuV
mzy5tTeP3MH8cUIzeThuyX0WJKH2QZYqjK15poZe/xgFiSQ3PLowJWHZ3ISG6qyx
4o67i5FEWQOxbrJh1REMHyYdVoGalj865L7G4y57RmZjboHvPDOQgeCWd+SrSSvl
2CFrYl1sHTJrCHCAAVJEE6gtQPOjoQxQDIF6dUoS4cl2VyEl3TwYF5dMwq+hZS0=
=TM+F
-----END PGP SIGNATURE-----

--------------enig3DD1BD332924F76C5B14A216--


From MAILER-DAEMON Mon Jan 10 16:16:11 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PcP63-0000Bm-CK
	for mharc-coreutils@gnu.org; Mon, 10 Jan 2011 16:16:11 -0500
Received: from [140.186.70.92] (port=38096 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PcP60-00009X-QM
	for coreutils@gnu.org; Mon, 10 Jan 2011 16:16:09 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <crockabiscuit@yahoo.com>) id 1PcP5z-00089A-M6
	for coreutils@gnu.org; Mon, 10 Jan 2011 16:16:08 -0500
Received: from nm13.bullet.mail.ne1.yahoo.com ([98.138.90.76]:33949)
	by eggs.gnu.org with smtp (Exim 4.71)
	(envelope-from <crockabiscuit@yahoo.com>) id 1PcP5z-000892-F9
	for coreutils@gnu.org; Mon, 10 Jan 2011 16:16:07 -0500
Received: from [98.138.90.52] by nm13.bullet.mail.ne1.yahoo.com with NNFMP;
	10 Jan 2011 21:16:06 -0000
Received: from [98.138.89.254] by tm5.bullet.mail.ne1.yahoo.com with NNFMP;
	10 Jan 2011 21:16:06 -0000
Received: from [127.0.0.1] by omp1046.mail.ne1.yahoo.com with NNFMP;
	10 Jan 2011 21:16:06 -0000
X-Yahoo-Newman-Property: ymail-3
X-Yahoo-Newman-Id: 840714.79355.bm@omp1046.mail.ne1.yahoo.com
Received: (qmail 36290 invoked by uid 60001); 10 Jan 2011 21:16:06 -0000
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024;
	t=1294694166; bh=DGqUjpfLJoStRC0DIGNJL6Kl46EXHu/5RxSRh5DClSQ=;
	h=Message-ID:X-YMail-OSG:Received:X-Mailer:References:Date:From:Subject:To:In-Reply-To:MIME-Version:Content-Type;
	b=HvcoIRVdWMapFi/h1jdjSfCp1Gpq5S6Hk+fW2QZdv+rlYhXgm6zT3IwNFoyCE94gxwTXPSeh54Ug7aIy3CYfkU1aqWj5GC6yYn2UixiCBRsKU9HlG3HYoawk6aPQeWvlHhTs72a0qpRjqWDAyjuT0FK0afM+41JubRFZkjPadQ0=
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com;
	h=Message-ID:X-YMail-OSG:Received:X-Mailer:References:Date:From:Subject:To:In-Reply-To:MIME-Version:Content-Type;
	b=Vu26+1mDpEvbDS1goO8g4oTzI1qSFnGl0bf1aYhgRk0naGtfQ5s2IHOxt90mM6cE5HiTboAckEQXcczPP3lR87d9822unipZkuSa00kar/+3Noq/zgJJiY56pGar+IShCB4k/K65v4uekA71UQ94WnnVdUtG9hHmO97PcFOpiM4=;
Message-ID: <737074.34855.qm@web120309.mail.ne1.yahoo.com>
X-YMail-OSG: p2czSO4VM1lJKzqTrtavXZN06uR1z5Hy.qFStVkhCABCMaz
	PFYIo9r.k.j5hbesxTdKbS8Gbmhxum62ytF93E_5sz.mhHDnkqPC_IXXL2x2
	tzTG5AcrkQX6LCQ68SC9XKFkw96SyXJMcTc36sYI2CnFpYRqOZ52JL_HnKKV
	ufGZTwtxkEmkj3Kemo2.AuIljMlP9N60_eYdY.JnlXmFvGBq0pGjL1U1NBgy
	rLTLLc5D2sJT0MidXjr97i_E-
Received: from [180.68.85.212] by web120309.mail.ne1.yahoo.com via HTTP;
	Mon, 10 Jan 2011 13:16:06 PST
X-Mailer: YahooMailRC/553 YahooMailWebService/0.8.107.285259
References: <766487.59657.qm@web120308.mail.ne1.yahoo.com>
	<4D2B6F77.9030805@redhat.com>
Date: Mon, 10 Jan 2011 13:16:06 -0800 (PST)
From: crocket <crockabiscuit@yahoo.com>
Subject: Re: [coreutils] "ls -lh" rounds up the size of files instead of
	rounding half up/down.
To: coreutils@gnu.org
In-Reply-To: <4D2B6F77.9030805@redhat.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 10 Jan 2011 21:16:09 -0000

Can you add an option to round half up/down?
There are times when users want to see close approximation rather than safely 
rounded up sizes.

----- Original Message ----
From: Eric Blake <eblake@redhat.com>
To: crocket <crockabiscuit@yahoo.com>
Cc: coreutils@gnu.org
Sent: Tue, January 11, 2011 5:43:35 AM
Subject: Re: [coreutils] "ls -lh" rounds up the size of files instead of 
rounding half up/down.

On 01/10/2011 01:30 PM, crocket wrote:
> There is a file whose size is 42949837312bytes.
> It's almost exactly 40.000153064727783GB.
> "ls -lh" displays it as 41GB.
> 
> It is resulted due to rounding up.
> 
> However, rounding half down/up is much more common than rounding up in the real 
>
>
> world.
> When I expect moderate file size approximation, this 1GB difference is not 
> ignorable.
> 
> Is there any reason for this behavior?

Yes.  -h is a GNU extension, but the comparable -k and -s options have
behavior specified by POSIX.  And POSIX requires that these options
round up to the next integral number of units.  Therefore, it is as if
once you cross a unit boundary, ls is treating that unit as the block
size and rounding up to the effective number of blocks of that unit
size, since you've lost the fractional portion (that is, it's better to
overestimate than underestimate when it comes to disk usage).

http://pubs.opengroup.org/onlinepubs/9699919799/utilities/ls.html

under STDOUT: "If any of the -l, -n, -s, [XSI] [Option Start] -g, or -o
[Option End] options is specified, each list of files within the
directory shall be preceded by a status line indicating the number of
file system blocks occupied by files in the directory in 512-byte units
if the -k option is not specified, or 1024-byte units if the -k option
is specified, rounded up to the next integral number of units".

> Or is it just a mistake?

No, it is intentional.

-- 
Eric Blake  eblake@redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org



      


From MAILER-DAEMON Mon Jan 10 16:21:18 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PcPB0-0003td-AM
	for mharc-coreutils@gnu.org; Mon, 10 Jan 2011 16:21:18 -0500
Received: from [140.186.70.92] (port=42743 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PcPAx-0003qi-6Z
	for coreutils@gnu.org; Mon, 10 Jan 2011 16:21:16 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1PcPAv-0000fo-MZ
	for coreutils@gnu.org; Mon, 10 Jan 2011 16:21:14 -0500
Received: from mx1.redhat.com ([209.132.183.28]:18110)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1PcPAv-0000fj-Ey
	for coreutils@gnu.org; Mon, 10 Jan 2011 16:21:13 -0500
Received: from int-mx12.intmail.prod.int.phx2.redhat.com
	(int-mx12.intmail.prod.int.phx2.redhat.com [10.5.11.25])
	by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id p0ALL9Ck013197
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK);
	Mon, 10 Jan 2011 16:21:09 -0500
Received: from [10.3.113.124] (ovpn-113-124.phx2.redhat.com [10.3.113.124])
	by int-mx12.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP
	id p0ALL9Rq007641; Mon, 10 Jan 2011 16:21:09 -0500
Message-ID: <4D2B7844.1080106@redhat.com>
Date: Mon, 10 Jan 2011 14:21:08 -0700
From: Eric Blake <eblake@redhat.com>
Organization: Red Hat
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US;
	rv:1.9.2.13) Gecko/20101209 Fedora/3.1.7-0.35.b3pre.fc14
	Lightning/1.0b3pre Mnenhy/0.8.3 Thunderbird/3.1.7
MIME-Version: 1.0
To: crocket <crockabiscuit@yahoo.com>, Coreutils <coreutils@gnu.org>
Subject: Re: [coreutils] "ls -lh" rounds up the size of files instead of
	rounding half up/down.
References: <766487.59657.qm@web120308.mail.ne1.yahoo.com>
	<4D2B6F77.9030805@redhat.com>
	<693084.39861.qm@web120307.mail.ne1.yahoo.com>
In-Reply-To: <693084.39861.qm@web120307.mail.ne1.yahoo.com>
X-Enigmail-Version: 1.1.2
OpenPGP: url=http://people.redhat.com/eblake/eblake.gpg
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature";
	boundary="------------enig6326B2975A6B150A9F403CD8"
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.25
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
Cc: 
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 10 Jan 2011 21:21:16 -0000

This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--------------enig6326B2975A6B150A9F403CD8
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

[re-adding the list, to publicize your request.  Also, it's nicer to
avoid top-posting on technical lists]

On 01/10/2011 02:14 PM, crocket wrote:
> Can you add an option to round half up/down?
> There are times when users want to see close approximation rather than =
safely=20
> rounded up sizes.

You are welcome to submit such a patch yourself.  Personally, rounding
up doesn't bother me, so I'm unlikely to take the time to write such a
patch myself when I have other more-pressing issues to tackle.

>> However, rounding half down/up is much more common than rounding up in=
 the real=20
>>
>>
>> world.
>> When I expect moderate file size approximation, this 1GB difference is=
 not=20
>> ignorable.
>>
>> Is there any reason for this behavior?
>=20
> Yes.  -h is a GNU extension, but the comparable -k and -s options have
> behavior specified by POSIX.  And POSIX requires that these options
> round up to the next integral number of units.

Meanwhile, if I'm really concerned about disk usage, I find 'df' to be
much more accurate than 'ls -l'; likewise, if quotas are a concern, I
find quota-specific tools to be more reliable than ls -l.  Sparse files
and hard links both come into play when determining disk usage, and
using 'ls' or 'du' to estimate disk usage is already inherently
inaccurate when compared to actually asking the operating system about
free disk space via 'df'.

--=20
Eric Blake   eblake@redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org


--------------enig6326B2975A6B150A9F403CD8
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Public key at http://people.redhat.com/eblake/eblake.gpg
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/

iQEcBAEBCAAGBQJNK3hEAAoJEKeha0olJ0Nq3p8H/jq2HkzQ8L82r3pT4LQED6QY
cPmvgGkDFHwqJKy8noS7DELA4/oSMhHjHU/jKo6D2PywzK0DWt9KbBxFZz2yN0yH
5h20qR/qq0bIkS/VCy50hTPlRBHH7NLGLDA0yNjl3DXfHdLSzW2kQ5H0CT7MnZK7
qhLTJ0Qljaai/5qZuaLNWDJOdjuD+AStJUMlwalsJ2ljRf2tFDP54SFcWCOaogOF
KQ6f2LxM5M3n6qChC5VmleW5FEevGNtsQWy17ulyN38eJL3dyCd07Lg+WIZfMs9Y
cpkE/D3aHotvHvoqNvR08y6FNB/VjSwiBVlgNSReMO/E5f+FY/yZnX/UxerAyME=
=Qh1D
-----END PGP SIGNATURE-----

--------------enig6326B2975A6B150A9F403CD8--


From MAILER-DAEMON Tue Jan 11 07:35:59 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PcdSB-0008Uq-Hy
	for mharc-coreutils@gnu.org; Tue, 11 Jan 2011 07:35:59 -0500
Received: from [140.186.70.92] (port=38293 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PcdS9-0008UO-7x
	for coreutils@gnu.org; Tue, 11 Jan 2011 07:35:58 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1PcdS7-0005Xl-Dv
	for coreutils@gnu.org; Tue, 11 Jan 2011 07:35:57 -0500
Received: from mail1.slb.deg.dub.stisp.net ([84.203.253.98]:3440)
	by eggs.gnu.org with smtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1PcdS7-0005XU-39
	for coreutils@gnu.org; Tue, 11 Jan 2011 07:35:55 -0500
Received: (qmail 1709 invoked from network); 11 Jan 2011 12:35:53 -0000
Received: from unknown (HELO ?192.168.2.25?) (84.203.137.218)
	by mail1.slb.deg.dub.stisp.net with SMTP; 11 Jan 2011 12:35:53 -0000
Message-ID: <4D2C4E91.1070908@draigBrady.com>
Date: Tue, 11 Jan 2011 12:35:29 +0000
From: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US;
	rv:1.9.1.8) Gecko/20100227 Thunderbird/3.0.3
MIME-Version: 1.0
To: coreutils@gnu.org
Subject: Re: [coreutils] join feature: auto-format
References: <4CACDEE5.6050908@gmail.com>	<4CAD0E47.7070005@draigBrady.com>	<4CAD9F55.6000001@draigBrady.com>	<4CAE06BA.2080402@gmail.com>
	<4CAE1099.4010702@draigBrady.com> <4D25AFED.1090902@draigBrady.com>
In-Reply-To: <4D25AFED.1090902@draigBrady.com>
X-Enigmail-Version: 1.0.1
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
X-detected-operating-system: by eggs.gnu.org: FreeBSD 4.6-4.9
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Tue, 11 Jan 2011 12:35:58 -0000

On 06/01/11 12:05, Pádraig Brady wrote:
> On 07/10/10 19:25, Pádraig Brady wrote:
>> On 07/10/10 18:43, Assaf Gordon wrote:
>>> Pádraig Brady wrote, On 10/07/2010 06:22 AM:
>>>> On 07/10/10 01:03, Pádraig Brady wrote:
>>>>> On 06/10/10 21:41, Assaf Gordon wrote:
>>>>>>
>>>>>> The "--auto-format" feature simply builds the "-o" format line automatically, based on the number of columns from both input files.
>>>>>
>>>>> Thanks for persisting with this and presenting a concise example.
>>>>> I agree that this is useful and can't think of a simple workaround.
>>>>> Perhaps the interface would be better as:
>>>>>
>>>>> -o {all (default), padded, FORMAT}
>>>>>
>>>>> where padded is the functionality you're suggesting?
>>>>
>>>> Thinking more about it, we mightn't need any new options at all.
>>>> Currently -e is redundant if -o is not specified.
>>>> So how about changing that so that if -e is specified
>>>> we operate as above by auto inserting empty fields?
>>>> Also I wouldn't base on the number of fields in the first line,
>>>> instead auto padding to the biggest number of fields
>>>> on the current lines under consideration.
>>>
>>> My concern is the principle of "least surprise" - if there are existing scripts/programs that specify "-e" without "-o" (doesn't make sense, but still possible) - this change will alter their behavior.
>>>
>>> Also, implying/forcing 'auto-format' when "-e" is used without "-o" might be a bit confusing.
>>
>> Well seeing as -e without -o currently does nothing,
>> I don't think we need to worry too much about changing that behavior.
>> Also to me, specifying -e EMPTY implicitly means I want
>> fields missing from one of the files replaced with EMPTY.
>>
>> Note POSIX is more explicit, and describes our current operation:
>>
>> -e EMPTY
>>   Replace empty output fields in the list selected by -o with EMPTY
>>
>> So changing that would be an extension to POSIX.
>> But I still think it makes sense.
>> I'll prepare a patch soon, to do as I describe above,
>> unless there are objections.
> 
> The attached changes `join` (from what's done on other platforms) so that...
> 
> `join -e` will automatically pad missing fields from one file
> so that the same number of fields are output from each file.
> Previously -e was only used for missing fields specified with -o or -j.
> 
> With this change join now does:
> 
> $ cat file1
> a 1 2
> b 1
> d 1 2
> 
> $ cat file2
> a 3 4
> b 3 4
> c 3 4
> 
> $ join -a1 -a2 -1 1 -2 1 -e. file1 file2
> a 1 2 3 4
> b 1 . 3 4
> c . . 3 4
> d 1 2 . .
> 
> $ join -a1 -a2 -1 1 -2 4 -e. file1 file2
> . . . . a 3 4
> . . . . b 3 4
> . . . . c 3 4
> a 1 2 . .
> b 1 .
> d 1 2 . .
> 
> $ join -a1 -a2 -1 4 -2 1 -e. file1 file2
> . a 1 2 . . .
> . b 1 . .
> . d 1 2 . . .
> a . . 3 4
> b . . 3 4
> c . . 3 4
> 
> $ join -a1 -a2 -1 4 -2 4 -e. file1 file2
> . a 1 2 a 3 4
> . a 1 2 b 3 4
> . a 1 2 c 3 4
> . b 1 . a 3 4
> . b 1 . b 3 4
> . b 1 . c 3 4
> . d 1 2 a 3 4
> . d 1 2 b 3 4
> . d 1 2 c 3 4
> 
> While -e without -o was previously a noop, and so could safely be extended IMHO,
> this will also change the behavior when with -e and -j are specified.
> Previously if -j > 1 was specified, and that field was missing,
> then -e would be used in its place, rather than the empty string.
> This still does that, but also does the padding.
> Without the -j issue I'd be 80:20 for just extending -e to auto pad,
> but given -j I'm 50:50.  The alternative it to select this with
> say '-o padded', but that's less discoverable, and complicates
> the interface somewhat.

Spending another few minutes on this, I realized
that we should not be trying to homogenize the number
of fields from each file, but rather the fields used
for a particular file in each line. The only sensible
basis for that is the first line as previously suggested.

The interface would be a little different for that.
I was thinking of:

  -o 'header'  Infer the format from the first line of each file

cheers,
Pádraig.


From MAILER-DAEMON Tue Jan 11 21:16:06 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PcqFq-00086C-Lr
	for mharc-coreutils@gnu.org; Tue, 11 Jan 2011 21:16:06 -0500
Received: from [140.186.70.92] (port=44113 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PcqFn-00085y-Oj
	for coreutils@gnu.org; Tue, 11 Jan 2011 21:16:04 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eggert@cs.ucla.edu>) id 1PcqFm-0007zy-5V
	for coreutils@gnu.org; Tue, 11 Jan 2011 21:16:03 -0500
Received: from smtp.cs.ucla.edu ([131.179.128.62]:39161)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eggert@cs.ucla.edu>) id 1PcqFl-0007zR-WA
	for coreutils@gnu.org; Tue, 11 Jan 2011 21:16:02 -0500
Received: from localhost (localhost.localdomain [127.0.0.1])
	by smtp.cs.ucla.edu (Postfix) with ESMTP id 8F7DA39E80DF
	for <coreutils@gnu.org>; Tue, 11 Jan 2011 18:16:00 -0800 (PST)
X-Virus-Scanned: amavisd-new at smtp.cs.ucla.edu
Received: from smtp.cs.ucla.edu ([127.0.0.1])
	by localhost (smtp.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10024)
	with ESMTP id d79valEAsZ9y for <coreutils@gnu.org>;
	Tue, 11 Jan 2011 18:15:59 -0800 (PST)
Received: from [192.168.1.10] (pool-71-189-109-235.lsanca.fios.verizon.net
	[71.189.109.235])
	by smtp.cs.ucla.edu (Postfix) with ESMTPSA id B664839E80DC
	for <coreutils@gnu.org>; Tue, 11 Jan 2011 18:15:59 -0800 (PST)
Message-ID: <4D2D0EDA.3060103@cs.ucla.edu>
Date: Tue, 11 Jan 2011 18:15:54 -0800
From: Paul Eggert <eggert@cs.ucla.edu>
Organization: UCLA Computer Science Department
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US;
	rv:1.9.2.13) Gecko/20101208 Thunderbird/3.1.7
MIME-Version: 1.0
To: Coreutils <coreutils@gnu.org>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
Subject: [coreutils] gnulib: Also use dtoastr and ldtoastr modules
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Wed, 12 Jan 2011 02:16:04 -0000

I installed this so that coreutils will bootstrap from the
latest gnulib's implementation of ftoastr etc.

>From 3292610bce67b331d7a99adc0171ed8b1b2f9b15 Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert@cs.ucla.edu>
Date: Tue, 11 Jan 2011 17:52:40 -0800
Subject: [PATCH 1/2] build: update gnulib submodule to latest

---
 gnulib |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/gnulib b/gnulib
index b3bff7a..104aca8 160000
--- a/gnulib
+++ b/gnulib
@@ -1 +1 @@
-Subproject commit b3bff7a868656967a8007faa831a89f3a1601eb2
+Subproject commit 104aca8300d01a80889f70f41a4dce7e044f9476
-- 
1.7.3


>From 11d864b36d83c7a60971b792581a5e61dab8bb40 Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert@cs.ucla.edu>
Date: Tue, 11 Jan 2011 18:07:15 -0800
Subject: [PATCH 2/2] gnulib: Also use dtoastr and ldtoastr modules.
 This adjusts to the recent splitting of the ftoastr module into 3
 <http://lists.gnu.org/archive/html/bug-gnulib/2011-01/msg00199.html>.
 * bootstrap.conf (gnulib_modules): Add dtoastr, ldtoastr,
 as coreutils needs all 3 modules now.

---
 bootstrap.conf |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)

diff --git a/bootstrap.conf b/bootstrap.conf
index 5f8171f..d994f52 100644
--- a/bootstrap.conf
+++ b/bootstrap.conf
@@ -61,6 +61,7 @@ gnulib_modules="
   dirfd
   dirname
   do-release-commit-and-tag
+  dtoastr
   dup2
   environ
   error
@@ -124,6 +125,7 @@ gnulib_modules="
   isblank
   lchmod
   lchown
+  ldtoastr
   lib-ignore
   linebuffer
   link
-- 
1.7.3



From MAILER-DAEMON Wed Jan 12 07:33:44 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PcztY-0001AD-4G
	for mharc-coreutils@gnu.org; Wed, 12 Jan 2011 07:33:44 -0500
Received: from [140.186.70.92] (port=40415 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PcztT-000183-PX
	for coreutils@gnu.org; Wed, 12 Jan 2011 07:33:42 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1PcztR-0003kL-0u
	for coreutils@gnu.org; Wed, 12 Jan 2011 07:33:38 -0500
Received: from mail1.slb.deg.dub.stisp.net ([84.203.253.98]:2787)
	by eggs.gnu.org with smtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1PcztQ-0003ip-KQ
	for coreutils@gnu.org; Wed, 12 Jan 2011 07:33:36 -0500
Received: (qmail 52911 invoked from network); 12 Jan 2011 12:33:33 -0000
Received: from unknown (HELO ?192.168.2.25?) (84.203.137.218)
	by mail1.slb.deg.dub.stisp.net with SMTP; 12 Jan 2011 12:33:33 -0000
Message-ID: <4D2D9F80.2060400@draigBrady.com>
Date: Wed, 12 Jan 2011 12:33:04 +0000
From: =?UTF-8?B?UMOhZHJhaWcgQnJhZHk=?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US;
	rv:1.9.1.8) Gecko/20100227 Thunderbird/3.0.3
MIME-Version: 1.0
To: Coreutils <coreutils@gnu.org>
X-Enigmail-Version: 1.0.1
Content-Type: multipart/mixed; boundary="------------060300060600050008010905"
X-detected-operating-system: by eggs.gnu.org: FreeBSD 4.6-4.9
Subject: [coreutils] [PATCH] maint: refactor to use read-file from gnulib
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Wed, 12 Jan 2011 12:33:42 -0000

This is a multi-part message in MIME format.
--------------060300060600050008010905
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Removes 100 lines of tricky code.

cheers,
PĂˇdraig.

--------------060300060600050008010905
Content-Type: text/x-patch;
 name="coreutils-read-file.diff"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename="coreutils-read-file.diff"

>From 85798e94a87b4c877da1b23a1c29ff15e642bfc7 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?P=C3=A1draig=20Brady?= <P@draigBrady.com>
Date: Tue, 11 Jan 2011 19:30:28 +0000
Subject: [PATCH] maint: refactor to use read-file from gnulib

* bootstrap.conf: Add the read-file module
* ptx.c: Replace the original code which would
needlessly read SIZE_MAX bytes of files larger than this.
* shuf.c: Replace the original code.
---
 bootstrap.conf |    1 +
 src/ptx.c      |   80 +++----------------------------------------------------
 src/shuf.c     |   48 +++------------------------------
 3 files changed, 11 insertions(+), 118 deletions(-)

diff --git a/bootstrap.conf b/bootstrap.conf
index 5f8171f..87de38f 100644
--- a/bootstrap.conf
+++ b/bootstrap.conf
@@ -170,6 +170,7 @@ gnulib_modules="
   quotearg
   randint
   randperm
+  read-file
   readlink
   readtokens
   readtokens0
diff --git a/src/ptx.c b/src/ptx.c
index 940f6e8..a45c4d8 100644
--- a/src/ptx.c
+++ b/src/ptx.c
@@ -28,6 +28,7 @@
 #include "fadvise.h"
 #include "quote.h"
 #include "quotearg.h"
+#include "read-file.h"
 #include "regex.h"
 #include "stdio--.h"
 #include "xstrtol.h"
@@ -62,10 +63,6 @@
    options.  Many of the "int" values below should be "size_t" or
    something else like that.  */
 
-/* Reallocation step when swallowing non regular files.  The value is not
-   the actual reallocation step, but its base two logarithm.  */
-#define SWALLOW_REALLOC_LOG 12
-
 /* Program options.  */
 
 enum Format
@@ -511,88 +508,21 @@ initialize_regex (void)
 static void
 swallow_file_in_memory (const char *file_name, BLOCK *block)
 {
-  int file_handle;		/* file descriptor number */
-  struct stat stat_block;	/* stat block for file */
-  size_t allocated_length;	/* allocated length of memory buffer */
   size_t used_length;		/* used length in memory buffer */
-  int read_length;		/* number of character gotten on last read */
 
   /* As special cases, a file name which is NULL or "-" indicates standard
      input, which is already opened.  In all other cases, open the file from
      its name.  */
   bool using_stdin = !file_name || !*file_name || STREQ (file_name, "-");
   if (using_stdin)
-    file_handle = STDIN_FILENO;
+    block->start = fread_file (stdin, &used_length);
   else
-    if ((file_handle = open (file_name, O_RDONLY)) < 0)
-      error (EXIT_FAILURE, errno, "%s", file_name);
-
-  /* If the file is a plain, regular file, allocate the memory buffer all at
-     once and swallow the file in one blow.  In other cases, read the file
-     repeatedly in smaller chunks until we have it all, reallocating memory
-     once in a while, as we go.  */
+    block->start = read_file (file_name, &used_length);
 
-  if (fstat (file_handle, &stat_block) < 0)
+  if (!block->start)
     error (EXIT_FAILURE, errno, "%s", file_name);
 
-  if (S_ISREG (stat_block.st_mode))
-    {
-      size_t in_memory_size;
-
-      fdadvise (file_handle, 0, 0, FADVISE_SEQUENTIAL);
-
-      block->start = xmalloc ((size_t) stat_block.st_size);
-
-      if ((in_memory_size = read (file_handle,
-                                  block->start, (size_t) stat_block.st_size))
-          != stat_block.st_size)
-        {
-#if MSDOS
-          /* On MSDOS, in memory size may be smaller than the file
-             size, because of end of line conversions.  But it can
-             never be smaller than half the file size, because the
-             minimum is when all lines are empty and terminated by
-             CR+LF.  */
-          if (in_memory_size != (size_t)-1
-              && in_memory_size >= stat_block.st_size / 2)
-            block->start = xrealloc (block->start, in_memory_size);
-          else
-#endif /* not MSDOS */
-
-            error (EXIT_FAILURE, errno, "%s", file_name);
-        }
-      block->end = block->start + in_memory_size;
-    }
-  else
-    {
-      block->start = xmalloc ((size_t) 1 << SWALLOW_REALLOC_LOG);
-      used_length = 0;
-      allocated_length = (1 << SWALLOW_REALLOC_LOG);
-
-      while (read_length = read (file_handle,
-                                 block->start + used_length,
-                                 allocated_length - used_length),
-             read_length > 0)
-        {
-          used_length += read_length;
-          if (used_length == allocated_length)
-            {
-              allocated_length += (1 << SWALLOW_REALLOC_LOG);
-              block->start
-                = xrealloc (block->start, allocated_length);
-            }
-        }
-
-      if (read_length < 0)
-        error (EXIT_FAILURE, errno, "%s", file_name);
-
-      block->end = block->start + used_length;
-    }
-
-  /* Close the file, but only if it was not the standard input.  */
-
-  if (! using_stdin && close (file_handle) != 0)
-    error (EXIT_FAILURE, errno, "%s", file_name);
+  block->end = block->start + used_length;
 }
 
 /* Sort and search routines.  */
diff --git a/src/shuf.c b/src/shuf.c
index 1d2261e..218ad6d 100644
--- a/src/shuf.c
+++ b/src/shuf.c
@@ -29,6 +29,7 @@
 #include "quotearg.h"
 #include "randint.h"
 #include "randperm.h"
+#include "read-file.h"
 #include "stdio--.h"
 #include "xstrtol.h"
 
@@ -147,52 +148,14 @@ read_input (FILE *in, char eolbyte, char ***pline)
 {
   char *p;
   char *buf = NULL;
+  size_t used;
   char *lim;
-  size_t alloc = 0;
-  size_t used = 0;
-  size_t next_alloc = (1 << 13) + 1;
-  size_t bytes_to_read;
-  size_t nread;
   char **line;
   size_t i;
   size_t n_lines;
-  int fread_errno;
-  struct stat instat;
 
-  if (fstat (fileno (in), &instat) == 0 && S_ISREG (instat.st_mode))
-    {
-      off_t file_size = instat.st_size;
-      off_t current_offset = ftello (in);
-      if (0 <= current_offset)
-        {
-          off_t remaining_size =
-            (current_offset < file_size ? file_size - current_offset : 0);
-          if (SIZE_MAX - 2 < remaining_size)
-            xalloc_die ();
-          next_alloc = remaining_size + 2;
-        }
-    }
-
-  do
-    {
-      if (alloc <= used + 1)
-        {
-          if (alloc == SIZE_MAX)
-            xalloc_die ();
-          alloc = next_alloc;
-          next_alloc = alloc * 2;
-          if (next_alloc < alloc)
-            next_alloc = SIZE_MAX;
-          buf = xrealloc (buf, alloc);
-        }
-
-      bytes_to_read = alloc - used - 1;
-      nread = fread (buf + used, sizeof (char), bytes_to_read, in);
-      used += nread;
-    }
-  while (nread == bytes_to_read);
-
-  fread_errno = errno;
+  if (!(buf = fread_file (in, &used)))
+    error (EXIT_FAILURE, errno, _("error reading input"));
 
   if (used && buf[used - 1] != eolbyte)
     buf[used++] = eolbyte;
@@ -209,7 +172,6 @@ read_input (FILE *in, char eolbyte, char ***pline)
   for (i = 1; i <= n_lines; i++)
     line[i] = p = next_line (p, eolbyte, lim - p);
 
-  errno = fread_errno;
   return n_lines;
 }
 
@@ -396,7 +358,7 @@ main (int argc, char **argv)
      doesn't have to worry about opening something other than
      stdin.  */
   if (! (echo || input_numbers_option_used (lo_input, hi_input))
-      && (ferror (stdin) || fclose (stdin) != 0))
+      && (fclose (stdin) != 0))
     error (EXIT_FAILURE, errno, _("read error"));
 
   permutation = randperm_new (randint_source, head_lines, n_lines);
-- 
1.7.3.4


--------------060300060600050008010905--


From MAILER-DAEMON Wed Jan 12 12:59:00 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1Pd4yK-0004BW-CI
	for mharc-coreutils@gnu.org; Wed, 12 Jan 2011 12:59:00 -0500
Received: from [140.186.70.92] (port=57783 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1Pd4yG-000473-Dj
	for coreutils@gnu.org; Wed, 12 Jan 2011 12:58:58 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Pd4y9-0006wR-R5
	for coreutils@gnu.org; Wed, 12 Jan 2011 12:58:50 -0500
Received: from mx.meyering.net ([82.230.74.64]:47458)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Pd4y9-0006w7-K1
	for coreutils@gnu.org; Wed, 12 Jan 2011 12:58:49 -0500
Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000)
	id EE421601B3; Wed, 12 Jan 2011 18:58:47 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: =?utf-8?Q?P=C3=A1draig?= Brady <P@draigBrady.com>
Subject: Re: [coreutils] [PATCH] maint: refactor to use read-file from gnulib
In-Reply-To: <4D2D9F80.2060400@draigBrady.com> (=?utf-8?Q?=22P=C3=A1draig?=
	Brady"'s message of "Wed, 12 Jan 2011 12:33:04 +0000")
References: <4D2D9F80.2060400@draigBrady.com>
Date: Wed, 12 Jan 2011 18:58:47 +0100
Message-ID: <87d3o2dmw8.fsf@meyering.net>
Lines: 23
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
Cc: Coreutils <coreutils@gnu.org>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Wed, 12 Jan 2011 17:58:59 -0000

P=C3=A1draig Brady wrote:
> Removes 100 lines of tricky code.
...
> Subject: [PATCH] maint: refactor to use read-file from gnulib
>
> * bootstrap.conf: Add the read-file module
> * ptx.c: Replace the original code which would
> needlessly read SIZE_MAX bytes of files larger than this.
> * shuf.c: Replace the original code.
> ---
>  bootstrap.conf |    1 +
>  src/ptx.c      |   80 +++-----------------------------------------------=
-----
>  src/shuf.c     |   48 +++------------------------------
>  3 files changed, 11 insertions(+), 118 deletions(-)

Nice.  It's not often we can remove so much code like that.

I glanced through it (looks fine), applied it, built
and ran a few smoke tests.

Push at will ;-)

Thanks!


From MAILER-DAEMON Wed Jan 12 13:04:19 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1Pd53S-0006gD-V4
	for mharc-coreutils@gnu.org; Wed, 12 Jan 2011 13:04:19 -0500
Received: from [140.186.70.92] (port=51954 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1Pd53P-0006aQ-OM
	for coreutils@gnu.org; Wed, 12 Jan 2011 13:04:17 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <assafgordon@gmail.com>) id 1Pd53L-00009S-Fs
	for coreutils@gnu.org; Wed, 12 Jan 2011 13:04:13 -0500
Received: from mail-qw0-f41.google.com ([209.85.216.41]:38025)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <assafgordon@gmail.com>) id 1Pd53L-00008z-CF
	for coreutils@gnu.org; Wed, 12 Jan 2011 13:04:11 -0500
Received: by qwa26 with SMTP id 26so844710qwa.0
	for <coreutils@gnu.org>; Wed, 12 Jan 2011 10:04:09 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma;
	h=domainkey-signature:message-id:disposition-notification-to:date
	:from:user-agent:mime-version:to:subject:references:in-reply-to
	:content-type:content-transfer-encoding;
	bh=iCn5NrUfzJUJEEph0vMuK84CKryTzvVx6OjNAT8CWXk=;
	b=cG2yAakRNVXrBQR7YrFXi8e3Bct3br94lw+GHCZ5eSw0/TVIYcxVQv7kIlJGt0pp9i
	mJG5BOJBfPuWwTXqqUvlYq6aGORGzEmqQ2ZzAOYGoYJEnPvpr4QKx5DpSeoegdWku7zP
	22RPjAbr/kSOrhWUie8kgRhqxRVzTeTYMtX9Q=
DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma;
	h=message-id:disposition-notification-to:date:from:user-agent
	:mime-version:to:subject:references:in-reply-to:content-type
	:content-transfer-encoding;
	b=YGlsjhKPlhhKaSXq5+7ggPEJiAKMw2KjTsSY7dQ6OFH0n5WxA1Ee660uU5JIIgh2SD
	2i1UaiorMbd1dj3fYSjpuv21JsonFBFATIgN46FcVFMAhvoF18ooQONUFWKoH277KXkl
	1DXGlWOFxwZTQt9p3MKk5/Gii94O2pRuNZqJU=
Received: by 10.224.45.132 with SMTP id e4mr1120411qaf.296.1294855449831;
	Wed, 12 Jan 2011 10:04:09 -0800 (PST)
Received: from [143.48.11.9] (tango.cshl.edu [143.48.11.9])
	by mx.google.com with ESMTPS id t7sm647969qcs.28.2011.01.12.10.04.07
	(version=TLSv1/SSLv3 cipher=RC4-MD5);
	Wed, 12 Jan 2011 10:04:07 -0800 (PST)
Message-ID: <4D2DED16.30101@gmail.com>
Date: Wed, 12 Jan 2011 13:04:06 -0500
From: Assaf Gordon <assafgordon@gmail.com>
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US;
	rv:1.9.2.9) Gecko/20100918 Icedove/3.1.4
MIME-Version: 1.0
To: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>, 
 coreutils@gnu.org
Subject: Re: [coreutils] join feature: auto-format
References: <4CACDEE5.6050908@gmail.com>	<4CAD0E47.7070005@draigBrady.com>	<4CAD9F55.6000001@draigBrady.com>	<4CAE06BA.2080402@gmail.com>	<4CAE1099.4010702@draigBrady.com>
	<4D25AFED.1090902@draigBrady.com> <4D2C4E91.1070908@draigBrady.com>
In-Reply-To: <4D2C4E91.1070908@draigBrady.com>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 2)
Cc: 
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Wed, 12 Jan 2011 18:04:17 -0000

P=E1draig Brady wrote, On 01/11/2011 07:35 AM:
>=20
> Spending another few minutes on this, I realized
> that we should not be trying to homogenize the number
> of fields from each file, but rather the fields used
> for a particular file in each line. The only sensible
> basis for that is the first line as previously suggested.
>=20
> The interface would be a little different for that.
> I was thinking of:
>=20
>   -o 'header'  Infer the format from the first line of each file
>=20
I second the idea of using the first line as the basis for the auto-forma=
tting,
but have reservation about the wording: '-o header' somewhat implies that=
 the first line has to be an actual header line (with column names or sim=
ilar), while it can just be the first line of actual data if the file doe=
sn't have a header line.

Something like '-o auto' might be less confusing.

Just my 2 cents,
 -gordon



From MAILER-DAEMON Wed Jan 12 13:58:50 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1Pd5uE-0001dj-KL
	for mharc-coreutils@gnu.org; Wed, 12 Jan 2011 13:58:50 -0500
Received: from [140.186.70.92] (port=58363 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1Pd5gR-00042e-Ak
	for coreutils@gnu.org; Wed, 12 Jan 2011 13:44:36 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jblaine@kickflop.net>) id 1Pd5fj-0000bq-UG
	for coreutils@gnu.org; Wed, 12 Jan 2011 13:43:53 -0500
Received: from caiajhbdcbef.dreamhost.com ([208.97.132.145]:59560
	helo=homiemail-a12.g.dreamhost.com)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jblaine@kickflop.net>) id 1Pd5fj-0000bD-OD
	for coreutils@gnu.org; Wed, 12 Jan 2011 13:43:51 -0500
Received: from homiemail-a12.g.dreamhost.com (localhost [127.0.0.1])
	by homiemail-a12.g.dreamhost.com (Postfix) with ESMTP id 67864714070
	for <coreutils@gnu.org>; Wed, 12 Jan 2011 10:43:48 -0800 (PST)
DomainKey-Signature: a=rsa-sha1; c=nofws; d=kickflop.net; h=message-id:date
	:from:mime-version:to:subject:content-type:
	content-transfer-encoding; q=dns; s=kickflop.net; b=HBiDZxIA5ujd
	BxzU/u3r5owPOiBu2zyHwl6x0x8UTEIpp6MoIzYmJcfsGVUGH3syvpbiELHMH68p
	KG4uPpSgJpUfxXMLsLDprM4RP7I2DjrhZTYJaGzvsexjRqkYmZY6HxkOL+rCN5ud
	rI3xiZBVcsyHC9xVR1997CMZ63ywi1Q=
DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=kickflop.net; h=message-id
	:date:from:mime-version:to:subject:content-type:
	content-transfer-encoding; s=kickflop.net; bh=Vr1+Fp08fAbDrboA/q
	0OCvLSwZ4=; b=7FSMKjPkN9atgM7CmaPv1q0Vzy6p50K0mSUXkQXhl8pq1doM4b
	35fWEEMga/77p/Tzmgtuyf1b+1abm2+c2grwKsc7HFFki3tWdmV17M6vAbfTc3qF
	hfmMufNewMLxaUX0BI08muH+BUlOXiUJ3+OnvywwLzKTTfuO+2s1rp7HA=
Received: from [192.168.1.3] (pool-108-9-57-66.tampfl.fios.verizon.net
	[108.9.57.66]) (using TLSv1 with cipher AES256-SHA (256/256 bits))
	(No client certificate requested)
	(Authenticated sender: jblaine@kickflop.net)
	by homiemail-a12.g.dreamhost.com (Postfix) with ESMTPSA id 3F48371406B
	for <coreutils@gnu.org>; Wed, 12 Jan 2011 10:43:48 -0800 (PST)
Message-ID: <4D2DF65B.2040500@kickflop.net>
Date: Wed, 12 Jan 2011 13:43:39 -0500
From: Jeff Blaine <jblaine@kickflop.net>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US;
	rv:1.9.2.13) Gecko/20101207 Thunderbird/3.1.7
MIME-Version: 1.0
To: coreutils@gnu.org
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
X-Mailman-Approved-At: Wed, 12 Jan 2011 13:58:48 -0500
Subject: [coreutils] basename/dirname can't handle stdin?
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Wed, 12 Jan 2011 18:44:37 -0000

I was surprised to find this today.

The following fails:

     echo /some/place/foo.txt | basename

This also fails:

     find /somewhere -print | xargs basename

This *does* work though, somehow, when the above does not:

     echo /some/place/foo.txt | xargs basename

What is going on here?  dirname is the same way.


From MAILER-DAEMON Wed Jan 12 14:02:24 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1Pd5xg-0003VQ-Jk
	for mharc-coreutils@gnu.org; Wed, 12 Jan 2011 14:02:24 -0500
Received: from [140.186.70.92] (port=43455 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1Pd5xe-0003Tz-Qx
	for coreutils@gnu.org; Wed, 12 Jan 2011 14:02:23 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1Pd5xa-0005IE-Nu
	for coreutils@gnu.org; Wed, 12 Jan 2011 14:02:22 -0500
Received: from mx1.redhat.com ([209.132.183.28]:7020)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1Pd5xa-0005I0-Gs
	for coreutils@gnu.org; Wed, 12 Jan 2011 14:02:18 -0500
Received: from int-mx02.intmail.prod.int.phx2.redhat.com
	(int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12])
	by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id p0CJ2GV9021491
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK);
	Wed, 12 Jan 2011 14:02:16 -0500
Received: from [10.3.113.124] (ovpn-113-124.phx2.redhat.com [10.3.113.124])
	by int-mx02.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP
	id p0CJ2FAx022918; Wed, 12 Jan 2011 14:02:15 -0500
Message-ID: <4D2DFAB7.6020809@redhat.com>
Date: Wed, 12 Jan 2011 12:02:15 -0700
From: Eric Blake <eblake@redhat.com>
Organization: Red Hat
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US;
	rv:1.9.2.13) Gecko/20101209 Fedora/3.1.7-0.35.b3pre.fc14
	Lightning/1.0b3pre Mnenhy/0.8.3 Thunderbird/3.1.7
MIME-Version: 1.0
To: Jeff Blaine <jblaine@kickflop.net>
Subject: Re: [coreutils] basename/dirname can't handle stdin?
References: <4D2DF65B.2040500@kickflop.net>
In-Reply-To: <4D2DF65B.2040500@kickflop.net>
X-Enigmail-Version: 1.1.2
OpenPGP: url=http://people.redhat.com/eblake/eblake.gpg
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature";
	boundary="------------enig94216D27BAFB098B941ED220"
X-Scanned-By: MIMEDefang 2.67 on 10.5.11.12
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Wed, 12 Jan 2011 19:02:23 -0000

This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--------------enig94216D27BAFB098B941ED220
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

On 01/12/2011 11:43 AM, Jeff Blaine wrote:
> I was surprised to find this today.
>=20
> The following fails:
>=20
>     echo /some/place/foo.txt | basename
>=20
> This also fails:
>=20
>     find /somewhere -print | xargs basename
>=20
> This *does* work though, somehow, when the above does not:
>=20
>     echo /some/place/foo.txt | xargs basename
>=20
> What is going on here?  dirname is the same way.

POSIX requires that basename and dirname be passed arguments; they are
not required to run as filters.

That said, it would be awesome if someone would contribute the patches
to emulate the BSD extensions of:

     basename [-a] [-s suffix] string [...]

=46rom there, it's not much harder to implement extensions that 0
arguments is an extension to operate in filter mode (although we may
want to require the use of basename --filter to make it obvious, and we
would certainly want basename --files-from0 to operate on find -print0
output).

But someone has to contribute the patch - are you willing?

--=20
Eric Blake   eblake@redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org


--------------enig94216D27BAFB098B941ED220
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Public key at http://people.redhat.com/eblake/eblake.gpg
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/

iQEcBAEBCAAGBQJNLfq3AAoJEKeha0olJ0NqLvsH/2wP/RJ1qgJtY73rrEOEU2oN
lFGYOBhzF4s8OA5yfYRhuvRURtzse/tjstFbqLZrIFO28ttFJ6hQNYoR7K5JffKr
UAgF0ejJ07LEnZf6o8B/34KMC/3ycj1IXW8ilCGDW8G+nsJPSE/cWkE83bwTRXkl
wWTzkjdBH3c0j/QffH9GR8fPIm9ep9AaZ01w70pb/QOCQpLYJB5x78zDE0IKVDdg
FFi9efj1m25AFZPG4tjzeLhiB1VWSn+v4UXXDEJMXpTNw59UR3lIoMOVCe99iV/Q
5MbK1+yEDrQRQQtM+yy66hIfcaHu268iSXF7Vm5pEeYbdvuFfKidu9HIGHEK7Os=
=6LLD
-----END PGP SIGNATURE-----

--------------enig94216D27BAFB098B941ED220--


From MAILER-DAEMON Wed Jan 12 14:04:03 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1Pd5zH-00046U-Nt
	for mharc-coreutils@gnu.org; Wed, 12 Jan 2011 14:04:03 -0500
Received: from [140.186.70.92] (port=44102 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1Pd5zF-000463-GR
	for coreutils@gnu.org; Wed, 12 Jan 2011 14:04:02 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jblaine@kickflop.net>) id 1Pd5zE-0005a7-54
	for coreutils@gnu.org; Wed, 12 Jan 2011 14:04:01 -0500
Received: from caiajhbdccac.dreamhost.com ([208.97.132.202]:46956
	helo=homiemail-a20.g.dreamhost.com)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jblaine@kickflop.net>) id 1Pd5zD-0005Zn-SL
	for coreutils@gnu.org; Wed, 12 Jan 2011 14:04:00 -0500
Received: from homiemail-a20.g.dreamhost.com (localhost [127.0.0.1])
	by homiemail-a20.g.dreamhost.com (Postfix) with ESMTP id 881A97EC072;
	Wed, 12 Jan 2011 11:03:58 -0800 (PST)
DomainKey-Signature: a=rsa-sha1; c=nofws; d=kickflop.net; h=message-id:date
	:from:mime-version:to:cc:subject:references:in-reply-to
	:content-type:content-transfer-encoding; q=dns; s=kickflop.net;
	b=JkWlbPoPK+J0VD9jFboi0TcNWJe71UBcDpA7c9LzNf/1d+yNZn4IkUCEfqVl9
	YIcaKAol0/3DJ1eJ3r8SFghuUaxAP7RIIFn1P8Bw+lmp8wC06JztcExx+Vu3faVf
	PeCVe6LmD7JX3rBzdhDwMRDzNww/4oU/vyQM3Y0B678SVY=
DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=kickflop.net; h=message-id
	:date:from:mime-version:to:cc:subject:references:in-reply-to
	:content-type:content-transfer-encoding; s=kickflop.net; bh=y4Vc
	9E+WmWekR3HIDnx8VyC3hUs=; b=f75+0UL3z8QuhGd50iRxFqyXTu4ivdRUu63V
	oxjbwvtFQrj1txUEu5jE5yE7x2Ta/DyZgD42j8zRKt0a2nGEptYaNbCpuBn/Njno
	z3+9UT6V5xGkpitpMIsS5emk1qAumocj9yTASfQENSAcrYVG5Sv7HSsPSFQ/daxK
	n1WWNI4=
Received: from [192.168.1.3] (pool-108-9-57-66.tampfl.fios.verizon.net
	[108.9.57.66]) (using TLSv1 with cipher AES256-SHA (256/256 bits))
	(No client certificate requested)
	(Authenticated sender: jblaine@kickflop.net)
	by homiemail-a20.g.dreamhost.com (Postfix) with ESMTPSA id 39AE97EC065; 
	Wed, 12 Jan 2011 11:03:58 -0800 (PST)
Message-ID: <4D2DFB15.2060902@kickflop.net>
Date: Wed, 12 Jan 2011 14:03:49 -0500
From: Jeff Blaine <jblaine@kickflop.net>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US;
	rv:1.9.2.13) Gecko/20101207 Thunderbird/3.1.7
MIME-Version: 1.0
To: Eric Blake <eblake@redhat.com>
Subject: Re: [coreutils] basename/dirname can't handle stdin?
References: <4D2DF65B.2040500@kickflop.net> <4D2DFAB7.6020809@redhat.com>
In-Reply-To: <4D2DFAB7.6020809@redhat.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Wed, 12 Jan 2011 19:04:02 -0000

Sure, I can try at least!

On 1/12/2011 2:02 PM, Eric Blake wrote:
> On 01/12/2011 11:43 AM, Jeff Blaine wrote:
>> I was surprised to find this today.
>>
>> The following fails:
>>
>>      echo /some/place/foo.txt | basename
>>
>> This also fails:
>>
>>      find /somewhere -print | xargs basename
>>
>> This *does* work though, somehow, when the above does not:
>>
>>      echo /some/place/foo.txt | xargs basename
>>
>> What is going on here?  dirname is the same way.
>
> POSIX requires that basename and dirname be passed arguments; they are
> not required to run as filters.
>
> That said, it would be awesome if someone would contribute the patches
> to emulate the BSD extensions of:
>
>       basename [-a] [-s suffix] string [...]
>
>  From there, it's not much harder to implement extensions that 0
> arguments is an extension to operate in filter mode (although we may
> want to require the use of basename --filter to make it obvious, and we
> would certainly want basename --files-from0 to operate on find -print0
> output).
>
> But someone has to contribute the patch - are you willing?
>


From MAILER-DAEMON Wed Jan 12 14:31:59 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1Pd6QJ-0004vm-0h
	for mharc-coreutils@gnu.org; Wed, 12 Jan 2011 14:31:59 -0500
Received: from [140.186.70.92] (port=49100 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1Pd6QD-0004qX-Bq
	for coreutils@gnu.org; Wed, 12 Jan 2011 14:31:57 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Pd6Pz-0003QP-Mo
	for coreutils@gnu.org; Wed, 12 Jan 2011 14:31:49 -0500
Received: from mx.meyering.net ([82.230.74.64]:57422)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Pd6Pz-0003Pn-Ev
	for coreutils@gnu.org; Wed, 12 Jan 2011 14:31:39 -0500
Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000)
	id 6202860154; Wed, 12 Jan 2011 20:31:37 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: Jeff Blaine <jblaine@kickflop.net>
Subject: Re: [coreutils] basename/dirname can't handle stdin?
In-Reply-To: <4D2DFB15.2060902@kickflop.net> (Jeff Blaine's message of "Wed,
	12 Jan 2011 14:03:49 -0500")
References: <4D2DF65B.2040500@kickflop.net> <4D2DFAB7.6020809@redhat.com>
	<4D2DFB15.2060902@kickflop.net>
Date: Wed, 12 Jan 2011 20:31:37 +0100
Message-ID: <871v4hex5y.fsf@meyering.net>
Lines: 29
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Wed, 12 Jan 2011 19:31:57 -0000

> On 1/12/2011 2:02 PM, Eric Blake wrote:
...
>> But someone has to contribute the patch - are you willing?

Jeff Blaine wrote:
> Sure, I can try at least!

Great!
Here are some coding guidelines:

    http://git.sv.gnu.org/cgit/coreutils.git/tree/HACKING

To start with, please look through the copyright assignment
section and start the process (assuming you're game), since
it can take a few weeks:

    http://git.sv.gnu.org/cgit/coreutils.git/tree/HACKING#n429


The arg-handling (and --files0-from=FILE handling) loop you'll use
should be very similar to the ones in wc.c, du.c and sort.c.
You'll probably want to use tests that are very similar to
the ones that test the behavior in those tools.

Note this bit at the top of tests/misc/wc-files0-from:

    #!/usr/bin/perl
    # Exercise wc's --files0-from option.
    # FIXME: keep this file in sync with tests/du/files0-from.


From MAILER-DAEMON Wed Jan 12 15:09:11 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1Pd70J-0003EW-JW
	for mharc-coreutils@gnu.org; Wed, 12 Jan 2011 15:09:11 -0500
Received: from [140.186.70.92] (port=46173 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1Pd70G-0003Df-Bn
	for coreutils@gnu.org; Wed, 12 Jan 2011 15:09:10 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <bob@proulx.com>) id 1Pd709-00036W-VW
	for coreutils@gnu.org; Wed, 12 Jan 2011 15:09:02 -0500
Received: from joseki.proulx.com ([216.17.153.58]:42176)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <bob@proulx.com>) id 1Pd709-00036K-Oy
	for coreutils@gnu.org; Wed, 12 Jan 2011 15:09:01 -0500
Received: from hysteria.proulx.com (hysteria.proulx.com [192.168.230.119])
	by joseki.proulx.com (Postfix) with ESMTP id AB19321311;
	Wed, 12 Jan 2011 13:09:00 -0700 (MST)
Received: by hysteria.proulx.com (Postfix, from userid 1000)
	id A59D92DD83; Wed, 12 Jan 2011 13:09:00 -0700 (MST)
Date: Wed, 12 Jan 2011 13:09:00 -0700
From: Bob Proulx <bob@proulx.com>
To: Jeff Blaine <jblaine@kickflop.net>
Subject: Re: [coreutils] basename/dirname can't handle stdin?
Message-ID: <20110112200900.GA27136@hysteria.proulx.com>
Mail-Followup-To: Jeff Blaine <jblaine@kickflop.net>, coreutils@gnu.org
References: <4D2DF65B.2040500@kickflop.net>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <4D2DF65B.2040500@kickflop.net>
User-Agent: Mutt/1.5.20 (2009-06-14)
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 1)
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Wed, 12 Jan 2011 20:09:10 -0000

Jeff Blaine wrote:
> The following fails:
>     echo /some/place/foo.txt | basename

As Eric noted, basename requires an argument.

> This also fails:
>     find /somewhere -print | xargs basename

Why did that fail?  Are you also getting an error message from find?

You usually want to include the xargs -r option if you think it might
not have any input at all.

  find /somewhere -print | xargs -r basename

> This *does* work though, somehow, when the above does not:
>     echo /some/place/foo.txt | xargs basename

That will invoke basename with one argument.  Which is fine.

You can verify what xargs will do by calling echo first.

  find /somewhere -print | xargs echo basename

  echo /some/place/foo.txt | xargs echo basename

Bob


From MAILER-DAEMON Wed Jan 12 17:09:26 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1Pd8sf-00037y-Vd
	for mharc-coreutils@gnu.org; Wed, 12 Jan 2011 17:09:26 -0500
Received: from [140.186.70.92] (port=52677 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1Pd8sd-00037s-Nq
	for coreutils@gnu.org; Wed, 12 Jan 2011 17:09:24 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jblaine@kickflop.net>) id 1Pd8sc-0001hy-CL
	for coreutils@gnu.org; Wed, 12 Jan 2011 17:09:23 -0500
Received: from mailbigip.dreamhost.com ([208.97.132.5]:53728
	helo=homiemail-a20.g.dreamhost.com)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jblaine@kickflop.net>) id 1Pd8sc-0001hq-3n
	for coreutils@gnu.org; Wed, 12 Jan 2011 17:09:22 -0500
Received: from homiemail-a20.g.dreamhost.com (localhost [127.0.0.1])
	by homiemail-a20.g.dreamhost.com (Postfix) with ESMTP id E2C407EC064;
	Wed, 12 Jan 2011 14:09:20 -0800 (PST)
DomainKey-Signature: a=rsa-sha1; c=nofws; d=kickflop.net; h=message-id:date
	:from:mime-version:to:cc:subject:references:in-reply-to
	:content-type:content-transfer-encoding; q=dns; s=kickflop.net;
	b=IFx5c4WsmeP+Bo+xQoOspUdzdDGG/mJLf44ex/OWIEixqARLl7x58e/fvpzv4
	IFb/enaDlaBpJTvU2v1J7JAL0Qqq5iKWnGPNjUFLg1mLw+3J1N/l4jiy9Bt57Gn0
	njpcJpbEKgE8ZRTXfDH9jAQSOUlhcgoBUWwUnzsF/NXWZg=
DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=kickflop.net; h=message-id
	:date:from:mime-version:to:cc:subject:references:in-reply-to
	:content-type:content-transfer-encoding; s=kickflop.net; bh=Npub
	A0XEOqi9DlaJfAIz/32ja9I=; b=mlL75BI47mpVzi77gIukV8lhCYJ3Eq+EmPN8
	Yt6aOLUNtAY8ul+JTPRlO+J9fxnS12i4Jtbbvt+JFqMj4Larbx1LTpmswXjEXRNl
	CtKKRvPCmW6p8N3J4EYiiMtf3PhPeuMejePlk9w8PxOJF1ZIaDim4KW5Mrru3v0P
	I8c9HMQ=
Received: from [192.168.1.3] (pool-108-9-57-66.tampfl.fios.verizon.net
	[108.9.57.66]) (using TLSv1 with cipher AES256-SHA (256/256 bits))
	(No client certificate requested)
	(Authenticated sender: jblaine@kickflop.net)
	by homiemail-a20.g.dreamhost.com (Postfix) with ESMTPSA id 98EE57EC061; 
	Wed, 12 Jan 2011 14:09:20 -0800 (PST)
Message-ID: <4D2E2687.9040107@kickflop.net>
Date: Wed, 12 Jan 2011 17:09:11 -0500
From: Jeff Blaine <jblaine@kickflop.net>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US;
	rv:1.9.2.13) Gecko/20101207 Thunderbird/3.1.7
MIME-Version: 1.0
To: Eric Blake <eblake@redhat.com>
Subject: Re: [coreutils] basename/dirname can't handle stdin?
References: <4D2DF65B.2040500@kickflop.net> <4D2DFAB7.6020809@redhat.com>
In-Reply-To: <4D2DFAB7.6020809@redhat.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Wed, 12 Jan 2011 22:09:24 -0000

Eric, now that I've got the prereqs in place,
built coreutils from git successfully, and read the
coreutils HACKING file, let's make sure I am on your
page:

In no order:

* Snarf getopt_long arg processing from wc.c or other
   you mentioned
* Add --filter (combined with the lack of file args) to handle
   processing of stdin
* Add --files-from0 per wc.c code (found it there)
* Add --suffix [suffix] to enable suffix killing (using
   SUFFIX as the 2nd arg will no longer work going forward)
* Update usage() obviously

Correct?  Anything else?

On 1/12/2011 2:02 PM, Eric Blake wrote:
> On 01/12/2011 11:43 AM, Jeff Blaine wrote:
>> I was surprised to find this today.
>>
>> The following fails:
>>
>>      echo /some/place/foo.txt | basename
>>
>> This also fails:
>>
>>      find /somewhere -print | xargs basename
>>
>> This *does* work though, somehow, when the above does not:
>>
>>      echo /some/place/foo.txt | xargs basename
>>
>> What is going on here?  dirname is the same way.
>
> POSIX requires that basename and dirname be passed arguments; they are
> not required to run as filters.
>
> That said, it would be awesome if someone would contribute the patches
> to emulate the BSD extensions of:
>
>       basename [-a] [-s suffix] string [...]
>
>  From there, it's not much harder to implement extensions that 0
> arguments is an extension to operate in filter mode (although we may
> want to require the use of basename --filter to make it obvious, and we
> would certainly want basename --files-from0 to operate on find -print0
> output).
>
> But someone has to contribute the patch - are you willing?
>


From MAILER-DAEMON Wed Jan 12 18:00:54 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1Pd9gU-00038N-H0
	for mharc-coreutils@gnu.org; Wed, 12 Jan 2011 18:00:54 -0500
Received: from [140.186.70.92] (port=36527 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1Pd9gP-000367-3R
	for coreutils@gnu.org; Wed, 12 Jan 2011 18:00:53 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Pd9gI-0003ut-QF
	for coreutils@gnu.org; Wed, 12 Jan 2011 18:00:48 -0500
Received: from mail1.slb.deg.dub.stisp.net ([84.203.253.98]:1227)
	by eggs.gnu.org with smtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Pd9gH-0003tm-FM
	for coreutils@gnu.org; Wed, 12 Jan 2011 18:00:42 -0500
Received: (qmail 60818 invoked from network); 12 Jan 2011 23:00:39 -0000
Received: from unknown (HELO ?192.168.2.25?) (84.203.137.218)
	by mail1.slb.deg.dub.stisp.net with SMTP; 12 Jan 2011 23:00:39 -0000
Message-ID: <4D2E3278.8040601@draigBrady.com>
Date: Wed, 12 Jan 2011 23:00:08 +0000
From: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US;
	rv:1.9.1.8) Gecko/20100227 Thunderbird/3.0.3
MIME-Version: 1.0
To: Assaf Gordon <assafgordon@gmail.com>
Subject: Re: [coreutils] join feature: auto-format
References: <4CACDEE5.6050908@gmail.com>	<4CAD0E47.7070005@draigBrady.com>	<4CAD9F55.6000001@draigBrady.com>	<4CAE06BA.2080402@gmail.com>	<4CAE1099.4010702@draigBrady.com>	<4D25AFED.1090902@draigBrady.com>
	<4D2C4E91.1070908@draigBrady.com> <4D2DED16.30101@gmail.com>
In-Reply-To: <4D2DED16.30101@gmail.com>
X-Enigmail-Version: 1.0.1
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
X-detected-operating-system: by eggs.gnu.org: FreeBSD 4.6-4.9
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Wed, 12 Jan 2011 23:00:53 -0000

On 12/01/11 18:04, Assaf Gordon wrote:
> Pádraig Brady wrote, On 01/11/2011 07:35 AM:
>>
>> Spending another few minutes on this, I realized
>> that we should not be trying to homogenize the number
>> of fields from each file, but rather the fields used
>> for a particular file in each line. The only sensible
>> basis for that is the first line as previously suggested.
>>
>> The interface would be a little different for that.
>> I was thinking of:
>>
>>   -o 'header'  Infer the format from the first line of each file
>>
> I second the idea of using the first line as the basis for the auto-formatting,
> but have reservation about the wording: '-o header' somewhat implies that the first line has to be an actual header line (with column names or similar), while it can just be the first line of actual data if the file doesn't have a header line.
> 
> Something like '-o auto' might be less confusing.

But less descriptive.
I'll go with "auto" as I can't think of anything better.

cheers,
Padraig.


From MAILER-DAEMON Thu Jan 13 02:53:07 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PdHzX-0007Xj-Lv
	for mharc-coreutils@gnu.org; Thu, 13 Jan 2011 02:53:07 -0500
Received: from [140.186.70.92] (port=51824 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PdHzU-0007W6-97
	for coreutils@gnu.org; Thu, 13 Jan 2011 02:53:06 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PdHzT-0005mK-8M
	for coreutils@gnu.org; Thu, 13 Jan 2011 02:53:04 -0500
Received: from mx.meyering.net ([82.230.74.64]:51281)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PdHzT-0005lx-0H
	for coreutils@gnu.org; Thu, 13 Jan 2011 02:53:03 -0500
Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000)
	id 859DF6008B; Thu, 13 Jan 2011 08:53:01 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: Jeff Blaine <jblaine@kickflop.net>
Subject: Re: [coreutils] basename/dirname can't handle stdin?
In-Reply-To: <4D2E2687.9040107@kickflop.net> (Jeff Blaine's message of "Wed,
	12 Jan 2011 17:09:11 -0500")
References: <4D2DF65B.2040500@kickflop.net> <4D2DFAB7.6020809@redhat.com>
	<4D2E2687.9040107@kickflop.net>
Date: Thu, 13 Jan 2011 08:53:01 +0100
Message-ID: <87bp3lck9u.fsf@meyering.net>
Lines: 50
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 13 Jan 2011 07:53:06 -0000

Jeff Blaine wrote:
> Eric, now that I've got the prereqs in place,
> built coreutils from git successfully, and read the
> coreutils HACKING file, let's make sure I am on your
> page:
>
> In no order:
>
> * Snarf getopt_long arg processing from wc.c or other
>   you mentioned

getopt_long?  There will be some of that (for the new options),
but most of the new code will be to process stdin, and you can
use code nearly identical to what wc uses:

  int i;
  ok = true;
  for (i = 0; /* */; i++)
    {
      bool skip_file = false;
      enum argv_iter_err ai_err;
      char *file_name = argv_iter (ai, &ai_err);
      ...

The preceding set-up code can be mostly-reused, too,

  bool read_tokens = false;
  struct argv_iterator *ai;
  if (files_from)
    {

with the exception that the "else"-block that sets up
to get names from argv.  I.e., while wc, du, sort can take
many arguments from argv, basename cannot.

> * Add --filter (combined with the lack of file args) to handle
>   processing of stdin
> * Add --files-from0 per wc.c code (found it there)

it's --files0-from

> * Add --suffix [suffix] to enable suffix killing (using
>   SUFFIX as the 2nd arg will no longer work going forward)
> * Update usage() obviously
>
> Correct?  Anything else?

  * Update NEWS to mention the new functionality.
  * Update doc/coreutils.texi
  * add tests to exercise the new behavior (as I mentioned earlier)


From MAILER-DAEMON Thu Jan 13 02:54:47 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PdI19-0007p5-Qt
	for mharc-coreutils@gnu.org; Thu, 13 Jan 2011 02:54:47 -0500
Received: from [140.186.70.92] (port=48328 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PdI17-0007oj-No
	for coreutils@gnu.org; Thu, 13 Jan 2011 02:54:46 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PdI16-0006BK-N1
	for coreutils@gnu.org; Thu, 13 Jan 2011 02:54:45 -0500
Received: from mx.meyering.net ([82.230.74.64]:51308)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PdI16-0006BB-9A
	for coreutils@gnu.org; Thu, 13 Jan 2011 02:54:44 -0500
Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000)
	id A59FB60114; Thu, 13 Jan 2011 08:54:43 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: Assaf Gordon <assafgordon@gmail.com>
Subject: Re: [coreutils] join feature: auto-format
In-Reply-To: <4D2E3278.8040601@draigBrady.com> (=?utf-8?Q?=22P=C3=A1draig?=
	Brady"'s message of "Wed, 12 Jan 2011 23:00:08 +0000")
References: <4CACDEE5.6050908@gmail.com> <4CAD0E47.7070005@draigBrady.com>
	<4CAD9F55.6000001@draigBrady.com> <4CAE06BA.2080402@gmail.com>
	<4CAE1099.4010702@draigBrady.com> <4D25AFED.1090902@draigBrady.com>
	<4D2C4E91.1070908@draigBrady.com> <4D2DED16.30101@gmail.com>
	<4D2E3278.8040601@draigBrady.com>
Date: Thu, 13 Jan 2011 08:54:43 +0100
Message-ID: <8762ttck70.fsf@meyering.net>
Lines: 29
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
Cc: =?utf-8?Q?P=C3=A1draig?= Brady <P@draigBrady.com>, coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 13 Jan 2011 07:54:46 -0000

P=C3=A1draig Brady wrote:

> On 12/01/11 18:04, Assaf Gordon wrote:
>> P=C3=A1draig Brady wrote, On 01/11/2011 07:35 AM:
>>>
>>> Spending another few minutes on this, I realized
>>> that we should not be trying to homogenize the number
>>> of fields from each file, but rather the fields used
>>> for a particular file in each line. The only sensible
>>> basis for that is the first line as previously suggested.
>>>
>>> The interface would be a little different for that.
>>> I was thinking of:
>>>
>>>   -o 'header'  Infer the format from the first line of each file
>>>
>> I second the idea of using the first line as the basis for the auto-form=
atting,
>> but have reservation about the wording: '-o header' somewhat implies
>> that the first line has to be an actual header line (with column
>> names or similar), while it can just be the first line of actual
>> data if the file doesn't have a header line.
>>
>> Something like '-o auto' might be less confusing.
>
> But less descriptive.
> I'll go with "auto" as I can't think of anything better.

-o auto is indeed more apt.
Thanks for the suggestion.


From MAILER-DAEMON Thu Jan 13 09:20:43 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PdO2d-0005FJ-KC
	for mharc-coreutils@gnu.org; Thu, 13 Jan 2011 09:20:43 -0500
Received: from [140.186.70.92] (port=54430 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PdO2a-0005CY-Rr
	for coreutils@gnu.org; Thu, 13 Jan 2011 09:20:42 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jblaine@kickflop.net>) id 1PdO2Z-0005MI-KC
	for coreutils@gnu.org; Thu, 13 Jan 2011 09:20:40 -0500
Received: from caiajhbdcaib.dreamhost.com ([208.97.132.81]:52772
	helo=homiemail-a23.g.dreamhost.com)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jblaine@kickflop.net>) id 1PdO2Z-0005M3-Cl
	for coreutils@gnu.org; Thu, 13 Jan 2011 09:20:39 -0500
Received: from homiemail-a23.g.dreamhost.com (localhost [127.0.0.1])
	by homiemail-a23.g.dreamhost.com (Postfix) with ESMTP id 6560B4B0058;
	Thu, 13 Jan 2011 06:20:38 -0800 (PST)
DomainKey-Signature: a=rsa-sha1; c=nofws; d=kickflop.net; h=message-id:date
	:from:mime-version:to:cc:subject:references:in-reply-to
	:content-type:content-transfer-encoding; q=dns; s=kickflop.net;
	b=aiCG4LNgzH/kOtZUFKsbtQz2vAevkroApqEYgeBY52vEDeBtT4Pqs5ySkb0Gf
	WVQw2gBpYIfbM5TcRwCAtLM8E3ecn5vExYtVtp1iEy+6nnuGO1UIT1u+5HWX3SnB
	jLe9e1CEHNviFZ3QgAOOE6/CpsrQfElMCiClfiJM7nsUIk=
DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=kickflop.net; h=message-id
	:date:from:mime-version:to:cc:subject:references:in-reply-to
	:content-type:content-transfer-encoding; s=kickflop.net; bh=t/Cf
	pHjseDlb0mfTWfjwjWgaJxM=; b=hxXbX2+cehcPlZr+bX3EoxKNp6FKsVJAtIc5
	GqZdr+1hdSDwT4FlGd8GNddjGt9H9sAE23ok7Z7x9BqOwa9NwS/3kU4tyc6V9/Ui
	fIUoGn8u12Rukbsjjzf1CBtz2GAuTkpEax8Qj7VKXFsvc9KqB4/qfXIbI14sRliS
	Mb9NZHs=
Received: from [192.168.1.3] (pool-108-9-57-66.tampfl.fios.verizon.net
	[108.9.57.66]) (using TLSv1 with cipher AES256-SHA (256/256 bits))
	(No client certificate requested)
	(Authenticated sender: jblaine@kickflop.net)
	by homiemail-a23.g.dreamhost.com (Postfix) with ESMTPSA id 0A8174B0071; 
	Thu, 13 Jan 2011 06:20:37 -0800 (PST)
Message-ID: <4D2F0A39.1080504@kickflop.net>
Date: Thu, 13 Jan 2011 09:20:41 -0500
From: Jeff Blaine <jblaine@kickflop.net>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US;
	rv:1.9.2.13) Gecko/20101207 Thunderbird/3.1.7
MIME-Version: 1.0
To: Jim Meyering <jim@meyering.net>
Subject: Re: [coreutils] basename/dirname can't handle stdin?
References: <4D2DF65B.2040500@kickflop.net>
	<4D2DFAB7.6020809@redhat.com>	<4D2E2687.9040107@kickflop.net>
	<87bp3lck9u.fsf@meyering.net>
In-Reply-To: <87bp3lck9u.fsf@meyering.net>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 13 Jan 2011 14:20:42 -0000

On 1/13/2011 2:53 AM, Jim Meyering wrote:
> Jeff Blaine wrote:
>> Eric, now that I've got the prereqs in place,
>> built coreutils from git successfully, and read the
>> coreutils HACKING file, let's make sure I am on your
>> page:
>>
>> In no order:
>>
>> * Snarf getopt_long arg processing from wc.c or other
>>    you mentioned
>
> getopt_long?  There will be some of that (for the new options),
> but most of the new code will be to process stdin, and you can
> use code nearly identical to what wc uses:
>
>    int i;
>    ok = true;
>    for (i = 0; /* */; i++)
>      {
>        bool skip_file = false;
>        enum argv_iter_err ai_err;
>        char *file_name = argv_iter (ai,&ai_err);
>        ...

Yup, already copied and reusing it.

> The preceding set-up code can be mostly-reused, too,
>
>    bool read_tokens = false;
>    struct argv_iterator *ai;
>    if (files_from)
>      {
>
> with the exception that the "else"-block that sets up
> to get names from argv.  I.e., while wc, du, sort can take
> many arguments from argv, basename cannot.

Yup, already copied and reusing it.

>> * Add --filter (combined with the lack of file args) to handle
>>    processing of stdin
>> * Add --files-from0 per wc.c code (found it there)
>
> it's --files0-from

Yes, sorry.

>> * Add --suffix [suffix] to enable suffix killing (using
>>    SUFFIX as the 2nd arg will no longer work going forward)
>> * Update usage() obviously
>>
>> Correct?  Anything else?
>
>    * Update NEWS to mention the new functionality.
>    * Update doc/coreutils.texi
>    * add tests to exercise the new behavior (as I mentioned earlier)

That goes without saying as it's in the HACKING file :)

This query to the list was to make sure we agreed on the
scope of what I was adding/changing.


From MAILER-DAEMON Thu Jan 13 10:00:16 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PdOeu-00028a-9c
	for mharc-coreutils@gnu.org; Thu, 13 Jan 2011 10:00:16 -0500
Received: from [140.186.70.92] (port=58509 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PdOeo-00025j-7t
	for coreutils@gnu.org; Thu, 13 Jan 2011 10:00:14 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <pth@suse.de>) id 1PdOei-0001j2-F2
	for coreutils@gnu.org; Thu, 13 Jan 2011 10:00:10 -0500
Received: from cantor.suse.de ([195.135.220.2]:59269 helo=mx1.suse.de)
	by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <pth@suse.de>)
	id 1PdOei-0001fl-88
	for coreutils@gnu.org; Thu, 13 Jan 2011 10:00:04 -0500
Received: from relay2.suse.de (charybdis-ext.suse.de [195.135.221.2])
	(using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
	(No client certificate requested)
	by mx1.suse.de (Postfix) with ESMTP id CE66493A00
	for <coreutils@gnu.org>; Thu, 13 Jan 2011 15:59:59 +0100 (CET)
Date: Thu, 13 Jan 2011 15:59:59 +0100
From: Philipp Thomas <pth@suse.de>
To: coreutils@gnu.org
Subject: Re: [coreutils] basename/dirname can't handle stdin?
Message-ID: <20110113145959.GD19224@paradies.suse.de>
References: <4D2DF65B.2040500@kickflop.net> <4D2DFAB7.6020809@redhat.com>
	<4D2E2687.9040107@kickflop.net> <87bp3lck9u.fsf@meyering.net>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <87bp3lck9u.fsf@meyering.net>
X-Operating-System: openSUSE - Kernel Linux 2.6.37-rc3-git6-8-desktop x86_64
Organization: SUSE LINUX Products GmbH, =?iso-8859-1?Q?G?=
	=?iso-8859-1?Q?F=3A_Markus_Rex=2C_HRB_16746_=28AG_N=FCrnberg=29?=
User-Agent: Mutt/1.5.21 (2010-09-15)
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.4-2.6
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 13 Jan 2011 15:00:14 -0000

* Jim Meyering (jim@meyering.net) [20110113 08:53]:

> > * Add --suffix [suffix] to enable suffix killing (using
> >   SUFFIX as the 2nd arg will no longer work going forward)

Using SUFFIX as the second argument IMO has to keep working as otherwise
you'll break tons of scripts!

Philipp


From MAILER-DAEMON Thu Jan 13 10:03:58 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PdOiT-0004Xg-Qj
	for mharc-coreutils@gnu.org; Thu, 13 Jan 2011 10:03:57 -0500
Received: from [140.186.70.92] (port=33076 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PdOiM-0004Si-80
	for coreutils@gnu.org; Thu, 13 Jan 2011 10:03:56 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PdOiG-0002vX-8F
	for coreutils@gnu.org; Thu, 13 Jan 2011 10:03:49 -0500
Received: from mx.meyering.net ([82.230.74.64]:55245)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PdOiG-0002vB-2y
	for coreutils@gnu.org; Thu, 13 Jan 2011 10:03:44 -0500
Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000)
	id CB76060194; Thu, 13 Jan 2011 16:03:42 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: Philipp Thomas <pth@suse.de>
Subject: Re: [coreutils] basename/dirname can't handle stdin?
In-Reply-To: <20110113145959.GD19224@paradies.suse.de> (Philipp Thomas's
	message of "Thu, 13 Jan 2011 15:59:59 +0100")
References: <4D2DF65B.2040500@kickflop.net> <4D2DFAB7.6020809@redhat.com>
	<4D2E2687.9040107@kickflop.net> <87bp3lck9u.fsf@meyering.net>
	<20110113145959.GD19224@paradies.suse.de>
Date: Thu, 13 Jan 2011 16:03:42 +0100
Message-ID: <87zkr46e29.fsf@meyering.net>
Lines: 15
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 13 Jan 2011 15:03:56 -0000

Philipp Thomas wrote:

> * Jim Meyering (jim@meyering.net) [20110113 08:53]:
>
>> > * Add --suffix [suffix] to enable suffix killing (using
>> >   SUFFIX as the 2nd arg will no longer work going forward)
>
> Using SUFFIX as the second argument IMO has to keep working as otherwise
> you'll break tons of scripts!

What I think he meant is that we'll need a --suffix=SUFF
option to specify a suffix, since when in --filter mode,
each input will be treated as a file/dir name.

Whereas, on the command line, we'll still accept only 1 or 2 args.


From MAILER-DAEMON Thu Jan 13 10:08:10 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PdOmX-0007CV-Ps
	for mharc-coreutils@gnu.org; Thu, 13 Jan 2011 10:08:09 -0500
Received: from [140.186.70.92] (port=58971 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PdOmU-00078t-1Z
	for coreutils@gnu.org; Thu, 13 Jan 2011 10:08:08 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1PdOmS-0003vC-K0
	for coreutils@gnu.org; Thu, 13 Jan 2011 10:08:05 -0500
Received: from mx1.redhat.com ([209.132.183.28]:22938)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1PdOmS-0003v0-Am
	for coreutils@gnu.org; Thu, 13 Jan 2011 10:08:04 -0500
Received: from int-mx01.intmail.prod.int.phx2.redhat.com
	(int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11])
	by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id p0DF83hI023795
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK);
	Thu, 13 Jan 2011 10:08:03 -0500
Received: from [10.3.113.124] (ovpn-113-124.phx2.redhat.com [10.3.113.124])
	by int-mx01.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP
	id p0DF82fH004890; Thu, 13 Jan 2011 10:08:03 -0500
Message-ID: <4D2F1551.4090001@redhat.com>
Date: Thu, 13 Jan 2011 08:08:01 -0700
From: Eric Blake <eblake@redhat.com>
Organization: Red Hat
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US;
	rv:1.9.2.13) Gecko/20101209 Fedora/3.1.7-0.35.b3pre.fc14
	Lightning/1.0b3pre Mnenhy/0.8.3 Thunderbird/3.1.7
MIME-Version: 1.0
To: Philipp Thomas <pth@suse.de>
Subject: Re: [coreutils] basename/dirname can't handle stdin?
References: <4D2DF65B.2040500@kickflop.net>
	<4D2DFAB7.6020809@redhat.com>	<4D2E2687.9040107@kickflop.net>
	<87bp3lck9u.fsf@meyering.net>
	<20110113145959.GD19224@paradies.suse.de>
In-Reply-To: <20110113145959.GD19224@paradies.suse.de>
X-Enigmail-Version: 1.1.2
OpenPGP: url=http://people.redhat.com/eblake/eblake.gpg
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature";
	boundary="------------enig3F33129144FB5B19AC7F4DA3"
X-Scanned-By: MIMEDefang 2.67 on 10.5.11.11
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 13 Jan 2011 15:08:09 -0000

This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--------------enig3F33129144FB5B19AC7F4DA3
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

On 01/13/2011 07:59 AM, Philipp Thomas wrote:
> * Jim Meyering (jim@meyering.net) [20110113 08:53]:
>=20
>>> * Add --suffix [suffix] to enable suffix killing (using
>>>   SUFFIX as the 2nd arg will no longer work going forward)
>=20
> Using SUFFIX as the second argument IMO has to keep working as otherwis=
e
> you'll break tons of scripts!

That's the whole point of the BSD -a/-s options, and extensible to our
proposal for --files0-from.

basename str1 str2: POSIX behavior (str2 is suffix)
basename -a str1 str2: filter behavior, print basenames of both str1 and
str2 without suffix
basename -s suffix: filter mode, reads from stdin, and chops suffix
basename: filter mode, reads from stdin
basename -s str2 str1: explicit suffix, same as basename str1 str2
basename -s suffix str1 str2: filter behavior, print basenames of both
str1 and str2 without suffix
basename -a -s suffix str1 str2: same (-a is only required for exactly
two arguements, and is implied for 0 or for more than 2 args)
basename --files0-from=3Dfile -s suffix: filter mode, input is NUL rather=

than newline terminated, so output is NUL terminated as well

--=20
Eric Blake   eblake@redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org


--------------enig3F33129144FB5B19AC7F4DA3
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Public key at http://people.redhat.com/eblake/eblake.gpg
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/

iQEcBAEBCAAGBQJNLxVSAAoJEKeha0olJ0NqkYwH/1nd4mvWbEF9EQDOS+O4Q393
OxTGtc277yxMmVBo+0s+Tkd3Ho8t/AgRgF2B/ePPAYdJCVuVQsvnBAtIhuGu3Ep2
5yXX7QIn6gkRwup/gCI6EOWL7/C1usOyjgpOR6bpnD8wmCYBMqRrWixD4NEfHJG+
7lMQwbuFWGN2tq7fb+sI8omdRr2iDaIwArXLX8TjT7M9Iu6g4bofq9REOhV4Z2Yb
0m7FxoEtMSyyDrJ/Tvd3ZgfOw2Q6dyViPEQORsemfNipen4ZHHXo5dedhbIqDdz6
uOMJTAtnf6gyHM6a8DWKTl3EIFnd/3MkgDdj2QRno7fgQ9rV0vuOlN/V2yePtPk=
=FNE8
-----END PGP SIGNATURE-----

--------------enig3F33129144FB5B19AC7F4DA3--


From MAILER-DAEMON Thu Jan 13 10:10:42 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PdOoz-0000Qa-U4
	for mharc-coreutils@gnu.org; Thu, 13 Jan 2011 10:10:42 -0500
Received: from [140.186.70.92] (port=41698 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PdOow-0000Oy-Qe
	for coreutils@gnu.org; Thu, 13 Jan 2011 10:10:39 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1PdOou-0004UL-DZ
	for coreutils@gnu.org; Thu, 13 Jan 2011 10:10:38 -0500
Received: from mx1.redhat.com ([209.132.183.28]:45773)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1PdOou-0004U9-5Y
	for coreutils@gnu.org; Thu, 13 Jan 2011 10:10:36 -0500
Received: from int-mx10.intmail.prod.int.phx2.redhat.com
	(int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23])
	by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id p0DFAZLV021314
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK);
	Thu, 13 Jan 2011 10:10:35 -0500
Received: from [10.3.113.124] (ovpn-113-124.phx2.redhat.com [10.3.113.124])
	by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP
	id p0DFAY6N010774; Thu, 13 Jan 2011 10:10:34 -0500
Message-ID: <4D2F15EA.30908@redhat.com>
Date: Thu, 13 Jan 2011 08:10:34 -0700
From: Eric Blake <eblake@redhat.com>
Organization: Red Hat
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US;
	rv:1.9.2.13) Gecko/20101209 Fedora/3.1.7-0.35.b3pre.fc14
	Lightning/1.0b3pre Mnenhy/0.8.3 Thunderbird/3.1.7
MIME-Version: 1.0
To: Jim Meyering <jim@meyering.net>
Subject: Re: [coreutils] basename/dirname can't handle stdin?
References: <4D2DF65B.2040500@kickflop.net>
	<4D2DFAB7.6020809@redhat.com>	<4D2E2687.9040107@kickflop.net>
	<87bp3lck9u.fsf@meyering.net>	<20110113145959.GD19224@paradies.suse.de>
	<87zkr46e29.fsf@meyering.net>
In-Reply-To: <87zkr46e29.fsf@meyering.net>
X-Enigmail-Version: 1.1.2
OpenPGP: url=http://people.redhat.com/eblake/eblake.gpg
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature";
	boundary="------------enig2EE31B68631E89415F7547CE"
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
Cc: Philipp Thomas <pth@suse.de>, coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 13 Jan 2011 15:10:40 -0000

This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--------------enig2EE31B68631E89415F7547CE
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

On 01/13/2011 08:03 AM, Jim Meyering wrote:
> Philipp Thomas wrote:
>=20
>> * Jim Meyering (jim@meyering.net) [20110113 08:53]:
>>
>>>> * Add --suffix [suffix] to enable suffix killing (using
>>>>   SUFFIX as the 2nd arg will no longer work going forward)
>>
>> Using SUFFIX as the second argument IMO has to keep working as otherwi=
se
>> you'll break tons of scripts!
>=20
> What I think he meant is that we'll need a --suffix=3DSUFF
> option to specify a suffix, since when in --filter mode,
> each input will be treated as a file/dir name.
>=20
> Whereas, on the command line, we'll still accept only 1 or 2 args.

BSD basename accepts more than 2 args.

basename arg1 arg2 arg3 - no suffix, and print out three lines

That is, the suffix is available if either:
you pass exactly two args, and -a is not used
you pass the -s/--suffix option

(well, the long option spelling of --suffix would be new to GNU, but
makes sense).

Likewise, BSD basename accepts -a (how about --all as the long option
spelling) to override 2-argument becoming a suffix situation.

--=20
Eric Blake   eblake@redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org


--------------enig2EE31B68631E89415F7547CE
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Public key at http://people.redhat.com/eblake/eblake.gpg
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/

iQEcBAEBCAAGBQJNLxXqAAoJEKeha0olJ0NqZ3wH+wXlvcLOyeNrFBDEh5bmKxzV
GBscpgxwfSlQZDmdCYZg/mzPfb8Ewhog9XDOPgjRd2Er576KqER5W5Bm9bo4Ottw
nWwY/YDjXppp9574FFLEKc6UoDSfOLtt8/LTQw1nzv07XAV6sgK8/KjAV10iBdEV
WUowdVzMKTCZelxFLfoPqE4qmuqFQ4oEzH/8Ju1ksMMfNdgdm6fZHzrZUne9Q4H5
tvM3NttNYz79PsESrr2JPmikl2AtXMo2JzdANJTSx7HCU3L3zBcpbA9LunPY+u8J
AJllbzhLRcX1PcqFGbXkNx7WcuMO70UbotkuoL8QilBeGFibBRKCgRIJsjkT3pI=
=jO6M
-----END PGP SIGNATURE-----

--------------enig2EE31B68631E89415F7547CE--


From MAILER-DAEMON Thu Jan 13 10:14:32 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PdOsh-0002m2-Pv
	for mharc-coreutils@gnu.org; Thu, 13 Jan 2011 10:14:31 -0500
Received: from [140.186.70.92] (port=43909 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PdOsg-0002kz-5i
	for coreutils@gnu.org; Thu, 13 Jan 2011 10:14:31 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PdOsd-0005OZ-K2
	for coreutils@gnu.org; Thu, 13 Jan 2011 10:14:30 -0500
Received: from mx.meyering.net ([82.230.74.64]:58709)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PdOsd-0005OL-Dj
	for coreutils@gnu.org; Thu, 13 Jan 2011 10:14:27 -0500
Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000)
	id E4D156008B; Thu, 13 Jan 2011 16:14:25 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: Eric Blake <eblake@redhat.com>
Subject: Re: [coreutils] basename/dirname can't handle stdin?
In-Reply-To: <4D2F15EA.30908@redhat.com> (Eric Blake's message of "Thu, 13 Jan
	2011 08:10:34 -0700")
References: <4D2DF65B.2040500@kickflop.net> <4D2DFAB7.6020809@redhat.com>
	<4D2E2687.9040107@kickflop.net> <87bp3lck9u.fsf@meyering.net>
	<20110113145959.GD19224@paradies.suse.de>
	<87zkr46e29.fsf@meyering.net> <4D2F15EA.30908@redhat.com>
Date: Thu, 13 Jan 2011 16:14:25 +0100
Message-ID: <87tyhc6dke.fsf@meyering.net>
Lines: 36
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
Cc: Philipp Thomas <pth@suse.de>, coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 13 Jan 2011 15:14:31 -0000

Eric Blake wrote:
> On 01/13/2011 08:03 AM, Jim Meyering wrote:
>> Philipp Thomas wrote:
>>
>>> * Jim Meyering (jim@meyering.net) [20110113 08:53]:
>>>
>>>>> * Add --suffix [suffix] to enable suffix killing (using
>>>>>   SUFFIX as the 2nd arg will no longer work going forward)
>>>
>>> Using SUFFIX as the second argument IMO has to keep working as otherwise
>>> you'll break tons of scripts!
>>
>> What I think he meant is that we'll need a --suffix=SUFF
>> option to specify a suffix, since when in --filter mode,
>> each input will be treated as a file/dir name.
>>
>> Whereas, on the command line, we'll still accept only 1 or 2 args.
>
> BSD basename accepts more than 2 args.

I missed the addition of -a to the scope, but agree that it is worth adding.
The patch to add that new option belongs in a separate change-set, of course.

> basename arg1 arg2 arg3 - no suffix, and print out three lines
>
> That is, the suffix is available if either:
> you pass exactly two args, and -a is not used
> you pass the -s/--suffix option
>
> (well, the long option spelling of --suffix would be new to GNU, but
> makes sense).
>
> Likewise, BSD basename accepts -a (how about --all as the long option
> spelling) to override 2-argument becoming a suffix situation.

Sure.


From MAILER-DAEMON Thu Jan 13 12:29:09 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PdQyy-0004BC-V0
	for mharc-coreutils@gnu.org; Thu, 13 Jan 2011 12:29:08 -0500
Received: from [140.186.70.92] (port=34941 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PdQyv-00048k-RB
	for coreutils@gnu.org; Thu, 13 Jan 2011 12:29:07 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1PdQyu-0000NI-AX
	for coreutils@gnu.org; Thu, 13 Jan 2011 12:29:05 -0500
Received: from mail1.slb.deg.dub.stisp.net ([84.203.253.98]:6990)
	by eggs.gnu.org with smtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1PdQyu-0000Mn-0P
	for coreutils@gnu.org; Thu, 13 Jan 2011 12:29:04 -0500
Received: (qmail 19565 invoked from network); 13 Jan 2011 17:29:00 -0000
Received: from unknown (HELO ?192.168.2.25?) (84.203.137.218)
	by mail1.slb.deg.dub.stisp.net with SMTP; 13 Jan 2011 17:29:00 -0000
Message-ID: <4D2F363A.6070408@draigBrady.com>
Date: Thu, 13 Jan 2011 17:28:26 +0000
From: =?UTF-8?B?UMOhZHJhaWcgQnJhZHk=?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US;
	rv:1.9.1.8) Gecko/20100227 Thunderbird/3.0.3
MIME-Version: 1.0
To: Coreutils <coreutils@gnu.org>
X-Enigmail-Version: 1.0.1
Content-Type: multipart/mixed; boundary="------------060503060003050805070904"
X-detected-operating-system: by eggs.gnu.org: FreeBSD 4.6-4.9
Subject: [coreutils] [PATCH] maint: trivial system header file cleanups
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 13 Jan 2011 17:29:07 -0000

This is a multi-part message in MIME format.
--------------060503060003050805070904
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit


--------------060503060003050805070904
Content-Type: text/x-patch;
 name="system-headers.diff"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename="system-headers.diff"

>From e1aaf8903db97f3240b1551fd6936ccdc652dfc8 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?P=C3=A1draig=20Brady?= <P@draigBrady.com>
Date: Thu, 13 Jan 2011 09:36:38 +0000
Subject: [PATCH] maint: trivial system header file cleanups

* src/system.h: Note where it should be included, and
make ordering check portable to GLIBC > 2
* src/copy.c: Move <sys/ioctl.h> along with other system headers
as is done elsewhere.
* src/install.c: Move <sys/wait.h> along with other system headers
as is done elsewhere.
* src/ptx.c: Include <regex.h> rather than "regex.h" as
is done elsewhere.  Note <regex.h> is kept after "system.h"
as per commit dba300a0.
---
 src/copy.c    |    3 +--
 src/install.c |    3 +--
 src/ptx.c     |    2 +-
 src/system.h  |    8 ++++++--
 4 files changed, 9 insertions(+), 7 deletions(-)

diff --git a/src/copy.c b/src/copy.c
index a160952..9a014ad 100644
--- a/src/copy.c
+++ b/src/copy.c
@@ -19,6 +19,7 @@
 #include <config.h>
 #include <stdio.h>
 #include <assert.h>
+#include <sys/ioctl.h>
 #include <sys/types.h>
 #include <selinux/selinux.h>
 
@@ -61,8 +62,6 @@
 # include "verror.h"
 #endif
 
-#include <sys/ioctl.h>
-
 #ifndef HAVE_FCHOWN
 # define HAVE_FCHOWN false
 # define fchown(fd, uid, gid) (-1)
diff --git a/src/install.c b/src/install.c
index c34e4dc..cebb642 100644
--- a/src/install.c
+++ b/src/install.c
@@ -24,6 +24,7 @@
 #include <pwd.h>
 #include <grp.h>
 #include <selinux/selinux.h>
+#include <sys/wait.h>
 
 #include "system.h"
 #include "backupfile.h"
@@ -48,8 +49,6 @@
 
 #define AUTHORS proper_name ("David MacKenzie")
 
-#include <sys/wait.h>
-
 static int selinux_enabled = 0;
 static bool use_default_selinux_context = true;
 
diff --git a/src/ptx.c b/src/ptx.c
index 940f6e8..6465618 100644
--- a/src/ptx.c
+++ b/src/ptx.c
@@ -22,13 +22,13 @@
 #include <getopt.h>
 #include <sys/types.h>
 #include "system.h"
+#include <regex.h>
 #include "argmatch.h"
 #include "diacrit.h"
 #include "error.h"
 #include "fadvise.h"
 #include "quote.h"
 #include "quotearg.h"
-#include "regex.h"
 #include "stdio--.h"
 #include "xstrtol.h"
 
diff --git a/src/system.h b/src/system.h
index 3b32cbd..b86e570 100644
--- a/src/system.h
+++ b/src/system.h
@@ -14,11 +14,15 @@
    You should have received a copy of the GNU General Public License
    along with this program.  If not, see <http://www.gnu.org/licenses/>.  */
 
+/* Include this file _after_ system headers if possible.  */
+
 #include <alloca.h>
 
-/* Include sys/types.h before this file.  */
+/* Include <sys/types.h> before this file.
+   Note this doesn't warn if we're included
+   before all system headers.  */
 
-#if 2 <= __GLIBC__ && 2 <= __GLIBC_MINOR__
+#if 2 < __GLIBC__ || ( 2 == ___GLIBC__ && 2 <= __GLIBC_MINOR__ )
 # if ! defined _SYS_TYPES_H
 you must include <sys/types.h> before including this file
 # endif
-- 
1.7.3.4


--------------060503060003050805070904--


From MAILER-DAEMON Thu Jan 13 18:15:30 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PdWOA-00048P-5i
	for mharc-coreutils@gnu.org; Thu, 13 Jan 2011 18:15:30 -0500
Received: from [140.186.70.92] (port=46711 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PdWO7-00047Y-Q1
	for coreutils@gnu.org; Thu, 13 Jan 2011 18:15:28 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jblaine@kickflop.net>) id 1PdWO6-0005Jx-52
	for coreutils@gnu.org; Thu, 13 Jan 2011 18:15:27 -0500
Received: from caiajhbdcbhh.dreamhost.com ([208.97.132.177]:54208
	helo=homiemail-a18.g.dreamhost.com)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jblaine@kickflop.net>) id 1PdWO5-0005JP-RK
	for coreutils@gnu.org; Thu, 13 Jan 2011 18:15:26 -0500
Received: from homiemail-a18.g.dreamhost.com (localhost [127.0.0.1])
	by homiemail-a18.g.dreamhost.com (Postfix) with ESMTP id 100BF250071
	for <coreutils@gnu.org>; Thu, 13 Jan 2011 15:15:24 -0800 (PST)
DomainKey-Signature: a=rsa-sha1; c=nofws; d=kickflop.net; h=message-id:date
	:from:mime-version:cc:subject:references:in-reply-to
	:content-type:content-transfer-encoding; q=dns; s=kickflop.net;
	b=jqe0tgt1k+5t5ulRJJC2aCVuwtqlT8ZlJUoavfjCwkA2wQsSNzXSpKgr/j0Ft
	+F79zA0Acblu5Hsr/Z39zbe7vOKYA68sxEiQ7STHEmVIST7yBkinqu04YSZDJfc7
	rL9DHASvE5il4gG8Y4r9GLxZtvyfURv6lkjNSejvp9iP78=
DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=kickflop.net; h=message-id
	:date:from:mime-version:cc:subject:references:in-reply-to
	:content-type:content-transfer-encoding; s=kickflop.net; bh=HSHt
	efYemtJeiZM1woPNsxXEUuM=; b=duKjAeeXobNg+yiA6jihDTXwE9S5Cbp1irh1
	QyW1RPaokk/UDpuvOTzEolc4fc3Z1ncAYllVpHihS8JP44uevS5p7orbSVvehIiQ
	xlUrtgW5clBZrhrM46xUw0XDnCenhVkCnwY4LNqgrcYiEWZpnKyPNzx1BL9Isga5
	hDs7YZg=
Received: from [192.168.1.3] (pool-108-9-57-66.tampfl.fios.verizon.net
	[108.9.57.66]) (using TLSv1 with cipher AES256-SHA (256/256 bits))
	(No client certificate requested)
	(Authenticated sender: jblaine@kickflop.net)
	by homiemail-a18.g.dreamhost.com (Postfix) with ESMTPSA id D2609250069
	for <coreutils@gnu.org>; Thu, 13 Jan 2011 15:15:23 -0800 (PST)
Message-ID: <4D2F8790.6010904@kickflop.net>
Date: Thu, 13 Jan 2011 18:15:28 -0500
From: Jeff Blaine <jblaine@kickflop.net>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US;
	rv:1.9.2.13) Gecko/20101207 Thunderbird/3.1.7
MIME-Version: 1.0
CC: coreutils@gnu.org
Subject: Re: [coreutils] basename/dirname can't handle stdin?
References: <4D2DF65B.2040500@kickflop.net>
	<4D2DFAB7.6020809@redhat.com>	<4D2E2687.9040107@kickflop.net>
	<87bp3lck9u.fsf@meyering.net>
In-Reply-To: <87bp3lck9u.fsf@meyering.net>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 13 Jan 2011 23:15:28 -0000

So then, please review.  One question at the bottom.

# Most basic usage
basename /foo/bar.txt                               => bar.txt


# Old/current "basename NAME SUFFIX" compat-
# ibility
basename /foo/bar.txt .txt                          => bar


# ERROR, one too many operands
basename /foo/bar.txt /x.txt /y.txt                 => ERROR


# BSD-adopted flag to signify no args are a
# suffix, process all
basename -a /foo/bar.txt /x/y.txt                   => bar.txt
                                                        y.txt


# For completeness, showing 3 args with -a
basename -a /foo/bar.txt /x/y.txt /a/b.txt          => bar.txt
                                                        y.txt
                                                        b.txt


basename -s .txt -a /foo/bar.txt /x/y.txt /a/b.txt  => bar
                                                        y
                                                        b


# No args means read stdin (-f,--filter mode)
cat filelist.txt | basename                         => bar.txt
                                                        y.txt
                                                        b.txt


# Only "-s <arg>" means read stdin (-f,--filter
# mode)
cat filelist.txt | basename -s .txt                 => bar
                                                        y
                                                        b


# Handle NUL-terminated stdin
find / -print | basename --file0-from=-             => bar.txt
                                                        y.txt
                                                        b.txt


# Handle NUL-terminated stdin with suffix strip
# (assuming /hh has our 3 files in it and is
# readable)
find /hh -print | basename --file0-from=- -s .txt   => bar
                                                        y
                                                        b


# Handle NUL-terminated FILE input
find / -print | basename --file0-from=FILE          => bar.txt
                                                        y.txt
                                                        b.txt

etc...

Is "-f,--filter" necessary?


From MAILER-DAEMON Thu Jan 13 18:30:25 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PdWcb-0001Vn-45
	for mharc-coreutils@gnu.org; Thu, 13 Jan 2011 18:30:25 -0500
Received: from [140.186.70.92] (port=33627 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PdWcY-0001VP-Sj
	for coreutils@gnu.org; Thu, 13 Jan 2011 18:30:24 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jblaine@kickflop.net>) id 1PdWcX-0008Cp-Py
	for coreutils@gnu.org; Thu, 13 Jan 2011 18:30:22 -0500
Received: from caiajhbdcaid.dreamhost.com ([208.97.132.83]:34597
	helo=homiemail-a75.g.dreamhost.com)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jblaine@kickflop.net>) id 1PdWcX-0008CT-JE
	for coreutils@gnu.org; Thu, 13 Jan 2011 18:30:21 -0500
Received: from homiemail-a75.g.dreamhost.com (localhost [127.0.0.1])
	by homiemail-a75.g.dreamhost.com (Postfix) with ESMTP id 04CCC5EC071
	for <coreutils@gnu.org>; Thu, 13 Jan 2011 15:30:19 -0800 (PST)
DomainKey-Signature: a=rsa-sha1; c=nofws; d=kickflop.net; h=message-id:date
	:from:mime-version:to:subject:references:in-reply-to
	:content-type:content-transfer-encoding; q=dns; s=kickflop.net;
	b=3nc3S6To/cEJRf5uXV1B2Q/fFL3quFzh2M6xQ3Et15vfvoyQ2JMk5Q1k3pFzU
	uEj7s1XwkQXd4Vt3RzDaHcI6vLMazRM/u4ZTb96Yr7VEtFoIpoa20lmoCO5Kdt27
	WcSAoma3xS7/HIF1xB+FptVhdCeAG4+I1l4C6JJAzfUW6k=
DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=kickflop.net; h=message-id
	:date:from:mime-version:to:subject:references:in-reply-to
	:content-type:content-transfer-encoding; s=kickflop.net; bh=8L9o
	AQmaf+wO0a3ClPEt/nNnCyY=; b=1B10v6WXyPuDv2OcQJwmAgGYtDwLnikt0S0l
	lp1W8dLmjbSErPhVdUsc45rNLx6H7m01usMw4BBf1j0Hmt36ccl55WTlo6d0qya0
	OVEP12vOPpnZb32mLmlSqVS8oBmfBv/Ac6UKIAfS0Fi0UgxyEDg9WevIWmdpt+4C
	Wq43q9Q=
Received: from [192.168.1.3] (pool-108-9-57-66.tampfl.fios.verizon.net
	[108.9.57.66]) (using TLSv1 with cipher AES256-SHA (256/256 bits))
	(No client certificate requested)
	(Authenticated sender: jblaine@kickflop.net)
	by homiemail-a75.g.dreamhost.com (Postfix) with ESMTPSA id B61DA5EC069
	for <coreutils@gnu.org>; Thu, 13 Jan 2011 15:30:18 -0800 (PST)
Message-ID: <4D2F8B0F.4010202@kickflop.net>
Date: Thu, 13 Jan 2011 18:30:23 -0500
From: Jeff Blaine <jblaine@kickflop.net>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US;
	rv:1.9.2.13) Gecko/20101207 Thunderbird/3.1.7
MIME-Version: 1.0
To: coreutils@gnu.org
Subject: Re: [coreutils] basename/dirname can't handle stdin?
References: <4D2DF65B.2040500@kickflop.net>
	<20110112200900.GA27136@hysteria.proulx.com>
In-Reply-To: <20110112200900.GA27136@hysteria.proulx.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
X-detected-operating-system: by eggs.gnu.org: Windows 98 (1)
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 13 Jan 2011 23:30:24 -0000

On 1/12/2011 3:09 PM, Bob Proulx wrote:
>> This also fails:
>>      find /somewhere -print | xargs basename
>
> Why did that fail?  Are you also getting an error message from find?

[jblaine@new-host src]$ find /home/jblaine -mtime -1 -print
/home/jblaine
/home/jblaine/.gconf
/home/jblaine/.gconf/apps
/home/jblaine/.gconf/apps/panel/applets/clock/prefs
/home/jblaine/.gconf/apps/panel/applets/clock/prefs/%gconf.xml
/home/jblaine/.gconf/apps/panel/applets/workspace_switcher/prefs
/home/jblaine/.gconf/apps/panel/applets/workspace_switcher/prefs/%gconf.xml
/home/jblaine/.gconf/apps/panel/applets/window_list/prefs
/home/jblaine/.gconf/apps/panel/applets/window_list/prefs/%gconf.xml
/home/jblaine/.gconf/apps/puplet
/home/jblaine/.gconf/apps/puplet/%gconf.xml
^C
[jblaine@new-host src]$ find /home/jblaine -mtime -1 -print | xargs 
/bin/basename
/bin/basename: extra operand `/home/jblaine/.gconf/apps'
Try `/bin/basename --help' for more information.
/bin/basename: extra operand `/home/jblaine/coreutils/lib/cloexec.c'
Try `/bin/basename --help' for more information.
/bin/basename: extra operand 
`/home/jblaine/coreutils/gnulib/modules/unicase/u8-is-lowercase-tests'
Try `/bin/basename --help' for more information.
/bin/basename: extra operand 
`/home/jblaine/coreutils/gnulib/lib/xgetdomainname.c'
Try `/bin/basename --help' for more information.
/bin/basename: extra operand 
`/home/jblaine/coreutils/gnulib/tests/unictype/test-digit.c'
Try `/bin/basename --help' for more information.
^C


From MAILER-DAEMON Thu Jan 13 18:35:46 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PdWhm-0003RA-Rr
	for mharc-coreutils@gnu.org; Thu, 13 Jan 2011 18:35:46 -0500
Received: from [140.186.70.92] (port=56278 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PdWhk-0003NZ-Fo
	for coreutils@gnu.org; Thu, 13 Jan 2011 18:35:45 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jblaine@kickflop.net>) id 1PdWhi-0000nv-Up
	for coreutils@gnu.org; Thu, 13 Jan 2011 18:35:44 -0500
Received: from caiajhbdccah.dreamhost.com ([208.97.132.207]:44419
	helo=homiemail-a13.g.dreamhost.com)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jblaine@kickflop.net>) id 1PdWhi-0000nl-Qv
	for coreutils@gnu.org; Thu, 13 Jan 2011 18:35:42 -0500
Received: from homiemail-a13.g.dreamhost.com (localhost [127.0.0.1])
	by homiemail-a13.g.dreamhost.com (Postfix) with ESMTP id 096CF6A806F
	for <coreutils@gnu.org>; Thu, 13 Jan 2011 15:35:42 -0800 (PST)
DomainKey-Signature: a=rsa-sha1; c=nofws; d=kickflop.net; h=message-id:date
	:from:mime-version:to:subject:references:in-reply-to
	:content-type:content-transfer-encoding; q=dns; s=kickflop.net;
	b=UIRVPd4Qp6l0W9xiOf8miNZBkUW56c0aoTkRLAefzUBSJtf5BWaqTcob6GaOp
	R9/sqVVV7D/Lhuag0p6hzB4y5FZ3D8QtN595udDfITelhF3m12V7UL0DnmWOGztH
	26L0odp6W8QAv2OzQXdIg9nnRnXPma7aQfQVwDg795rO10=
DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=kickflop.net; h=message-id
	:date:from:mime-version:to:subject:references:in-reply-to
	:content-type:content-transfer-encoding; s=kickflop.net; bh=4sfS
	yAFukkEidyULIR+hpTJePYs=; b=UUsQU4MCxtgbPqAH44NXsJyIHHDYdkBJJ/Kq
	DPcmU7S/Mz3+l92WBe2X+HXHABwtxlEXYgxvQhogZems3KDulo5KEdMKOH/SR6DA
	D5kNUCI5IEwd8LOYS6+zx345QG2QqWfcK75cuF+mHKev8czXr64DzpdlsS86FYzF
	N7hUeJQ=
Received: from [192.168.1.3] (pool-108-9-57-66.tampfl.fios.verizon.net
	[108.9.57.66]) (using TLSv1 with cipher AES256-SHA (256/256 bits))
	(No client certificate requested)
	(Authenticated sender: jblaine@kickflop.net)
	by homiemail-a13.g.dreamhost.com (Postfix) with ESMTPSA id CB67F6A805F
	for <coreutils@gnu.org>; Thu, 13 Jan 2011 15:35:41 -0800 (PST)
Message-ID: <4D2F8C52.9010105@kickflop.net>
Date: Thu, 13 Jan 2011 18:35:46 -0500
From: Jeff Blaine <jblaine@kickflop.net>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US;
	rv:1.9.2.13) Gecko/20101207 Thunderbird/3.1.7
MIME-Version: 1.0
To: coreutils@gnu.org
Subject: Re: [coreutils] basename/dirname can't handle stdin?
References: <4D2DF65B.2040500@kickflop.net>
	<4D2DFAB7.6020809@redhat.com>	<4D2E2687.9040107@kickflop.net>
	<87bp3lck9u.fsf@meyering.net> <4D2F8790.6010904@kickflop.net>
In-Reply-To: <4D2F8790.6010904@kickflop.net>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 13 Jan 2011 23:35:45 -0000

Eric wrote:
> basename -a -s suffix str1 str2: same (-a is only required for exactly
> two arguements, and is implied for 0 or for more than 2 args)

basename -a -s .txt foo.txt bar.txt baz.txt

So you're saying -a is redundant there?  Sorry, I'm not
familiar with BSD's basename at all.  Just looking to
implement what you want.

On 1/13/2011 6:15 PM, Jeff Blaine wrote:
> So then, please review. One question at the bottom.
>
> # Most basic usage
> basename /foo/bar.txt => bar.txt
>
>
> # Old/current "basename NAME SUFFIX" compat-
> # ibility
> basename /foo/bar.txt .txt => bar
>
>
> # ERROR, one too many operands
> basename /foo/bar.txt /x.txt /y.txt => ERROR
>
>
> # BSD-adopted flag to signify no args are a
> # suffix, process all
> basename -a /foo/bar.txt /x/y.txt => bar.txt
> y.txt
>
>
> # For completeness, showing 3 args with -a
> basename -a /foo/bar.txt /x/y.txt /a/b.txt => bar.txt
> y.txt
> b.txt
>
>
> basename -s .txt -a /foo/bar.txt /x/y.txt /a/b.txt => bar
> y
> b
>
>
> # No args means read stdin (-f,--filter mode)
> cat filelist.txt | basename => bar.txt
> y.txt
> b.txt
>
>
> # Only "-s <arg>" means read stdin (-f,--filter
> # mode)
> cat filelist.txt | basename -s .txt => bar
> y
> b
>
>
> # Handle NUL-terminated stdin
> find / -print | basename --file0-from=- => bar.txt
> y.txt
> b.txt
>
>
> # Handle NUL-terminated stdin with suffix strip
> # (assuming /hh has our 3 files in it and is
> # readable)
> find /hh -print | basename --file0-from=- -s .txt => bar
> y
> b
>
>
> # Handle NUL-terminated FILE input
> find / -print | basename --file0-from=FILE => bar.txt
> y.txt
> b.txt
>
> etc...
>
> Is "-f,--filter" necessary?


From MAILER-DAEMON Thu Jan 13 18:48:54 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PdWuU-0006Vo-1e
	for mharc-coreutils@gnu.org; Thu, 13 Jan 2011 18:48:54 -0500
Received: from [140.186.70.92] (port=40959 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PdWuR-0006Va-Kj
	for coreutils@gnu.org; Thu, 13 Jan 2011 18:48:52 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1PdWuQ-0002tB-0m
	for coreutils@gnu.org; Thu, 13 Jan 2011 18:48:51 -0500
Received: from mx1.redhat.com ([209.132.183.28]:37041)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1PdWuP-0002sy-KW
	for coreutils@gnu.org; Thu, 13 Jan 2011 18:48:49 -0500
Received: from int-mx10.intmail.prod.int.phx2.redhat.com
	(int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23])
	by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id p0DNmlBD021645
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK);
	Thu, 13 Jan 2011 18:48:47 -0500
Received: from [10.3.113.124] (ovpn-113-124.phx2.redhat.com [10.3.113.124])
	by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP
	id p0DNmkl9020033; Thu, 13 Jan 2011 18:48:47 -0500
Message-ID: <4D2F8F5E.5050701@redhat.com>
Date: Thu, 13 Jan 2011 16:48:46 -0700
From: Eric Blake <eblake@redhat.com>
Organization: Red Hat
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US;
	rv:1.9.2.13) Gecko/20101209 Fedora/3.1.7-0.35.b3pre.fc14
	Lightning/1.0b3pre Mnenhy/0.8.3 Thunderbird/3.1.7
MIME-Version: 1.0
To: Jeff Blaine <jblaine@kickflop.net>
Subject: Re: [coreutils] basename/dirname can't handle stdin?
References: <4D2DF65B.2040500@kickflop.net>	<4D2DFAB7.6020809@redhat.com>	<4D2E2687.9040107@kickflop.net>	<87bp3lck9u.fsf@meyering.net>
	<4D2F8790.6010904@kickflop.net>
In-Reply-To: <4D2F8790.6010904@kickflop.net>
X-Enigmail-Version: 1.1.2
OpenPGP: url=http://people.redhat.com/eblake/eblake.gpg
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature";
	boundary="------------enig092A45631B65455C42C4A85F"
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 13 Jan 2011 23:48:52 -0000

This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--------------enig092A45631B65455C42C4A85F
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

On 01/13/2011 04:15 PM, Jeff Blaine wrote:
> So then, please review.  One question at the bottom.
>=20
> # Most basic usage
> basename /foo/bar.txt                               =3D> bar.txt
>=20
>=20
> # Old/current "basename NAME SUFFIX" compat-
> # ibility
> basename /foo/bar.txt .txt                          =3D> bar
>=20
>=20
> # ERROR, one too many operands
> basename /foo/bar.txt /x.txt /y.txt                 =3D> ERROR

BSD does:

basename /foo/bar.txt /x.txt /y.txt                 =3D> bar.txt
                                                       x.txt
                                                       y.txt

No need to be gratuitously different (using 3 arguments falls outside
the bounds of POSIX, so we can do whatever we want - our current
behavior is to error, but we might as well match BSDs behavior of being
an argument filter without requiring -a, for all cases except 0 and 2
arguments)

>=20
>=20
> # BSD-adopted flag to signify no args are a
> # suffix, process all
> basename -a /foo/bar.txt /x/y.txt                   =3D> bar.txt
>                                                        y.txt

Correct, but this raises the issue of ambiguous output if input contains
newlines.  And since it is not a case of --files0-from, maybe it argues
that we need two options, such that:

basename -a -z /foo/bar.txt /x/y.txt =3D> bar.txt\0y.txt\0

and have --files0-from imply -z (similar to how sort has both
--files0-from and --zero-terminated).

>=20
>=20
> # For completeness, showing 3 args with -a
> basename -a /foo/bar.txt /x/y.txt /a/b.txt          =3D> bar.txt
>                                                        y.txt
>                                                        b.txt
>=20
>=20
> basename -s .txt -a /foo/bar.txt /x/y.txt /a/b.txt  =3D> bar
>                                                        y
>                                                        b

Also possible with 2 args and without -a:

basename -s .txt /foo/bar.txt /x/y.txt =3D> bar
                                          y

>=20
>=20
> # No args means read stdin (-f,--filter mode)
> cat filelist.txt | basename                         =3D> bar.txt
>                                                        y.txt
>                                                        b.txt

Useless use of cat :)

Hmm - BSD doesn't have a stdin filter mode.  Right now:

basename

triggers a usage message for both GNU and BSD basename, but turning it
into a stdin filter by default for 0 arguments means that you will then
hang waiting for stdin, that can lead to surprises for unsuspecting users=
=2E

So maybe it's better to require an explicit argument (but see more
thoughts below):

basename --filter < filelist.txt       =3D> bar.txt
                                          y.txt
                                          b.txt

basename                               =3D> usage error


>=20
> # Only "-s <arg>" means read stdin (-f,--filter
> # mode)
> cat filelist.txt | basename -s .txt                 =3D> bar
>                                                        y
>                                                        b

BSD rejects this usage, too.  With an explicit option:

basename --filter -s .txt < filelist.txt    =3D> bar
                                               y
                                               b



>=20
> # Handle NUL-terminated stdin
> find / -print | basename --file0-from=3D-             =3D> bar.txt
>                                                        y.txt
>                                                        b.txt

Typos: you meant 'find / -print0', and --files0-from.  But yes,
--files0-from implies being a stdin filter, so here's a case where you
don't need two options, one was enough.  On the other hand, since we
KNOW the input is NUL-terminated, it seems like the output should be
NUL-terminated as well (in wc and sort, where the data read from
--files0-from is not part of the output, and thus sort --files0-from
does not imply --sort -z; but for basename, it makes sense that the
implication should be there - if you were worried about newline
ambiguity on input, then you should also worry about it on output):

find / -print0 | basename --files0-from=3D- =3D> bar.txt\0y.txt\0b.txt\0

>=20
>=20
> # Handle NUL-terminated stdin with suffix strip
> # (assuming /hh has our 3 files in it and is
> # readable)
> find /hh -print | basename --file0-from=3D- -s .txt   =3D> bar
>                                                        y
>                                                        b

Same typos and issue with NUL-terminated output:

find /hh -print0 | basename --files0-from=3D- -s .txt   =3D> bar\0y\0b\0

>=20
>=20
> # Handle NUL-terminated FILE input
> find / -print | basename --file0-from=3DFILE          =3D> bar.txt
>                                                        y.txt
>                                                        b.txt

This should discard stdin input from find, and only process from FILE;
so that it is identical to:

basename --files0-from=3DFILE   =3D> bar.txt\0y.txt\0b.txt\0


>=20
> etc...
>=20
> Is "-f,--filter" necessary?

I don't want to burn -f until we have to; we have to burn -a and -s to
match BSD, but everything else can be a long option until we are sure it
is useful enough to promote to a short option.  Then again, if we add
both --files0-from and --filter, we are stuck with an ambiguous prefix,
and you can't use --f (it would have to be --file or --filt).  That
doesn't seem nice, either.

I guess it all boils down to whether we think:

basename

should hang or automatically become a filter.  If we want to avoid hangs
by requiring an explicit option, then we need something (but can we come
up with a better name than --filter, so that --files0-from can be
unambiguously shortened to --f)?  But since the context of 0 arguments
is unambiguous, and the most obvious choice of an explicit name is
--filter, I'm leaning 60:40 to having just --files0-from, and not --filte=
r.

--=20
Eric Blake   eblake@redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org


--------------enig092A45631B65455C42C4A85F
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Public key at http://people.redhat.com/eblake/eblake.gpg
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/

iQEcBAEBCAAGBQJNL49eAAoJEKeha0olJ0Nqp3IH/AiHJA4tW3XTvurxYAq7OX5n
0rQiT7pyjnf/t4ZSIYXDqI+f2qvtnBGIOGyIeqy+WD5R4Z+4HLYv4QJe3Krb/nT6
OYDcfM361D36HPMYG5dvB+tZ4PSOuK5LBd8I89lZN8fmtdYs5fKO6ccoEnaaLm6E
VfV6LfUQeRBZhISHTb7dxxLLHE6it84y71Gx5HjTFtrqVpBLuxxqDsnPYYefu3a0
bW1sGQ1HA1FRoGVc9W54IzW6AFgq4zqKfrSWju6jIDSekBM0zp/NtsJwEveVvaWx
hzXXyTkdHR9G2tX1mM6ZaGc6d9FC9ORe4D14kcA8YJsiD3kqGdxCP5T0M+TP1Uc=
=2fEJ
-----END PGP SIGNATURE-----

--------------enig092A45631B65455C42C4A85F--


From MAILER-DAEMON Thu Jan 13 18:51:36 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PdWx6-0006x7-CJ
	for mharc-coreutils@gnu.org; Thu, 13 Jan 2011 18:51:36 -0500
Received: from [140.186.70.92] (port=48111 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PdWx3-0006vV-SI
	for coreutils@gnu.org; Thu, 13 Jan 2011 18:51:34 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1PdWx2-0003HO-Kr
	for coreutils@gnu.org; Thu, 13 Jan 2011 18:51:33 -0500
Received: from mx1.redhat.com ([209.132.183.28]:41568)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1PdWx2-0003HE-DU
	for coreutils@gnu.org; Thu, 13 Jan 2011 18:51:32 -0500
Received: from int-mx09.intmail.prod.int.phx2.redhat.com
	(int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22])
	by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id p0DNpVjp010807
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK);
	Thu, 13 Jan 2011 18:51:31 -0500
Received: from [10.3.113.124] (ovpn-113-124.phx2.redhat.com [10.3.113.124])
	by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP
	id p0DNpUfc032696; Thu, 13 Jan 2011 18:51:31 -0500
Message-ID: <4D2F9002.5070201@redhat.com>
Date: Thu, 13 Jan 2011 16:51:30 -0700
From: Eric Blake <eblake@redhat.com>
Organization: Red Hat
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US;
	rv:1.9.2.13) Gecko/20101209 Fedora/3.1.7-0.35.b3pre.fc14
	Lightning/1.0b3pre Mnenhy/0.8.3 Thunderbird/3.1.7
MIME-Version: 1.0
To: Jeff Blaine <jblaine@kickflop.net>
Subject: Re: [coreutils] basename/dirname can't handle stdin?
References: <4D2DF65B.2040500@kickflop.net>	<4D2DFAB7.6020809@redhat.com>	<4D2E2687.9040107@kickflop.net>	<87bp3lck9u.fsf@meyering.net>
	<4D2F8790.6010904@kickflop.net> <4D2F8C52.9010105@kickflop.net>
In-Reply-To: <4D2F8C52.9010105@kickflop.net>
X-Enigmail-Version: 1.1.2
OpenPGP: url=http://people.redhat.com/eblake/eblake.gpg
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature";
	boundary="------------enig0CD5B88A77492AC37C690B66"
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 13 Jan 2011 23:51:35 -0000

This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--------------enig0CD5B88A77492AC37C690B66
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

On 01/13/2011 04:35 PM, Jeff Blaine wrote:
> Eric wrote:
>> basename -a -s suffix str1 str2: same (-a is only required for exactly=

>> two arguements, and is implied for 0 or for more than 2 args)
>=20
> basename -a -s .txt foo.txt bar.txt baz.txt
>=20
> So you're saying -a is redundant there?  Sorry, I'm not
> familiar with BSD's basename at all.  Just looking to
> implement what you want.

Yes, the -a is redundant the moment you have -s.  (It helps that I have
a login to a FreeBSD machine, and have been testing these cases as I
type these mails).  It is only required when you have exactly 2
arguments but don't want the second to be treated as a suffix.

--=20
Eric Blake   eblake@redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org


--------------enig0CD5B88A77492AC37C690B66
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Public key at http://people.redhat.com/eblake/eblake.gpg
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/

iQEcBAEBCAAGBQJNL5ACAAoJEKeha0olJ0NqgwAH/i0wfr4aVhLBzTKITWOe2pLX
jj1uzNqCjMzpxGFkjsQewCMGROGeB9ha3nwbSVMlUyXbD9s6QvL9KZYFGRk/Dv+Q
mp/240KjJGl/sK5MLIjaY8GcYf2abiqDWi/AP+a5eBFLBdRyBny8PozIrLgeEPzZ
ubfzOybzWEpb9gkSdUN5z00j/dAwhp9mbX8sb9fisak0i2q8cyaf6pQup+4/DbHi
YypZcroqezv6NdM37ysC694CvoTlzGY1GOz5LBwtvDVcspsXhIWEmIwDmoh3v5Vp
+nWe25p7sEJbqAcDqcZy5Q/S9jY91xO2nkJwDmOLMQtIL7hGtaIW6cat4IIUdI0=
=Fo12
-----END PGP SIGNATURE-----

--------------enig0CD5B88A77492AC37C690B66--


From MAILER-DAEMON Thu Jan 13 18:53:48 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PdWzE-0007WT-7n
	for mharc-coreutils@gnu.org; Thu, 13 Jan 2011 18:53:48 -0500
Received: from [140.186.70.92] (port=48558 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PdWzC-0007WI-HI
	for coreutils@gnu.org; Thu, 13 Jan 2011 18:53:47 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1PdWzB-0003UY-4G
	for coreutils@gnu.org; Thu, 13 Jan 2011 18:53:46 -0500
Received: from mail1.slb.deg.dub.stisp.net ([84.203.253.98]:17602)
	by eggs.gnu.org with smtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1PdWzA-0003UA-O1
	for coreutils@gnu.org; Thu, 13 Jan 2011 18:53:45 -0500
Received: (qmail 80758 invoked from network); 13 Jan 2011 23:53:43 -0000
Received: from unknown (HELO ?192.168.2.25?) (84.203.137.218)
	by mail1.slb.deg.dub.stisp.net with SMTP; 13 Jan 2011 23:53:43 -0000
Message-ID: <4D2F9063.7000803@draigBrady.com>
Date: Thu, 13 Jan 2011 23:53:07 +0000
From: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US;
	rv:1.9.1.8) Gecko/20100227 Thunderbird/3.0.3
MIME-Version: 1.0
To: Jeff Blaine <jblaine@kickflop.net>
Subject: Re: [coreutils] basename/dirname can't handle stdin?
References: <4D2DF65B.2040500@kickflop.net>	<4D2DFAB7.6020809@redhat.com>	<4D2E2687.9040107@kickflop.net>	<87bp3lck9u.fsf@meyering.net>
	<4D2F8790.6010904@kickflop.net>
In-Reply-To: <4D2F8790.6010904@kickflop.net>
X-Enigmail-Version: 1.0.1
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
X-detected-operating-system: by eggs.gnu.org: FreeBSD 4.6-4.9
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 13 Jan 2011 23:53:47 -0000

On 13/01/11 23:15, Jeff Blaine wrote:
> So then, please review.  One question at the bottom.
> 
> # Most basic usage
> basename /foo/bar.txt                               => bar.txt
> 
> 
> # Old/current "basename NAME SUFFIX" compat-
> # ibility
> basename /foo/bar.txt .txt                          => bar
> 
> 
> # ERROR, one too many operands
> basename /foo/bar.txt /x.txt /y.txt                 => ERROR
> 
> 
> # BSD-adopted flag to signify no args are a
> # suffix, process all
> basename -a /foo/bar.txt /x/y.txt                   => bar.txt
>                                                        y.txt
> 
> 
> # For completeness, showing 3 args with -a
> basename -a /foo/bar.txt /x/y.txt /a/b.txt          => bar.txt
>                                                        y.txt
>                                                        b.txt
> 
> 
> basename -s .txt -a /foo/bar.txt /x/y.txt /a/b.txt  => bar
>                                                        y
>                                                        b
> 
> 
> # No args means read stdin (-f,--filter mode)
> cat filelist.txt | basename                         => bar.txt
>                                                        y.txt
>                                                        b.txt
> 
> 
> # Only "-s <arg>" means read stdin (-f,--filter
> # mode)
> cat filelist.txt | basename -s .txt                 => bar
>                                                        y
>                                                        b
> 
> 
> # Handle NUL-terminated stdin
> find / -print | basename --file0-from=-             => bar.txt
>                                                        y.txt
>                                                        b.txt
> 
> 
> # Handle NUL-terminated stdin with suffix strip
> # (assuming /hh has our 3 files in it and is
> # readable)
> find /hh -print | basename --file0-from=- -s .txt   => bar
>                                                        y
>                                                        b
> 
> 
> # Handle NUL-terminated FILE input
> find / -print | basename --file0-from=FILE          => bar.txt
>                                                        y.txt
>                                                        b.txt
> 
> etc...
> 
> Is "-f,--filter" necessary?
> 
> 

It would be nice not to mandate it, but without it
a script containing the following for example could hang:

path=$(basename $path 2>/dev/null || echo default_path)

Perhaps we should only support --files0-from
and the normal filtering case can be handled with:

find / | xargs basename -a

In fact thinking more about it we might not need --files0-from either.
It's used in du,wc, and sort as they need to deal with all files
from a single process invocation (to generate totals etc.)
But that's not the case for basename.

So in summary, just implement -a and -s like BSD does?

cheers,
Pádraig.


From MAILER-DAEMON Thu Jan 13 19:02:48 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PdX7w-0000c1-Pj
	for mharc-coreutils@gnu.org; Thu, 13 Jan 2011 19:02:48 -0500
Received: from [140.186.70.92] (port=34015 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PdX7u-0000bt-3c
	for coreutils@gnu.org; Thu, 13 Jan 2011 19:02:47 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1PdX7s-0004iG-6K
	for coreutils@gnu.org; Thu, 13 Jan 2011 19:02:45 -0500
Received: from mx1.redhat.com ([209.132.183.28]:63753)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1PdX7r-0004i4-Ua
	for coreutils@gnu.org; Thu, 13 Jan 2011 19:02:44 -0500
Received: from int-mx12.intmail.prod.int.phx2.redhat.com
	(int-mx12.intmail.prod.int.phx2.redhat.com [10.5.11.25])
	by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id p0E02dhl023727
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK);
	Thu, 13 Jan 2011 19:02:39 -0500
Received: from [10.3.113.124] (ovpn-113-124.phx2.redhat.com [10.3.113.124])
	by int-mx12.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP
	id p0E02cNJ015346; Thu, 13 Jan 2011 19:02:39 -0500
Message-ID: <4D2F929E.7060607@redhat.com>
Date: Thu, 13 Jan 2011 17:02:38 -0700
From: Eric Blake <eblake@redhat.com>
Organization: Red Hat
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US;
	rv:1.9.2.13) Gecko/20101209 Fedora/3.1.7-0.35.b3pre.fc14
	Lightning/1.0b3pre Mnenhy/0.8.3 Thunderbird/3.1.7
MIME-Version: 1.0
To: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
Subject: Re: [coreutils] basename/dirname can't handle stdin?
References: <4D2DF65B.2040500@kickflop.net>	<4D2DFAB7.6020809@redhat.com>	<4D2E2687.9040107@kickflop.net>	<87bp3lck9u.fsf@meyering.net>	<4D2F8790.6010904@kickflop.net>
	<4D2F9063.7000803@draigBrady.com>
In-Reply-To: <4D2F9063.7000803@draigBrady.com>
X-Enigmail-Version: 1.1.2
OpenPGP: url=http://people.redhat.com/eblake/eblake.gpg
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature";
	boundary="------------enigD00F1BF65D63E63092C35F82"
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.25
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
Cc: Jeff Blaine <jblaine@kickflop.net>, coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 14 Jan 2011 00:02:47 -0000

This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--------------enigD00F1BF65D63E63092C35F82
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

On 01/13/2011 04:53 PM, P=E1draig Brady wrote:
>=20
> It would be nice not to mandate it, but without it
> a script containing the following for example could hang:
>=20
> path=3D$(basename $path 2>/dev/null || echo default_path)
>=20
> Perhaps we should only support --files0-from
> and the normal filtering case can be handled with:
>=20
> find / | xargs basename -a
>=20
> In fact thinking more about it we might not need --files0-from either.
> It's used in du,wc, and sort as they need to deal with all files
> from a single process invocation (to generate totals etc.)
> But that's not the case for basename.

The moment you output more than one file name, you need the ability to
choose to use NUL termination to avoid output ambiguity.

wc needs --files0-from because wc tracks totals over multiple files, but
'xargs -0 wc' might split the invocation resulting in multiple total
lines instead of the desired single total line.

sort needs --files0-from for the same reason - if broken into multiple
processes, the set of files is not sorted into a single result.

sort also has -z/--zero-terminated, to operate on NUL-terminated rather
than newline-terminated input/output lines.

basename does not change its output (no multiple total lines, and no
missed interpolation of input), so I agree that basename --files0-from
does not make sense, after all.  xargs -0 basename is sufficient for the
input side of things.

However, there's still the issue of the output side of things.

basename $'a\nb' results in two lines, even though it was only over one
file.  When basename only accepted one argument, there was no ambiguity.
 But even doing 'xargs -0 -n 1 basename' is ambiguous, and since $()
eats all trailing newlines rather than just one, it becomes difficult to
use shell constructs to get exactly one trailing byte stripped.  So that
argues that we need -z/--zero-terminated.

basename -z a =3D> $'a\0' (rather than the current $'a\n')
basename -z -a a b =3D> $'a\0b\0'

> So in summary, just implement -a and -s like BSD does?

I think you've persuaded me that a filter mode doesn't buy us much (it's
only useful if unambiguous, but xargs -0 works just as well at giving us
unambiguous input), but we do need -z in addition to BSD's -a and -s.

--=20
Eric Blake   eblake@redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org


--------------enigD00F1BF65D63E63092C35F82
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Public key at http://people.redhat.com/eblake/eblake.gpg
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/

iQEcBAEBCAAGBQJNL5KeAAoJEKeha0olJ0NqevwH+gMQZMb7CyospW+gfdRUg/Pw
e/2W66CrWd3/XkO29xEYw8QCrb/WN/bTCXLbtGqNowns+iBePB7OibKi7aWty/4G
CjXiP+m1+paVyNLjkjeVmVLfHO7x+yg0T9jd4Zr3H/b6bm1qRHCv1Mb1ozO4C+oL
XabKk0r2Wcletdqhrd9cdtgpv6cvcYHtj0afx67nC+MxUOoO1x634jeTAsh97WAM
DXbeiCR7p6rqxvFAt479CzIM2S56op0fq/EoD8fvGqUX+38lJ+yUrrGjsLX2uzsA
lnMLpRaKvOkDh4BjL8vAOEjOEJHDgfDJofJrs5sMpb16ZI6WT6FTVJXP2tJ/WKo=
=K0Xv
-----END PGP SIGNATURE-----

--------------enigD00F1BF65D63E63092C35F82--


From MAILER-DAEMON Thu Jan 13 19:56:03 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PdXxT-00046q-Ez
	for mharc-coreutils@gnu.org; Thu, 13 Jan 2011 19:56:03 -0500
Received: from [140.186.70.92] (port=42385 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PdXxR-00045u-3y
	for coreutils@gnu.org; Thu, 13 Jan 2011 19:56:02 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <bob@proulx.com>) id 1PdXxQ-0004SE-3L
	for coreutils@gnu.org; Thu, 13 Jan 2011 19:56:01 -0500
Received: from joseki.proulx.com ([216.17.153.58]:43684)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <bob@proulx.com>) id 1PdXxP-0004SA-Rz
	for coreutils@gnu.org; Thu, 13 Jan 2011 19:55:59 -0500
Received: from hysteria.proulx.com (hysteria.proulx.com [192.168.230.119])
	by joseki.proulx.com (Postfix) with ESMTP id 0FF5221311;
	Thu, 13 Jan 2011 17:55:57 -0700 (MST)
Received: by hysteria.proulx.com (Postfix, from userid 1000)
	id EFE902DD82; Thu, 13 Jan 2011 17:55:56 -0700 (MST)
Date: Thu, 13 Jan 2011 17:55:56 -0700
From: Bob Proulx <bob@proulx.com>
To: Jeff Blaine <jblaine@kickflop.net>
Subject: Re: [coreutils] basename/dirname can't handle stdin?
Message-ID: <20110114005556.GA21047@hysteria.proulx.com>
Mail-Followup-To: Jeff Blaine <jblaine@kickflop.net>, coreutils@gnu.org
References: <4D2DF65B.2040500@kickflop.net>
	<20110112200900.GA27136@hysteria.proulx.com>
	<4D2F8B0F.4010202@kickflop.net>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <4D2F8B0F.4010202@kickflop.net>
User-Agent: Mutt/1.5.20 (2009-06-14)
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 1)
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 14 Jan 2011 00:56:02 -0000

Jeff Blaine wrote:
> Bob Proulx wrote:
> > Why did that fail? ...
> 
> [jblaine@new-host src]$ find /home/jblaine -mtime -1 -print | xargs /bin/basename
> /bin/basename: extra operand `/home/jblaine/.gconf/apps'
> Try `/bin/basename --help' for more information.

Ah... Yes, too many arguments.  In that case ask xargs to not provide
that many.

Give this a try:

  find . -type f -print0 | xargs -r0 -I{} basename {} .txt

And with dirname something like this:

  find a -type f -print0 | xargs -r0 -L1 dirname

BTW...  You shouldn't hard code paths such as /bin/basename.  On some
systems basename is in /bin and and on some in /usr/bin.  It is always
better to avoid hard coded paths, which create portability problems,
and instead locate programs from PATH.

Bob


From MAILER-DAEMON Thu Jan 13 19:58:30 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PdXzq-0004db-Ll
	for mharc-coreutils@gnu.org; Thu, 13 Jan 2011 19:58:30 -0500
Received: from [140.186.70.92] (port=43206 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PdXzo-0004dS-9o
	for coreutils@gnu.org; Thu, 13 Jan 2011 19:58:29 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1PdXzm-0004sz-Qf
	for coreutils@gnu.org; Thu, 13 Jan 2011 19:58:27 -0500
Received: from mail1.slb.deg.dub.stisp.net ([84.203.253.98]:10298)
	by eggs.gnu.org with smtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1PdXzm-0004so-Bk
	for coreutils@gnu.org; Thu, 13 Jan 2011 19:58:26 -0500
Received: (qmail 88848 invoked from network); 14 Jan 2011 00:58:24 -0000
Received: from unknown (HELO ?192.168.2.25?) (84.203.137.218)
	by mail1.slb.deg.dub.stisp.net with SMTP; 14 Jan 2011 00:58:24 -0000
Message-ID: <4D2F9F8C.1000201@draigBrady.com>
Date: Fri, 14 Jan 2011 00:57:48 +0000
From: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US;
	rv:1.9.1.8) Gecko/20100227 Thunderbird/3.0.3
MIME-Version: 1.0
To: Eric Blake <eblake@redhat.com>
Subject: Re: [coreutils] basename/dirname can't handle stdin?
References: <4D2DF65B.2040500@kickflop.net>	<4D2DFAB7.6020809@redhat.com>	<4D2E2687.9040107@kickflop.net>	<87bp3lck9u.fsf@meyering.net>	<4D2F8790.6010904@kickflop.net>	<4D2F9063.7000803@draigBrady.com>
	<4D2F929E.7060607@redhat.com>
In-Reply-To: <4D2F929E.7060607@redhat.com>
X-Enigmail-Version: 1.0.1
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
X-detected-operating-system: by eggs.gnu.org: FreeBSD 4.6-4.9
Cc: Jeff Blaine <jblaine@kickflop.net>, coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 14 Jan 2011 00:58:29 -0000

On 14/01/11 00:02, Eric Blake wrote:
> On 01/13/2011 04:53 PM, Pádraig Brady wrote:
>> So in summary, just implement -a and -s like BSD does?
> 
> I think you've persuaded me that a filter mode doesn't buy us much (it's
> only useful if unambiguous, but xargs -0 works just as well at giving us
> unambiguous input), but we do need -z in addition to BSD's -a and -s.

ack




From MAILER-DAEMON Thu Jan 13 20:07:00 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PdY84-0006Rq-BZ
	for mharc-coreutils@gnu.org; Thu, 13 Jan 2011 20:07:00 -0500
Received: from [140.186.70.92] (port=51386 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PdY82-0006Qr-3B
	for coreutils@gnu.org; Thu, 13 Jan 2011 20:06:58 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jblaine@kickflop.net>) id 1PdY80-0005r8-Qe
	for coreutils@gnu.org; Thu, 13 Jan 2011 20:06:57 -0500
Received: from caiajhbdcaib.dreamhost.com ([208.97.132.81]:41834
	helo=homiemail-a19.g.dreamhost.com)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jblaine@kickflop.net>) id 1PdY80-0005qz-CY
	for coreutils@gnu.org; Thu, 13 Jan 2011 20:06:56 -0500
Received: from homiemail-a19.g.dreamhost.com (localhost [127.0.0.1])
	by homiemail-a19.g.dreamhost.com (Postfix) with ESMTP id 646D5604069
	for <coreutils@gnu.org>; Thu, 13 Jan 2011 17:06:55 -0800 (PST)
DomainKey-Signature: a=rsa-sha1; c=nofws; d=kickflop.net; h=message-id:date
	:from:mime-version:to:subject:references:in-reply-to
	:content-type:content-transfer-encoding; q=dns; s=kickflop.net;
	b=lk+FtnTsJjDJxb1oYrK4Pciy3D7hFjp4KYI25LLDl44d9R4h1R9f/TYZ6xF6B
	k+7xCNKXlUEs5PqIrzxMruqZdxhtHcU3m4TifhPt6z1EzmdWrlsWUmyGohBF1mbR
	5MRXpMu9Yh5+Nq+KKJo511H0qtFpLAgvAIYe0eJ8A8+8Uw=
DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=kickflop.net; h=message-id
	:date:from:mime-version:to:subject:references:in-reply-to
	:content-type:content-transfer-encoding; s=kickflop.net; bh=yGdP
	OfLbSWcklmuw/1kEInNdhIo=; b=nYSQvkxmbRiqTovhEIerv+mBYh6TVQrB1g+b
	t50OHJ2Th1Oic56os2XJG0FFYskgCR32psp2BJOZlKTMXYTeaniCa07eObPNWHZ2
	UX2dW7FKIxfS5TNJqHeai6AUBIhJYxpVRel0kegXUIvoLnncT1vHhq6n/CKDGfn4
	9/ryxsY=
Received: from [192.168.1.3] (pool-108-9-57-66.tampfl.fios.verizon.net
	[108.9.57.66]) (using TLSv1 with cipher AES256-SHA (256/256 bits))
	(No client certificate requested)
	(Authenticated sender: jblaine@kickflop.net)
	by homiemail-a19.g.dreamhost.com (Postfix) with ESMTPSA id 31D7A604014
	for <coreutils@gnu.org>; Thu, 13 Jan 2011 17:06:55 -0800 (PST)
Message-ID: <4D2FA1B4.90001@kickflop.net>
Date: Thu, 13 Jan 2011 20:07:00 -0500
From: Jeff Blaine <jblaine@kickflop.net>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US;
	rv:1.9.2.13) Gecko/20101207 Thunderbird/3.1.7
MIME-Version: 1.0
To: coreutils@gnu.org
Subject: Re: [coreutils] basename/dirname can't handle stdin?
References: <4D2DF65B.2040500@kickflop.net>
	<20110112200900.GA27136@hysteria.proulx.com>
	<4D2F8B0F.4010202@kickflop.net>
	<20110114005556.GA21047@hysteria.proulx.com>
In-Reply-To: <20110114005556.GA21047@hysteria.proulx.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 14 Jan 2011 01:06:59 -0000

On 1/13/2011 7:55 PM, Bob Proulx wrote:
> Jeff Blaine wrote:
>> Bob Proulx wrote:
>>> Why did that fail? ...
>>
>> [jblaine@new-host src]$ find /home/jblaine -mtime -1 -print | xargs /bin/basename

Yes, I've found there are a few ways around the lack of
"filter mode" in GNU basename.  That there are other
solutions doesn't negate the idea that it can/should be
able to work in "filter mode" like so many other chainable
applications, right?

> BTW...  You shouldn't hard code paths such as /bin/basename.  On some
> systems basename is in /bin and and on some in /usr/bin.  It is always
> better to avoid hard coded paths, which create portability problems,
> and instead locate programs from PATH.

I didn't hardcode anything :)  I specified the basename
binary I wanted to use here because there is another one in
my PATH before /bin (the basename code I am working on!  And
I want it there at the front)


From MAILER-DAEMON Thu Jan 13 21:43:11 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PdZd9-0002Ae-Jk
	for mharc-coreutils@gnu.org; Thu, 13 Jan 2011 21:43:11 -0500
Received: from [140.186.70.92] (port=45676 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PdZd8-0002AX-Bs
	for coreutils@gnu.org; Thu, 13 Jan 2011 21:43:11 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1PdZd7-0002X7-2z
	for coreutils@gnu.org; Thu, 13 Jan 2011 21:43:10 -0500
Received: from mx1.redhat.com ([209.132.183.28]:28983)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1PdZd6-0002Wv-Mm
	for coreutils@gnu.org; Thu, 13 Jan 2011 21:43:08 -0500
Received: from int-mx10.intmail.prod.int.phx2.redhat.com
	(int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23])
	by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id p0E2h7MG006560
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK)
	for <coreutils@gnu.org>; Thu, 13 Jan 2011 21:43:07 -0500
Received: from [10.3.113.124] (ovpn-113-124.phx2.redhat.com [10.3.113.124])
	by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP
	id p0E2h6Wt028791
	for <coreutils@gnu.org>; Thu, 13 Jan 2011 21:43:07 -0500
Message-ID: <4D2FB83A.6090005@redhat.com>
Date: Thu, 13 Jan 2011 19:43:06 -0700
From: Eric Blake <eblake@redhat.com>
Organization: Red Hat
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US;
	rv:1.9.2.13) Gecko/20101209 Fedora/3.1.7-0.35.b3pre.fc14
	Lightning/1.0b3pre Mnenhy/0.8.3 Thunderbird/3.1.7
MIME-Version: 1.0
To: Coreutils <coreutils@gnu.org>
X-Enigmail-Version: 1.1.2
OpenPGP: url=http://people.redhat.com/eblake/eblake.gpg
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature";
	boundary="------------enig1FACEBE844B685206AB97127"
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
Subject: [coreutils] another potential sort bug?
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 14 Jan 2011 02:43:11 -0000

This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--------------enig1FACEBE844B685206AB97127
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

POSIX is explicit that in a multi-threaded app, use of any
non-async-signal-safe function in between fork() and exec*()/exit() may
result in undefined behavior.  For example, if thread 1 is in the middle
of a malloc() and holds a mutex, and thread 2 calls fork(), then any
attempt by the child process to call malloc() will deadlock (the
malloc() in the child will attempt to obtain the mutex, but thread 1 no
longer exists in the child to release the mutex).

Most of the coreutils that use fork/exec (install, timeout) are
single-threaded, so this is not an issue.  But now that sort is
multi-threaded, the fact that it calls fork/exec means we have to audit
all the code for safety.  Right now, I see several potential calls to
error(_()) in the child process, and error() definitely is not
async-signal-safe (the call to gettext() may malloc, and error() itself
certainly has calls to malloc() and various stdio functions).

It is still possible to use error() in the child if we can guarantee
that no other thread can be in the middle of any of the
non-async-signal-safe functions used by the child, and thus no deadlock
can occur in any of those functions.

One thought is that pthread_atfork() can be used to provide such a
guarantee - POSIX admits that it doesn't scale well to libraries where
there is no idea what other functions/mutexes might be use in other
threads, but perhaps we can make it work fine in sort() where we can
easily audit all resources that must be locked correctly before the
child attempts an otherwise unsafe action, because we aren't linking to
multi-thread libraries.  But that would mean hiding all instances of
malloc() and stdio functions behind a mutex controlled by the
pthread_atfork handlers, which seems prohibitively expensive.

Another thought is to open a cloexec pipe from child to parent, where if
the child encounters any failure, instead of calling error() in the
child, we call write() to pass the (untranslated) message back to the
parent, and the parent then listens on the pipe, receives any
untranslated message, and calls error(_()) on the result, such that all
malloc() and stdio is removed from the child.  This is probably easier
to implement, and certainly has the benefit that it doesn't impact the
common case by surrounding all other unsafe actions in a mutex.  But it
does put more fd pressure (at least 2 fd, if we can share the error pipe
among all children, and up to 2*threads if it has to be one pipe per
thread calling fork).

Are we worried enough about the potential for a malloc mutex deadlock
that we should go ahead and use one of these methods to protect sort
from deadlock or other undefined behavior in the child process?

--=20
Eric Blake   eblake@redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org


--------------enig1FACEBE844B685206AB97127
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Public key at http://people.redhat.com/eblake/eblake.gpg
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/

iQEcBAEBCAAGBQJNL7g6AAoJEKeha0olJ0Nq9X0IAJzSwbF1KSWJs+jX1f4OQISg
Jicp9ePH4KR75WUEIFTkaaLmJL/G0iJW3SjE3iquO9WXjcL8/zmNCg64C4uslsvF
WI419WyCyK93y7HbYQ3I2AATMPx0Vcq9TvJFL6ikl+J36AuU730pnG7AtoGpcVpf
OqawhFSBKU/TSNQL07N39GYfnfnwVM9ssbNT3Oogvy8Lngp34xZU2f200m+SbCbU
/+io89gEiiYEpyNRdd0DsKBVbiOMu7hbv8BfQHRp+4lXDQL0VwxadYJJ8Stimpa3
JuBS1e1dR90gr/orDpwtxNjJk0gXcZGPtuMuKCGQNYh8qMajFzkLpcsFmLrokrU=
=7iFP
-----END PGP SIGNATURE-----

--------------enig1FACEBE844B685206AB97127--


From MAILER-DAEMON Fri Jan 14 00:58:58 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1Pdcgc-00049d-JA
	for mharc-coreutils@gnu.org; Fri, 14 Jan 2011 00:58:58 -0500
Received: from [140.186.70.92] (port=56187 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PdcgZ-00049Y-G9
	for coreutils@gnu.org; Fri, 14 Jan 2011 00:58:57 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <bob@proulx.com>) id 1PdcgY-0002jv-2m
	for coreutils@gnu.org; Fri, 14 Jan 2011 00:58:55 -0500
Received: from joseki.proulx.com ([216.17.153.58]:33884)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <bob@proulx.com>) id 1PdcgX-0002jr-SL
	for coreutils@gnu.org; Fri, 14 Jan 2011 00:58:54 -0500
Received: from hysteria.proulx.com (hysteria.proulx.com [192.168.230.119])
	by joseki.proulx.com (Postfix) with ESMTP id 9F45221311;
	Thu, 13 Jan 2011 22:58:52 -0700 (MST)
Received: by hysteria.proulx.com (Postfix, from userid 1000)
	id 8C97C2DD82; Thu, 13 Jan 2011 22:58:52 -0700 (MST)
Date: Thu, 13 Jan 2011 22:58:52 -0700
From: Bob Proulx <bob@proulx.com>
To: Jeff Blaine <jblaine@kickflop.net>
Subject: Re: [coreutils] basename/dirname can't handle stdin?
Message-ID: <20110114055852.GA20005@hysteria.proulx.com>
Mail-Followup-To: Jeff Blaine <jblaine@kickflop.net>, coreutils@gnu.org
References: <4D2DF65B.2040500@kickflop.net>
	<20110112200900.GA27136@hysteria.proulx.com>
	<4D2F8B0F.4010202@kickflop.net>
	<20110114005556.GA21047@hysteria.proulx.com>
	<4D2FA1B4.90001@kickflop.net>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <4D2FA1B4.90001@kickflop.net>
User-Agent: Mutt/1.5.20 (2009-06-14)
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 1)
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 14 Jan 2011 05:58:57 -0000

Jeff Blaine wrote:
> Yes, I've found there are a few ways around the lack of
> "filter mode" in GNU basename.  That there are other
> solutions doesn't negate the idea that it can/should be
> able to work in "filter mode" like so many other chainable
> applications, right?

After forty years it is hard to believe that basename really needs it.
Because basename really isn't a filter command.  Not like the classic
filter command sed for example.

  find . -print | sed 's/.*\/\([^/][^/]*\)/\1/;s/.txt$//'

That is a filter command.  Of course it can't handle arbitrary
filenames which is why I didn't suggest it as a solution.  And of
course bash and other shells can do the function using builtins too.

Certainly if you want the basename of files without removing suffixes
(which seemed to be the case of your original posting) and are already
using find (again as per your original posting) then you don't even
need basename.  That is more easily done with find.

basename-like:
  find . -printf '%f\n'

dirname-like:
  find . -printf '%h\n'

And zstring output too.

  find . -printf '%f\0' | xargs -0 -L1  # adding xargs -0 just to safe this

But I didn't say anything opposing making basename compatible with
BSD's basename.  Compatibility is good.  Even if it doesn't really
need the feature.  It certainly isn't difficult to do with the present
tools.  But it is a small thing and being compatible is useful.

To be clear I am not opposed to adding the *BSD compatibility options.

I am only opposed to thinking that it is needed.  :-)

> >BTW...  You shouldn't hard code paths such as /bin/basename.  On some
> >systems basename is in /bin and and on some in /usr/bin.  It is always
> >better to avoid hard coded paths, which create portability problems,
> >and instead locate programs from PATH.
> 
> I didn't hardcode anything :)  I specified the basename
> binary I wanted to use here because there is another one in
> my PATH before /bin (the basename code I am working on!  And
> I want it there at the front)

I am just working with the information provided.  :-) That is what you
wrote and so that is what I responded to.  And people have in the past
and do continue to hardcode paths such as /bin/basename.  Basename and
grep are two of the most problematic commands in that regard.  They
are all over the place on different systems.  I have hit that problem
many times.

Bob


From MAILER-DAEMON Fri Jan 14 02:22:28 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PddzQ-000392-Qr
	for mharc-coreutils@gnu.org; Fri, 14 Jan 2011 02:22:28 -0500
Received: from [140.186.70.92] (port=53852 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PddzP-00038v-4j
	for coreutils@gnu.org; Fri, 14 Jan 2011 02:22:28 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PddzN-0007Ou-Th
	for coreutils@gnu.org; Fri, 14 Jan 2011 02:22:27 -0500
Received: from mx.meyering.net ([82.230.74.64]:41477)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PddzN-0007Oi-KC
	for coreutils@gnu.org; Fri, 14 Jan 2011 02:22:25 -0500
Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000)
	id 8749D6010F; Fri, 14 Jan 2011 08:22:24 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: =?utf-8?Q?P=C3=A1draig?= Brady <P@draigBrady.com>
Subject: Re: [coreutils] [PATCH] maint: trivial system header file cleanups
In-Reply-To: <4D2F363A.6070408@draigBrady.com> (=?utf-8?Q?=22P=C3=A1draig?=
	Brady"'s message of "Thu, 13 Jan 2011 17:28:26 +0000")
References: <4D2F363A.6070408@draigBrady.com>
Date: Fri, 14 Jan 2011 08:22:24 +0100
Message-ID: <87r5cg3q6n.fsf@meyering.net>
Lines: 40
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
Cc: Coreutils <coreutils@gnu.org>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 14 Jan 2011 07:22:28 -0000

P=C3=A1draig Brady wrote:
>>From e1aaf8903db97f3240b1551fd6936ccdc652dfc8 Mon Sep 17 00:00:00 2001
> From: =3D?UTF-8?q?P=3DC3=3DA1draig=3D20Brady?=3D <P@draigBrady.com>
> Date: Thu, 13 Jan 2011 09:36:38 +0000
> Subject: [PATCH] maint: trivial system header file cleanups
>
> * src/system.h: Note where it should be included, and
> make ordering check portable to GLIBC > 2
> * src/copy.c: Move <sys/ioctl.h> along with other system headers
> as is done elsewhere.
> * src/install.c: Move <sys/wait.h> along with other system headers
> as is done elsewhere.
> * src/ptx.c: Include <regex.h> rather than "regex.h" as
> is done elsewhere.  Note <regex.h> is kept after "system.h"
> as per commit dba300a0.

Those look fine.  Thanks.

...
> diff --git a/src/system.h b/src/system.h
> index 3b32cbd..b86e570 100644
> --- a/src/system.h
> +++ b/src/system.h
> @@ -14,11 +14,15 @@
>     You should have received a copy of the GNU General Public License
>     along with this program.  If not, see <http://www.gnu.org/licenses/>.=
  */
>
> +/* Include this file _after_ system headers if possible.  */
> +
>  #include <alloca.h>
>
> -/* Include sys/types.h before this file.  */
> +/* Include <sys/types.h> before this file.
> +   Note this doesn't warn if we're included
> +   before all system headers.  */
>
> -#if 2 <=3D __GLIBC__ && 2 <=3D __GLIBC_MINOR__
> +#if 2 < __GLIBC__ || ( 2 =3D=3D ___GLIBC__ && 2 <=3D __GLIBC_MINOR__ )

Good one.


From MAILER-DAEMON Fri Jan 14 06:22:08 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PdhjL-0006Fu-U3
	for mharc-coreutils@gnu.org; Fri, 14 Jan 2011 06:22:07 -0500
Received: from [140.186.70.92] (port=35759 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PdhjJ-0006Fb-Fd
	for coreutils@gnu.org; Fri, 14 Jan 2011 06:22:06 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PdhjI-0007qE-IT
	for coreutils@gnu.org; Fri, 14 Jan 2011 06:22:05 -0500
Received: from mx.meyering.net ([82.230.74.64]:44709)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PdhjI-0007q2-DD
	for coreutils@gnu.org; Fri, 14 Jan 2011 06:22:04 -0500
Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000)
	id F205260125; Fri, 14 Jan 2011 12:22:02 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: coreutils@gnu.org
Date: Fri, 14 Jan 2011 12:22:02 +0100
Message-ID: <87k4i73f39.fsf@meyering.net>
Lines: 6
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
Subject: [coreutils] might as well release yet again
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 14 Jan 2011 11:22:06 -0000

I'm thinking of making yet another coreutils release.
The du bug isn't that big a deal, but still.
I prefer not to mix bug fixes and feature additions,
and a few of the latter are coming.

Does anyone have patches that would fit in a bug-fix release?


From MAILER-DAEMON Fri Jan 14 06:52:53 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PdiD6-0000be-Td
	for mharc-coreutils@gnu.org; Fri, 14 Jan 2011 06:52:52 -0500
Received: from [140.186.70.92] (port=59577 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PdiD3-0000a0-QB
	for coreutils@gnu.org; Fri, 14 Jan 2011 06:52:51 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1PdiD1-0004kJ-P9
	for coreutils@gnu.org; Fri, 14 Jan 2011 06:52:49 -0500
Received: from mail1.slb.deg.dub.stisp.net ([84.203.253.98]:15875)
	by eggs.gnu.org with smtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1PdiD1-0004ji-Bl
	for coreutils@gnu.org; Fri, 14 Jan 2011 06:52:47 -0500
Received: (qmail 73505 invoked from network); 14 Jan 2011 11:52:45 -0000
Received: from unknown (HELO ?192.168.2.25?) (84.203.137.218)
	by mail1.slb.deg.dub.stisp.net with SMTP; 14 Jan 2011 11:52:45 -0000
Message-ID: <4D3038E7.1020908@draigBrady.com>
Date: Fri, 14 Jan 2011 11:52:07 +0000
From: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US;
	rv:1.9.1.8) Gecko/20100227 Thunderbird/3.0.3
MIME-Version: 1.0
To: Jim Meyering <jim@meyering.net>
Subject: Re: [coreutils] might as well release yet again
References: <87k4i73f39.fsf@meyering.net>
In-Reply-To: <87k4i73f39.fsf@meyering.net>
X-Enigmail-Version: 1.0.1
Content-Type: multipart/mixed; boundary="------------030408040703070703060704"
X-detected-operating-system: by eggs.gnu.org: FreeBSD 4.6-4.9
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 14 Jan 2011 11:52:51 -0000

This is a multi-part message in MIME format.
--------------030408040703070703060704
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

On 14/01/11 11:22, Jim Meyering wrote:
> I'm thinking of making yet another coreutils release.
> The du bug isn't that big a deal, but still.
> I prefer not to mix bug fixes and feature additions,
> and a few of the latter are coming.

I'd be inclined not to bother with a new release
unless we can have more consequential fixes.

> Does anyone have patches that would fit in a bug-fix release?

I can't connect at the moment, but perhaps
Kamil's patch here is needed?
https://bugzilla.redhat.com/show_bug.cgi?id=537463

I noticed a small issue this morning.
Assaf, the following warning shouldn't be printed, right?
The attached patch addresses that.

printf 'test\n1\n' | join --header -a1 - /dev/null
test
join: file 1 is not in sorted order
1




--------------030408040703070703060704
Content-Type: text/x-patch;
 name="join-empty-header.diff"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename="join-empty-header.diff"

>From c36b20e51f028d13d06c27d86c0cf009313bacd3 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?P=C3=A1draig=20Brady?= <P@draigBrady.com>
Date: Fri, 14 Jan 2011 08:46:21 +0000
Subject: [PATCH] join: ensure --header skips the order check with empty files

* src/join.c: Skip the header even if one of the files is empty.
* tests/misc/join: Add a test case.
* NEWS: Mention the fix
---
 NEWS            |    3 +++
 src/join.c      |   12 ++++++++----
 tests/misc/join |    6 ++++++
 3 files changed, 17 insertions(+), 4 deletions(-)

diff --git a/NEWS b/NEWS
index 9ccad63..82565b0 100644
--- a/NEWS
+++ b/NEWS
@@ -13,6 +13,9 @@ GNU coreutils NEWS                                    -*- outline -*-
   rm -f no longer fails for EINVAL or EILSEQ on file systems that
   reject file names invalid for that file system.
 
+  join --header now skips the ordering check for the first line
+  even if the other file is empty.
+
 
 * Noteworthy changes in release 8.9 (2011-01-04) [stable]
 
diff --git a/src/join.c b/src/join.c
index afda5a1..07ac856 100644
--- a/src/join.c
+++ b/src/join.c
@@ -627,13 +627,17 @@ join (FILE *fp1, FILE *fp2)
   initseq (&seq2);
   getseq (fp2, &seq2, 2);
 
-  if (join_header_lines && seq1.count && seq2.count)
+  if (join_header_lines && (seq1.count || seq2.count))
     {
-      prjoin (seq1.lines[0], seq2.lines[0]);
+      struct line const *hline1 = seq1.count ? seq1.lines[0] : &uni_blank;
+      struct line const *hline2 = seq2.count ? seq2.lines[0] : &uni_blank;
+      prjoin (hline1, hline2);
       prevline[0] = NULL;
       prevline[1] = NULL;
-      advance_seq (fp1, &seq1, true, 1);
-      advance_seq (fp2, &seq2, true, 2);
+      if (seq1.count)
+        advance_seq (fp1, &seq1, true, 1);
+      if (seq2.count)
+        advance_seq (fp2, &seq2, true, 2);
     }
 
   while (seq1.count && seq2.count)
diff --git a/tests/misc/join b/tests/misc/join
index 3696a03..0299427 100755
--- a/tests/misc/join
+++ b/tests/misc/join
@@ -219,6 +219,12 @@ my @tv = (
  [ "ID1 Name\n1 A\n2 B\n", "ID2 Color\n1 red\n"],
    "ID1 Name Color\n1 A red\n", 0],
 
+# '--header' doesn't check order of a header
+# even if there is no header in the second file
+['header-6', '--header -a1',
+ [ "ID1 Name\n1 A\n", ""],
+   "ID1 Name\n1 A\n", 0],
+
 );
 
 # Convert the above old-style test vectors to the newer
-- 
1.7.3.4


--------------030408040703070703060704--


From MAILER-DAEMON Fri Jan 14 07:43:13 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1Pdizp-0007iJ-QS
	for mharc-coreutils@gnu.org; Fri, 14 Jan 2011 07:43:13 -0500
Received: from [140.186.70.92] (port=51002 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1Pdizm-0007i9-Pc
	for coreutils@gnu.org; Fri, 14 Jan 2011 07:43:12 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Pdizl-0006Gp-32
	for coreutils@gnu.org; Fri, 14 Jan 2011 07:43:10 -0500
Received: from mail1.slb.deg.dub.stisp.net ([84.203.253.98]:11533)
	by eggs.gnu.org with smtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Pdizk-0006GW-OQ
	for coreutils@gnu.org; Fri, 14 Jan 2011 07:43:09 -0500
Received: (qmail 84024 invoked from network); 14 Jan 2011 12:43:06 -0000
Received: from unknown (HELO ?192.168.2.25?) (84.203.137.218)
	by mail1.slb.deg.dub.stisp.net with SMTP; 14 Jan 2011 12:43:06 -0000
Message-ID: <4D3044B4.10700@draigBrady.com>
Date: Fri, 14 Jan 2011 12:42:28 +0000
From: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US;
	rv:1.9.1.8) Gecko/20100227 Thunderbird/3.0.3
MIME-Version: 1.0
To: Assaf Gordon <assafgordon@gmail.com>
Subject: Re: [coreutils] join feature: auto-format
References: <4CACDEE5.6050908@gmail.com>	<4CAD0E47.7070005@draigBrady.com>	<4CAD9F55.6000001@draigBrady.com>	<4CAE06BA.2080402@gmail.com>	<4CAE1099.4010702@draigBrady.com>	<4D25AFED.1090902@draigBrady.com>
	<4D2C4E91.1070908@draigBrady.com> <4D2DED16.30101@gmail.com>
In-Reply-To: <4D2DED16.30101@gmail.com>
X-Enigmail-Version: 1.0.1
Content-Type: multipart/mixed; boundary="------------040401020202040005020100"
X-detected-operating-system: by eggs.gnu.org: FreeBSD 4.6-4.9
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 14 Jan 2011 12:43:12 -0000

This is a multi-part message in MIME format.
--------------040401020202040005020100
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit

On 12/01/11 18:04, Assaf Gordon wrote:
> Pádraig Brady wrote, On 01/11/2011 07:35 AM:
>>
>> Spending another few minutes on this, I realized
>> that we should not be trying to homogenize the number
>> of fields from each file, but rather the fields used
>> for a particular file in each line. The only sensible
>> basis for that is the first line as previously suggested.
>>
>> The interface would be a little different for that.
>> I was thinking of:
>>
>>   -o 'header'  Infer the format from the first line of each file
>>
> I second the idea of using the first line as the basis for the auto-formatting,
> but have reservation about the wording: '-o header' somewhat implies that the first line has to be an actual header line (with column names or similar), while it can just be the first line of actual data if the file doesn't have a header line.
> 
> Something like '-o auto' might be less confusing.

I've attached a patch that implements the above.
I'll hold off applying for a short while as
we might be doing a bug fix release.

cheers,
Pádraig.

--------------040401020202040005020100
Content-Type: text/x-patch;
 name="join-auto-format.diff"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename="join-auto-format.diff"

>From 26a1a63dfc7210722c7e14b9bb4b301719ba7eb6 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?P=C3=A1draig=20Brady?= <P@draigBrady.com>
Date: Wed, 5 Jan 2011 11:52:54 +0000
Subject: [PATCH] join: add -o 'auto' to output a constant number of fields per line

Lines with a different number of fields than the first line,
will be truncated or padded.

* src/join.c (prfields): A new function refactored from prjoin(),
to output all but the join field.
(prjoin): Don't swap line1 and line2 when line1 is blank
so that the padding is applied to the right place.
(main): Handle the -o 'auto' option.
* tests/misc/join: Add 4 new cases to test the auto format.
* NEWS: Mention the change in behavior.
Suggestion from Assaf Gordon
---
 NEWS               |    6 +++
 doc/coreutils.texi |   17 ++++++---
 src/join.c         |   88 +++++++++++++++++++++++++++++++++------------------
 tests/misc/join    |   14 ++++++++
 4 files changed, 88 insertions(+), 37 deletions(-)

diff --git a/NEWS b/NEWS
index 9ccad63..a9fd2a9 100644
--- a/NEWS
+++ b/NEWS
@@ -13,6 +13,12 @@ GNU coreutils NEWS                                    -*- outline -*-
   rm -f no longer fails for EINVAL or EILSEQ on file systems that
   reject file names invalid for that file system.
 
+** Changes in behavior
+
+  join now supports -o 'auto' which will automatically infer the
+  output format from the first line in each file, to ensure
+  the same number of fields are output for each line.
+
 
 * Noteworthy changes in release 8.9 (2011-01-04) [stable]
 
diff --git a/doc/coreutils.texi b/doc/coreutils.texi
index 85d5201..41c61bb 100644
--- a/doc/coreutils.texi
+++ b/doc/coreutils.texi
@@ -5675,8 +5675,8 @@ Do not check that both input files are in sorted order.  This is the default.
 
 @item -e @var{string}
 @opindex -e
-Replace those output fields that are missing in the input with
-@var{string}.
+Replace those output fields that are missing in the input with @var{string}.
+I.E. missing fields specified with the @option{-12jo} options.
 
 @item --header
 @opindex --header
@@ -5707,10 +5707,15 @@ Join on field @var{field} (a positive integer) of file 2.
 Equivalent to @option{-1 @var{field} -2 @var{field}}.
 
 @item -o @var{field-list}
-Construct each output line according to the format in @var{field-list}.
-Each element in @var{field-list} is either the single character @samp{0} or
-has the form @var{m.n} where the file number, @var{m}, is @samp{1} or
-@samp{2} and @var{n} is a positive field number.
+@itemx -o auto
+If the keyword @samp{auto} is specified, infer the output format from
+the first line in each file.  This is the same as the default output format
+but also ensures the same number of fields are output for each line.
+
+Otherwise, construct each output line according to the format in
+@var{field-list}.  Each element in @var{field-list} is either the single
+character @samp{0} or has the form @var{m.n} where the file number, @var{m},
+is @samp{1} or @samp{2} and @var{n} is a positive field number.
 
 A field specification of @samp{0} denotes the join field.
 In most cases, the functionality of the @samp{0} field spec
diff --git a/src/join.c b/src/join.c
index afda5a1..bf7e908 100644
--- a/src/join.c
+++ b/src/join.c
@@ -112,6 +112,13 @@ static bool issued_disorder_warning[2];
 /* Empty output field filler.  */
 static char const *empty_filler;
 
+/* Whether to ensure the same number of fields are output from each line.  */
+static bool autoformat;
+/* The number of fields to output for each line.
+   Only significant when autoformat is true.  */
+static size_t autocount_1;
+static size_t autocount_2;
+
 /* Field to join on; SIZE_MAX means they haven't been determined yet.  */
 static size_t join_field_1 = SIZE_MAX;
 static size_t join_field_2 = SIZE_MAX;
@@ -210,7 +217,8 @@ else fields are separated by CHAR.  Any FIELD is a field number counted\n\
 from 1.  FORMAT is one or more comma or blank separated specifications,\n\
 each being `FILENUM.FIELD' or `0'.  Default FORMAT outputs the join field,\n\
 the remaining fields from FILE1, the remaining fields from FILE2, all\n\
-separated by CHAR.\n\
+separated by CHAR.  If FORMAT is the keyword 'auto', then the first\n\
+line of each file determines the number of fields output for each line.\n\
 \n\
 Important: FILE1 and FILE2 must be sorted on the join fields.\n\
 E.g., use ` sort -k 1b,1 ' if `join' has no options,\n\
@@ -527,6 +535,27 @@ prfield (size_t n, struct line const *line)
     fputs (empty_filler, stdout);
 }
 
+/* Output all the fields in line, other than the join field.  */
+
+static void
+prfields (struct line const *line, size_t join_field, size_t autocount)
+{
+  size_t i;
+  size_t nfields = autoformat ? autocount : line->nfields;
+  char output_separator = tab < 0 ? ' ' : tab;
+
+  for (i = 0; i < join_field && i < nfields; ++i)
+    {
+      putchar (output_separator);
+      prfield (i, line);
+    }
+  for (i = join_field + 1; i < nfields; ++i)
+    {
+      putchar (output_separator);
+      prfield (i, line);
+    }
+}
+
 /* Print the join of LINE1 and LINE2.  */
 
 static void
@@ -534,6 +563,8 @@ prjoin (struct line const *line1, struct line const *line2)
 {
   const struct outlist *outlist;
   char output_separator = tab < 0 ? ' ' : tab;
+  size_t field;
+  struct line const *line;
 
   outlist = outlist_head.next;
   if (outlist)
@@ -543,9 +574,6 @@ prjoin (struct line const *line1, struct line const *line2)
       o = outlist;
       while (1)
         {
-          size_t field;
-          struct line const *line;
-
           if (o->file == 0)
             {
               if (line1 == &uni_blank)
@@ -574,37 +602,24 @@ prjoin (struct line const *line1, struct line const *line2)
     }
   else
     {
-      size_t i;
-
       if (line1 == &uni_blank)
         {
-          struct line const *t;
-          t = line1;
-          line1 = line2;
-          line2 = t;
+          line = line2;
+          field = join_field_2;
         }
-      prfield (join_field_1, line1);
-      for (i = 0; i < join_field_1 && i < line1->nfields; ++i)
-        {
-          putchar (output_separator);
-          prfield (i, line1);
-        }
-      for (i = join_field_1 + 1; i < line1->nfields; ++i)
+      else
         {
-          putchar (output_separator);
-          prfield (i, line1);
+          line = line1;
+          field = join_field_1;
         }
 
-      for (i = 0; i < join_field_2 && i < line2->nfields; ++i)
-        {
-          putchar (output_separator);
-          prfield (i, line2);
-        }
-      for (i = join_field_2 + 1; i < line2->nfields; ++i)
-        {
-          putchar (output_separator);
-          prfield (i, line2);
-        }
+      /* Output the join field.  */
+      prfield (field, line);
+
+      /* Output other fields.  */
+      prfields (line1, join_field_1, autocount_1);
+      prfields (line2, join_field_2, autocount_2);
+
       putchar ('\n');
     }
 }
@@ -627,6 +642,12 @@ join (FILE *fp1, FILE *fp2)
   initseq (&seq2);
   getseq (fp2, &seq2, 2);
 
+  if (autoformat)
+    {
+      autocount_1 = seq1.count ? seq1.lines[0]->nfields : 0;
+      autocount_2 = seq2.count ? seq2.lines[0]->nfields : 0;
+    }
+
   if (join_header_lines && seq1.count && seq2.count)
     {
       prjoin (seq1.lines[0], seq2.lines[0]);
@@ -1037,8 +1058,13 @@ main (int argc, char **argv)
           break;
 
         case 'o':
-          add_field_list (optarg);
-          optc_status = MIGHT_BE_O_ARG;
+          if (STREQ (optarg, "auto"))
+            autoformat = true;
+          else
+            {
+              add_field_list (optarg);
+              optc_status = MIGHT_BE_O_ARG;
+            }
           break;
 
         case 't':
diff --git a/tests/misc/join b/tests/misc/join
index 3696a03..4a8c6b6 100755
--- a/tests/misc/join
+++ b/tests/misc/join
@@ -127,6 +127,20 @@ my @tv = (
 # From David Dyck
 ['9a', '', [" a 1\n b 2\n", " a Y\n b Z\n"], "a 1 Y\nb 2 Z\n", 0],
 
+# -o 'auto'
+['10a', '-a1 -a2 -e . -o auto',
+ ["a 1 2\nb 1\nd 1 2\n", "a 3 4\nb 3 4\nc 3 4\n"],
+ "a 1 2 3 4\nb 1 . 3 4\nc . . 3 4\nd 1 2 . .\n", 0],
+['10b', '-a1 -a2 -j3 -e . -o auto',
+ ["a 1 2\nb 1\nd 1 2\n", "a 3 4\nb 3 4\nc 3 4\n"],
+ "2 a 1 . .\n. b 1 . .\n2 d 1 . .\n4 . . a 3\n4 . . b 3\n4 . . c 3\n"],
+['10c', '-a1 -1 1 -2 4 -e. -o auto',
+ ["a 1 2\nb 1\nd 1 2\n", "a 3 4\nb 3 4\nc 3 4\n"],
+ "a 1 2 . . .\nb 1 . . . .\nd 1 2 . . .\n"],
+['10d', '-a2 -1 1 -2 4 -e. -o auto',
+ ["a 1 2\nb 1\nd 1 2\n", "a 3 4\nb 3 4\nc 3 4\n"],
+ ". . . a 3 4\n. . . b 3 4\n. . . c 3 4\n"],
+
 # From Tim Smithers: fixed in 1.22l
 ['trailing-sp', '-t: -1 1 -2 1', ["a:x \n", "a:y \n"], "a:x :y \n", 0],
 
-- 
1.7.3.4


--------------040401020202040005020100--


From MAILER-DAEMON Fri Jan 14 07:47:42 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1Pdj4A-0000bs-72
	for mharc-coreutils@gnu.org; Fri, 14 Jan 2011 07:47:42 -0500
Received: from [140.186.70.92] (port=41973 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1Pdj48-0000ba-Q5
	for coreutils@gnu.org; Fri, 14 Jan 2011 07:47:41 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Pdj47-0006w5-O0
	for coreutils@gnu.org; Fri, 14 Jan 2011 07:47:40 -0500
Received: from mx.meyering.net ([82.230.74.64]:34918)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Pdj47-0006vt-IX
	for coreutils@gnu.org; Fri, 14 Jan 2011 07:47:39 -0500
Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000)
	id 6241960338; Fri, 14 Jan 2011 13:47:38 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: =?utf-8?Q?P=C3=A1draig?= Brady <P@draigBrady.com>
Subject: Re: [coreutils] might as well release yet again
In-Reply-To: <4D3038E7.1020908@draigBrady.com> (=?utf-8?Q?=22P=C3=A1draig?=
	Brady"'s message of "Fri, 14 Jan 2011 11:52:07 +0000")
References: <87k4i73f39.fsf@meyering.net> <4D3038E7.1020908@draigBrady.com>
Date: Fri, 14 Jan 2011 13:47:38 +0100
Message-ID: <8739ov3b4l.fsf@meyering.net>
Lines: 39
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 14 Jan 2011 12:47:41 -0000

P=C3=A1draig Brady wrote:
> On 14/01/11 11:22, Jim Meyering wrote:
>> I'm thinking of making yet another coreutils release.
>> The du bug isn't that big a deal, but still.
>> I prefer not to mix bug fixes and feature additions,
>> and a few of the latter are coming.
>
> I'd be inclined not to bother with a new release
> unless we can have more consequential fixes.

Ok.  I'm happy to put off the release for a couple weeks,
and to include features like your join auto-format.
Besides, there's Eric's sort vs multi-thread safety point.

>> Does anyone have patches that would fit in a bug-fix release?
...
> I noticed a small issue this morning.
> Assaf, the following warning shouldn't be printed, right?
> The attached patch addresses that.
>
> printf 'test\n1\n' | join --header -a1 - /dev/null
> test
> join: file 1 is not in sorted order
> 1

Good catch.

> Subject: [PATCH] join: ensure --header skips the order check with empty f=
iles
>
> * src/join.c: Skip the header even if one of the files is empty.
> * tests/misc/join: Add a test case.
> * NEWS: Mention the fix
> ---
>  NEWS            |    3 +++
>  src/join.c      |   12 ++++++++----
>  tests/misc/join |    6 ++++++
>  3 files changed, 17 insertions(+), 4 deletions(-)

That looks perfect.  Thanks!


From MAILER-DAEMON Fri Jan 14 07:52:51 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1Pdj99-0001em-Il
	for mharc-coreutils@gnu.org; Fri, 14 Jan 2011 07:52:51 -0500
Received: from [140.186.70.92] (port=37262 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1Pdj97-0001e8-SW
	for coreutils@gnu.org; Fri, 14 Jan 2011 07:52:50 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Pdj96-0007nO-QY
	for coreutils@gnu.org; Fri, 14 Jan 2011 07:52:49 -0500
Received: from mail1.slb.deg.dub.stisp.net ([84.203.253.98]:9543)
	by eggs.gnu.org with smtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Pdj96-0007nI-H6
	for coreutils@gnu.org; Fri, 14 Jan 2011 07:52:48 -0500
Received: (qmail 86076 invoked from network); 14 Jan 2011 12:52:47 -0000
Received: from unknown (HELO ?192.168.2.25?) (84.203.137.218)
	by mail1.slb.deg.dub.stisp.net with SMTP; 14 Jan 2011 12:52:47 -0000
Message-ID: <4D3046F8.3020101@draigBrady.com>
Date: Fri, 14 Jan 2011 12:52:08 +0000
From: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US;
	rv:1.9.1.8) Gecko/20100227 Thunderbird/3.0.3
MIME-Version: 1.0
To: Assaf Gordon <assafgordon@gmail.com>
Subject: Re: [coreutils] join feature: auto-format
References: <4CACDEE5.6050908@gmail.com>	<4CAD0E47.7070005@draigBrady.com>	<4CAD9F55.6000001@draigBrady.com>	<4CAE06BA.2080402@gmail.com>	<4CAE1099.4010702@draigBrady.com>	<4D25AFED.1090902@draigBrady.com>	<4D2C4E91.1070908@draigBrady.com>
	<4D2DED16.30101@gmail.com> <4D3044B4.10700@draigBrady.com>
In-Reply-To: <4D3044B4.10700@draigBrady.com>
X-Enigmail-Version: 1.0.1
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
X-detected-operating-system: by eggs.gnu.org: FreeBSD 4.6-4.9
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 14 Jan 2011 12:52:50 -0000

On 14/01/11 12:42, Pádraig Brady wrote:
> On 12/01/11 18:04, Assaf Gordon wrote:
>> Pádraig Brady wrote, On 01/11/2011 07:35 AM:
>>>
>>> Spending another few minutes on this, I realized
>>> that we should not be trying to homogenize the number
>>> of fields from each file, but rather the fields used
>>> for a particular file in each line. The only sensible
>>> basis for that is the first line as previously suggested.
>>>
>>> The interface would be a little different for that.
>>> I was thinking of:
>>>
>>>   -o 'header'  Infer the format from the first line of each file
>>>
>> I second the idea of using the first line as the basis for the auto-formatting,
>> but have reservation about the wording: '-o header' somewhat implies that the first line has to be an actual header line (with column names or similar), while it can just be the first line of actual data if the file doesn't have a header line.
>>
>> Something like '-o auto' might be less confusing.
> 
> I've attached a patch that implements the above.
> I'll hold off applying for a short while as
> we might be doing a bug fix release.

Actually Jim said this feature is Ok to add now.
I also need to amend NEWS to describe it as a
"New Feature" rather than as a "Change in behavior".

cheers,
Pádraig.


From MAILER-DAEMON Fri Jan 14 09:32:31 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1Pdkhb-0002y7-Iz
	for mharc-coreutils@gnu.org; Fri, 14 Jan 2011 09:32:31 -0500
Received: from [140.186.70.92] (port=55508 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1Pdibw-0005eP-Dq
	for coreutils@gnu.org; Fri, 14 Jan 2011 07:18:33 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <ovasik@redhat.com>) id 1Pdibu-00018L-OG
	for coreutils@gnu.org; Fri, 14 Jan 2011 07:18:32 -0500
Received: from mx1.redhat.com ([209.132.183.28]:47685)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <ovasik@redhat.com>) id 1Pdibu-00017l-H7
	for coreutils@gnu.org; Fri, 14 Jan 2011 07:18:30 -0500
Received: from int-mx02.intmail.prod.int.phx2.redhat.com
	(int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12])
	by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id p0ECIPPJ030813
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK);
	Fri, 14 Jan 2011 07:18:25 -0500
Received: from [10.34.25.213] (dhcp-25-213.brq.redhat.com [10.34.25.213])
	by int-mx02.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP
	id p0ECIMMI005750; Fri, 14 Jan 2011 07:18:23 -0500
Subject: Re: [coreutils] might as well release yet again
From: Ondrej Vasik <ovasik@redhat.com>
To: =?ISO-8859-1?Q?P=E1draig?= Brady <P@draigBrady.com>
In-Reply-To: <4D3038E7.1020908@draigBrady.com>
References: <87k4i73f39.fsf@meyering.net>  <4D3038E7.1020908@draigBrady.com>
Content-Type: text/plain; charset="UTF-8"
Organization: Red Hat, Inc.
Date: Fri, 14 Jan 2011 13:19:53 +0100
Message-ID: <1295007593.3001.1.camel@dhcp-24-233.brq.redhat.com>
Mime-Version: 1.0
X-Scanned-By: MIMEDefang 2.67 on 10.5.11.12
Content-Transfer-Encoding: quoted-printable
X-MIME-Autoconverted: from 8bit to quoted-printable by mx1.redhat.com id
	p0ECIPPJ030813
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Mailman-Approved-At: Fri, 14 Jan 2011 09:32:30 -0500
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
Reply-To: ovasik@redhat.com
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 14 Jan 2011 12:18:33 -0000

P=C3=A1draig Brady wrote:
> On 14/01/11 11:22, Jim Meyering wrote:
> > I'm thinking of making yet another coreutils release.
> > The du bug isn't that big a deal, but still.
> > I prefer not to mix bug fixes and feature additions,
> > and a few of the latter are coming.
>=20
> I'd be inclined not to bother with a new release
> unless we can have more consequential fixes.
>=20
> > Does anyone have patches that would fit in a bug-fix release?
>=20
> I can't connect at the moment, but perhaps
> Kamil's patch here is needed?
> https://bugzilla.redhat.com/show_bug.cgi?id=3D537463

That one is already in gnulib -
http://git.savannah.gnu.org/gitweb/?p=3Dgnulib.git;a=3Dcommitdiff;h=3Dac6=
7780e88411e8fcb94cda0a9e0a9abab6f8369 , it was just RHEL-5 version of the=
 bug and easy backport.

Ondrej



From MAILER-DAEMON Fri Jan 14 13:55:08 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1Pdonk-00083Z-8W
	for mharc-coreutils@gnu.org; Fri, 14 Jan 2011 13:55:08 -0500
Received: from [140.186.70.92] (port=51016 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1Pdoei-0004HY-40
	for coreutils@gnu.org; Fri, 14 Jan 2011 13:45:49 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <public@plane.gmane.org>) id 1Pdoeg-00032e-QZ
	for coreutils@gnu.org; Fri, 14 Jan 2011 13:45:47 -0500
Received: from plane.gmane.org ([80.91.229.3]:36022)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <public@plane.gmane.org>) id 1Pdoeg-00032S-Iu
	for coreutils@gnu.org; Fri, 14 Jan 2011 13:45:46 -0500
Received: from public by plane.gmane.org with local (Exim 4.69)
	(envelope-from <public@plane.gmane.org>) id 1PdoeZ-0006Bg-QB
	for coreutils@gnu.org; Fri, 14 Jan 2011 19:45:39 +0100
Received: from mail-out.m-online.net ([212.18.0.10])
	by plane.gmane.org with esmtp (Exim 4.69)
	(envelope-from <whitebox@nefkom.net>)
	id 1PdoeN-000649-8m; Fri, 14 Jan 2011 19:45:27 +0100
Received: from frontend1.mail.m-online.net (frontend1.mail.intern.m-online.net
	[192.168.8.180])
	by mail-out.m-online.net (Postfix) with ESMTP id 6709C188B59A;
	Fri, 14 Jan 2011 19:45:26 +0100 (CET)
Received: from localhost (dynscan1.mnet-online.de [192.168.8.164])
	by mail.m-online.net (Postfix) with ESMTP id AF2521C0061C;
	Fri, 14 Jan 2011 19:45:26 +0100 (CET)
X-Virus-Scanned: amavisd-new at mnet-online.de
Received: from mail.mnet-online.de ([192.168.8.180])
	by localhost (dynscan1.mail.m-online.net [192.168.8.164]) (amavisd-new,
	port 10024)
	with ESMTP id JV9PKhZbpzlb; Fri, 14 Jan 2011 19:45:25 +0100 (CET)
Received: from igel.home (ppp-88-217-115-226.dynamic.mnet-online.de
	[88.217.115.226]) by mail.mnet-online.de (Postfix) with ESMTP;
	Fri, 14 Jan 2011 19:45:25 +0100 (CET)
Received: by igel.home (Postfix, from userid 501)
	id 1BAC6CA2A0; Fri, 14 Jan 2011 19:45:24 +0100 (CET)
From: Andreas Schwab <schwab@linux-m68k.org>
To: Eric Blake <public-eblake-H+wXaHxf7aLQT0dZR+AlfA@plane.gmane.org>
References: <4D2DF65B.2040500@kickflop.net> <4D2DFAB7.6020809@redhat.com>
	<4D2E2687.9040107@kickflop.net> <87bp3lck9u.fsf@meyering.net>
	<4D2F8790.6010904@kickflop.net> <4D2F8C52.9010105@kickflop.net>
	<4D2F9002.5070201@redhat.com>
X-Yow: ..  are the STEWED PRUNES still in the HAIR DRYER?
Date: Fri, 14 Jan 2011 19:45:24 +0100
In-Reply-To: <4D2F9002.5070201@redhat.com> (Eric Blake's message of "Thu, 13
	Jan 2011 16:51:30 -0700")
Message-ID: <m2d3nzcojf.fsf@igel.home>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.2.91 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Mailman-Approved-At: Fri, 14 Jan 2011 13:55:06 -0500
Cc: public-coreutils-mXXj517/zsQ@plane.gmane.org,
	Jeff Blaine <public-jblaine-GbE5gUWZ6k7k1uMJSBkQmQ@plane.gmane.org>
Subject: [coreutils] Re: basename/dirname can't handle stdin?
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 14 Jan 2011 18:45:49 -0000



Eric Blake <eblake-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> writes:

> On 01/13/2011 04:35 PM, Jeff Blaine wrote:
>> Eric wrote:
>>> basename -a -s suffix str1 str2: same (-a is only required for exactly
>>> two arguements, and is implied for 0 or for more than 2 args)
>> 
>> basename -a -s .txt foo.txt bar.txt baz.txt
>> 
>> So you're saying -a is redundant there?  Sorry, I'm not
>> familiar with BSD's basename at all.  Just looking to
>> implement what you want.
>
> Yes, the -a is redundant the moment you have -s.  (It helps that I have
> a login to a FreeBSD machine, and have been testing these cases as I
> type these mails).  It is only required when you have exactly 2
> arguments but don't want the second to be treated as a suffix.

The manpages are online under <http://www.freebsd.org/cgi/man.cgi>.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."



From MAILER-DAEMON Sun Jan 16 18:53:18 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PecPO-0005AB-D6
	for mharc-coreutils@gnu.org; Sun, 16 Jan 2011 18:53:18 -0500
Received: from [140.186.70.92] (port=55474 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PecPN-0005A5-0H
	for coreutils@gnu.org; Sun, 16 Jan 2011 18:53:18 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <kerolasa@gmail.com>) id 1PecPL-0002ld-CN
	for coreutils@gnu.org; Sun, 16 Jan 2011 18:53:16 -0500
Received: from mail-qw0-f41.google.com ([209.85.216.41]:40283)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <kerolasa@gmail.com>) id 1PecPL-0002lP-8l
	for coreutils@gnu.org; Sun, 16 Jan 2011 18:53:15 -0500
Received: by qwa26 with SMTP id 26so4083042qwa.0
	for <coreutils@gnu.org>; Sun, 16 Jan 2011 15:53:14 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma;
	h=domainkey-signature:mime-version:sender:reply-to:date
	:x-google-sender-auth:message-id:subject:from:to:content-type;
	bh=GIN5rsJ3ajDLcumKoJuQn77uPE2xMXL9R/UM/S7DMAg=;
	b=dKLGY/lzXShS8tdyyPRBSDt/bSb2MLpp+VGl6ogM1pjBRoh4t+vlS7xFGj0QJyclBd
	OMBaP8Ui54GlIlNVg+mIAVuBQjG7m3aNdcX7W1hJbX+nlGwum6o0pzA0t8o5vspk8nFM
	OtOVy0Sf/6O8NECBc8jrYwW1IDP+UVX+T1dko=
DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma;
	h=mime-version:sender:reply-to:date:x-google-sender-auth:message-id
	:subject:from:to:content-type;
	b=LmVqBZ11M12C2gZ2wSiF5CfEa/5Ip6aAQUKg4jtIO6tPTqRCqq9g77RmG/W0XpRsWB
	WHmhAEnl0kLc9E44NsBOLsAYq/qVITHgpMYk4g3e+30dBlxO/RIv4rxanZaefGP13hBb
	hBQfRMm+KcLn4nHjmsWh1x4GyNQlX/O1YKcK0=
MIME-Version: 1.0
Received: by 10.229.184.1 with SMTP id ci1mr3043836qcb.193.1295221993198; Sun,
	16 Jan 2011 15:53:13 -0800 (PST)
Sender: kerolasa@gmail.com
Received: by 10.229.97.207 with HTTP; Sun, 16 Jan 2011 15:53:12 -0800 (PST)
Date: Mon, 17 Jan 2011 00:53:12 +0100
X-Google-Sender-Auth: 6Zz-c-AMn5x0dh6IbajhqMjRdEc
Message-ID: <AANLkTikowrdL7D3FszcrZ_WVuFJntZko6CV7J_hUyBnL@mail.gmail.com>
From: Sami Kerola <kerolasa@iki.fi>
To: coreutils@gnu.org
Content-Type: multipart/mixed; boundary=0016e68b68e8cf561d0499ff5ef2
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 2)
Subject: [coreutils] [PATCH] uniq: don't continue field processing after end
	of line
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
Reply-To: kerolasa@gmail.com
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 16 Jan 2011 23:53:18 -0000

--0016e68b68e8cf561d0499ff5ef2
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Hi,

I notice uniq -f 'insane_large_number' will make process to be busy
long time without good reason. Attached patch should fix this symptom.

I found the bug after friend of mine asked why uniq does not allow
specifying field separator, similar way sort -t. I could not answer
anything rational, so I look the code and tested how it works. That
inspired me to send bug fix, which is obvious thing to do. But how
about that -t, do you think this would be worth while addition to
uniq?

--=20
=A0=A0 Sami Kerola
=A0=A0 http://www.iki.fi/kerolasa/

--0016e68b68e8cf561d0499ff5ef2
Content-Type: text/plain; charset=US-ASCII; 
	name="0001-uniq-don-t-continue-field-processing-after-end-of-line.txt"
Content-Disposition: attachment; 
	filename="0001-uniq-don-t-continue-field-processing-after-end-of-line.txt"
Content-Transfer-Encoding: base64
X-Attachment-Id: f_gj0laznn0

RnJvbSA0NjUyNDkzYzkwOTQwZTU0NjBjY2ZlYjhlZDNlMjMxZTA2ZWU0NmRjIE1vbiBTZXAgMTcg
MDA6MDA6MDAgMjAwMQpGcm9tOiBTYW1pIEtlcm9sYSA8a2Vyb2xhc2FAaWtpLmZpPgpEYXRlOiBN
b24sIDE3IEphbiAyMDExIDAwOjI3OjA2ICswMTAwClN1YmplY3Q6IFtQQVRDSF0gdW5pcTogZG9u
J3QgY29udGludWUgZmllbGQgcHJvY2Vzc2luZyBhZnRlciBlbmQgb2YgbGluZQoKKiBORVdTIChC
dWcgZml4ZXMpOiBNZW50aW9uIGl0LgoqIHNyYy91bmlxLmMgKGZpbmRfZmllbGQpOiBTdG9wIHBy
b2Nlc3NpbmcgbG9vcCB3aGVuIGVuZCBvZiBsaW5lCmlzIHJlYWNoZWQuIEJlZm9yZSB0aGlzIGZp
eCAndW5pcSAtZiAxMDAwMDAwMDAwMCAvZXRjL3Bhc3N3ZCcgcmFuCnF1aXRlIGxvbmcgdGltZS4K
LS0tCiBORVdTICAgICAgIHwgICAgMyArKysKIHNyYy91bmlxLmMgfCAgICAyICstCiAyIGZpbGVz
IGNoYW5nZWQsIDQgaW5zZXJ0aW9ucygrKSwgMSBkZWxldGlvbnMoLSkKCmRpZmYgLS1naXQgYS9O
RVdTIGIvTkVXUwppbmRleCA5Y2NhZDYzLi43MTAxOGRkIDEwMDY0NAotLS0gYS9ORVdTCisrKyBi
L05FV1MKQEAgLTEzLDYgKzEzLDkgQEAgR05VIGNvcmV1dGlscyBORVdTICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgLSotIG91dGxpbmUgLSotCiAgIHJtIC1mIG5vIGxvbmdlciBm
YWlscyBmb3IgRUlOVkFMIG9yIEVJTFNFUSBvbiBmaWxlIHN5c3RlbXMgdGhhdAogICByZWplY3Qg
ZmlsZSBuYW1lcyBpbnZhbGlkIGZvciB0aGF0IGZpbGUgc3lzdGVtLgogCisgIHVuaXEgLWYgTlVN
IHdpbGwgbm8gbG9uZ2VyIHRyeSB0byBwcm9jZXNzIGxpbmVzIGFmdGVyIGVuZCBvZgorICBsaW5l
IGlzIHJlYWNoLgorCiAKICogTm90ZXdvcnRoeSBjaGFuZ2VzIGluIHJlbGVhc2UgOC45ICgyMDEx
LTAxLTA0KSBbc3RhYmxlXQogCmRpZmYgLS1naXQgYS9zcmMvdW5pcS5jIGIvc3JjL3VuaXEuYwpp
bmRleCA3YmRiYzRmLi42ZmE2Y2M5IDEwMDY0NAotLS0gYS9zcmMvdW5pcS5jCisrKyBiL3NyYy91
bmlxLmMKQEAgLTIxNCw3ICsyMTQsNyBAQCBmaW5kX2ZpZWxkIChzdHJ1Y3QgbGluZWJ1ZmZlciBj
b25zdCAqbGluZSkKICAgc2l6ZV90IHNpemUgPSBsaW5lLT5sZW5ndGggLSAxOwogICBzaXplX3Qg
aSA9IDA7CiAKLSAgZm9yIChjb3VudCA9IDA7IGNvdW50IDwgc2tpcF9maWVsZHM7IGNvdW50Kysp
CisgIGZvciAoY291bnQgPSAwOyBjb3VudCA8IHNraXBfZmllbGRzICYmIGNvdW50IDwgbGluZS0+
bGVuZ3RoOyBjb3VudCsrKQogICAgIHsKICAgICAgIHdoaWxlIChpIDwgc2l6ZSAmJiBpc2JsYW5r
ICh0b191Y2hhciAobHBbaV0pKSkKICAgICAgICAgaSsrOwotLSAKMS43LjMuNQoK
--0016e68b68e8cf561d0499ff5ef2--


From MAILER-DAEMON Mon Jan 17 04:57:37 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PelqD-0004i5-QQ
	for mharc-coreutils@gnu.org; Mon, 17 Jan 2011 04:57:37 -0500
Received: from [140.186.70.92] (port=60461 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PelqB-0004hN-GD
	for coreutils@gnu.org; Mon, 17 Jan 2011 04:57:36 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1PelqA-0001jV-1W
	for coreutils@gnu.org; Mon, 17 Jan 2011 04:57:35 -0500
Received: from mail1.slb.deg.dub.stisp.net ([84.203.253.98]:17311)
	by eggs.gnu.org with smtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Pelq9-0001if-OF
	for coreutils@gnu.org; Mon, 17 Jan 2011 04:57:33 -0500
Received: (qmail 71405 invoked from network); 17 Jan 2011 09:57:30 -0000
Received: from unknown (HELO ?192.168.2.25?) (84.203.137.218)
	by mail1.slb.deg.dub.stisp.net with SMTP; 17 Jan 2011 09:57:30 -0000
Message-ID: <4D341256.9000703@draigBrady.com>
Date: Mon, 17 Jan 2011 09:56:38 +0000
From: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US;
	rv:1.9.1.8) Gecko/20100227 Thunderbird/3.0.3
MIME-Version: 1.0
To: kerolasa@gmail.com
Subject: Re: [coreutils] [PATCH] uniq: don't continue field processing after
	end	of line
References: <AANLkTikowrdL7D3FszcrZ_WVuFJntZko6CV7J_hUyBnL@mail.gmail.com>
In-Reply-To: <AANLkTikowrdL7D3FszcrZ_WVuFJntZko6CV7J_hUyBnL@mail.gmail.com>
X-Enigmail-Version: 1.0.1
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
X-detected-operating-system: by eggs.gnu.org: FreeBSD 4.6-4.9
Cc: Sami Kerola <kerolasa@iki.fi>, coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 17 Jan 2011 09:57:36 -0000

On 16/01/11 23:53, Sami Kerola wrote:
> Hi,
> 
> I notice uniq -f 'insane_large_number' will make process to be busy
> long time without good reason. Attached patch should fix this symptom.

I'd slightly amend that to the following,
to match the other limit checks in the function.

diff --git a/src/uniq.c b/src/uniq.c
index 7bdbc4f..9c7e37c 100644
--- a/src/uniq.c
+++ b/src/uniq.c
@@ -214,7 +214,7 @@ find_field (struct linebuffer const *line)
   size_t size = line->length - 1;
   size_t i = 0;

-  for (count = 0; count < skip_fields; count++)
+  for (count = 0; count < skip_fields && i < size; count++)
     {
       while (i < size && isblank (to_uchar (lp[i])))

> I found the bug after friend of mine asked why uniq does not allow
> specifying field separator, similar way sort -t. I could not answer
> anything rational, so I look the code and tested how it works. That
> inspired me to send bug fix, which is obvious thing to do. But how
> about that -t, do you think this would be worth while addition to
> uniq?

yes. Basically `uniq` should support the same -k and -t
functionality that `sort` does. See also http://debbugs.gnu.org/5832

cheers,
Pádraig.


From MAILER-DAEMON Mon Jan 17 05:36:29 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PemRp-0007ep-IW
	for mharc-coreutils@gnu.org; Mon, 17 Jan 2011 05:36:29 -0500
Received: from [140.186.70.92] (port=37963 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PemRn-0007ea-2W
	for coreutils@gnu.org; Mon, 17 Jan 2011 05:36:27 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PemRf-0001Gm-7o
	for coreutils@gnu.org; Mon, 17 Jan 2011 05:36:26 -0500
Received: from mx.meyering.net ([82.230.74.64]:58273)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PemRe-0001G7-Vm
	for coreutils@gnu.org; Mon, 17 Jan 2011 05:36:19 -0500
Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000)
	id 643B76010F; Mon, 17 Jan 2011 11:36:17 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: coreutils@gnu.org
Date: Mon, 17 Jan 2011 11:36:17 +0100
Message-ID: <87vd1nn7fi.fsf@meyering.net>
Lines: 38
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
Subject: [coreutils] [PATCH] doc: show how to shred using a single
	zero-writing pass
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 17 Jan 2011 10:36:28 -0000

I have just wiped a 250GB drive and didn't want to wait for
the default 3-pass process.  One pass is slow enough.
Having to use -n0 --zero seemed sufficiently un-obvious that
I added this example to the manual:

>From 7dc6335653afcdad9a3ffa327877571734644285 Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering@redhat.com>
Date: Mon, 17 Jan 2011 11:32:35 +0100
Subject: [PATCH] doc: show how to shred using a single zero-writing pass

* doc/coreutils.texi (shred invocation): Give an example showing how
to invoke shred in its most basic (fastest) write-only-zeros mode.
---
 doc/coreutils.texi |    9 +++++++++
 1 files changed, 9 insertions(+), 0 deletions(-)

diff --git a/doc/coreutils.texi b/doc/coreutils.texi
index 9c3e2ed..8fb9f0c 100644
--- a/doc/coreutils.texi
+++ b/doc/coreutils.texi
@@ -8892,6 +8892,15 @@ shred invocation
 shred --verbose /dev/sda5
 @end example

+On modern disks, a single pass that writes only zeros may be enough,
+and it will be much faster than the default.
+Use a command like this to tell @command{shred} to skip all random
+passes and to perform only a final zero-writing pass:
+
+@example
+shred --verbose -n0 --zero /dev/sda5
+@end example
+
 A @var{file} of @samp{-} denotes standard output.
 The intended use of this is to shred a removed temporary file.
 For example:
--
1.7.3.5


From MAILER-DAEMON Mon Jan 17 06:27:08 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PenEq-0008Bx-HX
	for mharc-coreutils@gnu.org; Mon, 17 Jan 2011 06:27:08 -0500
Received: from [140.186.70.92] (port=52046 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PenEo-0008Bs-7l
	for coreutils@gnu.org; Mon, 17 Jan 2011 06:27:07 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1PenEn-0003i3-3D
	for coreutils@gnu.org; Mon, 17 Jan 2011 06:27:06 -0500
Received: from mail1.slb.deg.dub.stisp.net ([84.203.253.98]:4095)
	by eggs.gnu.org with smtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1PenEm-0003hy-Jw
	for coreutils@gnu.org; Mon, 17 Jan 2011 06:27:04 -0500
Received: (qmail 90491 invoked from network); 17 Jan 2011 11:27:02 -0000
Received: from unknown (HELO ?192.168.2.25?) (84.203.137.218)
	by mail1.slb.deg.dub.stisp.net with SMTP; 17 Jan 2011 11:27:02 -0000
Message-ID: <4D342751.7090403@draigBrady.com>
Date: Mon, 17 Jan 2011 11:26:09 +0000
From: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US;
	rv:1.9.1.8) Gecko/20100227 Thunderbird/3.0.3
MIME-Version: 1.0
To: Jim Meyering <jim@meyering.net>
Subject: Re: [coreutils] [PATCH] doc: show how to shred using a
	single	zero-writing pass
References: <87vd1nn7fi.fsf@meyering.net>
In-Reply-To: <87vd1nn7fi.fsf@meyering.net>
X-Enigmail-Version: 1.0.1
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
X-detected-operating-system: by eggs.gnu.org: FreeBSD 4.6-4.9
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 17 Jan 2011 11:27:07 -0000

On 17/01/11 10:36, Jim Meyering wrote:
>>From 7dc6335653afcdad9a3ffa327877571734644285 Mon Sep 17 00:00:00 2001
> From: Jim Meyering <meyering@redhat.com>
> Date: Mon, 17 Jan 2011 11:32:35 +0100
> Subject: [PATCH] doc: show how to shred using a single zero-writing pass
> 
> * doc/coreutils.texi (shred invocation): Give an example showing how
> to invoke shred in its most basic (fastest) write-only-zeros mode.
> ---
>  doc/coreutils.texi |    9 +++++++++
>  1 files changed, 9 insertions(+), 0 deletions(-)
> 
> diff --git a/doc/coreutils.texi b/doc/coreutils.texi
> index 9c3e2ed..8fb9f0c 100644
> --- a/doc/coreutils.texi
> +++ b/doc/coreutils.texi
> @@ -8892,6 +8892,15 @@ shred invocation
>  shred --verbose /dev/sda5
>  @end example
> 
> +On modern disks, a single pass that writes only zeros may be enough,
> +and it will be much faster than the default.

Well only 3 times, due to the disk being the bottleneck
(since we changed to the fast internal PRNG by default).
Also for security, writing random data would probably be more effective.
So I'd reword the above sentence to:

"To simply clear a disk"

> +Use a command like this to tell @command{shred} to skip all random
> +passes and to perform only a final zero-writing pass:
> +
> +@example
> +shred --verbose -n0 --zero /dev/sda5
> +@end example

It's probably not worth noting the equivalent:
dd conv=fdatasync bs=2M < /dev/zero > /dev/sda5

cheers,
Pádraig.


From MAILER-DAEMON Mon Jan 17 06:31:00 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PenIa-0001As-OX
	for mharc-coreutils@gnu.org; Mon, 17 Jan 2011 06:31:00 -0500
Received: from [140.186.70.92] (port=38982 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PenIY-00019e-NV
	for coreutils@gnu.org; Mon, 17 Jan 2011 06:30:59 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PenIX-0004Nk-9J
	for coreutils@gnu.org; Mon, 17 Jan 2011 06:30:58 -0500
Received: from mx.meyering.net ([82.230.74.64]:35423)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PenIW-0004NP-Ve
	for coreutils@gnu.org; Mon, 17 Jan 2011 06:30:57 -0500
Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000)
	id 6895F6032C; Mon, 17 Jan 2011 12:30:54 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: =?iso-8859-1?Q?P=E1draig?= Brady <P@draigBrady.com>
Subject: Re: [coreutils] [PATCH] uniq: don't continue field processing after
	end	of line
In-Reply-To: <4D341256.9000703@draigBrady.com> (=?iso-8859-1?Q?=22P=E1drai?=
	=?iso-8859-1?Q?g?= Brady"'s message of
	"Mon, 17 Jan 2011 09:56:38 +0000")
References: <AANLkTikowrdL7D3FszcrZ_WVuFJntZko6CV7J_hUyBnL@mail.gmail.com>
	<4D341256.9000703@draigBrady.com>
Date: Mon, 17 Jan 2011 12:30:54 +0100
Message-ID: <87pqrvn4wh.fsf@meyering.net>
Lines: 131
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
Cc: coreutils@gnu.org, kerolasa@gmail.com, Sami Kerola <kerolasa@iki.fi>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 17 Jan 2011 11:30:59 -0000

P=E1draig Brady wrote:

> On 16/01/11 23:53, Sami Kerola wrote:
>> Hi,
>>
>> I notice uniq -f 'insane_large_number' will make process to be busy
>> long time without good reason. Attached patch should fix this symptom.
>
> I'd slightly amend that to the following,
> to match the other limit checks in the function.
>
> diff --git a/src/uniq.c b/src/uniq.c
> index 7bdbc4f..9c7e37c 100644
> --- a/src/uniq.c
> +++ b/src/uniq.c
> @@ -214,7 +214,7 @@ find_field (struct linebuffer const *line)
>    size_t size =3D line->length - 1;
>    size_t i =3D 0;
>
> -  for (count =3D 0; count < skip_fields; count++)
> +  for (count =3D 0; count < skip_fields && i < size; count++)
>      {
>        while (i < size && isblank (to_uchar (lp[i])))

Thank you!
I've also adjusted NEWS.
Here's your adjusted patch, followed by another to add a test
to exercise the bug/fix:

>From bf0ed321332b01fc38eb892d1deac16629aea07c Mon Sep 17 00:00:00 2001
From: Sami Kerola <kerolasa@iki.fi>
Date: Mon, 17 Jan 2011 00:27:06 +0100
Subject: [PATCH] uniq: don't continue field processing after end of line

* NEWS (Bug fixes): Mention it.
* src/uniq.c (find_field): Stop processing loop when end of line
is reached.  Before this fix, 'uniq -f 10000000000 /etc/passwd'
would run for a very long time.
---
 NEWS       |    2 ++
 src/uniq.c |    2 +-
 2 files changed, 3 insertions(+), 1 deletions(-)

diff --git a/NEWS b/NEWS
index 9ccad63..3ec35c7 100644
--- a/NEWS
+++ b/NEWS
@@ -13,6 +13,8 @@ GNU coreutils NEWS                                    -*-=
 outline -*-
   rm -f no longer fails for EINVAL or EILSEQ on file systems that
   reject file names invalid for that file system.

+  uniq -f NUM no longer tries to process fields after end of line.
+

 * Noteworthy changes in release 8.9 (2011-01-04) [stable]

diff --git a/src/uniq.c b/src/uniq.c
index 7bdbc4f..9c7e37c 100644
--- a/src/uniq.c
+++ b/src/uniq.c
@@ -214,7 +214,7 @@ find_field (struct linebuffer const *line)
   size_t size =3D line->length - 1;
   size_t i =3D 0;

-  for (count =3D 0; count < skip_fields; count++)
+  for (count =3D 0; count < skip_fields && i < size; count++)
     {
       while (i < size && isblank (to_uchar (lp[i])))
         i++;
--
1.7.3.5


>From 660d57085140c86387f84eb508f4e15caa972bee Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering@redhat.com>
Date: Mon, 17 Jan 2011 12:27:55 +0100
Subject: [PATCH] tests: add a test for today's uniq bug

* tests/misc/uniq-perf: New file.
* tests/Makefile.am (TESTS): Add it.
---
 tests/Makefile.am    |    1 +
 tests/misc/uniq-perf |   25 +++++++++++++++++++++++++
 2 files changed, 26 insertions(+), 0 deletions(-)
 create mode 100755 tests/misc/uniq-perf

diff --git a/tests/Makefile.am b/tests/Makefile.am
index a5dbd3e..1e4e300 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -286,6 +286,7 @@ TESTS =3D						\
   misc/tty-eof					\
   misc/unexpand					\
   misc/uniq					\
+  misc/uniq-perf				\
   misc/xattr					\
   tail-2/wait					\
   chmod/c-option				\
diff --git a/tests/misc/uniq-perf b/tests/misc/uniq-perf
new file mode 100755
index 0000000..f000e76
--- /dev/null
+++ b/tests/misc/uniq-perf
@@ -0,0 +1,25 @@
+#!/bin/sh
+# before coreutils-8.10, seq 100000|uniq -f 10000000000 would run for days
+
+# Copyright (C) 2011 Free Software Foundation, Inc.
+
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+. "${srcdir=3D.}/init.sh"; path_prepend_ ../src
+print_ver_ uniq
+
+seq 100 > in || fail=3D1
+timeout 1 uniq -f 10000000000 in || fail=3D1
+
+Exit $fail
--
1.7.3.5


From MAILER-DAEMON Mon Jan 17 06:45:17 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PenWP-0003uF-1A
	for mharc-coreutils@gnu.org; Mon, 17 Jan 2011 06:45:17 -0500
Received: from [140.186.70.92] (port=33330 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PenWM-0003tw-MK
	for coreutils@gnu.org; Mon, 17 Jan 2011 06:45:15 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PenWL-0006jB-Ad
	for coreutils@gnu.org; Mon, 17 Jan 2011 06:45:14 -0500
Received: from mx.meyering.net ([82.230.74.64]:43848)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PenWL-0006ip-5A
	for coreutils@gnu.org; Mon, 17 Jan 2011 06:45:13 -0500
Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000)
	id AC10A6010F; Mon, 17 Jan 2011 12:45:11 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: coreutils@gnu.org
Date: Mon, 17 Jan 2011 12:45:11 +0100
Message-ID: <87k4i3n48o.fsf@meyering.net>
Lines: 41
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
Subject: [coreutils] uniq: minor code/perf improvement
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 17 Jan 2011 11:45:15 -0000

I noticed another wasteful loop in that same function.
The loop can be replaced with a simple "MIN(...)" expression, and that gives
a 10% performance improvement for very large --skip-chars offsets:

    $ (n=1000000000; printf "a%0${n}d0\n" 0;printf "b%0${n}d1\n" 0) \
      | env time --f=%e ./uniq -s$n > /dev/null
    6.03

    $ (n=1000000000; printf "a%0${n}d0\n" 0;printf "b%0${n}d1\n" 0) \
      | env time --f=%e uniq -s$n > /dev/null
    6.80


>From 9aa02d21f75c3c0dd14b4e406ebd0dff0c75504e Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering@redhat.com>
Date: Mon, 17 Jan 2011 12:36:58 +0100
Subject: [PATCH] uniq: replace a wasteful loop with simple calculation

* src/uniq.c (find_field): Remove the byte-skipping loop altogether.
Instead, perform the simple calculation.  This results in a 10%
performance improvement for large byte offsets.
---
 src/uniq.c |    3 +--
 1 files changed, 1 insertions(+), 2 deletions(-)

diff --git a/src/uniq.c b/src/uniq.c
index 9c7e37c..b35938a 100644
--- a/src/uniq.c
+++ b/src/uniq.c
@@ -222,8 +222,7 @@ find_field (struct linebuffer const *line)
         i++;
     }

-  for (count = 0; count < skip_chars && i < size; count++)
-    i++;
+  i += MIN (skip_chars, size - i);

   return line->buffer + i;
 }
--
1.7.3.5


From MAILER-DAEMON Mon Jan 17 12:08:34 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PesZG-0003T5-93
	for mharc-coreutils@gnu.org; Mon, 17 Jan 2011 12:08:34 -0500
Received: from [140.186.70.92] (port=52279 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PesZC-0003Py-Bd
	for coreutils@gnu.org; Mon, 17 Jan 2011 12:08:32 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PesZ3-0003r9-K1
	for coreutils@gnu.org; Mon, 17 Jan 2011 12:08:24 -0500
Received: from mx.meyering.net ([82.230.74.64]:33262)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PesZ3-0003qI-87
	for coreutils@gnu.org; Mon, 17 Jan 2011 12:08:21 -0500
Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000)
	id 0A4906010F; Mon, 17 Jan 2011 18:08:18 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: =?iso-8859-1?Q?P=E1draig?= Brady <P@draigBrady.com>
Subject: Re: [coreutils] [PATCH] doc: show how to shred using a
	single	zero-writing pass
In-Reply-To: <4D342751.7090403@draigBrady.com> (=?iso-8859-1?Q?=22P=E1drai?=
	=?iso-8859-1?Q?g?= Brady"'s message of
	"Mon, 17 Jan 2011 11:26:09 +0000")
References: <87vd1nn7fi.fsf@meyering.net> <4D342751.7090403@draigBrady.com>
Date: Mon, 17 Jan 2011 18:08:18 +0100
Message-ID: <871v4bmpa5.fsf@meyering.net>
Lines: 117
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 17 Jan 2011 17:08:32 -0000

P=E1draig Brady wrote:
> On 17/01/11 10:36, Jim Meyering wrote:
>>>From 7dc6335653afcdad9a3ffa327877571734644285 Mon Sep 17 00:00:00 2001
>> From: Jim Meyering <meyering@redhat.com>
>> Date: Mon, 17 Jan 2011 11:32:35 +0100
>> Subject: [PATCH] doc: show how to shred using a single zero-writing pass
>>
>> * doc/coreutils.texi (shred invocation): Give an example showing how
>> to invoke shred in its most basic (fastest) write-only-zeros mode.
>> ---
>>  doc/coreutils.texi |    9 +++++++++
>>  1 files changed, 9 insertions(+), 0 deletions(-)
>>
>> diff --git a/doc/coreutils.texi b/doc/coreutils.texi
>> index 9c3e2ed..8fb9f0c 100644
>> --- a/doc/coreutils.texi
>> +++ b/doc/coreutils.texi
>> @@ -8892,6 +8892,15 @@ shred invocation
>>  shred --verbose /dev/sda5
>>  @end example
>>
>> +On modern disks, a single pass that writes only zeros may be enough,
>> +and it will be much faster than the default.
>
> Well only 3 times, due to the disk being the bottleneck
> (since we changed to the fast internal PRNG by default).
> Also for security, writing random data would probably be more effective.
> So I'd reword the above sentence to:
>
> "To simply clear a disk"

Regarding "only 3x" I agree that 3x is better than the 25x difference
when comparing with the default from coreutils-7.0 and earlier.
However, when it takes two hours with "-n0 --zero", using
the default would add 4 hours.

Interestingly, the difference is sometimes far greater than the expected 3x:
(this is with F14/ext4 on an i7-960 and an 60GB OCZ vertex-2 SSD)

    $ seq 10000000 > k
    $ env time --f=3D%e shred -n0 --zero k
    0.36
    $ env time --f=3D%e shred k
    3.75

Surprisingly, even with just -n1, it's still nearly 3x:
(I expected this to have minimal performance difference with
the -n0 --zero run):

    $ env time --f=3D%e shred -n1 k
    0.95

Let's compare syscalls (the only difference should be PRNG-related)

$ strace -c shred -n1 k
% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
 96.34    0.052992       52992         1           fdatasync
  2.72    0.001496           0      6421         1 write
  0.94    0.000518         259         2           read
  0.00    0.000000           0         4           open
  0.00    0.000000           0         6           close
  0.00    0.000000           0         3           fstat
  0.00    0.000000           0         1           lseek
  0.00    0.000000           0         7           mmap
  0.00    0.000000           0         3           mprotect
  0.00    0.000000           0         1           munmap
  0.00    0.000000           0         4           brk
  0.00    0.000000           0         1         1 access
  0.00    0.000000           0         1           execve
  0.00    0.000000           0         4           fcntl
  0.00    0.000000           0         1           arch_prctl
------ ----------- ----------- --------- --------- ----------------
100.00    0.055006                  6460         2 total

$ strace -c shred -n0 --zero k
% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
 96.57    0.053991       53991         1           fdatasync
  3.43    0.001916           0      6421         1 write
  0.00    0.000000           0         2           read
  0.00    0.000000           0         4           open
  0.00    0.000000           0         6           close
  0.00    0.000000           0         3           fstat
  0.00    0.000000           0         1           lseek
  0.00    0.000000           0         7           mmap
  0.00    0.000000           0         3           mprotect
  0.00    0.000000           0         1           munmap
  0.00    0.000000           0         4           brk
  0.00    0.000000           0         1         1 access
  0.00    0.000000           0         1           execve
  0.00    0.000000           0         4           fcntl
  0.00    0.000000           0         1           arch_prctl
------ ----------- ----------- --------- --------- ----------------
100.00    0.055907                  6460         2 total

Odd... no difference in CPU time or syscall counts.

I wonder if the SSD is doing something special with blocks of all zeros,
which is reminiscent of The Reg's article:
http://www.theregister.co.uk/2011/01/14/ocz_and_ddrdrive_performance_row/

>> +Use a command like this to tell @command{shred} to skip all random
>> +passes and to perform only a final zero-writing pass:
>> +
>> +@example
>> +shred --verbose -n0 --zero /dev/sda5
>> +@end example
>
> It's probably not worth noting the equivalent:
> dd conv=3Dfdatasync bs=3D2M < /dev/zero > /dev/sda5

I like shred's --verbose %-completion indicator, though even with dd,
you might be able to get a similar progress indicator using
the phantom-progress.bash script here:

  http://blog.ksplice.com/2011/01/solving-problems-with-proc/


From MAILER-DAEMON Mon Jan 17 14:47:11 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1Pev2l-0001Ut-2Y
	for mharc-coreutils@gnu.org; Mon, 17 Jan 2011 14:47:11 -0500
Received: from [140.186.70.92] (port=60647 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PeuTN-0001fo-BE
	for coreutils@gnu.org; Mon, 17 Jan 2011 14:10:38 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <public@plane.gmane.org>) id 1PeuTL-0002TX-SK
	for coreutils@gnu.org; Mon, 17 Jan 2011 14:10:37 -0500
Received: from plane.gmane.org ([80.91.229.3]:33549)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <public@plane.gmane.org>) id 1PeuTL-0002Sw-Mh
	for coreutils@gnu.org; Mon, 17 Jan 2011 14:10:35 -0500
Received: from public by plane.gmane.org with local (Exim 4.69)
	(envelope-from <public@plane.gmane.org>) id 1PeuTH-0002iY-Dz
	for coreutils@gnu.org; Mon, 17 Jan 2011 20:10:31 +0100
Received: from mail-out.m-online.net ([212.18.0.9])
	by plane.gmane.org with esmtp (Exim 4.69)
	(envelope-from <whitebox@nefkom.net>)
	id 1PeuT8-0002bl-62; Mon, 17 Jan 2011 20:10:22 +0100
Received: from frontend1.mail.m-online.net (unknown [192.168.8.180])
	by mail-out.m-online.net (Postfix) with ESMTP id 7FED71C08E09;
	Mon, 17 Jan 2011 20:10:21 +0100 (CET)
Received: from localhost (dynscan1.mnet-online.de [192.168.8.164])
	by mail.m-online.net (Postfix) with ESMTP id 3BA2B1C00169;
	Mon, 17 Jan 2011 20:10:21 +0100 (CET)
X-Virus-Scanned: amavisd-new at mnet-online.de
Received: from mail.mnet-online.de ([192.168.8.180])
	by localhost (dynscan1.mail.m-online.net [192.168.8.164]) (amavisd-new,
	port 10024)
	with ESMTP id 50MBlVKQ2gyM; Mon, 17 Jan 2011 20:10:20 +0100 (CET)
Received: from igel.home (ppp-88-217-105-154.dynamic.mnet-online.de
	[88.217.105.154]) by mail.mnet-online.de (Postfix) with ESMTP;
	Mon, 17 Jan 2011 20:10:20 +0100 (CET)
Received: by igel.home (Postfix, from userid 501)
	id B642DCA2A0; Mon, 17 Jan 2011 20:10:19 +0100 (CET)
From: Andreas Schwab <schwab@linux-m68k.org>
To: Jim Meyering <public-jim-Oxw1nKZkIVjk1uMJSBkQmQ@plane.gmane.org>
References: <AANLkTikowrdL7D3FszcrZ_WVuFJntZko6CV7J_hUyBnL@mail.gmail.com>
	<4D341256.9000703@draigBrady.com> <87pqrvn4wh.fsf@meyering.net>
X-Yow: I've gotta GO, now!!  I wanta tell you you're a GREAT bunch of guys
	but you ought to CHANGE your UNDERWEAR more often!!
Date: Mon, 17 Jan 2011 20:10:18 +0100
In-Reply-To: <87pqrvn4wh.fsf@meyering.net> (Jim Meyering's message of "Mon, 17
	Jan 2011 12:30:54 +0100")
Message-ID: <m2fwsrcpnp.fsf@igel.home>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.2.92 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Mailman-Approved-At: Mon, 17 Jan 2011 14:47:10 -0500
Cc: public-coreutils-mXXj517/zsQ@plane.gmane.org,
	=?utf-8?Q?P=C3=A1draig?= Brady <P@draigBrady.com>,
	public-kerolasa-Re5JQEeQqe8AvxtiuMwx3w@plane.gmane.org,
	Sami Kerola <public-kerolasa-X3B1VOXEql0@plane.gmane.org>
Subject: [coreutils] Re: [PATCH] uniq: don't continue field processing after
	end	of line
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 17 Jan 2011 19:10:38 -0000



Jim Meyering <jim-Oxw1nKZkIVjk1uMJSBkQmQ@public.gmane.org> writes:

> +timeout 1 uniq -f 10000000000 in || fail=1

"The Cray-3 is so fast it can execute an infinite loop in under 2
seconds!"

Andreas. :-)

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."



From MAILER-DAEMON Tue Jan 18 03:36:08 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1Pf72u-0006oW-AP
	for mharc-coreutils@gnu.org; Tue, 18 Jan 2011 03:36:08 -0500
Received: from [140.186.70.92] (port=55102 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1Pf72s-0006oQ-AQ
	for coreutils@gnu.org; Tue, 18 Jan 2011 03:36:07 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Pf72q-00050N-Uq
	for coreutils@gnu.org; Tue, 18 Jan 2011 03:36:06 -0500
Received: from mx.meyering.net ([82.230.74.64]:56501)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Pf72q-00050F-Ma
	for coreutils@gnu.org; Tue, 18 Jan 2011 03:36:04 -0500
Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000)
	id 547266010F; Tue, 18 Jan 2011 09:36:03 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: =?iso-8859-1?Q?P=E1draig?= Brady <P@draigBrady.com>
Subject: Re: [coreutils] [PATCH] doc: show how to shred using a
	single	zero-writing pass
In-Reply-To: <871v4bmpa5.fsf@meyering.net> (Jim Meyering's message of "Mon, 17
	Jan 2011 18:08:18 +0100")
References: <87vd1nn7fi.fsf@meyering.net> <4D342751.7090403@draigBrady.com>
	<871v4bmpa5.fsf@meyering.net>
Date: Tue, 18 Jan 2011 09:36:03 +0100
Message-ID: <87fwsqhamk.fsf@meyering.net>
Lines: 49
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Tue, 18 Jan 2011 08:36:07 -0000

Jim Meyering wrote:
...
> Odd... no difference in CPU time or syscall counts.
>
> I wonder if the SSD is doing something special with blocks of all zeros,
> which is reminiscent of The Reg's article:
> http://www.theregister.co.uk/2011/01/14/ocz_and_ddrdrive_performance_row/

I've redone the doc change accordingly:

>From 0f9cf6b545aa07d552538735182007ae240d6071 Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering@redhat.com>
Date: Mon, 17 Jan 2011 11:32:35 +0100
Subject: [PATCH] doc: show how to shred more efficiently

* doc/coreutils.texi (shred invocation): Give an example showing how
to invoke shred in single-pass mode, and warn that -n0 --zero may
be inadequate.
---
 doc/coreutils.texi |   14 ++++++++++++++
 1 files changed, 14 insertions(+), 0 deletions(-)

diff --git a/doc/coreutils.texi b/doc/coreutils.texi
index 9c3e2ed..8a1b3b6 100644
--- a/doc/coreutils.texi
+++ b/doc/coreutils.texi
@@ -8892,6 +8892,20 @@ shred invocation
 shred --verbose /dev/sda5
 @end example

+On modern disks, a single pass should be adequate,
+and it will take one third the time of the default three-pass approach.
+
+@example
+# 1 pass, write pseudo-random data; 3x faster than the default
+shred --verbose -n1 /dev/sda5
+@end example
+
+To be on the safe side, use at least one pass that overwrites using
+pseudo-random data.  I.e., don't be tempted to use @samp{-n0 --zero},
+in case some disk controller optimizes the process of writing blocks
+of all zeros, and thereby does not clear all bytes in a block.
+Some SSDs may do just that.
+
 A @var{file} of @samp{-} denotes standard output.
 The intended use of this is to shred a removed temporary file.
 For example:
--
1.7.3.5


From MAILER-DAEMON Tue Jan 18 03:56:55 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1Pf7N1-00040g-Kz
	for mharc-coreutils@gnu.org; Tue, 18 Jan 2011 03:56:55 -0500
Received: from [140.186.70.92] (port=54732 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1Pf7Mw-0003u4-2m
	for coreutils@gnu.org; Tue, 18 Jan 2011 03:56:54 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Pf7Mh-00025m-Ht
	for coreutils@gnu.org; Tue, 18 Jan 2011 03:56:38 -0500
Received: from mx.meyering.net ([82.230.74.64]:44609)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Pf7Mh-00025d-Af
	for coreutils@gnu.org; Tue, 18 Jan 2011 03:56:35 -0500
Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000)
	id 056986010F; Tue, 18 Jan 2011 09:56:33 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: =?iso-8859-1?Q?P=E1draig?= Brady <P@draigBrady.com>
Subject: Re: [coreutils] [PATCH] uniq: don't continue field processing after
	end	of line
In-Reply-To: <87pqrvn4wh.fsf@meyering.net> (Jim Meyering's message of "Mon, 17
	Jan 2011 12:30:54 +0100")
References: <AANLkTikowrdL7D3FszcrZ_WVuFJntZko6CV7J_hUyBnL@mail.gmail.com>
	<4D341256.9000703@draigBrady.com> <87pqrvn4wh.fsf@meyering.net>
Date: Tue, 18 Jan 2011 09:56:33 +0100
Message-ID: <87aaiyh9oe.fsf@meyering.net>
Lines: 79
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
Cc: Sami Kerola <kerolasa@iki.fi>, coreutils@gnu.org, kerolasa@gmail.com
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Tue, 18 Jan 2011 08:56:54 -0000

Jim Meyering wrote:
> P=E1draig Brady wrote:
>
>> On 16/01/11 23:53, Sami Kerola wrote:
>>> Hi,
>>>
>>> I notice uniq -f 'insane_large_number' will make process to be busy
>>> long time without good reason. Attached patch should fix this symptom.
>>
>> I'd slightly amend that to the following,
>> to match the other limit checks in the function.
>>
>> diff --git a/src/uniq.c b/src/uniq.c
>> index 7bdbc4f..9c7e37c 100644
>> --- a/src/uniq.c
>> +++ b/src/uniq.c
>> @@ -214,7 +214,7 @@ find_field (struct linebuffer const *line)
>>    size_t size =3D line->length - 1;
>>    size_t i =3D 0;
>>
>> -  for (count =3D 0; count < skip_fields; count++)
>> +  for (count =3D 0; count < skip_fields && i < size; count++)
>>      {
>>        while (i < size && isblank (to_uchar (lp[i])))
>
> Thank you!
> I've also adjusted NEWS.
> Here's your adjusted patch, followed by another to add a test
> to exercise the bug/fix:
>
> Subject: [PATCH] uniq: don't continue field processing after end of line
>
> * NEWS (Bug fixes): Mention it.
> * src/uniq.c (find_field): Stop processing loop when end of line
> is reached.  Before this fix, 'uniq -f 10000000000 /etc/passwd'
> would run for a very long time.
> ---
>  NEWS       |    2 ++
>  src/uniq.c |    2 +-
>  2 files changed, 3 insertions(+), 1 deletions(-)
>
> diff --git a/NEWS b/NEWS
> index 9ccad63..3ec35c7 100644
> --- a/NEWS
> +++ b/NEWS
> @@ -13,6 +13,8 @@ GNU coreutils NEWS                                    -=
*- outline -*-
>    rm -f no longer fails for EINVAL or EILSEQ on file systems that
>    reject file names invalid for that file system.
>
> +  uniq -f NUM no longer tries to process fields after end of line.

I pushed that, then realized that we hadn't noted the origin
of the bug:

>From 34ece0b0a1e19f1faef8edfe3d59c5a2f5209369 Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering@redhat.com>
Date: Tue, 18 Jan 2011 09:45:00 +0100
Subject: [PATCH] doc: update NEWS

* NEWS: Note when the uniq bug was introduced.
It was mine, commit 1d9b3de9, "uniq: remove redundant test".
---
 NEWS |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/NEWS b/NEWS
index 3ec35c7..e420cd9 100644
--- a/NEWS
+++ b/NEWS
@@ -14,6 +14,7 @@ GNU coreutils NEWS                                    -*-=
 outline -*-
   reject file names invalid for that file system.

   uniq -f NUM no longer tries to process fields after end of line.
+  [bug introduced in coreutils-7.0]


 * Noteworthy changes in release 8.9 (2011-01-04) [stable]
--
1.7.3.5


From MAILER-DAEMON Tue Jan 18 06:10:54 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1Pf9Sg-0003zB-Bx
	for mharc-coreutils@gnu.org; Tue, 18 Jan 2011 06:10:54 -0500
Received: from [140.186.70.92] (port=51243 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1Pf9LH-0001gN-Fz
	for coreutils@gnu.org; Tue, 18 Jan 2011 06:03:17 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <crockabiscuit@yahoo.com>) id 1Pf9LF-0007DU-30
	for coreutils@gnu.org; Tue, 18 Jan 2011 06:03:15 -0500
Received: from nm7-vm0.bullet.mail.ac4.yahoo.com ([98.139.52.228]:39201)
	by eggs.gnu.org with smtp (Exim 4.71)
	(envelope-from <crockabiscuit@yahoo.com>) id 1Pf9LE-0007Cu-UD
	for coreutils@gnu.org; Tue, 18 Jan 2011 06:03:13 -0500
Received: from [98.139.52.193] by nm7.bullet.mail.ac4.yahoo.com with NNFMP;
	18 Jan 2011 11:03:11 -0000
Received: from [98.139.52.137] by tm6.bullet.mail.ac4.yahoo.com with NNFMP;
	18 Jan 2011 11:03:11 -0000
Received: from [127.0.0.1] by omp1020.mail.ac4.yahoo.com with NNFMP;
	18 Jan 2011 11:03:11 -0000
X-Yahoo-Newman-Property: ymail-3
X-Yahoo-Newman-Id: 672542.33829.bm@omp1020.mail.ac4.yahoo.com
Received: (qmail 12864 invoked by uid 60001); 18 Jan 2011 11:03:11 -0000
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024;
	t=1295348591; bh=aCpkmxYd2yDIkL35wFoMC6Kss9CDpBw9GkV/ILMc+TE=;
	h=Message-ID:X-YMail-OSG:Received:X-Mailer:References:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding;
	b=czbUr/Dzb0Nltwt78CMM+RN8GYG3cDjmiaj8WAm9HJOONbQIXywwwaS6S6VMw4RkRbNBA2kEzMY6YE8N2UWRgje6THjqrLknt1Q2q+nuL0aichVhOM3GpLd0YUB9fht3QZArL4wTDDx0H9vbGwh4FcLJ6BEEdgtxS62+ZjQftAI=
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com;
	h=Message-ID:X-YMail-OSG:Received:X-Mailer:References:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding;
	b=B4SigXNLZE5V/85CzhPhI5biMtYmnzmGUOrn/U6vFka+LyzaF/y2xmG4fKOd5SmDT5eG0uFQQYa4sOs6F6piAmNUTTC8EEcTeOBMGqZKWDLN346S+uIiU618+3OxVaixBoOpcorhGUjRZt6oPCF/W/bqRzRJ9zVvPuE1J7UUOvw=;
Message-ID: <300854.98908.qm@web120304.mail.ne1.yahoo.com>
X-YMail-OSG: OhtPgPgVM1napINF.FM_cToPfkZv9JC3QEDjl_TsflW4SOM
	C2ybrSafRMiiL.XkxuRMtzrF22aJyIkZqt2GuHiqmnCSfuOFIu6Lp99dGIDM
	7zCRxtU07BvjNNUSOknId1NL1rDxE3jRTosvbN011PWjkxo8nr0UVQzrM2o4
	mHS21AKBVW.rydNbdrvYLDAyO5DOzjRoxFbTz.tcFco9keAhVdn6vtPAgjW_
	JT9zKqdZemQ--
Received: from [180.68.85.212] by web120304.mail.ne1.yahoo.com via HTTP;
	Tue, 18 Jan 2011 03:03:11 PST
X-Mailer: YahooMailRC/555 YahooMailWebService/0.8.107.285259
References: <AANLkTikowrdL7D3FszcrZ_WVuFJntZko6CV7J_hUyBnL@mail.gmail.com>
	<4D341256.9000703@draigBrady.com> <87pqrvn4wh.fsf@meyering.net>
	<m2fwsrcpnp.fsf@igel.home>
Date: Tue, 18 Jan 2011 03:03:11 -0800 (PST)
From: crocket <crockabiscuit@yahoo.com>
Subject: Re: [coreutils] Re: [PATCH] uniq: don't continue field processing
	after end of line
To: Andreas Schwab <schwab@linux-m68k.org>
In-Reply-To: <m2fwsrcpnp.fsf@igel.home>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Tue, 18 Jan 2011 11:10:53 -0000

The below message didn't go to my "coreutils mailing list" folder in my mai=
lbox.=0AIt was because the mssage didn't have "coreutils@gnu.org" as one of=
 recipients.=0AIs there a way to put such messages in the correct folder?=
=0A=0A----- Original Message ----=0AFrom: Andreas Schwab <schwab@linux-m68k=
.org>=0ATo: Jim Meyering <public-jim-Oxw1nKZkIVjk1uMJSBkQmQ@plane.gmane.org=
>=0ACc: public-coreutils-mXXj517/zsQ@plane.gmane.org; P=E1draig Brady =0A<P=
@draigBrady.com>; public-kerolasa-Re5JQEeQqe8AvxtiuMwx3w@plane.gmane.org; S=
ami =0A=0AKerola <public-kerolasa-X3B1VOXEql0@plane.gmane.org>=0ASent: Tue,=
 January 18, 2011 4:10:18 AM=0ASubject: [coreutils] Re: [PATCH] uniq: don't=
 continue field processing after end =0A=0Aof line=0A=0A=0A=0AJim Meyering =
<jim-Oxw1nKZkIVjk1uMJSBkQmQ@public.gmane.org> writes:=0A=0A> +timeout 1 uni=
q -f 10000000000 in || fail=3D1=0A=0A"The Cray-3 is so fast it can execute =
an infinite loop in under 2=0Aseconds!"=0A=0AAndreas. :-)=0A=0A-- =0AAndrea=
s Schwab, schwab@linux-m68k.org=0AGPG Key fingerprint =3D 58CA 54C7 6D53 94=
2B 1756  01D3 44D5 214B 8276 4ED5=0A"And now for something completely diffe=
rent."=0A=0A=0A      


From MAILER-DAEMON Tue Jan 18 16:36:34 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PfJEA-0006oY-DY
	for mharc-coreutils@gnu.org; Tue, 18 Jan 2011 16:36:34 -0500
Received: from [140.186.70.92] (port=47586 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PfJE2-0006ks-4A
	for coreutils@gnu.org; Tue, 18 Jan 2011 16:36:33 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PfJDv-00048e-Uu
	for coreutils@gnu.org; Tue, 18 Jan 2011 16:36:25 -0500
Received: from mx.meyering.net ([82.230.74.64]:59293)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PfJDv-00047h-PO
	for coreutils@gnu.org; Tue, 18 Jan 2011 16:36:19 -0500
Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000)
	id 085F5600D5; Tue, 18 Jan 2011 22:36:17 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: coreutils@gnu.org
Date: Tue, 18 Jan 2011 22:36:17 +0100
Message-ID: <87hbd5hp2m.fsf@meyering.net>
Lines: 42
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
Subject: [coreutils] [PATCH] tests: avoid FP failure in new du test
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Tue, 18 Jan 2011 21:36:33 -0000

I noticed that the new du test would occasionally fail
because du would complete too quickly.
Making the tree about ten times larger solved the problem:
(5x was not enough; see comments below)

>From 7e72d4fe9da159d42efe0845b0f36e2ea9d7c3b1 Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering@redhat.com>
Date: Tue, 18 Jan 2011 22:32:33 +0100
Subject: [PATCH] tests: avoid FP failure in new du test

* tests/du/move-dir-while-traversing: Create a larger tree to
avoid a false-positive failure due to du terminating before
the rename is triggered.
---
 tests/du/move-dir-while-traversing |   11 +++++++++--
 1 files changed, 9 insertions(+), 2 deletions(-)

diff --git a/tests/du/move-dir-while-traversing b/tests/du/move-dir-while-traversing
index fe1615c..9605831 100755
--- a/tests/du/move-dir-while-traversing
+++ b/tests/du/move-dir-while-traversing
@@ -58,9 +58,16 @@ notifier.loop()
 EOF
 chmod a+x inotify-watch-for-dir-access.py

-long=d/e/f/g/h/i/j/k/l/m/n/o/p/q/r/s/t/u/v/w/x/y/z
 t=T/U
-mkdir -p $t/3/a/b/c/$long d2 || framework_failure
+mkdir d2 || framework_failure
+long=d/e/f/g/h/i/j/k/l/m/n/o/p/q/r/s/t/u/v/w/x/y/z
+# One iteration of this loop creates a tree with which
+# du sometimes completes its traversal before the above rename.
+# Even 5 iterations was not enough in 2 of 7 "make -j20 check" runs on a
+# 6/12-core system.  However, using "10", I saw no failure in 20 trials.
+for i in $(seq 10); do
+  mkdir -p $t/3/a/b/c/$i/$long || framework_failure
+done
 timeout 6 ./inotify-watch-for-dir-access.py $t/3/a/b > start-msg &

 # Wait for the watcher to start...
--
1.7.3.5


From MAILER-DAEMON Tue Jan 18 18:45:22 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PfLEo-0005jt-0X
	for mharc-coreutils@gnu.org; Tue, 18 Jan 2011 18:45:22 -0500
Received: from [140.186.70.92] (port=44703 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PfLEl-0005jf-Vf
	for coreutils@gnu.org; Tue, 18 Jan 2011 18:45:20 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <bob@proulx.com>) id 1PfLEg-00015o-GX
	for coreutils@gnu.org; Tue, 18 Jan 2011 18:45:19 -0500
Received: from joseki.proulx.com ([216.17.153.58]:56955)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <bob@proulx.com>) id 1PfLEg-00015Y-Az
	for coreutils@gnu.org; Tue, 18 Jan 2011 18:45:14 -0500
Received: from hysteria.proulx.com (hysteria.proulx.com [192.168.230.119])
	by joseki.proulx.com (Postfix) with ESMTP id 2285021363
	for <coreutils@gnu.org>; Tue, 18 Jan 2011 16:45:11 -0700 (MST)
Received: by hysteria.proulx.com (Postfix, from userid 1000)
	id C60D92DCC5; Tue, 18 Jan 2011 16:45:10 -0700 (MST)
Date: Tue, 18 Jan 2011 16:45:10 -0700
From: Bob Proulx <bob@proulx.com>
To: coreutils@gnu.org
Subject: Filing Emails From Mailing Lists (was: [coreutils] Re: [PATCH]
	uniq: don't continue field processing after end of line)
Message-ID: <20110118234510.GA32527@hysteria.proulx.com>
Mail-Followup-To: coreutils@gnu.org
References: <AANLkTikowrdL7D3FszcrZ_WVuFJntZko6CV7J_hUyBnL@mail.gmail.com>
	<4D341256.9000703@draigBrady.com> <87pqrvn4wh.fsf@meyering.net>
	<m2fwsrcpnp.fsf@igel.home>
	<300854.98908.qm@web120304.mail.ne1.yahoo.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <300854.98908.qm@web120304.mail.ne1.yahoo.com>
User-Agent: Mutt/1.5.20 (2009-06-14)
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 1)
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Tue, 18 Jan 2011 23:45:21 -0000

crocket wrote:
> The below message didn't go to my "coreutils mailing list" folder in
> my mailbox.
> It was because the mssage didn't have "coreutils@gnu.org" as one of
> recipients.

In general the mailing list address should always be in the recipient
list in either the To: or Cc: fields.  It isn't strictly required.
But it makes it harder for people to reply when that isn't true.

Filing based upon the recipient address is fraught with traps and
pitfalls.  Better to use the mailing list headers intended for that
purpose.  Here are the mailing list headers from that message.

  X-BeenThere: coreutils@gnu.org
  X-Mailman-Version: 2.1.5
  Precedence: list
  List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
  List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
          <mailto:coreutils-request@gnu.org?subject=unsubscribe>
  List-Archive: <http://lists.gnu.org/archive/html/coreutils>
  List-Post: <mailto:coreutils@gnu.org>
  List-Help: <mailto:coreutils-request@gnu.org?subject=help>
  List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
          <mailto:coreutils-request@gnu.org?subject=subscribe>
  Sender: coreutils-bounces+bob=example.com@gnu.org
  Errors-To: coreutils-bounces+bob=proulx.com@gnu.org

The best header to use is List-Id.  That is a required header for all
mailing list and is specified by RFC 2919.

  http://www.apps.ietf.org/rfc/rfc2919.html

> Is there a way to put such messages in the correct folder?

What tool are you using to put things into the correct folders?

I use procmail and the following recipe will automatically put all
email into folders by mailing list automatically.

:0
* ^List-Id: .*<[-a-zA-Z0-9]+\.gnu\.org>
* ^List-Id: .*<\/[-a-zA-Z0-9]+
$MAILDIR/Lists/gnu/$MATCH/

:0
* ^List-Id: .*<[-a-zA-Z0-9]+\.nongnu\.org>
* ^List-Id: .*<\/[-a-zA-Z0-9]+
$MAILDIR/Lists/nongnu/$MATCH/

Bob


From MAILER-DAEMON Wed Jan 19 02:36:48 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PfSb2-0008TM-T4
	for mharc-coreutils@gnu.org; Wed, 19 Jan 2011 02:36:48 -0500
Received: from [140.186.70.92] (port=60552 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PfSb1-0008TD-86
	for coreutils@gnu.org; Wed, 19 Jan 2011 02:36:48 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PfSb0-0004RJ-3E
	for coreutils@gnu.org; Wed, 19 Jan 2011 02:36:47 -0500
Received: from mx.meyering.net ([82.230.74.64]:35539)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PfSaz-0004Qx-Tq
	for coreutils@gnu.org; Wed, 19 Jan 2011 02:36:46 -0500
Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000)
	id 48EBD600E1; Wed, 19 Jan 2011 08:36:44 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: coreutils@gnu.org
Date: Wed, 19 Jan 2011 08:36:44 +0100
Message-ID: <871v49gx9v.fsf@meyering.net>
Lines: 42
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
Subject: [coreutils] [PATCH] maint: use slightly more efficient process in
	README-release
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Wed, 19 Jan 2011 07:36:48 -0000

This makes the release process more efficient on a multi-core system:

>From 948828b208e4cd59ceb903afbb46f55c09d51e18 Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering@redhat.com>
Date: Tue, 4 Jan 2011 12:47:24 +0100
Subject: [PATCH] maint: use slightly more efficient process in README-release

* README-release: Run cheaper root-only tests first.
Use half of processing units (not just 1) for the expensive tests.
---
 README-release |   16 +++++++++-------
 1 files changed, 9 insertions(+), 7 deletions(-)

diff --git a/README-release b/README-release
index b2bd34c..729ab89 100644
--- a/README-release
+++ b/README-release
@@ -29,13 +29,15 @@ FIXME: enable excluded programs like arch? to get their manual pages?
   Run the following on at least one SELinux-enabled (enforcing) and
   one non-SELinux system:

-    make distcheck
-    make -j1 check RUN_VERY_EXPENSIVE_TESTS=yes RUN_EXPENSIVE_TESTS=yes
-    sudo env PATH="$PATH" NON_ROOT_USERNAME=$USER make -k check-root
-
-  Note the -j1 above.  If you use -jN, for larger N, some of the expensive
-  tests are likely to interfere with concurrent performance-measuring or
-  timing-sensitive tests, resulting in spurious failures.
+    n=$(( ($(nproc) + 1) / 2 ))
+    sudo env PATH="$PATH" NON_ROOT_USERNAME=$USER make -k -j$(nproc) check-root\
+      && make distcheck \
+      && make -j$n check RUN_VERY_EXPENSIVE_TESTS=yes RUN_EXPENSIVE_TESTS=yes
+
+  Note that the use of -j$n tells make to use approximately half of the
+  available processing units.  If you use -jN, for larger N, some of the
+  expensive tests are likely to interfere with concurrent performance-measuring
+  or timing-sensitive tests, resulting in spurious failures.

   If "make distcheck" doesn't run "make syntax-check" for you, then run
   it manually:
--
1.7.3.5


From MAILER-DAEMON Thu Jan 20 04:12:29 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PfqZB-0005C9-5p
	for mharc-coreutils@gnu.org; Thu, 20 Jan 2011 04:12:29 -0500
Received: from [140.186.70.92] (port=56724 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PfqZ8-0004jR-TA
	for coreutils@gnu.org; Thu, 20 Jan 2011 04:12:27 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PfqXT-000632-2n
	for coreutils@gnu.org; Thu, 20 Jan 2011 04:10:44 -0500
Received: from mx.meyering.net ([82.230.74.64]:44908)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PfqXS-00062m-TL
	for coreutils@gnu.org; Thu, 20 Jan 2011 04:10:43 -0500
Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000)
	id 51495601A4; Thu, 20 Jan 2011 10:10:41 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: Rob Vermaas <rob.vermaas@gmail.com>
In-Reply-To: <AANLkTik=WO6P1C=BpNRhLb45EG3pGLNGCRRFCYE6SH0R@mail.gmail.com>
	(Rob Vermaas's message of "Thu, 20 Jan 2011 09:57:37 +0100")
References: <AANLkTik=WO6P1C=BpNRhLb45EG3pGLNGCRRFCYE6SH0R@mail.gmail.com>
Date: Thu, 20 Jan 2011 10:10:41 +0100
Message-ID: <87ei88os8e.fsf@meyering.net>
Lines: 54
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
Cc: coreutils@gnu.org
Subject: [coreutils] Re: gnulib/coreutils: Error on coreutils git submodule
	update
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 20 Jan 2011 09:12:27 -0000

Rob Vermaas wrote:
> Hi Jim,
>
> sorry for mailing you directly, but I'm having trouble getting the
> latest coreutils code. I get this error on 'git submodule update':
>
> fatal: reference is not a tree: 568779402439589832419c90e3f8193496a39613
> Unable to checkout '568779402439589832419c90e3f8193496a39613' in
> submodule path 'gnulib'
>
> This has to do with this commit:
> http://git.savannah.gnu.org/gitweb/?p=coreutils.git;a=blobdiff;f=gnulib;h=568779402439589832419c90e3f8193496a39613;hp=104aca8300d01a80889f70f41a4dce7e044f9476;hb=5c75d1da85d8da86e7a0083d4fbbd8d4de50305b;hpb=34ece0b0a1e19f1faef8edfe3d59c5a2f5209369
>
> Somehow the commit is not in the gnulib git repository. Any idea
> what's going on there?

Hi Rob,

Thanks for the heads-up.
That is indeed my mistake.

I ran "git syncsub" vs. a local gnulib repository
in which I had committed-but-not-pushed a change.

We now have a public-submodule-commit make target in the gnulib-provided
maint.mk, and it checks for just that condition.  However, it is currently
run only at release time.  This makes it apparent that we should check
this all the time...  At least via "make distcheck", and probably via a
server-side hook to prevent the offending "push" as well.

I've just updated (careful to use a public commit,
not the mountlist pair that is waiting for an ACK ;-)
and pushed the fix:

>From 634440238940bd4b216af62c88658073a84e69ec Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering@redhat.com>
Date: Thu, 20 Jan 2011 10:00:42 +0100
Subject: [PATCH] build: update gnulib submodule to latest

The previous gnulib submodule reference was to a non-public commit.
Reported by Rob Vermaas.
---
 gnulib |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/gnulib b/gnulib
index 5687794..680c7ff 160000
--- a/gnulib
+++ b/gnulib
@@ -1 +1 @@
-Subproject commit 568779402439589832419c90e3f8193496a39613
+Subproject commit 680c7ff005a2eb96b0903e5200cc28de273555c7
--
1.7.3.5


From MAILER-DAEMON Thu Jan 20 04:56:20 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PfrFc-0006Fe-Px
	for mharc-coreutils@gnu.org; Thu, 20 Jan 2011 04:56:20 -0500
Received: from [140.186.70.92] (port=39865 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PfrFa-0006FS-PZ
	for coreutils@gnu.org; Thu, 20 Jan 2011 04:56:19 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PfrFZ-0005ns-MR
	for coreutils@gnu.org; Thu, 20 Jan 2011 04:56:18 -0500
Received: from mx.meyering.net ([82.230.74.64]:57431)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PfrFZ-0005nm-FP
	for coreutils@gnu.org; Thu, 20 Jan 2011 04:56:17 -0500
Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000)
	id 6E5356032C; Thu, 20 Jan 2011 10:56:15 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: Rob Vermaas <rob.vermaas@gmail.com>
In-Reply-To: <AANLkTimGHUHHpuX48eUDuHG+=4Y9=Cc32ivbNfw3Y2ED@mail.gmail.com>
	(Rob Vermaas's message of "Thu, 20 Jan 2011 10:18:58 +0100")
References: <AANLkTik=WO6P1C=BpNRhLb45EG3pGLNGCRRFCYE6SH0R@mail.gmail.com>
	<87ei88os8e.fsf@meyering.net>
	<AANLkTimGHUHHpuX48eUDuHG+=4Y9=Cc32ivbNfw3Y2ED@mail.gmail.com>
Date: Thu, 20 Jan 2011 10:56:15 +0100
Message-ID: <874o93q4ow.fsf@meyering.net>
Lines: 54
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
Cc: coreutils@gnu.org
Subject: [coreutils] Re: gnulib/coreutils: Error on coreutils git submodule
	update
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 20 Jan 2011 09:56:19 -0000

Rob Vermaas wrote:

> Hi Jim,
>
>> -Subproject commit 568779402439589832419c90e3f8193496a39613
>> +Subproject commit 680c7ff005a2eb96b0903e5200cc28de273555c7
>
> Did you make the same mistake again? or is it me this time?
>
> fatal: reference is not a tree: 680c7ff005a2eb96b0903e5200cc28de273555c7
> Unable to checkout '680c7ff005a2eb96b0903e5200cc28de273555c7' in
> submodule path 'gnulib'

Hmm...

I was careful to put my local gnulib repo into the right state,
then ran "git syncsub" in coreutils, committed, and pushed.

However, I now see that is not always adequate:
"git syncsub" did not do what I expected, and I did
not rerun "make public-submodule-commit" to verify.
It still fails.

The problem is that the gnulib submodule under my coreutils
build directory had a stray commit, and that was causing each syncsub
to do a non-ff update and thus to create a merge commit.

Thanks for the quick feedback!

I've just confirmed that this does solve the problem,
and *then* pushed it.

>From e0c6272ac38a8eaba49b31adb4415e31159b6dd4 Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering@redhat.com>
Date: Thu, 20 Jan 2011 10:50:11 +0100
Subject: [PATCH] build: update gnulib submodule to latest

The previous gnulib submodule reference was *still* to a
non-public commit.  My submodule had a stray commit, so
the reference was always to a local merge commit.
Reported by Rob Vermaas.
---
 gnulib |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/gnulib b/gnulib
index 680c7ff..ff4bb04 160000
--- a/gnulib
+++ b/gnulib
@@ -1 +1 @@
-Subproject commit 680c7ff005a2eb96b0903e5200cc28de273555c7
+Subproject commit ff4bb04bec0c8d7834bfb0590d6de08fa0ceefd0
--
1.7.3.5


From MAILER-DAEMON Thu Jan 20 11:05:59 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1Pfx1L-0007Z5-MJ
	for mharc-coreutils@gnu.org; Thu, 20 Jan 2011 11:05:59 -0500
Received: from [140.186.70.92] (port=54491 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1Pfx1I-0007Wy-D1
	for coreutils@gnu.org; Thu, 20 Jan 2011 11:05:57 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1Pfx1H-0005M6-7R
	for coreutils@gnu.org; Thu, 20 Jan 2011 11:05:56 -0500
Received: from mx1.redhat.com ([209.132.183.28]:10387)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1Pfx1G-0005Lr-GW
	for coreutils@gnu.org; Thu, 20 Jan 2011 11:05:55 -0500
Received: from int-mx09.intmail.prod.int.phx2.redhat.com
	(int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22])
	by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id p0KG5mqH002897
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK);
	Thu, 20 Jan 2011 11:05:49 -0500
Received: from [10.3.113.109] (ovpn-113-109.phx2.redhat.com [10.3.113.109])
	by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP
	id p0KG5mpl003689; Thu, 20 Jan 2011 11:05:48 -0500
Message-ID: <4D385D5B.60607@redhat.com>
Date: Thu, 20 Jan 2011 09:05:47 -0700
From: Eric Blake <eblake@redhat.com>
Organization: Red Hat
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US;
	rv:1.9.2.13) Gecko/20101209 Fedora/3.1.7-0.35.b3pre.fc14
	Lightning/1.0b3pre Mnenhy/0.8.3 Thunderbird/3.1.7
MIME-Version: 1.0
To: Jim Meyering <jim@meyering.net>
Subject: Re: [coreutils] Re: gnulib/coreutils: Error on coreutils git submodule
	update
References: <AANLkTik=WO6P1C=BpNRhLb45EG3pGLNGCRRFCYE6SH0R@mail.gmail.com>	<87ei88os8e.fsf@meyering.net>	<AANLkTimGHUHHpuX48eUDuHG+=4Y9=Cc32ivbNfw3Y2ED@mail.gmail.com>
	<874o93q4ow.fsf@meyering.net>
In-Reply-To: <874o93q4ow.fsf@meyering.net>
X-Enigmail-Version: 1.1.2
OpenPGP: url=http://people.redhat.com/eblake/eblake.gpg
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature";
	boundary="------------enig2537166827DF78637B9415B4"
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
Cc: Rob Vermaas <rob.vermaas@gmail.com>, coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 20 Jan 2011 16:05:57 -0000

This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--------------enig2537166827DF78637B9415B4
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

On 01/20/2011 02:56 AM, Jim Meyering wrote:
> However, I now see that is not always adequate:
> "git syncsub" did not do what I expected, and I did
> not rerun "make public-submodule-commit" to verify.
> It still fails.

Can you configure branch.master.rebase =3D true in your gnulib submodule
to avoid creating merge commits?

Meanwhile, I've just modified my ~/.gitconfig to alter my syncsub alias
to make it more apparent to me if this situation happens:

[alias]
    syncsub =3D git submodule foreach git --ff-only origin master

Note the addition of --ff-only.

--=20
Eric Blake   eblake@redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org


--------------enig2537166827DF78637B9415B4
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Public key at http://people.redhat.com/eblake/eblake.gpg
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/

iQEcBAEBCAAGBQJNOF1bAAoJEKeha0olJ0Nq/iIH/RzBaJlRqwLtYEMH9fFJ6p9r
tQH4zIgCSryaODBcIQ84fdfGrujV6JqC12BGjWn9f5R8tncaVnPKQ9eo04bBRZyR
3H4Rgp0Mv8qghP+FtdRyluS7M1EqBRQweVMRUJcCLrVImpLkZ4a1a5xocGQNHmh5
FBFJ9uUc5Iy1gfOMOwc9vfuEfQUCJHFgvoL1Ar0on1yT1vtbc2bOth+3jZDb+W4J
bYv6LLaWF19lD9dgmWlVc+fQbo7qdaVkyfQoe2mx8LKxWu0w7oAs/VKfiwauEVQ7
acjppb3hbeBzup2rhncKnLtZS++wOPmIt1WrPECyzuE0QchVRe0q9ZC1Zpph4iE=
=TDe3
-----END PGP SIGNATURE-----

--------------enig2537166827DF78637B9415B4--


From MAILER-DAEMON Thu Jan 20 11:09:42 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1Pfx4w-0000x2-5G
	for mharc-coreutils@gnu.org; Thu, 20 Jan 2011 11:09:42 -0500
Received: from [140.186.70.92] (port=53400 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1Pfx4t-0000vF-4U
	for coreutils@gnu.org; Thu, 20 Jan 2011 11:09:41 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Pfx4s-0005qV-5P
	for coreutils@gnu.org; Thu, 20 Jan 2011 11:09:39 -0500
Received: from mx.meyering.net ([82.230.74.64]:34489)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Pfx4r-0005qA-Vz
	for coreutils@gnu.org; Thu, 20 Jan 2011 11:09:38 -0500
Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000)
	id 445DD604F0; Thu, 20 Jan 2011 17:09:36 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: Eric Blake <eblake@redhat.com>
Subject: Re: [coreutils] Re: gnulib/coreutils: Error on coreutils git
	submodule update
In-Reply-To: <4D385D5B.60607@redhat.com> (Eric Blake's message of "Thu, 20 Jan
	2011 09:05:47 -0700")
References: <AANLkTik=WO6P1C=BpNRhLb45EG3pGLNGCRRFCYE6SH0R@mail.gmail.com>
	<87ei88os8e.fsf@meyering.net>
	<AANLkTimGHUHHpuX48eUDuHG+=4Y9=Cc32ivbNfw3Y2ED@mail.gmail.com>
	<874o93q4ow.fsf@meyering.net> <4D385D5B.60607@redhat.com>
Date: Thu, 20 Jan 2011 17:09:36 +0100
Message-ID: <87vd1jlfpb.fsf@meyering.net>
Lines: 25
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
Cc: Rob Vermaas <rob.vermaas@gmail.com>, coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 20 Jan 2011 16:09:41 -0000

Eric Blake wrote:

> On 01/20/2011 02:56 AM, Jim Meyering wrote:
>> However, I now see that is not always adequate:
>> "git syncsub" did not do what I expected, and I did
>> not rerun "make public-submodule-commit" to verify.
>> It still fails.
>
> Can you configure branch.master.rebase = true in your gnulib submodule
> to avoid creating merge commits?

Thanks.
I'll try that.

> Meanwhile, I've just modified my ~/.gitconfig to alter my syncsub alias
> to make it more apparent to me if this situation happens:
>
> [alias]
>     syncsub = git submodule foreach git --ff-only origin master
>
> Note the addition of --ff-only.

You must mean this (i.e., don't remove the "pull"):

    syncsub = git submodule foreach git pull --ff-only origin master


From MAILER-DAEMON Thu Jan 20 11:13:02 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1Pfx8A-00022W-EJ
	for mharc-coreutils@gnu.org; Thu, 20 Jan 2011 11:13:02 -0500
Received: from [140.186.70.92] (port=54575 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1Pfx87-00020J-H1
	for coreutils@gnu.org; Thu, 20 Jan 2011 11:13:00 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1Pfx86-0006Pm-6l
	for coreutils@gnu.org; Thu, 20 Jan 2011 11:12:59 -0500
Received: from mx1.redhat.com ([209.132.183.28]:58805)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1Pfx85-0006Pg-Tx
	for coreutils@gnu.org; Thu, 20 Jan 2011 11:12:58 -0500
Received: from int-mx12.intmail.prod.int.phx2.redhat.com
	(int-mx12.intmail.prod.int.phx2.redhat.com [10.5.11.25])
	by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id p0KGCuPD004609
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK);
	Thu, 20 Jan 2011 11:12:56 -0500
Received: from [10.3.113.109] (ovpn-113-109.phx2.redhat.com [10.3.113.109])
	by int-mx12.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP
	id p0KGCte4001773; Thu, 20 Jan 2011 11:12:55 -0500
Message-ID: <4D385F06.7040501@redhat.com>
Date: Thu, 20 Jan 2011 09:12:54 -0700
From: Eric Blake <eblake@redhat.com>
Organization: Red Hat
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US;
	rv:1.9.2.13) Gecko/20101209 Fedora/3.1.7-0.35.b3pre.fc14
	Lightning/1.0b3pre Mnenhy/0.8.3 Thunderbird/3.1.7
MIME-Version: 1.0
To: Jim Meyering <jim@meyering.net>
Subject: Re: [coreutils] Re: gnulib/coreutils: Error on coreutils git submodule
	update
References: <AANLkTik=WO6P1C=BpNRhLb45EG3pGLNGCRRFCYE6SH0R@mail.gmail.com>	<87ei88os8e.fsf@meyering.net>	<AANLkTimGHUHHpuX48eUDuHG+=4Y9=Cc32ivbNfw3Y2ED@mail.gmail.com>	<874o93q4ow.fsf@meyering.net>
	<4D385D5B.60607@redhat.com> <87vd1jlfpb.fsf@meyering.net>
In-Reply-To: <87vd1jlfpb.fsf@meyering.net>
X-Enigmail-Version: 1.1.2
OpenPGP: url=http://people.redhat.com/eblake/eblake.gpg
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature";
	boundary="------------enigF0264EAFDF48D139C06A34D4"
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.25
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
Cc: Rob Vermaas <rob.vermaas@gmail.com>, coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 20 Jan 2011 16:13:00 -0000

This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--------------enigF0264EAFDF48D139C06A34D4
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

On 01/20/2011 09:09 AM, Jim Meyering wrote:
>> Note the addition of --ff-only.
>=20
> You must mean this (i.e., don't remove the "pull"):
>=20
>     syncsub =3D git submodule foreach git pull --ff-only origin master

Yep (that's what I get for retyping it instead of doing copy-and-paste)

--=20
Eric Blake   eblake@redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org


--------------enigF0264EAFDF48D139C06A34D4
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Public key at http://people.redhat.com/eblake/eblake.gpg
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/

iQEcBAEBCAAGBQJNOF8GAAoJEKeha0olJ0NqvAAH/jrhxX5h3XD6tx4ChPTGTze8
JmC/yU9AQnWDrlkzKnSlXcgrybqfzzb7w1U2SZeVP9RTLjFKlJz6ZSPNf2xaskWL
l9vlJbF/4c+V9q/SKOJWlsSlw06b2sYMyq7Y5+Jc9QGJnvdnetOm1ANNjQNSzDey
LilDjoZN4eUl9R7wBSpZSFzkQs6NZ2qGNgeLWOKkLsvDB7j5zN9Bc3G6yPJgYZn7
ZppECQ/hU4rixVNfhJRJs3vn+3GGzDI2+LdbFZgPixIi4aYLsKSdf++fyX/RuG2D
sEzJqktiTvKKrtFLm2LbdVLBbJtlE7SeAZuSaqTALAFNQPASRlrD0ZwhT9jC/X0=
=p50u
-----END PGP SIGNATURE-----

--------------enigF0264EAFDF48D139C06A34D4--


From MAILER-DAEMON Mon Jan 24 04:05:57 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PhIN3-0000Pj-I7
	for mharc-coreutils@gnu.org; Mon, 24 Jan 2011 04:05:57 -0500
Received: from [140.186.70.92] (port=38519 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PhIN1-0000OM-98
	for coreutils@gnu.org; Mon, 24 Jan 2011 04:05:56 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PhIMz-0007Tk-GM
	for coreutils@gnu.org; Mon, 24 Jan 2011 04:05:54 -0500
Received: from mx.meyering.net ([82.230.74.64]:45651)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PhIMz-0007TT-AZ
	for coreutils@gnu.org; Mon, 24 Jan 2011 04:05:53 -0500
Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000)
	id C6E3C6016F; Mon, 24 Jan 2011 10:05:51 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: coreutils@gnu.org
Date: Mon, 24 Jan 2011 10:05:51 +0100
Message-ID: <87ei82zn68.fsf@meyering.net>
Lines: 77
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
Subject: [coreutils] two patches required for fedora rawhide
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 24 Jan 2011 09:05:56 -0000

I built on a just-updated rawhide system and encountered two problems.
First, split got a new warning about a used-uninitialized "files" variable.

  split.c: In function 'main':
  split.c:663:9: error: 'files' may be used uninitialized in this function \
    [-Werror=uninitialized]

That's spurious, so I added an IF_LINT initializer.

The next was a test failure due to fstatfs now failing with ENOSYS
(presumably due to the fact this is with linux-2.3.38 RC)
when applied to an input file descriptor that's open on a pipe.
This failure caused a diagnostic that made a test fail.
Since the fstatfs call is solely to determine whether the input
file is remote (not appropriate for inotify), we can treat this
particular failure (ENOSYS) just like the absence of inotify support.

>From 15ea577af7f04c0e9ddbd88fa72768e79a933b60 Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering@redhat.com>
Date: Mon, 24 Jan 2011 09:38:40 +0100
Subject: [PATCH 2/2] split: avoid a new, spurious warning from gcc-4.6.0

* src/split.c (lines_rr) [IF_LINT]: Initialize files, now that
rawhide's gcc-4.6.0 would otherwise warn about use-uninitialized.
---
 src/split.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/src/split.c b/src/split.c
index c11c032..364576a 100644
--- a/src/split.c
+++ b/src/split.c
@@ -660,7 +660,7 @@ lines_rr (uintmax_t k, uintmax_t n, char *buf, size_t bufsize)
 {
   bool file_limit;
   size_t i_file;
-  of_t *files;
+  of_t *files IF_LINT (= NULL);
   uintmax_t line_no;

   if (k)
--
1.7.3.5.38.gb312b

>From 8264fc615a1beb82e062949e5cf17c852fcfaaaf Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering@redhat.com>
Date: Mon, 24 Jan 2011 09:37:10 +0100
Subject: [PATCH 1/2] tail: avoid new diagnostic when applying -f to a pipe on linux-2.3.38

* src/tail.c (fremote): Do not print a diagnostic when
fstatfs (pipe_FD, &buf) fails, as it now does on linux-2.3.38.
This avoids the spurious failure of tests/misc/tail's f-pipe-1
test, when running in input-from-pipe mode.
---
 src/tail.c |    7 +++++--
 1 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/src/tail.c b/src/tail.c
index 9682a53..bcd2d99 100644
--- a/src/tail.c
+++ b/src/tail.c
@@ -887,8 +887,11 @@ fremote (int fd, const char *name)
   int err = fstatfs (fd, &buf);
   if (err != 0)
     {
-      error (0, errno, _("cannot determine location of %s. "
-                         "reverting to polling"), quote (name));
+      /* On at least linux-2.6.38, fstatfs fails with ENOSYS when FD
+         is open on a pipe.  Treat that like a remote file.  */
+      if (errno != ENOSYS)
+        error (0, errno, _("cannot determine location of %s. "
+                           "reverting to polling"), quote (name));
     }
   else
     {
--
1.7.3.5.38.gb312b


From MAILER-DAEMON Mon Jan 24 13:31:56 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PhRCm-00043S-2b
	for mharc-coreutils@gnu.org; Mon, 24 Jan 2011 13:31:56 -0500
Received: from [140.186.70.92] (port=42806 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PhRCj-00042N-TG
	for coreutils@gnu.org; Mon, 24 Jan 2011 13:31:54 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1PhRCh-0008Bi-Kt
	for coreutils@gnu.org; Mon, 24 Jan 2011 13:31:53 -0500
Received: from qmta13.westchester.pa.mail.comcast.net ([76.96.59.243]:34643)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1PhRCh-0008BU-IP
	for coreutils@gnu.org; Mon, 24 Jan 2011 13:31:51 -0500
Received: from omta06.westchester.pa.mail.comcast.net ([76.96.62.51])
	by qmta13.westchester.pa.mail.comcast.net with comcast
	id zWVd1f00D16LCl05DWXrvh; Mon, 24 Jan 2011 18:31:51 +0000
Received: from office.redhat.com ([24.10.251.25])
	by omta06.westchester.pa.mail.comcast.net with comcast
	id zWXn1f0030ZdyUg3SWXr0K; Mon, 24 Jan 2011 18:31:51 +0000
From: Eric Blake <eblake@redhat.com>
To: coreutils@gnu.org
Date: Mon, 24 Jan 2011 11:31:46 -0700
Message-Id: <1295893906-13522-1-git-send-email-eblake@redhat.com>
X-Mailer: git-send-email 1.7.3.4
In-Reply-To: <4D3DA293.90202@redhat.com>
References: <4D3DA293.90202@redhat.com>
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
Cc: bug-gnulib@gnu.org
Subject: [coreutils] [PATCH] build: speed up configure for releases
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 24 Jan 2011 18:31:55 -0000

* gnulib: Update to latest, for posixcheck improvement.
* configure.ac (gl_ASSERT_NO_GNULIB_POSIXCHECK): Conditionally
define, according to whether git-version-gen output included '-'.
---

What do you think of this patch?

It speeds up ./configure of a release tarball (no time at all
spent on any HAVE_RAW_DECL_* checks), while defaulting to
leaving that in during development.  It even makes the resulting
configure file smaller for the tarball!

About the only other thing I could think of is adding a new
./configure option:

./configure --enable-posix-check

and a shell test that skips the body of gl_WARN_ON_USE_PREPARE
if that option is not in effect, but which (unfortunately) has
to keep the full bulk of the configure tests around, even though
they are not executed without the configure option.  In other
words, a configure option is not the same as an m4 option, which
is why this patch goes for an m4 option, but automates it based
on whether the version indicates an intra-release devel version.

Hmm, it may still make sense to do both things - make it possible
to completely elide HAVE_RAW_DECL_* (this patch) for configure
built for a tarball, as well as to provide a configure option
(rather than the current ad-hoc approach of
make CFLAGS=-DGNULIB_POSIXCHECK=1) to make it more apparent when
a developer is explicitly building for posix check purposes
and to thus speed up builds when the option is not in effect.

 configure.ac |    6 ++++++
 gnulib       |    2 +-
 2 files changed, 7 insertions(+), 1 deletions(-)

diff --git a/configure.ac b/configure.ac
index bc9933d..4e781b7 100644
--- a/configure.ac
+++ b/configure.ac
@@ -35,6 +35,12 @@ AC_CONFIG_HEADERS([lib/config.h:lib/config.hin])
 AM_INIT_AUTOMAKE([1.11.1 dist-xz color-tests parallel-tests])
 AM_SILENT_RULES([yes]) # make --enable-silent-rules the default.

+dnl Allow the use of GNULIB_POSIXCHECK during development, but avoid
+dnl the configure time spent on otherwise unused HAVE_RAW_DECL_*
+dnl in released builds.
+m4_if(m4_index(m4_defn([AC_PACKAGE_VERSION]), [-]), [-1],
+  [gl_ASSERT_NO_GNULIB_POSIXCHECK])
+
 AC_PROG_CC_STDC
 AM_PROG_CC_C_O
 AC_PROG_CPP
diff --git a/gnulib b/gnulib
index ff4bb04..5e5009f 160000
--- a/gnulib
+++ b/gnulib
@@ -1 +1 @@
-Subproject commit ff4bb04bec0c8d7834bfb0590d6de08fa0ceefd0
+Subproject commit 5e5009f839875220d5dec7abdf5cf53d919f4d42
-- 
1.7.3.4



From MAILER-DAEMON Mon Jan 24 15:39:45 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PhTCT-0004JN-N3
	for mharc-coreutils@gnu.org; Mon, 24 Jan 2011 15:39:45 -0500
Received: from [140.186.70.92] (port=42003 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PhTCR-0004Gh-AJ
	for coreutils@gnu.org; Mon, 24 Jan 2011 15:39:44 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eggert@cs.ucla.edu>) id 1PhTCQ-0001FR-06
	for coreutils@gnu.org; Mon, 24 Jan 2011 15:39:43 -0500
Received: from smtp.cs.ucla.edu ([131.179.128.62]:48923)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eggert@cs.ucla.edu>)
	id 1PhTCJ-0001Cu-DD; Mon, 24 Jan 2011 15:39:35 -0500
Received: from localhost (localhost.localdomain [127.0.0.1])
	by smtp.cs.ucla.edu (Postfix) with ESMTP id DC3FD39E80E0;
	Mon, 24 Jan 2011 12:39:31 -0800 (PST)
X-Virus-Scanned: amavisd-new at smtp.cs.ucla.edu
Received: from smtp.cs.ucla.edu ([127.0.0.1])
	by localhost (smtp.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10024)
	with ESMTP id iGWfgDR3HWLn; Mon, 24 Jan 2011 12:39:30 -0800 (PST)
Received: from [131.179.64.200] (Penguin.CS.UCLA.EDU [131.179.64.200])
	by smtp.cs.ucla.edu (Postfix) with ESMTPSA id 27F4739E8083;
	Mon, 24 Jan 2011 12:39:30 -0800 (PST)
Message-ID: <4D3DE37B.8040702@cs.ucla.edu>
Date: Mon, 24 Jan 2011 12:39:23 -0800
From: Paul Eggert <eggert@cs.ucla.edu>
Organization: UCLA Computer Science Department
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US;
	rv:1.9.2.13) Gecko/20101208 Thunderbird/3.1.7
MIME-Version: 1.0
To: Eric Blake <eblake@redhat.com>
References: <4D3DA293.90202@redhat.com>
	<1295893906-13522-1-git-send-email-eblake@redhat.com>
In-Reply-To: <1295893906-13522-1-git-send-email-eblake@redhat.com>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
Cc: bug-gnulib@gnu.org, coreutils@gnu.org
Subject: [coreutils] Re: [PATCH] build: speed up configure for releases
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 24 Jan 2011 20:39:44 -0000

On 01/24/11 10:31, Eric Blake wrote:
> Hmm, it may still make sense to do both things - make it possible
> to completely elide HAVE_RAW_DECL_* (this patch) for configure
> built for a tarball, as well as to provide a configure option
> (rather than the current ad-hoc approach of
> make CFLAGS=-DGNULIB_POSIXCHECK=1) to make it more apparent when
> a developer is explicitly building for posix check purposes
> and to thus speed up builds when the option is not in effect.

That sort of thing sounds useful, yes.  I have another
idea or two which you may want to throw into the pot.

When I used gl_ASSERT_NO_GNULIB_POSIXCHECK in Emacs (see
<http://git.savannah.gnu.org/cgit/emacs.git/commit/?id=8d38eb490bdd94432de803575ac3663cef360285>)
aclocal helpfully informed me that m4/warn-on-use.m4 was unnecessary,
so I added a line "rm m4/warn-on-use.m4" to the Make rule
for "make sync-from-gnulib".  This is a bit ad-hoc and it'd
be nice if gnulib-tool would automate it.

More generally, it'd be nice if there were a package
"no-gnulib-posixcheck" that would do all this for me.
Or, perhaps even better, a package "gnulib-posixcheck" that would
enable the HAVE_RAW_DECL_* stuff, with the default
being to disable it.  The latter might be easier because
then its Files: section could list m4/warn-on-use.m4,
and we could remove m4/warn-on-use.m4 from the warn-on-use
module, and remove the need for gl_ASSERT_NO_GNULIB_POSIXCHECK.


From MAILER-DAEMON Mon Jan 24 19:52:09 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PhX8j-0005gf-Fv
	for mharc-coreutils@gnu.org; Mon, 24 Jan 2011 19:52:09 -0500
Received: from [140.186.70.92] (port=32864 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PhX8h-0005fH-0C
	for coreutils@gnu.org; Mon, 24 Jan 2011 19:52:07 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <bruno@clisp.org>) id 1PhX8f-0007Ac-Uk
	for coreutils@gnu.org; Mon, 24 Jan 2011 19:52:06 -0500
Received: from mo-p00-ob.rzone.de ([81.169.146.161]:8449)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <bruno@clisp.org>)
	id 1PhX8c-00079R-Kb; Mon, 24 Jan 2011 19:52:03 -0500
DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; t=1295916721; l=1857;
	s=domk; d=haible.de;
	h=Content-Transfer-Encoding:Content-Type:MIME-Version:In-Reply-To:
	References:Cc:Date:Subject:To:From:X-RZG-CLASS-ID:X-RZG-AUTH;
	bh=k+Z83OVU75p6UV+wEltWyHPQBF0=;
	b=BefqDc5rudj9i6zkiYKu+VZ1AIlcoja2Gn50L7vdHT1GFp6qpIBY+D2GJ6hUAd8aUez
	g2oJ/Z4cijwN6EjaNR9YVke1ggu2EOdyZ7mvUugBuqvkLQNMvlF4Kozor7Ihx1DFgHZly
	YKEMwM6vnyUuoE1v9RPqk8MHTlxnGovYYCs=
X-RZG-AUTH: :Ln4Re0+Ic/6oZXR1YgKryK8brksyK8dozXDwHXjf9hj/zDNRbfA44+iwyQ==
X-RZG-CLASS-ID: mo00
Received: from linuix.haible.de
	(dslb-088-068-046-137.pools.arcor-ip.net [88.68.46.137])
	by post.strato.de (fruni mo53) (RZmta 24.12)
	with ESMTP id 206bdbn0OLNc8k ; Tue, 25 Jan 2011 01:51:50 +0100 (MET)
From: Bruno Haible <bruno@clisp.org>
To: bug-gnulib@gnu.org
Date: Tue, 25 Jan 2011 01:51:48 +0100
User-Agent: KMail/1.9.9
References: <4D3DA293.90202@redhat.com>
	<1295893906-13522-1-git-send-email-eblake@redhat.com>
In-Reply-To: <1295893906-13522-1-git-send-email-eblake@redhat.com>
MIME-Version: 1.0
Content-Type: text/plain;
  charset="utf-8"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
Message-Id: <201101250151.49698.bruno@clisp.org>
X-detected-operating-system: by eggs.gnu.org: Solaris 10 (beta)
Cc: coreutils@gnu.org
Subject: [coreutils] Re: [PATCH] build: speed up configure for releases
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Tue, 25 Jan 2011 00:52:08 -0000

Hi Eric,

> What do you think of this patch?
> 
> It speeds up ./configure of a release tarball (no time at all
> spent on any HAVE_RAW_DECL_* checks), while defaulting to
> leaving that in during development.

I think it's a bad idea to make release tarballs work differently than
the development environment and intermediate snapshots. For three reasons:

1) Freedom to modify a package. It is an excellent property of GNU software
   that anyone can take a released tarball, make a small modification, do
   "./configure; make; make dist", and get a tarball that works like the
   original one, except for precisely the intended modification.

   If people cannot do that any more, if people cannot tinker with packages
   by themselves any more, then where is the freedom to modify gone?

2) For newbies to become contributors to a package easily, it's important
   that the perspective of a tarball consumer and the perspective of a
   developer are not too different. We have a README-HACKING which
   explains a lot of the practical knowledge. But that does not mean that
   adding more differences between these two perspectives is welcome.

3) Testability of the release process. If release tarballs are built
   differently than snapshot tarballs, by Murphy's law, you will at some
   point have a bug in the release tarball that was absent from all
   snapshot tarballs.

> About the only other thing I could think of is adding a new
> ./configure option:
> 
> ./configure --enable-posix-check

This is acceptable. But OTOH, why not just let the maintainer add
these two lines to configure.ac:

  # Comment this out if you want to use -DGNULIB_POSIXCHECK.
  gl_ASSERT_NO_GNULIB_POSIXCHECK

If it's rare enough for the maintainer to use -DGNULIB_POSIXCHECK,
he can tweak his configure.ac file and rerun configure and make.

Bruno


From MAILER-DAEMON Tue Jan 25 06:42:58 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PhhIY-0001fG-PO
	for mharc-coreutils@gnu.org; Tue, 25 Jan 2011 06:42:58 -0500
Received: from [140.186.70.92] (port=59609 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PhhIW-0001d0-46
	for coreutils@gnu.org; Tue, 25 Jan 2011 06:42:57 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PhhIS-0007D5-Re
	for coreutils@gnu.org; Tue, 25 Jan 2011 06:42:55 -0500
Received: from mx.meyering.net ([82.230.74.64]:32968)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PhhIS-0007Cf-K9
	for coreutils@gnu.org; Tue, 25 Jan 2011 06:42:52 -0500
Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000)
	id 33B97600E1; Tue, 25 Jan 2011 12:42:51 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: coreutils@gnu.org
Date: Tue, 25 Jan 2011 12:42:51 +0100
Message-ID: <877hdtus3o.fsf@meyering.net>
Lines: 70
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
Subject: [coreutils] [PATCH 1/2] tests: avoid rare FP failure in new du test
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Tue, 25 Jan 2011 11:42:57 -0000

I saw this test fail again.
Tripling the size of the tree seems to be enough...

While waiting for 200 trials to complete I suspended the job,
and upon resumption, one iteration failed.  The second patch
prevents that sort of problem by making the test script ignore
TSTP and STOP signals.

>From 5c501f6fba5c1e2c4b45ce3f44b2da58d559ebe7 Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering@redhat.com>
Date: Tue, 25 Jan 2011 11:09:27 +0100
Subject: [PATCH 1/2] tests: avoid rare FP failure in new du test

* tests/du/move-dir-while-traversing: Create an even larger tree
to avoid a false-positive failure due to du terminating before
the rename is triggered.
---
 tests/du/move-dir-while-traversing |    6 ++++--
 1 files changed, 4 insertions(+), 2 deletions(-)

diff --git a/tests/du/move-dir-while-traversing b/tests/du/move-dir-while-traversing
index 9605831..c711e02 100755
--- a/tests/du/move-dir-while-traversing
+++ b/tests/du/move-dir-while-traversing
@@ -63,9 +63,11 @@ mkdir d2 || framework_failure
 long=d/e/f/g/h/i/j/k/l/m/n/o/p/q/r/s/t/u/v/w/x/y/z
 # One iteration of this loop creates a tree with which
 # du sometimes completes its traversal before the above rename.
-# Even 5 iterations was not enough in 2 of 7 "make -j20 check" runs on a
+# Five iterations was not enough in 2 of 7 "make -j20 check" runs on a
 # 6/12-core system.  However, using "10", I saw no failure in 20 trials.
-for i in $(seq 10); do
+# Using 10 iterations was not enough, either.
+# Using 30, I saw no failure in 200 trials.
+for i in $(seq 30); do
   mkdir -p $t/3/a/b/c/$i/$long || framework_failure
 done
 timeout 6 ./inotify-watch-for-dir-access.py $t/3/a/b > start-msg &
--
1.7.3.5.38.gb312b


>From 56699864f63fe69259d4a5c1f94498762d8df78d Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering@redhat.com>
Date: Tue, 25 Jan 2011 12:36:22 +0100
Subject: [PATCH 2/2] tests: avoid FP failure due to suspension

* tests/du/move-dir-while-traversing: Prohibit suspension,
to avoid false-positive failure.
---
 tests/du/move-dir-while-traversing |    4 ++++
 1 files changed, 4 insertions(+), 0 deletions(-)

diff --git a/tests/du/move-dir-while-traversing b/tests/du/move-dir-while-traversing
index c711e02..2a60d6e 100755
--- a/tests/du/move-dir-while-traversing
+++ b/tests/du/move-dir-while-traversing
@@ -70,6 +70,10 @@ long=d/e/f/g/h/i/j/k/l/m/n/o/p/q/r/s/t/u/v/w/x/y/z
 for i in $(seq 30); do
   mkdir -p $t/3/a/b/c/$i/$long || framework_failure
 done
+
+# Prohibit suspension, which could otherwise cause a timeout-induced FP failure.
+trap '' STOP TSTP
+
 timeout 6 ./inotify-watch-for-dir-access.py $t/3/a/b > start-msg &

 # Wait for the watcher to start...
--
1.7.3.5.38.gb312b


From MAILER-DAEMON Tue Jan 25 12:19:30 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PhmYE-0003Io-Hc
	for mharc-coreutils@gnu.org; Tue, 25 Jan 2011 12:19:30 -0500
Received: from [140.186.70.92] (port=38890 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PhmXk-00034O-0F
	for coreutils@gnu.org; Tue, 25 Jan 2011 12:19:01 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <public@plane.gmane.org>) id 1PhmXi-0005HC-EL
	for coreutils@gnu.org; Tue, 25 Jan 2011 12:18:59 -0500
Received: from plane.gmane.org ([80.91.229.3]:44228)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <public@plane.gmane.org>) id 1PhmXi-0005Fn-8T
	for coreutils@gnu.org; Tue, 25 Jan 2011 12:18:58 -0500
Received: from public by plane.gmane.org with local (Exim 4.69)
	(envelope-from <public@plane.gmane.org>) id 1PhmXf-0002mv-4K
	for coreutils@gnu.org; Tue, 25 Jan 2011 18:18:55 +0100
Received: from mail-out.m-online.net ([212.18.0.9])
	by plane.gmane.org with esmtp (Exim 4.69)
	(envelope-from <whitebox@nefkom.net>)
	id 1PhmXW-0002dz-81; Tue, 25 Jan 2011 18:18:46 +0100
Received: from frontend1.mail.m-online.net (unknown [192.168.8.180])
	by mail-out.m-online.net (Postfix) with ESMTP id D14401C01EF5;
	Tue, 25 Jan 2011 18:18:45 +0100 (CET)
Received: from localhost (dynscan1.mnet-online.de [192.168.8.164])
	by mail.m-online.net (Postfix) with ESMTP id BDBC31C000BF;
	Tue, 25 Jan 2011 18:18:45 +0100 (CET)
X-Virus-Scanned: amavisd-new at mnet-online.de
Received: from mail.mnet-online.de ([192.168.8.180])
	by localhost (dynscan1.mail.m-online.net [192.168.8.164]) (amavisd-new,
	port 10024)
	with ESMTP id 5lzP8LTI4MMm; Tue, 25 Jan 2011 18:18:44 +0100 (CET)
Received: from igel.home (ppp-88-217-110-119.dynamic.mnet-online.de
	[88.217.110.119]) by mail.mnet-online.de (Postfix) with ESMTP;
	Tue, 25 Jan 2011 18:18:44 +0100 (CET)
Received: by igel.home (Postfix, from userid 501)
	id 6928ECA2A0; Tue, 25 Jan 2011 18:18:44 +0100 (CET)
From: Andreas Schwab <schwab@linux-m68k.org>
To: Jim Meyering <public-jim-Oxw1nKZkIVjk1uMJSBkQmQ@plane.gmane.org>
References: <877hdtus3o.fsf@meyering.net>
X-Yow: He is the MELBA-BEING...  the ANGEL CAKE... XEROX him...  XEROX him --
Date: Tue, 25 Jan 2011 18:18:44 +0100
In-Reply-To: <877hdtus3o.fsf@meyering.net> (Jim Meyering's message of "Tue, 25
	Jan 2011 12:42:51 +0100")
Message-ID: <m21v40oqa3.fsf@igel.home>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.2.92 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Mailman-Approved-At: Tue, 25 Jan 2011 12:19:29 -0500
Cc: public-coreutils-mXXj517/zsQ@plane.gmane.org
Subject: [coreutils] Re: [PATCH 1/2] tests: avoid rare FP failure in new du
	test
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Tue, 25 Jan 2011 17:19:01 -0000



> +# Prohibit suspension, which could otherwise cause a timeout-induced FP failure.
> +trap '' STOP TSTP

SIGSTOP cannot be caught, blocked, or ignored.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."



From MAILER-DAEMON Tue Jan 25 12:30:27 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1Phmip-0006U1-KZ
	for mharc-coreutils@gnu.org; Tue, 25 Jan 2011 12:30:27 -0500
Received: from [140.186.70.92] (port=41817 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1Phmii-0006RO-E3
	for coreutils@gnu.org; Tue, 25 Jan 2011 12:30:25 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Phmie-0008SO-BT
	for coreutils@gnu.org; Tue, 25 Jan 2011 12:30:19 -0500
Received: from mx.meyering.net ([82.230.74.64]:35645)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Phmie-0008Qc-5g
	for coreutils@gnu.org; Tue, 25 Jan 2011 12:30:16 -0500
Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000)
	id AA7D560162; Tue, 25 Jan 2011 18:30:13 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: Andreas Schwab <schwab@linux-m68k.org>
In-Reply-To: <m21v40oqa3.fsf@igel.home> (Andreas Schwab's message of "Tue, 25
	Jan 2011 18:18:44 +0100")
References: <877hdtus3o.fsf@meyering.net> <m21v40oqa3.fsf@igel.home>
Date: Tue, 25 Jan 2011 18:30:13 +0100
Message-ID: <87k4hssxga.fsf@meyering.net>
Lines: 37
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
Cc: coreutils@gnu.org
Subject: [coreutils] Re: [PATCH 1/2] tests: avoid rare FP failure in new du
	test
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Tue, 25 Jan 2011 17:30:25 -0000

Andreas Schwab wrote:

>> +# Prohibit suspension, which could otherwise cause a
>> timeout-induced FP failure.
>> +trap '' STOP TSTP
>
> SIGSTOP cannot be caught, blocked, or ignored.

Right, of course.  Thanks:

>From d255e40c371d3d132ac6529a7e3468dab6fdabfd Mon Sep 17 00:00:00 2001
From: Andreas Schwab <schwab@linux-m68k.org>
Date: Tue, 25 Jan 2011 18:29:07 +0100
Subject: [PATCH] tests: minor correction

* tests/du/move-dir-while-traversing: Ignoring SIGTSTP is enough;
don't also attempt to ignore SIGSTOP, it cannot be handled or ignored.
Spotted by Andreas Schwab.
---
 tests/du/move-dir-while-traversing |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/tests/du/move-dir-while-traversing b/tests/du/move-dir-while-traversing
index 2a60d6e..68302b8 100755
--- a/tests/du/move-dir-while-traversing
+++ b/tests/du/move-dir-while-traversing
@@ -72,7 +72,7 @@ for i in $(seq 30); do
 done

 # Prohibit suspension, which could otherwise cause a timeout-induced FP failure.
-trap '' STOP TSTP
+trap '' TSTP

 timeout 6 ./inotify-watch-for-dir-access.py $t/3/a/b > start-msg &

--
1.7.3.5.38.gb312b


From MAILER-DAEMON Wed Jan 26 02:41:51 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1Pi00l-0003FO-3e
	for mharc-coreutils@gnu.org; Wed, 26 Jan 2011 02:41:51 -0500
Received: from [140.186.70.92] (port=44728 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1Pi00i-0003F6-Qe
	for coreutils@gnu.org; Wed, 26 Jan 2011 02:41:49 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Pi00h-0002Re-Mc
	for coreutils@gnu.org; Wed, 26 Jan 2011 02:41:48 -0500
Received: from mx.meyering.net ([82.230.74.64]:34088)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Pi00h-0002RM-Gu
	for coreutils@gnu.org; Wed, 26 Jan 2011 02:41:47 -0500
Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000)
	id 8A6E1600E1; Wed, 26 Jan 2011 08:41:46 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: coreutils@gnu.org
Date: Wed, 26 Jan 2011 08:41:46 +0100
Message-ID: <87lj28qfgl.fsf@meyering.net>
Lines: 37
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
Subject: [coreutils] [PATCH] tests: don't hide all trace of the
	vc_exe_in_TESTS test
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Wed, 26 Jan 2011 07:41:49 -0000

FYI,

I was tempted to remove this rule altogether, since
test scripts no longer have to be executable.
However, it's not that expensive and does serve to
ensure consistency, and as a useful cross-check.

>From 034e496e983d286011fa5f6058b541b163ba9afd Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering@redhat.com>
Date: Tue, 25 Jan 2011 14:29:07 +0100
Subject: [PATCH] tests: don't hide all trace of the vc_exe_in_TESTS test

There was a non-negligible delay after running a single test.
Now, you'll know why when you see this test's name.
* tests/check.mk (vc_exe_in_TESTS): Don't @-hide commands.
Use $(AM_V_GEN) instead.
---
 tests/check.mk |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/tests/check.mk b/tests/check.mk
index cddfe89..f931806 100644
--- a/tests/check.mk
+++ b/tests/check.mk
@@ -20,8 +20,8 @@
 _v = TESTS
 _w = root_tests
 vc_exe_in_TESTS: Makefile
-	@rm -f t1 t2
-	@if test -d $(top_srcdir)/.git && test $(srcdir) = .; then	\
+	$(AM_V_GEN)rm -f t1 t2;						\
+	if test -d $(top_srcdir)/.git && test $(srcdir) = .; then	\
 	  { sed -n '/^$(_v) =[	 ]*\\$$/,/[^\]$$/p'			\
 		$(srcdir)/Makefile.am					\
 	    | sed 's/^  *//;/^\$$.*/d;/^$(_v) =/d';			\
--
1.7.3.5.38.gb312b


From MAILER-DAEMON Wed Jan 26 10:37:11 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1Pi7Qk-0008R4-Vu
	for mharc-coreutils@gnu.org; Wed, 26 Jan 2011 10:37:11 -0500
Received: from [140.186.70.92] (port=45772 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1Pi7Qj-0008Qs-1f
	for coreutils@gnu.org; Wed, 26 Jan 2011 10:37:10 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Pi7Qh-00025V-35
	for coreutils@gnu.org; Wed, 26 Jan 2011 10:37:08 -0500
Received: from mail1.slb.deg.dub.stisp.net ([84.203.253.98]:3334)
	by eggs.gnu.org with smtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Pi7Qg-00024x-Oq
	for coreutils@gnu.org; Wed, 26 Jan 2011 10:37:07 -0500
Received: (qmail 92939 invoked from network); 26 Jan 2011 15:37:04 -0000
Received: from unknown (HELO ?192.168.2.25?) (84.203.137.218)
	by mail1.slb.deg.dub.stisp.net with SMTP; 26 Jan 2011 15:37:04 -0000
Message-ID: <4D403F3E.9060408@draigBrady.com>
Date: Wed, 26 Jan 2011 15:35:26 +0000
From: =?UTF-8?B?UMOhZHJhaWcgQnJhZHk=?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US;
	rv:1.9.1.8) Gecko/20100227 Thunderbird/3.0.3
MIME-Version: 1.0
To: Coreutils <coreutils@gnu.org>
X-Enigmail-Version: 1.0.1
Content-Type: multipart/mixed; boundary="------------050409000007000906020001"
X-detected-operating-system: by eggs.gnu.org: FreeBSD 4.6-4.9
Subject: [coreutils] [PATCH] don't warn about disorder against empty input
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Wed, 26 Jan 2011 15:37:10 -0000

This is a multi-part message in MIME format.
--------------050409000007000906020001
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

I notice that join doesn't warn about disorder
when there are no mismatches between the two files,
which is a good thing.

$ join -a1 <(printf '1\n0\n') <(printf '1\n0\n')
1
0

However it does warn if there is no input
in one of the files. I'm inclined to think that
the following should be treated as above?

$ join -a1 <(printf '1\n0\n') /dev/null
1
join: file 1 is not in sorted order
0

Doing that would allow one to use join to
extract fields with awk like blank handling,
and reordering support, without needing
to specify --nocheck-order.

$ printf "1 2 3\n" | join -a1 - /dev/null -o 1.3,1.1
3 1

cheers,
PĂˇdraig.

--------------050409000007000906020001
Content-Type: text/x-patch;
 name="join-null-disorder.diff"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename="join-null-disorder.diff"

diff --git a/src/join.c b/src/join.c
index afda5a1..8fcd47b 100644
--- a/src/join.c
+++ b/src/join.c
@@ -354,7 +354,7 @@ keycmp (struct line const *line1, struct line const *line2,
 
 /* Check that successive input lines PREV and CURRENT from input file
    WHATFILE are presented in order, unless the user may be relying on
-   the GNU extension that input lines may be out of order if no input
+   the extension that input lines may be out of order if no input
    lines are unpairable.
 
    If the user specified --nocheck-order, the check is not made.
@@ -711,7 +711,7 @@ join (FILE *fp1, FILE *fp2)
         seq2.count = 0;
     }
 
-  /* If the user did not specify --check-order, then we read the
+  /* If the user did not specify --nocheck-order, then we read the
      tail ends of both inputs to verify that they are in order.  We
      skip the rest of the tail once we have issued a warning for that
      file, unless we actually need to print the unpairable lines.  */
@@ -726,7 +726,8 @@ join (FILE *fp1, FILE *fp2)
     {
       if (print_unpairables_1)
         prjoin (seq1.lines[0], &uni_blank);
-      seen_unpairable = true;
+      if (seq2.count)
+        seen_unpairable = true;
       while (get_line (fp1, &line, 1))
         {
           if (print_unpairables_1)
@@ -740,7 +741,8 @@ join (FILE *fp1, FILE *fp2)
     {
       if (print_unpairables_2)
         prjoin (&uni_blank, seq2.lines[0]);
-      seen_unpairable = true;
+      if (seq1.count)
+        seen_unpairable = true;
       while (get_line (fp2, &line, 2))
         {
           if (print_unpairables_2)

--------------050409000007000906020001--


From MAILER-DAEMON Fri Jan 28 13:17:10 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1Pissg-0007mO-Fk
	for mharc-coreutils@gnu.org; Fri, 28 Jan 2011 13:17:10 -0500
Received: from [140.186.70.92] (port=47912 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1Pissa-0007NK-Dr
	for coreutils@gnu.org; Fri, 28 Jan 2011 13:17:08 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PismU-00045e-TX
	for coreutils@gnu.org; Fri, 28 Jan 2011 13:10:47 -0500
Received: from mx.meyering.net ([82.230.74.64]:45747)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PismU-00045Z-O9
	for coreutils@gnu.org; Fri, 28 Jan 2011 13:10:46 -0500
Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000)
	id 26F176008B; Fri, 28 Jan 2011 19:10:46 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: coreutils@gnu.org
Date: Fri, 28 Jan 2011 19:10:46 +0100
Message-ID: <87pqrgc315.fsf@meyering.net>
Lines: 6
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
X-Received-From: 82.230.74.64
Subject: [coreutils] coreutils-8.10 next week
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 28 Jan 2011 18:17:09 -0000

I would like to release coreutils-8.10 next week, including the changes
on the fiemap-copy-2 branch (shortly to be extended).  I'll make a test
release as soon as the FIEMAP changes have been merged to master.

If anyone has additional changes that they would like to see included,
please let us know.


From MAILER-DAEMON Fri Jan 28 17:28:01 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PiwnR-0006HR-0S
	for mharc-coreutils@gnu.org; Fri, 28 Jan 2011 17:28:01 -0500
Received: from [140.186.70.92] (port=51066 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PiwnO-0006HB-Os
	for coreutils@gnu.org; Fri, 28 Jan 2011 17:27:59 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PiwnN-00054B-An
	for coreutils@gnu.org; Fri, 28 Jan 2011 17:27:58 -0500
Received: from mx.meyering.net ([82.230.74.64]:34527)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PiwnM-000540-RR
	for coreutils@gnu.org; Fri, 28 Jan 2011 17:27:57 -0500
Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000)
	id 0233D6008B; Fri, 28 Jan 2011 23:27:54 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: coreutils@gnu.org
Date: Fri, 28 Jan 2011 23:27:54 +0100
Message-ID: <877hdobr4l.fsf@meyering.net>
Lines: 167
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
X-Received-From: 82.230.74.64
Subject: [coreutils] [PATCH] tests: remove obsolete uses of "$$" in
	temporary file names
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 28 Jan 2011 22:27:59 -0000

FYI, here's some clean-up I pushed along the way...

>From 68a734d66bbe72d5d4133c386fa77069dd77fb8c Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering@redhat.com>
Date: Fri, 28 Jan 2011 19:23:21 +0100
Subject: [PATCH] tests: remove obsolete uses of "$$" in temporary file names

Those were useful when tests might have been run in the same
directory and in parallel.  Now, each test is run in a newly-
created empty directory.
* tests/cp/backup-1: Remove obsolete uses of "$$".
* tests/cp/same-file: Likewise.
* tests/dd/misc: Likewise.
* tests/mv/part-symlink: Likewise.
* tests/mv/to-symlink: Likewise.
* tests/touch/fail-diag: Likewise.
---
 tests/cp/backup-1     |    4 +---
 tests/cp/same-file    |    9 +++------
 tests/dd/misc         |    8 ++++----
 tests/mv/part-symlink |    9 +++------
 tests/mv/to-symlink   |    4 +---
 tests/touch/fail-diag |   10 +++-------
 6 files changed, 15 insertions(+), 29 deletions(-)

diff --git a/tests/cp/backup-1 b/tests/cp/backup-1
index a3ed9c3..34670bd 100755
--- a/tests/cp/backup-1
+++ b/tests/cp/backup-1
@@ -21,10 +21,8 @@
 print_ver_ cp

 suffix=.b
-file=b1.$$
+file=F
 file_backup="$file$suffix"
-temp_files="$file $file_backup"
-rm -f $temp_files

 echo test > $file || fail=1

diff --git a/tests/cp/same-file b/tests/cp/same-file
index 7bb605a..da4fce3 100755
--- a/tests/cp/same-file
+++ b/tests/cp/same-file
@@ -39,10 +39,7 @@ test $hard_link_to_symlink_does_the_deref = yes \
     && remove_these_sed='/^0 -[bf]*l .*sl1 ->/d' \
     || remove_these_sed='/^ELIDE NO TEST OUTPUT/d'

-actual=actual-$$
-expected=expected-$$
-
-exec 3>&1 1> $actual
+exec 3>&1 1> actual

 # FIXME: This should be bigger: like more than 8k
 contents=XYZ
@@ -130,7 +127,7 @@ for args in 'foo symlink' 'symlink foo' 'foo foo' 'sl1 sl2' 'foo hardlink'; do
   echo
 done

-cat <<\EOF | sed "$remove_these_sed" > $expected
+cat <<\EOF | sed "$remove_these_sed" > expected
 1 [cp: `foo' and `symlink' are the same file] (foo symlink -> foo)
 1 -d [cp: `foo' and `symlink' are the same file] (foo symlink -> foo)
 1 -f [cp: `foo' and `symlink' are the same file] (foo symlink -> foo)
@@ -218,6 +215,6 @@ EOF

 exec 1>&3 3>&-

-compare $expected $actual 1>&2 || fail=1
+compare expected actual 1>&2 || fail=1

 Exit $fail
diff --git a/tests/dd/misc b/tests/dd/misc
index f6efde2..820984c 100755
--- a/tests/dd/misc
+++ b/tests/dd/misc
@@ -20,10 +20,10 @@
 . "${srcdir=.}/init.sh"; path_prepend_ ../src
 print_ver_ dd

-tmp_in=dd-in.$$
-tmp_in2=dd-in2.$$
-tmp_sym=dd-sym.$$
-tmp_out=dd-out.$$
+tmp_in=dd-in
+tmp_in2=dd-in2
+tmp_sym=dd-sym
+tmp_out=dd-out

 warn=0
 echo data > $tmp_in || framework_failure
diff --git a/tests/mv/part-symlink b/tests/mv/part-symlink
index de8ece0..ec523b8 100755
--- a/tests/mv/part-symlink
+++ b/tests/mv/part-symlink
@@ -45,10 +45,7 @@ pwd_tmp=`pwd`
 # Exercise those four cases for each of
 # cp and mv, with lots of combinations of options.

-actual=actual-$$
-expected=expected-$$
-
-exec 1> $actual
+exec 1> actual

 # FIXME: This should be bigger: like more than 8k
 contents=XYZ
@@ -153,7 +150,7 @@ done
 test $fail = 1 &&
   { (exit 1); exit; }

-cat <<\EOF > $expected
+cat <<\EOF > expected
 1 cp loc_reg rem_sl
  [cp: `loc_reg' and `rem_sl' are the same file]
  (loc_reg) (rem_sl -> dir/loc_reg)
@@ -259,6 +256,6 @@ cat <<\EOF > $expected
 EOF

 # Redirect to stderr, since stdout is already taken.
-compare $expected $actual 1>&2 || fail=1
+compare expected actual 1>&2 || fail=1

 Exit $fail
diff --git a/tests/mv/to-symlink b/tests/mv/to-symlink
index ffc291c..600fde3 100755
--- a/tests/mv/to-symlink
+++ b/tests/mv/to-symlink
@@ -24,10 +24,8 @@ cleanup_() { rm -rf "$other_partition_tmpdir"; }

 rem_file="$other_partition_tmpdir/file"
 rem_symlink="$other_partition_tmpdir/symlink"
-file=to-sym-$$
+file=to-sym

-
-rm -f $file || framework_failure
 echo local > $file || framework_failure
 echo remote > $rem_file || framework_failure
 ln -s $rem_file $rem_symlink || framework_failure
diff --git a/tests/touch/fail-diag b/tests/touch/fail-diag
index a4c81c6..a2613fe 100755
--- a/tests/touch/fail-diag
+++ b/tests/touch/fail-diag
@@ -20,15 +20,11 @@
 print_ver_ touch
 skip_if_root_

-d1=no-$$
-dir=/$d1/such-dir
-# Ensure that $d1 doesn't already exist.
-ls -d $d1 2> /dev/null && framework_failure
+file=/no-such-dir/file

-
-touch $dir > out 2>&1 && fail=1
+touch $file > out 2>&1 && fail=1
 cat <<EOF > exp
-touch: cannot touch \`$dir': No such file or directory
+touch: cannot touch \`$file': No such file or directory
 EOF

 compare out exp || fail=1
--
1.7.3.5.44.g960a


From MAILER-DAEMON Fri Jan 28 17:49:10 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1Pix7t-0000Zn-T2
	for mharc-coreutils@gnu.org; Fri, 28 Jan 2011 17:49:09 -0500
Received: from [140.186.70.92] (port=42505 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1Pix7q-0000Wt-9t
	for coreutils@gnu.org; Fri, 28 Jan 2011 17:49:08 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Pix7n-0000Xu-T7
	for coreutils@gnu.org; Fri, 28 Jan 2011 17:49:06 -0500
Received: from mail1.slb.deg.dub.stisp.net ([84.203.253.98]:9712)
	by eggs.gnu.org with smtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Pix7n-0000XI-An
	for coreutils@gnu.org; Fri, 28 Jan 2011 17:49:03 -0500
Received: (qmail 75765 invoked from network); 28 Jan 2011 22:48:59 -0000
Received: from unknown (HELO ?192.168.2.25?) (84.203.137.218)
	by mail1.slb.deg.dub.stisp.net with SMTP; 28 Jan 2011 22:48:59 -0000
Message-ID: <4D43476F.9090604@draigBrady.com>
Date: Fri, 28 Jan 2011 22:47:11 +0000
From: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US;
	rv:1.9.1.8) Gecko/20100227 Thunderbird/3.0.3
MIME-Version: 1.0
To: coreutils@gnu.org
Subject: Re: [coreutils] coreutils-8.10 next week
References: <87pqrgc315.fsf@meyering.net>
In-Reply-To: <87pqrgc315.fsf@meyering.net>
X-Enigmail-Version: 1.0.1
Content-Type: multipart/mixed; boundary="------------000704070604000803020401"
X-detected-operating-system: by eggs.gnu.org: FreeBSD 4.6-4.9
X-Received-From: 84.203.253.98
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 28 Jan 2011 22:49:08 -0000

This is a multi-part message in MIME format.
--------------000704070604000803020401
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit

On 28/01/11 18:10, Jim Meyering wrote:
> I would like to release coreutils-8.10 next week, including the changes
> on the fiemap-copy-2 branch (shortly to be extended).  I'll make a test
> release as soon as the FIEMAP changes have been merged to master.
> 
> If anyone has additional changes that they would like to see included,
> please let us know.

I intend to apply the attached:

join: ensure --header skips the order check with empty files
join: don't report disorder against an empty file
join: add -o 'auto' to output a constant number of fields per line
doc: add alternatives for field processing not supported by cut

cheers,
Pádraig.

--------------000704070604000803020401
Content-Type: text/x-patch;
 name="join-null-header.diff"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename="join-null-header.diff"

>From c36b20e51f028d13d06c27d86c0cf009313bacd3 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?P=C3=A1draig=20Brady?= <P@draigBrady.com>
Date: Fri, 14 Jan 2011 08:46:21 +0000
Subject: [PATCH] join: ensure --header skips the order check with empty files

* src/join.c: Skip the header even if one of the files is empty.
* tests/misc/join: Add a test case.
* NEWS: Mention the fix
---
 NEWS            |    3 +++
 src/join.c      |   12 ++++++++----
 tests/misc/join |    6 ++++++
 3 files changed, 17 insertions(+), 4 deletions(-)

diff --git a/NEWS b/NEWS
index 9ccad63..82565b0 100644
--- a/NEWS
+++ b/NEWS
@@ -13,6 +13,9 @@ GNU coreutils NEWS                                    -*- outline -*-
   rm -f no longer fails for EINVAL or EILSEQ on file systems that
   reject file names invalid for that file system.
 
+  join --header now skips the ordering check for the first line
+  even if the other file is empty.
+
 
 * Noteworthy changes in release 8.9 (2011-01-04) [stable]
 
diff --git a/src/join.c b/src/join.c
index afda5a1..07ac856 100644
--- a/src/join.c
+++ b/src/join.c
@@ -627,13 +627,17 @@ join (FILE *fp1, FILE *fp2)
   initseq (&seq2);
   getseq (fp2, &seq2, 2);
 
-  if (join_header_lines && seq1.count && seq2.count)
+  if (join_header_lines && (seq1.count || seq2.count))
     {
-      prjoin (seq1.lines[0], seq2.lines[0]);
+      struct line const *hline1 = seq1.count ? seq1.lines[0] : &uni_blank;
+      struct line const *hline2 = seq2.count ? seq2.lines[0] : &uni_blank;
+      prjoin (hline1, hline2);
       prevline[0] = NULL;
       prevline[1] = NULL;
-      advance_seq (fp1, &seq1, true, 1);
-      advance_seq (fp2, &seq2, true, 2);
+      if (seq1.count)
+        advance_seq (fp1, &seq1, true, 1);
+      if (seq2.count)
+        advance_seq (fp2, &seq2, true, 2);
     }
 
   while (seq1.count && seq2.count)
diff --git a/tests/misc/join b/tests/misc/join
index 3696a03..0299427 100755
--- a/tests/misc/join
+++ b/tests/misc/join
@@ -219,6 +219,12 @@ my @tv = (
  [ "ID1 Name\n1 A\n2 B\n", "ID2 Color\n1 red\n"],
    "ID1 Name Color\n1 A red\n", 0],
 
+# '--header' doesn't check order of a header
+# even if there is no header in the second file
+['header-6', '--header -a1',
+ [ "ID1 Name\n1 A\n", ""],
+   "ID1 Name\n1 A\n", 0],
+
 );
 
 # Convert the above old-style test vectors to the newer
-- 
1.7.3.4


--------------000704070604000803020401
Content-Type: text/x-patch;
 name="join-order-null.diff"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename="join-order-null.diff"

>From 1ebde7af7d1e122bcb2d0935d0b37af21156f357 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?P=C3=A1draig=20Brady?= <P@draigBrady.com>
Date: Thu, 27 Jan 2011 07:17:16 +0000
Subject: [PATCH] join: don't report disorder against an empty file

This allows one to use join as a field extractor like:
  join -a1 -o 1.3,1.1 - /dev/null

* src/join.c (join): Don't flag unpairable lines when
one of the files is empty.
* tests/misc/join: Add a new test for empty input, and adjust
a previous test that was only checking against empty input.
* doc/coreutils.texi (join invocation): Document the change.
* NEWS: Likewise.
---
 NEWS               |    6 ++++++
 doc/coreutils.texi |   18 +++++++++++++-----
 src/join.c         |    8 +++++---
 tests/misc/join    |    6 +++++-
 4 files changed, 29 insertions(+), 9 deletions(-)

diff --git a/NEWS b/NEWS
index 9ccad63..422bbe6 100644
--- a/NEWS
+++ b/NEWS
@@ -13,6 +13,12 @@ GNU coreutils NEWS                                    -*- outline -*-
   rm -f no longer fails for EINVAL or EILSEQ on file systems that
   reject file names invalid for that file system.
 
+** Changes in behavior
+
+  join no longer reports disorder when one of the files is empty.
+  This allows one to use join as a field extractor like:
+  join -a1 -o 1.3,1.1 - /dev/null
+
 
 * Noteworthy changes in release 8.9 (2011-01-04) [stable]
 
diff --git a/doc/coreutils.texi b/doc/coreutils.texi
index 85d5201..c2a7580 100644
--- a/doc/coreutils.texi
+++ b/doc/coreutils.texi
@@ -4761,11 +4761,17 @@ If there is an error it exits with nonzero status.
 @macro checkOrderOption{cmd}
 If the @option{--check-order} option is given, unsorted inputs will
 cause a fatal error message.  If the option @option{--nocheck-order}
-is given, unsorted inputs will never cause an error message.  If
-neither of these options is given, wrongly sorted inputs are diagnosed
-only if an input file is found to contain unpairable lines.  If an
-input file is diagnosed as being unsorted, the @command{\cmd\} command
-will exit with a nonzero status (and the output should not be used).
+is given, unsorted inputs will never cause an error message.  If neither
+of these options is given, wrongly sorted inputs are diagnosed
+only if an input file is found to contain unpairable
+@ifset JOIN_COMMAND
+lines, and when both input files are non empty.
+@end ifset
+@ifclear JOIN_COMMAND
+lines.
+@end ifclear
+If an input file is diagnosed as being unsorted, the @command{\cmd\}
+command will exit with a nonzero status (and the output should not be used).
 
 Forcing @command{\cmd\} to process wrongly sorted input files
 containing unpairable lines by specifying @option{--nocheck-order} is
@@ -5646,7 +5652,9 @@ c c1 c2
 b b1 b2
 @end example
 
+@set JOIN_COMMAND
 @checkOrderOption{join}
+@clear JOIN_COMMAND
 
 The defaults are:
 @itemize
diff --git a/src/join.c b/src/join.c
index afda5a1..6e10f61 100644
--- a/src/join.c
+++ b/src/join.c
@@ -711,7 +711,7 @@ join (FILE *fp1, FILE *fp2)
         seq2.count = 0;
     }
 
-  /* If the user did not specify --check-order, then we read the
+  /* If the user did not specify --nocheck-order, then we read the
      tail ends of both inputs to verify that they are in order.  We
      skip the rest of the tail once we have issued a warning for that
      file, unless we actually need to print the unpairable lines.  */
@@ -726,7 +726,8 @@ join (FILE *fp1, FILE *fp2)
     {
       if (print_unpairables_1)
         prjoin (seq1.lines[0], &uni_blank);
-      seen_unpairable = true;
+      if (seq2.count)
+        seen_unpairable = true;
       while (get_line (fp1, &line, 1))
         {
           if (print_unpairables_1)
@@ -740,7 +741,8 @@ join (FILE *fp1, FILE *fp2)
     {
       if (print_unpairables_2)
         prjoin (&uni_blank, seq2.lines[0]);
-      seen_unpairable = true;
+      if (seq1.count)
+        seen_unpairable = true;
       while (get_line (fp2, &line, 2))
         {
           if (print_unpairables_2)
diff --git a/tests/misc/join b/tests/misc/join
index 3696a03..3ce267c 100755
--- a/tests/misc/join
+++ b/tests/misc/join
@@ -189,7 +189,11 @@ my @tv = (
 
 # Before 6.10.143, this would mistakenly fail with the diagnostic:
 # join: File 1 is not in sorted order
-['chkodr-7', '-12', ["2 a\n1 b\n", ""], "", 0],
+['chkodr-7', '-12', ["2 a\n1 b\n", "2 c\n1 d"], "", 0],
+
+# After 8.9, join doesn't report disorder by default
+# when comparing against an empty input file.
+['chkodr-8', '', ["2 a\n1 b\n", ""], "", 0],
 
 # Test '--header' feature
 ['header-1', '--header',
-- 
1.7.3.4


--------------000704070604000803020401
Content-Type: text/x-patch;
 name="join-auto-format.diff"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename="join-auto-format.diff"

>From 29746dc55e6176934388c772dbe70012859897ff Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?P=C3=A1draig=20Brady?= <P@draigBrady.com>
Date: Wed, 5 Jan 2011 11:52:54 +0000
Subject: [PATCH] join: add -o 'auto' to output a constant number of fields per line

Lines with a different number of fields than the first line,
will be truncated or padded.

* src/join.c (prfields): A new function refactored from prjoin(),
to output all but the join field.
(prjoin): Don't swap line1 and line2 when line1 is blank
so that the padding is applied to the right place.
(main): Handle the -o 'auto' option.
* tests/misc/join: Add 6 new cases to test the auto format.
* NEWS: Mention the change in behavior.
Suggestion from Assaf Gordon
---
 NEWS               |    6 +++
 doc/coreutils.texi |   19 ++++++++---
 src/join.c         |   88 +++++++++++++++++++++++++++++++++------------------
 tests/misc/join    |   20 ++++++++++++
 4 files changed, 96 insertions(+), 37 deletions(-)

diff --git a/NEWS b/NEWS
index 9ccad63..a9d329a 100644
--- a/NEWS
+++ b/NEWS
@@ -13,6 +13,12 @@ GNU coreutils NEWS                                    -*- outline -*-
   rm -f no longer fails for EINVAL or EILSEQ on file systems that
   reject file names invalid for that file system.
 
+** New features
+
+  join now supports -o 'auto' which will automatically infer the
+  output format from the first line in each file, to ensure
+  the same number of fields are output for each line.
+
 
 * Noteworthy changes in release 8.9 (2011-01-04) [stable]
 
diff --git a/doc/coreutils.texi b/doc/coreutils.texi
index 85d5201..9397ab3 100644
--- a/doc/coreutils.texi
+++ b/doc/coreutils.texi
@@ -5675,8 +5675,8 @@ Do not check that both input files are in sorted order.  This is the default.
 
 @item -e @var{string}
 @opindex -e
-Replace those output fields that are missing in the input with
-@var{string}.
+Replace those output fields that are missing in the input with @var{string}.
+I.E. missing fields specified with the @option{-12jo} options.
 
 @item --header
 @opindex --header
@@ -5707,10 +5707,17 @@ Join on field @var{field} (a positive integer) of file 2.
 Equivalent to @option{-1 @var{field} -2 @var{field}}.
 
 @item -o @var{field-list}
-Construct each output line according to the format in @var{field-list}.
-Each element in @var{field-list} is either the single character @samp{0} or
-has the form @var{m.n} where the file number, @var{m}, is @samp{1} or
-@samp{2} and @var{n} is a positive field number.
+@itemx -o auto
+If the keyword @samp{auto} is specified, infer the output format from
+the first line in each file.  This is the same as the default output format
+but also ensures the same number of fields are output for each line.
+Missing fields are replaced with the @option{-e} option and extra fields
+are discarded.
+
+Otherwise, construct each output line according to the format in
+@var{field-list}.  Each element in @var{field-list} is either the single
+character @samp{0} or has the form @var{m.n} where the file number, @var{m},
+is @samp{1} or @samp{2} and @var{n} is a positive field number.
 
 A field specification of @samp{0} denotes the join field.
 In most cases, the functionality of the @samp{0} field spec
diff --git a/src/join.c b/src/join.c
index afda5a1..bf7e908 100644
--- a/src/join.c
+++ b/src/join.c
@@ -112,6 +112,13 @@ static bool issued_disorder_warning[2];
 /* Empty output field filler.  */
 static char const *empty_filler;
 
+/* Whether to ensure the same number of fields are output from each line.  */
+static bool autoformat;
+/* The number of fields to output for each line.
+   Only significant when autoformat is true.  */
+static size_t autocount_1;
+static size_t autocount_2;
+
 /* Field to join on; SIZE_MAX means they haven't been determined yet.  */
 static size_t join_field_1 = SIZE_MAX;
 static size_t join_field_2 = SIZE_MAX;
@@ -210,7 +217,8 @@ else fields are separated by CHAR.  Any FIELD is a field number counted\n\
 from 1.  FORMAT is one or more comma or blank separated specifications,\n\
 each being `FILENUM.FIELD' or `0'.  Default FORMAT outputs the join field,\n\
 the remaining fields from FILE1, the remaining fields from FILE2, all\n\
-separated by CHAR.\n\
+separated by CHAR.  If FORMAT is the keyword 'auto', then the first\n\
+line of each file determines the number of fields output for each line.\n\
 \n\
 Important: FILE1 and FILE2 must be sorted on the join fields.\n\
 E.g., use ` sort -k 1b,1 ' if `join' has no options,\n\
@@ -527,6 +535,27 @@ prfield (size_t n, struct line const *line)
     fputs (empty_filler, stdout);
 }
 
+/* Output all the fields in line, other than the join field.  */
+
+static void
+prfields (struct line const *line, size_t join_field, size_t autocount)
+{
+  size_t i;
+  size_t nfields = autoformat ? autocount : line->nfields;
+  char output_separator = tab < 0 ? ' ' : tab;
+
+  for (i = 0; i < join_field && i < nfields; ++i)
+    {
+      putchar (output_separator);
+      prfield (i, line);
+    }
+  for (i = join_field + 1; i < nfields; ++i)
+    {
+      putchar (output_separator);
+      prfield (i, line);
+    }
+}
+
 /* Print the join of LINE1 and LINE2.  */
 
 static void
@@ -534,6 +563,8 @@ prjoin (struct line const *line1, struct line const *line2)
 {
   const struct outlist *outlist;
   char output_separator = tab < 0 ? ' ' : tab;
+  size_t field;
+  struct line const *line;
 
   outlist = outlist_head.next;
   if (outlist)
@@ -543,9 +574,6 @@ prjoin (struct line const *line1, struct line const *line2)
       o = outlist;
       while (1)
         {
-          size_t field;
-          struct line const *line;
-
           if (o->file == 0)
             {
               if (line1 == &uni_blank)
@@ -574,37 +602,24 @@ prjoin (struct line const *line1, struct line const *line2)
     }
   else
     {
-      size_t i;
-
       if (line1 == &uni_blank)
         {
-          struct line const *t;
-          t = line1;
-          line1 = line2;
-          line2 = t;
+          line = line2;
+          field = join_field_2;
         }
-      prfield (join_field_1, line1);
-      for (i = 0; i < join_field_1 && i < line1->nfields; ++i)
-        {
-          putchar (output_separator);
-          prfield (i, line1);
-        }
-      for (i = join_field_1 + 1; i < line1->nfields; ++i)
+      else
         {
-          putchar (output_separator);
-          prfield (i, line1);
+          line = line1;
+          field = join_field_1;
         }
 
-      for (i = 0; i < join_field_2 && i < line2->nfields; ++i)
-        {
-          putchar (output_separator);
-          prfield (i, line2);
-        }
-      for (i = join_field_2 + 1; i < line2->nfields; ++i)
-        {
-          putchar (output_separator);
-          prfield (i, line2);
-        }
+      /* Output the join field.  */
+      prfield (field, line);
+
+      /* Output other fields.  */
+      prfields (line1, join_field_1, autocount_1);
+      prfields (line2, join_field_2, autocount_2);
+
       putchar ('\n');
     }
 }
@@ -627,6 +642,12 @@ join (FILE *fp1, FILE *fp2)
   initseq (&seq2);
   getseq (fp2, &seq2, 2);
 
+  if (autoformat)
+    {
+      autocount_1 = seq1.count ? seq1.lines[0]->nfields : 0;
+      autocount_2 = seq2.count ? seq2.lines[0]->nfields : 0;
+    }
+
   if (join_header_lines && seq1.count && seq2.count)
     {
       prjoin (seq1.lines[0], seq2.lines[0]);
@@ -1037,8 +1058,13 @@ main (int argc, char **argv)
           break;
 
         case 'o':
-          add_field_list (optarg);
-          optc_status = MIGHT_BE_O_ARG;
+          if (STREQ (optarg, "auto"))
+            autoformat = true;
+          else
+            {
+              add_field_list (optarg);
+              optc_status = MIGHT_BE_O_ARG;
+            }
           break;
 
         case 't':
diff --git a/tests/misc/join b/tests/misc/join
index 3696a03..3cf278b 100755
--- a/tests/misc/join
+++ b/tests/misc/join
@@ -127,6 +127,26 @@ my @tv = (
 # From David Dyck
 ['9a', '', [" a 1\n b 2\n", " a Y\n b Z\n"], "a 1 Y\nb 2 Z\n", 0],
 
+# -o 'auto'
+['10a', '-a1 -a2 -e . -o auto',
+ ["a 1 2\nb 1\nd 1 2\n", "a 3 4\nb 3 4\nc 3 4\n"],
+ "a 1 2 3 4\nb 1 . 3 4\nc . . 3 4\nd 1 2 . .\n", 0],
+['10b', '-a1 -a2 -j3 -e . -o auto',
+ ["a 1 2\nb 1\nd 1 2\n", "a 3 4\nb 3 4\nc 3 4\n"],
+ "2 a 1 . .\n. b 1 . .\n2 d 1 . .\n4 . . a 3\n4 . . b 3\n4 . . c 3\n"],
+['10c', '-a1 -1 1 -2 4 -e. -o auto',
+ ["a 1 2\nb 1\nd 1 2\n", "a 3 4\nb 3 4\nc 3 4\n"],
+ "a 1 2 . . .\nb 1 . . . .\nd 1 2 . . .\n"],
+['10d', '-a2 -1 1 -2 4 -e. -o auto',
+ ["a 1 2\nb 1\nd 1 2\n", "a 3 4\nb 3 4\nc 3 4\n"],
+ ". . . a 3 4\n. . . b 3 4\n. . . c 3 4\n"],
+['10e', '-o auto',
+ ["a 1 2\nb 1 2 discard\n", "a 3 4\nb 3 4 discard\n"],
+ "a 1 2 3 4\nb 1 2 3 4\n"],
+['10f', '-t, -o auto',
+ ["a,1,,2\nb,1,2\n", "a,3,4\nb,3,4\n"],
+ "a,1,,2,3,4\nb,1,2,,3,4\n"],
+
 # From Tim Smithers: fixed in 1.22l
 ['trailing-sp', '-t: -1 1 -2 1', ["a:x \n", "a:y \n"], "a:x :y \n", 0],
 
-- 
1.7.3.4


--------------000704070604000803020401
Content-Type: text/x-patch;
 name="cut-awk-docs.diff"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename="cut-awk-docs.diff"

>From 97ae252c17fdf913cad04438fa873e840c397977 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?P=C3=A1draig=20Brady?= <P@draigBrady.com>
Date: Wed, 24 Nov 2010 08:37:23 +0000
Subject: [PATCH] doc: add alternatives for field processing not supported by cut

* doc/coreutils.texi (cut invocation): Remove the tr -s '[:blank:]'
example, as it doesn't handle leading and trailing blanks.  Add `awk`
examples for common field processing operations often asked about.
Also document a `join` hack, to achieve the same thing.
---
 doc/coreutils.texi |   23 +++++++++++++++++++----
 1 files changed, 19 insertions(+), 4 deletions(-)

diff --git a/doc/coreutils.texi b/doc/coreutils.texi
index 85d5201..b30b68f 100644
--- a/doc/coreutils.texi
+++ b/doc/coreutils.texi
@@ -5474,11 +5474,26 @@ Select for printing only the fields listed in @var{field-list}.
 Fields are separated by a TAB character by default.  Also print any
 line that contains no delimiter character, unless the
 @option{--only-delimited} (@option{-s}) option is specified.
-Note @command{cut} does not support specifying runs of whitespace as a
-delimiter, so to achieve that common functionality one can pre-process
-with @command{tr} like:
+
+Note @command{awk} supports more sophisticated field processing,
+and by default will use (and discard) runs of blank characters to
+separate fields, and ignore leading and trailing blanks.
+@example
+@verbatim
+awk '{print $2}'    # print the second field
+awk '{print $NF-1}' # print the penultimate field
+awk '{print $2,$1}' # reorder the first two fields
+@end verbatim
+@end example
+
+In the unlikely event that @command{awk} is unavailable,
+one can use the @command{join} command, to process blank
+characters as @command{awk} does above.
 @example
-tr -s '[:blank:]' '\t' | cut -f@dots{}
+@verbatim
+join -a1 - /dev/null -o 1.2     # print the second field
+join -a1 - /dev/null -o 1.2,1.1 # reorder the first two fields
+@end verbatim
 @end example
 
 @item -d @var{input_delim_byte}
-- 
1.7.3.4


--------------000704070604000803020401--


From MAILER-DAEMON Sun Jan 30 10:40:22 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PjZO2-0000Gp-JT
	for mharc-coreutils@gnu.org; Sun, 30 Jan 2011 10:40:22 -0500
Received: from [140.186.70.92] (port=40639 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PjZNy-0000Em-WA
	for coreutils@gnu.org; Sun, 30 Jan 2011 10:40:19 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PjZNx-0004EQ-R7
	for coreutils@gnu.org; Sun, 30 Jan 2011 10:40:18 -0500
Received: from mx.meyering.net ([82.230.74.64]:34080)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PjZNx-0004EL-Hu
	for coreutils@gnu.org; Sun, 30 Jan 2011 10:40:17 -0500
Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000)
	id 0C682600B5; Sun, 30 Jan 2011 16:40:15 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: coreutils@gnu.org
Date: Sun, 30 Jan 2011 16:40:14 +0100
Message-ID: <87fwsa763l.fsf@meyering.net>
Lines: 54
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
X-Received-From: 82.230.74.64
Subject: [coreutils] [PATCH] copy,
	tee: assume EINTR is always defined: remove #ifdefs
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 30 Jan 2011 15:40:19 -0000

FYI, I'm pushing this to the fiemap-copy branch
that will shortly be pulled to master.

>From 372479c4afb7f3da2cd72e200a2afe65d718cb42 Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering@redhat.com>
Date: Sun, 30 Jan 2011 16:12:56 +0100
Subject: [PATCH] copy, tee: assume EINTR is always defined: remove #ifdefs

Don't use "#ifdef EINTR".  dd.c has been doing that since 2004.
* src/copy.c (sparse_copy): Remove #ifdef...#endif around EINTR use.
* src/tee.c (tee_files): Remove #ifdef...#endif around EINTR use.
If we need it, add something like this in system.h:
/* When EINTR is not defined, define it to an improbable value
   so that each use does not have to be #ifdef'd.  */
 #ifndef EINTR
 # define EINTR 999988
 #endif
---
 src/copy.c |    2 --
 src/tee.c  |    2 --
 2 files changed, 0 insertions(+), 4 deletions(-)

diff --git a/src/copy.c b/src/copy.c
index 96bb35b..4e73e1b 100644
--- a/src/copy.c
+++ b/src/copy.c
@@ -160,10 +160,8 @@ sparse_copy (int src_fd, int dest_fd, char *buf, size_t buf_size,
       ssize_t n_read = read (src_fd, buf, MIN (max_n_read, buf_size));
       if (n_read < 0)
         {
-#ifdef EINTR
           if (errno == EINTR)
             continue;
-#endif
           error (0, errno, _("reading %s"), quote (src_name));
           return false;
         }
diff --git a/src/tee.c b/src/tee.c
index 0499442..0518b07 100644
--- a/src/tee.c
+++ b/src/tee.c
@@ -183,10 +183,8 @@ tee_files (int nfiles, const char **files)
   while (1)
     {
       bytes_read = read (0, buffer, sizeof buffer);
-#ifdef EINTR
       if (bytes_read < 0 && errno == EINTR)
         continue;
-#endif
       if (bytes_read <= 0)
         break;

--
1.7.3.5.44.g960a


From MAILER-DAEMON Sun Jan 30 11:00:15 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PjZhH-0005Pz-NJ
	for mharc-coreutils@gnu.org; Sun, 30 Jan 2011 11:00:15 -0500
Received: from [140.186.70.92] (port=49287 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PjZhE-0005OZ-CX
	for coreutils@gnu.org; Sun, 30 Jan 2011 11:00:14 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PjZh7-0008Lm-I5
	for coreutils@gnu.org; Sun, 30 Jan 2011 11:00:06 -0500
Received: from mx.meyering.net ([82.230.74.64]:34358)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PjZh7-0008LT-9E
	for coreutils@gnu.org; Sun, 30 Jan 2011 11:00:05 -0500
Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000)
	id EA4126010F; Sun, 30 Jan 2011 17:00:02 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: coreutils@gnu.org
In-Reply-To: <87fwsa763l.fsf@meyering.net> (Jim Meyering's message of "Sun, 30
	Jan 2011 16:40:14 +0100")
References: <87fwsa763l.fsf@meyering.net>
Date: Sun, 30 Jan 2011 17:00:02 +0100
Message-ID: <8762t6756l.fsf@meyering.net>
Lines: 3
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
X-Received-From: 82.230.74.64
Subject: Re: [PATCH] copy,
	tee: assume EINTR is always defined: remove #ifdefs
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 30 Jan 2011 16:00:14 -0000

FYI, I've just adjusted the mailman's subject_prefix setting
so that it no longer prepends "[coreutils] " to each Subject:
on this list.


From MAILER-DAEMON Sun Jan 30 15:17:04 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1Pjdho-0000JS-5b
	for mharc-coreutils@gnu.org; Sun, 30 Jan 2011 15:17:04 -0500
Received: from [140.186.70.92] (port=47751 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1Pjdhl-0000Is-8U
	for coreutils@gnu.org; Sun, 30 Jan 2011 15:17:02 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Pjdhk-00006r-4m
	for coreutils@gnu.org; Sun, 30 Jan 2011 15:17:01 -0500
Received: from mx.meyering.net ([82.230.74.64]:42992)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Pjdhj-00006n-Ov
	for coreutils@gnu.org; Sun, 30 Jan 2011 15:17:00 -0500
Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000)
	id 799A1600A5; Sun, 30 Jan 2011 21:16:58 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: =?iso-8859-1?Q?P=E1draig?= Brady <P@draigBrady.com>
In-Reply-To: <4D43476F.9090604@draigBrady.com> (=?iso-8859-1?Q?=22P=E1drai?=
	=?iso-8859-1?Q?g?= Brady"'s message of
	"Fri, 28 Jan 2011 22:47:11 +0000")
References: <87pqrgc315.fsf@meyering.net> <4D43476F.9090604@draigBrady.com>
Date: Sun, 30 Jan 2011 21:16:58 +0100
Message-ID: <87y6622ll1.fsf@meyering.net>
Lines: 20
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
X-Received-From: 82.230.74.64
Cc: coreutils@gnu.org
Subject: Re: [coreutils] coreutils-8.10 next week
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 30 Jan 2011 20:17:02 -0000

P=E1draig Brady wrote:
> On 28/01/11 18:10, Jim Meyering wrote:
>> I would like to release coreutils-8.10 next week, including the changes
>> on the fiemap-copy-2 branch (shortly to be extended).  I'll make a test
>> release as soon as the FIEMAP changes have been merged to master.
>>
>> If anyone has additional changes that they would like to see included,
>> please let us know.
>
> I intend to apply the attached:
>
> join: ensure --header skips the order check with empty files
> join: don't report disorder against an empty file
> join: add -o 'auto' to output a constant number of fields per line
> doc: add alternatives for field processing not supported by cut

Thanks!  Those all look good.

I've pulled the fiemap-copy series onto master and pushed the result,
but have not done any portability testing yet.


From MAILER-DAEMON Mon Jan 31 09:58:20 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PjvCu-00080R-NG
	for mharc-coreutils@gnu.org; Mon, 31 Jan 2011 09:58:20 -0500
Received: from [140.186.70.92] (port=46315 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PjvCg-0007r6-Ec
	for coreutils@gnu.org; Mon, 31 Jan 2011 09:58:19 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1PjvCR-00035R-SV
	for coreutils@gnu.org; Mon, 31 Jan 2011 09:58:06 -0500
Received: from mx1.redhat.com ([209.132.183.28]:65189)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1PjvCR-00035G-LB
	for coreutils@gnu.org; Mon, 31 Jan 2011 09:57:51 -0500
Received: from int-mx02.intmail.prod.int.phx2.redhat.com
	(int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12])
	by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id p0VEvoNQ021382
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK);
	Mon, 31 Jan 2011 09:57:50 -0500
Received: from [10.3.113.114] (ovpn-113-114.phx2.redhat.com [10.3.113.114])
	by int-mx02.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP
	id p0VEvoQB001902; Mon, 31 Jan 2011 09:57:50 -0500
Message-ID: <4D46CDED.6080102@redhat.com>
Date: Mon, 31 Jan 2011 07:57:49 -0700
From: Eric Blake <eblake@redhat.com>
Organization: Red Hat
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US;
	rv:1.9.2.13) Gecko/20101209 Fedora/3.1.7-0.35.b3pre.fc14
	Lightning/1.0b3pre Mnenhy/0.8.3 Thunderbird/3.1.7
MIME-Version: 1.0
To: Jim Meyering <jim@meyering.net>
References: <87fwsa763l.fsf@meyering.net>
In-Reply-To: <87fwsa763l.fsf@meyering.net>
X-Enigmail-Version: 1.1.2
OpenPGP: url=http://people.redhat.com/eblake/eblake.gpg
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature";
	boundary="------------enig302563E257A9DB36C32D6BCA"
X-Scanned-By: MIMEDefang 2.67 on 10.5.11.12
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.132.183.28
Cc: coreutils@gnu.org
Subject: Re: [coreutils] [PATCH] copy, tee: assume EINTR is always defined:
 remove #ifdefs
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 31 Jan 2011 14:58:19 -0000

This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--------------enig302563E257A9DB36C32D6BCA
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

On 01/30/2011 08:40 AM, Jim Meyering wrote:
> FYI, I'm pushing this to the fiemap-copy branch
> that will shortly be pulled to master.
>=20
>>From 372479c4afb7f3da2cd72e200a2afe65d718cb42 Mon Sep 17 00:00:00 2001
> From: Jim Meyering <meyering@redhat.com>
> Date: Sun, 30 Jan 2011 16:12:56 +0100
> Subject: [PATCH] copy, tee: assume EINTR is always defined: remove #ifd=
efs
>=20
> Don't use "#ifdef EINTR".  dd.c has been doing that since 2004.
> * src/copy.c (sparse_copy): Remove #ifdef...#endif around EINTR use.
> * src/tee.c (tee_files): Remove #ifdef...#endif around EINTR use.
> If we need it, add something like this in system.h:
> /* When EINTR is not defined, define it to an improbable value
>    so that each use does not have to be #ifdef'd.  */

Actually, the gnulib errno module (which we already use) guarantees
pretty much ALL posix-defined errno values.  There's probably other
#ifdef E* that you could scrub.

--=20
Eric Blake   eblake@redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org


--------------enig302563E257A9DB36C32D6BCA
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Public key at http://people.redhat.com/eblake/eblake.gpg
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/

iQEcBAEBCAAGBQJNRs3tAAoJEKeha0olJ0NqWbIH/3X3DyuXLLczwf8hdX7Zt2/j
YqUMmrVLrMqoNuRbEC/b0orpsZV44VnKlUetEszQoJCVMcmX/+x5x0WuXTufxqM7
t5rxF/CU6ym+q5v0qvGbK76rDu60CtcCa1s70nnvzc93dtaVqlcWV9y+sseZuk0F
QdfB+QqNk3lqfwlc1mpsfqOoXdsDdt8j7upwtilC/ETkX3Y2vQS7PBUrGsZ0U2a5
PO7yT269JJaPPfYgPSmDtK/8onrs+r7noH38ecU+URzkbwnkRzvuTGSzvjOZq0T+
9LiZb9JnDOp2iiWIL+U57c1NWbpw3YamnpDtDoqeCp6Tiqr5eysNWYJe3MNM5tg=
=2q1Q
-----END PGP SIGNATURE-----

--------------enig302563E257A9DB36C32D6BCA--


From MAILER-DAEMON Mon Jan 31 10:11:30 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PjvPe-00034T-J2
	for mharc-coreutils@gnu.org; Mon, 31 Jan 2011 10:11:30 -0500
Received: from [140.186.70.92] (port=38825 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PjvPb-00033M-Uf
	for coreutils@gnu.org; Mon, 31 Jan 2011 10:11:28 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PjvPa-00074I-SI
	for coreutils@gnu.org; Mon, 31 Jan 2011 10:11:27 -0500
Received: from mx.meyering.net ([82.230.74.64]:52198)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PjvPa-00073r-Jv
	for coreutils@gnu.org; Mon, 31 Jan 2011 10:11:26 -0500
Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000)
	id DEA0C600ED; Mon, 31 Jan 2011 16:11:24 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: Eric Blake <eblake@redhat.com>
In-Reply-To: <4D46CDED.6080102@redhat.com> (Eric Blake's message of "Mon, 31
	Jan 2011 07:57:49 -0700")
References: <87fwsa763l.fsf@meyering.net> <4D46CDED.6080102@redhat.com>
Date: Mon, 31 Jan 2011 16:11:24 +0100
Message-ID: <877hdl152b.fsf@meyering.net>
Lines: 37
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
X-Received-From: 82.230.74.64
Cc: coreutils@gnu.org
Subject: Re: [coreutils] [PATCH] copy,
	tee: assume EINTR is always defined: remove #ifdefs
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 31 Jan 2011 15:11:28 -0000

Eric Blake wrote:

> On 01/30/2011 08:40 AM, Jim Meyering wrote:
>> FYI, I'm pushing this to the fiemap-copy branch
>> that will shortly be pulled to master.
>>
>>>From 372479c4afb7f3da2cd72e200a2afe65d718cb42 Mon Sep 17 00:00:00 2001
>> From: Jim Meyering <meyering@redhat.com>
>> Date: Sun, 30 Jan 2011 16:12:56 +0100
>> Subject: [PATCH] copy, tee: assume EINTR is always defined: remove #ifdefs
>>
>> Don't use "#ifdef EINTR".  dd.c has been doing that since 2004.
>> * src/copy.c (sparse_copy): Remove #ifdef...#endif around EINTR use.
>> * src/tee.c (tee_files): Remove #ifdef...#endif around EINTR use.
>> If we need it, add something like this in system.h:
>> /* When EINTR is not defined, define it to an improbable value
>>    so that each use does not have to be #ifdef'd.  */
>
> Actually, the gnulib errno module (which we already use) guarantees
> pretty much ALL posix-defined errno values.

As you probably know, EINTR is not on gnulib's list.

> There's probably other
> #ifdef E* that you could scrub.

Actually no.  At least not that I could see:

  $ git grep -E 'ifn?def E' src|sort -u
  src/expr.c:#ifdef EVAL_TRACE
  src/extent-scan.h:#ifndef EXTENT_SCAN_H
  src/install.c:#ifdef ENABLE_MATCHPATHCON
  src/stty.c:#ifdef ECHOCTL
  src/stty.c:#ifdef ECHOKE
  src/stty.c:#ifdef ECHOPRT
  src/system.h:#ifndef ENODATA
  src/true.c:#ifndef EXIT_STATUS


From MAILER-DAEMON Mon Jan 31 10:14:48 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PjvSq-0004Sx-EA
	for mharc-coreutils@gnu.org; Mon, 31 Jan 2011 10:14:48 -0500
Received: from [140.186.70.92] (port=50828 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PjvSo-0004SB-OD
	for coreutils@gnu.org; Mon, 31 Jan 2011 10:14:47 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1PjvSn-0007iP-1y
	for coreutils@gnu.org; Mon, 31 Jan 2011 10:14:46 -0500
Received: from mx1.redhat.com ([209.132.183.28]:1026)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1PjvSm-0007hh-P7
	for coreutils@gnu.org; Mon, 31 Jan 2011 10:14:44 -0500
Received: from int-mx10.intmail.prod.int.phx2.redhat.com
	(int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23])
	by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id p0VFEh0A009617
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK);
	Mon, 31 Jan 2011 10:14:43 -0500
Received: from [10.3.113.114] (ovpn-113-114.phx2.redhat.com [10.3.113.114])
	by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP
	id p0VFEgO9029487; Mon, 31 Jan 2011 10:14:43 -0500
Message-ID: <4D46D1E2.5080708@redhat.com>
Date: Mon, 31 Jan 2011 08:14:42 -0700
From: Eric Blake <eblake@redhat.com>
Organization: Red Hat
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US;
	rv:1.9.2.13) Gecko/20101209 Fedora/3.1.7-0.35.b3pre.fc14
	Lightning/1.0b3pre Mnenhy/0.8.3 Thunderbird/3.1.7
MIME-Version: 1.0
To: Jim Meyering <jim@meyering.net>
References: <87fwsa763l.fsf@meyering.net> <4D46CDED.6080102@redhat.com>
	<877hdl152b.fsf@meyering.net>
In-Reply-To: <877hdl152b.fsf@meyering.net>
X-Enigmail-Version: 1.1.2
OpenPGP: url=http://people.redhat.com/eblake/eblake.gpg
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature";
	boundary="------------enig4A259DFCDCFDC7C8148EDC2A"
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.132.183.28
Cc: coreutils@gnu.org
Subject: Re: [coreutils] [PATCH] copy, tee: assume EINTR is always defined:
 remove #ifdefs
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 31 Jan 2011 15:14:47 -0000

This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--------------enig4A259DFCDCFDC7C8148EDC2A
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

On 01/31/2011 08:11 AM, Jim Meyering wrote:
>> Actually, the gnulib errno module (which we already use) guarantees
>> pretty much ALL posix-defined errno values.
>=20
> As you probably know, EINTR is not on gnulib's list.

Not in errno.in.h, but it IS in tests/test-errno.c, which means that
(with no failures of that test) that it is sufficiently portable that
errno.in.h need not worry about it.

> Actually no.  At least not that I could see:
>=20
>   $ git grep -E 'ifn?def E' src|sort -u

Nice - then you cleaned the last one :)

--=20
Eric Blake   eblake@redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org


--------------enig4A259DFCDCFDC7C8148EDC2A
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Public key at http://people.redhat.com/eblake/eblake.gpg
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/

iQEcBAEBCAAGBQJNRtHiAAoJEKeha0olJ0Nq0pcH/ijUSA3NnvRFTJNkX+bls39f
y8ZDPn9j2KDcaLhQuT1QDIF3zaZv4U6XEucTJQrrbEsxQ0QmyegthrEXgQLrCZIy
LmOl91MMczHwSzbUbd26kSiDibyYuK+rHNgEMsCVsT7mTZng0hd7dTOdJof2K4Ua
R612JKA7mn8PdwVIXpfhC2Dut5qPIqZQdsP9HZxmSD06g3P0A3IdGTDYlDXqpZ2V
gwnk23bNjm1vOGPeOeWsogjl4gM2jeqk+T/qgLzreDkpg4DeAnkBrH+MHXKGHg3P
EkBWojnxVcdFnmglASA6bv8IAGwLeQMFJ/k1FjVvu/fFbIg5YGmf/2QNYYE4nDM=
=BIlX
-----END PGP SIGNATURE-----

--------------enig4A259DFCDCFDC7C8148EDC2A--


From MAILER-DAEMON Mon Jan 31 12:30:49 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1PjxaT-0002Kx-Hk
	for mharc-coreutils@gnu.org; Mon, 31 Jan 2011 12:30:49 -0500
Received: from [140.186.70.92] (port=52444 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1PjxaQ-0002KV-4R
	for coreutils@gnu.org; Mon, 31 Jan 2011 12:30:48 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PjxaO-0003XS-1E
	for coreutils@gnu.org; Mon, 31 Jan 2011 12:30:45 -0500
Received: from mx.meyering.net ([82.230.74.64]:46079)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1PjxaN-0003Ww-Ic
	for coreutils@gnu.org; Mon, 31 Jan 2011 12:30:43 -0500
Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000)
	id 5081D600ED; Mon, 31 Jan 2011 18:30:42 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: coreutils@gnu.org
Date: Mon, 31 Jan 2011 18:30:42 +0100
Message-ID: <87r5btyo8t.fsf@meyering.net>
Lines: 288
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
X-Received-From: 82.230.74.64
Subject: minor test improvements, clean-up
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 31 Jan 2011 17:30:48 -0000

FYI,

>From 040b00ef19fb1bbc0f7d3e5bec6ff82e19a92d26 Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering@redhat.com>
Date: Mon, 31 Jan 2011 13:35:55 +0100
Subject: [PATCH 1/8] tests: remove duplicate fiemap-perf test

* tests/cp/fiemap-perf: Copy block-comparing code from sparse-fiemap.
* tests/cp/sparse-fiemap: The same test was here, alongside a much
more involved test.  Remove it, now that it is in its own file.
---
 tests/cp/fiemap-perf   |    4 ++++
 tests/cp/sparse-fiemap |   13 +------------
 2 files changed, 5 insertions(+), 12 deletions(-)

diff --git a/tests/cp/fiemap-perf b/tests/cp/fiemap-perf
index 6c588cb..bd8fab8 100755
--- a/tests/cp/fiemap-perf
+++ b/tests/cp/fiemap-perf
@@ -29,4 +29,8 @@ timeout 10 truncate -s1T f || framework_failure_
 # Nothing can read (much less write) that many bytes in so little time.
 timeout 10 cp f f2 || fail=1

+# Ensure that the sparse file copied through fiemap has the same size
+# in bytes as the original.
+test $(stat --printf %s sparse) = $(stat --printf %s fiemap) || fail=1
+
 Exit $fail
diff --git a/tests/cp/sparse-fiemap b/tests/cp/sparse-fiemap
index b6b1103..aecf9d2 100755
--- a/tests/cp/sparse-fiemap
+++ b/tests/cp/sparse-fiemap
@@ -1,7 +1,7 @@
 #!/bin/sh
 # Test cp --sparse=always through fiemap copy

-# Copyright (C) 2010 Free Software Foundation, Inc.
+# Copyright (C) 2010, 2011 Free Software Foundation, Inc.

 # This program is free software: you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
@@ -47,17 +47,6 @@ else
     skip_test_ "insufficient mount/ext4 support"
 fi

-# Create a 1TiB sparse file
-dd if=/dev/zero of=sparse bs=1k count=1 seek=1G || framework_failure
-
-# It takes many minutes to copy this sparse file using the old method.
-# By contrast, it takes far less than 1 second using FIEMAP-copy.
-timeout 10 cp --sparse=always sparse fiemap || fail=1
-
-# Ensure that the sparse file copied through fiemap has the same size
-# in bytes as the original.
-test $(stat --printf %s sparse) = $(stat --printf %s fiemap) || fail=1
-
 # =================================================
 # Ensure that we exercise the FIEMAP-copying code enough
 # to provoke at least two iterations of the do...while loop
--
1.7.3.5.44.g960a


>From 08b7bce540f12595f9995851338bb765783d264c Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering@redhat.com>
Date: Mon, 31 Jan 2011 13:40:26 +0100
Subject: [PATCH 2/8] tests: modernize sparse-fiemap test

* tests/cp/sparse-fiemap: Use print_ver_, not open-coded VERBOSE test.
---
 tests/cp/sparse-fiemap |    6 +-----
 1 files changed, 1 insertions(+), 5 deletions(-)

diff --git a/tests/cp/sparse-fiemap b/tests/cp/sparse-fiemap
index aecf9d2..c873645 100755
--- a/tests/cp/sparse-fiemap
+++ b/tests/cp/sparse-fiemap
@@ -16,12 +16,8 @@
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.

-if test "$VERBOSE" = yes; then
-  set -x
-  cp --version
-fi
-
 . "${srcdir=.}/init.sh"; path_prepend_ ../src
+print_ver_ cp

 if df -T -t btrfs -t xfs -t ext4 -t ocfs2 . ; then
   : # Current dir is on a partition with working extents.  Good!
--
1.7.3.5.44.g960a


>From 31dc21f693b5dc9c267be89934ea4cb306806e39 Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering@redhat.com>
Date: Mon, 31 Jan 2011 13:42:49 +0100
Subject: [PATCH 3/8] maint: update copyright year lists in new files

* src/extent-scan.h: Update copyright year list.
* src/extent-scan.c: Likewise.
* tests/cp/sparse-fiemap: Likewise.
---
 src/extent-scan.c      |    2 +-
 src/extent-scan.h      |    2 +-
 tests/cp/sparse-fiemap |    2 +-
 3 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/src/extent-scan.c b/src/extent-scan.c
index 3bb0d53..9f38877 100644
--- a/src/extent-scan.c
+++ b/src/extent-scan.c
@@ -1,5 +1,5 @@
 /* extent-scan.c -- core functions for scanning extents
-   Copyright (C) 2010 Free Software Foundation, Inc.
+   Copyright (C) 2010-2011 Free Software Foundation, Inc.

    This program is free software: you can redistribute it and/or modify
    it under the terms of the GNU General Public License as published by
diff --git a/src/extent-scan.h b/src/extent-scan.h
index ac9e500..4724b25 100644
--- a/src/extent-scan.h
+++ b/src/extent-scan.h
@@ -1,5 +1,5 @@
 /* core functions for efficient reading sparse files
-   Copyright (C) 2010 Free Software Foundation, Inc.
+   Copyright (C) 2010-2011 Free Software Foundation, Inc.

    This program is free software: you can redistribute it and/or modify
    it under the terms of the GNU General Public License as published by
diff --git a/tests/cp/sparse-fiemap b/tests/cp/sparse-fiemap
index c873645..7e1932b 100755
--- a/tests/cp/sparse-fiemap
+++ b/tests/cp/sparse-fiemap
@@ -1,7 +1,7 @@
 #!/bin/sh
 # Test cp --sparse=always through fiemap copy

-# Copyright (C) 2010, 2011 Free Software Foundation, Inc.
+# Copyright (C) 2010-2011 Free Software Foundation, Inc.

 # This program is free software: you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
--
1.7.3.5.44.g960a


>From 3aae01fa9b3759da90854e7423bfc75b8c634afd Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering@redhat.com>
Date: Mon, 31 Jan 2011 14:44:15 +0100
Subject: [PATCH 4/8] tests: factor fiemap-related predicate into init.cfg

* tests/init.cfg (fiemap_capable_): New function.
* tests/cp/fiemap-perf: Use it.
* tests/cp/sparse-fiemap: Likewise.
* tests/cp/fiemap-2: Likewise.
---
 tests/cp/fiemap-2      |    2 +-
 tests/cp/fiemap-perf   |    2 +-
 tests/cp/sparse-fiemap |    2 +-
 tests/init.cfg         |    7 +++++++
 4 files changed, 10 insertions(+), 3 deletions(-)

diff --git a/tests/cp/fiemap-2 b/tests/cp/fiemap-2
index e27f355..a17076c 100755
--- a/tests/cp/fiemap-2
+++ b/tests/cp/fiemap-2
@@ -20,7 +20,7 @@
 print_ver_ cp

 # Require a fiemap-enabled FS.
-df -T -t btrfs -t xfs -t ext4 -t ocfs2 . \
+fiemap_capable_ . \
   || skip_ "this file system lacks FIEMAP support"

 # Exercise the code that handles a file ending in a hole.
diff --git a/tests/cp/fiemap-perf b/tests/cp/fiemap-perf
index bd8fab8..6227243 100755
--- a/tests/cp/fiemap-perf
+++ b/tests/cp/fiemap-perf
@@ -20,7 +20,7 @@
 print_ver_ cp

 # Require a fiemap-enabled FS.
-df -T -t btrfs -t xfs -t ext4 -t ocfs2 . \
+fiemap_capable_ . \
   || skip_ "this file system lacks FIEMAP support"

 # Create a large-but-sparse file.
diff --git a/tests/cp/sparse-fiemap b/tests/cp/sparse-fiemap
index 7e1932b..f224b5b 100755
--- a/tests/cp/sparse-fiemap
+++ b/tests/cp/sparse-fiemap
@@ -19,7 +19,7 @@
 . "${srcdir=.}/init.sh"; path_prepend_ ../src
 print_ver_ cp

-if df -T -t btrfs -t xfs -t ext4 -t ocfs2 . ; then
+if fiemap_capable_ . ; then
   : # Current dir is on a partition with working extents.  Good!
 else
   # It's not;  we need to create one, hence we need root access.
diff --git a/tests/init.cfg b/tests/init.cfg
index a7860e7..fa15aa4 100644
--- a/tests/init.cfg
+++ b/tests/init.cfg
@@ -295,6 +295,13 @@ require_proc_pid_status_()
     kill $pid
 }

+# Return nonzero if the specified directory is on a file system
+# for which FIEMAP support exists.
+fiemap_capable_()
+{
+  df -T -t btrfs -t xfs -t ext4 -t ocfs2 "$@"
+}
+
 # Does the current (working-dir) file system support sparse files?
 require_sparse_support_()
 {
--
1.7.3.5.44.g960a


>From aeed6c1d8981e89d414f6ba0f4dd078a1ab755f0 Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering@redhat.com>
Date: Mon, 31 Jan 2011 15:17:12 +0100
Subject: [PATCH 5/8] maint: remove unused inclusion of "fiemap.h"

* src/copy.c: Don't include "fiemap.h".
---
 src/copy.c |    4 ----
 1 files changed, 0 insertions(+), 4 deletions(-)

diff --git a/src/copy.c b/src/copy.c
index 4e73e1b..d32a11c 100644
--- a/src/copy.c
+++ b/src/copy.c
@@ -63,10 +63,6 @@
 # include "verror.h"
 #endif

-#ifndef HAVE_FIEMAP
-# include "fiemap.h"
-#endif
-
 #ifndef HAVE_FCHOWN
 # define HAVE_FCHOWN false
 # define fchown(fd, uid, gid) (-1)
--
1.7.3.5.44.g960a


>From 96a93f9480e0a420c21db7b58662a6e3ddf16124 Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering@redhat.com>
Date: Mon, 31 Jan 2011 15:20:29 +0100
Subject: [PATCH 6/8] tests: gfs2 also supports FIEMAP

* tests/init.cfg (fiemap_capable_): Add gfs2.
Clarify comment.
---
 tests/init.cfg |    8 +++++---
 1 files changed, 5 insertions(+), 3 deletions(-)

diff --git a/tests/init.cfg b/tests/init.cfg
index fa15aa4..b2d1bab 100644
--- a/tests/init.cfg
+++ b/tests/init.cfg
@@ -295,11 +295,13 @@ require_proc_pid_status_()
     kill $pid
 }

-# Return nonzero if the specified directory is on a file system
-# for which FIEMAP support exists.
+# Return nonzero if the specified directory is on a file system for
+# which FIEMAP support exists, and the file system type is new enough
+# (unlike ext2 and ext3) that it is hard to find an instance *without*
+# FIEMAP support.
 fiemap_capable_()
 {
-  df -T -t btrfs -t xfs -t ext4 -t ocfs2 "$@"
+  df -T -t btrfs -t xfs -t ext4 -t ocfs2 -t gfs2 "$@"
 }

 # Does the current (working-dir) file system support sparse files?
--
1.7.3.5.44.g960a


From MAILER-DAEMON Mon Jan 31 12:31:03 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1Pjxah-0002Nn-Kx
	for mharc-coreutils@gnu.org; Mon, 31 Jan 2011 12:31:03 -0500
Received: from [140.186.70.92] (port=52694 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1Pjxae-0002NV-OD
	for coreutils@gnu.org; Mon, 31 Jan 2011 12:31:02 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Pjxad-0003eB-9Q
	for coreutils@gnu.org; Mon, 31 Jan 2011 12:31:00 -0500
Received: from mx.meyering.net ([82.230.74.64]:46095)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Pjxac-0003dt-SN
	for coreutils@gnu.org; Mon, 31 Jan 2011 12:30:59 -0500
Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000)
	id CF0466017A; Mon, 31 Jan 2011 18:30:57 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: coreutils@gnu.org
Date: Mon, 31 Jan 2011 18:30:57 +0100
Message-ID: <87oc6xyo8e.fsf@meyering.net>
Lines: 178
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
X-Received-From: 82.230.74.64
Subject: [PATCH 7/8] cp: fix copying a sparse file to a pipe
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 31 Jan 2011 17:31:02 -0000

I discovered that my recent changes had broken a corner of cp.
Here's a fix and a test to exercise the bug.

>From cc0645fc643d3ea5a06f12f5fad784a2b8097888 Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering@redhat.com>
Date: Mon, 31 Jan 2011 18:25:58 +0100
Subject: [PATCH 7/8] cp: fix copying a sparse file to a pipe

The recent FIEMAP-related changes made it so the unusual case of
copying a sparse file to a non-regular destination (e.g., a pipe)
would erroneously write one byte too many to that destination.
That happened because extent_copy assumed that it could use lseek
to obtain the number of bytes written to the output file descriptor.
That was valid only for regular files.
* src/copy.c (sparse_copy): Add a parameter, to be used by extent_copy,
but not by reg_copy.  Adjust callers.
(extent_copy): Maintain new local, dest_pos, using new arg, n_read.
Don't call lseek on dest_fd; use new var, dest_pos, instead.
(copy_reg): Add unused arg.
---
 src/copy.c |   27 ++++++++++++++++++++-------
 1 files changed, 20 insertions(+), 7 deletions(-)

diff --git a/src/copy.c b/src/copy.c
index d32a11c..8ba09e0 100644
--- a/src/copy.c
+++ b/src/copy.c
@@ -139,15 +139,18 @@ utimens_symlink (char const *file, struct timespec const *timespec)
    BUF must have sizeof(uintptr_t)-1 bytes of additional space
    beyond BUF[BUF_SIZE-1].
    Set *LAST_WRITE_MADE_HOLE to true if the final operation on
-   DEST_FD introduced a hole.  */
+   DEST_FD introduced a hole.  Set *TOTAL_N_READ to the number of
+   bytes read.  */
 static bool
 sparse_copy (int src_fd, int dest_fd, char *buf, size_t buf_size,
              bool make_holes,
              char const *src_name, char const *dst_name,
-             uintmax_t max_n_read, bool *last_write_made_hole)
+             uintmax_t max_n_read, off_t *total_n_read,
+             bool *last_write_made_hole)
 {
   typedef uintptr_t word;
   *last_write_made_hole = false;
+  *total_n_read = 0;

   while (max_n_read)
     {
@@ -164,6 +167,7 @@ sparse_copy (int src_fd, int dest_fd, char *buf, size_t buf_size,
       if (n_read == 0)
         break;
       max_n_read -= n_read;
+      *total_n_read += n_read;

       if (make_holes)
         {
@@ -313,6 +317,10 @@ extent_copy (int src_fd, int dest_fd, char *buf, size_t buf_size,
   off_t last_ext_start = 0;
   uint64_t last_ext_len = 0;

+  /* Keep track of the output position.
+     We may need this at the end, for a final ftruncate.  */
+  off_t dest_pos = 0;
+
   extent_scan_init (src_fd, &scan);

   bool wrote_hole_at_eof = true;
@@ -378,10 +386,14 @@ extent_copy (int src_fd, int dest_fd, char *buf, size_t buf_size,
           last_ext_start = ext_start;
           last_ext_len = ext_len;

+          off_t n_read;
           if ( ! sparse_copy (src_fd, dest_fd, buf, buf_size,
-                              make_holes, src_name, dst_name, ext_len,
+                              make_holes, src_name, dst_name,
+                              ext_len, &n_read,
                               &wrote_hole_at_eof))
             return false;
+
+          dest_pos = ext_start + n_read;
         }

       /* Release the space allocated to scan->ext_info.  */
@@ -398,11 +410,10 @@ extent_copy (int src_fd, int dest_fd, char *buf, size_t buf_size,
      In addition, if the final extent was a block of zeros at EOF and we've
      just converted them to a hole in the destination, we must call ftruncate
      here in order to record the proper length in the destination.  */
-  off_t dest_len = lseek (dest_fd, 0, SEEK_CUR);
-  if ((dest_len < src_total_size || wrote_hole_at_eof)
+  if ((dest_pos < src_total_size || wrote_hole_at_eof)
       && (make_holes
           ? ftruncate (dest_fd, src_total_size)
-          : ! write_zeros (dest_fd, src_total_size - dest_len)))
+          : ! write_zeros (dest_fd, src_total_size - dest_pos)))
     {
       error (0, errno, _("failed to extend %s"), quote (dst_name));
       return false;
@@ -981,9 +992,11 @@ copy_reg (char const *src_name, char const *dst_name,
           goto close_src_and_dst_desc;
         }

+      off_t n_read;
       bool wrote_hole_at_eof;
       if ( ! sparse_copy (source_desc, dest_desc, buf, buf_size,
-                          make_holes, src_name, dst_name, UINTMAX_MAX,
+                          make_holes, src_name, dst_name,
+                          UINTMAX_MAX, &n_read,
                           &wrote_hole_at_eof)
            || (wrote_hole_at_eof &&
                ! sparse_copy_finalize (dest_desc, dst_name)))
--
1.7.3.5.44.g960a


>From 43d739e99c508a7c0fdfabc3c1aae844b029766f Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering@redhat.com>
Date: Mon, 31 Jan 2011 17:18:13 +0100
Subject: [PATCH 8/8] tests: exercise a rarely-used corner of copy.c

* tests/cp/sparse-to-pipe: New test.
* tests/Makefile.am (TESTS): Add it.
---
 tests/Makefile.am       |    1 +
 tests/cp/sparse-to-pipe |   31 +++++++++++++++++++++++++++++++
 2 files changed, 32 insertions(+), 0 deletions(-)
 create mode 100755 tests/cp/sparse-to-pipe

diff --git a/tests/Makefile.am b/tests/Makefile.am
index 40d35ac..751b327 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -343,6 +343,7 @@ TESTS =						\
   cp/same-file					\
   cp/slink-2-slink				\
   cp/sparse					\
+  cp/sparse-to-pipe				\
   cp/special-f					\
   cp/src-base-dot				\
   cp/symlink-slash				\
diff --git a/tests/cp/sparse-to-pipe b/tests/cp/sparse-to-pipe
new file mode 100755
index 0000000..4d39458
--- /dev/null
+++ b/tests/cp/sparse-to-pipe
@@ -0,0 +1,31 @@
+#!/bin/sh
+# copy a sparse file to a pipe, to exercise some seldom-used parts of copy.c
+
+# Copyright (C) 2011 Free Software Foundation, Inc.
+
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+. "${srcdir=.}/init.sh"; path_prepend_ ../src
+print_ver_ cp
+
+require_sparse_support_
+
+mkfifo_or_skip_ pipe
+timeout 10 cat pipe > copy &
+
+truncate -s1M sparse || framework_failure_
+cp sparse pipe || fail=1
+cmp sparse copy || fail=1
+
+Exit $fail
--
1.7.3.5.44.g960a


From MAILER-DAEMON Mon Jan 31 15:19:21 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1Pk0DZ-0002LD-0T
	for mharc-coreutils@gnu.org; Mon, 31 Jan 2011 15:19:21 -0500
Received: from [140.186.70.92] (port=40906 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1Pk01T-00048C-TB
	for coreutils@gnu.org; Mon, 31 Jan 2011 15:06:52 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <hggdh2@gmail.com>) id 1Pk01S-0005vv-QB
	for coreutils@gnu.org; Mon, 31 Jan 2011 15:06:51 -0500
Received: from mail-vx0-f169.google.com ([209.85.220.169]:45311)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <hggdh2@gmail.com>) id 1Pk01S-0005vJ-JQ
	for coreutils@gnu.org; Mon, 31 Jan 2011 15:06:50 -0500
Received: by vxb41 with SMTP id 41so379361vxb.0
	for <coreutils@gnu.org>; Mon, 31 Jan 2011 12:06:49 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma;
	h=domainkey-signature:sender:message-id:date:from:user-agent
	:mime-version:to:cc:subject:x-enigmail-version:content-type;
	bh=ERdbhNhP4d0RwumVsPUpsJGE2bQWy7YoIXS5tPLYvi4=;
	b=Bn0sbBsBuLfprlnCYPwTbznVq75Q19UB9G22JRX72v6ntqcORIiN0MQf0fRlr0zrM8
	NTtmj6N/ue9Z3jkRhHlRsuVw7ZJFpeJI1oCvf4eAwXi4kVk9VlYoOYop3z+8qatknHUg
	HwKaC0bF/NPXwAkNvtFQSaPk5Bq/Jik7lm1Dk=
DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma;
	h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject
	:x-enigmail-version:content-type;
	b=gweoCeLE+YE4HjNXdzQo43590sebNS8neMxpDzTNHx43vWdVwIz8f1ciA0Ay41RVY0
	Onk54dDnj1hmoKmA6PtZa0rSx+gjh8FzgDtHH7DhTo9j4TWzRTXtDtoDzGDpnRpNsS6P
	V4mSlG3bcCnC1Xxbmosz6eV6Klbr5wToBYcnI=
Received: by 10.220.179.11 with SMTP id bo11mr1751324vcb.249.1296504407798;
	Mon, 31 Jan 2011 12:06:47 -0800 (PST)
Received: from [192.168.2.157] (pool-173-64-203-126.dllstx.fios.verizon.net
	[173.64.203.126])
	by mx.google.com with ESMTPS id i14sm6401080vcr.11.2011.01.31.12.06.46
	(version=SSLv3 cipher=RC4-MD5); Mon, 31 Jan 2011 12:06:46 -0800 (PST)
Sender: hggdh <hggdh2@gmail.com>
Message-ID: <4D471654.1010403@ubuntu.com>
Date: Mon, 31 Jan 2011 14:06:44 -0600
From: C de-Avillez <hggdh2@ubuntu.com>
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US;
	rv:1.9.2.14) Gecko/20110128 Thunderbird/3.1.8 ThunderBrowse/3.3.4
MIME-Version: 1.0
To: coreutils@gnu.org
X-Enigmail-Version: 1.1.2
Content-Type: multipart/signed; micalg=pgp-sha512;
	protocol="application/pgp-signature";
	boundary="------------enig2B7F96496F00964973B5026B"
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 2)
X-Received-From: 209.85.220.169
X-Mailman-Approved-At: Mon, 31 Jan 2011 15:19:19 -0500
Cc: Mike Bianchi <MBianchi@Foveal.com>
Subject: date and fuzzy calculations
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 31 Jan 2011 20:06:53 -0000

This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--------------enig2B7F96496F00964973B5026B
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Hello folks.

We got a bug on Ubuntu [1] where the OP (Mike, copied here) proposes
a rework of the fuzzy 'date' calculations; I am copying his last
comment here:

---- start comment ----

"This explanation does not quite square with the behavior. Witness:

          date
        Mon Jan 31 08:51:30 EST 2011

          date --date=3D'this month' +%j
        031

          date --date=3D'next month' +%j
        062

          date --date=3D'next month'
        Thu Mar 3 08:57:01 EST 2011

So it _appears_ that the algorithm is
        next month -> +1 month
        month -> this month's number of days
        this month's number of days -> 31

        Julian date +=3D 31

I find it hard to accept that the given behavior is "correct" even
if it is
"as documented".

So now the question becomes "am I motivated enough to dig into the code,
understand it thoroughly enough to see what _is_ going on, and then do
something about it."

The simplest correct solution would seem to be:

 manipulations of "month" must take place as manipulations of
 the month field as expressed by the date format %m .

 _if_ the result is an illegal date ( i.e. 31 February )
 _and_ the day of month ( %d ) or julian day ( %j ) will be output,
 _then_
                output an illegal date error
    date --date=3D'next month'
  date: invalid date `31 February'
 _else_
                output
                  date --date=3D'next month' +%B
  February

Would that be deemed acceptable behavior by the keepers of /bin/date ?"

---- end comment ----

Please note I closed the Ubuntu bug WONTFIX, since this is well
documented in both the info pages and on the FAQ.

Still, since Mike is willing to code, I considered it more
productive to contact you folks for a better analysis.

Please copy Mike on the replies, I am not sure he subscribes to the ML.

Cheers,

=2E.C..

[1] https://bugs.launchpad.net/bugs/710368


--------------enig2B7F96496F00964973B5026B
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQIcBAEBCgAGBQJNRxZUAAoJECWAD5grd+mNNPQP/A5oAVY6uA1Vdn03/a0ISAGV
WhCUxMGJird7nEkN8Ukvmpjr6+6elLMm/bHa7Zz6IHIlQfHMZRATQgV3Iz3aaikJ
Ylc6RXZVPEICXaUk83gi4z9tq2Wm9JdeqIfYvNxGJy5rx6ecW+CMGg5kixLEsuNG
idaoCohWNz4Dzvi58iRY6hxfjLvX+8br0tDZ9+ItEoCVC0kyn9Xwowmyb+JDKWSB
BycJB0fPSIlvClXpaD0IAQ3AJjn5dMyamca4IBnQcm43c/wLEGpkFheqvPoCOWTZ
v0TYtZK6oy0KIfAsXL48FZWVkq+U43rtZG+au30Pk7hmsoHLr09etrktef9SDaj+
vkHBN51IYjo//iAwj0cO2Mjdbq15Rzh7QDCfsX142q8gvq8wTSUWJjbrcQ8rH+gk
aaqSnqkUhbVjZPtCAciifuUwWfwykqUmcuAXd4vZKMwxH3AI2uQv75sIFhfHEasG
+hXFaDGg/UzyF7FEkqFN4OuGChhqbzM4OeMwv8Hro+t6B/3toK3UACnpv06zoTZH
A+ptD+ZD7duFaGUupmYP3Qsdk8rxLxSzGSSL09Lc6mdNzXCkv2nrDzAwgL/YHDp2
9O8C+SS3CCco/73jXKvltsDQOqcufRQpHNIuVEOOy+yWwbYzciQ87Bv16QihpD8v
yC+Y2uo1Ehswjh3GFZDc
=jJpW
-----END PGP SIGNATURE-----

--------------enig2B7F96496F00964973B5026B--


From MAILER-DAEMON Mon Jan 31 16:52:26 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1Pk1fe-0001z7-2C
	for mharc-coreutils@gnu.org; Mon, 31 Jan 2011 16:52:26 -0500
Received: from [140.186.70.92] (port=35518 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1Pk1fT-0001CO-Cf
	for coreutils@gnu.org; Mon, 31 Jan 2011 16:52:23 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Pk1aN-0003c2-GY
	for coreutils@gnu.org; Mon, 31 Jan 2011 16:47:00 -0500
Received: from mx.meyering.net ([82.230.74.64]:56129)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Pk1aM-0003bh-Vt
	for coreutils@gnu.org; Mon, 31 Jan 2011 16:46:59 -0500
Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000)
	id 34B7E600ED; Mon, 31 Jan 2011 22:46:58 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: coreutils@gnu.org
Date: Mon, 31 Jan 2011 22:46:58 +0100
Message-ID: <87aaigzqy5.fsf@meyering.net>
Lines: 237
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
X-Received-From: 82.230.74.64
Subject: RFC: new cp option: --efficient-sparse=HOW
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 31 Jan 2011 21:52:23 -0000

Now that we have can read sparse files efficiently,
what if I want to copy a 20PiB sparse file, and yet I want to
be sure that it does so efficiently.  Few people can afford
to wait around while a normal processor and storage system process
that much raw data.  But if it's a sparse file and the src and dest
file systems have the right support (FIEMAP ioctl), then it'll be
copied in the time it takes to make a few syscalls.

Currently, when the efficient sparse copy fails, cp falls back
on the regular, expensive, read-every-byte approach.

This proposal adds an option, --efficient-sparse=required,
to make cp fail if the initial attempt to read the sparse file fails,
rather than resorting to the regular (very slow in the above case) copy
procedure.

The default is --efficient-sparse=auto, and for symmetry,
I've provided --efficient-sparse=never, in case someone finds
a reason to want to skip the ioctl.

You can demonstrate this new feature on a tmpfs file system,
since it supports sparse files, but not the FIEMAP ioctl:

    $ cd /dev/shm
    $ truncate -s128K k
    $ cp --efficient=required k kk
    cp: unable to read sparse `k' efficiently
    [Exit 1]

Here's a preliminary patch
(not including texinfo changes)
I'll add tests, too, of course.

Feedback on the option name or anything else would be most welcome.
The sooner the better.


diff --git a/src/copy.c b/src/copy.c
index 04c678d..72425af 100644
--- a/src/copy.c
+++ b/src/copy.c
@@ -305,8 +305,8 @@ write_zeros (int fd, uint64_t n_bytes)
    copy, and thus makes copying sparse files much more efficient.
    Upon a successful copy, return true.  If the initial extent scan
    fails, set *NORMAL_COPY_REQUIRED to true and return false.
-   Upon any other failure, set *NORMAL_COPY_REQUIRED to false and
-   return false.  */
+   Upon any other failure, give a diagnostic, set *NORMAL_COPY_REQUIRED
+   to false and return false.  */
 static bool
 extent_copy (int src_fd, int dest_fd, char *buf, size_t buf_size,
              off_t src_total_size, bool make_holes,
@@ -931,6 +931,7 @@ copy_reg (char const *src_name, char const *dst_name,
       /* Deal with sparse files.  */
       bool make_holes = false;

+      bool src_is_sparse = false;
       if (S_ISREG (sb.st_mode))
         {
           /* Even with --sparse=always, try to create holes only
@@ -943,9 +944,13 @@ copy_reg (char const *src_name, char const *dst_name,
              blocks.  If the file has fewer blocks than would normally be
              needed for a file of its size, then at least one of the blocks in
              the file is a hole.  */
+
           if (x->sparse_mode == SPARSE_AUTO && S_ISREG (src_open_sb.st_mode)
               && ST_NBLOCKS (src_open_sb) < src_open_sb.st_size / ST_NBLOCKSIZE)
-            make_holes = true;
+            {
+              make_holes = true;
+              src_is_sparse = true;
+            }
 #endif
         }

@@ -977,18 +982,30 @@ copy_reg (char const *src_name, char const *dst_name,
       buf_alloc = xmalloc (buf_size + buf_alignment_slop);
       buf = ptr_align (buf_alloc, buf_alignment);

-      bool normal_copy_required;
+      bool normal_copy_required = true;
       /* Perform an efficient extent-based copy, falling back to the
          standard copy only if the initial extent scan fails.  If the
          '--sparse=never' option is specified, write all data but use
          any extents to read more efficiently.  */
-      if (extent_copy (source_desc, dest_desc, buf, buf_size,
-                       src_open_sb.st_size, make_holes,
-                       src_name, dst_name, &normal_copy_required))
+
+      if (x->sparse_efficiency != SPARSE_EFF_NEVER
+          && extent_copy (source_desc, dest_desc, buf, buf_size,
+                          src_open_sb.st_size, make_holes,
+                          src_name, dst_name, &normal_copy_required))
         goto preserve_metadata;

       if (! normal_copy_required)
         {
+          /* extent_copy already diagnosed the failure */
+          return_val = false;
+          goto close_src_and_dst_desc;
+        }
+
+      /* extent_copy failed, and we are instructed not to fall-back */
+      if (src_is_sparse && x->sparse_efficiency == SPARSE_EFF_REQUIRED)
+        {
+          error (0, 0, _("unable to read sparse %s efficiently"),
+                 quote (src_name));
           return_val = false;
           goto close_src_and_dst_desc;
         }
@@ -2519,6 +2536,7 @@ cp_options_default (struct cp_options *x)
 #else
   x->chown_privileges = x->owner_privileges = (geteuid () == 0);
 #endif
+  x->sparse_efficiency = SPARSE_EFF_AUTO;
 }

 /* Return true if it's OK for chown to fail, where errno is
diff --git a/src/copy.h b/src/copy.h
index 5014ea9..fab131b 100644
--- a/src/copy.h
+++ b/src/copy.h
@@ -22,6 +22,22 @@
 # include <stdbool.h>
 # include "hash.h"

+/* Control efficient reading of sparse files.  On some systems, you can
+   use the FIEMAP ioctl to read only the non-sparse parts of a file.  */
+enum Sparse_efficiency
+{
+  /* Do not attempt to treat sparse source files specially.  */
+  SPARSE_EFF_NEVER,
+
+  /* Attempt to read sparse files efficiently, but if that is not
+     possible, fall back on the regular, less-efficient approach.  */
+  SPARSE_EFF_AUTO,
+
+  /* Read sparse files efficiently, and if that is not possible,
+     then treat it as failure to copy.  */
+  SPARSE_EFF_REQUIRED
+};
+
 /* Control creation of sparse files (files with holes).  */
 enum Sparse_type
 {
@@ -110,6 +126,9 @@ struct cp_options
   /* Control creation of sparse files.  */
   enum Sparse_type sparse_mode;

+  /* Control efficient reading of sparse files.  */
+  enum Sparse_efficiency sparse_efficiency;
+
   /* Set the mode of the destination file to exactly this value
      if SET_MODE is nonzero.  */
   mode_t mode;
diff --git a/src/cp.c b/src/cp.c
index 859f21b..711e229 100644
--- a/src/cp.c
+++ b/src/cp.c
@@ -74,6 +74,7 @@ enum
 {
   ATTRIBUTES_ONLY_OPTION = CHAR_MAX + 1,
   COPY_CONTENTS_OPTION,
+  EFFICIENT_SPARSE_OPTION,
   NO_PRESERVE_ATTRIBUTES_OPTION,
   PARENTS_OPTION,
   PRESERVE_ATTRIBUTES_OPTION,
@@ -93,6 +94,16 @@ static bool parents_option = false;
 /* Remove any trailing slashes from each SOURCE argument.  */
 static bool remove_trailing_slashes;

+static char const *const eff_sparse_type_string[] =
+{
+  "never", "auto", "required", NULL
+};
+static enum Sparse_type const eff_sparse_type[] =
+{
+  SPARSE_EFF_NEVER, SPARSE_EFF_AUTO, SPARSE_EFF_REQUIRED
+};
+ARGMATCH_VERIFY (eff_sparse_type_string, eff_sparse_type);
+
 static char const *const sparse_type_string[] =
 {
   "never", "auto", "always", NULL
@@ -120,6 +131,7 @@ static struct option const long_opts[] =
   {"backup", optional_argument, NULL, 'b'},
   {"copy-contents", no_argument, NULL, COPY_CONTENTS_OPTION},
   {"dereference", no_argument, NULL, 'L'},
+  {"efficient-sparse", required_argument, NULL, EFFICIENT_SPARSE_OPTION},
   {"force", no_argument, NULL, 'f'},
   {"interactive", no_argument, NULL, 'i'},
   {"link", no_argument, NULL, 'l'},
@@ -177,6 +189,9 @@ Mandatory arguments to long options are mandatory for short options too.\n\
   -d                           same as --no-dereference --preserve=links\n\
 "), stdout);
       fputs (_("\
+      --efficient-sparse=HOW   control efficient reading of sparse files.\n\
+"), stdout);
+      fputs (_("\
   -f, --force                  if an existing destination file cannot be\n\
                                  opened, remove it and try again (redundant if\
 \n\
@@ -247,6 +262,21 @@ fails, or if --reflink=auto is specified, fall back to a standard copy.\n\
 "), stdout);
       fputs (_("\
 \n\
+By default, cp tries to read sparse SOURCE files efficiently, but if the\n\
+required capability is not available it resorts to copying the usual way.\n\
+--efficient-sparse=auto is the default.  One case in which you would not\n\
+want to fall back on the usual method is when you are copying a very large,\n\
+mostly-sparse file, and processing all bytes in the nominal size would take\n\
+too long.\
+"), stdout);
+      fputs (_("\
+  In that case, use --efficient-sparse=required to make cp fail if\n\
+the efficient method does not work.  I.e., tell cp not to resort to the\n\
+less-efficient method. Finally, --efficient-sparse=never makes cp skip the\n\
+attempt to copy efficiently.\n\
+"), stdout);
+      fputs (_("\
+\n\
 The backup suffix is `~', unless set with --suffix or SIMPLE_BACKUP_SUFFIX.\n\
 The version control method may be selected via the --backup option or through\n\
 the VERSION_CONTROL environment variable.  Here are the values:\n\
@@ -944,6 +974,12 @@ main (int argc, char **argv)
                                      sparse_type_string, sparse_type);
           break;

+        case EFFICIENT_SPARSE_OPTION:
+          x.sparse_efficiency = XARGMATCH ("--efficient-sparse", optarg,
+                                           eff_sparse_type_string,
+                                           eff_sparse_type);
+          break;
+
         case REFLINK_OPTION:
           if (optarg == NULL)
             x.reflink_mode = REFLINK_ALWAYS;


From MAILER-DAEMON Mon Jan 31 16:52:32 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1Pk1fj-00022H-Hs
	for mharc-coreutils@gnu.org; Mon, 31 Jan 2011 16:52:31 -0500
Received: from [140.186.70.92] (port=42741 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1Pk1fa-0001RZ-Bb
	for coreutils@gnu.org; Mon, 31 Jan 2011 16:52:30 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Pk1Y2-0003JY-Ar
	for coreutils@gnu.org; Mon, 31 Jan 2011 16:44:35 -0500
Received: from mx.meyering.net ([82.230.74.64]:43539)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Pk1Y2-0003J6-1v
	for coreutils@gnu.org; Mon, 31 Jan 2011 16:44:34 -0500
Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000)
	id 0400B600ED; Mon, 31 Jan 2011 22:44:30 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: coreutils@gnu.org
Date: Mon, 31 Jan 2011 22:44:30 +0100
Message-ID: <87d3nczr29.fsf@meyering.net>
Lines: 26
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
X-Received-From: 82.230.74.64
Subject: [PATCH] cp: always initialize extent_copy's output parameter
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 31 Jan 2011 21:52:30 -0000

Here's another bug fix:

>From f8f57dd76ba98793e131a88beb138a32d54e0aa9 Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering@redhat.com>
Date: Mon, 31 Jan 2011 20:55:34 +0100
Subject: [PATCH] cp: always initialize extent_copy's output parameter

* src/copy.c (extent_copy): Otherwise it would be used uninitialized.
---
 src/copy.c |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/src/copy.c b/src/copy.c
index 8ba09e0..04c678d 100644
--- a/src/copy.c
+++ b/src/copy.c
@@ -323,6 +323,7 @@ extent_copy (int src_fd, int dest_fd, char *buf, size_t buf_size,

   extent_scan_init (src_fd, &scan);

+  *require_normal_copy = false;
   bool wrote_hole_at_eof = true;
   do
     {
--
1.7.3.5.44.g960a


From MAILER-DAEMON Mon Jan 31 17:14:43 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1Pk21C-0001sx-LI
	for mharc-coreutils@gnu.org; Mon, 31 Jan 2011 17:14:42 -0500
Received: from [140.186.70.92] (port=55625 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1Pk217-0001ol-Rw
	for coreutils@gnu.org; Mon, 31 Jan 2011 17:14:41 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Pk214-00014I-El
	for coreutils@gnu.org; Mon, 31 Jan 2011 17:14:36 -0500
Received: from mx.meyering.net ([82.230.74.64]:41151)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Pk213-000148-Ut
	for coreutils@gnu.org; Mon, 31 Jan 2011 17:14:34 -0500
Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000)
	id 3397A600ED; Mon, 31 Jan 2011 23:14:32 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: coreutils@gnu.org
In-Reply-To: <87aaigzqy5.fsf@meyering.net> (Jim Meyering's message of "Mon, 31
	Jan 2011 22:46:58 +0100")
References: <87aaigzqy5.fsf@meyering.net>
Date: Mon, 31 Jan 2011 23:14:32 +0100
Message-ID: <874o8ozpo7.fsf@meyering.net>
Lines: 291
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
X-Received-From: 82.230.74.64
Subject: Re: RFC: new cp option: --efficient-sparse=HOW
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 31 Jan 2011 22:14:41 -0000

Jim Meyering wrote:

> Now that we have can read sparse files efficiently,
> what if I want to copy a 20PiB sparse file, and yet I want to
> be sure that it does so efficiently.  Few people can afford
> to wait around while a normal processor and storage system process
> that much raw data.  But if it's a sparse file and the src and dest
> file systems have the right support (FIEMAP ioctl), then it'll be
> copied in the time it takes to make a few syscalls.
>
> Currently, when the efficient sparse copy fails, cp falls back
> on the regular, expensive, read-every-byte approach.
>
> This proposal adds an option, --efficient-sparse=required,
> to make cp fail if the initial attempt to read the sparse file fails,
> rather than resorting to the regular (very slow in the above case) copy
> procedure.
>
> The default is --efficient-sparse=auto, and for symmetry,
> I've provided --efficient-sparse=never, in case someone finds
> a reason to want to skip the ioctl.
>
> You can demonstrate this new feature on a tmpfs file system,
> since it supports sparse files, but not the FIEMAP ioctl:
>
>     $ cd /dev/shm
>     $ truncate -s128K k
>     $ cp --efficient=required k kk
>     cp: unable to read sparse `k' efficiently
>     [Exit 1]
>
> Here's a preliminary patch
> (not including texinfo changes)
> I'll add tests, too, of course.

And NEWS.

Here's that same patch, but now with a proper ChangeLog:

>From c83ea420c64169a7db58189cce6d3e755eb7b717 Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering@redhat.com>
Date: Mon, 31 Jan 2011 23:13:36 +0100
Subject: [PATCH] cp: support new option: --efficient-sparse=HOW

Now that we have can read sparse files efficiently,
what if I want to copy a 20PiB sparse file, and yet I want to
be sure that it does so efficiently?  Few people can afford
to wait around while a normal processor and storage system process
that much raw data.  But if it's a sparse file and the src and dest
file systems have the right support (FIEMAP ioctl), then it'll be
copied in the time it takes to make a few syscalls.

Currently, when the efficient sparse copy fails, cp falls back
on the regular, expensive, read-every-byte approach.

This proposal adds an option, --efficient-sparse=required,
to make cp fail if the initial attempt to read the sparse file fails,
rather than resorting to the regular (very slow in the above case) copy
procedure.

The default is --efficient-sparse=auto, and for symmetry,
I've provided --efficient-sparse=never, in case someone finds
a reason to want to skip the ioctl.

You can demonstrate this new feature on a tmpfs file system,
since it supports sparse files, but not the FIEMAP ioctl:

    $ cd /dev/shm
    $ truncate -s128K k
    $ cp --efficient=required k kk
    cp: unable to read sparse `k' efficiently
    [Exit 1]

* src/copy.h (enum Sparse_efficiency): Declare.
(struct cp_options) [sparse_efficiency]: New member.
* src/copy.c (word, cp_options_default):
(extent_copy): Add description for a parameter.
(copy_reg): Remember result of src_is_sparse heuristic...
and test that when extent_copy fails.
Don't call extent_copy for SPARSE_EFF_NEVER.
(cp_options_default): Initialize new member.
* src/cp.c (eff_sparse_type, long_opts, main): Support new options.
(usage): Document them.
---
 src/copy.c |   32 +++++++++++++++++++++++++-------
 src/copy.h |   19 +++++++++++++++++++
 src/cp.c   |   36 ++++++++++++++++++++++++++++++++++++
 3 files changed, 80 insertions(+), 7 deletions(-)

diff --git a/src/copy.c b/src/copy.c
index 04c678d..72425af 100644
--- a/src/copy.c
+++ b/src/copy.c
@@ -305,8 +305,8 @@ write_zeros (int fd, uint64_t n_bytes)
    copy, and thus makes copying sparse files much more efficient.
    Upon a successful copy, return true.  If the initial extent scan
    fails, set *NORMAL_COPY_REQUIRED to true and return false.
-   Upon any other failure, set *NORMAL_COPY_REQUIRED to false and
-   return false.  */
+   Upon any other failure, give a diagnostic, set *NORMAL_COPY_REQUIRED
+   to false and return false.  */
 static bool
 extent_copy (int src_fd, int dest_fd, char *buf, size_t buf_size,
              off_t src_total_size, bool make_holes,
@@ -931,6 +931,7 @@ copy_reg (char const *src_name, char const *dst_name,
       /* Deal with sparse files.  */
       bool make_holes = false;

+      bool src_is_sparse = false;
       if (S_ISREG (sb.st_mode))
         {
           /* Even with --sparse=always, try to create holes only
@@ -943,9 +944,13 @@ copy_reg (char const *src_name, char const *dst_name,
              blocks.  If the file has fewer blocks than would normally be
              needed for a file of its size, then at least one of the blocks in
              the file is a hole.  */
+
           if (x->sparse_mode == SPARSE_AUTO && S_ISREG (src_open_sb.st_mode)
               && ST_NBLOCKS (src_open_sb) < src_open_sb.st_size / ST_NBLOCKSIZE)
-            make_holes = true;
+            {
+              make_holes = true;
+              src_is_sparse = true;
+            }
 #endif
         }

@@ -977,18 +982,30 @@ copy_reg (char const *src_name, char const *dst_name,
       buf_alloc = xmalloc (buf_size + buf_alignment_slop);
       buf = ptr_align (buf_alloc, buf_alignment);

-      bool normal_copy_required;
+      bool normal_copy_required = true;
       /* Perform an efficient extent-based copy, falling back to the
          standard copy only if the initial extent scan fails.  If the
          '--sparse=never' option is specified, write all data but use
          any extents to read more efficiently.  */
-      if (extent_copy (source_desc, dest_desc, buf, buf_size,
-                       src_open_sb.st_size, make_holes,
-                       src_name, dst_name, &normal_copy_required))
+
+      if (x->sparse_efficiency != SPARSE_EFF_NEVER
+          && extent_copy (source_desc, dest_desc, buf, buf_size,
+                          src_open_sb.st_size, make_holes,
+                          src_name, dst_name, &normal_copy_required))
         goto preserve_metadata;

       if (! normal_copy_required)
         {
+          /* extent_copy already diagnosed the failure */
+          return_val = false;
+          goto close_src_and_dst_desc;
+        }
+
+      /* extent_copy failed, and we are instructed not to fall-back */
+      if (src_is_sparse && x->sparse_efficiency == SPARSE_EFF_REQUIRED)
+        {
+          error (0, 0, _("unable to read sparse %s efficiently"),
+                 quote (src_name));
           return_val = false;
           goto close_src_and_dst_desc;
         }
@@ -2519,6 +2536,7 @@ cp_options_default (struct cp_options *x)
 #else
   x->chown_privileges = x->owner_privileges = (geteuid () == 0);
 #endif
+  x->sparse_efficiency = SPARSE_EFF_AUTO;
 }

 /* Return true if it's OK for chown to fail, where errno is
diff --git a/src/copy.h b/src/copy.h
index 5014ea9..fab131b 100644
--- a/src/copy.h
+++ b/src/copy.h
@@ -22,6 +22,22 @@
 # include <stdbool.h>
 # include "hash.h"

+/* Control efficient reading of sparse files.  On some systems, you can
+   use the FIEMAP ioctl to read only the non-sparse parts of a file.  */
+enum Sparse_efficiency
+{
+  /* Do not attempt to treat sparse source files specially.  */
+  SPARSE_EFF_NEVER,
+
+  /* Attempt to read sparse files efficiently, but if that is not
+     possible, fall back on the regular, less-efficient approach.  */
+  SPARSE_EFF_AUTO,
+
+  /* Read sparse files efficiently, and if that is not possible,
+     then treat it as failure to copy.  */
+  SPARSE_EFF_REQUIRED
+};
+
 /* Control creation of sparse files (files with holes).  */
 enum Sparse_type
 {
@@ -110,6 +126,9 @@ struct cp_options
   /* Control creation of sparse files.  */
   enum Sparse_type sparse_mode;

+  /* Control efficient reading of sparse files.  */
+  enum Sparse_efficiency sparse_efficiency;
+
   /* Set the mode of the destination file to exactly this value
      if SET_MODE is nonzero.  */
   mode_t mode;
diff --git a/src/cp.c b/src/cp.c
index 859f21b..711e229 100644
--- a/src/cp.c
+++ b/src/cp.c
@@ -74,6 +74,7 @@ enum
 {
   ATTRIBUTES_ONLY_OPTION = CHAR_MAX + 1,
   COPY_CONTENTS_OPTION,
+  EFFICIENT_SPARSE_OPTION,
   NO_PRESERVE_ATTRIBUTES_OPTION,
   PARENTS_OPTION,
   PRESERVE_ATTRIBUTES_OPTION,
@@ -93,6 +94,16 @@ static bool parents_option = false;
 /* Remove any trailing slashes from each SOURCE argument.  */
 static bool remove_trailing_slashes;

+static char const *const eff_sparse_type_string[] =
+{
+  "never", "auto", "required", NULL
+};
+static enum Sparse_type const eff_sparse_type[] =
+{
+  SPARSE_EFF_NEVER, SPARSE_EFF_AUTO, SPARSE_EFF_REQUIRED
+};
+ARGMATCH_VERIFY (eff_sparse_type_string, eff_sparse_type);
+
 static char const *const sparse_type_string[] =
 {
   "never", "auto", "always", NULL
@@ -120,6 +131,7 @@ static struct option const long_opts[] =
   {"backup", optional_argument, NULL, 'b'},
   {"copy-contents", no_argument, NULL, COPY_CONTENTS_OPTION},
   {"dereference", no_argument, NULL, 'L'},
+  {"efficient-sparse", required_argument, NULL, EFFICIENT_SPARSE_OPTION},
   {"force", no_argument, NULL, 'f'},
   {"interactive", no_argument, NULL, 'i'},
   {"link", no_argument, NULL, 'l'},
@@ -177,6 +189,9 @@ Mandatory arguments to long options are mandatory for short options too.\n\
   -d                           same as --no-dereference --preserve=links\n\
 "), stdout);
       fputs (_("\
+      --efficient-sparse=HOW   control efficient reading of sparse files.\n\
+"), stdout);
+      fputs (_("\
   -f, --force                  if an existing destination file cannot be\n\
                                  opened, remove it and try again (redundant if\
 \n\
@@ -247,6 +262,21 @@ fails, or if --reflink=auto is specified, fall back to a standard copy.\n\
 "), stdout);
       fputs (_("\
 \n\
+By default, cp tries to read sparse SOURCE files efficiently, but if the\n\
+required capability is not available it resorts to copying the usual way.\n\
+--efficient-sparse=auto is the default.  One case in which you would not\n\
+want to fall back on the usual method is when you are copying a very large,\n\
+mostly-sparse file, and processing all bytes in the nominal size would take\n\
+too long.\
+"), stdout);
+      fputs (_("\
+  In that case, use --efficient-sparse=required to make cp fail if\n\
+the efficient method does not work.  I.e., tell cp not to resort to the\n\
+less-efficient method. Finally, --efficient-sparse=never makes cp skip the\n\
+attempt to copy efficiently.\n\
+"), stdout);
+      fputs (_("\
+\n\
 The backup suffix is `~', unless set with --suffix or SIMPLE_BACKUP_SUFFIX.\n\
 The version control method may be selected via the --backup option or through\n\
 the VERSION_CONTROL environment variable.  Here are the values:\n\
@@ -944,6 +974,12 @@ main (int argc, char **argv)
                                      sparse_type_string, sparse_type);
           break;

+        case EFFICIENT_SPARSE_OPTION:
+          x.sparse_efficiency = XARGMATCH ("--efficient-sparse", optarg,
+                                           eff_sparse_type_string,
+                                           eff_sparse_type);
+          break;
+
         case REFLINK_OPTION:
           if (optarg == NULL)
             x.reflink_mode = REFLINK_ALWAYS;
--
1.7.3.5.44.g960a


From MAILER-DAEMON Mon Jan 31 17:18:00 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1Pk24O-0002kw-D4
	for mharc-coreutils@gnu.org; Mon, 31 Jan 2011 17:18:00 -0500
Received: from [140.186.70.92] (port=56744 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1Pk24L-0002kX-9h
	for coreutils@gnu.org; Mon, 31 Jan 2011 17:17:59 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1Pk24I-0001Zu-3D
	for coreutils@gnu.org; Mon, 31 Jan 2011 17:17:55 -0500
Received: from mx1.redhat.com ([209.132.183.28]:12291)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1Pk24H-0001Zo-Qo
	for coreutils@gnu.org; Mon, 31 Jan 2011 17:17:54 -0500
Received: from int-mx12.intmail.prod.int.phx2.redhat.com
	(int-mx12.intmail.prod.int.phx2.redhat.com [10.5.11.25])
	by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id p0VMHqlH028032
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK);
	Mon, 31 Jan 2011 17:17:53 -0500
Received: from [10.3.113.114] (ovpn-113-114.phx2.redhat.com [10.3.113.114])
	by int-mx12.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP
	id p0VMHqQZ019601; Mon, 31 Jan 2011 17:17:52 -0500
Message-ID: <4D47350F.8060407@redhat.com>
Date: Mon, 31 Jan 2011 15:17:51 -0700
From: Eric Blake <eblake@redhat.com>
Organization: Red Hat
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US;
	rv:1.9.2.13) Gecko/20101209 Fedora/3.1.7-0.35.b3pre.fc14
	Lightning/1.0b3pre Mnenhy/0.8.3 Thunderbird/3.1.7
MIME-Version: 1.0
To: Jim Meyering <jim@meyering.net>
References: <87aaigzqy5.fsf@meyering.net>
In-Reply-To: <87aaigzqy5.fsf@meyering.net>
X-Enigmail-Version: 1.1.2
OpenPGP: url=http://people.redhat.com/eblake/eblake.gpg
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature";
	boundary="------------enigB447D60EE392ACD149C8B3EE"
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.25
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.132.183.28
Cc: coreutils@gnu.org
Subject: Re: RFC: new cp option: --efficient-sparse=HOW
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 31 Jan 2011 22:17:59 -0000

This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--------------enigB447D60EE392ACD149C8B3EE
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

On 01/31/2011 02:46 PM, Jim Meyering wrote:
> Now that we have can read sparse files efficiently,
> what if I want to copy a 20PiB sparse file, and yet I want to
> be sure that it does so efficiently.  Few people can afford
> to wait around while a normal processor and storage system process
> that much raw data.  But if it's a sparse file and the src and dest
> file systems have the right support (FIEMAP ioctl), then it'll be
> copied in the time it takes to make a few syscalls.
>=20
> Currently, when the efficient sparse copy fails, cp falls back
> on the regular, expensive, read-every-byte approach.
>=20
> This proposal adds an option, --efficient-sparse=3Drequired,
> to make cp fail if the initial attempt to read the sparse file fails,
> rather than resorting to the regular (very slow in the above case) copy=

> procedure.
>=20
> The default is --efficient-sparse=3Dauto, and for symmetry,
> I've provided --efficient-sparse=3Dnever, in case someone finds
> a reason to want to skip the ioctl.

Conversely, what happens if I have a file that contains large blocks of
zeros but is NOT fully sparse (plausible, since we're still facing the
fact that it is still not easy to punch holes into existing files when
data in that portion of the file is no longer needed)?  Does all the new
fiemap code still have the ability for me to request that the cp code
specifically look for large blocks of zero in the source, rather than
trusting the fiemap, so that I can create a copy that is more sparse
than the original?  Does that also need a tunable; and if so, should we
try to combine it into this tunable or is it orthogonal?

--=20
Eric Blake   eblake@redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org


--------------enigB447D60EE392ACD149C8B3EE
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Public key at http://people.redhat.com/eblake/eblake.gpg
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/

iQEcBAEBCAAGBQJNRzUPAAoJEKeha0olJ0NqdfgH/0arQ9XneF23z8FG3S1Xu53Y
VvJTxIcs3iYDhPArhDBHbtfHyDljYbQ90sonRHxqxki8DfYZzEfgBkXOtqSEZU+5
c6wAvM+XwV0j8K9EdjAiWrIAz3n7OVdOCyf3LLmgq/ssWD1QoCZ5SLJwHD2SNK1c
yfzgGSP3MvwYM7DP2X+k4/MwlbzJJhNEIUtCoR2qHgzJh+4eRdzkB27QIyiYeYjC
N35ORR2QH8mP8MQJAzvBtJdTOhR1h4fkCLkAm4B+E8yW4bUIuh0zUAT9dyd1XKXF
VD+1Eo7ypEPNXeZhwpPegtoAkNzZ36DtrSk4uH2Ja+0rmndTkzGR7dMKnF7nQmY=
=ecu7
-----END PGP SIGNATURE-----

--------------enigB447D60EE392ACD149C8B3EE--


From MAILER-DAEMON Mon Jan 31 17:27:52 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1Pk2Dw-0005BZ-5g
	for mharc-coreutils@gnu.org; Mon, 31 Jan 2011 17:27:52 -0500
Received: from [140.186.70.92] (port=43091 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1Pk2Ds-0005A6-Vv
	for coreutils@gnu.org; Mon, 31 Jan 2011 17:27:50 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Pk2Dq-0003He-I6
	for coreutils@gnu.org; Mon, 31 Jan 2011 17:27:47 -0500
Received: from mx.meyering.net ([82.230.74.64]:43326)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Pk2Dq-0003HQ-BP
	for coreutils@gnu.org; Mon, 31 Jan 2011 17:27:46 -0500
Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000)
	id D0F1460131; Mon, 31 Jan 2011 23:27:44 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: Eric Blake <eblake@redhat.com>
In-Reply-To: <4D47350F.8060407@redhat.com> (Eric Blake's message of "Mon, 31
	Jan 2011 15:17:51 -0700")
References: <87aaigzqy5.fsf@meyering.net> <4D47350F.8060407@redhat.com>
Date: Mon, 31 Jan 2011 23:27:44 +0100
Message-ID: <87wrlkyahr.fsf@meyering.net>
Lines: 53
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
X-Received-From: 82.230.74.64
Cc: coreutils@gnu.org
Subject: Re: RFC: new cp option: --efficient-sparse=HOW
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 31 Jan 2011 22:27:50 -0000

Eric Blake wrote:

> On 01/31/2011 02:46 PM, Jim Meyering wrote:
>> Now that we have can read sparse files efficiently,
>> what if I want to copy a 20PiB sparse file, and yet I want to
>> be sure that it does so efficiently.  Few people can afford
>> to wait around while a normal processor and storage system process
>> that much raw data.  But if it's a sparse file and the src and dest
>> file systems have the right support (FIEMAP ioctl), then it'll be
>> copied in the time it takes to make a few syscalls.
>>
>> Currently, when the efficient sparse copy fails, cp falls back
>> on the regular, expensive, read-every-byte approach.
>>
>> This proposal adds an option, --efficient-sparse=required,
>> to make cp fail if the initial attempt to read the sparse file fails,
>> rather than resorting to the regular (very slow in the above case) copy
>> procedure.
>>
>> The default is --efficient-sparse=auto, and for symmetry,
>> I've provided --efficient-sparse=never, in case someone finds
>> a reason to want to skip the ioctl.
>
> Conversely, what happens if I have a file that contains large blocks of
> zeros but is NOT fully sparse (plausible, since we're still facing the
> fact that it is still not easy to punch holes into existing files when
> data in that portion of the file is no longer needed)?  Does all the new
> fiemap code still have the ability for me to request that the cp code
> specifically look for large blocks of zero in the source, rather than
> trusting the fiemap, so that I can create a copy that is more sparse
> than the original?  Does that also need a tunable; and if so, should we
> try to combine it into this tunable or is it orthogonal?

It's orthogonal.

--sparse=always still does the hole-punching, independently
of whether we're copying normally or via the efficient FIEMAP-based
code.

E.g., if you have a sparse file, where one non-sparse chunk
contains all-zero blocks (currently 32KiB minimum), then --sparse=always
will convert those blocks to holes, with or without
--efficient-sparse=never.

--efficient-sparse=... controls efficiency while reading
--sparse=...           controls hole-punching (or preserving)

BTW, that the existing hole-punching behavior works for no
sequence shorter than 32KiB is a bug that I will fix very soon.
I think that was introduced as an unwanted side-effect when
increasing buffer size for efficiency.

Thanks for the feedback.


From MAILER-DAEMON Mon Jan 31 17:29:38 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1Pk2Fe-0005Ud-73
	for mharc-coreutils@gnu.org; Mon, 31 Jan 2011 17:29:38 -0500
Received: from [140.186.70.92] (port=42479 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1Pk2Fa-0005TO-Vc
	for coreutils@gnu.org; Mon, 31 Jan 2011 17:29:37 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1Pk2FZ-0003aK-Hd
	for coreutils@gnu.org; Mon, 31 Jan 2011 17:29:34 -0500
Received: from mx1.redhat.com ([209.132.183.28]:2517)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1Pk2FZ-0003aB-9T
	for coreutils@gnu.org; Mon, 31 Jan 2011 17:29:33 -0500
Received: from int-mx02.intmail.prod.int.phx2.redhat.com
	(int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12])
	by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id p0VMTVBw003035
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK);
	Mon, 31 Jan 2011 17:29:32 -0500
Received: from [10.3.113.114] (ovpn-113-114.phx2.redhat.com [10.3.113.114])
	by int-mx02.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP
	id p0VMTVPD011721; Mon, 31 Jan 2011 17:29:31 -0500
Message-ID: <4D4737CA.7030405@redhat.com>
Date: Mon, 31 Jan 2011 15:29:30 -0700
From: Eric Blake <eblake@redhat.com>
Organization: Red Hat
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US;
	rv:1.9.2.13) Gecko/20101209 Fedora/3.1.7-0.35.b3pre.fc14
	Lightning/1.0b3pre Mnenhy/0.8.3 Thunderbird/3.1.7
MIME-Version: 1.0
References: <87aaigzqy5.fsf@meyering.net> <4D47350F.8060407@redhat.com>
In-Reply-To: <4D47350F.8060407@redhat.com>
X-Enigmail-Version: 1.1.2
OpenPGP: url=http://people.redhat.com/eblake/eblake.gpg
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature";
	boundary="------------enigAEA634CC2395B22902EB5BE5"
X-Scanned-By: MIMEDefang 2.67 on 10.5.11.12
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.132.183.28
Cc: coreutils@gnu.org
Subject: Re: RFC: new cp option: --efficient-sparse=HOW
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 31 Jan 2011 22:29:37 -0000

This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--------------enigAEA634CC2395B22902EB5BE5
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

On 01/31/2011 03:17 PM, Eric Blake wrote:
>> The default is --efficient-sparse=3Dauto, and for symmetry,
>> I've provided --efficient-sparse=3Dnever, in case someone finds
>> a reason to want to skip the ioctl.
>=20
> Conversely, what happens if I have a file that contains large blocks of=

> zeros but is NOT fully sparse (plausible, since we're still facing the
> fact that it is still not easy to punch holes into existing files when
> data in that portion of the file is no longer needed)?  Does all the ne=
w
> fiemap code still have the ability for me to request that the cp code
> specifically look for large blocks of zero in the source, rather than
> trusting the fiemap, so that I can create a copy that is more sparse
> than the original?  Does that also need a tunable; and if so, should we=

> try to combine it into this tunable or is it orthogonal?

Maybe something like this matrix, introducing --sparse=3Dsearch as a step=

beyond --sparse=3Dalways to request that the (non-sparse) portions of the=

input be specifically searched for possible further blocks of 0 that can
be made sparse in the destination:

--sparse=3Dalways --efficient-sparse=3Dalways
    require fiemap, destination will only be as sparse as input
--sparse=3Dsearch --efficient-sparse=3Dalways
    require fiemap, destination might be more sparse than input
--sparse=3Dalways --efficient-sparse=3Dauto
    if fiemap is supported, destination will be as sparse as input
    if fiemap is not supported, destination might be more sparse
--sparse=3Dsearch --efficient-sparse=3Dauto
    destination might be more sparse than input
--sparse=3Dauto --efficient-sparse=3Dalways
    if input is sparse, require fiemap, destination will match input
sparseness
--sparse=3Dauto --efficient-sparse=3Dauto
    destination will be sparse if input was sparse; if fiemap is
supported then destination will be equally sparse, otherwise, it might
be more sparse than the input

--=20
Eric Blake   eblake@redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org


--------------enigAEA634CC2395B22902EB5BE5
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Public key at http://people.redhat.com/eblake/eblake.gpg
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/

iQEcBAEBCAAGBQJNRzfKAAoJEKeha0olJ0NqCIUH/0eXDS0QaeuACAFeLSR03PSr
eY9HUwej8aSsc1n7C8bKr1kNEo/D3lUjJHsa8it9G9FcZsGt1DbI7uFcZBcR5jFg
y9kYhrNhiFRYAsM9BIyxe+p8KIcKKPciX8kSjagDFE0p1EWgEl3kmEkPu6dVlWF1
qvZSsvMOW2Ks/E08WOg+nwzhwyDhNTJ6tLaRm2RYHbIS5e8fr9G/b4Ly5Sq0FMjg
mFtm4m8q7RKSicsxRJdpzlghtT2Uk8TSBDnh49xlxGCet+En/uVcr5oTz8zpShGN
RgiGWILpbB8s8X3PfoTnoUjbxjMdS1CfMWSV2bO81k4PrGgH69iSidPl6Mzkn8M=
=cCDO
-----END PGP SIGNATURE-----

--------------enigAEA634CC2395B22902EB5BE5--


From MAILER-DAEMON Mon Jan 31 17:53:42 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1Pk2cv-0001yV-Rb
	for mharc-coreutils@gnu.org; Mon, 31 Jan 2011 17:53:41 -0500
Received: from [140.186.70.92] (port=44494 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1Pk1z9-0001D4-CM
	for coreutils@gnu.org; Mon, 31 Jan 2011 17:12:36 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Pk1z8-0000dg-9S
	for coreutils@gnu.org; Mon, 31 Jan 2011 17:12:35 -0500
Received: from mail1.slb.deg.dub.stisp.net ([84.203.253.98]:9880)
	by eggs.gnu.org with smtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Pk1z7-0000cW-UV
	for coreutils@gnu.org; Mon, 31 Jan 2011 17:12:34 -0500
Received: (qmail 46287 invoked from network); 31 Jan 2011 22:12:31 -0000
Received: from unknown (HELO ?192.168.2.25?) (84.203.137.218)
	by mail1.slb.deg.dub.stisp.net with SMTP; 31 Jan 2011 22:12:31 -0000
Message-ID: <4D473354.8000403@draigBrady.com>
Date: Mon, 31 Jan 2011 22:10:28 +0000
From: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US;
	rv:1.9.1.8) Gecko/20100227 Thunderbird/3.0.3
MIME-Version: 1.0
To: coreutils@gnu.org
References: <87d3nczr29.fsf@meyering.net>
In-Reply-To: <87d3nczr29.fsf@meyering.net>
X-Enigmail-Version: 1.0.1
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
X-detected-operating-system: by eggs.gnu.org: FreeBSD 4.6-4.9
X-Received-From: 84.203.253.98
Subject: [PATCH] cp: use a bigger buffer size when writing zeros
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 31 Jan 2011 22:53:40 -0000

Another buglet:

commit 4a64fd8efe00629b0424bb59c8ec8d9d3ef4542f
Author: Pádraig Brady <P@draigBrady.com>
Date:   Mon Jan 31 22:04:35 2011 +0000

    cp: use a bigger buffer size when writing zeros

    * src/copy.c (write_zeros): This bug caused 4 or 8 bytes to
    be written at a time which is very inefficient.  One could
    trigger the issue with `cp --sparse=never sparse non-sparse`
    on a file system that supports fiemap.

diff --git a/src/copy.c b/src/copy.c
index 4e73e1b..65c18f4 100644
--- a/src/copy.c
+++ b/src/copy.c
@@ -291,7 +291,7 @@ write_zeros (int fd, uint64_t n_bytes)

   while (n_bytes)
     {
-      uint64_t n = MIN (sizeof nz, n_bytes);
+      uint64_t n = MIN (nz, n_bytes);
       if ((full_write (fd, zeros, n)) != n)
         return false;
       n_bytes -= n;


From MAILER-DAEMON Mon Jan 31 18:10:41 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1Pk2tN-00056e-DC
	for mharc-coreutils@gnu.org; Mon, 31 Jan 2011 18:10:41 -0500
Received: from [140.186.70.92] (port=39517 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1Pk2tJ-00053w-Lc
	for coreutils@gnu.org; Mon, 31 Jan 2011 18:10:39 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1Pk2tC-0002RE-4D
	for coreutils@gnu.org; Mon, 31 Jan 2011 18:10:31 -0500
Received: from mx1.redhat.com ([209.132.183.28]:51100)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1Pk2tB-0002R5-QF
	for coreutils@gnu.org; Mon, 31 Jan 2011 18:10:30 -0500
Received: from int-mx02.intmail.prod.int.phx2.redhat.com
	(int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12])
	by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id p0VNAQFu012661
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK);
	Mon, 31 Jan 2011 18:10:26 -0500
Received: from [10.3.113.114] (ovpn-113-114.phx2.redhat.com [10.3.113.114])
	by int-mx02.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP
	id p0VNAOY3022422; Mon, 31 Jan 2011 18:10:24 -0500
Message-ID: <4D474160.5000504@redhat.com>
Date: Mon, 31 Jan 2011 16:10:24 -0700
From: Eric Blake <eblake@redhat.com>
Organization: Red Hat
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US;
	rv:1.9.2.13) Gecko/20101209 Fedora/3.1.7-0.35.b3pre.fc14
	Lightning/1.0b3pre Mnenhy/0.8.3 Thunderbird/3.1.7
MIME-Version: 1.0
To: Jim Meyering <jim@meyering.net>
References: <87aaigzqy5.fsf@meyering.net> <4D47350F.8060407@redhat.com>
	<87wrlkyahr.fsf@meyering.net>
In-Reply-To: <87wrlkyahr.fsf@meyering.net>
X-Enigmail-Version: 1.1.2
OpenPGP: url=http://people.redhat.com/eblake/eblake.gpg
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature";
	boundary="------------enig6D2763D3E7F4CFD9A11F2BCE"
X-Scanned-By: MIMEDefang 2.67 on 10.5.11.12
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.132.183.28
Cc: coreutils@gnu.org
Subject: Re: RFC: new cp option: --efficient-sparse=HOW
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 31 Jan 2011 23:10:39 -0000

This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--------------enig6D2763D3E7F4CFD9A11F2BCE
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

On 01/31/2011 03:27 PM, Jim Meyering wrote:
> It's orthogonal.
>=20
> --sparse=3Dalways still does the hole-punching, independently
> of whether we're copying normally or via the efficient FIEMAP-based
> code.
>=20
> E.g., if you have a sparse file, where one non-sparse chunk
> contains all-zero blocks (currently 32KiB minimum), then --sparse=3Dalw=
ays
> will convert those blocks to holes, with or without
> --efficient-sparse=3Dnever.
>=20
> --efficient-sparse=3D... controls efficiency while reading
> --sparse=3D...           controls hole-punching (or preserving)

Still thinking about it.  Isn't there still a bit of a penalty between
reading a block without inspecting its contents, vs. reading a block and
checking if it is all zeros (whether or not there is an optimization of
skipping the read in the first place thanks to fiemap)?  That is,
shouldn't I be able to specify that I want the fastest copying possible,
even if it results in a less than optimal output size, by telling the
read side to not check for all zero blocks?  Or is that check much
smaller than the corresponding I/O savings of not writing the block but
just seeking ahead, at which point punching holes in the output is
inherently more efficient even if it takes longer on the input to tell
if holes were present?

Supposing that we stick with your proposal of just 3 levels for the two
options, then my matrix of the previous mail could be rewritten as
follows for the above 9 combinations:

--sparse=3Dalways --efficient-sparse=3Dalways
    if input is sparse, require fiemap; always scan for holes to punch
--sparse=3Dalways --efficient-sparse=3Dauto
    always scan for holes to punch; use fiemap if available but not
required (smallest output)
--sparse=3Dalways --efficient-sparse=3Dnever
    always scan for holes to punch; avoid fiemap
--sparse=3Dauto --efficient-sparse=3Dalways
    if input is sparse, require fiemap; output is equally sparse as
input because we skip scanning for additional holes (fastest), even if
additional holes could be made
--sparse=3Dauto --efficient-sparse=3Dauto
    if input is sparse, then scan for holes; if fiemap is available then
output will match input sparseness, otherwise output might be smaller as
a result
--sparse=3Dauto --efficient-sparse=3Dnever
    if input is sparse, then scan for holes; avoid fiemap
--sparse=3Dnever --efficient-sparse=3Dalways
    output will not be sparse, but if input is sparse, then require
fiemap to make reading faster
--sparse=3Dnever --efficient-sparse=3Dauto
    output will not be sparse, use fiemap if available
--sparse=3Dnever --efficient-sparse=3Dnever
    output will not be sparse and fiemap is avoided (slowest, but great
way to exercise I/O)

--=20
Eric Blake   eblake@redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org


--------------enig6D2763D3E7F4CFD9A11F2BCE
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Public key at http://people.redhat.com/eblake/eblake.gpg
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/

iQEcBAEBCAAGBQJNR0FgAAoJEKeha0olJ0NqJSEH/jCLYzVsK6rYQVkNKKKv8cKy
PIQz/JQRi3b1xoqCq1n1BLYFABREjzoFDSg/nVkDSW83VSm21LrKLuoWAcEFSaGc
u22MfmESEoxEOpkEO4LRO4Bhjv/QyMFK3IBLeTbbWaGJHxX/B8U4ueY0YjJGaJDA
sxOY7jCUQDREJuzQZ7jMn6uJH1KG3+jfCKydprBQ9l3WsJndNaSJKz4SdVRqGCYe
VRUeQVj+3IGTDJZj1uqJ+rNyZI0M5tNFFetMT69rEE7MfWFGtus3iCt84fWP2/sn
mIZzu8Jh/nRwZ1BNmdWWcmuJV0mzC7tZAiv2z+3NxCrJZ7fMXwHGh6Qny6ZtC5s=
=NtM0
-----END PGP SIGNATURE-----

--------------enig6D2763D3E7F4CFD9A11F2BCE--


From MAILER-DAEMON Mon Jan 31 20:37:39 2011
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1Pk5Bb-0006nn-4G
	for mharc-coreutils@gnu.org; Mon, 31 Jan 2011 20:37:39 -0500
Received: from [140.186.70.92] (port=39656 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1Pk5BZ-0006ni-7Z
	for coreutils@gnu.org; Mon, 31 Jan 2011 20:37:38 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Pk5BY-0006M7-4t
	for coreutils@gnu.org; Mon, 31 Jan 2011 20:37:37 -0500
Received: from mail1.slb.deg.dub.stisp.net ([84.203.253.98]:9403)
	by eggs.gnu.org with smtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Pk5BX-0006M3-Me
	for coreutils@gnu.org; Mon, 31 Jan 2011 20:37:36 -0500
Received: (qmail 75030 invoked from network); 1 Feb 2011 01:37:33 -0000
Received: from unknown (HELO ?192.168.2.25?) (84.203.137.218)
	by mail1.slb.deg.dub.stisp.net with SMTP; 1 Feb 2011 01:37:33 -0000
Message-ID: <4D476361.8080705@draigBrady.com>
Date: Tue, 01 Feb 2011 01:35:29 +0000
From: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US;
	rv:1.9.1.8) Gecko/20100227 Thunderbird/3.0.3
MIME-Version: 1.0
To: Jim Meyering <jim@meyering.net>
References: <87aaigzqy5.fsf@meyering.net>
In-Reply-To: <87aaigzqy5.fsf@meyering.net>
X-Enigmail-Version: 1.0.1
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
X-detected-operating-system: by eggs.gnu.org: FreeBSD 4.6-4.9
X-Received-From: 84.203.253.98
Cc: coreutils@gnu.org
Subject: Re: RFC: new cp option: --efficient-sparse=HOW
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Tue, 01 Feb 2011 01:37:38 -0000

On 31/01/11 21:46, Jim Meyering wrote:
> Now that we have can read sparse files efficiently,
> what if I want to copy a 20PiB sparse file, and yet I want to
> be sure that it does so efficiently.  Few people can afford
> to wait around while a normal processor and storage system process
> that much raw data.  But if it's a sparse file and the src and dest
> file systems have the right support (FIEMAP ioctl), then it'll be
> copied in the time it takes to make a few syscalls.
> 
> Currently, when the efficient sparse copy fails, cp falls back
> on the regular, expensive, read-every-byte approach.
> 
> This proposal adds an option, --efficient-sparse=required,
> to make cp fail if the initial attempt to read the sparse file fails,
> rather than resorting to the regular (very slow in the above case) copy
> procedure.
> 
> The default is --efficient-sparse=auto, and for symmetry,
> I've provided --efficient-sparse=never, in case someone finds
> a reason to want to skip the ioctl.

I'm 50:50 on whether we should add this option.
If a user doesn't want to copy the file if it's going
to be too slow, then what other options do they have?
Also the sparseness of a file can change over time.

Given the fairly specialized need for such an option,
perhaps it's best to foist to other tools, like:

test $(du f) -lt $limit && df -T -t myfs f && cp f f2

or perhaps

timeout $time_limit cp f f2 || { rm f2; echo "oops" >&2; }

cheers,
Pádraig.


