From MAILER-DAEMON Sun Feb 05 18:57:48 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RuBxs-00078h-PE
	for mharc-coreutils@gnu.org; Sun, 05 Feb 2012 18:57:48 -0500
Received: from eggs.gnu.org ([140.186.70.92]:50229)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1Ru19z-0004Pl-05
	for coreutils@gnu.org; Sun, 05 Feb 2012 07:25:41 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1Ru19v-0003vd-4L
	for coreutils@gnu.org; Sun, 05 Feb 2012 07:25:34 -0500
Received: from mail-wi0-f169.google.com ([209.85.212.169]:51030)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1Ru19u-0003vX-PR
	for coreutils@gnu.org; Sun, 05 Feb 2012 07:25:31 -0500
Received: by wibhj13 with SMTP id hj13so5076933wib.0
	for <coreutils@gnu.org>; Sun, 05 Feb 2012 04:25:29 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma;
	h=from:to:subject:cc:date:message-id:mime-version:content-type;
	bh=oSGSo7mP9Xgj2mveGTfn64lHmd52d362r9rXMo7Kj4g=;
	b=vsOOcXlt73T0mOlWNAew93oh4HKqile2vHyGt75MtyI6UlK3UelOe29/NokvdTyF/2
	zNTyyE+RiXtbKahSnVG04tCXwbPTpvnp+CJrjUZZk0UhK8mBDhANsS5HrVRZRoHenI+M
	PyRpipJCIsYVEMZs8RV0O7+d10TEfztwZhy9o=
Received: by 10.180.78.130 with SMTP id b2mr7427673wix.1.1328444729465;
	Sun, 05 Feb 2012 04:25:29 -0800 (PST)
Received: from Apollo.jerryland.fr ([85.69.251.50])
	by mx.google.com with ESMTPS id q7sm18526579wix.5.2012.02.05.04.25.27
	(version=TLSv1/SSLv3 cipher=OTHER);
	Sun, 05 Feb 2012 04:25:27 -0800 (PST)
From: =?utf-8?Q?J=C3=A9r=C3=A9my_Compostella?= <jeremy.compostella@gmail.com>
To: coreutils@gnu.org
Subject: dd: add 'skip_bytes' and 'count_bytes' operands
Date: Sun, 05 Feb 2012 13:25:46 +0100
Message-ID: <877h015uwl.fsf@Apollo.jerryland.fr>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="=-=-="
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.85.212.169
X-Mailman-Approved-At: Sun, 05 Feb 2012 18:57:45 -0500
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 05 Feb 2012 12:25:41 -0000

--=-=-=
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

All,

I worked on the 'skip_bytes' and 'count_bytes' features reported in the
following mail thread:
http://lists.gnu.org/archive/html/bug-coreutils/2009-10/msg00061.html

If anybody have started working on it or have guidance for it, please
let me know.

I wrote the attached patch which implements these features. As I'm new
in this project and I'm not really comfortable with project specific
good practices do not hesitate to comment this patch. I will take into
account whatever you want.

I added a special test suite to satisfy these new operands. tests/dd/bytes

Writing this patch I thought it would be nice having the seek_bytes
operand too. What is your opinion about this ?

Cheers,

J=C3=A9r=C3=A9my


--=-=-=
Content-Type: text/x-diff
Content-Disposition: inline;
 filename=0001-dd-add-skip_bytes-and-count_bytes-operands.patch
Content-Description: dd: add skip_bytes and count_bytes operands patch

>From 732f14abfc8b1b5aa711a41cc4dd7fdc0b3d045a Mon Sep 17 00:00:00 2001
From: Jeremy Compostella <jeremy.compostella@gmail.com>
Date: Sat, 4 Feb 2012 15:25:54 +0100
Subject: [PATCH] dd: add skip_bytes and count_bytes operands

dd now accepts:
- the skip_bytes=N operand. When specified, dd skip N bytes before
  copying. 'skip' and 'skip_bytes" operands are mutually exclusive.
- the count_bytes=N operand. When specified, dd copy N bytes. 'count'
  and 'count_bytes' are mutually exclusive.

Signed-off-by: Jeremy Compostella <jeremy.compostella@gmail.com>
---
 NEWS               |    8 ++++
 doc/coreutils.texi |   11 ++++++
 src/dd.c           |   99 +++++++++++++++++++++++++++++++++++++++------------
 tests/Makefile.am  |    1 +
 tests/dd/bytes     |   37 +++++++++++++++++++
 5 files changed, 132 insertions(+), 24 deletions(-)
 create mode 100755 tests/dd/bytes

diff --git a/NEWS b/NEWS
index 2b0926f..445d87b 100644
--- a/NEWS
+++ b/NEWS
@@ -2,6 +2,14 @@ GNU coreutils NEWS                                    -*- outline -*-
 
 * Noteworthy changes in release ?.? (????-??-??) [?]
 
+** New features
+
+  dd now accepts:
+  - the skip_bytes=N operand. When specified, dd skip N
+    bytes before copying. 'skip' and 'skip_bytes' operands are mutually
+    exclusive.
+  - the count_bytes=N operand. When specified, dd copy N bytes. 'count'
+    and 'count_bytes' are mutually exclusive.
 
 * Noteworthy changes in release 8.15 (2012-01-06) [stable]
 
diff --git a/doc/coreutils.texi b/doc/coreutils.texi
index 0d3b739..4124927 100644
--- a/doc/coreutils.texi
+++ b/doc/coreutils.texi
@@ -8061,6 +8061,11 @@ use @var{bytes} as the fixed record length.
 @opindex skip
 Skip @var{blocks} @samp{ibs}-byte blocks in the input file before copying.
 
+@item skip_bytes=@var{n}
+@opindex skip_bytes
+Skip @var{n} bytes in the input file before copying.
+@samp{skip_bytes} and @samp{skip} are mutually exclusive.
+
 @item seek=@var{blocks}
 @opindex seek
 Skip @var{blocks} @samp{obs}-byte blocks in the output file before copying.
@@ -8070,6 +8075,12 @@ Skip @var{blocks} @samp{obs}-byte blocks in the output file before copying.
 Copy @var{blocks} @samp{ibs}-byte blocks from the input file, instead
 of everything until the end of the file.
 
+@item count_bytes=@var{n}
+@opindex count
+Copy @var{n} bytes from the input file, instead of everything until
+the end of the file. @samp{count} and @samp{count_bytes} are mutually
+exclusive.
+
 @item status=noxfer
 @opindex status
 Do not print the overall transfer rate and volume statistics
diff --git a/src/dd.c b/src/dd.c
index 8c687b3..461b9e0 100644
--- a/src/dd.c
+++ b/src/dd.c
@@ -156,12 +156,19 @@ static size_t conversion_blocksize = 0;
 /* Skip this many records of 'input_blocksize' bytes before input. */
 static uintmax_t skip_records = 0;
 
+/* Skip this many bytes before input in addition of 'skip_records'
+   records.  */
+static size_t skip_bytes = 0;
+
 /* Skip this many records of 'output_blocksize' bytes before output. */
 static uintmax_t seek_records = 0;
 
 /* Copy only this many records.  The default is effectively infinity.  */
 static uintmax_t max_records = (uintmax_t) -1;
 
+/* Copy this many bytes in addition to 'max_records' records.  */
+static size_t max_bytes = 0;
+
 /* Bit vector of conversions to apply. */
 static int conversions_mask = 0;
 
@@ -490,6 +497,8 @@ Copy a file, converting and formatting according to the operands.\n\
   cbs=BYTES       convert BYTES bytes at a time\n\
   conv=CONVS      convert the file as per the comma separated symbol list\n\
   count=BLOCKS    copy only BLOCKS input blocks\n\
+  count_bytes=N   copy only N bytes. count and count_bytes operands are mutually\n\
+                  exclusive\n\
   ibs=BYTES       read up to BYTES bytes at a time (default: 512)\n\
 "), stdout);
       fputs (_("\
@@ -500,6 +509,8 @@ Copy a file, converting and formatting according to the operands.\n\
   oflag=FLAGS     write as per the comma separated symbol list\n\
   seek=BLOCKS     skip BLOCKS obs-sized blocks at start of output\n\
   skip=BLOCKS     skip BLOCKS ibs-sized blocks at start of input\n\
+  skip_bytes=N    skip N bytes at start of input. skip and skip_bytes operands\n\
+                  are mutually exclusive\n\
   status=noxfer   suppress transfer statistics\n\
 "), stdout);
       fputs (_("\
@@ -1120,6 +1131,8 @@ scanargs (int argc, char *const *argv)
 {
   int i;
   size_t blocksize = 0;
+  uintmax_t count_in_bytes = 0;
+  uintmax_t skip_in_bytes = 0;
 
   for (i = optind; i < argc; i++)
     {
@@ -1176,10 +1189,14 @@ scanargs (int argc, char *const *argv)
             }
           else if (operand_is (name, "skip"))
             skip_records = n;
+          else if (operand_is (name, "skip_bytes"))
+            skip_in_bytes = n;
           else if (operand_is (name, "seek"))
             seek_records = n;
           else if (operand_is (name, "count"))
             max_records = n;
+          else if (operand_is (name, "count_bytes"))
+            count_in_bytes = n;
           else
             {
               error (0, 0, _("unrecognized operand %s"), quote (name));
@@ -1216,6 +1233,30 @@ scanargs (int argc, char *const *argv)
       usage (EXIT_FAILURE);
     }
 
+  if (skip_in_bytes != 0 && skip_records != 0)
+    {
+      error (0, 0, _("'skip_bytes' and 'skip' operands are mutually exclusive"));
+      usage (EXIT_FAILURE);
+    }
+
+  if (skip_in_bytes != 0)
+    {
+      skip_records = skip_in_bytes / input_blocksize;
+      skip_bytes = skip_in_bytes % input_blocksize;
+    }
+
+  if (count_in_bytes != 0 && max_records != (uintmax_t) -1)
+    {
+      error (0, 0, _("'count_bytes' and 'count' operands are mutually exclusive"));
+      usage (EXIT_FAILURE);
+    }
+
+  if (count_in_bytes != 0)
+    {
+      max_records = count_in_bytes / input_blocksize;
+      max_bytes = count_in_bytes % input_blocksize;
+    }
+
   /* Warn about partial reads if bs=SIZE is given and iflag=fullblock
      is not, and if counting or skipping bytes or using direct I/O.
      This helps to avoid confusion with miscounts, and to avoid issues
@@ -1411,18 +1452,19 @@ skip_via_lseek (char const *filename, int fdesc, off_t offset, int whence)
 # define skip_via_lseek(Filename, Fd, Offset, Whence) lseek (Fd, Offset, Whence)
 #endif
 
-/* Throw away RECORDS blocks of BLOCKSIZE bytes on file descriptor FDESC,
-   which is open with read permission for FILE.  Store up to BLOCKSIZE
-   bytes of the data at a time in BUF, if necessary.  RECORDS must be
-   nonzero.  If fdesc is STDIN_FILENO, advance the input offset.
-   Return the number of records remaining, i.e., that were not skipped
-   because EOF was reached.  */
+/* Throw away RECORDS blocks of BLOCKSIZE bytes plus BYTES bytes on
+   file descriptor FDESC, which is open with read permission for FILE.
+   Store up to BLOCKSIZE bytes of the data at a time in BUF, if
+   necessary. RECORDS or BYTES must be nonzero. If FDESC is
+   STDIN_FILENO, advance the input offset. Return the number of
+   records remaining, i.e., that were not skipped because EOF was
+   reached.  */
 
 static uintmax_t
 skip (int fdesc, char const *file, uintmax_t records, size_t blocksize,
-      char *buf)
+      size_t bytes, char *buf)
 {
-  uintmax_t offset = records * blocksize;
+  uintmax_t offset = records * blocksize + bytes;
 
   /* Try lseek and if an error indicates it was an inappropriate operation --
      or if the file offset is not representable as an off_t --
@@ -1491,29 +1533,35 @@ skip (int fdesc, char const *file, uintmax_t records, size_t blocksize,
 
       do
         {
-          ssize_t nread = iread_fnc (fdesc, buf, blocksize);
+          ssize_t nread;
+          if (records != 0)
+            nread = iread_fnc (fdesc, buf, blocksize);
+          else
+            nread = iread_fnc (fdesc, buf, bytes);
+
           if (nread < 0)
             {
               if (fdesc == STDIN_FILENO)
                 {
                   error (0, errno, _("reading %s"), quote (file));
                   if (conversions_mask & C_NOERROR)
-                    {
-                      print_stats ();
-                      continue;
-                    }
+                    print_stats ();
                 }
               else
                 error (0, lseek_errno, _("%s: cannot seek"), quote (file));
               quit (EXIT_FAILURE);
             }
-
-          if (nread == 0)
+          else if (nread == 0)
             break;
-          if (fdesc == STDIN_FILENO)
+          else if (fdesc == STDIN_FILENO)
             advance_input_offset (nread);
+
+          if (records != 0)
+            --records;
+          else
+            bytes = 0;
         }
-      while (--records != 0);
+      while (records != 0 || bytes != 0);
 
       return records;
     }
@@ -1777,11 +1825,11 @@ dd_copy (void)
       obuf = ibuf;
     }
 
-  if (skip_records != 0)
+  if (skip_records != 0 || skip_bytes != 0)
     {
-      uintmax_t us_bytes = input_offset + (skip_records * input_blocksize);
+      uintmax_t us_bytes = input_offset + (skip_records * input_blocksize) + skip_bytes;
       uintmax_t us_blocks = skip (STDIN_FILENO, input_file,
-                                  skip_records, input_blocksize, ibuf);
+                                  skip_records, input_blocksize, skip_bytes, ibuf);
       us_bytes -= input_offset;
 
       /* POSIX doesn't say what to do when dd detects it has been
@@ -1800,7 +1848,7 @@ dd_copy (void)
   if (seek_records != 0)
     {
       uintmax_t write_records = skip (STDOUT_FILENO, output_file,
-                                      seek_records, output_blocksize, obuf);
+                                      seek_records, output_blocksize, 0, obuf);
 
       if (write_records != 0)
         {
@@ -1817,12 +1865,12 @@ dd_copy (void)
         }
     }
 
-  if (max_records == 0)
+  if (max_records == 0 && max_bytes == 0)
     return exit_status;
 
   while (1)
     {
-      if (r_partial + r_full >= max_records)
+      if (r_partial + r_full >= max_records + (max_bytes ? 1 : 0))
         break;
 
       /* Zero the buffer before reading, so that if we get a read error,
@@ -1833,7 +1881,10 @@ dd_copy (void)
                 (conversions_mask & (C_BLOCK | C_UNBLOCK)) ? ' ' : '\0',
                 input_blocksize);
 
-      nread = iread_fnc (STDIN_FILENO, ibuf, input_blocksize);
+      if (r_partial + r_full >= max_records)
+        nread = iread_fnc (STDIN_FILENO, ibuf, max_bytes);
+      else
+        nread = iread_fnc (STDIN_FILENO, ibuf, input_blocksize);
 
       if (nread >= 0 && i_nocache)
         invalidate_cache (STDIN_FILENO, nread);
diff --git a/tests/Makefile.am b/tests/Makefile.am
index 8b670fc..c6b8737 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -369,6 +369,7 @@ TESTS =						\
   dd/reblock					\
   dd/skip-seek					\
   dd/skip-seek2					\
+  dd/bytes					\
   dd/skip-seek-past-file			\
   dd/stderr					\
   dd/unblock					\
diff --git a/tests/dd/bytes b/tests/dd/bytes
new file mode 100755
index 0000000..d0c72f8
--- /dev/null
+++ b/tests/dd/bytes
@@ -0,0 +1,37 @@
+#!/bin/sh
+
+# Copyright (C) 2012 Free Software Foundation, Inc.
+
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+. "${srcdir=.}/init.sh"; path_prepend_ ../src
+print_ver_ dd
+
+# count_bytes
+echo 0123456789abcefghijklm > in || fail=1
+(dd count_bytes=14 conv=swab) < in > out 2> /dev/null || fail=1
+case `cat out` in
+  1032547698baec) ;;
+  *) fail=1 ;;
+esac
+
+# skip_bytes
+echo 0123456789abcefghijklm > in || fail=1
+(dd skip_bytes=10) < in > out 2> /dev/null || fail=1
+case `cat out` in
+  abcefghijklm) ;;
+  *) fail=1 ;;
+esac
+
+Exit $fail
-- 
1.7.2.5


--=-=-=--


From MAILER-DAEMON Sun Feb 05 18:57:48 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RuBxs-000792-Rw
	for mharc-coreutils@gnu.org; Sun, 05 Feb 2012 18:57:48 -0500
Received: from eggs.gnu.org ([140.186.70.92]:35584)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1Ru5Ph-0000pd-KV
	for coreutils@gnu.org; Sun, 05 Feb 2012 11:58:09 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1Ru5Pa-0007CI-NJ
	for coreutils@gnu.org; Sun, 05 Feb 2012 11:58:05 -0500
Received: from mail-we0-f169.google.com ([74.125.82.169]:47698)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1Ru5Pa-0007C0-Gx
	for coreutils@gnu.org; Sun, 05 Feb 2012 11:57:58 -0500
Received: by wera13 with SMTP id a13so4896524wer.0
	for <coreutils@gnu.org>; Sun, 05 Feb 2012 08:57:57 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma;
	h=from:to:subject:cc:date:message-id:mime-version:content-type;
	bh=8t7KhZbgn3BXIYin2i6LTPQBmCqpOFVOd1dzLkrDf68=;
	b=j47JQ4zpTXsKarCcIMAMwZ8rvlRSJnCI2DOpo7+Xqc/K50D1UdB7/Cuuv0ci/jsbBk
	P5dM3F7g8ILIYgooE+WAQYvmIr0UbSN9TIDKWFTHCcbyL/FHhdd1He4rsewLbX/VfmHW
	0uWjiXdaQVAPnCMG8gzot6Z+8Yr5fZ5bp4WRo=
Received: by 10.216.132.219 with SMTP id o69mr2124783wei.55.1328461076936;
	Sun, 05 Feb 2012 08:57:56 -0800 (PST)
Received: from Apollo.jerryland.fr ([85.69.251.50])
	by mx.google.com with ESMTPS id y1sm38020119wiw.6.2012.02.05.08.57.55
	(version=TLSv1/SSLv3 cipher=OTHER);
	Sun, 05 Feb 2012 08:57:56 -0800 (PST)
From: =?utf-8?Q?J=C3=A9r=C3=A9my_Compostella?= <jeremy.compostella@gmail.com>
To: coreutils@debbugs.gnu.org
Subject: dd: add braces as requested by HACKING coding standard
Date: Sun, 05 Feb 2012 17:58:13 +0100
Message-ID: <874nv55iai.fsf@Apollo.jerryland.fr>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="=-=-="
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 74.125.82.169
X-Mailman-Approved-At: Sun, 05 Feb 2012 18:57:47 -0500
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 05 Feb 2012 16:58:09 -0000

--=-=-=
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

All,

Working on dd new features, I found out that a do/while statement wasn't
respecting the HACKING coding standard.

I attached the patch which fix this.

Cheers,

J=C3=A9r=C3=A9my
---

--=-=-=
Content-Type: text/x-diff
Content-Disposition: inline;
 filename=0001-dd-add-braces-as-requested-by-HACKING-coding-standar.patch
Content-Description: dd: add braces as requested by HACKING coding standard
 patch

>From c34ba70d5bbdb91fce01a58bef1d61df7b191880 Mon Sep 17 00:00:00 2001
From: Jeremy Compostella <jeremy.compostella@gmail.com>
Date: Sun, 5 Feb 2012 17:53:55 +0100
Subject: [PATCH] dd: add braces as requested by HACKING coding standard

Braces was missing for this do/while statement.

Signed-off-by: Jeremy Compostella <jeremy.compostella@gmail.com>
---
 src/dd.c |   14 ++++++++------
 1 files changed, 8 insertions(+), 6 deletions(-)

diff --git a/src/dd.c b/src/dd.c
index 8c687b3..9b18a72 100644
--- a/src/dd.c
+++ b/src/dd.c
@@ -1807,12 +1807,14 @@ dd_copy (void)
           memset (obuf, 0, output_blocksize);
 
           do
-            if (iwrite (STDOUT_FILENO, obuf, output_blocksize)
-                != output_blocksize)
-              {
-                error (0, errno, _("writing to %s"), quote (output_file));
-                quit (EXIT_FAILURE);
-              }
+            {
+              if (iwrite (STDOUT_FILENO, obuf, output_blocksize)
+                  != output_blocksize)
+                {
+                  error (0, errno, _("writing to %s"), quote (output_file));
+                  quit (EXIT_FAILURE);
+                }
+            }
           while (--write_records != 0);
         }
     }
-- 
1.7.2.5


--=-=-=--


From MAILER-DAEMON Sun Feb 05 20:55:33 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RuDnp-00085I-QM
	for mharc-coreutils@gnu.org; Sun, 05 Feb 2012 20:55:33 -0500
Received: from eggs.gnu.org ([140.186.70.92]:48276)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RuDnm-00085C-9P
	for coreutils@gnu.org; Sun, 05 Feb 2012 20:55:32 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RuDnk-0000LC-OQ
	for coreutils@gnu.org; Sun, 05 Feb 2012 20:55:30 -0500
Received: from mx1.redhat.com ([209.132.183.28]:2842)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RuDnk-0000L5-HQ
	for coreutils@gnu.org; Sun, 05 Feb 2012 20:55:28 -0500
Received: from int-mx01.intmail.prod.int.phx2.redhat.com
	(int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id q161tOw5002464
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK);
	Sun, 5 Feb 2012 20:55:24 -0500
Received: from [10.36.116.26] (ovpn-116-26.ams2.redhat.com [10.36.116.26])
	by int-mx01.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP
	id q161tMJe009587
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO);
	Sun, 5 Feb 2012 20:55:24 -0500
Message-ID: <4F2F3308.3020506@draigBrady.com>
Date: Mon, 06 Feb 2012 01:55:20 +0000
From: =?UTF-8?B?UMOhZHJhaWcgQnJhZHk=?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:6.0) Gecko/20110816 Thunderbird/6.0
MIME-Version: 1.0
To: =?UTF-8?B?SsOpcsOpbXkgQ29tcG9zdGVsbGE=?=
 <jeremy.compostella@gmail.com>
Subject: Re: dd: add braces as requested by HACKING coding standard
References: <874nv55iai.fsf@Apollo.jerryland.fr>
In-Reply-To: <874nv55iai.fsf@Apollo.jerryland.fr>
X-Enigmail-Version: 1.3.2
Content-Type: multipart/mixed; boundary="------------060900000205050002080601"
X-Scanned-By: MIMEDefang 2.67 on 10.5.11.11
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.132.183.28
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 06 Feb 2012 01:55:32 -0000

This is a multi-part message in MIME format.
--------------060900000205050002080601
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
X-MIME-Autoconverted: from 8bit to quoted-printable by mx1.redhat.com id q161tOw5002464

On 02/05/2012 04:58 PM, J=C3=A9r=C3=A9my Compostella wrote:
> All,
>=20
> Working on dd new features, I found out that a do/while statement wasn'=
t
> respecting the HACKING coding standard.
>=20
> I attached the patch which fix this.

I made a couple of meta changes:

1. Changed the tag of the commit message from dd: to maint:
since this has no functional change to dd.  This helps when
scanning the auto generated summary of changes.

2. You inconsistently use =C3=A9 and e in your name between git config
and your emailer and patches on github.  I adjusted to using =C3=A9
as I presume that's more correct?

3. I removed the redundant Signed-Off line

4. I summarised the file and function where the change was made

5. I removed the debbugs email address from the CC of this mail.
coreutils@gnu.org is best for patches.

I hope these changes don't seem pedantic.
Keeping a consistent commit structure does help with ongoing maintenance.
Also using a consistent format is quicker as less thinking is involved.
Also tools like http://www.gnu.org/software/vc-dwim/ can be used
if doing many such patches.

I'll push the attached if you concur.

cheers,
P=C3=A1draig.

--------------060900000205050002080601
Content-Type: text/x-patch;
 name="0001-maint-add-braces-as-requested-by-HACKING-coding-stan.patch"
Content-Disposition: attachment;
 filename*0="0001-maint-add-braces-as-requested-by-HACKING-coding-stan.pa";
 filename*1="tch"
Content-Transfer-Encoding: 7bit

>From 31eeead93842c6581e0a9b21f6e8eb964d0c9309 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?J=C3=A9r=C3=A9my=20Compostella?=
 <jeremy.compostella@gmail.com>
Date: Sun, 5 Feb 2012 17:53:55 +0100
Subject: [PATCH] maint: add braces as requested by HACKING coding standard

* src/dd.c (dd_copy): Braces were missing for this do/while statement.
---
 src/dd.c |   14 ++++++++------
 1 files changed, 8 insertions(+), 6 deletions(-)

diff --git a/src/dd.c b/src/dd.c
index 2c545c2..9d24791 100644
--- a/src/dd.c
+++ b/src/dd.c
@@ -1807,12 +1807,14 @@ dd_copy (void)
           memset (obuf, 0, output_blocksize);
 
           do
-            if (iwrite (STDOUT_FILENO, obuf, output_blocksize)
-                != output_blocksize)
-              {
-                error (0, errno, _("writing to %s"), quote (output_file));
-                quit (EXIT_FAILURE);
-              }
+            {
+              if (iwrite (STDOUT_FILENO, obuf, output_blocksize)
+                  != output_blocksize)
+                {
+                  error (0, errno, _("writing to %s"), quote (output_file));
+                  quit (EXIT_FAILURE);
+                }
+            }
           while (--write_records != 0);
         }
     }
-- 
1.7.6.4


--------------060900000205050002080601--


From MAILER-DAEMON Mon Feb 06 01:26:22 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RuI1u-0005b5-7E
	for mharc-coreutils@gnu.org; Mon, 06 Feb 2012 01:26:22 -0500
Received: from eggs.gnu.org ([140.186.70.92]:38934)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1RuI1r-0005YD-C2
	for coreutils@gnu.org; Mon, 06 Feb 2012 01:26:20 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1RuI1q-0000F8-4s
	for coreutils@gnu.org; Mon, 06 Feb 2012 01:26:19 -0500
Received: from mail-ww0-f49.google.com ([74.125.82.49]:48702)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1RuI1p-0000F4-VS
	for coreutils@gnu.org; Mon, 06 Feb 2012 01:26:18 -0500
Received: by wgbdt13 with SMTP id dt13so4510980wgb.30
	for <coreutils@gnu.org>; Sun, 05 Feb 2012 22:26:17 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma;
	h=from:to:subject:cc:date:message-id:mime-version:content-type
	:content-transfer-encoding;
	bh=xE+ebzmT0InjxFBT3QByXffoTBTtxWlb39WRosIe6Ac=;
	b=g8kr7dY2gu5MLez2xeA5GdoQeIQjSI19bM/rJDvN5kJteD69W3LKsILzFHj5d4yDQw
	hF8O2I9x2Yf5+DSi3YoQVP8m0amyuGA+1T4//PkOgpU38CKh0TLEoCoWkEz542jOCzrY
	mLvGSYakE3KPkyLdfsAUYipNL0OI1FXFR5AQU=
Received: by 10.180.107.34 with SMTP id gz2mr25058000wib.21.1328509577043;
	Sun, 05 Feb 2012 22:26:17 -0800 (PST)
Received: from Apollo.jerryland.fr ([85.69.251.50])
	by mx.google.com with ESMTPS id q2sm23075253wiy.7.2012.02.05.22.26.14
	(version=TLSv1/SSLv3 cipher=OTHER);
	Sun, 05 Feb 2012 22:26:15 -0800 (PST)
From: =?utf-8?Q?J=C3=A9r=C3=A9my_Compostella?= <jeremy.compostella@gmail.com>
To: =?utf-8?Q?P=C3=A1draig?= Brady <P@draigbrady.com>
Subject: Re: dd: add braces as requested by HACKING coding standard
Date: Mon, 06 Feb 2012 07:26:34 +0100
Message-ID: <871uq85vfp.fsf@Apollo.jerryland.fr>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 74.125.82.49
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 06 Feb 2012 06:26:20 -0000

> I made a couple of meta changes:
>=20
> 1. Changed the tag of the commit message from dd: to maint:
> since this has no functional change to dd.  This helps when
> scanning the auto generated summary of changes.
>=20
> 2. You inconsistently use =C3=A9 and e in your name between git config
> and your emailer and patches on github.  I adjusted to using =C3=A9
> as I presume that's more correct?
You presume right. I stopped to use accent in my git config a few months
ago because I noticed that sometime when my patch were integrated in
some projects, my name was badly written due to accent
encoding. However, I really do prefer do get my name correctly written
(with accent).

> 3. I removed the redundant Signed-Off line
Sorry, an habit from the Linux kernel. I will remove it from my config
for coreutils.

>=20
> 4. I summarised the file and function where the change was made
And fixed my bad grammar ... ;)

> 5. I removed the debbugs email address from the CC of this mail.
> coreutils@gnu.org is best for patches.
>=20
> I hope these changes don't seem pedantic.
Absolutely not. You're perfectly right and moreover you take time to
explain what's wrong which is very rewarding for me.

> Keeping a consistent commit structure does help with ongoing
> maintenance.
If all developers in the world could hear you ...
=20
> Also using a consistent format is quicker as less thinking is involved.
> Also tools like http://www.gnu.org/software/vc-dwim/ can be used
> if doing many such patches.

> I'll push the attached if you concur.
Let's got for it.


From MAILER-DAEMON Mon Feb 06 04:07:30 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RuKXq-0004zg-Id
	for mharc-coreutils@gnu.org; Mon, 06 Feb 2012 04:07:30 -0500
Received: from eggs.gnu.org ([140.186.70.92]:45028)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RuKXk-0004zL-9j
	for coreutils@gnu.org; Mon, 06 Feb 2012 04:07:28 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RuKXe-0001Tz-2B
	for coreutils@gnu.org; Mon, 06 Feb 2012 04:07:24 -0500
Received: from mx1.redhat.com ([209.132.183.28]:29840)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RuKXd-0001Tp-Q6
	for coreutils@gnu.org; Mon, 06 Feb 2012 04:07:18 -0500
Received: from int-mx11.intmail.prod.int.phx2.redhat.com
	(int-mx11.intmail.prod.int.phx2.redhat.com [10.5.11.24])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id q1697EEK006550
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK);
	Mon, 6 Feb 2012 04:07:15 -0500
Received: from [10.36.116.56] (ovpn-116-56.ams2.redhat.com [10.36.116.56])
	by int-mx11.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP
	id q16977IU021787
	(version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO);
	Mon, 6 Feb 2012 04:07:09 -0500
Message-ID: <4F2F983A.3080509@draigBrady.com>
Date: Mon, 06 Feb 2012 09:07:06 +0000
From: =?UTF-8?B?UMOhZHJhaWcgQnJhZHk=?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:6.0) Gecko/20110816 Thunderbird/6.0
MIME-Version: 1.0
To: =?UTF-8?B?SsOpcsOpbXkgQ29tcG9zdGVsbGE=?=
 <jeremy.compostella@gmail.com>
Subject: Re: dd: add braces as requested by HACKING coding standard
References: <871uq85vfp.fsf@Apollo.jerryland.fr>
In-Reply-To: <871uq85vfp.fsf@Apollo.jerryland.fr>
X-Enigmail-Version: 1.3.2
Content-Type: text/plain; charset=UTF-8
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.24
Content-Transfer-Encoding: quoted-printable
X-MIME-Autoconverted: from 8bit to quoted-printable by mx1.redhat.com id
	q1697EEK006550
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.132.183.28
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 06 Feb 2012 09:07:28 -0000

On 02/06/2012 06:26 AM, J=C3=A9r=C3=A9my Compostella wrote:
>>
>> You inconsistently use =C3=A9 and e in your name between git config
>> and your emailer and patches on github.  I adjusted to using =C3=A9
>> as I presume that's more correct?
> You presume right. I stopped to use accent in my git config a few month=
s
> ago because I noticed that sometime when my patch were integrated in
> some projects, my name was badly written due to accent
> encoding. However, I really do prefer do get my name correctly written
> (with accent).

Cool.
Note you can set per project git config by editing .git/config

>> I'll push the attached if you concur.
> Let's got for it.

Pushed.

cheers,
P=C3=A1draig.


From MAILER-DAEMON Mon Feb 06 08:55:21 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RuP2P-0000HY-6E
	for mharc-coreutils@gnu.org; Mon, 06 Feb 2012 08:55:21 -0500
Received: from eggs.gnu.org ([140.186.70.92]:55415)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1RuP2I-0000ED-7A
	for coreutils@gnu.org; Mon, 06 Feb 2012 08:55:19 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1RuP2D-0003tY-3R
	for coreutils@gnu.org; Mon, 06 Feb 2012 08:55:14 -0500
Received: from mail-wi0-f169.google.com ([209.85.212.169]:62519)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1RuP2C-0003se-Fu
	for coreutils@gnu.org; Mon, 06 Feb 2012 08:55:08 -0500
Received: by wibhj13 with SMTP id hj13so5900928wib.0
	for <coreutils@gnu.org>; Mon, 06 Feb 2012 05:55:06 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma;
	h=from:to:subject:cc:date:message-id:mime-version:content-type;
	bh=IenkGplFdwurVG1qqxYfbFvexUVqWZpG3t9DAD+Fmdc=;
	b=TtfpxcoNMbmLY9VZlD/0HB5vQMpAGFET26qDyoM85bARxrQ4bkk1BoORFupUHIH6DA
	hHUWjXk40zOm7Yak1/iOg1enVtMVwfAk+qGIaDvimTl0chzqcIXuUYUsvoI2UYGIOzCn
	7gWIlMnMEaOkwI0sWs8BnpPdlvagIOnJY0xUg=
Received: by 10.180.79.229 with SMTP id m5mr4126180wix.6.1328536506852;
	Mon, 06 Feb 2012 05:55:06 -0800 (PST)
Received: from Apollo.jerryland.fr ([85.69.251.50])
	by mx.google.com with ESMTPS id s8sm16693111wiz.8.2012.02.06.05.55.04
	(version=TLSv1/SSLv3 cipher=OTHER);
	Mon, 06 Feb 2012 05:55:05 -0800 (PST)
From: =?utf-8?Q?J=C3=A9r=C3=A9my_Compostella?= <jeremy.compostella@gmail.com>
To: =?utf-8?Q?P=C3=A1draig?= Brady <P@draigbrady.com>
Subject: Re: bug#6554: [PATCH] split: Additional suffix for split (bug#6554)
Date: Mon, 06 Feb 2012 14:55:24 +0100
Message-ID: <87y5sg3w37.fsf@Apollo.jerryland.fr>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="=-=-="
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.85.212.169
Cc: 6554@debbugs.gnu.org, coreutils@gnu.org, Peng Yu <pengyu.ut@gmail.com>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 06 Feb 2012 13:55:19 -0000

--=-=-=
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

P=C3=A1draig, all,

I took into account general comments on my commits. I attached the
improved patch for feature. Improvements are:
- Add by file description,
- I put back accents on my name (J=C3=A9r=C3=A9my instead of Jeremy),
- I referenced from who this feature was requested.

Cheers,

J=C3=A9r=C3=A9my
---

--=-=-=
Content-Type: text/x-diff
Content-Disposition: inline;
 filename=0001-split-Additional-suffix-for-split-bug-6554.patch
Content-Description: split: Additional suffix for split (bug#6554) patch

>From 81e12245be60ef4e9b06ab18fa9e92bce81ea63c Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?J=C3=A9r=C3=A9my=20Compostella?= <jeremy.compostella@gmail.com>
Date: Fri, 27 Jan 2012 18:14:34 +0100
Subject: [PATCH] split: Additional suffix for split (bug#6554)

Add support to an additionnal suffix with the new `--suffix=SUFF'
option.  SUFF is appended to each output filename right after the
dynamic suffix.

* src/split.c (next_file_name): Append fixed_suffix to outfile.
(main): Handle --suffix new option.
* NEWS (New features): Mention it.
* doc/coreutils.texi (split invocation): Mention it.
* tests/split/suffix: New file. --suffix option tests.
* tests/Makefile.am (TESTS): Add it.
Requested by Peng Yu.
---
 NEWS               |    6 ++++++
 doc/coreutils.texi |    6 ++++++
 src/split.c        |   24 ++++++++++++++++++++++--
 tests/Makefile.am  |    1 +
 tests/split/suffix |   41 +++++++++++++++++++++++++++++++++++++++++
 5 files changed, 76 insertions(+), 2 deletions(-)
 create mode 100755 tests/split/suffix

diff --git a/NEWS b/NEWS
index 9eebbf6..fa056e1 100644
--- a/NEWS
+++ b/NEWS
@@ -2,6 +2,12 @@ GNU coreutils NEWS                                    -*- outline -*-
 
 * Noteworthy changes in release ?.? (????-??-??) [?]
 
+** New features
+
+  split now accepts the new --suffix=SUFF option. When SUFF is
+  specified, SUFF is append to each output filenames right after the
+  dynamic suffix.
+
 ** Bug fixes
 
   mv now lets you move a symlink onto a same-inode destination file that
diff --git a/doc/coreutils.texi b/doc/coreutils.texi
index 52838e7..db12dbb 100644
--- a/doc/coreutils.texi
+++ b/doc/coreutils.texi
@@ -3083,6 +3083,12 @@ and so can be a pipe for example.
 @opindex --suffix-length
 Use suffixes of length @var{length}.  The default @var{length} is 2.
 
+@itemx --suffix=@var{suff}
+@opindex -a
+@opindex --suffix-length
+Append @var{suff} to each output filename right after the dynamic
+suffix. @var{suff} must not contain slash.
+
 @item -d
 @itemx --numeric-suffixes
 @opindex -d
diff --git a/src/split.c b/src/split.c
index 1d0310c..af532fa 100644
--- a/src/split.c
+++ b/src/split.c
@@ -80,6 +80,13 @@ static size_t suffix_length;
 /* Alphabet of characters to use in suffix.  */
 static char const *suffix_alphabet = "abcdefghijklmnopqrstuvwxyz";
 
+/* Length of fixed suffix. */
+static size_t fixed_suffix_len = 0;
+
+/* Fixed suffix to append to OUTFILE right after the dynamic
+   suffix.  */
+static char const *fixed_suffix;
+
 /* Name of input file.  May be "-".  */
 static char *infile;
 
@@ -110,7 +117,8 @@ enum
 {
   VERBOSE_OPTION = CHAR_MAX + 1,
   FILTER_OPTION,
-  IO_BLKSIZE_OPTION
+  IO_BLKSIZE_OPTION,
+  SUFFIX_OPTION
 };
 
 static struct option const longopts[] =
@@ -122,6 +130,7 @@ static struct option const longopts[] =
   {"elide-empty-files", no_argument, NULL, 'e'},
   {"unbuffered", no_argument, NULL, 'u'},
   {"suffix-length", required_argument, NULL, 'a'},
+  {"suffix", required_argument, NULL, SUFFIX_OPTION},
   {"numeric-suffixes", no_argument, NULL, 'd'},
   {"filter", required_argument, NULL, FILTER_OPTION},
   {"verbose", no_argument, NULL, VERBOSE_OPTION},
@@ -193,6 +202,8 @@ Mandatory arguments to long options are mandatory for short options too.\n\
 "), stdout);
       fprintf (stdout, _("\
   -a, --suffix-length=N   use suffixes of length N (default %d)\n\
+      --suffix=SUFF       append SUFF to each output filename right after\n\
+                          the dynamic suffix. SUFF must not contain slash\n\
   -b, --bytes=SIZE        put SIZE bytes per output file\n\
   -C, --line-bytes=SIZE   put at most SIZE bytes of lines per output file\n\
   -d, --numeric-suffixes  use numeric suffixes instead of alphabetic\n\
@@ -237,13 +248,15 @@ next_file_name (void)
       /* Allocate and initialize the first file name.  */
 
       size_t outbase_length = strlen (outbase);
-      size_t outfile_length = outbase_length + suffix_length;
+      size_t outfile_length = outbase_length + suffix_length + fixed_suffix_len;
       if (outfile_length + 1 < outbase_length)
         xalloc_die ();
       outfile = xmalloc (outfile_length + 1);
       outfile_mid = outfile + outbase_length;
       memcpy (outfile, outbase, outbase_length);
       memset (outfile_mid, suffix_alphabet[0], suffix_length);
+      if (fixed_suffix_len)
+        memcpy (outfile_mid + suffix_length, fixed_suffix, fixed_suffix_len);
       outfile[outfile_length] = 0;
       sufindex = xcalloc (suffix_length, sizeof *sufindex);
 
@@ -1036,6 +1049,13 @@ main (int argc, char **argv)
           }
           break;
 
+        case SUFFIX_OPTION:
+          {
+            fixed_suffix = optarg;
+            fixed_suffix_len = strlen (fixed_suffix);
+          }
+          break;
+
         case 'b':
           if (split_type != type_undef)
             FAIL_ONLY_ONE_WAY ();
diff --git a/tests/Makefile.am b/tests/Makefile.am
index a94aaa2..28a7ef6 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -254,6 +254,7 @@ TESTS =						\
   misc/sort-NaN-infloop				\
   split/filter					\
   split/suffix-length				\
+  split/suffix					\
   split/b-chunk					\
   split/fail					\
   split/lines					\
diff --git a/tests/split/suffix b/tests/split/suffix
new file mode 100755
index 0000000..8282359
--- /dev/null
+++ b/tests/split/suffix
@@ -0,0 +1,41 @@
+#!/bin/sh
+# show that 'split --suffix=SUFF' works.
+
+# Copyright (C) 2012 Free Software Foundation, Inc.
+
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+. "${srcdir=.}/init.sh"; path_prepend_ ../src
+print_ver_ split
+
+printf '1\n2\n3\n4\n5\n' > in || framework_failure_
+
+split --lines=2 --suffix=.txt in > out || fail=1
+cat <<\EOF > exp-1
+1
+2
+EOF
+cat <<\EOF > exp-2
+3
+4
+EOF
+cat <<\EOF > exp-3
+5
+EOF
+
+compare exp-1 xaa.txt || fail=1
+compare exp-2 xab.txt || fail=1
+compare exp-3 xac.txt || fail=1
+
+Exit $fail
-- 
1.7.2.5


--=-=-=--


From MAILER-DAEMON Mon Feb 06 08:56:26 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RuP3S-0000Nq-D3
	for mharc-coreutils@gnu.org; Mon, 06 Feb 2012 08:56:26 -0500
Received: from eggs.gnu.org ([140.186.70.92]:40059)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <sven.breuner@itwm.fraunhofer.de>) id 1RuOyL-0007in-QC
	for coreutils@gnu.org; Mon, 06 Feb 2012 08:51:14 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <sven.breuner@itwm.fraunhofer.de>) id 1RuOyD-0002w2-7G
	for coreutils@gnu.org; Mon, 06 Feb 2012 08:51:09 -0500
Received: from mailgw1.uni-kl.de ([131.246.120.220]:41801)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <sven.breuner@itwm.fraunhofer.de>) id 1RuOyC-0002vm-R9
	for coreutils@gnu.org; Mon, 06 Feb 2012 08:51:01 -0500
Received: from itwm2.itwm.fhg.de (itwm2.itwm.fhg.de [131.246.191.3])
	by mailgw1.uni-kl.de (8.14.3/8.14.3/Debian-5+lenny1) with ESMTP id
	q16Doq2P030867
	(version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=NOT)
	for <coreutils@gnu.org>; Mon, 6 Feb 2012 14:50:52 +0100
Received: from mail2.itwm.fhg.de ([131.246.191.79]:55677)
	by itwm2.itwm.fhg.de with esmtps (TLSv1:DES-CBC3-SHA:168)
	(/C=DE/ST=Rheinland-Pfalz/L=Kaiserslautern/O=Fraunhofer
	ITWM/OU=SLG/CN=mail2.itwm.fhg.de)(verified=1) (Exim 4.74 #1)
	id 1RuOy3-0003mY-S1
	for coreutils@gnu.org; Mon, 06 Feb 2012 14:50:51 +0100
Message-ID: <4F2FDABB.6080800@itwm.fraunhofer.de>
Date: Mon, 06 Feb 2012 14:50:51 +0100
From: Sven Breuner <sven.breuner@itwm.fraunhofer.de>
User-Agent: Mozilla/5.0 (X11; Linux i686 on x86_64;
	rv:10.0) Gecko/20120129 Thunderbird/10.0
MIME-Version: 1.0
To: coreutils@gnu.org
Subject: "ls -l": Avoid unnecessary getxattr() overhead
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
X-ITWM-CharSet: ISO-8859-1
X-ITWM-Scanned-By: mail2.itwm.fhg.de
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
X-Received-From: 131.246.120.220
X-Mailman-Approved-At: Mon, 06 Feb 2012 08:56:24 -0500
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 06 Feb 2012 13:51:14 -0000

Hi,

when strace'ing "ls -l" on Linux, I see that it runs three syscalls for 
every file in the directory, e.g.:

$ strace ls -l
...
lstat("test", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
lgetxattr("test", "security.selinux", 0x62d130, 255) = -1 ENODATA (No 
data available)
getxattr("test", "system.posix_acl_access", 0x0, 0) = -1 EOPNOTSUPP 
(Operation not supported)

lstat("test2", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
lgetxattr("test2", "security.selinux", 0x62b350, 255) = -1 ENODATA (No 
data available)
getxattr("test2", "system.posix_acl_access", 0x0, 0) = -1 EOPNOTSUPP 
(Operation not supported)

lstat("test3", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
lgetxattr("test3", "security.selinux", 0x62d150, 255) = -1 ENODATA (No 
data available)
getxattr("test3", "system.posix_acl_access", 0x0, 0) = -1 EOPNOTSUPP 
(Operation not supported)
...

The first operation is lstat(), which definitely makes sense. The other 
two operations are
1) lgetxattr(..., "security.selinux", ...): Doesn't make sense imho, 
because selinux isn't even enabled on the system on which I ran "ls -l".
2) getxattr(..., "system.posix_acl_access", ...): Doesn't make sense, as 
ACLs are not supported on this file system (=> EOPNOTSUPP).

While these extra calls might not really be a problem for local file 
systems, they are a real pain for remote file systems, such as NFS or 
FhGFS (on which I'm working) and probably also for FUSE-based file systems.
The problem is that these extra syscalls lead to an inode revalidation 
inside the (Linux) kernel and thus result in extra remote calls for 
every file, so it would be really great if they could be avoided after 
detecting that they don't make sense in a particular directory.

I don't want to start a philosophical discussion about large 
directories, but as a matter of fact, we often have users with tens of 
thousands of files inside a single directory and those users need to run 
"ls -l" in these directories frequently to monitor the status of thei 
compute job and things like that.
Currently, they need to wait three times longer than necessary to see 
the result, due to the two extra syscalls for each file.

What I would like to suggest is detecting whether these extra calls make 
sense and disable them if they don't make sense:
1) For lgetxattr(..., "security.selinux", ...), ls could simply check 
whether selinux is enabled and skip the calls if it isn't enabled.
2) For getxattr(..., "system.posix_acl_access", ...), ls could check 
whether the first call returns EOPNOTSUPP and then skip this call for 
the remaining directory entries, because those will just have the same 
return value.

Best regards,
Sven

---

Sven Breuner
Fraunhofer HPC
FhGFS Team
http://www.fhgfs.com


From MAILER-DAEMON Mon Feb 06 08:58:54 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RuP5q-0004Hu-78
	for mharc-coreutils@gnu.org; Mon, 06 Feb 2012 08:58:54 -0500
Received: from eggs.gnu.org ([140.186.70.92]:56376)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1RuP5h-0003re-4T
	for coreutils@gnu.org; Mon, 06 Feb 2012 08:58:51 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1RuP5b-0004QM-OU
	for coreutils@gnu.org; Mon, 06 Feb 2012 08:58:45 -0500
Received: from mail-we0-f169.google.com ([74.125.82.169]:37111)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1RuP5b-0004QE-CB
	for coreutils@gnu.org; Mon, 06 Feb 2012 08:58:39 -0500
Received: by wera13 with SMTP id a13so5559788wer.0
	for <coreutils@gnu.org>; Mon, 06 Feb 2012 05:58:38 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma;
	h=from:to:subject:cc:date:message-id:mime-version:content-type;
	bh=UXhUA3Mkw3QTBFYdslDPRV8XgvZF9sxn+HuXP/a5sko=;
	b=Y4eSNYiYIR/eFgU1xR09e6w/l3IXmIIHuY7r0/yZ29ZsiMnwFfSafe78G4wlITJY4Q
	6U7zQUsDLJNYDcE9bZ6WFnakBeNbLRHylcEHYFkRZeO8D6Q8jLJq9f02OvZ7N8WctjHy
	p0tAkf6i2Gr7/rVOkYqo5pTO4Bh95wrvPNwrc=
Received: by 10.216.139.147 with SMTP id c19mr7164951wej.11.1328536718410;
	Mon, 06 Feb 2012 05:58:38 -0800 (PST)
Received: from Apollo.jerryland.fr ([85.69.251.50])
	by mx.google.com with ESMTPS id y6sm13232892wix.10.2012.02.06.05.58.36
	(version=TLSv1/SSLv3 cipher=OTHER);
	Mon, 06 Feb 2012 05:58:37 -0800 (PST)
From: =?utf-8?Q?J=C3=A9r=C3=A9my_Compostella?= <jeremy.compostella@gmail.com>
To: =?utf-8?Q?P=C3=A1draig?= Brady <P@draigbrady.com>
Subject: Re: bug#9085: 'split' feature request: an option to uses e.g. '.001'
	as first suffix.
Date: Mon, 06 Feb 2012 14:58:55 +0100
Message-ID: <87vcnk3vxc.fsf@Apollo.jerryland.fr>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="=-=-="
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 74.125.82.169
Cc: 9085@debbugs.gnu.org, coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 06 Feb 2012 13:58:51 -0000

--=-=-=
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

P=C3=A1draig, all,

I took into account general comments on my commits. I attached the
improved patch for feature. Improvements are:
- Add by file description,
- I put back accents on my name (J=C3=A9r=C3=A9my instead of Jeremy),
- I referenced from who this feature was requested.

I added a new dedicated tests suite too.

Cheers,

J=C3=A9r=C3=A9my
---

--=-=-=
Content-Type: text/x-diff
Content-Disposition: inline;
 filename=0001-split-numeric-suffixes-new-optional-from-argument-bu.patch
Content-Description: split: --numeric-suffixes new optional "from" argument
 (bug#9085) patch

>From 71612b3319e1dbed84d1182470397718afccf846 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?J=C3=A9r=C3=A9my=20Compostella?= <jeremy.compostella@gmail.com>
Date: Sun, 29 Jan 2012 15:20:31 +0100
Subject: [PATCH] split: --numeric-suffixes new optional "from" argument (bug#9085)

The split command now accepts a new optional "from" argument for the
--numeric-suffixes option. If this argument is specified, the numeric
suffix counts from this value, otherwise, like before, it counts from
0.

* src/split.c (next_file_name): Initialize the suffix index
accordingly to the count from value.
(main): Check that the suffix length is greater enough for the
numerical suffix count from value.
* doc/coreutils.texi (split invocation): Mention it.
* NEWS (New features): Mention it.
* tests/split/numeric: New file. --numeric-suffixes[=FROM] option
  tests.
* tests/Makefile.am (TESTS): Add it.
Requested by SciFi.
---
 NEWS                |    7 ++++++
 doc/coreutils.texi  |    5 ++-
 src/split.c         |   60 ++++++++++++++++++++++++++++++++++++++++++++++----
 tests/Makefile.am   |    1 +
 tests/split/numeric |   61 +++++++++++++++++++++++++++++++++++++++++++++++++++
 5 files changed, 127 insertions(+), 7 deletions(-)
 create mode 100755 tests/split/numeric

diff --git a/NEWS b/NEWS
index 9eebbf6..f09d55d 100644
--- a/NEWS
+++ b/NEWS
@@ -2,6 +2,13 @@ GNU coreutils NEWS                                    -*- outline -*-
 
 * Noteworthy changes in release ?.? (????-??-??) [?]
 
+** New features
+
+   split now accept an optional "from" value for the
+   --numeric-suffixes option. If this argument is specified, the
+   numeric suffix counts from this value, otherwise, like before, it
+   counts from 0.
+
 ** Bug fixes
 
   mv now lets you move a symlink onto a same-inode destination file that
diff --git a/doc/coreutils.texi b/doc/coreutils.texi
index 52838e7..b9fb482 100644
--- a/doc/coreutils.texi
+++ b/doc/coreutils.texi
@@ -3084,10 +3084,11 @@ and so can be a pipe for example.
 Use suffixes of length @var{length}.  The default @var{length} is 2.
 
 @item -d
-@itemx --numeric-suffixes
+@itemx --numeric-suffixes[=@var{from}]
 @opindex -d
 @opindex --numeric-suffixes
-Use digits in suffixes rather than lower-case letters.
+Use digits in suffixes rather than lower-case letters. The numerical
+suffix counts from @var{from} if specified, 0 otherwise.
 
 @item -e
 @itemx --elide-empty-files
diff --git a/src/split.c b/src/split.c
index 1d0310c..c8e1a51 100644
--- a/src/split.c
+++ b/src/split.c
@@ -80,6 +80,9 @@ static size_t suffix_length;
 /* Alphabet of characters to use in suffix.  */
 static char const *suffix_alphabet = "abcdefghijklmnopqrstuvwxyz";
 
+/* Numerical suffix count from value.  */
+static unsigned long suffix_count_from;
+
 /* Name of input file.  May be "-".  */
 static char *infile;
 
@@ -122,7 +125,7 @@ static struct option const longopts[] =
   {"elide-empty-files", no_argument, NULL, 'e'},
   {"unbuffered", no_argument, NULL, 'u'},
   {"suffix-length", required_argument, NULL, 'a'},
-  {"numeric-suffixes", no_argument, NULL, 'd'},
+  {"numeric-suffixes", optional_argument, NULL, 'd'},
   {"filter", required_argument, NULL, FILTER_OPTION},
   {"verbose", no_argument, NULL, VERBOSE_OPTION},
   {"-io-blksize", required_argument, NULL,
@@ -195,7 +198,8 @@ Mandatory arguments to long options are mandatory for short options too.\n\
   -a, --suffix-length=N   use suffixes of length N (default %d)\n\
   -b, --bytes=SIZE        put SIZE bytes per output file\n\
   -C, --line-bytes=SIZE   put at most SIZE bytes of lines per output file\n\
-  -d, --numeric-suffixes  use numeric suffixes instead of alphabetic\n\
+  -d, --numeric-suffixes[=FROM]  use numeric suffixes instead of alphabetic.\n\
+                           When specified, start counting from FROM, 0 otherwise\n\
   -e, --elide-empty-files  do not generate empty output files with '-n'\n\
       --filter=COMMAND    write to shell COMMAND; file name is $FILE\n\
   -l, --lines=NUMBER      put NUMBER lines per output file\n\
@@ -231,6 +235,7 @@ next_file_name (void)
 {
   /* Index in suffix_alphabet of each character in the suffix.  */
   static size_t *sufindex;
+  size_t i = suffix_length;
 
   if (! outfile)
     {
@@ -243,9 +248,23 @@ next_file_name (void)
       outfile = xmalloc (outfile_length + 1);
       outfile_mid = outfile + outbase_length;
       memcpy (outfile, outbase, outbase_length);
-      memset (outfile_mid, suffix_alphabet[0], suffix_length);
-      outfile[outfile_length] = 0;
       sufindex = xcalloc (suffix_length, sizeof *sufindex);
+      /* Initialize the suffix index accordingly to the count from
+         value.  */
+      {
+        unsigned long left = suffix_count_from;
+        while (i-- != 0)
+          {
+            if (left)
+              {
+                sufindex[i] = left % 10;
+                left /= 10;
+              }
+            outfile_mid[i] = suffix_alphabet[sufindex[i]];
+          }
+      }
+
+      outfile[outfile_length] = 0;
 
 #if ! _POSIX_NO_TRUNC && HAVE_PATHCONF && defined _PC_NAME_MAX
       /* POSIX requires that if the output file name is too long for
@@ -265,7 +284,6 @@ next_file_name (void)
     {
       /* Increment the suffix in place, if possible.  */
 
-      size_t i = suffix_length;
       while (i-- != 0)
         {
           sufindex[i]++;
@@ -1142,6 +1160,18 @@ main (int argc, char **argv)
 
         case 'd':
           suffix_alphabet = "0123456789";
+          if (optarg)
+            {
+              unsigned long tmp;
+              if (xstrtoul (optarg, NULL, 10, &tmp, "") != LONGINT_OK)
+                {
+                  error (0, 0, _("%s: invalid count from numerical suffix number"),
+                         optarg);
+                  usage (EXIT_FAILURE);
+                }
+              else
+                suffix_count_from = tmp;
+            }
           break;
 
         case 'e':
@@ -1212,6 +1242,26 @@ main (int argc, char **argv)
       usage (EXIT_FAILURE);
     }
 
+  /* Check that the suffix length is greater enough for the numerical
+     suffix count from value.  */
+  if (suffix_count_from)
+    {
+      unsigned long start = suffix_count_from;
+      size_t length = suffix_length;
+
+      while (start)
+        {
+          if (length == 0)
+            {
+              error (0, 0, _("numerical suffix FROM number too hight\
+ for the suffix length"));
+              usage (EXIT_FAILURE);
+            }
+          start /= 10;
+          length--;
+        }
+    }
+
   /* Open the input file.  */
   if (! STREQ (infile, "-")
       && fd_reopen (STDIN_FILENO, infile, O_RDONLY, 0) < 0)
diff --git a/tests/Makefile.am b/tests/Makefile.am
index a94aaa2..62bbb66 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -259,6 +259,7 @@ TESTS =						\
   split/lines					\
   split/l-chunk					\
   split/r-chunk					\
+  split/numeric					\
   misc/stat-birthtime				\
   misc/stat-fmt					\
   misc/stat-hyphen				\
diff --git a/tests/split/numeric b/tests/split/numeric
new file mode 100755
index 0000000..5550d6f
--- /dev/null
+++ b/tests/split/numeric
@@ -0,0 +1,61 @@
+#!/bin/sh
+# Show that split --numeric-suffixes[=from] works.
+
+# Copyright (C) 2012 Free Software Foundation, Inc.
+
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+. "${srcdir=.}/init.sh"; path_prepend_ ../src
+print_ver_ split
+
+# Check default start from 0
+printf '1\n2\n3\n4\n5\n' > in || framework_failure_
+split --numeric-suffixes --lines=2 in > out || fail=1
+cat <<\EOF > exp-1
+1
+2
+EOF
+cat <<\EOF > exp-2
+3
+4
+EOF
+cat <<\EOF > exp-3
+5
+EOF
+compare exp-1 x00 || fail=1
+compare exp-2 x01 || fail=1
+compare exp-3 x02 || fail=1
+
+# Check --numeric-suffixes=X
+split --numeric-suffixes=1 --lines=2 in > out || fail=1
+cat <<\EOF > exp-1
+1
+2
+EOF
+cat <<\EOF > exp-2
+3
+4
+EOF
+cat <<\EOF > exp-3
+5
+EOF
+compare exp-1 x01 || fail=1
+compare exp-2 x02 || fail=1
+compare exp-3 x03 || fail=1
+
+# Check that split failed when suffix length is not greater enough for
+# the numerical suffix count from value
+fail=1 && split -a 3 --numeric-suffixes=1000 in > out 2> /dev/null || fail=0
+
+Exit $fail
-- 
1.7.2.5


--=-=-=--


From MAILER-DAEMON Mon Feb 06 09:01:12 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RuP84-0006WH-SJ
	for mharc-coreutils@gnu.org; Mon, 06 Feb 2012 09:01:12 -0500
Received: from eggs.gnu.org ([140.186.70.92]:52762)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1RuP7w-00063F-5L
	for coreutils@gnu.org; Mon, 06 Feb 2012 09:01:10 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1RuP7q-00058G-Tq
	for coreutils@gnu.org; Mon, 06 Feb 2012 09:01:04 -0500
Received: from mail-ww0-f49.google.com ([74.125.82.49]:39349)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1RuP7q-00058B-Km
	for coreutils@gnu.org; Mon, 06 Feb 2012 09:00:58 -0500
Received: by wgbdt13 with SMTP id dt13so4789623wgb.30
	for <coreutils@gnu.org>; Mon, 06 Feb 2012 06:00:57 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma;
	h=from:to:subject:cc:date:message-id:mime-version:content-type;
	bh=SqJh1GkA/gqQ1AyfOSoqtuFagvyZzi6clUdSgTSMKCY=;
	b=TK3mRQlUZ1L8vAvO9DeMTimxv5RXqWPnygoWJNmBYUcoG5WZK3Erhkh6SPHJ9hBvbn
	NoexRfb2y9dFAKB2TGYwhCSSIN7Om3LzOCkMk3IqHu0vyIQZZyVy5iI+J15DA1xh/HzN
	CUUSezAPoS3LWksQ2xQZV6KM5KqpOy1EodLyA=
Received: by 10.180.14.129 with SMTP id p1mr27636257wic.16.1328536857630;
	Mon, 06 Feb 2012 06:00:57 -0800 (PST)
Received: from Apollo.jerryland.fr ([85.69.251.50])
	by mx.google.com with ESMTPS id fv6sm46700900wib.8.2012.02.06.06.00.55
	(version=TLSv1/SSLv3 cipher=OTHER);
	Mon, 06 Feb 2012 06:00:56 -0800 (PST)
From: =?utf-8?Q?J=C3=A9r=C3=A9my_Compostella?= <jeremy.compostella@gmail.com>
To: =?utf-8?Q?P=C3=A1draig?= Brady <P@draigbrady.com>
Subject: Re: dd: add 'skip_bytes' and 'count_bytes' operands
Date: Mon, 06 Feb 2012 15:01:15 +0100
Message-ID: <87sjio3vtg.fsf@Apollo.jerryland.fr>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="=-=-="
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 74.125.82.49
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 06 Feb 2012 14:01:11 -0000

--=-=-=
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

P=C3=A1draig, all,

I took into account general comments on my commits. I attached the
improved patch for feature. Improvements are:
- Add by file description,
- I put back accents on my name (J=C3=A9r=C3=A9my instead of Jeremy),

Cheers,

J=C3=A9r=C3=A9my
---

--=-=-=
Content-Type: text/x-diff
Content-Disposition: inline;
 filename=0001-dd-add-skip_bytes-and-count_bytes-operands.patch
Content-Description: dd: add skip_bytes and count_bytes operands patch

>From 53c65c4bc125217b935c824c193449e592011a74 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?J=C3=A9r=C3=A9my=20Compostella?= <jeremy.compostella@gmail.com>
Date: Sat, 4 Feb 2012 15:25:54 +0100
Subject: [PATCH] dd: add skip_bytes and count_bytes operands

dd now accepts:
- the skip_bytes=N operand: When specified, dd skip N bytes before
  copying. 'skip' and 'skip_bytes" operands are mutually exclusive.
- the count_bytes=N operand: When specified, dd copy N bytes. 'count'
  and 'count_bytes' are mutually exclusive.

* src/dd.c (scanargs): Compute skip_records and skip_bytes when
'skip_bytes' operand is used. Compute max_records and max_bytes when
'count_bytes' operand is used. Raise a "mutually exclusive" error when
'skip' and 'skip_bytes' are both used or when 'count' and
'count_bytes' are both used.
(skip_via_lseek): Use new 'bytes' parameter and handle potential
'records' equals to zero. Update the header comments.
(dd_copy): Skip accordingly to skip_records AND skip_bytes. Count
accordingly to max_records AND max_bytes.
* NEWS (New features): Mention it.
* doc/coreutils.texi (New features): Detail new operands and
  behaviors.
* tests/dd/bytes: New file. Tests for these two new operands.
* tests/Makefile.am (TESTS): Add it.
---
 NEWS               |   10 +++++-
 doc/coreutils.texi |   11 ++++++
 src/dd.c           |   99 +++++++++++++++++++++++++++++++++++++++------------
 tests/Makefile.am  |    1 +
 tests/dd/bytes     |   37 +++++++++++++++++++
 5 files changed, 133 insertions(+), 25 deletions(-)
 create mode 100755 tests/dd/bytes

diff --git a/NEWS b/NEWS
index 9eebbf6..adf6140 100644
--- a/NEWS
+++ b/NEWS
@@ -2,6 +2,15 @@ GNU coreutils NEWS                                    -*- outline -*-
 
 * Noteworthy changes in release ?.? (????-??-??) [?]
 
+** New features
+
+  dd now accepts:
+  - the skip_bytes=N operand. When specified, dd skip N
+    bytes before copying. 'skip' and 'skip_bytes' operands are mutually
+    exclusive.
+  - the count_bytes=N operand. When specified, dd copy N bytes. 'count'
+    and 'count_bytes' are mutually exclusive.
+
 ** Bug fixes
 
   mv now lets you move a symlink onto a same-inode destination file that
@@ -11,7 +20,6 @@ GNU coreutils NEWS                                    -*- outline -*-
   referent, there is no risk of data loss, since the symlink will
   typically still point to one of the hard links.
 
-
 * Noteworthy changes in release 8.15 (2012-01-06) [stable]
 
 ** New programs
diff --git a/doc/coreutils.texi b/doc/coreutils.texi
index 52838e7..e671a8b 100644
--- a/doc/coreutils.texi
+++ b/doc/coreutils.texi
@@ -8061,6 +8061,11 @@ use @var{bytes} as the fixed record length.
 @opindex skip
 Skip @var{blocks} @samp{ibs}-byte blocks in the input file before copying.
 
+@item skip_bytes=@var{n}
+@opindex skip_bytes
+Skip @var{n} bytes in the input file before copying.
+@samp{skip_bytes} and @samp{skip} are mutually exclusive.
+
 @item seek=@var{blocks}
 @opindex seek
 Skip @var{blocks} @samp{obs}-byte blocks in the output file before copying.
@@ -8070,6 +8075,12 @@ Skip @var{blocks} @samp{obs}-byte blocks in the output file before copying.
 Copy @var{blocks} @samp{ibs}-byte blocks from the input file, instead
 of everything until the end of the file.
 
+@item count_bytes=@var{n}
+@opindex count
+Copy @var{n} bytes from the input file, instead of everything until
+the end of the file. @samp{count} and @samp{count_bytes} are mutually
+exclusive.
+
 @item status=noxfer
 @opindex status
 Do not print the overall transfer rate and volume statistics
diff --git a/src/dd.c b/src/dd.c
index 9d24791..7c65653 100644
--- a/src/dd.c
+++ b/src/dd.c
@@ -156,12 +156,19 @@ static size_t conversion_blocksize = 0;
 /* Skip this many records of 'input_blocksize' bytes before input. */
 static uintmax_t skip_records = 0;
 
+/* Skip this many bytes before input in addition of 'skip_records'
+   records.  */
+static size_t skip_bytes = 0;
+
 /* Skip this many records of 'output_blocksize' bytes before output. */
 static uintmax_t seek_records = 0;
 
 /* Copy only this many records.  The default is effectively infinity.  */
 static uintmax_t max_records = (uintmax_t) -1;
 
+/* Copy this many bytes in addition to 'max_records' records.  */
+static size_t max_bytes = 0;
+
 /* Bit vector of conversions to apply. */
 static int conversions_mask = 0;
 
@@ -490,6 +497,8 @@ Copy a file, converting and formatting according to the operands.\n\
   cbs=BYTES       convert BYTES bytes at a time\n\
   conv=CONVS      convert the file as per the comma separated symbol list\n\
   count=BLOCKS    copy only BLOCKS input blocks\n\
+  count_bytes=N   copy only N bytes. count and count_bytes operands are mutually\n\
+                  exclusive\n\
   ibs=BYTES       read up to BYTES bytes at a time (default: 512)\n\
 "), stdout);
       fputs (_("\
@@ -500,6 +509,8 @@ Copy a file, converting and formatting according to the operands.\n\
   oflag=FLAGS     write as per the comma separated symbol list\n\
   seek=BLOCKS     skip BLOCKS obs-sized blocks at start of output\n\
   skip=BLOCKS     skip BLOCKS ibs-sized blocks at start of input\n\
+  skip_bytes=N    skip N bytes at start of input. skip and skip_bytes operands\n\
+                  are mutually exclusive\n\
   status=noxfer   suppress transfer statistics\n\
 "), stdout);
       fputs (_("\
@@ -1120,6 +1131,8 @@ scanargs (int argc, char *const *argv)
 {
   int i;
   size_t blocksize = 0;
+  uintmax_t count_in_bytes = 0;
+  uintmax_t skip_in_bytes = 0;
 
   for (i = optind; i < argc; i++)
     {
@@ -1176,10 +1189,14 @@ scanargs (int argc, char *const *argv)
             }
           else if (operand_is (name, "skip"))
             skip_records = n;
+          else if (operand_is (name, "skip_bytes"))
+            skip_in_bytes = n;
           else if (operand_is (name, "seek"))
             seek_records = n;
           else if (operand_is (name, "count"))
             max_records = n;
+          else if (operand_is (name, "count_bytes"))
+            count_in_bytes = n;
           else
             {
               error (0, 0, _("unrecognized operand %s"), quote (name));
@@ -1216,6 +1233,30 @@ scanargs (int argc, char *const *argv)
       usage (EXIT_FAILURE);
     }
 
+  if (skip_in_bytes != 0 && skip_records != 0)
+    {
+      error (0, 0, _("'skip_bytes' and 'skip' operands are mutually exclusive"));
+      usage (EXIT_FAILURE);
+    }
+
+  if (skip_in_bytes != 0)
+    {
+      skip_records = skip_in_bytes / input_blocksize;
+      skip_bytes = skip_in_bytes % input_blocksize;
+    }
+
+  if (count_in_bytes != 0 && max_records != (uintmax_t) -1)
+    {
+      error (0, 0, _("'count_bytes' and 'count' operands are mutually exclusive"));
+      usage (EXIT_FAILURE);
+    }
+
+  if (count_in_bytes != 0)
+    {
+      max_records = count_in_bytes / input_blocksize;
+      max_bytes = count_in_bytes % input_blocksize;
+    }
+
   /* Warn about partial reads if bs=SIZE is given and iflag=fullblock
      is not, and if counting or skipping bytes or using direct I/O.
      This helps to avoid confusion with miscounts, and to avoid issues
@@ -1411,18 +1452,19 @@ skip_via_lseek (char const *filename, int fdesc, off_t offset, int whence)
 # define skip_via_lseek(Filename, Fd, Offset, Whence) lseek (Fd, Offset, Whence)
 #endif
 
-/* Throw away RECORDS blocks of BLOCKSIZE bytes on file descriptor FDESC,
-   which is open with read permission for FILE.  Store up to BLOCKSIZE
-   bytes of the data at a time in BUF, if necessary.  RECORDS must be
-   nonzero.  If fdesc is STDIN_FILENO, advance the input offset.
-   Return the number of records remaining, i.e., that were not skipped
-   because EOF was reached.  */
+/* Throw away RECORDS blocks of BLOCKSIZE bytes plus BYTES bytes on
+   file descriptor FDESC, which is open with read permission for FILE.
+   Store up to BLOCKSIZE bytes of the data at a time in BUF, if
+   necessary. RECORDS or BYTES must be nonzero. If FDESC is
+   STDIN_FILENO, advance the input offset. Return the number of
+   records remaining, i.e., that were not skipped because EOF was
+   reached.  */
 
 static uintmax_t
 skip (int fdesc, char const *file, uintmax_t records, size_t blocksize,
-      char *buf)
+      size_t bytes, char *buf)
 {
-  uintmax_t offset = records * blocksize;
+  uintmax_t offset = records * blocksize + bytes;
 
   /* Try lseek and if an error indicates it was an inappropriate operation --
      or if the file offset is not representable as an off_t --
@@ -1491,29 +1533,35 @@ skip (int fdesc, char const *file, uintmax_t records, size_t blocksize,
 
       do
         {
-          ssize_t nread = iread_fnc (fdesc, buf, blocksize);
+          ssize_t nread;
+          if (records != 0)
+            nread = iread_fnc (fdesc, buf, blocksize);
+          else
+            nread = iread_fnc (fdesc, buf, bytes);
+
           if (nread < 0)
             {
               if (fdesc == STDIN_FILENO)
                 {
                   error (0, errno, _("reading %s"), quote (file));
                   if (conversions_mask & C_NOERROR)
-                    {
-                      print_stats ();
-                      continue;
-                    }
+                    print_stats ();
                 }
               else
                 error (0, lseek_errno, _("%s: cannot seek"), quote (file));
               quit (EXIT_FAILURE);
             }
-
-          if (nread == 0)
+          else if (nread == 0)
             break;
-          if (fdesc == STDIN_FILENO)
+          else if (fdesc == STDIN_FILENO)
             advance_input_offset (nread);
+
+          if (records != 0)
+            --records;
+          else
+            bytes = 0;
         }
-      while (--records != 0);
+      while (records != 0 || bytes != 0);
 
       return records;
     }
@@ -1777,11 +1825,11 @@ dd_copy (void)
       obuf = ibuf;
     }
 
-  if (skip_records != 0)
+  if (skip_records != 0 || skip_bytes != 0)
     {
-      uintmax_t us_bytes = input_offset + (skip_records * input_blocksize);
+      uintmax_t us_bytes = input_offset + (skip_records * input_blocksize) + skip_bytes;
       uintmax_t us_blocks = skip (STDIN_FILENO, input_file,
-                                  skip_records, input_blocksize, ibuf);
+                                  skip_records, input_blocksize, skip_bytes, ibuf);
       us_bytes -= input_offset;
 
       /* POSIX doesn't say what to do when dd detects it has been
@@ -1800,7 +1848,7 @@ dd_copy (void)
   if (seek_records != 0)
     {
       uintmax_t write_records = skip (STDOUT_FILENO, output_file,
-                                      seek_records, output_blocksize, obuf);
+                                      seek_records, output_blocksize, 0, obuf);
 
       if (write_records != 0)
         {
@@ -1819,12 +1867,12 @@ dd_copy (void)
         }
     }
 
-  if (max_records == 0)
+  if (max_records == 0 && max_bytes == 0)
     return exit_status;
 
   while (1)
     {
-      if (r_partial + r_full >= max_records)
+      if (r_partial + r_full >= max_records + (max_bytes ? 1 : 0))
         break;
 
       /* Zero the buffer before reading, so that if we get a read error,
@@ -1835,7 +1883,10 @@ dd_copy (void)
                 (conversions_mask & (C_BLOCK | C_UNBLOCK)) ? ' ' : '\0',
                 input_blocksize);
 
-      nread = iread_fnc (STDIN_FILENO, ibuf, input_blocksize);
+      if (r_partial + r_full >= max_records)
+        nread = iread_fnc (STDIN_FILENO, ibuf, max_bytes);
+      else
+        nread = iread_fnc (STDIN_FILENO, ibuf, input_blocksize);
 
       if (nread >= 0 && i_nocache)
         invalidate_cache (STDIN_FILENO, nread);
diff --git a/tests/Makefile.am b/tests/Makefile.am
index a94aaa2..31bb050 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -369,6 +369,7 @@ TESTS =						\
   dd/reblock					\
   dd/skip-seek					\
   dd/skip-seek2					\
+  dd/bytes					\
   dd/skip-seek-past-file			\
   dd/stderr					\
   dd/unblock					\
diff --git a/tests/dd/bytes b/tests/dd/bytes
new file mode 100755
index 0000000..d0c72f8
--- /dev/null
+++ b/tests/dd/bytes
@@ -0,0 +1,37 @@
+#!/bin/sh
+
+# Copyright (C) 2012 Free Software Foundation, Inc.
+
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+. "${srcdir=.}/init.sh"; path_prepend_ ../src
+print_ver_ dd
+
+# count_bytes
+echo 0123456789abcefghijklm > in || fail=1
+(dd count_bytes=14 conv=swab) < in > out 2> /dev/null || fail=1
+case `cat out` in
+  1032547698baec) ;;
+  *) fail=1 ;;
+esac
+
+# skip_bytes
+echo 0123456789abcefghijklm > in || fail=1
+(dd skip_bytes=10) < in > out 2> /dev/null || fail=1
+case `cat out` in
+  abcefghijklm) ;;
+  *) fail=1 ;;
+esac
+
+Exit $fail
-- 
1.7.2.5


--=-=-=--


From MAILER-DAEMON Mon Feb 06 09:51:47 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RuPv1-0004iL-9d
	for mharc-coreutils@gnu.org; Mon, 06 Feb 2012 09:51:47 -0500
Received: from eggs.gnu.org ([140.186.70.92]:39908)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RuPuy-0004gI-2E
	for coreutils@gnu.org; Mon, 06 Feb 2012 09:51:46 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RuPup-00084I-SM
	for coreutils@gnu.org; Mon, 06 Feb 2012 09:51:44 -0500
Received: from mx1.redhat.com ([209.132.183.28]:51988)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RuPup-000841-HE
	for coreutils@gnu.org; Mon, 06 Feb 2012 09:51:35 -0500
Received: from int-mx01.intmail.prod.int.phx2.redhat.com
	(int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id q16EpWpO019745
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK);
	Mon, 6 Feb 2012 09:51:32 -0500
Received: from [10.36.116.60] (ovpn-116-60.ams2.redhat.com [10.36.116.60])
	by int-mx01.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP
	id q16EpU5X027170
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO);
	Mon, 6 Feb 2012 09:51:31 -0500
Message-ID: <4F2FE8F0.5070007@draigBrady.com>
Date: Mon, 06 Feb 2012 14:51:28 +0000
From: =?UTF-8?B?UMOhZHJhaWcgQnJhZHk=?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:6.0) Gecko/20110816 Thunderbird/6.0
MIME-Version: 1.0
To: =?UTF-8?B?SsOpcsOpbXkgQ29tcG9zdGVsbGE=?=
 <jeremy.compostella@gmail.com>
Subject: Re: dd: add 'skip_bytes' and 'count_bytes' operands
References: <87sjio3vtg.fsf@Apollo.jerryland.fr>
In-Reply-To: <87sjio3vtg.fsf@Apollo.jerryland.fr>
X-Enigmail-Version: 1.3.2
Content-Type: text/plain; charset=UTF-8
X-Scanned-By: MIMEDefang 2.67 on 10.5.11.11
Content-Transfer-Encoding: quoted-printable
X-MIME-Autoconverted: from 8bit to quoted-printable by mx1.redhat.com id
	q16EpWpO019745
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.132.183.28
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 06 Feb 2012 14:51:46 -0000

On 02/06/2012 02:01 PM, J=C3=A9r=C3=A9my Compostella wrote:
> P=C3=A1draig, all,
>=20
> I took into account general comments on my commits. I attached the
> improved patch for feature. Improvements are:
> - Add by file description,
> - I put back accents on my name (J=C3=A9r=C3=A9my instead of Jeremy),

Great thanks.

Hmm, shouldn't there bee a seek_bytes param too for consistency?


I'd change the NEWS to something simpler like:
** New features
  dd now accepts the skip_bytes and count_bytes parameters,
  to more easily allow processing portions of a file.

This additional doc change is necessary I think:
--- a/doc/coreutils.texi
+++ b/doc/coreutils.texi
@@ -8354,8 +8354,10 @@ should not be too large---values larger than a few=
 megabytes
 are generally wasteful or (as in the gigabyte..exabyte case) downright
 counterproductive or error-inducing.

-Use different @command{dd} invocations to use different block sizes for
-skipping and I/O@.  For example, the following shell commands copy data
+To use different block sizes for skipping and I/O@
+you can use the @samp{skip_bytes} and @samp{seek_bytes} options,
+or the traditional method of separate @command{dd} invocations.
+For example, the following shell commands copy data
 in 512 KiB blocks between a disk and a tape, but do not save or restore =
a
 4 KiB label at the start of the disk:

Your change to skip() looks good, but I would suggest the
following style adjustment.
diff --git a/src/dd.c b/src/dd.c
index 7c65653..6798725 100644
--- a/src/dd.c
+++ b/src/dd.c
@@ -1533,12 +1533,7 @@ skip (int fdesc, char const *file, uintmax_t recor=
ds, size_t blocksize,

       do
         {
-          ssize_t nread;
-          if (records !=3D 0)
-            nread =3D iread_fnc (fdesc, buf, blocksize);
-          else
-            nread =3D iread_fnc (fdesc, buf, bytes);
-
+          ssize_t nread =3D iread_fnc (fdesc, buf, records ? blocksize :=
 bytes);
           if (nread < 0)
             {
               if (fdesc =3D=3D STDIN_FILENO)
@@ -1557,11 +1552,11 @@ skip (int fdesc, char const *file, uintmax_t reco=
rds, size_t blocksize,
             advance_input_offset (nread);

           if (records !=3D 0)
-            --records;
+            records--;
           else
             bytes =3D 0;
         }
-      while (records !=3D 0 || bytes !=3D 0);
+      while (records || bytes);

       return records;
     }



The test doesn't exercise the above, so I added
an extra part to do so. I also removed redundant
subshells, and used framework_failure_ where appropriate:
# count_bytes
echo 0123456789abcdefghijklm > in || framework_failure_
dd count_bytes=3D14 conv=3Dswab < in > out 2> /dev/null || fail=3D1
case `cat out` in
  1032547698badc) ;;
  *) fail=3D1 ;;
esac

# skip_bytes
echo 0123456789abcdefghijklm > in || framework_failure_
dd skip_bytes=3D10 < in > out 2> /dev/null || fail=3D1
case `cat out` in
  abcdefghijklm) ;;
  *) fail=3D1 ;;
esac

# skip records and bytes from pipe
echo 0123456789abcdefghijklm |
  dd skip_bytes=3D10 bs=3D2 > out 2> /dev/null || fail=3D1
case `cat out` in
  abcdefghijklm) ;;
  *) fail=3D1 ;;
esac


cheers,
P=C3=A1draig


From MAILER-DAEMON Mon Feb 06 10:09:31 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RuQCB-00013Q-84
	for mharc-coreutils@gnu.org; Mon, 06 Feb 2012 10:09:31 -0500
Received: from eggs.gnu.org ([140.186.70.92]:40566)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1RuQC3-0000zQ-CV
	for coreutils@gnu.org; Mon, 06 Feb 2012 10:09:28 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1RuQBp-00032u-Vt
	for coreutils@gnu.org; Mon, 06 Feb 2012 10:09:23 -0500
Received: from mail-wi0-f169.google.com ([209.85.212.169]:44831)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1RuQBp-00032m-Qt
	for coreutils@gnu.org; Mon, 06 Feb 2012 10:09:09 -0500
Received: by wibhj13 with SMTP id hj13so5976099wib.0
	for <coreutils@gnu.org>; Mon, 06 Feb 2012 07:09:09 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma;
	h=from:to:subject:cc:date:message-id:mime-version:content-type
	:content-transfer-encoding;
	bh=IR5h7zgqZFjfZJ2dS0Sg137AQJ6PbtzkOOU9N7Am+64=;
	b=t3tSsryQXXWVa8XepF+NBQWxFJtVrKEgMyXGUsRJLiTvkFn3nwX7e9az+S4WHRrHdO
	ksfyW9qP8jfyi1KADmNs+Ck1oS9mkOoYl3GNEkhh6QsVr4JzMHfnPdqJ6E+Y1tXXu5jU
	r/CGzHR0yiC3y3dQloHwjp8dXdhjUlznF0OEY=
Received: by 10.180.96.8 with SMTP id do8mr13444844wib.21.1328540949025;
	Mon, 06 Feb 2012 07:09:09 -0800 (PST)
Received: from Apollo.jerryland.fr ([85.69.251.50])
	by mx.google.com with ESMTPS id l6sm25840801wiv.11.2012.02.06.07.09.07
	(version=TLSv1/SSLv3 cipher=OTHER);
	Mon, 06 Feb 2012 07:09:07 -0800 (PST)
From: =?utf-8?Q?J=C3=A9r=C3=A9my_Compostella?= <jeremy.compostella@gmail.com>
To: Sven Breuner <sven.breuner@itwm.fraunhofer.de>
Subject: Re: "ls -l": Avoid unnecessary getxattr() overhead
Date: Mon, 06 Feb 2012 16:09:26 +0100
Message-ID: <87pqds3snt.fsf@Apollo.jerryland.fr>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.85.212.169
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 06 Feb 2012 15:09:29 -0000

Sven,

As I'm stuck home today (unusual snow at Bordeaux, France), I made some
tests:

1. time ls -l > /dev/null > /dev/null (with a 10 thousands files in
   current directory over NFS)

  a) With coreutils 8.5-1

real	0m4.242s
user	0m0.132s
sys	0m0.504s

  b) With coreutils devel 8.15

real	0m4.162s
user	0m0.116s
sys	0m0.392s

2. strace -c ls -l > /dev/null (on the same directory)

  a) With coreutils 8.5-1

% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
 60.27    0.044688           4     10000           lstat
 37.77    0.028002        2546        11           getdents
  1.18    0.000876           0     20000     20000 getxattr
  0.65    0.000480           0     10000     10000 lgetxattr
  0.13    0.000095           0      9999           stat
  0.00    0.000000           0        19           read
  0.00    0.000000           0       122           write
[...]

  b) With coreutils devel 8.15

% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
 65.22    0.037658           4     10000           lstat
 34.64    0.020002        1818        11           getdents
  0.13    0.000076           0      9999           stat
  0.00    0.000000           0        13           read
  0.00    0.000000           0       122           write
[...)

=3D> First, it looks like that the devel version does not call getxattr()
and lgetxattr() when unneeded. Second, the getxattr and lgetxattr
syscall are not the cause of the speed problem you talk about.

Cheers,

J=C3=A9r=C3=A9my


From MAILER-DAEMON Mon Feb 06 10:24:25 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RuQQb-0005Sd-NN
	for mharc-coreutils@gnu.org; Mon, 06 Feb 2012 10:24:25 -0500
Received: from eggs.gnu.org ([140.186.70.92]:34818)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1RuQQU-00051N-Ac
	for coreutils@gnu.org; Mon, 06 Feb 2012 10:24:24 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1RuQQM-0006D7-AA
	for coreutils@gnu.org; Mon, 06 Feb 2012 10:24:18 -0500
Received: from mail-wi0-f169.google.com ([209.85.212.169]:35451)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1RuQQM-0006Cx-4S
	for coreutils@gnu.org; Mon, 06 Feb 2012 10:24:10 -0500
Received: by wibhj13 with SMTP id hj13so5992082wib.0
	for <coreutils@gnu.org>; Mon, 06 Feb 2012 07:24:08 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma;
	h=from:to:subject:cc:date:message-id:mime-version:content-type
	:content-transfer-encoding;
	bh=e6Yk7Hu3fnmXyPfbPRPfYAgIk8qL+kyAEDthO5if+Eo=;
	b=vuzzlXmYddK14phsVnQguDvP4aNtghHZFo03VbOaPsM26hHFMKRqGGlkFw2Y/Z8UIc
	cUHpckWDFYZ65WtYTK7z7SyxhB9PB64dWLSUPnmeRcBfjSGzD3XP1PK5AypvdkJC7Z2x
	V/aJDHRzpYVbmM47FPxjrJQ540Tbx2H1S083I=
Received: by 10.180.82.227 with SMTP id l3mr26261135wiy.1.1328541848653;
	Mon, 06 Feb 2012 07:24:08 -0800 (PST)
Received: from Apollo.jerryland.fr ([85.69.251.50])
	by mx.google.com with ESMTPS id cb8sm25977082wib.0.2012.02.06.07.24.07
	(version=TLSv1/SSLv3 cipher=OTHER);
	Mon, 06 Feb 2012 07:24:07 -0800 (PST)
From: =?utf-8?Q?J=C3=A9r=C3=A9my_Compostella?= <jeremy.compostella@gmail.com>
To: =?utf-8?Q?P=C3=A1draig?= Brady <P@draigbrady.com>
Subject: Re: dd: add 'skip_bytes' and 'count_bytes' operands
Date: Mon, 06 Feb 2012 16:24:25 +0100
Message-ID: <87mx8w3ryu.fsf@Apollo.jerryland.fr>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.85.212.169
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 06 Feb 2012 15:24:24 -0000

> Hmm, shouldn't there be a seek_bytes param too for consistency?
That was effectively my first mail question. As you talk about it in
your explanation addition in coreutils.texi I guess I should start
implementing it ?

> I'd change the NEWS to something simpler like:
> ** New features
>  dd now accepts the skip_bytes and count_bytes parameters,
>  to more easily allow processing portions of a file.
OK.

> This additional doc change is necessary I think:
> --- a/doc/coreutils.texi
> +++ b/doc/coreutils.texi
> @@ -8354,8 +8354,10 @@ should not be too large---values larger than a few=
 megabytes
>  are generally wasteful or (as in the gigabyte..exabyte case) downright
>  counterproductive or error-inducing.
>=20
> -Use different @command{dd} invocations to use different block sizes for
> -skipping and I/O@.  For example, the following shell commands copy data
> +To use different block sizes for skipping and I/O@
> +you can use the @samp{skip_bytes} and @samp{seek_bytes} options,
> +or the traditional method of separate @command{dd} invocations.
> +For example, the following shell commands copy data
>  in 512 KiB blocks between a disk and a tape, but do not save or restore a
>  4 KiB label at the start of the disk:
OK. I would add something about count_bytes too, no ?

> Your change to skip() looks good, but I would suggest the
> following style adjustment.
> diff --git a/src/dd.c b/src/dd.c
> index 7c65653..6798725 100644
> --- a/src/dd.c
> +++ b/src/dd.c
> @@ -1533,12 +1533,7 @@ skip (int fdesc, char const *file, uintmax_t recor=
ds, size_t blocksize,
>=20
>       do
>         {
> -          ssize_t nread;
> -          if (records !=3D 0)
> -            nread =3D iread_fnc (fdesc, buf, blocksize);
> -          else
> -            nread =3D iread_fnc (fdesc, buf, bytes);
> -
> +          ssize_t nread =3D iread_fnc (fdesc, buf, records ? blocksize :=
 bytes);
>           if (nread < 0)
>             {
>               if (fdesc =3D=3D STDIN_FILENO)
> @@ -1557,11 +1552,11 @@ skip (int fdesc, char const *file, uintmax_t reco=
rds, size_t blocksize,
>             advance_input_offset (nread);
>=20
>           if (records !=3D 0)
> -            --records;
> +            records--;
>           else
>             bytes =3D 0;
>         }
> -      while (records !=3D 0 || bytes !=3D 0);
> +      while (records || bytes);
>=20
>       return records;
>     }
OK.

> The test doesn't exercise the above, so I added
> an extra part to do so. I also removed redundant
> subshells, and used framework_failure_ where appropriate:
> # count_bytes
> echo 0123456789abcdefghijklm > in || framework_failure_
> dd count_bytes=3D14 conv=3Dswab < in > out 2> /dev/null || fail=3D1
> case `cat out` in
>  1032547698badc) ;;
>  *) fail=3D1 ;;
> esac
>=20
> # skip_bytes
> echo 0123456789abcdefghijklm > in || framework_failure_
> dd skip_bytes=3D10 < in > out 2> /dev/null || fail=3D1
> case `cat out` in
>  abcdefghijklm) ;;
>  *) fail=3D1 ;;
> esac
>=20
> # skip records and bytes from pipe
> echo 0123456789abcdefghijklm |
>  dd skip_bytes=3D10 bs=3D2 > out 2> /dev/null || fail=3D1
> case `cat out` in
>  abcdefghijklm) ;;
>  *) fail=3D1 ;;
> esac

Should I import your patch suggestions in my patch or do you will make
an additional commit on top of it ? That's particularly relevant if I
implement the seek_bytes operand.

Cheers,

J=C3=A9r=C3=A9my


From MAILER-DAEMON Mon Feb 06 10:47:14 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RuQmg-0002zn-Dx
	for mharc-coreutils@gnu.org; Mon, 06 Feb 2012 10:47:14 -0500
Received: from eggs.gnu.org ([140.186.70.92]:51632)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RuQmZ-0002zR-2G
	for coreutils@gnu.org; Mon, 06 Feb 2012 10:47:12 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RuQmT-0003At-2S
	for coreutils@gnu.org; Mon, 06 Feb 2012 10:47:06 -0500
Received: from mx1.redhat.com ([209.132.183.28]:51113)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RuQmS-0003Ae-Pp
	for coreutils@gnu.org; Mon, 06 Feb 2012 10:47:01 -0500
Received: from int-mx01.intmail.prod.int.phx2.redhat.com
	(int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id q16Fkwfv023950
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK);
	Mon, 6 Feb 2012 10:46:58 -0500
Received: from [10.36.116.60] (ovpn-116-60.ams2.redhat.com [10.36.116.60])
	by int-mx01.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP
	id q16FkumZ015177
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO);
	Mon, 6 Feb 2012 10:46:57 -0500
Message-ID: <4F2FF5EE.9010906@draigBrady.com>
Date: Mon, 06 Feb 2012 15:46:54 +0000
From: =?UTF-8?B?UMOhZHJhaWcgQnJhZHk=?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:6.0) Gecko/20110816 Thunderbird/6.0
MIME-Version: 1.0
To: =?UTF-8?B?SsOpcsOpbXkgQ29tcG9zdGVsbGE=?=
 <jeremy.compostella@gmail.com>
Subject: Re: dd: add 'skip_bytes' and 'count_bytes' operands
References: <87mx8w3ryu.fsf@Apollo.jerryland.fr>
In-Reply-To: <87mx8w3ryu.fsf@Apollo.jerryland.fr>
X-Enigmail-Version: 1.3.2
Content-Type: text/plain; charset=UTF-8
X-Scanned-By: MIMEDefang 2.67 on 10.5.11.11
Content-Transfer-Encoding: quoted-printable
X-MIME-Autoconverted: from 8bit to quoted-printable by mx1.redhat.com id
	q16Fkwfv023950
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.132.183.28
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 06 Feb 2012 15:47:12 -0000

On 02/06/2012 03:24 PM, J=C3=A9r=C3=A9my Compostella wrote:
>> Hmm, shouldn't there be a seek_bytes param too for consistency?
> That was effectively my first mail question. As you talk about it in
> your explanation addition in coreutils.texi I guess I should start
> implementing it ?

I think so.

Note it annoys me to be adding 3 new options to dd,
though I can't think of how to simplify off hand.
Anyway if we do think of a better way it will probably only
involve minor interface code changes.

>> This additional doc change is necessary I think:
>> --- a/doc/coreutils.texi
>> +++ b/doc/coreutils.texi
>> @@ -8354,8 +8354,10 @@ should not be too large---values larger than a =
few megabytes
>>  are generally wasteful or (as in the gigabyte..exabyte case) downrigh=
t
>>  counterproductive or error-inducing.
>>
>> -Use different @command{dd} invocations to use different block sizes f=
or
>> -skipping and I/O@.  For example, the following shell commands copy da=
ta
>> +To use different block sizes for skipping and I/O@
>> +you can use the @samp{skip_bytes} and @samp{seek_bytes} options,
>> +or the traditional method of separate @command{dd} invocations.
>> +For example, the following shell commands copy data
>>  in 512 KiB blocks between a disk and a tape, but do not save or resto=
re a
>>  4 KiB label at the start of the disk:
> OK. I would add something about count_bytes too, no ?

Well this example was just about skipping,
but it's probably best to mention all related args there.
How about:

"
To process data that is at an offset or size that is not a
multiple of the I/O@ block size, you can use the @samp{skip_bytes},
@samp{seek_bytes} and @samp{count_bytes} options.  Alternatively
the traditional method of separate @command{dd} invocations can be used.
For example, the following shell commands copy data
"

> Should I import your patch suggestions in my patch or do you will make
> an additional commit on top of it ? That's particularly relevant if I
> implement the seek_bytes operand.

It's best if we can --amend everything into your patch.

cheers,
P=C3=A1draig.


From MAILER-DAEMON Mon Feb 06 13:48:29 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RuTc5-00058p-1a
	for mharc-coreutils@gnu.org; Mon, 06 Feb 2012 13:48:29 -0500
Received: from eggs.gnu.org ([140.186.70.92]:57006)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <sven.breuner@itwm.fraunhofer.de>) id 1RuTbz-00057m-D1
	for coreutils@gnu.org; Mon, 06 Feb 2012 13:48:27 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <sven.breuner@itwm.fraunhofer.de>) id 1RuTbs-00038s-4a
	for coreutils@gnu.org; Mon, 06 Feb 2012 13:48:23 -0500
Received: from mailgw1.uni-kl.de ([131.246.120.220]:60253)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <sven.breuner@itwm.fraunhofer.de>) id 1RuTbr-00038L-Pm
	for coreutils@gnu.org; Mon, 06 Feb 2012 13:48:16 -0500
Received: from itwm2.itwm.fhg.de (itwm2.itwm.fhg.de [131.246.191.3])
	by mailgw1.uni-kl.de (8.14.3/8.14.3/Debian-5+lenny1) with ESMTP id
	q16ImCYb002747
	(version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=NOT)
	for <coreutils@gnu.org>; Mon, 6 Feb 2012 19:48:13 +0100
Received: from mail2.itwm.fhg.de ([131.246.191.79]:59020)
	by itwm2.itwm.fhg.de with esmtps (TLSv1:DES-CBC3-SHA:168)
	(/C=DE/ST=Rheinland-Pfalz/L=Kaiserslautern/O=Fraunhofer
	ITWM/OU=SLG/CN=mail2.itwm.fhg.de)(verified=1) (Exim 4.74 #1)
	id 1RuTbo-00054M-FC; Mon, 06 Feb 2012 19:48:12 +0100
Message-ID: <4F30206B.1020802@itwm.fraunhofer.de>
Date: Mon, 06 Feb 2012 19:48:11 +0100
From: Sven Breuner <sven.breuner@itwm.fraunhofer.de>
User-Agent: Mozilla/5.0 (X11; Linux i686 on x86_64;
	rv:10.0) Gecko/20120129 Thunderbird/10.0
MIME-Version: 1.0
To: =?UTF-8?B?SsOpcsOpbXkgQ29tcG9zdGVsbGE=?=
 <jeremy.compostella@gmail.com>
Subject: Re: "ls -l": Avoid unnecessary getxattr() overhead
References: <87pqds3snt.fsf@Apollo.jerryland.fr>
In-Reply-To: <87pqds3snt.fsf@Apollo.jerryland.fr>
Content-Type: text/plain; charset=UTF-8; format=flowed
X-ITWM-CharSet: UTF-8
X-ITWM-Scanned-By: mail2.itwm.fhg.de
Content-Transfer-Encoding: quoted-printable
X-MIME-Autoconverted: from 8bit to quoted-printable by mailgw1.uni-kl.de id
	q16ImCYb002747
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
X-Received-From: 131.246.120.220
Cc: Bernd Schubert <bernd.schubert@itwm.fraunhofer.de>, coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 06 Feb 2012 18:48:27 -0000

Hi J=C3=A9r=C3=A9my,

thanks for running the tests.

J=C3=A9r=C3=A9my Compostella wrote on 02/06/2012 04:09 PM:
> As I'm stuck home today (unusual snow at Bordeaux, France), I made some
> tests:

Hm, Bordeaux - You're not working at the University, are you? Then you=20
would even have access to a fhgfs file system to run the tests ;)

> 1. time ls -l>  /dev/null>  /dev/null (with a 10 thousands files in
>     current directory over NFS)
>
>    a) With coreutils 8.5-1
>
> real	0m4.242s
>
>    b) With coreutils devel 8.15
>
> real	0m4.162s

Ok, nfs was probably not the best example (my fault), as it just uses=20
cached information and doesn't care much about revalidation. But there=20
are certainly other file systems out there, fhgfs being one of them,=20
which try to deliver up-to-date information and thus have to go an extra=20
mile here.
My tests were based on a fhgfs benchmark with bonnie++, in which bonnie=20
stat'ed all 60,0000 files in a directory in about 10-12 seconds and an=20
"ls -l" in that directory took >30 seconds over a GigE connection on=20
RHEL 6.2.


> 2. strace -c ls -l>  /dev/null (on the same directory)
>
>    a) With coreutils 8.5-1
> [...]
>
>    b) With coreutils devel 8.15
> [...]
>
> =3D>  First, it looks like that the devel version does not call getxatt=
r()
> and lgetxattr() when unneeded.

I don't see any obvious commit with "git log src/ls.c" that could have=20
changed this behaviour.
Did you compile coreutils 8.15 with selinux and acl support?

Best regards,
Sven


From MAILER-DAEMON Mon Feb 06 14:01:16 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RuToS-0007dC-0Q
	for mharc-coreutils@gnu.org; Mon, 06 Feb 2012 14:01:16 -0500
Received: from eggs.gnu.org ([140.186.70.92]:53263)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1RuToM-0007WU-Da
	for coreutils@gnu.org; Mon, 06 Feb 2012 14:01:14 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1RuToH-0005fC-DT
	for coreutils@gnu.org; Mon, 06 Feb 2012 14:01:10 -0500
Received: from mail-iy0-f169.google.com ([209.85.210.169]:54096)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1RuToG-0005ey-Q9
	for coreutils@gnu.org; Mon, 06 Feb 2012 14:01:05 -0500
Received: by iagz16 with SMTP id z16so11425731iag.0
	for <coreutils@gnu.org>; Mon, 06 Feb 2012 11:01:04 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma;
	h=mime-version:in-reply-to:references:from:date:message-id:subject:to
	:cc:content-type;
	bh=wJzr4wKLuuLq45xvVhncEA0wrHfVeOTI4tEILdeHkYk=;
	b=LSpQ4GIPpAdRDGmq5/LkdKjkGmF/pWi8I+RHYlaaH1iYAHIVd87P5a0H6EKGZxJcrA
	j1ywUMEuwo5kIpj3LiRc74C8vXu7lACRrkO1vDBPgnjmqpbtETBwXUNcQF+cD6mJEbIp
	xQzJNizyqZvM96TthxMHyUSznzdmiZHBEh7QY=
Received: by 10.42.235.137 with SMTP id kg9mr18097131icb.47.1328554864219;
	Mon, 06 Feb 2012 11:01:04 -0800 (PST)
MIME-Version: 1.0
Received: by 10.42.29.196 with HTTP; Mon, 6 Feb 2012 11:00:44 -0800 (PST)
In-Reply-To: <4F30206B.1020802@itwm.fraunhofer.de>
References: <87pqds3snt.fsf@Apollo.jerryland.fr>
	<4F30206B.1020802@itwm.fraunhofer.de>
From: =?ISO-8859-1?Q?J=E9r=E9my_Compostella?= <jeremy.compostella@gmail.com>
Date: Mon, 6 Feb 2012 20:00:44 +0100
Message-ID: <CA+TATJHorgsuY7pxmpxxAeWUHRP5MfFfzQP05o=1H6C4ttswdA@mail.gmail.com>
Subject: Re: "ls -l": Avoid unnecessary getxattr() overhead
To: Sven Breuner <sven.breuner@itwm.fraunhofer.de>
Content-Type: multipart/alternative; boundary=20cf3026697abf166104b850488d
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.85.210.169
Cc: Bernd Schubert <bernd.schubert@itwm.fraunhofer.de>, coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 06 Feb 2012 19:01:14 -0000

--20cf3026697abf166104b850488d
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

2012/2/6 Sven Breuner <sven.breuner@itwm.fraunhofer.de>

> Hi J=E9r=E9my,
>
> thanks for running the tests.
>
> J=E9r=E9my Compostella wrote on 02/06/2012 04:09 PM:
>
>  As I'm stuck home today (unusual snow at Bordeaux, France), I made some
>> tests:
>>
>
> Hm, Bordeaux - You're not working at the University, are you? Then you
> would even have access to a fhgfs file system to run the tests ;)
>
No, I don't.

>
>  1. time ls -l>  /dev/null>  /dev/null (with a 10 thousands files in
>>    current directory over NFS)
>>
>>   a) With coreutils 8.5-1
>>
>> real    0m4.242s
>>
>>   b) With coreutils devel 8.15
>>
>> real    0m4.162s
>>
>
> Ok, nfs was probably not the best example (my fault), as it just uses
> cached information and doesn't care much about revalidation. But there ar=
e
> certainly other file systems out there, fhgfs being one of them, which tr=
y
> to deliver up-to-date information and thus have to go an extra mile here.
> My tests were based on a fhgfs benchmark with bonnie++, in which bonnie
> stat'ed all 60,0000 files in a directory in about 10-12 seconds and an "l=
s
> -l" in that directory took >30 seconds over a GigE connection on RHEL 6.2=
.
>
>
>  2. strace -c ls -l>  /dev/null (on the same directory)
>>
>>   a) With coreutils 8.5-1
>> [...]
>>
>>   b) With coreutils devel 8.15
>> [...]
>>
>>
>> =3D>  First, it looks like that the devel version does not call getxattr=
()
>> and lgetxattr() when unneeded.
>>
>
> I don't see any obvious commit with "git log src/ls.c" that could have
> changed this behaviour.
> Did you compile coreutils 8.15 with selinux and acl support?
>
Looking at config.log, I can affirm that I don't compile with both of them
... ;)
And that perfectly explain what I did not find any commit relative to this
too.

Anyway, you should maybe provide a "strace -c" output in both cases in
order to know
how much would be the speed improvement.

Cheers,

J=E9r=E9my

--20cf3026697abf166104b850488d
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div class=3D"gmail_quote">2012/2/6 Sven Breuner <span dir=3D"ltr">&lt;<a h=
ref=3D"mailto:sven.breuner@itwm.fraunhofer.de">sven.breuner@itwm.fraunhofer=
.de</a>&gt;</span><br><blockquote class=3D"gmail_quote" style=3D"margin:0 0=
 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">

Hi J=E9r=E9my,<br>
<br>
thanks for running the tests.<br>
<br>
J=E9r=E9my Compostella wrote on 02/06/2012 04:09 PM:<div class=3D"im"><br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">
As I&#39;m stuck home today (unusual snow at Bordeaux, France), I made some=
<br>
tests:<br>
</blockquote>
<br></div>
Hm, Bordeaux - You&#39;re not working at the University, are you? Then you =
would even have access to a fhgfs file system to run the tests ;)<br></bloc=
kquote><div>No, I don&#39;t.=A0</div><blockquote class=3D"gmail_quote" styl=
e=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">


<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex"><div class=3D"im">
1. time ls -l&gt; =A0/dev/null&gt; =A0/dev/null (with a 10 thousands files =
in<br>
 =A0 =A0current directory over NFS)<br>
<br>
 =A0 a) With coreutils 8.5-1<br>
<br>
real =A0 =A00m4.242s<br>
<br></div><div class=3D"im">
 =A0 b) With coreutils devel 8.15<br>
<br>
real =A0 =A00m4.162s<br>
</div></blockquote>
<br>
Ok, nfs was probably not the best example (my fault), as it just uses cache=
d information and doesn&#39;t care much about revalidation. But there are c=
ertainly other file systems out there, fhgfs being one of them, which try t=
o deliver up-to-date information and thus have to go an extra mile here.<br=
>


My tests were based on a fhgfs benchmark with bonnie++, in which bonnie sta=
t&#39;ed all 60,0000 files in a directory in about 10-12 seconds and an &qu=
ot;ls -l&quot; in that directory took &gt;30 seconds over a GigE connection=
 on RHEL 6.2.<br>


<br>
<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex"><div class=3D"im">
2. strace -c ls -l&gt; =A0/dev/null (on the same directory)<br>
<br>
 =A0 a) With coreutils 8.5-1<br></div><div class=3D"im">
[...]<br>
<br>
 =A0 b) With coreutils devel 8.15<br></div>
[...]<div class=3D"im"><br>
<br>
=3D&gt; =A0First, it looks like that the devel version does not call getxat=
tr()<br>
and lgetxattr() when unneeded.<br>
</div></blockquote>
<br>
I don&#39;t see any obvious commit with &quot;git log src/ls.c&quot; that c=
ould have changed this behaviour.<br>
Did you compile coreutils 8.15 with selinux and acl support?<br></blockquot=
e><div>Looking at config.log, I can affirm that I don&#39;t compile with bo=
th of them ... ;)</div><div>And that perfectly explain what I did not find =
any commit relative to this too.</div>

<div><br></div><div>Anyway, you should maybe provide a &quot;strace -c&quot=
; output in both cases in order to know</div><div>how much would be the spe=
ed improvement.</div><div><br></div><div>Cheers,</div><div><br></div><div>

J=E9r=E9my</div></div><br>

--20cf3026697abf166104b850488d--


From MAILER-DAEMON Mon Feb 06 15:06:32 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RuUpc-0007C6-N3
	for mharc-coreutils@gnu.org; Mon, 06 Feb 2012 15:06:32 -0500
Received: from eggs.gnu.org ([140.186.70.92]:49124)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RuUpa-0007Bg-2u
	for coreutils@gnu.org; Mon, 06 Feb 2012 15:06:31 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RuUpU-0001hO-L5
	for coreutils@gnu.org; Mon, 06 Feb 2012 15:06:30 -0500
Received: from mail3.vodafone.ie ([213.233.128.45]:43052)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RuUpU-0001h1-Gi
	for coreutils@gnu.org; Mon, 06 Feb 2012 15:06:24 -0500
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: ApIBAMsxME9tTdw1/2dsb2JhbAAMN7JBAQEEDCYBRhALDRQlDwJGBg0BBwEBiAGwfgSLLgEEAgECAgkCAgEGBwQGAQsJLgIBAgGCcR0DDBcFUgIMg2QEmyKMYQ
Received: from unknown (HELO [192.168.1.79]) ([109.77.220.53])
	by mail3.vodafone.ie with ESMTP; 06 Feb 2012 20:06:22 +0000
Message-ID: <4F3032BC.50702@draigBrady.com>
Date: Mon, 06 Feb 2012 20:06:20 +0000
From: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:6.0) Gecko/20110816 Thunderbird/6.0
MIME-Version: 1.0
To: =?ISO-8859-1?Q?J=E9r=E9my_Compostella?=
 <jeremy.compostella@gmail.com>
Subject: Re: "ls -l": Avoid unnecessary getxattr() overhead
References: <87pqds3snt.fsf@Apollo.jerryland.fr>
	<4F30206B.1020802@itwm.fraunhofer.de>
	<CA+TATJHorgsuY7pxmpxxAeWUHRP5MfFfzQP05o=1H6C4ttswdA@mail.gmail.com>
In-Reply-To: <CA+TATJHorgsuY7pxmpxxAeWUHRP5MfFfzQP05o=1H6C4ttswdA@mail.gmail.com>
X-Enigmail-Version: 1.3.2
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 213.233.128.45
Cc: Sven Breuner <sven.breuner@itwm.fraunhofer.de>,
	Bernd Schubert <bernd.schubert@itwm.fraunhofer.de>, coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 06 Feb 2012 20:06:31 -0000

Thanks for taking a look Jérémy.
This seems to have some related info.
https://bugzilla.redhat.com/show_bug.cgi?id=662011
We might be able to cache something.
We'll investigate.

cheers,
Pádraig.


From MAILER-DAEMON Mon Feb 06 18:29:37 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RuY09-0004Ia-L9
	for mharc-coreutils@gnu.org; Mon, 06 Feb 2012 18:29:37 -0500
Received: from eggs.gnu.org ([140.186.70.92]:55116)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <sven.breuner@itwm.fraunhofer.de>) id 1RuY07-0004IJ-9j
	for coreutils@gnu.org; Mon, 06 Feb 2012 18:29:36 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <sven.breuner@itwm.fraunhofer.de>) id 1RuY06-000379-7X
	for coreutils@gnu.org; Mon, 06 Feb 2012 18:29:35 -0500
Received: from mailgw1.uni-kl.de ([131.246.120.220]:53490)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <sven.breuner@itwm.fraunhofer.de>) id 1RuY05-00036Y-MV
	for coreutils@gnu.org; Mon, 06 Feb 2012 18:29:34 -0500
Received: from itwm2.itwm.fhg.de (itwm2.itwm.fhg.de [131.246.191.3])
	by mailgw1.uni-kl.de (8.14.3/8.14.3/Debian-5+lenny1) with ESMTP id
	q16NTUXw032399
	(version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=NOT)
	for <coreutils@gnu.org>; Tue, 7 Feb 2012 00:29:31 +0100
Received: from mail2.itwm.fhg.de ([131.246.191.79]:60571)
	by itwm2.itwm.fhg.de with esmtps (TLSv1:DES-CBC3-SHA:168)
	(/C=DE/ST=Rheinland-Pfalz/L=Kaiserslautern/O=Fraunhofer
	ITWM/OU=SLG/CN=mail2.itwm.fhg.de)(verified=1) (Exim 4.74 #1)
	id 1RuY02-00066H-GL; Tue, 07 Feb 2012 00:29:30 +0100
Message-ID: <4F30625A.6000001@itwm.fraunhofer.de>
Date: Tue, 07 Feb 2012 00:29:30 +0100
From: Sven Breuner <sven.breuner@itwm.fraunhofer.de>
User-Agent: Mozilla/5.0 (X11; Linux i686 on x86_64;
	rv:10.0) Gecko/20120129 Thunderbird/10.0
MIME-Version: 1.0
To: =?ISO-8859-1?Q?J=E9r=E9my_Compostella?=
 <jeremy.compostella@gmail.com>
Subject: Re: "ls -l": Avoid unnecessary getxattr() overhead
References: <87pqds3snt.fsf@Apollo.jerryland.fr>
	<4F30206B.1020802@itwm.fraunhofer.de>
	<CA+TATJHorgsuY7pxmpxxAeWUHRP5MfFfzQP05o=1H6C4ttswdA@mail.gmail.com>
In-Reply-To: <CA+TATJHorgsuY7pxmpxxAeWUHRP5MfFfzQP05o=1H6C4ttswdA@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
X-ITWM-CharSet: ISO-8859-1
X-ITWM-Scanned-By: mail2.itwm.fhg.de
Content-Transfer-Encoding: quoted-printable
X-MIME-Autoconverted: from 8bit to quoted-printable by mailgw1.uni-kl.de id
	q16NTUXw032399
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
X-Received-From: 131.246.120.220
Cc: Bernd Schubert <bernd.schubert@itwm.fraunhofer.de>, coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 06 Feb 2012 23:29:36 -0000

J=E9r=E9my Compostella wrote on 02/06/2012 08:00 PM:
> Anyway, you should maybe provide a "strace -c" output in both cases in
> order to know
> how much would be the speed improvement.

on a directory with 61441 files:

$ strace -c ls -l >/dev/null
% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
  32.64    0.128275           2     61441           lstat
  27.77    0.109109           2     61441           lgetxattr
  27.46    0.107915           2     61441     61441 getxattr
  11.70    0.045993         613        75           getdents
[...]

Best regards,
Sven


From MAILER-DAEMON Mon Feb 06 18:32:48 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RuY3E-0004hr-FE
	for mharc-coreutils@gnu.org; Mon, 06 Feb 2012 18:32:48 -0500
Received: from eggs.gnu.org ([140.186.70.92]:55930)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <sven.breuner@itwm.fraunhofer.de>) id 1RuY3C-0004hd-7o
	for coreutils@gnu.org; Mon, 06 Feb 2012 18:32:47 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <sven.breuner@itwm.fraunhofer.de>) id 1RuY3A-0003XV-DW
	for coreutils@gnu.org; Mon, 06 Feb 2012 18:32:46 -0500
Received: from mailgw1.uni-kl.de ([131.246.120.220]:47618)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <sven.breuner@itwm.fraunhofer.de>) id 1RuY3A-0003XH-5k
	for coreutils@gnu.org; Mon, 06 Feb 2012 18:32:44 -0500
Received: from itwm2.itwm.fhg.de (itwm2.itwm.fhg.de [131.246.191.3])
	by mailgw1.uni-kl.de (8.14.3/8.14.3/Debian-5+lenny1) with ESMTP id
	q16NWhnh000969
	(version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=NOT)
	for <coreutils@gnu.org>; Tue, 7 Feb 2012 00:32:43 +0100
Received: from mail2.itwm.fhg.de ([131.246.191.79]:60588)
	by itwm2.itwm.fhg.de with esmtps (TLSv1:DES-CBC3-SHA:168)
	(/C=DE/ST=Rheinland-Pfalz/L=Kaiserslautern/O=Fraunhofer
	ITWM/OU=SLG/CN=mail2.itwm.fhg.de)(verified=1) (Exim 4.74 #1)
	id 1RuY38-00066z-Gr; Tue, 07 Feb 2012 00:32:42 +0100
Message-ID: <4F30631A.2070009@itwm.fraunhofer.de>
Date: Tue, 07 Feb 2012 00:32:42 +0100
From: Sven Breuner <sven.breuner@itwm.fraunhofer.de>
User-Agent: Mozilla/5.0 (X11; Linux i686 on x86_64;
	rv:10.0) Gecko/20120129 Thunderbird/10.0
MIME-Version: 1.0
To: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
Subject: Re: "ls -l": Avoid unnecessary getxattr() overhead
References: <87pqds3snt.fsf@Apollo.jerryland.fr>
	<4F30206B.1020802@itwm.fraunhofer.de>
	<CA+TATJHorgsuY7pxmpxxAeWUHRP5MfFfzQP05o=1H6C4ttswdA@mail.gmail.com>
	<4F3032BC.50702@draigBrady.com>
In-Reply-To: <4F3032BC.50702@draigBrady.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
X-ITWM-CharSet: ISO-8859-1
X-ITWM-Scanned-By: mail2.itwm.fhg.de
Content-Transfer-Encoding: quoted-printable
X-MIME-Autoconverted: from 8bit to quoted-printable by mailgw1.uni-kl.de id
	q16NWhnh000969
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
X-Received-From: 131.246.120.220
Cc: Bernd Schubert <bernd.schubert@itwm.fraunhofer.de>,
	=?ISO-8859-1?Q?J=E9r=E9my_Compostella?= <jeremy.compostella@gmail.com>,
	coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 06 Feb 2012 23:32:47 -0000

P=E1draig Brady wrote on 02/06/2012 09:06 PM:
> This seems to have some related info.
> https://bugzilla.redhat.com/show_bug.cgi?id=3D662011
> We might be able to cache something.
> We'll investigate.

Thanks, P=E1draig.

I assume that bugzilla report is only related on a very abstract level:
The reporter of that ticket is suggesting special behavior after a=20
getxattr(acl_stuff) call returned ENODATA, in which case following=20
getxattr(acl_stuff) calls might of course have other results.

I'm referring to getxattr(acl_stuff) calls that return EOPNOTSUPP,=20
because the file system does not support ACLs / extended attributes, in=20
which case all following getxattr(acl_stuff) calls will definitely have=20
the same result (plus the other lgetxattr(selinux_stuff) calls, which=20
simply cannot provide any interesting information when selinux is=20
completely disabled on a machine).

Best regards,
Sven


From MAILER-DAEMON Mon Feb 06 19:33:38 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RuZ06-0004sE-0I
	for mharc-coreutils@gnu.org; Mon, 06 Feb 2012 19:33:38 -0500
Received: from eggs.gnu.org ([140.186.70.92]:34288)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1RuZ00-0004r2-Kr
	for coreutils@gnu.org; Mon, 06 Feb 2012 19:33:36 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1RuYzw-0004qU-UW
	for coreutils@gnu.org; Mon, 06 Feb 2012 19:33:32 -0500
Received: from qmta05.emeryville.ca.mail.comcast.net ([76.96.30.48]:60427)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1RuYzw-0004qN-Lh
	for coreutils@gnu.org; Mon, 06 Feb 2012 19:33:28 -0500
Received: from omta01.emeryville.ca.mail.comcast.net ([76.96.30.11])
	by qmta05.emeryville.ca.mail.comcast.net with comcast
	id WoQd1i0030EPchoA5oZTcQ; Tue, 07 Feb 2012 00:33:27 +0000
Received: from office.redhat.com ([24.10.251.25])
	by omta01.emeryville.ca.mail.comcast.net with comcast
	id WoZP1i00h0ZdyUg8MoZSYs; Tue, 07 Feb 2012 00:33:26 +0000
From: Eric Blake <eblake@redhat.com>
To: coreutils@gnu.org
Subject: [PATCH] test: expose recent gnulib canonicalize bug
Date: Mon,  6 Feb 2012 17:33:23 -0700
Message-Id: <1328574803-25243-1-git-send-email-eblake@redhat.com>
X-Mailer: git-send-email 1.7.7.6
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 76.96.30.48
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Tue, 07 Feb 2012 00:33:36 -0000

https://lists.gnu.org/archive/html/bug-gnulib/2012-02/msg00038.html
detailed a couple of bugs in gnulib's canonicalize that were visible
through coreutils' readlink, but only on systems where // is distinct
from /.  This particular test assumes that POSIX will be fixed to
require canonicalization of a symlink containing just slashes will
behave as if slashes separating the symlink from the rest of the
name are elided (see http://austingroupbugs.net/view.php?id=541),
as that is the only useful (and current) behavior on Cygwin.  That is,

ln -s / root
ls root/dev

must list the contents of /dev, not //dev.

* tests/misc/readlink-root: New test.
* tests/Makefile.am (TESTS): Run it.
---

Partway through my promise to test my recent patches to realpath,
by starting with missing readlink coverage.

I have not yet had a chance to run coreutils under valgrind or
Coverity after configuring with gl_cv_double_slash_root=yes.

 tests/Makefile.am        |    1 +
 tests/misc/readlink-root |   92 ++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 93 insertions(+), 0 deletions(-)
 create mode 100644 tests/misc/readlink-root

diff --git a/tests/Makefile.am b/tests/Makefile.am
index a94aaa2..39b17ab 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -220,6 +220,7 @@ TESTS =						\
   misc/printf-surprise				\
   misc/pwd-long					\
   misc/readlink-fp-loop				\
+  misc/readlink-root				\
   misc/realpath					\
   misc/runcon-no-reorder			\
   misc/sha1sum					\
diff --git a/tests/misc/readlink-root b/tests/misc/readlink-root
new file mode 100644
index 0000000..3c30cec
--- /dev/null
+++ b/tests/misc/readlink-root
@@ -0,0 +1,92 @@
+#!/bin/sh
+# tests for canonicalize-existing mode (readlink -e) on /.
+
+# Copyright (C) 2012 Free Software Foundation, Inc.
+
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+. "${srcdir=.}/init.sh"; path_prepend_ ../src
+print_ver_ readlink
+
+stat_single=$(stat -c %d:%i /) || framework_failure_
+stat_double=$(stat -c %d:%i //) || framework_failure_
+double_slash=/
+if test x"$stat_single" = x"stat_double"; then
+  double_slash=//
+fi
+
+test -d /dev || framework_failure_
+
+ln -s / one || framework_failure_
+ln -s // two || framework_failure_
+ln -s /// three || framework_failure_
+ln -s /./..// one-dots || framework_failure_
+ln -s //./..// two-dots || framework_failure_
+ln -s ///./..// three-dots || framework_failure_
+ln -s /dev one-dev || framework_failure_
+ln -s //dev two-dev || framework_failure_
+ln -s ///dev three-dev || framework_failure_
+
+cat >exp <<EOF || framework_failure_
+/
+$double_slash
+/
+/
+$double_slash
+/
+/
+$double_slash
+/
+/
+$double_slash
+/
+/dev
+${double_slash}dev
+/dev
+/dev
+${double_slash}dev
+/dev
+/dev
+${double_slash}dev
+/dev
+EOF
+
+{
+  readlink -e / || fail=1
+  readlink -e // || fail=1
+  readlink -e /// || fail=1
+  readlink -e /.//.. || fail=1
+  readlink -e //.//.. || fail=1
+  readlink -e ///.//.. || fail=1
+  readlink -e one || fail=1
+  readlink -e two || fail=1
+  readlink -e three || fail=1
+  readlink -e one-dots || fail=1
+  readlink -e two-dots || fail=1
+  readlink -e three-dots || fail=1
+  readlink -e one-dev || fail=1
+  # We know /dev exists, but cannot assume //dev exists
+  readlink -f two-dev || fail=1
+  readlink -e three-dev || fail=1
+  readlink -e one/dev || fail=1
+  readlink -f two/dev || fail=1
+  readlink -e three/dev || fail=1
+  readlink -e one-dots/dev || fail=1
+  readlink -f two-dots/dev || fail=1
+  readlink -e three-dots/dev || fail=1
+} > out
+
+compare exp out || fail=1
+
+Exit $fail
-- 
1.7.7.6



From MAILER-DAEMON Mon Feb 06 22:37:02 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1Rubra-0002Sr-Fs
	for mharc-coreutils@gnu.org; Mon, 06 Feb 2012 22:37:02 -0500
Received: from eggs.gnu.org ([140.186.70.92]:45069)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <pengyu.ut@gmail.com>) id 1RubrY-0002Sk-3J
	for coreutils@gnu.org; Mon, 06 Feb 2012 22:37:01 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <pengyu.ut@gmail.com>) id 1RubrX-0006l8-1a
	for coreutils@gnu.org; Mon, 06 Feb 2012 22:36:59 -0500
Received: from mail-tul01m020-f169.google.com ([209.85.214.169]:60712)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <pengyu.ut@gmail.com>) id 1RubrW-0006kt-Ui
	for coreutils@gnu.org; Mon, 06 Feb 2012 22:36:59 -0500
Received: by obbta7 with SMTP id ta7so9758636obb.0
	for <coreutils@gnu.org>; Mon, 06 Feb 2012 19:36:58 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma;
	h=mime-version:date:message-id:subject:from:to:content-type;
	bh=/Irl03X9eFRl7tCTXmleJH5yMi86+iJtuuR9bGRr4+o=;
	b=dYKEyVidtsisl739eBR9nN8sue1ghKJmdBuGQECueeSvHQIh+S/l/9/jq7PIfAGsAe
	EtjUEAhUQNa9dgr72ZQqfY4GB5vgKzXPG+HXWY+/vPBKoMIHP+p8gn/3iHBQBhtvQi+O
	no6bVE+hsHbDV74ZCI5y56ehUCIVLxQh2OvrM=
MIME-Version: 1.0
Received: by 10.182.11.41 with SMTP id n9mr19427074obb.24.1328585818290; Mon,
	06 Feb 2012 19:36:58 -0800 (PST)
Received: by 10.182.75.202 with HTTP; Mon, 6 Feb 2012 19:36:58 -0800 (PST)
Date: Mon, 6 Feb 2012 21:36:58 -0600
Message-ID: <CABrM6wmAm=0btpW0JT_A4=8W-bi3PAZ8SDHv1sCjcBOXyQeimQ@mail.gmail.com>
Subject: Adding humanize_number to coreutiles?
From: Peng Yu <pengyu.ut@gmail.com>
To: Coreutils <coreutils@gnu.org>
Content-Type: text/plain; charset=ISO-8859-1
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.85.214.169
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Tue, 07 Feb 2012 03:37:01 -0000

Hi,

Several commands in coreutils have the -h option. I'm wondering
whether anybody in the develop team also thinks that it is worthwhile
to export it as a standalone command. If so, I'd recommend add such
convenient command in coreutiles. As I don't find it anywhere else as
a stand alone command.

http://siarzhuk.dyndns.org/haiku/doxygen/coreutils_2lib_2human_8c_source.html#l00154

-- 
Regards,
Peng


From MAILER-DAEMON Tue Feb 07 04:46:08 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1Ruhcm-0005KJ-TF
	for mharc-coreutils@gnu.org; Tue, 07 Feb 2012 04:46:08 -0500
Received: from eggs.gnu.org ([140.186.70.92]:38721)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Ruhch-0005K1-HN
	for coreutils@gnu.org; Tue, 07 Feb 2012 04:46:07 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Ruhcb-0001i7-UB
	for coreutils@gnu.org; Tue, 07 Feb 2012 04:46:03 -0500
Received: from mail3.vodafone.ie ([213.233.128.45]:45005)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Ruhcb-0001hy-Pg
	for coreutils@gnu.org; Tue, 07 Feb 2012 04:45:57 -0500
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: ApMBAAzyME9tTdw1/2dsb2JhbAAMN7JCAQEBBAECLwFGEAsNCwkWDwkDAgECARYvBg0BBwEBiAG7CosmDwEEAgECAgkCAgEGBwQGAQsJMAOCch0DDBcFUgIMg2UEkwSIIoM2iS0
Received: from unknown (HELO [192.168.1.79]) ([109.77.220.53])
	by mail3.vodafone.ie with ESMTP; 07 Feb 2012 09:45:54 +0000
Message-ID: <4F30F2D0.6010009@draigBrady.com>
Date: Tue, 07 Feb 2012 09:45:52 +0000
From: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:6.0) Gecko/20110816 Thunderbird/6.0
MIME-Version: 1.0
To: Peng Yu <pengyu.ut@gmail.com>
Subject: Re: Adding humanize_number to coreutiles?
References: <CABrM6wmAm=0btpW0JT_A4=8W-bi3PAZ8SDHv1sCjcBOXyQeimQ@mail.gmail.com>
In-Reply-To: <CABrM6wmAm=0btpW0JT_A4=8W-bi3PAZ8SDHv1sCjcBOXyQeimQ@mail.gmail.com>
X-Enigmail-Version: 1.3.2
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 213.233.128.45
Cc: Coreutils <coreutils@gnu.org>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Tue, 07 Feb 2012 09:46:07 -0000

On 02/07/2012 03:36 AM, Peng Yu wrote:
> Hi,
> 
> Several commands in coreutils have the -h option. I'm wondering
> whether anybody in the develop team also thinks that it is worthwhile
> to export it as a standalone command. If so, I'd recommend add such
> convenient command in coreutiles. As I don't find it anywhere else as
> a stand alone command.
> 
> http://siarzhuk.dyndns.org/haiku/doxygen/coreutils_2lib_2human_8c_source.html#l00154

I've needed such functionality many times.
I'm thinking a printf format would be best to expose this:
http://lists.gnu.org/archive/html/coreutils/2011-08/msg00029.html

%H seems like it might cause compat problems in future.
%{human} is more descriptive and extensible, so I'm leaning towards that.
Any other suggestions appreciated.

I'll work on it this week.

cheers,
Pádraig.


From MAILER-DAEMON Tue Feb 07 10:56:25 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RunP7-0000tO-Gm
	for mharc-coreutils@gnu.org; Tue, 07 Feb 2012 10:56:25 -0500
Received: from eggs.gnu.org ([140.186.70.92]:45139)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <pengyu.ut@gmail.com>) id 1RunP0-0000pj-DT
	for coreutils@gnu.org; Tue, 07 Feb 2012 10:56:23 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <pengyu.ut@gmail.com>) id 1RunOv-0000q4-SP
	for coreutils@gnu.org; Tue, 07 Feb 2012 10:56:18 -0500
Received: from mail-tul01m020-f169.google.com ([209.85.214.169]:38297)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <pengyu.ut@gmail.com>) id 1RunOv-0000py-PO
	for coreutils@gnu.org; Tue, 07 Feb 2012 10:56:13 -0500
Received: by obbta7 with SMTP id ta7so10765729obb.0
	for <coreutils@gnu.org>; Tue, 07 Feb 2012 07:56:13 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma;
	h=mime-version:date:message-id:subject:from:to:content-type;
	bh=QdAvgrpqqbSbF/9S/xMbREGIsg0C3ZvLG7+EIa58B5E=;
	b=tbcxis+K1Z8eVoUKAhQ/zAbt491o06En+J1IjnxbTlE0rw9QDe6sY1EXXDkYxnGXhA
	B35R8yCTwGX8RxD2RSoJ2v/dTYRqxhQUQog8NBRebro4lAAiD9A4N0c7ODdDn/nn/bln
	sBChwH7RMvqkhMFuJUxrQjKJsrRtYmKtxi8pw=
MIME-Version: 1.0
Received: by 10.182.38.7 with SMTP id c7mr21430918obk.44.1328630173082; Tue,
	07 Feb 2012 07:56:13 -0800 (PST)
Received: by 10.182.75.202 with HTTP; Tue, 7 Feb 2012 07:56:13 -0800 (PST)
Date: Tue, 7 Feb 2012 09:56:13 -0600
Message-ID: <CABrM6wnELk+kuX6jd11cY2DY4xjUUs1U5N7qYJmJSOVAt0NNEA@mail.gmail.com>
Subject: uniq --accumulate
From: Peng Yu <pengyu.ut@gmail.com>
To: Coreutils <coreutils@gnu.org>
Content-Type: text/plain; charset=ISO-8859-1
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.85.214.169
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Tue, 07 Feb 2012 15:56:23 -0000

Hi,

Suppose that I have a table of the following, where the last column is
a number. I'd like to accumulate the number of rows that are the same
for all the remaining columns.

A 1
A 3
X 2
X 3
Y 3

The result will be the following. Although this is can be easily done
by awk, etc, I'm wondering if it is worthwhile to add an additional
option '--accumulate' to uniq. I'd recommend adding such an option if
anybody in coreutils develop team also think this is useful.

A 4
X 5
Y 3

-- 
Regards,
Peng


From MAILER-DAEMON Tue Feb 07 11:00:59 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RunTX-0002oM-1W
	for mharc-coreutils@gnu.org; Tue, 07 Feb 2012 11:00:59 -0500
Received: from eggs.gnu.org ([140.186.70.92]:39218)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RunTO-0002mN-EG
	for coreutils@gnu.org; Tue, 07 Feb 2012 11:00:58 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RunTH-0001xd-E6
	for coreutils@gnu.org; Tue, 07 Feb 2012 11:00:50 -0500
Received: from mail3.vodafone.ie ([213.233.128.45]:46530)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RunTH-0001xA-3g
	for coreutils@gnu.org; Tue, 07 Feb 2012 11:00:43 -0500
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: ApMBAO9JMU9tTdw1/2dsb2JhbAAMN60rhSEBAQEEMgFGEAsNCwkWDwkDAgECAUUGDQEHAQHCRIs2AQQCAQICCQICAQYHBAYBCwkwAwGCcR0DDBcFUgIMg2UEmyaMYw
Received: from unknown (HELO [192.168.1.79]) ([109.77.220.53])
	by mail3.vodafone.ie with ESMTP; 07 Feb 2012 16:00:41 +0000
Message-ID: <4F314AA7.3060508@draigBrady.com>
Date: Tue, 07 Feb 2012 16:00:39 +0000
From: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:6.0) Gecko/20110816 Thunderbird/6.0
MIME-Version: 1.0
To: Peng Yu <pengyu.ut@gmail.com>
Subject: Re: uniq --accumulate
References: <CABrM6wnELk+kuX6jd11cY2DY4xjUUs1U5N7qYJmJSOVAt0NNEA@mail.gmail.com>
In-Reply-To: <CABrM6wnELk+kuX6jd11cY2DY4xjUUs1U5N7qYJmJSOVAt0NNEA@mail.gmail.com>
X-Enigmail-Version: 1.3.2
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 213.233.128.45
Cc: Coreutils <coreutils@gnu.org>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Tue, 07 Feb 2012 16:00:58 -0000

On 02/07/2012 03:56 PM, Peng Yu wrote:
> Hi,
> 
> Suppose that I have a table of the following, where the last column is
> a number. I'd like to accumulate the number of rows that are the same
> for all the remaining columns.
> 
> A 1
> A 3
> X 2
> X 3
> Y 3
> 
> The result will be the following. Although this is can be easily done
> by awk, etc, I'm wondering if it is worthwhile to add an additional
> option '--accumulate' to uniq. I'd recommend adding such an option if
> anybody in coreutils develop team also think this is useful.
> 
> A 4
> X 5
> Y 3
> 

Thanks for the suggestion,
but this is too specialized for coreutils I think.

cheers,
Pádraig.


From MAILER-DAEMON Tue Feb 07 11:21:17 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RunnB-0004kD-6c
	for mharc-coreutils@gnu.org; Tue, 07 Feb 2012 11:21:17 -0500
Received: from eggs.gnu.org ([140.186.70.92]:47965)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <assafgordon@gmail.com>) id 1Runn5-0004jM-3Z
	for coreutils@gnu.org; Tue, 07 Feb 2012 11:21:16 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <assafgordon@gmail.com>) id 1Runn3-0006Kb-Di
	for coreutils@gnu.org; Tue, 07 Feb 2012 11:21:11 -0500
Received: from mail-qy0-f169.google.com ([209.85.216.169]:39289)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <assafgordon@gmail.com>) id 1Runn3-0006KU-BO
	for coreutils@gnu.org; Tue, 07 Feb 2012 11:21:09 -0500
Received: by qcsd16 with SMTP id d16so4616191qcs.0
	for <coreutils@gnu.org>; Tue, 07 Feb 2012 08:21:08 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma;
	h=message-id:date:from:user-agent:mime-version:to:cc:subject
	:references:in-reply-to:content-type:content-transfer-encoding;
	bh=mUsNcxkrgXEujb08Upou36Oabw9aiz33RHLG0eOHbqI=;
	b=mm6Ern3hNeLSbSyNOymytdYgFVmTJvgqo7W8Mvd1YNeeW6UBXrvTxRhHEDYfrEFbo1
	QvAA+A4soXJ/Yp4eNFlKPKQdVYLheuJ0Gy6V0xiz8Uk6/HAQ/glFvXEl+5WEVm3Hsgzy
	z4zBO0cMMJQ4m9aQbfbZcUFO+VFLj1iXKzosA=
Received: by 10.224.188.140 with SMTP id da12mr22648591qab.49.1328631668677;
	Tue, 07 Feb 2012 08:21:08 -0800 (PST)
Received: from [143.48.11.9] (tango.cshl.edu. [143.48.11.9])
	by mx.google.com with ESMTPS id eb5sm41732511qab.10.2012.02.07.08.21.07
	(version=TLSv1/SSLv3 cipher=OTHER);
	Tue, 07 Feb 2012 08:21:08 -0800 (PST)
Message-ID: <4F314F73.7020401@gmail.com>
Date: Tue, 07 Feb 2012 11:21:07 -0500
From: Assaf Gordon <assafgordon@gmail.com>
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US;
	rv:1.9.2.21) Gecko/20110831 Icedove/3.1.13
MIME-Version: 1.0
To: Peng Yu <pengyu.ut@gmail.com>
Subject: Re: uniq --accumulate
References: <CABrM6wnELk+kuX6jd11cY2DY4xjUUs1U5N7qYJmJSOVAt0NNEA@mail.gmail.com>
	<4F314AA7.3060508@draigBrady.com>
In-Reply-To: <4F314AA7.3060508@draigBrady.com>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.85.216.169
Cc: Coreutils <coreutils@gnu.org>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Tue, 07 Feb 2012 16:21:16 -0000

P=E1draig Brady wrote, On 02/07/2012 11:00 AM:
> On 02/07/2012 03:56 PM, Peng Yu wrote:
>>
>> Suppose that I have a table of the following, where the last column is=

>> a number. I'd like to accumulate the number of rows that are the same
>> for all the remaining columns.
>>
>=20
> Thanks for the suggestion,
> but this is too specialized for coreutils I think.

Slightly off-topic for coreutils,

but a package called "BEDTools" ( http://code.google.com/p/bedtools/ ) pr=
ovides a program called "groupBy", which does exactly that, and more.

Akin to SQL's "group by" command, the program can group a text file by a =
specified column, and perform operations (count,sum,mean,median,etc.) on =
another column.

-gordon



From MAILER-DAEMON Tue Feb 07 11:43:28 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1Ruo8e-00017e-CC
	for mharc-coreutils@gnu.org; Tue, 07 Feb 2012 11:43:28 -0500
Received: from eggs.gnu.org ([140.186.70.92]:45500)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <pengyu.ut@gmail.com>) id 1Ruo8c-00016a-15
	for coreutils@gnu.org; Tue, 07 Feb 2012 11:43:27 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <pengyu.ut@gmail.com>) id 1Ruo8W-0002RY-Bc
	for coreutils@gnu.org; Tue, 07 Feb 2012 11:43:25 -0500
Received: from mail-gy0-f169.google.com ([209.85.160.169]:46363)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <pengyu.ut@gmail.com>) id 1Ruo8W-0002RT-68
	for coreutils@gnu.org; Tue, 07 Feb 2012 11:43:20 -0500
Received: by ghrr18 with SMTP id r18so3892918ghr.0
	for <coreutils@gnu.org>; Tue, 07 Feb 2012 08:43:19 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma;
	h=mime-version:in-reply-to:references:date:message-id:subject:from:to
	:cc:content-type:content-transfer-encoding;
	bh=B+ZoEfYQ9XmdEwDw3a4zq6r0Nm4J9aMx62NIIYscSSA=;
	b=mONNRvuzvjd+BR4gDA4es4hxMfgg+adDvrMDIlZeXbBs5gtxrkp7Q+S7NMbI2nwhKp
	AZpBKkWHiu/xuIjer4GK3xNd7GtKfaJLzH8ccjdwKRVzDDsi69PjrB/GAFxZYXf8+lnA
	uQLqGiIDSf8hXUu3AYhumN4idnGatWIUCsf/U=
MIME-Version: 1.0
Received: by 10.182.76.169 with SMTP id l9mr2434862obw.34.1328632999163; Tue,
	07 Feb 2012 08:43:19 -0800 (PST)
Received: by 10.182.75.202 with HTTP; Tue, 7 Feb 2012 08:43:19 -0800 (PST)
In-Reply-To: <4F314F73.7020401@gmail.com>
References: <CABrM6wnELk+kuX6jd11cY2DY4xjUUs1U5N7qYJmJSOVAt0NNEA@mail.gmail.com>
	<4F314AA7.3060508@draigBrady.com> <4F314F73.7020401@gmail.com>
Date: Tue, 7 Feb 2012 10:43:19 -0600
Message-ID: <CABrM6wmA+VnjHdykCiz3bjYdHvuMqpQNHFssjizKoEghVxyB9A@mail.gmail.com>
Subject: Re: uniq --accumulate
From: Peng Yu <pengyu.ut@gmail.com>
To: Assaf Gordon <assafgordon@gmail.com>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.85.160.169
Cc: Coreutils <coreutils@gnu.org>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Tue, 07 Feb 2012 16:43:27 -0000

On Tue, Feb 7, 2012 at 10:21 AM, Assaf Gordon <assafgordon@gmail.com> wrote=
:
> P=E1draig Brady wrote, On 02/07/2012 11:00 AM:
>> On 02/07/2012 03:56 PM, Peng Yu wrote:
>>>
>>> Suppose that I have a table of the following, where the last column is
>>> a number. I'd like to accumulate the number of rows that are the same
>>> for all the remaining columns.
>>>
>>
>> Thanks for the suggestion,
>> but this is too specialized for coreutils I think.
>
> Slightly off-topic for coreutils,
>
> but a package called "BEDTools" ( http://code.google.com/p/bedtools/ ) pr=
ovides a program called "groupBy", which does exactly that, and more.
>
> Akin to SQL's "group by" command, the program can group a text file by a =
specified column, and perform operations (count,sum,mean,median,etc.) on an=
other column.

OK. This is what I'm looking for. The latest groupBy is from filo. I
think that the field separator is not documented. My guess is that it
is '\t'. But users can not change it. Would you please document it and
add an option so that users can change it? Thanks!

~$ groupBy -h

Program: groupBy (v1.1.0)
Authors: Aaron Quinlan (aaronquinlan@gmail.com)
         Assaf Gordon
Summary: Summarizes a dataset column based upon
	 common column groupings. Akin to the SQL "group by" command.

Usage:	 groupBy -i [FILE] -g [group_column(s)] -c [op_column(s)] -o [ops]
	 cat [FILE] | groupBy -g [group_column(s)] -c [op_column(s)] -o [ops]

Options:
	-i		Input file. Assumes "stdin" if omitted.

	-g -grp		Specify the columns (1-based) for the grouping.
			The columns must be comma separated.
			- Default: 1,2,3

	-c -opCols	Specify the column (1-based) that should be summarized.
			- Required.

	-o -ops		Specify the operation that should be applied to opCol.
			Valid operations:
			    sum, count, min, max,
			    mean, median, mode, antimode,
			    stdev, sstdev (sample standard dev.),
			    collapse (i.e., print a comma separated list (duplicates allowed)),
			    distinct (i.e., print a comma separated list (NO duplicates allowed)=
),
			    concat   (i.e., merge values into a single, non-delimited string),
			    freqdesc (i.e., print desc. list of values:freq)
			    freqasc (i.e., print asc. list of values:freq)
			- Default: sum

	-full		Print all columns from input file.
			Default: print only grouped columns.

	-inheader	Input file has a header line - the first line will be ignored.

	-outheader	Print header line in the output, detailing the column names.
			If the input file has headers (-inheader), the output file
			will use the input's column names.
			If the input file has no headers, the output file
			will use "col_1", "col_2", etc. as the column names.

	-header		same as '-inheader -outheader'

	-ignorecase	Group values regardless of upper/lower case.

Examples:
	$ cat ex1.out
	chr1 10  20  A   chr1    15  25  B.1 1000    ATAT
	chr1 10  20  A   chr1    25  35  B.2 10000   CGCG

	$ groupBy -i ex1.out -g 1,2,3,4 -c 9 -o sum
	chr1 10  20  A   11000

	$ groupBy -i ex1.out -grp 1,2,3,4 -opCols 9,9 -ops sum,max
	chr1 10  20  A   11000   10000

	$ groupBy -i ex1.out -g 1,2,3,4 -c 8,9 -o collapse,mean
	chr1 10  20  A   B.1,B.2,    5500

	$ cat ex1.out | groupBy -g 1,2,3,4 -c 8,9 -o collapse,mean
	chr1 10  20  A   B.1,B.2,    5500

	$ cat ex1.out | groupBy -g 1,2,3,4 -c 10 -o concat
	chr1 10  20  A   ATATCGCG

Notes:
	(1)  The input file/stream should be sorted/grouped by the -grp. columns
	(2)  If -i is unspecified, input is assumed to come from stdin.


--=20
Regards,
Peng


From MAILER-DAEMON Tue Feb 07 11:46:48 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RuoBs-0004F4-GI
	for mharc-coreutils@gnu.org; Tue, 07 Feb 2012 11:46:48 -0500
Received: from eggs.gnu.org ([140.186.70.92]:44422)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <pengyu.ut@gmail.com>) id 1RuoBk-00043y-Tm
	for coreutils@gnu.org; Tue, 07 Feb 2012 11:46:46 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <pengyu.ut@gmail.com>) id 1RuoBb-0003P1-Ky
	for coreutils@gnu.org; Tue, 07 Feb 2012 11:46:40 -0500
Received: from mail-tul01m020-f169.google.com ([209.85.214.169]:36407)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <pengyu.ut@gmail.com>) id 1RuoBb-0003Os-DK
	for coreutils@gnu.org; Tue, 07 Feb 2012 11:46:31 -0500
Received: by obbta7 with SMTP id ta7so10836562obb.0
	for <coreutils@gnu.org>; Tue, 07 Feb 2012 08:46:30 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma;
	h=mime-version:in-reply-to:references:date:message-id:subject:from:to
	:cc:content-type:content-transfer-encoding;
	bh=F48nXIMg/yXlkma1IoPnwPg+CQxqrWNP4li9pK5WCIE=;
	b=jfvL0gxyJ/cnB+drOF9mqJn5xY04Mxczw+fFzxmcBhD1iPT8jIcVVpYNYgMz5qXhN8
	zM1nx9yalbaOiip9vEWesGj4SlucNLJ7Zrl2A303tI50Q4k53ccNlm4yCVPAXYinqkbf
	0IqsgjL7RvCshHavAv2iXeKlwQdOrfXZ5WtQI=
MIME-Version: 1.0
Received: by 10.182.76.169 with SMTP id l9mr2446216obw.34.1328633190655; Tue,
	07 Feb 2012 08:46:30 -0800 (PST)
Received: by 10.182.75.202 with HTTP; Tue, 7 Feb 2012 08:46:30 -0800 (PST)
In-Reply-To: <CABrM6wmA+VnjHdykCiz3bjYdHvuMqpQNHFssjizKoEghVxyB9A@mail.gmail.com>
References: <CABrM6wnELk+kuX6jd11cY2DY4xjUUs1U5N7qYJmJSOVAt0NNEA@mail.gmail.com>
	<4F314AA7.3060508@draigBrady.com> <4F314F73.7020401@gmail.com>
	<CABrM6wmA+VnjHdykCiz3bjYdHvuMqpQNHFssjizKoEghVxyB9A@mail.gmail.com>
Date: Tue, 7 Feb 2012 10:46:30 -0600
Message-ID: <CABrM6wkkXE=QeNnpqo+chnKhUUUVzMMA4V-OiDOOo13epu+AFw@mail.gmail.com>
Subject: Re: uniq --accumulate
From: Peng Yu <pengyu.ut@gmail.com>
To: Assaf Gordon <assafgordon@gmail.com>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.85.214.169
Cc: Coreutils <coreutils@gnu.org>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Tue, 07 Feb 2012 16:46:46 -0000

> =A0 =A0 =A0 =A0(1) =A0The input file/stream should be sorted/grouped by t=
he -grp. columns

Do you actually mean that the rows are treated just like uniq does?
I.e, only adjacent rows are subjected to grouping? If so, the document
should be reworded to reflect this meaning.

--=20
Regards,
Peng


From MAILER-DAEMON Tue Feb 07 11:54:13 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RuoJ3-0002MX-Hs
	for mharc-coreutils@gnu.org; Tue, 07 Feb 2012 11:54:13 -0500
Received: from eggs.gnu.org ([140.186.70.92]:42491)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <pengyu.ut@gmail.com>) id 1RuoIu-0001qs-Tf
	for coreutils@gnu.org; Tue, 07 Feb 2012 11:54:12 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <pengyu.ut@gmail.com>) id 1RuoIm-0004g8-PH
	for coreutils@gnu.org; Tue, 07 Feb 2012 11:54:04 -0500
Received: from mail-yw0-f41.google.com ([209.85.213.41]:46140)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <pengyu.ut@gmail.com>) id 1RuoIm-0004fz-Mm
	for coreutils@gnu.org; Tue, 07 Feb 2012 11:53:56 -0500
Received: by yhoo22 with SMTP id o22so3645619yho.0
	for <coreutils@gnu.org>; Tue, 07 Feb 2012 08:53:56 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma;
	h=mime-version:in-reply-to:references:date:message-id:subject:from:to
	:cc:content-type:content-transfer-encoding;
	bh=UE0oKgfqtzB+zfAzqKXsalLbVgD+7OjHOvhUi+hTJ4g=;
	b=hP66YXjOxMgnsRScAmPQX1NqMTk9z9HoYAz8UcOPpTuqwOCLBL22B9uXC5JWw33BvO
	wPTLMuw2BLww35wvujoUKtmvdbCjf/aiiFwBhV4In5r4dn5v1qy+BFcPMa/H7LJoOKb8
	N/YS+G3Yayf9CXQj5sng/Zh73HPshu2ZmudQ8=
MIME-Version: 1.0
Received: by 10.182.38.7 with SMTP id c7mr21641225obk.44.1328633636051; Tue,
	07 Feb 2012 08:53:56 -0800 (PST)
Received: by 10.182.75.202 with HTTP; Tue, 7 Feb 2012 08:53:55 -0800 (PST)
In-Reply-To: <CABrM6wmA+VnjHdykCiz3bjYdHvuMqpQNHFssjizKoEghVxyB9A@mail.gmail.com>
References: <CABrM6wnELk+kuX6jd11cY2DY4xjUUs1U5N7qYJmJSOVAt0NNEA@mail.gmail.com>
	<4F314AA7.3060508@draigBrady.com> <4F314F73.7020401@gmail.com>
	<CABrM6wmA+VnjHdykCiz3bjYdHvuMqpQNHFssjizKoEghVxyB9A@mail.gmail.com>
Date: Tue, 7 Feb 2012 10:53:55 -0600
Message-ID: <CABrM6wkv4bdjKkRq_EeETjMvq_MZvHy7bq6c_iY89Onf8Q47wA@mail.gmail.com>
Subject: Re: uniq --accumulate
From: Peng Yu <pengyu.ut@gmail.com>
To: Assaf Gordon <assafgordon@gmail.com>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.85.213.41
Cc: Coreutils <coreutils@gnu.org>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Tue, 07 Feb 2012 16:54:12 -0000

> =A0 =A0 =A0 =A0-o -ops =A0 =A0 =A0 =A0 Specify the operation that should =
be applied to opCol.
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0Valid operations:
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0sum, count, min, m=
ax,
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0mean, median, mode=
, antimode,
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0stdev, sstdev (sam=
ple standard dev.),
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0collapse (i.e., pr=
int a comma separated list (duplicates allowed)),
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0distinct (i.e., pr=
int a comma separated list (NO duplicates allowed)),
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0concat =A0 (i.e., =
merge values into a single, non-delimited string),
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0freqdesc (i.e., pr=
int desc. list of values:freq)
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0freqasc (i.e., pri=
nt asc. list of values:freq)
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0- Default: sum

I'm not sure that I understand this option. I assume that if I don't
specify it I should by default have the sum option. By the following
test case demonstrates that it is not the case.


~$ ./main.sh
groupBy -g 1,2,3 -c 4 -i input1.txt 2>/dev/null
groupBy -g 1,2,3 -c 4 -o sum -i input1.txt
chr1	-	1	3
chr1	+	1	2

~$ cat input1.txt
chr1	-	1	1
chr1	-	1	2
chr1	+	1	2
~$ cat main.sh
#!/usr/bin/env bash

set -v
groupBy -g 1,2,3 -c 4 -i input1.txt 2>/dev/null
groupBy -g 1,2,3 -c 4 -o sum -i input1.txt


--=20
Regards,
Peng


From MAILER-DAEMON Tue Feb 07 13:26:32 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RupkO-0002wV-HA
	for mharc-coreutils@gnu.org; Tue, 07 Feb 2012 13:26:32 -0500
Received: from eggs.gnu.org ([140.186.70.92]:55469)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1RupkI-0002tJ-Kn
	for coreutils@gnu.org; Tue, 07 Feb 2012 13:26:31 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1RupkE-0007Jt-2b
	for coreutils@gnu.org; Tue, 07 Feb 2012 13:26:26 -0500
Received: from mail-iy0-f169.google.com ([209.85.210.169]:51491)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1RupkD-0007Jf-Qo
	for coreutils@gnu.org; Tue, 07 Feb 2012 13:26:22 -0500
Received: by iagz16 with SMTP id z16so13466973iag.0
	for <coreutils@gnu.org>; Tue, 07 Feb 2012 10:26:20 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma;
	h=mime-version:in-reply-to:references:from:date:message-id:subject:to
	:cc:content-type;
	bh=Y9msOrl3a/PNfjitnzoLShr78O66mZegPMmwTs4QkxA=;
	b=dHIR6NDmVOGcxGlPWy0PtVlK6Hb8cdeuo8MWK0rGKqYxzJFxCM/3OwWfHGgoI9Nlj3
	lH2xgqs+DCC4qKlonAPjNr254jCjBY2jaYr7bHhX3q43tH/AtrWgZHFa1DG9BDlXE5Es
	+re323h9iDpqLjmbZqudYpMSIWR26d007pDMc=
Received: by 10.42.235.137 with SMTP id kg9mr22811568icb.47.1328639180313;
	Tue, 07 Feb 2012 10:26:20 -0800 (PST)
MIME-Version: 1.0
Received: by 10.42.29.196 with HTTP; Tue, 7 Feb 2012 10:25:59 -0800 (PST)
In-Reply-To: <4F2FF5EE.9010906@draigBrady.com>
References: <87mx8w3ryu.fsf@Apollo.jerryland.fr>
	<4F2FF5EE.9010906@draigBrady.com>
From: =?ISO-8859-1?Q?J=E9r=E9my_Compostella?= <jeremy.compostella@gmail.com>
Date: Tue, 7 Feb 2012 19:25:59 +0100
Message-ID: <CA+TATJF4w4Ue44Za4hgGR_0RxgEZGBm5nzhHRSdGYxzzfKWNLg@mail.gmail.com>
Subject: Re: dd: add 'skip_bytes' and 'count_bytes' operands
To: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigbrady.com>
Content-Type: multipart/alternative; boundary=20cf3026697a6095fa04b863eadc
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.85.210.169
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Tue, 07 Feb 2012 18:26:31 -0000

--20cf3026697a6095fa04b863eadc
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

> On 02/06/2012 03:24 PM, J=E9r=E9my Compostella wrote:
> >> Hmm, shouldn't there be a seek_bytes param too for consistency?
> > That was effectively my first mail question. As you talk about it in
> > your explanation addition in coreutils.texi I guess I should start
> > implementing it ?
>
> I think so.
>
> Note it annoys me to be adding 3 new options to dd,
> though I can't think of how to simplify off hand.
> Anyway if we do think of a better way it will probably only
> involve minor interface code changes.
Today, I thought about it a little bit and I did not find a veritable
good solution. The only one I thought about was introducing a "modal"
(bytes or block) option but it will probably add more complexity in both
code and documentation.

> >> This additional doc change is necessary I think:
> >> --- a/doc/coreutils.texi
> >> +++ b/doc/coreutils.texi
> >> @@ -8354,8 +8354,10 @@ should not be too large---values larger than a
few megabytes
> >>  are generally wasteful or (as in the gigabyte..exabyte case) downrigh=
t
> >>  counterproductive or error-inducing.
> >>
> >> -Use different @command{dd} invocations to use different block sizes
for
> >> -skipping and I/O@.  For example, the following shell commands copy
data
> >> +To use different block sizes for skipping and I/O@
> >> +you can use the @samp{skip_bytes} and @samp{seek_bytes} options,
> >> +or the traditional method of separate @command{dd} invocations.
> >> +For example, the following shell commands copy data
> >>  in 512 KiB blocks between a disk and a tape, but do not save or
restore a
> >>  4 KiB label at the start of the disk:
> > OK. I would add something about count_bytes too, no ?
>
> Well this example was just about skipping,
> but it's probably best to mention all related args there.
> How about:
>
> "
> To process data that is at an offset or size that is not a
> multiple of the I/O@ block size, you can use the @samp{skip_bytes},
> @samp{seek_bytes} and @samp{count_bytes} options.  Alternatively
> the traditional method of separate @command{dd} invocations can be used.
> For example, the following shell commands copy data
> "
It looks fine to me.

>
> > Should I import your patch suggestions in my patch or do you will make
> > an additional commit on top of it ? That's particularly relevant if I
> > implement the seek_bytes operand.
>
> It's best if we can --amend everything into your patch.
OK.

I thought a little bit more about the mutually exclusive bytes and not
bytes operands. Indeed, we could let the user use both bytes and not
bytes operands simultaneously in a cumulative way (for example
seek_bytes and seek operands in the dd call). The corresponding code
should not be hard to write and it could offer more
flexibility. However, it could be more confusing for the user and the
documentation should explain this behavior. Since you explain me that we
have to keep options and documentation as simple as possible, I'm not
sure it's a so good idea. What is your opinion about this ?

J=E9r=E9my

--20cf3026697a6095fa04b863eadc
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div>&gt; On 02/06/2012 03:24 PM, J=E9r=E9my Compostella wrote:</div><div>&=
gt; &gt;&gt; Hmm, shouldn&#39;t there be a seek_bytes param too for consist=
ency?</div><div>&gt; &gt; That was effectively my first mail question. As y=
ou talk about it in</div>

<div>&gt; &gt; your explanation addition in coreutils.texi I guess I should=
 start</div><div>&gt; &gt; implementing it ?</div><div>&gt;=A0</div><div>&g=
t; I think so.</div><div>&gt;=A0</div><div>&gt; Note it annoys me to be add=
ing 3 new options to dd,</div>

<div>&gt; though I can&#39;t think of how to simplify off hand.</div><div>&=
gt; Anyway if we do think of a better way it will probably only</div><div>&=
gt; involve minor interface code changes.</div><div>Today, I thought about =
it a little bit and I did not find a veritable</div>

<div>good solution. The only one I thought about was introducing a &quot;mo=
dal&quot;</div><div>(bytes or block) option but it will probably add more c=
omplexity in both</div><div>code and documentation.</div><div><br></div>

<div>&gt; &gt;&gt; This additional doc change is necessary I think:</div><d=
iv>&gt; &gt;&gt; --- a/doc/coreutils.texi</div><div>&gt; &gt;&gt; +++ b/doc=
/coreutils.texi</div><div>&gt; &gt;&gt; @@ -8354,8 +8354,10 @@ should not b=
e too large---values larger than a few megabytes</div>

<div>&gt; &gt;&gt; =A0are generally wasteful or (as in the gigabyte..exabyt=
e case) downright</div><div>&gt; &gt;&gt; =A0counterproductive or error-ind=
ucing.</div><div>&gt; &gt;&gt;</div><div>&gt; &gt;&gt; -Use different @comm=
and{dd} invocations to use different block sizes for</div>

<div>&gt; &gt;&gt; -skipping and I/O@. =A0For example, the following shell =
commands copy data</div><div>&gt; &gt;&gt; +To use different block sizes fo=
r skipping and I/O@</div><div>&gt; &gt;&gt; +you can use the @samp{skip_byt=
es} and @samp{seek_bytes} options,</div>

<div>&gt; &gt;&gt; +or the traditional method of separate @command{dd} invo=
cations.</div><div>&gt; &gt;&gt; +For example, the following shell commands=
 copy data</div><div>&gt; &gt;&gt; =A0in 512 KiB blocks between a disk and =
a tape, but do not save or restore a</div>

<div>&gt; &gt;&gt; =A04 KiB label at the start of the disk:</div><div>&gt; =
&gt; OK. I would add something about count_bytes too, no ?</div><div>&gt;=
=A0</div><div>&gt; Well this example was just about skipping,</div><div>&gt=
; but it&#39;s probably best to mention all related args there.</div>

<div>&gt; How about:</div><div>&gt;=A0</div><div>&gt; &quot;</div><div>&gt;=
 To process data that is at an offset or size that is not a</div><div>&gt; =
multiple of the I/O@ block size, you can use the @samp{skip_bytes},</div>

<div>&gt; @samp{seek_bytes} and @samp{count_bytes} options. =A0Alternativel=
y</div><div>&gt; the traditional method of separate @command{dd} invocation=
s can be used.</div><div>&gt; For example, the following shell commands cop=
y data</div>

<div>&gt; &quot;</div><div>It looks fine to me.</div><div><br></div><div>&g=
t;=A0</div><div>&gt; &gt; Should I import your patch suggestions in my patc=
h or do you will make</div><div>&gt; &gt; an additional commit on top of it=
 ? That&#39;s particularly relevant if I</div>

<div>&gt; &gt; implement the seek_bytes operand.</div><div>&gt;=A0</div><di=
v>&gt; It&#39;s best if we can --amend everything into your patch.</div><di=
v>OK.</div><div><br></div><div>I thought a little bit more about the mutual=
ly exclusive bytes and not</div>

<div>bytes operands. Indeed, we could let the user use both bytes and not</=
div><div>bytes operands simultaneously in a cumulative way (for example</di=
v><div>seek_bytes and seek operands in the dd call). The corresponding code=
</div>

<div>should not be hard to write and it could offer more</div><div>flexibil=
ity. However, it could be more confusing for the user and the</div><div>doc=
umentation should explain this behavior. Since you explain me that we</div>

<div>have to keep options and documentation as simple as possible, I&#39;m =
not</div><div>sure it&#39;s a so good idea. What is your opinion about this=
 ?</div><div><br></div><div>J=E9r=E9my</div><div><br></div>

--20cf3026697a6095fa04b863eadc--


From MAILER-DAEMON Tue Feb 07 14:35:03 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1Ruqoh-0007ff-GH
	for mharc-coreutils@gnu.org; Tue, 07 Feb 2012 14:35:03 -0500
Received: from eggs.gnu.org ([140.186.70.92]:34443)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Ruqof-0007e1-4O
	for coreutils@gnu.org; Tue, 07 Feb 2012 14:35:02 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Ruqod-0003pf-Fq
	for coreutils@gnu.org; Tue, 07 Feb 2012 14:35:01 -0500
Received: from mail3.vodafone.ie ([213.233.128.45]:37074)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Ruqod-0003pZ-BM
	for coreutils@gnu.org; Tue, 07 Feb 2012 14:34:59 -0500
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: ApMBAFV8MU9tTdw1/2dsb2JhbAAMN60shSMBAQEEDCYBRhALDQsJFg8JAwIBAgFFBg0BBwEBF8ILizYBBAIBAgIJAgIBBgcEBgELCYMlHQMMFwVSAgyDZQSbJoM2iS0
Received: from unknown (HELO [192.168.1.79]) ([109.77.220.53])
	by mail3.vodafone.ie with ESMTP; 07 Feb 2012 19:34:57 +0000
Message-ID: <4F317CDF.2040909@draigBrady.com>
Date: Tue, 07 Feb 2012 19:34:55 +0000
From: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:6.0) Gecko/20110816 Thunderbird/6.0
MIME-Version: 1.0
To: =?ISO-8859-1?Q?J=E9r=E9my_Compostella?=
 <jeremy.compostella@gmail.com>
Subject: Re: dd: add 'skip_bytes' and 'count_bytes' operands
References: <87mx8w3ryu.fsf@Apollo.jerryland.fr>
	<4F2FF5EE.9010906@draigBrady.com>
	<CA+TATJF4w4Ue44Za4hgGR_0RxgEZGBm5nzhHRSdGYxzzfKWNLg@mail.gmail.com>
In-Reply-To: <CA+TATJF4w4Ue44Za4hgGR_0RxgEZGBm5nzhHRSdGYxzzfKWNLg@mail.gmail.com>
X-Enigmail-Version: 1.3.2
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 213.233.128.45
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Tue, 07 Feb 2012 19:35:02 -0000

On 02/07/2012 06:25 PM, Jérémy Compostella wrote:
>> On 02/06/2012 03:24 PM, Jérémy Compostella wrote:
>> >> Hmm, shouldn't there be a seek_bytes param too for consistency?
>> > That was effectively my first mail question. As you talk about it in
>> > your explanation addition in coreutils.texi I guess I should start
>> > implementing it ?
>> 
>> I think so.
>> 
>> Note it annoys me to be adding 3 new options to dd,
>> though I can't think of how to simplify off hand.
>> Anyway if we do think of a better way it will probably only
>> involve minor interface code changes.
> Today, I thought about it a little bit and I did not find a veritable
> good solution. The only one I thought about was introducing a "modal"
> (bytes or block) option but it will probably add more complexity in both
> code and documentation.
> 
>> >> This additional doc change is necessary I think:
>> >> --- a/doc/coreutils.texi
>> >> +++ b/doc/coreutils.texi
>> >> @@ -8354,8 +8354,10 @@ should not be too large---values larger than a few megabytes
>> >>  are generally wasteful or (as in the gigabyte..exabyte case) downright
>> >>  counterproductive or error-inducing.
>> >>
>> >> -Use different @command{dd} invocations to use different block sizes for
>> >> -skipping and I/O@.  For example, the following shell commands copy data
>> >> +To use different block sizes for skipping and I/O@
>> >> +you can use the @samp{skip_bytes} and @samp{seek_bytes} options,
>> >> +or the traditional method of separate @command{dd} invocations.
>> >> +For example, the following shell commands copy data
>> >>  in 512 KiB blocks between a disk and a tape, but do not save or restore a
>> >>  4 KiB label at the start of the disk:
>> > OK. I would add something about count_bytes too, no ?
>> 
>> Well this example was just about skipping,
>> but it's probably best to mention all related args there.
>> How about:
>> 
>> "
>> To process data that is at an offset or size that is not a
>> multiple of the I/O@ block size, you can use the @samp{skip_bytes},
>> @samp{seek_bytes} and @samp{count_bytes} options.  Alternatively
>> the traditional method of separate @command{dd} invocations can be used.
>> For example, the following shell commands copy data
>> "
> It looks fine to me.
> 
>> 
>> > Should I import your patch suggestions in my patch or do you will make
>> > an additional commit on top of it ? That's particularly relevant if I
>> > implement the seek_bytes operand.
>> 
>> It's best if we can --amend everything into your patch.
> OK.
> 
> I thought a little bit more about the mutually exclusive bytes and not
> bytes operands. Indeed, we could let the user use both bytes and not
> bytes operands simultaneously in a cumulative way (for example
> seek_bytes and seek operands in the dd call). The corresponding code
> should not be hard to write and it could offer more
> flexibility. However, it could be more confusing for the user and the
> documentation should explain this behavior. Since you explain me that we
> have to keep options and documentation as simple as possible, I'm not
> sure it's a so good idea. What is your opinion about this ?

I considered that, but I think your initial implementation is fine.
I.E. mutually exclusive is fine.  I can't see a need to use
both methods to skip and count.

That leads onto the only other possible implementation I suppose.
I.E. since we don't need to specify skip and skip_bytes together,
the option value could select between each mode. I.E.

skip=2      => skip    2 blocks,
skip=2b     => skip 1024 blocks,
skip=2bytes => skip    2 bytes

Hmm, that's not too bad at all really.

cheers,
Pádraig.


From MAILER-DAEMON Wed Feb 08 05:24:32 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1Rv4hU-00055B-7j
	for mharc-coreutils@gnu.org; Wed, 08 Feb 2012 05:24:32 -0500
Received: from eggs.gnu.org ([140.186.70.92]:35493)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Rv4hJ-00054u-CG
	for coreutils@gnu.org; Wed, 08 Feb 2012 05:24:30 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Rv4hB-0001IJ-8d
	for coreutils@gnu.org; Wed, 08 Feb 2012 05:24:21 -0500
Received: from mail3.vodafone.ie ([213.233.128.45]:21452)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Rv4hB-0001I1-43
	for coreutils@gnu.org; Wed, 08 Feb 2012 05:24:13 -0500
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: ApMBAPFMMk9tTc6T/2dsb2JhbAAMHRqyVAcBAQEEAQIvAUYQCw0BCgkWDwkDAgECARYvBg0BBwEBBYU/gj0puQeLPAwBBAIBAgIJAgIBBgcEBgELCYMlHQMMFwVSAgwZAhCDOgSTBIgihRWHTg
Received: from unknown (HELO [192.168.1.79]) ([109.77.206.147])
	by mail3.vodafone.ie with ESMTP; 08 Feb 2012 10:24:12 +0000
Message-ID: <4F324D4A.2000600@draigBrady.com>
Date: Wed, 08 Feb 2012 10:24:10 +0000
From: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:6.0) Gecko/20110816 Thunderbird/6.0
MIME-Version: 1.0
To: Eric Blake <eblake@redhat.com>
Subject: Re: [PATCH] test: expose recent gnulib canonicalize bug
References: <1328574803-25243-1-git-send-email-eblake@redhat.com>
In-Reply-To: <1328574803-25243-1-git-send-email-eblake@redhat.com>
X-Enigmail-Version: 1.3.2
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 213.233.128.45
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Wed, 08 Feb 2012 10:24:30 -0000

On 02/07/2012 12:33 AM, Eric Blake wrote:
> https://lists.gnu.org/archive/html/bug-gnulib/2012-02/msg00038.html
> detailed a couple of bugs in gnulib's canonicalize that were visible
> through coreutils' readlink, but only on systems where // is distinct
> from /.  This particular test assumes that POSIX will be fixed to
> require canonicalization of a symlink containing just slashes will
> behave as if slashes separating the symlink from the rest of the
> name are elided (see http://austingroupbugs.net/view.php?id=541),
> as that is the only useful (and current) behavior on Cygwin.  That is,
> 
> ln -s / root
> ls root/dev
> 
> must list the contents of /dev, not //dev.
> 
> * tests/misc/readlink-root: New test.
> * tests/Makefile.am (TESTS): Run it.
> ---
> 
> Partway through my promise to test my recent patches to realpath,
> by starting with missing readlink coverage.
> 
> I have not yet had a chance to run coreutils under valgrind or
> Coverity after configuring with gl_cv_double_slash_root=yes.
> 
>  tests/Makefile.am        |    1 +
>  tests/misc/readlink-root |   92 ++++++++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 93 insertions(+), 0 deletions(-)
>  create mode 100644 tests/misc/readlink-root
> 
> diff --git a/tests/Makefile.am b/tests/Makefile.am
> index a94aaa2..39b17ab 100644
> --- a/tests/Makefile.am
> +++ b/tests/Makefile.am
> @@ -220,6 +220,7 @@ TESTS =						\
>    misc/printf-surprise				\
>    misc/pwd-long					\
>    misc/readlink-fp-loop				\
> +  misc/readlink-root				\
>    misc/realpath					\
>    misc/runcon-no-reorder			\
>    misc/sha1sum					\
> diff --git a/tests/misc/readlink-root b/tests/misc/readlink-root
> new file mode 100644
> index 0000000..3c30cec
> --- /dev/null
> +++ b/tests/misc/readlink-root
> @@ -0,0 +1,92 @@
> +#!/bin/sh
> +# tests for canonicalize-existing mode (readlink -e) on /.
> +
> +# Copyright (C) 2012 Free Software Foundation, Inc.
> +
> +# This program is free software: you can redistribute it and/or modify
> +# it under the terms of the GNU General Public License as published by
> +# the Free Software Foundation, either version 3 of the License, or
> +# (at your option) any later version.
> +
> +# This program is distributed in the hope that it will be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +# GNU General Public License for more details.
> +
> +# You should have received a copy of the GNU General Public License
> +# along with this program.  If not, see <http://www.gnu.org/licenses/>.
> +
> +. "${srcdir=.}/init.sh"; path_prepend_ ../src
> +print_ver_ readlink
> +
> +stat_single=$(stat -c %d:%i /) || framework_failure_
> +stat_double=$(stat -c %d:%i //) || framework_failure_
> +double_slash=/
> +if test x"$stat_single" = x"stat_double"; then
> +  double_slash=//
> +fi
> +
> +test -d /dev || framework_failure_

Maybe just skip_ above?

> +  readlink -e / || fail=1
> +  readlink -e // || fail=1
> +  readlink -e /// || fail=1
> +  readlink -e /.//.. || fail=1
> +  readlink -e //.//.. || fail=1
> +  readlink -e ///.//.. || fail=1
> +  readlink -e one || fail=1
> +  readlink -e two || fail=1
> +  readlink -e three || fail=1
> +  readlink -e one-dots || fail=1
> +  readlink -e two-dots || fail=1
> +  readlink -e three-dots || fail=1
> +  readlink -e one-dev || fail=1
> +  # We know /dev exists, but cannot assume //dev exists
> +  readlink -f two-dev || fail=1
> +  readlink -e three-dev || fail=1
> +  readlink -e one/dev || fail=1
> +  readlink -f two/dev || fail=1
> +  readlink -e three/dev || fail=1
> +  readlink -e one-dots/dev || fail=1
> +  readlink -f two-dots/dev || fail=1
> +  readlink -e three-dots/dev || fail=1

cool.

thanks,
Pádraig.


From MAILER-DAEMON Wed Feb 08 10:15:26 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1Rv9F0-0006Gi-Hl
	for mharc-coreutils@gnu.org; Wed, 08 Feb 2012 10:15:26 -0500
Received: from eggs.gnu.org ([140.186.70.92]:51438)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1Rv9Es-0005xs-FO
	for coreutils@gnu.org; Wed, 08 Feb 2012 10:15:24 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1Rv9El-0006Jc-O3
	for coreutils@gnu.org; Wed, 08 Feb 2012 10:15:18 -0500
Received: from mail-yw0-f41.google.com ([209.85.213.41]:54793)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1Rv9El-0006JO-JD
	for coreutils@gnu.org; Wed, 08 Feb 2012 10:15:11 -0500
Received: by yhoo22 with SMTP id o22so331463yho.0
	for <coreutils@gnu.org>; Wed, 08 Feb 2012 07:15:10 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma;
	h=mime-version:in-reply-to:references:from:date:message-id:subject:to
	:cc:content-type;
	bh=z1rwKQzKKVUuSOn8oAwfvsIyjKOBSwMg+vSs+p2zJ9k=;
	b=O6Di5qzQHMaM+FqaSyj6qnBCldKDIvp3xeZ2T94Xf37TmHAip6fI86C8IqffCT65a+
	f1GxPY7gtBLYUKf95HAknBno8F005NM6w5+O4oLrSZxpiW2XK5yoj7PCZKZlG/spyPT1
	9e//PguLThosDOc+/KB5llknFbN5Aheebnpbk=
Received: by 10.50.194.199 with SMTP id hy7mr23623176igc.26.1328714110326;
	Wed, 08 Feb 2012 07:15:10 -0800 (PST)
MIME-Version: 1.0
Received: by 10.42.29.196 with HTTP; Wed, 8 Feb 2012 07:14:49 -0800 (PST)
In-Reply-To: <4F317CDF.2040909@draigBrady.com>
References: <87mx8w3ryu.fsf@Apollo.jerryland.fr>
	<4F2FF5EE.9010906@draigBrady.com>
	<CA+TATJF4w4Ue44Za4hgGR_0RxgEZGBm5nzhHRSdGYxzzfKWNLg@mail.gmail.com>
	<4F317CDF.2040909@draigBrady.com>
From: =?ISO-8859-1?Q?J=E9r=E9my_Compostella?= <jeremy.compostella@gmail.com>
Date: Wed, 8 Feb 2012 16:14:49 +0100
Message-ID: <CA+TATJFU7yOGzzeZseHUFTqxxGAtz8zUoQccz=5m0Eu=CqEtRQ@mail.gmail.com>
Subject: Re: dd: add 'skip_bytes' and 'count_bytes' operands
To: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigbrady.com>
Content-Type: multipart/alternative; boundary=14dae9399c1f8dd6a904b8755c8a
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.85.213.41
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Wed, 08 Feb 2012 15:15:24 -0000

--14dae9399c1f8dd6a904b8755c8a
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

> That leads onto the only other possible implementation I suppose.
> I.E. since we don't need to specify skip and skip_bytes together,
> the option value could select between each mode. I.E.
>
> skip=3D2      =3D> skip    2 blocks,
> skip=3D2b     =3D> skip 1024 blocks,
> skip=3D2bytes =3D> skip    2 bytes
>
> Hmm, that's not too bad at all really
I do agree, it looks like a very interesting idea indeed. However, I
have two questions:
1. Why skip=3D2b =3D> 1024 blocks ?
2. Do you know if there is a utility function somewhere in coreutils or
gnulib which could help me to parse this ? If such a function exists it
could be able to manage more units (bytes, kbytes, megabytes, ...). If
have to write such a function we have to discuss the possibilities I have t=
o
provide.

Cheers,

J=E9r=E9my

--14dae9399c1f8dd6a904b8755c8a
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div><div>&gt; That leads onto the only other possible implementation I sup=
pose.</div><div>&gt; I.E. since we don&#39;t need to specify skip and skip_=
bytes together,</div><div>&gt; the option value could select between each m=
ode. I.E.</div>

<div>&gt;=A0</div><div>&gt; skip=3D2 =A0 =A0 =A0=3D&gt; skip =A0 =A02 block=
s,</div><div>&gt; skip=3D2b =A0 =A0 =3D&gt; skip 1024 blocks,</div><div>&gt=
; skip=3D2bytes =3D&gt; skip =A0 =A02 bytes</div><div>&gt;=A0</div><div>&gt=
; Hmm, that&#39;s not too bad at all really</div>

<div>I do agree, it looks like a very interesting idea indeed. However, I</=
div><div>have two questions:</div><div>1. Why skip=3D2b =3D&gt; 1024 blocks=
 ?</div><div>2. Do you know if there is a utility function somewhere in cor=
eutils or</div>

<div>gnulib which could help me to parse this ? If such a function exists i=
t</div><div>could be able to manage more units (bytes, kbytes, megabytes, .=
..). If</div><div>have to write such a function we have to discuss the poss=
ibilities I have to</div>

<div>provide.</div><div><br></div><div>Cheers,</div><div><br></div><div>J=
=E9r=E9my</div></div><div><br></div>

--14dae9399c1f8dd6a904b8755c8a--


From MAILER-DAEMON Wed Feb 08 11:01:24 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1Rv9xU-00006J-CH
	for mharc-coreutils@gnu.org; Wed, 08 Feb 2012 11:01:24 -0500
Received: from eggs.gnu.org ([140.186.70.92]:46518)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1Rv9xR-0008Vh-FZ
	for coreutils@gnu.org; Wed, 08 Feb 2012 11:01:22 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1Rv9xI-0006ye-9c
	for coreutils@gnu.org; Wed, 08 Feb 2012 11:01:21 -0500
Received: from mail-gy0-f169.google.com ([209.85.160.169]:47734)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1Rv9xI-0006yZ-5o
	for coreutils@gnu.org; Wed, 08 Feb 2012 11:01:12 -0500
Received: by ghrr18 with SMTP id r18so375331ghr.0
	for <coreutils@gnu.org>; Wed, 08 Feb 2012 08:01:11 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma;
	h=mime-version:in-reply-to:references:from:date:message-id:subject:to
	:cc:content-type;
	bh=hLhmMFKJ/Bj00y/c1rJYYidxAkw1kr0C4ydRP4abTPo=;
	b=ZdFQ3iUsu42Yms7k49NO69yuYdr1+jZaGwgWcOOXfdp0B/AxJO3kW1f1FoUypZSIAf
	qFgth34cMCIXOutzicV3QjzyM0Fk4CT2VIdC+jcWk/fWu9ebCGYSi+P5aCJhp6lPeQvH
	sg6l0qy8if9PAVc62ns3CPRtpuQYSBbcE/p1I=
Received: by 10.50.153.198 with SMTP id vi6mr34372578igb.30.1328716871500;
	Wed, 08 Feb 2012 08:01:11 -0800 (PST)
MIME-Version: 1.0
Received: by 10.42.29.196 with HTTP; Wed, 8 Feb 2012 08:00:51 -0800 (PST)
In-Reply-To: <CA+TATJFU7yOGzzeZseHUFTqxxGAtz8zUoQccz=5m0Eu=CqEtRQ@mail.gmail.com>
References: <87mx8w3ryu.fsf@Apollo.jerryland.fr>
	<4F2FF5EE.9010906@draigBrady.com>
	<CA+TATJF4w4Ue44Za4hgGR_0RxgEZGBm5nzhHRSdGYxzzfKWNLg@mail.gmail.com>
	<4F317CDF.2040909@draigBrady.com>
	<CA+TATJFU7yOGzzeZseHUFTqxxGAtz8zUoQccz=5m0Eu=CqEtRQ@mail.gmail.com>
From: =?ISO-8859-1?Q?J=E9r=E9my_Compostella?= <jeremy.compostella@gmail.com>
Date: Wed, 8 Feb 2012 17:00:51 +0100
Message-ID: <CA+TATJGGiAUg=5vNRnzfYv-NPqRdm-A+AduEPwHNvghhJ503HQ@mail.gmail.com>
Subject: Re: dd: add 'skip_bytes' and 'count_bytes' operands
To: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigbrady.com>
Content-Type: multipart/alternative; boundary=e89a8f22c68122018204b87601d3
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.85.160.169
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Wed, 08 Feb 2012 16:01:22 -0000

--e89a8f22c68122018204b87601d3
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: quoted-printable

[...]
> > Hmm, that's not too bad at all really
> I do agree, it looks like a very interesting idea indeed. However, I
> have two questions:
> 1. Why skip=3D2b =3D> 1024 blocks ?
> 2. Do you know if there is a utility function somewhere in coreutils or
> gnulib which could help me to parse this ? If such a function exists it
> could be able to manage more units (bytes, kbytes, megabytes, ...). If
> have to write such a function we have to discuss the possibilities I have
to
> provide.
About #2, I was thinking that we could be consistent with the truncate
--size
option for example. Adding the missing B for bytes.

truncate --size option documentation extract:
      =91KB=92 =3D> 1000 (KiloBytes)
      =91K=92 =3D> 1024 (KibiBytes)
      =91MB=92 =3D> 1000*1000 (MegaBytes)
      =91M=92 =3D> 1024*1024 (MebiBytes)
      =91GB=92 =3D> 1000*1000*1000 (GigaBytes)
      =91G=92 =3D> 1024*1024*1024 (GibiBytes)
and so on for =91T=92, =91P=92, =91E=92, =91Z=92, and =91Y=92.

Cheers,

J=E9r=E9my

--e89a8f22c68122018204b87601d3
Content-Type: text/html; charset=windows-1252
Content-Transfer-Encoding: quoted-printable

<div>[...]</div><div>&gt; &gt; Hmm, that&#39;s not too bad at all really</d=
iv><div>&gt; I do agree, it looks like a very interesting idea indeed. Howe=
ver, I</div><div>&gt; have two questions:</div><div>&gt; 1. Why skip=3D2b =
=3D&gt; 1024 blocks ?</div>

<div>&gt; 2. Do you know if there is a utility function somewhere in coreut=
ils or</div><div>&gt; gnulib which could help me to parse this ? If such a =
function exists it</div><div>&gt; could be able to manage more units (bytes=
, kbytes, megabytes, ...). If</div>

<div>&gt; have to write such a function we have to discuss the possibilitie=
s I have to</div><div>&gt; provide.</div><div>About #2, I was thinking that=
 we could be consistent with the truncate --size</div><div>option=A0for exa=
mple. Adding the missing B for bytes.</div>

<div><br></div><div>truncate --size option documentation extract:</div><div=
>=A0=A0 =A0 =A0=91KB=92 =3D&gt; 1000 (KiloBytes)</div><div>=A0=A0 =A0 =A0=
=91K=92 =3D&gt; 1024 (KibiBytes)</div><div>=A0=A0 =A0 =A0=91MB=92 =3D&gt; 1=
000*1000 (MegaBytes)</div><div>=A0=A0 =A0 =A0=91M=92 =3D&gt; 1024*1024 (Meb=
iBytes)</div>

<div>=A0=A0 =A0 =A0=91GB=92 =3D&gt; 1000*1000*1000 (GigaBytes)</div><div>=
=A0=A0 =A0 =A0=91G=92 =3D&gt; 1024*1024*1024 (GibiBytes)</div><div>and so o=
n for =91T=92, =91P=92, =91E=92, =91Z=92, and =91Y=92.</div><div><br></div>=
<div>Cheers,</div><div><br></div><div>J=E9r=E9my</div>



--e89a8f22c68122018204b87601d3--


From MAILER-DAEMON Wed Feb 08 11:06:02 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RvA1y-00023O-JF
	for mharc-coreutils@gnu.org; Wed, 08 Feb 2012 11:06:02 -0500
Received: from eggs.gnu.org ([140.186.70.92]:59616)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1RvA1s-000200-7L
	for coreutils@gnu.org; Wed, 08 Feb 2012 11:06:00 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1RvA1l-0007y7-JN
	for coreutils@gnu.org; Wed, 08 Feb 2012 11:05:56 -0500
Received: from mx1.redhat.com ([209.132.183.28]:6216)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1RvA1l-0007y1-Cd
	for coreutils@gnu.org; Wed, 08 Feb 2012 11:05:49 -0500
Received: from int-mx12.intmail.prod.int.phx2.redhat.com
	(int-mx12.intmail.prod.int.phx2.redhat.com [10.5.11.25])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id q18G5ice006414
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK);
	Wed, 8 Feb 2012 11:05:45 -0500
Received: from [10.3.113.116] (ovpn-113-116.phx2.redhat.com [10.3.113.116])
	by int-mx12.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP
	id q18G5iWE026862; Wed, 8 Feb 2012 11:05:44 -0500
Message-ID: <4F329D57.4050002@redhat.com>
Date: Wed, 08 Feb 2012 09:05:43 -0700
From: Eric Blake <eblake@redhat.com>
Organization: Red Hat
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:10.0) Gecko/20120131 Thunderbird/10.0
MIME-Version: 1.0
To: =?ISO-8859-1?Q?J=E9r=E9my_Compostella?=
 <jeremy.compostella@gmail.com>
Subject: Re: dd: add 'skip_bytes' and 'count_bytes' operands
References: <87mx8w3ryu.fsf@Apollo.jerryland.fr>
	<4F2FF5EE.9010906@draigBrady.com>
	<CA+TATJF4w4Ue44Za4hgGR_0RxgEZGBm5nzhHRSdGYxzzfKWNLg@mail.gmail.com>
	<4F317CDF.2040909@draigBrady.com>
	<CA+TATJFU7yOGzzeZseHUFTqxxGAtz8zUoQccz=5m0Eu=CqEtRQ@mail.gmail.com>
In-Reply-To: <CA+TATJFU7yOGzzeZseHUFTqxxGAtz8zUoQccz=5m0Eu=CqEtRQ@mail.gmail.com>
X-Enigmail-Version: 1.3.5
OpenPGP: url=http://people.redhat.com/eblake/eblake.gpg
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature";
	boundary="------------enig53260B6C3C7B80CC6776ABFA"
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.25
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.132.183.28
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Wed, 08 Feb 2012 16:06:00 -0000

This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--------------enig53260B6C3C7B80CC6776ABFA
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

On 02/08/2012 08:14 AM, J=E9r=E9my Compostella wrote:
>> That leads onto the only other possible implementation I suppose.
>> I.E. since we don't need to specify skip and skip_bytes together,
>> the option value could select between each mode. I.E.
>>
>> skip=3D2      =3D> skip    2 blocks,
>> skip=3D2b     =3D> skip 1024 blocks,
>> skip=3D2bytes =3D> skip    2 bytes
>>
>> Hmm, that's not too bad at all really
> I do agree, it looks like a very interesting idea indeed. However, I
> have two questions:
> 1. Why skip=3D2b =3D> 1024 blocks ?

Because POSIX requires that one.
http://pubs.opengroup.org/onlinepubs/9699919799/utilities/dd.html

For the bs=3D, cbs=3D, ibs=3D, and obs=3D operands, the application shall=
 supply
an expression specifying a size in bytes. The expression, expr, can be:

    A positive decimal number

    A positive decimal number followed by k, specifying multiplication
by 1024

    A positive decimal number followed by b, specifying multiplication
by 512

    Two or more positive decimal numbers (with or without k or b)
separated by x, specifying the product of the indicated values

> 2. Do you know if there is a utility function somewhere in coreutils or=

> gnulib which could help me to parse this ? If such a function exists it=

> could be able to manage more units (bytes, kbytes, megabytes, ...). If
> have to write such a function we have to discuss the possibilities I ha=
ve to
> provide.

See how parse_integer() is already using xstrtoumax; it would just be a
matter of special-casing 'bytes', differently than 'b', and more like
'x' is special-cased.

--=20
Eric Blake   eblake@redhat.com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


--------------enig53260B6C3C7B80CC6776ABFA
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Public key at http://people.redhat.com/eblake/eblake.gpg
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQEcBAEBCAAGBQJPMp1YAAoJEKeha0olJ0Nq2yUH/iOfr4vs9u/48g/w7J9oL2Ur
P9QhRmWlEEov8syzY9l+ZBLwGl/RkrmAAg5Uq3ElXdzs5XPgFBeMe0ORq4GZla+7
DaEH7CbP1+SJQwnUjiqwzjO7auDhyToK3dEYyLy7iyPFnCa51CdbGB0j8GgTgjRK
w3VW2Xmpv56cMe58KycrGtE1WnDcgtmtdNMZxr2OzNHwVFSFJASdOP0lfumCyVjL
7JQClmSQAgYQPDRrNtva/X1MuURshNKw4T020B2Kj96O3vaxlVplfyzXhpMZVhn/
tQfP/clVAvF7uBCbW+hvaTms/oWAOdK8paRG660W+GIAuJVsH7rHb7+6AqMTf2I=
=qwdz
-----END PGP SIGNATURE-----

--------------enig53260B6C3C7B80CC6776ABFA--


From MAILER-DAEMON Wed Feb 08 11:15:07 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RvAAl-0004mm-NM
	for mharc-coreutils@gnu.org; Wed, 08 Feb 2012 11:15:07 -0500
Received: from eggs.gnu.org ([140.186.70.92]:43509)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1RvAAe-0004iT-HH
	for coreutils@gnu.org; Wed, 08 Feb 2012 11:15:05 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1RvAAc-00018l-W7
	for coreutils@gnu.org; Wed, 08 Feb 2012 11:15:00 -0500
Received: from mx1.redhat.com ([209.132.183.28]:23460)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1RvAAc-00018f-O5
	for coreutils@gnu.org; Wed, 08 Feb 2012 11:14:58 -0500
Received: from int-mx12.intmail.prod.int.phx2.redhat.com
	(int-mx12.intmail.prod.int.phx2.redhat.com [10.5.11.25])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id q18GEu6D001715
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK);
	Wed, 8 Feb 2012 11:14:56 -0500
Received: from [10.3.113.116] (ovpn-113-116.phx2.redhat.com [10.3.113.116])
	by int-mx12.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP
	id q18GEtMd030734; Wed, 8 Feb 2012 11:14:55 -0500
Message-ID: <4F329F7E.4090002@redhat.com>
Date: Wed, 08 Feb 2012 09:14:54 -0700
From: Eric Blake <eblake@redhat.com>
Organization: Red Hat
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:10.0) Gecko/20120131 Thunderbird/10.0
MIME-Version: 1.0
To: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
Subject: Re: [PATCH] test: expose recent gnulib canonicalize bug
References: <1328574803-25243-1-git-send-email-eblake@redhat.com>
	<4F324D4A.2000600@draigBrady.com>
In-Reply-To: <4F324D4A.2000600@draigBrady.com>
X-Enigmail-Version: 1.3.5
OpenPGP: url=http://people.redhat.com/eblake/eblake.gpg
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature";
	boundary="------------enig6B2C0C84FF79D33659ACA4B9"
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.25
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.132.183.28
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Wed, 08 Feb 2012 16:15:05 -0000

This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--------------enig6B2C0C84FF79D33659ACA4B9
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

On 02/08/2012 03:24 AM, P=E1draig Brady wrote:
> On 02/07/2012 12:33 AM, Eric Blake wrote:
>> https://lists.gnu.org/archive/html/bug-gnulib/2012-02/msg00038.html
>> detailed a couple of bugs in gnulib's canonicalize that were visible
>> through coreutils' readlink, but only on systems where // is distinct
>> from /.  This particular test assumes that POSIX will be fixed to
>> require canonicalization of a symlink containing just slashes will
>> behave as if slashes separating the symlink from the rest of the
>> name are elided (see http://austingroupbugs.net/view.php?id=3D541),
>> as that is the only useful (and current) behavior on Cygwin.  That is,=

>>
>> ln -s / root
>> ls root/dev
>>
>> must list the contents of /dev, not //dev.
>>

>> +
>> +test -d /dev || framework_failure_
>=20
> Maybe just skip_ above?

Nah.  POSIX requires /dev/null to exist, and you'd be hard-pressed to
find a system where this test fails.

>=20
> cool.

Good to hear; that means I now need to write up my realpath test for //.

Question before I push:  Right now, we have:

tests/readlink/can-{e,f,m}
tests/readlink/rl-1
tests/misc/readlink-fp-loop

This feels inconsistent; should we be consolidating all readlink-related
tests into the same directory, and if so, which directory?

--=20
Eric Blake   eblake@redhat.com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


--------------enig6B2C0C84FF79D33659ACA4B9
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Public key at http://people.redhat.com/eblake/eblake.gpg
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQEcBAEBCAAGBQJPMp9/AAoJEKeha0olJ0NqH/AH/jfeo+DjRR24wiZETEdJvjj1
rcj4R+UcBoQVK3GVz1l9r/tYfYshDhj/9Acj46el3qJVpIPDmqnhOyUe7di+UOAf
S8XwB6TW5oU1ZbJFj+Cp79QBc3O887QTxiBuXMF0yCScSxRd5sbB20ZFTWzfVpLI
JL6KkZ/iZbmEnKSxyJ/sHwIFwYX3fpHKVy03yWUdZcte0uIIAjdb+9z8kk+A2xV/
9hPYaCFc3BWBgwBVoeLSFy2MZrfrIDKz0zVapi9SzzUIm7QsqMWp4zFMNho4NoTu
ETpl7eBXuFJY8IUc1LegMLAIOzKcB4gNqPig8+L2J+PnHgsC044/besluE66TYQ=
=R5d+
-----END PGP SIGNATURE-----

--------------enig6B2C0C84FF79D33659ACA4B9--


From MAILER-DAEMON Wed Feb 08 11:16:26 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RvAC2-0005v2-TH
	for mharc-coreutils@gnu.org; Wed, 08 Feb 2012 11:16:26 -0500
Received: from eggs.gnu.org ([140.186.70.92]:43810)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RvABw-0005sW-Jb
	for coreutils@gnu.org; Wed, 08 Feb 2012 11:16:24 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RvABq-0001PY-Hk
	for coreutils@gnu.org; Wed, 08 Feb 2012 11:16:20 -0500
Received: from mail3.vodafone.ie ([213.233.128.45]:52427)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RvABq-0001OU-At
	for coreutils@gnu.org; Wed, 08 Feb 2012 11:16:14 -0500
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: ApMBAOaeMk9tTc6T/2dsb2JhbAAMOK0ThSQBAQEDAQwmAUYFCwsNCwkWDwkDAgECAUUGDQEHAQGHeLlVi0sBBAIBAgIJAgIBBgcEBgELCYMlHQMMFwVSAgyDZQSbJ4M2iS0
Received: from unknown (HELO [192.168.1.79]) ([109.77.206.147])
	by mail3.vodafone.ie with ESMTP; 08 Feb 2012 16:16:11 +0000
Message-ID: <4F329FC9.4030403@draigBrady.com>
Date: Wed, 08 Feb 2012 16:16:09 +0000
From: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:6.0) Gecko/20110816 Thunderbird/6.0
MIME-Version: 1.0
To: =?ISO-8859-1?Q?J=E9r=E9my_Compostella?=
 <jeremy.compostella@gmail.com>
Subject: Re: dd: add 'skip_bytes' and 'count_bytes' operands
References: <87mx8w3ryu.fsf@Apollo.jerryland.fr>
	<4F2FF5EE.9010906@draigBrady.com>
	<CA+TATJF4w4Ue44Za4hgGR_0RxgEZGBm5nzhHRSdGYxzzfKWNLg@mail.gmail.com>
	<4F317CDF.2040909@draigBrady.com>
	<CA+TATJFU7yOGzzeZseHUFTqxxGAtz8zUoQccz=5m0Eu=CqEtRQ@mail.gmail.com>
In-Reply-To: <CA+TATJFU7yOGzzeZseHUFTqxxGAtz8zUoQccz=5m0Eu=CqEtRQ@mail.gmail.com>
X-Enigmail-Version: 1.3.2
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 213.233.128.45
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Wed, 08 Feb 2012 16:16:25 -0000

On 02/08/2012 03:14 PM, Jérémy Compostella wrote:
>> That leads onto the only other possible implementation I suppose.
>> I.E. since we don't need to specify skip and skip_bytes together,
>> the option value could select between each mode. I.E.
>> 
>> skip=2      => skip    2 blocks,
>> skip=2b     => skip 1024 blocks,
>> skip=2bytes => skip    2 bytes
>> 
>> Hmm, that's not too bad at all really
> I do agree, it looks like a very interesting idea indeed. However, I
> have two questions:

> 1. Why skip=2b => 1024 blocks ?

Old legacy stuff we must support,
and just presented for illustration.

  2b = 2 blocks = 2x512

Another anomaly with the existing number syntax is that
'B' is used to indicate base 10 which is awkward.
Ideally M would be base 10 and Mi base 2. I.E.

skip=2      => skip    2 blocks
skip=2K     => skip 2000 blocks
skip=2Ki    => skip 2048 blocks
skip=2KiB   => skip 2048 bytes

For completeness what the above actually mean are:

skip=2      => skip    2 blocks
skip=2K     => skip 2048 blocks
skip=2Ki    => invalid
skip=2KiB   => skip 2048 blocks

But I'm sure that 'M' etc. is really old and hence
we have this awkwardness for backwards compat reasons.
A prime example of why we should be careful of the
interface before committing.

Anyway the existing 'B' means we could have this awkward form:

  skip=2MBbytes

Maybe it's better to use '_bytes' as the tag.

  skip=3KB_bytes
  skip=2_bytes

Not too bad. A variation to tagging the number directly,
would be to add flags:

  iflag=skip_bytes  # Treat skip as number of bytes not blocks
  iflag=count_bytes # Treat count as number of bytes not blocks
  oflag=seek_bytes  # Treat seek as number of bytes not blocks

While cleaner it's a bit less direct.
Though since this is slightly off the main use case,
I guess flags are OK.

> 2. Do you know if there is a utility function somewhere in coreutils or
> gnulib which could help me to parse this ? If such a function exists it
> could be able to manage more units (bytes, kbytes, megabytes, ...). If
> have to write such a function we have to discuss the possibilities I have to
> provide.

All was done already. See parse_integer().
If going with the '_bytes' tag then all that would
need to be done is something like:

  if opt in ('skip', 'seek', 'count'):
    if optarg.endswith('_bytes'):
      opt_bytes_mode = True
      optarg = optarg[:-6] # strip tag
    parse_integer(optarg)
    if opt_bytes_mode:
      handle_as_in_previous_patch()
    else:
      handle_value()

Though as stated above, the flag's option is probably best.

Sorry for the churn on this, but I've not had
time to think about it and I'm just reacting to your
questions on the fly.  At least the decision
process for the interface is documented.

cheers,
Pádraig.


From MAILER-DAEMON Wed Feb 08 11:27:27 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RvAMh-00028j-Fu
	for mharc-coreutils@gnu.org; Wed, 08 Feb 2012 11:27:27 -0500
Received: from eggs.gnu.org ([140.186.70.92]:53523)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RvAMa-000243-0n
	for coreutils@gnu.org; Wed, 08 Feb 2012 11:27:25 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RvAMR-0003zG-6T
	for coreutils@gnu.org; Wed, 08 Feb 2012 11:27:20 -0500
Received: from mail3.vodafone.ie ([213.233.128.45]:52134)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RvAMR-0003yc-1y
	for coreutils@gnu.org; Wed, 08 Feb 2012 11:27:11 -0500
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: ApMBAF2hMk9tTc6T/2dsb2JhbAAMOKxZhSQBAQEEMgFGEAsNAQoJFg8JAwIBAgFFBg0BBwEBwUKLSwEEAgECAgkCAgEGBwQGAQsJMAMBgnEdAwwXBVICDBkCEIM6BJsnjGM
Received: from unknown (HELO [192.168.1.79]) ([109.77.206.147])
	by mail3.vodafone.ie with ESMTP; 08 Feb 2012 16:27:09 +0000
Message-ID: <4F32A25A.3080704@draigBrady.com>
Date: Wed, 08 Feb 2012 16:27:06 +0000
From: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:6.0) Gecko/20110816 Thunderbird/6.0
MIME-Version: 1.0
To: Eric Blake <eblake@redhat.com>
Subject: Re: [PATCH] test: expose recent gnulib canonicalize bug
References: <1328574803-25243-1-git-send-email-eblake@redhat.com>
	<4F324D4A.2000600@draigBrady.com> <4F329F7E.4090002@redhat.com>
In-Reply-To: <4F329F7E.4090002@redhat.com>
X-Enigmail-Version: 1.3.2
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 213.233.128.45
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Wed, 08 Feb 2012 16:27:25 -0000

On 02/08/2012 04:14 PM, Eric Blake wrote:
> Question before I push:  Right now, we have:
> 
> tests/readlink/can-{e,f,m}
> tests/readlink/rl-1
> tests/misc/readlink-fp-loop
> 
> This feels inconsistent; should we be consolidating all readlink-related
> tests into the same directory, and if so, which directory?

test/readlink/ I think.
tests/misc is fine for isolated tests (like test/misc/realpath for example).
Though once we get a few (> 3?) related tests it's best to group
to their own dir, like was done recently for the split tests in commit 963d809ae

cheers,
Pádraig.


From MAILER-DAEMON Wed Feb 08 12:19:26 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RvBB0-0003A1-PS
	for mharc-coreutils@gnu.org; Wed, 08 Feb 2012 12:19:26 -0500
Received: from eggs.gnu.org ([140.186.70.92]:48266)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1RvBAs-0002rs-Vd
	for coreutils@gnu.org; Wed, 08 Feb 2012 12:19:25 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1RvBAo-00065a-KI
	for coreutils@gnu.org; Wed, 08 Feb 2012 12:19:18 -0500
Received: from mx1.redhat.com ([209.132.183.28]:29818)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>)
	id 1RvBAc-00063g-6u; Wed, 08 Feb 2012 12:19:02 -0500
Received: from int-mx12.intmail.prod.int.phx2.redhat.com
	(int-mx12.intmail.prod.int.phx2.redhat.com [10.5.11.25])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id q18HJ0P8004063
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK);
	Wed, 8 Feb 2012 12:19:00 -0500
Received: from [10.3.113.116] (ovpn-113-116.phx2.redhat.com [10.3.113.116])
	by int-mx12.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP
	id q18HIx8L002834; Wed, 8 Feb 2012 12:18:59 -0500
Message-ID: <4F32AE82.4040109@redhat.com>
Date: Wed, 08 Feb 2012 10:18:58 -0700
From: Eric Blake <eblake@redhat.com>
Organization: Red Hat
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:10.0) Gecko/20120131 Thunderbird/10.0
MIME-Version: 1.0
To: Paul Eggert <eggert@cs.ucla.edu>
Subject: Re: [PATCH 1/4] grep: don't say "write error" over and over
References: <1327053573-28534-1-git-send-email-jskarvad@redhat.com>
	<4F197CF7.70702@gnu.org> <4F199529.9050509@redhat.com>
	<4F19A9EA.8020609@cs.ucla.edu> <4F1A63F0.1080506@cs.ucla.edu>
	<4F32144D.1060703@cs.ucla.edu>
In-Reply-To: <4F32144D.1060703@cs.ucla.edu>
X-Enigmail-Version: 1.3.5
OpenPGP: url=http://people.redhat.com/eblake/eblake.gpg
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature";
	boundary="------------enig6A8CBDE5CF36FB6B3F83378C"
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.25
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.132.183.28
Cc: bug-grep@gnu.org, Coreutils <coreutils@gnu.org>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Wed, 08 Feb 2012 17:19:25 -0000

This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--------------enig6A8CBDE5CF36FB6B3F83378C
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

[adding coreutils]

On 02/07/2012 11:21 PM, Paul Eggert wrote:
> On 01/20/2012 11:06 PM, Paul Eggert wrote:
>> I took a look and came up with four related patches to fix these.
>> Here's the first one.  It fixes the originally-reported bug.
>=20
> I found a problem in that patch's test case: it assumes
> that SIGPIPE has the default action when the test is
> run, which isn't a portable assumption.  I installed the
> following further patch to fix this bug in the test case.

Alas, you _cannot_ use trap to undo a shell that is started with SIGPIPE
ignored.  POSIX requires that:

"Signals that were ignored on entry to a non-interactive shell cannot be
trapped or reset, although no error need be reported when attempting to
do so."

http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#=
trap

$ (trap '' PIPE; bash -c 'trap; trap - PIPE; trap')
trap -- '' SIGPIPE
trap -- '' SIGPIPE

Even more interesting, ksh refuses to tell you that it is ignoring a
trap, until you try to change it:

$ (trap '' PIPE; ksh -c 'trap; echo 0; trap "echo 1" PIPE; trap; \
     echo 2; kill -s PIPE $$; trap; echo 3')
0
trap -- '' PIPE
2
trap -- '' PIPE
3

Then there is dash, which obeys POSIX on ignoring changes to the trap,
but fails to inform you (I'm filing a bug against dash, as that is just
outright mean to lie):

$ (trap '' PIPE; dash -c 'trap; echo 0; trap "echo 1" PIPE; trap; \
     echo 2; kill -s PIPE $$; trap; echo 3')
0
trap -- 'echo 1' PIPE
2
trap -- 'echo 1' PIPE
3

>=20
> tests: work even if SIGPIPE is ignored
> * tests/epipe: Do not infinite-loop if SIGPIPE is already ignored.
> It could be that the invoker of 'make check' ignores SIGPIPE,
> for example.
> diff --git a/tests/epipe b/tests/epipe
> index 336d994..f9cb3f6 100755
> --- a/tests/epipe
> +++ b/tests/epipe
> @@ -6,7 +6,7 @@
>=20
>  if
>     (
> -     while :; do echo x; done |
> +     (trap - PIPE; while echo x; do :; done) 3>&- |

So this won't work reliably, and can still inf-loop if epipe is executed
from an environment with SIGPIPE ignored.

The _only_ way to reset SIGPIPE out of not being ignored is to execute
an intermediary program that doesn't share the shell's lame POSIX
restriction.

I almost wonder if coreutils should add a way to reset signals, perhaps
as a new option to env, as in:

env --reset-signal=3DPIPE echo x

Obviously, if we add that extension to coreutils, you'd have to mark
this particular grep test as skipped on platforms that lack env
--reset-signal, and where we can detect either detect that SIGPIPE is
ignored by the shell, or where we can detect that the shell is not know
for its ability to tell us the truth about whether SIGPIPE is ignored.

--=20
Eric Blake   eblake@redhat.com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


--------------enig6A8CBDE5CF36FB6B3F83378C
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Public key at http://people.redhat.com/eblake/eblake.gpg
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQEcBAEBCAAGBQJPMq6DAAoJEKeha0olJ0NqCCwH/2OYpwKk6ClBQZXBzfFWIlfL
Cql/H4KfHU8Ja7Qf7KgdMnzNOuOkLhjqBC5j2w5QZF1UXK5ZtjYqgtMD96+7Upe6
IO/wYU+dyZcpL1uILriT9B1ECuFqGLWV+j89KR9cDpmb/7RxLgJz8F0Q02TTCUqu
moI2GvIfLm+dDFVKzSuBDPs5KygqJIP46i303wDpmNNkoOmpdYnAAnVUTWCeyPES
gHvX445BJp6dLld9gUzjFrMy0a2XdRhn89iI9Q0QDxwx836WbnMb2DB42/5zAfUz
aeglGqxl2cggQXjf5N7/3SBn/rPm/anOVPqgjuc15NGKCdKRq1I1l6fsuc93Tso=
=CasA
-----END PGP SIGNATURE-----

--------------enig6A8CBDE5CF36FB6B3F83378C--


From MAILER-DAEMON Wed Feb 08 13:20:22 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RvC7y-0005bA-Ox
	for mharc-coreutils@gnu.org; Wed, 08 Feb 2012 13:20:22 -0500
Received: from eggs.gnu.org ([140.186.70.92]:40627)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1RvC7t-0005aW-5h
	for coreutils@gnu.org; Wed, 08 Feb 2012 13:20:21 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1RvC7o-0000SV-Iq
	for coreutils@gnu.org; Wed, 08 Feb 2012 13:20:17 -0500
Received: from mail-yw0-f41.google.com ([209.85.213.41]:63086)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1RvC7o-0000SL-9A
	for coreutils@gnu.org; Wed, 08 Feb 2012 13:20:12 -0500
Received: by yhoo22 with SMTP id o22so515959yho.0
	for <coreutils@gnu.org>; Wed, 08 Feb 2012 10:20:11 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma;
	h=mime-version:in-reply-to:references:from:date:message-id:subject:to
	:cc:content-type;
	bh=3SR2Gn0L9Yqs/c+T7eucNISrDqZvB6FuNjRmklH3328=;
	b=STXyFB1EIN/VTZpVU0A4IXuvX2PsNdIDR8iL8WfNwIOZmbJpS2CCpmnqThSUgktoXO
	wSN5rHpdL/iimMav4MhTDJMHJEf54/uKQfeNZlf7RFwv53CA/z3wXyLwzD8cE0n/b7/+
	AS/Wku3AO3ovIn+zLahs9tr0QdYAw6pN2wedI=
Received: by 10.50.89.232 with SMTP id br8mr24859028igb.30.1328725211365; Wed,
	08 Feb 2012 10:20:11 -0800 (PST)
MIME-Version: 1.0
Received: by 10.42.29.196 with HTTP; Wed, 8 Feb 2012 10:19:51 -0800 (PST)
In-Reply-To: <4F329FC9.4030403@draigBrady.com>
References: <87mx8w3ryu.fsf@Apollo.jerryland.fr>
	<4F2FF5EE.9010906@draigBrady.com>
	<CA+TATJF4w4Ue44Za4hgGR_0RxgEZGBm5nzhHRSdGYxzzfKWNLg@mail.gmail.com>
	<4F317CDF.2040909@draigBrady.com>
	<CA+TATJFU7yOGzzeZseHUFTqxxGAtz8zUoQccz=5m0Eu=CqEtRQ@mail.gmail.com>
	<4F329FC9.4030403@draigBrady.com>
From: =?ISO-8859-1?Q?J=E9r=E9my_Compostella?= <jeremy.compostella@gmail.com>
Date: Wed, 8 Feb 2012 19:19:51 +0100
Message-ID: <CA+TATJFkn8=Kd3iqrtk=z2pkkzKRGwnAsquw61r0kqSMx76zgQ@mail.gmail.com>
Subject: Re: dd: add 'skip_bytes' and 'count_bytes' operands
To: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigbrady.com>
Content-Type: multipart/alternative; boundary=e89a8f3ba4293a3f7304b877f232
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.85.213.41
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Wed, 08 Feb 2012 18:20:21 -0000

--e89a8f3ba4293a3f7304b877f232
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

First, thank you  Eric for the explanation and the guidance.

[...]
> On 02/08/2012 03:14 PM, J=E9r=E9my Compostella wrote:
> Not too bad. A variation to tagging the number directly, would be to
> add flags:
>
>  iflag=3Dskip_bytes # Treat skip as number of bytes not blocks
>  iflag=3Dcount_bytes # Treat count as number of bytes not blocks
>  oflag=3Dseek_bytes # Treat seek as number of bytes not blocks
>
> While cleaner it's a bit less direct.  Though since this is slightly
> off the main use case, I guess flags are OK.
I think too. Moreover, IMHO the "_bytes" and "KB_bytes" suffixes are not
really natural and although flags are indirect it looks clearer with
flags. I will re-write my patch with the flags solution.

> > 2. Do you know if there is a utility function somewhere in coreutils
> > or gnulib which could help me to parse this ? If such a function
> > exists it could be able to manage more units (bytes, kbytes,
> > megabytes, ...). If have to write such a function we have to discuss
> > the possibilities I have to provide.
>
> All was done already. See parse_integer().
OK. I will take a look for my personal interest.

[...]
> Sorry for the churn on this, but I've not had time to think about it
> and I'm just reacting to your questions on the fly.  At least the
> decision process for the interface is documented.
It's absolutely not a problem for me to examine all the solutions even
losing time with the bad one. If you directly provided me the
appropriate solution I probably would not take time to look at the
legacy and POSIX constraints what would be a shame for me. And since I
do it on my pretty small free time and just for the pleasure, I'm not in
a hurry long and reasonable long discussion is not a problem at all.

J=E9r=E9my

--e89a8f3ba4293a3f7304b877f232
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div>First, thank you =A0Eric for the explanation and the guidance.</div><d=
iv><br></div><div>[...]</div><div>&gt; On 02/08/2012 03:14 PM, J=E9r=E9my C=
ompostella wrote:</div><div>&gt; Not too bad. A variation to tagging the nu=
mber directly, would be to</div>

<div>&gt; add flags:</div><div>&gt;=A0</div><div>&gt; =A0iflag=3Dskip_bytes=
 # Treat skip as number of bytes not blocks</div><div>&gt; =A0iflag=3Dcount=
_bytes # Treat count as number of bytes not blocks</div><div>&gt; =A0oflag=
=3Dseek_bytes # Treat seek as number of bytes not blocks</div>

<div>&gt;=A0</div><div>&gt; While cleaner it&#39;s a bit less direct. =A0Th=
ough since this is slightly</div><div>&gt; off the main use case, I guess f=
lags are OK.</div><div>I think too. Moreover, IMHO the &quot;_bytes&quot; a=
nd &quot;KB_bytes&quot; suffixes are not</div>

<div>really natural and although flags are indirect it looks clearer with</=
div><div>flags. I will re-write my patch with the flags solution.</div><div=
><br></div><div>&gt; &gt; 2. Do you know if there is a utility function som=
ewhere in coreutils</div>

<div>&gt; &gt; or gnulib which could help me to parse this ? If such a func=
tion</div><div>&gt; &gt; exists it could be able to manage more units (byte=
s, kbytes,</div><div>&gt; &gt; megabytes, ...). If have to write such a fun=
ction we have to discuss</div>

<div>&gt; &gt; the possibilities I have to provide.</div><div>&gt;=A0</div>=
<div>&gt; All was done already. See parse_integer().</div><div>OK. I will t=
ake a look for my personal interest.</div><div><br></div><div>[...]</div>

<div>&gt; Sorry for the churn on this, but I&#39;ve not had time to think a=
bout it</div><div>&gt; and I&#39;m just reacting to your questions on the f=
ly. =A0At least the</div><div>&gt; decision process for the interface is do=
cumented.</div>

<div>It&#39;s absolutely not a problem for me to examine all the solutions =
even</div><div>losing time with the bad one. If you directly provided me th=
e</div><div>appropriate solution I probably would not take time to look at =
the</div>

<div>legacy and POSIX constraints what would be a shame for me. And since I=
</div><div>do it on my pretty small free time and just for the pleasure, I&=
#39;m not in</div><div>a hurry long and reasonable long discussion is not a=
 problem at all.</div>

<div><br></div><div>J=E9r=E9my</div><div><br></div>

--e89a8f3ba4293a3f7304b877f232--


From MAILER-DAEMON Wed Feb 08 13:20:58 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RvC8Y-0005e0-Sn
	for mharc-coreutils@gnu.org; Wed, 08 Feb 2012 13:20:58 -0500
Received: from eggs.gnu.org ([140.186.70.92]:40730)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <auerswal@unix-ag.uni-kl.de>) id 1RvC8S-0005dq-76
	for coreutils@gnu.org; Wed, 08 Feb 2012 13:20:57 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <auerswal@unix-ag.uni-kl.de>) id 1RvC8M-0000WK-Bk
	for coreutils@gnu.org; Wed, 08 Feb 2012 13:20:52 -0500
Received: from mailgw1.uni-kl.de ([131.246.120.220]:35873)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <auerswal@unix-ag.uni-kl.de>) id 1RvC8L-0000W8-W5
	for coreutils@gnu.org; Wed, 08 Feb 2012 13:20:46 -0500
Received: from sushi.unix-ag.uni-kl.de (sushi.unix-ag.uni-kl.de
	[IPv6:2001:638:208:ef34:0:ff:fe00:65])
	by mailgw1.uni-kl.de (8.14.3/8.14.3/Debian-5+lenny1) with ESMTP id
	q18IKgQf003607
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT)
	for <coreutils@gnu.org>; Wed, 8 Feb 2012 19:20:42 +0100
Received: from sushi.unix-ag.uni-kl.de (ip6-localhost [IPv6:::1])
	by sushi.unix-ag.uni-kl.de (8.14.3/8.14.3/Debian-5+lenny1) with ESMTP
	id q18IKgs9030313
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO)
	for <coreutils@gnu.org>; Wed, 8 Feb 2012 19:20:42 +0100
Received: (from auerswal@localhost)
	by sushi.unix-ag.uni-kl.de (8.14.3/8.14.3/Submit) id q18IKg48030312
	for coreutils@gnu.org; Wed, 8 Feb 2012 19:20:42 +0100
Date: Wed, 8 Feb 2012 19:20:42 +0100
From: Erik Auerswald <auerswal@unix-ag.uni-kl.de>
To: coreutils@gnu.org
Subject: Re: dd: add 'skip_bytes' and 'count_bytes' operands
Message-ID: <20120208182042.GA29530@sushi.unix-ag.uni-kl.de>
References: <87mx8w3ryu.fsf@Apollo.jerryland.fr>
	<4F2FF5EE.9010906@draigBrady.com>
	<CA+TATJF4w4Ue44Za4hgGR_0RxgEZGBm5nzhHRSdGYxzzfKWNLg@mail.gmail.com>
	<4F317CDF.2040909@draigBrady.com>
	<CA+TATJFU7yOGzzeZseHUFTqxxGAtz8zUoQccz=5m0Eu=CqEtRQ@mail.gmail.com>
	<4F329D57.4050002@redhat.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
In-Reply-To: <4F329D57.4050002@redhat.com>
User-Agent: Mutt/1.5.18 (2008-05-17)
Content-Transfer-Encoding: quoted-printable
X-MIME-Autoconverted: from 8bit to quoted-printable by mailgw1.uni-kl.de id
	q18IKgQf003607
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
X-Received-From: 131.246.120.220
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Wed, 08 Feb 2012 18:20:57 -0000

Hi,

On Wed, Feb 08, 2012 at 09:05:43AM -0700, Eric Blake wrote:
> On 02/08/2012 08:14 AM, J=E9r=E9my Compostella wrote:
> >> That leads onto the only other possible implementation I suppose.
> >> I.E. since we don't need to specify skip and skip_bytes together,
> >> the option value could select between each mode. I.E.
> >>
> >> skip=3D2      =3D> skip    2 blocks,
> >> skip=3D2b     =3D> skip 1024 blocks,
> >> skip=3D2bytes =3D> skip    2 bytes
> >>
> >> Hmm, that's not too bad at all really
> > I do agree, it looks like a very interesting idea indeed. However, I
> > have two questions:
> > 1. Why skip=3D2b =3D> 1024 blocks ?
>=20
> Because POSIX requires that one.
> http://pubs.opengroup.org/onlinepubs/9699919799/utilities/dd.html

I don't think so.

> For the bs=3D, cbs=3D, ibs=3D, and obs=3D operands,

skip=3D is not in this list.

> the application shall supply
> an expression specifying a size in bytes. The expression, expr, can be:

skip=3Dn
  Skip n input blocks

The expression given to skip is specified as being in blocks. POSIX does
not specify that skip recognizes a b suffix. It does not specify that b
means blocks^2 in this case neither.

IMHO skip=3D2b should be the same as skip=3D2 (or result in an error, if
silently ignoring the superfluous b suffix is not acceptable).

>     A positive decimal number
>=20
>     A positive decimal number followed by k, specifying multiplication
> by 1024
>=20
>     A positive decimal number followed by b, specifying multiplication
> by 512
>=20
>     Two or more positive decimal numbers (with or without k or b)
> separated by x, specifying the product of the indicated values

All of the above do not apply to skip.

Thanks,
Erik


From MAILER-DAEMON Wed Feb 08 14:22:43 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RvD6J-00017q-6U
	for mharc-coreutils@gnu.org; Wed, 08 Feb 2012 14:22:43 -0500
Received: from eggs.gnu.org ([140.186.70.92]:42912)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eggert@cs.ucla.edu>) id 1RvD6H-00015X-7Y
	for coreutils@gnu.org; Wed, 08 Feb 2012 14:22:42 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eggert@cs.ucla.edu>) id 1RvD6G-00042O-0F
	for coreutils@gnu.org; Wed, 08 Feb 2012 14:22:41 -0500
Received: from smtp.cs.ucla.edu ([131.179.128.62]:59926)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eggert@cs.ucla.edu>)
	id 1RvD67-00040t-Sr; Wed, 08 Feb 2012 14:22:31 -0500
Received: from localhost (localhost.localdomain [127.0.0.1])
	by smtp.cs.ucla.edu (Postfix) with ESMTP id 8214D39E800C;
	Wed,  8 Feb 2012 11:22:30 -0800 (PST)
X-Virus-Scanned: amavisd-new at smtp.cs.ucla.edu
Received: from smtp.cs.ucla.edu ([127.0.0.1])
	by localhost (smtp.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10024)
	with ESMTP id 9+LGjVcL60lQ; Wed,  8 Feb 2012 11:22:29 -0800 (PST)
Received: from penguin.cs.ucla.edu (Penguin.CS.UCLA.EDU [131.179.64.200])
	by smtp.cs.ucla.edu (Postfix) with ESMTPSA id 7E68739E8007;
	Wed,  8 Feb 2012 11:22:29 -0800 (PST)
Message-ID: <4F32CB75.9020408@cs.ucla.edu>
Date: Wed, 08 Feb 2012 11:22:29 -0800
From: Paul Eggert <eggert@cs.ucla.edu>
Organization: UCLA Computer Science Department
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:9.0) Gecko/20111222 Thunderbird/9.0
MIME-Version: 1.0
To: Eric Blake <eblake@redhat.com>
Subject: Re: [PATCH 1/4] grep: don't say "write error" over and over
References: <1327053573-28534-1-git-send-email-jskarvad@redhat.com>
	<4F197CF7.70702@gnu.org> <4F199529.9050509@redhat.com>
	<4F19A9EA.8020609@cs.ucla.edu> <4F1A63F0.1080506@cs.ucla.edu>
	<4F32144D.1060703@cs.ucla.edu> <4F32AE82.4040109@redhat.com>
In-Reply-To: <4F32AE82.4040109@redhat.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
X-Received-From: 131.179.128.62
Cc: bug-grep@gnu.org, Coreutils <coreutils@gnu.org>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Wed, 08 Feb 2012 19:22:42 -0000

Thanks for reviewing that.  What a mess!
An odd thing is that the patch *did* fix things for
Fedora 15.  Maybe its shell is busted?  But that's a
different matter.

I think we can address this particular problem without
relying on a new coreutils feature, though the feature
you mention does sound useful for other reasons.

I pushed this:

tests: work portably even if SIGPIPE is ignored
* tests/epipe: Don't rely on "trap - PIPE"; that's not portable.
Problem reported by Eric Blake in
<http://lists.gnu.org/archive/html/bug-grep/2012-02/msg00017.html>.
Also, use "ls -al" rather than "echo", in case "echo" is done by a
buggy shell that ignores write errors.  And close grep's fd 3, as
a sanity check.
diff --git a/tests/epipe b/tests/epipe
index f9cb3f6..1d9ef05 100755
--- a/tests/epipe
+++ b/tests/epipe
@@ -6,8 +6,8 @@

 if
    (
-     (trap - PIPE; while echo x; do :; done) 3>&- |
-     (trap '' PIPE; exec grep x 2>&3) |
+     while ls -al; do :; done 3>&- |
+     (trap '' PIPE; exec grep . 2>&3 3>&-) |
      :
    ) 3>&1 | (
      read line1 && echo >&2 "$line1" &&


From MAILER-DAEMON Wed Feb 08 15:59:21 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RvEbp-0003Bh-Jg
	for mharc-coreutils@gnu.org; Wed, 08 Feb 2012 15:59:21 -0500
Received: from eggs.gnu.org ([140.186.70.92]:42564)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <stefano.lattarini@gmail.com>) id 1RvEbl-00037g-Gh
	for coreutils@gnu.org; Wed, 08 Feb 2012 15:59:19 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <stefano.lattarini@gmail.com>) id 1RvEbk-00064q-23
	for coreutils@gnu.org; Wed, 08 Feb 2012 15:59:17 -0500
Received: from mail-ee0-f41.google.com ([74.125.83.41]:51069)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <stefano.lattarini@gmail.com>)
	id 1RvEbh-00061r-IH; Wed, 08 Feb 2012 15:59:13 -0500
Received: by eekc13 with SMTP id c13so348062eek.0
	for <multiple recipients>; Wed, 08 Feb 2012 12:59:12 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma;
	h=message-id:date:from:mime-version:to:cc:subject:references
	:in-reply-to:content-type:content-transfer-encoding;
	bh=57e7rGmr8ydOvoUebt+fzMrFm3d/QbP17CfmzM0ci1Q=;
	b=L2BMwfMoGeuMIo8G8mvCX3xwYrxNDM/uh9uwU+hzN1PjI1YYH85iSoxCaPxjEpwAGa
	7oABfHu2ZWoPXWHnuFhK6D3YkD+fnnJiR5CxI3sGkJOXMi4xJYdzOZWe593FZrb47udM
	mmhEFOniiA6x55spz7WJ4VM1LY/7Nml5HcPzs=
Received: by 10.14.99.195 with SMTP id x43mr8831393eef.46.1328734752478;
	Wed, 08 Feb 2012 12:59:12 -0800 (PST)
Received: from [87.14.90.217]
	(host217-90-dynamic.14-87-r.retail.telecomitalia.it. [87.14.90.217])
	by mx.google.com with ESMTPS id o49sm1399232eei.0.2012.02.08.12.59.10
	(version=SSLv3 cipher=OTHER); Wed, 08 Feb 2012 12:59:11 -0800 (PST)
Message-ID: <4F32E21D.4030306@gmail.com>
Date: Wed, 08 Feb 2012 21:59:09 +0100
From: Stefano Lattarini <stefano.lattarini@gmail.com>
MIME-Version: 1.0
To: Paul Eggert <eggert@cs.ucla.edu>
Subject: Re: [PATCH 1/4] grep: don't say "write error" over and over
References: <1327053573-28534-1-git-send-email-jskarvad@redhat.com>	<4F197CF7.70702@gnu.org>
	<4F199529.9050509@redhat.com>	<4F19A9EA.8020609@cs.ucla.edu>
	<4F1A63F0.1080506@cs.ucla.edu>	<4F32144D.1060703@cs.ucla.edu>
	<4F32AE82.4040109@redhat.com> <4F32CB75.9020408@cs.ucla.edu>
In-Reply-To: <4F32CB75.9020408@cs.ucla.edu>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 74.125.83.41
Cc: bug-grep@gnu.org, Coreutils <coreutils@gnu.org>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Wed, 08 Feb 2012 20:59:19 -0000

Hi Paul, just a quick outsider nit ...

On 02/08/2012 08:22 PM, Paul Eggert wrote:
> Thanks for reviewing that.  What a mess!
> An odd thing is that the patch *did* fix things for
> Fedora 15.  Maybe its shell is busted?  But that's a
> different matter.
> 
> I think we can address this particular problem without
> relying on a new coreutils feature, though the feature
> you mention does sound useful for other reasons.
> 
> I pushed this:
> 
> tests: work portably even if SIGPIPE is ignored
> * tests/epipe: Don't rely on "trap - PIPE"; that's not portable.
> Problem reported by Eric Blake in
> <http://lists.gnu.org/archive/html/bug-grep/2012-02/msg00017.html>.
> Also, use "ls -al" rather than "echo", in case "echo" is done by a
> buggy shell that ignores write errors.
>
I think this explanation ought to be added in a code comment as well,
rather than only in the ChangeLog entry.

> And close grep's fd 3, as a sanity check.
>
And this as well, probably.

Regards,
  Stefano


From MAILER-DAEMON Thu Feb 09 02:20:56 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RvOJM-0001Wz-FA
	for mharc-coreutils@gnu.org; Thu, 09 Feb 2012 02:20:56 -0500
Received: from eggs.gnu.org ([140.186.70.92]:43968)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <bernhard.voelker@siemens-enterprise.com>)
	id 1RvOJG-0001Va-5j
	for coreutils@gnu.org; Thu, 09 Feb 2012 02:20:54 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <bernhard.voelker@siemens-enterprise.com>)
	id 1RvOJA-0002n1-GW
	for coreutils@gnu.org; Thu, 09 Feb 2012 02:20:50 -0500
Received: from senmx11-mx.siemens-enterprise.com ([62.134.46.9]:37750)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <bernhard.voelker@siemens-enterprise.com>)
	id 1RvOJA-0002mZ-Bn
	for coreutils@gnu.org; Thu, 09 Feb 2012 02:20:44 -0500
Received: from MCHP064A.global-ad.net (unknown [172.29.37.63])
	by senmx11-mx.siemens-enterprise.com (Server) with ESMTP id
	DE9471EB848E; Thu,  9 Feb 2012 08:20:41 +0100 (CET)
Received: from MCHP058A.global-ad.net ([172.29.37.55]) by
	MCHP064A.global-ad.net ([172.29.37.63]) with mapi;
	Thu, 9 Feb 2012 08:20:42 +0100
From: "Voelker, Bernhard" <bernhard.voelker@siemens-enterprise.com>
To: Eric Blake <eblake@redhat.com>, =?iso-8859-1?Q?J=E9r=E9my_Compostella?=
	<jeremy.compostella@gmail.com>
Date: Thu, 9 Feb 2012 08:20:40 +0100
Subject: RE: dd: add 'skip_bytes' and 'count_bytes' operands
Thread-Topic: dd: add 'skip_bytes' and 'count_bytes' operands
Thread-Index: Aczme4+CdheO6w9GTmSDT33cu1Qe5wAezJjQ
Message-ID: <7856072A9D04C24B82DFE2B1112FE38A0D8E0B1B13@MCHP058A.global-ad.net>
References: <87mx8w3ryu.fsf@Apollo.jerryland.fr>
	<4F2FF5EE.9010906@draigBrady.com>
	<CA+TATJF4w4Ue44Za4hgGR_0RxgEZGBm5nzhHRSdGYxzzfKWNLg@mail.gmail.com>
	<4F317CDF.2040909@draigBrady.com>
	<CA+TATJFU7yOGzzeZseHUFTqxxGAtz8zUoQccz=5m0Eu=CqEtRQ@mail.gmail.com>
	<4F329D57.4050002@redhat.com>
In-Reply-To: <4F329D57.4050002@redhat.com>
Accept-Language: de-DE, en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
acceptlanguage: de-DE, en-US
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
X-Received-From: 62.134.46.9
Cc: "coreutils@gnu.org" <coreutils@gnu.org>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 09 Feb 2012 07:20:55 -0000

Eric Blake wrote:
> Because POSIX requires that one.
> http://pubs.opengroup.org/onlinepubs/9699919799/utilities/dd.html
>=20
> For the bs=3D, cbs=3D, ibs=3D, and obs=3D operands, the application shall=
 supply
> an expression specifying a size in bytes. The expression, expr, can be:
...
>     Two or more positive decimal numbers (with or without k or b)
> separated by x, specifying the product of the indicated values

Cool, that's new to me!

Maybe this form should be made more prominent, as it outclasses
all the other forms: if you misremember the suffix e.g. for 2048,
then you can simply use the multiplication with 'x':

  dd if=3Dfile1 of=3Dfile2 count=3D1 bs=3D32x2048

I didn't find that form in the info page. Is it documented there?

Have a nice day,
Berny



From MAILER-DAEMON Thu Feb 09 06:32:40 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RvSEy-00038M-60
	for mharc-coreutils@gnu.org; Thu, 09 Feb 2012 06:32:40 -0500
Received: from eggs.gnu.org ([140.186.70.92]:46107)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1RvSEq-000385-Ao
	for coreutils@gnu.org; Thu, 09 Feb 2012 06:32:38 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1RvSEo-0003Pp-VS
	for coreutils@gnu.org; Thu, 09 Feb 2012 06:32:32 -0500
Received: from mx.meyering.net ([88.168.87.75]:38013)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1RvSEo-0003Pj-IW
	for coreutils@gnu.org; Thu, 09 Feb 2012 06:32:30 -0500
Received: from rho.meyering.net (localhost.localdomain [127.0.0.1])
	by rho.meyering.net (Acme Bit-Twister) with ESMTP id 98FD560061;
	Thu,  9 Feb 2012 12:32:29 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: Sven Breuner <sven.breuner@itwm.fraunhofer.de>
Subject: Re: "ls -l": Avoid unnecessary getxattr() overhead
In-Reply-To: <4F30631A.2070009@itwm.fraunhofer.de> (Sven Breuner's message of
	"Tue, 07 Feb 2012 00:32:42 +0100")
References: <87pqds3snt.fsf@Apollo.jerryland.fr>
	<4F30206B.1020802@itwm.fraunhofer.de>
	<CA+TATJHorgsuY7pxmpxxAeWUHRP5MfFfzQP05o=1H6C4ttswdA@mail.gmail.com>
	<4F3032BC.50702@draigBrady.com> <4F30631A.2070009@itwm.fraunhofer.de>
Date: Thu, 09 Feb 2012 12:32:29 +0100
Message-ID: <87y5scfdiq.fsf@rho.meyering.net>
Lines: 147
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 88.168.87.75
Cc: Bernd Schubert <bernd.schubert@itwm.fraunhofer.de>,
	=?iso-8859-1?Q?J=E9r=E9my?= Compostella <jeremy.compostella@gmail.com>,
	coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 09 Feb 2012 11:32:38 -0000

Sven Breuner wrote:
> P=E1draig Brady wrote on 02/06/2012 09:06 PM:
>> This seems to have some related info.
>> https://bugzilla.redhat.com/show_bug.cgi?id=3D662011
>> We might be able to cache something.
>> We'll investigate.
>
> Thanks, P=E1draig.
>
> I assume that bugzilla report is only related on a very abstract level:
> The reporter of that ticket is suggesting special behavior after a
> getxattr(acl_stuff) call returned ENODATA, in which case following
> getxattr(acl_stuff) calls might of course have other results.
>
> I'm referring to getxattr(acl_stuff) calls that return EOPNOTSUPP,
> because the file system does not support ACLs / extended attributes,
> in which case all following getxattr(acl_stuff) calls will definitely
> have the same result (plus the other lgetxattr(selinux_stuff) calls,
> which simply cannot provide any interesting information when selinux
> is completely disabled on a machine).

Here's a proof-of-concept patch:
When, for a given directory (being read via readdir),
lgetfilecon fails with ENOTSUP or EOPNOTSUPP the first
time we call it on a file, we skip that function for
all remaining entries in the same directory (but only at the same level).
Technically, we could probably exempt all files on that device, but
until/unless we convert ls.c to use fts, it's probably not worth
trying to do that.

Even this tiny patch...
I'm not sure it's worth it.  What ls.c really needs is a revamp,
to make it use fts like all other dir-tree traversal programs in
coreutils.  However, that is sure to be a big task.

It avoids 33% of the *getxattr calls here, and results
in that saving only when not following symlinks.

diff --git a/src/ls.c b/src/ls.c
index f5cd37a..4ee8cef 100644
--- a/src/ls.c
+++ b/src/ls.c
@@ -244,7 +244,8 @@ static size_t quote_name (FILE *out, const char *name,
 static char *make_link_name (char const *name, char const *linkname);
 static int decode_switches (int argc, char **argv);
 static bool file_ignored (char const *name);
-static uintmax_t gobble_file (char const *name, enum filetype type,
+static uintmax_t gobble_file (bool first_entry, char const *name,
+                              enum filetype type,
                               ino_t inode, bool command_line_arg,
                               char const *dirname);
 static bool print_color_indicator (const struct fileinfo *f,
@@ -1388,13 +1389,13 @@ main (int argc, char **argv)
   if (n_files <=3D 0)
     {
       if (immediate_dirs)
-        gobble_file (".", directory, NOT_AN_INODE_NUMBER, true, "");
+        gobble_file (true, ".", directory, NOT_AN_INODE_NUMBER, true, "");
       else
         queue_directory (".", NULL, true);
     }
   else
     do
-      gobble_file (argv[i++], unknown, NOT_AN_INODE_NUMBER, true, "");
+      gobble_file (true, argv[i++], unknown, NOT_AN_INODE_NUMBER, true, ""=
);
     while (i < argc);

   if (cwd_n_used)
@@ -2563,6 +2564,7 @@ print_dir (char const *name, char const *realname, bo=
ol command_line_arg)

   clear_files ();

+  bool first_entry =3D true;
   while (1)
     {
       /* Set errno to zero so we can distinguish between a readdir failure
@@ -2590,9 +2592,10 @@ print_dir (char const *name, char const *realname, b=
ool command_line_arg)
 # endif
                 }
 #endif
-              total_blocks +=3D gobble_file (next->d_name, type,
+              total_blocks +=3D gobble_file (first_entry, next->d_name, ty=
pe,
                                            RELIABLE_D_INO (next),
                                            false, name);
+              first_entry =3D false;

               /* In this narrow case, print out each name right away, so
                  ls uses constant memory while processing the entries of
@@ -2777,13 +2780,23 @@ clear_files (void)
   file_size_width =3D 0;
 }

+static bool
+errno_unsupported (int err)
+{
+  return err =3D=3D ENOTSUP || err =3D=3D EOPNOTSUPP;
+}
+
 /* Add a file to the current table of files.
    Verify that the file exists, and print an error message if it does not.
-   Return the number of blocks that the file occupies.  */
+   Return the number of blocks that the file occupies.
+   Set FIRST_ENTRY to false for second and subsequent entries when
+   calling from a readdir loop.  Otherwise, set it to true.  This permits
+   an optimization (when false) to avoid repeatedly calling a function that
+   fails with an indication of no support.  */

 static uintmax_t
-gobble_file (char const *name, enum filetype type, ino_t inode,
-             bool command_line_arg, char const *dirname)
+gobble_file (bool first_entry, char const *name, enum filetype type,
+             ino_t inode, bool command_line_arg, char const *dirname)
 {
   uintmax_t blocks =3D 0;
   struct fileinfo *f;
@@ -2798,6 +2811,10 @@ gobble_file (char const *name, enum filetype type, i=
no_t inode,
       cwd_n_alloc *=3D 2;
     }

+  static bool first_lgetfilecon =3D true;
+  if (first_entry)
+    first_lgetfilecon =3D true;
+
   f =3D &cwd_file[cwd_n_used];
   memset (f, '\0', sizeof *f);
   f->stat.st_ino =3D inode;
@@ -2918,10 +2935,17 @@ gobble_file (char const *name, enum filetype type, =
ino_t inode,
         {
           bool have_selinux =3D false;
           bool have_acl =3D false;
-          int attr_len =3D (do_deref
-                          ?  getfilecon (absolute_name, &f->scontext)
-                          : lgetfilecon (absolute_name, &f->scontext));
+          static int prev_errno;
+          int attr_len =3D
+            (do_deref
+             ? getfilecon (absolute_name, &f->scontext)
+             : (( ! command_line_arg && ! first_lgetfilecon
+                  && errno_unsupported (prev_errno))
+                ? (errno =3D ENOTSUP), -1
+                : lgetfilecon (absolute_name, &f->scontext)));
+          first_lgetfilecon =3D false;
           err =3D (attr_len < 0);
+          prev_errno =3D err ? errno : 0;

           if (err =3D=3D 0)
             have_selinux =3D ! STREQ ("unlabeled", f->scontext);


From MAILER-DAEMON Thu Feb 09 07:02:49 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RvSi9-00028X-Cz
	for mharc-coreutils@gnu.org; Thu, 09 Feb 2012 07:02:49 -0500
Received: from eggs.gnu.org ([140.186.70.92]:58576)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RvSi4-00026s-JU
	for coreutils@gnu.org; Thu, 09 Feb 2012 07:02:48 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RvShx-0001Cg-8v
	for coreutils@gnu.org; Thu, 09 Feb 2012 07:02:44 -0500
Received: from mail3.vodafone.ie ([213.233.128.45]:57022)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RvShx-0001BA-4X
	for coreutils@gnu.org; Thu, 09 Feb 2012 07:02:37 -0500
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: ApwBAK+0M09tTc6T/2dsb2JhbAAMN6x3hSUBAQEEJwsBRhALDQsJFg8JAwIBAgFFBg0BBwEBuzGLTCAPCwgGBAMEBAcOBgEDCAENGYNlhBcEmyyMZg
Received: from unknown (HELO [192.168.1.79]) ([109.77.206.147])
	by mail3.vodafone.ie with ESMTP; 09 Feb 2012 12:02:33 +0000
Message-ID: <4F33B5D7.3010909@draigBrady.com>
Date: Thu, 09 Feb 2012 12:02:31 +0000
From: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:6.0) Gecko/20110816 Thunderbird/6.0
MIME-Version: 1.0
To: Jim Meyering <jim@meyering.net>
Subject: Re: "ls -l": Avoid unnecessary getxattr() overhead
References: <87pqds3snt.fsf@Apollo.jerryland.fr>
	<4F30206B.1020802@itwm.fraunhofer.de>
	<CA+TATJHorgsuY7pxmpxxAeWUHRP5MfFfzQP05o=1H6C4ttswdA@mail.gmail.com>
	<4F3032BC.50702@draigBrady.com>
	<4F30631A.2070009@itwm.fraunhofer.de>
	<87y5scfdiq.fsf@rho.meyering.net>
In-Reply-To: <87y5scfdiq.fsf@rho.meyering.net>
X-Enigmail-Version: 1.3.2
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 213.233.128.45
Cc: Sven Breuner <sven.breuner@itwm.fraunhofer.de>,
	Bernd Schubert <bernd.schubert@itwm.fraunhofer.de>,
	=?ISO-8859-1?Q?J=E9r=E9my_Compostella?= <jeremy.compostella@gmail.com>,
	coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 09 Feb 2012 12:02:48 -0000

On 02/09/2012 11:32 AM, Jim Meyering wrote:
> @@ -2918,10 +2935,17 @@ gobble_file (char const *name, enum filetype type, ino_t inode,
>          {
>            bool have_selinux = false;
>            bool have_acl = false;
> -          int attr_len = (do_deref
> -                          ?  getfilecon (absolute_name, &f->scontext)
> -                          : lgetfilecon (absolute_name, &f->scontext));
> +          static int prev_errno;
> +          int attr_len =
> +            (do_deref
> +             ? getfilecon (absolute_name, &f->scontext)
> +             : (( ! command_line_arg && ! first_lgetfilecon
> +                  && errno_unsupported (prev_errno))
> +                ? (errno = ENOTSUP), -1
> +                : lgetfilecon (absolute_name, &f->scontext)));
> +          first_lgetfilecon = false;
>            err = (attr_len < 0);
> +          prev_errno = err ? errno : 0;
> 
>            if (err == 0)
>              have_selinux = ! STREQ ("unlabeled", f->scontext);
> 

So you avoid symlinks as they can point outside the device.
Unfortunately so can bind mounts, so you probably have to key on device?
Here is an example for which the above would break I think:

# touch file /dev/shm/file
# setfattr -n "user.foo" -v "bar" file /dev/shm/file
setfattr: /dev/shm/file: Operation not supported
# stat -c %D file /dev/shm/file
812
11
# mount --bind file /dev/shm/file
# stat -c %D file /dev/shm/file
812
812
# getfattr -m- -d /dev/shm/file
getfattr: Removing leading '/' from absolute path names
# file: dev/shm/file
security.selinux="unconfined_u:object_r:admin_home_t:s0"
user.foo="bar"

cheers,
Pádraig.


From MAILER-DAEMON Thu Feb 09 09:17:34 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RvUoY-0004q5-Kf
	for mharc-coreutils@gnu.org; Thu, 09 Feb 2012 09:17:34 -0500
Received: from eggs.gnu.org ([140.186.70.92]:43695)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1RvUoR-0004pt-I1
	for coreutils@gnu.org; Thu, 09 Feb 2012 09:17:33 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1RvUoL-000177-GM
	for coreutils@gnu.org; Thu, 09 Feb 2012 09:17:27 -0500
Received: from mx.meyering.net ([88.168.87.75]:38432)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1RvUoL-00016v-3t
	for coreutils@gnu.org; Thu, 09 Feb 2012 09:17:21 -0500
Received: from rho.meyering.net (localhost.localdomain [127.0.0.1])
	by rho.meyering.net (Acme Bit-Twister) with ESMTP id 1721A600A1;
	Thu,  9 Feb 2012 15:17:20 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: =?iso-8859-1?Q?P=E1draig?= Brady <P@draigBrady.com>
Subject: Re: "ls -l": Avoid unnecessary getxattr() overhead
In-Reply-To: <4F33B5D7.3010909@draigBrady.com> (=?iso-8859-1?Q?=22P=E1drai?=
	=?iso-8859-1?Q?g?= Brady"'s message of
	"Thu, 09 Feb 2012 12:02:31 +0000")
References: <87pqds3snt.fsf@Apollo.jerryland.fr>
	<4F30206B.1020802@itwm.fraunhofer.de>
	<CA+TATJHorgsuY7pxmpxxAeWUHRP5MfFfzQP05o=1H6C4ttswdA@mail.gmail.com>
	<4F3032BC.50702@draigBrady.com> <4F30631A.2070009@itwm.fraunhofer.de>
	<87y5scfdiq.fsf@rho.meyering.net> <4F33B5D7.3010909@draigBrady.com>
Date: Thu, 09 Feb 2012 15:17:19 +0100
Message-ID: <87mx8sf5w0.fsf@rho.meyering.net>
Lines: 81
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 88.168.87.75
Cc: Sven Breuner <sven.breuner@itwm.fraunhofer.de>,
	Bernd Schubert <bernd.schubert@itwm.fraunhofer.de>,
	=?iso-8859-1?Q?J=E9r=E9my?= Compostella <jeremy.compostella@gmail.com>,
	coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 09 Feb 2012 14:17:33 -0000

P=E1draig Brady wrote:
> On 02/09/2012 11:32 AM, Jim Meyering wrote:
>> Technically, we could probably exempt all files on that device, but
>> ...
>
> So you avoid symlinks as they can point outside the device.
> Unfortunately so can bind mounts, so you probably have to key on device?

Thanks.  That confirms my impression that my patch was not worthwhile:
It was buggy, too.

I suppose we could use a little hash table, whose entries are
<dev_t device_number, bool getfilecon_required> pairs.

Or even a single static dev_t selinux_challenged_device,
that if equal to f->stat.st_dev, we can skip the *getfilecon call.
That should be good enough for most uses.

Here's that simpler patch:

diff --git a/src/ls.c b/src/ls.c
index f5cd37a..cb9f834 100644
--- a/src/ls.c
+++ b/src/ls.c
@@ -2777,6 +2777,45 @@ clear_files (void)
   file_size_width =3D 0;
 }

+static bool
+errno_unsupported (int err)
+{
+  return err =3D=3D ENOTSUP || err =3D=3D ENODATA || err =3D=3D EOPNOTSUPP;
+}
+
+/* st_dev of the most recently processed device for which
+   we've found that getfilecon or lgetfilecon fails with
+   e.g., ENOTSUP or EOPNOTSUPP.  */
+static dev_t selinux_challenged_device;
+
+static int
+getfilecon_cache (char const *file, struct fileinfo *f)
+{
+  if (f->stat.st_dev =3D=3D selinux_challenged_device)
+    {
+      errno =3D ENOTSUP;
+      return -1;
+    }
+  int r =3D getfilecon (file, &f->scontext);
+  if (r < 0 && errno_unsupported (errno))
+    selinux_challenged_device =3D f->stat.st_dev;
+  return r;
+}
+
+static int
+lgetfilecon_cache (char const *file, struct fileinfo *f)
+{
+  if (f->stat.st_dev =3D=3D selinux_challenged_device)
+    {
+      errno =3D ENOTSUP;
+      return -1;
+    }
+  int r =3D lgetfilecon (file, &f->scontext);
+  if (r < 0 && errno_unsupported (errno))
+    selinux_challenged_device =3D f->stat.st_dev;
+  return r;
+}
+
 /* Add a file to the current table of files.
    Verify that the file exists, and print an error message if it does not.
    Return the number of blocks that the file occupies.  */
@@ -2919,8 +2958,8 @@ gobble_file (char const *name, enum filetype type, in=
o_t inode,
           bool have_selinux =3D false;
           bool have_acl =3D false;
           int attr_len =3D (do_deref
-                          ?  getfilecon (absolute_name, &f->scontext)
-                          : lgetfilecon (absolute_name, &f->scontext));
+                          ? getfilecon_cache (absolute_name, f)
+                          : lgetfilecon_cache (absolute_name, f));
           err =3D (attr_len < 0);

           if (err =3D=3D 0)


From MAILER-DAEMON Thu Feb 09 09:53:20 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RvVNA-0002oU-JX
	for mharc-coreutils@gnu.org; Thu, 09 Feb 2012 09:53:20 -0500
Received: from eggs.gnu.org ([140.186.70.92]:43247)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RvVN3-0002nr-ML
	for coreutils@gnu.org; Thu, 09 Feb 2012 09:53:19 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RvVMx-0000pa-Kx
	for coreutils@gnu.org; Thu, 09 Feb 2012 09:53:13 -0500
Received: from mail3.vodafone.ie ([213.233.128.45]:15369)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RvVMx-0000ox-CY
	for coreutils@gnu.org; Thu, 09 Feb 2012 09:53:07 -0500
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: ApwBAHTcM09tTc6T/2dsb2JhbAAMOKx7hSUBAQEDAScLAUYQCw0LCRYPCQMCAQIBRQYNAQcBAYd4ugmLTgYPBg8PAwQGBAMEBAcOBgEDCAENg36EFwSbLIxm
Received: from unknown (HELO [192.168.1.79]) ([109.77.206.147])
	by mail3.vodafone.ie with ESMTP; 09 Feb 2012 14:53:04 +0000
Message-ID: <4F33DDCF.7080307@draigBrady.com>
Date: Thu, 09 Feb 2012 14:53:03 +0000
From: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:6.0) Gecko/20110816 Thunderbird/6.0
MIME-Version: 1.0
To: Jim Meyering <jim@meyering.net>
Subject: Re: "ls -l": Avoid unnecessary getxattr() overhead
References: <87pqds3snt.fsf@Apollo.jerryland.fr>
	<4F30206B.1020802@itwm.fraunhofer.de>
	<CA+TATJHorgsuY7pxmpxxAeWUHRP5MfFfzQP05o=1H6C4ttswdA@mail.gmail.com>
	<4F3032BC.50702@draigBrady.com>
	<4F30631A.2070009@itwm.fraunhofer.de>
	<87y5scfdiq.fsf@rho.meyering.net> <4F33B5D7.3010909@draigBrady.com>
	<87mx8sf5w0.fsf@rho.meyering.net>
In-Reply-To: <87mx8sf5w0.fsf@rho.meyering.net>
X-Enigmail-Version: 1.3.2
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 213.233.128.45
Cc: Sven Breuner <sven.breuner@itwm.fraunhofer.de>,
	Bernd Schubert <bernd.schubert@itwm.fraunhofer.de>,
	=?ISO-8859-1?Q?J=E9r=E9my_Compostella?= <jeremy.compostella@gmail.com>,
	coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 09 Feb 2012 14:53:19 -0000

On 02/09/2012 02:17 PM, Jim Meyering wrote:
> Pádraig Brady wrote:
>> On 02/09/2012 11:32 AM, Jim Meyering wrote:
>>> Technically, we could probably exempt all files on that device, but
>>> ...
>>
>> So you avoid symlinks as they can point outside the device.
>> Unfortunately so can bind mounts, so you probably have to key on device?
> 
> Thanks.  That confirms my impression that my patch was not worthwhile:
> It was buggy, too.
> 
> I suppose we could use a little hash table, whose entries are
> <dev_t device_number, bool getfilecon_required> pairs.
> 
> Or even a single static dev_t selinux_challenged_device,
> that if equal to f->stat.st_dev, we can skip the *getfilecon call.
> That should be good enough for most uses.
> 
> Here's that simpler patch:

That's nice and clean and looks correct.

> 
> diff --git a/src/ls.c b/src/ls.c
> index f5cd37a..cb9f834 100644
> --- a/src/ls.c
> +++ b/src/ls.c
> @@ -2777,6 +2777,45 @@ clear_files (void)
>    file_size_width = 0;
>  }
> 
> +static bool
> +errno_unsupported (int err)
> +{
> +  return err == ENOTSUP || err == ENODATA || err == EOPNOTSUPP;
> +}

Hmm, could one get ENODATA for only some files on a file system?
>From getfilecon(2) "If the context does not exist, or the process
has no access to this attribute, errno is set to ENODATA.".

> +
> +/* st_dev of the most recently processed device for which
> +   we've found that getfilecon or lgetfilecon fails with
> +   e.g., ENOTSUP or EOPNOTSUPP.  */
> +static dev_t selinux_challenged_device;
> +
> +static int
> +getfilecon_cache (char const *file, struct fileinfo *f)
> +{
> +  if (f->stat.st_dev == selinux_challenged_device)
> +    {
> +      errno = ENOTSUP;
> +      return -1;
> +    }
> +  int r = getfilecon (file, &f->scontext);
> +  if (r < 0 && errno_unsupported (errno))
> +    selinux_challenged_device = f->stat.st_dev;
> +  return r;
> +}
> +
> +static int
> +lgetfilecon_cache (char const *file, struct fileinfo *f)
> +{
> +  if (f->stat.st_dev == selinux_challenged_device)
> +    {
> +      errno = ENOTSUP;
> +      return -1;
> +    }
> +  int r = lgetfilecon (file, &f->scontext);
> +  if (r < 0 && errno_unsupported (errno))
> +    selinux_challenged_device = f->stat.st_dev;
> +  return r;
> +}
> +
>  /* Add a file to the current table of files.
>     Verify that the file exists, and print an error message if it does not.
>     Return the number of blocks that the file occupies.  */
> @@ -2919,8 +2958,8 @@ gobble_file (char const *name, enum filetype type, ino_t inode,
>            bool have_selinux = false;
>            bool have_acl = false;
>            int attr_len = (do_deref
> -                          ?  getfilecon (absolute_name, &f->scontext)
> -                          : lgetfilecon (absolute_name, &f->scontext));
> +                          ? getfilecon_cache (absolute_name, f)
> +                          : lgetfilecon_cache (absolute_name, f));
>            err = (attr_len < 0);
> 
>            if (err == 0)
> 


The same logic could be applied to file_has_acl()
and cap_get_file(), which both return errno=ENOTSUP
when I tested against /proc here.

cheers,
Pádraig.


From MAILER-DAEMON Thu Feb 09 10:09:11 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RvVcV-0002sk-Kd
	for mharc-coreutils@gnu.org; Thu, 09 Feb 2012 10:09:11 -0500
Received: from eggs.gnu.org ([140.186.70.92]:52569)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1RvVcP-0002r3-QG
	for coreutils@gnu.org; Thu, 09 Feb 2012 10:09:10 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1RvVcG-0004Q4-3U
	for coreutils@gnu.org; Thu, 09 Feb 2012 10:09:05 -0500
Received: from mx.meyering.net ([88.168.87.75]:38563)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1RvVcF-0004OA-MV
	for coreutils@gnu.org; Thu, 09 Feb 2012 10:08:56 -0500
Received: from rho.meyering.net (localhost.localdomain [127.0.0.1])
	by rho.meyering.net (Acme Bit-Twister) with ESMTP id BE1FF6011A;
	Thu,  9 Feb 2012 16:08:54 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: =?iso-8859-1?Q?P=E1draig?= Brady <P@draigBrady.com>
Subject: Re: "ls -l": Avoid unnecessary getxattr() overhead
In-Reply-To: <4F33DDCF.7080307@draigBrady.com> (=?iso-8859-1?Q?=22P=E1drai?=
	=?iso-8859-1?Q?g?= Brady"'s message of
	"Thu, 09 Feb 2012 14:53:03 +0000")
References: <87pqds3snt.fsf@Apollo.jerryland.fr>
	<4F30206B.1020802@itwm.fraunhofer.de>
	<CA+TATJHorgsuY7pxmpxxAeWUHRP5MfFfzQP05o=1H6C4ttswdA@mail.gmail.com>
	<4F3032BC.50702@draigBrady.com> <4F30631A.2070009@itwm.fraunhofer.de>
	<87y5scfdiq.fsf@rho.meyering.net> <4F33B5D7.3010909@draigBrady.com>
	<87mx8sf5w0.fsf@rho.meyering.net> <4F33DDCF.7080307@draigBrady.com>
Date: Thu, 09 Feb 2012 16:08:54 +0100
Message-ID: <87ehu4f3i1.fsf@rho.meyering.net>
Lines: 84
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 88.168.87.75
Cc: Sven Breuner <sven.breuner@itwm.fraunhofer.de>,
	Bernd Schubert <bernd.schubert@itwm.fraunhofer.de>,
	=?iso-8859-1?Q?J=E9r=E9my?= Compostella <jeremy.compostella@gmail.com>,
	coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 09 Feb 2012 15:09:10 -0000

P=E1draig Brady wrote:

> On 02/09/2012 02:17 PM, Jim Meyering wrote:
>> P=E1draig Brady wrote:
>>> On 02/09/2012 11:32 AM, Jim Meyering wrote:
>>>> Technically, we could probably exempt all files on that device, but
>>>> ...
>>>
>>> So you avoid symlinks as they can point outside the device.
>>> Unfortunately so can bind mounts, so you probably have to key on device?
>>
>> Thanks.  That confirms my impression that my patch was not worthwhile:
>> It was buggy, too.
>>
>> I suppose we could use a little hash table, whose entries are
>> <dev_t device_number, bool getfilecon_required> pairs.
>>
>> Or even a single static dev_t selinux_challenged_device,
>> that if equal to f->stat.st_dev, we can skip the *getfilecon call.
>> That should be good enough for most uses.
>>
>> Here's that simpler patch:
>
> That's nice and clean and looks correct.

Thanks for the speedy review.
Now I'll have to write comments, log, and tests.

>> diff --git a/src/ls.c b/src/ls.c
>> index f5cd37a..cb9f834 100644
>> --- a/src/ls.c
>> +++ b/src/ls.c
>> @@ -2777,6 +2777,45 @@ clear_files (void)
>>    file_size_width =3D 0;
>>  }
>>
>> +static bool
>> +errno_unsupported (int err)
>> +{
>> +  return err =3D=3D ENOTSUP || err =3D=3D ENODATA || err =3D=3D EOPNOTS=
UPP;
>> +}
>
> Hmm, could one get ENODATA for only some files on a file system?

Yes, I suspect so.

>>From getfilecon(2) "If the context does not exist, or the process
> has no access to this attribute, errno is set to ENODATA.".

Good catch.
I had changed that to remove the ENODATA term on one system (and in
the comment you see below) but somehow used the old version of that
function above.  That'll teach me not to copy diffs around, rather than
real change-sets.

>>  /* Add a file to the current table of files.
>>     Verify that the file exists, and print an error message if it does n=
ot.
>>     Return the number of blocks that the file occupies.  */
>> @@ -2919,8 +2958,8 @@ gobble_file (char const *name, enum filetype type,=
 ino_t inode,
>>            bool have_selinux =3D false;
>>            bool have_acl =3D false;
>>            int attr_len =3D (do_deref
>> -                          ?  getfilecon (absolute_name, &f->scontext)
>> -                          : lgetfilecon (absolute_name, &f->scontext));
>> +                          ? getfilecon_cache (absolute_name, f)
>> +                          : lgetfilecon_cache (absolute_name, f));
>>            err =3D (attr_len < 0);
>>
>>            if (err =3D=3D 0)
>>
>
>
> The same logic could be applied to file_has_acl()
> and cap_get_file(), which both return errno=3DENOTSUP
> when I tested against /proc here.

Yep, at least for file_has_acl which I looked at,
though at least in the few file systems where I tried it,
I always got ENODATA, which is what prompted me to remove ENODATA
from one definition of errno_unsupported.

I'm not as enthusiastic about adding code to optimize away
the calls in file_has_acl, unless there are mainstream system/FS
for which it makes a difference.


From MAILER-DAEMON Thu Feb 09 16:17:13 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RvbMf-0006Cv-2O
	for mharc-coreutils@gnu.org; Thu, 09 Feb 2012 16:17:13 -0500
Received: from eggs.gnu.org ([140.186.70.92]:54798)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <sven.breuner@itwm.fraunhofer.de>) id 1RvbMb-00068H-SA
	for coreutils@gnu.org; Thu, 09 Feb 2012 16:17:11 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <sven.breuner@itwm.fraunhofer.de>) id 1RvbMa-0004Ox-Tk
	for coreutils@gnu.org; Thu, 09 Feb 2012 16:17:09 -0500
Received: from mailgw1.uni-kl.de ([131.246.120.220]:57227)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <sven.breuner@itwm.fraunhofer.de>) id 1RvbMa-0004OL-Iw
	for coreutils@gnu.org; Thu, 09 Feb 2012 16:17:08 -0500
Received: from itwm2.itwm.fhg.de (itwm2.itwm.fhg.de [131.246.191.3])
	by mailgw1.uni-kl.de (8.14.3/8.14.3/Debian-5+lenny1) with ESMTP id
	q19LH4JY017465
	(version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=NOT)
	for <coreutils@gnu.org>; Thu, 9 Feb 2012 22:17:04 +0100
Received: from mail2.itwm.fhg.de ([131.246.191.79]:57827)
	by itwm2.itwm.fhg.de with esmtps (TLSv1:DES-CBC3-SHA:168)
	(/C=DE/ST=Rheinland-Pfalz/L=Kaiserslautern/O=Fraunhofer
	ITWM/OU=SLG/CN=mail2.itwm.fhg.de)(verified=1) (Exim 4.74 #1)
	id 1RvbMU-0005E3-LX; Thu, 09 Feb 2012 22:17:02 +0100
Message-ID: <4F3437CE.4050308@itwm.fraunhofer.de>
Date: Thu, 09 Feb 2012 22:17:02 +0100
From: Sven Breuner <sven.breuner@itwm.fraunhofer.de>
User-Agent: Mozilla/5.0 (X11; Linux i686 on x86_64;
	rv:10.0) Gecko/20120129 Thunderbird/10.0
MIME-Version: 1.0
To: Jim Meyering <jim@meyering.net>
Subject: Re: "ls -l": Avoid unnecessary getxattr() overhead
References: <87pqds3snt.fsf@Apollo.jerryland.fr>
	<4F30206B.1020802@itwm.fraunhofer.de>
	<CA+TATJHorgsuY7pxmpxxAeWUHRP5MfFfzQP05o=1H6C4ttswdA@mail.gmail.com>
	<4F3032BC.50702@draigBrady.com>
	<4F30631A.2070009@itwm.fraunhofer.de>
	<87y5scfdiq.fsf@rho.meyering.net> <4F33B5D7.3010909@draigBrady.com>
	<87mx8sf5w0.fsf@rho.meyering.net>
In-Reply-To: <87mx8sf5w0.fsf@rho.meyering.net>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
X-ITWM-CharSet: ISO-8859-1
X-ITWM-Scanned-By: mail2.itwm.fhg.de
Content-Transfer-Encoding: quoted-printable
X-MIME-Autoconverted: from 8bit to quoted-printable by mailgw1.uni-kl.de id
	q19LH4JY017465
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
X-Received-From: 131.246.120.220
Cc: Bernd Schubert <bernd.schubert@itwm.fraunhofer.de>,
	=?ISO-8859-1?Q?J=E9r=E9my_Compostella?= <jeremy.compostella@gmail.com>,
	coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 09 Feb 2012 21:17:11 -0000

Jim Meyering wrote on 02/09/2012 03:17 PM:
> P=E1draig Brady wrote:
>> On 02/09/2012 11:32 AM, Jim Meyering wrote:
>>> Technically, we could probably exempt all files on that device, but
>>> ...
>>
>> So you avoid symlinks as they can point outside the device.
>> Unfortunately so can bind mounts, so you probably have to key on devic=
e?
>
> Thanks.  That confirms my impression that my patch was not worthwhile:
> It was buggy, too.
>
> I suppose we could use a little hash table, whose entries are
> <dev_t device_number, bool getfilecon_required>  pairs.
>
> Or even a single static dev_t selinux_challenged_device,
> that if equal to f->stat.st_dev, we can skip the *getfilecon call.
> That should be good enough for most uses.
>
> Here's that simpler patch:
>
> diff --git a/src/ls.c b/src/ls.c
> index f5cd37a..cb9f834 100644
> [...]


Just tried it out and the difference is very significant, as expected.
(Test case with 61,441 files in a directory.)

Current git version without the patch...

$ time ~/tmp/ls-8.15.34-31eee -l >/dev/null

real    0m31.135s
user    0m0.859s
sys     0m4.276s

$ strace -c ~/tmp/ls-8.15.34-31eee -l >/dev/null
% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
  31.13    0.303648           5     61441           lstat
  30.81    0.300585           5     61441     61441 getxattr
  30.65    0.298937           5     61441           lgetxattr
   7.17    0.069987         933        75           getdents
[...]

Current git version patched...

$ time ~/tmp/ls-8.15.34-31eee_patched -l >/dev/null

real    0m21.254s
user    0m0.565s
sys     0m2.496s

$ strace -c ~/tmp/ls-8.15.34-31eee_patched -l >/dev/null
% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
  45.50    0.313386           5     61441           lstat
  43.63    0.300524           5     61441     61441 getxattr
  10.60    0.072986         973        75           getdents
[...]

Thanks!
Sven


From MAILER-DAEMON Thu Feb 09 19:35:14 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RveSI-0007l5-RZ
	for mharc-coreutils@gnu.org; Thu, 09 Feb 2012 19:35:14 -0500
Received: from eggs.gnu.org ([140.186.70.92]:57537)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <sven.breuner@itwm.fraunhofer.de>) id 1RveSG-0007kt-1T
	for coreutils@gnu.org; Thu, 09 Feb 2012 19:35:13 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <sven.breuner@itwm.fraunhofer.de>) id 1RveSE-0002aX-88
	for coreutils@gnu.org; Thu, 09 Feb 2012 19:35:11 -0500
Received: from mailgw1.uni-kl.de ([131.246.120.220]:60038)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <sven.breuner@itwm.fraunhofer.de>) id 1RveSD-0002ZX-RA
	for coreutils@gnu.org; Thu, 09 Feb 2012 19:35:10 -0500
Received: from itwm2.itwm.fhg.de (itwm2.itwm.fhg.de [131.246.191.3])
	by mailgw1.uni-kl.de (8.14.3/8.14.3/Debian-5+lenny1) with ESMTP id
	q1A0Z613019400
	(version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=NOT)
	for <coreutils@gnu.org>; Fri, 10 Feb 2012 01:35:06 +0100
Received: from mail2.itwm.fhg.de ([131.246.191.79]:58423)
	by itwm2.itwm.fhg.de with esmtps (TLSv1:DES-CBC3-SHA:168)
	(/C=DE/ST=Rheinland-Pfalz/L=Kaiserslautern/O=Fraunhofer
	ITWM/OU=SLG/CN=mail2.itwm.fhg.de)(verified=1) (Exim 4.74 #1)
	id 1RveS8-0005xR-P7; Fri, 10 Feb 2012 01:35:04 +0100
Message-ID: <4F346638.9090304@itwm.fraunhofer.de>
Date: Fri, 10 Feb 2012 01:35:04 +0100
From: Sven Breuner <sven.breuner@itwm.fraunhofer.de>
User-Agent: Mozilla/5.0 (X11; Linux i686 on x86_64;
	rv:10.0) Gecko/20120129 Thunderbird/10.0
MIME-Version: 1.0
To: Jim Meyering <jim@meyering.net>
Subject: Re: "ls -l": Avoid unnecessary getxattr() overhead
References: <87pqds3snt.fsf@Apollo.jerryland.fr>
	<4F30206B.1020802@itwm.fraunhofer.de>
	<CA+TATJHorgsuY7pxmpxxAeWUHRP5MfFfzQP05o=1H6C4ttswdA@mail.gmail.com>
	<4F3032BC.50702@draigBrady.com>
	<4F30631A.2070009@itwm.fraunhofer.de>
	<87y5scfdiq.fsf@rho.meyering.net> <4F33B5D7.3010909@draigBrady.com>
	<87mx8sf5w0.fsf@rho.meyering.net> <4F33DDCF.7080307@draigBrady.com>
	<87ehu4f3i1.fsf@rho.meyering.net>
In-Reply-To: <87ehu4f3i1.fsf@rho.meyering.net>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
X-ITWM-CharSet: ISO-8859-1
X-ITWM-Scanned-By: mail2.itwm.fhg.de
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
X-Received-From: 131.246.120.220
Cc: Bernd Schubert <bernd.schubert@itwm.fraunhofer.de>,
	=?ISO-8859-1?Q?J=E9r=E9my_Compostella?= <jeremy.compostella@gmail.com>,
	coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 10 Feb 2012 00:35:13 -0000

Jim Meyering wrote on 02/09/2012 04:08 PM:
> I'm not as enthusiastic about adding code to optimize away
> the calls in file_has_acl, unless there are mainstream system/FS
> for which it makes a difference.

Jim Meyering wrote on 02/09/2012 03:17 PM:
 > Here's that simpler patch:
 >
 > diff --git a/src/ls.c b/src/ls.c
 > index f5cd37a..cb9f834 100644
 > [...]


Does cifs count as mainstream in this context?

Here's a test run on rhel 6.2 (one machine exporting an ext4, the other 
one mounting via cifs with default settings), selinux_challenged_device 
patch already applied to ls:

$ strace -c ~/tmp/ls-8.15.34-31eee_patched -l >/dev/null
% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
  44.56    0.186147           3     61440           lstat
  38.94    0.162680           3     61440     61440 getxattr
  15.80    0.065989         122       540           getdents
[...]

While this kind of large directory might not be very typical as a use 
case for cifs, I know at least quite a few cifs filer admins that would 
be happy about every single network request they could keep off of their 
filers to reduce load.

On the other hand, you say you don't feel enthusiastic about such a 
change, so you probably have good reasons for that (I have to admit I 
didn't take a closer look at that code yet).

Best regards,
Sven


From MAILER-DAEMON Thu Feb 09 20:03:16 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RvetQ-0005p8-Q7
	for mharc-coreutils@gnu.org; Thu, 09 Feb 2012 20:03:16 -0500
Received: from eggs.gnu.org ([140.186.70.92]:41455)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RvetI-0005mD-BM
	for coreutils@gnu.org; Thu, 09 Feb 2012 20:03:13 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RvetG-0006Zr-Up
	for coreutils@gnu.org; Thu, 09 Feb 2012 20:03:08 -0500
Received: from mail3.vodafone.ie ([213.233.128.45]:16173)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RvetG-0006ZQ-N3
	for coreutils@gnu.org; Thu, 09 Feb 2012 20:03:06 -0500
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: Aq8BAHVrNE9tTcN5/2dsb2JhbAAMN60lhTEBAQEEJwsBRhALDQsJFg8JAwIBAgFFBg0BBwEBwQCLPxcGCRYGBAMEBAcOBgEDCAENg36EFwSbK4xm
Received: from unknown (HELO [192.168.1.79]) ([109.77.195.121])
	by mail3.vodafone.ie with ESMTP; 10 Feb 2012 01:03:04 +0000
Message-ID: <4F346CC7.4030608@draigBrady.com>
Date: Fri, 10 Feb 2012 01:03:03 +0000
From: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:6.0) Gecko/20110816 Thunderbird/6.0
MIME-Version: 1.0
To: Sven Breuner <sven.breuner@itwm.fraunhofer.de>
Subject: Re: "ls -l": Avoid unnecessary getxattr() overhead
References: <87pqds3snt.fsf@Apollo.jerryland.fr>
	<4F30206B.1020802@itwm.fraunhofer.de>
	<CA+TATJHorgsuY7pxmpxxAeWUHRP5MfFfzQP05o=1H6C4ttswdA@mail.gmail.com>
	<4F3032BC.50702@draigBrady.com>
	<4F30631A.2070009@itwm.fraunhofer.de>
	<87y5scfdiq.fsf@rho.meyering.net> <4F33B5D7.3010909@draigBrady.com>
	<87mx8sf5w0.fsf@rho.meyering.net>
	<4F3437CE.4050308@itwm.fraunhofer.de>
In-Reply-To: <4F3437CE.4050308@itwm.fraunhofer.de>
X-Enigmail-Version: 1.3.2
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 213.233.128.45
Cc: Bernd Schubert <bernd.schubert@itwm.fraunhofer.de>,
	=?ISO-8859-1?Q?J=E9r=E9my_Compostella?= <jeremy.compostella@gmail.com>,
	coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 10 Feb 2012 01:03:14 -0000

On 02/09/2012 09:17 PM, Sven Breuner wrote:
> Jim Meyering wrote on 02/09/2012 03:17 PM:
>> Pádraig Brady wrote:
>>> On 02/09/2012 11:32 AM, Jim Meyering wrote:
>>>> Technically, we could probably exempt all files on that device, but
>>>> ...
>>>
>>> So you avoid symlinks as they can point outside the device.
>>> Unfortunately so can bind mounts, so you probably have to key on device?
>>
>> Thanks.  That confirms my impression that my patch was not worthwhile:
>> It was buggy, too.
>>
>> I suppose we could use a little hash table, whose entries are
>> <dev_t device_number, bool getfilecon_required>  pairs.
>>
>> Or even a single static dev_t selinux_challenged_device,
>> that if equal to f->stat.st_dev, we can skip the *getfilecon call.
>> That should be good enough for most uses.
>>
>> Here's that simpler patch:
>>
>> diff --git a/src/ls.c b/src/ls.c
>> index f5cd37a..cb9f834 100644
>> [...]
> 
> 
> Just tried it out and the difference is very significant, as expected.
> (Test case with 61,441 files in a directory.)
> 
> Current git version without the patch...
> 
> $ time ~/tmp/ls-8.15.34-31eee -l >/dev/null
> 
> real    0m31.135s
> user    0m0.859s
> sys     0m4.276s
> 
> $ strace -c ~/tmp/ls-8.15.34-31eee -l >/dev/null
> % time     seconds  usecs/call     calls    errors syscall
> ------ ----------- ----------- --------- --------- ----------------
>  31.13    0.303648           5     61441           lstat
>  30.81    0.300585           5     61441     61441 getxattr
>  30.65    0.298937           5     61441           lgetxattr
>   7.17    0.069987         933        75           getdents
> [...]
> 
> Current git version patched...
> 
> $ time ~/tmp/ls-8.15.34-31eee_patched -l >/dev/null
> 
> real    0m21.254s
> user    0m0.565s
> sys     0m2.496s
> 
> $ strace -c ~/tmp/ls-8.15.34-31eee_patched -l >/dev/null
> % time     seconds  usecs/call     calls    errors syscall
> ------ ----------- ----------- --------- --------- ----------------
>  45.50    0.313386           5     61441           lstat
>  43.63    0.300524           5     61441     61441 getxattr
>  10.60    0.072986         973        75           getdents

So you're still getting all those errors.
Is that from file_has_acl() or cap_get_file()?
Oh it must be from file_has_acl() as you've
not specified --color. What is the errno in that case?

Note you can avoid the cap_get_file() calls at least,
that would be enabled with --color, by unsetting capability
coloring, which you can test by running the following
before your ls command.

eval $(dircolors | sed 's/ca[^:]*:/ca=:/')

cheers,
Pádraig.


From MAILER-DAEMON Fri Feb 10 03:16:16 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RvleS-0001O2-47
	for mharc-coreutils@gnu.org; Fri, 10 Feb 2012 03:16:16 -0500
Received: from eggs.gnu.org ([140.186.70.92]:36404)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1RvleM-0001Nw-4T
	for coreutils@gnu.org; Fri, 10 Feb 2012 03:16:14 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1RvleK-0003Sq-S4
	for coreutils@gnu.org; Fri, 10 Feb 2012 03:16:10 -0500
Received: from mx.meyering.net ([88.168.87.75]:41073)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1RvleK-0003Sl-Gp
	for coreutils@gnu.org; Fri, 10 Feb 2012 03:16:08 -0500
Received: from rho.meyering.net (localhost.localdomain [127.0.0.1])
	by rho.meyering.net (Acme Bit-Twister) with ESMTP id 73AAE60061;
	Fri, 10 Feb 2012 09:16:07 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: =?iso-8859-1?Q?P=E1draig?= Brady <P@draigBrady.com>
Subject: Re: "ls -l": Avoid unnecessary getxattr() overhead
In-Reply-To: <4F346CC7.4030608@draigBrady.com> (=?iso-8859-1?Q?=22P=E1drai?=
	=?iso-8859-1?Q?g?= Brady"'s message of
	"Fri, 10 Feb 2012 01:03:03 +0000")
References: <87pqds3snt.fsf@Apollo.jerryland.fr>
	<4F30206B.1020802@itwm.fraunhofer.de>
	<CA+TATJHorgsuY7pxmpxxAeWUHRP5MfFfzQP05o=1H6C4ttswdA@mail.gmail.com>
	<4F3032BC.50702@draigBrady.com> <4F30631A.2070009@itwm.fraunhofer.de>
	<87y5scfdiq.fsf@rho.meyering.net> <4F33B5D7.3010909@draigBrady.com>
	<87mx8sf5w0.fsf@rho.meyering.net>
	<4F3437CE.4050308@itwm.fraunhofer.de>
	<4F346CC7.4030608@draigBrady.com>
Date: Fri, 10 Feb 2012 09:16:07 +0100
Message-ID: <87r4y3dry0.fsf@rho.meyering.net>
Lines: 83
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 88.168.87.75
Cc: Sven Breuner <sven.breuner@itwm.fraunhofer.de>,
	Bernd Schubert <bernd.schubert@itwm.fraunhofer.de>,
	=?iso-8859-1?Q?J=E9r=E9my?= Compostella <jeremy.compostella@gmail.com>,
	coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 10 Feb 2012 08:16:14 -0000

P=E1draig Brady wrote:

> On 02/09/2012 09:17 PM, Sven Breuner wrote:
>> Jim Meyering wrote on 02/09/2012 03:17 PM:
>>> P=E1draig Brady wrote:
>>>> On 02/09/2012 11:32 AM, Jim Meyering wrote:
>>>>> Technically, we could probably exempt all files on that device, but
>>>>> ...
>>>>
>>>> So you avoid symlinks as they can point outside the device.
>>>> Unfortunately so can bind mounts, so you probably have to key on devic=
e?
>>>
>>> Thanks.  That confirms my impression that my patch was not worthwhile:
>>> It was buggy, too.
>>>
>>> I suppose we could use a little hash table, whose entries are
>>> <dev_t device_number, bool getfilecon_required>  pairs.
>>>
>>> Or even a single static dev_t selinux_challenged_device,
>>> that if equal to f->stat.st_dev, we can skip the *getfilecon call.
>>> That should be good enough for most uses.
>>>
>>> Here's that simpler patch:
>>>
>>> diff --git a/src/ls.c b/src/ls.c
>>> index f5cd37a..cb9f834 100644
>>> [...]
>>
>>
>> Just tried it out and the difference is very significant, as expected.
>> (Test case with 61,441 files in a directory.)
>>
>> Current git version without the patch...
>>
>> $ time ~/tmp/ls-8.15.34-31eee -l >/dev/null
>>
>> real    0m31.135s
>> user    0m0.859s
>> sys     0m4.276s
>>
>> $ strace -c ~/tmp/ls-8.15.34-31eee -l >/dev/null
>> % time     seconds  usecs/call     calls    errors syscall
>> ------ ----------- ----------- --------- --------- ----------------
>>  31.13    0.303648           5     61441           lstat
>>  30.81    0.300585           5     61441     61441 getxattr
>>  30.65    0.298937           5     61441           lgetxattr
>>   7.17    0.069987         933        75           getdents
>> [...]
>>
>> Current git version patched...
>>
>> $ time ~/tmp/ls-8.15.34-31eee_patched -l >/dev/null
>>
>> real    0m21.254s
>> user    0m0.565s
>> sys     0m2.496s
>>
>> $ strace -c ~/tmp/ls-8.15.34-31eee_patched -l >/dev/null
>> % time     seconds  usecs/call     calls    errors syscall
>> ------ ----------- ----------- --------- --------- ----------------
>>  45.50    0.313386           5     61441           lstat
>>  43.63    0.300524           5     61441     61441 getxattr
>>  10.60    0.072986         973        75           getdents
>
> So you're still getting all those errors.
> Is that from file_has_acl() or cap_get_file()?
> Oh it must be from file_has_acl() as you've
> not specified --color. What is the errno in that case?

So we can avoid all but one of those failing getxattr calls
in this case.  It's a little tricky because of how file_has_acl works.
We may have to set errno=3D0 just prior, and then test errno
when file_has_acl returns <=3D0.  Sven, can you tell us whether
file_has_acl is returning -1 or 0 in your case?  And confirm
that the errno value is propagated out to ls.c?

So either print both "n" and "errno" right after this line in ls.c

              int n =3D file_has_acl (absolute_name, &f->stat);

or inspect those values in gdb?
If n is always negative for you, then I think we can skip the
kludgy errno=3D0 initialization.


From MAILER-DAEMON Fri Feb 10 03:18:52 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1Rvlgy-0002D6-RP
	for mharc-coreutils@gnu.org; Fri, 10 Feb 2012 03:18:52 -0500
Received: from eggs.gnu.org ([140.186.70.92]:36953)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Rvlgv-0002AA-Mc
	for coreutils@gnu.org; Fri, 10 Feb 2012 03:18:50 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Rvlgr-0003jC-OY
	for coreutils@gnu.org; Fri, 10 Feb 2012 03:18:49 -0500
Received: from mx.meyering.net ([88.168.87.75]:41085)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Rvlgr-0003j7-Gy
	for coreutils@gnu.org; Fri, 10 Feb 2012 03:18:45 -0500
Received: from rho.meyering.net (localhost.localdomain [127.0.0.1])
	by rho.meyering.net (Acme Bit-Twister) with ESMTP id B54B4601FB;
	Fri, 10 Feb 2012 09:18:44 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: Sven Breuner <sven.breuner@itwm.fraunhofer.de>
Subject: Re: "ls -l": Avoid unnecessary getxattr() overhead
In-Reply-To: <4F346638.9090304@itwm.fraunhofer.de> (Sven Breuner's message of
	"Fri, 10 Feb 2012 01:35:04 +0100")
References: <87pqds3snt.fsf@Apollo.jerryland.fr>
	<4F30206B.1020802@itwm.fraunhofer.de>
	<CA+TATJHorgsuY7pxmpxxAeWUHRP5MfFfzQP05o=1H6C4ttswdA@mail.gmail.com>
	<4F3032BC.50702@draigBrady.com> <4F30631A.2070009@itwm.fraunhofer.de>
	<87y5scfdiq.fsf@rho.meyering.net> <4F33B5D7.3010909@draigBrady.com>
	<87mx8sf5w0.fsf@rho.meyering.net> <4F33DDCF.7080307@draigBrady.com>
	<87ehu4f3i1.fsf@rho.meyering.net>
	<4F346638.9090304@itwm.fraunhofer.de>
Date: Fri, 10 Feb 2012 09:18:44 +0100
Message-ID: <87liobdrtn.fsf@rho.meyering.net>
Lines: 37
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 88.168.87.75
Cc: Bernd Schubert <bernd.schubert@itwm.fraunhofer.de>,
	=?iso-8859-1?Q?J=E9r=E9my?= Compostella <jeremy.compostella@gmail.com>,
	coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 10 Feb 2012 08:18:50 -0000

Sven Breuner wrote:
> Jim Meyering wrote on 02/09/2012 04:08 PM:
>> I'm not as enthusiastic about adding code to optimize away
>> the calls in file_has_acl, unless there are mainstream system/FS
>> for which it makes a difference.
>
> Jim Meyering wrote on 02/09/2012 03:17 PM:
>> Here's that simpler patch:
>>
>> diff --git a/src/ls.c b/src/ls.c
>> index f5cd37a..cb9f834 100644
>> [...]
>
>
> Does cifs count as mainstream in this context?

Yes.
Which means it is worth making the change to cache failed file_has_acl, too.

> Here's a test run on rhel 6.2 (one machine exporting an ext4, the
> other one mounting via cifs with default settings),
> selinux_challenged_device patch already applied to ls:
>
> $ strace -c ~/tmp/ls-8.15.34-31eee_patched -l >/dev/null
> % time     seconds  usecs/call     calls    errors syscall
> ------ ----------- ----------- --------- --------- ----------------
>  44.56    0.186147           3     61440           lstat
>  38.94    0.162680           3     61440     61440 getxattr
>  15.80    0.065989         122       540           getdents
> [...]
>
> While this kind of large directory might not be very typical as a use
> case for cifs, I know at least quite a few cifs filer admins that
> would be happy about every single network request they could keep off
> of their filers to reduce load.

cifs is a big market.  Thanks for the info.


From MAILER-DAEMON Fri Feb 10 11:21:41 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RvtED-00055T-Ck
	for mharc-coreutils@gnu.org; Fri, 10 Feb 2012 11:21:41 -0500
Received: from eggs.gnu.org ([140.186.70.92]:48005)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1RvtE7-00053f-FE
	for coreutils@gnu.org; Fri, 10 Feb 2012 11:21:39 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1RvtDx-0002sH-3y
	for coreutils@gnu.org; Fri, 10 Feb 2012 11:21:35 -0500
Received: from mail-tul01m020-f169.google.com ([209.85.214.169]:57711)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1RvtDw-0002s2-S4
	for coreutils@gnu.org; Fri, 10 Feb 2012 11:21:25 -0500
Received: by obbta7 with SMTP id ta7so4914021obb.0
	for <coreutils@gnu.org>; Fri, 10 Feb 2012 08:21:23 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma;
	h=mime-version:in-reply-to:references:from:date:message-id:subject:to
	:cc:content-type;
	bh=TARgFkN+TWiuaIJK0PYkdyYxnv84OFRnfbCP2Ok34LM=;
	b=wKRWoi16IJq3fdXnBETTnJdHQAxwFICpsqyYJOJ4e3l8Vv7/DO2kIXVkdWu6D69A75
	6xvEUPu3p68v1Nf2wf8PGPk8nwgDQQi3zvWeWazrFeNO4n8MMJ+6leaxFAGRP/yaiZqH
	nSDLCu/S//jH35bUimerXNDTLgvC22tLo3yxI=
Received: by 10.50.171.40 with SMTP id ar8mr12098993igc.26.1328890883287; Fri,
	10 Feb 2012 08:21:23 -0800 (PST)
MIME-Version: 1.0
Received: by 10.42.29.196 with HTTP; Fri, 10 Feb 2012 08:21:03 -0800 (PST)
In-Reply-To: <CA+TATJFkn8=Kd3iqrtk=z2pkkzKRGwnAsquw61r0kqSMx76zgQ@mail.gmail.com>
References: <87mx8w3ryu.fsf@Apollo.jerryland.fr>
	<4F2FF5EE.9010906@draigBrady.com>
	<CA+TATJF4w4Ue44Za4hgGR_0RxgEZGBm5nzhHRSdGYxzzfKWNLg@mail.gmail.com>
	<4F317CDF.2040909@draigBrady.com>
	<CA+TATJFU7yOGzzeZseHUFTqxxGAtz8zUoQccz=5m0Eu=CqEtRQ@mail.gmail.com>
	<4F329FC9.4030403@draigBrady.com>
	<CA+TATJFkn8=Kd3iqrtk=z2pkkzKRGwnAsquw61r0kqSMx76zgQ@mail.gmail.com>
From: =?ISO-8859-1?Q?J=E9r=E9my_Compostella?= <jeremy.compostella@gmail.com>
Date: Fri, 10 Feb 2012 17:21:03 +0100
Message-ID: <CA+TATJF2uW6eHqF4fkwacHq4xm1XPY=3Ent3RHDNPuMA8GOZVA@mail.gmail.com>
Subject: Re: dd: add 'skip_bytes' and 'count_bytes' operands
To: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigbrady.com>
Content-Type: multipart/mixed; boundary=e89a8f234cd30b2a7404b89e85cf
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.85.214.169
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 10 Feb 2012 16:21:39 -0000

--e89a8f234cd30b2a7404b89e85cf
Content-Type: multipart/alternative; boundary=e89a8f234cd30b2a7004b89e85cd

--e89a8f234cd30b2a7004b89e85cd
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

P=E1draig, others,

First, I added the missing seek_bytes feature.
Second, I re-wrote the patch in order to replace the _bytes operands by the
corresponding input/output flags.
Finally, I added some tests to check the new seek_bytes flag.

I successfully run the entire tests suite.

I attached the new patch. Do not hesitate to comment it.

Cheers,

J=E9r=E9my

--e89a8f234cd30b2a7004b89e85cd
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div>P=E1draig, others,</div><div><br></div><div>First, I added the missing=
 seek_bytes feature.</div><div>Second, I re-wrote the patch in order to rep=
lace the _bytes operands by the</div><div>corresponding input/output flags.=
</div>

<div>Finally, I added some tests to check the new seek_bytes flag.</div><di=
v><br></div><div>I successfully run the entire tests suite.</div><div><br><=
/div><div>I attached the new patch. Do not hesitate to comment it.</div>

<div><br></div><div>Cheers,</div><div><br></div><div>J=E9r=E9my</div><div><=
br></div>

--e89a8f234cd30b2a7004b89e85cd--
--e89a8f234cd30b2a7404b89e85cf
Content-Type: text/x-patch; charset=US-ASCII; 
	name="0001-dd-add-count_bytes-skip_bytes-and-seek_bytes-flags.patch"
Content-Disposition: attachment; 
	filename="0001-dd-add-count_bytes-skip_bytes-and-seek_bytes-flags.patch"
Content-Transfer-Encoding: base64
X-Attachment-Id: f_gyhfa4vy0

RnJvbSBmZjBhMTg2Y2MwMzEyNjMxZmZjOWU0MjA1YTg3YWNhMWEwMzY1OTcyIE1vbiBTZXAgMTcg
MDA6MDA6MDAgMjAwMQpGcm9tOiA9P1VURi04P3E/Sj1DMz1BOXI9QzM9QTlteT0yMENvbXBvc3Rl
bGxhPz0gPGplcmVteS5jb21wb3N0ZWxsYUBnbWFpbC5jb20+CkRhdGU6IFNhdCwgNCBGZWIgMjAx
MiAxNToyNTo1NCArMDEwMApTdWJqZWN0OiBbUEFUQ0hdIGRkOiBhZGQgY291bnRfYnl0ZXMsIHNr
aXBfYnl0ZXMgYW5kIHNlZWtfYnl0ZXMgZmxhZ3MKCmRkIG5vdyBhY2NlcHRzIHRoZSBjb3VudF9i
eXRlcyBhbmQgc2tpcF9ieXRlcyBpbnB1dCBmbGFnIGFuZCB0aGUKc2Vla19ieXRlcyBvdXRwdXQg
ZmxhZywgdG8gbW9yZSBlYXNpbHkgYWxsb3cgcHJvY2Vzc2luZyBwb3J0aW9ucyBvZiBhCmZpbGUu
CgoqIHNyYy9kZC5jIChzY2FuYXJncyk6IENvbXB1dGUgc2tpcF9yZWNvcmRzIGFuZCBza2lwX2J5
dGVzIHdoZW4KJ3NraXBfYnl0ZXMnIGlmbGFnIGlzIHVzZWQuIENvbXB1dGUgbWF4X3JlY29yZHMg
YW5kIG1heF9ieXRlcyB3aGVuCidjb3VudF9ieXRlcycgaWZsYWcgaXMgdXNlZC4gQ29tcHV0ZSBz
ZWVrX3JlY29yZHMgYW5kIHNlZWtfYnl0ZXMKd2hlbiAnc2Vla19ieXRlcycgb2ZsYWcgaXMgdXNl
ZC4KKHNraXBfdmlhX2xzZWVrKTogVXNlIG5ldyAnYnl0ZXMnIHBhcmFtZXRlciBhbmQgaGFuZGxl
IHBvdGVudGlhbAoncmVjb3JkcycgZXF1YWxzIHRvIHplcm8uIFVwZGF0ZSB0aGUgYnl0ZXMgcGFy
YW1ldGVyIHdoZW4gY2FsbGVkIHdpdGgKJ2ZkZXNjJyBlcXVhbCB0byBTVERPVVRfRklMRU5PLiBV
cGRhdGUgdGhlIGhlYWRlciBjb21tZW50cy4KKGRkX2NvcHkpOiBTa2lwIGFjY29yZGluZ2x5IHRv
IHNraXBfcmVjb3JkcyBBTkQgc2tpcF9ieXRlcy4gQ291bnQKYWNjb3JkaW5nbHkgdG8gbWF4X3Jl
Y29yZHMgQU5EIG1heF9ieXRlcy4gU2VlayBvbiBvdXRwdXQgYWNjb3JkaW5nbHkKdG8gc2Vla19y
ZWNvcmRzIEFORCBzZWVrX2J5dGVzLgoqIE5FV1MgKE5ldyBmZWF0dXJlcyk6IE1lbnRpb24gaXQu
CiogZG9jL2NvcmV1dGlscy50ZXhpIChOZXcgZmVhdHVyZXMpOiBEZXRhaWwgbmV3IGZsYWdzIGFu
ZCBiZWhhdmlvcnMuCiogdGVzdHMvZGQvYnl0ZXM6IE5ldyBmaWxlLiBUZXN0cyBmb3IgdGhlc2Ug
bmV3IGZsYWdzLgoqIHRlc3RzL01ha2VmaWxlLmFtIChURVNUUyk6IEFkZCBpdC4KLS0tCiBORVdT
ICAgICAgICAgICAgICAgfCAgICA1ICsrCiBkb2MvY29yZXV0aWxzLnRleGkgfCAgIDQyICsrKysr
KysrLS0tCiBzcmMvZGQuYyAgICAgICAgICAgfCAgMjAxICsrKysrKysrKysrKysrKysrKysrKysr
KysrKysrKysrKysrKysrLS0tLS0tLS0tLS0tLS0KIHRlc3RzL01ha2VmaWxlLmFtICB8ICAgIDEg
KwogdGVzdHMvZGQvYnl0ZXMgICAgIHwgICA1NyArKysrKysrKysrKysrKysKIDUgZmlsZXMgY2hh
bmdlZCwgMjQyIGluc2VydGlvbnMoKyksIDY0IGRlbGV0aW9ucygtKQogY3JlYXRlIG1vZGUgMTAw
NzU1IHRlc3RzL2RkL2J5dGVzCgpkaWZmIC0tZ2l0IGEvTkVXUyBiL05FV1MKaW5kZXggOWVlYmJm
Ni4uZTcwNWU3MSAxMDA2NDQKLS0tIGEvTkVXUworKysgYi9ORVdTCkBAIC0yLDYgKzIsMTEgQEAg
R05VIGNvcmV1dGlscyBORVdTICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgLSot
IG91dGxpbmUgLSotCiAKICogTm90ZXdvcnRoeSBjaGFuZ2VzIGluIHJlbGVhc2UgPy4/ICg/Pz8/
LT8/LT8/KSBbP10KIAorKiogTmV3IGZlYXR1cmVzCisKKyAgZGQgbm93IGFjY2VwdHMgdGhlIGNv
dW50X2J5dGVzLCBza2lwX2J5dGVzIGlmbGFncyBhbmQgdGhlIGNvdW50X2J5dGVzCisgIG9mbGFn
LCB0byBtb3JlIGVhc2lseSBhbGxvdyBwcm9jZXNzaW5nIHBvcnRpb25zIG9mIGEgZmlsZS4KKwog
KiogQnVnIGZpeGVzCiAKICAgbXYgbm93IGxldHMgeW91IG1vdmUgYSBzeW1saW5rIG9udG8gYSBz
YW1lLWlub2RlIGRlc3RpbmF0aW9uIGZpbGUgdGhhdApkaWZmIC0tZ2l0IGEvZG9jL2NvcmV1dGls
cy50ZXhpIGIvZG9jL2NvcmV1dGlscy50ZXhpCmluZGV4IDUyODM4ZTcuLjZlZmM1Y2MgMTAwNjQ0
Ci0tLSBhL2RvYy9jb3JldXRpbHMudGV4aQorKysgYi9kb2MvY29yZXV0aWxzLnRleGkKQEAgLTgw
NTcsMTcgKzgwNTcsMTcgQEAgV2hlbiBjb252ZXJ0aW5nIHZhcmlhYmxlLWxlbmd0aCByZWNvcmRz
IHRvIGZpeGVkLWxlbmd0aCBvbmVzCiAoQG9wdGlvbntjb252PWJsb2NrfSkgb3IgdGhlIHJldmVy
c2UgKEBvcHRpb257Y29udj11bmJsb2NrfSksCiB1c2UgQHZhcntieXRlc30gYXMgdGhlIGZpeGVk
IHJlY29yZCBsZW5ndGguCiAKLUBpdGVtIHNraXA9QHZhcntibG9ja3N9CitAaXRlbSBza2lwPUB2
YXJ7bn0KIEBvcGluZGV4IHNraXAKLVNraXAgQHZhcntibG9ja3N9IEBzYW1we2lic30tYnl0ZSBi
bG9ja3MgaW4gdGhlIGlucHV0IGZpbGUgYmVmb3JlIGNvcHlpbmcuCitTa2lwIEB2YXJ7bn0gQHNh
bXB7aWJzfS1ieXRlIGJsb2NrcyBpbiB0aGUgaW5wdXQgZmlsZSBiZWZvcmUgY29weWluZy4KIAot
QGl0ZW0gc2Vlaz1AdmFye2Jsb2Nrc30KK0BpdGVtIHNlZWs9QHZhcntufQogQG9waW5kZXggc2Vl
awotU2tpcCBAdmFye2Jsb2Nrc30gQHNhbXB7b2JzfS1ieXRlIGJsb2NrcyBpbiB0aGUgb3V0cHV0
IGZpbGUgYmVmb3JlIGNvcHlpbmcuCitTa2lwIEB2YXJ7bn0gQHNhbXB7b2JzfS1ieXRlIGJsb2Nr
cyBpbiB0aGUgb3V0cHV0IGZpbGUgYmVmb3JlIGNvcHlpbmcuCiAKLUBpdGVtIGNvdW50PUB2YXJ7
YmxvY2tzfQorQGl0ZW0gY291bnQ9QHZhcntufQogQG9waW5kZXggY291bnQKLUNvcHkgQHZhcnti
bG9ja3N9IEBzYW1we2lic30tYnl0ZSBibG9ja3MgZnJvbSB0aGUgaW5wdXQgZmlsZSwgaW5zdGVh
ZAorQ29weSBAdmFye259IEBzYW1we2lic30tYnl0ZSBibG9ja3MgZnJvbSB0aGUgaW5wdXQgZmls
ZSwgaW5zdGVhZAogb2YgZXZlcnl0aGluZyB1bnRpbCB0aGUgZW5kIG9mIHRoZSBmaWxlLgogCiBA
aXRlbSBzdGF0dXM9bm94ZmVyCkBAIC04MzIxLDYgKzgzMjEsMjQgQEAgV2hlbiB0aGF0IGhhcHBl
bnMsIGNvbnRpbnVlIGNhbGxpbmcgQGNvZGV7cmVhZH0gdG8gZmlsbCB0aGUgcmVtYWluZGVyCiBv
ZiB0aGUgYmxvY2suCiBUaGlzIGZsYWcgY2FuIGJlIHVzZWQgb25seSB3aXRoIEBjb2Rle2lmbGFn
fS4KIAorQGl0ZW0gY291bnRfYnl0ZXMKK0BvcGluZGV4IGNvdW50X2J5dGVzCitVc2UgYnl0ZSB1
bml0IGluc3RlYWQgb2YgQHNhbXB7aWJzfS1ieXRlIGJsb2NrIHVuaXQgZm9yIHRoZQorQHNhbXB7
Y291bnQ9bn0gb3BlcmFuZC4KK1RoaXMgZmxhZyBjYW4gYmUgdXNlZCBvbmx5IHdpdGggQGNvZGV7
aWZsYWd9LgorCitAaXRlbSBza2lwX2J5dGVzCitAb3BpbmRleCBza2lwX2J5dGVzCitVc2UgYnl0
ZSB1bml0IGluc3RlYWQgb2YgQHNhbXB7aWJzfS1ieXRlIGJsb2NrIHVuaXQgZm9yIHRoZQorQHNh
bXB7c2tpcD1ufSBvcGVyYW5kLgorVGhpcyBmbGFnIGNhbiBiZSB1c2VkIG9ubHkgd2l0aCBAY29k
ZXtpZmxhZ30uCisKK0BpdGVtIHNlZWtfYnl0ZXMKK0BvcGluZGV4IHNlZWtfYnl0ZXMKK1VzZSBi
eXRlIHVuaXQgaW5zdGVhZCBvZiBAc2FtcHtvYnN9LWJ5dGUgYmxvY2sgdW5pdCBmb3IgdGhlCitA
c2FtcHtzZWVrPW59IG9wZXJhbmQuCitUaGlzIGZsYWcgY2FuIGJlIHVzZWQgb25seSB3aXRoIEBj
b2Rle29mbGFnfS4KKwogQGVuZCB0YWJsZQogCiBUaGVzZSBmbGFncyBhcmUgbm90IHN1cHBvcnRl
ZCBvbiBhbGwgc3lzdGVtcywgYW5kIEBzYW1we2RkfSByZWplY3RzCkBAIC04MzQzLDEwICs4MzYx
LDE0IEBAIHNob3VsZCBub3QgYmUgdG9vIGxhcmdlLS0tdmFsdWVzIGxhcmdlciB0aGFuIGEgZmV3
IG1lZ2FieXRlcwogYXJlIGdlbmVyYWxseSB3YXN0ZWZ1bCBvciAoYXMgaW4gdGhlIGdpZ2FieXRl
Li5leGFieXRlIGNhc2UpIGRvd25yaWdodAogY291bnRlcnByb2R1Y3RpdmUgb3IgZXJyb3ItaW5k
dWNpbmcuCiAKLVVzZSBkaWZmZXJlbnQgQGNvbW1hbmR7ZGR9IGludm9jYXRpb25zIHRvIHVzZSBk
aWZmZXJlbnQgYmxvY2sgc2l6ZXMgZm9yCi1za2lwcGluZyBhbmQgSS9PQC4gIEZvciBleGFtcGxl
LCB0aGUgZm9sbG93aW5nIHNoZWxsIGNvbW1hbmRzIGNvcHkgZGF0YQotaW4gNTEyIEtpQiBibG9j
a3MgYmV0d2VlbiBhIGRpc2sgYW5kIGEgdGFwZSwgYnV0IGRvIG5vdCBzYXZlIG9yIHJlc3RvcmUg
YQotNCBLaUIgbGFiZWwgYXQgdGhlIHN0YXJ0IG9mIHRoZSBkaXNrOgorVXNlIGRpZmZlcmVudCBA
Y29tbWFuZHtkZH0gaW52b2NhdGlvbnMgdG8gdXNlIGRpZmZlcmVudCBibG9jayBzaXplcworZm9y
IHNraXBwaW5nIGFuZCBJL09ALiBUbyBwcm9jZXNzIGRhdGEgdGhhdCBpcyBhdCBhbiBvZmZzZXQg
b3Igc2l6ZQordGhhdCBpcyBub3QgYSBtdWx0aXBsZSBvZiB0aGUgSS9PQCBibG9jayBzaXplLCB5
b3UgY2FuIHVzZSB0aGUKK2NvdW50X2J5dGVzIGFuZCBza2lwX2J5dGVzIGlucHV0IGZsYWcgYW5k
IHRoZSBzZWVrX2J5dGVzIG91dHB1dCBmbGFnLgorQWx0ZXJuYXRpdmVseSB0aGUgdHJhZGl0aW9u
YWwgbWV0aG9kIG9mIHNlcGFyYXRlIEBjb21tYW5ke2RkfQoraW52b2NhdGlvbnMgY2FuIGJlIHVz
ZWQuICBGb3IgZXhhbXBsZSwgdGhlIGZvbGxvd2luZyBzaGVsbCBjb21tYW5kcworY29weSBkYXRh
IGluIDUxMiBLaUIgYmxvY2tzIGJldHdlZW4gYSBkaXNrIGFuZCBhIHRhcGUsIGJ1dCBkbyBub3Qg
c2F2ZQorb3IgcmVzdG9yZSBhIDQgS2lCIGxhYmVsIGF0IHRoZSBzdGFydCBvZiB0aGUgZGlzazoK
IAogQGV4YW1wbGUKIGRpc2s9L2Rldi9yZHNrL2MwdDFkMHMyCmRpZmYgLS1naXQgYS9zcmMvZGQu
YyBiL3NyYy9kZC5jCmluZGV4IDlkMjQ3OTEuLjdhYjdjYjIgMTAwNjQ0Ci0tLSBhL3NyYy9kZC5j
CisrKyBiL3NyYy9kZC5jCkBAIC0xNTYsMTIgKzE1NiwyMyBAQCBzdGF0aWMgc2l6ZV90IGNvbnZl
cnNpb25fYmxvY2tzaXplID0gMDsKIC8qIFNraXAgdGhpcyBtYW55IHJlY29yZHMgb2YgJ2lucHV0
X2Jsb2Nrc2l6ZScgYnl0ZXMgYmVmb3JlIGlucHV0LiAqLwogc3RhdGljIHVpbnRtYXhfdCBza2lw
X3JlY29yZHMgPSAwOwogCisvKiBTa2lwIHRoaXMgbWFueSBieXRlcyBiZWZvcmUgaW5wdXQgaW4g
YWRkaXRpb24gb2YgJ3NraXBfcmVjb3JkcycKKyAgIHJlY29yZHMuICAqLworc3RhdGljIHNpemVf
dCBza2lwX2J5dGVzID0gMDsKKwogLyogU2tpcCB0aGlzIG1hbnkgcmVjb3JkcyBvZiAnb3V0cHV0
X2Jsb2Nrc2l6ZScgYnl0ZXMgYmVmb3JlIG91dHB1dC4gKi8KIHN0YXRpYyB1aW50bWF4X3Qgc2Vl
a19yZWNvcmRzID0gMDsKIAorLyogU2tpcCB0aGlzIG1hbnkgYnl0ZXMgaW4gYWRkaXRpb24gdG8g
J3NlZWtfcmVjb3JkcycgcmVjb3JkcyBiZWZvcmUKKyAgIG91dHB1dC4gICovCitzdGF0aWMgdWlu
dG1heF90IHNlZWtfYnl0ZXMgPSAwOworCiAvKiBDb3B5IG9ubHkgdGhpcyBtYW55IHJlY29yZHMu
ICBUaGUgZGVmYXVsdCBpcyBlZmZlY3RpdmVseSBpbmZpbml0eS4gICovCiBzdGF0aWMgdWludG1h
eF90IG1heF9yZWNvcmRzID0gKHVpbnRtYXhfdCkgLTE7CiAKKy8qIENvcHkgdGhpcyBtYW55IGJ5
dGVzIGluIGFkZGl0aW9uIHRvICdtYXhfcmVjb3JkcycgcmVjb3Jkcy4gICovCitzdGF0aWMgc2l6
ZV90IG1heF9ieXRlcyA9IDA7CisKIC8qIEJpdCB2ZWN0b3Igb2YgY29udmVyc2lvbnMgdG8gYXBw
bHkuICovCiBzdGF0aWMgaW50IGNvbnZlcnNpb25zX21hc2sgPSAwOwogCkBAIC0yNDEsNyArMjUy
LDcgQEAgc3RhdGljIGJvb2wgaV9ub2NhY2hlLCBvX25vY2FjaGU7CiBzdGF0aWMgc3NpemVfdCAo
KmlyZWFkX2ZuYykgKGludCBmZCwgY2hhciAqYnVmLCBzaXplX3Qgc2l6ZSk7CiAKIC8qIEEgbG9u
Z2VzdCBzeW1ib2wgaW4gdGhlIHN0cnVjdCBzeW1ib2xfdmFsdWVzIHRhYmxlcyBiZWxvdy4gICov
Ci0jZGVmaW5lIExPTkdFU1RfU1lNQk9MICJmZGF0YXN5bmMiCisjZGVmaW5lIExPTkdFU1RfU1lN
Qk9MICJjb3VudF9ieXRlcyIKIAogLyogQSBzeW1ib2wgYW5kIHRoZSBjb3JyZXNwb25kaW5nIGlu
dGVnZXIgdmFsdWUuICAqLwogc3RydWN0IHN5bWJvbF92YWx1ZQpAQCAtMjk2LDM3ICszMDcsNTUg
QEAgZW51bQogICAgIE9fRlVMTEJMT0NLID0gRkZTX01BU0sgKHYpLAogICAgIHYyID0gdiBeIE9f
RlVMTEJMT0NLLAogCi0gICAgT19OT0NBQ0hFID0gRkZTX01BU0sgKHYyKQorICAgIE9fTk9DQUNI
RSA9IEZGU19NQVNLICh2MiksCisgICAgdjMgPSB2MiBeIE9fTk9DQUNIRSwKKworICAgIE9fQ09V
TlRfQllURVMgPSBGRlNfTUFTSyAodjMpLAorICAgIHY0ID0gdjMgXiBPX0NPVU5UX0JZVEVTLAor
CisgICAgT19TS0lQX0JZVEVTID0gRkZTX01BU0sgKHY0KSwKKyAgICB2NSA9IHY0IF4gT19TS0lQ
X0JZVEVTLAorCisgICAgT19TRUVLX0JZVEVTID0gRkZTX01BU0sgKHY1KQogICB9OwogCiAvKiBF
bnN1cmUgdGhhdCB3ZSBnb3Qgc29tZXRoaW5nLiAgKi8KIHZlcmlmeSAoT19GVUxMQkxPQ0sgIT0g
MCk7CiB2ZXJpZnkgKE9fTk9DQUNIRSAhPSAwKTsKK3ZlcmlmeSAoT19DT1VOVF9CWVRFUyAhPSAw
KTsKK3ZlcmlmeSAoT19TS0lQX0JZVEVTICE9IDApOwordmVyaWZ5IChPX1NFRUtfQllURVMgIT0g
MCk7CiAKICNkZWZpbmUgTVVMVElQTEVfQklUU19TRVQoaSkgKCgoaSkgJiAoKGkpIC0gMSkpICE9
IDApCiAKIC8qIEVuc3VyZSB0aGF0IHRoaXMgaXMgYSBzaW5nbGUtYml0IHZhbHVlLiAgKi8KIHZl
cmlmeSAoICEgTVVMVElQTEVfQklUU19TRVQgKE9fRlVMTEJMT0NLKSk7CiB2ZXJpZnkgKCAhIE1V
TFRJUExFX0JJVFNfU0VUIChPX05PQ0FDSEUpKTsKK3ZlcmlmeSAoICEgTVVMVElQTEVfQklUU19T
RVQgKE9fQ09VTlRfQllURVMpKTsKK3ZlcmlmeSAoICEgTVVMVElQTEVfQklUU19TRVQgKE9fU0tJ
UF9CWVRFUykpOwordmVyaWZ5ICggISBNVUxUSVBMRV9CSVRTX1NFVCAoT19TRUVLX0JZVEVTKSk7
CiAKIC8qIEZsYWdzLCBmb3IgaWZsYWc9Ii4uLiIgYW5kIG9mbGFnPSIuLi4iLiAgKi8KIHN0YXRp
YyBzdHJ1Y3Qgc3ltYm9sX3ZhbHVlIGNvbnN0IGZsYWdzW10gPQogewotICB7ImFwcGVuZCIsCU9f
QVBQRU5EfSwKLSAgeyJiaW5hcnkiLAlPX0JJTkFSWX0sCi0gIHsiY2lvIiwJT19DSU99LAotICB7
ImRpcmVjdCIsCU9fRElSRUNUfSwKLSAgeyJkaXJlY3RvcnkiLAlPX0RJUkVDVE9SWX0sCi0gIHsi
ZHN5bmMiLAlPX0RTWU5DfSwKLSAgeyJub2F0aW1lIiwJT19OT0FUSU1FfSwKLSAgeyJub2NhY2hl
IiwJT19OT0NBQ0hFfSwgICAvKiBEaXNjYXJkIGNhY2hlLiAgKi8KLSAgeyJub2N0dHkiLAlPX05P
Q1RUWX0sCi0gIHsibm9mb2xsb3ciLAlIQVZFX1dPUktJTkdfT19OT0ZPTExPVyA/IE9fTk9GT0xM
T1cgOiAwfSwKLSAgeyJub2xpbmtzIiwJT19OT0xJTktTfSwKLSAgeyJub25ibG9jayIsCU9fTk9O
QkxPQ0t9LAotICB7InN5bmMiLAlPX1NZTkN9LAotICB7InRleHQiLAlPX1RFWFR9LAotICB7ImZ1
bGxibG9jayIsIE9fRlVMTEJMT0NLfSwgLyogQWNjdW11bGF0ZSBmdWxsIGJsb2NrcyBmcm9tIGlu
cHV0LiAgKi8KKyAgeyJhcHBlbmQiLAkgIE9fQVBQRU5EfSwKKyAgeyJiaW5hcnkiLAkgIE9fQklO
QVJZfSwKKyAgeyJjaW8iLAkgIE9fQ0lPfSwKKyAgeyJkaXJlY3QiLAkgIE9fRElSRUNUfSwKKyAg
eyJkaXJlY3RvcnkiLCAgIE9fRElSRUNUT1JZfSwKKyAgeyJkc3luYyIsCSAgT19EU1lOQ30sCisg
IHsibm9hdGltZSIsCSAgT19OT0FUSU1FfSwKKyAgeyJub2NhY2hlIiwJICBPX05PQ0FDSEV9LCAg
IC8qIERpc2NhcmQgY2FjaGUuICAqLworICB7Im5vY3R0eSIsCSAgT19OT0NUVFl9LAorICB7Im5v
Zm9sbG93IiwJICBIQVZFX1dPUktJTkdfT19OT0ZPTExPVyA/IE9fTk9GT0xMT1cgOiAwfSwKKyAg
eyJub2xpbmtzIiwJICBPX05PTElOS1N9LAorICB7Im5vbmJsb2NrIiwJICBPX05PTkJMT0NLfSwK
KyAgeyJzeW5jIiwJICBPX1NZTkN9LAorICB7InRleHQiLAkgIE9fVEVYVH0sCisgIHsiZnVsbGJs
b2NrIiwgICBPX0ZVTExCTE9DS30sIC8qIEFjY3VtdWxhdGUgZnVsbCBibG9ja3MgZnJvbSBpbnB1
dC4gICovCisgIHsiY291bnRfYnl0ZXMiLCBPX0NPVU5UX0JZVEVTfSwKKyAgeyJza2lwX2J5dGVz
IiwgIE9fU0tJUF9CWVRFU30sCisgIHsic2Vla19ieXRlcyIsICBPX1NFRUtfQllURVN9LAogICB7
IiIsCQkwfQogfTsKIApAQCAtNDg5LDcgKzUxOCw3IEBAIENvcHkgYSBmaWxlLCBjb252ZXJ0aW5n
IGFuZCBmb3JtYXR0aW5nIGFjY29yZGluZyB0byB0aGUgb3BlcmFuZHMuXG5cCiAgIGJzPUJZVEVT
ICAgICAgICByZWFkIGFuZCB3cml0ZSB1cCB0byBCWVRFUyBieXRlcyBhdCBhIHRpbWVcblwKICAg
Y2JzPUJZVEVTICAgICAgIGNvbnZlcnQgQllURVMgYnl0ZXMgYXQgYSB0aW1lXG5cCiAgIGNvbnY9
Q09OVlMgICAgICBjb252ZXJ0IHRoZSBmaWxlIGFzIHBlciB0aGUgY29tbWEgc2VwYXJhdGVkIHN5
bWJvbCBsaXN0XG5cCi0gIGNvdW50PUJMT0NLUyAgICBjb3B5IG9ubHkgQkxPQ0tTIGlucHV0IGJs
b2Nrc1xuXAorICBjb3VudD1OICAgICAgICAgY29weSBvbmx5IE4gaW5wdXQgYmxvY2tzXG5cCiAg
IGlicz1CWVRFUyAgICAgICByZWFkIHVwIHRvIEJZVEVTIGJ5dGVzIGF0IGEgdGltZSAoZGVmYXVs
dDogNTEyKVxuXAogIiksIHN0ZG91dCk7CiAgICAgICBmcHV0cyAoXygiXApAQCAtNDk4LDggKzUy
Nyw4IEBAIENvcHkgYSBmaWxlLCBjb252ZXJ0aW5nIGFuZCBmb3JtYXR0aW5nIGFjY29yZGluZyB0
byB0aGUgb3BlcmFuZHMuXG5cCiAgIG9icz1CWVRFUyAgICAgICB3cml0ZSBCWVRFUyBieXRlcyBh
dCBhIHRpbWUgKGRlZmF1bHQ6IDUxMilcblwKICAgb2Y9RklMRSAgICAgICAgIHdyaXRlIHRvIEZJ
TEUgaW5zdGVhZCBvZiBzdGRvdXRcblwKICAgb2ZsYWc9RkxBR1MgICAgIHdyaXRlIGFzIHBlciB0
aGUgY29tbWEgc2VwYXJhdGVkIHN5bWJvbCBsaXN0XG5cCi0gIHNlZWs9QkxPQ0tTICAgICBza2lw
IEJMT0NLUyBvYnMtc2l6ZWQgYmxvY2tzIGF0IHN0YXJ0IG9mIG91dHB1dFxuXAotICBza2lwPUJM
T0NLUyAgICAgc2tpcCBCTE9DS1MgaWJzLXNpemVkIGJsb2NrcyBhdCBzdGFydCBvZiBpbnB1dFxu
XAorICBzZWVrPU4gICAgICAgICAgc2tpcCBOIG9icy1zaXplZCBibG9ja3MgYXQgc3RhcnQgb2Yg
b3V0cHV0XG5cCisgIHNraXA9TiAgICAgICAgICBza2lwIE4gaWJzLXNpemVkIGJsb2NrcyBhdCBz
dGFydCBvZiBpbnB1dFxuXAogICBzdGF0dXM9bm94ZmVyICAgc3VwcHJlc3MgdHJhbnNmZXIgc3Rh
dGlzdGljc1xuXAogIiksIHN0ZG91dCk7CiAgICAgICBmcHV0cyAoXygiXApAQCAtNTY4LDYgKzU5
NywxNSBAQCBFYWNoIEZMQUcgc3ltYm9sIG1heSBiZTpcblwKICAgICAgICAgZnB1dHMgKF8oIiAg
YmluYXJ5ICAgIHVzZSBiaW5hcnkgSS9PIGZvciBkYXRhXG4iKSwgc3Rkb3V0KTsKICAgICAgIGlm
IChPX1RFWFQpCiAgICAgICAgIGZwdXRzIChfKCIgIHRleHQgICAgICB1c2UgdGV4dCBJL08gZm9y
IGRhdGFcbiIpLCBzdGRvdXQpOworICAgICAgaWYgKE9fQ09VTlRfQllURVMpCisgICAgICAgIGZw
dXRzIChfKCIgIGNvdW50X2J5dGVzICB1c2UgYnl0ZSB1bml0IGZvciB0aGUgJ2NvdW50PU4nIG9w
ZXJhbmRcCisgKGlmbGFnIG9ubHkpXG4iKSwgc3Rkb3V0KTsKKyAgICAgIGlmIChPX1NLSVBfQllU
RVMpCisgICAgICAgIGZwdXRzIChfKCIgIHNraXBfYnl0ZXMgIHVzZSBieXRlIHVuaXQgZm9yIHRo
ZSAnc2tpcD1OJyBvcGVyYW5kXAorIChpZmxhZyBvbmx5KVxuIiksIHN0ZG91dCk7CisgICAgICBp
ZiAoT19TRUVLX0JZVEVTKQorICAgICAgICBmcHV0cyAoXygiICBzZWVrX2J5dGVzICB1c2UgYnl0
ZSB1bml0IGZvciB0aGUgJ3NlZWs9Ticgb3BlcmFuZFwKKyAob2ZsYWcgb25seSlcbiIpLCBzdGRv
dXQpOwogCiAgICAgICB7CiAgICAgICAgIGNoYXIgY29uc3QgKnNpZ2luZm9fbmFtZSA9IChTSUdJ
TkZPID09IFNJR1VTUjEgPyAiVVNSMSIgOiAiSU5GTyIpOwpAQCAtMTEyMCw2ICsxMTU4LDkgQEAg
c2NhbmFyZ3MgKGludCBhcmdjLCBjaGFyICpjb25zdCAqYXJndikKIHsKICAgaW50IGk7CiAgIHNp
emVfdCBibG9ja3NpemUgPSAwOworICB1aW50bWF4X3QgY291bnQgPSAodWludG1heF90KSAtMTsK
KyAgdWludG1heF90IHNraXAgPSAwOworICB1aW50bWF4X3Qgc2VlayA9IDA7CiAKICAgZm9yIChp
ID0gb3B0aW5kOyBpIDwgYXJnYzsgaSsrKQogICAgIHsKQEAgLTExNzUsMTEgKzEyMTYsMTEgQEAg
c2NhbmFyZ3MgKGludCBhcmdjLCBjaGFyICpjb25zdCAqYXJndikKICAgICAgICAgICAgICAgY29u
dmVyc2lvbl9ibG9ja3NpemUgPSBuOwogICAgICAgICAgICAgfQogICAgICAgICAgIGVsc2UgaWYg
KG9wZXJhbmRfaXMgKG5hbWUsICJza2lwIikpCi0gICAgICAgICAgICBza2lwX3JlY29yZHMgPSBu
OworICAgICAgICAgICAgc2tpcCA9IG47CiAgICAgICAgICAgZWxzZSBpZiAob3BlcmFuZF9pcyAo
bmFtZSwgInNlZWsiKSkKLSAgICAgICAgICAgIHNlZWtfcmVjb3JkcyA9IG47CisgICAgICAgICAg
ICBzZWVrID0gbjsKICAgICAgICAgICBlbHNlIGlmIChvcGVyYW5kX2lzIChuYW1lLCAiY291bnQi
KSkKLSAgICAgICAgICAgIG1heF9yZWNvcmRzID0gbjsKKyAgICAgICAgICAgIGNvdW50ID0gbjsK
ICAgICAgICAgICBlbHNlCiAgICAgICAgICAgICB7CiAgICAgICAgICAgICAgIGVycm9yICgwLCAw
LCBfKCJ1bnJlY29nbml6ZWQgb3BlcmFuZCAlcyIpLCBxdW90ZSAobmFtZSkpOwpAQCAtMTIxNiw2
ICsxMjU3LDQzIEBAIHNjYW5hcmdzIChpbnQgYXJnYywgY2hhciAqY29uc3QgKmFyZ3YpCiAgICAg
ICB1c2FnZSAoRVhJVF9GQUlMVVJFKTsKICAgICB9CiAKKyAgaWYgKGlucHV0X2ZsYWdzICYgT19T
RUVLX0JZVEVTKQorICAgIHsKKyAgICAgIGVycm9yICgwLCAwLCAiJXM6ICVzIiwgXygiaW52YWxp
ZCBpbnB1dCBmbGFnIiksICInc2Vla19ieXRlcyciKTsKKyAgICAgIHVzYWdlIChFWElUX0ZBSUxV
UkUpOworICAgIH0KKworICBpZiAob3V0cHV0X2ZsYWdzICYgKE9fQ09VTlRfQllURVMgfCBPX1NL
SVBfQllURVMpKQorICAgIHsKKyAgICAgIGVycm9yICgwLCAwLCAiJXM6ICVzIiwgXygiaW52YWxp
ZCBvdXRwdXQgZmxhZyIpLAorICAgICAgICAgICAgIG91dHB1dF9mbGFncyAmIE9fQ09VTlRfQllU
RVMgPyAiJ2NvdW50X2J5dGVzJyIgOiAiJ3NraXBfYnl0ZXMnIik7CisgICAgICB1c2FnZSAoRVhJ
VF9GQUlMVVJFKTsKKyAgICB9CisKKyAgaWYgKGlucHV0X2ZsYWdzICYgT19TS0lQX0JZVEVTICYm
IHNraXAgIT0gMCkKKyAgICB7CisgICAgICBza2lwX3JlY29yZHMgPSBza2lwIC8gaW5wdXRfYmxv
Y2tzaXplOworICAgICAgc2tpcF9ieXRlcyA9IHNraXAgJSBpbnB1dF9ibG9ja3NpemU7CisgICAg
fQorICBlbHNlIGlmIChza2lwICE9IDApCisgICAgc2tpcF9yZWNvcmRzID0gc2tpcDsKKworICBp
ZiAoaW5wdXRfZmxhZ3MgJiBPX0NPVU5UX0JZVEVTICYmIGNvdW50ICE9ICh1aW50bWF4X3QpIC0x
KQorICAgIHsKKyAgICAgIG1heF9yZWNvcmRzID0gY291bnQgLyBpbnB1dF9ibG9ja3NpemU7Cisg
ICAgICBtYXhfYnl0ZXMgPSBjb3VudCAlIGlucHV0X2Jsb2Nrc2l6ZTsKKyAgICB9CisgIGVsc2Ug
aWYgKGNvdW50ICE9ICh1aW50bWF4X3QpIC0xKQorICAgIG1heF9yZWNvcmRzID0gY291bnQ7CisK
KyAgaWYgKG91dHB1dF9mbGFncyAmIE9fU0VFS19CWVRFUyAmJiBzZWVrICE9IDApCisgICAgewor
ICAgICAgc2Vla19yZWNvcmRzID0gc2VlayAvIG91dHB1dF9ibG9ja3NpemU7CisgICAgICBzZWVr
X2J5dGVzID0gc2VlayAlIG91dHB1dF9ibG9ja3NpemU7CisgICAgfQorICBlbHNlIGlmIChzZWVr
ICE9IDApCisgICAgc2Vla19yZWNvcmRzID0gc2VlazsKKwogICAvKiBXYXJuIGFib3V0IHBhcnRp
YWwgcmVhZHMgaWYgYnM9U0laRSBpcyBnaXZlbiBhbmQgaWZsYWc9ZnVsbGJsb2NrCiAgICAgIGlz
IG5vdCwgYW5kIGlmIGNvdW50aW5nIG9yIHNraXBwaW5nIGJ5dGVzIG9yIHVzaW5nIGRpcmVjdCBJ
L08uCiAgICAgIFRoaXMgaGVscHMgdG8gYXZvaWQgY29uZnVzaW9uIHdpdGggbWlzY291bnRzLCBh
bmQgdG8gYXZvaWQgaXNzdWVzCkBAIC0xNDExLDE4ICsxNDg5LDIwIEBAIHNraXBfdmlhX2xzZWVr
IChjaGFyIGNvbnN0ICpmaWxlbmFtZSwgaW50IGZkZXNjLCBvZmZfdCBvZmZzZXQsIGludCB3aGVu
Y2UpCiAjIGRlZmluZSBza2lwX3ZpYV9sc2VlayhGaWxlbmFtZSwgRmQsIE9mZnNldCwgV2hlbmNl
KSBsc2VlayAoRmQsIE9mZnNldCwgV2hlbmNlKQogI2VuZGlmCiAKLS8qIFRocm93IGF3YXkgUkVD
T1JEUyBibG9ja3Mgb2YgQkxPQ0tTSVpFIGJ5dGVzIG9uIGZpbGUgZGVzY3JpcHRvciBGREVTQywK
LSAgIHdoaWNoIGlzIG9wZW4gd2l0aCByZWFkIHBlcm1pc3Npb24gZm9yIEZJTEUuICBTdG9yZSB1
cCB0byBCTE9DS1NJWkUKLSAgIGJ5dGVzIG9mIHRoZSBkYXRhIGF0IGEgdGltZSBpbiBCVUYsIGlm
IG5lY2Vzc2FyeS4gIFJFQ09SRFMgbXVzdCBiZQotICAgbm9uemVyby4gIElmIGZkZXNjIGlzIFNU
RElOX0ZJTEVOTywgYWR2YW5jZSB0aGUgaW5wdXQgb2Zmc2V0LgotICAgUmV0dXJuIHRoZSBudW1i
ZXIgb2YgcmVjb3JkcyByZW1haW5pbmcsIGkuZS4sIHRoYXQgd2VyZSBub3Qgc2tpcHBlZAotICAg
YmVjYXVzZSBFT0Ygd2FzIHJlYWNoZWQuICAqLworLyogVGhyb3cgYXdheSBSRUNPUkRTIGJsb2Nr
cyBvZiBCTE9DS1NJWkUgYnl0ZXMgcGx1cyBCWVRFUyBieXRlcyBvbgorICAgZmlsZSBkZXNjcmlw
dG9yIEZERVNDLCB3aGljaCBpcyBvcGVuIHdpdGggcmVhZCBwZXJtaXNzaW9uIGZvciBGSUxFLgor
ICAgU3RvcmUgdXAgdG8gQkxPQ0tTSVpFIGJ5dGVzIG9mIHRoZSBkYXRhIGF0IGEgdGltZSBpbiBC
VUYsIGlmCisgICBuZWNlc3NhcnkuIFJFQ09SRFMgb3IgQllURVMgbXVzdCBiZSBub256ZXJvLiBJ
ZiBGREVTQyBpcworICAgU1RESU5fRklMRU5PLCBhZHZhbmNlIHRoZSBpbnB1dCBvZmZzZXQuIFJl
dHVybiB0aGUgbnVtYmVyIG9mCisgICByZWNvcmRzIHJlbWFpbmluZywgaS5lLiwgdGhhdCB3ZXJl
IG5vdCBza2lwcGVkIGJlY2F1c2UgRU9GIHdhcworICAgcmVhY2hlZC4gIElmIEZERVNDIGlzIFNU
RE9VVF9GSUxFTk8sIG9uIHJldHVybiwgQllURVMgaXMgdGhlCisgICByZW1haW5pbmcgYnl0ZXMg
aW4gYWRkaXRpb24gdG8gdGhlIHJlbWFpbmluZyByZWNvcmRzLiAgKi8KIAogc3RhdGljIHVpbnRt
YXhfdAogc2tpcCAoaW50IGZkZXNjLCBjaGFyIGNvbnN0ICpmaWxlLCB1aW50bWF4X3QgcmVjb3Jk
cywgc2l6ZV90IGJsb2Nrc2l6ZSwKLSAgICAgIGNoYXIgKmJ1ZikKKyAgICAgIHNpemVfdCAqYnl0
ZXMsIGNoYXIgKmJ1ZikKIHsKLSAgdWludG1heF90IG9mZnNldCA9IHJlY29yZHMgKiBibG9ja3Np
emU7CisgIHVpbnRtYXhfdCBvZmZzZXQgPSByZWNvcmRzICogYmxvY2tzaXplICsgKmJ5dGVzOwog
CiAgIC8qIFRyeSBsc2VlayBhbmQgaWYgYW4gZXJyb3IgaW5kaWNhdGVzIGl0IHdhcyBhbiBpbmFw
cHJvcHJpYXRlIG9wZXJhdGlvbiAtLQogICAgICBvciBpZiB0aGUgZmlsZSBvZmZzZXQgaXMgbm90
IHJlcHJlc2VudGFibGUgYXMgYW4gb2ZmX3QgLS0KQEAgLTE0NTAsNyArMTUzMCwxMCBAQCBza2lw
IChpbnQgZmRlc2MsIGNoYXIgY29uc3QgKmZpbGUsIHVpbnRtYXhfdCByZWNvcmRzLCBzaXplX3Qg
YmxvY2tzaXplLAogICAgICAgICAgICBhZHZhbmNlX2lucHV0X29mZnNldCAob2Zmc2V0KTsKICAg
ICAgICAgfQogICAgICAgZWxzZQotICAgICAgICByZWNvcmRzID0gMDsKKyAgICAgICAgeworICAg
ICAgICAgIHJlY29yZHMgPSAwOworICAgICAgICAgICpieXRlcyA9IDA7CisgICAgICAgIH0KICAg
ICAgIHJldHVybiByZWNvcmRzOwogICAgIH0KICAgZWxzZQpAQCAtMTQ5MSwyOSArMTU3NCwzMCBA
QCBza2lwIChpbnQgZmRlc2MsIGNoYXIgY29uc3QgKmZpbGUsIHVpbnRtYXhfdCByZWNvcmRzLCBz
aXplX3QgYmxvY2tzaXplLAogCiAgICAgICBkbwogICAgICAgICB7Ci0gICAgICAgICAgc3NpemVf
dCBucmVhZCA9IGlyZWFkX2ZuYyAoZmRlc2MsIGJ1ZiwgYmxvY2tzaXplKTsKKyAgICAgICAgICBz
c2l6ZV90IG5yZWFkID0gaXJlYWRfZm5jIChmZGVzYywgYnVmLCByZWNvcmRzID8gYmxvY2tzaXpl
IDogKmJ5dGVzKTsKICAgICAgICAgICBpZiAobnJlYWQgPCAwKQogICAgICAgICAgICAgewogICAg
ICAgICAgICAgICBpZiAoZmRlc2MgPT0gU1RESU5fRklMRU5PKQogICAgICAgICAgICAgICAgIHsK
ICAgICAgICAgICAgICAgICAgIGVycm9yICgwLCBlcnJubywgXygicmVhZGluZyAlcyIpLCBxdW90
ZSAoZmlsZSkpOwogICAgICAgICAgICAgICAgICAgaWYgKGNvbnZlcnNpb25zX21hc2sgJiBDX05P
RVJST1IpCi0gICAgICAgICAgICAgICAgICAgIHsKLSAgICAgICAgICAgICAgICAgICAgICBwcmlu
dF9zdGF0cyAoKTsKLSAgICAgICAgICAgICAgICAgICAgICBjb250aW51ZTsKLSAgICAgICAgICAg
ICAgICAgICAgfQorICAgICAgICAgICAgICAgICAgICBwcmludF9zdGF0cyAoKTsKICAgICAgICAg
ICAgICAgICB9CiAgICAgICAgICAgICAgIGVsc2UKICAgICAgICAgICAgICAgICBlcnJvciAoMCwg
bHNlZWtfZXJybm8sIF8oIiVzOiBjYW5ub3Qgc2VlayIpLCBxdW90ZSAoZmlsZSkpOwogICAgICAg
ICAgICAgICBxdWl0IChFWElUX0ZBSUxVUkUpOwogICAgICAgICAgICAgfQotCi0gICAgICAgICAg
aWYgKG5yZWFkID09IDApCisgICAgICAgICAgZWxzZSBpZiAobnJlYWQgPT0gMCkKICAgICAgICAg
ICAgIGJyZWFrOwotICAgICAgICAgIGlmIChmZGVzYyA9PSBTVERJTl9GSUxFTk8pCisgICAgICAg
ICAgZWxzZSBpZiAoZmRlc2MgPT0gU1RESU5fRklMRU5PKQogICAgICAgICAgICAgYWR2YW5jZV9p
bnB1dF9vZmZzZXQgKG5yZWFkKTsKKworICAgICAgICAgIGlmIChyZWNvcmRzICE9IDApCisgICAg
ICAgICAgICByZWNvcmRzLS07CisgICAgICAgICAgZWxzZQorICAgICAgICAgICAgKmJ5dGVzID0g
MDsKICAgICAgICAgfQotICAgICAgd2hpbGUgKC0tcmVjb3JkcyAhPSAwKTsKKyAgICAgIHdoaWxl
IChyZWNvcmRzIHx8ICpieXRlcyk7CiAKICAgICAgIHJldHVybiByZWNvcmRzOwogICAgIH0KQEAg
LTE3NzcsMTEgKzE4NjEsMTEgQEAgZGRfY29weSAodm9pZCkKICAgICAgIG9idWYgPSBpYnVmOwog
ICAgIH0KIAotICBpZiAoc2tpcF9yZWNvcmRzICE9IDApCisgIGlmIChza2lwX3JlY29yZHMgIT0g
MCB8fCBza2lwX2J5dGVzICE9IDApCiAgICAgewotICAgICAgdWludG1heF90IHVzX2J5dGVzID0g
aW5wdXRfb2Zmc2V0ICsgKHNraXBfcmVjb3JkcyAqIGlucHV0X2Jsb2Nrc2l6ZSk7CisgICAgICB1
aW50bWF4X3QgdXNfYnl0ZXMgPSBpbnB1dF9vZmZzZXQgKyAoc2tpcF9yZWNvcmRzICogaW5wdXRf
YmxvY2tzaXplKSArIHNraXBfYnl0ZXM7CiAgICAgICB1aW50bWF4X3QgdXNfYmxvY2tzID0gc2tp
cCAoU1RESU5fRklMRU5PLCBpbnB1dF9maWxlLAotICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIHNraXBfcmVjb3JkcywgaW5wdXRfYmxvY2tzaXplLCBpYnVmKTsKKyAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICBza2lwX3JlY29yZHMsIGlucHV0X2Jsb2Nrc2l6ZSwgJnNr
aXBfYnl0ZXMsIGlidWYpOwogICAgICAgdXNfYnl0ZXMgLT0gaW5wdXRfb2Zmc2V0OwogCiAgICAg
ICAvKiBQT1NJWCBkb2Vzbid0IHNheSB3aGF0IHRvIGRvIHdoZW4gZGQgZGV0ZWN0cyBpdCBoYXMg
YmVlbgpAQCAtMTc5NywzNCArMTg4MSw0MCBAQCBkZF9jb3B5ICh2b2lkKQogICAgICAgICB9CiAg
ICAgfQogCi0gIGlmIChzZWVrX3JlY29yZHMgIT0gMCkKKyAgaWYgKHNlZWtfcmVjb3JkcyAhPSAw
IHx8IHNlZWtfYnl0ZXMgIT0gMCkKICAgICB7CisgICAgICBzaXplX3QgYnl0ZXMgPSBzZWVrX2J5
dGVzOwogICAgICAgdWludG1heF90IHdyaXRlX3JlY29yZHMgPSBza2lwIChTVERPVVRfRklMRU5P
LCBvdXRwdXRfZmlsZSwKLSAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgc2Vl
a19yZWNvcmRzLCBvdXRwdXRfYmxvY2tzaXplLCBvYnVmKTsKKyAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgc2Vla19yZWNvcmRzLCBvdXRwdXRfYmxvY2tzaXplLCAmYnl0ZXMs
IG9idWYpOwogCi0gICAgICBpZiAod3JpdGVfcmVjb3JkcyAhPSAwKQorICAgICAgaWYgKHdyaXRl
X3JlY29yZHMgIT0gMCB8fCBieXRlcyAhPSAwKQogICAgICAgICB7Ci0gICAgICAgICAgbWVtc2V0
IChvYnVmLCAwLCBvdXRwdXRfYmxvY2tzaXplKTsKKyAgICAgICAgICBtZW1zZXQgKG9idWYsIDAs
IHdyaXRlX3JlY29yZHMgPyBvdXRwdXRfYmxvY2tzaXplIDogYnl0ZXMpOwogCiAgICAgICAgICAg
ZG8KICAgICAgICAgICAgIHsKLSAgICAgICAgICAgICAgaWYgKGl3cml0ZSAoU1RET1VUX0ZJTEVO
Tywgb2J1Ziwgb3V0cHV0X2Jsb2Nrc2l6ZSkKLSAgICAgICAgICAgICAgICAgICE9IG91dHB1dF9i
bG9ja3NpemUpCisgICAgICAgICAgICAgIHNpemVfdCBzaXplID0gd3JpdGVfcmVjb3JkcyA/IG91
dHB1dF9ibG9ja3NpemUgOiBieXRlczsKKyAgICAgICAgICAgICAgaWYgKGl3cml0ZSAoU1RET1VU
X0ZJTEVOTywgb2J1Ziwgc2l6ZSkgIT0gc2l6ZSkKICAgICAgICAgICAgICAgICB7CiAgICAgICAg
ICAgICAgICAgICBlcnJvciAoMCwgZXJybm8sIF8oIndyaXRpbmcgdG8gJXMiKSwgcXVvdGUgKG91
dHB1dF9maWxlKSk7CiAgICAgICAgICAgICAgICAgICBxdWl0IChFWElUX0ZBSUxVUkUpOwogICAg
ICAgICAgICAgICAgIH0KKworICAgICAgICAgICAgICBpZiAod3JpdGVfcmVjb3JkcyAhPSAwKQor
ICAgICAgICAgICAgICAgIHdyaXRlX3JlY29yZHMtLTsKKyAgICAgICAgICAgICAgZWxzZQorICAg
ICAgICAgICAgICAgIGJ5dGVzID0gMDsKICAgICAgICAgICAgIH0KLSAgICAgICAgICB3aGlsZSAo
LS13cml0ZV9yZWNvcmRzICE9IDApOworICAgICAgICAgIHdoaWxlICh3cml0ZV9yZWNvcmRzIHx8
IGJ5dGVzKTsKICAgICAgICAgfQogICAgIH0KIAotICBpZiAobWF4X3JlY29yZHMgPT0gMCkKKyAg
aWYgKG1heF9yZWNvcmRzID09IDAgJiYgbWF4X2J5dGVzID09IDApCiAgICAgcmV0dXJuIGV4aXRf
c3RhdHVzOwogCiAgIHdoaWxlICgxKQogICAgIHsKLSAgICAgIGlmIChyX3BhcnRpYWwgKyByX2Z1
bGwgPj0gbWF4X3JlY29yZHMpCisgICAgICBpZiAocl9wYXJ0aWFsICsgcl9mdWxsID49IG1heF9y
ZWNvcmRzICsgKG1heF9ieXRlcyA/IDEgOiAwKSkKICAgICAgICAgYnJlYWs7CiAKICAgICAgIC8q
IFplcm8gdGhlIGJ1ZmZlciBiZWZvcmUgcmVhZGluZywgc28gdGhhdCBpZiB3ZSBnZXQgYSByZWFk
IGVycm9yLApAQCAtMTgzNSw3ICsxOTI1LDEwIEBAIGRkX2NvcHkgKHZvaWQpCiAgICAgICAgICAg
ICAgICAgKGNvbnZlcnNpb25zX21hc2sgJiAoQ19CTE9DSyB8IENfVU5CTE9DSykpID8gJyAnIDog
J1wwJywKICAgICAgICAgICAgICAgICBpbnB1dF9ibG9ja3NpemUpOwogCi0gICAgICBucmVhZCA9
IGlyZWFkX2ZuYyAoU1RESU5fRklMRU5PLCBpYnVmLCBpbnB1dF9ibG9ja3NpemUpOworICAgICAg
aWYgKHJfcGFydGlhbCArIHJfZnVsbCA+PSBtYXhfcmVjb3JkcykKKyAgICAgICAgbnJlYWQgPSBp
cmVhZF9mbmMgKFNURElOX0ZJTEVOTywgaWJ1ZiwgbWF4X2J5dGVzKTsKKyAgICAgIGVsc2UKKyAg
ICAgICAgbnJlYWQgPSBpcmVhZF9mbmMgKFNURElOX0ZJTEVOTywgaWJ1ZiwgaW5wdXRfYmxvY2tz
aXplKTsKIAogICAgICAgaWYgKG5yZWFkID49IDAgJiYgaV9ub2NhY2hlKQogICAgICAgICBpbnZh
bGlkYXRlX2NhY2hlIChTVERJTl9GSUxFTk8sIG5yZWFkKTsKZGlmZiAtLWdpdCBhL3Rlc3RzL01h
a2VmaWxlLmFtIGIvdGVzdHMvTWFrZWZpbGUuYW0KaW5kZXggYTk0YWFhMi4uMzFiYjA1MCAxMDA2
NDQKLS0tIGEvdGVzdHMvTWFrZWZpbGUuYW0KKysrIGIvdGVzdHMvTWFrZWZpbGUuYW0KQEAgLTM2
OSw2ICszNjksNyBAQCBURVNUUyA9CQkJCQkJXAogICBkZC9yZWJsb2NrCQkJCQlcCiAgIGRkL3Nr
aXAtc2VlawkJCQkJXAogICBkZC9za2lwLXNlZWsyCQkJCQlcCisgIGRkL2J5dGVzCQkJCQlcCiAg
IGRkL3NraXAtc2Vlay1wYXN0LWZpbGUJCQlcCiAgIGRkL3N0ZGVycgkJCQkJXAogICBkZC91bmJs
b2NrCQkJCQlcCmRpZmYgLS1naXQgYS90ZXN0cy9kZC9ieXRlcyBiL3Rlc3RzL2RkL2J5dGVzCm5l
dyBmaWxlIG1vZGUgMTAwNzU1CmluZGV4IDAwMDAwMDAuLjYwMzg3NDIKLS0tIC9kZXYvbnVsbAor
KysgYi90ZXN0cy9kZC9ieXRlcwpAQCAtMCwwICsxLDU3IEBACisjIS9iaW4vc2gKKworIyBDb3B5
cmlnaHQgKEMpIDIwMTIgRnJlZSBTb2Z0d2FyZSBGb3VuZGF0aW9uLCBJbmMuCisKKyMgVGhpcyBw
cm9ncmFtIGlzIGZyZWUgc29mdHdhcmU6IHlvdSBjYW4gcmVkaXN0cmlidXRlIGl0IGFuZC9vciBt
b2RpZnkKKyMgaXQgdW5kZXIgdGhlIHRlcm1zIG9mIHRoZSBHTlUgR2VuZXJhbCBQdWJsaWMgTGlj
ZW5zZSBhcyBwdWJsaXNoZWQgYnkKKyMgdGhlIEZyZWUgU29mdHdhcmUgRm91bmRhdGlvbiwgZWl0
aGVyIHZlcnNpb24gMyBvZiB0aGUgTGljZW5zZSwgb3IKKyMgKGF0IHlvdXIgb3B0aW9uKSBhbnkg
bGF0ZXIgdmVyc2lvbi4KKworIyBUaGlzIHByb2dyYW0gaXMgZGlzdHJpYnV0ZWQgaW4gdGhlIGhv
cGUgdGhhdCBpdCB3aWxsIGJlIHVzZWZ1bCwKKyMgYnV0IFdJVEhPVVQgQU5ZIFdBUlJBTlRZOyB3
aXRob3V0IGV2ZW4gdGhlIGltcGxpZWQgd2FycmFudHkgb2YKKyMgTUVSQ0hBTlRBQklMSVRZIG9y
IEZJVE5FU1MgRk9SIEEgUEFSVElDVUxBUiBQVVJQT1NFLiAgU2VlIHRoZQorIyBHTlUgR2VuZXJh
bCBQdWJsaWMgTGljZW5zZSBmb3IgbW9yZSBkZXRhaWxzLgorCisjIFlvdSBzaG91bGQgaGF2ZSBy
ZWNlaXZlZCBhIGNvcHkgb2YgdGhlIEdOVSBHZW5lcmFsIFB1YmxpYyBMaWNlbnNlCisjIGFsb25n
IHdpdGggdGhpcyBwcm9ncmFtLiAgSWYgbm90LCBzZWUgPGh0dHA6Ly93d3cuZ251Lm9yZy9saWNl
bnNlcy8+LgorCisuICIke3NyY2Rpcj0ufS9pbml0LnNoIjsgcGF0aF9wcmVwZW5kXyAuLi9zcmMK
K3ByaW50X3Zlcl8gZGQKKworIyBjb3VudF9ieXRlcworZWNobyAwMTIzNDU2Nzg5YWJjZGVmZ2hp
amtsbSA+IGluIHx8IGZyYW1ld29ya19mYWlsdXJlXworZGQgY291bnQ9MTQgY29udj1zd2FiIGlm
bGFnPWNvdW50X2J5dGVzIDwgaW4gPiBvdXQgMj4gL2Rldi9udWxsIHx8IGZhaWw9MQorY2FzZSBg
Y2F0IG91dGAgaW4KKyAxMDMyNTQ3Njk4YmFkYykgOzsKKyAqKSBmYWlsPTEgOzsKK2VzYWMKKwor
IyBza2lwX2J5dGVzCitlY2hvIDAxMjM0NTY3ODlhYmNkZWZnaGlqa2xtID4gaW4gfHwgZnJhbWV3
b3JrX2ZhaWx1cmVfCitkZCBza2lwPTEwIGlmbGFnPXNraXBfYnl0ZXMgPCBpbiA+IG91dCAyPiAv
ZGV2L251bGwgfHwgZmFpbD0xCitjYXNlIGBjYXQgb3V0YCBpbgorIGFiY2RlZmdoaWprbG0pIDs7
CisgKikgZmFpbD0xIDs7Citlc2FjCisKKyMgc2tpcCByZWNvcmRzIGFuZCBieXRlcyBmcm9tIHBp
cGUKK2VjaG8gMDEyMzQ1Njc4OWFiY2RlZmdoaWprbG0gfAorIGRkIHNraXA9MTAgYnM9MiBpZmxh
Zz1za2lwX2J5dGVzID4gb3V0IDI+IC9kZXYvbnVsbCB8fCBmYWlsPTEKK2Nhc2UgYGNhdCBvdXRg
IGluCisgYWJjZGVmZ2hpamtsbSkgOzsKKyAqKSBmYWlsPTEgOzsKK2VzYWMKKworIyBzZWVrIGJ5
dGVzCitlY2hvIGFiY2RlZmdoaWprbG0gfAorIGRkIGJzPTUgc2Vlaz04IG9mbGFnPXNlZWtfYnl0
ZXMgPiBvdXQgMj4gL2Rldi9udWxsIHx8IGZhaWw9MQorZWNobyBhYmNkZWZnaGlqa2xtIHwKKyBk
ZCBicz00IHNlZWs9MiA+IGV4cGVjdGVkIDI+IC9kZXYvbnVsbCB8fCBmYWlsPTEKK2NvbXBhcmUg
ZXhwZWN0ZWQgb3V0IHx8IGZhaWw9MQorCisjIHNlZWsgYnl0ZXMgb24gZW1wdHkgZmlsZQorZWNo
byBhYmNkZWZnaGlqa2xtIHwKKyBkZCBicz01IHNlZWs9OCBvZmxhZz1zZWVrX2J5dGVzID4gb3V0
MiAyPiAvZGV2L251bGwgfHwgZmFpbD0xCitjb21wYXJlIGV4cGVjdGVkIG91dDIgfHwgZmFpbD0x
CisKK0V4aXQgJGZhaWwKLS0gCjEuNy4yLjUKCg==
--e89a8f234cd30b2a7404b89e85cf--


From MAILER-DAEMON Fri Feb 10 11:58:06 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RvtnS-000658-IT
	for mharc-coreutils@gnu.org; Fri, 10 Feb 2012 11:58:06 -0500
Received: from eggs.gnu.org ([140.186.70.92]:59603)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RvtnM-0005zJ-UI
	for coreutils@gnu.org; Fri, 10 Feb 2012 11:58:05 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RvtnL-0000dr-I9
	for coreutils@gnu.org; Fri, 10 Feb 2012 11:58:00 -0500
Received: from mail1.vodafone.ie ([213.233.128.43]:25557)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RvtnL-0000dS-Da
	for coreutils@gnu.org; Fri, 10 Feb 2012 11:57:59 -0500
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: ApsBAIJLNU9tTfGC/2dsb2JhbAAMOK0khTEBAQEEDG0QCw0LCRYPCQMCAQIBRQYNAQcBAcFLi1IXUgENg36EFwSOSIEbi0mDOIkw
Received: from unknown (HELO [192.168.1.79]) ([109.77.241.130])
	by mail1.vodafone.ie with ESMTP; 10 Feb 2012 16:57:56 +0000
Message-ID: <4F354C8E.9040203@draigBrady.com>
Date: Fri, 10 Feb 2012 16:57:50 +0000
From: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:6.0) Gecko/20110816 Thunderbird/6.0
MIME-Version: 1.0
To: =?ISO-8859-1?Q?J=E9r=E9my_Compostella?=
 <jeremy.compostella@gmail.com>
Subject: Re: dd: add 'skip_bytes' and 'count_bytes' operands
References: <87mx8w3ryu.fsf@Apollo.jerryland.fr>
	<4F2FF5EE.9010906@draigBrady.com>
	<CA+TATJF4w4Ue44Za4hgGR_0RxgEZGBm5nzhHRSdGYxzzfKWNLg@mail.gmail.com>
	<4F317CDF.2040909@draigBrady.com>
	<CA+TATJFU7yOGzzeZseHUFTqxxGAtz8zUoQccz=5m0Eu=CqEtRQ@mail.gmail.com>
	<4F329FC9.4030403@draigBrady.com>
	<CA+TATJFkn8=Kd3iqrtk=z2pkkzKRGwnAsquw61r0kqSMx76zgQ@mail.gmail.com>
	<CA+TATJF2uW6eHqF4fkwacHq4xm1XPY=3Ent3RHDNPuMA8GOZVA@mail.gmail.com>
In-Reply-To: <CA+TATJF2uW6eHqF4fkwacHq4xm1XPY=3Ent3RHDNPuMA8GOZVA@mail.gmail.com>
X-Enigmail-Version: 1.3.2
Content-Type: multipart/mixed; boundary="------------060805010306010505060404"
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 213.233.128.43
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 10 Feb 2012 16:58:05 -0000

This is a multi-part message in MIME format.
--------------060805010306010505060404
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit

On 02/10/2012 04:21 PM, Jérémy Compostella wrote:
> Pádraig, others,
> 
> First, I added the missing seek_bytes feature.
> Second, I re-wrote the patch in order to replace the _bytes operands by the
> corresponding input/output flags.
> Finally, I added some tests to check the new seek_bytes flag.
> 
> I successfully run the entire tests suite.

Excellent.
I'll review fully later.

> * doc/coreutils.texi (New features): Detail new flags and behaviors.

s/New features/dd invocation/

I've also attached a diff of some adjustments to --amend to the texinfo.

cheers,
Pádraig.

--------------060805010306010505060404
Content-Type: text/plain;
 name="dd_bytes_desc.diff"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename="dd_bytes_desc.diff"

diff --git a/doc/coreutils.texi b/doc/coreutils.texi
index 6efc5cc..02c3a2a 100644
--- a/doc/coreutils.texi
+++ b/doc/coreutils.texi
@@ -8060,15 +8060,21 @@ use @var{bytes} as the fixed record length.
 @item skip=@var{n}
 @opindex skip
 Skip @var{n} @samp{ibs}-byte blocks in the input file before copying.
+If @samp{iflag=skip_bytes} is specified, @var{n} is interpreted
+as a byte count rather than a block count.
 
 @item seek=@var{n}
 @opindex seek
 Skip @var{n} @samp{obs}-byte blocks in the output file before copying.
+if @samp{oflag=seek_bytes} is specified, @var{n} is interpreted
+as a byte count rather than a block count.
 
 @item count=@var{n}
 @opindex count
 Copy @var{n} @samp{ibs}-byte blocks from the input file, instead
 of everything until the end of the file.
+if @samp{iflag=count_bytes} is specified, @var{n} is interpreted
+as a byte count rather than a block count.
 
 @item status=noxfer
 @opindex status
@@ -8323,20 +8329,23 @@ This flag can be used only with @code{iflag}.
 
 @item count_bytes
 @opindex count_bytes
-Use byte unit instead of @samp{ibs}-byte block unit for the
-@samp{count=n} operand.
+Interpret the @samp{count=} operand as a byte count,
+rather than a block count, which allows specifying
+a length that is not a multiple of the I/O block size.
 This flag can be used only with @code{iflag}.
 
 @item skip_bytes
 @opindex skip_bytes
-Use byte unit instead of @samp{ibs}-byte block unit for the
-@samp{skip=n} operand.
+Interpret the @samp{skip=} operand as a byte count,
+rather than a block count, which allows specifying
+an offset that is not a multiple of the I/O block size.
 This flag can be used only with @code{iflag}.
 
 @item seek_bytes
 @opindex seek_bytes
-Use byte unit instead of @samp{obs}-byte block unit for the
-@samp{seek=n} operand.
+Interpret the @samp{seek=} operand as a byte count,
+rather than a block count, which allows specifying
+an offset that is not a multiple of the I/O block size.
 This flag can be used only with @code{oflag}.
 
 @end table
@@ -8361,13 +8370,12 @@ should not be too large---values larger than a few megabytes
 are generally wasteful or (as in the gigabyte..exabyte case) downright
 counterproductive or error-inducing.
 
-Use different @command{dd} invocations to use different block sizes
-for skipping and I/O@. To process data that is at an offset or size
-that is not a multiple of the I/O@ block size, you can use the
-count_bytes and skip_bytes input flag and the seek_bytes output flag.
-Alternatively the traditional method of separate @command{dd}
-invocations can be used.  For example, the following shell commands
-copy data in 512 KiB blocks between a disk and a tape, but do not save
+To process data that is at an offset or size that is not a
+multiple of the I/O@ block size, you can use the @samp{skip_bytes},
+@samp{seek_bytes} and @samp{count_bytes} flags.  Alternatively
+the traditional method of separate @command{dd} invocations can be used.
+For example, the following shell commands copy data
+in 512 KiB blocks between a disk and a tape, but do not save
 or restore a 4 KiB label at the start of the disk:
 
 @example

--------------060805010306010505060404--


From MAILER-DAEMON Sat Feb 11 06:24:25 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RwB45-0007MS-N7
	for mharc-coreutils@gnu.org; Sat, 11 Feb 2012 06:24:25 -0500
Received: from eggs.gnu.org ([140.186.70.92]:54437)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1RwB42-0007MJ-Rf
	for coreutils@gnu.org; Sat, 11 Feb 2012 06:24:24 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1RwB41-0000eU-Fy
	for coreutils@gnu.org; Sat, 11 Feb 2012 06:24:22 -0500
Received: from mx.meyering.net ([88.168.87.75]:45280)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1RwB41-0000eQ-3P
	for coreutils@gnu.org; Sat, 11 Feb 2012 06:24:21 -0500
Received: from rho.meyering.net (localhost.localdomain [127.0.0.1])
	by rho.meyering.net (Acme Bit-Twister) with ESMTP id 317F060094
	for <coreutils@gnu.org>; Sat, 11 Feb 2012 12:24:20 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: coreutils@gnu.org
Subject: [PATCH] build: update gnulib, bootstrap, gl/lib/regcomp.c.diff
Date: Sat, 11 Feb 2012 12:24:20 +0100
Message-ID: <87zkcpa9zv.fsf@rho.meyering.net>
Lines: 177
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 88.168.87.75
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sat, 11 Feb 2012 11:24:24 -0000

FYI,

>From a8478ac32c7e987b392973d12dc4c97dc31202d7 Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering@redhat.com>
Date: Sat, 11 Feb 2012 10:43:50 +0100
Subject: [PATCH] build: update gnulib, bootstrap, gl/lib/regcomp.c.diff

* gnulib: Update submodule to latest.
* bootstrap: Update from gnulib.
* gl/lib/regcomp.c.diff: Adapt to gnulib changes.
---
 bootstrap             |   30 +++++++++++++++++++++---------
 gl/lib/regcomp.c.diff |   28 ++++++++++++++++------------
 gnulib                |    2 +-
 3 files changed, 38 insertions(+), 22 deletions(-)

diff --git a/bootstrap b/bootstrap
index 53b73d0..c8ee3cc 100755
--- a/bootstrap
+++ b/bootstrap
@@ -1,6 +1,6 @@
 #! /bin/sh
 # Print a version string.
-scriptversion=2011-12-17.15; # UTC
+scriptversion=2012-02-11.09; # UTC

 # Bootstrap this package from checked-out sources.

@@ -87,9 +87,9 @@ gnulib_files=
 : ${AUTOPOINT=autopoint}
 : ${AUTORECONF=autoreconf}

-# A function to be called to edit gnulib.mk right after it's created.
+# A function to be called right after gnulib-tool is run.
 # Override it via your own definition in bootstrap.conf.
-gnulib_mk_hook() { :; }
+bootstrap_post_import_hook() { :; }

 # A function to be called after everything else in this script.
 # Override it via your own definition in bootstrap.conf.
@@ -316,8 +316,8 @@ insert_vc_ignore() {
   pattern="$2"
   case $vc_ignore_file in
   *.gitignore)
-    # A .gitignore entry that does not start with `/' applies
-    # recursively to subdirectories, so prepend `/' to every
+    # A .gitignore entry that does not start with '/' applies
+    # recursively to subdirectories, so prepend '/' to every
     # .gitignore entry.
     pattern=`echo "$pattern" | sed s,^,/,`;;
   esac
@@ -604,7 +604,7 @@ if $bootstrap_sync; then
 fi

 gnulib_tool=$GNULIB_SRCDIR/gnulib-tool
-<$gnulib_tool || exit
+<$gnulib_tool || exit $?

 # Get translations.

@@ -758,7 +758,7 @@ fi
 # Autoreconf runs aclocal before libtoolize, which causes spurious
 # warnings if the initial aclocal is confused by the libtoolized
 # (or worse out-of-date) macro directory.
-if grep '^[    ]*LT_INIT' configure.ac >/dev/null; then
+if test $use_libtool = 1; then
   echo "running: $LIBTOOLIZE --copy --install"
   $LIBTOOLIZE --copy --install
 fi
@@ -807,6 +807,9 @@ for file in $gnulib_files; do
   symlink_to_dir "$GNULIB_SRCDIR" $file || exit
 done

+bootstrap_post_import_hook \
+  || { echo >&2 "$me: bootstrap_post_import_hook failed"; exit 1; }
+
 # Remove any dangling symlink matching "*.m4" or "*.[ch]" in some
 # gnulib-populated directories.  Such .m4 files would cause aclocal to fail.
 # The following requires GNU find 4.2.3 or newer.  Considering the usual
@@ -819,11 +822,20 @@ find "$m4_base" "$source_base" \
   -depth \( -name '*.m4' -o -name '*.[ch]' \) \
   -type l -xtype l -delete > /dev/null 2>&1

+# Some systems (RHEL 5) are using ancient autotools, for which the
+# --no-recursive option had not been invented.  Detect that lack and
+# omit the option when it's not supported.  FIXME in 2017: remove this
+# hack when RHEL 5 autotools are updated, or when they become irrelevant.
+no_recursive=
+case $($AUTORECONF --help) in
+  *--no-recursive*) no_recursive=--no-recursive;;
+esac
+
 # Tell autoreconf not to invoke autopoint or libtoolize; they were run above.
 echo "running: AUTOPOINT=true LIBTOOLIZE=true " \
-    "$AUTORECONF --verbose --install --no-recursive -I $m4_base"
+    "$AUTORECONF --verbose --install $no_recursive -I $m4_base $ACLOCAL_FLAGS"
 AUTOPOINT=true LIBTOOLIZE=true \
-    $AUTORECONF --verbose --install --no-recursive -I $m4_base \
+    $AUTORECONF --verbose --install $no_recursive -I $m4_base $ACLOCAL_FLAGS \
   || exit 1

 # Get some extra files from gnulib, overriding existing files.
diff --git a/gl/lib/regcomp.c.diff b/gl/lib/regcomp.c.diff
index 63fc187..d79c7b2 100644
--- a/gl/lib/regcomp.c.diff
+++ b/gl/lib/regcomp.c.diff
@@ -11,7 +11,7 @@ index d5968bd..4926676 100644
  	  char *_Restrict_ errbuf, size_t errbuf_size)
  #endif
  {
-@@ -1383,7 +1383,7 @@ calc_first (void *extra, bin_tree_t *node)
+@@ -1415,7 +1415,7 @@ calc_first (void *extra, bin_tree_t *node)

  /* Pass 2: compute NEXT on the tree.  Preorder visit.  */
  static reg_errcode_t
@@ -20,17 +20,20 @@ index d5968bd..4926676 100644
  {
    switch (node->token.type)
      {
-@@ -2744,7 +2744,8 @@ static reg_errcode_t
+@@ -2792,8 +2792,10 @@ build_range_exp (const reg_syntax_t synt
+ static reg_errcode_t
  internal_function
- build_collating_symbol (bitset_t sbcset,
  # ifdef RE_ENABLE_I18N
--			re_charset_t *mbcset, Idx *coll_sym_alloc,
-+			re_charset_t *mbcset _UNUSED_PARAMETER_,
+-build_collating_symbol (bitset_t sbcset, re_charset_t *mbcset,
+-			Idx *coll_sym_alloc, const unsigned char *name)
++build_collating_symbol (bitset_t sbcset,
++                        re_charset_t *mbcset _UNUSED_PARAMETER_,
 +			Idx *coll_sym_alloc _UNUSED_PARAMETER_,
- # endif
- 			const unsigned char *name)
- {
-@@ -3323,7 +3324,8 @@ parse_bracket_exp (re_string_t *regexp, re_dfa_t *dfa, re_token_t *token,
++                        const unsigned char *name)
+ # else /* not RE_ENABLE_I18N */
+ build_collating_symbol (bitset_t sbcset, const unsigned char *name)
+ # endif /* not RE_ENABLE_I18N */
+@@ -3377,7 +3379,8 @@ parse_bracket_exp (re_string_t *regexp,

  static reg_errcode_t
  parse_bracket_element (bracket_elem_t *elem, re_string_t *regexp,
@@ -40,19 +43,20 @@ index d5968bd..4926676 100644
  		       reg_syntax_t syntax, bool accept_hyphen)
  {
  #ifdef RE_ENABLE_I18N
-@@ -3410,8 +3412,9 @@ parse_bracket_symbol (bracket_elem_t *elem, re_string_t *regexp,
+@@ -3464,8 +3467,10 @@ parse_bracket_symbol (bracket_elem_t *elem, re_string_t *regexp,

  static reg_errcode_t
  #ifdef RE_ENABLE_I18N
 -build_equiv_class (bitset_t sbcset, re_charset_t *mbcset,
 -		   Idx *equiv_class_alloc, const unsigned char *name)
-+build_equiv_class (bitset_t sbcset, re_charset_t *mbcset _UNUSED_PARAMETER_,
++build_equiv_class (bitset_t sbcset,
++                   re_charset_t *mbcset _UNUSED_PARAMETER_,
 +		   Idx *equiv_class_alloc _UNUSED_PARAMETER_,
 +		   const unsigned char *name)
  #else /* not RE_ENABLE_I18N */
  build_equiv_class (bitset_t sbcset, const unsigned char *name)
  #endif /* not RE_ENABLE_I18N */
-@@ -3816,7 +3819,7 @@ free_token (re_token_t *node)
+@@ -3869,1 +3874,1 @@ free_token (re_token_t *node)
     and its children. */

  static reg_errcode_t
diff --git a/gnulib b/gnulib
index bfb5863..de009b6 160000
--- a/gnulib
+++ b/gnulib
@@ -1 +1 @@
-Subproject commit bfb5863e350590d8fe9a643d840c7d0b5f3d934b
+Subproject commit de009b65198c00dbe0ecd796ce75d5d9f5346103
--
1.7.9.190.gf1d33


From MAILER-DAEMON Sat Feb 11 11:20:15 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RwFgN-0001LG-Gw
	for mharc-coreutils@gnu.org; Sat, 11 Feb 2012 11:20:15 -0500
Received: from eggs.gnu.org ([140.186.70.92]:59909)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <pengyu.ut@gmail.com>) id 1RwFgL-0001LA-KV
	for coreutils@gnu.org; Sat, 11 Feb 2012 11:20:14 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <pengyu.ut@gmail.com>) id 1RwFgK-0007pz-NE
	for coreutils@gnu.org; Sat, 11 Feb 2012 11:20:13 -0500
Received: from mail-tul01m020-f169.google.com ([209.85.214.169]:35246)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <pengyu.ut@gmail.com>) id 1RwFgK-0007po-GU
	for coreutils@gnu.org; Sat, 11 Feb 2012 11:20:12 -0500
Received: by obbta7 with SMTP id ta7so6129406obb.0
	for <coreutils@gnu.org>; Sat, 11 Feb 2012 08:20:11 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma;
	h=mime-version:date:message-id:subject:from:to:content-type;
	bh=Z7j8N1U9eHAB7MLMDX4PSWMrRqODfujeAo58LggMU8Y=;
	b=TK8pCs2j81qR/r404u1hWq41QC+LLOl7gJ5cu/CrHBTflS2ctMbIqGWNEJWqKdsEDQ
	aKw82w0IZGZGex8eMACpahjBfpAkbc2fkDDrTlXzXZj4Bx/RQSSneUKG8v7EEmQhGnlk
	jA5XoShnItsUtj0ZOqQExqSiIsTV0duqVvwCk=
MIME-Version: 1.0
Received: by 10.182.76.229 with SMTP id n5mr7071274obw.14.1328977211242; Sat,
	11 Feb 2012 08:20:11 -0800 (PST)
Received: by 10.182.75.202 with HTTP; Sat, 11 Feb 2012 08:20:11 -0800 (PST)
Date: Sat, 11 Feb 2012 10:20:11 -0600
Message-ID: <CABrM6wmZC1Smw85J5hd4=jhZmCysU4DZeu6Cuv2gWjaae9cY+w@mail.gmail.com>
Subject: sort complexity on nearly sorted input
From: Peng Yu <pengyu.ut@gmail.com>
To: Coreutils <coreutils@gnu.org>
Content-Type: text/plain; charset=ISO-8859-1
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.85.214.169
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sat, 11 Feb 2012 16:20:14 -0000

Hi,

I assume the time complexity of 'sort' is log N, where N is the input size.

But I'm not familiar with 'sort' enough to tell the complexity of
sorting a nearly sorted input. Suppose that I have a listed of N
numbers, there only k numbers (k << N, say k=N/100) that are not in
the correct position and all the other N-k numbers are in the correct
position. Does anybody knows the time complexity of this case, or it
is still log N?

-- 
Regards,
Peng


From MAILER-DAEMON Sat Feb 11 11:53:29 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RwGCX-0007mF-7u
	for mharc-coreutils@gnu.org; Sat, 11 Feb 2012 11:53:29 -0500
Received: from eggs.gnu.org ([140.186.70.92]:51919)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <dave_br@gmx.com>) id 1RwGCU-0007lv-KS
	for coreutils@gnu.org; Sat, 11 Feb 2012 11:53:27 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <dave_br@gmx.com>) id 1RwGCS-0003hB-Ju
	for coreutils@gnu.org; Sat, 11 Feb 2012 11:53:26 -0500
Received: from mailout-eu.gmx.com ([213.165.64.42]:54183)
	by eggs.gnu.org with smtp (Exim 4.71)
	(envelope-from <dave_br@gmx.com>) id 1RwGCS-0003gk-AA
	for coreutils@gnu.org; Sat, 11 Feb 2012 11:53:24 -0500
Received: (qmail invoked by alias); 11 Feb 2012 16:52:53 -0000
Received: from 76.Red-213-98-80.staticIP.rima-tde.net (EHLO fozzie.sax)
	[213.98.80.76]
	by mail.gmx.com (mp-eu005) with SMTP; 11 Feb 2012 17:52:53 +0100
X-Authenticated: #48875277
X-Provags-ID: V01U2FsdGVkX1+qO8pCyEGq36w0Zxlpf5QSjuBgVNzy37KPjRaL/9
	2TudNSoEu89Qnx
Date: Sat, 11 Feb 2012 17:52:44 +0100
From: Davide Brini <dave_br@gmx.com>
To: coreutils@gnu.org
Subject: Re: sort complexity on nearly sorted input
In-Reply-To: <CABrM6wmZC1Smw85J5hd4=jhZmCysU4DZeu6Cuv2gWjaae9cY+w@mail.gmail.com>
References: <CABrM6wmZC1Smw85J5hd4=jhZmCysU4DZeu6Cuv2gWjaae9cY+w@mail.gmail.com>
X-Mailer: Claws Mail 3.8.0 (GTK+ 2.24.9; i686-pc-linux-gnu)
Mime-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
X-Y-GMX-Trusted: 0
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
Message-ID: <E1RwGCS-0003gk-AA@eggs.gnu.org>
X-Received-From: 213.165.64.42
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sat, 11 Feb 2012 16:53:27 -0000

On Sat, 11 Feb 2012 10:20:11 -0600, Peng Yu <pengyu.ut@gmail.com> wrote:

> Hi,
> 
> I assume the time complexity of 'sort' is log N, where N is the input
> size.
>
> But I'm not familiar with 'sort' enough to tell the complexity of
> sorting a nearly sorted input. Suppose that I have a listed of N
> numbers, there only k numbers (k << N, say k=N/100) that are not in
> the correct position and all the other N-k numbers are in the correct
> position. Does anybody knows the time complexity of this case, or it
> is still log N?

The complexity of an algorithm provides an upper bound for the worst case
which holds regardless of how bad the input is. (on the other hand, the
actual operation *may* cost less, depending on the specific algorithm and
input).

BTW, I don't know what sorting method "sort" uses, but for most sorting
algorithms the complexity (that is, the number of comparisons the algorithm
has to make to sort the data) is O(N log(N)). (Simplifying a lot, "O(Y)"
is the notation to say "at most Y").

Quick sort is a special case among the popular sort algorithms, in that its
complexity is officially O(N^2) with an *average* complexity of O(N
log(N)), just like the others; it can be modified and forced to be O(N
log(N)), but the reward of doing that isn't usually worth the cost; in
practice, the "normal" version performs as good as or better than other
guaranteed O(N log(N)) algorithms.


From MAILER-DAEMON Sat Feb 11 13:11:58 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RwHQU-0003En-Qz
	for mharc-coreutils@gnu.org; Sat, 11 Feb 2012 13:11:58 -0500
Received: from eggs.gnu.org ([140.186.70.92]:48109)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <pengyu.ut@gmail.com>) id 1RwHQS-0003Ee-Qw
	for coreutils@gnu.org; Sat, 11 Feb 2012 13:11:57 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <pengyu.ut@gmail.com>) id 1RwHQR-00084X-VF
	for coreutils@gnu.org; Sat, 11 Feb 2012 13:11:56 -0500
Received: from mail-tul01m020-f169.google.com ([209.85.214.169]:62186)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <pengyu.ut@gmail.com>) id 1RwHQR-00084T-SF
	for coreutils@gnu.org; Sat, 11 Feb 2012 13:11:55 -0500
Received: by obbta7 with SMTP id ta7so6211283obb.0
	for <coreutils@gnu.org>; Sat, 11 Feb 2012 10:11:55 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma;
	h=mime-version:in-reply-to:references:date:message-id:subject:from:to
	:content-type; bh=CkTtIETRzu7/kNaa/bWmTSEL09sFHAF8UqAo2BBvaEs=;
	b=E6RBby7MqI+Ken1kf2zy+LnKj1cg5LRRjC8C2fweQ/2Dm9c775IzZIH3x9kZwnW9rh
	FY3Mp9TnfKVy6CXIMXrxIlBletwMZZuertwemDPuBb475itImZnkAxfhLDGNfWa1uCyM
	3AqObUvfF/f2Ci9yQnyTl0NFcu/BtqoJK8Du4=
MIME-Version: 1.0
Received: by 10.60.22.169 with SMTP id e9mr2068784oef.24.1328983915275; Sat,
	11 Feb 2012 10:11:55 -0800 (PST)
Received: by 10.182.75.202 with HTTP; Sat, 11 Feb 2012 10:11:55 -0800 (PST)
In-Reply-To: <CABrM6wmZC1Smw85J5hd4=jhZmCysU4DZeu6Cuv2gWjaae9cY+w@mail.gmail.com>
References: <CABrM6wmZC1Smw85J5hd4=jhZmCysU4DZeu6Cuv2gWjaae9cY+w@mail.gmail.com>
Date: Sat, 11 Feb 2012 12:11:55 -0600
Message-ID: <CABrM6wkbFcPqmmp=8h16ai_jM7urZAXb-SyXL2qnYEv2picVrg@mail.gmail.com>
Subject: Re: sort complexity on nearly sorted input
From: Peng Yu <pengyu.ut@gmail.com>
To: Coreutils <coreutils@gnu.org>
Content-Type: text/plain; charset=ISO-8859-1
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.85.214.169
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sat, 11 Feb 2012 18:11:57 -0000

On Sat, Feb 11, 2012 at 10:20 AM, Peng Yu <pengyu.ut@gmail.com> wrote:
> Hi,
>
> I assume the time complexity of 'sort' is log N, where N is the input size.
                                                                  ^
typo. Should be N log N
>
> But I'm not familiar with 'sort' enough to tell the complexity of
> sorting a nearly sorted input. Suppose that I have a listed of N
> numbers, there only k numbers (k << N, say k=N/100) that are not in
> the correct position and all the other N-k numbers are in the correct
> position. Does anybody knows the time complexity of this case, or it
> is still log N?
>
> --
> Regards,
> Peng



-- 
Regards,
Peng


From MAILER-DAEMON Sat Feb 11 19:06:16 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RwMxM-00070H-CW
	for mharc-coreutils@gnu.org; Sat, 11 Feb 2012 19:06:16 -0500
Received: from eggs.gnu.org ([140.186.70.92]:59086)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RwMxJ-00070B-EG
	for coreutils@gnu.org; Sat, 11 Feb 2012 19:06:14 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RwMxI-0005fB-6Y
	for coreutils@gnu.org; Sat, 11 Feb 2012 19:06:13 -0500
Received: from mail3.vodafone.ie ([213.233.128.45]:37539)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RwMxH-0005f2-VA
	for coreutils@gnu.org; Sat, 11 Feb 2012 19:06:12 -0500
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: ApMBAMYBN09tTC6//2dsb2JhbAAMN7JtAQEBBDIBRhALDQsJFg8JAwIBAgFFBg0BBwEBiAG3Q4t0BQoHBgQDBAQHDgYBAwgBDYN+FAEFDBgFg1QEmy6MaA
Received: from unknown (HELO [192.168.1.79]) ([109.76.46.191])
	by mail3.vodafone.ie with ESMTP; 12 Feb 2012 00:06:08 +0000
Message-ID: <4F37026E.8050206@draigBrady.com>
Date: Sun, 12 Feb 2012 00:06:06 +0000
From: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:6.0) Gecko/20110816 Thunderbird/6.0
MIME-Version: 1.0
To: Peng Yu <pengyu.ut@gmail.com>
Subject: Re: sort complexity on nearly sorted input
References: <CABrM6wmZC1Smw85J5hd4=jhZmCysU4DZeu6Cuv2gWjaae9cY+w@mail.gmail.com>
In-Reply-To: <CABrM6wmZC1Smw85J5hd4=jhZmCysU4DZeu6Cuv2gWjaae9cY+w@mail.gmail.com>
X-Enigmail-Version: 1.3.2
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 213.233.128.45
Cc: Coreutils <coreutils@gnu.org>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 12 Feb 2012 00:06:14 -0000

On 02/11/2012 04:20 PM, Peng Yu wrote:
> Hi,
> 
> I assume the time complexity of 'sort' is log N, where N is the input size.
> 
> But I'm not familiar with 'sort' enough to tell the complexity of
> sorting a nearly sorted input. Suppose that I have a listed of N
> numbers, there only k numbers (k << N, say k=N/100) that are not in
> the correct position and all the other N-k numbers are in the correct
> position. Does anybody knows the time complexity of this case, or it
> is still log N?
> 

Well discounting all the memory, thread and data type management
that `sort` gives you, centrally there is this description
on sequential_sort():

   Use a recursive divide-and-conquer algorithm, in the style
   suggested by Knuth volume 3 (2nd edition), exercise 5.2.4-23.  Use
   the optimization suggested by exercise 5.2.4-10; this requires room
   for only 1.5*N lines, rather than the usual 2*N lines.  Knuth
   writes that this memory optimization was originally published by
   D. A. Bell, Comp J. 1 (1958), 75.

I searched for the above text out of interest, and noticed
this good overview of the operation of `sort`:
http://philip.greenspun.com/software/abstraction-filtration-comparison/sort/


It's trivial to empirically measure how sort behaves with different inputs:

$ for i in $(seq 4 7); do
>   seq -w $((10**$i)) > sorted.$i
>   shuf < sorted.$i > unsorted.$i
> done

$ export OMP_NUM_THREADS=1
$ export LANG=C

$ for i in $(seq 4 7); do
>   time sort sorted.$i > /dev/null
>   time sort unsorted.$i > /dev/null
> done

We can also compare python's in place "timsort",
given that the data set fits in RAM and timsort
is said to be especially good for mostly sorted data:

$ for type in sorted unsorted; do
>   time python -c "
> import sys
> a=sys.stdin.readlines()
> a.sort()
> sys.stdout.writelines(a)
> " > /dev/null < $type.7
> done

Results:
N         sorted   unsorted  py_sorted  py_unsorted
10000      0.015      0.021
100000     0.058      0.088
1000000    0.522      1.105
10000000   7.020     15.644     2.882        26.285

cheers,
Pádraig.


From MAILER-DAEMON Sat Feb 11 20:01:44 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RwNp2-0001YT-Kw
	for mharc-coreutils@gnu.org; Sat, 11 Feb 2012 20:01:44 -0500
Received: from eggs.gnu.org ([140.186.70.92]:50426)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <sven.breuner@itwm.fraunhofer.de>) id 1RwNoz-0001Tj-Kx
	for coreutils@gnu.org; Sat, 11 Feb 2012 20:01:43 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <sven.breuner@itwm.fraunhofer.de>) id 1RwNox-0005Ov-Gk
	for coreutils@gnu.org; Sat, 11 Feb 2012 20:01:41 -0500
Received: from mailgw1.uni-kl.de ([131.246.120.220]:51927)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <sven.breuner@itwm.fraunhofer.de>) id 1RwNox-0005Ol-92
	for coreutils@gnu.org; Sat, 11 Feb 2012 20:01:39 -0500
Received: from itwm2.itwm.fhg.de (itwm2.itwm.fhg.de [131.246.191.3])
	by mailgw1.uni-kl.de (8.14.3/8.14.3/Debian-5+lenny1) with ESMTP id
	q1C11ZrM000885
	(version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=NOT)
	for <coreutils@gnu.org>; Sun, 12 Feb 2012 02:01:35 +0100
Received: from mail2.itwm.fhg.de ([131.246.191.79]:42356)
	by itwm2.itwm.fhg.de with esmtps (TLSv1:DES-CBC3-SHA:168)
	(/C=DE/ST=Rheinland-Pfalz/L=Kaiserslautern/O=Fraunhofer
	ITWM/OU=SLG/CN=mail2.itwm.fhg.de)(verified=1) (Exim 4.74 #1)
	id 1RwNoq-0008Ic-U9; Sun, 12 Feb 2012 02:01:32 +0100
Message-ID: <4F370F6C.4090407@itwm.fraunhofer.de>
Date: Sun, 12 Feb 2012 02:01:32 +0100
From: Sven Breuner <sven.breuner@itwm.fraunhofer.de>
User-Agent: Mozilla/5.0 (X11; Linux i686 on x86_64;
	rv:10.0) Gecko/20120129 Thunderbird/10.0
MIME-Version: 1.0
To: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
Subject: Re: "ls -l": Avoid unnecessary getxattr() overhead
References: <87pqds3snt.fsf@Apollo.jerryland.fr>
	<4F30206B.1020802@itwm.fraunhofer.de>
	<CA+TATJHorgsuY7pxmpxxAeWUHRP5MfFfzQP05o=1H6C4ttswdA@mail.gmail.com>
	<4F3032BC.50702@draigBrady.com>
	<4F30631A.2070009@itwm.fraunhofer.de>
	<87y5scfdiq.fsf@rho.meyering.net> <4F33B5D7.3010909@draigBrady.com>
	<87mx8sf5w0.fsf@rho.meyering.net>
	<4F3437CE.4050308@itwm.fraunhofer.de>
	<4F346CC7.4030608@draigBrady.com>
In-Reply-To: <4F346CC7.4030608@draigBrady.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
X-ITWM-CharSet: ISO-8859-1
X-ITWM-Scanned-By: mail2.itwm.fhg.de
Content-Transfer-Encoding: quoted-printable
X-MIME-Autoconverted: from 8bit to quoted-printable by mailgw1.uni-kl.de id
	q1C11ZrM000885
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
X-Received-From: 131.246.120.220
Cc: Bernd Schubert <bernd.schubert@itwm.fraunhofer.de>,
	=?ISO-8859-1?Q?J=E9r=E9my_Compostella?= <jeremy.compostella@gmail.com>,
	coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 12 Feb 2012 01:01:43 -0000

P=E1draig Brady wrote on 02/10/2012 02:03 AM:
> On 02/09/2012 09:17 PM, Sven Breuner wrote:
>> Current git version patched...
>>
>> $ time ~/tmp/ls-8.15.34-31eee_patched -l>/dev/null
>>
>> real    0m21.254s
>> user    0m0.565s
>> sys     0m2.496s
>>
>> $ strace -c ~/tmp/ls-8.15.34-31eee_patched -l>/dev/null
>> % time     seconds  usecs/call     calls    errors syscall
>> ------ ----------- ----------- --------- --------- ----------------
>>   45.50    0.313386           5     61441           lstat
>>   43.63    0.300524           5     61441     61441 getxattr
>>   10.60    0.072986         973        75           getdents
>
> So you're still getting all those errors.
> Is that from file_has_acl() or cap_get_file()?
> Oh it must be from file_has_acl() as you've
> not specified --color. What is the errno in that case?
>
> Note you can avoid the cap_get_file() calls at least,
> that would be enabled with --color, by unsetting capability
> coloring, which you can test by running the following
> before your ls command.
>
> eval $(dircolors | sed 's/ca[^:]*:/ca=3D:/')

Thanks, P=E1draig - you're right. Indeed, I didn't consider that coloring=
=20
was disabled during my tests and that turning it on might lead to more=20
fs calls.

So here's the strace of a test with colors enabled:
(ls with Jim's selinux optimization patch applied)

$ strace -c ~/tmp/ls-8.15.34-31eee_patched -l --color=3Dalways >/dev/null
% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
  56.63    0.139981           1    122882    122882 getxattr
  38.43    0.094985         176       540           getdents
   3.03    0.007478           0     61441           lstat
   1.35    0.003341           0     61441     61441 capget
[...]

=3D> Two getxattr calls per file in the directory, both returning errors.

Using your suggestion regarding cap_get_file:

$ eval $(dircolors | sed 's/ca[^:]*:/ca=3D:/')
$ strace -c  ~/tmp/ls-8.15.34-31eee_patched -l  --color=3Dalways >/dev/nu=
ll
% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
  43.96    0.173020           3     61441           lstat
  37.75    0.148590           2     61441     61441 getxattr
  18.04    0.070994         131       540           getdents
[...]

=3D> Back to one getxattr call per file in the directory.

Even though it's nice to have the option to avoid the cap_get_file and=20
associated getxattr call, I guess it would be best to optimize the=20
default case. So that means there are two getxattr calls per directory=20
entry left that could be optimized.

Here's an strace excerpt with colors enabled and default color settings=20
on rhel 6.2:
$ strace ~/tmp/ls-8.15.34-31eee_patched -l --color=3Dalways
[...]
lstat("1", {st_mode=3DS_IFREG|0644, st_size=3D0, ...}) =3D 0
capget(0x20080522, 0, NULL)             =3D -1 EFAULT (Bad address)
getxattr("1", "security.capability", 0x7fffffffda30, 20) =3D -1 EOPNOTSUP=
P=20
(Operation not supported)
getxattr("1", "system.posix_acl_access", 0x0, 0) =3D -1 EOPNOTSUPP=20
(Operation not supported)

lstat("dir1", {st_mode=3DS_IFDIR|0755, st_size=3D0, ...}) =3D 0
getxattr("dir1", "system.posix_acl_access", 0x0, 0) =3D -1 EOPNOTSUPP=20
(Operation not supported)

=3D> Two getxattr calls per file in the directory, one getxattr per=20
subdirectory

Best regards,
Sven


From MAILER-DAEMON Sat Feb 11 20:07:29 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RwNub-0002dm-P6
	for mharc-coreutils@gnu.org; Sat, 11 Feb 2012 20:07:29 -0500
Received: from eggs.gnu.org ([140.186.70.92]:54233)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <sven.breuner@itwm.fraunhofer.de>) id 1RwNuZ-0002cA-G5
	for coreutils@gnu.org; Sat, 11 Feb 2012 20:07:28 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <sven.breuner@itwm.fraunhofer.de>) id 1RwNuX-0006Ag-Nf
	for coreutils@gnu.org; Sat, 11 Feb 2012 20:07:27 -0500
Received: from mailgw1.uni-kl.de ([131.246.120.220]:39624)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <sven.breuner@itwm.fraunhofer.de>) id 1RwNuX-0006AH-D6
	for coreutils@gnu.org; Sat, 11 Feb 2012 20:07:25 -0500
Received: from itwm2.itwm.fhg.de (itwm2.itwm.fhg.de [131.246.191.3])
	by mailgw1.uni-kl.de (8.14.3/8.14.3/Debian-5+lenny1) with ESMTP id
	q1C17NOg002558
	(version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=NOT)
	for <coreutils@gnu.org>; Sun, 12 Feb 2012 02:07:23 +0100
Received: from mail2.itwm.fhg.de ([131.246.191.79]:42366)
	by itwm2.itwm.fhg.de with esmtps (TLSv1:DES-CBC3-SHA:168)
	(/C=DE/ST=Rheinland-Pfalz/L=Kaiserslautern/O=Fraunhofer
	ITWM/OU=SLG/CN=mail2.itwm.fhg.de)(verified=1) (Exim 4.74 #1)
	id 1RwNuU-0008Js-G3; Sun, 12 Feb 2012 02:07:22 +0100
Message-ID: <4F3710CA.8020402@itwm.fraunhofer.de>
Date: Sun, 12 Feb 2012 02:07:22 +0100
From: Sven Breuner <sven.breuner@itwm.fraunhofer.de>
User-Agent: Mozilla/5.0 (X11; Linux i686 on x86_64;
	rv:10.0) Gecko/20120129 Thunderbird/10.0
MIME-Version: 1.0
To: Jim Meyering <jim@meyering.net>
Subject: Re: "ls -l": Avoid unnecessary getxattr() overhead
References: <87pqds3snt.fsf@Apollo.jerryland.fr>
	<4F30206B.1020802@itwm.fraunhofer.de>
	<CA+TATJHorgsuY7pxmpxxAeWUHRP5MfFfzQP05o=1H6C4ttswdA@mail.gmail.com>
	<4F3032BC.50702@draigBrady.com>
	<4F30631A.2070009@itwm.fraunhofer.de>
	<87y5scfdiq.fsf@rho.meyering.net> <4F33B5D7.3010909@draigBrady.com>
	<87mx8sf5w0.fsf@rho.meyering.net>
	<4F3437CE.4050308@itwm.fraunhofer.de>
	<4F346CC7.4030608@draigBrady.com> <87r4y3dry0.fsf@rho.meyering.net>
In-Reply-To: <87r4y3dry0.fsf@rho.meyering.net>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
X-ITWM-CharSet: ISO-8859-1
X-ITWM-Scanned-By: mail2.itwm.fhg.de
Content-Transfer-Encoding: quoted-printable
X-MIME-Autoconverted: from 8bit to quoted-printable by mailgw1.uni-kl.de id
	q1C17NOg002558
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
X-Received-From: 131.246.120.220
Cc: Bernd Schubert <bernd.schubert@itwm.fraunhofer.de>,
	=?ISO-8859-1?Q?J=E9r=E9my_Compostella?= <jeremy.compostella@gmail.com>,
	coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 12 Feb 2012 01:07:29 -0000

Jim Meyering wrote on 02/10/2012 09:16 AM:
> P=E1draig Brady wrote:
>>> Current git version patched...
>>>
>>> $ strace -c ~/tmp/ls-8.15.34-31eee_patched -l>/dev/null
>>> % time     seconds  usecs/call     calls    errors syscall
>>> ------ ----------- ----------- --------- --------- ----------------
>>>   45.50    0.313386           5     61441           lstat
>>>   43.63    0.300524           5     61441     61441 getxattr
>>>   10.60    0.072986         973        75           getdents
>>
>> So you're still getting all those errors.
>> Is that from file_has_acl() or cap_get_file()?
>> Oh it must be from file_has_acl() as you've
>> not specified --color. What is the errno in that case?
>
> So we can avoid all but one of those failing getxattr calls
> in this case.

Yes, I would say so, too. If ls gets EOPNOTSUPP from a=20
getxattr(acl_stuff) call on a file, it can skip further getxattr calls=20
on the corresponding file system.

> It's a little tricky because of how file_has_acl works.
> We may have to set errno=3D0 just prior, and then test errno
> when file_has_acl returns<=3D0.  Sven, can you tell us whether
> file_has_acl is returning -1 or 0 in your case?  And confirm
> that the errno value is propagated out to ls.c?
>
> So either print both "n" and "errno" right after this line in ls.c
>
>                int n =3D file_has_acl (absolute_name,&f->stat);
>
> or inspect those values in gdb?
> If n is always negative for you, then I think we can skip the
> kludgy errno=3D0 initialization.

I used ls from git with the selinux optimization patch applied and added=20
the following printf:

               int n =3D file_has_acl (absolute_name, &f->stat);
               printf("after file_has_acl(%s): n=3D%d errno=3D%d (%s)\n",
                      absolute_name, n, errno, strerror(errno) );


Here's the result from a small test directory:
$ ~/git/coreutils/src/ls -l --color=3Dalways
after file_has_acl(file3): n=3D0 errno=3D95 (Operation not supported)
after file_has_acl(dirlink): n=3D0 errno=3D95 (Operation not supported)
after file_has_acl(file1): n=3D0 errno=3D95 (Operation not supported)
after file_has_acl(file2): n=3D0 errno=3D95 (Operation not supported)
after file_has_acl(filelink): n=3D0 errno=3D95 (Operation not supported)
after file_has_acl(dir): n=3D0 errno=3D95 (Operation not supported)
after file_has_acl(dangling_link): n=3D0 errno=3D95 (Operation not suppor=
ted)
total 0
lrwxrwxrwx 1 breuner hpc 6 Feb 12 00:46 dangling_link -> nofile
drwxr-xr-x 2 breuner hpc 0 Feb 12 00:42 dir
lrwxrwxrwx 1 breuner hpc 3 Feb 12 01:32 dirlink -> dir
-rw-r--r-- 1 breuner hpc 0 Feb 12 00:44 file1
-rw-r--r-- 1 breuner hpc 0 Feb 12 00:44 file2
-rw-r--r-- 1 breuner hpc 0 Feb 12 00:44 file3
lrwxrwxrwx 1 breuner hpc 5 Feb 12 00:46 filelink -> file1

=3D> n is not negative
So that is probably not in sync with the comment of file_has_acl(),=20
which says "Return [...] -1 (setting errno) on error."


In case it's relevant: errno after file_has_acl() is not always=20
EOPNOTSUPP. Here's the result from a directory with only a single=20
dangling symlink in it:

$~/git/coreutils/src/ls -l --color=3Dalways
after file_has_acl(link1): n=3D0 errno=3D61 (No data available)
total 0
lrwxrwxrwx 1 breuner hpc 5 Feb 12 00:59 link1 -> file1

(Tested with similar results on cifs, ext4, fhgfs.)

Best regards,
Sven


From MAILER-DAEMON Sat Feb 11 20:13:23 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RwO0J-0003qQ-1Q
	for mharc-coreutils@gnu.org; Sat, 11 Feb 2012 20:13:23 -0500
Received: from eggs.gnu.org ([140.186.70.92]:47593)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RwO0F-0003qK-Ls
	for coreutils@gnu.org; Sat, 11 Feb 2012 20:13:22 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RwO0E-0006wQ-69
	for coreutils@gnu.org; Sat, 11 Feb 2012 20:13:19 -0500
Received: from mail2.vodafone.ie ([213.233.128.44]:2478)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RwO0E-0006wB-0R
	for coreutils@gnu.org; Sat, 11 Feb 2012 20:13:18 -0500
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: ApMBANQQN09tTC6//2dsb2JhbAAMNq07hTIBAQEEMgFGEAsNCwkWDwkDAgECAUUGDQEHAQG/QItIBiYWBgQDBAQHDgYBAwgBDYN+hBcEmy6MaA
Received: from unknown (HELO [192.168.1.79]) ([109.76.46.191])
	by mail2.vodafone.ie with ESMTP; 12 Feb 2012 01:13:15 +0000
Message-ID: <4F371229.8050804@draigBrady.com>
Date: Sun, 12 Feb 2012 01:13:13 +0000
From: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:6.0) Gecko/20110816 Thunderbird/6.0
MIME-Version: 1.0
To: Sven Breuner <sven.breuner@itwm.fraunhofer.de>
Subject: Re: "ls -l": Avoid unnecessary getxattr() overhead
References: <87pqds3snt.fsf@Apollo.jerryland.fr>
	<4F30206B.1020802@itwm.fraunhofer.de>
	<CA+TATJHorgsuY7pxmpxxAeWUHRP5MfFfzQP05o=1H6C4ttswdA@mail.gmail.com>
	<4F3032BC.50702@draigBrady.com>
	<4F30631A.2070009@itwm.fraunhofer.de>
	<87y5scfdiq.fsf@rho.meyering.net> <4F33B5D7.3010909@draigBrady.com>
	<87mx8sf5w0.fsf@rho.meyering.net>
	<4F3437CE.4050308@itwm.fraunhofer.de>
	<4F346CC7.4030608@draigBrady.com>
	<4F370F6C.4090407@itwm.fraunhofer.de>
In-Reply-To: <4F370F6C.4090407@itwm.fraunhofer.de>
X-Enigmail-Version: 1.3.2
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 213.233.128.44
Cc: Bernd Schubert <bernd.schubert@itwm.fraunhofer.de>,
	=?ISO-8859-1?Q?J=E9r=E9my_Compostella?= <jeremy.compostella@gmail.com>,
	coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 12 Feb 2012 01:13:22 -0000

On 02/12/2012 01:01 AM, Sven Breuner wrote:
> Pádraig Brady wrote on 02/10/2012 02:03 AM:
>> On 02/09/2012 09:17 PM, Sven Breuner wrote:
>>> Current git version patched...
>>>
>>> $ time ~/tmp/ls-8.15.34-31eee_patched -l>/dev/null
>>>
>>> real    0m21.254s
>>> user    0m0.565s
>>> sys     0m2.496s
>>>
>>> $ strace -c ~/tmp/ls-8.15.34-31eee_patched -l>/dev/null
>>> % time     seconds  usecs/call     calls    errors syscall
>>> ------ ----------- ----------- --------- --------- ----------------
>>>   45.50    0.313386           5     61441           lstat
>>>   43.63    0.300524           5     61441     61441 getxattr
>>>   10.60    0.072986         973        75           getdents
>>
>> So you're still getting all those errors.
>> Is that from file_has_acl() or cap_get_file()?
>> Oh it must be from file_has_acl() as you've
>> not specified --color. What is the errno in that case?
>>
>> Note you can avoid the cap_get_file() calls at least,
>> that would be enabled with --color, by unsetting capability
>> coloring, which you can test by running the following
>> before your ls command.
>>
>> eval $(dircolors | sed 's/ca[^:]*:/ca=:/')
> 
> Thanks, Pádraig - you're right. Indeed, I didn't consider that coloring was disabled during my tests and that turning it on might lead to more fs calls.
> 
> So here's the strace of a test with colors enabled:
> (ls with Jim's selinux optimization patch applied)
> 
> $ strace -c ~/tmp/ls-8.15.34-31eee_patched -l --color=always >/dev/null
> % time     seconds  usecs/call     calls    errors syscall
> ------ ----------- ----------- --------- --------- ----------------
>  56.63    0.139981           1    122882    122882 getxattr
>  38.43    0.094985         176       540           getdents
>   3.03    0.007478           0     61441           lstat
>   1.35    0.003341           0     61441     61441 capget
> [...]
> 
> => Two getxattr calls per file in the directory, both returning errors.
> 
> Using your suggestion regarding cap_get_file:
> 
> $ eval $(dircolors | sed 's/ca[^:]*:/ca=:/')
> $ strace -c  ~/tmp/ls-8.15.34-31eee_patched -l  --color=always >/dev/null
> % time     seconds  usecs/call     calls    errors syscall
> ------ ----------- ----------- --------- --------- ----------------
>  43.96    0.173020           3     61441           lstat
>  37.75    0.148590           2     61441     61441 getxattr
>  18.04    0.070994         131       540           getdents
> [...]
> 
> => Back to one getxattr call per file in the directory.
> 
> Even though it's nice to have the option to avoid the cap_get_file and associated getxattr call, I guess it would be best to optimize the default case. So that means there are two getxattr calls per directory entry left that could be optimized.
> 
> Here's an strace excerpt with colors enabled and default color settings on rhel 6.2:
> $ strace ~/tmp/ls-8.15.34-31eee_patched -l --color=always
> [...]
> lstat("1", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
> capget(0x20080522, 0, NULL)             = -1 EFAULT (Bad address)
> getxattr("1", "security.capability", 0x7fffffffda30, 20) = -1 EOPNOTSUPP (Operation not supported)
> getxattr("1", "system.posix_acl_access", 0x0, 0) = -1 EOPNOTSUPP (Operation not supported)
> 
> lstat("dir1", {st_mode=S_IFDIR|0755, st_size=0, ...}) = 0
> getxattr("dir1", "system.posix_acl_access", 0x0, 0) = -1 EOPNOTSUPP (Operation not supported)
> 
> => Two getxattr calls per file in the directory, one getxattr per subdirectory

Thanks for the nice summary.
I think it would be worth applying the same avoidance
technique to file_has_acl() and cap_get_file() so,
given their non ubiquity and high overhead.

cheers,
Pádraig.


From MAILER-DAEMON Sat Feb 11 20:16:29 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RwO3J-00040V-AN
	for mharc-coreutils@gnu.org; Sat, 11 Feb 2012 20:16:29 -0500
Received: from eggs.gnu.org ([140.186.70.92]:56246)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RwO3G-00040I-7C
	for coreutils@gnu.org; Sat, 11 Feb 2012 20:16:28 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RwO3E-0007TG-6T
	for coreutils@gnu.org; Sat, 11 Feb 2012 20:16:26 -0500
Received: from mail3.vodafone.ie ([213.233.128.45]:52812)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RwO3D-0007TA-Qw
	for coreutils@gnu.org; Sat, 11 Feb 2012 20:16:24 -0500
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: ApMBAPURN09tTC6//2dsb2JhbAAMNq07hTIBAQEEDH0LDQsJFg8JAwIBAgFFBgEMCAEBv0CLXRdSAQ2DfoQXBI5JgRuLSoM4iTA
Received: from unknown (HELO [192.168.1.79]) ([109.76.46.191])
	by mail3.vodafone.ie with ESMTP; 12 Feb 2012 01:16:22 +0000
Message-ID: <4F3712E4.2080101@draigBrady.com>
Date: Sun, 12 Feb 2012 01:16:20 +0000
From: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:6.0) Gecko/20110816 Thunderbird/6.0
MIME-Version: 1.0
To: coreutils@gnu.org, =?ISO-8859-1?Q?J=E9r=E9my_Compostella?=
	<jeremy.compostella@gmail.com>
Subject: Re: dd: add 'skip_bytes' and 'count_bytes' operands
References: <87mx8w3ryu.fsf@Apollo.jerryland.fr>
	<4F2FF5EE.9010906@draigBrady.com>
	<CA+TATJF4w4Ue44Za4hgGR_0RxgEZGBm5nzhHRSdGYxzzfKWNLg@mail.gmail.com>
	<4F317CDF.2040909@draigBrady.com>
	<CA+TATJFU7yOGzzeZseHUFTqxxGAtz8zUoQccz=5m0Eu=CqEtRQ@mail.gmail.com>
	<4F329FC9.4030403@draigBrady.com>
	<CA+TATJFkn8=Kd3iqrtk=z2pkkzKRGwnAsquw61r0kqSMx76zgQ@mail.gmail.com>
	<CA+TATJF2uW6eHqF4fkwacHq4xm1XPY=3Ent3RHDNPuMA8GOZVA@mail.gmail.com>
	<4F354C8E.9040203@draigBrady.com>
In-Reply-To: <4F354C8E.9040203@draigBrady.com>
X-Enigmail-Version: 1.3.2
Content-Type: multipart/mixed; boundary="------------070506090207050404060403"
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 213.233.128.45
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 12 Feb 2012 01:16:28 -0000

This is a multi-part message in MIME format.
--------------070506090207050404060403
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit

On 02/10/2012 04:57 PM, Pádraig Brady wrote:
> On 02/10/2012 04:21 PM, Jérémy Compostella wrote:
>> Pádraig, others,
>>
>> First, I added the missing seek_bytes feature.
>> Second, I re-wrote the patch in order to replace the _bytes operands by the
>> corresponding input/output flags.
>> Finally, I added some tests to check the new seek_bytes flag.
>>
>> I successfully run the entire tests suite.
> 
> Excellent.
> I'll review fully later.
> 
>> * doc/coreutils.texi (New features): Detail new flags and behaviors.
> 
> s/New features/dd invocation/
> 
> I've also attached a diff of some adjustments to --amend to the texinfo.

I've updated the diff slightly to add a couple more doc tweaks,
and ensure all lines are < 80 chars.
Note `make syntax-check` would have cought that for you.

I can't see any issue with the logic.
A very nice patch all round.

I'll apply this in the morning.

cheers,
Pádraig.

--------------070506090207050404060403
Content-Type: text/plain;
 name="dd_bytes_desc.diff"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename="dd_bytes_desc.diff"

diff --git a/doc/coreutils.texi b/doc/coreutils.texi
index 6efc5cc..02c3a2a 100644
--- a/doc/coreutils.texi
+++ b/doc/coreutils.texi
@@ -8060,15 +8060,21 @@ use @var{bytes} as the fixed record length.
 @item skip=@var{n}
 @opindex skip
 Skip @var{n} @samp{ibs}-byte blocks in the input file before copying.
+If @samp{iflag=skip_bytes} is specified, @var{n} is interpreted
+as a byte count rather than a block count.
 
 @item seek=@var{n}
 @opindex seek
 Skip @var{n} @samp{obs}-byte blocks in the output file before copying.
+if @samp{oflag=seek_bytes} is specified, @var{n} is interpreted
+as a byte count rather than a block count.
 
 @item count=@var{n}
 @opindex count
 Copy @var{n} @samp{ibs}-byte blocks from the input file, instead
 of everything until the end of the file.
+if @samp{iflag=count_bytes} is specified, @var{n} is interpreted
+as a byte count rather than a block count.
 
 @item status=noxfer
 @opindex status
@@ -8323,20 +8329,23 @@ This flag can be used only with @code{iflag}.
 
 @item count_bytes
 @opindex count_bytes
-Use byte unit instead of @samp{ibs}-byte block unit for the
-@samp{count=n} operand.
+Interpret the @samp{count=} operand as a byte count,
+rather than a block count, which allows specifying
+a length that is not a multiple of the I/O block size.
 This flag can be used only with @code{iflag}.
 
 @item skip_bytes
 @opindex skip_bytes
-Use byte unit instead of @samp{ibs}-byte block unit for the
-@samp{skip=n} operand.
+Interpret the @samp{skip=} operand as a byte count,
+rather than a block count, which allows specifying
+an offset that is not a multiple of the I/O block size.
 This flag can be used only with @code{iflag}.
 
 @item seek_bytes
 @opindex seek_bytes
-Use byte unit instead of @samp{obs}-byte block unit for the
-@samp{seek=n} operand.
+Interpret the @samp{seek=} operand as a byte count,
+rather than a block count, which allows specifying
+an offset that is not a multiple of the I/O block size.
 This flag can be used only with @code{oflag}.
 
 @end table
@@ -8361,13 +8370,12 @@ should not be too large---values larger than a few megabytes
 are generally wasteful or (as in the gigabyte..exabyte case) downright
 counterproductive or error-inducing.
 
-Use different @command{dd} invocations to use different block sizes
-for skipping and I/O@. To process data that is at an offset or size
-that is not a multiple of the I/O@ block size, you can use the
-count_bytes and skip_bytes input flag and the seek_bytes output flag.
-Alternatively the traditional method of separate @command{dd}
-invocations can be used.  For example, the following shell commands
-copy data in 512 KiB blocks between a disk and a tape, but do not save
+To process data that is at an offset or size that is not a
+multiple of the I/O@ block size, you can use the @samp{skip_bytes},
+@samp{seek_bytes} and @samp{count_bytes} flags.  Alternatively
+the traditional method of separate @command{dd} invocations can be used.
+For example, the following shell commands copy data
+in 512 KiB blocks between a disk and a tape, but do not save
 or restore a 4 KiB label at the start of the disk:
 
 @example
diff --git a/src/dd.c b/src/dd.c
index 7ab7cb2..e7f4037 100644
--- a/src/dd.c
+++ b/src/dd.c
@@ -598,14 +598,14 @@ Each FLAG symbol may be:\n\
       if (O_TEXT)
         fputs (_("  text      use text I/O for data\n"), stdout);
       if (O_COUNT_BYTES)
-        fputs (_("  count_bytes  use byte unit for the 'count=N' operand\
- (iflag only)\n"), stdout);
+        fputs (_("  count_bytes  treat 'count=N' as a byte count (iflag only)\n\
+"), stdout);
       if (O_SKIP_BYTES)
-        fputs (_("  skip_bytes  use byte unit for the 'skip=N' operand\
- (iflag only)\n"), stdout);
+        fputs (_("  skip_bytes  treat 'skip=N' as a byte count (iflag only)\n\
+"), stdout);
       if (O_SEEK_BYTES)
-        fputs (_("  seek_bytes  use byte unit for the 'seek=N' operand\
- (oflag only)\n"), stdout);
+        fputs (_("  seek_bytes  treat 'seek=N' as a byte count (oflag only)\n\
+"), stdout);
 
       {
         char const *siginfo_name = (SIGINFO == SIGUSR1 ? "USR1" : "INFO");
@@ -1863,9 +1863,11 @@ dd_copy (void)
 
   if (skip_records != 0 || skip_bytes != 0)
     {
-      uintmax_t us_bytes = input_offset + (skip_records * input_blocksize) + skip_bytes;
+      uintmax_t us_bytes = input_offset + (skip_records * input_blocksize)
+                           + skip_bytes;
       uintmax_t us_blocks = skip (STDIN_FILENO, input_file,
-                                  skip_records, input_blocksize, &skip_bytes, ibuf);
+                                  skip_records, input_blocksize, &skip_bytes,
+                                  ibuf);
       us_bytes -= input_offset;
 
       /* POSIX doesn't say what to do when dd detects it has been
@@ -1885,7 +1887,8 @@ dd_copy (void)
     {
       size_t bytes = seek_bytes;
       uintmax_t write_records = skip (STDOUT_FILENO, output_file,
-                                      seek_records, output_blocksize, &bytes, obuf);
+                                      seek_records, output_blocksize, &bytes,
+                                      obuf);
 
       if (write_records != 0 || bytes != 0)
         {

--------------070506090207050404060403--


From MAILER-DAEMON Sat Feb 11 21:37:11 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RwPJP-0007B7-1I
	for mharc-coreutils@gnu.org; Sat, 11 Feb 2012 21:37:11 -0500
Received: from eggs.gnu.org ([140.186.70.92]:49051)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <goodman.m.w@gmail.com>) id 1RwPJK-0007Ao-Sm
	for coreutils@gnu.org; Sat, 11 Feb 2012 21:37:09 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <goodman.m.w@gmail.com>) id 1RwPJH-00005y-Ue
	for coreutils@gnu.org; Sat, 11 Feb 2012 21:37:06 -0500
Received: from mail-yx0-f169.google.com ([209.85.213.169]:43944)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <goodman.m.w@gmail.com>) id 1RwPJH-00005t-Qe
	for coreutils@gnu.org; Sat, 11 Feb 2012 21:37:03 -0500
Received: by yenl5 with SMTP id l5so2418903yen.0
	for <coreutils@gnu.org>; Sat, 11 Feb 2012 18:37:02 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma;
	h=mime-version:from:date:message-id:subject:to:content-type;
	bh=LoHiEyrGrLGzLtr1Xd/SpsLchgU1CERZFMTCLmQEUBo=;
	b=JgZO5A4ddRq0odrD/bxxUfFQ9ERDk/e0DS13LYVzTzkYxwVnF5wYD0HiZ7Yz3stLK1
	gEetVci1nXIvqaYMbd3xE2Wm8OB8s5Y0iPprYFwualyrC87J3nV3mZMvrfqyuOPxXlVH
	+c6sPCmQL2Di+38fNTpmuYSkfjGbZV0Cg2FaM=
Received: by 10.236.154.232 with SMTP id h68mr15293119yhk.51.1329014201605;
	Sat, 11 Feb 2012 18:36:41 -0800 (PST)
MIME-Version: 1.0
Received: by 10.101.99.9 with HTTP; Sat, 11 Feb 2012 18:36:21 -0800 (PST)
From: "goodman.m.w@gmail.com" <goodman.m.w@gmail.com>
Date: Sat, 11 Feb 2012 18:36:21 -0800
Message-ID: <CAGXBFArPO=HHy31epb9LpEipd8dB4a88YY550-=v13k9Y-XPjg@mail.gmail.com>
Subject: Paste more than two files
To: coreutils@gnu.org
Content-Type: text/plain; charset=UTF-8
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.85.213.169
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 12 Feb 2012 02:37:09 -0000

Hi Coreutils,

I posted to the list about a month ago and haven't gotten any
response. Perhaps it got overlooked because of the holidays (or
post-holiday email glut), so I'm reposting it:

---

Hello,

I fairly recently discovered the joys of join, but now I wonder why it
is limited to two files?

In other words, I would like to do the following:

join file1 file2 ... fileN

While I CAN achieve this through other methods, they are not ideal.
For instance, paste works with multiple files, but then I must cut out
the repeated key columns. The following also works, but doesn't
generalize to filename expansions (e.g. `join file*`):

join file1 file2 | join - file3 | ... | join - fileN

As for my use case, I am working with data files containing the
results of multiple systems running over the same test items. I would
like to compare the results of all systems for each item by putting
them side-by-side.

I don't know the history of the command, so I am not aware of any
technical or ideological reasons why it shouldn't support more than
two files. Any explanation appreciated!

---

Thanks, and I'm looking forward to a response!

-- 
-Michael Wayne Goodman


From MAILER-DAEMON Sun Feb 12 05:41:07 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RwWrj-0002RU-JC
	for mharc-coreutils@gnu.org; Sun, 12 Feb 2012 05:41:07 -0500
Received: from eggs.gnu.org ([140.186.70.92]:50115)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RwWrh-0002QA-1i
	for coreutils@gnu.org; Sun, 12 Feb 2012 05:41:05 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RwWrf-0002sJ-Sx
	for coreutils@gnu.org; Sun, 12 Feb 2012 05:41:04 -0500
Received: from mail3.vodafone.ie ([213.233.128.45]:8561)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RwWrf-0002s3-Li
	for coreutils@gnu.org; Sun, 12 Feb 2012 05:41:03 -0500
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: ApMBAGyWN09tTC6//2dsb2JhbAAMNoUQnh2PQQEBAQQjDwFGEAsNCwICBRYLAgIJAwIBAgEPNgYNAQUCAQGuRJBigS+HFYJ7DwgbCAUFBwQCBAMECgECBwUGAQMIAQ0Zg2WDAYEWBJsuhRSHVA
Received: from unknown (HELO [192.168.1.79]) ([109.76.46.191])
	by mail3.vodafone.ie with ESMTP; 12 Feb 2012 10:41:01 +0000
Message-ID: <4F37973B.7030706@draigBrady.com>
Date: Sun, 12 Feb 2012 10:40:59 +0000
From: =?UTF-8?B?UMOhZHJhaWcgQnJhZHk=?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:6.0) Gecko/20110816 Thunderbird/6.0
MIME-Version: 1.0
To: "goodman.m.w@gmail.com" <goodman.m.w@gmail.com>
Subject: Re: Paste more than two files
References: <CAGXBFArPO=HHy31epb9LpEipd8dB4a88YY550-=v13k9Y-XPjg@mail.gmail.com>
In-Reply-To: <CAGXBFArPO=HHy31epb9LpEipd8dB4a88YY550-=v13k9Y-XPjg@mail.gmail.com>
X-Enigmail-Version: 1.3.2
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 213.233.128.45
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 12 Feb 2012 10:41:06 -0000

On 02/12/2012 02:36 AM, goodman.m.w@gmail.com wrote:
> Hi Coreutils,
> 
> I posted to the list about a month ago and haven't gotten any
> response. Perhaps it got overlooked because of the holidays (or
> post-holiday email glut), so I'm reposting it:
> 
> ---
> 
> Hello,
> 
> I fairly recently discovered the joys of join, but now I wonder why it
> is limited to two files?
> 
> In other words, I would like to do the following:
> 
> join file1 file2 ... fileN
> 
> While I CAN achieve this through other methods, they are not ideal.
> For instance, paste works with multiple files, but then I must cut out
> the repeated key columns. The following also works, but doesn't
> generalize to filename expansions (e.g. `join file*`):
> 
> join file1 file2 | join - file3 | ... | join - fileN
> 
> As for my use case, I am working with data files containing the
> results of multiple systems running over the same test items. I would
> like to compare the results of all systems for each item by putting
> them side-by-side.
> 
> I don't know the history of the command, so I am not aware of any
> technical or ideological reasons why it shouldn't support more than
> two files. Any explanation appreciated!

Sorry I missed your previous mail.

Well a join operation is not scalable across many files.
You'd have to stream from each file (hence keep file
descriptors open), and maintain internal comparison buffers
for each file. Hence join is restricted to 2 "tables",
which can be combined more generally as you've shown above.

Note for many files, and to handle globbing,
you might want to use temp files, which would
also help with scalability, like:

base=$(mktemp)
cp base $base
for file in file*; do
   nbase=$(mktemp)
  join $base $file > $nbase
  rm $base
done
cat $nbase
rm $nbase

cheers,
PÃ¡draig.


From MAILER-DAEMON Sun Feb 12 08:38:51 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RwZdj-0001y6-Oc
	for mharc-coreutils@gnu.org; Sun, 12 Feb 2012 08:38:51 -0500
Received: from eggs.gnu.org ([140.186.70.92]:48330)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1RwZdh-0001vJ-Ce
	for coreutils@gnu.org; Sun, 12 Feb 2012 08:38:50 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1RwZdg-0002Yi-7R
	for coreutils@gnu.org; Sun, 12 Feb 2012 08:38:49 -0500
Received: from mx.meyering.net ([88.168.87.75]:48826)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1RwZdf-0002WO-VU
	for coreutils@gnu.org; Sun, 12 Feb 2012 08:38:48 -0500
Received: from rho.meyering.net (localhost.localdomain [127.0.0.1])
	by rho.meyering.net (Acme Bit-Twister) with ESMTP id 3909460149
	for <coreutils@gnu.org>; Sun, 12 Feb 2012 14:38:46 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: coreutils@gnu.org
Subject: [PATCH] build: correct @@-,
	length values in last hunk of regcomp.c.diff
Date: Sun, 12 Feb 2012 14:38:46 +0100
Message-ID: <87zkco893t.fsf@rho.meyering.net>
Lines: 34
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 88.168.87.75
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 12 Feb 2012 13:38:50 -0000

My updated-to-match-gnulib regcomp.c.diff file had bogus @@-,length values.
That made bootstrap fail.  Thanks to the hydra auto-builder for
reporting the failure so promptly:

    http://hydra.nixos.org/jobset/gnu/coreutils-master


>From da2b32e7af52f41501ded604dd84c1078583a8d0 Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering@redhat.com>
Date: Sun, 12 Feb 2012 14:37:00 +0100
Subject: [PATCH] build: correct @@-,length values in last hunk of
 regcomp.c.diff

* gl/lib/regcomp.c.diff: Correct invalid hunk lengths (s/1/7/g)
in the last hunk's @@-line.
---
 gl/lib/regcomp.c.diff |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/gl/lib/regcomp.c.diff b/gl/lib/regcomp.c.diff
index d79c7b2..e965673 100644
--- a/gl/lib/regcomp.c.diff
+++ b/gl/lib/regcomp.c.diff
@@ -56,7 +56,7 @@ index d5968bd..4926676 100644
  #else /* not RE_ENABLE_I18N */
  build_equiv_class (bitset_t sbcset, const unsigned char *name)
  #endif /* not RE_ENABLE_I18N */
-@@ -3869,1 +3874,1 @@ free_token (re_token_t *node)
+@@ -3869,7 +3874,7 @@ free_token (re_token_t *node)
     and its children. */

  static reg_errcode_t
--
1.7.9.210.g21ac7a


From MAILER-DAEMON Sun Feb 12 22:40:33 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RwmmH-0002Lj-8u
	for mharc-coreutils@gnu.org; Sun, 12 Feb 2012 22:40:33 -0500
Received: from eggs.gnu.org ([140.186.70.92]:53257)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <pengyu.ut@gmail.com>) id 1RwmmF-0002KK-BZ
	for coreutils@gnu.org; Sun, 12 Feb 2012 22:40:32 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <pengyu.ut@gmail.com>) id 1RwmmE-0003bT-C0
	for coreutils@gnu.org; Sun, 12 Feb 2012 22:40:31 -0500
Received: from mail-tul01m020-f169.google.com ([209.85.214.169]:46085)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <pengyu.ut@gmail.com>) id 1RwmmE-0003bN-8v
	for coreutils@gnu.org; Sun, 12 Feb 2012 22:40:30 -0500
Received: by obbta7 with SMTP id ta7so7369808obb.0
	for <coreutils@gnu.org>; Sun, 12 Feb 2012 19:40:29 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma;
	h=mime-version:date:message-id:subject:from:to:content-type;
	bh=2Mh2ZvjoUqtwXOa20pXfzXDG6JfaFujv4geMSGqFyEg=;
	b=xtWvGUAZDHtpC54umM+3scJuhHtzgaVebpU36Byi/xrHaESsSg2Kgqkc53UkODVbjn
	EuVX79JJVph4852p40UcJ+uOFwq5tIPBLkg3bUCvIG/JApWiMyt4cReJMOYv8+z+N4r7
	6/io0AJZvQ7aNbTJ7Cs30vZ27JdIwnxetCwBE=
MIME-Version: 1.0
Received: by 10.182.76.229 with SMTP id n5mr10581253obw.14.1329104429121; Sun,
	12 Feb 2012 19:40:29 -0800 (PST)
Received: by 10.182.75.202 with HTTP; Sun, 12 Feb 2012 19:40:29 -0800 (PST)
Date: Sun, 12 Feb 2012 21:40:29 -0600
Message-ID: <CABrM6wmmkFpw3QVX7oYQfA_RB7k8yVnDphzsLX8A9Hv6Pzqitw@mail.gmail.com>
Subject: join on multiple field
From: Peng Yu <pengyu.ut@gmail.com>
To: Coreutils <coreutils@gnu.org>
Content-Type: text/plain; charset=ISO-8859-1
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.85.214.169
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 13 Feb 2012 03:40:32 -0000

Hi,

It seems that 'join' only allow joining on 1 field. It will be very
useful to allow multiple field join. Is anybody in the develop team
interested in adding such a feature in join?

http://lists.gnu.org/archive/html/bug-coreutils/2004-03/msg00089.html

-- 
Regards,
Peng


From MAILER-DAEMON Sun Feb 12 22:45:31 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1Rwmr5-0002VA-MQ
	for mharc-coreutils@gnu.org; Sun, 12 Feb 2012 22:45:31 -0500
Received: from eggs.gnu.org ([140.186.70.92]:37122)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <pengyu.ut@gmail.com>) id 1Rwmr2-0002V3-LV
	for coreutils@gnu.org; Sun, 12 Feb 2012 22:45:29 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <pengyu.ut@gmail.com>) id 1Rwmr1-0004FW-4g
	for coreutils@gnu.org; Sun, 12 Feb 2012 22:45:28 -0500
Received: from mail-tul01m020-f169.google.com ([209.85.214.169]:41576)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <pengyu.ut@gmail.com>) id 1Rwmr0-0004FS-Ut
	for coreutils@gnu.org; Sun, 12 Feb 2012 22:45:27 -0500
Received: by obbta7 with SMTP id ta7so7373675obb.0
	for <coreutils@gnu.org>; Sun, 12 Feb 2012 19:45:26 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma;
	h=mime-version:in-reply-to:references:date:message-id:subject:from:to
	:cc:content-type:content-transfer-encoding;
	bh=ZnM68h9YJeaLcmSnJs53pz6q3kffsUt66atHJhZriO8=;
	b=l3WaYPWcbI1S/Opzc667NQZUz1kMTUAbW/cJNfUUmTbm6wirhoiPLtgGMgy/uT14H/
	NVmIro9eIrRWBWbpYh0ynLruecD75I3ceCmOUxNVfBXKr3IxdkhLpPXtzlYD7hrCkkf6
	QP9cRWcHmrsZygn80/Hu+fSORk8QSg25vKmLw=
MIME-Version: 1.0
Received: by 10.182.76.229 with SMTP id n5mr10589542obw.14.1329104726431; Sun,
	12 Feb 2012 19:45:26 -0800 (PST)
Received: by 10.182.75.202 with HTTP; Sun, 12 Feb 2012 19:45:26 -0800 (PST)
In-Reply-To: <4F30F2D0.6010009@draigBrady.com>
References: <CABrM6wmAm=0btpW0JT_A4=8W-bi3PAZ8SDHv1sCjcBOXyQeimQ@mail.gmail.com>
	<4F30F2D0.6010009@draigBrady.com>
Date: Sun, 12 Feb 2012 21:45:26 -0600
Message-ID: <CABrM6w=WteqGuyPB0R_+UjgNnwaqGR2UtFN0YcwwO_UbWmbhtg@mail.gmail.com>
Subject: Re: Adding humanize_number to coreutiles?
From: Peng Yu <pengyu.ut@gmail.com>
To: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigbrady.com>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.85.214.169
Cc: Coreutils <coreutils@gnu.org>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 13 Feb 2012 03:45:29 -0000

2012/2/7 P=E1draig Brady <P@draigbrady.com>:
> On 02/07/2012 03:36 AM, Peng Yu wrote:
>> Hi,
>>
>> Several commands in coreutils have the -h option. I'm wondering
>> whether anybody in the develop team also thinks that it is worthwhile
>> to export it as a standalone command. If so, I'd recommend add such
>> convenient command in coreutiles. As I don't find it anywhere else as
>> a stand alone command.
>>
>> http://siarzhuk.dyndns.org/haiku/doxygen/coreutils_2lib_2human_8c_source=
.html#l00154
>
> I've needed such functionality many times.
> I'm thinking a printf format would be best to expose this:
> http://lists.gnu.org/archive/html/coreutils/2011-08/msg00029.html
>
> %H seems like it might cause compat problems in future.
> %{human} is more descriptive and extensible, so I'm leaning towards that.
> Any other suggestions appreciated.
>
> I'll work on it this week.

I'm not sure %{human} is enough for configuring all the possible ways
of printing a humanized number. See my code below, there are binary
and decimal humanized numbers. Also, you'd better allow a space (or
not) between the numbers and the letters (such as 'T', 'G').

Also embedding it in printf will make it hard to be found, I'd
recommend to create a new command like humanizenumber, just as I did.

/tmp$ cat `which humanizenumber.sh `
#!/usr/bin/env bash

script_name=3D`basename "$0" .sh`

TEMP=3D`getopt -o hbd:sn --long
help,binary,number_of_decimal_places:,space,newline -n
"${script_name}.sh" -- "$@"`

if [ $? !=3D 0 ] ; then printf "Terminating...\n" >&2 ; exit 1 ; fi

eval set -- "$TEMP"

abspath_script=3D`readlink -f -e "$0"`
script_absdir=3D`dirname "$abspath_script"`

number_of_decimal_places=3D0
while true ; do
  case "$1" in
    -h|--help)
      cat "$script_absdir"/${script_name}_help.txt
      exit
      ;;
    -b|--binary)
      binary=3Dx
      shift
      ;;
    -d|--number_of_decimal_places)
      number_of_decimal_places=3D"$2"
      shift 2
      ;;
    -s|--space)
      space=3D' '
      shift
      ;;
    -n|--newline)
      newline=3D'\n'
      shift
      ;;
    --)
      shift
      break
      ;;
    *)
      printf "Internal error!\n">&2
      exit 1
      ;;
  esac
done

if [ $# -ne 0 ]
then
  n=3D"$1"
  if [ -n "$binary" ]
  then
    awk -v sum=3D$n \
      -v space=3D"$space" \
      -v newline=3D"$newline" \
      -v number_of_decimal_places=3D$number_of_decimal_places '
    BEGIN{
    hum[1024**5]=3D"P"
    hum[1024**4]=3D"T"
    hum[1024**3]=3D"G"
    hum[1024**2]=3D"M"
    hum[1024]=3D"K"
    for (x=3D1024**5; x>=3D1024; x/=3D1024) {
      if (sum>=3Dx) {
        printf "%." number_of_decimal_places "f" space "%s" newline,
sum/x, hum[x]
        break
      }
    }
  }'
else
  awk -v sum=3D$n \
    -v space=3D"$space" \
    -v newline=3D"$newline" \
    -v number_of_decimal_places=3D$number_of_decimal_places '
  BEGIN{
  hum[1000**5]=3D"P"
  hum[1000**4]=3D"T"
  hum[1000**3]=3D"G"
  hum[1000**2]=3D"M"
  hum[1000]=3D"k"
  for (x=3D1000**5; x>=3D1000; x/=3D1000) {
    if (sum>=3Dx) {
      printf "%." number_of_decimal_places "f" space "%s" newline, sum/x, h=
um[x]
      break
    }
  }
}'
  fi
fi
/tmp$  humanizenumber.sh -h
Description:
  Humanize number(s)

Usage:
  humanizenumber.sh [Options] [NUMBER]

    NUMBER                            If not specifed, then do nothing.

Options:
  -h|--help                           Help message.
  -b|--binary                         Default: decimal.
  -s|--space                          Default: nospace.

Examples:
  humanizenumber.sh 456456456
  humanizenumber.sh -d 2 456456456
  humanizenumber.sh -s 456456456
  humanizenumber.sh -b 456456456
  humanizenumber.sh -n 456456456

Author:
  Peng Yu <pengyu.ut@gmail.com>



--=20
Regards,
Peng


From MAILER-DAEMON Mon Feb 13 01:16:10 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RwpCs-0005Xo-KP
	for mharc-coreutils@gnu.org; Mon, 13 Feb 2012 01:16:10 -0500
Received: from eggs.gnu.org ([140.186.70.92]:50941)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <pengyu.ut@gmail.com>) id 1RwpCp-0005Xi-TW
	for coreutils@gnu.org; Mon, 13 Feb 2012 01:16:08 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <pengyu.ut@gmail.com>) id 1RwpCo-0001vt-Kb
	for coreutils@gnu.org; Mon, 13 Feb 2012 01:16:07 -0500
Received: from mail-tul01m020-f169.google.com ([209.85.214.169]:61704)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <pengyu.ut@gmail.com>) id 1RwpCo-0001vp-E8
	for coreutils@gnu.org; Mon, 13 Feb 2012 01:16:06 -0500
Received: by obbta7 with SMTP id ta7so7520330obb.0
	for <coreutils@gnu.org>; Sun, 12 Feb 2012 22:16:06 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma;
	h=mime-version:date:message-id:subject:from:to:content-type
	:content-transfer-encoding;
	bh=V5X7onxCWRvEKmMx3g2JHfN3OX3znBKxn1ak58El1Ow=;
	b=JSfmsvowRev4s7rxT6xPXCOjKA/I8GmZ4aLyXR9G1yFGSuNDTWdQvb57cbSRPG+AZ4
	DcrLCxn3rhDaro2j+mbTnq8Wb/6QDc11Ps+TK8ZwleUUvKLt7HDSpwisEM6ObUB65MQw
	tzRxi16Cbr7HL8j8UdgnGsDtWxaOH/zlLWxGw=
MIME-Version: 1.0
Received: by 10.182.144.68 with SMTP id sk4mr10992551obb.4.1329113765993; Sun,
	12 Feb 2012 22:16:05 -0800 (PST)
Received: by 10.182.75.202 with HTTP; Sun, 12 Feb 2012 22:16:05 -0800 (PST)
Date: Mon, 13 Feb 2012 00:16:05 -0600
Message-ID: <CABrM6wm_=7rZkyA7jNCyaLzS+_GpFrciuLcgky7bgNrdUO4esA@mail.gmail.com>
Subject: Confusion in 'join' manual?
From: Peng Yu <pengyu.ut@gmail.com>
To: Coreutils <coreutils@gnu.org>
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.85.214.169
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 13 Feb 2012 06:16:09 -0000

Hi,

The following description is confusing to me. Since "--nocheck-order"
is default, I'm wondering how can I not to give it?

I tried specifying --nocheck-order in the command line, the warnings
disappear. So by default, --nocheck-order is not specified?

".. If the option =91--nocheck-order=92 is given, unsorted inputs will
never cause an error message.
If neither of these options is given, wrongly sorted inputs are
diagnosed only if an input file
is found to contain unpairable lines, and when both input files are non emp=
ty."

"=91--nocheck-order=92
Do not check that both input files are in sorted order. This is the default=
."

Suppose that the joining field in both files are sorted by some
ordering that 'join' don't know (for example, the joining field is a
concatenation of a string and a number), what options should be used
to join files with such ordered joining fields?

--=20
Regards,
Peng


From MAILER-DAEMON Mon Feb 13 02:50:34 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RwqgE-0000Mb-Ea
	for mharc-coreutils@gnu.org; Mon, 13 Feb 2012 02:50:34 -0500
Received: from eggs.gnu.org ([140.186.70.92]:36704)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1RwqgB-0000Jn-Tw
	for coreutils@gnu.org; Mon, 13 Feb 2012 02:50:32 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1Rwqg8-0006wT-NZ
	for coreutils@gnu.org; Mon, 13 Feb 2012 02:50:31 -0500
Received: from moutng.kundenserver.de ([212.227.126.171]:65375)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1Rwqg8-0006vx-Fb
	for coreutils@gnu.org; Mon, 13 Feb 2012 02:50:28 -0500
Received: from [192.168.2.108] (p4FF753D4.dip.t-dialin.net [79.247.83.212])
	by mrelayeu.kundenserver.de (node=mreu0) with ESMTP (Nemesis)
	id 0LfWft-1SGqV71JYv-00p3MX; Mon, 13 Feb 2012 08:50:24 +0100
Message-ID: <4F38C0BF.2010205@bernhard-voelker.de>
Date: Mon, 13 Feb 2012 08:50:23 +0100
From: Bernhard Voelker <mail@bernhard-voelker.de>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:10.0) Gecko/20120129 Thunderbird/10.0
MIME-Version: 1.0
To: =?ISO-8859-1?Q?J=E9r=E9my_Compostella?=
 <jeremy.compostella@gmail.com>
Subject: Re: dd: add 'skip_bytes' and 'count_bytes' operands
References: <87mx8w3ryu.fsf@Apollo.jerryland.fr>
	<4F2FF5EE.9010906@draigBrady.com>
	<CA+TATJF4w4Ue44Za4hgGR_0RxgEZGBm5nzhHRSdGYxzzfKWNLg@mail.gmail.com>
	<4F317CDF.2040909@draigBrady.com>
	<CA+TATJFU7yOGzzeZseHUFTqxxGAtz8zUoQccz=5m0Eu=CqEtRQ@mail.gmail.com>
	<4F329FC9.4030403@draigBrady.com>
	<CA+TATJFkn8=Kd3iqrtk=z2pkkzKRGwnAsquw61r0kqSMx76zgQ@mail.gmail.com>
	<CA+TATJF2uW6eHqF4fkwacHq4xm1XPY=3Ent3RHDNPuMA8GOZVA@mail.gmail.com>
In-Reply-To: <CA+TATJF2uW6eHqF4fkwacHq4xm1XPY=3Ent3RHDNPuMA8GOZVA@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
X-Provags-ID: V02:K0:6kmF/YSAeCw8du8ns8ISyRwxISMV/1tMjVgmhH94YIB
	VXJ94iGRt1M0Og2s+CNKW/W852Vr6ZmMV90uLKTMvXu0OaBbPu
	kgbhFEj111ANwKCcXDSu3NmWKNTyj5mNwPkwnhcYZu5iHhqDU5
	fuDHiUWSNJ6SM7SZ7Rt7Nq6bTv5OTDwFrKt6ate49EFcvWRptr
	z0PAkE3dN2v0inzug2Yc9c8yEaqJDYorO/P/3Zq3fHzGglaiw5
	LIhKUsDP9cG08bfHdjREALOqazaigI4WhllD+R3xq2bMR/as2g
	TJL6jd/M8rHclmWPcM6rtYSrJuT3ou3hih0um/nsRX66NCs/Sp
	frLRkiC/8UfVT2uE4efuInf5RaDyboA9EhVg8rCcr
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 212.227.126.171
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 13 Feb 2012 07:50:32 -0000

On 02/10/2012 05:21 PM, Jérémy Compostella wrote:

Sorry, this is a bit late, but I have 2 notes about the
new dd/bytes test:

> diff --git a/tests/dd/bytes b/tests/dd/bytes
> new file mode 100755
> index 0000000..6038742
> --- /dev/null
> +++ b/tests/dd/bytes
> @@ -0,0 +1,57 @@

> +# seek bytes
> +echo abcdefghijklm |
> + dd bs=5 seek=8 oflag=seek_bytes > out 2> /dev/null || fail=1
> +echo abcdefghijklm |
> + dd bs=4 seek=2 > expected 2> /dev/null || fail=1
> +compare expected out || fail=1

I don't know why, but I somehow do not have a good feeling
about a test comparing the result of the program to be tested
with another run. If something cowardly fails, then this test
would succeed.

> +# seek bytes on empty file
> +echo abcdefghijklm |
> + dd bs=5 seek=8 oflag=seek_bytes > out2 2> /dev/null || fail=1
> +compare expected out2 || fail=1

The comment is wrong here, isn't it?

Have a nice day,
Berny


From MAILER-DAEMON Mon Feb 13 05:00:54 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RwsiM-0003aK-Qs
	for mharc-coreutils@gnu.org; Mon, 13 Feb 2012 05:00:54 -0500
Received: from eggs.gnu.org ([140.186.70.92]:60091)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RwsiF-0003XZ-Nm
	for coreutils@gnu.org; Mon, 13 Feb 2012 05:00:52 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Rwsi9-0004e3-Gw
	for coreutils@gnu.org; Mon, 13 Feb 2012 05:00:47 -0500
Received: from mail1.vodafone.ie ([213.233.128.43]:53541)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Rwsi9-0004dS-8R
	for coreutils@gnu.org; Mon, 13 Feb 2012 05:00:41 -0500
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: ApkBAP3eOE9daxc1/2dsb2JhbAAMN61BhTQBAQEEDCYBRhALDQEKCRYPCQMCAQIBRQYNAQcBAcBgjAoGBAMEBAcOBgEDCAENGYNlhBcEmy6DOIkw
Received: from unknown (HELO [192.168.1.79]) ([93.107.23.53])
	by mail1.vodafone.ie with ESMTP; 13 Feb 2012 10:00:37 +0000
Message-ID: <4F38DF43.2000106@draigBrady.com>
Date: Mon, 13 Feb 2012 10:00:35 +0000
From: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:6.0) Gecko/20110816 Thunderbird/6.0
MIME-Version: 1.0
To: Bernhard Voelker <mail@bernhard-voelker.de>
Subject: Re: dd: add 'skip_bytes' and 'count_bytes' operands
References: <87mx8w3ryu.fsf@Apollo.jerryland.fr>
	<4F2FF5EE.9010906@draigBrady.com>
	<CA+TATJF4w4Ue44Za4hgGR_0RxgEZGBm5nzhHRSdGYxzzfKWNLg@mail.gmail.com>
	<4F317CDF.2040909@draigBrady.com>
	<CA+TATJFU7yOGzzeZseHUFTqxxGAtz8zUoQccz=5m0Eu=CqEtRQ@mail.gmail.com>
	<4F329FC9.4030403@draigBrady.com>
	<CA+TATJFkn8=Kd3iqrtk=z2pkkzKRGwnAsquw61r0kqSMx76zgQ@mail.gmail.com>
	<CA+TATJF2uW6eHqF4fkwacHq4xm1XPY=3Ent3RHDNPuMA8GOZVA@mail.gmail.com>
	<4F38C0BF.2010205@bernhard-voelker.de>
In-Reply-To: <4F38C0BF.2010205@bernhard-voelker.de>
X-Enigmail-Version: 1.3.2
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 213.233.128.43
Cc: =?ISO-8859-1?Q?J=E9r=E9my_Compostella?= <jeremy.compostella@gmail.com>,
	coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 13 Feb 2012 10:00:52 -0000

On 02/13/2012 07:50 AM, Bernhard Voelker wrote:
> On 02/10/2012 05:21 PM, Jérémy Compostella wrote:
> 
> Sorry, this is a bit late, but I have 2 notes about the
> new dd/bytes test:
> 
>> diff --git a/tests/dd/bytes b/tests/dd/bytes
>> new file mode 100755
>> index 0000000..6038742
>> --- /dev/null
>> +++ b/tests/dd/bytes
>> @@ -0,0 +1,57 @@
> 
>> +# seek bytes
>> +echo abcdefghijklm |
>> + dd bs=5 seek=8 oflag=seek_bytes > out 2> /dev/null || fail=1
>> +echo abcdefghijklm |
>> + dd bs=4 seek=2 > expected 2> /dev/null || fail=1
>> +compare expected out || fail=1
> 
> I don't know why, but I somehow do not have a good feeling
> about a test comparing the result of the program to be tested
> with another run. If something cowardly fails, then this test
> would succeed.

Fair point.

>> +# seek bytes on empty file
>> +echo abcdefghijklm |
>> + dd bs=5 seek=8 oflag=seek_bytes > out2 2> /dev/null || fail=1
>> +compare expected out2 || fail=1
> 
> The comment is wrong here, isn't it?

Well out2 will be a new empty file.
Though the case is no different to the previous
(since conv=notrunc was not specified).
So I'll just remove it.

cheers,
Pádraig.

diff --git a/tests/dd/bytes b/tests/dd/bytes
index 6038742..10f3618 100755
--- a/tests/dd/bytes
+++ b/tests/dd/bytes
@@ -45,13 +45,7 @@ esac
 # seek bytes
 echo abcdefghijklm |
  dd bs=5 seek=8 oflag=seek_bytes > out 2> /dev/null || fail=1
-echo abcdefghijklm |
- dd bs=4 seek=2 > expected 2> /dev/null || fail=1
+printf '\0\0\0\0\0\0\0\0abcdefghijklm\n' > expected
 compare expected out || fail=1

-# seek bytes on empty file
-echo abcdefghijklm |
- dd bs=5 seek=8 oflag=seek_bytes > out2 2> /dev/null || fail=1
-compare expected out2 || fail=1
-
 Exit $fail


From MAILER-DAEMON Mon Feb 13 05:13:26 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RwsuU-0005dO-Tf
	for mharc-coreutils@gnu.org; Mon, 13 Feb 2012 05:13:26 -0500
Received: from eggs.gnu.org ([140.186.70.92]:49000)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1RwsuP-0005aK-26
	for coreutils@gnu.org; Mon, 13 Feb 2012 05:13:25 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1RwsuI-0006fa-Fn
	for coreutils@gnu.org; Mon, 13 Feb 2012 05:13:21 -0500
Received: from moutng.kundenserver.de ([212.227.126.186]:53199)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1RwsuI-0006fN-42
	for coreutils@gnu.org; Mon, 13 Feb 2012 05:13:14 -0500
Received: from [192.168.2.108] (p4FF753D4.dip.t-dialin.net [79.247.83.212])
	by mrelayeu.kundenserver.de (node=mrbap4) with ESMTP (Nemesis)
	id 0MXGy5-1S2ZwQ436L-00WwiG; Mon, 13 Feb 2012 11:13:12 +0100
Message-ID: <4F38E236.8050903@bernhard-voelker.de>
Date: Mon, 13 Feb 2012 11:13:10 +0100
From: Bernhard Voelker <mail@bernhard-voelker.de>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:10.0) Gecko/20120129 Thunderbird/10.0
MIME-Version: 1.0
To: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
Subject: Re: dd: add 'skip_bytes' and 'count_bytes' operands
References: <87mx8w3ryu.fsf@Apollo.jerryland.fr>
	<4F2FF5EE.9010906@draigBrady.com>
	<CA+TATJF4w4Ue44Za4hgGR_0RxgEZGBm5nzhHRSdGYxzzfKWNLg@mail.gmail.com>
	<4F317CDF.2040909@draigBrady.com>
	<CA+TATJFU7yOGzzeZseHUFTqxxGAtz8zUoQccz=5m0Eu=CqEtRQ@mail.gmail.com>
	<4F329FC9.4030403@draigBrady.com>
	<CA+TATJFkn8=Kd3iqrtk=z2pkkzKRGwnAsquw61r0kqSMx76zgQ@mail.gmail.com>
	<CA+TATJF2uW6eHqF4fkwacHq4xm1XPY=3Ent3RHDNPuMA8GOZVA@mail.gmail.com>
	<4F38C0BF.2010205@bernhard-voelker.de>
	<4F38DF43.2000106@draigBrady.com>
In-Reply-To: <4F38DF43.2000106@draigBrady.com>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
X-Provags-ID: V02:K0:961Poir22y6xqc9wczp1I4Ur3cFKxj1PjTNAPnw1/XU
	DTMwAKv5R4Cm/hsnDGi4XQXQPCA64W0vNRFJeEznnKIG/XmwwM
	oUaBAJdy2dTOaS4w/e5qKb0/v8av1kcpskf6NS6xneCHJel9NN
	V8X0n5vhJ93RPWGTMg+ZhTeFZ4mfkDHB0Ram7GALJUgg+/HaAh
	Zv32FEKEK+bxe1tYGAh/2fsIbiYqMaKACzu1/rTzOKbHw304yh
	DlYPoPN1j6aA5VZiRAXWNyg9VOgQyRzMisd0OOf20Uv+rGH/XJ
	Fqf6EsVFJzyMmHPe3yynVRQMQqXvduTAYy+QOPK3UczZJZk3AR
	i+P2Nzja8/bPRD5ifh6S4M+APUdl2OSicugXp8rOX
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 212.227.126.186
Cc: =?ISO-8859-1?Q?J=E9r=E9my_Compostella?= <jeremy.compostella@gmail.com>,
	coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 13 Feb 2012 10:13:25 -0000



On 02/13/2012 11:00 AM, Pádraig Brady wrote:
> On 02/13/2012 07:50 AM, Bernhard Voelker wrote:
>> On 02/10/2012 05:21 PM, Jérémy Compostella wrote:
>>
>> Sorry, this is a bit late, but I have 2 notes about the
>> new dd/bytes test:
>>
>>> diff --git a/tests/dd/bytes b/tests/dd/bytes
>>> new file mode 100755
>>> index 0000000..6038742
>>> --- /dev/null
>>> +++ b/tests/dd/bytes
>>> @@ -0,0 +1,57 @@
>>
>>> +# seek bytes
>>> +echo abcdefghijklm |
>>> + dd bs=5 seek=8 oflag=seek_bytes > out 2> /dev/null || fail=1
>>> +echo abcdefghijklm |
>>> + dd bs=4 seek=2 > expected 2> /dev/null || fail=1
>>> +compare expected out || fail=1
>>
>> I don't know why, but I somehow do not have a good feeling
>> about a test comparing the result of the program to be tested
>> with another run. If something cowardly fails, then this test
>> would succeed.
> 
> Fair point.

Thanks for the patch.

>>> +# seek bytes on empty file
>>> +echo abcdefghijklm |
>>> + dd bs=5 seek=8 oflag=seek_bytes > out2 2> /dev/null || fail=1
>>> +compare expected out2 || fail=1
>>
>> The comment is wrong here, isn't it?
> 
> Well out2 will be a new empty file.
> Though the case is no different to the previous
> (since conv=notrunc was not specified).
> So I'll just remove it.

Ah, I see. Thanks for the clarification.

BTW: shouldn't
   echo abcdefghijklm | src/dd bs=5 seek=8 oflag=seek_bytes
receive EPIPE? When run on a terminal, dd waits in read() forever.

Have a nice day,
Berny


From MAILER-DAEMON Mon Feb 13 08:09:53 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RwvfF-0002lF-Sr
	for mharc-coreutils@gnu.org; Mon, 13 Feb 2012 08:09:53 -0500
Received: from eggs.gnu.org ([140.186.70.92]:54426)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Rwvf8-0002hB-LT
	for coreutils@gnu.org; Mon, 13 Feb 2012 08:09:50 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Rwvf4-0000zm-6u
	for coreutils@gnu.org; Mon, 13 Feb 2012 08:09:46 -0500
Received: from mail1.vodafone.ie ([213.233.128.43]:51054)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Rwvf4-0000zS-1y
	for coreutils@gnu.org; Mon, 13 Feb 2012 08:09:42 -0500
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: ApMBAFgKOU9daxc1/2dsb2JhbAAMN61NhTMBAQEEDCYBRhALDQEKCRYPCQMCAQIBRQYNAQcBAcEIi04wDAYEAwQEBw4GAQMIAQ0Zg2WEFwSbLoM4iTA
Received: from unknown (HELO [192.168.1.79]) ([93.107.23.53])
	by mail1.vodafone.ie with ESMTP; 13 Feb 2012 13:09:39 +0000
Message-ID: <4F390B91.5070903@draigBrady.com>
Date: Mon, 13 Feb 2012 13:09:37 +0000
From: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:6.0) Gecko/20110816 Thunderbird/6.0
MIME-Version: 1.0
To: Bernhard Voelker <mail@bernhard-voelker.de>
Subject: Re: dd: add 'skip_bytes' and 'count_bytes' operands
References: <87mx8w3ryu.fsf@Apollo.jerryland.fr>
	<4F2FF5EE.9010906@draigBrady.com>
	<CA+TATJF4w4Ue44Za4hgGR_0RxgEZGBm5nzhHRSdGYxzzfKWNLg@mail.gmail.com>
	<4F317CDF.2040909@draigBrady.com>
	<CA+TATJFU7yOGzzeZseHUFTqxxGAtz8zUoQccz=5m0Eu=CqEtRQ@mail.gmail.com>
	<4F329FC9.4030403@draigBrady.com>
	<CA+TATJFkn8=Kd3iqrtk=z2pkkzKRGwnAsquw61r0kqSMx76zgQ@mail.gmail.com>
	<CA+TATJF2uW6eHqF4fkwacHq4xm1XPY=3Ent3RHDNPuMA8GOZVA@mail.gmail.com>
	<4F38C0BF.2010205@bernhard-voelker.de>
	<4F38DF43.2000106@draigBrady.com>
	<4F38E236.8050903@bernhard-voelker.de>
In-Reply-To: <4F38E236.8050903@bernhard-voelker.de>
X-Enigmail-Version: 1.3.2
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 213.233.128.43
Cc: =?ISO-8859-1?Q?J=E9r=E9my_Compostella?= <jeremy.compostella@gmail.com>,
	coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 13 Feb 2012 13:09:50 -0000

On 02/13/2012 10:13 AM, Bernhard Voelker wrote:
> 
> 
> On 02/13/2012 11:00 AM, Pádraig Brady wrote:
>> On 02/13/2012 07:50 AM, Bernhard Voelker wrote:
>>> On 02/10/2012 05:21 PM, Jérémy Compostella wrote:
>>>
>>> Sorry, this is a bit late, but I have 2 notes about the
>>> new dd/bytes test:
>>>
>>>> diff --git a/tests/dd/bytes b/tests/dd/bytes
>>>> new file mode 100755
>>>> index 0000000..6038742
>>>> --- /dev/null
>>>> +++ b/tests/dd/bytes
>>>> @@ -0,0 +1,57 @@
>>>
>>>> +# seek bytes
>>>> +echo abcdefghijklm |
>>>> + dd bs=5 seek=8 oflag=seek_bytes > out 2> /dev/null || fail=1
>>>> +echo abcdefghijklm |
>>>> + dd bs=4 seek=2 > expected 2> /dev/null || fail=1
>>>> +compare expected out || fail=1
>>>
>>> I don't know why, but I somehow do not have a good feeling
>>> about a test comparing the result of the program to be tested
>>> with another run. If something cowardly fails, then this test
>>> would succeed.
>>
>> Fair point.
> 
> Thanks for the patch.
> 
>>>> +# seek bytes on empty file
>>>> +echo abcdefghijklm |
>>>> + dd bs=5 seek=8 oflag=seek_bytes > out2 2> /dev/null || fail=1
>>>> +compare expected out2 || fail=1
>>>
>>> The comment is wrong here, isn't it?
>>
>> Well out2 will be a new empty file.
>> Though the case is no different to the previous
>> (since conv=notrunc was not specified).
>> So I'll just remove it.
> 
> Ah, I see. Thanks for the clarification.
> 
> BTW: shouldn't
>    echo abcdefghijklm | src/dd bs=5 seek=8 oflag=seek_bytes
> receive EPIPE? When run on a terminal, dd waits in read() forever.

True. That's because we try to seek using read()
on non seekable files, and stdout is readable by default,
so we'll hang waiting for input from the terminal.
I.E. this will block:

  dd seek=1 if=/dev/null

Not sure what to do there.

cheers,
Pádraig.


From MAILER-DAEMON Mon Feb 13 08:13:15 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RwviV-0003Vw-LD
	for mharc-coreutils@gnu.org; Mon, 13 Feb 2012 08:13:15 -0500
Received: from eggs.gnu.org ([140.186.70.92]:55125)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RwviN-0003Hr-Lf
	for coreutils@gnu.org; Mon, 13 Feb 2012 08:13:11 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RwviH-0001Th-Oy
	for coreutils@gnu.org; Mon, 13 Feb 2012 08:13:07 -0500
Received: from mail1.vodafone.ie ([213.233.128.43]:41573)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RwviH-0001TV-JI
	for coreutils@gnu.org; Mon, 13 Feb 2012 08:13:01 -0500
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: ApMBAHwLOU9daxc1/2dsb2JhbAAMN61NhTQBAQQyAVYLDRQWDwkDAgECAUUGAQwIAQHBDYwKBgQDBAQHDgYBAwgBDRYDg2WEFwSbLoM4iTA
Received: from unknown (HELO [192.168.1.79]) ([93.107.23.53])
	by mail1.vodafone.ie with ESMTP; 13 Feb 2012 13:13:01 +0000
Message-ID: <4F390C5B.4070307@draigBrady.com>
Date: Mon, 13 Feb 2012 13:12:59 +0000
From: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:6.0) Gecko/20110816 Thunderbird/6.0
MIME-Version: 1.0
To: coreutils@gnu.org, =?ISO-8859-1?Q?J=E9r=E9my_Compostella?=
	<jeremy.compostella@gmail.com>
Subject: Re: dd: add 'skip_bytes' and 'count_bytes' operands
References: <87mx8w3ryu.fsf@Apollo.jerryland.fr>
	<4F2FF5EE.9010906@draigBrady.com>
	<CA+TATJF4w4Ue44Za4hgGR_0RxgEZGBm5nzhHRSdGYxzzfKWNLg@mail.gmail.com>
	<4F317CDF.2040909@draigBrady.com>
	<CA+TATJFU7yOGzzeZseHUFTqxxGAtz8zUoQccz=5m0Eu=CqEtRQ@mail.gmail.com>
	<4F329FC9.4030403@draigBrady.com>
	<CA+TATJFkn8=Kd3iqrtk=z2pkkzKRGwnAsquw61r0kqSMx76zgQ@mail.gmail.com>
	<CA+TATJF2uW6eHqF4fkwacHq4xm1XPY=3Ent3RHDNPuMA8GOZVA@mail.gmail.com>
	<4F354C8E.9040203@draigBrady.com> <4F3712E4.2080101@draigBrady.com>
In-Reply-To: <4F3712E4.2080101@draigBrady.com>
X-Enigmail-Version: 1.3.2
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 213.233.128.43
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 13 Feb 2012 13:13:11 -0000

I've just noticed a couple of edge case issues.

The first hunk is for when not doing I/O,
ftruncate() should be on the full size.

The second, will not invalidate cache for the whole file,
if 0 < count < ibs

cheers,
Pádraig.

--- a/src/dd.c
+++ b/src/dd.c
@@ -2170,7 +2170,7 @@ main (int argc, char **argv)

       if (seek_records != 0 && !(conversions_mask & C_NOTRUNC))
         {
-          uintmax_t size = seek_records * output_blocksize;
+          uintmax_t size = seek_records * output_blocksize + seek_bytes;
           unsigned long int obs = output_blocksize;

           if (OFF_T_MAX / output_blocksize < seek_records)
@@ -2207,7 +2207,7 @@ main (int argc, char **argv)

   exit_status = dd_copy ();

-  if (max_records == 0)
+  if (max_records == 0 && max_bytes == 0)
     {
       /* Special case to invalidate cache to end of file.  */
       if (i_nocache && !invalidate_cache (STDIN_FILENO, 0))



From MAILER-DAEMON Mon Feb 13 08:38:04 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1Rww6W-0005SP-4h
	for mharc-coreutils@gnu.org; Mon, 13 Feb 2012 08:38:04 -0500
Received: from eggs.gnu.org ([140.186.70.92]:47872)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1Rww6O-0005SG-Jj
	for coreutils@gnu.org; Mon, 13 Feb 2012 08:38:02 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1Rww6I-0007fM-Rs
	for coreutils@gnu.org; Mon, 13 Feb 2012 08:37:56 -0500
Received: from mx1.redhat.com ([209.132.183.28]:25265)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1Rww6I-0007fB-L9
	for coreutils@gnu.org; Mon, 13 Feb 2012 08:37:50 -0500
Received: from int-mx10.intmail.prod.int.phx2.redhat.com
	(int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id q1DDbm7F016113
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK);
	Mon, 13 Feb 2012 08:37:48 -0500
Received: from [10.3.113.104] (ovpn-113-104.phx2.redhat.com [10.3.113.104])
	by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP
	id q1DDbmZs023749; Mon, 13 Feb 2012 08:37:48 -0500
Message-ID: <4F391226.3060508@redhat.com>
Date: Mon, 13 Feb 2012 06:37:42 -0700
From: Eric Blake <eblake@redhat.com>
Organization: Red Hat
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:10.0) Gecko/20120131 Thunderbird/10.0
MIME-Version: 1.0
To: Peng Yu <pengyu.ut@gmail.com>, Coreutils <coreutils@gnu.org>
Subject: Re: join on multiple field
References: <CABrM6wmmkFpw3QVX7oYQfA_RB7k8yVnDphzsLX8A9Hv6Pzqitw@mail.gmail.com>
In-Reply-To: <CABrM6wmmkFpw3QVX7oYQfA_RB7k8yVnDphzsLX8A9Hv6Pzqitw@mail.gmail.com>
X-Enigmail-Version: 1.3.5
OpenPGP: url=http://people.redhat.com/eblake/eblake.gpg
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature";
	boundary="------------enigAAA3C7C61C42120B153FA0ED"
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.132.183.28
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 13 Feb 2012 13:38:02 -0000

This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--------------enigAAA3C7C61C42120B153FA0ED
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

On 02/12/2012 08:40 PM, Peng Yu wrote:
> Hi,
>=20
> It seems that 'join' only allow joining on 1 field.

Correct, that's the POSIX-specified behavior.

> It will be very
> useful to allow multiple field join.

Can't you already do this with awk?

> Is anybody in the develop team
> interested in adding such a feature in join?

Not really, unless _you_ want to submit the patches.  It's very hard to
justify bloating 'join' to do something that 'awk' can already do, since
it is easier to find a machine with a solution that works now with awk
than it is to find a machine with a new version of coreutils installed
with your proposed extension to 'join'.

>=20
> http://lists.gnu.org/archive/html/bug-coreutils/2004-03/msg00089.html

The same comments from that thread still apply:

> Send patches. (send unified diffs, please -- i.e. diff -u format) If yo=
ur
> changes fix bugs, the bar is quite low in that I don't need much more
> than to understand what the original problem was.  However, it helps
> a lot if you can give me enough information to reproduce the problem.
>=20
> On the other hand, if you're adding new features, please follow the
> guidelines below:
>=20
>   - convince me that this is a useful change/addition
>       (if you're adding yet another option to ls, the above is pretty h=
ard)
>=20
>   - convince other people of the same thing
>       One way to do that is to send mail to address@hidden

[the mail archive munged that; it would be bug-gnu-utils@gnu.org]

>       (aka the gnu.utils.bug news group) including as much description
>       and justification as you can.  Based on the feedback that generat=
es,
>       you may be able to convince me.

--=20
Eric Blake   eblake@redhat.com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


--------------enigAAA3C7C61C42120B153FA0ED
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)
Comment: Public key at http://people.redhat.com/eblake/eblake.gpg
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQEcBAEBCAAGBQJPORIrAAoJEKeha0olJ0NqILYH/i+N0lJUVbzwt+zMvSP59/HI
j4J86eqPM+iR38qkzasw17PrGJiXFmOY1ozuEigbBf7PZLcIN0oVc9l1n4PvC3vM
b5sfCKuFAnmltEeLvqlALBTA1nwSq4MNunag+da0W77Js1lMrBIK34hl36Z9rBk0
CFF4sesBNM8qP56lQE20cyE3S4VKtdCsxLaGqSEqc8RPLnHajCp/3WQc3C8wTM/v
SStj7Hk3Hn1J5Vpr0wBH4Co87lyq1CFnTIpQERHjkx9Z9/Udj9iY59saho3rVcs5
FiRk4F7ImWJQlLzIZ//QJ+uZXEps6uyqOdd6eMVl3e743RIAjd8VfzkHcyDU9yk=
=UhqM
-----END PGP SIGNATURE-----

--------------enigAAA3C7C61C42120B153FA0ED--


From MAILER-DAEMON Mon Feb 13 08:54:22 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RwwMI-000095-Rf
	for mharc-coreutils@gnu.org; Mon, 13 Feb 2012 08:54:22 -0500
Received: from eggs.gnu.org ([140.186.70.92]:43286)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <pengyu.ut@gmail.com>) id 1RwwMC-00008W-I1
	for coreutils@gnu.org; Mon, 13 Feb 2012 08:54:20 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <pengyu.ut@gmail.com>) id 1RwwM6-00033r-T3
	for coreutils@gnu.org; Mon, 13 Feb 2012 08:54:16 -0500
Received: from mail-tul01m020-f169.google.com ([209.85.214.169]:50487)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <pengyu.ut@gmail.com>) id 1RwwM6-00033k-Kp
	for coreutils@gnu.org; Mon, 13 Feb 2012 08:54:10 -0500
Received: by obbta7 with SMTP id ta7so8158644obb.0
	for <coreutils@gnu.org>; Mon, 13 Feb 2012 05:54:09 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma;
	h=mime-version:in-reply-to:references:date:message-id:subject:from:to
	:cc:content-type;
	bh=WNzzd2b7kTaN0OWs6GkncuesRJnfWAtX+mt2zCEUN+g=;
	b=t5M02B0iVPBeHsjBKhAfQEI/o+KoswlnrPLNdx3558QzSQVK8HUbhIFUoR53oWJM4y
	9O7Yh+ECsLrv2fmKYtYKJsRPGnKgrXOJdXXHgAftQ6RplldiPrYWUhZx4KnXNzmP98/L
	jwr7OudbocL71LLGpeXY0KRMNUBd3VUzNx8oM=
MIME-Version: 1.0
Received: by 10.182.144.68 with SMTP id sk4mr11937825obb.4.1329141249599; Mon,
	13 Feb 2012 05:54:09 -0800 (PST)
Received: by 10.182.75.202 with HTTP; Mon, 13 Feb 2012 05:54:09 -0800 (PST)
In-Reply-To: <4F391226.3060508@redhat.com>
References: <CABrM6wmmkFpw3QVX7oYQfA_RB7k8yVnDphzsLX8A9Hv6Pzqitw@mail.gmail.com>
	<4F391226.3060508@redhat.com>
Date: Mon, 13 Feb 2012 07:54:09 -0600
Message-ID: <CABrM6w=ggV7=i5qmbiAqcsZwRruXTgZRPLknpNbLn4eapTrRYQ@mail.gmail.com>
Subject: Re: join on multiple field
From: Peng Yu <pengyu.ut@gmail.com>
To: Eric Blake <eblake@redhat.com>
Content-Type: text/plain; charset=ISO-8859-1
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.85.214.169
Cc: Coreutils <coreutils@gnu.org>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 13 Feb 2012 13:54:20 -0000

> Can't you already do this with awk?

This is not a very useful comment. Everything that coreutils do can be
done with some other software. Since 'join' can do 1 field join, it is
natural to ask for multiple field join. After all, people use
coreutils for its convenience in certain aspects. Why not make it more
convenient?

The current working flow in coreutils is like from the stone age,
while many other open source projects use more advanced dvcs like
github or bitbucket.

I had tried to send some patch before. Besides the above mentioned
inconvenience, I have to get my signature from GNU office, which they
never reply my email.

If coreutils wants me people to contribute, the patching/submission
processes must be improved. The current process is way inconvenience
for new contributors.

-- 
Regards,
Peng


From MAILER-DAEMON Mon Feb 13 09:28:42 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RwwtW-0006nV-UN
	for mharc-coreutils@gnu.org; Mon, 13 Feb 2012 09:28:42 -0500
Received: from eggs.gnu.org ([140.186.70.92]:54792)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1RwwtU-0006n0-Ud
	for coreutils@gnu.org; Mon, 13 Feb 2012 09:28:41 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1RwwtP-0001Or-96
	for coreutils@gnu.org; Mon, 13 Feb 2012 09:28:40 -0500
Received: from moutng.kundenserver.de ([212.227.126.187]:60382)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1RwwtO-0001OX-Vo
	for coreutils@gnu.org; Mon, 13 Feb 2012 09:28:35 -0500
Received: from [192.168.2.108] (p4FF753D4.dip.t-dialin.net [79.247.83.212])
	by mrelayeu.kundenserver.de (node=mreu2) with ESMTP (Nemesis)
	id 0MQrwA-1S5Hqg3Z8A-00UlF4; Mon, 13 Feb 2012 15:28:33 +0100
Message-ID: <4F391E0F.5000302@bernhard-voelker.de>
Date: Mon, 13 Feb 2012 15:28:31 +0100
From: Bernhard Voelker <mail@bernhard-voelker.de>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:10.0) Gecko/20120129 Thunderbird/10.0
MIME-Version: 1.0
To: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
Subject: Re: dd: add 'skip_bytes' and 'count_bytes' operands
References: <87mx8w3ryu.fsf@Apollo.jerryland.fr>
	<4F2FF5EE.9010906@draigBrady.com>
	<CA+TATJF4w4Ue44Za4hgGR_0RxgEZGBm5nzhHRSdGYxzzfKWNLg@mail.gmail.com>
	<4F317CDF.2040909@draigBrady.com>
	<CA+TATJFU7yOGzzeZseHUFTqxxGAtz8zUoQccz=5m0Eu=CqEtRQ@mail.gmail.com>
	<4F329FC9.4030403@draigBrady.com>
	<CA+TATJFkn8=Kd3iqrtk=z2pkkzKRGwnAsquw61r0kqSMx76zgQ@mail.gmail.com>
	<CA+TATJF2uW6eHqF4fkwacHq4xm1XPY=3Ent3RHDNPuMA8GOZVA@mail.gmail.com>
	<4F38C0BF.2010205@bernhard-voelker.de>
	<4F38DF43.2000106@draigBrady.com>
	<4F38E236.8050903@bernhard-voelker.de>
	<4F390B91.5070903@draigBrady.com>
In-Reply-To: <4F390B91.5070903@draigBrady.com>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
X-Provags-ID: V02:K0:SOsPv9MIK04WwUPtvbayAwuxZx31LATmWTIYJGi7IBD
	0NbNoG2AxXZyMA8kD2TAZIA4fP1dP08bnHElyATU4X1fm+5mBM
	LTno9IZxVeRtC3x+yv7RW+b8yVCHduYq7YlS4U73P0pfigVR1e
	1M32hkxzOZ2NPMW3I3ZFmplhGPlcNv5lA031CQ6rKo2Bt6YdOO
	ImHz9gho6Zr17UxUo77TLXTj3FYk0qjecHNs7q/28F1U5CB3hK
	DW7M2HUV00Emqycdp+wc/Vqol1WEvu9nMRkjwHKLj8jMdHkzm3
	DOFOdahgoxp0h4pnUQT0KkyQnJ6AUraSBtVHs5nDst1m6TKJs2
	FfHhs8CFt4KuJtkpHuepHDu9p85Ipr02WbytXZsjH
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 212.227.126.187
Cc: =?ISO-8859-1?Q?J=E9r=E9my_Compostella?= <jeremy.compostella@gmail.com>,
	coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 13 Feb 2012 14:28:41 -0000



On 02/13/2012 02:09 PM, Pádraig Brady wrote:
>> BTW: shouldn't
>> >    echo abcdefghijklm | src/dd bs=5 seek=8 oflag=seek_bytes
>> > receive EPIPE? When run on a terminal, dd waits in read() forever.
> True. That's because we try to seek using read()
> on non seekable files, and stdout is readable by default,
> so we'll hang waiting for input from the terminal.
> I.E. this will block:
> 
>   dd seek=1 if=/dev/null
> 
> Not sure what to do there.

Hmm, the process writing into the pipe has already exited,
so I don't get why read() doesn't return eof.

Using O_NONBLOCK if reading from stdin?
Strangely, using iflag=nonblock also blocks ;-(

  echo abcdefghijklm | src/dd bs=5 seek=8 iflag=nonblock

Have a nice day,
Berny


From MAILER-DAEMON Mon Feb 13 09:31:32 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RwwwG-0000Iv-7V
	for mharc-coreutils@gnu.org; Mon, 13 Feb 2012 09:31:32 -0500
Received: from eggs.gnu.org ([140.186.70.92]:36361)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Rwww8-0008Sq-Ej
	for coreutils@gnu.org; Mon, 13 Feb 2012 09:31:30 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Rwwvx-0002Je-63
	for coreutils@gnu.org; Mon, 13 Feb 2012 09:31:24 -0500
Received: from mail1.vodafone.ie ([213.233.128.43]:61666)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Rwwvw-0002Ih-Sz
	for coreutils@gnu.org; Mon, 13 Feb 2012 09:31:13 -0500
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: ApMBAE4eOU9daxc1/2dsb2JhbAAMN61PhTUBAQEDATIBRhALDQEKCRYPCQMCAQIBRQYNAQcBAYd4uS6LTisFDAYEAwQEBw4GAQMIAQ0WA4NlhBcEmy6DOIkw
Received: from unknown (HELO [192.168.1.79]) ([93.107.23.53])
	by mail1.vodafone.ie with ESMTP; 13 Feb 2012 14:31:10 +0000
Message-ID: <4F391EAB.4020206@draigBrady.com>
Date: Mon, 13 Feb 2012 14:31:07 +0000
From: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:6.0) Gecko/20110816 Thunderbird/6.0
MIME-Version: 1.0
To: Bernhard Voelker <mail@bernhard-voelker.de>
Subject: Re: dd: add 'skip_bytes' and 'count_bytes' operands
References: <87mx8w3ryu.fsf@Apollo.jerryland.fr>
	<4F2FF5EE.9010906@draigBrady.com>
	<CA+TATJF4w4Ue44Za4hgGR_0RxgEZGBm5nzhHRSdGYxzzfKWNLg@mail.gmail.com>
	<4F317CDF.2040909@draigBrady.com>
	<CA+TATJFU7yOGzzeZseHUFTqxxGAtz8zUoQccz=5m0Eu=CqEtRQ@mail.gmail.com>
	<4F329FC9.4030403@draigBrady.com>
	<CA+TATJFkn8=Kd3iqrtk=z2pkkzKRGwnAsquw61r0kqSMx76zgQ@mail.gmail.com>
	<CA+TATJF2uW6eHqF4fkwacHq4xm1XPY=3Ent3RHDNPuMA8GOZVA@mail.gmail.com>
	<4F38C0BF.2010205@bernhard-voelker.de>
	<4F38DF43.2000106@draigBrady.com>
	<4F38E236.8050903@bernhard-voelker.de>
	<4F390B91.5070903@draigBrady.com>
	<4F391E0F.5000302@bernhard-voelker.de>
In-Reply-To: <4F391E0F.5000302@bernhard-voelker.de>
X-Enigmail-Version: 1.3.2
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 213.233.128.43
Cc: =?ISO-8859-1?Q?J=E9r=E9my_Compostella?= <jeremy.compostella@gmail.com>,
	coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 13 Feb 2012 14:31:30 -0000

On 02/13/2012 02:28 PM, Bernhard Voelker wrote:
> 
> 
> On 02/13/2012 02:09 PM, Pádraig Brady wrote:
>>> BTW: shouldn't
>>>>    echo abcdefghijklm | src/dd bs=5 seek=8 oflag=seek_bytes
>>>> receive EPIPE? When run on a terminal, dd waits in read() forever.
>> True. That's because we try to seek using read()
>> on non seekable files, and stdout is readable by default,
>> so we'll hang waiting for input from the terminal.
>> I.E. this will block:
>>
>>   dd seek=1 if=/dev/null
>>
>> Not sure what to do there.

In other words, I'm not sure this is an issue

> 
> Hmm, the process writing into the pipe has already exited,
> so I don't get why read() doesn't return eof.
> 
> Using O_NONBLOCK if reading from stdin?
> Strangely, using iflag=nonblock also blocks ;-(
> 
>   echo abcdefghijklm | src/dd bs=5 seek=8 iflag=nonblock

seek is for output, skip is for input.

cheers,
Pádraig.


From MAILER-DAEMON Mon Feb 13 10:06:32 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RwxU8-0000o8-Pw
	for mharc-coreutils@gnu.org; Mon, 13 Feb 2012 10:06:32 -0500
Received: from eggs.gnu.org ([140.186.70.92]:40126)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1RwxTz-0000lA-QT
	for coreutils@gnu.org; Mon, 13 Feb 2012 10:06:31 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1RwxTy-0001eS-8V
	for coreutils@gnu.org; Mon, 13 Feb 2012 10:06:23 -0500
Received: from moutng.kundenserver.de ([212.227.126.171]:65373)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1RwxTx-0001eC-Vd
	for coreutils@gnu.org; Mon, 13 Feb 2012 10:06:22 -0500
Received: from [192.168.2.108] (p4FF753D4.dip.t-dialin.net [79.247.83.212])
	by mrelayeu.kundenserver.de (node=mreu1) with ESMTP (Nemesis)
	id 0LnX58-1SRMYO37RG-00h7TB; Mon, 13 Feb 2012 16:06:20 +0100
Message-ID: <4F3926E9.2040807@bernhard-voelker.de>
Date: Mon, 13 Feb 2012 16:06:17 +0100
From: Bernhard Voelker <mail@bernhard-voelker.de>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:10.0) Gecko/20120129 Thunderbird/10.0
MIME-Version: 1.0
To: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
Subject: Re: dd: add 'skip_bytes' and 'count_bytes' operands
References: <87mx8w3ryu.fsf@Apollo.jerryland.fr>
	<4F2FF5EE.9010906@draigBrady.com>
	<CA+TATJF4w4Ue44Za4hgGR_0RxgEZGBm5nzhHRSdGYxzzfKWNLg@mail.gmail.com>
	<4F317CDF.2040909@draigBrady.com>
	<CA+TATJFU7yOGzzeZseHUFTqxxGAtz8zUoQccz=5m0Eu=CqEtRQ@mail.gmail.com>
	<4F329FC9.4030403@draigBrady.com>
	<CA+TATJFkn8=Kd3iqrtk=z2pkkzKRGwnAsquw61r0kqSMx76zgQ@mail.gmail.com>
	<CA+TATJF2uW6eHqF4fkwacHq4xm1XPY=3Ent3RHDNPuMA8GOZVA@mail.gmail.com>
	<4F38C0BF.2010205@bernhard-voelker.de>
	<4F38DF43.2000106@draigBrady.com>
	<4F38E236.8050903@bernhard-voelker.de>
	<4F390B91.5070903@draigBrady.com>
	<4F391E0F.5000302@bernhard-voelker.de>
	<4F391EAB.4020206@draigBrady.com>
In-Reply-To: <4F391EAB.4020206@draigBrady.com>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
X-Provags-ID: V02:K0:jDSFkgMjrOnAsoz90Tt+zG4I45r54it4N18o5MOGLud
	yV8WT+dWm8QePdZ5hbANz8w/GQvR92KyB5lr9cNpOI8M7S8uuq
	ob9HEIIS3FxKDfcz3JaDQC77coBYmBaVGDhGtDnaIMypSOZ44w
	VIkQ2Lycg6PTmbGZR2tLeQMiMmfGctcOZSMnseVyv2rY5kgDR6
	qXqH1ExT2WGwGMsSAIA3geHUr85IY+RHFzA+TW2Ei4TTVkQB3L
	uKri1XhzvRFhZWaJ/Nbl4DOc+kBezKy7NWXjGcbzpCYTkxyi50
	vSX4/yDO7dxhA7Dae1btB4XkUPZjrA/IpnbJuUDW9aPo3WmS6Q
	eXbxsqfPAbr+eH6xhn0L7kkrRrB4D6HacEvYykzBi
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 212.227.126.171
Cc: =?ISO-8859-1?Q?J=E9r=E9my_Compostella?= <jeremy.compostella@gmail.com>,
	coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 13 Feb 2012 15:06:31 -0000

On 02/13/2012 03:31 PM, Pádraig Brady wrote:
> On 02/13/2012 02:28 PM, Bernhard Voelker wrote:
>>
>>
>> On 02/13/2012 02:09 PM, Pádraig Brady wrote:
>>>> BTW: shouldn't
>>>>>    echo abcdefghijklm | src/dd bs=5 seek=8 oflag=seek_bytes
>>>>> receive EPIPE? When run on a terminal, dd waits in read() forever.
>>> True. That's because we try to seek using read()
>>> on non seekable files, and stdout is readable by default,
>>> so we'll hang waiting for input from the terminal.
>>> I.E. this will block:
>>>
>>>   dd seek=1 if=/dev/null
>>>
>>> Not sure what to do there.
> 
> In other words, I'm not sure this is an issue
> 
>>
>> Hmm, the process writing into the pipe has already exited,
>> so I don't get why read() doesn't return eof.
>>
>> Using O_NONBLOCK if reading from stdin?
>> Strangely, using iflag=nonblock also blocks ;-(
>>
>>   echo abcdefghijklm | src/dd bs=5 seek=8 iflag=nonblock
> 
> seek is for output, skip is for input.


Now, I see it: it's reading from *stdout*, not from stdin:

lseek(1, 0, SEEK_END)                   = -1 ESPIPE (Illegal seek)
read(1, 0x616000, 512)                  = ?

It could issue an error if "seek=..." is used when fdesc is a tty.

Have a nice day,
Berny



From MAILER-DAEMON Mon Feb 13 10:53:23 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RwyDT-0007Z4-TU
	for mharc-coreutils@gnu.org; Mon, 13 Feb 2012 10:53:23 -0500
Received: from eggs.gnu.org ([140.186.70.92]:39675)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1RwyDN-0007Yl-HK
	for coreutils@gnu.org; Mon, 13 Feb 2012 10:53:23 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1RwyDJ-0004yu-EJ
	for coreutils@gnu.org; Mon, 13 Feb 2012 10:53:17 -0500
Received: from moutng.kundenserver.de ([212.227.17.8]:50617)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1RwyDI-0004yO-Vh
	for coreutils@gnu.org; Mon, 13 Feb 2012 10:53:13 -0500
Received: from [192.168.2.108] (p4FF753D4.dip.t-dialin.net [79.247.83.212])
	by mrelayeu.kundenserver.de (node=mreu0) with ESMTP (Nemesis)
	id 0M371n-1SpLuu1wBz-00sUnt; Mon, 13 Feb 2012 16:53:09 +0100
Message-ID: <4F3931E2.5040102@bernhard-voelker.de>
Date: Mon, 13 Feb 2012 16:53:06 +0100
From: Bernhard Voelker <mail@bernhard-voelker.de>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:10.0) Gecko/20120129 Thunderbird/10.0
MIME-Version: 1.0
To: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
Subject: Re: dd: add 'skip_bytes' and 'count_bytes' operands
References: <87mx8w3ryu.fsf@Apollo.jerryland.fr>
	<4F2FF5EE.9010906@draigBrady.com>
	<CA+TATJF4w4Ue44Za4hgGR_0RxgEZGBm5nzhHRSdGYxzzfKWNLg@mail.gmail.com>
	<4F317CDF.2040909@draigBrady.com>
	<CA+TATJFU7yOGzzeZseHUFTqxxGAtz8zUoQccz=5m0Eu=CqEtRQ@mail.gmail.com>
	<4F329FC9.4030403@draigBrady.com>
	<CA+TATJFkn8=Kd3iqrtk=z2pkkzKRGwnAsquw61r0kqSMx76zgQ@mail.gmail.com>
	<CA+TATJF2uW6eHqF4fkwacHq4xm1XPY=3Ent3RHDNPuMA8GOZVA@mail.gmail.com>
	<4F38C0BF.2010205@bernhard-voelker.de>
	<4F38DF43.2000106@draigBrady.com>
	<4F38E236.8050903@bernhard-voelker.de>
	<4F390B91.5070903@draigBrady.com>
	<4F391E0F.5000302@bernhard-voelker.de>
	<4F391EAB.4020206@draigBrady.com>
	<4F3926E9.2040807@bernhard-voelker.de>
In-Reply-To: <4F3926E9.2040807@bernhard-voelker.de>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
X-Provags-ID: V02:K0:ajbYrotspzCHHCvya7HRNvywHpuQ3niZRsgn9PXKTaz
	vASaJ6GMrvUeVJqT8yYvpPm/pZle+jlNsNQdG8y1+L1SWPZFpd
	nghJcLXjgGjzp7SDg0Q5qF76LCgRaNzKBOUD4uStXRECyP45RF
	ibQyvBpy3KzX465ANyFpwivN1vPOaB0NAAXnmANJ6vbTcPdGEg
	n3ihLFdYjVQpTjoqsQmGkWBHSuL59kD3J2RNlrOyw4MA5s2hcf
	+3eDe2W+A/IoubcaC1ACIjjWg122op4BUBKe2FQZ7kjQntg+9y
	fjaFdcnGCNTNxykjiz5EdMady0aMGbtYFNmO7T5Gd+neiA2pJx
	aaIGaashpWvgJyUvxAgp3l7ZKK3jV/0UwhE00m3j1
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 212.227.17.8
Cc: =?ISO-8859-1?Q?J=E9r=E9my_Compostella?= <jeremy.compostella@gmail.com>,
	coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 13 Feb 2012 15:53:23 -0000

On 02/13/2012 04:06 PM, Bernhard Voelker wrote:
> Now, I see it: it's reading from *stdout*, not from stdin:
> 
> lseek(1, 0, SEEK_END)                   = -1 ESPIPE (Illegal seek)
> read(1, 0x616000, 512)                  = ?
> 
> It could issue an error if "seek=..." is used when fdesc is a tty.

Interestingly, read returns with EBADF if stdout is
redirected to /dev/tty:

  /usr/bin/printf "abc" | strace  src/dd seek=1 > /dev/tty
  ...
  lseek(1, 0, SEEK_END)                   = -1 ESPIPE (Illegal seek)
  read(1, 0x616000, 512)                  = -1 EBADF (Bad file descriptor)

but it blocks again when of=/dev/tty is given:

  /usr/bin/printf "abc" | strace src/dd seek=1 of=/dev/tty

I'm getting the impression that this is a kernel problem.

Have a nice day,
Berny


From MAILER-DAEMON Mon Feb 13 13:42:15 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1Rx0qt-00038u-5s
	for mharc-coreutils@gnu.org; Mon, 13 Feb 2012 13:42:15 -0500
Received: from eggs.gnu.org ([140.186.70.92]:46951)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1Rx0qr-00038c-CO
	for coreutils@gnu.org; Mon, 13 Feb 2012 13:42:14 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1Rx0qq-0001iR-4S
	for coreutils@gnu.org; Mon, 13 Feb 2012 13:42:13 -0500
Received: from mail-iy0-f169.google.com ([209.85.210.169]:37918)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1Rx0qq-0001fF-1P
	for coreutils@gnu.org; Mon, 13 Feb 2012 13:42:12 -0500
Received: by mail-iy0-f169.google.com with SMTP id z16so5534938iag.0
	for <coreutils@gnu.org>; Mon, 13 Feb 2012 10:42:11 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma;
	h=mime-version:in-reply-to:references:from:date:message-id:subject:to
	:cc:content-type;
	bh=01jQFvvct/RFnhHxiBqrI1/0GOtDyNYdXY4UssboFW4=;
	b=e8Mf6k6WaF+BXAL022ZkpGv3r6x7YLvXaT6EMn29IX7iG16ZOFezeaObRD7gXTjqDp
	Jao6TmUQbDwZt9YbKjqPfNi8H66dObwwSPFCkE7n2jSBvcE4Emrsj41ZPaAjqK3MdsIl
	u4ejQbRDMkdGR95w71QKB1ZuD8oESfbssMUHA=
Received: by 10.42.235.137 with SMTP id kg9mr23704072icb.47.1329158531099;
	Mon, 13 Feb 2012 10:42:11 -0800 (PST)
MIME-Version: 1.0
Received: by 10.42.29.196 with HTTP; Mon, 13 Feb 2012 10:41:49 -0800 (PST)
In-Reply-To: <4F3931E2.5040102@bernhard-voelker.de>
References: <87mx8w3ryu.fsf@Apollo.jerryland.fr>
	<4F2FF5EE.9010906@draigBrady.com>
	<CA+TATJF4w4Ue44Za4hgGR_0RxgEZGBm5nzhHRSdGYxzzfKWNLg@mail.gmail.com>
	<4F317CDF.2040909@draigBrady.com>
	<CA+TATJFU7yOGzzeZseHUFTqxxGAtz8zUoQccz=5m0Eu=CqEtRQ@mail.gmail.com>
	<4F329FC9.4030403@draigBrady.com>
	<CA+TATJFkn8=Kd3iqrtk=z2pkkzKRGwnAsquw61r0kqSMx76zgQ@mail.gmail.com>
	<CA+TATJF2uW6eHqF4fkwacHq4xm1XPY=3Ent3RHDNPuMA8GOZVA@mail.gmail.com>
	<4F38C0BF.2010205@bernhard-voelker.de>
	<4F38DF43.2000106@draigBrady.com>
	<4F38E236.8050903@bernhard-voelker.de>
	<4F390B91.5070903@draigBrady.com>
	<4F391E0F.5000302@bernhard-voelker.de>
	<4F391EAB.4020206@draigBrady.com>
	<4F3926E9.2040807@bernhard-voelker.de>
	<4F3931E2.5040102@bernhard-voelker.de>
From: =?ISO-8859-1?Q?J=E9r=E9my_Compostella?= <jeremy.compostella@gmail.com>
Date: Mon, 13 Feb 2012 19:41:49 +0100
Message-ID: <CA+TATJEgfzq4LOgcm+p=pKZc3qdQtxdjULbwZiPZtG_F7ZP1cQ@mail.gmail.com>
Subject: Re: dd: add 'skip_bytes' and 'count_bytes' operands
To: Bernhard Voelker <mail@bernhard-voelker.de>
Content-Type: multipart/alternative; boundary=20cf3026697a18aa7404b8dcd672
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.85.210.169
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 13 Feb 2012 18:42:14 -0000

--20cf3026697a18aa7404b8dcd672
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

P=E1draig,

I took a closer look to the last test case. I wrote it during the
development phase. At this time, I forced the skip() function to failed
in order to test the line 1896 algorithm.

I just made some test and was unable to make the lseek() call failed on
my system. So I guess we could remove this last test case. Except if
someone have a good idea to easily make this call failed.

Cheers,

J=E9r=E9my

--20cf3026697a18aa7404b8dcd672
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div>P=E1draig,</div><div><br></div><div>I took a closer look to the last t=
est case. I wrote it during the</div><div>development phase. At this time, =
I forced the skip() function to failed</div><div>in order to test the line =
1896 algorithm.</div>

<div><br></div><div>I just made some test and was unable to make the lseek(=
) call failed on</div><div>my system. So I guess we could remove this last =
test case. Except if</div><div>someone have a good idea to easily make this=
 call failed.</div>

<div><br></div><div>Cheers,</div><div><br></div><div>J=E9r=E9my</div>

--20cf3026697a18aa7404b8dcd672--


From MAILER-DAEMON Mon Feb 13 17:06:45 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1Rx42m-0000Zi-VB
	for mharc-coreutils@gnu.org; Mon, 13 Feb 2012 17:06:44 -0500
Received: from eggs.gnu.org ([140.186.70.92]:34806)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Rx42k-0000YC-Ej
	for coreutils@gnu.org; Mon, 13 Feb 2012 17:06:43 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Rx42j-0000Ws-2A
	for coreutils@gnu.org; Mon, 13 Feb 2012 17:06:42 -0500
Received: from mail1.vodafone.ie ([213.233.128.43]:24862)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Rx42i-0000WJ-Tp
	for coreutils@gnu.org; Mon, 13 Feb 2012 17:06:41 -0500
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: ApMBAOmIOU9daxc1/2dsb2JhbAAMN7MJAQEBBAxtEAsNBAMBAgEJFg8JAwIBAgE9CAYNAQUCAQHBMItAAzEFCkMBDYN+hBcEjkmBG4tKgziJMA
Received: from unknown (HELO [192.168.1.79]) ([93.107.23.53])
	by mail1.vodafone.ie with ESMTP; 13 Feb 2012 22:06:38 +0000
Message-ID: <4F39896C.3040804@draigBrady.com>
Date: Mon, 13 Feb 2012 22:06:36 +0000
From: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:6.0) Gecko/20110816 Thunderbird/6.0
MIME-Version: 1.0
To: =?ISO-8859-1?Q?J=E9r=E9my_Compostella?=
 <jeremy.compostella@gmail.com>
Subject: Re: dd: add 'skip_bytes' and 'count_bytes' operands
References: <87mx8w3ryu.fsf@Apollo.jerryland.fr>
	<4F2FF5EE.9010906@draigBrady.com>
	<CA+TATJF4w4Ue44Za4hgGR_0RxgEZGBm5nzhHRSdGYxzzfKWNLg@mail.gmail.com>
	<4F317CDF.2040909@draigBrady.com>
	<CA+TATJFU7yOGzzeZseHUFTqxxGAtz8zUoQccz=5m0Eu=CqEtRQ@mail.gmail.com>
	<4F329FC9.4030403@draigBrady.com>
	<CA+TATJFkn8=Kd3iqrtk=z2pkkzKRGwnAsquw61r0kqSMx76zgQ@mail.gmail.com>
	<CA+TATJF2uW6eHqF4fkwacHq4xm1XPY=3Ent3RHDNPuMA8GOZVA@mail.gmail.com>
	<4F38C0BF.2010205@bernhard-voelker.de>
	<4F38DF43.2000106@draigBrady.com>
	<4F38E236.8050903@bernhard-voelker.de>
	<4F390B91.5070903@draigBrady.com>
	<4F391E0F.5000302@bernhard-voelker.de>
	<4F391EAB.4020206@draigBrady.com>
	<4F3926E9.2040807@bernhard-voelker.de>
	<4F3931E2.5040102@bernhard-voelker.de>
	<CA+TATJEgfzq4LOgcm+p=pKZc3qdQtxdjULbwZiPZtG_F7ZP1cQ@mail.gmail.com>
In-Reply-To: <CA+TATJEgfzq4LOgcm+p=pKZc3qdQtxdjULbwZiPZtG_F7ZP1cQ@mail.gmail.com>
X-Enigmail-Version: 1.3.2
Content-Type: multipart/mixed; boundary="------------090602010206010707020704"
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 213.233.128.43
Cc: Bernhard Voelker <mail@bernhard-voelker.de>, coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 13 Feb 2012 22:06:43 -0000

This is a multi-part message in MIME format.
--------------090602010206010707020704
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit

On 02/13/2012 06:41 PM, Jérémy Compostella wrote:
> Pádraig,
> 
> I took a closer look to the last test case. I wrote it during the
> development phase. At this time, I forced the skip() function to failed
> in order to test the line 1896 algorithm.
> 
> I just made some test and was unable to make the lseek() call failed on
> my system. So I guess we could remove this last test case. Except if
> someone have a good idea to easily make this call failed.

Ok I replaced that test with one checking the truncate functionality.

I've pushed the attached.

cheers,
Pádraig.

--------------090602010206010707020704
Content-Type: text/plain;
 name="seek_bytes_adjustment.diff"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename="seek_bytes_adjustment.diff"

>From 4bb5dbcfef55c4f66755dd3c48b906e4e9030b96 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?P=C3=A1draig=20Brady?= <P@draigBrady.com>
Date: Mon, 13 Feb 2012 21:47:28 +0000
Subject: [PATCH] dd: fix issues in the count_bytes and seek_bytes flags
 change

These edge cases were missed in the previous commit 140eca15c.

* src/dd.c (main): Include the bytes slop when truncating
without further I/O.  Don't invalidate the whole file cache
in the case where 0 < count < ibs.
* tests/dd/bytes: Change to using the independent truncate
command to generate the file for comparison.  Remove a redundant
test case and replace with one testing the truncation only logic.
---
 src/dd.c       |    8 ++++----
 tests/dd/bytes |   11 +++++------
 2 files changed, 9 insertions(+), 10 deletions(-)

diff --git a/src/dd.c b/src/dd.c
index e7f4037..fe44a30 100644
--- a/src/dd.c
+++ b/src/dd.c
@@ -1917,7 +1917,7 @@ dd_copy (void)
 
   while (1)
     {
-      if (r_partial + r_full >= max_records + (max_bytes ? 1 : 0))
+      if (r_partial + r_full >= max_records + !!max_bytes)
         break;
 
       /* Zero the buffer before reading, so that if we get a read error,
@@ -2170,7 +2170,7 @@ main (int argc, char **argv)
 
       if (seek_records != 0 && !(conversions_mask & C_NOTRUNC))
         {
-          uintmax_t size = seek_records * output_blocksize;
+          uintmax_t size = seek_records * output_blocksize + seek_bytes;
           unsigned long int obs = output_blocksize;
 
           if (OFF_T_MAX / output_blocksize < seek_records)
@@ -2207,7 +2207,7 @@ main (int argc, char **argv)
 
   exit_status = dd_copy ();
 
-  if (max_records == 0)
+  if (max_records == 0 && max_bytes == 0)
     {
       /* Special case to invalidate cache to end of file.  */
       if (i_nocache && !invalidate_cache (STDIN_FILENO, 0))
@@ -2225,7 +2225,7 @@ main (int argc, char **argv)
     }
   else if (max_records != (uintmax_t) -1)
     {
-      /* Invalidate any pending region less that page size,
+      /* Invalidate any pending region less than page size,
          in case the kernel might round up.  */
       if (i_nocache)
         invalidate_cache (STDIN_FILENO, 0);
diff --git a/tests/dd/bytes b/tests/dd/bytes
index 6038742..15755aa 100755
--- a/tests/dd/bytes
+++ b/tests/dd/bytes
@@ -45,13 +45,12 @@ esac
 # seek bytes
 echo abcdefghijklm |
  dd bs=5 seek=8 oflag=seek_bytes > out 2> /dev/null || fail=1
-echo abcdefghijklm |
- dd bs=4 seek=2 > expected 2> /dev/null || fail=1
+printf '\0\0\0\0\0\0\0\0abcdefghijklm\n' > expected
 compare expected out || fail=1
 
-# seek bytes on empty file
-echo abcdefghijklm |
- dd bs=5 seek=8 oflag=seek_bytes > out2 2> /dev/null || fail=1
-compare expected out2 || fail=1
+# Just truncation, no I/O
+dd bs=5 seek=8 oflag=seek_bytes of=out2 count=0 2> /dev/null || fail=1
+truncate -s8 expected2
+compare expected2 out2 || fail=1
 
 Exit $fail
-- 
1.7.6.4


--------------090602010206010707020704--


From MAILER-DAEMON Mon Feb 13 20:06:31 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1Rx6ql-0003GL-4X
	for mharc-coreutils@gnu.org; Mon, 13 Feb 2012 20:06:31 -0500
Received: from eggs.gnu.org ([140.186.70.92]:33047)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Rx6qh-0003Ff-Lo
	for coreutils@gnu.org; Mon, 13 Feb 2012 20:06:29 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Rx6qg-00048X-2A
	for coreutils@gnu.org; Mon, 13 Feb 2012 20:06:27 -0500
Received: from mx1.redhat.com ([209.132.183.28]:22172)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Rx6qf-00048Q-QL
	for coreutils@gnu.org; Mon, 13 Feb 2012 20:06:26 -0500
Received: from int-mx11.intmail.prod.int.phx2.redhat.com
	(int-mx11.intmail.prod.int.phx2.redhat.com [10.5.11.24])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id q1E16MBW020194
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK);
	Mon, 13 Feb 2012 20:06:23 -0500
Received: from [10.36.116.55] (ovpn-116-55.ams2.redhat.com [10.36.116.55])
	by int-mx11.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP
	id q1E16KX8004109
	(version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO);
	Mon, 13 Feb 2012 20:06:21 -0500
Message-ID: <4F39B38A.6070603@draigBrady.com>
Date: Tue, 14 Feb 2012 01:06:18 +0000
From: =?UTF-8?B?UMOhZHJhaWcgQnJhZHk=?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:6.0) Gecko/20110816 Thunderbird/6.0
MIME-Version: 1.0
To: Peng Yu <pengyu.ut@gmail.com>
Subject: Re: Adding humanize_number to coreutiles?
References: <CABrM6wmAm=0btpW0JT_A4=8W-bi3PAZ8SDHv1sCjcBOXyQeimQ@mail.gmail.com>
	<4F30F2D0.6010009@draigBrady.com>
	<CABrM6w=WteqGuyPB0R_+UjgNnwaqGR2UtFN0YcwwO_UbWmbhtg@mail.gmail.com>
In-Reply-To: <CABrM6w=WteqGuyPB0R_+UjgNnwaqGR2UtFN0YcwwO_UbWmbhtg@mail.gmail.com>
X-Enigmail-Version: 1.3.2
Content-Type: text/plain; charset=UTF-8
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.24
Content-Transfer-Encoding: quoted-printable
X-MIME-Autoconverted: from 8bit to quoted-printable by mx1.redhat.com id
	q1E16MBW020194
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.132.183.28
Cc: Coreutils <coreutils@gnu.org>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Tue, 14 Feb 2012 01:06:29 -0000

On 02/13/2012 03:45 AM, Peng Yu wrote:
> 2012/2/7 P=C3=A1draig Brady <P@draigbrady.com>:
>> On 02/07/2012 03:36 AM, Peng Yu wrote:
>>> Hi,
>>>
>>> Several commands in coreutils have the -h option. I'm wondering
>>> whether anybody in the develop team also thinks that it is worthwhile
>>> to export it as a standalone command. If so, I'd recommend add such
>>> convenient command in coreutiles. As I don't find it anywhere else as
>>> a stand alone command.
>>>
>>> http://siarzhuk.dyndns.org/haiku/doxygen/coreutils_2lib_2human_8c_sou=
rce.html#l00154
>>
>> I've needed such functionality many times.
>> I'm thinking a printf format would be best to expose this:
>> http://lists.gnu.org/archive/html/coreutils/2011-08/msg00029.html
>>
>> %H seems like it might cause compat problems in future.
>> %{human} is more descriptive and extensible, so I'm leaning towards th=
at.
>> Any other suggestions appreciated.
>>
>> I'll work on it this week.
>=20
> I'm not sure %{human} is enough for configuring all the possible ways
> of printing a humanized number. See my code below, there are binary
> and decimal humanized numbers. Also, you'd better allow a space (or
> not) between the numbers and the letters (such as 'T', 'G').
>=20
> Also embedding it in printf will make it hard to be found, I'd
> recommend to create a new command like humanizenumber, just as I did.
>=20
> /tmp$ cat `which humanizenumber.sh `
> #!/usr/bin/env bash
>=20
> script_name=3D`basename "$0" .sh`
>=20
> TEMP=3D`getopt -o hbd:sn --long
> help,binary,number_of_decimal_places:,space,newline -n
> "${script_name}.sh" -- "$@"`
>=20
> if [ $? !=3D 0 ] ; then printf "Terminating...\n" >&2 ; exit 1 ; fi
>=20
> eval set -- "$TEMP"
>=20
> abspath_script=3D`readlink -f -e "$0"`
> script_absdir=3D`dirname "$abspath_script"`
>=20
> number_of_decimal_places=3D0
> while true ; do
>   case "$1" in
>     -h|--help)
>       cat "$script_absdir"/${script_name}_help.txt
>       exit
>       ;;
>     -b|--binary)
>       binary=3Dx
>       shift
>       ;;
>     -d|--number_of_decimal_places)
>       number_of_decimal_places=3D"$2"
>       shift 2
>       ;;
>     -s|--space)
>       space=3D' '
>       shift
>       ;;
>     -n|--newline)
>       newline=3D'\n'
>       shift
>       ;;
>     --)
>       shift
>       break
>       ;;
>     *)
>       printf "Internal error!\n">&2
>       exit 1
>       ;;
>   esac
> done
>=20
> if [ $# -ne 0 ]
> then
>   n=3D"$1"
>   if [ -n "$binary" ]
>   then
>     awk -v sum=3D$n \
>       -v space=3D"$space" \
>       -v newline=3D"$newline" \
>       -v number_of_decimal_places=3D$number_of_decimal_places '
>     BEGIN{
>     hum[1024**5]=3D"P"
>     hum[1024**4]=3D"T"
>     hum[1024**3]=3D"G"
>     hum[1024**2]=3D"M"
>     hum[1024]=3D"K"
>     for (x=3D1024**5; x>=3D1024; x/=3D1024) {
>       if (sum>=3Dx) {
>         printf "%." number_of_decimal_places "f" space "%s" newline,
> sum/x, hum[x]
>         break
>       }
>     }
>   }'
> else
>   awk -v sum=3D$n \
>     -v space=3D"$space" \
>     -v newline=3D"$newline" \
>     -v number_of_decimal_places=3D$number_of_decimal_places '
>   BEGIN{
>   hum[1000**5]=3D"P"
>   hum[1000**4]=3D"T"
>   hum[1000**3]=3D"G"
>   hum[1000**2]=3D"M"
>   hum[1000]=3D"k"
>   for (x=3D1000**5; x>=3D1000; x/=3D1000) {
>     if (sum>=3Dx) {
>       printf "%." number_of_decimal_places "f" space "%s" newline, sum/=
x, hum[x]
>       break
>     }
>   }
> }'
>   fi
> fi
> /tmp$  humanizenumber.sh -h
> Description:
>   Humanize number(s)
>=20
> Usage:
>   humanizenumber.sh [Options] [NUMBER]
>=20
>     NUMBER                            If not specifed, then do nothing.
>=20
> Options:
>   -h|--help                           Help message.
>   -b|--binary                         Default: decimal.
>   -s|--space                          Default: nospace.
>=20
> Examples:
>   humanizenumber.sh 456456456
>   humanizenumber.sh -d 2 456456456
>   humanizenumber.sh -s 456456456
>   humanizenumber.sh -b 456456456
>   humanizenumber.sh -n 456456456
>=20
> Author:
>   Peng Yu <pengyu.ut@gmail.com>

Looking more at this, you might be right.
Now printf already has related formatting functionality:

$ env LANG=3Dfa_IR.utf8 printf "%I'd\n" 1234
=DB=B1=D9=AC=DB=B2=DB=B3=DB=B4

I was thinking it would be appropriate to add "human" into the mix like

$ env LANG=3Dfa_IR.utf8 printf "%Hd\n" 1234
1K

$ env LANG=3Dfa_IR.utf8 printf "%HId\n" 1234
=DB=B1K

But as you say there are options for humanizing.
So would there be enough cohesive functionality one could add to such a u=
til?
I suppose so, since one could add field processing and
multiplier support for example.

Also what to call it? humanize_number is too long I think.
Perhaps we could use a more general name. Drats I was
thinking of `numconv`, but that's taken:
http://www.unixref.com/manPages/numconv.html
Maybe `convnum`, anyway...

A tentative design could be:


convnum [OPTIONS] [NUM]...

Numbers are processed from stdin or the command options.

--from=3D{auto,SI,IEC}
If not specified, suffixes are ignored
auto =3D> 1K -> 1000, 1Ki -> 1024
SI =3D> 1K* -> 1000
IEC =3D> 1K* -> 1024

--from-unit=3D<NUMBER>
Specify the unit size.
--from-unit=3D1 is implied if not specified

--to=3D{SI,IEC,<NUMBER>}
Auto scale the numbers to SI (powers of 1000),
or IEC (powers of 1024), so at most 3 digits are output.
Note output will be standard, without a B suffix.
I.E. 123K or 123Ki for SI and IEC respectively.
If <NUMBER> is specified use this as the scale.

--to-unit=3D<NUMBER>
Specify the output unit size.
--to-unit=3D1 is implied if not specified

--round=3D{ceiling, floor, nearest}
--round=3Dceiling is implied if not specified

--number-format=3DFORMAT
--number-format=3D%d is implied if not specified
You can use this to specify a space after the number
You can also use this to perform grouping (with %'d)
You can also use this to select alternative number forms (with %Id) etc.

--suffix=3DSUFFIX
Example --suffix=3DB

--field=3DNUM
replace the number in the portion of the line delimited by whitespace.
If the new number is narrower, then pad to the same field width with spac=
es.

<NUMBER>s specified above can be numeric
with an optional suffix, like K, Ki.
(Note K should probably be SI here, unlike other coreutils).


cheers,
P=C3=A1draig


From MAILER-DAEMON Tue Feb 14 07:37:38 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RxHda-00064K-9s
	for mharc-coreutils@gnu.org; Tue, 14 Feb 2012 07:37:38 -0500
Received: from eggs.gnu.org ([140.186.70.92]:59040)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RxHdO-0005xI-Cz
	for coreutils@gnu.org; Tue, 14 Feb 2012 07:37:35 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RxHdI-0006oD-Fl
	for coreutils@gnu.org; Tue, 14 Feb 2012 07:37:26 -0500
Received: from mx1.redhat.com ([209.132.183.28]:50794)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RxHdI-0006nz-9A
	for coreutils@gnu.org; Tue, 14 Feb 2012 07:37:20 -0500
Received: from int-mx02.intmail.prod.int.phx2.redhat.com
	(int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id q1ECbG6v017449
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK);
	Tue, 14 Feb 2012 07:37:16 -0500
Received: from [10.36.116.51] (ovpn-116-51.ams2.redhat.com [10.36.116.51])
	by int-mx02.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP
	id q1ECbDrf032225
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO);
	Tue, 14 Feb 2012 07:37:15 -0500
Message-ID: <4F3A5578.9080102@draigBrady.com>
Date: Tue, 14 Feb 2012 12:37:12 +0000
From: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:6.0) Gecko/20110816 Thunderbird/6.0
MIME-Version: 1.0
To: Peng Yu <pengyu.ut@gmail.com>
Subject: Re: join on multiple field
References: <CABrM6wmmkFpw3QVX7oYQfA_RB7k8yVnDphzsLX8A9Hv6Pzqitw@mail.gmail.com>
	<4F391226.3060508@redhat.com>
	<CABrM6w=ggV7=i5qmbiAqcsZwRruXTgZRPLknpNbLn4eapTrRYQ@mail.gmail.com>
In-Reply-To: <CABrM6w=ggV7=i5qmbiAqcsZwRruXTgZRPLknpNbLn4eapTrRYQ@mail.gmail.com>
X-Enigmail-Version: 1.3.2
Content-Type: text/plain; charset=ISO-8859-1
X-Scanned-By: MIMEDefang 2.67 on 10.5.11.12
Content-Transfer-Encoding: quoted-printable
X-MIME-Autoconverted: from 8bit to quoted-printable by mx1.redhat.com id
	q1ECbG6v017449
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.132.183.28
Cc: Coreutils <coreutils@gnu.org>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Tue, 14 Feb 2012 12:37:35 -0000

On 02/13/2012 01:54 PM, Peng Yu wrote:
>> Can't you already do this with awk?
>=20
> This is not a very useful comment.

Take things on this list as discussion,
rather than a mandate.
We're all very open here.

> Everything that coreutils do can be
> done with some other software. Since 'join' can do 1 field join, it is
> natural to ask for multiple field join. After all, people use
> coreutils for its convenience in certain aspects. Why not make it more
> convenient?

Sure. There is always a trade-off in adding new logic.
Extra docs for users to parse, and extra code to maintain, etc.
Discussion, is needed to aid decisions.

> The current working flow in coreutils is like from the stone age,
> while many other open source projects use more advanced dvcs like
> github or bitbucket.
>=20
> I had tried to send some patch before. Besides the above mentioned
> inconvenience, I have to get my signature from GNU office, which they
> never reply my email.
>=20
> If coreutils wants me people to contribute, the patching/submission
> processes must be improved. The current process is way inconvenience
> for new contributors.

The last assignment was completed in a few days.
Could you perhaps follow up with copyright-clerk@fsf.org
in case something got lost along the way?

cheers,
P=E1draig.


From MAILER-DAEMON Tue Feb 14 07:59:20 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RxHya-0007Tr-BG
	for mharc-coreutils@gnu.org; Tue, 14 Feb 2012 07:59:20 -0500
Received: from eggs.gnu.org ([140.186.70.92]:33093)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1RxHyX-0007Tl-PS
	for coreutils@gnu.org; Tue, 14 Feb 2012 07:59:18 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1RxHyS-0002gs-44
	for coreutils@gnu.org; Tue, 14 Feb 2012 07:59:17 -0500
Received: from mx.meyering.net ([88.168.87.75]:55769)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1RxHyR-0002gh-Mw
	for coreutils@gnu.org; Tue, 14 Feb 2012 07:59:12 -0500
Received: from rho.meyering.net (localhost.localdomain [127.0.0.1])
	by rho.meyering.net (Acme Bit-Twister) with ESMTP id 9AF7E601FB;
	Tue, 14 Feb 2012 13:59:10 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: =?iso-8859-1?Q?P=E1draig?= Brady <P@draigBrady.com>
Subject: Re: Adding humanize_number to coreutiles?
In-Reply-To: <4F39B38A.6070603@draigBrady.com> (=?iso-8859-1?Q?=22P=E1drai?=
	=?iso-8859-1?Q?g?= Brady"'s message of
	"Tue, 14 Feb 2012 01:06:18 +0000")
References: <CABrM6wmAm=0btpW0JT_A4=8W-bi3PAZ8SDHv1sCjcBOXyQeimQ@mail.gmail.com>
	<4F30F2D0.6010009@draigBrady.com>
	<CABrM6w=WteqGuyPB0R_+UjgNnwaqGR2UtFN0YcwwO_UbWmbhtg@mail.gmail.com>
	<4F39B38A.6070603@draigBrady.com>
Date: Tue, 14 Feb 2012 13:59:10 +0100
Message-ID: <87k43pzi3l.fsf@rho.meyering.net>
Lines: 39
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 88.168.87.75
Cc: Coreutils <coreutils@gnu.org>, Peng Yu <pengyu.ut@gmail.com>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Tue, 14 Feb 2012 12:59:19 -0000

P=C3=A1draig Brady wrote:
...
> Looking more at this, you might be right.
> Now printf already has related formatting functionality:
>
> $ env LANG=3Dfa_IR.utf8 printf "%I'd\n" 1234
> =DB=B1=D9=AC=DB=B2=DB=B3=DB=B4
>
> I was thinking it would be appropriate to add "human" into the mix like
>
> $ env LANG=3Dfa_IR.utf8 printf "%Hd\n" 1234
> 1K
>
> $ env LANG=3Dfa_IR.utf8 printf "%HId\n" 1234
> =DB=B1K
>
> But as you say there are options for humanizing.
> So would there be enough cohesive functionality one could add to such a u=
til?
> I suppose so, since one could add field processing and
> multiplier support for example.
>
> Also what to call it? humanize_number is too long I think.
> Perhaps we could use a more general name. Drats I was
> thinking of `numconv`, but that's taken:
> http://www.unixref.com/manPages/numconv.html
> Maybe `convnum`, anyway...

human-readable   too long and hyphenated
hr               short, but in the unix tradition (like od, dd, etc.)

> A tentative design could be:
>
> convnum [OPTIONS] [NUM]...
>
> Numbers are processed from stdin or the command options.

Perhaps this, instead:

  Convert numbers to human readable form.


From MAILER-DAEMON Tue Feb 14 08:31:03 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RxITH-0007ZM-Rk
	for mharc-coreutils@gnu.org; Tue, 14 Feb 2012 08:31:03 -0500
Received: from eggs.gnu.org ([140.186.70.92]:48431)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RxITC-0007ZD-3K
	for coreutils@gnu.org; Tue, 14 Feb 2012 08:31:02 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RxIT2-0000Pw-2n
	for coreutils@gnu.org; Tue, 14 Feb 2012 08:30:57 -0500
Received: from mx1.redhat.com ([209.132.183.28]:1025)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RxIT1-0000Pn-P8
	for coreutils@gnu.org; Tue, 14 Feb 2012 08:30:47 -0500
Received: from int-mx01.intmail.prod.int.phx2.redhat.com
	(int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id q1EDUirA006745
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK);
	Tue, 14 Feb 2012 08:30:44 -0500
Received: from [10.36.116.51] (ovpn-116-51.ams2.redhat.com [10.36.116.51])
	by int-mx01.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP
	id q1EDUg7G028978
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO);
	Tue, 14 Feb 2012 08:30:43 -0500
Message-ID: <4F3A6200.1020605@draigBrady.com>
Date: Tue, 14 Feb 2012 13:30:40 +0000
From: =?UTF-8?B?UMOhZHJhaWcgQnJhZHk=?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:6.0) Gecko/20110816 Thunderbird/6.0
MIME-Version: 1.0
To: Jim Meyering <jim@meyering.net>
Subject: Re: Adding humanize_number to coreutiles?
References: <CABrM6wmAm=0btpW0JT_A4=8W-bi3PAZ8SDHv1sCjcBOXyQeimQ@mail.gmail.com>
	<4F30F2D0.6010009@draigBrady.com>
	<CABrM6w=WteqGuyPB0R_+UjgNnwaqGR2UtFN0YcwwO_UbWmbhtg@mail.gmail.com>
	<4F39B38A.6070603@draigBrady.com> <87k43pzi3l.fsf@rho.meyering.net>
In-Reply-To: <87k43pzi3l.fsf@rho.meyering.net>
X-Enigmail-Version: 1.3.2
Content-Type: text/plain; charset=UTF-8
X-Scanned-By: MIMEDefang 2.67 on 10.5.11.11
Content-Transfer-Encoding: quoted-printable
X-MIME-Autoconverted: from 8bit to quoted-printable by mx1.redhat.com id
	q1EDUirA006745
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.132.183.28
Cc: Coreutils <coreutils@gnu.org>, Peng Yu <pengyu.ut@gmail.com>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Tue, 14 Feb 2012 13:31:02 -0000

On 02/14/2012 12:59 PM, Jim Meyering wrote:
> P=C3=A1draig Brady wrote:
> ...
>> Looking more at this, you might be right.
>> Now printf already has related formatting functionality:
>>
>> $ env LANG=3Dfa_IR.utf8 printf "%I'd\n" 1234
>> =DB=B1=D9=AC=DB=B2=DB=B3=DB=B4
>>
>> I was thinking it would be appropriate to add "human" into the mix lik=
e
>>
>> $ env LANG=3Dfa_IR.utf8 printf "%Hd\n" 1234
>> 1K
>>
>> $ env LANG=3Dfa_IR.utf8 printf "%HId\n" 1234
>> =DB=B1K
>>
>> But as you say there are options for humanizing.
>> So would there be enough cohesive functionality one could add to such =
a util?
>> I suppose so, since one could add field processing and
>> multiplier support for example.
>>
>> Also what to call it? humanize_number is too long I think.
>> Perhaps we could use a more general name. Drats I was
>> thinking of `numconv`, but that's taken:
>> http://www.unixref.com/manPages/numconv.html
>> Maybe `convnum`, anyway...
>=20
> human-readable   too long and hyphenated
> hr               short, but in the unix tradition (like od, dd, etc.)

Hmm, well "humanize" might be a bit of an amorphous term
to have as the base verb for the command.
Perhaps adding a --human option is clear enough,
since it would not be the only mode of operation.
I.E. support `convnum -h`, where:

  -h, --human  Same as --to=3DSI

I'm not that happy with convnum though.
What about `numfmt`. Shorter than convnum,
we already have `fmt`, and the util presented
has a central --format option.
So what I have so far is:


numfmt [OPTIONS] [NUMBER]...

Reformat NUMBER(s)
Numbers can be processed either from stdin or command arguments

--from=3D{auto,SI,IEC}
If not specified, suffixes are ignored
auto =3D> 1K -> 1000, 1Ki -> 1024
SI =3D> 1K* -> 1000
IEC =3D> 1K* -> 1024

--from-unit=3D<NUMBER>
Specify the unit size.
--from-unit=3D1 is implied if not specified

--to=3D{SI,IEC,<NUMBER>}
Auto scale the numbers to SI (powers of 1000),
or IEC (powers of 1024), so at most 3 digits are output.
Note output will be standard, without a B suffix.
I.E. 123K or 123Ki for SI and IEC respectively.
If <NUMBER> is specified use this as the scale.

--to-unit=3D<NUMBER>
Specify the output unit size.
--to-unit=3D1 is implied if not specified

--round=3D{ceiling, floor, nearest}
--round=3Dceiling is implied if not specified

-f, --format=3DFORMAT
--format=3D%d is implied if not specified
You can use this to specify a space after the number
You can also use this to perform grouping (with %'d)
You can also use this to select alternative number forms (with %Id) etc.

--suffix=3DSUFFIX
Example --suffix=3DB

--field=3DNUM
replace the number in the portion of the line delimited by whitespace.
If the new number is narrower, then pad to the same field width with spac=
es.

<NUMBER>s specified above can be numeric
with an optional suffix, like K, Ki.
(Note K should probably be SI here, unlike other coreutils).

cheers,
P=C3=A1draig.


From MAILER-DAEMON Tue Feb 14 09:58:26 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RxJpq-0000f4-R1
	for mharc-coreutils@gnu.org; Tue, 14 Feb 2012 09:58:26 -0500
Received: from eggs.gnu.org ([140.186.70.92]:36220)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <reinhard@fsfe.org>) id 1RxFdJ-00061t-TY
	for coreutils@gnu.org; Tue, 14 Feb 2012 05:29:19 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <reinhard@fsfe.org>) id 1RxFdE-0006tx-6A
	for coreutils@gnu.org; Tue, 14 Feb 2012 05:29:13 -0500
Received: from bsmtp4.bon.at ([195.3.86.186]:14884 helo=bsmtp.bon.at)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <reinhard@fsfe.org>) id 1RxFdE-0006ti-1C
	for coreutils@gnu.org; Tue, 14 Feb 2012 05:29:08 -0500
Received: from [192.168.16.153] (unknown [80.120.117.86])
	by bsmtp.bon.at (Postfix) with ESMTP id E4431CDFB8
	for <coreutils@gnu.org>; Tue, 14 Feb 2012 11:30:25 +0100 (CET)
Subject: Thank you for this great collection of tools!
From: Reinhard =?ISO-8859-1?Q?M=FCller?= <reinhard@fsfe.org>
To: coreutils@gnu.org
Content-Type: multipart/signed; micalg="pgp-sha1";
	protocol="application/pgp-signature";
	boundary="=-9Bt1W3U2/c22x+1lVJec"
Date: Tue, 14 Feb 2012 11:29:03 +0100
Message-ID: <1329215343.1901.87.camel@dublin>
Mime-Version: 1.0
X-Mailer: Evolution 2.28.3 
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 195.3.86.186
X-Mailman-Approved-At: Tue, 14 Feb 2012 09:58:23 -0500
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Tue, 14 Feb 2012 10:29:19 -0000


--=-9Bt1W3U2/c22x+1lVJec
Content-Type: text/plain; charset="ISO-8859-15"
Content-Transfer-Encoding: quoted-printable

Sending virtual hugs to everybody involved in creating and maintaining
this great collection of software! How awful, if not impossible, would
life be without ls, cat, echo, and wc!

Reinhard

http://ilovefs.org

--=-9Bt1W3U2/c22x+1lVJec
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Dies ist ein digital signierter Nachrichtenteil

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)

iJwEAAECAAYFAk86N28ACgkQKm/Tcod8puP9ggQAjLt4sja5lMbMoFNDHNU9EapK
he75Blh0coEsdYdwJeOH3mxl/TXn8qEzM5yiRxbJhm2JGrKKoy554OF0hFQgz00N
8u0tL4+xKsv7Ffe4S2N2MwU4Dyryd5B3FWyilPEEVfw1p5tVx8tIj6prQPzG1Cse
VM/5JzJORzNe6D80wEU=
=EbUj
-----END PGP SIGNATURE-----

--=-9Bt1W3U2/c22x+1lVJec--



From MAILER-DAEMON Tue Feb 14 10:25:10 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RxKFi-0004fX-5E
	for mharc-coreutils@gnu.org; Tue, 14 Feb 2012 10:25:10 -0500
Received: from eggs.gnu.org ([140.186.70.92]:34060)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1RxKFX-0004dX-3e
	for coreutils@gnu.org; Tue, 14 Feb 2012 10:25:08 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1RxKFW-0007If-7t
	for coreutils@gnu.org; Tue, 14 Feb 2012 10:24:59 -0500
Received: from mx.meyering.net ([88.168.87.75]:56154)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1RxKFW-0007IZ-1D
	for coreutils@gnu.org; Tue, 14 Feb 2012 10:24:58 -0500
Received: from rho.meyering.net (localhost.localdomain [127.0.0.1])
	by rho.meyering.net (Acme Bit-Twister) with ESMTP id 3B43360096;
	Tue, 14 Feb 2012 16:24:57 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: Reinhard =?iso-8859-1?Q?M=FCller?= <reinhard@fsfe.org>
Subject: Re: Thank you for this great collection of tools!
In-Reply-To: <1329215343.1901.87.camel@dublin> ("Reinhard
	\=\?iso-8859-1\?Q\?M\?\=
	\=\?iso-8859-1\?Q\?\=FCller\=22's\?\= message of
	"Tue, 14 Feb 2012 11:29:03 +0100")
References: <1329215343.1901.87.camel@dublin>
Date: Tue, 14 Feb 2012 16:24:57 +0100
Message-ID: <87mx8lxws6.fsf@rho.meyering.net>
Lines: 5
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 88.168.87.75
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Tue, 14 Feb 2012 15:25:08 -0000

> Sending virtual hugs to everybody involved in creating and maintaining
> this great collection of software! How awful, if not impossible, would
> life be without ls, cat, echo, and wc!

Thanks for the kind words.


From MAILER-DAEMON Tue Feb 14 11:19:39 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RxL6R-0001qn-Ss
	for mharc-coreutils@gnu.org; Tue, 14 Feb 2012 11:19:39 -0500
Received: from eggs.gnu.org ([140.186.70.92]:50760)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1RxL6J-0001if-95
	for coreutils@gnu.org; Tue, 14 Feb 2012 11:19:36 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1RxL6E-0002xc-HK
	for coreutils@gnu.org; Tue, 14 Feb 2012 11:19:31 -0500
Received: from mx.meyering.net ([88.168.87.75]:56307)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1RxL6E-0002xN-Ao
	for coreutils@gnu.org; Tue, 14 Feb 2012 11:19:26 -0500
Received: from rho.meyering.net (localhost.localdomain [127.0.0.1])
	by rho.meyering.net (Acme Bit-Twister) with ESMTP id DB6BA60096;
	Tue, 14 Feb 2012 17:19:24 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: =?iso-8859-1?Q?P=E1draig?= Brady <P@draigBrady.com>
Subject: Re: Adding humanize_number to coreutiles?
In-Reply-To: <4F3A6200.1020605@draigBrady.com> (=?iso-8859-1?Q?=22P=E1drai?=
	=?iso-8859-1?Q?g?= Brady"'s message of
	"Tue, 14 Feb 2012 13:30:40 +0000")
References: <CABrM6wmAm=0btpW0JT_A4=8W-bi3PAZ8SDHv1sCjcBOXyQeimQ@mail.gmail.com>
	<4F30F2D0.6010009@draigBrady.com>
	<CABrM6w=WteqGuyPB0R_+UjgNnwaqGR2UtFN0YcwwO_UbWmbhtg@mail.gmail.com>
	<4F39B38A.6070603@draigBrady.com> <87k43pzi3l.fsf@rho.meyering.net>
	<4F3A6200.1020605@draigBrady.com>
Date: Tue, 14 Feb 2012 17:19:24 +0100
Message-ID: <87bop1xu9f.fsf@rho.meyering.net>
Lines: 57
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 88.168.87.75
Cc: Coreutils <coreutils@gnu.org>, Peng Yu <pengyu.ut@gmail.com>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Tue, 14 Feb 2012 16:19:37 -0000

P=C3=A1draig Brady wrote:
> On 02/14/2012 12:59 PM, Jim Meyering wrote:
>> P=C3=A1draig Brady wrote:
>> ...
>>> Looking more at this, you might be right.
>>> Now printf already has related formatting functionality:
>>>
>>> $ env LANG=3Dfa_IR.utf8 printf "%I'd\n" 1234
>>> =DB=B1=D9=AC=DB=B2=DB=B3=DB=B4
>>>
>>> I was thinking it would be appropriate to add "human" into the mix like
>>>
>>> $ env LANG=3Dfa_IR.utf8 printf "%Hd\n" 1234
>>> 1K
>>>
>>> $ env LANG=3Dfa_IR.utf8 printf "%HId\n" 1234
>>> =DB=B1K
>>>
>>> But as you say there are options for humanizing.
>>> So would there be enough cohesive functionality one could add to such a=
 util?
>>> I suppose so, since one could add field processing and
>>> multiplier support for example.
>>>
>>> Also what to call it? humanize_number is too long I think.
>>> Perhaps we could use a more general name. Drats I was
>>> thinking of `numconv`, but that's taken:
>>> http://www.unixref.com/manPages/numconv.html
>>> Maybe `convnum`, anyway...
>>
>> human-readable   too long and hyphenated
>> hr               short, but in the unix tradition (like od, dd, etc.)
>
> Hmm, well "humanize" might be a bit of an amorphous term
> to have as the base verb for the command.
> Perhaps adding a --human option is clear enough,
> since it would not be the only mode of operation.
> I.E. support `convnum -h`, where:
>
>   -h, --human  Same as --to=3DSI
>
> I'm not that happy with convnum though.
> What about `numfmt`. Shorter than convnum,
> we already have `fmt`, and the util presented
> has a central --format option.
> So what I have so far is:
>
> numfmt [OPTIONS] [NUMBER]...
>
> Reformat NUMBER(s)
> Numbers can be processed either from stdin or command arguments

So far, numfmt is my favorite, too.
Good that it starts with "num", and not too long.

You might want to say something about readability of the result.
"Reformat" all by itself is just too vague unless you give an example like
"Reformat numbers like 11505426432 to the more human-readable 11G"


From MAILER-DAEMON Thu Feb 16 08:35:10 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1Ry1UM-0006f8-97
	for mharc-coreutils@gnu.org; Thu, 16 Feb 2012 08:35:10 -0500
Received: from eggs.gnu.org ([140.186.70.92]:42049)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <guido@trentalancia.com>) id 1Rxvf7-000262-1P
	for coreutils@gnu.org; Thu, 16 Feb 2012 02:21:56 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <guido@trentalancia.com>) id 1Rxvf0-0006Tx-5a
	for coreutils@gnu.org; Thu, 16 Feb 2012 02:21:52 -0500
Received: from cp-out7.libero.it ([212.52.84.107]:47107)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <guido@trentalancia.com>) id 1Rxvez-0006Tc-OF
	for coreutils@gnu.org; Thu, 16 Feb 2012 02:21:46 -0500
X-CTCH-Spam: Unknown
X-CTCH-RefID: str=0001.0A0B0204.4F3CAE88.0046,ss=1,re=0.000,fgs=0
X-libjamoibt: 1555
Received: from [192.168.2.2] (151.76.42.7) by cp-out7.libero.it (8.5.133)
	id 4F3A7FA0003E9D7A for coreutils@gnu.org;
	Thu, 16 Feb 2012 08:23:39 +0100
Message-ID: <1329376903.12559.20.camel@vortex>
Subject: [PATCH]: add an option for quiet copy operations
From: Guido Trentalancia <guido@trentalancia.com>
To: coreutils@gnu.org
Date: Thu, 16 Feb 2012 08:21:43 +0100
Content-Type: text/plain; charset="ISO-8859-15"
X-Mailer: Evolution 3.2.3 
Content-Transfer-Encoding: 7bit
Mime-Version: 1.0
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
X-Received-From: 212.52.84.107
X-Mailman-Approved-At: Thu, 16 Feb 2012 08:35:07 -0500
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 16 Feb 2012 07:21:56 -0000

Hello.

I am writing to propose a simple patch for review. The patch adds an
option "-q" or "--quiet" to cp, mv and install so that they can support
quiet operation on missing SOURCE files.

Such behavior might be useful for example when using any of the above
commands from scripts.

The patch can be applied both to the git tree and to the latest release.

Add an option "-q, --quiet" for cp, mv and install so that the program
does not complain about any missing SOURCE file.

Signed-off-by: Guido Trentalancia <guido@trentalancia.com>
---
 doc/coreutils.texi |   14 ++++++++++++++
 src/copy.c         |    3 ++-
 src/copy.h         |    3 +++
 src/cp.c           |   11 ++++++++++-
 src/install.c      |    8 +++++++-
 src/mv.c           |   10 +++++++++-
 6 files changed, 45 insertions(+), 4 deletions(-)

diff -pru coreutils-8.15-original/doc/coreutils.texi coreutils-8.15/doc/coreutils.texi
--- coreutils-8.15-original/doc/coreutils.texi	2012-01-03 16:48:48.000000000 +0100
+++ coreutils-8.15/doc/coreutils.texi	2012-02-16 07:00:21.482160794 +0100
@@ -7837,6 +7837,13 @@ cp --parents a/b/c existing_dir
 copies the file @file{a/b/c} to @file{existing_dir/a/b/c}, creating
 any missing intermediate directories.
 
+@item -q
+@itemx --quiet
+@opindex -q
+@opindex --quiet
+Do not produce an error message if any of the SOURCE arguments does not
+exist.
+
 @item -R
 @itemx -r
 @itemx --recursive
@@ -8527,6 +8534,13 @@ This option is useful if you want to use
 of installed files to keep track of when they were last built as opposed
 to when they were last installed.
 
+@item -q
+@itemx --quiet
+@opindex -q
+@opindex --quiet
+Do not produce an error message if any of the SOURCE arguments does not
+exist.
+
 @item -s
 @itemx --strip
 @opindex -s
diff -pru coreutils-8.15-original/src/copy.c coreutils-8.15/src/copy.c
--- coreutils-8.15-original/src/copy.c	2012-01-05 15:24:19.000000000 +0100
+++ coreutils-8.15/src/copy.c	2012-02-16 06:00:41.074544259 +0100
@@ -1572,7 +1572,8 @@ copy_internal (char const *src_name, cha
 
   if (XSTAT (x, src_name, &src_sb) != 0)
     {
-      error (0, errno, _("cannot stat %s"), quote (src_name));
+      if (!x->quiet)
+        error (0, errno, _("cannot stat %s"), quote (src_name));
       return false;
     }
 
diff -pru coreutils-8.15-original/src/copy.h coreutils-8.15/src/copy.h
--- coreutils-8.15-original/src/copy.h	2012-01-01 10:04:06.000000000 +0100
+++ coreutils-8.15/src/copy.h	2012-02-16 05:50:26.251633846 +0100
@@ -215,6 +215,9 @@ struct cp_options
    */
   bool reduce_diagnostics;
 
+  /* If true, do not complain when a SOURCE argument does not exist. */
+  bool quiet;
+
   /* If true, copy directories recursively and copy special files
      as themselves rather than copying their contents. */
   bool recursive;
diff -pru coreutils-8.15-original/src/cp.c coreutils-8.15/src/cp.c
--- coreutils-8.15-original/src/cp.c	2012-01-01 10:04:06.000000000 +0100
+++ coreutils-8.15/src/cp.c	2012-02-16 06:41:26.235023924 +0100
@@ -131,6 +131,7 @@ static struct option const long_opts[] =
   {"parents", no_argument, NULL, PARENTS_OPTION},
   {"path", no_argument, NULL, PARENTS_OPTION},   /* Deprecated.  */
   {"preserve", optional_argument, NULL, PRESERVE_ATTRIBUTES_OPTION},
+  {"quiet", no_argument, NULL, 'q'},
   {"recursive", no_argument, NULL, 'R'},
   {"remove-destination", no_argument, NULL, UNLINK_DEST_BEFORE_OPENING},
   {"sparse", required_argument, NULL, SPARSE_OPTION},
@@ -208,6 +209,9 @@ Mandatory arguments to long options are
       --parents                use full source file name under DIRECTORY\n\
 "), stdout);
       fputs (_("\
+  -q, --quiet                  do not complain about missing SOURCE files\n\
+"), stdout);
+      fputs (_("\
   -R, -r, --recursive          copy directories recursively\n\
       --reflink[=WHEN]         control clone/CoW copies. See below\n\
       --remove-destination     remove each existing destination file before\n\
@@ -792,6 +796,7 @@ cp_option_init (struct cp_options *x)
 
   x->data_copy_required = true;
   x->require_preserve = false;
+  x->quiet = false;
   x->recursive = false;
   x->sparse_mode = SPARSE_AUTO;
   x->symbolic_link = false;
@@ -933,7 +938,7 @@ main (int argc, char **argv)
      we'll actually use backup_suffix_string.  */
   backup_suffix_string = getenv ("SIMPLE_BACKUP_SUFFIX");
 
-  while ((c = getopt_long (argc, argv, "abdfHilLnprst:uvxPRS:T",
+  while ((c = getopt_long (argc, argv, "abdfHilLnpqrst:uvxPRS:T",
                            long_opts, NULL))
          != -1)
     {
@@ -1041,6 +1046,10 @@ main (int argc, char **argv)
           parents_option = true;
           break;
 
+        case 'q':
+          x.quiet = true;
+          break;
+
         case 'r':
         case 'R':
           x.recursive = true;
diff -pru coreutils-8.15-original/src/install.c coreutils-8.15/src/install.c
--- coreutils-8.15-original/src/install.c	2012-02-16 06:54:21.258922905 +0100
+++ coreutils-8.15/src/install.c	2012-02-16 06:55:19.465285193 +0100
@@ -129,6 +129,7 @@ static struct option const long_options[
   {"owner", required_argument, NULL, 'o'},
   {"preserve-timestamps", no_argument, NULL, 'p'},
   {"preserve-context", no_argument, NULL, PRESERVE_CONTEXT_OPTION},
+  {"quiet", no_argument, NULL, 'q'},
   {"strip", no_argument, NULL, 's'},
   {"strip-program", required_argument, NULL, STRIP_PROGRAM_OPTION},
   {"suffix", required_argument, NULL, 'S'},
@@ -280,6 +281,7 @@ cp_option_init (struct cp_options *x)
   x->require_preserve = false;
   x->require_preserve_context = false;
   x->require_preserve_xattr = false;
+  x->quiet = false;
   x->recursive = false;
   x->sparse_mode = SPARSE_AUTO;
   x->symbolic_link = false;
@@ -632,6 +634,7 @@ Mandatory arguments to long options are
       fputs (_("\
   -p, --preserve-timestamps   apply access/modification times of SOURCE files\n\
                         to corresponding destination files\n\
+  -q, --quiet         do not complain about missing SOURCE files\n\
   -s, --strip         strip symbol tables\n\
       --strip-program=PROGRAM  program used to strip binaries\n\
   -S, --suffix=SUFFIX  override the usual backup suffix\n\
@@ -783,7 +786,7 @@ main (int argc, char **argv)
      we'll actually use backup_suffix_string.  */
   backup_suffix_string = getenv ("SIMPLE_BACKUP_SUFFIX");
 
-  while ((optc = getopt_long (argc, argv, "bcCsDdg:m:o:pt:TvS:Z:", long_options,
+  while ((optc = getopt_long (argc, argv, "bcCsDdg:m:o:pqt:TvS:Z:", long_options,
                               NULL)) != -1)
     {
       switch (optc)
@@ -830,6 +833,9 @@ main (int argc, char **argv)
         case 'p':
           x.preserve_timestamps = true;
           break;
+        case 'q':
+          x.quiet = true;
+          break;
         case 'S':
           make_backups = true;
           backup_suffix_string = optarg;
diff -pru coreutils-8.15-original/src/mv.c coreutils-8.15/src/mv.c
--- coreutils-8.15-original/src/mv.c	2012-02-16 06:34:23.893273201 +0100
+++ coreutils-8.15/src/mv.c	2012-02-16 06:54:10.913858428 +0100
@@ -59,6 +59,7 @@ static struct option const long_options[
   {"interactive", no_argument, NULL, 'i'},
   {"no-clobber", no_argument, NULL, 'n'},
   {"no-target-directory", no_argument, NULL, 'T'},
+  {"quiet", no_argument, NULL, 'q'},
   {"strip-trailing-slashes", no_argument, NULL, STRIP_TRAILING_SLASHES_OPTION},
   {"suffix", required_argument, NULL, 'S'},
   {"target-directory", required_argument, NULL, 't'},
@@ -124,6 +125,7 @@ cp_option_init (struct cp_options *x)
   x->require_preserve_context = false;
   x->preserve_xattr = true;
   x->require_preserve_xattr = false;
+  x->quiet = false;
   x->recursive = true;
   x->sparse_mode = SPARSE_AUTO;  /* FIXME: maybe make this an option */
   x->symbolic_link = false;
@@ -305,6 +307,9 @@ Mandatory arguments to long options are
 If you specify more than one of -i, -f, -n, only the final one takes effect.\n\
 "), stdout);
       fputs (_("\
+  -q, --quiet                  do not complain about missing SOURCE files\n\
+"), stdout);
+      fputs (_("\
       --strip-trailing-slashes  remove any trailing slashes from each SOURCE\n\
                                  argument\n\
   -S, --suffix=SUFFIX          override the usual backup suffix\n\
@@ -368,7 +373,7 @@ main (int argc, char **argv)
      we'll actually use backup_suffix_string.  */
   backup_suffix_string = getenv ("SIMPLE_BACKUP_SUFFIX");
 
-  while ((c = getopt_long (argc, argv, "bfint:uvS:T", long_options, NULL))
+  while ((c = getopt_long (argc, argv, "bfinqt:uvS:T", long_options, NULL))
          != -1)
     {
       switch (c)
@@ -390,6 +395,9 @@ main (int argc, char **argv)
         case STRIP_TRAILING_SLASHES_OPTION:
           remove_trailing_slashes = true;
           break;
+        case 'q':
+          x.quiet = true;
+          break;
         case 't':
           if (target_directory)
             error (EXIT_FAILURE, 0, _("multiple target directories specified"));




From MAILER-DAEMON Thu Feb 16 09:02:01 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1Ry1uL-0003sE-Fi
	for mharc-coreutils@gnu.org; Thu, 16 Feb 2012 09:02:01 -0500
Received: from eggs.gnu.org ([140.186.70.92]:46719)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Ry1uB-0003rb-Lu
	for coreutils@gnu.org; Thu, 16 Feb 2012 09:01:59 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Ry1u1-00038l-Te
	for coreutils@gnu.org; Thu, 16 Feb 2012 09:01:51 -0500
Received: from mx1.redhat.com ([209.132.183.28]:58536)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Ry1u1-00038a-KZ
	for coreutils@gnu.org; Thu, 16 Feb 2012 09:01:41 -0500
Received: from int-mx01.intmail.prod.int.phx2.redhat.com
	(int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id q1GE1dfH003196
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK);
	Thu, 16 Feb 2012 09:01:39 -0500
Received: from [10.36.116.36] (ovpn-116-36.ams2.redhat.com [10.36.116.36])
	by int-mx01.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP
	id q1GE1aTk030383
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO);
	Thu, 16 Feb 2012 09:01:38 -0500
Message-ID: <4F3D0C40.2070407@draigBrady.com>
Date: Thu, 16 Feb 2012 14:01:36 +0000
From: =?ISO-8859-15?Q?P=E1draig_Brady?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:6.0) Gecko/20110816 Thunderbird/6.0
MIME-Version: 1.0
To: Guido Trentalancia <guido@trentalancia.com>
Subject: Re: [PATCH]: add an option for quiet copy operations
References: <1329376903.12559.20.camel@vortex>
In-Reply-To: <1329376903.12559.20.camel@vortex>
X-Enigmail-Version: 1.3.2
Content-Type: text/plain; charset=ISO-8859-15
X-Scanned-By: MIMEDefang 2.67 on 10.5.11.11
Content-Transfer-Encoding: quoted-printable
X-MIME-Autoconverted: from 8bit to quoted-printable by mx1.redhat.com id
	q1GE1dfH003196
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.132.183.28
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 16 Feb 2012 14:02:00 -0000

On 02/16/2012 07:21 AM, Guido Trentalancia wrote:
> Hello.
>=20
> I am writing to propose a simple patch for review. The patch adds an
> option "-q" or "--quiet" to cp, mv and install so that they can support
> quiet operation on missing SOURCE files.
>=20
> Such behavior might be useful for example when using any of the above
> commands from scripts.
>=20
> The patch can be applied both to the git tree and to the latest release.
>=20
> Add an option "-q, --quiet" for cp, mv and install so that the program
> does not complain about any missing SOURCE file.
>=20
> Signed-off-by: Guido Trentalancia <guido@trentalancia.com>

Thanks a lot for taking time to do the patch.
I'm not sure about it though.
You could easily use other tools to filter non existent arguments.
Something like:

  find file1 file2 -print0 2>/dev/null |
  xargs -r0 cp --target=3Ddest/

Or alternatively something like:

  cp $(realpath -e file1 file2 2>/dev/null) dest/

cheers,
P=E1draig.

p.s. Hmm, I wonder should `realpath -e` suppress ENOENT.
readlink -e does


From MAILER-DAEMON Thu Feb 16 09:20:49 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1Ry2CX-00077S-2n
	for mharc-coreutils@gnu.org; Thu, 16 Feb 2012 09:20:49 -0500
Received: from eggs.gnu.org ([140.186.70.92]:47653)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Ry2CP-00076o-Ns
	for coreutils@gnu.org; Thu, 16 Feb 2012 09:20:47 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Ry2CG-0006t1-Ul
	for coreutils@gnu.org; Thu, 16 Feb 2012 09:20:41 -0500
Received: from mx1.redhat.com ([209.132.183.28]:59061)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Ry2CG-0006sU-MH
	for coreutils@gnu.org; Thu, 16 Feb 2012 09:20:32 -0500
Received: from int-mx02.intmail.prod.int.phx2.redhat.com
	(int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id q1GEKU1T013847
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK);
	Thu, 16 Feb 2012 09:20:30 -0500
Received: from [10.36.116.36] (ovpn-116-36.ams2.redhat.com [10.36.116.36])
	by int-mx02.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP
	id q1GEKRgE032212
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO);
	Thu, 16 Feb 2012 09:20:29 -0500
Message-ID: <4F3D10AB.7000200@draigBrady.com>
Date: Thu, 16 Feb 2012 14:20:27 +0000
From: =?ISO-8859-15?Q?P=E1draig_Brady?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:6.0) Gecko/20110816 Thunderbird/6.0
MIME-Version: 1.0
To: Guido Trentalancia <guido@trentalancia.com>
Subject: Re: [PATCH]: add an option for quiet copy operations
References: <1329376903.12559.20.camel@vortex>
	<4F3D0C40.2070407@draigBrady.com>
In-Reply-To: <4F3D0C40.2070407@draigBrady.com>
X-Enigmail-Version: 1.3.2
Content-Type: text/plain; charset=ISO-8859-15
X-Scanned-By: MIMEDefang 2.67 on 10.5.11.12
Content-Transfer-Encoding: quoted-printable
X-MIME-Autoconverted: from 8bit to quoted-printable by mx1.redhat.com id
	q1GEKU1T013847
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.132.183.28
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 16 Feb 2012 14:20:47 -0000

On 02/16/2012 02:01 PM, P=E1draig Brady wrote:
> On 02/16/2012 07:21 AM, Guido Trentalancia wrote:
>> Hello.
>>
>> I am writing to propose a simple patch for review. The patch adds an
>> option "-q" or "--quiet" to cp, mv and install so that they can suppor=
t
>> quiet operation on missing SOURCE files.
>>
>> Such behavior might be useful for example when using any of the above
>> commands from scripts.
>>
>> The patch can be applied both to the git tree and to the latest releas=
e.
>>
>> Add an option "-q, --quiet" for cp, mv and install so that the program
>> does not complain about any missing SOURCE file.
>>
>> Signed-off-by: Guido Trentalancia <guido@trentalancia.com>
>=20
> Thanks a lot for taking time to do the patch.
> I'm not sure about it though.
> You could easily use other tools to filter non existent arguments.
> Something like:
>=20
>   find file1 file2 -print0 2>/dev/null |
>   xargs -r0 cp --target=3Ddest/
>=20
> Or alternatively something like:
>=20
>   cp $(realpath -e file1 file2 2>/dev/null) dest/
>=20
> cheers,
> P=E1draig.
>=20
> p.s. Hmm, I wonder should `realpath -e` suppress ENOENT.
> readlink -e does

Ah now I remember.

readlink is --quiet by default (so the --quiet and --silent
options are largely redundant)

realpath diagnoses ENOENT etc. by default, unless --quiet is specified.
So a better form of the above command is:

  cp $(realpath -qe file1 file2) dest/

A caveat of the above, is that it doesn't handle spaces in filenames.
To do that you'd have to use something of the original form, like:

  realpath -zqe "file 1" "file2" |
  xargs -r0 cp --target=3Ddest/

cheers,
P=E1draig.


From MAILER-DAEMON Thu Feb 16 11:59:47 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1Ry4gN-0008Jq-Mt
	for mharc-coreutils@gnu.org; Thu, 16 Feb 2012 11:59:47 -0500
Received: from eggs.gnu.org ([140.186.70.92]:40006)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <guido@trentalancia.com>) id 1Ry4gG-0008CB-HV
	for coreutils@gnu.org; Thu, 16 Feb 2012 11:59:46 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <guido@trentalancia.com>) id 1Ry4gF-0003cU-Cn
	for coreutils@gnu.org; Thu, 16 Feb 2012 11:59:40 -0500
Received: from cp-out9.libero.it ([212.52.84.109]:36889)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <guido@trentalancia.com>) id 1Ry4gF-0003cF-1n
	for coreutils@gnu.org; Thu, 16 Feb 2012 11:59:39 -0500
Received: from [192.168.2.2] (151.76.42.7) by cp-out9.libero.it (8.5.133)
	id 4F3A7992005689BB; Thu, 16 Feb 2012 17:59:37 +0100
Message-ID: <1329411576.2166.4.camel@vortex>
Subject: Re: [PATCH]: add an option for quiet copy operations
From: Guido Trentalancia <guido@trentalancia.com>
To: coreutils@gnu.org
Date: Thu, 16 Feb 2012 17:59:36 +0100
In-Reply-To: <4F3D10AB.7000200@draigBrady.com>
References: <1329376903.12559.20.camel@vortex>
	<4F3D0C40.2070407@draigBrady.com> <4F3D10AB.7000200@draigBrady.com>
Content-Type: text/plain; charset="ISO-8859-15"
X-Mailer: Evolution 3.2.3 
Mime-Version: 1.0
Content-Transfer-Encoding: Quoted-Printable
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
X-Received-From: 212.52.84.109
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 16 Feb 2012 16:59:46 -0000

Hello P=E1draig.

On Thu, 2012-02-16 at 14:20 +0000, P=E1draig Brady wrote:
> On 02/16/2012 02:01 PM, P=E1draig Brady wrote:
> > On 02/16/2012 07:21 AM, Guido Trentalancia wrote:
> >> Hello.
> >>
> >> I am writing to propose a simple patch for review. The patch adds an
> >> option "-q" or "--quiet" to cp, mv and install so that they can supp=
ort
> >> quiet operation on missing SOURCE files.

[cut]

> > Thanks a lot for taking time to do the patch.
> > I'm not sure about it though.
> > You could easily use other tools to filter non existent arguments.
> > Something like:
> >=20
> >   find file1 file2 -print0 2>/dev/null |
> >   xargs -r0 cp --target=3Ddest/
> >=20
> > Or alternatively something like:
> >=20
> >   cp $(realpath -e file1 file2 2>/dev/null) dest/

It's not very concise and it might not be straightforward to remember...

> > cheers,
> > P=E1draig.
> >=20
> > p.s. Hmm, I wonder should `realpath -e` suppress ENOENT.
> > readlink -e does
>=20
> Ah now I remember.

You see, it's definitely not straightforward to remember, as it took
about 19 minutes for you since the first message appeared on the list.

> readlink is --quiet by default (so the --quiet and --silent
> options are largely redundant)
>=20
> realpath diagnoses ENOENT etc. by default, unless --quiet is specified.
> So a better form of the above command is:
>=20
>   cp $(realpath -qe file1 file2) dest/
>=20
> A caveat of the above, is that it doesn't handle spaces in filenames.
> To do that you'd have to use something of the original form, like:
>=20
>   realpath -zqe "file 1" "file2" |
>   xargs -r0 cp --target=3Ddest/

How about "cp file1 file2 > /dev/null 2>&1" ???

> cheers,
> P=E1draig.

Regards,

Guido



From MAILER-DAEMON Thu Feb 16 14:39:47 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1Ry7BD-0005kn-Gq
	for mharc-coreutils@gnu.org; Thu, 16 Feb 2012 14:39:47 -0500
Received: from eggs.gnu.org ([140.186.70.92]:40179)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Ry7B6-0005On-62
	for coreutils@gnu.org; Thu, 16 Feb 2012 14:39:45 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Ry7B0-00045s-Vu
	for coreutils@gnu.org; Thu, 16 Feb 2012 14:39:40 -0500
Received: from mx.meyering.net ([88.168.87.75]:36171)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Ry7B0-00045l-EC
	for coreutils@gnu.org; Thu, 16 Feb 2012 14:39:34 -0500
Received: from rho.meyering.net (localhost.localdomain [127.0.0.1])
	by rho.meyering.net (Acme Bit-Twister) with ESMTP id 569B560064;
	Thu, 16 Feb 2012 20:39:33 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: Sven Breuner <sven.breuner@itwm.fraunhofer.de>
Subject: Re: "ls -l": Avoid unnecessary getxattr() overhead
In-Reply-To: <4F3710CA.8020402@itwm.fraunhofer.de> (Sven Breuner's message of
	"Sun, 12 Feb 2012 02:07:22 +0100")
References: <87pqds3snt.fsf@Apollo.jerryland.fr>
	<4F30206B.1020802@itwm.fraunhofer.de>
	<CA+TATJHorgsuY7pxmpxxAeWUHRP5MfFfzQP05o=1H6C4ttswdA@mail.gmail.com>
	<4F3032BC.50702@draigBrady.com> <4F30631A.2070009@itwm.fraunhofer.de>
	<87y5scfdiq.fsf@rho.meyering.net> <4F33B5D7.3010909@draigBrady.com>
	<87mx8sf5w0.fsf@rho.meyering.net>
	<4F3437CE.4050308@itwm.fraunhofer.de>
	<4F346CC7.4030608@draigBrady.com> <87r4y3dry0.fsf@rho.meyering.net>
	<4F3710CA.8020402@itwm.fraunhofer.de>
Date: Thu, 16 Feb 2012 20:39:33 +0100
Message-ID: <8739aao9e2.fsf@rho.meyering.net>
Lines: 360
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 88.168.87.75
Cc: Bernd Schubert <bernd.schubert@itwm.fraunhofer.de>,
	=?iso-8859-1?Q?J=E9r=E9my?= Compostella <jeremy.compostella@gmail.com>,
	"Daniel P. Berrange" <berrange@redhat.com>, coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 16 Feb 2012 19:39:45 -0000

Sven Breuner wrote:
...
> (Tested with similar results on cifs, ext4, fhgfs.)

Thanks for the testing.

Here are two patches to eliminate all unnecessary getxattr calls
(you've already seen the first) and one more to add a test script.
Sven, it should be interesting to see your speed-up numbers ;-)

The test is interesting in that it uses LD_PRELOAD, and is expected
to be skipped (or fail) on systems that don't support that.
Which makes me realize that I don't yet have code to make the test
skip when $CC and ld succeed but where LD_PRELOAD is not supported.

At first I wanted to write a root-only test that would
create a file system of some type for which *getxattr would always
fail, but with linux-kernel-based systems, that may be a challenge,
since all non-network FS types seem to have some amount of support, now.
I am avoiding tests involving network file systems: that would be
a little too invasive for my taste.

But then Daniel Berrang=E9 suggested to use LD_PRELOAD.  That's better
because the test does not require root privileges (most mount-related
tests would), but comes with a few challenges.  I'll skip the
portability ones for now.  The part that I found surprising is that
from the .so wrapper calls, while I could indeed count the number of
intercepted *getxattr calls, I could *not* access that static count
from an __attribute__((destructor)) function that would simply print
that static variable.  The printed number was always zero, because, as I
found, when the destructor is invoked, the static variable has a different
address than the one seen by the intercepted calls.  My current method
is to write a single line to file descriptor 33 from each intercepted call,
redirect that FD, and to count the lines of the resulting file.
Definitely worthy of the "FIXME" comment in that script.



>From ac6dc056f8a2ed00b86c63af50e949e1ac30dae0 Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering@redhat.com>
Date: Sat, 11 Feb 2012 12:36:57 +0100
Subject: [PATCH 1/3] ls: optimize for when getfilecon would often fail (~30%
 speed-up)

On systems or file systems without SELinux support, all getfilecon
and lgetfilecon calls would fail due to lack of support.  We can non-
invasively cache such failure (on most recently accessed device) and
avoid the vast majority of the failing underlying getxattr syscalls.
* src/ls.c (errno_unsupported): New function.
(selinux_challenged_device): New file-scoped global.
(getfilecon_cache, lgetfilecon_cache): New error-caching wrapper
functions.
(gobble_file): Use the caching wrappers, for when many *getfilecon
calls would fail with ENOTSUP or EOPNOTSUPP.
Suggested by Sven Breuner in
http://thread.gmane.org/gmane.comp.gnu.coreutils.general/2187
---
 src/ls.c |   52 +++++++++++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 49 insertions(+), 3 deletions(-)

diff --git a/src/ls.c b/src/ls.c
index f5cd37a..9e8806f 100644
--- a/src/ls.c
+++ b/src/ls.c
@@ -2777,10 +2777,56 @@ clear_files (void)
   file_size_width =3D 0;
 }

+/* Return true if ERR implies lack-of-support failure by a
+   getxattr-calling function like getfilecon.  */
+static bool
+errno_unsupported (int err)
+{
+  return err =3D=3D ENOTSUP || err =3D=3D EOPNOTSUPP;
+}
+
+/* st_dev of the most recently processed device for which
+   we've found that getfilecon or lgetfilecon fails with
+   e.g., ENOTSUP or EOPNOTSUPP.  */
+static dev_t selinux_challenged_device;
+
+/* Cache getfilecon failure, when it's trivial to do.
+   Like getfilecon, but when F's st_dev says it's on a known-
+   SELinux-challenged file system, fail with ENOTSUP immediately.  */
+static int
+getfilecon_cache (char const *file, struct fileinfo *f)
+{
+  if (f->stat.st_dev =3D=3D selinux_challenged_device)
+    {
+      errno =3D ENOTSUP;
+      return -1;
+    }
+  int r =3D getfilecon (file, &f->scontext);
+  if (r < 0 && errno_unsupported (errno))
+    selinux_challenged_device =3D f->stat.st_dev;
+  return r;
+}
+
+/* Cache lgetfilecon failure, when it's trivial to do.
+   Like lgetfilecon, but when F's st_dev says it's on a known-
+   SELinux-challenged file system, fail with ENOTSUP immediately.  */
+static int
+lgetfilecon_cache (char const *file, struct fileinfo *f)
+{
+  if (f->stat.st_dev =3D=3D selinux_challenged_device)
+    {
+      errno =3D ENOTSUP;
+      return -1;
+    }
+  int r =3D lgetfilecon (file, &f->scontext);
+  if (r < 0 && errno_unsupported (errno))
+    selinux_challenged_device =3D f->stat.st_dev;
+  return r;
+}
+
 /* Add a file to the current table of files.
    Verify that the file exists, and print an error message if it does not.
    Return the number of blocks that the file occupies.  */
-
 static uintmax_t
 gobble_file (char const *name, enum filetype type, ino_t inode,
              bool command_line_arg, char const *dirname)
@@ -2919,8 +2965,8 @@ gobble_file (char const *name, enum filetype type, in=
o_t inode,
           bool have_selinux =3D false;
           bool have_acl =3D false;
           int attr_len =3D (do_deref
-                          ?  getfilecon (absolute_name, &f->scontext)
-                          : lgetfilecon (absolute_name, &f->scontext));
+                          ? getfilecon_cache (absolute_name, f)
+                          : lgetfilecon_cache (absolute_name, f));
           err =3D (attr_len < 0);

           if (err =3D=3D 0)
--
1.7.9.1.232.g1a183


>From 796b05173ad9d34a3403e2201152a1a81fd973a1 Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering@redhat.com>
Date: Mon, 13 Feb 2012 12:05:40 +0100
Subject: [PATCH 2/3] ls: also cache ACL- and CAP-querying syscall failures

Like the optimization to avoid always-failing getfilecon calls,
this change avoids always-failing queries for whether a file has
a nontrivial ACL and for whether a file has certain "capabilities".
Those queries often resolve to getxattr syscalls, and when they
fail for one query (indicating no support), they are known always
to fail that way for the affected device.  With this change-set, we
have thus eliminated nearly all failing-unsupported getxattr syscalls.
* src/ls.c (has_capability) [!HAVE_CAP]: Set errno to ENOTSUP.
(errno_unsupported): Expand the list of E* errno values to match
that of lib/acl-internal.h's ACL_NOT_WELL_SUPPORTED macro.
(file_has_acl_cache, has_capability_cache): New functions.
(gobble_file): Use them in place of non-caching ones.
Suggested by Sven Breuner in
http://thread.gmane.org/gmane.comp.gnu.coreutils.general/2187
---
 src/ls.c |   61 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++-=
---
 1 file changed, 57 insertions(+), 4 deletions(-)

diff --git a/src/ls.c b/src/ls.c
index 9e8806f..b7c7af6 100644
--- a/src/ls.c
+++ b/src/ls.c
@@ -2736,6 +2736,7 @@ has_capability (char const *name)
 static bool
 has_capability (char const *name ATTRIBUTE_UNUSED)
 {
+  errno =3D ENOTSUP;
   return false;
 }
 #endif
@@ -2778,11 +2779,16 @@ clear_files (void)
 }

 /* Return true if ERR implies lack-of-support failure by a
-   getxattr-calling function like getfilecon.  */
+   getxattr-calling function like getfilecon or file_has_acl.  */
 static bool
 errno_unsupported (int err)
 {
-  return err =3D=3D ENOTSUP || err =3D=3D EOPNOTSUPP;
+  return (err =3D=3D EBUSY
+          || err =3D=3D EINVAL
+          || err =3D=3D ENOENT
+          || err =3D=3D ENOSYS
+          || err =3D=3D ENOTSUP
+          || err =3D=3D EOPNOTSUPP);
 }

 /* st_dev of the most recently processed device for which
@@ -2824,6 +2830,53 @@ lgetfilecon_cache (char const *file, struct fileinfo=
 *f)
   return r;
 }

+/* Cache file_has_acl failure, when it's trivial to do.
+   Like file_has_acl, but when F's st_dev says it's on a file
+   system lacking ACL support, return 0 with ENOTSUP immediately.  */
+static int
+file_has_acl_cache (char const *file, struct fileinfo *f)
+{
+  /* st_dev of the most recently processed device for which we've
+     found that file_has_acl fails indicating lack of support.  */
+  static dev_t unsupported_device;
+
+  if (f->stat.st_dev =3D=3D unsupported_device)
+    {
+      errno =3D ENOTSUP;
+      return 0;
+    }
+
+  /* Zero errno so that we can distinguish between two 0-returning cases:
+     "has-ACL-support, but only a default ACL" and "no ACL support". */
+  errno =3D 0;
+  int n =3D file_has_acl (file, &f->stat);
+  if (n <=3D 0 && errno_unsupported (errno))
+    unsupported_device =3D f->stat.st_dev;
+  return n;
+}
+
+/* Cache has_capability failure, when it's trivial to do.
+   Like has_capability, but when F's st_dev says it's on a file
+   system lacking capability support, return 0 with ENOTSUP immediately.  =
*/
+static bool
+has_capability_cache (char const *file, struct fileinfo *f)
+{
+  /* st_dev of the most recently processed device for which we've
+     found that has_capability fails indicating lack of support.  */
+  static dev_t unsupported_device;
+
+  if (f->stat.st_dev =3D=3D unsupported_device)
+    {
+      errno =3D ENOTSUP;
+      return 0;
+    }
+
+  bool b =3D has_capability (file);
+  if ( !b && errno_unsupported (errno))
+    unsupported_device =3D f->stat.st_dev;
+  return b;
+}
+
 /* Add a file to the current table of files.
    Verify that the file exists, and print an error message if it does not.
    Return the number of blocks that the file occupies.  */
@@ -2958,7 +3011,7 @@ gobble_file (char const *name, enum filetype type, in=
o_t inode,
       /* Note has_capability() adds around 30% runtime to 'ls --color'  */
       if ((type =3D=3D normal || S_ISREG (f->stat.st_mode))
           && print_with_color && is_colored (C_CAP))
-        f->has_capability =3D has_capability (absolute_name);
+        f->has_capability =3D has_capability_cache (absolute_name, f);

       if (format =3D=3D long_format || print_scontext)
         {
@@ -2985,7 +3038,7 @@ gobble_file (char const *name, enum filetype type, in=
o_t inode,

           if (err =3D=3D 0 && format =3D=3D long_format)
             {
-              int n =3D file_has_acl (absolute_name, &f->stat);
+              int n =3D file_has_acl_cache (absolute_name, f);
               err =3D (n < 0);
               have_acl =3D (0 < n);
             }
--
1.7.9.1.232.g1a183


>From 92556f46359798797963b9606e60994b9d31b1b9 Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering@redhat.com>
Date: Mon, 13 Feb 2012 12:52:19 +0100
Subject: [PATCH 3/3] tests: test for getxattr speed-up

* tests/ls/getxattr-speedup: New test.
* tests/Makefile.am (TESTS): Add it.
---
 tests/Makefile.am         |    1 +
 tests/ls/getxattr-speedup |   60 +++++++++++++++++++++++++++++++++++++++++=
++++
 2 files changed, 61 insertions(+)
 create mode 100755 tests/ls/getxattr-speedup

diff --git a/tests/Makefile.am b/tests/Makefile.am
index 7b53681..625b636 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -426,6 +426,7 @@ TESTS =3D						\
   ls/dired					\
   ls/file-type					\
   ls/follow-slink				\
+  ls/getxattr-speedup				\
   ls/infloop					\
   ls/inode					\
   ls/m-option					\
diff --git a/tests/ls/getxattr-speedup b/tests/ls/getxattr-speedup
new file mode 100755
index 0000000..f3dc5bb
--- /dev/null
+++ b/tests/ls/getxattr-speedup
@@ -0,0 +1,60 @@
+#!/bin/sh
+# Show that we've eliminated most of ls' failing getxattr syscalls,
+# regardless of how many files are in a directory we list.
+# This test is skipped on systems that lack LD_PRELOAD support; that's fin=
e.
+
+# Copyright (C) 2012 Free Software Foundation, Inc.
+
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+. "${srcdir=3D.}/init.sh"; path_prepend_ ../src
+print_ver_ ls
+
+fd=3D33
+
+# Replace each getxattr and lgetxattr call with a call to these stubs.
+# Each writes a single line to the specified FD so that we can later
+# determine the total number of calls.  FIXME: there must be a better way.
+cat > k.c <<'EOF' || framework_failure_
+#include <errno.h>
+#include <unistd.h>
+static void
+write_1 (void)
+{
+  int fd =3D @FD@;
+  write (fd, "x\n", 2);
+}
+ssize_t getxattr (const char *path, const char *name, void *value, size_t =
size)
+{ write_1 (); errno =3D ENOTSUP; return -1; }
+ssize_t lgetxattr(const char *path, const char *name, void *value, size_t =
size)
+{ write_1 (); errno =3D ENOTSUP; return -1; }
+EOF
+sed "s/@FD@/$fd/" k.c > k || framework_failure_
+mv k k.c || framework_failure_
+
+# Then compile/link it:
+$CC -fPIC -O2 -c k.c || framework_failure_ 'failed to compile with -fPIC'
+ld -G k.o -o k.so || framework_failure_ 'failed to invoke ld -G ...'
+
+# create a few files
+seq 100 | xargs touch || framework_failure_
+
+# Finally, run the test, redirecting
+eval "LD_PRELOAD=3D$PWD/k.so ls --color=3Dalways -l . $fd>x" || fail=3D1
+
+# Ensure that there were no more than 3 *getxattr calls.
+n_calls=3D$(wc -l < x)
+test "$n_calls" -le 3 || fail=3D1
+
+Exit $fail
--
1.7.9.1.232.g1a183


From MAILER-DAEMON Thu Feb 16 16:30:54 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1Ry8uk-0006Tg-2D
	for mharc-coreutils@gnu.org; Thu, 16 Feb 2012 16:30:54 -0500
Received: from eggs.gnu.org ([140.186.70.92]:44314)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1Ry8uf-0006Oi-LE
	for coreutils@gnu.org; Thu, 16 Feb 2012 16:30:51 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1Ry8uW-0003b6-TR
	for coreutils@gnu.org; Thu, 16 Feb 2012 16:30:49 -0500
Received: from mail-yw0-f41.google.com ([209.85.213.41]:36680)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1Ry8uW-0003av-NO
	for coreutils@gnu.org; Thu, 16 Feb 2012 16:30:40 -0500
Received: by yhoo22 with SMTP id o22so1890843yho.0
	for <coreutils@gnu.org>; Thu, 16 Feb 2012 13:30:39 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma;
	h=mime-version:in-reply-to:references:from:date:message-id:subject:to
	:cc:content-type;
	bh=CVZdrUzw7w4O8Sapo3fle8G7MwDvA8glh5RKXrTRM70=;
	b=ftDJ4IgZ0c9EmYoVHZBFPywIPY/SROygiJ9YRz2p+E6gnRWOBJr/FTDjUI4XEblOMr
	Zi+/RqBpNcV4o4Ibns+uTZTMeHlJmUWd2JbP24qv7bue/gNnGL4jBo+aNIEam/IVefaV
	ddqTWd3y//T4l/dkCguDxDJSsGccyN+XvgmXY=
Received: by 10.50.153.198 with SMTP id vi6mr5263124igb.30.1329427839425; Thu,
	16 Feb 2012 13:30:39 -0800 (PST)
MIME-Version: 1.0
Received: by 10.42.29.196 with HTTP; Thu, 16 Feb 2012 13:30:19 -0800 (PST)
In-Reply-To: <87vcnk3vxc.fsf@Apollo.jerryland.fr>
References: <87vcnk3vxc.fsf@Apollo.jerryland.fr>
From: =?ISO-8859-1?Q?J=E9r=E9my_Compostella?= <jeremy.compostella@gmail.com>
Date: Thu, 16 Feb 2012 22:30:19 +0100
Message-ID: <CA+TATJFys_1jcxP=UuxykbTNA_ntY3CMhNp4WWHAqbWjbPhaVw@mail.gmail.com>
Subject: Re: bug#9085: 'split' feature request: an option to uses e.g. '.001'
	as first suffix.
To: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigbrady.com>
Content-Type: multipart/mixed; boundary=e89a8f22c6811f9d7504b91b8ab8
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.85.213.41
Cc: 9085@debbugs.gnu.org, coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 16 Feb 2012 21:30:51 -0000

--e89a8f22c6811f9d7504b91b8ab8
Content-Type: multipart/alternative; boundary=e89a8f22c6811f9d7104b91b8ab6

--e89a8f22c6811f9d7104b91b8ab6
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

P=E1draig, all,

I rebased my branch for this feature and make the syntax-check
success. I attached the new patch which I hope will satisfy you.

Feel free to comment it, I will take into account whatever you want.

Cheers,

J=E9r=E9my

--e89a8f22c6811f9d7104b91b8ab6
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div>P=E1draig, all,</div><div><br></div><div>I rebased my branch for this =
feature and make the syntax-check</div><div>success. I attached the new pat=
ch which I hope will satisfy you.</div><div><br></div><div>Feel free to com=
ment it, I will take into account whatever you want.</div>

<div><br></div><div>Cheers,</div><div><br></div><div>J=E9r=E9my</div>

--e89a8f22c6811f9d7104b91b8ab6--
--e89a8f22c6811f9d7504b91b8ab8
Content-Type: text/x-patch; charset=US-ASCII; 
	name="0001-split-numeric-suffixes-new-optional-from-argument-bu.patch"
Content-Disposition: attachment; 
	filename="0001-split-numeric-suffixes-new-optional-from-argument-bu.patch"
Content-Transfer-Encoding: base64
X-Attachment-Id: f_gyqaywgq1

RnJvbSA5ZTkwZjZmMjZhYjg1ZmFjMmIxNTk5ODA0ZTMxYjc4YWFhYzQyMzcwIE1vbiBTZXAgMTcg
MDA6MDA6MDAgMjAwMQpGcm9tOiA9P1VURi04P3E/Sj1DMz1BOXI9QzM9QTlteT0yMENvbXBvc3Rl
bGxhPz0gPGplcmVteS5jb21wb3N0ZWxsYUBnbWFpbC5jb20+CkRhdGU6IFN1biwgMjkgSmFuIDIw
MTIgMTU6MjA6MzEgKzAxMDAKU3ViamVjdDogW1BBVENIXSBzcGxpdDogLS1udW1lcmljLXN1ZmZp
eGVzIG5ldyBvcHRpb25hbCAiZnJvbSIgYXJndW1lbnQgKGJ1ZyM5MDg1KQoKVGhlIHNwbGl0IGNv
bW1hbmQgbm93IGFjY2VwdHMgYSBuZXcgb3B0aW9uYWwgImZyb20iIGFyZ3VtZW50IGZvciB0aGUK
LS1udW1lcmljLXN1ZmZpeGVzIG9wdGlvbi4gSWYgdGhpcyBhcmd1bWVudCBpcyBzcGVjaWZpZWQs
IHRoZSBudW1lcmljCnN1ZmZpeCBjb3VudHMgZnJvbSB0aGlzIHZhbHVlLCBvdGhlcndpc2UsIGxp
a2UgYmVmb3JlLCBpdCBjb3VudHMgZnJvbQowLgoKKiBzcmMvc3BsaXQuYyAobmV4dF9maWxlX25h
bWUpOiBJbml0aWFsaXplIHRoZSBzdWZmaXggaW5kZXgKYWNjb3JkaW5nbHkgdG8gdGhlIGNvdW50
IGZyb20gdmFsdWUuCihtYWluKTogQ2hlY2sgdGhhdCB0aGUgc3VmZml4IGxlbmd0aCBpcyBncmVh
dGVyIGVub3VnaCBmb3IgdGhlCm51bWVyaWNhbCBzdWZmaXggY291bnQgZnJvbSB2YWx1ZS4KKiBk
b2MvY29yZXV0aWxzLnRleGkgKHNwbGl0IGludm9jYXRpb24pOiBNZW50aW9uIGl0LgoqIE5FV1Mg
KE5ldyBmZWF0dXJlcyk6IE1lbnRpb24gaXQuCiogdGVzdHMvc3BsaXQvbnVtZXJpYzogTmV3IGZp
bGUuIC0tbnVtZXJpYy1zdWZmaXhlc1s9RlJPTV0gb3B0aW9uCiAgdGVzdHMuCiogdGVzdHMvTWFr
ZWZpbGUuYW0gKFRFU1RTKTogQWRkIGl0LgpSZXF1ZXN0ZWQgYnkgU2NpRmkuCi0tLQogTkVXUyAg
ICAgICAgICAgICAgICB8ICAgIDUgKysrKwogZG9jL2NvcmV1dGlscy50ZXhpICB8ICAgIDUgKyst
CiBzcmMvc3BsaXQuYyAgICAgICAgIHwgICA2MSArKysrKysrKysrKysrKysrKysrKysrKysrKysr
KysrKysrKysrKysrKysrKysrLS0tLQogdGVzdHMvTWFrZWZpbGUuYW0gICB8ICAgIDEgKwogdGVz
dHMvc3BsaXQvbnVtZXJpYyB8ICAgNjEgKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysr
KysrKysrKysrKysrKysrKysrCiA1IGZpbGVzIGNoYW5nZWQsIDEyNiBpbnNlcnRpb25zKCspLCA3
IGRlbGV0aW9ucygtKQogY3JlYXRlIG1vZGUgMTAwNzU1IHRlc3RzL3NwbGl0L251bWVyaWMKCmRp
ZmYgLS1naXQgYS9ORVdTIGIvTkVXUwppbmRleCAyMDgwNzkwLi5mYWU2ZWNjIDEwMDY0NAotLS0g
YS9ORVdTCisrKyBiL05FV1MKQEAgLTcsNiArNywxMSBAQCBHTlUgY29yZXV0aWxzIE5FV1MgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAtKi0gb3V0bGluZSAtKi0KICAgZGQgbm93
IGFjY2VwdHMgdGhlIGNvdW50X2J5dGVzLCBza2lwX2J5dGVzIGlmbGFncyBhbmQgdGhlIGNvdW50
X2J5dGVzCiAgIG9mbGFnLCB0byBtb3JlIGVhc2lseSBhbGxvdyBwcm9jZXNzaW5nIHBvcnRpb25z
IG9mIGEgZmlsZS4KIAorICBzcGxpdCBub3cgYWNjZXB0IGFuIG9wdGlvbmFsICJmcm9tIiB2YWx1
ZSBmb3IgdGhlCisgIC0tbnVtZXJpYy1zdWZmaXhlcyBvcHRpb24uIElmIHRoaXMgYXJndW1lbnQg
aXMgc3BlY2lmaWVkLCB0aGUKKyAgbnVtZXJpYyBzdWZmaXggY291bnRzIGZyb20gdGhpcyB2YWx1
ZSwgb3RoZXJ3aXNlLCBsaWtlIGJlZm9yZSwgaXQKKyAgY291bnRzIGZyb20gMC4KKwogKiogQnVn
IGZpeGVzCiAKICAgbXYgbm93IGxldHMgeW91IG1vdmUgYSBzeW1saW5rIG9udG8gYSBzYW1lLWlu
b2RlIGRlc3RpbmF0aW9uIGZpbGUgdGhhdApkaWZmIC0tZ2l0IGEvZG9jL2NvcmV1dGlscy50ZXhp
IGIvZG9jL2NvcmV1dGlscy50ZXhpCmluZGV4IDhjNmEyODcuLjAzYTMxOTEgMTAwNjQ0Ci0tLSBh
L2RvYy9jb3JldXRpbHMudGV4aQorKysgYi9kb2MvY29yZXV0aWxzLnRleGkKQEAgLTMwODQsMTAg
KzMwODQsMTEgQEAgYW5kIHNvIGNhbiBiZSBhIHBpcGUgZm9yIGV4YW1wbGUuCiBVc2Ugc3VmZml4
ZXMgb2YgbGVuZ3RoIEB2YXJ7bGVuZ3RofS4gIFRoZSBkZWZhdWx0IEB2YXJ7bGVuZ3RofSBpcyAy
LgogCiBAaXRlbSAtZAotQGl0ZW14IC0tbnVtZXJpYy1zdWZmaXhlcworQGl0ZW14IC0tbnVtZXJp
Yy1zdWZmaXhlc1s9QHZhcntmcm9tfV0KIEBvcGluZGV4IC1kCiBAb3BpbmRleCAtLW51bWVyaWMt
c3VmZml4ZXMKLVVzZSBkaWdpdHMgaW4gc3VmZml4ZXMgcmF0aGVyIHRoYW4gbG93ZXItY2FzZSBs
ZXR0ZXJzLgorVXNlIGRpZ2l0cyBpbiBzdWZmaXhlcyByYXRoZXIgdGhhbiBsb3dlci1jYXNlIGxl
dHRlcnMuIFRoZSBudW1lcmljYWwKK3N1ZmZpeCBjb3VudHMgZnJvbSBAdmFye2Zyb219IGlmIHNw
ZWNpZmllZCwgMCBvdGhlcndpc2UuCiAKIEBpdGVtIC1lCiBAaXRlbXggLS1lbGlkZS1lbXB0eS1m
aWxlcwpkaWZmIC0tZ2l0IGEvc3JjL3NwbGl0LmMgYi9zcmMvc3BsaXQuYwppbmRleCAxZDAzMTBj
Li4wM2ZlYjlmIDEwMDY0NAotLS0gYS9zcmMvc3BsaXQuYworKysgYi9zcmMvc3BsaXQuYwpAQCAt
ODAsNiArODAsOSBAQCBzdGF0aWMgc2l6ZV90IHN1ZmZpeF9sZW5ndGg7CiAvKiBBbHBoYWJldCBv
ZiBjaGFyYWN0ZXJzIHRvIHVzZSBpbiBzdWZmaXguICAqLwogc3RhdGljIGNoYXIgY29uc3QgKnN1
ZmZpeF9hbHBoYWJldCA9ICJhYmNkZWZnaGlqa2xtbm9wcXJzdHV2d3h5eiI7CiAKKy8qIE51bWVy
aWNhbCBzdWZmaXggY291bnQgZnJvbSB2YWx1ZS4gICovCitzdGF0aWMgdW5zaWduZWQgbG9uZyBz
dWZmaXhfY291bnRfZnJvbTsKKwogLyogTmFtZSBvZiBpbnB1dCBmaWxlLiAgTWF5IGJlICItIi4g
ICovCiBzdGF0aWMgY2hhciAqaW5maWxlOwogCkBAIC0xMjIsNyArMTI1LDcgQEAgc3RhdGljIHN0
cnVjdCBvcHRpb24gY29uc3QgbG9uZ29wdHNbXSA9CiAgIHsiZWxpZGUtZW1wdHktZmlsZXMiLCBu
b19hcmd1bWVudCwgTlVMTCwgJ2UnfSwKICAgeyJ1bmJ1ZmZlcmVkIiwgbm9fYXJndW1lbnQsIE5V
TEwsICd1J30sCiAgIHsic3VmZml4LWxlbmd0aCIsIHJlcXVpcmVkX2FyZ3VtZW50LCBOVUxMLCAn
YSd9LAotICB7Im51bWVyaWMtc3VmZml4ZXMiLCBub19hcmd1bWVudCwgTlVMTCwgJ2QnfSwKKyAg
eyJudW1lcmljLXN1ZmZpeGVzIiwgb3B0aW9uYWxfYXJndW1lbnQsIE5VTEwsICdkJ30sCiAgIHsi
ZmlsdGVyIiwgcmVxdWlyZWRfYXJndW1lbnQsIE5VTEwsIEZJTFRFUl9PUFRJT059LAogICB7InZl
cmJvc2UiLCBub19hcmd1bWVudCwgTlVMTCwgVkVSQk9TRV9PUFRJT059LAogICB7Ii1pby1ibGtz
aXplIiwgcmVxdWlyZWRfYXJndW1lbnQsIE5VTEwsCkBAIC0xOTUsNyArMTk4LDggQEAgTWFuZGF0
b3J5IGFyZ3VtZW50cyB0byBsb25nIG9wdGlvbnMgYXJlIG1hbmRhdG9yeSBmb3Igc2hvcnQgb3B0
aW9ucyB0b28uXG5cCiAgIC1hLCAtLXN1ZmZpeC1sZW5ndGg9TiAgIHVzZSBzdWZmaXhlcyBvZiBs
ZW5ndGggTiAoZGVmYXVsdCAlZClcblwKICAgLWIsIC0tYnl0ZXM9U0laRSAgICAgICAgcHV0IFNJ
WkUgYnl0ZXMgcGVyIG91dHB1dCBmaWxlXG5cCiAgIC1DLCAtLWxpbmUtYnl0ZXM9U0laRSAgIHB1
dCBhdCBtb3N0IFNJWkUgYnl0ZXMgb2YgbGluZXMgcGVyIG91dHB1dCBmaWxlXG5cCi0gIC1kLCAt
LW51bWVyaWMtc3VmZml4ZXMgIHVzZSBudW1lcmljIHN1ZmZpeGVzIGluc3RlYWQgb2YgYWxwaGFi
ZXRpY1xuXAorICAtZCwgLS1udW1lcmljLXN1ZmZpeGVzWz1GUk9NXSAgdXNlIG51bWVyaWMgc3Vm
Zml4ZXMgaW5zdGVhZCBvZiBhbHBoYWJldGljLlxuXAorICAgICAgICAgICAgICAgICAgICAgICAg
V2hlbiBzcGVjaWZpZWQsIHN0YXJ0IGNvdW50aW5nIGZyb20gRlJPTSwgMCBvdGhlcndpc2VcblwK
ICAgLWUsIC0tZWxpZGUtZW1wdHktZmlsZXMgIGRvIG5vdCBnZW5lcmF0ZSBlbXB0eSBvdXRwdXQg
ZmlsZXMgd2l0aCAnLW4nXG5cCiAgICAgICAtLWZpbHRlcj1DT01NQU5EICAgIHdyaXRlIHRvIHNo
ZWxsIENPTU1BTkQ7IGZpbGUgbmFtZSBpcyAkRklMRVxuXAogICAtbCwgLS1saW5lcz1OVU1CRVIg
ICAgICBwdXQgTlVNQkVSIGxpbmVzIHBlciBvdXRwdXQgZmlsZVxuXApAQCAtMjMxLDYgKzIzNSw3
IEBAIG5leHRfZmlsZV9uYW1lICh2b2lkKQogewogICAvKiBJbmRleCBpbiBzdWZmaXhfYWxwaGFi
ZXQgb2YgZWFjaCBjaGFyYWN0ZXIgaW4gdGhlIHN1ZmZpeC4gICovCiAgIHN0YXRpYyBzaXplX3Qg
KnN1ZmluZGV4OworICBzaXplX3QgaSA9IHN1ZmZpeF9sZW5ndGg7CiAKICAgaWYgKCEgb3V0Zmls
ZSkKICAgICB7CkBAIC0yNDMsOSArMjQ4LDIzIEBAIG5leHRfZmlsZV9uYW1lICh2b2lkKQogICAg
ICAgb3V0ZmlsZSA9IHhtYWxsb2MgKG91dGZpbGVfbGVuZ3RoICsgMSk7CiAgICAgICBvdXRmaWxl
X21pZCA9IG91dGZpbGUgKyBvdXRiYXNlX2xlbmd0aDsKICAgICAgIG1lbWNweSAob3V0ZmlsZSwg
b3V0YmFzZSwgb3V0YmFzZV9sZW5ndGgpOwotICAgICAgbWVtc2V0IChvdXRmaWxlX21pZCwgc3Vm
Zml4X2FscGhhYmV0WzBdLCBzdWZmaXhfbGVuZ3RoKTsKLSAgICAgIG91dGZpbGVbb3V0ZmlsZV9s
ZW5ndGhdID0gMDsKICAgICAgIHN1ZmluZGV4ID0geGNhbGxvYyAoc3VmZml4X2xlbmd0aCwgc2l6
ZW9mICpzdWZpbmRleCk7CisgICAgICAvKiBJbml0aWFsaXplIHRoZSBzdWZmaXggaW5kZXggYWNj
b3JkaW5nbHkgdG8gdGhlIGNvdW50IGZyb20KKyAgICAgICAgIHZhbHVlLiAgKi8KKyAgICAgIHsK
KyAgICAgICAgdW5zaWduZWQgbG9uZyBsZWZ0ID0gc3VmZml4X2NvdW50X2Zyb207CisgICAgICAg
IHdoaWxlIChpLS0gIT0gMCkKKyAgICAgICAgICB7CisgICAgICAgICAgICBpZiAobGVmdCkKKyAg
ICAgICAgICAgICAgeworICAgICAgICAgICAgICAgIHN1ZmluZGV4W2ldID0gbGVmdCAlIDEwOwor
ICAgICAgICAgICAgICAgIGxlZnQgLz0gMTA7CisgICAgICAgICAgICAgIH0KKyAgICAgICAgICAg
IG91dGZpbGVfbWlkW2ldID0gc3VmZml4X2FscGhhYmV0W3N1ZmluZGV4W2ldXTsKKyAgICAgICAg
ICB9CisgICAgICB9CisKKyAgICAgIG91dGZpbGVbb3V0ZmlsZV9sZW5ndGhdID0gMDsKIAogI2lm
ICEgX1BPU0lYX05PX1RSVU5DICYmIEhBVkVfUEFUSENPTkYgJiYgZGVmaW5lZCBfUENfTkFNRV9N
QVgKICAgICAgIC8qIFBPU0lYIHJlcXVpcmVzIHRoYXQgaWYgdGhlIG91dHB1dCBmaWxlIG5hbWUg
aXMgdG9vIGxvbmcgZm9yCkBAIC0yNjUsNyArMjg0LDYgQEAgbmV4dF9maWxlX25hbWUgKHZvaWQp
CiAgICAgewogICAgICAgLyogSW5jcmVtZW50IHRoZSBzdWZmaXggaW4gcGxhY2UsIGlmIHBvc3Np
YmxlLiAgKi8KIAotICAgICAgc2l6ZV90IGkgPSBzdWZmaXhfbGVuZ3RoOwogICAgICAgd2hpbGUg
KGktLSAhPSAwKQogICAgICAgICB7CiAgICAgICAgICAgc3VmaW5kZXhbaV0rKzsKQEAgLTExNDIs
NiArMTE2MCwxOSBAQCBtYWluIChpbnQgYXJnYywgY2hhciAqKmFyZ3YpCiAKICAgICAgICAgY2Fz
ZSAnZCc6CiAgICAgICAgICAgc3VmZml4X2FscGhhYmV0ID0gIjAxMjM0NTY3ODkiOworICAgICAg
ICAgIGlmIChvcHRhcmcpCisgICAgICAgICAgICB7CisgICAgICAgICAgICAgIHVuc2lnbmVkIGxv
bmcgdG1wOworICAgICAgICAgICAgICBpZiAoeHN0cnRvdWwgKG9wdGFyZywgTlVMTCwgMTAsICZ0
bXAsICIiKSAhPSBMT05HSU5UX09LKQorICAgICAgICAgICAgICAgIHsKKyAgICAgICAgICAgICAg
ICAgIGVycm9yICgwLCAwLAorICAgICAgICAgICAgICAgICAgICAgICAgIF8oIiVzOiBpbnZhbGlk
IGNvdW50IGZyb20gbnVtZXJpY2FsIHN1ZmZpeCBudW1iZXIiKSwKKyAgICAgICAgICAgICAgICAg
ICAgICAgICBvcHRhcmcpOworICAgICAgICAgICAgICAgICAgdXNhZ2UgKEVYSVRfRkFJTFVSRSk7
CisgICAgICAgICAgICAgICAgfQorICAgICAgICAgICAgICBlbHNlCisgICAgICAgICAgICAgICAg
c3VmZml4X2NvdW50X2Zyb20gPSB0bXA7CisgICAgICAgICAgICB9CiAgICAgICAgICAgYnJlYWs7
CiAKICAgICAgICAgY2FzZSAnZSc6CkBAIC0xMjEyLDYgKzEyNDMsMjYgQEAgbWFpbiAoaW50IGFy
Z2MsIGNoYXIgKiphcmd2KQogICAgICAgdXNhZ2UgKEVYSVRfRkFJTFVSRSk7CiAgICAgfQogCisg
IC8qIENoZWNrIHRoYXQgdGhlIHN1ZmZpeCBsZW5ndGggaXMgZ3JlYXRlciBlbm91Z2ggZm9yIHRo
ZSBudW1lcmljYWwKKyAgICAgc3VmZml4IGNvdW50IGZyb20gdmFsdWUuICAqLworICBpZiAoc3Vm
Zml4X2NvdW50X2Zyb20pCisgICAgeworICAgICAgdW5zaWduZWQgbG9uZyBzdGFydCA9IHN1ZmZp
eF9jb3VudF9mcm9tOworICAgICAgc2l6ZV90IGxlbmd0aCA9IHN1ZmZpeF9sZW5ndGg7CisKKyAg
ICAgIHdoaWxlIChzdGFydCkKKyAgICAgICAgeworICAgICAgICAgIGlmIChsZW5ndGggPT0gMCkK
KyAgICAgICAgICAgIHsKKyAgICAgICAgICAgICAgZXJyb3IgKDAsIDAsIF8oIm51bWVyaWNhbCBz
dWZmaXggRlJPTSBudW1iZXIgdG9vIGhpZ2h0XAorIGZvciB0aGUgc3VmZml4IGxlbmd0aCIpKTsK
KyAgICAgICAgICAgICAgdXNhZ2UgKEVYSVRfRkFJTFVSRSk7CisgICAgICAgICAgICB9CisgICAg
ICAgICAgc3RhcnQgLz0gMTA7CisgICAgICAgICAgbGVuZ3RoLS07CisgICAgICAgIH0KKyAgICB9
CisKICAgLyogT3BlbiB0aGUgaW5wdXQgZmlsZS4gICovCiAgIGlmICghIFNUUkVRIChpbmZpbGUs
ICItIikKICAgICAgICYmIGZkX3Jlb3BlbiAoU1RESU5fRklMRU5PLCBpbmZpbGUsIE9fUkRPTkxZ
LCAwKSA8IDApCmRpZmYgLS1naXQgYS90ZXN0cy9NYWtlZmlsZS5hbSBiL3Rlc3RzL01ha2VmaWxl
LmFtCmluZGV4IDdiNTM2ODEuLmVlZDZlZDYgMTAwNjQ0Ci0tLSBhL3Rlc3RzL01ha2VmaWxlLmFt
CisrKyBiL3Rlc3RzL01ha2VmaWxlLmFtCkBAIC0yNTksNiArMjU5LDcgQEAgVEVTVFMgPQkJCQkJ
CVwKICAgc3BsaXQvbGluZXMJCQkJCVwKICAgc3BsaXQvbC1jaHVuawkJCQkJXAogICBzcGxpdC9y
LWNodW5rCQkJCQlcCisgIHNwbGl0L251bWVyaWMJCQkJCVwKICAgbWlzYy9zdGF0LWJpcnRodGlt
ZQkJCQlcCiAgIG1pc2Mvc3RhdC1mbXQJCQkJCVwKICAgbWlzYy9zdGF0LWh5cGhlbgkJCQlcCmRp
ZmYgLS1naXQgYS90ZXN0cy9zcGxpdC9udW1lcmljIGIvdGVzdHMvc3BsaXQvbnVtZXJpYwpuZXcg
ZmlsZSBtb2RlIDEwMDc1NQppbmRleCAwMDAwMDAwLi41NTUwZDZmCi0tLSAvZGV2L251bGwKKysr
IGIvdGVzdHMvc3BsaXQvbnVtZXJpYwpAQCAtMCwwICsxLDYxIEBACisjIS9iaW4vc2gKKyMgU2hv
dyB0aGF0IHNwbGl0IC0tbnVtZXJpYy1zdWZmaXhlc1s9ZnJvbV0gd29ya3MuCisKKyMgQ29weXJp
Z2h0IChDKSAyMDEyIEZyZWUgU29mdHdhcmUgRm91bmRhdGlvbiwgSW5jLgorCisjIFRoaXMgcHJv
Z3JhbSBpcyBmcmVlIHNvZnR3YXJlOiB5b3UgY2FuIHJlZGlzdHJpYnV0ZSBpdCBhbmQvb3IgbW9k
aWZ5CisjIGl0IHVuZGVyIHRoZSB0ZXJtcyBvZiB0aGUgR05VIEdlbmVyYWwgUHVibGljIExpY2Vu
c2UgYXMgcHVibGlzaGVkIGJ5CisjIHRoZSBGcmVlIFNvZnR3YXJlIEZvdW5kYXRpb24sIGVpdGhl
ciB2ZXJzaW9uIDMgb2YgdGhlIExpY2Vuc2UsIG9yCisjIChhdCB5b3VyIG9wdGlvbikgYW55IGxh
dGVyIHZlcnNpb24uCisKKyMgVGhpcyBwcm9ncmFtIGlzIGRpc3RyaWJ1dGVkIGluIHRoZSBob3Bl
IHRoYXQgaXQgd2lsbCBiZSB1c2VmdWwsCisjIGJ1dCBXSVRIT1VUIEFOWSBXQVJSQU5UWTsgd2l0
aG91dCBldmVuIHRoZSBpbXBsaWVkIHdhcnJhbnR5IG9mCisjIE1FUkNIQU5UQUJJTElUWSBvciBG
SVRORVNTIEZPUiBBIFBBUlRJQ1VMQVIgUFVSUE9TRS4gIFNlZSB0aGUKKyMgR05VIEdlbmVyYWwg
UHVibGljIExpY2Vuc2UgZm9yIG1vcmUgZGV0YWlscy4KKworIyBZb3Ugc2hvdWxkIGhhdmUgcmVj
ZWl2ZWQgYSBjb3B5IG9mIHRoZSBHTlUgR2VuZXJhbCBQdWJsaWMgTGljZW5zZQorIyBhbG9uZyB3
aXRoIHRoaXMgcHJvZ3JhbS4gIElmIG5vdCwgc2VlIDxodHRwOi8vd3d3LmdudS5vcmcvbGljZW5z
ZXMvPi4KKworLiAiJHtzcmNkaXI9Ln0vaW5pdC5zaCI7IHBhdGhfcHJlcGVuZF8gLi4vc3JjCitw
cmludF92ZXJfIHNwbGl0CisKKyMgQ2hlY2sgZGVmYXVsdCBzdGFydCBmcm9tIDAKK3ByaW50ZiAn
MVxuMlxuM1xuNFxuNVxuJyA+IGluIHx8IGZyYW1ld29ya19mYWlsdXJlXworc3BsaXQgLS1udW1l
cmljLXN1ZmZpeGVzIC0tbGluZXM9MiBpbiA+IG91dCB8fCBmYWlsPTEKK2NhdCA8PFxFT0YgPiBl
eHAtMQorMQorMgorRU9GCitjYXQgPDxcRU9GID4gZXhwLTIKKzMKKzQKK0VPRgorY2F0IDw8XEVP
RiA+IGV4cC0zCis1CitFT0YKK2NvbXBhcmUgZXhwLTEgeDAwIHx8IGZhaWw9MQorY29tcGFyZSBl
eHAtMiB4MDEgfHwgZmFpbD0xCitjb21wYXJlIGV4cC0zIHgwMiB8fCBmYWlsPTEKKworIyBDaGVj
ayAtLW51bWVyaWMtc3VmZml4ZXM9WAorc3BsaXQgLS1udW1lcmljLXN1ZmZpeGVzPTEgLS1saW5l
cz0yIGluID4gb3V0IHx8IGZhaWw9MQorY2F0IDw8XEVPRiA+IGV4cC0xCisxCisyCitFT0YKK2Nh
dCA8PFxFT0YgPiBleHAtMgorMworNAorRU9GCitjYXQgPDxcRU9GID4gZXhwLTMKKzUKK0VPRgor
Y29tcGFyZSBleHAtMSB4MDEgfHwgZmFpbD0xCitjb21wYXJlIGV4cC0yIHgwMiB8fCBmYWlsPTEK
K2NvbXBhcmUgZXhwLTMgeDAzIHx8IGZhaWw9MQorCisjIENoZWNrIHRoYXQgc3BsaXQgZmFpbGVk
IHdoZW4gc3VmZml4IGxlbmd0aCBpcyBub3QgZ3JlYXRlciBlbm91Z2ggZm9yCisjIHRoZSBudW1l
cmljYWwgc3VmZml4IGNvdW50IGZyb20gdmFsdWUKK2ZhaWw9MSAmJiBzcGxpdCAtYSAzIC0tbnVt
ZXJpYy1zdWZmaXhlcz0xMDAwIGluID4gb3V0IDI+IC9kZXYvbnVsbCB8fCBmYWlsPTAKKworRXhp
dCAkZmFpbAotLSAKMS43LjIuNQoK
--e89a8f22c6811f9d7504b91b8ab8--


From MAILER-DAEMON Thu Feb 16 21:13:20 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RyDK4-00082k-AZ
	for mharc-coreutils@gnu.org; Thu, 16 Feb 2012 21:13:20 -0500
Received: from eggs.gnu.org ([140.186.70.92]:57010)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RyDK1-00081s-T4
	for coreutils@gnu.org; Thu, 16 Feb 2012 21:13:18 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RyDK0-0004Kr-Ji
	for coreutils@gnu.org; Thu, 16 Feb 2012 21:13:17 -0500
Received: from mail3.vodafone.ie ([213.233.128.45]:30944)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RyDK0-0004Kc-60
	for coreutils@gnu.org; Thu, 16 Feb 2012 21:13:16 -0500
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: AskBAF22PU9tTBcG/2dsb2JhbAAMOK4fhUYBAQEEDG0QCw0LCQwIAg8JAwIBAgFFBg0BBwEBFsEci2sKARtBAYQagykEjk6MaIxp
Received: from unknown (HELO [192.168.1.79]) ([109.76.23.6])
	by mail3.vodafone.ie with ESMTP; 17 Feb 2012 02:13:13 +0000
Message-ID: <4F3DB7B9.6020201@draigBrady.com>
Date: Fri, 17 Feb 2012 02:13:13 +0000
From: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:6.0) Gecko/20110816 Thunderbird/6.0
MIME-Version: 1.0
To: =?ISO-8859-1?Q?J=E9r=E9my_Compostella?=
 <jeremy.compostella@gmail.com>
Subject: Re: bug#9085: 'split' feature request: an option to uses e.g. '.001'
	as first suffix.
References: <87vcnk3vxc.fsf@Apollo.jerryland.fr>
	<CA+TATJFys_1jcxP=UuxykbTNA_ntY3CMhNp4WWHAqbWjbPhaVw@mail.gmail.com>
In-Reply-To: <CA+TATJFys_1jcxP=UuxykbTNA_ntY3CMhNp4WWHAqbWjbPhaVw@mail.gmail.com>
X-Enigmail-Version: 1.3.2
Content-Type: multipart/mixed; boundary="------------020508000004040103070801"
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 213.233.128.45
Cc: 9085@debbugs.gnu.org, coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 17 Feb 2012 02:13:19 -0000

This is a multi-part message in MIME format.
--------------020508000004040103070801
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit

On 02/16/2012 09:30 PM, Jérémy Compostella wrote:
> Pádraig, all,
> 
> I rebased my branch for this feature and make the syntax-check
> success. I attached the new patch which I hope will satisfy you.
> 
> Feel free to comment it, I will take into account whatever you want.

Thanks for continuing with this.
One general thing that might both improve
and simplify the implementation, is to
not to convert from string to int at all.

I.E. when processing the arg, just validate like:
if (strlen (optarg) != strspn (optarg, suffix_alphabet))
  error()
else
   /* skip over any leading 0, and use this as the start directly. */

Then the subsequent check for length and
the initialization of the file name should be simplified.

Also this removes the limitation of size of an unsigned int,
though that's not really a practical concern I suppose.

I've also attached some string and test cleanups,
to --amend into your patch.

cheers,
Pádraig.

--------------020508000004040103070801
Content-Type: text/plain;
 name="split-numeric-suffix-adjusments.diff"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename="split-numeric-suffix-adjusments.diff"

diff --git a/NEWS b/NEWS
index fae6ecc..4662919 100644
--- a/NEWS
+++ b/NEWS
@@ -7,10 +7,8 @@ GNU coreutils NEWS                                    -*- outline -*-
   dd now accepts the count_bytes, skip_bytes iflags and the count_bytes
   oflag, to more easily allow processing portions of a file.
 
-  split now accept an optional "from" value for the
-  --numeric-suffixes option. If this argument is specified, the
-  numeric suffix counts from this value, otherwise, like before, it
-  counts from 0.
+  split now accepts an optional "from" argument to --numeric-suffixes,
+  which changes the start number from the default of 0.
 
 ** Bug fixes
 
diff --git a/doc/coreutils.texi b/doc/coreutils.texi
index f5616d8..d7a43d6 100644
--- a/doc/coreutils.texi
+++ b/doc/coreutils.texi
@@ -3087,7 +3087,7 @@ Use suffixes of length @var{length}.  The default @var{length} is 2.
 @itemx --numeric-suffixes[=@var{from}]
 @opindex -d
 @opindex --numeric-suffixes
-Use digits in suffixes rather than lower-case letters. The numerical
+Use digits in suffixes rather than lower-case letters.  The numerical
 suffix counts from @var{from} if specified, 0 otherwise.
 
 @item -e
diff --git a/src/split.c b/src/split.c
index 03feb9f..e7c61bb 100644
--- a/src/split.c
+++ b/src/split.c
@@ -249,8 +249,7 @@ next_file_name (void)
       outfile_mid = outfile + outbase_length;
       memcpy (outfile, outbase, outbase_length);
       sufindex = xcalloc (suffix_length, sizeof *sufindex);
-      /* Initialize the suffix index accordingly to the count from
-         value.  */
+      /* Initialize the suffix index according to start value.  */
       {
         unsigned long left = suffix_count_from;
         while (i-- != 0)
@@ -1166,7 +1165,7 @@ main (int argc, char **argv)
               if (xstrtoul (optarg, NULL, 10, &tmp, "") != LONGINT_OK)
                 {
                   error (0, 0,
-                         _("%s: invalid count from numerical suffix number"),
+                         _("%s: invalid start value for numerical suffix"),
                          optarg);
                   usage (EXIT_FAILURE);
                 }
@@ -1243,7 +1242,7 @@ main (int argc, char **argv)
       usage (EXIT_FAILURE);
     }
 
-  /* Check that the suffix length is greater enough for the numerical
+  /* Check that the suffix length is large enough for the numerical
      suffix count from value.  */
   if (suffix_count_from)
     {
@@ -1254,8 +1253,8 @@ main (int argc, char **argv)
         {
           if (length == 0)
             {
-              error (0, 0, _("numerical suffix FROM number too hight\
- for the suffix length"));
+              error (0, 0, _("numerical suffix start value is too large "
+                             "for the suffix length"));
               usage (EXIT_FAILURE);
             }
           start /= 10;
diff --git a/tests/split/numeric b/tests/split/numeric
index 5550d6f..ad22df6 100755
--- a/tests/split/numeric
+++ b/tests/split/numeric
@@ -21,7 +21,7 @@ print_ver_ split
 
 # Check default start from 0
 printf '1\n2\n3\n4\n5\n' > in || framework_failure_
-split --numeric-suffixes --lines=2 in > out || fail=1
+split --numeric-suffixes --lines=2 in || fail=1
 cat <<\EOF > exp-1
 1
 2
@@ -38,7 +38,7 @@ compare exp-2 x01 || fail=1
 compare exp-3 x02 || fail=1
 
 # Check --numeric-suffixes=X
-split --numeric-suffixes=1 --lines=2 in > out || fail=1
+split --numeric-suffixes=1 --lines=2 in || fail=1
 cat <<\EOF > exp-1
 1
 2
@@ -54,8 +54,12 @@ compare exp-1 x01 || fail=1
 compare exp-2 x02 || fail=1
 compare exp-3 x03 || fail=1
 
-# Check that split failed when suffix length is not greater enough for
-# the numerical suffix count from value
-fail=1 && split -a 3 --numeric-suffixes=1000 in > out 2> /dev/null || fail=0
+# Check that split failed when suffix length is not large enough for
+# the numerical suffix start value
+split -a 3 --numeric-suffixes=1000 in 2> /dev/null && fail=1
+
+# check invalid --numeric-suffixes start values are flagged
+split --numeric-suffixes=-1 in 2> /dev/null && fail=1
+split --numeric-suffixes=one in 2> /dev/null && fail=1
 
 Exit $fail

--------------020508000004040103070801--


From MAILER-DAEMON Fri Feb 17 03:42:49 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RyJOz-0003Fy-Nu
	for mharc-coreutils@gnu.org; Fri, 17 Feb 2012 03:42:49 -0500
Received: from eggs.gnu.org ([140.186.70.92]:40652)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1RyJOs-0003Fq-KJ
	for coreutils@gnu.org; Fri, 17 Feb 2012 03:42:48 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1RyJOo-0001fl-3t
	for coreutils@gnu.org; Fri, 17 Feb 2012 03:42:42 -0500
Received: from moutng.kundenserver.de ([212.227.126.186]:57345)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1RyJOn-0001fR-NR
	for coreutils@gnu.org; Fri, 17 Feb 2012 03:42:38 -0500
Received: from [192.168.2.108] (p4FF76CC3.dip.t-dialin.net [79.247.108.195])
	by mrelayeu.kundenserver.de (node=mrbap2) with ESMTP (Nemesis)
	id 0LcnP5-1SNVfB1d0J-00jGsi; Fri, 17 Feb 2012 09:42:34 +0100
Message-ID: <4F3E12F9.6080102@bernhard-voelker.de>
Date: Fri, 17 Feb 2012 09:42:33 +0100
From: Bernhard Voelker <mail@bernhard-voelker.de>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:10.0.1) Gecko/20120208 Thunderbird/10.0.1
MIME-Version: 1.0
To: Jim Meyering <jim@meyering.net>
Subject: Re: "ls -l": Avoid unnecessary getxattr() overhead
References: <87pqds3snt.fsf@Apollo.jerryland.fr>
	<4F30206B.1020802@itwm.fraunhofer.de>
	<CA+TATJHorgsuY7pxmpxxAeWUHRP5MfFfzQP05o=1H6C4ttswdA@mail.gmail.com>
	<4F3032BC.50702@draigBrady.com>
	<4F30631A.2070009@itwm.fraunhofer.de>
	<87y5scfdiq.fsf@rho.meyering.net> <4F33B5D7.3010909@draigBrady.com>
	<87mx8sf5w0.fsf@rho.meyering.net>
	<4F3437CE.4050308@itwm.fraunhofer.de>
	<4F346CC7.4030608@draigBrady.com> <87r4y3dry0.fsf@rho.meyering.net>
	<4F3710CA.8020402@itwm.fraunhofer.de>
	<8739aao9e2.fsf@rho.meyering.net>
In-Reply-To: <8739aao9e2.fsf@rho.meyering.net>
Content-Type: multipart/mixed; boundary="------------000202050308090600090606"
X-Provags-ID: V02:K0:9XLtu2rvcN5QumijsxxHSRMWepru3QRdoOjB6AKOSZ3
	MmDOoGpoHdMWpP9UcD526N+uhh2RFGvNfMYc6dDsFodsq0p02F
	fJFYvyiibtW590zLlwuZr/vhZsaZRtdcL5Fy8xgEkiA/Smct0D
	HeZ4W6v8le2LGKk/7Pq2QleipVSswMK1KfajQCOAw1Zh9WfFDh
	KF4O/fQu3NGWi9iEU9LU7kYAzw2bbHb4jFgO+prsJCzFtBfmYy
	LzG+1M7t6wJDBegutrMlMhaQsMjQQxSNWna/wFEpzcdt54m9pp
	2dvZYy7OHGwWIi/hqCwm6B1LSyQkKvxCxA6UV+cJvfTbNA92aK
	lP1BhtRfhqDqY7bOmXIoKBcfXQG4wjQh23m+8Oavh
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 212.227.126.186
Cc: Sven Breuner <sven.breuner@itwm.fraunhofer.de>,
	Bernd Schubert <bernd.schubert@itwm.fraunhofer.de>,
	=?ISO-8859-1?Q?J=E9r=E9my_Compostella?= <jeremy.compostella@gmail.com>,
	"Daniel P. Berrange" <berrange@redhat.com>, coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 17 Feb 2012 08:42:48 -0000

This is a multi-part message in MIME format.
--------------000202050308090600090606
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit

On 02/16/2012 08:39 PM, Jim Meyering wrote:
> Sven Breuner wrote:
> ...
>> (Tested with similar results on cifs, ext4, fhgfs.)
> 
> Thanks for the testing.
> 
> Here are two patches to eliminate all unnecessary getxattr calls
> (you've already seen the first) and one more to add a test script.
> Sven, it should be interesting to see your speed-up numbers ;-)
> 
> The test is interesting in that it uses LD_PRELOAD, and is expected
> to be skipped (or fail) on systems that don't support that.
> Which makes me realize that I don't yet have code to make the test
> skip when $CC and ld succeed but where LD_PRELOAD is not supported.
> 
> At first I wanted to write a root-only test that would
> create a file system of some type for which *getxattr would always
> fail, but with linux-kernel-based systems, that may be a challenge,
> since all non-network FS types seem to have some amount of support, now.
> I am avoiding tests involving network file systems: that would be
> a little too invasive for my taste.
> 
> But then Daniel Berrangé suggested to use LD_PRELOAD.  That's better

That's clever.

> because the test does not require root privileges (most mount-related
> tests would), but comes with a few challenges.  I'll skip the
> portability ones for now.  The part that I found surprising is that
> from the .so wrapper calls, while I could indeed count the number of
> intercepted *getxattr calls, I could *not* access that static count
> from an __attribute__((destructor)) function that would simply print
> that static variable.  The printed number was always zero, because, as I
> found, when the destructor is invoked, the static variable has a different
> address than the one seen by the intercepted calls.  My current method
> is to write a single line to file descriptor 33 from each intercepted call,
> redirect that FD, and to count the lines of the resulting file.
> Definitely worthy of the "FIXME" comment in that script.

Well, the test could open "x" and write directly into it, but that
doesn't make a big difference (only the test would be slightly slower).

Alternatively, it could write a certain line to stdout or stderr,
and for comparison, this pattern would have to be counted.
There's a raw patch attached to demonstrate it.


> From ac6dc056f8a2ed00b86c63af50e949e1ac30dae0 Mon Sep 17 00:00:00 2001
> From: Jim Meyering <meyering@redhat.com>
> Date: Sat, 11 Feb 2012 12:36:57 +0100
> Subject: [PATCH 1/3] ls: optimize for when getfilecon would often fail (~30%
>  speed-up)

> +static dev_t selinux_challenged_device;

Having worked with different compilers, I would rather like to see such
variables to be initialized.

Likewise in:


> Subject: [PATCH 2/3] ls: also cache ACL- and CAP-querying syscall failures

> +static int
> +file_has_acl_cache (char const *file, struct fileinfo *f)
> +{
> +  /* st_dev of the most recently processed device for which we've
> +     found that file_has_acl fails indicating lack of support.  */
> +  static dev_t unsupported_device;

and here:

> +static bool
> +has_capability_cache (char const *file, struct fileinfo *f)
> +{
> +  /* st_dev of the most recently processed device for which we've
> +     found that has_capability fails indicating lack of support.  */
> +  static dev_t unsupported_device;


Have a nice day,
Berny

--------------000202050308090600090606
Content-Type: text/x-patch;
 name="getxattr2.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename="getxattr2.patch"

diff --git a/tests/ls/getxattr-speedup b/tests/ls/getxattr-speedup
index f3dc5bb..9127a28 100755
--- a/tests/ls/getxattr-speedup
+++ b/tests/ls/getxattr-speedup
@@ -21,7 +21,8 @@
 . "${srcdir=.}/init.sh"; path_prepend_ ../src
 print_ver_ ls
 
-fd=33
+speeduptext="__SPEEDUP__"
+speeduptextlen=$( echo "$speeduptext" | wc -c )  # also count newline.
 
 # Replace each getxattr and lgetxattr call with a call to these stubs.
 # Each writes a single line to the specified FD so that we can later
@@ -32,29 +33,27 @@ cat > k.c <<'EOF' || framework_failure_
 static void
 write_1 (void)
 {
-  int fd = @FD@;
-  write (fd, "x\n", 2);
+  write (1, SPEEDUPTEXT "\n", SPEEDUPTEXTLEN);
 }
 ssize_t getxattr (const char *path, const char *name, void *value, size_t size)
 { write_1 (); errno = ENOTSUP; return -1; }
 ssize_t lgetxattr(const char *path, const char *name, void *value, size_t size)
 { write_1 (); errno = ENOTSUP; return -1; }
 EOF
-sed "s/@FD@/$fd/" k.c > k || framework_failure_
-mv k k.c || framework_failure_
 
 # Then compile/link it:
-$CC -fPIC -O2 -c k.c || framework_failure_ 'failed to compile with -fPIC'
+$CC -DSPEEDUPTEXT="\"$speeduptext\"" -DSPEEDUPTEXTLEN="$speeduptextlen" \
+  -fPIC -O2 -c k.c || framework_failure_ 'failed to compile with -fPIC'
 ld -G k.o -o k.so || framework_failure_ 'failed to invoke ld -G ...'
 
 # create a few files
 seq 100 | xargs touch || framework_failure_
 
 # Finally, run the test, redirecting
-eval "LD_PRELOAD=$PWD/k.so ls --color=always -l . $fd>x" || fail=1
+eval "LD_PRELOAD=$PWD/k.so ls --color=always -l . >x" || fail=1
 
 # Ensure that there were no more than 3 *getxattr calls.
-n_calls=$(wc -l < x)
+n_calls=$( grep "^$speeduptext\$" x | wc -l)
 test "$n_calls" -le 3 || fail=1
 
 Exit $fail

--------------000202050308090600090606--


From MAILER-DAEMON Fri Feb 17 04:19:01 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RyJy1-0003gl-4V
	for mharc-coreutils@gnu.org; Fri, 17 Feb 2012 04:19:01 -0500
Received: from eggs.gnu.org ([140.186.70.92]:41496)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1RyJxu-0003gC-Jl
	for coreutils@gnu.org; Fri, 17 Feb 2012 04:19:00 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1RyJxo-0007oR-7N
	for coreutils@gnu.org; Fri, 17 Feb 2012 04:18:54 -0500
Received: from mx.meyering.net ([88.168.87.75]:38294)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1RyJxn-0007o6-Qb
	for coreutils@gnu.org; Fri, 17 Feb 2012 04:18:48 -0500
Received: from rho.meyering.net (localhost.localdomain [127.0.0.1])
	by rho.meyering.net (Acme Bit-Twister) with ESMTP id 4BDDF60220;
	Fri, 17 Feb 2012 10:18:46 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: Bernhard Voelker <mail@bernhard-voelker.de>
Subject: Re: "ls -l": Avoid unnecessary getxattr() overhead
In-Reply-To: <4F3E12F9.6080102@bernhard-voelker.de> (Bernhard Voelker's
	message of "Fri, 17 Feb 2012 09:42:33 +0100")
References: <87pqds3snt.fsf@Apollo.jerryland.fr>
	<4F30206B.1020802@itwm.fraunhofer.de>
	<CA+TATJHorgsuY7pxmpxxAeWUHRP5MfFfzQP05o=1H6C4ttswdA@mail.gmail.com>
	<4F3032BC.50702@draigBrady.com> <4F30631A.2070009@itwm.fraunhofer.de>
	<87y5scfdiq.fsf@rho.meyering.net> <4F33B5D7.3010909@draigBrady.com>
	<87mx8sf5w0.fsf@rho.meyering.net>
	<4F3437CE.4050308@itwm.fraunhofer.de>
	<4F346CC7.4030608@draigBrady.com> <87r4y3dry0.fsf@rho.meyering.net>
	<4F3710CA.8020402@itwm.fraunhofer.de>
	<8739aao9e2.fsf@rho.meyering.net>
	<4F3E12F9.6080102@bernhard-voelker.de>
Date: Fri, 17 Feb 2012 10:18:46 +0100
Message-ID: <87obsxlsw9.fsf@rho.meyering.net>
Lines: 100
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 88.168.87.75
Cc: Sven Breuner <sven.breuner@itwm.fraunhofer.de>,
	Bernd Schubert <bernd.schubert@itwm.fraunhofer.de>,
	=?iso-8859-1?Q?J=E9r=E9my?= Compostella <jeremy.compostella@gmail.com>,
	"Daniel P. Berrange" <berrange@redhat.com>, coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 17 Feb 2012 09:19:00 -0000

Bernhard Voelker wrote:
...
>> +static dev_t selinux_challenged_device;
>
> Having worked with different compilers, I would rather like to see such
> variables to be initialized.
...

Depending on static-scoped variables to be initialized to 0
is so fundamental, and has been that way forever, if
you've seen a compiler that gets it wrong, it could not have
been good for anything other than (poor) demonstration purposes.

In any declaration like "static int VAR = 0;" the "= 0" part is
strictly optional, even from a let's-be-ultra-portable standpoint.
At least as far as I know.  Can you provide details on a compiler
that gets that wrong?

You can argue that it makes the code slightly more readable,
especially if one fails to notice the "static" part.

I just checked and see that there are plenty of instances in coreutils:

  git grep -E 'static.* = (0|NULL);'

but most are in the dustier corners, and there are well over 200
that omit the initializer.

  src/cat.c:static int newlines2 = 0;
  src/csplit.c:static struct buffer_record *head = NULL;
  src/csplit.c:static char *hold_area = NULL;
  src/csplit.c:static size_t hold_count = 0;
  src/csplit.c:static uintmax_t last_line_number = 0;
  src/csplit.c:static uintmax_t current_line = 0;
  src/csplit.c:static char *volatile filename_space = NULL;
  src/csplit.c:static char const *volatile prefix = NULL;
  src/csplit.c:static char *volatile suffix = NULL;
  src/csplit.c:static unsigned int volatile files_created = 0;
  src/csplit.c:static FILE *output_stream = NULL;
  src/csplit.c:static char *output_filename = NULL;
  src/csplit.c:  static struct buffer_record *prev_buf = NULL;
  src/csplit.c:  static size_t control_allocated = 0; /* Total space allocated. */
  src/csplit.c:  static uintmax_t last_val = 0;
  src/dd.c:static char const *input_file = NULL;
  src/dd.c:static char const *output_file = NULL;
  src/dd.c:static size_t input_blocksize = 0;
  src/dd.c:static size_t output_blocksize = 0;
  src/dd.c:static size_t conversion_blocksize = 0;
  src/dd.c:static uintmax_t skip_records = 0;
  src/dd.c:static size_t skip_bytes = 0;
  src/dd.c:static uintmax_t seek_records = 0;
  src/dd.c:static uintmax_t seek_bytes = 0;
  src/dd.c:static size_t max_bytes = 0;
  src/dd.c:static int conversions_mask = 0;
  src/dd.c:static int input_flags = 0;
  src/dd.c:static int output_flags = 0;
  src/dd.c:static int status_flags = 0;
  src/dd.c:static uintmax_t w_partial = 0;
  src/dd.c:static uintmax_t w_full = 0;
  src/dd.c:static uintmax_t r_partial = 0;
  src/dd.c:static uintmax_t r_full = 0;
  src/dd.c:static uintmax_t w_bytes = 0;
  src/dd.c:static uintmax_t r_truncate = 0;
  src/dd.c:static size_t oc = 0;
  src/dd.c:static size_t col = 0;
  src/dd.c:  static size_t pending_spaces = 0;
  src/du.c:static char const *time_style = NULL;
  src/du.c:static char const *time_format = NULL;
  src/factor.c:static mpz_t *factor = NULL;
  src/factor.c:static size_t nfactors_found = 0;
  src/factor.c:static size_t nfactors_allocated = 0;
  src/fold.c:  static char *line_out = NULL;
  src/fold.c:  static size_t allocated_out = 0;
  src/id.c:static int just_context = 0;
  src/id.c:static security_context_t context = NULL;
  src/install.c:static int selinux_enabled = 0;
  src/ls.c:static struct color_ext_type *color_ext_list = NULL;
  src/nl.c:static struct re_pattern_buffer *current_regex = NULL;
  src/nl.c:static char *header_del = NULL;
  src/nl.c:static char *body_del = NULL;
  src/nl.c:static char *footer_del = NULL;
  src/nl.c:static char *print_no_line_fmt = NULL;
  src/nl.c:  static intmax_t blank_lines = 0;	/* Consecutive blank lines so far. */
  src/od.c:static uintmax_t n_bytes_to_skip = 0;
  src/pinky.c:  static time_t now = 0;
  src/pr.c:static int chars_per_margin = 0;
  src/pr.c:static uintmax_t first_page_number = 0;
  src/pr.c:static int files_ready_to_read = 0;
  src/pr.c:static int total_files = 0;
  src/pr.c:static int col_sep_length = 0;
  src/ptx.c:static const char *break_file = NULL;	/* name of the 'Break chars' file */
  src/ptx.c:static const char *only_file = NULL;	/* name of the 'Only words' file */
  src/ptx.c:static const char *ignore_file = NULL;	/* name of the 'Ignore words' file */
  src/tac.c:  static size_t bytes_in_buffer = 0;
  src/tac.c:  static char *tempfile = NULL;
  src/test.c:   static int test_error_return = 0;
  src/tsort.c:static struct item *head = NULL;
  src/tsort.c:static struct item *zeros = NULL;
  src/tsort.c:static struct item *loop = NULL;
  src/tsort.c:static size_t n_strings = 0;


From MAILER-DAEMON Fri Feb 17 04:48:06 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RyKQA-0004rr-Oc
	for mharc-coreutils@gnu.org; Fri, 17 Feb 2012 04:48:06 -0500
Received: from eggs.gnu.org ([140.186.70.92]:46763)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1RyKQ3-0004pF-Qk
	for coreutils@gnu.org; Fri, 17 Feb 2012 04:48:05 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1RyKPx-0003hr-WA
	for coreutils@gnu.org; Fri, 17 Feb 2012 04:47:59 -0500
Received: from moutng.kundenserver.de ([212.227.17.9]:50546)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1RyKPx-0003hc-Lr
	for coreutils@gnu.org; Fri, 17 Feb 2012 04:47:53 -0500
Received: from [192.168.2.108] (p4FF76CC3.dip.t-dialin.net [79.247.108.195])
	by mrelayeu.kundenserver.de (node=mrbap0) with ESMTP (Nemesis)
	id 0MDn2s-1RhH0017oE-00HRXI; Fri, 17 Feb 2012 10:47:47 +0100
Message-ID: <4F3E2240.2060903@bernhard-voelker.de>
Date: Fri, 17 Feb 2012 10:47:44 +0100
From: Bernhard Voelker <mail@bernhard-voelker.de>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:10.0.1) Gecko/20120208 Thunderbird/10.0.1
MIME-Version: 1.0
To: Jim Meyering <jim@meyering.net>
Subject: Re: "ls -l": Avoid unnecessary getxattr() overhead
References: <87pqds3snt.fsf@Apollo.jerryland.fr>
	<4F30206B.1020802@itwm.fraunhofer.de>
	<CA+TATJHorgsuY7pxmpxxAeWUHRP5MfFfzQP05o=1H6C4ttswdA@mail.gmail.com>
	<4F3032BC.50702@draigBrady.com>
	<4F30631A.2070009@itwm.fraunhofer.de>
	<87y5scfdiq.fsf@rho.meyering.net> <4F33B5D7.3010909@draigBrady.com>
	<87mx8sf5w0.fsf@rho.meyering.net>
	<4F3437CE.4050308@itwm.fraunhofer.de>
	<4F346CC7.4030608@draigBrady.com> <87r4y3dry0.fsf@rho.meyering.net>
	<4F3710CA.8020402@itwm.fraunhofer.de>
	<8739aao9e2.fsf@rho.meyering.net>
	<4F3E12F9.6080102@bernhard-voelker.de>
	<87obsxlsw9.fsf@rho.meyering.net>
In-Reply-To: <87obsxlsw9.fsf@rho.meyering.net>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
X-Provags-ID: V02:K0:tEIXOofvi495ejN3B3E9jcaZsbZmjmswwemB1dcxfy0
	UiAMYdazZB5B6Hh8glhBrTeHoiJFw16929r/xflawVgz3Z0Bd3
	fPb4mVPwOPo6UdbT5rIyfNVHaxDB24zyKsq9EMf39fBiKJ7P0/
	A5HOXGT9f1cBEhq0cU6JvwflU0qSA8qwOt8Kg0ONY7UfdGgvNa
	M32JaSDZOQMtPF7qSOKqAV8VloRfLWqvDtMkEDl6naiK5/DeXG
	2NpNLLu/EKqfLqE8Aj7z8zlaS5IqZf/7wch0sxMJTJbbHcPpH5
	SvKD/qnfOC4Mv8ALejtbbC81b/wsHK/C8qbFAWP1ej+lpush+y
	odIxkdEVHIAcO2wcFllktGc915PdisGgy7keQdy2e
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 212.227.17.9
Cc: Sven Breuner <sven.breuner@itwm.fraunhofer.de>,
	Bernd Schubert <bernd.schubert@itwm.fraunhofer.de>,
	=?ISO-8859-1?Q?J=E9r=E9my_Compostella?= <jeremy.compostella@gmail.com>,
	"Daniel P. Berrange" <berrange@redhat.com>, coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 17 Feb 2012 09:48:05 -0000

On 02/17/2012 10:18 AM, Jim Meyering wrote:
> Bernhard Voelker wrote:
> ...
>>> +static dev_t selinux_challenged_device;
>>
>> Having worked with different compilers, I would rather like to see such
>> variables to be initialized.
> ...
> 
> Depending on static-scoped variables to be initialized to 0
> is so fundamental, and has been that way forever, if
> you've seen a compiler that gets it wrong, it could not have
> been good for anything other than (poor) demonstration purposes.

Admitted, you're right.

> In any declaration like "static int VAR = 0;" the "= 0" part is
> strictly optional, even from a let's-be-ultra-portable standpoint.
> At least as far as I know.  Can you provide details on a compiler
> that gets that wrong?

I thought I remembered issues on very old machines (~1998),
either a Sun, HP-UX or Sinix, but I don't have access to them
any more. Sinix isn't a supported platform for coreutils anyway,
isn't it?

> You can argue that it makes the code slightly more readable,
> especially if one fails to notice the "static" part.
> 
> I just checked and see that there are plenty of instances in coreutils:
> 
>   git grep -E 'static.* = (0|NULL);'
> 
> but most are in the dustier corners, and there are well over 200
> that omit the initializer.

Even if it's not readability, I think it's more about consistency.
Should we remove the instances you found?

Have a nice day,
Berny


From MAILER-DAEMON Fri Feb 17 05:35:34 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RyLA6-0002JB-DR
	for mharc-coreutils@gnu.org; Fri, 17 Feb 2012 05:35:34 -0500
Received: from eggs.gnu.org ([140.186.70.92]:43791)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1RyL9y-0002Ip-Dm
	for coreutils@gnu.org; Fri, 17 Feb 2012 05:35:32 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1RyL9o-0004tB-Cg
	for coreutils@gnu.org; Fri, 17 Feb 2012 05:35:26 -0500
Received: from mx.meyering.net ([88.168.87.75]:38493)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1RyL9o-0004su-29
	for coreutils@gnu.org; Fri, 17 Feb 2012 05:35:16 -0500
Received: from rho.meyering.net (localhost.localdomain [127.0.0.1])
	by rho.meyering.net (Acme Bit-Twister) with ESMTP id 2D97F60097;
	Fri, 17 Feb 2012 11:35:14 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: Bernhard Voelker <mail@bernhard-voelker.de>
Subject: Re: "ls -l": Avoid unnecessary getxattr() overhead
In-Reply-To: <4F3E12F9.6080102@bernhard-voelker.de> (Bernhard Voelker's
	message of "Fri, 17 Feb 2012 09:42:33 +0100")
References: <87pqds3snt.fsf@Apollo.jerryland.fr>
	<4F30206B.1020802@itwm.fraunhofer.de>
	<CA+TATJHorgsuY7pxmpxxAeWUHRP5MfFfzQP05o=1H6C4ttswdA@mail.gmail.com>
	<4F3032BC.50702@draigBrady.com> <4F30631A.2070009@itwm.fraunhofer.de>
	<87y5scfdiq.fsf@rho.meyering.net> <4F33B5D7.3010909@draigBrady.com>
	<87mx8sf5w0.fsf@rho.meyering.net>
	<4F3437CE.4050308@itwm.fraunhofer.de>
	<4F346CC7.4030608@draigBrady.com> <87r4y3dry0.fsf@rho.meyering.net>
	<4F3710CA.8020402@itwm.fraunhofer.de>
	<8739aao9e2.fsf@rho.meyering.net>
	<4F3E12F9.6080102@bernhard-voelker.de>
Date: Fri, 17 Feb 2012 11:35:14 +0100
Message-ID: <8762f5lpct.fsf@rho.meyering.net>
Lines: 72
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 88.168.87.75
Cc: Sven Breuner <sven.breuner@itwm.fraunhofer.de>,
	Bernd Schubert <bernd.schubert@itwm.fraunhofer.de>,
	=?iso-8859-1?Q?J=E9r=E9my?= Compostella <jeremy.compostella@gmail.com>,
	"Daniel P. Berrange" <berrange@redhat.com>, coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 17 Feb 2012 10:35:32 -0000

Bernhard Voelker wrote:
> Alternatively, it could write a certain line to stdout or stderr,
> and for comparison, this pattern would have to be counted.
> There's a raw patch attached to demonstrate it.
>
> diff --git a/tests/ls/getxattr-speedup b/tests/ls/getxattr-speedup
> index f3dc5bb..9127a28 100755
> --- a/tests/ls/getxattr-speedup
> +++ b/tests/ls/getxattr-speedup
> @@ -21,7 +21,8 @@
>  . "${srcdir=.}/init.sh"; path_prepend_ ../src
>  print_ver_ ls
>
> -fd=33
> +speeduptext="__SPEEDUP__"
> +speeduptextlen=$( echo "$speeduptext" | wc -c )  # also count newline.
...

Thanks for the patch.
However, while I don't particularly like using fd 33, and might
use stderr instead, I do want to avoid stdout.  With it, there seems
to be too much risk that mixing wrapper-induced output with ls's normal
output would cause trouble, e.g., if your __SPEEDUP__ token were not
written right after a newline.

Instead, I've realized that I can eliminate the need for the wc post-
processing by using the fact that our contrived output file is seekable:
rather than simply writing to fd, first seek to position 0 each time,
and write the count:

Here's a stand-alone test script.
When it completes, the output file, "x"
contains either nothing or a count of the number of calls.

  # Pretend *getxattr always fails with ENOTSUP
  fd=33

  cat > k.c <<'EOF' || exit 1
  #include <errno.h>
  #include <unistd.h>
  #include <stdio.h>
  #include <string.h>
  static unsigned long int n_calls;

  static void
  write_1 (void)
  {
    ++n_calls;
    int fd = @FD@;
    if (lseek (fd, 0, SEEK_SET) < 0)
      return;
    char buf[30];
    sprintf (buf, "%lu\n", n_calls);
    write (fd, buf, strlen (buf));
  }
  ssize_t getxattr (const char *path, const char *name, void *value, size_t size)
  { write_1 (); errno = ENOTSUP; return -1; }
  ssize_t lgetxattr(const char *path, const char *name, void *value, size_t size)
  { write_1 (); errno = ENOTSUP; return -1; }
EOF
  sed "s/@FD@/$fd/" k.c > k || exit 1
  mv k k.c || exit 1

  # Then compile/link it:
  gcc -Wall -fPIC -O2 -c k.c && ld -G k.o -o k.so

  # Finally, to test it, do this:
  eval "LD_PRELOAD=$PWD/k.so ls --color=always -l . $fd>x" || exit 1

Hmm... that means the case of no getxattr calls would still
require a little special handling to map "empty file" to "0".
With that, I still prefer the simpler, original write_1 function.


From MAILER-DAEMON Fri Feb 17 06:11:28 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RyLiq-0006R5-Nl
	for mharc-coreutils@gnu.org; Fri, 17 Feb 2012 06:11:28 -0500
Received: from eggs.gnu.org ([140.186.70.92]:58190)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1RyLik-0006B8-64
	for coreutils@gnu.org; Fri, 17 Feb 2012 06:11:27 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1RyLid-00037q-O2
	for coreutils@gnu.org; Fri, 17 Feb 2012 06:11:22 -0500
Received: from moutng.kundenserver.de ([212.227.17.8]:60823)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1RyLid-00037f-9y
	for coreutils@gnu.org; Fri, 17 Feb 2012 06:11:15 -0500
Received: from [192.168.2.108] (p4FF76CC3.dip.t-dialin.net [79.247.108.195])
	by mrelayeu.kundenserver.de (node=mreu0) with ESMTP (Nemesis)
	id 0MgJSU-1S9wp01BLS-00NgC8; Fri, 17 Feb 2012 12:11:08 +0100
Message-ID: <4F3E35C9.7050707@bernhard-voelker.de>
Date: Fri, 17 Feb 2012 12:11:05 +0100
From: Bernhard Voelker <mail@bernhard-voelker.de>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:10.0.1) Gecko/20120208 Thunderbird/10.0.1
MIME-Version: 1.0
To: Jim Meyering <jim@meyering.net>
Subject: Re: "ls -l": Avoid unnecessary getxattr() overhead
References: <87pqds3snt.fsf@Apollo.jerryland.fr>
	<4F30206B.1020802@itwm.fraunhofer.de>
	<CA+TATJHorgsuY7pxmpxxAeWUHRP5MfFfzQP05o=1H6C4ttswdA@mail.gmail.com>
	<4F3032BC.50702@draigBrady.com>
	<4F30631A.2070009@itwm.fraunhofer.de>
	<87y5scfdiq.fsf@rho.meyering.net> <4F33B5D7.3010909@draigBrady.com>
	<87mx8sf5w0.fsf@rho.meyering.net>
	<4F3437CE.4050308@itwm.fraunhofer.de>
	<4F346CC7.4030608@draigBrady.com> <87r4y3dry0.fsf@rho.meyering.net>
	<4F3710CA.8020402@itwm.fraunhofer.de>
	<8739aao9e2.fsf@rho.meyering.net>
	<4F3E12F9.6080102@bernhard-voelker.de>
	<8762f5lpct.fsf@rho.meyering.net>
In-Reply-To: <8762f5lpct.fsf@rho.meyering.net>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
X-Provags-ID: V02:K0:UdkQhRzk9tPoQNsKVW/XpEVhTUxRe8ZocFiF/RFmwXz
	KopV/NEcexJV+eHI3QLEINd55Tlf+qu9zL6Iu8yRjYtCwAHrfp
	yOBbJsTkQ9UAdJOGLd7f/gyhWGhCxHIqozqfUDFORuofeRB6Lr
	ijpRKjHU8Ab7LPnw7mFr+lJOk+U1swMhVo1Fx3VGnoo2DKqaZQ
	LR1Tz8pc7d0dXboZOdOb/Po8jz85DdWRV20bd2OoU7peP88ep1
	u3zRRCh3v6AN/40hvLtVhA6bGJd5KPcESb3nvIAR0Gri9j5ldK
	kbE+5QAoJZAm3ElCZ3g+VITAHoFrGIdvZS1lSgELB8+lQE+684
	LanKp9bu0f2JjwEO9bFMmGj+fVuy4+Yf/mZOhCqAw
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 212.227.17.8
Cc: Sven Breuner <sven.breuner@itwm.fraunhofer.de>,
	Bernd Schubert <bernd.schubert@itwm.fraunhofer.de>,
	=?ISO-8859-1?Q?J=E9r=E9my_Compostella?= <jeremy.compostella@gmail.com>,
	"Daniel P. Berrange" <berrange@redhat.com>, coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 17 Feb 2012 11:11:27 -0000

On 02/17/2012 11:35 AM, Jim Meyering wrote:
> Bernhard Voelker wrote:
>> Alternatively, it could write a certain line to stdout or stderr,
>> and for comparison, this pattern would have to be counted.

> Thanks for the patch.
> However, while I don't particularly like using fd 33, and might
> use stderr instead, I do want to avoid stdout.  With it, there seems
> to be too much risk that mixing wrapper-induced output with ls's normal
> output would cause trouble, e.g., if your __SPEEDUP__ token were not
> written right after a newline.
> 
> Instead, I've realized that I can eliminate the need for the wc post-
> processing by using the fact that our contrived output file is seekable:
> rather than simply writing to fd, first seek to position 0 each time,
> and write the count:
> 
> Here's a stand-alone test script.
> When it completes, the output file, "x"
> contains either nothing or a count of the number of calls.
> 
...
> Hmm... that means the case of no getxattr calls would still
> require a little special handling to map "empty file" to "0".
> With that, I still prefer the simpler, original write_1 function.

I agree.
We could maybe avoid the sed+mv by using a preprocessor define,
similar as I did in my previous patch:


diff --git a/tests/ls/getxattr-speedup b/tests/ls/getxattr-speedup
index f3dc5bb..58fd90a 100755
--- a/tests/ls/getxattr-speedup
+++ b/tests/ls/getxattr-speedup
@@ -32,19 +32,16 @@ cat > k.c <<'EOF' || framework_failure_
 static void
 write_1 (void)
 {
-  int fd = @FD@;
-  write (fd, "x\n", 2);
+  write (FD, "x\n", 2);
 }
 ssize_t getxattr (const char *path, const char *name, void *value, size_t size)
 { write_1 (); errno = ENOTSUP; return -1; }
 ssize_t lgetxattr(const char *path, const char *name, void *value, size_t size)
 { write_1 (); errno = ENOTSUP; return -1; }
 EOF
-sed "s/@FD@/$fd/" k.c > k || framework_failure_
-mv k k.c || framework_failure_

 # Then compile/link it:
-$CC -fPIC -O2 -c k.c || framework_failure_ 'failed to compile with -fPIC'
+$CC -DFD=$fd -fPIC -O2 -c k.c || framework_failure_ 'failed to compile with -fPIC'
 ld -G k.o -o k.so || framework_failure_ 'failed to invoke ld -G ...'

 # create a few files


BTW: I have another question: what if someone has used
"./configure --disable-xattr"? ... surprisingly nothing:
ls doesn't honor USE_XATTR. Is that correct?

Have a nice day,
Berny


From MAILER-DAEMON Fri Feb 17 06:26:45 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RyLxd-0000gK-JS
	for mharc-coreutils@gnu.org; Fri, 17 Feb 2012 06:26:45 -0500
Received: from eggs.gnu.org ([140.186.70.92]:57439)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RyLxW-0000ec-C2
	for coreutils@gnu.org; Fri, 17 Feb 2012 06:26:43 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RyLxQ-0005vJ-Kg
	for coreutils@gnu.org; Fri, 17 Feb 2012 06:26:38 -0500
Received: from mail3.vodafone.ie ([213.233.128.45]:46420)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RyLxQ-0005vC-F0
	for coreutils@gnu.org; Fri, 17 Feb 2012 06:26:32 -0500
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: ApMBAMw4Pk9tTBcG/2dsb2JhbAAMOLQBAQEBAwEnCwFGBQsLDQQDAQIKFg8JAwIBAgE9CAYNAQUCAQGHfLhdi30yCQMCAwcEBAICAwkBE4NpFAWDLwSbNoxp
Received: from unknown (HELO [192.168.1.79]) ([109.76.23.6])
	by mail3.vodafone.ie with ESMTP; 17 Feb 2012 11:26:30 +0000
Message-ID: <4F3E3966.3010803@draigBrady.com>
Date: Fri, 17 Feb 2012 11:26:30 +0000
From: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:6.0) Gecko/20110816 Thunderbird/6.0
MIME-Version: 1.0
To: Jim Meyering <jim@meyering.net>
Subject: Re: "ls -l": Avoid unnecessary getxattr() overhead
References: <87pqds3snt.fsf@Apollo.jerryland.fr>
	<4F30206B.1020802@itwm.fraunhofer.de>
	<CA+TATJHorgsuY7pxmpxxAeWUHRP5MfFfzQP05o=1H6C4ttswdA@mail.gmail.com>
	<4F3032BC.50702@draigBrady.com>
	<4F30631A.2070009@itwm.fraunhofer.de>
	<87y5scfdiq.fsf@rho.meyering.net> <4F33B5D7.3010909@draigBrady.com>
	<87mx8sf5w0.fsf@rho.meyering.net>
	<4F3437CE.4050308@itwm.fraunhofer.de>
	<4F346CC7.4030608@draigBrady.com> <87r4y3dry0.fsf@rho.meyering.net>
	<4F3710CA.8020402@itwm.fraunhofer.de>
	<8739aao9e2.fsf@rho.meyering.net>
In-Reply-To: <8739aao9e2.fsf@rho.meyering.net>
X-Enigmail-Version: 1.3.2
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 213.233.128.45
Cc: Sven Breuner <sven.breuner@itwm.fraunhofer.de>,
	Bernd Schubert <bernd.schubert@itwm.fraunhofer.de>,
	"Daniel P. Berrange" <berrange@redhat.com>,
	=?ISO-8859-1?Q?J=E9r=E9my_Compostella?= <jeremy.compostella@gmail.com>,
	coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 17 Feb 2012 11:26:44 -0000

On 02/16/2012 07:39 PM, Jim Meyering wrote:

> diff --git a/src/ls.c b/src/ls.c

> +/* st_dev of the most recently processed device for which
> +   we've found that getfilecon or lgetfilecon fails with
> +   e.g., ENOTSUP or EOPNOTSUPP.  */
> +static dev_t selinux_challenged_device;
> +
> +/* Cache getfilecon failure, when it's trivial to do.
> +   Like getfilecon, but when F's st_dev says it's on a known-
> +   SELinux-challenged file system, fail with ENOTSUP immediately.  */
> +static int
> +getfilecon_cache (char const *file, struct fileinfo *f)
> +{
> +  if (f->stat.st_dev == selinux_challenged_device)
> +    {
> +      errno = ENOTSUP;
> +      return -1;
> +    }
> +  int r = getfilecon (file, &f->scontext);
> +  if (r < 0 && errno_unsupported (errno))
> +    selinux_challenged_device = f->stat.st_dev;
> +  return r;
> +}
> +
> +/* Cache lgetfilecon failure, when it's trivial to do.
> +   Like lgetfilecon, but when F's st_dev says it's on a known-
> +   SELinux-challenged file system, fail with ENOTSUP immediately.  */
> +static int
> +lgetfilecon_cache (char const *file, struct fileinfo *f)
> +{
> +  if (f->stat.st_dev == selinux_challenged_device)
> +    {
> +      errno = ENOTSUP;
> +      return -1;
> +    }
> +  int r = lgetfilecon (file, &f->scontext);
> +  if (r < 0 && errno_unsupported (errno))
> +    selinux_challenged_device = f->stat.st_dev;
> +  return r;
> +}

I'd be inclined to refactor the above 2 functions to one like:

static int
getfilecon_cache (char const *file, struct fileinfo *f, bool do_deref)
{
  /* st_dev of the most recently processed device for which we've
     found that [l]getfilecon fails indicating lack of support.  */
  static dev_t unsupported_device;

  if (f->stat.st_dev == unsupported_device)
    {
      errno = ENOTSUP;
      return -1;
    }
  int r = (do_deref ? getfilecon : lgetfilecon) (file, &f->scontext);
  if (r < 0 && errno_unsupported (errno))
    unsupported_device = f->stat.st_dev;
  return r;
}

>>From 796b05173ad9d34a3403e2201152a1a81fd973a1 Mon Sep 17 00:00:00 2001
> From: Jim Meyering <meyering@redhat.com>
> Date: Mon, 13 Feb 2012 12:05:40 +0100
> Subject: [PATCH 2/3] ls: also cache ACL- and CAP-querying syscall failures

patch series are not apparent in git shortlog summaries etc.,
so I'd s/also//

looks good!

cheers,
Pádraig.


From MAILER-DAEMON Fri Feb 17 06:35:26 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RyM62-00067S-Uk
	for mharc-coreutils@gnu.org; Fri, 17 Feb 2012 06:35:26 -0500
Received: from eggs.gnu.org ([140.186.70.92]:39036)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RyM5u-0005sE-I5
	for coreutils@gnu.org; Fri, 17 Feb 2012 06:35:24 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RyM5o-0007mO-Sa
	for coreutils@gnu.org; Fri, 17 Feb 2012 06:35:18 -0500
Received: from mx1.redhat.com ([209.132.183.28]:63576)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RyM5o-0007mI-Ly
	for coreutils@gnu.org; Fri, 17 Feb 2012 06:35:12 -0500
Received: from int-mx09.intmail.prod.int.phx2.redhat.com
	(int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id q1HBZ9Ie014064
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK);
	Fri, 17 Feb 2012 06:35:09 -0500
Received: from pb-n5110.redhat.com (ovpn-116-60.ams2.redhat.com [10.36.116.60])
	by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP
	id q1HBZ7AM029972; Fri, 17 Feb 2012 06:35:08 -0500
From: =?UTF-8?q?P=C3=A1draig=20Brady?= <P@draigBrady.com>
To: coreutils@gnu.org
Subject: [PATCH] doc: fix a typo about seek_bytes in NEWS
Date: Fri, 17 Feb 2012 11:35:02 +0000
Message-Id: <1329478502-30135-1-git-send-email-P@draigBrady.com>
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.132.183.28
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 17 Feb 2012 11:35:24 -0000

* NEWS: s/count_bytes/seek_bytes/2
---
 NEWS |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/NEWS b/NEWS
index 2080790..7973b55 100644
--- a/NEWS
+++ b/NEWS
@@ -4,7 +4,7 @@ GNU coreutils NEWS                                    -*- outline -*-
 
 ** New features
 
-  dd now accepts the count_bytes, skip_bytes iflags and the count_bytes
+  dd now accepts the count_bytes, skip_bytes iflags and the seek_bytes
   oflag, to more easily allow processing portions of a file.
 
 ** Bug fixes
-- 
1.7.6.4



From MAILER-DAEMON Fri Feb 17 06:41:08 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RyMBY-0000lE-A8
	for mharc-coreutils@gnu.org; Fri, 17 Feb 2012 06:41:08 -0500
Received: from eggs.gnu.org ([140.186.70.92]:53169)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1RyMBR-0000X5-2F
	for coreutils@gnu.org; Fri, 17 Feb 2012 06:41:06 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1RyMBC-0000YZ-8o
	for coreutils@gnu.org; Fri, 17 Feb 2012 06:41:00 -0500
Received: from mx.meyering.net ([88.168.87.75]:38659)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1RyMBB-0000YT-TP
	for coreutils@gnu.org; Fri, 17 Feb 2012 06:40:46 -0500
Received: from rho.meyering.net (localhost.localdomain [127.0.0.1])
	by rho.meyering.net (Acme Bit-Twister) with ESMTP id 8E4596008E;
	Fri, 17 Feb 2012 12:40:44 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: =?iso-8859-1?Q?P=E1draig?= Brady <P@draigBrady.com>
Subject: Re: "ls -l": Avoid unnecessary getxattr() overhead
In-Reply-To: <4F3E3966.3010803@draigBrady.com> (=?iso-8859-1?Q?=22P=E1drai?=
	=?iso-8859-1?Q?g?= Brady"'s message of
	"Fri, 17 Feb 2012 11:26:30 +0000")
References: <87pqds3snt.fsf@Apollo.jerryland.fr>
	<4F30206B.1020802@itwm.fraunhofer.de>
	<CA+TATJHorgsuY7pxmpxxAeWUHRP5MfFfzQP05o=1H6C4ttswdA@mail.gmail.com>
	<4F3032BC.50702@draigBrady.com> <4F30631A.2070009@itwm.fraunhofer.de>
	<87y5scfdiq.fsf@rho.meyering.net> <4F33B5D7.3010909@draigBrady.com>
	<87mx8sf5w0.fsf@rho.meyering.net>
	<4F3437CE.4050308@itwm.fraunhofer.de>
	<4F346CC7.4030608@draigBrady.com> <87r4y3dry0.fsf@rho.meyering.net>
	<4F3710CA.8020402@itwm.fraunhofer.de>
	<8739aao9e2.fsf@rho.meyering.net> <4F3E3966.3010803@draigBrady.com>
Date: Fri, 17 Feb 2012 12:40:44 +0100
Message-ID: <87sji9k7r7.fsf@rho.meyering.net>
Lines: 78
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 88.168.87.75
Cc: Sven Breuner <sven.breuner@itwm.fraunhofer.de>,
	Bernd Schubert <bernd.schubert@itwm.fraunhofer.de>, "Daniel
	P. Berrange" <berrange@redhat.com>,
	=?iso-8859-1?Q?J=E9r=E9my?= Compostella <jeremy.compostella@gmail.com>,
	coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 17 Feb 2012 11:41:06 -0000

P=E1draig Brady wrote:
> On 02/16/2012 07:39 PM, Jim Meyering wrote:
>
>> diff --git a/src/ls.c b/src/ls.c
>
>> +/* st_dev of the most recently processed device for which
>> +   we've found that getfilecon or lgetfilecon fails with
>> +   e.g., ENOTSUP or EOPNOTSUPP.  */
>> +static dev_t selinux_challenged_device;
>> +
>> +/* Cache getfilecon failure, when it's trivial to do.
>> +   Like getfilecon, but when F's st_dev says it's on a known-
>> +   SELinux-challenged file system, fail with ENOTSUP immediately.  */
>> +static int
>> +getfilecon_cache (char const *file, struct fileinfo *f)
>> +{
>> +  if (f->stat.st_dev =3D=3D selinux_challenged_device)
>> +    {
>> +      errno =3D ENOTSUP;
>> +      return -1;
>> +    }
>> +  int r =3D getfilecon (file, &f->scontext);
>> +  if (r < 0 && errno_unsupported (errno))
>> +    selinux_challenged_device =3D f->stat.st_dev;
>> +  return r;
>> +}
>> +
>> +/* Cache lgetfilecon failure, when it's trivial to do.
>> +   Like lgetfilecon, but when F's st_dev says it's on a known-
>> +   SELinux-challenged file system, fail with ENOTSUP immediately.  */
>> +static int
>> +lgetfilecon_cache (char const *file, struct fileinfo *f)
>> +{
>> +  if (f->stat.st_dev =3D=3D selinux_challenged_device)
>> +    {
>> +      errno =3D ENOTSUP;
>> +      return -1;
>> +    }
>> +  int r =3D lgetfilecon (file, &f->scontext);
>> +  if (r < 0 && errno_unsupported (errno))
>> +    selinux_challenged_device =3D f->stat.st_dev;
>> +  return r;
>> +}
>
> I'd be inclined to refactor the above 2 functions to one like:

Very inclined.
Nice improvement.  Thank you!

> static int
> getfilecon_cache (char const *file, struct fileinfo *f, bool do_deref)
> {
>   /* st_dev of the most recently processed device for which we've
>      found that [l]getfilecon fails indicating lack of support.  */
>   static dev_t unsupported_device;
>
>   if (f->stat.st_dev =3D=3D unsupported_device)
>     {
>       errno =3D ENOTSUP;
>       return -1;
>     }
>   int r =3D (do_deref ? getfilecon : lgetfilecon) (file, &f->scontext);
>   if (r < 0 && errno_unsupported (errno))
>     unsupported_device =3D f->stat.st_dev;
>   return r;
> }
>
>>>From 796b05173ad9d34a3403e2201152a1a81fd973a1 Mon Sep 17 00:00:00 2001
>> From: Jim Meyering <meyering@redhat.com>
>> Date: Mon, 13 Feb 2012 12:05:40 +0100
>> Subject: [PATCH 2/3] ls: also cache ACL- and CAP-querying syscall failur=
es
>
> patch series are not apparent in git shortlog summaries etc.,
> so I'd s/also//

Thanks again.

Plus, I still have to mention this in NEWS.


From MAILER-DAEMON Fri Feb 17 07:12:02 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RyMfS-0006Q7-59
	for mharc-coreutils@gnu.org; Fri, 17 Feb 2012 07:12:02 -0500
Received: from eggs.gnu.org ([140.186.70.92]:36033)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1RyMfI-00064E-2M
	for coreutils@gnu.org; Fri, 17 Feb 2012 07:11:58 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1RyMf9-0006ge-QU
	for coreutils@gnu.org; Fri, 17 Feb 2012 07:11:51 -0500
Received: from mx.meyering.net ([88.168.87.75]:38744)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1RyMf9-0006gW-19
	for coreutils@gnu.org; Fri, 17 Feb 2012 07:11:43 -0500
Received: from rho.meyering.net (localhost.localdomain [127.0.0.1])
	by rho.meyering.net (Acme Bit-Twister) with ESMTP id 067FD60064;
	Fri, 17 Feb 2012 13:11:41 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: Bernhard Voelker <mail@bernhard-voelker.de>
Subject: Re: "ls -l": Avoid unnecessary getxattr() overhead
In-Reply-To: <4F3E35C9.7050707@bernhard-voelker.de> (Bernhard Voelker's
	message of "Fri, 17 Feb 2012 12:11:05 +0100")
References: <87pqds3snt.fsf@Apollo.jerryland.fr>
	<4F30206B.1020802@itwm.fraunhofer.de>
	<CA+TATJHorgsuY7pxmpxxAeWUHRP5MfFfzQP05o=1H6C4ttswdA@mail.gmail.com>
	<4F3032BC.50702@draigBrady.com> <4F30631A.2070009@itwm.fraunhofer.de>
	<87y5scfdiq.fsf@rho.meyering.net> <4F33B5D7.3010909@draigBrady.com>
	<87mx8sf5w0.fsf@rho.meyering.net>
	<4F3437CE.4050308@itwm.fraunhofer.de>
	<4F346CC7.4030608@draigBrady.com> <87r4y3dry0.fsf@rho.meyering.net>
	<4F3710CA.8020402@itwm.fraunhofer.de>
	<8739aao9e2.fsf@rho.meyering.net>
	<4F3E12F9.6080102@bernhard-voelker.de>
	<8762f5lpct.fsf@rho.meyering.net>
	<4F3E35C9.7050707@bernhard-voelker.de>
Date: Fri, 17 Feb 2012 13:11:41 +0100
Message-ID: <87mx8hk6bm.fsf@rho.meyering.net>
Lines: 409
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 88.168.87.75
Cc: Sven Breuner <sven.breuner@itwm.fraunhofer.de>,
	Bernd Schubert <bernd.schubert@itwm.fraunhofer.de>,
	=?iso-8859-1?Q?J=E9r=E9my?= Compostella <jeremy.compostella@gmail.com>,
	"Daniel P. Berrange" <berrange@redhat.com>, coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 17 Feb 2012 12:11:58 -0000

Bernhard Voelker wrote:
> On 02/17/2012 11:35 AM, Jim Meyering wrote:
>> Bernhard Voelker wrote:
>>> Alternatively, it could write a certain line to stdout or stderr,
>>> and for comparison, this pattern would have to be counted.
>
>> Thanks for the patch.
>> However, while I don't particularly like using fd 33, and might
>> use stderr instead, I do want to avoid stdout.  With it, there seems
>> to be too much risk that mixing wrapper-induced output with ls's normal
>> output would cause trouble, e.g., if your __SPEEDUP__ token were not
>> written right after a newline.
>>
>> Instead, I've realized that I can eliminate the need for the wc post-
>> processing by using the fact that our contrived output file is seekable:
>> rather than simply writing to fd, first seek to position 0 each time,
>> and write the count:
>>
>> Here's a stand-alone test script.
>> When it completes, the output file, "x"
>> contains either nothing or a count of the number of calls.
>>
> ...
>> Hmm... that means the case of no getxattr calls would still
>> require a little special handling to map "empty file" to "0".
>> With that, I still prefer the simpler, original write_1 function.
>
> I agree.
> We could maybe avoid the sed+mv by using a preprocessor define,
> similar as I did in my previous patch:
>
>
> diff --git a/tests/ls/getxattr-speedup b/tests/ls/getxattr-speedup
> index f3dc5bb..58fd90a 100755
> --- a/tests/ls/getxattr-speedup
> +++ b/tests/ls/getxattr-speedup
> @@ -32,19 +32,16 @@ cat > k.c <<'EOF' || framework_failure_
>  static void
>  write_1 (void)
>  {
> -  int fd =3D @FD@;
> -  write (fd, "x\n", 2);
> +  write (FD, "x\n", 2);
>  }
>  ssize_t getxattr (const char *path, const char *name, void *value, size_=
t size)
>  { write_1 (); errno =3D ENOTSUP; return -1; }
>  ssize_t lgetxattr(const char *path, const char *name, void *value, size_=
t size)
>  { write_1 (); errno =3D ENOTSUP; return -1; }
>  EOF
> -sed "s/@FD@/$fd/" k.c > k || framework_failure_
> -mv k k.c || framework_failure_
>
>  # Then compile/link it:
> -$CC -fPIC -O2 -c k.c || framework_failure_ 'failed to compile with -fPIC'
> +$CC -DFD=3D$fd -fPIC -O2 -c k.c || framework_failure_ 'failed to
> compile with -fPIC'
>  ld -G k.o -o k.so || framework_failure_ 'failed to invoke ld -G ...'

Definitely an improvement.  Thank you!

> BTW: I have another question: what if someone has used
> "./configure --disable-xattr"? ... surprisingly nothing:
> ls doesn't honor USE_XATTR. Is that correct?

That option is intended to disable copy.c's attribute-copying code.
I vaguely recall that initial libxattr implementations had problems.
It would not be feasible (or sensible, imho) to eliminate all getxattr
uses from ls.c.

Thanks again for all of the review comments.
Here's the updates series, along with this addition to NEWS:

** Improvements

  ls can be much more efficient, especially with large directories on file
  systems for which getfilecon-, ACL-check- and XATTR-check-induced syscalls
  fail with ENOTSUP or similar.


>From 783f67f9ba1823414e8ff12492f5a73aeb93a627 Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering@redhat.com>
Date: Sat, 11 Feb 2012 12:36:57 +0100
Subject: [PATCH 1/3] ls: optimize for when getfilecon would often fail (~33%
 perf. gain)
MIME-Version: 1.0
Content-Type: text/plain; charset=3DUTF-8
Content-Transfer-Encoding: 8bit

On systems or file systems without SELinux support, all getfilecon
and lgetfilecon calls would fail due to lack of support.  We can non-
invasively cache such failure (on most recently accessed device) and
avoid the vast majority of the failing underlying getxattr syscalls.
* src/ls.c (errno_unsupported): New function.
(selinux_challenged_device): New file-scoped global.
(getfilecon_cache, lgetfilecon_cache): New error-caching wrapper
functions.
(gobble_file): Use the caching wrappers, for when many *getfilecon
calls would fail with ENOTSUP or EOPNOTSUPP.
Suggested by Sven Breuner in
http://thread.gmane.org/gmane.comp.gnu.coreutils.general/2187
Improved-by: P=E1draig Brady.
---
 src/ls.c |   36 ++++++++++++++++++++++++++++++++----
 1 file changed, 32 insertions(+), 4 deletions(-)

diff --git a/src/ls.c b/src/ls.c
index f5cd37a..8a9cb41 100644
--- a/src/ls.c
+++ b/src/ls.c
@@ -2777,10 +2777,40 @@ clear_files (void)
   file_size_width =3D 0;
 }

+/* Return true if ERR implies lack-of-support failure by a
+   getxattr-calling function like getfilecon.  */
+static bool
+errno_unsupported (int err)
+{
+  return err =3D=3D ENOTSUP || err =3D=3D EOPNOTSUPP;
+}
+
+/* Cache *getfilecon failure, when it's trivial to do so.
+   Like getfilecon/lgetfilecon, but when F's st_dev says it's on a known-
+   SELinux-challenged file system, fail with ENOTSUP immediately.  */
+static int
+getfilecon_cache (char const *file, struct fileinfo *f, bool deref)
+{
+  /* st_dev of the most recently processed device for which we've
+     found that [l]getfilecon fails indicating lack of support.  */
+  static dev_t unsupported_device;
+
+  if (f->stat.st_dev =3D=3D unsupported_device)
+    {
+      errno =3D ENOTSUP;
+      return -1;
+    }
+  int r =3D (deref
+           ? getfilecon (file, &f->scontext)
+           : lgetfilecon (file, &f->scontext));
+  if (r < 0 && errno_unsupported (errno))
+    unsupported_device =3D f->stat.st_dev;
+  return r;
+}
+
 /* Add a file to the current table of files.
    Verify that the file exists, and print an error message if it does not.
    Return the number of blocks that the file occupies.  */
-
 static uintmax_t
 gobble_file (char const *name, enum filetype type, ino_t inode,
              bool command_line_arg, char const *dirname)
@@ -2918,9 +2948,7 @@ gobble_file (char const *name, enum filetype type, in=
o_t inode,
         {
           bool have_selinux =3D false;
           bool have_acl =3D false;
-          int attr_len =3D (do_deref
-                          ?  getfilecon (absolute_name, &f->scontext)
-                          : lgetfilecon (absolute_name, &f->scontext));
+          int attr_len =3D getfilecon_cache (absolute_name, f, do_deref);
           err =3D (attr_len < 0);

           if (err =3D=3D 0)
--
1.7.9.1.245.gbbe07


>From 14abb248030e1a708170d25957159598e2892165 Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering@redhat.com>
Date: Mon, 13 Feb 2012 12:05:40 +0100
Subject: [PATCH 2/3] ls: cache ACL- and CAP-querying syscall failures

Like the optimization to avoid always-failing getfilecon calls,
this change avoids always-failing queries for whether a file has
a nontrivial ACL and for whether a file has certain "capabilities".
When such a query fails for one file (indicating no support), we know it
will always fail that way for the affected device.  With this change, we
have thus eliminated nearly all failing-unsupported getxattr syscalls.
* src/ls.c (has_capability) [!HAVE_CAP]: Set errno to ENOTSUP.
(errno_unsupported): Expand the list of E* errno values to match
that of lib/acl-internal.h's ACL_NOT_WELL_SUPPORTED macro.
(file_has_acl_cache, has_capability_cache): New functions.
(gobble_file): Use them in place of non-caching ones.
* NEWS (Improvements): Mention it.
Suggested by Sven Breuner in
http://thread.gmane.org/gmane.comp.gnu.coreutils.general/2187
While eliminating most getfilecon calls saved about 33%,
eliminating these other calls can save almost all of the
remaining ~67% cost, on some remote file systems.
---
 NEWS     |    6 ++++++
 src/ls.c |   61 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++-=
---
 2 files changed, 63 insertions(+), 4 deletions(-)

diff --git a/NEWS b/NEWS
index 2080790..133f8fe 100644
--- a/NEWS
+++ b/NEWS
@@ -22,6 +22,12 @@ GNU coreutils NEWS                                    -*=
- outline -*-
   surprising rename no-op behavior).  Now, mv handles this case by skipping
   the usually-useless rename and simply unlinking A.

+** Improvements
+
+  ls can be much more efficient, especially with large directories on file
+  systems for which getfilecon-, ACL-check- and XATTR-check-induced syscal=
ls
+  fail with ENOTSUP or similar.
+

 * Noteworthy changes in release 8.15 (2012-01-06) [stable]

diff --git a/src/ls.c b/src/ls.c
index 8a9cb41..92b17a4 100644
--- a/src/ls.c
+++ b/src/ls.c
@@ -2736,6 +2736,7 @@ has_capability (char const *name)
 static bool
 has_capability (char const *name ATTRIBUTE_UNUSED)
 {
+  errno =3D ENOTSUP;
   return false;
 }
 #endif
@@ -2778,11 +2779,16 @@ clear_files (void)
 }

 /* Return true if ERR implies lack-of-support failure by a
-   getxattr-calling function like getfilecon.  */
+   getxattr-calling function like getfilecon or file_has_acl.  */
 static bool
 errno_unsupported (int err)
 {
-  return err =3D=3D ENOTSUP || err =3D=3D EOPNOTSUPP;
+  return (err =3D=3D EBUSY
+          || err =3D=3D EINVAL
+          || err =3D=3D ENOENT
+          || err =3D=3D ENOSYS
+          || err =3D=3D ENOTSUP
+          || err =3D=3D EOPNOTSUPP);
 }

 /* Cache *getfilecon failure, when it's trivial to do so.
@@ -2808,6 +2814,53 @@ getfilecon_cache (char const *file, struct fileinfo =
*f, bool deref)
   return r;
 }

+/* Cache file_has_acl failure, when it's trivial to do.
+   Like file_has_acl, but when F's st_dev says it's on a file
+   system lacking ACL support, return 0 with ENOTSUP immediately.  */
+static int
+file_has_acl_cache (char const *file, struct fileinfo *f)
+{
+  /* st_dev of the most recently processed device for which we've
+     found that file_has_acl fails indicating lack of support.  */
+  static dev_t unsupported_device;
+
+  if (f->stat.st_dev =3D=3D unsupported_device)
+    {
+      errno =3D ENOTSUP;
+      return 0;
+    }
+
+  /* Zero errno so that we can distinguish between two 0-returning cases:
+     "has-ACL-support, but only a default ACL" and "no ACL support". */
+  errno =3D 0;
+  int n =3D file_has_acl (file, &f->stat);
+  if (n <=3D 0 && errno_unsupported (errno))
+    unsupported_device =3D f->stat.st_dev;
+  return n;
+}
+
+/* Cache has_capability failure, when it's trivial to do.
+   Like has_capability, but when F's st_dev says it's on a file
+   system lacking capability support, return 0 with ENOTSUP immediately.  =
*/
+static bool
+has_capability_cache (char const *file, struct fileinfo *f)
+{
+  /* st_dev of the most recently processed device for which we've
+     found that has_capability fails indicating lack of support.  */
+  static dev_t unsupported_device;
+
+  if (f->stat.st_dev =3D=3D unsupported_device)
+    {
+      errno =3D ENOTSUP;
+      return 0;
+    }
+
+  bool b =3D has_capability (file);
+  if ( !b && errno_unsupported (errno))
+    unsupported_device =3D f->stat.st_dev;
+  return b;
+}
+
 /* Add a file to the current table of files.
    Verify that the file exists, and print an error message if it does not.
    Return the number of blocks that the file occupies.  */
@@ -2942,7 +2995,7 @@ gobble_file (char const *name, enum filetype type, in=
o_t inode,
       /* Note has_capability() adds around 30% runtime to 'ls --color'  */
       if ((type =3D=3D normal || S_ISREG (f->stat.st_mode))
           && print_with_color && is_colored (C_CAP))
-        f->has_capability =3D has_capability (absolute_name);
+        f->has_capability =3D has_capability_cache (absolute_name, f);

       if (format =3D=3D long_format || print_scontext)
         {
@@ -2967,7 +3020,7 @@ gobble_file (char const *name, enum filetype type, in=
o_t inode,

           if (err =3D=3D 0 && format =3D=3D long_format)
             {
-              int n =3D file_has_acl (absolute_name, &f->stat);
+              int n =3D file_has_acl_cache (absolute_name, f);
               err =3D (n < 0);
               have_acl =3D (0 < n);
             }
--
1.7.9.1.245.gbbe07


>From e1951a002eae92b33ea3310a394a065858670468 Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering@redhat.com>
Date: Mon, 13 Feb 2012 12:52:19 +0100
Subject: [PATCH 3/3] tests: test for ls speed-up

* tests/ls/getxattr-speedup: New test.
* tests/Makefile.am (TESTS): Add it.
Improved-by: Bernhard Voelker.
---
 tests/Makefile.am         |    1 +
 tests/ls/getxattr-speedup |   58 +++++++++++++++++++++++++++++++++++++++++=
++++
 2 files changed, 59 insertions(+)
 create mode 100755 tests/ls/getxattr-speedup

diff --git a/tests/Makefile.am b/tests/Makefile.am
index 7b53681..625b636 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -426,6 +426,7 @@ TESTS =3D						\
   ls/dired					\
   ls/file-type					\
   ls/follow-slink				\
+  ls/getxattr-speedup				\
   ls/infloop					\
   ls/inode					\
   ls/m-option					\
diff --git a/tests/ls/getxattr-speedup b/tests/ls/getxattr-speedup
new file mode 100755
index 0000000..ecdd126
--- /dev/null
+++ b/tests/ls/getxattr-speedup
@@ -0,0 +1,58 @@
+#!/bin/sh
+# Show that we've eliminated most of ls' failing getxattr syscalls,
+# regardless of how many files are in a directory we list.
+# This test is skipped on systems that lack LD_PRELOAD support; that's fin=
e.
+
+# Copyright (C) 2012 Free Software Foundation, Inc.
+
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+. "${srcdir=3D.}/init.sh"; path_prepend_ ../src
+print_ver_ ls
+
+fd=3D33
+
+# Replace each getxattr and lgetxattr call with a call to these stubs.
+# Each writes a single line to the specified FD so that we can later
+# determine the total number of calls.  FIXME: there must be a better way.
+cat > k.c <<'EOF' || framework_failure_
+#include <errno.h>
+#include <unistd.h>
+static void
+write_1 (void)
+{
+  write (FD, "x\n", 2);
+}
+ssize_t getxattr (const char *path, const char *name, void *value, size_t =
size)
+{ write_1 (); errno =3D ENOTSUP; return -1; }
+ssize_t lgetxattr(const char *path, const char *name, void *value, size_t =
size)
+{ write_1 (); errno =3D ENOTSUP; return -1; }
+EOF
+
+# Then compile/link it:
+$CC -DFD=3D$fd -fPIC -O2 -c k.c \
+  || framework_failure_ 'failed to compile with -fPIC'
+ld -G k.o -o k.so || framework_failure_ 'failed to invoke ld -G ...'
+
+# create a few files
+seq 100 | xargs touch || framework_failure_
+
+# Finally, run the test, redirecting
+eval "LD_PRELOAD=3D$PWD/k.so ls --color=3Dalways -l . $fd>x" || fail=3D1
+
+# Ensure that there were no more than 3 *getxattr calls.
+n_calls=3D$(wc -l < x)
+test "$n_calls" -le 3 || fail=3D1
+
+Exit $fail
--
1.7.9.1.245.gbbe07


From MAILER-DAEMON Fri Feb 17 07:54:16 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RyNKK-0003qx-SD
	for mharc-coreutils@gnu.org; Fri, 17 Feb 2012 07:54:16 -0500
Received: from eggs.gnu.org ([140.186.70.92]:37503)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1RyNKD-0003oe-Bq
	for coreutils@gnu.org; Fri, 17 Feb 2012 07:54:15 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1RyNK7-0006Hx-7c
	for coreutils@gnu.org; Fri, 17 Feb 2012 07:54:09 -0500
Received: from moutng.kundenserver.de ([212.227.17.9]:54194)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1RyNK6-0006HP-UF
	for coreutils@gnu.org; Fri, 17 Feb 2012 07:54:03 -0500
Received: from [192.168.2.108] (p4FF76CC3.dip.t-dialin.net [79.247.108.195])
	by mrelayeu.kundenserver.de (node=mreu4) with ESMTP (Nemesis)
	id 0LzzMe-1SlvXa1OUm-00uqOJ; Fri, 17 Feb 2012 13:53:54 +0100
Message-ID: <4F3E4DE1.5080505@bernhard-voelker.de>
Date: Fri, 17 Feb 2012 13:53:53 +0100
From: Bernhard Voelker <mail@bernhard-voelker.de>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:10.0.1) Gecko/20120208 Thunderbird/10.0.1
MIME-Version: 1.0
To: Jim Meyering <jim@meyering.net>
Subject: Re: "ls -l": Avoid unnecessary getxattr() overhead
References: <87pqds3snt.fsf@Apollo.jerryland.fr>
	<4F30206B.1020802@itwm.fraunhofer.de>
	<CA+TATJHorgsuY7pxmpxxAeWUHRP5MfFfzQP05o=1H6C4ttswdA@mail.gmail.com>
	<4F3032BC.50702@draigBrady.com>
	<4F30631A.2070009@itwm.fraunhofer.de>
	<87y5scfdiq.fsf@rho.meyering.net> <4F33B5D7.3010909@draigBrady.com>
	<87mx8sf5w0.fsf@rho.meyering.net>
	<4F3437CE.4050308@itwm.fraunhofer.de>
	<4F346CC7.4030608@draigBrady.com> <87r4y3dry0.fsf@rho.meyering.net>
	<4F3710CA.8020402@itwm.fraunhofer.de>
	<8739aao9e2.fsf@rho.meyering.net>
	<4F3E12F9.6080102@bernhard-voelker.de>
	<8762f5lpct.fsf@rho.meyering.net>
	<4F3E35C9.7050707@bernhard-voelker.de>
	<87mx8hk6bm.fsf@rho.meyering.net>
In-Reply-To: <87mx8hk6bm.fsf@rho.meyering.net>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
X-Provags-ID: V02:K0:DJ8RVB1GR6/1d2xYHnOhFCRd3k7dmnlWMB5+CecG8WO
	LUcMMXDwxE+VbIrWAmFsaZHSXo8pkrl4aRFOROVBhd7lknBB2f
	eGDxGihyCKUs99QjHOwlq5s6Ubeqma+19JiOYnt6lRIadoogrw
	efoXFXudOEva86C6SaESO47ET7QfG2p1Gf0CKDPFlIeBdofyLI
	KU8bznc9mDMX2D1Ss1iYnN9WZOhv9saAfMV1Jxanh2viKytjBk
	aBFDv+6fL57UqlyCzRMuPqDGIrD1AjoffnQi/F5aBY0YzKTj0T
	STbTdWy8pqTIkx2g1XFSdJuy9DUHHnuVV6mL0GHGhPORIB1gaa
	t08cDx4mev/bDL4FUjy4esp4N9GK6uAOsXi7qfuC+
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 212.227.17.9
Cc: Sven Breuner <sven.breuner@itwm.fraunhofer.de>,
	Bernd Schubert <bernd.schubert@itwm.fraunhofer.de>,
	=?ISO-8859-1?Q?J=E9r=E9my_Compostella?= <jeremy.compostella@gmail.com>,
	"Daniel P. Berrange" <berrange@redhat.com>, coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 17 Feb 2012 12:54:15 -0000

On 02/17/2012 01:11 PM, Jim Meyering wrote:
> Bernhard Voelker wrote:
>> BTW: I have another question: what if someone has used
>> "./configure --disable-xattr"? ... surprisingly nothing:
>> ls doesn't honor USE_XATTR. Is that correct?
> 
> That option is intended to disable copy.c's attribute-copying code.
> I vaguely recall that initial libxattr implementations had problems.
> It would not be feasible (or sensible, imho) to eliminate all getxattr
> uses from ls.c.

Thanks for the explanation.

> Thanks again for all of the review comments.

Thanks for your patience. ;-)

> Here's the updates series, along with this addition to NEWS:
> 
> ** Improvements
> 
>   ls can be much more efficient, especially with large directories on file
>   systems for which getfilecon-, ACL-check- and XATTR-check-induced syscalls
>   fail with ENOTSUP or similar.

Do you think it's worth to distinguish between ENOTSUP and EOPNOTSUPP?

I.e. on system returning EOPNOTSUPP, when the files reside on N different
filesystems (or when processed in an unlucky order), then ls would still
try to get these attributes although we already know that the system does
not support it.

If so, then besides another flag, this would mean to s/ENOTSUP/EOPNOTSUPP/
here:

> diff --git a/src/ls.c b/src/ls.c
> index 8a9cb41..92b17a4 100644
> --- a/src/ls.c
> +++ b/src/ls.c
> @@ -2736,6 +2736,7 @@ has_capability (char const *name)
>  static bool
>  has_capability (char const *name ATTRIBUTE_UNUSED)
>  {
> +  errno = ENOTSUP;
>    return false;
>  }
>  #endif

Have a nice day,
Berny


From MAILER-DAEMON Fri Feb 17 08:23:55 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RyNn1-0003O9-2I
	for mharc-coreutils@gnu.org; Fri, 17 Feb 2012 08:23:55 -0500
Received: from eggs.gnu.org ([140.186.70.92]:57809)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1RyNmt-0003N7-AI
	for coreutils@gnu.org; Fri, 17 Feb 2012 08:23:53 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1RyNmq-0004br-E7
	for coreutils@gnu.org; Fri, 17 Feb 2012 08:23:47 -0500
Received: from mx.meyering.net ([88.168.87.75]:38923)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1RyNmq-0004bO-5w
	for coreutils@gnu.org; Fri, 17 Feb 2012 08:23:44 -0500
Received: from rho.meyering.net (localhost.localdomain [127.0.0.1])
	by rho.meyering.net (Acme Bit-Twister) with ESMTP id AA92860336;
	Fri, 17 Feb 2012 14:23:42 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: Bernhard Voelker <mail@bernhard-voelker.de>
Subject: Re: "ls -l": Avoid unnecessary getxattr() overhead
In-Reply-To: <4F3E4DE1.5080505@bernhard-voelker.de> (Bernhard Voelker's
	message of "Fri, 17 Feb 2012 13:53:53 +0100")
References: <87pqds3snt.fsf@Apollo.jerryland.fr>
	<4F30206B.1020802@itwm.fraunhofer.de>
	<CA+TATJHorgsuY7pxmpxxAeWUHRP5MfFfzQP05o=1H6C4ttswdA@mail.gmail.com>
	<4F3032BC.50702@draigBrady.com> <4F30631A.2070009@itwm.fraunhofer.de>
	<87y5scfdiq.fsf@rho.meyering.net> <4F33B5D7.3010909@draigBrady.com>
	<87mx8sf5w0.fsf@rho.meyering.net>
	<4F3437CE.4050308@itwm.fraunhofer.de>
	<4F346CC7.4030608@draigBrady.com> <87r4y3dry0.fsf@rho.meyering.net>
	<4F3710CA.8020402@itwm.fraunhofer.de>
	<8739aao9e2.fsf@rho.meyering.net>
	<4F3E12F9.6080102@bernhard-voelker.de>
	<8762f5lpct.fsf@rho.meyering.net>
	<4F3E35C9.7050707@bernhard-voelker.de>
	<87mx8hk6bm.fsf@rho.meyering.net>
	<4F3E4DE1.5080505@bernhard-voelker.de>
Date: Fri, 17 Feb 2012 14:23:42 +0100
Message-ID: <87booxk2zl.fsf@rho.meyering.net>
Lines: 62
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 88.168.87.75
Cc: Sven Breuner <sven.breuner@itwm.fraunhofer.de>,
	Bernd Schubert <bernd.schubert@itwm.fraunhofer.de>,
	=?iso-8859-1?Q?J=E9r=E9my?= Compostella <jeremy.compostella@gmail.com>,
	"Daniel P. Berrange" <berrange@redhat.com>, coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 17 Feb 2012 13:23:53 -0000

Bernhard Voelker wrote:

> On 02/17/2012 01:11 PM, Jim Meyering wrote:
>> Bernhard Voelker wrote:
>>> BTW: I have another question: what if someone has used
>>> "./configure --disable-xattr"? ... surprisingly nothing:
>>> ls doesn't honor USE_XATTR. Is that correct?
>>
>> That option is intended to disable copy.c's attribute-copying code.
>> I vaguely recall that initial libxattr implementations had problems.
>> It would not be feasible (or sensible, imho) to eliminate all getxattr
>> uses from ls.c.
>
> Thanks for the explanation.
>
>> Thanks again for all of the review comments.
>
> Thanks for your patience. ;-)
>
>> Here's the updates series, along with this addition to NEWS:
>>
>> ** Improvements
>>
>>   ls can be much more efficient, especially with large directories on file
>>   systems for which getfilecon-, ACL-check- and XATTR-check-induced syscalls
>>   fail with ENOTSUP or similar.
>
> Do you think it's worth to distinguish between ENOTSUP and EOPNOTSUPP?

No, since AFAIK, they can be used interchangeably in this particular context.
i.e., getfilecon can fail with either, and in each case the result
is the same for us: we know that all further getfilecon calls
on the same device will fail with that same result.

> I.e. on system returning EOPNOTSUPP, when the files reside on N different
> filesystems (or when processed in an unlucky order), then ls would still

Perhaps I have not understood your example?
If the files are on different file systems, then we cannot and do not
use cached failure/errno-values.

> try to get these attributes although we already know that the system does
> not support it.
>
> If so, then besides another flag, this would mean to s/ENOTSUP/EOPNOTSUPP/
> here:
>
>> diff --git a/src/ls.c b/src/ls.c
>> index 8a9cb41..92b17a4 100644
>> --- a/src/ls.c
>> +++ b/src/ls.c
>> @@ -2736,6 +2736,7 @@ has_capability (char const *name)
>>  static bool
>>  has_capability (char const *name ATTRIBUTE_UNUSED)
>>  {
>> +  errno = ENOTSUP;
>>    return false;
>>  }
>>  #endif
>
> Have a nice day,
> Berny


From MAILER-DAEMON Fri Feb 17 08:39:34 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RyO2A-0000Gg-F4
	for mharc-coreutils@gnu.org; Fri, 17 Feb 2012 08:39:34 -0500
Received: from eggs.gnu.org ([140.186.70.92]:52508)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1RyO23-0000GU-4t
	for coreutils@gnu.org; Fri, 17 Feb 2012 08:39:32 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1RyO1x-00088q-HT
	for coreutils@gnu.org; Fri, 17 Feb 2012 08:39:27 -0500
Received: from moutng.kundenserver.de ([212.227.17.9]:63655)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1RyO1x-00088j-56
	for coreutils@gnu.org; Fri, 17 Feb 2012 08:39:21 -0500
Received: from [192.168.2.108] (p4FF76CC3.dip.t-dialin.net [79.247.108.195])
	by mrelayeu.kundenserver.de (node=mreu1) with ESMTP (Nemesis)
	id 0M1CM0-1Sn2k604mO-00tGTo; Fri, 17 Feb 2012 14:39:14 +0100
Message-ID: <4F3E5881.1030203@bernhard-voelker.de>
Date: Fri, 17 Feb 2012 14:39:13 +0100
From: Bernhard Voelker <mail@bernhard-voelker.de>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:10.0.1) Gecko/20120208 Thunderbird/10.0.1
MIME-Version: 1.0
To: Jim Meyering <jim@meyering.net>
Subject: Re: "ls -l": Avoid unnecessary getxattr() overhead
References: <87pqds3snt.fsf@Apollo.jerryland.fr>
	<4F30206B.1020802@itwm.fraunhofer.de>
	<CA+TATJHorgsuY7pxmpxxAeWUHRP5MfFfzQP05o=1H6C4ttswdA@mail.gmail.com>
	<4F3032BC.50702@draigBrady.com>
	<4F30631A.2070009@itwm.fraunhofer.de>
	<87y5scfdiq.fsf@rho.meyering.net> <4F33B5D7.3010909@draigBrady.com>
	<87mx8sf5w0.fsf@rho.meyering.net>
	<4F3437CE.4050308@itwm.fraunhofer.de>
	<4F346CC7.4030608@draigBrady.com> <87r4y3dry0.fsf@rho.meyering.net>
	<4F3710CA.8020402@itwm.fraunhofer.de>
	<8739aao9e2.fsf@rho.meyering.net>
	<4F3E12F9.6080102@bernhard-voelker.de>
	<8762f5lpct.fsf@rho.meyering.net>
	<4F3E35C9.7050707@bernhard-voelker.de>
	<87mx8hk6bm.fsf@rho.meyering.net>
	<4F3E4DE1.5080505@bernhard-voelker.de>
	<87booxk2zl.fsf@rho.meyering.net>
In-Reply-To: <87booxk2zl.fsf@rho.meyering.net>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
X-Provags-ID: V02:K0:4BaJouRQTN6VOrvKA8KcoBHUXtFB9M17fSTc9oLYrjt
	CLwJspzHGmYPGyMZgMXt2887DIPbubmR/jDrxOSfG93oyBNQWT
	1BP8qgkJW0fu5So4zQr/KT4IRwAuMFwm70x6EF/dmWyuEaVH2v
	9Ff1cuIMl5Z4JZKI5rju5+JUypuD7Z+8aQB53PCAe7ClYwRj5C
	hzsreQE/sSb+hUEUc89HKWFQ+1Lr8LERb8xLyy7ArjjasOV5th
	fFKm9Y2UIKl97gnobOryTQdKgZ8XLbw1ucYR+lynv2VsfzCg94
	3MVO1TAm20MFEPnWWYhKgkkon2DoaG65IYwrOFl2p6EpZvQs71
	aH5FaUJ/JrHzfWzgUiZG+Ms6bQqIrqn8kRL1MzVsN
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 212.227.17.9
Cc: Sven Breuner <sven.breuner@itwm.fraunhofer.de>,
	Bernd Schubert <bernd.schubert@itwm.fraunhofer.de>,
	=?ISO-8859-1?Q?J=E9r=E9my_Compostella?= <jeremy.compostella@gmail.com>,
	"Daniel P. Berrange" <berrange@redhat.com>, coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 17 Feb 2012 13:39:32 -0000


On 02/17/2012 02:23 PM, Jim Meyering wrote:
>> Do you think it's worth to distinguish between ENOTSUP and EOPNOTSUPP?
> No, since AFAIK, they can be used interchangeably in this particular context.
> i.e., getfilecon can fail with either, and in each case the result
> is the same for us: we know that all further getfilecon calls
> on the same device will fail with that same result.
> 
>> > I.e. on system returning EOPNOTSUPP, when the files reside on N different
>> > filesystems (or when processed in an unlucky order), then ls would still
> Perhaps I have not understood your example?
> If the files are on different file systems, then we cannot and do not
> use cached failure/errno-values.

Hmm, I was mislead that EOPNOTSUPP would mean "this system does not support
the operation" - at all. I think I was wrong. Scratch it.

Have a nice day,
Berny


From MAILER-DAEMON Fri Feb 17 08:53:17 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RyOFR-0005ez-Dy
	for mharc-coreutils@gnu.org; Fri, 17 Feb 2012 08:53:17 -0500
Received: from eggs.gnu.org ([140.186.70.92]:34767)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1RyOFO-0005TX-40
	for coreutils@gnu.org; Fri, 17 Feb 2012 08:53:15 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1RyOFI-0002CK-0d
	for coreutils@gnu.org; Fri, 17 Feb 2012 08:53:14 -0500
Received: from mx1.redhat.com ([209.132.183.28]:9156)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1RyOFH-0002CA-Ob
	for coreutils@gnu.org; Fri, 17 Feb 2012 08:53:07 -0500
Received: from int-mx10.intmail.prod.int.phx2.redhat.com
	(int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id q1HDr24Y006507
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK);
	Fri, 17 Feb 2012 08:53:03 -0500
Received: from [10.3.113.18] ([10.3.113.18])
	by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP
	id q1HDr2cu023283; Fri, 17 Feb 2012 08:53:02 -0500
Message-ID: <4F3E5BBD.80809@redhat.com>
Date: Fri, 17 Feb 2012 06:53:01 -0700
From: Eric Blake <eblake@redhat.com>
Organization: Red Hat
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:10.0) Gecko/20120131 Thunderbird/10.0
MIME-Version: 1.0
To: Jim Meyering <jim@meyering.net>
Subject: Re: "ls -l": Avoid unnecessary getxattr() overhead
References: <87pqds3snt.fsf@Apollo.jerryland.fr>
	<4F30206B.1020802@itwm.fraunhofer.de>
	<CA+TATJHorgsuY7pxmpxxAeWUHRP5MfFfzQP05o=1H6C4ttswdA@mail.gmail.com>
	<4F3032BC.50702@draigBrady.com>
	<4F30631A.2070009@itwm.fraunhofer.de>
	<87y5scfdiq.fsf@rho.meyering.net> <4F33B5D7.3010909@draigBrady.com>
	<87mx8sf5w0.fsf@rho.meyering.net>
	<4F3437CE.4050308@itwm.fraunhofer.de>
	<4F346CC7.4030608@draigBrady.com> <87r4y3dry0.fsf@rho.meyering.net>
	<4F3710CA.8020402@itwm.fraunhofer.de>
	<8739aao9e2.fsf@rho.meyering.net>
	<4F3E12F9.6080102@bernhard-voelker.de>
	<8762f5lpct.fsf@rho.meyering.net>
In-Reply-To: <8762f5lpct.fsf@rho.meyering.net>
X-Enigmail-Version: 1.3.5
OpenPGP: url=http://people.redhat.com/eblake/eblake.gpg
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature";
	boundary="------------enig02B2F426699EE06A1952AE17"
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.132.183.28
Cc: "Daniel P. Berrange" <berrange@redhat.com>,
	Bernhard Voelker <mail@bernhard-voelker.de>,
	Bernd Schubert <bernd.schubert@itwm.fraunhofer.de>, coreutils@gnu.org,
	Sven Breuner <sven.breuner@itwm.fraunhofer.de>,
	=?UTF-8?B?SsOpcsOpbXk=?= =?UTF-8?B?IENvbXBvc3RlbGxh?=
	<jeremy.compostella@gmail.com>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 17 Feb 2012 13:53:15 -0000

This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--------------enig02B2F426699EE06A1952AE17
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

On 02/17/2012 03:35 AM, Jim Meyering wrote:

> Instead, I've realized that I can eliminate the need for the wc post-
> processing by using the fact that our contrived output file is seekable=
:
> rather than simply writing to fd, first seek to position 0 each time,
> and write the count:

>=20
>   # Finally, to test it, do this:
>   eval "LD_PRELOAD=3D$PWD/k.so ls --color=3Dalways -l . $fd>x" || exit =
1
>=20
> Hmm... that means the case of no getxattr calls would still
> require a little special handling to map "empty file" to "0".

No.  Just add 'echo 0 >x' prior to the actual test, so that x is never
empty.

--=20
Eric Blake   eblake@redhat.com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


--------------enig02B2F426699EE06A1952AE17
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)
Comment: Public key at http://people.redhat.com/eblake/eblake.gpg
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQEcBAEBCAAGBQJPPlu9AAoJEKeha0olJ0NqfVAH+gLxWH2u1lKKhU2aADPJQPLV
jn8Ss6Xw1V98e3vuiLKIYb8ArNWeyBhhXHs6+s9aRT+RnndInqu9J18hYD8qXmNr
CQtCBLWGjr+ZrblirKM0ILugxbDiVwuIJo9uHifMc3chBb0FLJ1767jpUMHxYl7M
8fYo/LurE2WuB2vKiS2xZK4SAgT46zQmFWfCuwrZRqU9jdbFJyoD2X0pR1WgjV62
S/FJm5tRHw0Q4N0DXS1bFdFbHKFEIh4Slggu/MyLGAozPPe+Zo3nc4XcKKdq0Kxx
PnmSiXUEBIOXk2OokLmd32oE8Nw/+6QULiAalOr28nisJUeQfvzhpIrlMx2lnwk=
=JY4l
-----END PGP SIGNATURE-----

--------------enig02B2F426699EE06A1952AE17--


From MAILER-DAEMON Fri Feb 17 08:59:03 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RyOL1-00078D-04
	for mharc-coreutils@gnu.org; Fri, 17 Feb 2012 08:59:03 -0500
Received: from eggs.gnu.org ([140.186.70.92]:60406)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1RyOKx-00076t-R6
	for coreutils@gnu.org; Fri, 17 Feb 2012 08:59:00 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1RyOKt-0003Zb-L4
	for coreutils@gnu.org; Fri, 17 Feb 2012 08:58:59 -0500
Received: from mx.meyering.net ([88.168.87.75]:39036)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1RyOKt-0003Y5-Ds
	for coreutils@gnu.org; Fri, 17 Feb 2012 08:58:55 -0500
Received: from rho.meyering.net (localhost.localdomain [127.0.0.1])
	by rho.meyering.net (Acme Bit-Twister) with ESMTP id 440D260336;
	Fri, 17 Feb 2012 14:58:54 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: Eric Blake <eblake@redhat.com>
Subject: Re: "ls -l": Avoid unnecessary getxattr() overhead
In-Reply-To: <4F3E5BBD.80809@redhat.com> (Eric Blake's message of "Fri, 17 Feb
	2012 06:53:01 -0700")
References: <87pqds3snt.fsf@Apollo.jerryland.fr>
	<4F30206B.1020802@itwm.fraunhofer.de>
	<CA+TATJHorgsuY7pxmpxxAeWUHRP5MfFfzQP05o=1H6C4ttswdA@mail.gmail.com>
	<4F3032BC.50702@draigBrady.com> <4F30631A.2070009@itwm.fraunhofer.de>
	<87y5scfdiq.fsf@rho.meyering.net> <4F33B5D7.3010909@draigBrady.com>
	<87mx8sf5w0.fsf@rho.meyering.net>
	<4F3437CE.4050308@itwm.fraunhofer.de>
	<4F346CC7.4030608@draigBrady.com> <87r4y3dry0.fsf@rho.meyering.net>
	<4F3710CA.8020402@itwm.fraunhofer.de>
	<8739aao9e2.fsf@rho.meyering.net>
	<4F3E12F9.6080102@bernhard-voelker.de>
	<8762f5lpct.fsf@rho.meyering.net> <4F3E5BBD.80809@redhat.com>
Date: Fri, 17 Feb 2012 14:58:54 +0100
Message-ID: <874nupk1cx.fsf@rho.meyering.net>
Lines: 39
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 88.168.87.75
Cc: "Daniel P. Berrange" <berrange@redhat.com>,
	Bernhard Voelker <mail@bernhard-voelker.de>,
	Bernd Schubert <bernd.schubert@itwm.fraunhofer.de>, coreutils@gnu.org,
	Sven Breuner <sven.breuner@itwm.fraunhofer.de>,
	=?iso-8859-1?Q?J=E9r=E9my?= Compostella <jeremy.compostella@gmail.com>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 17 Feb 2012 13:59:00 -0000

Eric Blake wrote:

> On 02/17/2012 03:35 AM, Jim Meyering wrote:
>
>> Instead, I've realized that I can eliminate the need for the wc post-
>> processing by using the fact that our contrived output file is seekable:
>> rather than simply writing to fd, first seek to position 0 each time,
>> and write the count:
>
>>
>>   # Finally, to test it, do this:
>>   eval "LD_PRELOAD=$PWD/k.so ls --color=always -l . $fd>x" || exit 1
>>
>> Hmm... that means the case of no getxattr calls would still
>> require a little special handling to map "empty file" to "0".
>
> No.  Just add 'echo 0 >x' prior to the actual test, so that x is never
> empty.

But the eval'd code will then truncate "x".
On some systems there will be no intercepted *getxattr call,
and hence x will still be empty.

Ahh... or maybe you meant to suggest that the eval'd code append to x?
That would work.

On a related note, I've just run "make distcheck", which exposed a bug
in this code:

  eval "LD_PRELOAD=$PWD/k.so ls --color=always -l . $fd>x" || exit 1

That fails when $PWD contains a space.  coreutils' make distcheck test
does actually build with such a working directory, to test for precisely
this type of bug.

The fix is trivial:

  -eval "LD_PRELOAD=$PWD/k.so ls --color=always -l . $fd>x" || exit 1
  +eval "LD_PRELOAD=./k.so ls --color=always -l . $fd>x" || exit 1


From MAILER-DAEMON Fri Feb 17 09:05:21 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RyOR7-0008PE-4o
	for mharc-coreutils@gnu.org; Fri, 17 Feb 2012 09:05:21 -0500
Received: from eggs.gnu.org ([140.186.70.92]:41496)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1RyOQy-0008MM-4x
	for coreutils@gnu.org; Fri, 17 Feb 2012 09:05:19 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1RyOQw-0004uR-Aw
	for coreutils@gnu.org; Fri, 17 Feb 2012 09:05:11 -0500
Received: from mx1.redhat.com ([209.132.183.28]:7182)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1RyOQv-0004tW-Mh
	for coreutils@gnu.org; Fri, 17 Feb 2012 09:05:10 -0500
Received: from int-mx12.intmail.prod.int.phx2.redhat.com
	(int-mx12.intmail.prod.int.phx2.redhat.com [10.5.11.25])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id q1HE53eg000655
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK);
	Fri, 17 Feb 2012 09:05:04 -0500
Received: from [10.3.113.18] ([10.3.113.18])
	by int-mx12.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP
	id q1HE5249006245; Fri, 17 Feb 2012 09:05:03 -0500
Message-ID: <4F3E5E8E.2040802@redhat.com>
Date: Fri, 17 Feb 2012 07:05:02 -0700
From: Eric Blake <eblake@redhat.com>
Organization: Red Hat
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:10.0) Gecko/20120131 Thunderbird/10.0
MIME-Version: 1.0
To: Jim Meyering <jim@meyering.net>
Subject: Re: "ls -l": Avoid unnecessary getxattr() overhead
References: <87pqds3snt.fsf@Apollo.jerryland.fr>
	<4F30206B.1020802@itwm.fraunhofer.de>
	<CA+TATJHorgsuY7pxmpxxAeWUHRP5MfFfzQP05o=1H6C4ttswdA@mail.gmail.com>
	<4F3032BC.50702@draigBrady.com>
	<4F30631A.2070009@itwm.fraunhofer.de>
	<87y5scfdiq.fsf@rho.meyering.net> <4F33B5D7.3010909@draigBrady.com>
	<87mx8sf5w0.fsf@rho.meyering.net>
	<4F3437CE.4050308@itwm.fraunhofer.de>
	<4F346CC7.4030608@draigBrady.com> <87r4y3dry0.fsf@rho.meyering.net>
	<4F3710CA.8020402@itwm.fraunhofer.de>
	<8739aao9e2.fsf@rho.meyering.net>
	<4F3E12F9.6080102@bernhard-voelker.de>
	<8762f5lpct.fsf@rho.meyering.net> <4F3E5BBD.80809@redhat.com>
	<874nupk1cx.fsf@rho.meyering.net>
In-Reply-To: <874nupk1cx.fsf@rho.meyering.net>
X-Enigmail-Version: 1.3.5
OpenPGP: url=http://people.redhat.com/eblake/eblake.gpg
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature";
	boundary="------------enig2804DE393BD30559CD4554A4"
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.25
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.132.183.28
Cc: "Daniel P. Berrange" <berrange@redhat.com>,
	Bernhard Voelker <mail@bernhard-voelker.de>,
	Bernd Schubert <bernd.schubert@itwm.fraunhofer.de>, coreutils@gnu.org,
	Sven Breuner <sven.breuner@itwm.fraunhofer.de>,
	=?UTF-8?B?SsOpcsOpbXk=?= =?UTF-8?B?IENvbXBvc3RlbGxh?=
	<jeremy.compostella@gmail.com>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 17 Feb 2012 14:05:19 -0000

This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--------------enig2804DE393BD30559CD4554A4
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

On 02/17/2012 06:58 AM, Jim Meyering wrote:

>>>   eval "LD_PRELOAD=3D$PWD/k.so ls --color=3Dalways -l . $fd>x" || exi=
t 1
>>>
>>> Hmm... that means the case of no getxattr calls would still
>>> require a little special handling to map "empty file" to "0".
>>
>> No.  Just add 'echo 0 >x' prior to the actual test, so that x is never=

>> empty.
>=20
> But the eval'd code will then truncate "x".

Oh right.  But not the end of the world - in the eval, use $fd<>x to
avoid the truncation.

> On some systems there will be no intercepted *getxattr call,
> and hence x will still be empty.
>=20
> Ahh... or maybe you meant to suggest that the eval'd code append to x?

Alas, $fd>>x opens x with O_APPEND, so seeking to position 0 then
calling write() will still append, which is not what we want.

And hopefully no one is running this on the buggy version of dash, since
it was only about 18 months ago where dash had a bug fix to quit using
O_TRUNCATE for $fd<>x (bash and ksh have always followed that part of
POSIX).

Hmm, if we can't rely on $fd<>x, maybe we _can_ use $fd>>x coupled with
fcntl() on fd to reset the O_APPEND before doing any write()s, but that
starts to feel complicated.  I'd rather go with $fd<>x and tell users to
upgrade to a fixed version of dash.

--=20
Eric Blake   eblake@redhat.com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


--------------enig2804DE393BD30559CD4554A4
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)
Comment: Public key at http://people.redhat.com/eblake/eblake.gpg
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQEcBAEBCAAGBQJPPl6OAAoJEKeha0olJ0NqsAcH/1uxqU61uQlUB/C9wGU7nkyT
/9WtfM4yzT8JSUi7M13qcG61KpNgxpWOgx4RyYbQejUl6F+SNUMp4XLvvjopPfN1
N2cBMeOj1Y4xVKelawmRC95p+jixv1hXDxUU2dWnM/KXMOfseXGNjhCCgXOHcpWk
YZNwsxtJm9pbOhWC5hUa/rwfNHQqdomriQUcPai75Xvl1LKpnKY3MbRR2APCAz5O
kAlZoLFyQ48NsEk6dpezG7V0sCSLcR65p2e9aEOOldJOhLe4DLcMTnzlYOJTFmMs
0m0Fk2zXaky6kbXp+1WTAruLkTSe0lZ0SD/iTeLP2sK2ep3tYPy8GLEQZOBjNjI=
=vyRN
-----END PGP SIGNATURE-----

--------------enig2804DE393BD30559CD4554A4--


From MAILER-DAEMON Fri Feb 17 10:16:03 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RyPXX-0004vS-24
	for mharc-coreutils@gnu.org; Fri, 17 Feb 2012 10:16:03 -0500
Received: from eggs.gnu.org ([140.186.70.92]:41660)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1RyPXP-0004vG-2n
	for coreutils@gnu.org; Fri, 17 Feb 2012 10:16:00 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1RyPXK-0001bh-N8
	for coreutils@gnu.org; Fri, 17 Feb 2012 10:15:55 -0500
Received: from mx.meyering.net ([88.168.87.75]:39258)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1RyPXK-0001bc-A0
	for coreutils@gnu.org; Fri, 17 Feb 2012 10:15:50 -0500
Received: from rho.meyering.net (localhost.localdomain [127.0.0.1])
	by rho.meyering.net (Acme Bit-Twister) with ESMTP id E242360064;
	Fri, 17 Feb 2012 16:15:48 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: Eric Blake <eblake@redhat.com>
Subject: Re: "ls -l": Avoid unnecessary getxattr() overhead
In-Reply-To: <4F3E5E8E.2040802@redhat.com> (Eric Blake's message of "Fri, 17
	Feb 2012 07:05:02 -0700")
References: <87pqds3snt.fsf@Apollo.jerryland.fr>
	<4F30206B.1020802@itwm.fraunhofer.de>
	<CA+TATJHorgsuY7pxmpxxAeWUHRP5MfFfzQP05o=1H6C4ttswdA@mail.gmail.com>
	<4F3032BC.50702@draigBrady.com> <4F30631A.2070009@itwm.fraunhofer.de>
	<87y5scfdiq.fsf@rho.meyering.net> <4F33B5D7.3010909@draigBrady.com>
	<87mx8sf5w0.fsf@rho.meyering.net>
	<4F3437CE.4050308@itwm.fraunhofer.de>
	<4F346CC7.4030608@draigBrady.com> <87r4y3dry0.fsf@rho.meyering.net>
	<4F3710CA.8020402@itwm.fraunhofer.de>
	<8739aao9e2.fsf@rho.meyering.net>
	<4F3E12F9.6080102@bernhard-voelker.de>
	<8762f5lpct.fsf@rho.meyering.net> <4F3E5BBD.80809@redhat.com>
	<874nupk1cx.fsf@rho.meyering.net> <4F3E5E8E.2040802@redhat.com>
Date: Fri, 17 Feb 2012 16:15:48 +0100
Message-ID: <87lio1ij8b.fsf@rho.meyering.net>
Lines: 106
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 88.168.87.75
Cc: "Daniel P. Berrange" <berrange@redhat.com>,
	Bernhard Voelker <mail@bernhard-voelker.de>,
	Bernd Schubert <bernd.schubert@itwm.fraunhofer.de>, coreutils@gnu.org,
	Sven Breuner <sven.breuner@itwm.fraunhofer.de>,
	=?iso-8859-1?Q?J=E9r=E9my?= Compostella <jeremy.compostella@gmail.com>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 17 Feb 2012 15:16:00 -0000

Eric Blake wrote:
> On 02/17/2012 06:58 AM, Jim Meyering wrote:
>>>>   eval "LD_PRELOAD=$PWD/k.so ls --color=always -l . $fd>x" || exit 1
>>>>
>>>> Hmm... that means the case of no getxattr calls would still
>>>> require a little special handling to map "empty file" to "0".
>>>
>>> No.  Just add 'echo 0 >x' prior to the actual test, so that x is never
>>> empty.
>>
>> But the eval'd code will then truncate "x".
>
> Oh right.  But not the end of the world - in the eval, use $fd<>x to
> avoid the truncation.
>
>> On some systems there will be no intercepted *getxattr call,
>> and hence x will still be empty.
>>
>> Ahh... or maybe you meant to suggest that the eval'd code append to x?
>
> Alas, $fd>>x opens x with O_APPEND, so seeking to position 0 then
> calling write() will still append, which is not what we want.
>
> And hopefully no one is running this on the buggy version of dash, since
> it was only about 18 months ago where dash had a bug fix to quit using
> O_TRUNCATE for $fd<>x (bash and ksh have always followed that part of
> POSIX).
>
> Hmm, if we can't rely on $fd<>x, maybe we _can_ use $fd>>x coupled with
> fcntl() on fd to reset the O_APPEND before doing any write()s, but that
> starts to feel complicated.  I'd rather go with $fd<>x and tell users to
> upgrade to a fixed version of dash.

Thanks for the suggestions, but they go beyond my keep-it-simple
threshold :-)  It might seem odd to worry about simplicity at the
shell-redirect level in a test that uses LD_PRELOAD and gcc's destructor
attribute, but hey, I'll take it where I can.

I added a slightly different destructor than what I'd tried before,
and now it works:

Here's the stand-alone test:
----------------
fd=33
cat <<'EOF' > k.c
#include <errno.h>
#include <stdio.h>
static unsigned long int n_calls;
ssize_t getxattr (const char *path, const char *name, void *value, size_t size)
{ ++n_calls; errno = ENOTSUP; return -1; }
ssize_t lgetxattr(const char *path, const char *name, void *value, size_t size)
{ ++n_calls; errno = ENOTSUP; return -1; }
__attribute__((destructor)) void p()
{
  FILE *fp = fdopen (FD, "a"); if (!fp) return;
  fprintf (fp, "%lu\n", n_calls);
  fclose (fp);
}
EOF
gcc -DFD=$fd -Wall -fPIC -O2 -c k.c && ld -G k.o -o k.so &&
eval "LD_PRELOAD=./k.so ls -l --color=always . $fd>x" &&
cat x
----------------

And here's the patch I've just squashed into the test c-set.

diff --git a/tests/ls/getxattr-speedup b/tests/ls/getxattr-speedup
index eaa3342..a3d47c1 100755
--- a/tests/ls/getxattr-speedup
+++ b/tests/ls/getxattr-speedup
@@ -28,16 +28,18 @@ fd=33
 # determine the total number of calls.  FIXME: there must be a better way.
 cat > k.c <<'EOF' || framework_failure_
 #include <errno.h>
-#include <unistd.h>
-static void
-write_1 (void)
-{
-  write (FD, "x\n", 2);
-}
+#include <stdio.h>
+static unsigned long int n_calls;
 ssize_t getxattr (const char *path, const char *name, void *value, size_t size)
-{ write_1 (); errno = ENOTSUP; return -1; }
+{ ++n_calls; errno = ENOTSUP; return -1; }
 ssize_t lgetxattr(const char *path, const char *name, void *value, size_t size)
-{ write_1 (); errno = ENOTSUP; return -1; }
+{ ++n_calls; errno = ENOTSUP; return -1; }
+__attribute__((destructor)) void p()
+{
+  FILE *fp = fdopen (FD, "a"); if (!fp) return;
+  fprintf (fp, "%lu\n", n_calls);
+  fclose (fp);
+}
 EOF

 # Then compile/link it:
@@ -52,7 +54,7 @@ seq 100 | xargs touch || framework_failure_
 eval "LD_PRELOAD=./k.so ls --color=always -l . $fd>x" || fail=1

 # Ensure that there were no more than 3 *getxattr calls.
-n_calls=$(wc -l < x)
+n_calls=$(cat x)
 test "$n_calls" -le 3 || fail=1

 Exit $fail


From MAILER-DAEMON Fri Feb 17 11:07:14 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RyQL4-0004ID-Ed
	for mharc-coreutils@gnu.org; Fri, 17 Feb 2012 11:07:14 -0500
Received: from eggs.gnu.org ([140.186.70.92]:50112)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1RyQKx-0004FI-Dq
	for coreutils@gnu.org; Fri, 17 Feb 2012 11:07:11 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1RyQKr-0003U3-Dd
	for coreutils@gnu.org; Fri, 17 Feb 2012 11:07:07 -0500
Received: from moutng.kundenserver.de ([212.227.126.171]:64672)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1RyQKr-0003Tx-48
	for coreutils@gnu.org; Fri, 17 Feb 2012 11:07:01 -0500
Received: from [192.168.2.108] (p4FF76CC3.dip.t-dialin.net [79.247.108.195])
	by mrelayeu.kundenserver.de (node=mrbap4) with ESMTP (Nemesis)
	id 0Lmujs-1SQ2RX3gbM-00gwH3; Fri, 17 Feb 2012 17:06:56 +0100
Message-ID: <4F3E7B1E.3090300@bernhard-voelker.de>
Date: Fri, 17 Feb 2012 17:06:54 +0100
From: Bernhard Voelker <mail@bernhard-voelker.de>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:10.0.1) Gecko/20120208 Thunderbird/10.0.1
MIME-Version: 1.0
To: Jim Meyering <jim@meyering.net>
Subject: Re: "ls -l": Avoid unnecessary getxattr() overhead
References: <87pqds3snt.fsf@Apollo.jerryland.fr>
	<4F30206B.1020802@itwm.fraunhofer.de>
	<CA+TATJHorgsuY7pxmpxxAeWUHRP5MfFfzQP05o=1H6C4ttswdA@mail.gmail.com>
	<4F3032BC.50702@draigBrady.com>
	<4F30631A.2070009@itwm.fraunhofer.de>
	<87y5scfdiq.fsf@rho.meyering.net> <4F33B5D7.3010909@draigBrady.com>
	<87mx8sf5w0.fsf@rho.meyering.net>
	<4F3437CE.4050308@itwm.fraunhofer.de>
	<4F346CC7.4030608@draigBrady.com> <87r4y3dry0.fsf@rho.meyering.net>
	<4F3710CA.8020402@itwm.fraunhofer.de>
	<8739aao9e2.fsf@rho.meyering.net>
	<4F3E12F9.6080102@bernhard-voelker.de>
	<8762f5lpct.fsf@rho.meyering.net> <4F3E5BBD.80809@redhat.com>
	<874nupk1cx.fsf@rho.meyering.net> <4F3E5E8E.2040802@redhat.com>
	<87lio1ij8b.fsf@rho.meyering.net>
In-Reply-To: <87lio1ij8b.fsf@rho.meyering.net>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
X-Provags-ID: V02:K0:amiidDZ7WJ91L5rcGz6rbajqN34JhWu8fUf4+t6n38H
	CvyaUS0ZlGL4JpuEKCEdM4CE/SQiAKiduF4pvKpsK9xizjDoGB
	AcRaE7M1tKAE8yRTLPSG6iwXjj/xbt8WSVjr7TDr5ElrIhurBe
	c38krrnYA9/lNhTsMcOzjd7OOk9uZzsICO+cdgCD2zmqzf92XS
	dFZ7hgnsTmmYgTk7H+OPfYj+NqFXeKJeNR4gi5zofeWQfTYZUE
	iaiL7i7TwTKC4H3sDVCGeenN1FP6DlEiiORZOpncZWs2yqZD0t
	L5eoF6nRffCMU4saZuulTLQrAbaedCygwEORYrCzY1TYoYgQur
	6hCk/jUAG0x7dZxlhPzD79/YUdfmOMTWzV3CRxPIs
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 212.227.126.171
Cc: "Daniel P. Berrange" <berrange@redhat.com>,
	Bernd Schubert <bernd.schubert@itwm.fraunhofer.de>, coreutils@gnu.org,
	Sven Breuner <sven.breuner@itwm.fraunhofer.de>,
	=?ISO-8859-1?Q?J=E9r=E9my_Compostella?= <jeremy.compostella@gmail.com>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 17 Feb 2012 16:07:11 -0000

On 02/17/2012 04:15 PM, Jim Meyering wrote:
> And here's the patch I've just squashed into the test c-set.
> 
> diff --git a/tests/ls/getxattr-speedup b/tests/ls/getxattr-speedup
> index eaa3342..a3d47c1 100755
> --- a/tests/ls/getxattr-speedup
> +++ b/tests/ls/getxattr-speedup
> @@ -28,16 +28,18 @@ fd=33
>  # determine the total number of calls.  FIXME: there must be a better way.
>  cat > k.c <<'EOF' || framework_failure_
>  #include <errno.h>
> -#include <unistd.h>
> -static void
> -write_1 (void)
> -{
> -  write (FD, "x\n", 2);
> -}
> +#include <stdio.h>
> +static unsigned long int n_calls;
>  ssize_t getxattr (const char *path, const char *name, void *value, size_t size)
> -{ write_1 (); errno = ENOTSUP; return -1; }
> +{ ++n_calls; errno = ENOTSUP; return -1; }
>  ssize_t lgetxattr(const char *path, const char *name, void *value, size_t size)
> -{ write_1 (); errno = ENOTSUP; return -1; }
> +{ ++n_calls; errno = ENOTSUP; return -1; }
> +__attribute__((destructor)) void p()
> +{
> +  FILE *fp = fdopen (FD, "a"); if (!fp) return;
> +  fprintf (fp, "%lu\n", n_calls);
> +  fclose (fp);
> +}
>  EOF
> 
>  # Then compile/link it:
> @@ -52,7 +54,7 @@ seq 100 | xargs touch || framework_failure_
>  eval "LD_PRELOAD=./k.so ls --color=always -l . $fd>x" || fail=1
> 
>  # Ensure that there were no more than 3 *getxattr calls.
> -n_calls=$(wc -l < x)
> +n_calls=$(cat x)
>  test "$n_calls" -le 3 || fail=1
> 
>  Exit $fail

Hmm, this doesn't work here - "x" is not created :-(

++ cat x
+ n_calls=
+ test '' -le 3
./ls/getxattr-speedup: line 58: test: : integer expression expected

gcc (SUSE Linux) 4.6.2
openSUSE 12.1 (x86_64)

Another suggestion: writing into "x" in write_1(). By this, you can
also get rid of fd=33 ... admittedly to the cost of maybe several
fopen calls. On my system, n_calls doesn't get more than 1.
We could yet s/seq 100/seq 20/.


diff --git a/tests/ls/getxattr-speedup b/tests/ls/getxattr-speedup
index ecdd126..60a72ed 100755
--- a/tests/ls/getxattr-speedup
+++ b/tests/ls/getxattr-speedup
@@ -21,18 +21,18 @@
 . "${srcdir=.}/init.sh"; path_prepend_ ../src
 print_ver_ ls

-fd=33
-
 # Replace each getxattr and lgetxattr call with a call to these stubs.
-# Each writes a single line to the specified FD so that we can later
-# determine the total number of calls.  FIXME: there must be a better way.
+# Each writes the number of calls into "x".
 cat > k.c <<'EOF' || framework_failure_
 #include <errno.h>
-#include <unistd.h>
+#include <stdio.h>
 static void
 write_1 (void)
 {
-  write (FD, "x\n", 2);
+  static unsigned long int n_calls;
+  ++n_calls;
+  FILE *fp = fopen ("x", "w"); if (!fp) return;
+  fprintf (fp, "%lu\n", n_calls);
 }
 ssize_t getxattr (const char *path, const char *name, void *value, size_t size)
 { write_1 (); errno = ENOTSUP; return -1; }
@@ -41,7 +41,7 @@ ssize_t lgetxattr(const char *path, const char *name, void *value, size_t size)
 EOF

 # Then compile/link it:
-$CC -DFD=$fd -fPIC -O2 -c k.c \
+$CC -fPIC -O2 -c k.c \
   || framework_failure_ 'failed to compile with -fPIC'
 ld -G k.o -o k.so || framework_failure_ 'failed to invoke ld -G ...'

@@ -49,10 +49,10 @@ ld -G k.o -o k.so || framework_failure_ 'failed to invoke ld -G ...'
 seq 100 | xargs touch || framework_failure_

 # Finally, run the test, redirecting
-eval "LD_PRELOAD=$PWD/k.so ls --color=always -l . $fd>x" || fail=1
+eval "LD_PRELOAD=./k.so ls --color=always -l ." || fail=1

 # Ensure that there were no more than 3 *getxattr calls.
-n_calls=$(wc -l < x)
+n_calls=$(cat x)
 test "$n_calls" -le 3 || fail=1

 Exit $fail



From MAILER-DAEMON Fri Feb 17 11:20:21 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RyQXl-0007WS-4S
	for mharc-coreutils@gnu.org; Fri, 17 Feb 2012 11:20:21 -0500
Received: from eggs.gnu.org ([140.186.70.92]:59710)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1RyQXd-0007Va-A1
	for coreutils@gnu.org; Fri, 17 Feb 2012 11:20:19 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1RyQXY-0005vt-3Z
	for coreutils@gnu.org; Fri, 17 Feb 2012 11:20:13 -0500
Received: from mx.meyering.net ([88.168.87.75]:39429)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1RyQXX-0005uu-PU
	for coreutils@gnu.org; Fri, 17 Feb 2012 11:20:08 -0500
Received: from rho.meyering.net (localhost.localdomain [127.0.0.1])
	by rho.meyering.net (Acme Bit-Twister) with ESMTP id DC2BD60064;
	Fri, 17 Feb 2012 17:20:05 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: Bernhard Voelker <mail@bernhard-voelker.de>
Subject: Re: "ls -l": Avoid unnecessary getxattr() overhead
In-Reply-To: <4F3E7B1E.3090300@bernhard-voelker.de> (Bernhard Voelker's
	message of "Fri, 17 Feb 2012 17:06:54 +0100")
References: <87pqds3snt.fsf@Apollo.jerryland.fr>
	<4F30206B.1020802@itwm.fraunhofer.de>
	<CA+TATJHorgsuY7pxmpxxAeWUHRP5MfFfzQP05o=1H6C4ttswdA@mail.gmail.com>
	<4F3032BC.50702@draigBrady.com> <4F30631A.2070009@itwm.fraunhofer.de>
	<87y5scfdiq.fsf@rho.meyering.net> <4F33B5D7.3010909@draigBrady.com>
	<87mx8sf5w0.fsf@rho.meyering.net>
	<4F3437CE.4050308@itwm.fraunhofer.de>
	<4F346CC7.4030608@draigBrady.com> <87r4y3dry0.fsf@rho.meyering.net>
	<4F3710CA.8020402@itwm.fraunhofer.de>
	<8739aao9e2.fsf@rho.meyering.net>
	<4F3E12F9.6080102@bernhard-voelker.de>
	<8762f5lpct.fsf@rho.meyering.net> <4F3E5BBD.80809@redhat.com>
	<874nupk1cx.fsf@rho.meyering.net> <4F3E5E8E.2040802@redhat.com>
	<87lio1ij8b.fsf@rho.meyering.net>
	<4F3E7B1E.3090300@bernhard-voelker.de>
Date: Fri, 17 Feb 2012 17:20:05 +0100
Message-ID: <8739a9ig96.fsf@rho.meyering.net>
Lines: 203
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 88.168.87.75
Cc: "Daniel P. Berrange" <berrange@redhat.com>,
	Bernd Schubert <bernd.schubert@itwm.fraunhofer.de>, coreutils@gnu.org,
	Sven Breuner <sven.breuner@itwm.fraunhofer.de>,
	=?iso-8859-1?Q?J=E9r=E9my?= Compostella <jeremy.compostella@gmail.com>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 17 Feb 2012 16:20:19 -0000

Bernhard Voelker wrote:

> On 02/17/2012 04:15 PM, Jim Meyering wrote:
>> And here's the patch I've just squashed into the test c-set.
>>
>> diff --git a/tests/ls/getxattr-speedup b/tests/ls/getxattr-speedup
>> index eaa3342..a3d47c1 100755
>> --- a/tests/ls/getxattr-speedup
>> +++ b/tests/ls/getxattr-speedup
>> @@ -28,16 +28,18 @@ fd=33
>>  # determine the total number of calls.  FIXME: there must be a better way.
>>  cat > k.c <<'EOF' || framework_failure_
>>  #include <errno.h>
>> -#include <unistd.h>
>> -static void
>> -write_1 (void)
>> -{
>> -  write (FD, "x\n", 2);
>> -}
>> +#include <stdio.h>
>> +static unsigned long int n_calls;
>>  ssize_t getxattr (const char *path, const char *name, void *value, size_t size)
>> -{ write_1 (); errno = ENOTSUP; return -1; }
>> +{ ++n_calls; errno = ENOTSUP; return -1; }
>>  ssize_t lgetxattr(const char *path, const char *name, void *value, size_t size)
>> -{ write_1 (); errno = ENOTSUP; return -1; }
>> +{ ++n_calls; errno = ENOTSUP; return -1; }
>> +__attribute__((destructor)) void p()
>> +{
>> +  FILE *fp = fdopen (FD, "a"); if (!fp) return;
>> +  fprintf (fp, "%lu\n", n_calls);
>> +  fclose (fp);
>> +}
>>  EOF
>>
>>  # Then compile/link it:
>> @@ -52,7 +54,7 @@ seq 100 | xargs touch || framework_failure_
>>  eval "LD_PRELOAD=./k.so ls --color=always -l . $fd>x" || fail=1
>>
>>  # Ensure that there were no more than 3 *getxattr calls.
>> -n_calls=$(wc -l < x)
>> +n_calls=$(cat x)
>>  test "$n_calls" -le 3 || fail=1
>>
>>  Exit $fail
>
> Hmm, this doesn't work here - "x" is not created :-(
>
> ++ cat x
> + n_calls=
> + test '' -le 3
> ./ls/getxattr-speedup: line 58: test: : integer expression expected
>
> gcc (SUSE Linux) 4.6.2
> openSUSE 12.1 (x86_64)
>
> Another suggestion: writing into "x" in write_1(). By this, you can
> also get rid of fd=33 ... admittedly to the cost of maybe several
> fopen calls. On my system, n_calls doesn't get more than 1.
> We could yet s/seq 100/seq 20/.
>
>
> diff --git a/tests/ls/getxattr-speedup b/tests/ls/getxattr-speedup
> index ecdd126..60a72ed 100755
> --- a/tests/ls/getxattr-speedup
> +++ b/tests/ls/getxattr-speedup
> @@ -21,18 +21,18 @@
>  . "${srcdir=.}/init.sh"; path_prepend_ ../src
>  print_ver_ ls
>
> -fd=33
> -
>  # Replace each getxattr and lgetxattr call with a call to these stubs.
> -# Each writes a single line to the specified FD so that we can later
> -# determine the total number of calls.  FIXME: there must be a better way.
> +# Each writes the number of calls into "x".
>  cat > k.c <<'EOF' || framework_failure_
>  #include <errno.h>
> -#include <unistd.h>
> +#include <stdio.h>
>  static void
>  write_1 (void)
>  {
> -  write (FD, "x\n", 2);
> +  static unsigned long int n_calls;
> +  ++n_calls;
> +  FILE *fp = fopen ("x", "w"); if (!fp) return;
> +  fprintf (fp, "%lu\n", n_calls);
>  }
>  ssize_t getxattr (const char *path, const char *name, void *value, size_t size)
>  { write_1 (); errno = ENOTSUP; return -1; }
> @@ -41,7 +41,7 @@ ssize_t lgetxattr(const char *path, const char *name, void *value, size_t size)
>  EOF
>
>  # Then compile/link it:
> -$CC -DFD=$fd -fPIC -O2 -c k.c \
> +$CC -fPIC -O2 -c k.c \
>    || framework_failure_ 'failed to compile with -fPIC'
>  ld -G k.o -o k.so || framework_failure_ 'failed to invoke ld -G ...'
>
> @@ -49,10 +49,10 @@ ld -G k.o -o k.so || framework_failure_ 'failed to invoke ld -G ...'
>  seq 100 | xargs touch || framework_failure_
>
>  # Finally, run the test, redirecting
> -eval "LD_PRELOAD=$PWD/k.so ls --color=always -l . $fd>x" || fail=1
> +eval "LD_PRELOAD=./k.so ls --color=always -l ." || fail=1

Good idea.
And with that, we can also drop the eval "...".
I have a slight preference for writing the file only from the destructor.
WDYT?

>From db3e5c56bb191c5991d2852d4396517e844bb8f3 Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering@redhat.com>
Date: Mon, 13 Feb 2012 12:52:19 +0100
Subject: [PATCH] tests: test for ls speed-up

* tests/ls/getxattr-speedup: New test.
* tests/Makefile.am (TESTS): Add it.
Improved-by: Bernhard Voelker.
---
 tests/Makefile.am         |    1 +
 tests/ls/getxattr-speedup |   57 +++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 58 insertions(+)
 create mode 100755 tests/ls/getxattr-speedup

diff --git a/tests/Makefile.am b/tests/Makefile.am
index 7b53681..625b636 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -426,6 +426,7 @@ TESTS =						\
   ls/dired					\
   ls/file-type					\
   ls/follow-slink				\
+  ls/getxattr-speedup				\
   ls/infloop					\
   ls/inode					\
   ls/m-option					\
diff --git a/tests/ls/getxattr-speedup b/tests/ls/getxattr-speedup
new file mode 100755
index 0000000..927cbe8
--- /dev/null
+++ b/tests/ls/getxattr-speedup
@@ -0,0 +1,57 @@
+#!/bin/sh
+# Show that we've eliminated most of ls' failing getxattr syscalls,
+# regardless of how many files are in a directory we list.
+# This test is skipped on systems that lack LD_PRELOAD support; that's fine.
+
+# Copyright (C) 2012 Free Software Foundation, Inc.
+
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+. "${srcdir=.}/init.sh"; path_prepend_ ../src
+print_ver_ ls
+
+# Replace each getxattr and lgetxattr call with a call to these stubs.
+# Count those and write the total number of calls to the file "x"
+# via a global destructor.
+cat > k.c <<'EOF' || framework_failure_
+#include <errno.h>
+#include <stdio.h>
+static unsigned long int n_calls;
+ssize_t getxattr (const char *path, const char *name, void *value, size_t size)
+{ ++n_calls; errno = ENOTSUP; return -1; }
+ssize_t lgetxattr(const char *path, const char *name, void *value, size_t size)
+{ ++n_calls; errno = ENOTSUP; return -1; }
+__attribute__((destructor)) void p()
+{
+  FILE *fp = fopen ("x", "w"); if (!fp) return;
+  fprintf (fp, "%lu\n", n_calls);
+  fclose (fp);
+}
+EOF
+
+# Then compile/link it:
+$CC -fPIC -O2 -c k.c || framework_failure_ 'failed to compile with -fPIC'
+ld -G k.o -o k.so || framework_failure_ 'failed to invoke ld -G ...'
+
+# create a few files
+seq 100 | xargs touch || framework_failure_
+
+# Finally, run the test, redirecting
+LD_PRELOAD=./k.so ls --color=always -l . || fail=1
+
+# Ensure that there were no more than 3 *getxattr calls.
+n_calls=$(cat x)
+test "$n_calls" -le 3 || fail=1
+
+Exit $fail
--
1.7.9.1.245.gbbe07


From MAILER-DAEMON Fri Feb 17 12:57:30 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RyS3m-0002c5-5a
	for mharc-coreutils@gnu.org; Fri, 17 Feb 2012 12:57:30 -0500
Received: from eggs.gnu.org ([140.186.70.92]:39386)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1RyS3e-0002Xg-7J
	for coreutils@gnu.org; Fri, 17 Feb 2012 12:57:28 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1RyS3X-0007Og-Gs
	for coreutils@gnu.org; Fri, 17 Feb 2012 12:57:22 -0500
Received: from mail-iy0-f169.google.com ([209.85.210.169]:45909)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1RyS3X-0007OY-4F
	for coreutils@gnu.org; Fri, 17 Feb 2012 12:57:15 -0500
Received: by iagz16 with SMTP id z16so5959973iag.0
	for <coreutils@gnu.org>; Fri, 17 Feb 2012 09:57:14 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma;
	h=mime-version:in-reply-to:references:from:date:message-id:subject:to
	:cc:content-type;
	bh=qL/oDvaF880b5kmB/BAAz/mHiMyATn5izm1BUtbxi7A=;
	b=TmFB6NMA5DCNj22wMfWdSOyjAYOO3nMa9GWdqfCoJQkqEJe9s95nspycuN6dAHOUfd
	RfCiu4bCAR1hXgBPWN0KUm6cT9KwMTQbtKaczO1Pl7HlgTkRkvhOXbyfZV8C4ds3O6Wi
	3WmbKtxTyjsaIRlLY+7LsTM0C8heIad9+ugEU=
Received: by 10.42.235.137 with SMTP id kg9mr7919956icb.47.1329501434336; Fri,
	17 Feb 2012 09:57:14 -0800 (PST)
MIME-Version: 1.0
Received: by 10.42.29.196 with HTTP; Fri, 17 Feb 2012 09:56:54 -0800 (PST)
In-Reply-To: <4F3DB7B9.6020201@draigBrady.com>
References: <87vcnk3vxc.fsf@Apollo.jerryland.fr>
	<CA+TATJFys_1jcxP=UuxykbTNA_ntY3CMhNp4WWHAqbWjbPhaVw@mail.gmail.com>
	<4F3DB7B9.6020201@draigBrady.com>
From: =?ISO-8859-1?Q?J=E9r=E9my_Compostella?= <jeremy.compostella@gmail.com>
Date: Fri, 17 Feb 2012 18:56:54 +0100
Message-ID: <CA+TATJGTaP0rOOhWqdwYM8CgTqe8ZdKW7ucMzNFDBvxokz2wqQ@mail.gmail.com>
Subject: Re: bug#9085: 'split' feature request: an option to uses e.g. '.001'
	as first suffix.
To: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigbrady.com>
Content-Type: multipart/mixed; boundary=20cf3026697ab8d64004b92cacb6
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.85.210.169
Cc: 9085@debbugs.gnu.org, coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 17 Feb 2012 17:57:28 -0000

--20cf3026697ab8d64004b92cacb6
Content-Type: multipart/alternative; boundary=20cf3026697ab8d63a04b92cacb4

--20cf3026697ab8d63a04b92cacb4
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

> Thanks for continuing with this.
> One general thing that might both improve
> and simplify the implementation, is to
> not to convert from string to int at all.
>
> I.E. when processing the arg, just validate like:
> if (strlen (optarg) !=3D strspn (optarg, suffix_alphabet))
>  error()
> else
>   /* skip over any leading 0, and use this as the start directly. */
>
> Then the subsequent check for length and
> the initialization of the file name should be simplified.
This is more like my first implementation indeed. But since I didn't
know about the strspn() function the code was a little bit more complex
without converting to int.

I attached the new patch. I tried to make it as simple as possible but
feel free to comment.

> Also this removes the limitation of size of an unsigned int,
> though that's not really a practical concern I suppose.
Fair point. IMHO, even if it's not a really practical concern
it's stupid to have such an easy to avoid restriction.

> I've also attached some string and test cleanups,
> to --amend into your patch.
Applied.

Cheers,

J=E9r=E9my

--20cf3026697ab8d63a04b92cacb4
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div><div>&gt; Thanks for continuing with this.</div><div>&gt; One general =
thing that might both improve</div><div>&gt; and simplify the implementatio=
n, is to</div><div>&gt; not to convert from string to int at all.</div><div=
>

&gt;=A0</div><div>&gt; I.E. when processing the arg, just validate like:</d=
iv><div>&gt; if (strlen (optarg) !=3D strspn (optarg, suffix_alphabet))</di=
v><div>&gt; =A0error()</div><div>&gt; else</div><div>&gt; =A0 /* skip over =
any leading 0, and use this as the start directly. */</div>

<div>&gt;=A0</div><div>&gt; Then the subsequent check for length and</div><=
div>&gt; the initialization of the file name should be simplified.</div><di=
v>This is more like my first implementation indeed. But since I didn&#39;t<=
/div>

<div>know about the strspn() function the code was a little bit more comple=
x</div><div>without converting to int.</div><div><br></div><div>I attached =
the new patch. I tried to make it as simple as possible but</div><div>
feel free to comment.</div>
<div><br></div><div>&gt; Also this removes the limitation of size of an uns=
igned int,</div><div>&gt; though that&#39;s not really a practical concern =
I suppose.</div><div>Fair point. IMHO, even if it&#39;s not a really practi=
cal concern</div>

<div>it&#39;s stupid to have=A0such an easy to avoid restriction.=A0</div><=
div><br></div><div>&gt; I&#39;ve also attached some string and test cleanup=
s,</div><div>&gt; to --amend into your patch.</div><div>Applied.</div></div=
>

<div><br></div><div>Cheers,</div><div><br></div><div>J=E9r=E9my</div>

--20cf3026697ab8d63a04b92cacb4--
--20cf3026697ab8d64004b92cacb6
Content-Type: text/x-patch; charset=US-ASCII; 
	name="0001-split-numeric-suffixes-new-optional-from-argument-bu.patch"
Content-Disposition: attachment; 
	filename="0001-split-numeric-suffixes-new-optional-from-argument-bu.patch"
Content-Transfer-Encoding: base64
X-Attachment-Id: f_gyrimvwo1

RnJvbSBlODQ0YjQ3NGNlMWE1YTM1MzMwZTI1OThjZWIyNzFiMmEyMmUwMDJkIE1vbiBTZXAgMTcg
MDA6MDA6MDAgMjAwMQpGcm9tOiA9P1VURi04P3E/Sj1DMz1BOXI9QzM9QTlteT0yMENvbXBvc3Rl
bGxhPz0gPGplcmVteS5jb21wb3N0ZWxsYUBnbWFpbC5jb20+CkRhdGU6IFN1biwgMjkgSmFuIDIw
MTIgMTU6MjA6MzEgKzAxMDAKU3ViamVjdDogW1BBVENIXSBzcGxpdDogLS1udW1lcmljLXN1ZmZp
eGVzIG5ldyBvcHRpb25hbCAiZnJvbSIgYXJndW1lbnQgKGJ1ZyM5MDg1KQoKc3BsaXQgbm93IGFj
Y2VwdHMgYW4gb3B0aW9uYWwgImZyb20iIGFyZ3VtZW50IHRvIC0tbnVtZXJpYy1zdWZmaXhlcywK
d2hpY2ggY2hhbmdlcyB0aGUgc3RhcnQgbnVtYmVyIGZyb20gdGhlIGRlZmF1bHQgb2YgMC4KCiog
c3JjL3NwbGl0LmMgKG5leHRfZmlsZV9uYW1lKTogSW5pdGlhbGl6ZSB0aGUgc3VmZml4IGluZGV4
CmFuZCB0aGUgb3V0cHV0IGZpbGVuYW1lIGFjY29yZGluZyB0byBzdGFydCB2YWx1ZS4KKG1haW4p
OiBDaGVjayB0aGF0IHRoZSBzdWZmaXggbGVuZ3RoIGlzIGxhcmdlIGVub3VnaCBmb3IgdGhlCm51
bWVyaWNhbCBzdWZmaXggc3RhcnQgdmFsdWUuCiogZG9jL2NvcmV1dGlscy50ZXhpIChzcGxpdCBp
bnZvY2F0aW9uKTogTWVudGlvbiBpdC4KKiBORVdTIChOZXcgZmVhdHVyZXMpOiBNZW50aW9uIGl0
LgoqIHRlc3RzL3NwbGl0L251bWVyaWM6IE5ldyBmaWxlLiAtLW51bWVyaWMtc3VmZml4ZXNbPUZS
T01dIG9wdGlvbgogIHRlc3RzLgoqIHRlc3RzL01ha2VmaWxlLmFtIChURVNUUyk6IEFkZCBpdC4K
LS0tCiBORVdTICAgICAgICAgICAgICAgIHwgICAgMyArKwogZG9jL2NvcmV1dGlscy50ZXhpICB8
ICAgIDUgKystCiBzcmMvc3BsaXQuYyAgICAgICAgIHwgICA0NiArKysrKysrKysrKysrKysrKysr
KysrKysrKysrKysrKystLQogdGVzdHMvTWFrZWZpbGUuYW0gICB8ICAgIDEgKwogdGVzdHMvc3Bs
aXQvbnVtZXJpYyB8ICAgNjUgKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysr
KysrKysrKysrKysrCiA1IGZpbGVzIGNoYW5nZWQsIDExNSBpbnNlcnRpb25zKCspLCA1IGRlbGV0
aW9ucygtKQogY3JlYXRlIG1vZGUgMTAwNzU1IHRlc3RzL3NwbGl0L251bWVyaWMKCmRpZmYgLS1n
aXQgYS9ORVdTIGIvTkVXUwppbmRleCAyMDgwNzkwLi40NjYyOTE5IDEwMDY0NAotLS0gYS9ORVdT
CisrKyBiL05FV1MKQEAgLTcsNiArNyw5IEBAIEdOVSBjb3JldXRpbHMgTkVXUyAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIC0qLSBvdXRsaW5lIC0qLQogICBkZCBub3cgYWNjZXB0
cyB0aGUgY291bnRfYnl0ZXMsIHNraXBfYnl0ZXMgaWZsYWdzIGFuZCB0aGUgY291bnRfYnl0ZXMK
ICAgb2ZsYWcsIHRvIG1vcmUgZWFzaWx5IGFsbG93IHByb2Nlc3NpbmcgcG9ydGlvbnMgb2YgYSBm
aWxlLgogCisgIHNwbGl0IG5vdyBhY2NlcHRzIGFuIG9wdGlvbmFsICJmcm9tIiBhcmd1bWVudCB0
byAtLW51bWVyaWMtc3VmZml4ZXMsCisgIHdoaWNoIGNoYW5nZXMgdGhlIHN0YXJ0IG51bWJlciBm
cm9tIHRoZSBkZWZhdWx0IG9mIDAuCisKICoqIEJ1ZyBmaXhlcwogCiAgIG12IG5vdyBsZXRzIHlv
dSBtb3ZlIGEgc3ltbGluayBvbnRvIGEgc2FtZS1pbm9kZSBkZXN0aW5hdGlvbiBmaWxlIHRoYXQK
ZGlmZiAtLWdpdCBhL2RvYy9jb3JldXRpbHMudGV4aSBiL2RvYy9jb3JldXRpbHMudGV4aQppbmRl
eCA4YzZhMjg3Li45ZjM3OGU4IDEwMDY0NAotLS0gYS9kb2MvY29yZXV0aWxzLnRleGkKKysrIGIv
ZG9jL2NvcmV1dGlscy50ZXhpCkBAIC0zMDg0LDEwICszMDg0LDExIEBAIGFuZCBzbyBjYW4gYmUg
YSBwaXBlIGZvciBleGFtcGxlLgogVXNlIHN1ZmZpeGVzIG9mIGxlbmd0aCBAdmFye2xlbmd0aH0u
ICBUaGUgZGVmYXVsdCBAdmFye2xlbmd0aH0gaXMgMi4KIAogQGl0ZW0gLWQKLUBpdGVteCAtLW51
bWVyaWMtc3VmZml4ZXMKK0BpdGVteCAtLW51bWVyaWMtc3VmZml4ZXNbPUB2YXJ7ZnJvbX1dCiBA
b3BpbmRleCAtZAogQG9waW5kZXggLS1udW1lcmljLXN1ZmZpeGVzCi1Vc2UgZGlnaXRzIGluIHN1
ZmZpeGVzIHJhdGhlciB0aGFuIGxvd2VyLWNhc2UgbGV0dGVycy4KK1VzZSBkaWdpdHMgaW4gc3Vm
Zml4ZXMgcmF0aGVyIHRoYW4gbG93ZXItY2FzZSBsZXR0ZXJzLiAgVGhlIG51bWVyaWNhbAorc3Vm
Zml4IGNvdW50cyBmcm9tIEB2YXJ7ZnJvbX0gaWYgc3BlY2lmaWVkLCAwIG90aGVyd2lzZS4KIAog
QGl0ZW0gLWUKIEBpdGVteCAtLWVsaWRlLWVtcHR5LWZpbGVzCmRpZmYgLS1naXQgYS9zcmMvc3Bs
aXQuYyBiL3NyYy9zcGxpdC5jCmluZGV4IDFkMDMxMGMuLjgyNjZjZmQgMTAwNjQ0Ci0tLSBhL3Ny
Yy9zcGxpdC5jCisrKyBiL3NyYy9zcGxpdC5jCkBAIC04MCw2ICs4MCw5IEBAIHN0YXRpYyBzaXpl
X3Qgc3VmZml4X2xlbmd0aDsKIC8qIEFscGhhYmV0IG9mIGNoYXJhY3RlcnMgdG8gdXNlIGluIHN1
ZmZpeC4gICovCiBzdGF0aWMgY2hhciBjb25zdCAqc3VmZml4X2FscGhhYmV0ID0gImFiY2RlZmdo
aWprbG1ub3BxcnN0dXZ3eHl6IjsKIAorLyogTnVtZXJpY2FsIHN1ZmZpeCBzdGFydCB2YWx1ZS4g
ICovCitzdGF0aWMgY29uc3QgY2hhciAqbnVtZXJpY19zdWZmaXhfc3RhcnQ7CisKIC8qIE5hbWUg
b2YgaW5wdXQgZmlsZS4gIE1heSBiZSAiLSIuICAqLwogc3RhdGljIGNoYXIgKmluZmlsZTsKIApA
QCAtMTIyLDcgKzEyNSw3IEBAIHN0YXRpYyBzdHJ1Y3Qgb3B0aW9uIGNvbnN0IGxvbmdvcHRzW10g
PQogICB7ImVsaWRlLWVtcHR5LWZpbGVzIiwgbm9fYXJndW1lbnQsIE5VTEwsICdlJ30sCiAgIHsi
dW5idWZmZXJlZCIsIG5vX2FyZ3VtZW50LCBOVUxMLCAndSd9LAogICB7InN1ZmZpeC1sZW5ndGgi
LCByZXF1aXJlZF9hcmd1bWVudCwgTlVMTCwgJ2EnfSwKLSAgeyJudW1lcmljLXN1ZmZpeGVzIiwg
bm9fYXJndW1lbnQsIE5VTEwsICdkJ30sCisgIHsibnVtZXJpYy1zdWZmaXhlcyIsIG9wdGlvbmFs
X2FyZ3VtZW50LCBOVUxMLCAnZCd9LAogICB7ImZpbHRlciIsIHJlcXVpcmVkX2FyZ3VtZW50LCBO
VUxMLCBGSUxURVJfT1BUSU9OfSwKICAgeyJ2ZXJib3NlIiwgbm9fYXJndW1lbnQsIE5VTEwsIFZF
UkJPU0VfT1BUSU9OfSwKICAgeyItaW8tYmxrc2l6ZSIsIHJlcXVpcmVkX2FyZ3VtZW50LCBOVUxM
LApAQCAtMTk1LDcgKzE5OCw4IEBAIE1hbmRhdG9yeSBhcmd1bWVudHMgdG8gbG9uZyBvcHRpb25z
IGFyZSBtYW5kYXRvcnkgZm9yIHNob3J0IG9wdGlvbnMgdG9vLlxuXAogICAtYSwgLS1zdWZmaXgt
bGVuZ3RoPU4gICB1c2Ugc3VmZml4ZXMgb2YgbGVuZ3RoIE4gKGRlZmF1bHQgJWQpXG5cCiAgIC1i
LCAtLWJ5dGVzPVNJWkUgICAgICAgIHB1dCBTSVpFIGJ5dGVzIHBlciBvdXRwdXQgZmlsZVxuXAog
ICAtQywgLS1saW5lLWJ5dGVzPVNJWkUgICBwdXQgYXQgbW9zdCBTSVpFIGJ5dGVzIG9mIGxpbmVz
IHBlciBvdXRwdXQgZmlsZVxuXAotICAtZCwgLS1udW1lcmljLXN1ZmZpeGVzICB1c2UgbnVtZXJp
YyBzdWZmaXhlcyBpbnN0ZWFkIG9mIGFscGhhYmV0aWNcblwKKyAgLWQsIC0tbnVtZXJpYy1zdWZm
aXhlc1s9RlJPTV0gIHVzZSBudW1lcmljIHN1ZmZpeGVzIGluc3RlYWQgb2YgYWxwaGFiZXRpYy5c
blwKKyAgICAgICAgICAgICAgICAgICAgICAgIFdoZW4gc3BlY2lmaWVkLCBzdGFydCBjb3VudGlu
ZyBmcm9tIEZST00sIDAgb3RoZXJ3aXNlXG5cCiAgIC1lLCAtLWVsaWRlLWVtcHR5LWZpbGVzICBk
byBub3QgZ2VuZXJhdGUgZW1wdHkgb3V0cHV0IGZpbGVzIHdpdGggJy1uJ1xuXAogICAgICAgLS1m
aWx0ZXI9Q09NTUFORCAgICB3cml0ZSB0byBzaGVsbCBDT01NQU5EOyBmaWxlIG5hbWUgaXMgJEZJ
TEVcblwKICAgLWwsIC0tbGluZXM9TlVNQkVSICAgICAgcHV0IE5VTUJFUiBsaW5lcyBwZXIgb3V0
cHV0IGZpbGVcblwKQEAgLTI0Myw5ICsyNDcsMTkgQEAgbmV4dF9maWxlX25hbWUgKHZvaWQpCiAg
ICAgICBvdXRmaWxlID0geG1hbGxvYyAob3V0ZmlsZV9sZW5ndGggKyAxKTsKICAgICAgIG91dGZp
bGVfbWlkID0gb3V0ZmlsZSArIG91dGJhc2VfbGVuZ3RoOwogICAgICAgbWVtY3B5IChvdXRmaWxl
LCBvdXRiYXNlLCBvdXRiYXNlX2xlbmd0aCk7CisgICAgICBzdWZpbmRleCA9IHhjYWxsb2MgKHN1
ZmZpeF9sZW5ndGgsIHNpemVvZiAqc3VmaW5kZXgpOwogICAgICAgbWVtc2V0IChvdXRmaWxlX21p
ZCwgc3VmZml4X2FscGhhYmV0WzBdLCBzdWZmaXhfbGVuZ3RoKTsKKyAgICAgIGlmIChudW1lcmlj
X3N1ZmZpeF9zdGFydCkKKyAgICAgICAgeworICAgICAgICAgIC8qIEluaXRpYWxpemUgdGhlIHN1
ZmZpeCBpbmRleCBhbmQgdGhlIG91dHB1dCBmaWxlbmFtZQorICAgICAgICAgICAgIGFjY29yZGlu
ZyB0byBzdGFydCB2YWx1ZS4gICovCisgICAgICAgICAgc2l6ZV90IGkgPSBzdHJsZW4gKG51bWVy
aWNfc3VmZml4X3N0YXJ0KTsKKyAgICAgICAgICBzaXplX3QgKnN1ZmluZGV4X2VuZCA9IHN1Zmlu
ZGV4ICsgc3VmZml4X2xlbmd0aDsKKyAgICAgICAgICBtZW1jcHkgKG91dGZpbGVfbWlkICsgc3Vm
Zml4X2xlbmd0aCAtIGksIG51bWVyaWNfc3VmZml4X3N0YXJ0LCBpKTsKKyAgICAgICAgICB3aGls
ZSAoaS0tICE9IDApCisgICAgICAgICAgICAqLS1zdWZpbmRleF9lbmQgPSBudW1lcmljX3N1ZmZp
eF9zdGFydFtpXSAtICcwJzsKKyAgICAgICAgfQogICAgICAgb3V0ZmlsZVtvdXRmaWxlX2xlbmd0
aF0gPSAwOwotICAgICAgc3VmaW5kZXggPSB4Y2FsbG9jIChzdWZmaXhfbGVuZ3RoLCBzaXplb2Yg
KnN1ZmluZGV4KTsKIAogI2lmICEgX1BPU0lYX05PX1RSVU5DICYmIEhBVkVfUEFUSENPTkYgJiYg
ZGVmaW5lZCBfUENfTkFNRV9NQVgKICAgICAgIC8qIFBPU0lYIHJlcXVpcmVzIHRoYXQgaWYgdGhl
IG91dHB1dCBmaWxlIG5hbWUgaXMgdG9vIGxvbmcgZm9yCkBAIC0xMTQyLDYgKzExNTYsMjMgQEAg
bWFpbiAoaW50IGFyZ2MsIGNoYXIgKiphcmd2KQogCiAgICAgICAgIGNhc2UgJ2QnOgogICAgICAg
ICAgIHN1ZmZpeF9hbHBoYWJldCA9ICIwMTIzNDU2Nzg5IjsKKyAgICAgICAgICBpZiAob3B0YXJn
KQorICAgICAgICAgICAgeworICAgICAgICAgICAgICBpZiAoc3RybGVuIChvcHRhcmcpICE9IHN0
cnNwbiAob3B0YXJnLCBzdWZmaXhfYWxwaGFiZXQpKQorICAgICAgICAgICAgICAgIHsKKyAgICAg
ICAgICAgICAgICAgIGVycm9yICgwLCAwLAorICAgICAgICAgICAgICAgICAgICAgICAgIF8oIiVz
OiBpbnZhbGlkIHN0YXJ0IHZhbHVlIGZvciBudW1lcmljYWwgc3VmZml4IiksCisgICAgICAgICAg
ICAgICAgICAgICAgICAgb3B0YXJnKTsKKyAgICAgICAgICAgICAgICAgIHVzYWdlIChFWElUX0ZB
SUxVUkUpOworICAgICAgICAgICAgICAgIH0KKyAgICAgICAgICAgICAgZWxzZQorICAgICAgICAg
ICAgICAgIHsKKyAgICAgICAgICAgICAgICAgIC8qIFNraXAgYW55IGxlYWRpbmcgemVyby4gICov
CisgICAgICAgICAgICAgICAgICB3aGlsZSAoKm9wdGFyZyA9PSAnMCcgJiYgKihvcHRhcmcgKyAx
KSAhPSAnXDAnKQorICAgICAgICAgICAgICAgICAgICBvcHRhcmcrKzsKKyAgICAgICAgICAgICAg
ICAgIG51bWVyaWNfc3VmZml4X3N0YXJ0ID0gb3B0YXJnOworICAgICAgICAgICAgICAgIH0KKyAg
ICAgICAgICAgIH0KICAgICAgICAgICBicmVhazsKIAogICAgICAgICBjYXNlICdlJzoKQEAgLTEy
MTIsNiArMTI0MywxNSBAQCBtYWluIChpbnQgYXJnYywgY2hhciAqKmFyZ3YpCiAgICAgICB1c2Fn
ZSAoRVhJVF9GQUlMVVJFKTsKICAgICB9CiAKKyAgLyogQ2hlY2sgdGhhdCB0aGUgc3VmZml4IGxl
bmd0aCBpcyBsYXJnZSBlbm91Z2ggZm9yIHRoZSBudW1lcmljYWwKKyAgICAgc3VmZml4IHN0YXJ0
IHZhbHVlLiAgKi8KKyAgaWYgKG51bWVyaWNfc3VmZml4X3N0YXJ0ICYmIHN0cmxlbiAobnVtZXJp
Y19zdWZmaXhfc3RhcnQpID4gc3VmZml4X2xlbmd0aCkKKyAgICB7CisgICAgICBlcnJvciAoMCwg
MCwgXygibnVtZXJpY2FsIHN1ZmZpeCBzdGFydCB2YWx1ZSBpcyB0b28gbGFyZ2UgIgorICAgICAg
ICAgICAgICAgICAgICAgImZvciB0aGUgc3VmZml4IGxlbmd0aCIpKTsKKyAgICAgIHVzYWdlIChF
WElUX0ZBSUxVUkUpOworICAgIH0KKwogICAvKiBPcGVuIHRoZSBpbnB1dCBmaWxlLiAgKi8KICAg
aWYgKCEgU1RSRVEgKGluZmlsZSwgIi0iKQogICAgICAgJiYgZmRfcmVvcGVuIChTVERJTl9GSUxF
Tk8sIGluZmlsZSwgT19SRE9OTFksIDApIDwgMCkKZGlmZiAtLWdpdCBhL3Rlc3RzL01ha2VmaWxl
LmFtIGIvdGVzdHMvTWFrZWZpbGUuYW0KaW5kZXggN2I1MzY4MS4uZWVkNmVkNiAxMDA2NDQKLS0t
IGEvdGVzdHMvTWFrZWZpbGUuYW0KKysrIGIvdGVzdHMvTWFrZWZpbGUuYW0KQEAgLTI1OSw2ICsy
NTksNyBAQCBURVNUUyA9CQkJCQkJXAogICBzcGxpdC9saW5lcwkJCQkJXAogICBzcGxpdC9sLWNo
dW5rCQkJCQlcCiAgIHNwbGl0L3ItY2h1bmsJCQkJCVwKKyAgc3BsaXQvbnVtZXJpYwkJCQkJXAog
ICBtaXNjL3N0YXQtYmlydGh0aW1lCQkJCVwKICAgbWlzYy9zdGF0LWZtdAkJCQkJXAogICBtaXNj
L3N0YXQtaHlwaGVuCQkJCVwKZGlmZiAtLWdpdCBhL3Rlc3RzL3NwbGl0L251bWVyaWMgYi90ZXN0
cy9zcGxpdC9udW1lcmljCm5ldyBmaWxlIG1vZGUgMTAwNzU1CmluZGV4IDAwMDAwMDAuLmFkMjJk
ZjYKLS0tIC9kZXYvbnVsbAorKysgYi90ZXN0cy9zcGxpdC9udW1lcmljCkBAIC0wLDAgKzEsNjUg
QEAKKyMhL2Jpbi9zaAorIyBTaG93IHRoYXQgc3BsaXQgLS1udW1lcmljLXN1ZmZpeGVzWz1mcm9t
XSB3b3Jrcy4KKworIyBDb3B5cmlnaHQgKEMpIDIwMTIgRnJlZSBTb2Z0d2FyZSBGb3VuZGF0aW9u
LCBJbmMuCisKKyMgVGhpcyBwcm9ncmFtIGlzIGZyZWUgc29mdHdhcmU6IHlvdSBjYW4gcmVkaXN0
cmlidXRlIGl0IGFuZC9vciBtb2RpZnkKKyMgaXQgdW5kZXIgdGhlIHRlcm1zIG9mIHRoZSBHTlUg
R2VuZXJhbCBQdWJsaWMgTGljZW5zZSBhcyBwdWJsaXNoZWQgYnkKKyMgdGhlIEZyZWUgU29mdHdh
cmUgRm91bmRhdGlvbiwgZWl0aGVyIHZlcnNpb24gMyBvZiB0aGUgTGljZW5zZSwgb3IKKyMgKGF0
IHlvdXIgb3B0aW9uKSBhbnkgbGF0ZXIgdmVyc2lvbi4KKworIyBUaGlzIHByb2dyYW0gaXMgZGlz
dHJpYnV0ZWQgaW4gdGhlIGhvcGUgdGhhdCBpdCB3aWxsIGJlIHVzZWZ1bCwKKyMgYnV0IFdJVEhP
VVQgQU5ZIFdBUlJBTlRZOyB3aXRob3V0IGV2ZW4gdGhlIGltcGxpZWQgd2FycmFudHkgb2YKKyMg
TUVSQ0hBTlRBQklMSVRZIG9yIEZJVE5FU1MgRk9SIEEgUEFSVElDVUxBUiBQVVJQT1NFLiAgU2Vl
IHRoZQorIyBHTlUgR2VuZXJhbCBQdWJsaWMgTGljZW5zZSBmb3IgbW9yZSBkZXRhaWxzLgorCisj
IFlvdSBzaG91bGQgaGF2ZSByZWNlaXZlZCBhIGNvcHkgb2YgdGhlIEdOVSBHZW5lcmFsIFB1Ymxp
YyBMaWNlbnNlCisjIGFsb25nIHdpdGggdGhpcyBwcm9ncmFtLiAgSWYgbm90LCBzZWUgPGh0dHA6
Ly93d3cuZ251Lm9yZy9saWNlbnNlcy8+LgorCisuICIke3NyY2Rpcj0ufS9pbml0LnNoIjsgcGF0
aF9wcmVwZW5kXyAuLi9zcmMKK3ByaW50X3Zlcl8gc3BsaXQKKworIyBDaGVjayBkZWZhdWx0IHN0
YXJ0IGZyb20gMAorcHJpbnRmICcxXG4yXG4zXG40XG41XG4nID4gaW4gfHwgZnJhbWV3b3JrX2Zh
aWx1cmVfCitzcGxpdCAtLW51bWVyaWMtc3VmZml4ZXMgLS1saW5lcz0yIGluIHx8IGZhaWw9MQor
Y2F0IDw8XEVPRiA+IGV4cC0xCisxCisyCitFT0YKK2NhdCA8PFxFT0YgPiBleHAtMgorMworNAor
RU9GCitjYXQgPDxcRU9GID4gZXhwLTMKKzUKK0VPRgorY29tcGFyZSBleHAtMSB4MDAgfHwgZmFp
bD0xCitjb21wYXJlIGV4cC0yIHgwMSB8fCBmYWlsPTEKK2NvbXBhcmUgZXhwLTMgeDAyIHx8IGZh
aWw9MQorCisjIENoZWNrIC0tbnVtZXJpYy1zdWZmaXhlcz1YCitzcGxpdCAtLW51bWVyaWMtc3Vm
Zml4ZXM9MSAtLWxpbmVzPTIgaW4gfHwgZmFpbD0xCitjYXQgPDxcRU9GID4gZXhwLTEKKzEKKzIK
K0VPRgorY2F0IDw8XEVPRiA+IGV4cC0yCiszCis0CitFT0YKK2NhdCA8PFxFT0YgPiBleHAtMwor
NQorRU9GCitjb21wYXJlIGV4cC0xIHgwMSB8fCBmYWlsPTEKK2NvbXBhcmUgZXhwLTIgeDAyIHx8
IGZhaWw9MQorY29tcGFyZSBleHAtMyB4MDMgfHwgZmFpbD0xCisKKyMgQ2hlY2sgdGhhdCBzcGxp
dCBmYWlsZWQgd2hlbiBzdWZmaXggbGVuZ3RoIGlzIG5vdCBsYXJnZSBlbm91Z2ggZm9yCisjIHRo
ZSBudW1lcmljYWwgc3VmZml4IHN0YXJ0IHZhbHVlCitzcGxpdCAtYSAzIC0tbnVtZXJpYy1zdWZm
aXhlcz0xMDAwIGluIDI+IC9kZXYvbnVsbCAmJiBmYWlsPTEKKworIyBjaGVjayBpbnZhbGlkIC0t
bnVtZXJpYy1zdWZmaXhlcyBzdGFydCB2YWx1ZXMgYXJlIGZsYWdnZWQKK3NwbGl0IC0tbnVtZXJp
Yy1zdWZmaXhlcz0tMSBpbiAyPiAvZGV2L251bGwgJiYgZmFpbD0xCitzcGxpdCAtLW51bWVyaWMt
c3VmZml4ZXM9b25lIGluIDI+IC9kZXYvbnVsbCAmJiBmYWlsPTEKKworRXhpdCAkZmFpbAotLSAK
MS43LjIuNQoK
--20cf3026697ab8d64004b92cacb6--


From MAILER-DAEMON Fri Feb 17 20:05:08 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RyYjc-00089e-8x
	for mharc-coreutils@gnu.org; Fri, 17 Feb 2012 20:05:08 -0500
Received: from eggs.gnu.org ([140.186.70.92]:51069)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1RyYjY-00086M-5c
	for coreutils@gnu.org; Fri, 17 Feb 2012 20:05:05 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1RyYjW-0007QT-5A
	for coreutils@gnu.org; Fri, 17 Feb 2012 20:05:04 -0500
Received: from moutng.kundenserver.de ([212.227.17.8]:57225)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1RyYjV-0007Ok-Qx
	for coreutils@gnu.org; Fri, 17 Feb 2012 20:05:02 -0500
Received: from [192.168.2.108] (p4FF7347E.dip.t-dialin.net [79.247.52.126])
	by mrelayeu.kundenserver.de (node=mrbap0) with ESMTP (Nemesis)
	id 0MP1RX-1Rslxw0g1n-0067lI; Sat, 18 Feb 2012 02:04:55 +0100
Message-ID: <4F3EF935.2070700@bernhard-voelker.de>
Date: Sat, 18 Feb 2012 02:04:53 +0100
From: Bernhard Voelker <mail@bernhard-voelker.de>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:10.0.1) Gecko/20120208 Thunderbird/10.0.1
MIME-Version: 1.0
To: Jim Meyering <jim@meyering.net>
Subject: Re: "ls -l": Avoid unnecessary getxattr() overhead
References: <87pqds3snt.fsf@Apollo.jerryland.fr>
	<4F30206B.1020802@itwm.fraunhofer.de>
	<CA+TATJHorgsuY7pxmpxxAeWUHRP5MfFfzQP05o=1H6C4ttswdA@mail.gmail.com>
	<4F3032BC.50702@draigBrady.com>
	<4F30631A.2070009@itwm.fraunhofer.de>
	<87y5scfdiq.fsf@rho.meyering.net> <4F33B5D7.3010909@draigBrady.com>
	<87mx8sf5w0.fsf@rho.meyering.net>
	<4F3437CE.4050308@itwm.fraunhofer.de>
	<4F346CC7.4030608@draigBrady.com> <87r4y3dry0.fsf@rho.meyering.net>
	<4F3710CA.8020402@itwm.fraunhofer.de>
	<8739aao9e2.fsf@rho.meyering.net>
	<4F3E12F9.6080102@bernhard-voelker.de>
	<8762f5lpct.fsf@rho.meyering.net> <4F3E5BBD.80809@redhat.com>
	<874nupk1cx.fsf@rho.meyering.net> <4F3E5E8E.2040802@redhat.com>
	<87lio1ij8b.fsf@rho.meyering.net>
	<4F3E7B1E.3090300@bernhard-voelker.de>
	<8739a9ig96.fsf@rho.meyering.net>
In-Reply-To: <8739a9ig96.fsf@rho.meyering.net>
Content-Type: multipart/mixed; boundary="------------020409090401000507020000"
X-Provags-ID: V02:K0:eB7Qqxh64iqXQFYD0oYOf/Bc4q8ykfmUasbxx3NDNby
	LSuVysqDsU+s7z1iapZbNoxLfppM2S6AQ8WlcUCNcr/S4yJ4yV
	Nj1GLAKZv4lhHkM3HmjlI0xKsBm1q03id62jR26YQpW+IgEN/n
	ZeCtoggJbAZUnnrWyV84qvHccGbl4s6gbpAjBG40WFF4O0VXhS
	/UbkZFk2xsf2XhBtZWOIO+hKbemcA1sHUNUjFXfnYtQMU3kr7U
	2/G6meSR+7t4hsDuxjc3av9PgYhOqs4MlrxahETjABJvcjEChP
	M4zfKXccQrzTU7UfVPPQGv59Nrl4N1xy7tlMhkyDuRrgs9WPc4
	jeMxhh+yejowIUKXF3Cve2vKU0/Xx6zgc89vNk+/4
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 212.227.17.8
Cc: "Daniel P. Berrange" <berrange@redhat.com>,
	Bernd Schubert <bernd.schubert@itwm.fraunhofer.de>, coreutils@gnu.org,
	Sven Breuner <sven.breuner@itwm.fraunhofer.de>,
	=?ISO-8859-1?Q?J=E9r=E9my_Compostella?= <jeremy.compostella@gmail.com>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sat, 18 Feb 2012 01:05:05 -0000

This is a multi-part message in MIME format.
--------------020409090401000507020000
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

On 02/17/2012 05:20 PM, Jim Meyering wrote:
> Good idea.
> And with that, we can also drop the eval "...".

Great.

> I have a slight preference for writing the file only from the destructor.
> WDYT?

You convinced me ... but unfortunately not my PC.

See the attached getxattr-speedup.log.gz - I added a "cat k.c" so that
you can check that I got your patch right.
It seems that GCC doesn't like our "__attribute__((destructor)) void p()".
It basically works in a standalone program, but not with LD_PRELOAD.
I didn't find much about this. Can this be a 64-bit issue?

I worked on another alternative: using atexit(). I had to add -lc
to pull atexit from glibc, of course. And I made p() and count_1()
static to avoid potential name clashes with original ls symbols.
Finally, I changed the comment "run the test, redirecting", as
we're obvisouly no longer redirecting. ;-)
What do you think?

Ah yes, and another aspect came to my mind: the test does not
cover the other side of the changes in ls.c: it maybe should
prove that ls tries to get the attributes again when the cache
functions start having success for another file, i.e. that we
didn't break the normal XATTR, CAP and ACL queries. Is this
covered by other tests?

Have a nice weekend,
Berny


>From 31c4976c5787222c48b7a27e7c7dfe983a1a40ad Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering@redhat.com>
Date: Sat, 18 Feb 2012 01:30:26 +0100
Subject: [PATCH] tests: test for ls speed-up

* tests/ls/getxattr-speedup: New test.
* tests/Makefile.am (TESTS): Add it.
Improved-by: Bernhard Voelker.
---
 tests/Makefile.am         |    1 +
 tests/ls/getxattr-speedup |   64 +++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 65 insertions(+), 0 deletions(-)
 create mode 100755 tests/ls/getxattr-speedup

diff --git a/tests/Makefile.am b/tests/Makefile.am
index 7b53681..625b636 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -426,6 +426,7 @@ TESTS =                                             \
   ls/dired                                     \
   ls/file-type                                 \
   ls/follow-slink                              \
+  ls/getxattr-speedup                          \
   ls/infloop                                   \
   ls/inode                                     \
   ls/m-option                                  \
diff --git a/tests/ls/getxattr-speedup b/tests/ls/getxattr-speedup
new file mode 100755
index 0000000..8166af1
--- /dev/null
+++ b/tests/ls/getxattr-speedup
@@ -0,0 +1,64 @@
+#!/bin/sh
+# Show that we've eliminated most of ls' failing getxattr syscalls,
+# regardless of how many files are in a directory we list.
+# This test is skipped on systems that lack LD_PRELOAD support; that's fine.
+
+# Copyright (C) 2012 Free Software Foundation, Inc.
+
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+. "${srcdir=.}/init.sh"; path_prepend_ ../src
+print_ver_ ls
+
+# Replace each getxattr and lgetxattr call with a call to these stubs.
+# Count those and write the total number of calls to the file "x"
+# after ls has terminated via a function registered with atexit.
+cat > k.c <<'EOF' || framework_failure_
+#include <errno.h>
+#include <stdio.h>
+#include <stdlib.h>
+static unsigned long int n_calls;
+static void p()
+{
+  FILE *fp = fopen ("x", "w"); if (!fp) return;
+  fprintf (fp, "%lu\n", n_calls);
+  fclose (fp);
+}
+static ssize_t count_1 (void)
+{
+  if (!n_calls)
+    atexit (p);
+  ++n_calls; errno = ENOTSUP; return -1;
+}
+ssize_t getxattr (const char *path, const char *name, void *value, size_t size)
+{ return count_1 (); }
+ssize_t lgetxattr(const char *path, const char *name, void *value, size_t size)
+{ return count_1 (); }
+EOF
+
+# Then compile/link it:
+$CC -fPIC -O2 -c k.c || framework_failure_ 'failed to compile with -fPIC'
+ld -G k.o -lc -o k.so || framework_failure_ 'failed to invoke ld -G ...'
+
+# create a few files
+seq 100 | xargs touch || framework_failure_
+
+# Finally, run the test, with the stubs and counting from above.
+LD_PRELOAD=./k.so ls --color=always -l . || fail=1
+
+# Ensure that there were no more than 3 *getxattr calls.
+n_calls=$(cat x)
+test "$n_calls" -le 3 || fail=1
+
+Exit $fail
-- 
1.7.7

--------------020409090401000507020000
Content-Type: application/x-gzip;
 name="getxattr-speedup.log.gz"
Content-Transfer-Encoding: base64
Content-Disposition: attachment;
 filename="getxattr-speedup.log.gz"

H4sIAMbePk8AA72ZbW/bNhCAP09/YV+uXQfbcSRbL35NXSBonC5YmgROimZYAUGRKJuoLGkU
lTgd+t93pGzXabPYtwAzHIc8He+O9/BFII8PT06HkBStKZOLQEphFjljUZlDnS24HILdMEbb
P4bRbAJPueRB4od3kT9qzbI5a90wkd63ply2wkywUnL0JFkhC6UfBzwZtbHUBCWLuPBzwWK+
8LUsFzyVMUyl0s3jhT/SxSbMP0s2zyMftvjApub33bKu8aOthEHB4NUvGLWuRssIdghchTvP
k0AyFdITLt4fXvtXk5PxpT/yNnyuXK1cV9K1zaW4CWVaMAlX7y+OTiaVpOo5mBGYEsz8eQmI
tnW19Vhz99Qbb/Zl1QnVQgX2HJtNHIhoY5o904oUcAlmNYiYmBejWiTuFqb+gANee9CFY3YD
dh/a7aHt/Gd3tY1MaE8PstGGEVTDm4Wz5/UJBJOlULa1aX+XkfqUQQifRQqmia9mu1/M/DTx
RzVDZQJOji9HNfipqsSZgIJP1XAGG7Pugo3fTpWPRS5Q2ATb6StddhskUJMiyKH2qVYb48qD
jwaqDHaV5Orh+gnYO/lwtvhw29qH86MPfALOTj7cbT5s7cN9xIeN4t1ytc2JVzmxH/GCj1C+
m5vONjdu5abziBu3VtmqpILNs1vmy3nuq0kAeSBnannPWYort2W1ChEay4liQ+0FDpy1mh7d
a5Xl9Fo9qKbYLgN4ZWHr/Kv8BDeF/839Nu0qqgdtdGRwcXj129Oth7s/veHpsFUWopVkYZB8
q+rCZq11bdtVBQuT7rdn02DOimGrWAtUScUvmNps2ILhJObz4qnddLPH4+vx+Pqq6mq1KGlu
aCOWxnrhW/PEYZUJqZNiLLd0/5ZhrpJipX3PClwo8RcFupvAjWqscjVQa6/2aspQeqt3BxNb
FzxLDazU3519gHWYDehbdsdy+6bttVlovM3ye8GnMwn1tw1w2rjOHwvG4DKL5V0gGBxnZRoF
Eo3tw0kaWsYpDxnuufDu4vTWbQ5BmccyLF3ifMGg1B4t4PVMynzYak3T0srEtJVUTXHVzBNr
JufJG8u4mnHcEAqIlddi6XUI91kJyr0WywzCWZBOGQRphAmNeCEFvyklAy6VCYaaaOPsHD4e
TiaHZ1d/7KtGcsYwt5KlUm9xXEoWwc09RndnGcZHoQSpEkw42hcRvLfgUgZJMg9S7eoouOUo
DcLfWfqFM0sDltUvfLZC4xeehkkZMXjNhEgza/ZmQ6ReYLSokJjAUL2q8GmKISRZOkVqElIf
SSbFgVEU/AuuA7DaS6AeZiliV2HBnpo8+7ApSXHA7sNthtHt4RJUYmVpQf1rGH9Ds7kyDjo0
HD3js/Oryw8XB6shadoH8HXtOln5/v9c+77yp0H6fr2uXvlEGcpMNBqVg7yO9gyA45PTMezF
OVqKM1wcof5y8XIfXt69bBwAj6H+Is4bS9sHqB8vX4vrcY5avyblpxTVl1E1tEaYZDiIUQGr
X5HoNAzBRGIjHKyDAZjxxclbMM8dMENNGqcdvlC+w3IGZob/ikzNaPYX2G01hxeBmBY46Mpw
hrXTI/9iMj49PzwaWa2lrp6YYZZkYhQkd8E9VhOwDJnhiAO7a5jizhSm+uJar1cYKAucVICf
9sNXMZuk3KZp09RpoTgkbZek7ZG0OyRtGp0eSbtP0h5QtEnpdkjgHRJ4hxYJCbxDAu+QwDsk
8A4JvEMC75DAkxLoksC7JPAuCbxLi5sE3iWBd0ngXRJ4lwTeJYEnpcQjgfdI4D0SeI8E3qP1
kgTeI4H3SOA9EniPBJ7UyQ4JfIcEvkMC3yGB75DAd2g5IYHvkMB3SOA7JPCksLsk8F0S+C4J
fJcEvksC3yWB79IySALfJYHvksCTAumRwPdI4Hsk8D0S+B4JfI8EvkcC36PlmwS+RwJPMt0n
ge+TwPdJ4Psk8H0S+D4JfJ8Evk8C36fRIYGnKZPAD0jgByTwAxL4AQn8gAR+QAI/IIEfkMAP
nmLpdb67elMHUP+q7vTcH9Qzpb4Qpvp+p971evZD9Z//bM/xzz5wnbk6q1J1fXsQSFgY+DuE
xRDOMijKcAYxT5g64o24YOqs7t5ork7WRqvT6loNTNRyDav1yK35EBKeMnzhGmrtIQzVYSib
MqEvWFihz5GxiPZZZCzvwG0sVJco+txNQpPpo/OlZF3YuFPBmu8XUjcNExakpZYNjV1uRpTS
bJ5FYE6gbGI2n3EXCGIOpoifY2HZv38AA/aSsosgAAA=
--------------020409090401000507020000--


From MAILER-DAEMON Fri Feb 17 21:17:31 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RyZrf-0005GA-VJ
	for mharc-coreutils@gnu.org; Fri, 17 Feb 2012 21:17:31 -0500
Received: from eggs.gnu.org ([140.186.70.92]:60485)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RyZrb-0005Cb-M9
	for coreutils@gnu.org; Fri, 17 Feb 2012 21:17:28 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RyZra-0000p0-Sr
	for coreutils@gnu.org; Fri, 17 Feb 2012 21:17:27 -0500
Received: from mail3.vodafone.ie ([213.233.128.45]:49254)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RyZra-0000mj-Lp
	for coreutils@gnu.org; Fri, 17 Feb 2012 21:17:26 -0500
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: ApMBAFIJP09tTdvu/2dsb2JhbAAMOK9BhV0BAQEEDCYBRhALDQsJJQ8CRgYNAQcBAcBwjC8JAwIDBwQEAgIDCQENAQIBAocxBJs2jGo
Received: from unknown (HELO [192.168.1.79]) ([109.77.219.238])
	by mail3.vodafone.ie with ESMTP; 18 Feb 2012 02:17:23 +0000
Message-ID: <4F3F0A33.40602@draigBrady.com>
Date: Sat, 18 Feb 2012 02:17:23 +0000
From: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:6.0) Gecko/20110816 Thunderbird/6.0
MIME-Version: 1.0
To: =?ISO-8859-1?Q?J=E9r=E9my_Compostella?=
 <jeremy.compostella@gmail.com>
Subject: Re: bug#9085: 'split' feature request: an option to uses e.g. '.001'
	as first suffix.
References: <87vcnk3vxc.fsf@Apollo.jerryland.fr>
	<CA+TATJFys_1jcxP=UuxykbTNA_ntY3CMhNp4WWHAqbWjbPhaVw@mail.gmail.com>
	<4F3DB7B9.6020201@draigBrady.com>
	<CA+TATJGTaP0rOOhWqdwYM8CgTqe8ZdKW7ucMzNFDBvxokz2wqQ@mail.gmail.com>
In-Reply-To: <CA+TATJGTaP0rOOhWqdwYM8CgTqe8ZdKW7ucMzNFDBvxokz2wqQ@mail.gmail.com>
X-Enigmail-Version: 1.3.2
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 213.233.128.45
Cc: 9085-done@debbugs.gnu.org, coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sat, 18 Feb 2012 02:17:28 -0000

On 02/17/2012 05:56 PM, Jérémy Compostella wrote:
> I attached the new patch. I tried to make it as simple as possible but
> feel free to comment.

Thanks!
I just grouped the operations in next_file_name() a little for clarity,
tweaked a couple of strings, and pushed.

cheers,
Pádraig.


From MAILER-DAEMON Sat Feb 18 07:53:02 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1Ryjmg-0005fA-9P
	for mharc-coreutils@gnu.org; Sat, 18 Feb 2012 07:53:02 -0500
Received: from eggs.gnu.org ([140.186.70.92]:38992)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1Ryjmd-0005f1-4d
	for coreutils@gnu.org; Sat, 18 Feb 2012 07:53:00 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1Ryjmc-0002C6-3t
	for coreutils@gnu.org; Sat, 18 Feb 2012 07:52:59 -0500
Received: from mail-iy0-f169.google.com ([209.85.210.169]:44010)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1Ryjmb-0002Bs-W1
	for coreutils@gnu.org; Sat, 18 Feb 2012 07:52:58 -0500
Received: by iagz16 with SMTP id z16so7059208iag.0
	for <coreutils@gnu.org>; Sat, 18 Feb 2012 04:52:56 -0800 (PST)
Received-SPF: pass (google.com: domain of jeremy.compostella@gmail.com
	designates 10.50.153.198 as permitted sender)
	client-ip=10.50.153.198; 
Authentication-Results: mr.google.com; spf=pass (google.com: domain of
	jeremy.compostella@gmail.com designates 10.50.153.198 as
	permitted sender) smtp.mail=jeremy.compostella@gmail.com;
	dkim=pass header.i=jeremy.compostella@gmail.com
Received: from mr.google.com ([10.50.153.198])
	by 10.50.153.198 with SMTP id vi6mr2524861igb.30.1329569576624
	(num_hops = 1); Sat, 18 Feb 2012 04:52:56 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma;
	h=mime-version:in-reply-to:references:from:date:message-id:subject:to
	:cc:content-type;
	bh=ZWefN1M6RavwGq/ffKaBoFUuhWLFLqGzg79QzyVPPlY=;
	b=BPn2vDPfcLHETtQ5XScl3AkEwGrvCszXwcBZR1tNfPVmoendBgBTRRJI5ExMklwM0c
	VYFpJxqAFIjlkj5DUrfC0ls191kqYB0nV8AJE9c73mnLzsYlDYV1TbTqpOoxmy4uqKWj
	uhUCoC4Ttr/hJrIsoGzTkHwmj1kNPlwqvIRi0=
Received: by 10.50.153.198 with SMTP id vi6mr2041883igb.30.1329569576584; Sat,
	18 Feb 2012 04:52:56 -0800 (PST)
MIME-Version: 1.0
Received: by 10.42.29.196 with HTTP; Sat, 18 Feb 2012 04:52:35 -0800 (PST)
In-Reply-To: <87sjiyd79n.fsf@Apollo.jerryland.fr>
References: <87sjiyd79n.fsf@Apollo.jerryland.fr>
From: =?ISO-8859-1?Q?J=E9r=E9my_Compostella?= <jeremy.compostella@gmail.com>
Date: Sat, 18 Feb 2012 13:52:35 +0100
Message-ID: <CA+TATJFssEZWcfiVBUsKEeazhH8rOoV_cA-kDBucWf73xq-i-A@mail.gmail.com>
Subject: Re: bug#6554: [PATCH] split: Additional suffix for split (bug#6554)
To: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigbrady.com>
Content-Type: multipart/alternative; boundary=e89a8f22c68151050904b93c8ac7
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.85.210.169
Cc: 6554@debbugs.gnu.org, coreutils@gnu.org, Peng Yu <pengyu.ut@gmail.com>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sat, 18 Feb 2012 12:53:00 -0000

--e89a8f22c68151050904b93c8ac7
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

All,

I'm continuing with this feature implementation. Getting advantage on my
newly acquired experience on this project I made some changes in my
implementation.

Now I have a concern about the option name "--suffix". I think the
--suffix-length, --numeric-suffixes and the new --suffix are confusing
for the user since the two first are related to the count suffix and the
last one is an additional suffix appended to the whole output file
name. I think it would be clearer to replace "--suffix" with something
like "--additional-suffix". What do you think ?

Cheers,

J=E9r=E9my

--e89a8f22c68151050904b93c8ac7
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div>All,</div><div><br></div><div>I&#39;m continuing with this feature imp=
lementation. Getting advantage on my</div><div>newly acquired experience on=
 this project I made some changes in my</div><div>implementation.</div><div=
>

<br></div><div>Now I have a concern about the option name &quot;--suffix&qu=
ot;. I think the</div><div>--suffix-length, --numeric-suffixes and the new =
--suffix are confusing</div><div>for the user since the two first are relat=
ed to the count suffix and the</div>

<div>last one is an additional suffix appended to the whole output file</di=
v><div>name. I think it would be clearer to replace &quot;--suffix&quot; wi=
th something</div><div>like &quot;--additional-suffix&quot;. What do you th=
ink ?</div>

<div><br></div><div>Cheers,</div><div><br></div><div>J=E9r=E9my</div>

--e89a8f22c68151050904b93c8ac7--


From MAILER-DAEMON Sat Feb 18 08:06:56 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1Ryk08-0008FV-Fz
	for mharc-coreutils@gnu.org; Sat, 18 Feb 2012 08:06:56 -0500
Received: from eggs.gnu.org ([140.186.70.92]:36334)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Ryk05-0008Cl-QQ
	for coreutils@gnu.org; Sat, 18 Feb 2012 08:06:55 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Ryk03-0005p7-ED
	for coreutils@gnu.org; Sat, 18 Feb 2012 08:06:53 -0500
Received: from mx.meyering.net ([88.168.87.75]:42517)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Ryk03-0005p1-1G
	for coreutils@gnu.org; Sat, 18 Feb 2012 08:06:51 -0500
Received: from rho.meyering.net (localhost.localdomain [127.0.0.1])
	by rho.meyering.net (Acme Bit-Twister) with ESMTP id EFEE460061;
	Sat, 18 Feb 2012 14:06:48 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: Bernhard Voelker <mail@bernhard-voelker.de>
Subject: Re: "ls -l": Avoid unnecessary getxattr() overhead
In-Reply-To: <4F3EF935.2070700@bernhard-voelker.de> (Bernhard Voelker's
	message of "Sat, 18 Feb 2012 02:04:53 +0100")
References: <87pqds3snt.fsf@Apollo.jerryland.fr>
	<CA+TATJHorgsuY7pxmpxxAeWUHRP5MfFfzQP05o=1H6C4ttswdA@mail.gmail.com>
	<4F3032BC.50702@draigBrady.com> <4F30631A.2070009@itwm.fraunhofer.de>
	<87y5scfdiq.fsf@rho.meyering.net> <4F33B5D7.3010909@draigBrady.com>
	<87mx8sf5w0.fsf@rho.meyering.net>
	<4F3437CE.4050308@itwm.fraunhofer.de>
	<4F346CC7.4030608@draigBrady.com> <87r4y3dry0.fsf@rho.meyering.net>
	<4F3710CA.8020402@itwm.fraunhofer.de>
	<8739aao9e2.fsf@rho.meyering.net>
	<4F3E12F9.6080102@bernhard-voelker.de>
	<8762f5lpct.fsf@rho.meyering.net> <4F3E5BBD.80809@redhat.com>
	<874nupk1cx.fsf@rho.meyering.net> <4F3E5E8E.2040802@redhat.com>
	<87lio1ij8b.fsf@rho.meyering.net>
	<4F3E7B1E.3090300@bernhard-voelker.de>
	<8739a9ig96.fsf@rho.meyering.net>
	<4F3EF935.2070700@bernhard-voelker.de>
Date: Sat, 18 Feb 2012 14:06:48 +0100
Message-ID: <878vk0gujb.fsf@rho.meyering.net>
Lines: 157
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 88.168.87.75
Cc: "Daniel P. Berrange" <berrange@redhat.com>,
	Bernd Schubert <bernd.schubert@itwm.fraunhofer.de>, coreutils@gnu.org,
	Sven Breuner <sven.breuner@itwm.fraunhofer.de>,
	=?iso-8859-1?Q?J=E9r=E9my?= Compostella <jeremy.compostella@gmail.com>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sat, 18 Feb 2012 13:06:55 -0000

Bernhard Voelker wrote:
> On 02/17/2012 05:20 PM, Jim Meyering wrote:
>> Good idea.
>> And with that, we can also drop the eval "...".
>
> Great.
>
>> I have a slight preference for writing the file only from the destructor.
>> WDYT?
>
> You convinced me ... but unfortunately not my PC.
>
> See the attached getxattr-speedup.log.gz - I added a "cat k.c" so that
> you can check that I got your patch right.
> It seems that GCC doesn't like our "__attribute__((destructor)) void p()".
> It basically works in a standalone program, but not with LD_PRELOAD.
> I didn't find much about this. Can this be a 64-bit issue?

Is that an open-suse system?
I tried on an opensuse system with 2.6.37.6-0.11-desktop,
and since there were no getxattr calls at all, no wrapper
would run, and hence, your atexit call would never be reached.
I've added this comment at the top:

  # This test is skipped on systems that lack LD_PRELOAD support; that's fine.
+ # Similarly, on a system that lacks getxattr altogether, skipping it is fine.

and this new skip_ use below:

test -f x || skip_ "internal test failure: maybe LD_PRELOAD doesn't work?"

Which may mean this test will simply be skipped on your system.
But that is fine.

> I worked on another alternative: using atexit(). I had to add -lc
> to pull atexit from glibc, of course. And I made p() and count_1()
> static to avoid potential name clashes with original ls symbols.

See above.

> Finally, I changed the comment "run the test, redirecting", as
> we're obvisouly no longer redirecting. ;-)
> What do you think?

Good idea.  I've updated that comment.

> Ah yes, and another aspect came to my mind: the test does not
> cover the other side of the changes in ls.c: it maybe should
> prove that ls tries to get the attributes again when the cache
> functions start having success for another file, i.e. that we
> didn't break the normal XATTR, CAP and ACL queries. Is this
> covered by other tests?

No, but the dev-caching logic is very simple, so probably not worth
a test for now.

I expect to push this series shortly.


>From b29db6767612cf70e9d4c7eb6ede9822f6173fca Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering@redhat.com>
Date: Mon, 13 Feb 2012 12:52:19 +0100
Subject: [PATCH] tests: test for ls speed-up

* tests/ls/getxattr-speedup: New test.
* tests/Makefile.am (TESTS): Add it.
Improved-by: Bernhard Voelker
---
 tests/Makefile.am         |    1 +
 tests/ls/getxattr-speedup |   64 +++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 65 insertions(+)
 create mode 100755 tests/ls/getxattr-speedup

diff --git a/tests/Makefile.am b/tests/Makefile.am
index eed6ed6..2fee97d 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -427,6 +427,7 @@ TESTS =						\
   ls/dired					\
   ls/file-type					\
   ls/follow-slink				\
+  ls/getxattr-speedup				\
   ls/infloop					\
   ls/inode					\
   ls/m-option					\
diff --git a/tests/ls/getxattr-speedup b/tests/ls/getxattr-speedup
new file mode 100755
index 0000000..2b7a1f3
--- /dev/null
+++ b/tests/ls/getxattr-speedup
@@ -0,0 +1,64 @@
+#!/bin/sh
+# Show that we've eliminated most of ls' failing getxattr syscalls,
+# regardless of how many files are in a directory we list.
+# This test is skipped on systems that lack LD_PRELOAD support; that's fine.
+# Similarly, on a system that lacks getxattr altogether, skipping it is fine.
+
+# Copyright (C) 2012 Free Software Foundation, Inc.
+
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+. "${srcdir=.}/init.sh"; path_prepend_ ../src
+print_ver_ ls
+
+# Replace each getxattr and lgetxattr call with a call to these stubs.
+# Count those and write the total number of calls to the file "x"
+# via a global destructor.
+cat > k.c <<'EOF' || framework_failure_
+#include <errno.h>
+#include <stdio.h>
+
+static unsigned long int n_calls;
+
+static void __attribute__ ((destructor))
+print_call_count (void)
+{
+  FILE *fp = fopen ("x", "w"); if (!fp) return;
+  fprintf (fp, "%lu\n", n_calls); fclose (fp);
+}
+
+static ssize_t incr () { ++n_calls; errno = ENOTSUP; return -1; }
+ssize_t getxattr (const char *path, const char *name, void *value, size_t size)
+{ return incr (); }
+ssize_t lgetxattr(const char *path, const char *name, void *value, size_t size)
+{ return incr (); }
+EOF
+
+# Then compile/link it:
+$CC -fPIC -O2 -c k.c || framework_failure_ 'failed to compile with -fPIC'
+ld -G k.o -lc -o k.so || framework_failure_ 'failed to invoke ld -G ...'
+
+# Create a few files:
+seq 20 | xargs touch || framework_failure_
+
+# Finally, run the test:
+LD_PRELOAD=./k.so ls --color=always -l . || fail=1
+
+test -f x || skip_ "internal test failure: maybe LD_PRELOAD doesn't work?"
+
+# Ensure that there were no more than 3 *getxattr calls.
+n_calls=$(cat x)
+test "$n_calls" -le 3 || fail=1
+
+Exit $fail
--
1.7.9.1.245.gbbe07


From MAILER-DAEMON Sat Feb 18 09:24:52 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RylDY-0006w5-N4
	for mharc-coreutils@gnu.org; Sat, 18 Feb 2012 09:24:52 -0500
Received: from eggs.gnu.org ([140.186.70.92]:56342)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RylDW-0006vy-S4
	for coreutils@gnu.org; Sat, 18 Feb 2012 09:24:51 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RylDV-00062P-Iy
	for coreutils@gnu.org; Sat, 18 Feb 2012 09:24:50 -0500
Received: from mail3.vodafone.ie ([213.233.128.45]:12277)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RylDV-000629-2z
	for coreutils@gnu.org; Sat, 18 Feb 2012 09:24:49 -0500
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: ApMBAFG0P09tTdvu/2dsb2JhbAAMNq9KhV0BAQEEDCYBRhALDQsJFg8JAwIBAgFFBg0BBwEBvX+MKAMCAQ4JAwIDBwQEAgIDCQEQA4cxBJs2jG4
Received: from unknown (HELO [192.168.1.79]) ([109.77.219.238])
	by mail3.vodafone.ie with ESMTP; 18 Feb 2012 14:24:41 +0000
Message-ID: <4F3FB4A9.8080302@draigBrady.com>
Date: Sat, 18 Feb 2012 14:24:41 +0000
From: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:6.0) Gecko/20110816 Thunderbird/6.0
MIME-Version: 1.0
To: =?ISO-8859-1?Q?J=E9r=E9my_Compostella?=
 <jeremy.compostella@gmail.com>
Subject: Re: bug#6554: [PATCH] split: Additional suffix for split (bug#6554)
References: <87sjiyd79n.fsf@Apollo.jerryland.fr>
	<CA+TATJFssEZWcfiVBUsKEeazhH8rOoV_cA-kDBucWf73xq-i-A@mail.gmail.com>
In-Reply-To: <CA+TATJFssEZWcfiVBUsKEeazhH8rOoV_cA-kDBucWf73xq-i-A@mail.gmail.com>
X-Enigmail-Version: 1.3.2
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 213.233.128.45
Cc: 6554@debbugs.gnu.org, coreutils@gnu.org, Peng Yu <pengyu.ut@gmail.com>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sat, 18 Feb 2012 14:24:51 -0000

On 02/18/2012 12:52 PM, Jérémy Compostella wrote:
> All,
> 
> I'm continuing with this feature implementation. Getting advantage on my
> newly acquired experience on this project I made some changes in my
> implementation.
> 
> Now I have a concern about the option name "--suffix". I think the
> --suffix-length, --numeric-suffixes and the new --suffix are confusing
> for the user since the two first are related to the count suffix and the
> last one is an additional suffix appended to the whole output file
> name. I think it would be clearer to replace "--suffix" with something
> like "--additional-suffix". What do you think ?

That's a good point. --additional-suffix is better.

Note csplit has a more general --suffix-format option
(which can be shortened to --suffix).
So one can do: --suffix="%02d.txt".
However that's only possible, because csplit
supports only numeric suffixes.

However, this is starting to make me think
this marginal feature isn't worth it.
For the record, summarising the disadvantages so far:

 Backwards incompat with other systems (including older coreutils).
 Some complexity in understanding/describing various suffix options.
 Interoperability issue with possible future auto suffix len feature.

cheers,
Pádraig.


From MAILER-DAEMON Sat Feb 18 09:51:53 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1Ryldh-0003WO-Ua
	for mharc-coreutils@gnu.org; Sat, 18 Feb 2012 09:51:53 -0500
Received: from eggs.gnu.org ([140.186.70.92]:54812)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1Rylde-0003TU-9Z
	for coreutils@gnu.org; Sat, 18 Feb 2012 09:51:51 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1Ryldd-0002JV-4w
	for coreutils@gnu.org; Sat, 18 Feb 2012 09:51:50 -0500
Received: from mail-iy0-f169.google.com ([209.85.210.169]:60234)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1Ryldd-0002JR-0X
	for coreutils@gnu.org; Sat, 18 Feb 2012 09:51:49 -0500
Received: by iagz16 with SMTP id z16so7176540iag.0
	for <coreutils@gnu.org>; Sat, 18 Feb 2012 06:51:48 -0800 (PST)
Received-SPF: pass (google.com: domain of jeremy.compostella@gmail.com
	designates 10.50.89.196 as permitted sender)
	client-ip=10.50.89.196; 
Authentication-Results: mr.google.com; spf=pass (google.com: domain of
	jeremy.compostella@gmail.com designates 10.50.89.196 as
	permitted sender) smtp.mail=jeremy.compostella@gmail.com;
	dkim=pass header.i=jeremy.compostella@gmail.com
Received: from mr.google.com ([10.50.89.196])
	by 10.50.89.196 with SMTP id bq4mr3062907igb.26.1329576708302 (num_hops
	= 1); Sat, 18 Feb 2012 06:51:48 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma;
	h=mime-version:in-reply-to:references:from:date:message-id:subject:to
	:cc:content-type;
	bh=V8kNIn6/zpKFo03Ql4oYFj+PTWONRaPLYslQ1d5nTvQ=;
	b=EwgUh3pc/NDbBSbqbxAkZd6l0x2caaTmfS3wyiiayB2sv2pS85YOaJWC8QzooCtrOg
	GPaJLCyyIvAnLqPhBgDcA6cZRwTsyfZ/x+JDyrB6gRHLn+xCfKeFFloz2xG8eJ7JQCFO
	5kd1SDtYKuEyDqveH9dSN/rbCl3WCaOw/Rgdw=
Received: by 10.50.89.196 with SMTP id bq4mr2476328igb.26.1329576708264; Sat,
	18 Feb 2012 06:51:48 -0800 (PST)
MIME-Version: 1.0
Received: by 10.42.29.196 with HTTP; Sat, 18 Feb 2012 06:51:28 -0800 (PST)
In-Reply-To: <4F3FB4A9.8080302@draigBrady.com>
References: <87sjiyd79n.fsf@Apollo.jerryland.fr>
	<CA+TATJFssEZWcfiVBUsKEeazhH8rOoV_cA-kDBucWf73xq-i-A@mail.gmail.com>
	<4F3FB4A9.8080302@draigBrady.com>
From: =?ISO-8859-1?Q?J=E9r=E9my_Compostella?= <jeremy.compostella@gmail.com>
Date: Sat, 18 Feb 2012 15:51:28 +0100
Message-ID: <CA+TATJHRG7uhrma7JtPvc1Xy=fjkwf8kKyVNHd-Hgbod6Nn2nA@mail.gmail.com>
Subject: Re: bug#6554: [PATCH] split: Additional suffix for split (bug#6554)
To: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigbrady.com>
Content-Type: multipart/alternative; boundary=e89a8f3ba68165d23004b93e333c
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.85.210.169
Cc: 6554@debbugs.gnu.org, coreutils@gnu.org, Peng Yu <pengyu.ut@gmail.com>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sat, 18 Feb 2012 14:51:51 -0000

--e89a8f3ba68165d23004b93e333c
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

> That's a good point. --additional-suffix is better.
>
> Note csplit has a more general --suffix-format option
> (which can be shortened to --suffix).
> So one can do: --suffix=3D"%02d.txt".
> However that's only possible, because csplit
> supports only numeric suffixes.
>
> However, this is starting to make me think
> this marginal feature isn't worth it.
> For the record, summarising the disadvantages so far:
>
>  Backwards incompat with other systems (including older coreutils).
>  Some complexity in understanding/describing various suffix options.
With the new "--additional-suffix" it looks clear to me.
>  Interoperability issue with possible future auto suffix len feature.
If I remember well the issue is related with the alphabetic file
order. This issue could be easily handled by the user.

IMHO, this feature is maybe marginal but it is very convenient to have
it. Anyway, it would be great to state on it as I need to know if I
should let it go or not. It's always hard to trash work but that's the
game ! :)

Cheers,

J=E9r=E9my

--e89a8f3ba68165d23004b93e333c
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div>&gt; That&#39;s a good point. --additional-suffix is better.</div><div=
>&gt;=A0</div><div>&gt; Note csplit has a more general --suffix-format opti=
on</div><div>&gt; (which can be shortened to --suffix).</div><div>&gt; So o=
ne can do: --suffix=3D&quot;%02d.txt&quot;.</div>

<div>&gt; However that&#39;s only possible, because csplit</div><div>&gt; s=
upports only numeric suffixes.</div><div>&gt;=A0</div><div>&gt; However, th=
is is starting to make me think</div><div>&gt; this marginal feature isn&#3=
9;t worth it.</div>

<div>&gt; For the record, summarising the disadvantages so far:</div><div>&=
gt;=A0</div><div>&gt; =A0Backwards incompat with other systems (including o=
lder coreutils).</div><div>&gt; =A0Some complexity in understanding/describ=
ing various suffix options.</div>

<div>With the new &quot;--additional-suffix&quot; it looks clear to me.</di=
v><div>&gt; =A0Interoperability issue with possible future auto suffix len =
feature.</div><div>If I remember well the issue is related with the alphabe=
tic file</div>

<div>order. This issue could be easily handled by the user.</div><div><br><=
/div><div>IMHO, this feature is maybe marginal but it is very convenient to=
 have</div><div>it. Anyway, it would be great to state on it as I need to k=
now if I</div>

<div>should let it go or not. It&#39;s always hard to trash work but that&#=
39;s the</div><div>game ! :)</div><div><br></div><div>Cheers,</div><div><br=
></div><div>J=E9r=E9my</div>

--e89a8f3ba68165d23004b93e333c--


From MAILER-DAEMON Sat Feb 18 09:57:37 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RyljF-0005fC-6O
	for mharc-coreutils@gnu.org; Sat, 18 Feb 2012 09:57:37 -0500
Received: from eggs.gnu.org ([140.186.70.92]:34151)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RyljD-0005dy-0a
	for coreutils@gnu.org; Sat, 18 Feb 2012 09:57:35 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RyljC-0003rk-0X
	for coreutils@gnu.org; Sat, 18 Feb 2012 09:57:34 -0500
Received: from mail3.vodafone.ie ([213.233.128.45]:23256)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RyljB-0003ra-Qk
	for coreutils@gnu.org; Sat, 18 Feb 2012 09:57:33 -0500
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: ApMBAGS7P09tTdvu/2dsb2JhbAAMOK9EhVsBAQEEDCYBRhALDQsJFg8JAwIBAgFFBg0BBwEBwQiMChoJAQ4JAwIDBwQEAgIDCQEQA4cxBJs2jG4
Received: from unknown (HELO [192.168.1.79]) ([109.77.219.238])
	by mail3.vodafone.ie with ESMTP; 18 Feb 2012 14:57:31 +0000
Message-ID: <4F3FBC5B.3010202@draigBrady.com>
Date: Sat, 18 Feb 2012 14:57:31 +0000
From: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:6.0) Gecko/20110816 Thunderbird/6.0
MIME-Version: 1.0
To: =?ISO-8859-1?Q?J=E9r=E9my_Compostella?=
 <jeremy.compostella@gmail.com>
Subject: Re: bug#6554: [PATCH] split: Additional suffix for split (bug#6554)
References: <87sjiyd79n.fsf@Apollo.jerryland.fr>
	<CA+TATJFssEZWcfiVBUsKEeazhH8rOoV_cA-kDBucWf73xq-i-A@mail.gmail.com>
	<4F3FB4A9.8080302@draigBrady.com>
	<CA+TATJHRG7uhrma7JtPvc1Xy=fjkwf8kKyVNHd-Hgbod6Nn2nA@mail.gmail.com>
In-Reply-To: <CA+TATJHRG7uhrma7JtPvc1Xy=fjkwf8kKyVNHd-Hgbod6Nn2nA@mail.gmail.com>
X-Enigmail-Version: 1.3.2
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 213.233.128.45
Cc: 6554@debbugs.gnu.org, coreutils@gnu.org, Peng Yu <pengyu.ut@gmail.com>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sat, 18 Feb 2012 14:57:36 -0000

On 02/18/2012 02:51 PM, Jérémy Compostella wrote:
>> That's a good point. --additional-suffix is better.
>> 
>> Note csplit has a more general --suffix-format option
>> (which can be shortened to --suffix).
>> So one can do: --suffix="%02d.txt".
>> However that's only possible, because csplit
>> supports only numeric suffixes.
>> 
>> However, this is starting to make me think
>> this marginal feature isn't worth it.
>> For the record, summarising the disadvantages so far:
>> 
>>  Backwards incompat with other systems (including older coreutils).
>>  Some complexity in understanding/describing various suffix options.
> With the new "--additional-suffix" it looks clear to me.
>>  Interoperability issue with possible future auto suffix len feature.
> If I remember well the issue is related with the alphabetic file
> order. This issue could be easily handled by the user.
> 
> IMHO, this feature is maybe marginal but it is very convenient to have
> it. Anyway, it would be great to state on it as I need to know if I
> should let it go or not. It's always hard to trash work but that's the
> game ! :)

Sure, please go ahead since you've done most of the work,
and others have expressed a desire for this.
I'm just stating the disadvantages for the record,
and perhaps a prompt to others to voice any
reservations they may have.
We always have to be wary when adding new stuff.

cheers,
Pádraig.


From MAILER-DAEMON Sat Feb 18 14:03:18 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RypZ0-0002G8-51
	for mharc-coreutils@gnu.org; Sat, 18 Feb 2012 14:03:18 -0500
Received: from eggs.gnu.org ([140.186.70.92]:49681)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1RypYx-0002G2-T3
	for coreutils@gnu.org; Sat, 18 Feb 2012 14:03:16 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1RypYw-0002EL-QH
	for coreutils@gnu.org; Sat, 18 Feb 2012 14:03:15 -0500
Received: from mail-iy0-f169.google.com ([209.85.210.169]:44071)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1RypYw-0002EF-LJ
	for coreutils@gnu.org; Sat, 18 Feb 2012 14:03:14 -0500
Received: by iagz16 with SMTP id z16so7425554iag.0
	for <coreutils@gnu.org>; Sat, 18 Feb 2012 11:03:14 -0800 (PST)
Received-SPF: pass (google.com: domain of jeremy.compostella@gmail.com
	designates 10.42.235.137 as permitted sender)
	client-ip=10.42.235.137; 
Authentication-Results: mr.google.com; spf=pass (google.com: domain of
	jeremy.compostella@gmail.com designates 10.42.235.137 as
	permitted sender) smtp.mail=jeremy.compostella@gmail.com;
	dkim=pass header.i=jeremy.compostella@gmail.com
Received: from mr.google.com ([10.42.235.137])
	by 10.42.235.137 with SMTP id kg9mr14639554icb.47.1329591794198
	(num_hops = 1); Sat, 18 Feb 2012 11:03:14 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma;
	h=mime-version:from:date:message-id:subject:to:content-type;
	bh=jpxrzlOfTbv8c0AhSWvZvqP4tbT359To1GhA/hj3N4Y=;
	b=BmjDJOuzFBFMltaQd0Y4pRvYdQbTzeJDSSof60Od0q91MN/XXODKmqcyG0Ottpgzf7
	BUw/1/bgPu1ZkBaHzOwuIpr1XzyhMIZCjQerP2Oi3vnz5G3KJy8ZfFvglkP6dmof6Pm9
	HR2Zl+QuyKetAGc7bn/CSOhGo2GLngdRDlHv8=
Received: by 10.42.235.137 with SMTP id kg9mr11602197icb.47.1329591794146;
	Sat, 18 Feb 2012 11:03:14 -0800 (PST)
MIME-Version: 1.0
Received: by 10.42.29.196 with HTTP; Sat, 18 Feb 2012 11:02:53 -0800 (PST)
From: =?ISO-8859-1?Q?J=E9r=E9my_Compostella?= <jeremy.compostella@gmail.com>
Date: Sat, 18 Feb 2012 20:02:53 +0100
Message-ID: <CA+TATJF2n7FE0=1xsaHjkZD4VF28A-1rQOJ7wksi9OMW9N-raQ@mail.gmail.com>
Subject: [PATCH] doc: fix filenames in split --filter=command example
To: coreutils@gnu.org
Content-Type: multipart/mixed; boundary=20cf3026697a961e3f04b941b66b
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.85.210.169
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sat, 18 Feb 2012 19:03:16 -0000

--20cf3026697a961e3f04b941b66b
Content-Type: multipart/alternative; boundary=20cf3026697a961e3c04b941b669

--20cf3026697a961e3c04b941b669
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

All,

A very small fix in documentation.

Cheers,

J=E9r=E9my

--20cf3026697a961e3c04b941b669
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

All,<div><br></div><div>A very small fix in documentation.</div><div><br></=
div><div>Cheers,</div><div><br></div><div>J=E9r=E9my</div>

--20cf3026697a961e3c04b941b669--
--20cf3026697a961e3f04b941b66b
Content-Type: text/x-patch; charset=US-ASCII; 
	name="0001-doc-fix-filenames-in-split-filter-command-example.patch"
Content-Disposition: attachment; 
	filename="0001-doc-fix-filenames-in-split-filter-command-example.patch"
Content-Transfer-Encoding: base64
X-Attachment-Id: f_gyt0kl4a0

RnJvbSAzZmNlMDkwMjgzM2ZkYjBhNDNhYzNmZmYzZjBjZTM1ZWM5ZTU4Yjk4IE1vbiBTZXAgMTcg
MDA6MDA6MDAgMjAwMQpGcm9tOiA9P1VURi04P3E/Sj1DMz1BOXI9QzM9QTlteT0yMENvbXBvc3Rl
bGxhPz0gPGplcmVteS5jb21wb3N0ZWxsYUBnbWFpbC5jb20+CkRhdGU6IFNhdCwgMTggRmViIDIw
MTIgMTk6NTk6NTIgKzAxMDAKU3ViamVjdDogW1BBVENIXSBkb2M6IGZpeCBmaWxlbmFtZXMgaW4g
c3BsaXQgLS1maWx0ZXI9Y29tbWFuZCBleGFtcGxlCgoqIGRvYy9jb3JldXRpbHMudGV4aSAoc3Bs
aXQgaW52b2thdGlvbik6IE91dHB1dCBmaWxlbmFtZXMgc2hvdWxkIGJlCnByZWZpeGVkIHdpdGgg
YGJpZy0nIG5vdCBgYmlnLXgnLgotLS0KIGRvYy9jb3JldXRpbHMudGV4aSB8ICAgIDIgKy0KIDEg
ZmlsZXMgY2hhbmdlZCwgMSBpbnNlcnRpb25zKCspLCAxIGRlbGV0aW9ucygtKQoKZGlmZiAtLWdp
dCBhL2RvYy9jb3JldXRpbHMudGV4aSBiL2RvYy9jb3JldXRpbHMudGV4aQppbmRleCA5ZjM3OGU4
Li41NjI0NDQzIDEwMDY0NAotLS0gYS9kb2MvY29yZXV0aWxzLnRleGkKKysrIGIvZG9jL2NvcmV1
dGlscy50ZXhpCkBAIC0zMDQwLDcgKzMwNDAsNyBAQCB4eiAtZGMgQklHLnh6IHwgc3BsaXQgLWIy
MDBHIC0tZmlsdGVyPSd4eiA+ICRGSUxFLnh6JyAtIGJpZy0KIEBlbmQgZXhhbXBsZQogCiBBc3N1
bWluZyBhIDEwOjEgY29tcHJlc3Npb24gcmF0aW8sIHRoYXQgd291bGQgY3JlYXRlIGFib3V0IGZp
ZnR5IDIwR2lCIGZpbGVzCi13aXRoIG5hbWVzIEBmaWxle2JpZy14YWEueHp9LCBAZmlsZXtiaWct
eGFiLnh6fSwgQGZpbGV7YmlnLXhhYy54en0sIGV0Yy4KK3dpdGggbmFtZXMgQGZpbGV7YmlnLWFh
Lnh6fSwgQGZpbGV7YmlnLWFiLnh6fSwgQGZpbGV7YmlnLWFjLnh6fSwgZXRjLgogCiBAaXRlbSAt
biBAdmFye2NodW5rc30KIEBpdGVteCAtLW51bWJlcj1AdmFye2NodW5rc30KLS0gCjEuNy4yLjUK
Cg==
--20cf3026697a961e3f04b941b66b--


From MAILER-DAEMON Sat Feb 18 14:33:35 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1Ryq2J-0007RC-PA
	for mharc-coreutils@gnu.org; Sat, 18 Feb 2012 14:33:35 -0500
Received: from eggs.gnu.org ([140.186.70.92]:59247)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Ryq2H-0007R6-Hs
	for coreutils@gnu.org; Sat, 18 Feb 2012 14:33:34 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Ryq2G-00081x-Fy
	for coreutils@gnu.org; Sat, 18 Feb 2012 14:33:33 -0500
Received: from mail3.vodafone.ie ([213.233.128.45]:44009)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Ryq2G-00081t-AE
	for coreutils@gnu.org; Sat, 18 Feb 2012 14:33:32 -0500
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: ApMBAAr8P09tTdvu/2dsb2JhbAAMN69IhVsBAQEEDCYBRhALDQsJFg8JAwIBAgFFBg0BBwEBvWWMPAkDAgMHBAQCAgMJAQsBAQECAQKHMQSbNoxu
Received: from unknown (HELO [192.168.1.79]) ([109.77.219.238])
	by mail3.vodafone.ie with ESMTP; 18 Feb 2012 19:33:30 +0000
Message-ID: <4F3FFD09.7030706@draigBrady.com>
Date: Sat, 18 Feb 2012 19:33:29 +0000
From: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:6.0) Gecko/20110816 Thunderbird/6.0
MIME-Version: 1.0
To: =?ISO-8859-1?Q?J=E9r=E9my_Compostella?=
 <jeremy.compostella@gmail.com>
Subject: Re: [PATCH] doc: fix filenames in split --filter=command example
References: <CA+TATJF2n7FE0=1xsaHjkZD4VF28A-1rQOJ7wksi9OMW9N-raQ@mail.gmail.com>
In-Reply-To: <CA+TATJF2n7FE0=1xsaHjkZD4VF28A-1rQOJ7wksi9OMW9N-raQ@mail.gmail.com>
X-Enigmail-Version: 1.3.2
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 213.233.128.45
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sat, 18 Feb 2012 19:33:34 -0000

On 02/18/2012 07:02 PM, Jérémy Compostella wrote:
> All,
> 
> A very small fix in documentation.

Good catch. pushed.

cheers,
Pádraig.


From MAILER-DAEMON Sat Feb 18 22:17:50 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RyxHa-0007Wr-Oa
	for mharc-coreutils@gnu.org; Sat, 18 Feb 2012 22:17:50 -0500
Received: from eggs.gnu.org ([140.186.70.92]:44063)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <sven.breuner@itwm.fraunhofer.de>) id 1RyxHZ-0007Wj-2D
	for coreutils@gnu.org; Sat, 18 Feb 2012 22:17:49 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <sven.breuner@itwm.fraunhofer.de>) id 1RyxHY-0003J3-1A
	for coreutils@gnu.org; Sat, 18 Feb 2012 22:17:49 -0500
Received: from mailgw1.uni-kl.de ([131.246.120.220]:60416)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <sven.breuner@itwm.fraunhofer.de>) id 1RyxHX-0003Ii-Pe
	for coreutils@gnu.org; Sat, 18 Feb 2012 22:17:47 -0500
Received: from itwm2.itwm.fhg.de (itwm2.itwm.fhg.de [131.246.191.3])
	by mailgw1.uni-kl.de (8.14.3/8.14.3/Debian-5+lenny1) with ESMTP id
	q1J3HhYH031475
	(version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=NOT)
	for <coreutils@gnu.org>; Sun, 19 Feb 2012 04:17:43 +0100
Received: from mail2.itwm.fhg.de ([131.246.191.79]:37642)
	by itwm2.itwm.fhg.de with esmtps (TLSv1:DES-CBC3-SHA:168)
	(/C=DE/ST=Rheinland-Pfalz/L=Kaiserslautern/O=Fraunhofer
	ITWM/OU=SLG/CN=mail2.itwm.fhg.de)(verified=1) (Exim 4.74 #1)
	id 1RyxHQ-0005kz-MH; Sun, 19 Feb 2012 04:17:40 +0100
Message-ID: <4F4069D3.5000806@itwm.fraunhofer.de>
Date: Sun, 19 Feb 2012 04:17:39 +0100
From: Sven Breuner <sven.breuner@itwm.fraunhofer.de>
User-Agent: Mozilla/5.0 (X11; Linux i686 on x86_64;
	rv:10.0.2) Gecko/20120216 Thunderbird/10.0.2
MIME-Version: 1.0
To: Jim Meyering <jim@meyering.net>
Subject: Re: "ls -l": Avoid unnecessary getxattr() overhead
References: <87pqds3snt.fsf@Apollo.jerryland.fr>
	<4F30206B.1020802@itwm.fraunhofer.de>
	<CA+TATJHorgsuY7pxmpxxAeWUHRP5MfFfzQP05o=1H6C4ttswdA@mail.gmail.com>
	<4F3032BC.50702@draigBrady.com>
	<4F30631A.2070009@itwm.fraunhofer.de>
	<87y5scfdiq.fsf@rho.meyering.net> <4F33B5D7.3010909@draigBrady.com>
	<87mx8sf5w0.fsf@rho.meyering.net>
	<4F3437CE.4050308@itwm.fraunhofer.de>
	<4F346CC7.4030608@draigBrady.com> <87r4y3dry0.fsf@rho.meyering.net>
	<4F3710CA.8020402@itwm.fraunhofer.de>
	<8739aao9e2.fsf@rho.meyering.net>
	<4F3E12F9.6080102@bernhard-voelker.de>
	<8762f5lpct.fsf@rho.meyering.net>
	<4F3E35C9.7050707@bernhard-voelker.de>
	<87mx8hk6bm.fsf@rho.meyering.net>
In-Reply-To: <87mx8hk6bm.fsf@rho.meyering.net>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
X-ITWM-CharSet: ISO-8859-1
X-ITWM-Scanned-By: mail2.itwm.fhg.de
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
X-Received-From: 131.246.120.220
Cc: "Daniel P. Berrange" <berrange@redhat.com>,
	Bernhard Voelker <mail@bernhard-voelker.de>,
	Bernd Schubert <bernd.schubert@itwm.fraunhofer.de>, coreutils@gnu.org,
	=?ISO-8859-1?Q?J=E9r=E9my_Compostella?= <jeremy.compostella@gmail.com>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 19 Feb 2012 03:17:50 -0000

Jim Meyering wrote on 02/09/2012 03:17 PM:
 > Here's that simpler patch:
 > [...]

Jim Meyering wrote on 02/17/2012 01:11 PM:
> Here's the updates series, along with this addition to NEWS:
>[...]

Thanks for all this optimization work so far!

My "ls -l" test case with >60K files in a single directory is now down 
from about 50s to about 20s already. (When the problem below is solved, 
it should be down to about 10s).

Unfortunately, some getxattr selinux calls seem to be back with the 
current coreutils git version.
Those calls were gone after Jim's initial patch from 02/09/2012.

With only the patch from 02/09/2012 applied, strace showed this output 
for the first three files:

$ strace ~/tmp/ls_20120209 -l --color=always
[...]
lstat("file9", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
capget(0x20080522, 0, NULL)             = -1 EFAULT (Bad address)
getxattr("file9", "security.capability", 0x7fffffffda30, 20) = -1 
EOPNOTSUPP (Operation not supported)
lgetxattr("file9", "security.selinux", "unlabeled", 255) = 10
getxattr("file9", "system.posix_acl_access", 0x0, 0) = -1 EOPNOTSUPP 
(Operation not supported)

lstat("file2", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
capget(0x20080522, 0, NULL)             = -1 EFAULT (Bad address)
getxattr("file2", "security.capability", 0x7fffffffda30, 20) = -1 
EOPNOTSUPP (Operation not supported)
getxattr("file2", "system.posix_acl_access", 0x0, 0) = -1 EOPNOTSUPP 
(Operation not supported)

lstat("file7", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
capget(0x20080522, 0, NULL)             = -1 EFAULT (Bad address)
getxattr("file7", "security.capability", 0x7fffffffda30, 20) = -1 
EOPNOTSUPP (Operation not supported)
getxattr("file7", "system.posix_acl_access", 0x0, 0) = -1 EOPNOTSUPP 
(Operation not supported)
[...]

=> selinux.security getxattr call for first file, but not for following 
files


Now with the current git version, strace shows this output for the first 
three files:

$ strace ~/git/coreutils/src/ls -l --color=always
[...]
lstat("file9", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
capget(0x20080522, 0, NULL)             = -1 EFAULT (Bad address)
getxattr("file9", "security.capability", 0x7fffffffda40, 20) = -1 
EOPNOTSUPP (Operation not supported)
lgetxattr("file9", "security.selinux", "unlabeled", 255) = 10
getxattr("file9", "system.posix_acl_access", 0x0, 0) = -1 EOPNOTSUPP 
(Operation not supported)

lstat("file2", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
lgetxattr("file2", "security.selinux", "unlabeled", 255) = 10

lstat("file7", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
lgetxattr("file7", "security.selinux", "unlabeled", 255) = 10
[...]

=> selinux.security getxattr call for all files


Best regards,
Sven


From MAILER-DAEMON Sun Feb 19 03:57:33 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1Rz2aL-0004Aa-H4
	for mharc-coreutils@gnu.org; Sun, 19 Feb 2012 03:57:33 -0500
Received: from eggs.gnu.org ([140.186.70.92]:39438)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Rz2aH-00047x-NF
	for coreutils@gnu.org; Sun, 19 Feb 2012 03:57:31 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Rz2aG-0000Ik-Jg
	for coreutils@gnu.org; Sun, 19 Feb 2012 03:57:29 -0500
Received: from mx.meyering.net ([88.168.87.75]:45301)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Rz2aG-0000If-Dm
	for coreutils@gnu.org; Sun, 19 Feb 2012 03:57:28 -0500
Received: from rho.meyering.net (localhost.localdomain [127.0.0.1])
	by rho.meyering.net (Acme Bit-Twister) with ESMTP id 49E9960094;
	Sun, 19 Feb 2012 09:57:27 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: Sven Breuner <sven.breuner@itwm.fraunhofer.de>
Subject: Re: "ls -l": Avoid unnecessary getxattr() overhead
In-Reply-To: <4F4069D3.5000806@itwm.fraunhofer.de> (Sven Breuner's message of
	"Sun, 19 Feb 2012 04:17:39 +0100")
References: <87pqds3snt.fsf@Apollo.jerryland.fr>
	<4F30206B.1020802@itwm.fraunhofer.de>
	<CA+TATJHorgsuY7pxmpxxAeWUHRP5MfFfzQP05o=1H6C4ttswdA@mail.gmail.com>
	<4F3032BC.50702@draigBrady.com> <4F30631A.2070009@itwm.fraunhofer.de>
	<87y5scfdiq.fsf@rho.meyering.net> <4F33B5D7.3010909@draigBrady.com>
	<87mx8sf5w0.fsf@rho.meyering.net>
	<4F3437CE.4050308@itwm.fraunhofer.de>
	<4F346CC7.4030608@draigBrady.com> <87r4y3dry0.fsf@rho.meyering.net>
	<4F3710CA.8020402@itwm.fraunhofer.de>
	<8739aao9e2.fsf@rho.meyering.net>
	<4F3E12F9.6080102@bernhard-voelker.de>
	<8762f5lpct.fsf@rho.meyering.net>
	<4F3E35C9.7050707@bernhard-voelker.de>
	<87mx8hk6bm.fsf@rho.meyering.net>
	<4F4069D3.5000806@itwm.fraunhofer.de>
Date: Sun, 19 Feb 2012 09:57:27 +0100
Message-ID: <87pqdbcia0.fsf@rho.meyering.net>
Lines: 40
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 88.168.87.75
Cc: "Daniel P. Berrange" <berrange@redhat.com>,
	Bernhard Voelker <mail@bernhard-voelker.de>,
	Bernd Schubert <bernd.schubert@itwm.fraunhofer.de>, coreutils@gnu.org,
	=?iso-8859-1?Q?J=E9r=E9my?= Compostella <jeremy.compostella@gmail.com>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 19 Feb 2012 08:57:31 -0000

Sven Breuner wrote:

> Jim Meyering wrote on 02/09/2012 03:17 PM:
>> Here's that simpler patch:
>> [...]
>
> Jim Meyering wrote on 02/17/2012 01:11 PM:
>> Here's the updates series, along with this addition to NEWS:
>>[...]
>
> Thanks for all this optimization work so far!
>
> My "ls -l" test case with >60K files in a single directory is now down
> from about 50s to about 20s already. (When the problem below is
> solved, it should be down to about 10s).
>
> Unfortunately, some getxattr selinux calls seem to be back with the
> current coreutils git version.
> Those calls were gone after Jim's initial patch from 02/09/2012.
>
> With only the patch from 02/09/2012 applied, strace showed this output
> for the first three files:

The initial patch interpreted ENODATA as "not supported",
but that shouldn't matter, since we cached the failure only
when getfilecon returned -1.  Here, it's always returning 10
(sizeof "unlabeled") and hence is not a failure.

I think it is legitimate to cache that case (success with "unlabeled")
as well.  Presuming I can confirm that, I'll make that change, too.

> $ strace ~/tmp/ls_20120209 -l --color=always
> [...]
> lstat("file9", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
> capget(0x20080522, 0, NULL)             = -1 EFAULT (Bad address)
> getxattr("file9", "security.capability", 0x7fffffffda30, 20) = -1
> EOPNOTSUPP (Operation not supported)
> lgetxattr("file9", "security.selinux", "unlabeled", 255) = 10
> getxattr("file9", "system.posix_acl_access", 0x0, 0) = -1 EOPNOTSUPP
> (Operation not supported)


From MAILER-DAEMON Sun Feb 19 05:35:21 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1Rz46z-000535-GF
	for mharc-coreutils@gnu.org; Sun, 19 Feb 2012 05:35:21 -0500
Received: from eggs.gnu.org ([140.186.70.92]:46993)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Rz46x-00052y-OF
	for coreutils@gnu.org; Sun, 19 Feb 2012 05:35:20 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Rz46w-0008Me-LW
	for coreutils@gnu.org; Sun, 19 Feb 2012 05:35:19 -0500
Received: from mail3.vodafone.ie ([213.233.128.45]:14046)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1Rz46w-0008MT-FC
	for coreutils@gnu.org; Sun, 19 Feb 2012 05:35:18 -0500
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: Ar0BACjQQE9tTFkb/2dsb2JhbAAMN69NhVoBAQEDATIBRhALDQsJFg8JAwIBAgFFBg0BBwEBFodmtTqMLwkGAgMGAwcCCQEPAodYBJs2jG4
Received: from unknown (HELO [192.168.1.79]) ([109.76.89.27])
	by mail3.vodafone.ie with ESMTP; 19 Feb 2012 10:35:15 +0000
Message-ID: <4F40D061.3000206@draigBrady.com>
Date: Sun, 19 Feb 2012 10:35:13 +0000
From: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:6.0) Gecko/20110816 Thunderbird/6.0
MIME-Version: 1.0
To: Jim Meyering <jim@meyering.net>
Subject: Re: "ls -l": Avoid unnecessary getxattr() overhead
References: <87pqds3snt.fsf@Apollo.jerryland.fr>
	<4F30206B.1020802@itwm.fraunhofer.de>
	<CA+TATJHorgsuY7pxmpxxAeWUHRP5MfFfzQP05o=1H6C4ttswdA@mail.gmail.com>
	<4F3032BC.50702@draigBrady.com>
	<4F30631A.2070009@itwm.fraunhofer.de>
	<87y5scfdiq.fsf@rho.meyering.net> <4F33B5D7.3010909@draigBrady.com>
	<87mx8sf5w0.fsf@rho.meyering.net>
	<4F3437CE.4050308@itwm.fraunhofer.de>
	<4F346CC7.4030608@draigBrady.com> <87r4y3dry0.fsf@rho.meyering.net>
	<4F3710CA.8020402@itwm.fraunhofer.de>
	<8739aao9e2.fsf@rho.meyering.net>
	<4F3E12F9.6080102@bernhard-voelker.de>
	<8762f5lpct.fsf@rho.meyering.net>
	<4F3E35C9.7050707@bernhard-voelker.de>
	<87mx8hk6bm.fsf@rho.meyering.net>
	<4F4069D3.5000806@itwm.fraunhofer.de>
	<87pqdbcia0.fsf@rho.meyering.net>
In-Reply-To: <87pqdbcia0.fsf@rho.meyering.net>
X-Enigmail-Version: 1.3.2
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 213.233.128.45
Cc: "Daniel P. Berrange" <berrange@redhat.com>,
	Bernhard Voelker <mail@bernhard-voelker.de>,
	Bernd Schubert <bernd.schubert@itwm.fraunhofer.de>, coreutils@gnu.org,
	Sven Breuner <sven.breuner@itwm.fraunhofer.de>,
	=?ISO-8859-1?Q?J=E9r?= =?ISO-8859-1?Q?=E9my_Compostella?=
	<jeremy.compostella@gmail.com>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 19 Feb 2012 10:35:20 -0000

On 02/19/2012 08:57 AM, Jim Meyering wrote:
> Sven Breuner wrote:
> 
>> Jim Meyering wrote on 02/09/2012 03:17 PM:
>>> Here's that simpler patch:
>>> [...]
>>
>> Jim Meyering wrote on 02/17/2012 01:11 PM:
>>> Here's the updates series, along with this addition to NEWS:
>>> [...]
>>
>> Thanks for all this optimization work so far!
>>
>> My "ls -l" test case with >60K files in a single directory is now down
>> from about 50s to about 20s already. (When the problem below is
>> solved, it should be down to about 10s).
>>
>> Unfortunately, some getxattr selinux calls seem to be back with the
>> current coreutils git version.
>> Those calls were gone after Jim's initial patch from 02/09/2012.
>>
>> With only the patch from 02/09/2012 applied, strace showed this output
>> for the first three files:
> 
> The initial patch interpreted ENODATA as "not supported",
> but that shouldn't matter, since we cached the failure only
> when getfilecon returned -1.  Here, it's always returning 10
> (sizeof "unlabeled") and hence is not a failure.
> 
> I think it is legitimate to cache that case (success with "unlabeled")
> as well.  Presuming I can confirm that, I'll make that change, too.

Hmm I'm not sure.
Couldn't you get that for some files and not others on some systems?

Also I was a bit surprised to see EBUSY and ENOENT in errno_unsupported().
Can one assume there are no other attributes on a device if you get those?

>> $ strace ~/tmp/ls_20120209 -l --color=always
>> [...]
>> lstat("file9", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
>> capget(0x20080522, 0, NULL)             = -1 EFAULT (Bad address)
>> getxattr("file9", "security.capability", 0x7fffffffda30, 20) = -1
>> EOPNOTSUPP (Operation not supported)
>> lgetxattr("file9", "security.selinux", "unlabeled", 255) = 10

>From Sven's strace it seems that lgetfilecon() is interpreting
the "unlabeled" return from lgetxattr() and converting that
into a -1 return with ENODATA?  That's the only logical explanation
I can see for how ENODATA is significant. As mentioned before
I'm not sure we could add ENODATA to errno_unsupported().

>> getxattr("file9", "system.posix_acl_access", 0x0, 0) = -1 EOPNOTSUPP
>> (Operation not supported)

cheers,
Pádraig.


From MAILER-DAEMON Sun Feb 19 07:54:56 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1Rz6I4-0007D1-GO
	for mharc-coreutils@gnu.org; Sun, 19 Feb 2012 07:54:56 -0500
Received: from eggs.gnu.org ([140.186.70.92]:46558)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1Rz6I1-0007Cv-FG
	for coreutils@gnu.org; Sun, 19 Feb 2012 07:54:54 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1Rz6Hz-0006xX-Er
	for coreutils@gnu.org; Sun, 19 Feb 2012 07:54:53 -0500
Received: from mail-iy0-f169.google.com ([209.85.210.169]:63719)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1Rz6Hz-0006xQ-8y
	for coreutils@gnu.org; Sun, 19 Feb 2012 07:54:51 -0500
Received: by iagz16 with SMTP id z16so8132389iag.0
	for <coreutils@gnu.org>; Sun, 19 Feb 2012 04:54:50 -0800 (PST)
Received-SPF: pass (google.com: domain of jeremy.compostella@gmail.com
	designates 10.50.89.232 as permitted sender)
	client-ip=10.50.89.232; 
Authentication-Results: mr.google.com; spf=pass (google.com: domain of
	jeremy.compostella@gmail.com designates 10.50.89.232 as
	permitted sender) smtp.mail=jeremy.compostella@gmail.com;
	dkim=pass header.i=jeremy.compostella@gmail.com
Received: from mr.google.com ([10.50.89.232])
	by 10.50.89.232 with SMTP id br8mr7095987igb.30.1329656090290 (num_hops
	= 1); Sun, 19 Feb 2012 04:54:50 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma;
	h=mime-version:in-reply-to:references:from:date:message-id:subject:to
	:cc:content-type;
	bh=REcsc84gtJlIXlnvEEw4pdJc0MsI3rr3iM5QfRpUe2g=;
	b=lyV2rFN4A5cUE2CZ1vqJQp7MEE3U3sSbWgdNqjWcKSy8G0+40xozzqxujuqieOnd/O
	lKQ3VdsoEPRZnl2wYyE0FqUrhnMdx1BS4mswYEX0MSyLSQW24GHh0xmBYT7x9BQ4oQmI
	tv4tHd79+IJwxHrZwUPnTvGR1bYDw5WBFXnoU=
Received: by 10.50.89.232 with SMTP id br8mr5732998igb.30.1329656090259; Sun,
	19 Feb 2012 04:54:50 -0800 (PST)
MIME-Version: 1.0
Received: by 10.42.29.196 with HTTP; Sun, 19 Feb 2012 04:54:30 -0800 (PST)
In-Reply-To: <4F3FBC5B.3010202@draigBrady.com>
References: <87sjiyd79n.fsf@Apollo.jerryland.fr>
	<CA+TATJFssEZWcfiVBUsKEeazhH8rOoV_cA-kDBucWf73xq-i-A@mail.gmail.com>
	<4F3FB4A9.8080302@draigBrady.com>
	<CA+TATJHRG7uhrma7JtPvc1Xy=fjkwf8kKyVNHd-Hgbod6Nn2nA@mail.gmail.com>
	<4F3FBC5B.3010202@draigBrady.com>
From: =?ISO-8859-1?Q?J=E9r=E9my_Compostella?= <jeremy.compostella@gmail.com>
Date: Sun, 19 Feb 2012 13:54:30 +0100
Message-ID: <CA+TATJFyPFF+tLizCfGbhmEoRyoXodrH2sWDryqKKTMRVNhksw@mail.gmail.com>
Subject: Re: bug#6554: [PATCH] split: Additional suffix for split (bug#6554)
To: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigbrady.com>
Content-Type: multipart/mixed; boundary=e89a8f3ba429eef11004b950ae2c
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.85.210.169
Cc: 6554@debbugs.gnu.org, coreutils@gnu.org, Peng Yu <pengyu.ut@gmail.com>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 19 Feb 2012 12:54:54 -0000

--e89a8f3ba429eef11004b950ae2c
Content-Type: multipart/alternative; boundary=e89a8f3ba429eef10d04b950ae2a

--e89a8f3ba429eef10d04b950ae2a
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

P=E1draig,

I replaced --suffix with --additional-suffix and made some cleaning.

Feel free to comment the attached patch.

Cheers,

J=E9r=E9my

--e89a8f3ba429eef10d04b950ae2a
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div>P=E1draig,</div><div><br></div><div>I replaced --suffix with --additio=
nal-suffix and made some cleaning.</div><div><br></div><div>Feel free to co=
mment the attached patch.</div><div><br></div><div>Cheers,</div><div><br></=
div>

<div>J=E9r=E9my</div>

--e89a8f3ba429eef10d04b950ae2a--
--e89a8f3ba429eef11004b950ae2c
Content-Type: text/x-patch; charset=US-ASCII; 
	name="0001-split-additional-suffix-for-split-bug-6554.patch"
Content-Disposition: attachment; 
	filename="0001-split-additional-suffix-for-split-bug-6554.patch"
Content-Transfer-Encoding: base64
X-Attachment-Id: f_gyu2vroy0

RnJvbSBiZDgwYWM4NzZjNjRmYTBmYWM0MDliMjQzMTFkMzBiZDQzZDM0ZGJkIE1vbiBTZXAgMTcg
MDA6MDA6MDAgMjAwMQpGcm9tOiA9P1VURi04P3E/Sj1DMz1BOXI9QzM9QTlteT0yMENvbXBvc3Rl
bGxhPz0gPGplcmVteS5jb21wb3N0ZWxsYUBnbWFpbC5jb20+CkRhdGU6IFN1biwgMTkgRmViIDIw
MTIgMTM6NTI6NDcgKzAxMDAKU3ViamVjdDogW1BBVENIXSBzcGxpdDogYWRkaXRpb25hbCBzdWZm
aXggZm9yIHNwbGl0IChidWcjNjU1NCkKCkFkZCB0aGUgLS1hZGRpdGlvbmFsLXN1ZmZpeCBvcHRp
b24sIHRvIGFwcGVuZCBhbiBhZGRpdGlvbmFsIHN1ZmZpeCB0bwpvdXRwdXQgZmlsZSBuYW1lcy4K
Ciogc3JjL3NwbGl0LmMgKG5leHRfZmlsZV9uYW1lKTogQXBwZW5kIGFkZGl0aW9ubmFsX3N1ZmZp
eCB0byBvdXRwdXQKICBmaWxlIG5hbWVzLgoobWFpbik6IEhhbmRsZSBuZXcgLS1hZGRpdGlvbmFs
LXN1ZmZpeCBvcHRpb24uCiogTkVXUyAoTmV3IGZlYXR1cmVzKTogTWVudGlvbiBpdC4KKiBkb2Mv
Y29yZXV0aWxzLnRleGkgKHNwbGl0IGludm9jYXRpb24pOiBNZW50aW9uIGl0LgoqIHRlc3RzL3Nw
bGl0L2FkZGl0aW9uYWwtc3VmZml4OiBOZXcgZmlsZS4gLS1hZGRpdGlvbmFsLXN1ZmZpeCBvcHRp
b24KICB0ZXN0cy4KKiB0ZXN0cy9NYWtlZmlsZS5hbSAoVEVTVFMpOiBBZGQgaXQuClJlcXVlc3Rl
ZCBieSBQZW5nIFl1LgotLS0KIE5FV1MgICAgICAgICAgICAgICAgICAgICAgICAgIHwgICAgMyAr
KwogZG9jL2NvcmV1dGlscy50ZXhpICAgICAgICAgICAgfCAgICA1ICsrKysKIHNyYy9zcGxpdC5j
ICAgICAgICAgICAgICAgICAgIHwgICAyNiArKysrKysrKysrKysrKysrKysrKysrLQogdGVzdHMv
TWFrZWZpbGUuYW0gICAgICAgICAgICAgfCAgICAxICsKIHRlc3RzL3NwbGl0L2FkZGl0aW9uYWwt
c3VmZml4IHwgICA0NCArKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKwog
NSBmaWxlcyBjaGFuZ2VkLCA3NyBpbnNlcnRpb25zKCspLCAyIGRlbGV0aW9ucygtKQogY3JlYXRl
IG1vZGUgMTAwNzU1IHRlc3RzL3NwbGl0L2FkZGl0aW9uYWwtc3VmZml4CgpkaWZmIC0tZ2l0IGEv
TkVXUyBiL05FV1MKaW5kZXggM2IxNWQzOS4uZTZjYTYxZiAxMDA2NDQKLS0tIGEvTkVXUworKysg
Yi9ORVdTCkBAIC0xMCw2ICsxMCw5IEBAIEdOVSBjb3JldXRpbHMgTkVXUyAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIC0qLSBvdXRsaW5lIC0qLQogICBzcGxpdCBub3cgYWNjZXB0
cyBhbiBvcHRpb25hbCAiZnJvbSIgYXJndW1lbnQgdG8gLS1udW1lcmljLXN1ZmZpeGVzLAogICB3
aGljaCBjaGFuZ2VzIHRoZSBzdGFydCBudW1iZXIgZnJvbSB0aGUgZGVmYXVsdCBvZiAwLgogCisg
IHNwbGl0IG5vdyBhY2NlcHRzIHRoZSAtLWFkZGl0aW9uYWwtc3VmZml4IG9wdGlvbiwgdG8gYXBw
ZW5kIGFuCisgIGFkZGl0aW9uYWwgc3VmZml4IHRvIG91dHB1dCBmaWxlIG5hbWVzLgorCiAqKiBC
dWcgZml4ZXMKIAogICBtdiBub3cgbGV0cyB5b3UgbW92ZSBhIHN5bWxpbmsgb250byBhIHNhbWUt
aW5vZGUgZGVzdGluYXRpb24gZmlsZSB0aGF0CmRpZmYgLS1naXQgYS9kb2MvY29yZXV0aWxzLnRl
eGkgYi9kb2MvY29yZXV0aWxzLnRleGkKaW5kZXggNTYyNDQ0My4uY2MzMDBhOCAxMDA2NDQKLS0t
IGEvZG9jL2NvcmV1dGlscy50ZXhpCisrKyBiL2RvYy9jb3JldXRpbHMudGV4aQpAQCAtMzA5MCw2
ICszMDkwLDExIEBAIFVzZSBzdWZmaXhlcyBvZiBsZW5ndGggQHZhcntsZW5ndGh9LiAgVGhlIGRl
ZmF1bHQgQHZhcntsZW5ndGh9IGlzIDIuCiBVc2UgZGlnaXRzIGluIHN1ZmZpeGVzIHJhdGhlciB0
aGFuIGxvd2VyLWNhc2UgbGV0dGVycy4gIFRoZSBudW1lcmljYWwKIHN1ZmZpeCBjb3VudHMgZnJv
bSBAdmFye2Zyb219IGlmIHNwZWNpZmllZCwgMCBvdGhlcndpc2UuCiAKK0BpdGVteCAtLWFkZGl0
aW9uYWwtc3VmZml4PUB2YXJ7c3VmZml4fQorQG9waW5kZXggLS1hZGRpdGlvbmFsLXN1ZmZpeAor
QXBwZW5kIGFuIGFkZGl0aW9uYWwgQHZhcntzdWZmaXh9IHRvIG91dHB1dCBmaWxlIG5hbWVzLiBA
dmFye3N1ZmZpeH0KK211c3Qgbm90IGNvbnRhaW4gc2xhc2guCisKIEBpdGVtIC1lCiBAaXRlbXgg
LS1lbGlkZS1lbXB0eS1maWxlcwogQG9waW5kZXggLWUKZGlmZiAtLWdpdCBhL3NyYy9zcGxpdC5j
IGIvc3JjL3NwbGl0LmMKaW5kZXggMGU2NTAwMS4uYzIxODcwYyAxMDA2NDQKLS0tIGEvc3JjL3Nw
bGl0LmMKKysrIGIvc3JjL3NwbGl0LmMKQEAgLTgzLDYgKzgzLDkgQEAgc3RhdGljIGNoYXIgY29u
c3QgKnN1ZmZpeF9hbHBoYWJldCA9ICJhYmNkZWZnaGlqa2xtbm9wcXJzdHV2d3h5eiI7CiAvKiBO
dW1lcmljYWwgc3VmZml4IHN0YXJ0IHZhbHVlLiAgKi8KIHN0YXRpYyBjb25zdCBjaGFyICpudW1l
cmljX3N1ZmZpeF9zdGFydDsKIAorLyogQWRkaXRpb25hbCBzdWZmaXggdG8gYXBwZW5kIHRvIG91
dHB1dCBmaWxlIG5hbWVzLiAgKi8KK3N0YXRpYyBjaGFyIGNvbnN0ICphZGRpdGlvbmFsX3N1ZmZp
eDsKKwogLyogTmFtZSBvZiBpbnB1dCBmaWxlLiAgTWF5IGJlICItIi4gICovCiBzdGF0aWMgY2hh
ciAqaW5maWxlOwogCkBAIC0xMTMsNyArMTE2LDggQEAgZW51bQogewogICBWRVJCT1NFX09QVElP
TiA9IENIQVJfTUFYICsgMSwKICAgRklMVEVSX09QVElPTiwKLSAgSU9fQkxLU0laRV9PUFRJT04K
KyAgSU9fQkxLU0laRV9PUFRJT04sCisgIEFERElUSU9OQUxfU1VGRklYX09QVElPTgogfTsKIAog
c3RhdGljIHN0cnVjdCBvcHRpb24gY29uc3QgbG9uZ29wdHNbXSA9CkBAIC0xMjUsNiArMTI5LDgg
QEAgc3RhdGljIHN0cnVjdCBvcHRpb24gY29uc3QgbG9uZ29wdHNbXSA9CiAgIHsiZWxpZGUtZW1w
dHktZmlsZXMiLCBub19hcmd1bWVudCwgTlVMTCwgJ2UnfSwKICAgeyJ1bmJ1ZmZlcmVkIiwgbm9f
YXJndW1lbnQsIE5VTEwsICd1J30sCiAgIHsic3VmZml4LWxlbmd0aCIsIHJlcXVpcmVkX2FyZ3Vt
ZW50LCBOVUxMLCAnYSd9LAorICB7ImFkZGl0aW9uYWwtc3VmZml4IiwgcmVxdWlyZWRfYXJndW1l
bnQsIE5VTEwsCisgICBBRERJVElPTkFMX1NVRkZJWF9PUFRJT059LAogICB7Im51bWVyaWMtc3Vm
Zml4ZXMiLCBvcHRpb25hbF9hcmd1bWVudCwgTlVMTCwgJ2QnfSwKICAgeyJmaWx0ZXIiLCByZXF1
aXJlZF9hcmd1bWVudCwgTlVMTCwgRklMVEVSX09QVElPTn0sCiAgIHsidmVyYm9zZSIsIG5vX2Fy
Z3VtZW50LCBOVUxMLCBWRVJCT1NFX09QVElPTn0sCkBAIC0xOTYsNiArMjAyLDggQEAgTWFuZGF0
b3J5IGFyZ3VtZW50cyB0byBsb25nIG9wdGlvbnMgYXJlIG1hbmRhdG9yeSBmb3Igc2hvcnQgb3B0
aW9ucyB0b28uXG5cCiAiKSwgc3Rkb3V0KTsKICAgICAgIGZwcmludGYgKHN0ZG91dCwgXygiXAog
ICAtYSwgLS1zdWZmaXgtbGVuZ3RoPU4gICB1c2Ugc3VmZml4ZXMgb2YgbGVuZ3RoIE4gKGRlZmF1
bHQgJWQpXG5cCisgICAgICAtLWFkZGl0aW9uYWwtc3VmZml4PVNVRkZJWCAgYXBwZW5kIGFuIGFk
ZGl0aW9uYWwgU1VGRklYIHRvIG91dHB1dCBmaWxlXG5cCisgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICBuYW1lcy4gU1VGRklYIG11c3Qgbm90IGNvbnRhaW4gc2xhc2guXG5c
CiAgIC1iLCAtLWJ5dGVzPVNJWkUgICAgICAgIHB1dCBTSVpFIGJ5dGVzIHBlciBvdXRwdXQgZmls
ZVxuXAogICAtQywgLS1saW5lLWJ5dGVzPVNJWkUgICBwdXQgYXQgbW9zdCBTSVpFIGJ5dGVzIG9m
IGxpbmVzIHBlciBvdXRwdXQgZmlsZVxuXAogICAtZCwgLS1udW1lcmljLXN1ZmZpeGVzWz1GUk9N
XSAgdXNlIG51bWVyaWMgc3VmZml4ZXMgaW5zdGVhZCBvZiBhbHBoYWJldGljLlxuXApAQCAtMjQx
LDEzICsyNDksMTYgQEAgbmV4dF9maWxlX25hbWUgKHZvaWQpCiAgICAgICAvKiBBbGxvY2F0ZSBh
bmQgaW5pdGlhbGl6ZSB0aGUgZmlyc3QgZmlsZSBuYW1lLiAgKi8KIAogICAgICAgc2l6ZV90IG91
dGJhc2VfbGVuZ3RoID0gc3RybGVuIChvdXRiYXNlKTsKLSAgICAgIHNpemVfdCBvdXRmaWxlX2xl
bmd0aCA9IG91dGJhc2VfbGVuZ3RoICsgc3VmZml4X2xlbmd0aDsKKyAgICAgIHNpemVfdCBhZGRz
dWZfbGVuZ3RoID0gYWRkaXRpb25hbF9zdWZmaXggPyBzdHJsZW4gKGFkZGl0aW9uYWxfc3VmZml4
KSA6IDA7CisgICAgICBzaXplX3Qgb3V0ZmlsZV9sZW5ndGggPSBvdXRiYXNlX2xlbmd0aCArIHN1
ZmZpeF9sZW5ndGggKyBhZGRzdWZfbGVuZ3RoOwogICAgICAgaWYgKG91dGZpbGVfbGVuZ3RoICsg
MSA8IG91dGJhc2VfbGVuZ3RoKQogICAgICAgICB4YWxsb2NfZGllICgpOwogICAgICAgb3V0Zmls
ZSA9IHhtYWxsb2MgKG91dGZpbGVfbGVuZ3RoICsgMSk7CiAgICAgICBvdXRmaWxlX21pZCA9IG91
dGZpbGUgKyBvdXRiYXNlX2xlbmd0aDsKICAgICAgIG1lbWNweSAob3V0ZmlsZSwgb3V0YmFzZSwg
b3V0YmFzZV9sZW5ndGgpOwogICAgICAgbWVtc2V0IChvdXRmaWxlX21pZCwgc3VmZml4X2FscGhh
YmV0WzBdLCBzdWZmaXhfbGVuZ3RoKTsKKyAgICAgIGlmIChhZGRpdGlvbmFsX3N1ZmZpeCkKKyAg
ICAgICAgbWVtY3B5IChvdXRmaWxlX21pZCArIHN1ZmZpeF9sZW5ndGgsIGFkZGl0aW9uYWxfc3Vm
Zml4LCBhZGRzdWZfbGVuZ3RoKTsKICAgICAgIG91dGZpbGVbb3V0ZmlsZV9sZW5ndGhdID0gMDsK
ICAgICAgIHN1ZmluZGV4ID0geGNhbGxvYyAoc3VmZml4X2xlbmd0aCwgc2l6ZW9mICpzdWZpbmRl
eCk7CiAKQEAgLTEwNTIsNiArMTA2MywxNyBAQCBtYWluIChpbnQgYXJnYywgY2hhciAqKmFyZ3Yp
CiAgICAgICAgICAgfQogICAgICAgICAgIGJyZWFrOwogCisgICAgICAgIGNhc2UgQURESVRJT05B
TF9TVUZGSVhfT1BUSU9OOgorICAgICAgICAgIGlmIChsYXN0X2NvbXBvbmVudCAob3B0YXJnKSAh
PSBvcHRhcmcpCisgICAgICAgICAgICB7CisgICAgICAgICAgICAgIGVycm9yICgwLCAwLAorICAg
ICAgICAgICAgICAgICAgICAgXygiaW52YWxpZCBzdWZmaXggJXMsIGNvbnRhaW5zIGRpcmVjdG9y
eSBzZXBhcmF0b3IiKSwKKyAgICAgICAgICAgICAgICAgICAgIHF1b3RlIChvcHRhcmcpKTsKKyAg
ICAgICAgICAgICAgdXNhZ2UgKEVYSVRfRkFJTFVSRSk7CisgICAgICAgICAgICB9CisgICAgICAg
ICAgYWRkaXRpb25hbF9zdWZmaXggPSBvcHRhcmc7CisgICAgICAgICAgYnJlYWs7CisKICAgICAg
ICAgY2FzZSAnYic6CiAgICAgICAgICAgaWYgKHNwbGl0X3R5cGUgIT0gdHlwZV91bmRlZikKICAg
ICAgICAgICAgIEZBSUxfT05MWV9PTkVfV0FZICgpOwpkaWZmIC0tZ2l0IGEvdGVzdHMvTWFrZWZp
bGUuYW0gYi90ZXN0cy9NYWtlZmlsZS5hbQppbmRleCAyZmVlOTdkLi43NGZmNDcwIDEwMDY0NAot
LS0gYS90ZXN0cy9NYWtlZmlsZS5hbQorKysgYi90ZXN0cy9NYWtlZmlsZS5hbQpAQCAtMjU0LDYg
KzI1NCw3IEBAIFRFU1RTID0JCQkJCQlcCiAgIG1pc2Mvc29ydC1OYU4taW5mbG9vcAkJCQlcCiAg
IHNwbGl0L2ZpbHRlcgkJCQkJXAogICBzcGxpdC9zdWZmaXgtbGVuZ3RoCQkJCVwKKyAgc3BsaXQv
YWRkaXRpb25hbC1zdWZmaXgJCQlcCiAgIHNwbGl0L2ItY2h1bmsJCQkJCVwKICAgc3BsaXQvZmFp
bAkJCQkJXAogICBzcGxpdC9saW5lcwkJCQkJXApkaWZmIC0tZ2l0IGEvdGVzdHMvc3BsaXQvYWRk
aXRpb25hbC1zdWZmaXggYi90ZXN0cy9zcGxpdC9hZGRpdGlvbmFsLXN1ZmZpeApuZXcgZmlsZSBt
b2RlIDEwMDc1NQppbmRleCAwMDAwMDAwLi44Y2ZkM2MxCi0tLSAvZGV2L251bGwKKysrIGIvdGVz
dHMvc3BsaXQvYWRkaXRpb25hbC1zdWZmaXgKQEAgLTAsMCArMSw0NCBAQAorIyEvYmluL3NoCisj
IHNob3cgdGhhdCAnc3BsaXQgLS1hZGRpdGlvbmFsLXN1ZmZpeD1TVUZGSVgnIHdvcmtzLgorCisj
IENvcHlyaWdodCAoQykgMjAxMiBGcmVlIFNvZnR3YXJlIEZvdW5kYXRpb24sIEluYy4KKworIyBU
aGlzIHByb2dyYW0gaXMgZnJlZSBzb2Z0d2FyZTogeW91IGNhbiByZWRpc3RyaWJ1dGUgaXQgYW5k
L29yIG1vZGlmeQorIyBpdCB1bmRlciB0aGUgdGVybXMgb2YgdGhlIEdOVSBHZW5lcmFsIFB1Ymxp
YyBMaWNlbnNlIGFzIHB1Ymxpc2hlZCBieQorIyB0aGUgRnJlZSBTb2Z0d2FyZSBGb3VuZGF0aW9u
LCBlaXRoZXIgdmVyc2lvbiAzIG9mIHRoZSBMaWNlbnNlLCBvcgorIyAoYXQgeW91ciBvcHRpb24p
IGFueSBsYXRlciB2ZXJzaW9uLgorCisjIFRoaXMgcHJvZ3JhbSBpcyBkaXN0cmlidXRlZCBpbiB0
aGUgaG9wZSB0aGF0IGl0IHdpbGwgYmUgdXNlZnVsLAorIyBidXQgV0lUSE9VVCBBTlkgV0FSUkFO
VFk7IHdpdGhvdXQgZXZlbiB0aGUgaW1wbGllZCB3YXJyYW50eSBvZgorIyBNRVJDSEFOVEFCSUxJ
VFkgb3IgRklUTkVTUyBGT1IgQSBQQVJUSUNVTEFSIFBVUlBPU0UuICBTZWUgdGhlCisjIEdOVSBH
ZW5lcmFsIFB1YmxpYyBMaWNlbnNlIGZvciBtb3JlIGRldGFpbHMuCisKKyMgWW91IHNob3VsZCBo
YXZlIHJlY2VpdmVkIGEgY29weSBvZiB0aGUgR05VIEdlbmVyYWwgUHVibGljIExpY2Vuc2UKKyMg
YWxvbmcgd2l0aCB0aGlzIHByb2dyYW0uICBJZiBub3QsIHNlZSA8aHR0cDovL3d3dy5nbnUub3Jn
L2xpY2Vuc2VzLz4uCisKKy4gIiR7c3JjZGlyPS59L2luaXQuc2giOyBwYXRoX3ByZXBlbmRfIC4u
L3NyYworcHJpbnRfdmVyXyBzcGxpdAorCitwcmludGYgJzFcbjJcbjNcbjRcbjVcbicgPiBpbiB8
fCBmcmFtZXdvcmtfZmFpbHVyZV8KKworc3BsaXQgLS1saW5lcz0yIC0tYWRkaXRpb25hbC1zdWZm
aXg9LnR4dCBpbiA+IG91dCB8fCBmYWlsPTEKK2NhdCA8PFxFT0YgPiBleHAtMQorMQorMgorRU9G
CitjYXQgPDxcRU9GID4gZXhwLTIKKzMKKzQKK0VPRgorY2F0IDw8XEVPRiA+IGV4cC0zCis1CitF
T0YKKworY29tcGFyZSBleHAtMSB4YWEudHh0IHx8IGZhaWw9MQorY29tcGFyZSBleHAtMiB4YWIu
dHh0IHx8IGZhaWw9MQorY29tcGFyZSBleHAtMyB4YWMudHh0IHx8IGZhaWw9MQorCisjIEFkZGl0
aW9uYWwgc3VmZml4IG11c3Qgbm90IGNvbnRhaW4gc2xhc2gKK3NwbGl0IC0tbGluZXM9MiAtLWFk
ZGl0aW9uYWwtc3VmZml4PWEvYiBpbiAyPi9kZXYvbnVsbCA+IG91dCAmJiBmYWlsPTEKKworRXhp
dCAkZmFpbAotLSAKMS43LjIuNQoK
--e89a8f3ba429eef11004b950ae2c--


From MAILER-DAEMON Sun Feb 19 09:43:47 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1Rz7zP-00059d-Kk
	for mharc-coreutils@gnu.org; Sun, 19 Feb 2012 09:43:47 -0500
Received: from eggs.gnu.org ([140.186.70.92]:48689)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1Rz7zN-00059X-J2
	for coreutils@gnu.org; Sun, 19 Feb 2012 09:43:46 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1Rz7zM-0002v9-GC
	for coreutils@gnu.org; Sun, 19 Feb 2012 09:43:45 -0500
Received: from mail-ww0-f49.google.com ([74.125.82.49]:63231)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1Rz7zM-0002v2-Bh
	for coreutils@gnu.org; Sun, 19 Feb 2012 09:43:44 -0500
Received: by wgbdt13 with SMTP id dt13so3239038wgb.30
	for <coreutils@gnu.org>; Sun, 19 Feb 2012 06:43:43 -0800 (PST)
Received-SPF: pass (google.com: domain of jeremy.compostella@gmail.com
	designates 10.180.8.164 as permitted sender)
	client-ip=10.180.8.164; 
Authentication-Results: mr.google.com; spf=pass (google.com: domain of
	jeremy.compostella@gmail.com designates 10.180.8.164 as
	permitted sender) smtp.mail=jeremy.compostella@gmail.com;
	dkim=pass header.i=jeremy.compostella@gmail.com
Received: from mr.google.com ([10.180.8.164])
	by 10.180.8.164 with SMTP id s4mr10161125wia.6.1329662623590 (num_hops
	= 1); Sun, 19 Feb 2012 06:43:43 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma;
	h=from:to:subject:date:message-id:mime-version:content-type;
	bh=RNZVIkzYh+/DBzyzBCIYNNgnalQwWgpBzcZUH7FeWfg=;
	b=KZIwkExR9tg6mtk+oSIzjPyIEP9UU3OcOEuhPx5S1NdHpv5Hh1OfYn0b6dDcc781oW
	WIRAXxvub++sLWRngSizhzccdekV6pCMxGzMKGljlanLtSxROUTQmJmRvDMIeLwqPzio
	juO0g3+EMtTY4qk4rj8p3jhn7kzUO0iReGNNg=
Received: by 10.180.8.164 with SMTP id s4mr8513275wia.6.1329662623542;
	Sun, 19 Feb 2012 06:43:43 -0800 (PST)
Received: from Apollo.jerryland.fr ([85.69.251.50])
	by mx.google.com with ESMTPS id y6sm9687406wix.10.2012.02.19.06.43.42
	(version=TLSv1/SSLv3 cipher=OTHER);
	Sun, 19 Feb 2012 06:43:42 -0800 (PST)
From: =?utf-8?Q?J=C3=A9r=C3=A9my_Compostella?= <jeremy.compostella@gmail.com>
To: coreutils@gnu.org
Subject: [PATCH] tests: add missing sys/types.h for ls speed-up test
Date: Sun, 19 Feb 2012 15:44:04 +0100
Message-ID: <87pqda3mtn.fsf@Apollo.jerryland.fr>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="=-=-="
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 74.125.82.49
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 19 Feb 2012 14:43:46 -0000

--=-=-=
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

All,

I have the following error message in getxattr-speedup.log:

  FAIL: ls/getxattr-speedup (exit: 99)
  =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
=20=20
  k.c:13: error: expected '=3D', ',', ';', 'asm' or '__attribute__' before =
'incr'
  k.c:14: error: expected '=3D', ',', ';', 'asm' or '__attribute__' before =
'getxattr'
  k.c:16: error: expected '=3D', ',', ';', 'asm' or '__attribute__' before =
'lgetxattr'
  getxattr-speedup: set-up failure: failed to compile with -fPIC

I added the missing sys/types.h include in ls/getxattr-speedup.
See the corresponding patch in attachment.

Cheers,

J=C3=A9r=C3=A9my
---

--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=0001-tests-add-missing-sys-types.h-for-ls-speed-up-test.patch
Content-Description: tests: add missing sys/types.h for ls speed-up test
 patch

>From 409e1c27fa056f52583a002a9f28f9c45469165a Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?J=C3=A9r=C3=A9my=20Compostella?= <jeremy.compostella@gmail.com>
Date: Sun, 19 Feb 2012 15:38:15 +0100
Subject: [PATCH] tests: add missing sys/types.h for ls speed-up test

* tests/ls/getxattr-speedup: add missing sys/types.h header include.
---
 tests/ls/getxattr-speedup |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/tests/ls/getxattr-speedup b/tests/ls/getxattr-speedup
index 2b7a1f3..a35dd69 100755
--- a/tests/ls/getxattr-speedup
+++ b/tests/ls/getxattr-speedup
@@ -28,6 +28,7 @@ print_ver_ ls
 cat > k.c <<'EOF' || framework_failure_
 #include <errno.h>
 #include <stdio.h>
+#include <sys/types.h>
 
 static unsigned long int n_calls;
 
-- 
1.7.2.5


--=-=-=--


From MAILER-DAEMON Sun Feb 19 09:58:58 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1Rz8E6-00011H-Uk
	for mharc-coreutils@gnu.org; Sun, 19 Feb 2012 09:58:58 -0500
Received: from eggs.gnu.org ([140.186.70.92]:35383)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Rz8E4-00011B-Kj
	for coreutils@gnu.org; Sun, 19 Feb 2012 09:58:57 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Rz8E3-0005ax-Kd
	for coreutils@gnu.org; Sun, 19 Feb 2012 09:58:56 -0500
Received: from mx.meyering.net ([88.168.87.75]:46093)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1Rz8E3-0005at-De
	for coreutils@gnu.org; Sun, 19 Feb 2012 09:58:55 -0500
Received: from rho.meyering.net (localhost.localdomain [127.0.0.1])
	by rho.meyering.net (Acme Bit-Twister) with ESMTP id 99BE060094;
	Sun, 19 Feb 2012 15:58:54 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: =?iso-8859-1?Q?J=E9r=E9my?= Compostella <jeremy.compostella@gmail.com>
Subject: Re: [PATCH] tests: add missing sys/types.h for ls speed-up test
In-Reply-To: <87pqda3mtn.fsf@Apollo.jerryland.fr> (=?iso-8859-1?Q?=22J=E9r?=
	=?iso-8859-1?Q?=E9my?= Compostella"'s
	message of "Sun, 19 Feb 2012 15:44:04 +0100")
References: <87pqda3mtn.fsf@Apollo.jerryland.fr>
Date: Sun, 19 Feb 2012 15:58:54 +0100
Message-ID: <877gzidg41.fsf@rho.meyering.net>
Lines: 49
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 88.168.87.75
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 19 Feb 2012 14:58:57 -0000

J=E9r=E9my Compostella wrote:

> All,
>
> I have the following error message in getxattr-speedup.log:
>
>   FAIL: ls/getxattr-speedup (exit: 99)
>   =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>
>   k.c:13: error: expected '=3D', ',', ';', 'asm' or '__attribute__' befor=
e 'incr'
>   k.c:14: error: expected '=3D', ',', ';', 'asm' or '__attribute__'
> before 'getxattr'
>   k.c:16: error: expected '=3D', ',', ';', 'asm' or '__attribute__'
> before 'lgetxattr'
>   getxattr-speedup: set-up failure: failed to compile with -fPIC
>
> I added the missing sys/types.h include in ls/getxattr-speedup.
> See the corresponding patch in attachment.

Hi J=E9r=E9my,

Thank you for the report and the patch.
Please tell us what type/version of compiler/system you're using.
It's good to include that information, especially since ssize_t
is supposed to be defined via stdio.h.  Then it is clear that this
change is working around a problem with non-conforming systems.


>>From 409e1c27fa056f52583a002a9f28f9c45469165a Mon Sep 17 00:00:00 2001
> From: =3D?UTF-8?q?J=3DC3=3DA9r=3DC3=3DA9my=3D20Compostella?=3D <jeremy.co=
mpostella@gmail.com>
> Date: Sun, 19 Feb 2012 15:38:15 +0100
> Subject: [PATCH] tests: add missing sys/types.h for ls speed-up test
>
> * tests/ls/getxattr-speedup: add missing sys/types.h header include.
> ---
>  tests/ls/getxattr-speedup |    1 +
>  1 files changed, 1 insertions(+), 0 deletions(-)
>
> diff --git a/tests/ls/getxattr-speedup b/tests/ls/getxattr-speedup
> index 2b7a1f3..a35dd69 100755
> --- a/tests/ls/getxattr-speedup
> +++ b/tests/ls/getxattr-speedup
> @@ -28,6 +28,7 @@ print_ver_ ls
>  cat > k.c <<'EOF' || framework_failure_
>  #include <errno.h>
>  #include <stdio.h>
> +#include <sys/types.h>
>
>  static unsigned long int n_calls;


From MAILER-DAEMON Sun Feb 19 10:07:46 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1Rz8Mb-0003C9-Vw
	for mharc-coreutils@gnu.org; Sun, 19 Feb 2012 10:07:45 -0500
Received: from eggs.gnu.org ([140.186.70.92]:38426)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1Rz8MZ-0003AS-Lh
	for coreutils@gnu.org; Sun, 19 Feb 2012 10:07:44 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1Rz8MY-0007Zv-I4
	for coreutils@gnu.org; Sun, 19 Feb 2012 10:07:43 -0500
Received: from mail-iy0-f169.google.com ([209.85.210.169]:54540)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1Rz8MY-0007Zr-AX
	for coreutils@gnu.org; Sun, 19 Feb 2012 10:07:42 -0500
Received: by iagz16 with SMTP id z16so8241722iag.0
	for <coreutils@gnu.org>; Sun, 19 Feb 2012 07:07:41 -0800 (PST)
Received-SPF: pass (google.com: domain of jeremy.compostella@gmail.com
	designates 10.42.235.137 as permitted sender)
	client-ip=10.42.235.137; 
Authentication-Results: mr.google.com; spf=pass (google.com: domain of
	jeremy.compostella@gmail.com designates 10.42.235.137 as
	permitted sender) smtp.mail=jeremy.compostella@gmail.com;
	dkim=pass header.i=jeremy.compostella@gmail.com
Received: from mr.google.com ([10.42.235.137])
	by 10.42.235.137 with SMTP id kg9mr17518050icb.47.1329664061799
	(num_hops = 1); Sun, 19 Feb 2012 07:07:41 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma;
	h=mime-version:in-reply-to:references:from:date:message-id:subject:to
	:cc:content-type;
	bh=fef7gyR8CsCAU3H/8PnWCZHQ9JFayJgdaM/8UT8Ve2w=;
	b=N8mMoFZQk6dClZE3BWL24gbo5nrOps3CH+4pG3D94PU6nV/iOEuucOR5A2Saj0fNuI
	zC6L4+40oSiLjGqLGsa5Y+pj5hVHcF5maHhYCbOZJsXxT4GS8SipzK18XB24EmhqRnp3
	SF0cX8bphsfPP8qhAE7yKeDxeW1wRzh5qiv/E=
Received: by 10.42.235.137 with SMTP id kg9mr13928040icb.47.1329664061656;
	Sun, 19 Feb 2012 07:07:41 -0800 (PST)
MIME-Version: 1.0
Received: by 10.42.29.196 with HTTP; Sun, 19 Feb 2012 07:07:21 -0800 (PST)
In-Reply-To: <877gzidg41.fsf@rho.meyering.net>
References: <87pqda3mtn.fsf@Apollo.jerryland.fr>
	<877gzidg41.fsf@rho.meyering.net>
From: =?ISO-8859-1?Q?J=E9r=E9my_Compostella?= <jeremy.compostella@gmail.com>
Date: Sun, 19 Feb 2012 16:07:21 +0100
Message-ID: <CA+TATJEUtf0513Cdk-XmCiHBfXGrFPYwRSBJj=jV=tfK79szDw@mail.gmail.com>
Subject: Re: [PATCH] tests: add missing sys/types.h for ls speed-up test
To: Jim Meyering <jim@meyering.net>
Content-Type: multipart/alternative; boundary=20cf3026697a10ce7504b9528a63
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.85.210.169
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 19 Feb 2012 15:07:44 -0000

--20cf3026697a10ce7504b9528a63
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

> Thank you for the report and the patch.
> Please tell us what type/version of compiler/system you're using.
> It's good to include that information, especially since ssize_t
> is supposed to be defined via stdio.h.  Then it is clear that this
> change is working around a problem with non-conforming systems.
Here the requested information:
- linux kernel: Linux Apollo 3.2.0-rc7+ #75 SMP Thu Jan 5 18:14:54 CET
  2012 x86_64 GNU/Linux
- gcc (Debian 4.4.5-8) 4.4.5
- libc6-dev 2.11.2-10

Cheers,

J=E9r=E9my

--20cf3026697a10ce7504b9528a63
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div>&gt; Thank you for the report and the patch.</div><div>&gt; Please tel=
l us what type/version of compiler/system you&#39;re using.</div><div>&gt; =
It&#39;s good to include that information, especially since ssize_t</div>

<div>&gt; is supposed to be defined via stdio.h. =A0Then it is clear that t=
his</div><div>&gt; change is working around a problem with non-conforming s=
ystems.</div><div>Here the requested information:</div><div>- linux kernel:=
 Linux Apollo 3.2.0-rc7+ #75 SMP Thu Jan 5 18:14:54 CET</div>

<div>=A0=A02012 x86_64 GNU/Linux</div><div>- gcc (Debian 4.4.5-8) 4.4.5</di=
v><div>- libc6-dev 2.11.2-10</div><div><br></div><div>Cheers,</div><div><br=
></div><div>J=E9r=E9my</div>

--20cf3026697a10ce7504b9528a63--


From MAILER-DAEMON Sun Feb 19 12:01:02 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RzA8E-0006QO-Ea
	for mharc-coreutils@gnu.org; Sun, 19 Feb 2012 12:01:02 -0500
Received: from eggs.gnu.org ([140.186.70.92]:54806)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1RzA8A-0006Lo-P9
	for coreutils@gnu.org; Sun, 19 Feb 2012 12:01:00 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1RzA85-0002dz-AT
	for coreutils@gnu.org; Sun, 19 Feb 2012 12:00:58 -0500
Received: from mx.meyering.net ([88.168.87.75]:46363)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1RzA84-0002de-FO
	for coreutils@gnu.org; Sun, 19 Feb 2012 12:00:52 -0500
Received: from rho.meyering.net (localhost.localdomain [127.0.0.1])
	by rho.meyering.net (Acme Bit-Twister) with ESMTP id D977660051;
	Sun, 19 Feb 2012 18:00:50 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: =?iso-8859-1?Q?P=E1draig?= Brady <P@draigBrady.com>
Subject: Re: "ls -l": Avoid unnecessary getxattr() overhead
In-Reply-To: <4F40D061.3000206@draigBrady.com> (=?iso-8859-1?Q?=22P=E1drai?=
	=?iso-8859-1?Q?g?= Brady"'s message of
	"Sun, 19 Feb 2012 10:35:13 +0000")
References: <87pqds3snt.fsf@Apollo.jerryland.fr>
	<4F30206B.1020802@itwm.fraunhofer.de>
	<CA+TATJHorgsuY7pxmpxxAeWUHRP5MfFfzQP05o=1H6C4ttswdA@mail.gmail.com>
	<4F3032BC.50702@draigBrady.com> <4F30631A.2070009@itwm.fraunhofer.de>
	<87y5scfdiq.fsf@rho.meyering.net> <4F33B5D7.3010909@draigBrady.com>
	<87mx8sf5w0.fsf@rho.meyering.net>
	<4F3437CE.4050308@itwm.fraunhofer.de>
	<4F346CC7.4030608@draigBrady.com> <87r4y3dry0.fsf@rho.meyering.net>
	<4F3710CA.8020402@itwm.fraunhofer.de>
	<8739aao9e2.fsf@rho.meyering.net>
	<4F3E12F9.6080102@bernhard-voelker.de>
	<8762f5lpct.fsf@rho.meyering.net>
	<4F3E35C9.7050707@bernhard-voelker.de>
	<87mx8hk6bm.fsf@rho.meyering.net>
	<4F4069D3.5000806@itwm.fraunhofer.de>
	<87pqdbcia0.fsf@rho.meyering.net> <4F40D061.3000206@draigBrady.com>
Date: Sun, 19 Feb 2012 18:00:50 +0100
Message-ID: <871upqdagt.fsf@rho.meyering.net>
Lines: 83
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 88.168.87.75
Cc: "Daniel P. Berrange" <berrange@redhat.com>,
	Bernhard Voelker <mail@bernhard-voelker.de>,
	Bernd Schubert <bernd.schubert@itwm.fraunhofer.de>, coreutils@gnu.org,
	Sven Breuner <sven.breuner@itwm.fraunhofer.de>,
	=?iso-8859-1?Q?J=E9r=E9my?= Compostella <jeremy.compostella@gmail.com>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 19 Feb 2012 17:01:00 -0000

P=E1draig Brady wrote:
> On 02/19/2012 08:57 AM, Jim Meyering wrote:
>> Sven Breuner wrote:
>>
>>> Jim Meyering wrote on 02/09/2012 03:17 PM:
>>>> Here's that simpler patch:
>>>> [...]
>>>
>>> Jim Meyering wrote on 02/17/2012 01:11 PM:
>>>> Here's the updates series, along with this addition to NEWS:
>>>> [...]
>>>
>>> Thanks for all this optimization work so far!
>>>
>>> My "ls -l" test case with >60K files in a single directory is now down
>>> from about 50s to about 20s already. (When the problem below is
>>> solved, it should be down to about 10s).
>>>
>>> Unfortunately, some getxattr selinux calls seem to be back with the
>>> current coreutils git version.
>>> Those calls were gone after Jim's initial patch from 02/09/2012.
>>>
>>> With only the patch from 02/09/2012 applied, strace showed this output
>>> for the first three files:
>>
>> The initial patch interpreted ENODATA as "not supported",
>> but that shouldn't matter, since we cached the failure only
>> when getfilecon returned -1.  Here, it's always returning 10
>> (sizeof "unlabeled") and hence is not a failure.
>>
>> I think it is legitimate to cache that case (success with "unlabeled")
>> as well.  Presuming I can confirm that, I'll make that change, too.
>
> Hmm I'm not sure.
> Couldn't you get that for some files and not others on some systems?

Technically, you can get a security context of "unlabeled" on one file,
and a regular context for the next file in the same directory, but in
practice it seems to be all or nothing, at a per-device level.
Here's where it'd be nice to have a user-specified format string,
so that those for whom it is inordinately expensive could easily
omit the "alternate access method flag" (the ".", "+", "" or "?" after
the 10-byte type+permission indicator), for which we incur most of this
overhead.

I'm afraid that we can't unilaterally interpret <10,"unlabeled">
as a per-device condition.

> Also I was a bit surprised to see EBUSY and ENOENT in errno_unsupported().
> Can one assume there are no other attributes on a device if you get those?

I added those to align with ACL_NOT_WELL_SUPPORTED from
lib/acl-internal.h, but now that you mention it, I suppose we'll need
different errno-checking functions for ACL-related than for getfilecon
failures.  I suppose ENOENT can arise when the file is removed between
when we stat'd it and when we perform one of these three file-name-using
tests.

>>> $ strace ~/tmp/ls_20120209 -l --color=3Dalways
>>> [...]
>>> lstat("file9", {st_mode=3DS_IFREG|0644, st_size=3D0, ...}) =3D 0
>>> capget(0x20080522, 0, NULL)             =3D -1 EFAULT (Bad address)
>>> getxattr("file9", "security.capability", 0x7fffffffda30, 20) =3D -1
>>> EOPNOTSUPP (Operation not supported)
>>> lgetxattr("file9", "security.selinux", "unlabeled", 255) =3D 10
>
>>From Sven's strace it seems that lgetfilecon() is interpreting
> the "unlabeled" return from lgetxattr() and converting that
> into a -1 return with ENODATA?  That's the only logical explanation
> I can see for how ENODATA is significant. As mentioned before
> I'm not sure we could add ENODATA to errno_unsupported().

Right.  We definitely don't want to add ENODATA.
I removed it for this sort of reason.

The gnulib getfilecon wrapper maps 10,"unlabeled" to -1,ENODATA.

Back when I wrote that wrapper, it sounded like 10,"unlabeled"
would happen only with kernels deemed crufty back then (in 2009).

  http://thread.gmane.org/gmane.comp.gnu.coreutils.bugs/18378/focus=3D18384

That we're still seeing such syscall traces in 2012...


From MAILER-DAEMON Sun Feb 19 12:45:33 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RzApJ-0005E6-Db
	for mharc-coreutils@gnu.org; Sun, 19 Feb 2012 12:45:33 -0500
Received: from eggs.gnu.org ([140.186.70.92]:32916)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1RzApH-0005AQ-36
	for coreutils@gnu.org; Sun, 19 Feb 2012 12:45:32 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1RzApG-0001vq-5e
	for coreutils@gnu.org; Sun, 19 Feb 2012 12:45:31 -0500
Received: from mx.meyering.net ([88.168.87.75]:46490)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1RzApF-0001vm-Un
	for coreutils@gnu.org; Sun, 19 Feb 2012 12:45:30 -0500
Received: from rho.meyering.net (localhost.localdomain [127.0.0.1])
	by rho.meyering.net (Acme Bit-Twister) with ESMTP id 1144C60064;
	Sun, 19 Feb 2012 18:45:29 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: =?iso-8859-1?Q?J=E9r=E9my?= Compostella <jeremy.compostella@gmail.com>
Subject: Re: [PATCH] tests: add missing sys/types.h for ls speed-up test
In-Reply-To: <CA+TATJEUtf0513Cdk-XmCiHBfXGrFPYwRSBJj=jV=tfK79szDw@mail.gmail.com>
	(=?iso-8859-1?Q?=22J=E9r=E9my?= Compostella"'s message of "Sun,
	19 Feb 2012 16:07:21 +0100")
References: <87pqda3mtn.fsf@Apollo.jerryland.fr>
	<877gzidg41.fsf@rho.meyering.net>
	<CA+TATJEUtf0513Cdk-XmCiHBfXGrFPYwRSBJj=jV=tfK79szDw@mail.gmail.com>
Date: Sun, 19 Feb 2012 18:45:28 +0100
Message-ID: <87vcn2bttz.fsf@rho.meyering.net>
Lines: 42
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 88.168.87.75
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 19 Feb 2012 17:45:32 -0000

J=E9r=E9my Compostella wrote:

>> Thank you for the report and the patch.
>> Please tell us what type/version of compiler/system you're using.
>> It's good to include that information, especially since ssize_t
>> is supposed to be defined via stdio.h. =A0Then it is clear that this
>> change is working around a problem with non-conforming systems.
> Here the requested information:
> - linux kernel: Linux Apollo 3.2.0-rc7+ #75 SMP Thu Jan 5 18:14:54 CET
> =A0=A02012 x86_64 GNU/Linux
> - gcc (Debian 4.4.5-8) 4.4.5
> - libc6-dev 2.11.2-10

Thanks.
I've adjusted your commit log to explain why:

>From 933989eca67bd82f011595937002cc284c98772d Mon Sep 17 00:00:00 2001
From: =3D?UTF-8?q?J=3DC3=3DA9r=3DC3=3DA9my=3D20Compostella?=3D <jeremy.comp=
ostella@gmail.com>
Date: Sun, 19 Feb 2012 15:38:15 +0100
Subject: [PATCH] tests: ls/getxattr-speedup: include <sys/types.h> for olde=
r headers

Modern <stdio.h> provides ssize_t, but the one from Debian's
libc6-dev 2.11.2-10 apparently does not.
* tests/ls/getxattr-speedup: Also include <sys/types.h>.
---
 tests/ls/getxattr-speedup |    1 +
 1 file changed, 1 insertion(+)

diff --git a/tests/ls/getxattr-speedup b/tests/ls/getxattr-speedup
index 2b7a1f3..a35dd69 100755
--- a/tests/ls/getxattr-speedup
+++ b/tests/ls/getxattr-speedup
@@ -28,6 +28,7 @@ print_ver_ ls
 cat > k.c <<'EOF' || framework_failure_
 #include <errno.h>
 #include <stdio.h>
+#include <sys/types.h>

 static unsigned long int n_calls;

--
1.7.9.1.245.gbbe07


From MAILER-DAEMON Sun Feb 19 12:47:53 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RzArZ-0006YO-PW
	for mharc-coreutils@gnu.org; Sun, 19 Feb 2012 12:47:53 -0500
Received: from eggs.gnu.org ([140.186.70.92]:33243)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1RzArW-0006W3-Bw
	for coreutils@gnu.org; Sun, 19 Feb 2012 12:47:51 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1RzArV-0002Bc-9w
	for coreutils@gnu.org; Sun, 19 Feb 2012 12:47:50 -0500
Received: from mail-iy0-f169.google.com ([209.85.210.169]:36736)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1RzArU-0002BV-V6
	for coreutils@gnu.org; Sun, 19 Feb 2012 12:47:49 -0500
Received: by iagz16 with SMTP id z16so8396288iag.0
	for <coreutils@gnu.org>; Sun, 19 Feb 2012 09:47:48 -0800 (PST)
Received-SPF: pass (google.com: domain of jeremy.compostella@gmail.com
	designates 10.42.144.196 as permitted sender)
	client-ip=10.42.144.196; 
Authentication-Results: mr.google.com; spf=pass (google.com: domain of
	jeremy.compostella@gmail.com designates 10.42.144.196 as
	permitted sender) smtp.mail=jeremy.compostella@gmail.com;
	dkim=pass header.i=jeremy.compostella@gmail.com
Received: from mr.google.com ([10.42.144.196])
	by 10.42.144.196 with SMTP id c4mr17779311icv.39.1329673668407
	(num_hops = 1); Sun, 19 Feb 2012 09:47:48 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma;
	h=mime-version:in-reply-to:references:from:date:message-id:subject:to
	:cc:content-type;
	bh=+TvxrueRZTqiY6msVQlws+lhI81kRXHkAv9K2dj2pqw=;
	b=XEFRd7fUDuaCooKhGSdzw2YloEP+sPCPw3EYySzv4hMXSqEw7k2RhCqRyR0H9Zat+5
	kmiLFXPewk19/47ArGA4P0WMDOisA6l/3PPJ6cGNmyhiQ21EHDMmnapMJVJ4khveARsw
	gK8etIFfvA20ux7vz8/UPR03m8UI+y62wx4NI=
Received: by 10.42.144.196 with SMTP id c4mr14152716icv.39.1329673668355; Sun,
	19 Feb 2012 09:47:48 -0800 (PST)
MIME-Version: 1.0
Received: by 10.42.29.196 with HTTP; Sun, 19 Feb 2012 09:47:28 -0800 (PST)
In-Reply-To: <CA+TATJEUtf0513Cdk-XmCiHBfXGrFPYwRSBJj=jV=tfK79szDw@mail.gmail.com>
References: <87pqda3mtn.fsf@Apollo.jerryland.fr>
	<877gzidg41.fsf@rho.meyering.net>
	<CA+TATJEUtf0513Cdk-XmCiHBfXGrFPYwRSBJj=jV=tfK79szDw@mail.gmail.com>
From: =?ISO-8859-1?Q?J=E9r=E9my_Compostella?= <jeremy.compostella@gmail.com>
Date: Sun, 19 Feb 2012 18:47:28 +0100
Message-ID: <CA+TATJHYy35NK5ftgCh+7aJodhhgnzn74aHGs3uGz3frU_gbQQ@mail.gmail.com>
Subject: Re: [PATCH] tests: add missing sys/types.h for ls speed-up test
To: Jim Meyering <jim@meyering.net>
Content-Type: multipart/alternative; boundary=90e6ba21224bab653504b954c6be
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.85.210.169
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 19 Feb 2012 17:47:51 -0000

--90e6ba21224bab653504b954c6be
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Jim,

> > Thank you for the report and the patch.
> > Please tell us what type/version of compiler/system you're using.
> > It's good to include that information, especially since ssize_t
> > is supposed to be defined via stdio.h.  Then it is clear that this
> > change is working around a problem with non-conforming systems.
> Here the requested information:
> - linux kernel: Linux Apollo 3.2.0-rc7+ #75 SMP Thu Jan 5 18:14:54 CET
>   2012 x86_64 GNU/Linux
> - gcc (Debian 4.4.5-8) 4.4.5
> - libc6-dev 2.11.2-10
I made some search. This issue is Debian related (I got another computer
with a recent Ubuntu and the problem is gone, version 2.13-0ubuntu13). I
am able to reproduce this issue on my Debian server too. I tested it
because my current laptop is a little bit customized and I don't trust
it at some points.

According to http://sourceware.org/bugzilla/show_bug.cgi?id=3D11125, this
issue has been reported on version 2.11 which is the current Debian GNU
libc distributed version and has been fixed. I just make the last Debian
update I always have this issue.

This has been fixed in glibc commit
cd2f000c074b07931bd78ab5ff5fa3c0f7db628a released with the 2.12 glibc
version.

So, the patch I proposed is clearly a Debian squeeze (2.11) workaround.
I will try to raise a bug report to the Debian team to see if they can pick
this commit in their branch.

Cheers,

J=E9r=E9my

--90e6ba21224bab653504b954c6be
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div>Jim,</div><div><br></div><div>&gt; &gt; Thank you for the report and t=
he patch.</div><div>&gt; &gt; Please tell us what type/version of compiler/=
system you&#39;re using.</div><div>&gt; &gt; It&#39;s good to include that =
information, especially since ssize_t</div>

<div>&gt; &gt; is supposed to be defined via stdio.h. =A0Then it is clear t=
hat this</div><div>&gt; &gt; change is working around a problem with non-co=
nforming systems.</div><div>&gt; Here the requested information:</div><div>

&gt; - linux kernel: Linux Apollo 3.2.0-rc7+ #75 SMP Thu Jan 5 18:14:54 CET=
</div><div>&gt; =A0 2012 x86_64 GNU/Linux</div><div>&gt; - gcc (Debian 4.4.=
5-8) 4.4.5</div><div>&gt; - libc6-dev 2.11.2-10</div><div>I made some searc=
h. This issue is Debian related (I got another computer</div>

<div>with a recent Ubuntu and the problem is gone, version 2.13-0ubuntu13).=
 I</div><div>am able to reproduce this issue on my Debian server too. I tes=
ted it</div><div>because my current laptop is a little bit customized and I=
 don&#39;t trust</div>

<div>it at some points.</div><div><br></div><div>According to <a href=3D"ht=
tp://sourceware.org/bugzilla/show_bug.cgi?id=3D11125">http://sourceware.org=
/bugzilla/show_bug.cgi?id=3D11125</a>, this</div><div>issue has been report=
ed on version 2.11 which is the current Debian GNU</div>

<div>libc distributed version and has been fixed. I just make the last Debi=
an</div><div>update I always have this issue.</div><div><br></div><div>This=
 has been fixed in glibc commit</div><div>cd2f000c074b07931bd78ab5ff5fa3c0f=
7db628a released with the 2.12 glibc</div>

<div>version.</div><div><br></div><div>So, the patch I proposed is clearly =
a Debian squeeze (2.11) workaround.</div><div>I will try to raise a bug rep=
ort to the Debian team to see if they can pick</div><div>this commit in the=
ir branch.</div>

<div><br></div><div>Cheers,</div><div><br></div><div>J=E9r=E9my</div>

--90e6ba21224bab653504b954c6be--


From MAILER-DAEMON Sun Feb 19 12:55:41 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RzAz7-0000UG-2P
	for mharc-coreutils@gnu.org; Sun, 19 Feb 2012 12:55:41 -0500
Received: from eggs.gnu.org ([140.186.70.92]:41574)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1RzAz4-0000TR-Di
	for coreutils@gnu.org; Sun, 19 Feb 2012 12:55:39 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1RzAz3-00048g-3V
	for coreutils@gnu.org; Sun, 19 Feb 2012 12:55:38 -0500
Received: from mx.meyering.net ([88.168.87.75]:46511)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1RzAz2-00048R-Th
	for coreutils@gnu.org; Sun, 19 Feb 2012 12:55:37 -0500
Received: from rho.meyering.net (localhost.localdomain [127.0.0.1])
	by rho.meyering.net (Acme Bit-Twister) with ESMTP id 104B860051;
	Sun, 19 Feb 2012 18:55:36 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: =?iso-8859-1?Q?J=E9r=E9my?= Compostella <jeremy.compostella@gmail.com>
Subject: Re: [PATCH] tests: add missing sys/types.h for ls speed-up test
In-Reply-To: <CA+TATJHYy35NK5ftgCh+7aJodhhgnzn74aHGs3uGz3frU_gbQQ@mail.gmail.com>
	(=?iso-8859-1?Q?=22J=E9r=E9my?= Compostella"'s message of "Sun,
	19 Feb 2012 18:47:28 +0100")
References: <87pqda3mtn.fsf@Apollo.jerryland.fr>
	<877gzidg41.fsf@rho.meyering.net>
	<CA+TATJEUtf0513Cdk-XmCiHBfXGrFPYwRSBJj=jV=tfK79szDw@mail.gmail.com>
	<CA+TATJHYy35NK5ftgCh+7aJodhhgnzn74aHGs3uGz3frU_gbQQ@mail.gmail.com>
Date: Sun, 19 Feb 2012 18:55:35 +0100
Message-ID: <87pqdabtd4.fsf@rho.meyering.net>
Lines: 44
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 88.168.87.75
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 19 Feb 2012 17:55:39 -0000

J=E9r=E9my Compostella wrote:

> Jim,
>
>> > Thank you for the report and the patch.
>> > Please tell us what type/version of compiler/system you're using.
>> > It's good to include that information, especially since ssize_t
>> > is supposed to be defined via stdio.h. =A0Then it is clear that this
>> > change is working around a problem with non-conforming systems.
>> Here the requested information:
>> - linux kernel: Linux Apollo 3.2.0-rc7+ #75 SMP Thu Jan 5 18:14:54 CET
>> =A0 2012 x86_64 GNU/Linux
>> - gcc (Debian 4.4.5-8) 4.4.5
>> - libc6-dev 2.11.2-10
> I made some search. This issue is Debian related (I got another computer
> with a recent Ubuntu and the problem is gone, version 2.13-0ubuntu13). I
> am able to reproduce this issue on my Debian server too. I tested it
> because my current laptop is a little bit customized and I don't trust
> it at some points.
>
> According to http://sourceware.org/bugzilla/show_bug.cgi?id=3D11125, this
> issue has been reported on version 2.11 which is the current Debian GNU
> libc distributed version and has been fixed. I just make the last Debian
> update I always have this issue.
>
> This has been fixed in glibc commit
> cd2f000c074b07931bd78ab5ff5fa3c0f7db628a released with the 2.12 glibc
> version.
>
> So, the patch I proposed is clearly a Debian squeeze (2.11) workaround.
> I will try to raise a bug report to the Debian team to see if they can pi=
ck
> this commit in their branch.

Thanks.
For a more standards-conforming development environment, I recommend the
latest Fedora.  I build/test everything I release on at least Fedora 16
(soon Fedora 17) and Fedora rawhide.  I also test regularly on Debian
unstable and, less frequently, on a few other platforms.

For the record, here are a few distro/version number pairs:

  Fedora Rawhide          glibc-2.15-2
  Fedora 16               glibc-2.14.90-24
  Debian unstable     libc6-dev 2.13-26


From MAILER-DAEMON Sun Feb 19 13:16:26 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RzBJC-00005G-LU
	for mharc-coreutils@gnu.org; Sun, 19 Feb 2012 13:16:26 -0500
Received: from eggs.gnu.org ([140.186.70.92]:42154)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RzBJ9-0008Tm-TJ
	for coreutils@gnu.org; Sun, 19 Feb 2012 13:16:24 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RzBJ9-0007jN-29
	for coreutils@gnu.org; Sun, 19 Feb 2012 13:16:23 -0500
Received: from mail3.vodafone.ie ([213.233.128.45]:49651)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RzBJ8-0007jD-Sg
	for coreutils@gnu.org; Sun, 19 Feb 2012 13:16:23 -0500
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: ApgBAPE7QU9tTFkb/2dsb2JhbAAMNq9NhVoBAQEEDCYBRhALDQsJFg8JAwIBAgFFBg0BBwEBvT6MBikJBgIDBgMHAgkBCQMBAgKHWASbNoxu
Received: from unknown (HELO [192.168.1.79]) ([109.76.89.27])
	by mail3.vodafone.ie with ESMTP; 19 Feb 2012 18:16:20 +0000
Message-ID: <4F413C74.1020104@draigBrady.com>
Date: Sun, 19 Feb 2012 18:16:20 +0000
From: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:6.0) Gecko/20110816 Thunderbird/6.0
MIME-Version: 1.0
To: =?ISO-8859-1?Q?J=E9r=E9my_Compostella?=
 <jeremy.compostella@gmail.com>
Subject: Re: bug#6554: [PATCH] split: Additional suffix for split (bug#6554)
References: <87sjiyd79n.fsf@Apollo.jerryland.fr>
	<CA+TATJFssEZWcfiVBUsKEeazhH8rOoV_cA-kDBucWf73xq-i-A@mail.gmail.com>
	<4F3FB4A9.8080302@draigBrady.com>
	<CA+TATJHRG7uhrma7JtPvc1Xy=fjkwf8kKyVNHd-Hgbod6Nn2nA@mail.gmail.com>
	<4F3FBC5B.3010202@draigBrady.com>
	<CA+TATJFyPFF+tLizCfGbhmEoRyoXodrH2sWDryqKKTMRVNhksw@mail.gmail.com>
In-Reply-To: <CA+TATJFyPFF+tLizCfGbhmEoRyoXodrH2sWDryqKKTMRVNhksw@mail.gmail.com>
X-Enigmail-Version: 1.3.2
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 213.233.128.45
Cc: coreutils@gnu.org, 6554-done@debbugs.gnu.org, Peng Yu <pengyu.ut@gmail.com>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 19 Feb 2012 18:16:24 -0000

On 02/19/2012 12:54 PM, Jérémy Compostella wrote:
> Pádraig,
> 
> I replaced --suffix with --additional-suffix and made some cleaning.
> 
> Feel free to comment the attached patch.

Logic is perfect again.

I adjusted the help output a little like:

-  -a, --suffix-length=N   use suffixes of length N (default %d)\n\
-      --additional-suffix=SUFFIX  append an additional SUFFIX to output file\n\
-                                       names. SUFFIX must not contain slash.\n\
+  -a, --suffix-length=N   generate suffixes of length N (default %d)\n\
+      --additional-suffix=SUFFIX  append an additional SUFFIX to file names.\n\

I'll push later on.

cheers,
Pádraig.


From MAILER-DAEMON Sun Feb 19 13:17:47 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RzBKV-0001TH-5m
	for mharc-coreutils@gnu.org; Sun, 19 Feb 2012 13:17:47 -0500
Received: from eggs.gnu.org ([140.186.70.92]:42338)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1RzBKS-0001T8-Qx
	for coreutils@gnu.org; Sun, 19 Feb 2012 13:17:45 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1RzBKR-0007tC-WC
	for coreutils@gnu.org; Sun, 19 Feb 2012 13:17:44 -0500
Received: from mail-iy0-f169.google.com ([209.85.210.169]:64026)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1RzBKR-0007t7-RV
	for coreutils@gnu.org; Sun, 19 Feb 2012 13:17:43 -0500
Received: by iagz16 with SMTP id z16so8426601iag.0
	for <coreutils@gnu.org>; Sun, 19 Feb 2012 10:17:43 -0800 (PST)
Received-SPF: pass (google.com: domain of jeremy.compostella@gmail.com
	designates 10.42.235.137 as permitted sender)
	client-ip=10.42.235.137; 
Authentication-Results: mr.google.com; spf=pass (google.com: domain of
	jeremy.compostella@gmail.com designates 10.42.235.137 as
	permitted sender) smtp.mail=jeremy.compostella@gmail.com;
	dkim=pass header.i=jeremy.compostella@gmail.com
Received: from mr.google.com ([10.42.235.137])
	by 10.42.235.137 with SMTP id kg9mr18069616icb.47.1329675463391
	(num_hops = 1); Sun, 19 Feb 2012 10:17:43 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma;
	h=mime-version:in-reply-to:references:from:date:message-id:subject:to
	:cc:content-type;
	bh=f+j9tsoqs173rAMSnXf/G3REtUHEUA9OXhEi+aEgVpA=;
	b=Tad9uOa8JWjA+ZXcvl5GrAL2qPa/pB9xDb/nCYbKP1tUy0XjWpZyl6t4+9YYuCIMKF
	DNIkjuDX2BNEBpqHwU66unxOkgkFzmEM6Hv6/uV21jE5U0RTVHp2oH54oNNBUqtirJqX
	2mhr2El2L9Zp0WskWsPD/SOH5whXpXxbBV9gk=
Received: by 10.42.235.137 with SMTP id kg9mr14385231icb.47.1329675463351;
	Sun, 19 Feb 2012 10:17:43 -0800 (PST)
MIME-Version: 1.0
Received: by 10.42.29.196 with HTTP; Sun, 19 Feb 2012 10:17:23 -0800 (PST)
In-Reply-To: <87pqdabtd4.fsf@rho.meyering.net>
References: <87pqda3mtn.fsf@Apollo.jerryland.fr>
	<877gzidg41.fsf@rho.meyering.net>
	<CA+TATJEUtf0513Cdk-XmCiHBfXGrFPYwRSBJj=jV=tfK79szDw@mail.gmail.com>
	<CA+TATJHYy35NK5ftgCh+7aJodhhgnzn74aHGs3uGz3frU_gbQQ@mail.gmail.com>
	<87pqdabtd4.fsf@rho.meyering.net>
From: =?ISO-8859-1?Q?J=E9r=E9my_Compostella?= <jeremy.compostella@gmail.com>
Date: Sun, 19 Feb 2012 19:17:23 +0100
Message-ID: <CA+TATJGA+vY=nYjDwFAYXTUdOLa=1tcU=Vht1xet_pOOjgUQVg@mail.gmail.com>
Subject: Re: [PATCH] tests: add missing sys/types.h for ls speed-up test
To: Jim Meyering <jim@meyering.net>
Content-Type: multipart/alternative; boundary=20cf3026697aa8de9204b95531de
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.85.210.169
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 19 Feb 2012 18:17:45 -0000

--20cf3026697aa8de9204b95531de
Content-Type: text/plain; charset=ISO-8859-1

Thanks for the recommendation. I will consider it.

FYI, I sent a bug report to the Debian team bug#660526.

--20cf3026697aa8de9204b95531de
Content-Type: text/html; charset=ISO-8859-1

<div>Thanks for the recommendation. I will consider it.</div><div><br></div>FYI, I sent a bug report to the Debian team bug#660526.<div><br>
</div>

--20cf3026697aa8de9204b95531de--


From MAILER-DAEMON Sun Feb 19 13:30:43 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RzBX1-0004uB-Cn
	for mharc-coreutils@gnu.org; Sun, 19 Feb 2012 13:30:43 -0500
Received: from eggs.gnu.org ([140.186.70.92]:58428)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1RzBWw-0004qX-FG
	for coreutils@gnu.org; Sun, 19 Feb 2012 13:30:40 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1RzBWv-0001no-AV
	for coreutils@gnu.org; Sun, 19 Feb 2012 13:30:38 -0500
Received: from mail-iy0-f169.google.com ([209.85.210.169]:58256)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1RzBWu-0001ne-TV
	for coreutils@gnu.org; Sun, 19 Feb 2012 13:30:37 -0500
Received: by iagz16 with SMTP id z16so8438930iag.0
	for <coreutils@gnu.org>; Sun, 19 Feb 2012 10:30:36 -0800 (PST)
Received-SPF: pass (google.com: domain of jeremy.compostella@gmail.com
	designates 10.50.89.232 as permitted sender)
	client-ip=10.50.89.232; 
Authentication-Results: mr.google.com; spf=pass (google.com: domain of
	jeremy.compostella@gmail.com designates 10.50.89.232 as
	permitted sender) smtp.mail=jeremy.compostella@gmail.com;
	dkim=pass header.i=jeremy.compostella@gmail.com
Received: from mr.google.com ([10.50.89.232])
	by 10.50.89.232 with SMTP id br8mr8306558igb.30.1329676236313 (num_hops
	= 1); Sun, 19 Feb 2012 10:30:36 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma;
	h=mime-version:in-reply-to:references:from:date:message-id:subject:to
	:cc:content-type;
	bh=U74zCqdm3tI+so82iSKFqA5CpdY/PtsoRl0ZczUsY3I=;
	b=V07bF/62+onMiM78zpBx1u/FdPRw6mjeqoTYI0oCMq3oxr5O7+mjZPthyAwXYrnzD7
	NvTmRvHsoYdsQg6Sf1EnMBo43opmjJmsk6b3pbmNGIRXF3L1gG8rmCGx02LxmMNg2oYE
	9ZYB3lu/ImrqJJWNY34c2b18D7JjWsLpn2l8U=
Received: by 10.50.89.232 with SMTP id br8mr6734308igb.30.1329676236264; Sun,
	19 Feb 2012 10:30:36 -0800 (PST)
MIME-Version: 1.0
Received: by 10.42.29.196 with HTTP; Sun, 19 Feb 2012 10:30:16 -0800 (PST)
In-Reply-To: <4F413C74.1020104@draigBrady.com>
References: <87sjiyd79n.fsf@Apollo.jerryland.fr>
	<CA+TATJFssEZWcfiVBUsKEeazhH8rOoV_cA-kDBucWf73xq-i-A@mail.gmail.com>
	<4F3FB4A9.8080302@draigBrady.com>
	<CA+TATJHRG7uhrma7JtPvc1Xy=fjkwf8kKyVNHd-Hgbod6Nn2nA@mail.gmail.com>
	<4F3FBC5B.3010202@draigBrady.com>
	<CA+TATJFyPFF+tLizCfGbhmEoRyoXodrH2sWDryqKKTMRVNhksw@mail.gmail.com>
	<4F413C74.1020104@draigBrady.com>
From: =?ISO-8859-1?Q?J=E9r=E9my_Compostella?= <jeremy.compostella@gmail.com>
Date: Sun, 19 Feb 2012 19:30:16 +0100
Message-ID: <CA+TATJFZuYkzKRhkYBDWWFJJSc1PEGNM3dVXFRPRS3Jpj0q-FA@mail.gmail.com>
Subject: Re: bug#6554: [PATCH] split: Additional suffix for split (bug#6554)
To: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigbrady.com>
Content-Type: multipart/alternative; boundary=e89a8f3ba429ba96aa04b9555fe2
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.85.210.169
Cc: coreutils@gnu.org, 6554-done@debbugs.gnu.org, Peng Yu <pengyu.ut@gmail.com>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 19 Feb 2012 18:30:40 -0000

--e89a8f3ba429ba96aa04b9555fe2
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

> On 02/19/2012 12:54 PM, J=E9r=E9my Compostella wrote:
> > P=E1draig,
> >
> > I replaced --suffix with --additional-suffix and made some cleaning.
> >
> > Feel free to comment the attached patch.
>
> Logic is perfect again.
Thanks :)

> I adjusted the help output a little like:
>
> -  -a, --suffix-length=3DN   use suffixes of length N (default %d)\n\
> -      --additional-suffix=3DSUFFIX  append an additional SUFFIX to outpu=
t
file\n\
> -                                       names. SUFFIX must not contain
slash.\n\
> +  -a, --suffix-length=3DN   generate suffixes of length N (default %d)\n=
\
> +      --additional-suffix=3DSUFFIX  append an additional SUFFIX to file
names.\n\
I'm just wondering why did you removed the slash warning ? This warning
is maybe too much in the command usage but IMHO the man page should
mention it. Moreover, the mktemp mention this "restriction" on its
--suffix option this way.

Cheers,

J=E9r=E9my

--e89a8f3ba429ba96aa04b9555fe2
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div>&gt; On 02/19/2012 12:54 PM, J=E9r=E9my Compostella wrote:</div><div>&=
gt; &gt; P=E1draig,</div><div>&gt; &gt;</div><div>&gt; &gt; I replaced --su=
ffix with --additional-suffix and made some cleaning.</div><div>&gt; &gt;</=
div>

<div>&gt; &gt; Feel free to comment the attached patch.</div><div>&gt;=A0</=
div><div>&gt; Logic is perfect again.</div><div>Thanks :)</div><div><br></d=
iv><div>&gt; I adjusted the help output a little like:</div><div>&gt;=A0</d=
iv>

<div>&gt; - =A0-a, --suffix-length=3DN =A0 use suffixes of length N (defaul=
t %d)\n\</div><div>&gt; - =A0 =A0 =A0--additional-suffix=3DSUFFIX =A0append=
 an additional SUFFIX to output file\n\</div><div>&gt; - =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 names. SUFFIX m=
ust not contain slash.\n\</div>

<div>&gt; + =A0-a, --suffix-length=3DN =A0 generate suffixes of length N (d=
efault %d)\n\</div><div>&gt; + =A0 =A0 =A0--additional-suffix=3DSUFFIX =A0a=
ppend an additional SUFFIX to file names.\n\</div><div>I&#39;m just wonderi=
ng why did you removed the slash warning ? This warning</div>

<div>is maybe too much in the command usage but IMHO the man page should</d=
iv><div>mention it. Moreover, the mktemp mention this &quot;restriction&quo=
t; on its</div><div>--suffix option this way.</div><div><br></div><div>

Cheers,</div><div><br></div><div>J=E9r=E9my</div>

--e89a8f3ba429ba96aa04b9555fe2--


From MAILER-DAEMON Sun Feb 19 15:17:01 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RzDBt-0003nC-Gv
	for mharc-coreutils@gnu.org; Sun, 19 Feb 2012 15:17:01 -0500
Received: from eggs.gnu.org ([140.186.70.92]:53597)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RzDBr-0003ms-FH
	for coreutils@gnu.org; Sun, 19 Feb 2012 15:17:00 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RzDBn-00011x-Ms
	for coreutils@gnu.org; Sun, 19 Feb 2012 15:16:59 -0500
Received: from mail3.vodafone.ie ([213.233.128.45]:28487)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RzDBn-00011l-F5
	for coreutils@gnu.org; Sun, 19 Feb 2012 15:16:55 -0500
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: ApgBABdYQU9tTFkb/2dsb2JhbAAMN69NhVoBAQEEDCYBRhALDQsJFg8JAwIBAgFFBg0BBwEBvUmMBgkgCQYCAwYDBwIJAQwDAodYBJs2jG4
Received: from unknown (HELO [192.168.1.79]) ([109.76.89.27])
	by mail3.vodafone.ie with ESMTP; 19 Feb 2012 20:16:53 +0000
Message-ID: <4F4158B4.2040201@draigBrady.com>
Date: Sun, 19 Feb 2012 20:16:52 +0000
From: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:6.0) Gecko/20110816 Thunderbird/6.0
MIME-Version: 1.0
To: =?ISO-8859-1?Q?J=E9r=E9my_Compostella?=
 <jeremy.compostella@gmail.com>
Subject: Re: bug#6554: [PATCH] split: Additional suffix for split (bug#6554)
References: <87sjiyd79n.fsf@Apollo.jerryland.fr>
	<CA+TATJFssEZWcfiVBUsKEeazhH8rOoV_cA-kDBucWf73xq-i-A@mail.gmail.com>
	<4F3FB4A9.8080302@draigBrady.com>
	<CA+TATJHRG7uhrma7JtPvc1Xy=fjkwf8kKyVNHd-Hgbod6Nn2nA@mail.gmail.com>
	<4F3FBC5B.3010202@draigBrady.com>
	<CA+TATJFyPFF+tLizCfGbhmEoRyoXodrH2sWDryqKKTMRVNhksw@mail.gmail.com>
	<4F413C74.1020104@draigBrady.com>
	<CA+TATJFZuYkzKRhkYBDWWFJJSc1PEGNM3dVXFRPRS3Jpj0q-FA@mail.gmail.com>
In-Reply-To: <CA+TATJFZuYkzKRhkYBDWWFJJSc1PEGNM3dVXFRPRS3Jpj0q-FA@mail.gmail.com>
X-Enigmail-Version: 1.3.2
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 213.233.128.45
Cc: coreutils@gnu.org, 6554-done@debbugs.gnu.org, Peng Yu <pengyu.ut@gmail.com>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 19 Feb 2012 20:17:00 -0000

On 02/19/2012 06:30 PM, Jérémy Compostella wrote:
>> On 02/19/2012 12:54 PM, Jérémy Compostella wrote:
>> > Pádraig,
>> >
>> > I replaced --suffix with --additional-suffix and made some cleaning.
>> >
>> > Feel free to comment the attached patch.
>> 
>> Logic is perfect again.
> Thanks :)
> 
>> I adjusted the help output a little like:
>> 
>> -  -a, --suffix-length=N   use suffixes of length N (default %d)\n\
>> -      --additional-suffix=SUFFIX  append an additional SUFFIX to output file\n\
>> -                                       names. SUFFIX must not contain slash.\n\
>> +  -a, --suffix-length=N   generate suffixes of length N (default %d)\n\
>> +      --additional-suffix=SUFFIX  append an additional SUFFIX to file names.\n\
> I'm just wondering why did you removed the slash warning ? This warning
> is maybe too much in the command usage but IMHO the man page should
> mention it. Moreover, the mktemp mention this "restriction" on its
> --suffix option this way.

I thought it wasn't worth mentioning, especially since
it messed up the output alignment a bit.
Who is going to use a slash?
the .0001% who do will immediately get an error anyway.

cheers,
Pádraig.


From MAILER-DAEMON Sun Feb 19 17:22:49 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RzF9d-0008TD-Ot
	for mharc-coreutils@gnu.org; Sun, 19 Feb 2012 17:22:49 -0500
Received: from eggs.gnu.org ([140.186.70.92]:38873)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <sven.breuner@itwm.fraunhofer.de>) id 1RzF9a-0008Qq-N1
	for coreutils@gnu.org; Sun, 19 Feb 2012 17:22:47 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <sven.breuner@itwm.fraunhofer.de>) id 1RzF9Z-0004dL-8o
	for coreutils@gnu.org; Sun, 19 Feb 2012 17:22:46 -0500
Received: from mailgw1.uni-kl.de ([131.246.120.220]:58444)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <sven.breuner@itwm.fraunhofer.de>) id 1RzF9Y-0004dD-VK
	for coreutils@gnu.org; Sun, 19 Feb 2012 17:22:45 -0500
Received: from itwm2.itwm.fhg.de (itwm2.itwm.fhg.de [131.246.191.3])
	by mailgw1.uni-kl.de (8.14.3/8.14.3/Debian-5+lenny1) with ESMTP id
	q1JMMfUA008036
	(version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=NOT)
	for <coreutils@gnu.org>; Sun, 19 Feb 2012 23:22:41 +0100
Received: from mail2.itwm.fhg.de ([131.246.191.79]:40630)
	by itwm2.itwm.fhg.de with esmtps (TLSv1:DES-CBC3-SHA:168)
	(/C=DE/ST=Rheinland-Pfalz/L=Kaiserslautern/O=Fraunhofer
	ITWM/OU=SLG/CN=mail2.itwm.fhg.de)(verified=1) (Exim 4.74 #1)
	id 1RzF9S-0001bd-OK; Sun, 19 Feb 2012 23:22:38 +0100
Message-ID: <4F41762E.1030801@itwm.fraunhofer.de>
Date: Sun, 19 Feb 2012 23:22:38 +0100
From: Sven Breuner <sven.breuner@itwm.fraunhofer.de>
User-Agent: Mozilla/5.0 (X11; Linux i686 on x86_64;
	rv:10.0.2) Gecko/20120216 Thunderbird/10.0.2
MIME-Version: 1.0
To: Jim Meyering <jim@meyering.net>
Subject: Re: "ls -l": Avoid unnecessary getxattr() overhead
References: <87pqds3snt.fsf@Apollo.jerryland.fr>
	<4F30206B.1020802@itwm.fraunhofer.de>
	<CA+TATJHorgsuY7pxmpxxAeWUHRP5MfFfzQP05o=1H6C4ttswdA@mail.gmail.com>
	<4F3032BC.50702@draigBrady.com>
	<4F30631A.2070009@itwm.fraunhofer.de>
	<87y5scfdiq.fsf@rho.meyering.net> <4F33B5D7.3010909@draigBrady.com>
	<87mx8sf5w0.fsf@rho.meyering.net>
	<4F3437CE.4050308@itwm.fraunhofer.de>
	<4F346CC7.4030608@draigBrady.com> <87r4y3dry0.fsf@rho.meyering.net>
	<4F3710CA.8020402@itwm.fraunhofer.de>
	<8739aao9e2.fsf@rho.meyering.net>
	<4F3E12F9.6080102@bernhard-voelker.de>
	<8762f5lpct.fsf@rho.meyering.net>
	<4F3E35C9.7050707@bernhard-voelker.de>
	<87mx8hk6bm.fsf@rho.meyering.net>
	<4F4069D3.5000806@itwm.fraunhofer.de>
	<87pqdbcia0.fsf@rho.meyering.net> <4F40D061.3000206@draigBrady.com>
	<871upqdagt.fsf@rho.meyering.net>
In-Reply-To: <871upqdagt.fsf@rho.meyering.net>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
X-ITWM-CharSet: ISO-8859-1
X-ITWM-Scanned-By: mail2.itwm.fhg.de
Content-Transfer-Encoding: quoted-printable
X-MIME-Autoconverted: from 8bit to quoted-printable by mailgw1.uni-kl.de id
	q1JMMfUA008036
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
X-Received-From: 131.246.120.220
Cc: "Daniel P. Berrange" <berrange@redhat.com>,
	Bernhard Voelker <mail@bernhard-voelker.de>,
	Bernd Schubert <bernd.schubert@itwm.fraunhofer.de>, coreutils@gnu.org,
	=?ISO-8859-1?Q?J=E9r?= =?ISO-8859-1?Q?=E9my_Compostella?=
	<jeremy.compostella@gmail.com>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 19 Feb 2012 22:22:47 -0000

Jim Meyering wrote on 02/19/2012 06:00 PM:
> P=E1draig Brady wrote:
>> > From Sven's strace it seems that lgetfilecon() is interpreting
>> the "unlabeled" return from lgetxattr() and converting that
>> into a -1 return with ENODATA?  That's the only logical explanation
>> I can see for how ENODATA is significant. As mentioned before
>> I'm not sure we could add ENODATA to errno_unsupported().
>
> Right.  We definitely don't want to add ENODATA.
> I removed it for this sort of reason.
>
> The gnulib getfilecon wrapper maps 10,"unlabeled" to -1,ENODATA.

Is there maybe another check that could be used to find out if selinux=20
is disabled? (e.g. is_selinux_enabled(3) )

> Back when I wrote that wrapper, it sounded like 10,"unlabeled"
> would happen only with kernels deemed crufty back then (in 2009).
>
>    http://thread.gmane.org/gmane.comp.gnu.coreutils.bugs/18378/focus=3D=
18384
>
> That we're still seeing such syscall traces in 2012...

I guess that means it's more or less intended that userspace does get=20
ENODATA or "unlabled" instead of EOPNOTSUPP - to show that the system=20
uses a kernel which is generally selinux compatible, even though selinux=20
is disabled.

On the other hand, all the scientific compute clusters I know have=20
selinux permanently disabled, because they just don't want to have=20
anything to do with it. Since these are typically the guys with the=20
large directories, I would say it's not right to let them wait longer=20
for their "ls -l" result, just because their kernel is generally selinux=20
compatible.

So would skipping these calls for "ls -l" be an option on a system with=20
disabled selinux? I assume the people who want to check their selinux=20
labels could use "ls -Z" or have other tools available to check this=20
information. And those people probably wouldn't set selinux disabled,=20
but rather use permissive mode instead.

Best regards,
Sven


From MAILER-DAEMON Sun Feb 19 18:54:31 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RzGaN-00065i-2I
	for mharc-coreutils@gnu.org; Sun, 19 Feb 2012 18:54:31 -0500
Received: from eggs.gnu.org ([140.186.70.92]:43164)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RzGaJ-00065U-Kz
	for coreutils@gnu.org; Sun, 19 Feb 2012 18:54:29 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RzGaH-0002Hu-Pu
	for coreutils@gnu.org; Sun, 19 Feb 2012 18:54:27 -0500
Received: from mail3.vodafone.ie ([213.233.128.45]:65214)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RzGaH-0002Ha-LS
	for coreutils@gnu.org; Sun, 19 Feb 2012 18:54:25 -0500
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: ApMBAISKQU9tTFkb/2dsb2JhbAAMOLUjAQEBAwEyAUYFCwsNCwklDwJGBg0BBwEBh3wJuGCLfwwODggJBgIDBgMHAgkBEYQUg0QEmzaMbg
Received: from unknown (HELO [192.168.1.79]) ([109.76.89.27])
	by mail3.vodafone.ie with ESMTP; 19 Feb 2012 23:54:22 +0000
Message-ID: <4F418BAE.20703@draigBrady.com>
Date: Sun, 19 Feb 2012 23:54:22 +0000
From: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:6.0) Gecko/20110816 Thunderbird/6.0
MIME-Version: 1.0
To: Sven Breuner <sven.breuner@itwm.fraunhofer.de>
Subject: Re: "ls -l": Avoid unnecessary getxattr() overhead
References: <87pqds3snt.fsf@Apollo.jerryland.fr>
	<4F30206B.1020802@itwm.fraunhofer.de>
	<CA+TATJHorgsuY7pxmpxxAeWUHRP5MfFfzQP05o=1H6C4ttswdA@mail.gmail.com>
	<4F3032BC.50702@draigBrady.com>
	<4F30631A.2070009@itwm.fraunhofer.de>
	<87y5scfdiq.fsf@rho.meyering.net> <4F33B5D7.3010909@draigBrady.com>
	<87mx8sf5w0.fsf@rho.meyering.net>
	<4F3437CE.4050308@itwm.fraunhofer.de>
	<4F346CC7.4030608@draigBrady.com> <87r4y3dry0.fsf@rho.meyering.net>
	<4F3710CA.8020402@itwm.fraunhofer.de>
	<8739aao9e2.fsf@rho.meyering.net>
	<4F3E12F9.6080102@bernhard-voelker.de>
	<8762f5lpct.fsf@rho.meyering.net>
	<4F3E35C9.7050707@bernhard-voelker.de>
	<87mx8hk6bm.fsf@rho.meyering.net>
	<4F4069D3.5000806@itwm.fraunhofer.de>
	<87pqdbcia0.fsf@rho.meyering.net> <4F40D061.3000206@draigBrady.com>
	<871upqdagt.fsf@rho.meyering.net>
	<4F41762E.1030801@itwm.fraunhofer.de>
In-Reply-To: <4F41762E.1030801@itwm.fraunhofer.de>
X-Enigmail-Version: 1.3.2
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 213.233.128.45
Cc: "Daniel P. Berrange" <berrange@redhat.com>,
	Bernhard Voelker <mail@bernhard-voelker.de>,
	Bernd Schubert <bernd.schubert@itwm.fraunhofer.de>, coreutils@gnu.org,
	=?ISO-8859-1?Q?J=E9r?= =?ISO-8859-1?Q?=E9my_Compostella?=
	<jeremy.compostella@gmail.com>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 19 Feb 2012 23:54:29 -0000

On 02/19/2012 10:22 PM, Sven Breuner wrote:
> Jim Meyering wrote on 02/19/2012 06:00 PM:
>> Pádraig Brady wrote:
>>> > From Sven's strace it seems that lgetfilecon() is interpreting
>>> the "unlabeled" return from lgetxattr() and converting that
>>> into a -1 return with ENODATA?  That's the only logical explanation
>>> I can see for how ENODATA is significant. As mentioned before
>>> I'm not sure we could add ENODATA to errno_unsupported().
>>
>> Right.  We definitely don't want to add ENODATA.
>> I removed it for this sort of reason.
>>
>> The gnulib getfilecon wrapper maps 10,"unlabeled" to -1,ENODATA.
> 
> Is there maybe another check that could be used to find out if selinux is disabled? (e.g. is_selinux_enabled(3) )

I'm not sure you could use that.
Even though it's currently disabled,
you might be surprised that ls didn't
indicate selinux attributes as normal.

> 
>> Back when I wrote that wrapper, it sounded like 10,"unlabeled"
>> would happen only with kernels deemed crufty back then (in 2009).
>>
>>    http://thread.gmane.org/gmane.comp.gnu.coreutils.bugs/18378/focus=18384
>>
>> That we're still seeing such syscall traces in 2012...
> 
> I guess that means it's more or less intended that userspace does get ENODATA or "unlabled" instead of EOPNOTSUPP - to show that the system uses a kernel which is generally selinux compatible, even though selinux is disabled.

In the referenced thread above, there is a 2006 kernel patch
which seems to remove this "unlabeled" thing.
Though that doesn't seem to correlate with your testing against ~/tmp/ls_20120209.

I just booted RHEL 6.2 with selinux=0 on the kernel command line,
so as I could test new files without a context created, and...

getxattr("security.selinux") on existing files on ext4, returns the existing attribute
getxattr("security.selinux") on new files on ext4, returns ENODATA
getxattr("security.selinux") on new files on tmpfs, returns ENOTSUP

So is this "unlabeled" coming from some really old kernel?
I thought you were on RHEL 6.2? Actually I went back to your
original report (I presume also on RHEL 6.2) and saw:

lgetxattr("test", "security.selinux", 0x62d130, 255) = -1 ENODATA (No data available)

I.E. "unlabeled" was not returned there either.
What's changed?

In any case "unlabeled" is mapped to ENODATA which is correct I think,
and ENODATA can't be taken as device wide as stated before.

> On the other hand, all the scientific compute clusters I know have selinux permanently disabled, because they just don't want to have anything to do with it. Since these are typically the guys with the large directories, I would say it's not right to let them wait longer for their "ls -l" result, just because their kernel is generally selinux compatible.
> 
> So would skipping these calls for "ls -l" be an option on a system with disabled selinux? I assume the people who want to check their selinux labels could use "ls -Z" or have other tools available to check this information. And those people probably wouldn't set selinux disabled, but rather use permissive mode instead.

So the options we have now are:

1. If the file system really doesn't support selinux attributes, it should return ENOTSUP,
like tmpfs does above. That will trigger the cache in the existing code.

2. We may in future support a --format option in ls, to give greater
control over what is output (and thus queried).

cheers,
Pádraig.


From MAILER-DAEMON Sun Feb 19 21:06:31 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RzIe7-0007h3-Nh
	for mharc-coreutils@gnu.org; Sun, 19 Feb 2012 21:06:31 -0500
Received: from eggs.gnu.org ([140.186.70.92]:55115)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <sven.breuner@itwm.fraunhofer.de>) id 1RzIe5-0007fn-40
	for coreutils@gnu.org; Sun, 19 Feb 2012 21:06:30 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <sven.breuner@itwm.fraunhofer.de>) id 1RzIe3-0006j5-O7
	for coreutils@gnu.org; Sun, 19 Feb 2012 21:06:29 -0500
Received: from mailgw1.uni-kl.de ([131.246.120.220]:39578)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <sven.breuner@itwm.fraunhofer.de>) id 1RzIe3-0006iz-D3
	for coreutils@gnu.org; Sun, 19 Feb 2012 21:06:27 -0500
Received: from itwm2.itwm.fhg.de (itwm2.itwm.fhg.de [131.246.191.3])
	by mailgw1.uni-kl.de (8.14.3/8.14.3/Debian-5+lenny1) with ESMTP id
	q1K26O4D013006
	(version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=NOT)
	for <coreutils@gnu.org>; Mon, 20 Feb 2012 03:06:24 +0100
Received: from mail2.itwm.fhg.de ([131.246.191.79]:41304)
	by itwm2.itwm.fhg.de with esmtps (TLSv1:DES-CBC3-SHA:168)
	(/C=DE/ST=Rheinland-Pfalz/L=Kaiserslautern/O=Fraunhofer
	ITWM/OU=SLG/CN=mail2.itwm.fhg.de)(verified=1) (Exim 4.74 #1)
	id 1RzIdx-0002VL-KG; Mon, 20 Feb 2012 03:06:21 +0100
Message-ID: <4F41AA9D.2060007@itwm.fraunhofer.de>
Date: Mon, 20 Feb 2012 03:06:21 +0100
From: Sven Breuner <sven.breuner@itwm.fraunhofer.de>
User-Agent: Mozilla/5.0 (X11; Linux i686 on x86_64;
	rv:10.0.2) Gecko/20120216 Thunderbird/10.0.2
MIME-Version: 1.0
To: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
Subject: Re: "ls -l": Avoid unnecessary getxattr() overhead
References: <87pqds3snt.fsf@Apollo.jerryland.fr>
	<4F30206B.1020802@itwm.fraunhofer.de>
	<CA+TATJHorgsuY7pxmpxxAeWUHRP5MfFfzQP05o=1H6C4ttswdA@mail.gmail.com>
	<4F3032BC.50702@draigBrady.com>
	<4F30631A.2070009@itwm.fraunhofer.de>
	<87y5scfdiq.fsf@rho.meyering.net> <4F33B5D7.3010909@draigBrady.com>
	<87mx8sf5w0.fsf@rho.meyering.net>
	<4F3437CE.4050308@itwm.fraunhofer.de>
	<4F346CC7.4030608@draigBrady.com> <87r4y3dry0.fsf@rho.meyering.net>
	<4F3710CA.8020402@itwm.fraunhofer.de>
	<8739aao9e2.fsf@rho.meyering.net>
	<4F3E12F9.6080102@bernhard-voelker.de>
	<8762f5lpct.fsf@rho.meyering.net>
	<4F3E35C9.7050707@bernhard-voelker.de>
	<87mx8hk6bm.fsf@rho.meyering.net>
	<4F4069D3.5000806@itwm.fraunhofer.de>
	<87pqdbcia0.fsf@rho.meyering.net> <4F40D061.3000206@draigBrady.com>
	<871upqdagt.fsf@rho.meyering.net>
	<4F41762E.1030801@itwm.fraunhofer.de>
	<4F418BAE.20703@draigBrady.com>
In-Reply-To: <4F418BAE.20703@draigBrady.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
X-ITWM-CharSet: ISO-8859-1
X-ITWM-Scanned-By: mail2.itwm.fhg.de
Content-Transfer-Encoding: quoted-printable
X-MIME-Autoconverted: from 8bit to quoted-printable by mailgw1.uni-kl.de id
	q1K26O4D013006
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
X-Received-From: 131.246.120.220
Cc: "Daniel P. Berrange" <berrange@redhat.com>,
	Bernhard Voelker <mail@bernhard-voelker.de>,
	Bernd Schubert <bernd.schubert@itwm.fraunhofer.de>, coreutils@gnu.org,
	=?ISO-8859-1?Q?J=E9r?= =?ISO-8859-1?Q?=E9my_Compostella?=
	<jeremy.compostella@gmail.com>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 20 Feb 2012 02:06:30 -0000

P=E1draig Brady wrote on 02/20/2012 12:54 AM:
> On 02/19/2012 10:22 PM, Sven Breuner wrote:
>> Is there maybe another check that could be used to find out if selinux=
 is disabled? (e.g. is_selinux_enabled(3) )
>
> I'm not sure you could use that.
> Even though it's currently disabled,
> you might be surprised that ls didn't
> indicate selinux attributes as normal.

Agreed, changing behavior that someone might be relying on wouldn't be=20
good. So I would say the question is whether someone with disabled=20
selinux would really rely on ls reporting selinux attributes in the "ls=20
-l" output.


>>> Back when I wrote that wrapper, it sounded like 10,"unlabeled"
>>> would happen only with kernels deemed crufty back then (in 2009).
>>>
>>>     http://thread.gmane.org/gmane.comp.gnu.coreutils.bugs/18378/focus=
=3D18384
>>>
>>> That we're still seeing such syscall traces in 2012...
>>
>> I guess that means it's more or less intended that userspace does get =
ENODATA or "unlabled" instead of EOPNOTSUPP - to show that the system use=
s a kernel which is generally selinux compatible, even though selinux is =
disabled.
>
> In the referenced thread above, there is a 2006 kernel patch
> which seems to remove this "unlabeled" thing.
> Though that doesn't seem to correlate with your testing against ~/tmp/l=
s_20120209.

The way I understood it, that patch should just avoid storing those=20
attributes on disk. It wasn't intended to completely remove the=20
existence of "unlabled".

$ git show 8aad38752e81d1d4de67e3d8e2524618ce7c9276
[...]
SELinux always maintains a safe internal label if there is none, so with=20
this patch, we just stick with that and wait until a policy is loaded=20
before adding a persistent label on disk.
[...]


> I just booted RHEL 6.2 with selinux=3D0 on the kernel command line,
> so as I could test new files without a context created, and...
>
> getxattr("security.selinux") on existing files on ext4, returns the exi=
sting attribute
> getxattr("security.selinux") on new files on ext4, returns ENODATA
> getxattr("security.selinux") on new files on tmpfs, returns ENOTSUP
>
> So is this "unlabeled" coming from some really old kernel?
> I thought you were on RHEL 6.2? Actually I went back to your
> original report (I presume also on RHEL 6.2) and saw:
>
> lgetxattr("test", "security.selinux", 0x62d130, 255) =3D -1 ENODATA (No=
 data available)
>
> I.E. "unlabeled" was not returned there either.
> What's changed?

Seems to be inconsistent. Some of our RHEL 6.x systems return "unlabled"=20
on a tmpfs:

a) RHEL 6.1...
$ uname -r
2.6.32-131.21.1.el6.x86_64
$ sestatus
SELinux status:                 disabled
$ mount -t tmpfs none /mnt/tmp
$ touch /mnt/tmp/file
$ strace ls -l /mnt/tmp/file 2>&1 | grep security
lgetxattr("/mnt/tmp/file", "security.selinux", "unlabeled", 255) =3D 10

b) Same "unlabeled" result for another machine with RHEL 6.2 kernel=20
2.6.32-220.4.1.el6.x86_64.

c) But then the third machine with RHEL kernel 2.6.32-220.2.1.el6.x86_64=20
shows EOPNOTSUPP in the same tmpfs test case:
$ strace ls -l /mnt/tmp/file 2>&1 | grep security
lgetxattr("/mnt/tmp/file", "security.selinux", 0x623300, 255) =3D -1=20
EOPNOTSUPP (Operation not supported)

All of these systems are fresh RHEL 6 installs, which never ran on an=20
old kernel and never had selinux enabled (though that probably shouldn't=20
make a difference on a tmpfs).

The fhgfs result in these cases is similar to the tmpfs result:=20
"unlabled" on machines a/b and EOPNOTSUPP on machine c.

Strange. The only thing I can conclude from these results is that there=20
is a difference in the selinux configuration of these machines that is=20
effective even though selinux is disabled. (Just don't know where to=20
look for that difference to verify this assumption.)

If we assume that there is something wrong with machines a/b and all=20
those systems actually should be returning EOPNOTSUPP in a "normal"=20
setup, then the current cache implementation would work perfectly fine,=20
of course.

Best regards,
Sven


From MAILER-DAEMON Mon Feb 20 03:32:38 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RzOfm-00079B-1c
	for mharc-coreutils@gnu.org; Mon, 20 Feb 2012 03:32:38 -0500
Received: from eggs.gnu.org ([140.186.70.92]:47534)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1RzOfe-00077P-GA
	for coreutils@gnu.org; Mon, 20 Feb 2012 03:32:35 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1RzOfd-00051x-8G
	for coreutils@gnu.org; Mon, 20 Feb 2012 03:32:30 -0500
Received: from moutng.kundenserver.de ([212.227.17.8]:58966)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1RzOfc-00050i-Sf
	for coreutils@gnu.org; Mon, 20 Feb 2012 03:32:29 -0500
Received: from [192.168.2.108] (p4FF77ECC.dip.t-dialin.net [79.247.126.204])
	by mrelayeu.kundenserver.de (node=mrbap1) with ESMTP (Nemesis)
	id 0M6P87-1SNYQ74AJv-00y4r6; Mon, 20 Feb 2012 09:32:25 +0100
Message-ID: <4F420517.2050804@bernhard-voelker.de>
Date: Mon, 20 Feb 2012 09:32:23 +0100
From: Bernhard Voelker <mail@bernhard-voelker.de>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:10.0.1) Gecko/20120208 Thunderbird/10.0.1
MIME-Version: 1.0
To: Jim Meyering <jim@meyering.net>
Subject: Re: "ls -l": Avoid unnecessary getxattr() overhead
References: <87pqds3snt.fsf@Apollo.jerryland.fr>
	<CA+TATJHorgsuY7pxmpxxAeWUHRP5MfFfzQP05o=1H6C4ttswdA@mail.gmail.com>
	<4F3032BC.50702@draigBrady.com>
	<4F30631A.2070009@itwm.fraunhofer.de>
	<87y5scfdiq.fsf@rho.meyering.net> <4F33B5D7.3010909@draigBrady.com>
	<87mx8sf5w0.fsf@rho.meyering.net>
	<4F3437CE.4050308@itwm.fraunhofer.de>
	<4F346CC7.4030608@draigBrady.com> <87r4y3dry0.fsf@rho.meyering.net>
	<4F3710CA.8020402@itwm.fraunhofer.de>
	<8739aao9e2.fsf@rho.meyering.net>
	<4F3E12F9.6080102@bernhard-voelker.de>
	<8762f5lpct.fsf@rho.meyering.net> <4F3E5BBD.80809@redhat.com>
	<874nupk1cx.fsf@rho.meyering.net> <4F3E5E8E.2040802@redhat.com>
	<87lio1ij8b.fsf@rho.meyering.net>
	<4F3E7B1E.3090300@bernhard-voelker.de>
	<8739a9ig96.fsf@rho.meyering.net>
	<4F3EF935.2070700@bernhard-voelker.de>
	<878vk0gujb.fsf@rho.meyering.net>
In-Reply-To: <878vk0gujb.fsf@rho.meyering.net>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
X-Provags-ID: V02:K0:n2361COKU2Elre4Ptps+wfsAKq8BflTDtBdaTBd7v+V
	keG+C3OE34PXLU9c2cOe5NpELMzSDjd0ZUnqk6My6yiq1+j9+m
	sJ7nHoim3QbkoCzZOyUh+OmFsqtkAbYehzLtLnAhWjZ19uHABf
	1ULLwCeKgDsSe1awod6aQZ+uvOqwSBPSAc/Z07lBKZpgNZtMG2
	D4dkYbCBZPn7KIpXC4zVaZtxFMcLYv/xb8iNr8Q2+KHXOxjCw3
	fGJ3OzCFVgORPPNVKrb44xKEq6TZgcrLEV4HVn6vLa3BhqbPIn
	6xWaxfTm5snH3ylIofx8Y66twska4u4oyzv5L9gBW51y8URd6p
	Ab99+PBcvFyP3694TT89oKsp148p5/1th/sKASOKq
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 212.227.17.8
Cc: Bernd Schubert <bernd.schubert@itwm.fraunhofer.de>,
	Sven Breuner <sven.breuner@itwm.fraunhofer.de>,
	=?ISO-8859-1?Q?J=E9r=E9my_Compostella?= <jeremy.compostella@gmail.com>,
	"Daniel P. Berrange" <berrange@redhat.com>, coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 20 Feb 2012 08:32:36 -0000

On 02/18/2012 02:06 PM, Jim Meyering wrote:
>> > See the attached getxattr-speedup.log.gz - I added a "cat k.c" so that
>> > you can check that I got your patch right.
>> > It seems that GCC doesn't like our "__attribute__((destructor)) void p()".
>> > It basically works in a standalone program, but not with LD_PRELOAD.
>> > I didn't find much about this. Can this be a 64-bit issue?
>
> Is that an open-suse system?
> I tried on an opensuse system with 2.6.37.6-0.11-desktop,
> and since there were no getxattr calls at all, no wrapper
> would run, and hence, your atexit call would never be reached.
> I've added this comment at the top:

Yes, it is a OpenSuSE 12.1 with kernel 3.1.9-1.4-desktop.

>   # This test is skipped on systems that lack LD_PRELOAD support; that's fine.

It doesn't lack LD_PRELOAD support, it simply doesn't call the destructor.


> + # Similarly, on a system that lacks getxattr altogether, skipping it is fine.
> 
> and this new skip_ use below:
> 
> test -f x || skip_ "internal test failure: maybe LD_PRELOAD doesn't work?"
> 
> Which may mean this test will simply be skipped on your system.
> But that is fine.

Maybe, but why is LD_PRELOAD honoring the "__attribute__((destructor))",
and on some systems not?

> +ld -G k.o -lc -o k.so || framework_failure_ 'failed to invoke ld -G ...'

I just added "-lc" for pulling in atexit(). It's not needed in your version.

Have a nice day,
Berny


From MAILER-DAEMON Mon Feb 20 05:55:28 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RzQu0-0004tC-Do
	for mharc-coreutils@gnu.org; Mon, 20 Feb 2012 05:55:28 -0500
Received: from eggs.gnu.org ([140.186.70.92]:59455)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RzQtt-0004sq-DN
	for coreutils@gnu.org; Mon, 20 Feb 2012 05:55:27 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RzQth-0006uz-Md
	for coreutils@gnu.org; Mon, 20 Feb 2012 05:55:21 -0500
Received: from mail3.vodafone.ie ([213.233.128.45]:54724)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RzQth-0006t4-CD
	for coreutils@gnu.org; Mon, 20 Feb 2012 05:55:09 -0500
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: ApIBAF0mQk9tTDS1/2dsb2JhbAAMOLRIAQEBBDIBOwsQCw0LCSUPAkYGDQEHAQGIBblmi38JERYJBgIDBgMHAgkBEYQUg0QEmzaMbg
Received: from unknown (HELO [192.168.1.79]) ([109.76.52.181])
	by mail3.vodafone.ie with ESMTP; 20 Feb 2012 10:54:56 +0000
Message-ID: <4F42267F.60205@draigBrady.com>
Date: Mon, 20 Feb 2012 10:54:55 +0000
From: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:6.0) Gecko/20110816 Thunderbird/6.0
MIME-Version: 1.0
To: Sven Breuner <sven.breuner@itwm.fraunhofer.de>
Subject: Re: "ls -l": Avoid unnecessary getxattr() overhead
References: <87pqds3snt.fsf@Apollo.jerryland.fr>
	<4F30206B.1020802@itwm.fraunhofer.de>
	<CA+TATJHorgsuY7pxmpxxAeWUHRP5MfFfzQP05o=1H6C4ttswdA@mail.gmail.com>
	<4F3032BC.50702@draigBrady.com>
	<4F30631A.2070009@itwm.fraunhofer.de>
	<87y5scfdiq.fsf@rho.meyering.net> <4F33B5D7.3010909@draigBrady.com>
	<87mx8sf5w0.fsf@rho.meyering.net>
	<4F3437CE.4050308@itwm.fraunhofer.de>
	<4F346CC7.4030608@draigBrady.com> <87r4y3dry0.fsf@rho.meyering.net>
	<4F3710CA.8020402@itwm.fraunhofer.de>
	<8739aao9e2.fsf@rho.meyering.net>
	<4F3E12F9.6080102@bernhard-voelker.de>
	<8762f5lpct.fsf@rho.meyering.net>
	<4F3E35C9.7050707@bernhard-voelker.de>
	<87mx8hk6bm.fsf@rho.meyering.net>
	<4F4069D3.5000806@itwm.fraunhofer.de>
	<87pqdbcia0.fsf@rho.meyering.net> <4F40D061.3000206@draigBrady.com>
	<871upqdagt.fsf@rho.meyering.net>
	<4F41762E.1030801@itwm.fraunhofer.de>
	<4F418BAE.20703@draigBrady.com>
	<4F41AA9D.2060007@itwm.fraunhofer.de>
In-Reply-To: <4F41AA9D.2060007@itwm.fraunhofer.de>
X-Enigmail-Version: 1.3.2
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 213.233.128.45
Cc: "Daniel P. Berrange" <berrange@redhat.com>,
	Bernhard Voelker <mail@bernhard-voelker.de>,
	Bernd Schubert <bernd.schubert@itwm.fraunhofer.de>, coreutils@gnu.org,
	=?ISO-8859-1?Q?J=E9r?= =?ISO-8859-1?Q?=E9my_Compostella?=
	<jeremy.compostella@gmail.com>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 20 Feb 2012 10:55:27 -0000

On 02/20/2012 02:06 AM, Sven Breuner wrote:
> Pádraig Brady wrote on 02/20/2012 12:54 AM:
>> On 02/19/2012 10:22 PM, Sven Breuner wrote:
>>> Is there maybe another check that could be used to find out if selinux is disabled? (e.g. is_selinux_enabled(3) )
>>
>> I'm not sure you could use that.
>> Even though it's currently disabled,
>> you might be surprised that ls didn't
>> indicate selinux attributes as normal.
> 
> Agreed, changing behavior that someone might be relying on wouldn't be good. So I would say the question is whether someone with disabled selinux would really rely on ls reporting selinux attributes in the "ls -l" output.
> 
> 
>>>> Back when I wrote that wrapper, it sounded like 10,"unlabeled"
>>>> would happen only with kernels deemed crufty back then (in 2009).
>>>>
>>>>     http://thread.gmane.org/gmane.comp.gnu.coreutils.bugs/18378/focus=18384
>>>>
>>>> That we're still seeing such syscall traces in 2012...
>>>
>>> I guess that means it's more or less intended that userspace does get ENODATA or "unlabled" instead of EOPNOTSUPP - to show that the system uses a kernel which is generally selinux compatible, even though selinux is disabled.
>>
>> In the referenced thread above, there is a 2006 kernel patch
>> which seems to remove this "unlabeled" thing.
>> Though that doesn't seem to correlate with your testing against ~/tmp/ls_20120209.
> 
> The way I understood it, that patch should just avoid storing those attributes on disk. It wasn't intended to completely remove the existence of "unlabled".
> 
> $ git show 8aad38752e81d1d4de67e3d8e2524618ce7c9276
> [...]
> SELinux always maintains a safe internal label if there is none, so with this patch, we just stick with that and wait until a policy is loaded before adding a persistent label on disk.
> [...]
> 
> 
>> I just booted RHEL 6.2 with selinux=0 on the kernel command line,
>> so as I could test new files without a context created, and...
>>
>> getxattr("security.selinux") on existing files on ext4, returns the existing attribute
>> getxattr("security.selinux") on new files on ext4, returns ENODATA
>> getxattr("security.selinux") on new files on tmpfs, returns ENOTSUP
>>
>> So is this "unlabeled" coming from some really old kernel?
>> I thought you were on RHEL 6.2? Actually I went back to your
>> original report (I presume also on RHEL 6.2) and saw:
>>
>> lgetxattr("test", "security.selinux", 0x62d130, 255) = -1 ENODATA (No data available)
>>
>> I.E. "unlabeled" was not returned there either.
>> What's changed?
> 
> Seems to be inconsistent. Some of our RHEL 6.x systems return "unlabled" on a tmpfs:
> 
> a) RHEL 6.1...
> $ uname -r
> 2.6.32-131.21.1.el6.x86_64
> $ sestatus
> SELinux status:                 disabled
> $ mount -t tmpfs none /mnt/tmp
> $ touch /mnt/tmp/file
> $ strace ls -l /mnt/tmp/file 2>&1 | grep security
> lgetxattr("/mnt/tmp/file", "security.selinux", "unlabeled", 255) = 10
> 
> b) Same "unlabeled" result for another machine with RHEL 6.2 kernel 2.6.32-220.4.1.el6.x86_64.
> 
> c) But then the third machine with RHEL kernel 2.6.32-220.2.1.el6.x86_64 shows EOPNOTSUPP in the same tmpfs test case:
> $ strace ls -l /mnt/tmp/file 2>&1 | grep security
> lgetxattr("/mnt/tmp/file", "security.selinux", 0x623300, 255) = -1 EOPNOTSUPP (Operation not supported)
> 
> All of these systems are fresh RHEL 6 installs, which never ran on an old kernel and never had selinux enabled (though that probably shouldn't make a difference on a tmpfs).
> 
> The fhgfs result in these cases is similar to the tmpfs result: "unlabled" on machines a/b and EOPNOTSUPP on machine c.
> 
> Strange. The only thing I can conclude from these results is that there is a difference in the selinux configuration of these machines that is effective even though selinux is disabled. (Just don't know where to look for that difference to verify this assumption.)
> 
> If we assume that there is something wrong with machines a/b and all those systems actually should be returning EOPNOTSUPP in a "normal" setup, then the current cache implementation would work perfectly fine, of course.

Yep. That's what I'm thinking is appropriate for fhgfs.
My kernel is 2.6.32-221 BTW.

I thought the difference might be selinux=0 boot param,
versus selinux=disabled in /etc/selinux/config, but that's not significant.

I notice that mount options can give a default context.
Is that being used in your "unlabeled" cases?
I.E. are there any *context options present?

If you can't spot the difference, then this is a good question
for an selinux mailing list I think.

cheers,
Pádraig.


From MAILER-DAEMON Mon Feb 20 06:50:12 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RzRky-0005cx-ON
	for mharc-coreutils@gnu.org; Mon, 20 Feb 2012 06:50:12 -0500
Received: from eggs.gnu.org ([140.186.70.92]:56105)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <sven.breuner@itwm.fraunhofer.de>) id 1RzRkq-0005X1-C9
	for coreutils@gnu.org; Mon, 20 Feb 2012 06:50:11 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <sven.breuner@itwm.fraunhofer.de>) id 1RzRkc-0003Oe-Dj
	for coreutils@gnu.org; Mon, 20 Feb 2012 06:50:03 -0500
Received: from mailgw1.uni-kl.de ([131.246.120.220]:47432)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <sven.breuner@itwm.fraunhofer.de>) id 1RzRkb-0003NK-Jg
	for coreutils@gnu.org; Mon, 20 Feb 2012 06:49:50 -0500
Received: from itwm2.itwm.fhg.de (itwm2.itwm.fhg.de [131.246.191.3])
	by mailgw1.uni-kl.de (8.14.3/8.14.3/Debian-5+lenny1) with ESMTP id
	q1KBniUw003413
	(version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=NOT)
	for <coreutils@gnu.org>; Mon, 20 Feb 2012 12:49:45 +0100
Received: from mail2.itwm.fhg.de ([131.246.191.79]:45263)
	by itwm2.itwm.fhg.de with esmtps (TLSv1:DES-CBC3-SHA:168)
	(/C=DE/ST=Rheinland-Pfalz/L=Kaiserslautern/O=Fraunhofer
	ITWM/OU=SLG/CN=mail2.itwm.fhg.de)(verified=1) (Exim 4.74 #1)
	id 1RzRkU-0004yd-H6; Mon, 20 Feb 2012 12:49:42 +0100
Message-ID: <4F423355.1080603@itwm.fraunhofer.de>
Date: Mon, 20 Feb 2012 12:49:41 +0100
From: Sven Breuner <sven.breuner@itwm.fraunhofer.de>
User-Agent: Mozilla/5.0 (X11; Linux i686 on x86_64;
	rv:10.0.2) Gecko/20120216 Thunderbird/10.0.2
MIME-Version: 1.0
To: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
Subject: Re: "ls -l": Avoid unnecessary getxattr() overhead
References: <87pqds3snt.fsf@Apollo.jerryland.fr>
	<4F30206B.1020802@itwm.fraunhofer.de>
	<CA+TATJHorgsuY7pxmpxxAeWUHRP5MfFfzQP05o=1H6C4ttswdA@mail.gmail.com>
	<4F3032BC.50702@draigBrady.com>
	<4F30631A.2070009@itwm.fraunhofer.de>
	<87y5scfdiq.fsf@rho.meyering.net> <4F33B5D7.3010909@draigBrady.com>
	<87mx8sf5w0.fsf@rho.meyering.net>
	<4F3437CE.4050308@itwm.fraunhofer.de>
	<4F346CC7.4030608@draigBrady.com> <87r4y3dry0.fsf@rho.meyering.net>
	<4F3710CA.8020402@itwm.fraunhofer.de>
	<8739aao9e2.fsf@rho.meyering.net>
	<4F3E12F9.6080102@bernhard-voelker.de>
	<8762f5lpct.fsf@rho.meyering.net>
	<4F3E35C9.7050707@bernhard-voelker.de>
	<87mx8hk6bm.fsf@rho.meyering.net>
	<4F4069D3.5000806@itwm.fraunhofer.de>
	<87pqdbcia0.fsf@rho.meyering.net> <4F40D061.3000206@draigBrady.com>
	<871upqdagt.fsf@rho.meyering.net>
	<4F41762E.1030801@itwm.fraunhofer.de>
	<4F418BAE.20703@draigBrady.com>
	<4F41AA9D.2060007@itwm.fraunhofer.de>
	<4F42267F.60205@draigBrady.com>
In-Reply-To: <4F42267F.60205@draigBrady.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
X-ITWM-CharSet: ISO-8859-1
X-ITWM-Scanned-By: mail2.itwm.fhg.de
Content-Transfer-Encoding: quoted-printable
X-MIME-Autoconverted: from 8bit to quoted-printable by mailgw1.uni-kl.de id
	q1KBniUw003413
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
X-Received-From: 131.246.120.220
Cc: "Daniel P. Berrange" <berrange@redhat.com>,
	Bernhard Voelker <mail@bernhard-voelker.de>,
	Bernd Schubert <bernd.schubert@itwm.fraunhofer.de>, coreutils@gnu.org,
	=?ISO-8859-1?Q?J=E9r?= =?ISO-8859-1?Q?=E9my_Compostella?=
	<jeremy.compostella@gmail.com>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 20 Feb 2012 11:50:11 -0000

P=E1draig Brady wrote on 02/20/2012 11:54 AM:
> On 02/20/2012 02:06 AM, Sven Breuner wrote:
>> Seems to be inconsistent. Some of our RHEL 6.x systems return "unlable=
d" on a tmpfs:
>> [...]
>> The fhgfs result in these cases is similar to the tmpfs result: "unlab=
led" on machines a/b and EOPNOTSUPP on machine c.
>>
>> Strange. The only thing I can conclude from these results is that ther=
e is a difference in the selinux configuration of these machines that is =
effective even though selinux is disabled. (Just don't know where to look=
 for that difference to verify this assumption.)
>>
>> If we assume that there is something wrong with machines a/b and all t=
hose systems actually should be returning EOPNOTSUPP in a "normal" setup,=
 then the current cache implementation would work perfectly fine, of cour=
se.
>
> Yep. That's what I'm thinking is appropriate for fhgfs.
> My kernel is 2.6.32-221 BTW.
>
> I thought the difference might be selinux=3D0 boot param,
> versus selinux=3Ddisabled in /etc/selinux/config, but that's not signif=
icant.
>
> I notice that mount options can give a default context.
> Is that being used in your "unlabeled" cases?
> I.E. are there any *context options present?

At least "mount" and "cat /proc/mounts" don't show any special options.

> If you can't spot the difference, then this is a good question
> for an selinux mailing list I think.

Yep :)

Best regards,
Sven


From MAILER-DAEMON Mon Feb 20 07:46:01 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RzScz-0005lE-PG
	for mharc-coreutils@gnu.org; Mon, 20 Feb 2012 07:46:01 -0500
Received: from eggs.gnu.org ([140.186.70.92]:35019)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RzSct-0005l4-Lg
	for coreutils@gnu.org; Mon, 20 Feb 2012 07:46:01 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RzScp-0006Ct-2P
	for coreutils@gnu.org; Mon, 20 Feb 2012 07:45:55 -0500
Received: from mail3.vodafone.ie ([213.233.128.45]:47793)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1RzSco-0006Cf-Pv
	for coreutils@gnu.org; Mon, 20 Feb 2012 07:45:50 -0500
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: ApMBACJAQk9tTDS1/2dsb2JhbAAMLwi0UQEBAQQBAi8BRhALDQcECRYPCQMCAQIBFi8GDQEFAgEBiAW2VolKgmUJBgIDBgMHAgkBDAMChBSDRASTD4gnjG4
Received: from unknown (HELO [192.168.1.79]) ([109.76.52.181])
	by mail3.vodafone.ie with ESMTP; 20 Feb 2012 12:45:48 +0000
Message-ID: <4F42407C.7010809@draigBrady.com>
Date: Mon, 20 Feb 2012 12:45:48 +0000
From: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:6.0) Gecko/20110816 Thunderbird/6.0
MIME-Version: 1.0
To: Jim Meyering <jim@meyering.net>
Subject: Re: "ls -l": Avoid unnecessary getxattr() overhead
References: <87pqds3snt.fsf@Apollo.jerryland.fr>
	<4F30206B.1020802@itwm.fraunhofer.de>
	<CA+TATJHorgsuY7pxmpxxAeWUHRP5MfFfzQP05o=1H6C4ttswdA@mail.gmail.com>
	<4F3032BC.50702@draigBrady.com>
	<4F30631A.2070009@itwm.fraunhofer.de>
	<87y5scfdiq.fsf@rho.meyering.net> <4F33B5D7.3010909@draigBrady.com>
	<87mx8sf5w0.fsf@rho.meyering.net>
	<4F3437CE.4050308@itwm.fraunhofer.de>
	<4F346CC7.4030608@draigBrady.com> <87r4y3dry0.fsf@rho.meyering.net>
	<4F3710CA.8020402@itwm.fraunhofer.de>
	<8739aao9e2.fsf@rho.meyering.net>
	<4F3E12F9.6080102@bernhard-voelker.de>
	<8762f5lpct.fsf@rho.meyering.net>
	<4F3E35C9.7050707@bernhard-voelker.de>
	<87mx8hk6bm.fsf@rho.meyering.net>
	<4F4069D3.5000806@itwm.fraunhofer.de>
	<87pqdbcia0.fsf@rho.meyering.net> <4F40D061.3000206@draigBrady.com>
	<871upqdagt.fsf@rho.meyering.net>
In-Reply-To: <871upqdagt.fsf@rho.meyering.net>
X-Enigmail-Version: 1.3.2
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 213.233.128.45
Cc: Bernd Schubert <bernd.schubert@itwm.fraunhofer.de>,
	Sven Breuner <sven.breuner@itwm.fraunhofer.de>, coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 20 Feb 2012 12:46:01 -0000

On 02/19/2012 05:00 PM, Jim Meyering wrote:
> Pádraig Brady wrote:
>> Also I was a bit surprised to see EBUSY and ENOENT in errno_unsupported().
>> Can one assume there are no other attributes on a device if you get those?
> 
> I added those to align with ACL_NOT_WELL_SUPPORTED from
> lib/acl-internal.h, but now that you mention it, I suppose we'll need
> different errno-checking functions for ACL-related than for getfilecon
> failures.  I suppose ENOENT can arise when the file is removed between
> when we stat'd it and when we perform one of these three file-name-using
> tests.

OK to push this so?

commit 7f48aa570d93347aa72d86a034e36377db6a22b9
Author: Pádraig Brady <P@draigBrady.com>
Date:   Mon Feb 20 12:33:17 2012 +0000

    ls: adjust errnos indicating device non support for xattrs

    * src/ls.c (errno_unsupported): Remove EBUSY, as this caters for
    the case where ACLs can't be accessed because the _file_ is locked.
    http://lists.gnu.org/archive/html/bug-coreutils/2005-06/msg00191.html
    Also ENOENT is not safe to include as you get that if the _file_
    is removed between the stat() and subsequent querying of xattrs.

diff --git a/src/ls.c b/src/ls.c
index 92b17a4..f1dfb1e 100644
--- a/src/ls.c
+++ b/src/ls.c
@@ -2783,9 +2783,7 @@ clear_files (void)
 static bool
 errno_unsupported (int err)
 {
-  return (err == EBUSY
-          || err == EINVAL
-          || err == ENOENT
+  return (err == EINVAL
           || err == ENOSYS
           || err == ENOTSUP
           || err == EOPNOTSUPP);


From MAILER-DAEMON Mon Feb 20 07:50:42 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RzShW-0007T6-Nk
	for mharc-coreutils@gnu.org; Mon, 20 Feb 2012 07:50:42 -0500
Received: from eggs.gnu.org ([140.186.70.92]:36576)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1RzShQ-0007SS-FR
	for coreutils@gnu.org; Mon, 20 Feb 2012 07:50:41 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1RzShK-00075F-Dh
	for coreutils@gnu.org; Mon, 20 Feb 2012 07:50:36 -0500
Received: from mx.meyering.net ([88.168.87.75]:49245)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1RzShK-00075B-6h
	for coreutils@gnu.org; Mon, 20 Feb 2012 07:50:30 -0500
Received: from rho.meyering.net (localhost.localdomain [127.0.0.1])
	by rho.meyering.net (Acme Bit-Twister) with ESMTP id 411F260064;
	Mon, 20 Feb 2012 13:50:29 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: =?iso-8859-1?Q?P=E1draig?= Brady <P@draigBrady.com>
Subject: Re: "ls -l": Avoid unnecessary getxattr() overhead
In-Reply-To: <4F42407C.7010809@draigBrady.com> (=?iso-8859-1?Q?=22P=E1drai?=
	=?iso-8859-1?Q?g?= Brady"'s message of
	"Mon, 20 Feb 2012 12:45:48 +0000")
References: <87pqds3snt.fsf@Apollo.jerryland.fr>
	<CA+TATJHorgsuY7pxmpxxAeWUHRP5MfFfzQP05o=1H6C4ttswdA@mail.gmail.com>
	<4F3032BC.50702@draigBrady.com> <4F30631A.2070009@itwm.fraunhofer.de>
	<87y5scfdiq.fsf@rho.meyering.net> <4F33B5D7.3010909@draigBrady.com>
	<87mx8sf5w0.fsf@rho.meyering.net>
	<4F3437CE.4050308@itwm.fraunhofer.de>
	<4F346CC7.4030608@draigBrady.com> <87r4y3dry0.fsf@rho.meyering.net>
	<4F3710CA.8020402@itwm.fraunhofer.de>
	<8739aao9e2.fsf@rho.meyering.net>
	<4F3E12F9.6080102@bernhard-voelker.de>
	<8762f5lpct.fsf@rho.meyering.net>
	<4F3E35C9.7050707@bernhard-voelker.de>
	<87mx8hk6bm.fsf@rho.meyering.net>
	<4F4069D3.5000806@itwm.fraunhofer.de>
	<87pqdbcia0.fsf@rho.meyering.net> <4F40D061.3000206@draigBrady.com>
	<871upqdagt.fsf@rho.meyering.net> <4F42407C.7010809@draigBrady.com>
Date: Mon, 20 Feb 2012 13:50:29 +0100
Message-ID: <871upp8y96.fsf@rho.meyering.net>
Lines: 28
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 88.168.87.75
Cc: Bernd Schubert <bernd.schubert@itwm.fraunhofer.de>,
	Sven Breuner <sven.breuner@itwm.fraunhofer.de>, coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 20 Feb 2012 12:50:41 -0000

P=E1draig Brady wrote:
> On 02/19/2012 05:00 PM, Jim Meyering wrote:
>> P=E1draig Brady wrote:
>>> Also I was a bit surprised to see EBUSY and ENOENT in errno_unsupported=
().
>>> Can one assume there are no other attributes on a device if you get tho=
se?
>>
>> I added those to align with ACL_NOT_WELL_SUPPORTED from
>> lib/acl-internal.h, but now that you mention it, I suppose we'll need
>> different errno-checking functions for ACL-related than for getfilecon
>> failures.  I suppose ENOENT can arise when the file is removed between
>> when we stat'd it and when we perform one of these three file-name-using
>> tests.
>
> OK to push this so?

Yes.  Thank you!

> commit 7f48aa570d93347aa72d86a034e36377db6a22b9
> Author: P=E1draig Brady <P@draigBrady.com>
> Date:   Mon Feb 20 12:33:17 2012 +0000
>
>     ls: adjust errnos indicating device non support for xattrs
>
>     * src/ls.c (errno_unsupported): Remove EBUSY, as this caters for
>     the case where ACLs can't be accessed because the _file_ is locked.
>     http://lists.gnu.org/archive/html/bug-coreutils/2005-06/msg00191.html
>     Also ENOENT is not safe to include as you get that if the _file_
>     is removed between the stat() and subsequent querying of xattrs.


From MAILER-DAEMON Mon Feb 20 08:51:43 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RzTeZ-00059r-SL
	for mharc-coreutils@gnu.org; Mon, 20 Feb 2012 08:51:43 -0500
Received: from eggs.gnu.org ([140.186.70.92]:57690)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1RzTeR-00056x-8t
	for coreutils@gnu.org; Mon, 20 Feb 2012 08:51:40 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1RzTeL-0001EL-0L
	for coreutils@gnu.org; Mon, 20 Feb 2012 08:51:35 -0500
Received: from mx1.redhat.com ([209.132.183.28]:26443)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>)
	id 1RzTeK-0001E5-Q4; Mon, 20 Feb 2012 08:51:28 -0500
Received: from int-mx10.intmail.prod.int.phx2.redhat.com
	(int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id q1KDpQaT011086
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK);
	Mon, 20 Feb 2012 08:51:26 -0500
Received: from [10.3.113.114] (ovpn-113-114.phx2.redhat.com [10.3.113.114])
	by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP
	id q1KDpP21020936; Mon, 20 Feb 2012 08:51:26 -0500
Message-ID: <4F424FD7.80701@redhat.com>
Date: Mon, 20 Feb 2012 06:51:19 -0700
From: Eric Blake <eblake@redhat.com>
Organization: Red Hat
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:10.0.1) Gecko/20120209 Thunderbird/10.0.1
MIME-Version: 1.0
To: Jim Meyering <jim@meyering.net>
Subject: Re: [PATCH] tests: add missing sys/types.h for ls speed-up test
References: <87pqda3mtn.fsf@Apollo.jerryland.fr>
	<877gzidg41.fsf@rho.meyering.net>
	<CA+TATJEUtf0513Cdk-XmCiHBfXGrFPYwRSBJj=jV=tfK79szDw@mail.gmail.com>
	<87vcn2bttz.fsf@rho.meyering.net>
In-Reply-To: <87vcn2bttz.fsf@rho.meyering.net>
X-Enigmail-Version: 1.3.5
OpenPGP: url=http://people.redhat.com/eblake/eblake.gpg
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature";
	boundary="------------enig46A06B40DEE0ED09B3A21BB7"
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.132.183.28
Cc: bug-gnulib <bug-gnulib@gnu.org>,
	=?ISO-8859-1?Q?J=E9r=E9my_Compostella?= <jeremy.compostella@gmail.com>,
	coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 20 Feb 2012 13:51:40 -0000

This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--------------enig46A06B40DEE0ED09B3A21BB7
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

On 02/19/2012 10:45 AM, Jim Meyering wrote:
> J=E9r=E9my Compostella wrote:
>=20
>>> Thank you for the report and the patch.
>>> Please tell us what type/version of compiler/system you're using.
>>> It's good to include that information, especially since ssize_t
>>> is supposed to be defined via stdio.h.  Then it is clear that this
>>> change is working around a problem with non-conforming systems.
>> Here the requested information:
>> - linux kernel: Linux Apollo 3.2.0-rc7+ #75 SMP Thu Jan 5 18:14:54 CET=

>>   2012 x86_64 GNU/Linux
>> - gcc (Debian 4.4.5-8) 4.4.5
>> - libc6-dev 2.11.2-10
>=20
> Thanks.
> I've adjusted your commit log to explain why:
>=20
>>From 933989eca67bd82f011595937002cc284c98772d Mon Sep 17 00:00:00 2001
> From: =3D?UTF-8?q?J=3DC3=3DA9r=3DC3=3DA9my=3D20Compostella?=3D <jeremy.=
compostella@gmail.com>
> Date: Sun, 19 Feb 2012 15:38:15 +0100
> Subject: [PATCH] tests: ls/getxattr-speedup: include <sys/types.h> for =
older headers
>=20
> Modern <stdio.h> provides ssize_t, but the one from Debian's
> libc6-dev 2.11.2-10 apparently does not.
> * tests/ls/getxattr-speedup: Also include <sys/types.h>.

Shouldn't we be fixing this in gnulib, rather than making every client
package (including coreutils) work around it?

--=20
Eric Blake   eblake@redhat.com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


--------------enig46A06B40DEE0ED09B3A21BB7
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)
Comment: Public key at http://people.redhat.com/eblake/eblake.gpg
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQEcBAEBCAAGBQJPQk/dAAoJEKeha0olJ0NqEP0IAILflWOctN8EggqbIfGYzzSZ
nkkFC1tmCfRpFE2z1Sn2zxUuwN+ScxdZF2tRs+LX9UiWmnm9otAiyOyHPBGRsvvO
uqv2wiLetF/XTwtRGJvCkDsxL21SGY6KbOHngxKHg4820uOjx8vl2iyFvKFyS9Ze
WCOPUOJX0TssK4P0LF+t0Zr59VO8eJweY5QOZa/jJh3bFG1IwNGIc5V0jE+5E21y
t+SFg+ORE4SncQqK903IlolH0xIrTg8PQgJ1DyJqQY7WVRVssPzDcZWaGkbH5GmI
w39CErBlo5yf2M2OZ1TOIM0K8QSI7pZ3Zo3L+ng0eeCsaHoXvdSckBCNWcik8nE=
=3S/+
-----END PGP SIGNATURE-----

--------------enig46A06B40DEE0ED09B3A21BB7--


From MAILER-DAEMON Mon Feb 20 08:54:22 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RzTh8-0005SE-Dv
	for mharc-coreutils@gnu.org; Mon, 20 Feb 2012 08:54:22 -0500
Received: from eggs.gnu.org ([140.186.70.92]:58239)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1RzTh1-0005O1-7f
	for coreutils@gnu.org; Mon, 20 Feb 2012 08:54:20 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1RzTgr-0001d9-LP
	for coreutils@gnu.org; Mon, 20 Feb 2012 08:54:15 -0500
Received: from mx.meyering.net ([88.168.87.75]:49408)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>)
	id 1RzTgk-0001ag-P9; Mon, 20 Feb 2012 08:53:58 -0500
Received: from rho.meyering.net (localhost.localdomain [127.0.0.1])
	by rho.meyering.net (Acme Bit-Twister) with ESMTP id 394D060108;
	Mon, 20 Feb 2012 14:53:57 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: Eric Blake <eblake@redhat.com>
Subject: Re: [PATCH] tests: add missing sys/types.h for ls speed-up test
In-Reply-To: <4F424FD7.80701@redhat.com> (Eric Blake's message of "Mon, 20 Feb
	2012 06:51:19 -0700")
References: <87pqda3mtn.fsf@Apollo.jerryland.fr>
	<877gzidg41.fsf@rho.meyering.net>
	<CA+TATJEUtf0513Cdk-XmCiHBfXGrFPYwRSBJj=jV=tfK79szDw@mail.gmail.com>
	<87vcn2bttz.fsf@rho.meyering.net> <4F424FD7.80701@redhat.com>
Date: Mon, 20 Feb 2012 14:53:57 +0100
Message-ID: <87ehtp7gqy.fsf@rho.meyering.net>
Lines: 36
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 88.168.87.75
Cc: bug-gnulib <bug-gnulib@gnu.org>,
	=?iso-8859-1?Q?J=E9r=E9my?= Compostella <jeremy.compostella@gmail.com>,
	coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 20 Feb 2012 13:54:20 -0000

Eric Blake wrote:

> On 02/19/2012 10:45 AM, Jim Meyering wrote:
>> J=E9r=E9my Compostella wrote:
>>
>>>> Thank you for the report and the patch.
>>>> Please tell us what type/version of compiler/system you're using.
>>>> It's good to include that information, especially since ssize_t
>>>> is supposed to be defined via stdio.h.  Then it is clear that this
>>>> change is working around a problem with non-conforming systems.
>>> Here the requested information:
>>> - linux kernel: Linux Apollo 3.2.0-rc7+ #75 SMP Thu Jan 5 18:14:54 CET
>>>   2012 x86_64 GNU/Linux
>>> - gcc (Debian 4.4.5-8) 4.4.5
>>> - libc6-dev 2.11.2-10
>>
>> Thanks.
>> I've adjusted your commit log to explain why:
>>
>>>From 933989eca67bd82f011595937002cc284c98772d Mon Sep 17 00:00:00 2001
>> From: =3D?UTF-8?q?J=3DC3=3DA9r=3DC3=3DA9my=3D20Compostella?=3D <jeremy.c=
ompostella@gmail.com>
>> Date: Sun, 19 Feb 2012 15:38:15 +0100
>> Subject: [PATCH] tests: ls/getxattr-speedup: include <sys/types.h>
>> for older headers
>>
>> Modern <stdio.h> provides ssize_t, but the one from Debian's
>> libc6-dev 2.11.2-10 apparently does not.
>> * tests/ls/getxattr-speedup: Also include <sys/types.h>.
>
> Shouldn't we be fixing this in gnulib, rather than making every client
> package (including coreutils) work around it?

Is it even a problem in gnulib?

Notice that that test is unusual in that it uses $CC without the usual
-I options and on code that does not include <config.h>.


From MAILER-DAEMON Mon Feb 20 10:43:31 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RzVOl-0000Nl-RF
	for mharc-coreutils@gnu.org; Mon, 20 Feb 2012 10:43:31 -0500
Received: from eggs.gnu.org ([140.186.70.92]:60857)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1RzVOc-0000NE-Uq
	for coreutils@gnu.org; Mon, 20 Feb 2012 10:43:30 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1RzVOX-0002dl-B4
	for coreutils@gnu.org; Mon, 20 Feb 2012 10:43:22 -0500
Received: from mx1.redhat.com ([209.132.183.28]:65181)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>)
	id 1RzVOW-0002de-Vo; Mon, 20 Feb 2012 10:43:17 -0500
Received: from int-mx01.intmail.prod.int.phx2.redhat.com
	(int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id q1KFhEkT009881
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK);
	Mon, 20 Feb 2012 10:43:15 -0500
Received: from [10.3.113.151] (ovpn-113-151.phx2.redhat.com [10.3.113.151])
	by int-mx01.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP
	id q1KFhEDI018684; Mon, 20 Feb 2012 10:43:14 -0500
Message-ID: <4F426A12.7060505@redhat.com>
Date: Mon, 20 Feb 2012 08:43:14 -0700
From: Eric Blake <eblake@redhat.com>
Organization: Red Hat
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:10.0.1) Gecko/20120209 Thunderbird/10.0.1
MIME-Version: 1.0
To: Jim Meyering <jim@meyering.net>
Subject: Re: [PATCH] tests: add missing sys/types.h for ls speed-up test
References: <87pqda3mtn.fsf@Apollo.jerryland.fr>
	<877gzidg41.fsf@rho.meyering.net>
	<CA+TATJEUtf0513Cdk-XmCiHBfXGrFPYwRSBJj=jV=tfK79szDw@mail.gmail.com>
	<87vcn2bttz.fsf@rho.meyering.net> <4F424FD7.80701@redhat.com>
	<87ehtp7gqy.fsf@rho.meyering.net>
In-Reply-To: <87ehtp7gqy.fsf@rho.meyering.net>
X-Enigmail-Version: 1.3.5
OpenPGP: url=http://people.redhat.com/eblake/eblake.gpg
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature";
	boundary="------------enig9B15F4093F50CA8A60BBA2D2"
X-Scanned-By: MIMEDefang 2.67 on 10.5.11.11
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.132.183.28
Cc: bug-gnulib <bug-gnulib@gnu.org>,
	=?ISO-8859-1?Q?J=E9r=E9my_Compostella?= <jeremy.compostella@gmail.com>,
	coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 20 Feb 2012 15:43:30 -0000

This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--------------enig9B15F4093F50CA8A60BBA2D2
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

On 02/20/2012 06:53 AM, Jim Meyering wrote:

>>> Modern <stdio.h> provides ssize_t, but the one from Debian's
>>> libc6-dev 2.11.2-10 apparently does not.
>>> * tests/ls/getxattr-speedup: Also include <sys/types.h>.
>>
>> Shouldn't we be fixing this in gnulib, rather than making every client=

>> package (including coreutils) work around it?
>=20
> Is it even a problem in gnulib?

Ah, it's already fixed there; from lib/stdio.in.h, there is this
unconditional include:

/* Get off_t and ssize_t.  Needed on many systems, including glibc 2.8
   and eglibc 2.11.2.  */
#include <sys/types.h>

>=20
> Notice that that test is unusual in that it uses $CC without the usual
> -I options and on code that does not include <config.h>.

You've convinced me, this is a patch specific to and appropriate for
just coreutils.

--=20
Eric Blake   eblake@redhat.com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


--------------enig9B15F4093F50CA8A60BBA2D2
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)
Comment: Public key at http://people.redhat.com/eblake/eblake.gpg
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQEcBAEBCAAGBQJPQmoSAAoJEKeha0olJ0NqrscH/19quvL6NHejFeVhZgCRdGo2
hXr3/l5msBY6TdK+nBIF52Ws7CmBJVBo5BMHQ+jHArLekaVYgtahJ4PIm7qgSgYR
0JtfecVRRRUxVURlFxmnidQBhaIWOk06u+/LjFGHi4HFPrhQrPEiZMEAuraBRLWQ
8Je1WgrADfhN6CV6+HAA5YS46JzzpgqlFB701+PgiHXL/myRK1s6pejoUiqV3nwE
8ppD8zHl2v07tbLFq7p5ChVrV7h8Bwt3NU1TQKRAZC44m/RJSlBPvlb+X8V/h3sp
6PlEIpu2KBbkBUJBKD1muZ+X0SeL3/oJa3qflanYTZNtlwddGK/PbOd071s+A9E=
=ziG0
-----END PGP SIGNATURE-----

--------------enig9B15F4093F50CA8A60BBA2D2--


From MAILER-DAEMON Mon Feb 20 17:06:23 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RzbNH-0003GW-NO
	for mharc-coreutils@gnu.org; Mon, 20 Feb 2012 17:06:23 -0500
Received: from eggs.gnu.org ([140.186.70.92]:40147)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <sachin_ohal@cable.comcast.com>) id 1RzbCo-0001Di-0C
	for coreutils@gnu.org; Mon, 20 Feb 2012 16:55:35 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <sachin_ohal@cable.comcast.com>) id 1RzbCm-0004Jc-Fy
	for coreutils@gnu.org; Mon, 20 Feb 2012 16:55:33 -0500
Received: from pacdcimo01.cable.comcast.com ([24.40.8.145]:56047)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <sachin_ohal@cable.comcast.com>) id 1RzbCm-0004JW-D8
	for coreutils@gnu.org; Mon, 20 Feb 2012 16:55:32 -0500
Received: from ([24.40.56.114])
	by pacdcimo01.cable.comcast.com with ESMTP  id 5503620.151858509;
	Mon, 20 Feb 2012 16:55:28 -0500
Received: from PACDCEXMB15.cable.comcast.com ([169.254.2.132]) by
	PACDCEXHUB01.cable.comcast.com ([fe80::d1e7:20b5:9b63:21a6%11]) with
	mapi id 14.01.0355.002; Mon, 20 Feb 2012 16:55:28 -0500
From: "Ohal, Sachin" <Sachin_Ohal@cable.comcast.com>
To: "coreutils@gnu.org" <coreutils@gnu.org>
Subject: Is SED support the 64 Bit Windows 2003 server operating system? 
Thread-Topic: Is SED support the 64 Bit Windows 2003 server operating system? 
Thread-Index: AczwGlrccgeg8c8QQnysjNsSPxUxJg==
Date: Mon, 20 Feb 2012 21:55:27 +0000
Message-ID: <4D87B132553FE94182E8E74A76F4285F0FE41E6D@PACDCEXMB15.cable.comcast.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [24.40.56.176]
Content-Type: multipart/alternative;
	boundary="_000_4D87B132553FE94182E8E74A76F4285F0FE41E6DPACDCEXMB15cabl_"
MIME-Version: 1.0
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 24.40.8.145
X-Mailman-Approved-At: Mon, 20 Feb 2012 17:06:22 -0500
Cc: "Ohal, Sachin" <Sachin_Ohal@cable.comcast.com>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 20 Feb 2012 21:55:35 -0000

--_000_4D87B132553FE94182E8E74A76F4285F0FE41E6DPACDCEXMB15cabl_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

Hi Team,

Greetings for the day!

Is SED support the 64 Bit Windows 2003 server operating system?

Regards,
Sachin O<mailto:sachin_ohal@cable.comcast.com>
Performance Center  & SiteScope<mailto:PerformanceCenterAdministrators@cabl=
e.comcast.com>
ParaSoft : CET Tools Admin Team<mailto:ToolsAdminTeam_CET@cable.comcast.com=
>
CET Tools Admin Team WebSite<http://147.191.121.98:136/>
Performance Testing Team SharePoint<http://workgroups.teamcomcast.com/team/=
cet1/intranet/CET_Testing/CET_PT/Performance%20Testing%20%20Validation/Form=
s/AllItems.aspx>
Desk Phone: 610-732-2314<Desk%20Phone:%20610-732-2314>
HighFive Me?<http://www.teamcomcast.com/home/HighFive/AddHighFive.aspx?Full=
Name=3DCABLE\sohal00c>


--_000_4D87B132553FE94182E8E74A76F4285F0FE41E6DPACDCEXMB15cabl_
Content-Type: text/html; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

<html xmlns:v=3D"urn:schemas-microsoft-com:vml" xmlns:o=3D"urn:schemas-micr=
osoft-com:office:office" xmlns:w=3D"urn:schemas-microsoft-com:office:word" =
xmlns:x=3D"urn:schemas-microsoft-com:office:excel" xmlns:m=3D"http://schema=
s.microsoft.com/office/2004/12/omml" xmlns=3D"http://www.w3.org/TR/REC-html=
40">
<head>
<meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dus-ascii"=
>
<meta name=3D"Generator" content=3D"Microsoft Word 14 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
	{font-family:Calibri;
	panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
	{margin:0in;
	margin-bottom:.0001pt;
	font-size:11.0pt;
	font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
	{mso-style-priority:99;
	color:blue;
	text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
	{mso-style-priority:99;
	color:purple;
	text-decoration:underline;}
span.EmailStyle17
	{mso-style-type:personal-compose;
	font-family:"Calibri","sans-serif";
	color:windowtext;}
.MsoChpDefault
	{mso-style-type:export-only;
	font-family:"Calibri","sans-serif";}
@page WordSection1
	{size:8.5in 11.0in;
	margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
	{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext=3D"edit" spidmax=3D"1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext=3D"edit">
<o:idmap v:ext=3D"edit" data=3D"1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang=3D"EN-US" link=3D"blue" vlink=3D"purple">
<div class=3D"WordSection1">
<p class=3D"MsoNormal">Hi Team,<o:p></o:p></p>
<p class=3D"MsoNormal"><o:p>&nbsp;</o:p></p>
<p class=3D"MsoNormal">Greetings for the day!<o:p></o:p></p>
<p class=3D"MsoNormal"><o:p>&nbsp;</o:p></p>
<p class=3D"MsoNormal">Is SED support the 64 Bit Windows 2003 server operat=
ing system?
<o:p></o:p></p>
<p class=3D"MsoNormal"><o:p>&nbsp;</o:p></p>
<p class=3D"MsoNormal">Regards,<o:p></o:p></p>
<p class=3D"MsoNormal"><a href=3D"mailto:sachin_ohal@cable.comcast.com"><sp=
an style=3D"color:blue">Sachin O</span></a><o:p></o:p></p>
<p class=3D"MsoNormal"><a href=3D"mailto:PerformanceCenterAdministrators@ca=
ble.comcast.com"><span style=3D"color:blue">Performance Center&nbsp; &amp; =
SiteScope</span></a><o:p></o:p></p>
<p class=3D"MsoNormal"><a href=3D"mailto:ToolsAdminTeam_CET@cable.comcast.c=
om"><span style=3D"color:blue">ParaSoft : CET Tools Admin Team</span></a><o=
:p></o:p></p>
<p class=3D"MsoNormal"><a href=3D"http://147.191.121.98:136/"><span style=
=3D"color:blue">CET Tools Admin Team WebSite</span></a><o:p></o:p></p>
<p class=3D"MsoNormal"><a href=3D"http://workgroups.teamcomcast.com/team/ce=
t1/intranet/CET_Testing/CET_PT/Performance%20Testing%20%20Validation/Forms/=
AllItems.aspx"><span style=3D"color:blue">Performance Testing Team SharePoi=
nt</span></a><o:p></o:p></p>
<p class=3D"MsoNormal"><a href=3D"Desk%20Phone:%20610-732-2314"><span style=
=3D"color:blue">Desk Phone: 610-732-2314</span></a><o:p></o:p></p>
<p class=3D"MsoNormal"><a href=3D"http://www.teamcomcast.com/home/HighFive/=
AddHighFive.aspx?FullName=3DCABLE\sohal00c"><span style=3D"color:blue">High=
Five Me?</span></a><o:p></o:p></p>
<p class=3D"MsoNormal"><o:p>&nbsp;</o:p></p>
</div>
</body>
</html>

--_000_4D87B132553FE94182E8E74A76F4285F0FE41E6DPACDCEXMB15cabl_--


From MAILER-DAEMON Mon Feb 20 17:13:11 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RzbTr-0004rP-7Q
	for mharc-coreutils@gnu.org; Mon, 20 Feb 2012 17:13:11 -0500
Received: from eggs.gnu.org ([140.186.70.92]:43560)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1RzbTo-0004gy-4x
	for coreutils@gnu.org; Mon, 20 Feb 2012 17:13:09 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1RzbTm-00077R-MH
	for coreutils@gnu.org; Mon, 20 Feb 2012 17:13:08 -0500
Received: from mx1.redhat.com ([209.132.183.28]:59061)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1RzbTm-00077F-4S
	for coreutils@gnu.org; Mon, 20 Feb 2012 17:13:06 -0500
Received: from int-mx12.intmail.prod.int.phx2.redhat.com
	(int-mx12.intmail.prod.int.phx2.redhat.com [10.5.11.25])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id q1KMD3mg001621
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK);
	Mon, 20 Feb 2012 17:13:03 -0500
Received: from [10.3.113.161] (ovpn-113-161.phx2.redhat.com [10.3.113.161])
	by int-mx12.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP
	id q1KMD2jL021897; Mon, 20 Feb 2012 17:13:03 -0500
Message-ID: <4F42C56E.9010202@redhat.com>
Date: Mon, 20 Feb 2012 15:13:02 -0700
From: Eric Blake <eblake@redhat.com>
Organization: Red Hat
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:10.0.1) Gecko/20120209 Thunderbird/10.0.1
MIME-Version: 1.0
To: "Ohal, Sachin" <Sachin_Ohal@cable.comcast.com>
Subject: Re: Is SED support the 64 Bit Windows 2003 server operating system?
References: <4D87B132553FE94182E8E74A76F4285F0FE41E6D@PACDCEXMB15.cable.comcast.com>
In-Reply-To: <4D87B132553FE94182E8E74A76F4285F0FE41E6D@PACDCEXMB15.cable.comcast.com>
X-Enigmail-Version: 1.3.5
OpenPGP: url=http://people.redhat.com/eblake/eblake.gpg
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature";
	boundary="------------enig4723D7ACEEE622AD6C412117"
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.25
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.132.183.28
Cc: "coreutils@gnu.org" <coreutils@gnu.org>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 20 Feb 2012 22:13:09 -0000

This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--------------enig4723D7ACEEE622AD6C412117
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

On 02/20/2012 02:55 PM, Ohal, Sachin wrote:
> Hi Team,
>=20
> Greetings for the day!
>=20
> Is SED support the 64 Bit Windows 2003 server operating system?

Wrong list - sed is not part of GNU coreutils.

But the answer is yes, sed can be built for 64-bit windows; and in fact,
if you are looking for a pre-compiled version of sed for that platform,
I would suggest you go to cygwin.com.

Followup questions should be directed to lists dedicated to porting GNU
software to windows, such as the cygwin list.

--=20
Eric Blake   eblake@redhat.com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


--------------enig4723D7ACEEE622AD6C412117
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)
Comment: Public key at http://people.redhat.com/eblake/eblake.gpg
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQEcBAEBCAAGBQJPQsVuAAoJEKeha0olJ0NqEzkIAK4IgGBGBwYCZsud7g+7nyUx
Td4rqT8NcVf7Fh+2el/SyGmaipkvU1tH8/lPMDtklzFc0f5ETYNXHE66b51OiRsl
9ORF0Y6z+cP9Jl8KGQninTN5pH6nrJ5z47q4yIaNv+GHVVHJ7DB/arQ8oMb5hCVZ
YmjgRV+isBV9r+Q+5G6tNMg0ZFMip8TElzvWzNS7xX/K5knW7YRK/LA503Sq/qTJ
uppiOJo6lZ3wUdTFUgRkrQE1Nu6TxEzBporj3VeGcaegKqjTGGzQFWlSmlZXvewl
YsswMm3ioUhGvFE/UiGQgn1yyVYALSLvuIysm7jE2ATS0VHMDMncES4leYFKQCA=
=qjYa
-----END PGP SIGNATURE-----

--------------enig4723D7ACEEE622AD6C412117--


From MAILER-DAEMON Tue Feb 21 16:23:35 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RzxBP-0005Ou-5S
	for mharc-coreutils@gnu.org; Tue, 21 Feb 2012 16:23:35 -0500
Received: from eggs.gnu.org ([140.186.70.92]:46515)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <sachin_ohal@cable.comcast.com>) id 1RzxBJ-0005Ol-8K
	for coreutils@gnu.org; Tue, 21 Feb 2012 16:23:33 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <sachin_ohal@cable.comcast.com>) id 1RzxBE-0002SW-OL
	for coreutils@gnu.org; Tue, 21 Feb 2012 16:23:29 -0500
Received: from pacdcimo01.cable.comcast.com ([24.40.8.145]:49267)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <sachin_ohal@cable.comcast.com>) id 1RzxBE-0002SQ-F1
	for coreutils@gnu.org; Tue, 21 Feb 2012 16:23:24 -0500
Received: from ([24.40.56.115])
	by pacdcimo01.cable.comcast.com with ESMTP  id 5503620.151928456;
	Tue, 21 Feb 2012 16:09:19 -0500
Received: from PACDCEXMB15.cable.comcast.com ([169.254.2.132]) by
	PACDCEXHUB02.cable.comcast.com ([fe80::492e:3fa1:c2ad:e04e%17]) with
	mapi id 14.01.0355.002; Tue, 21 Feb 2012 16:09:19 -0500
From: "Ohal, Sachin" <Sachin_Ohal@cable.comcast.com>
To: Eric Blake <eblake@redhat.com>
Subject: RE: Is SED support the 64 Bit Windows 2003 server operating system?
Thread-Topic: Is SED support the 64 Bit Windows 2003 server operating system?
Thread-Index: AQHM8BzU6BZ0sT68+Uu1x5ednV4mGZZH2bHA
Date: Tue, 21 Feb 2012 21:09:18 +0000
Message-ID: <4D87B132553FE94182E8E74A76F4285F0FE428D2@PACDCEXMB15.cable.comcast.com>
References: <4D87B132553FE94182E8E74A76F4285F0FE41E6D@PACDCEXMB15.cable.comcast.com>
	<4F42C56E.9010202@redhat.com>
In-Reply-To: <4F42C56E.9010202@redhat.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [24.40.56.176]
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 24.40.8.145
Cc: "Ohal, Sachin" <Sachin_Ohal@cable.comcast.com>,
	"coreutils@gnu.org" <coreutils@gnu.org>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Tue, 21 Feb 2012 21:23:33 -0000

Hi Eric,

Greetings for the day!

Thank you for below information

However, I am unable to find the link where I can download the 64 bit Windo=
ws 2003 serve SED version=20

Regards,
Sachin O
Performance Center  & SiteScope
ParaSoft : CET Tools Admin Team
CET Tools Admin Team WebSite
Performance Testing Team SharePoint
Desk Phone: 610-732-2314
HighFive Me?

-----Original Message-----
From: Eric Blake [mailto:eblake@redhat.com]=20
Sent: Monday, February 20, 2012 5:13 PM
To: Ohal, Sachin
Cc: coreutils@gnu.org
Subject: Re: Is SED support the 64 Bit Windows 2003 server operating system=
?

On 02/20/2012 02:55 PM, Ohal, Sachin wrote:
> Hi Team,
>=20
> Greetings for the day!
>=20
> Is SED support the 64 Bit Windows 2003 server operating system?

Wrong list - sed is not part of GNU coreutils.

But the answer is yes, sed can be built for 64-bit windows; and in fact, if=
 you are looking for a pre-compiled version of sed for that platform, I wou=
ld suggest you go to cygwin.com.

Followup questions should be directed to lists dedicated to porting GNU sof=
tware to windows, such as the cygwin list.

--=20
Eric Blake   eblake@redhat.com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org



From MAILER-DAEMON Tue Feb 21 16:33:16 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1RzxKm-0006wj-2x
	for mharc-coreutils@gnu.org; Tue, 21 Feb 2012 16:33:16 -0500
Received: from eggs.gnu.org ([140.186.70.92]:49897)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1RzxKj-0006wL-23
	for coreutils@gnu.org; Tue, 21 Feb 2012 16:33:14 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1RzxKf-0004D4-5N
	for coreutils@gnu.org; Tue, 21 Feb 2012 16:33:13 -0500
Received: from mx1.redhat.com ([209.132.183.28]:58851)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1RzxKe-0004Cv-V5
	for coreutils@gnu.org; Tue, 21 Feb 2012 16:33:09 -0500
Received: from int-mx10.intmail.prod.int.phx2.redhat.com
	(int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id q1LLX6aX002325
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK);
	Tue, 21 Feb 2012 16:33:06 -0500
Received: from [10.3.113.9] ([10.3.113.9])
	by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP
	id q1LLX62l030296; Tue, 21 Feb 2012 16:33:06 -0500
Message-ID: <4F440D8C.7040102@redhat.com>
Date: Tue, 21 Feb 2012 14:33:00 -0700
From: Eric Blake <eblake@redhat.com>
Organization: Red Hat
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:10.0.1) Gecko/20120209 Thunderbird/10.0.1
MIME-Version: 1.0
To: "Ohal, Sachin" <Sachin_Ohal@cable.comcast.com>
Subject: Re: Is SED support the 64 Bit Windows 2003 server operating system?
References: <4D87B132553FE94182E8E74A76F4285F0FE41E6D@PACDCEXMB15.cable.comcast.com>
	<4F42C56E.9010202@redhat.com>
	<4D87B132553FE94182E8E74A76F4285F0FE428D2@PACDCEXMB15.cable.comcast.com>
In-Reply-To: <4D87B132553FE94182E8E74A76F4285F0FE428D2@PACDCEXMB15.cable.comcast.com>
X-Enigmail-Version: 1.3.5
OpenPGP: url=http://people.redhat.com/eblake/eblake.gpg
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature";
	boundary="------------enig10AAB8007C4FAF9D89C59AAC"
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.132.183.28
Cc: "coreutils@gnu.org" <coreutils@gnu.org>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Tue, 21 Feb 2012 21:33:14 -0000

This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--------------enig10AAB8007C4FAF9D89C59AAC
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

[please don't top-post on technical lists]

On 02/21/2012 02:09 PM, Ohal, Sachin wrote:
> Hi Eric,
>=20
> Greetings for the day!
>=20
> Thank you for below information
>=20
> However, I am unable to find the link where I can download the 64 bit W=
indows 2003 serve SED version=20

I already gave you a link:

> if you are looking for a pre-compiled version of sed for that platform,=
 I would suggest you go to cygwin.com.

http://cygwin.com

And I still stand by this statement:

>=20
> Followup questions should be directed to lists dedicated to porting GNU=
 software to windows, such as the cygwin list.
>=20

--=20
Eric Blake   eblake@redhat.com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


--------------enig10AAB8007C4FAF9D89C59AAC
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)
Comment: Public key at http://people.redhat.com/eblake/eblake.gpg
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQEcBAEBCAAGBQJPRA2MAAoJEKeha0olJ0NqwtAH/3td0HZFbwHxE4yrvwcTh+0R
3Crfl6aBVHgL/XQAUwjptbTZ1yAi/4Wx31zzHpNSrkiWDJzeq2pPP2CeCOrs3qOG
rxf3QrWtwKWm4CCvUbxup3kc2Y96aGFPgVt0MvLpsmyqQPjIoW2r9SnvOXhG/q9V
E7ASNipW4Rv0QLgZb3yQpcmDwB7isculLeM+19DVNydZB/Y5WzntQDj50y8W7rjl
MinFeJ5ogwzm+02KzjQmrvmFdoiEyqgoVL4QMPLzDuSr2Nn4eztsXFZxZkV1v+FR
M+F15Qols+3kghSV4Biw5rVWzInt1Ymx0djDNihy94EQD7Lo80VzV8xUj0q0bAk=
=2xZN
-----END PGP SIGNATURE-----

--------------enig10AAB8007C4FAF9D89C59AAC--


From MAILER-DAEMON Wed Feb 22 18:40:26 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1S0LnO-0000n2-3y
	for mharc-coreutils@gnu.org; Wed, 22 Feb 2012 18:40:26 -0500
Received: from eggs.gnu.org ([140.186.70.92]:50996)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1S0LnK-0000cs-CM
	for coreutils@gnu.org; Wed, 22 Feb 2012 18:40:23 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1S0LnH-0002oS-5Q
	for coreutils@gnu.org; Wed, 22 Feb 2012 18:40:21 -0500
Received: from mail-ww0-f49.google.com ([74.125.82.49]:58089)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1S0LnG-0002nv-JC
	for coreutils@gnu.org; Wed, 22 Feb 2012 18:40:18 -0500
Received: by wgbdt13 with SMTP id dt13so445895wgb.30
	for <coreutils@gnu.org>; Wed, 22 Feb 2012 15:40:16 -0800 (PST)
Received-SPF: pass (google.com: domain of jeremy.compostella@gmail.com
	designates 10.180.90.225 as permitted sender)
	client-ip=10.180.90.225; 
Authentication-Results: mr.google.com; spf=pass (google.com: domain of
	jeremy.compostella@gmail.com designates 10.180.90.225 as
	permitted sender) smtp.mail=jeremy.compostella@gmail.com;
	dkim=pass header.i=jeremy.compostella@gmail.com
Received: from mr.google.com ([10.180.90.225])
	by 10.180.90.225 with SMTP id bz1mr828684wib.5.1329954016942 (num_hops
	= 1); Wed, 22 Feb 2012 15:40:16 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma;
	h=from:to:subject:cc:date:message-id:mime-version:content-type;
	bh=ADXWFjk7ks65DwOSAuQ2FA8Zph4X3xoL0HSwHnLDGto=;
	b=Qe43HxPyflCAahyvXdcvakhUsteOx2/3vJbUuyg4AXAnOtueTjhmWjiEqddAKC3ESv
	R3EDcEuEm2RHF/qASUBAMNQA4juX1g3c0NBmEtIxm61JaE6SKSDiNjHjQ5ygfsg/jpmZ
	+FSQy9fLPNNy6HDii5eDivbuqpnR2Ubon9Hk4=
Received: by 10.180.90.225 with SMTP id bz1mr677951wib.5.1329954016866;
	Wed, 22 Feb 2012 15:40:16 -0800 (PST)
Received: from Apollo.jerryland.fr ([85.69.251.50])
	by mx.google.com with ESMTPS id cs4sm2059126wib.8.2012.02.22.15.40.13
	(version=TLSv1/SSLv3 cipher=OTHER);
	Wed, 22 Feb 2012 15:40:14 -0800 (PST)
From: =?utf-8?Q?J=C3=A9r=C3=A9my_Compostella?= <jeremy.compostella@gmail.com>
To: Erik Auerswald <auerswal@unix-ag.uni-kl.de>
Subject: Re: bug#8736: chmod -p --parents
Date: Thu, 23 Feb 2012 00:40:37 +0100
Message-ID: <878vju3096.fsf@Apollo.jerryland.fr>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="=-=-="
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 74.125.82.49
Cc: 8736@debbugs.gnu.org, coreutils@gnu.org, francky.leyn@telenet.be
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Wed, 22 Feb 2012 23:40:24 -0000

--=-=-=
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

> Hi,
>=20
> On Thu, May 26, 2011 at 10:13:05PM +0200, francky.leyn <at> telenet.be wr=
ote:
> >=20
> > In the past, I was an advocate of the -p --parents option for=20
> > mkdir. By now this is realised. Now I'm doing the same for chmod.=20
> > [...]
> > I want to be able to execute the following:=20
> >=20
> > chmod a+rx -p ~/dir1/dir2/dir3/file.ext=20
>=20
> I like the proposal.
> > It should be more refined however: the dirs=20
> > should be rx, but the file only r.=20
>=20
> You can use a capital X to automatically handle the directory case. But
> please take a look at the docs for the _exact_ meaning of X.
I made a first implementation of this chmod --parents feature (see the
attached patch). I do not support the second part since 'X' is already
in used for the "Conditional Executability" feature. I have no idea how
to support it.

I don't have written the documentation for now. I would like to have
your opinion about this patch first as I maybe missed some corner case
or the code should be written another way or even something else ...

I really hope you will find time to look at it.

Cheers,

J=C3=A9r=C3=A9my
---
Sent from my Emacs


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=0001-chmod-add-parents-option.patch
Content-Description: chmod: --parents option patch

>From 5de3629e2f181779460947ea8693561692425d75 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?J=C3=A9r=C3=A9my=20Compostella?= <jeremy.compostella@gmail.com>
Date: Thu, 23 Feb 2012 00:11:41 +0100
Subject: [PATCH] chmod: add --parents option

* chmod.c (main): Handle --parents option.
---
 src/chmod.c |   51 ++++++++++++++++++++++++++++++++++++++++++++++++---
 1 files changed, 48 insertions(+), 3 deletions(-)

diff --git a/src/chmod.c b/src/chmod.c
index a134e3f..4378459 100644
--- a/src/chmod.c
+++ b/src/chmod.c
@@ -96,6 +96,7 @@ static struct option const long_options[] =
 {
   {"changes", no_argument, NULL, 'c'},
   {"recursive", no_argument, NULL, 'R'},
+  {"parents", no_argument, NULL, 'p'},
   {"no-preserve-root", no_argument, NULL, NO_PRESERVE_ROOT},
   {"preserve-root", no_argument, NULL, PRESERVE_ROOT},
   {"quiet", no_argument, NULL, 'f'},
@@ -416,6 +417,7 @@ main (int argc, char **argv)
   size_t mode_alloc = 0;
   bool ok;
   bool preserve_root = false;
+  bool parents = false;
   char const *reference_file = NULL;
   int c;
 
@@ -430,7 +432,7 @@ main (int argc, char **argv)
   recurse = force_silent = diagnose_surprises = false;
 
   while ((c = getopt_long (argc, argv,
-                           "Rcfvr::w::x::X::s::t::u::g::o::a::,::+::=::",
+                           "Rpcfvr::w::x::X::s::t::u::g::o::a::,::+::=::",
                            long_options, NULL))
          != -1)
     {
@@ -484,6 +486,9 @@ main (int argc, char **argv)
         case REFERENCE_FILE_OPTION:
           reference_file = optarg;
           break;
+        case 'p':
+          parents = true;
+          break;
         case 'R':
           recurse = true;
           break;
@@ -544,6 +549,12 @@ main (int argc, char **argv)
       umask_value = umask (0);
     }
 
+  if (recurse && parents)
+    {
+      error (0, 0, _("--recurse and --parents are mutually exclusive"));
+      usage (EXIT_FAILURE);
+    }
+
   if (recurse && preserve_root)
     {
       static struct dev_ino dev_ino_buf;
@@ -557,8 +568,42 @@ main (int argc, char **argv)
       root_dev_ino = NULL;
     }
 
-  ok = process_files (argv + optind,
-                      FTS_COMFOLLOW | FTS_PHYSICAL | FTS_DEFER_STAT);
+  if (! parents)
+    ok = process_files (argv + optind,
+                        FTS_COMFOLLOW | FTS_PHYSICAL | FTS_DEFER_STAT);
+  else
+  {
+    char **first = argv + optind;
+    char **end = argv + argc;
+    while (end != first)
+      {
+        ok = process_files (first,
+                            FTS_COMFOLLOW | FTS_PHYSICAL | FTS_DEFER_STAT);
+        if (! ok)
+          break;
+
+        char **current = end - 1;
+        do
+          {
+            char *last = last_component (*current);
+            if (last != *current) {
+              /* Skiping slashes preceding the last component */
+              while (ISSLASH (*--last) && last != *current)
+                ;
+              if (! ISSLASH (*last))
+                *++last = '\0';
+            }
+            if (last == *current)
+              {
+                /* Remove CURRENT by shifting ARGV to left from there */
+                end = current - 1;
+                while (*end && end++)
+                  *end = *(end + 1);
+              }
+          }
+        while (current-- != first);
+      }
+  }
 
   exit (ok ? EXIT_SUCCESS : EXIT_FAILURE);
 }
-- 
1.7.2.5


--=-=-=--


From MAILER-DAEMON Thu Feb 23 06:04:32 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1S0WTQ-0000sE-NM
	for mharc-coreutils@gnu.org; Thu, 23 Feb 2012 06:04:32 -0500
Received: from eggs.gnu.org ([140.186.70.92]:40062)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1S0WTN-0000re-LY
	for coreutils@gnu.org; Thu, 23 Feb 2012 06:04:30 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1S0WTJ-0006EY-9v
	for coreutils@gnu.org; Thu, 23 Feb 2012 06:04:29 -0500
Received: from mx.meyering.net ([88.168.87.75]:60296)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1S0WTJ-0006Dz-35
	for coreutils@gnu.org; Thu, 23 Feb 2012 06:04:25 -0500
Received: from rho.meyering.net (localhost.localdomain [127.0.0.1])
	by rho.meyering.net (Acme Bit-Twister) with ESMTP id 0A6F460052;
	Thu, 23 Feb 2012 12:04:23 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: =?iso-8859-1?Q?J=E9r=E9my?= Compostella <jeremy.compostella@gmail.com>
Subject: Re: bug#8736: chmod -p --parents
In-Reply-To: <878vju3096.fsf@Apollo.jerryland.fr> (=?iso-8859-1?Q?=22J=E9r?=
	=?iso-8859-1?Q?=E9my?= Compostella"'s
	message of "Thu, 23 Feb 2012 00:40:37 +0100")
References: <878vju3096.fsf@Apollo.jerryland.fr>
Date: Thu, 23 Feb 2012 12:04:22 +0100
Message-ID: <8739a1u7yh.fsf@rho.meyering.net>
Lines: 51
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 88.168.87.75
Cc: 8736@debbugs.gnu.org, coreutils@gnu.org, francky.leyn@telenet.be
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 23 Feb 2012 11:04:30 -0000

J=E9r=E9my Compostella wrote:
>> On Thu, May 26, 2011 at 10:13:05PM +0200, francky.leyn <at> telenet.be w=
rote:
>> >
>> > In the past, I was an advocate of the -p --parents option for
>> > mkdir. By now this is realised. Now I'm doing the same for chmod.
>> > [...]
>> > I want to be able to execute the following:
>> >
>> > chmod a+rx -p ~/dir1/dir2/dir3/file.ext
>>
>> I like the proposal.
>> > It should be more refined however: the dirs
>> > should be rx, but the file only r.
>>
>> You can use a capital X to automatically handle the directory case. But
>> please take a look at the docs for the _exact_ meaning of X.
> I made a first implementation of this chmod --parents feature (see the
> attached patch). I do not support the second part since 'X' is already
> in used for the "Conditional Executability" feature. I have no idea how
> to support it.
>
> I don't have written the documentation for now. I would like to have
> your opinion about this patch first as I maybe missed some corner case
> or the code should be written another way or even something else ...
>
> I really hope you will find time to look at it.

Hi J=E9r=E9my,

Thank you for the patch, but as usual with option-adding patches,
we have to justify it, i.e., to ask "why not do it some other way?"
Or, "is this too specialized to merit an option?"

Here's a little bash/zsh script that does it:

chmod-parents()
{
  local p=3D$2
  while :; do
    chmod "$1" "$p" || break
    test "$p" =3D . && break
    p=3D$(dirname "$p")
  done
}

And an example showing how to use it:

  p=3Dd1/d2/d3
  mkdir -p $p
  touch $p/file
  chmod-parents 505 $p/file


From MAILER-DAEMON Thu Feb 23 14:05:34 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1S0dyw-0005RR-NG
	for mharc-coreutils@gnu.org; Thu, 23 Feb 2012 14:05:34 -0500
Received: from eggs.gnu.org ([140.186.70.92]:60869)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1S0dyt-0005QP-Na
	for coreutils@gnu.org; Thu, 23 Feb 2012 14:05:33 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1S0dys-0002Vh-20
	for coreutils@gnu.org; Thu, 23 Feb 2012 14:05:31 -0500
Received: from mail-gx0-f169.google.com ([209.85.161.169]:41166)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1S0dyr-0002VX-TY
	for coreutils@gnu.org; Thu, 23 Feb 2012 14:05:30 -0500
Received: by ggnf1 with SMTP id f1so922963ggn.0
	for <coreutils@gnu.org>; Thu, 23 Feb 2012 11:05:29 -0800 (PST)
Received-SPF: pass (google.com: domain of jeremy.compostella@gmail.com
	designates 10.50.89.232 as permitted sender)
	client-ip=10.50.89.232; 
Authentication-Results: mr.google.com; spf=pass (google.com: domain of
	jeremy.compostella@gmail.com designates 10.50.89.232 as
	permitted sender) smtp.mail=jeremy.compostella@gmail.com;
	dkim=pass header.i=jeremy.compostella@gmail.com
Received: from mr.google.com ([10.50.89.232])
	by 10.50.89.232 with SMTP id br8mr3994359igb.30.1330023929493 (num_hops
	= 1); Thu, 23 Feb 2012 11:05:29 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma;
	h=mime-version:in-reply-to:references:from:date:message-id:subject:to
	:cc:content-type;
	bh=JI657FvzGsDEhk0oeyIMIgYIvrvtNjzSqw2bi0gOeOU=;
	b=aulj3sWDPaUyaHvLioo6vXRuUkOUjg/GpDuKBbjnsErhASB870u/bAa6/RX446SOie
	xIANPr2B/omV6tt9RQ0o14WA9ouo5O8K7gySR6UXIpzmARITyP3lmu2uKMlUhllWZOQu
	prJOalrr0kJDWIgZh5AXHhK0fZM/gRq1z8fDg=
Received: by 10.50.89.232 with SMTP id br8mr3319244igb.30.1330023929287; Thu,
	23 Feb 2012 11:05:29 -0800 (PST)
MIME-Version: 1.0
Received: by 10.42.29.196 with HTTP; Thu, 23 Feb 2012 11:05:09 -0800 (PST)
In-Reply-To: <8739a1u7yh.fsf@rho.meyering.net>
References: <878vju3096.fsf@Apollo.jerryland.fr>
	<8739a1u7yh.fsf@rho.meyering.net>
From: =?ISO-8859-1?Q?J=E9r=E9my_Compostella?= <jeremy.compostella@gmail.com>
Date: Thu, 23 Feb 2012 20:05:09 +0100
Message-ID: <CA+TATJEK_-m3E1-LWsEpiRy_0a+CkYingn02A89_G-hAZ+9yCQ@mail.gmail.com>
Subject: Re: bug#8736: chmod -p --parents
To: Jim Meyering <jim@meyering.net>
Content-Type: multipart/alternative; boundary=e89a8f3ba429d915de04b9a65355
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.85.161.169
Cc: 8736@debbugs.gnu.org, coreutils@gnu.org, francky.leyn@telenet.be
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 23 Feb 2012 19:05:33 -0000

--e89a8f3ba429d915de04b9a65355
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

OK, I understand. My answers to both questions:

1. why not do it some other way?

First, it is very convenient to do it that way because it is very simple
and it is more consistent with other commands that already provide it:

$ mkdir -p a/b/c d/e
$ chmod g+w -p a/b/c d/e

IMHO, the way like the one you propose misses the consistency point.

Don't misunderstand me, I don't want to force a new feature if someone
has a pertinent argument to oppose to. But this one has already been
proposed by other people and I really think it is worth to have it too.

2. Is this too specialized to merit an option?

I don't think so. This option already exists for other commands like
`mkdir' and `cp' to deserve the same purpose.

Cheers,

J=E9r=E9my

--e89a8f3ba429d915de04b9a65355
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div>OK, I understand. My answers to both questions:</div><div><br></div><d=
iv>1. why not do it some other way?</div><div><br></div><div>First, it is v=
ery convenient to do it that way because it is very simple</div><div>and it=
 is more consistent with other commands that already provide it:</div>

<div><br></div><div>$ mkdir -p a/b/c d/e</div><div>$ chmod g+w -p a/b/c d/e=
</div><div><br></div><div>IMHO, the way like the one you propose misses the=
 consistency point.</div><div><br></div><div>Don&#39;t misunderstand me, I =
don&#39;t want to force a new feature if someone</div>

<div>has a pertinent argument to oppose to. But this one has already been</=
div><div>proposed by other people and I really think it is worth to have it=
 too.</div><div><br></div><div>2. Is this too specialized to merit an optio=
n?</div>

<div><br></div><div>I don&#39;t think so. This option already exists for ot=
her commands like</div><div>`mkdir&#39; and `cp&#39; to deserve the same pu=
rpose.</div><div><br></div><div>Cheers,</div><div><br></div><div>J=E9r=E9my=
</div>

<div><br></div>

--e89a8f3ba429d915de04b9a65355--


From MAILER-DAEMON Thu Feb 23 14:40:29 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1S0eWj-0001dF-6E
	for mharc-coreutils@gnu.org; Thu, 23 Feb 2012 14:40:29 -0500
Received: from eggs.gnu.org ([140.186.70.92]:58220)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1S0eWd-0001d9-Qg
	for coreutils@gnu.org; Thu, 23 Feb 2012 14:40:27 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1S0eWc-0000VS-Q9
	for coreutils@gnu.org; Thu, 23 Feb 2012 14:40:23 -0500
Received: from mx.meyering.net ([88.168.87.75]:33487)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1S0eWc-0000VK-JF
	for coreutils@gnu.org; Thu, 23 Feb 2012 14:40:22 -0500
Received: from rho.meyering.net (localhost.localdomain [127.0.0.1])
	by rho.meyering.net (Acme Bit-Twister) with ESMTP id 5082460052;
	Thu, 23 Feb 2012 20:40:21 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: =?iso-8859-1?Q?J=E9r=E9my?= Compostella <jeremy.compostella@gmail.com>
Subject: Re: bug#8736: chmod -p --parents
In-Reply-To: <CA+TATJEK_-m3E1-LWsEpiRy_0a+CkYingn02A89_G-hAZ+9yCQ@mail.gmail.com>
	(=?iso-8859-1?Q?=22J=E9r=E9my?= Compostella"'s message of "Thu,
	23 Feb 2012 20:05:09 +0100")
References: <878vju3096.fsf@Apollo.jerryland.fr>
	<8739a1u7yh.fsf@rho.meyering.net>
	<CA+TATJEK_-m3E1-LWsEpiRy_0a+CkYingn02A89_G-hAZ+9yCQ@mail.gmail.com>
Date: Thu, 23 Feb 2012 20:40:21 +0100
Message-ID: <87ipixqqxm.fsf@rho.meyering.net>
Lines: 34
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 88.168.87.75
Cc: 8736@debbugs.gnu.org, coreutils@gnu.org, francky.leyn@telenet.be
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 23 Feb 2012 19:40:27 -0000

J=E9r=E9my Compostella wrote:
> OK, I understand. My answers to both questions:
>
> 1. why not do it some other way?
>
> First, it is very convenient to do it that way because it is very simple
> and it is more consistent with other commands that already provide it:
>
> $ mkdir -p a/b/c d/e
> $ chmod g+w -p a/b/c d/e
>
> IMHO, the way like the one you propose misses the consistency point.
>
> Don't misunderstand me, I don't want to force a new feature if someone
> has a pertinent argument to oppose to. But this one has already been
> proposed by other people and I really think it is worth to have it too.

When a small bash/perl/python script can provide the same functionality,
that means the proposed feature requires more justification.

By your arguments, chown, chgrp and chcon should also support this new opti=
on.
Sorry, but "consistency" and "convenience" are not sufficient, here.
It's just as convenient to use a small script on those rare occasions
that one requires this behavior.

Sorry to reject the idea, but if we were to accept new options too easily
many of the coreutils programs would be larger and harder to maintain,
for relatively little added value.  Part of our job as maintainers is
to ensure that each new feature really is well justified.

> 2. Is this too specialized to merit an option?
>
> I don't think so. This option already exists for other commands like
> `mkdir' and `cp' to deserve the same purpose.


From MAILER-DAEMON Thu Feb 23 15:00:19 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1S0epu-0005Nd-UK
	for mharc-coreutils@gnu.org; Thu, 23 Feb 2012 15:00:18 -0500
Received: from eggs.gnu.org ([140.186.70.92]:60999)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1S0epj-00055D-Q3
	for coreutils@gnu.org; Thu, 23 Feb 2012 15:00:15 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1S0eph-0004Cy-55
	for coreutils@gnu.org; Thu, 23 Feb 2012 15:00:07 -0500
Received: from mx.meyering.net ([88.168.87.75]:33563)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1S0epg-0004Ai-U7
	for coreutils@gnu.org; Thu, 23 Feb 2012 15:00:05 -0500
Received: from rho.meyering.net (localhost.localdomain [127.0.0.1])
	by rho.meyering.net (Acme Bit-Twister) with ESMTP id EC05F60188;
	Thu, 23 Feb 2012 21:00:02 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: =?iso-8859-1?Q?J=E9r=E9my?= Compostella <jeremy.compostella@gmail.com>
Subject: Re: bug#8736: chmod -p --parents
In-Reply-To: <87ipixqqxm.fsf@rho.meyering.net> (Jim Meyering's message of
	"Thu, 23 Feb 2012 20:40:21 +0100")
References: <878vju3096.fsf@Apollo.jerryland.fr>
	<8739a1u7yh.fsf@rho.meyering.net>
	<CA+TATJEK_-m3E1-LWsEpiRy_0a+CkYingn02A89_G-hAZ+9yCQ@mail.gmail.com>
	<87ipixqqxm.fsf@rho.meyering.net>
Date: Thu, 23 Feb 2012 21:00:02 +0100
Message-ID: <8739a1qq0t.fsf@rho.meyering.net>
Lines: 10
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 88.168.87.75
Cc: 8736@debbugs.gnu.org, coreutils@gnu.org, francky.leyn@telenet.be
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 23 Feb 2012 20:00:16 -0000

Jim Meyering wrote:
...
> Sorry to reject the idea, but if we were to accept new options too easily
> many of the coreutils programs would be larger and harder to maintain,
> for relatively little added value.  Part of our job as maintainers is
> to ensure that each new feature really is well justified.

Also, please ping the list before investing in even a small project
like this.  It's easier to discuss a proposal like this when there
isn't an already-written patch hanging in the balance.


From MAILER-DAEMON Thu Feb 23 15:09:18 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1S0eyc-0006Sc-G6
	for mharc-coreutils@gnu.org; Thu, 23 Feb 2012 15:09:18 -0500
Received: from eggs.gnu.org ([140.186.70.92]:51944)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <harald@redhat.com>) id 1S0dx9-0005Gv-5X
	for coreutils@gnu.org; Thu, 23 Feb 2012 14:03:47 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <harald@redhat.com>) id 1S0dx2-0001z8-Eh
	for coreutils@gnu.org; Thu, 23 Feb 2012 14:03:42 -0500
Received: from mx1.redhat.com ([209.132.183.28]:45065)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <harald@redhat.com>) id 1S0dx2-0001z1-0C
	for coreutils@gnu.org; Thu, 23 Feb 2012 14:03:36 -0500
Received: from int-mx10.intmail.prod.int.phx2.redhat.com
	(int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id q1NJ3YDZ014299
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK)
	for <coreutils@gnu.org>; Thu, 23 Feb 2012 14:03:34 -0500
Received: from lenovo.str.redhat.com (vpn1-6-11.ams2.redhat.com [10.36.6.11])
	by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with
	ESMTP id q1NJ3XAY006103; Thu, 23 Feb 2012 14:03:33 -0500
From: harald@redhat.com
To: coreutils@gnu.org
Subject: [PATCH] ln: add the --relative option
Date: Thu, 23 Feb 2012 20:03:24 +0100
Message-Id: <1330023804-20695-1-git-send-email-harald@redhat.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23
Content-Transfer-Encoding: quoted-printable
X-MIME-Autoconverted: from 8bit to quoted-printable by mx1.redhat.com id
	q1NJ3YDZ014299
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.132.183.28
X-Mailman-Approved-At: Thu, 23 Feb 2012 15:09:16 -0500
Cc: Harald Hoyer <harald@redhat.com>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 23 Feb 2012 19:03:47 -0000

From: Harald Hoyer <harald@redhat.com>

With the "--relative --symbolic" options, ln computes the relative
symbolic link for the user.

So, ln works just as cp, but creates relative symbolic links instead
of copying the file.

I miss this feature since the beginning of using ln.

$ tree ./
./
`-- usr
    |-- bin
    `-- lib
        `-- foo
            `-- foo

4 directories, 1 file

$ ln -s -v --relative usr/lib/foo/foo usr/bin/foo
=E2=80=98usr/bin/foo=E2=80=99 -> =E2=80=98../lib/foo/foo=E2=80=99

$ tree ./
./
`-- usr
    |-- bin
    |   `-- foo -> ../lib/foo/foo
    `-- lib
        `-- foo
            `-- foo

4 directories, 2 files

$ ln -s -v --relative usr/bin/foo usr/lib/foo/link-to-foo
=E2=80=98usr/lib/foo/link-to-foo=E2=80=99 -> =E2=80=98foo=E2=80=99

$ tree ./
./
`-- usr
    |-- bin
    |   `-- foo -> ../lib/foo/foo
    `-- lib
        `-- foo
            |-- link-to-foo -> foo
            `-- foo

4 directories, 3 files
---
 src/ln.c |   89 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++=
+++++-
 1 files changed, 88 insertions(+), 1 deletions(-)

diff --git a/src/ln.c b/src/ln.c
index d984fd7..6bd6e68 100644
--- a/src/ln.c
+++ b/src/ln.c
@@ -31,6 +31,7 @@
 #include "quote.h"
 #include "same.h"
 #include "yesno.h"
+#include "canonicalize.h"
=20
 /* The official name of this program (e.g., no 'g' prefix).  */
 #define PROGRAM_NAME "ln"
@@ -45,6 +46,9 @@ static enum backup_type backup_type;
 /* If true, make symbolic links; otherwise, make hard links.  */
 static bool symbolic_link;
=20
+/* If true, make symbolic links relative  */
+static bool relative;
+
 /* If true, hard links are logical rather than physical.  */
 static bool logical =3D !!LINK_FOLLOWS_SYMLINKS;
=20
@@ -90,6 +94,7 @@ static struct option const long_options[] =3D
   {"target-directory", required_argument, NULL, 't'},
   {"logical", no_argument, NULL, 'L'},
   {"physical", no_argument, NULL, 'P'},
+  {"relative", no_argument, NULL, 'r'},
   {"symbolic", no_argument, NULL, 's'},
   {"verbose", no_argument, NULL, 'v'},
   {GETOPT_HELP_OPTION_DECL},
@@ -120,6 +125,71 @@ target_directory_operand (char const *file)
   return is_a_dir;
 }
=20
+static const char *
+convert_abs_rel (const char *from, const char *target)
+{
+  /* we use the 4*MAXPATHLEN, which should not overrun */
+  char relative_from[MAXPATHLEN*4];
+  char *realtarget=3DNULL, *realfrom=3DNULL;
+  int level=3D0, fromlevel=3D0, targetlevel=3D0;
+  int l, i, rl;
+
+  realtarget =3D xstrdup(canonicalize_filename_mode (target, CAN_MISSING=
));
+  realfrom =3D xstrdup(canonicalize_filename_mode (from, CAN_MISSING));
+
+  /* now calculate the relative path from <from> to <target> and
+     store it in <relative_from>
+  */
+  relative_from[0] =3D 0;
+  rl =3D 0;
+
+  /* count the pathname elements of realtarget */
+  for(targetlevel=3D0, i =3D 0; realtarget[i]; i++)
+    if (realtarget[i] =3D=3D '/')
+      targetlevel++;
+
+  /* count the pathname elements of realfrom */
+  for(fromlevel=3D0, i =3D 0; realfrom[i]; i++)
+    if (realfrom[i] =3D=3D '/')
+      fromlevel++;
+
+  /* count the pathname elements, which are common for both paths */
+  for(level=3D0, i =3D 0;
+      realtarget[i] && (realtarget[i] =3D=3D realfrom[i]);
+      i++)
+    if (realtarget[i] =3D=3D '/')
+      level++;
+
+  /* add "../" to the relative_from path, until the common pathname is
+     reached */
+  for(i =3D level; i < targetlevel; i++)
+    {
+      if (i !=3D level)
+	relative_from[rl++] =3D '/';
+      relative_from[rl++] =3D '.';
+      relative_from[rl++] =3D '.';
+    }
+
+  /* set l to the next uncommon pathname element in realfrom */
+  for(l =3D 1, i =3D 1; i < level; i++)
+    for(l++; realfrom[l] && realfrom[l] !=3D '/'; l++);
+  /* skip next '/' */
+  l++;
+
+  /* append the uncommon rest of realfrom to the relative_from path */
+  for(i =3D level; i <=3D fromlevel; i++)
+    {
+      if(rl)
+	relative_from[rl++] =3D '/';
+      while(realfrom[l] && realfrom[l] !=3D '/')
+	relative_from[rl++] =3D realfrom[l++];
+      l++;
+    }
+  relative_from[rl] =3D 0;
+
+  return xstrdup(relative_from);
+}
+
 /* Make a link DEST to the (usually) existing file SOURCE.
    Symbolic links to nonexistent files are allowed.
    Return true if successful.  */
@@ -246,6 +316,12 @@ do_link (const char *source, const char *dest)
         }
     }
=20
+  if (relative)
+    {
+      source =3D convert_abs_rel(source, dest);
+    }
+
+
   ok =3D ((symbolic_link ? symlink (source, dest)
          : linkat (AT_FDCWD, source, AT_FDCWD, dest,
                    logical ? AT_SYMLINK_FOLLOW : 0))
@@ -367,6 +443,7 @@ Mandatory arguments to long options are mandatory for=
 short options too.\n\
   -n, --no-dereference        treat LINK_NAME as a normal file if\n\
                                 it is a symbolic link to a directory\n\
   -P, --physical              make hard links directly to symbolic links=
\n\
+  -r, --relative              create softlink relative to LINK_NAME\n\
   -s, --symbolic              make symbolic links instead of hard links\=
n\
 "), stdout);
       fputs (_("\
@@ -429,7 +506,7 @@ main (int argc, char **argv)
   symbolic_link =3D remove_existing_files =3D interactive =3D verbose
     =3D hard_dir_link =3D false;
=20
-  while ((c =3D getopt_long (argc, argv, "bdfinst:vFLPS:T", long_options=
, NULL))
+  while ((c =3D getopt_long (argc, argv, "bdfinrst:vFLPS:T", long_option=
s, NULL))
          !=3D -1)
     {
       switch (c)
@@ -460,6 +537,9 @@ main (int argc, char **argv)
         case 'P':
           logical =3D false;
           break;
+        case 'r':
+          relative =3D true;
+          break;
         case 's':
           symbolic_link =3D true;
           break;
@@ -539,6 +619,13 @@ main (int argc, char **argv)
                  ? xget_version (_("backup type"), version_control_strin=
g)
                  : no_backups);
=20
+  if (relative && !symbolic_link)
+    {
+        error (EXIT_FAILURE, 0,
+               _("cannot do --relative without --symbolic"));
+    }
+
+
   if (target_directory)
     {
       int i;
--=20
1.7.9.1



From MAILER-DAEMON Thu Feb 23 15:09:18 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1S0eyc-0006TG-MP
	for mharc-coreutils@gnu.org; Thu, 23 Feb 2012 15:09:18 -0500
Received: from eggs.gnu.org ([140.186.70.92]:46894)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <francky.leyn@telenet.be>) id 1S0eb6-0002xr-QP
	for coreutils@gnu.org; Thu, 23 Feb 2012 14:45:05 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <francky.leyn@telenet.be>) id 1S0eb5-0001FN-Po
	for coreutils@gnu.org; Thu, 23 Feb 2012 14:45:00 -0500
Received: from juliette.telenet-ops.be ([195.130.137.74]:38638)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <francky.leyn@telenet.be>) id 1S0eb5-0001FA-Ga
	for coreutils@gnu.org; Thu, 23 Feb 2012 14:44:59 -0500
Received: from zcsnocm15.telenet-ops.be ([213.224.144.79])
	by juliette.telenet-ops.be with bizsmtp
	id dXkx1i0011j046l06Xkxlf; Thu, 23 Feb 2012 20:44:57 +0100
Date: Thu, 23 Feb 2012 20:44:56 +0100 (CET)
From: francky.leyn@telenet.be
To: Jim Meyering <jim@meyering.net>
Subject: Re: bug#8736: chmod -p --parents
Message-ID: <65b4578a-bbf1-4e6f-8723-470c432cc5e4@chioma.telenet-ops.be>
In-Reply-To: <87ipixqqxm.fsf@rho.meyering.net>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-Originating-IP: [213.224.144.4]
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 195.130.137.74
X-Mailman-Approved-At: Thu, 23 Feb 2012 15:09:16 -0500
Cc: 8736@debbugs.gnu.org, Francky Leyn <francky@leyn.eu>,
	=?utf-8?B?SsOpcsOpbXk=?= Compostella <jeremy.compostella@gmail.com>,
	coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 23 Feb 2012 19:45:05 -0000

Hello,

I for sure hope the proposal gets through.

Best regards,

Francky

----- Oorspronkelijk e-mail -----
> J=C3=A9r=C3=A9my Compostella wrote:
> > OK, I understand. My answers to both questions:
> >
> > 1. why not do it some other way?
> >
> > First, it is very convenient to do it that way because it is very
> > simple
> > and it is more consistent with other commands that already provide
> > it:
> >
> > $ mkdir -p a/b/c d/e
> > $ chmod g+w -p a/b/c d/e
> >
> > IMHO, the way like the one you propose misses the consistency
> > point.
> >
> > Don't misunderstand me, I don't want to force a new feature if
> > someone
> > has a pertinent argument to oppose to. But this one has already
> > been
> > proposed by other people and I really think it is worth to have it
> > too.
>=20
> When a small bash/perl/python script can provide the same
> functionality,
> that means the proposed feature requires more justification.
>=20
> By your arguments, chown, chgrp and chcon should also support this
> new option.
> Sorry, but "consistency" and "convenience" are not sufficient, here.
> It's just as convenient to use a small script on those rare occasions
> that one requires this behavior.
>=20
> Sorry to reject the idea, but if we were to accept new options too
> easily
> many of the coreutils programs would be larger and harder to
> maintain,
> for relatively little added value.  Part of our job as maintainers is
> to ensure that each new feature really is well justified.
>=20
> > 2. Is this too specialized to merit an option?
> >
> > I don't think so. This option already exists for other commands
> > like
> > `mkdir' and `cp' to deserve the same purpose.
>=20


From MAILER-DAEMON Thu Feb 23 15:09:18 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1S0eyc-0006Tj-PB
	for mharc-coreutils@gnu.org; Thu, 23 Feb 2012 15:09:18 -0500
Received: from eggs.gnu.org ([140.186.70.92]:54822)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <harald@redhat.com>) id 1S0egp-0003fk-Is
	for coreutils@gnu.org; Thu, 23 Feb 2012 14:50:56 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <harald@redhat.com>) id 1S0egg-0002KG-D1
	for coreutils@gnu.org; Thu, 23 Feb 2012 14:50:55 -0500
Received: from mx1.redhat.com ([209.132.183.28]:47194)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <harald@redhat.com>) id 1S0egf-0002J9-PM
	for coreutils@gnu.org; Thu, 23 Feb 2012 14:50:46 -0500
Received: from int-mx10.intmail.prod.int.phx2.redhat.com
	(int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id q1NJohi2003800
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK)
	for <coreutils@gnu.org>; Thu, 23 Feb 2012 14:50:43 -0500
Received: from lenovo.str.redhat.com (vpn1-6-11.ams2.redhat.com [10.36.6.11])
	by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with
	ESMTP id q1NJog6r025501; Thu, 23 Feb 2012 14:50:43 -0500
From: harald@redhat.com
To: coreutils@gnu.org
Subject: [PATCH v2] ln: add the --relative option
Date: Thu, 23 Feb 2012 20:50:27 +0100
Message-Id: <1330026627-21526-1-git-send-email-harald@redhat.com>
In-Reply-To: <1330023804-20695-1-git-send-email-harald@redhat.com>
References: <1330023804-20695-1-git-send-email-harald@redhat.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23
Content-Transfer-Encoding: quoted-printable
X-MIME-Autoconverted: from 8bit to quoted-printable by mx1.redhat.com id
	q1NJohi2003800
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.132.183.28
X-Mailman-Approved-At: Thu, 23 Feb 2012 15:09:16 -0500
Cc: Harald Hoyer <harald@redhat.com>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 23 Feb 2012 19:50:56 -0000

From: Harald Hoyer <harald@redhat.com>

With the "--relative --symbolic" options, ln computes the relative
symbolic link for the user.

So, ln works just as cp, but creates relative symbolic links instead
of copying the file.

I miss this feature since the beginning of using ln.

$ tree ./
./
`-- usr
    |-- bin
    `-- lib
        `-- foo
            `-- foo

4 directories, 1 file

$ ln -s -v --relative usr/lib/foo/foo usr/bin/foo
=E2=80=98usr/bin/foo=E2=80=99 -> =E2=80=98../lib/foo/foo=E2=80=99

$ tree ./
./
`-- usr
    |-- bin
    |   `-- foo -> ../lib/foo/foo
    `-- lib
        `-- foo
            `-- foo

4 directories, 2 files

$ ln -s -v --relative usr/bin/foo usr/lib/foo/link-to-foo
=E2=80=98usr/lib/foo/link-to-foo=E2=80=99 -> =E2=80=98foo=E2=80=99

$ tree ./
./
`-- usr
    |-- bin
    |   `-- foo -> ../lib/foo/foo
    `-- lib
        `-- foo
            |-- link-to-foo -> foo
            `-- foo

4 directories, 3 files
---
 src/ln.c |   99 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++=
+++++-
 1 files changed, 98 insertions(+), 1 deletions(-)

diff --git a/src/ln.c b/src/ln.c
index d984fd7..77965af 100644
--- a/src/ln.c
+++ b/src/ln.c
@@ -31,6 +31,7 @@
 #include "quote.h"
 #include "same.h"
 #include "yesno.h"
+#include "canonicalize.h"
=20
 /* The official name of this program (e.g., no 'g' prefix).  */
 #define PROGRAM_NAME "ln"
@@ -45,6 +46,9 @@ static enum backup_type backup_type;
 /* If true, make symbolic links; otherwise, make hard links.  */
 static bool symbolic_link;
=20
+/* If true, make symbolic links relative  */
+static bool relative;
+
 /* If true, hard links are logical rather than physical.  */
 static bool logical =3D !!LINK_FOLLOWS_SYMLINKS;
=20
@@ -90,6 +94,7 @@ static struct option const long_options[] =3D
   {"target-directory", required_argument, NULL, 't'},
   {"logical", no_argument, NULL, 'L'},
   {"physical", no_argument, NULL, 'P'},
+  {"relative", no_argument, NULL, 'r'},
   {"symbolic", no_argument, NULL, 's'},
   {"verbose", no_argument, NULL, 'v'},
   {GETOPT_HELP_OPTION_DECL},
@@ -120,6 +125,81 @@ target_directory_operand (char const *file)
   return is_a_dir;
 }
=20
+static const char *
+convert_abs_rel (const char *from, const char *target)
+{
+  /* we use the 4*MAXPATHLEN, which should not overrun */
+  char relative_from[MAXPATHLEN*4];
+  char *realtarget=3DNULL, *realfrom=3DNULL;
+  int level=3D0, fromlevel=3D0, targetlevel=3D0;
+  int l, i, rl;
+
+  realtarget =3D xstrdup(canonicalize_filename_mode (target, CAN_MISSING=
));
+  realfrom =3D xstrdup(canonicalize_filename_mode (from, CAN_MISSING));
+
+  if ((realtarget =3D=3D NULL) || (realfrom =3D=3D NULL))
+    {
+      free(realtarget);
+      free(realfrom);
+      return from;
+    }
+
+  /* now calculate the relative path from <from> to <target> and
+     store it in <relative_from>
+  */
+  relative_from[0] =3D 0;
+  rl =3D 0;
+
+  /* count the pathname elements of realtarget */
+  for(targetlevel=3D0, i =3D 0; realtarget[i]; i++)
+    if (realtarget[i] =3D=3D '/')
+      targetlevel++;
+
+  /* count the pathname elements of realfrom */
+  for(fromlevel=3D0, i =3D 0; realfrom[i]; i++)
+    if (realfrom[i] =3D=3D '/')
+      fromlevel++;
+
+  /* count the pathname elements, which are common for both paths */
+  for(level=3D0, i =3D 0;
+      realtarget[i] && (realtarget[i] =3D=3D realfrom[i]);
+      i++)
+    if (realtarget[i] =3D=3D '/')
+      level++;
+
+  /* add "../" to the relative_from path, until the common pathname is
+     reached */
+  for(i =3D level; i < targetlevel; i++)
+    {
+      if (i !=3D level)
+	relative_from[rl++] =3D '/';
+      relative_from[rl++] =3D '.';
+      relative_from[rl++] =3D '.';
+    }
+
+  /* set l to the next uncommon pathname element in realfrom */
+  for(l =3D 1, i =3D 1; i < level; i++)
+    for(l++; realfrom[l] && realfrom[l] !=3D '/'; l++);
+  /* skip next '/' */
+  l++;
+
+  /* append the uncommon rest of realfrom to the relative_from path */
+  for(i =3D level; i <=3D fromlevel; i++)
+    {
+      if(rl)
+	relative_from[rl++] =3D '/';
+      while(realfrom[l] && realfrom[l] !=3D '/')
+	relative_from[rl++] =3D realfrom[l++];
+      l++;
+    }
+  relative_from[rl] =3D 0;
+
+  free(realtarget);
+  free(realfrom);
+
+  return xstrdup(relative_from);
+}
+
 /* Make a link DEST to the (usually) existing file SOURCE.
    Symbolic links to nonexistent files are allowed.
    Return true if successful.  */
@@ -246,6 +326,12 @@ do_link (const char *source, const char *dest)
         }
     }
=20
+  if (relative)
+    {
+      source =3D convert_abs_rel(source, dest);
+    }
+
+
   ok =3D ((symbolic_link ? symlink (source, dest)
          : linkat (AT_FDCWD, source, AT_FDCWD, dest,
                    logical ? AT_SYMLINK_FOLLOW : 0))
@@ -367,6 +453,7 @@ Mandatory arguments to long options are mandatory for=
 short options too.\n\
   -n, --no-dereference        treat LINK_NAME as a normal file if\n\
                                 it is a symbolic link to a directory\n\
   -P, --physical              make hard links directly to symbolic links=
\n\
+  -r, --relative              create softlink relative to LINK_NAME\n\
   -s, --symbolic              make symbolic links instead of hard links\=
n\
 "), stdout);
       fputs (_("\
@@ -429,7 +516,7 @@ main (int argc, char **argv)
   symbolic_link =3D remove_existing_files =3D interactive =3D verbose
     =3D hard_dir_link =3D false;
=20
-  while ((c =3D getopt_long (argc, argv, "bdfinst:vFLPS:T", long_options=
, NULL))
+  while ((c =3D getopt_long (argc, argv, "bdfinrst:vFLPS:T", long_option=
s, NULL))
          !=3D -1)
     {
       switch (c)
@@ -460,6 +547,9 @@ main (int argc, char **argv)
         case 'P':
           logical =3D false;
           break;
+        case 'r':
+          relative =3D true;
+          break;
         case 's':
           symbolic_link =3D true;
           break;
@@ -539,6 +629,13 @@ main (int argc, char **argv)
                  ? xget_version (_("backup type"), version_control_strin=
g)
                  : no_backups);
=20
+  if (relative && !symbolic_link)
+    {
+        error (EXIT_FAILURE, 0,
+               _("cannot do --relative without --symbolic"));
+    }
+
+
   if (target_directory)
     {
       int i;
--=20
1.7.9.1



From MAILER-DAEMON Thu Feb 23 16:28:49 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1S0gDZ-0005Jm-Ms
	for mharc-coreutils@gnu.org; Thu, 23 Feb 2012 16:28:49 -0500
Received: from eggs.gnu.org ([140.186.70.92]:40245)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1S0gDX-0005Jf-4U
	for coreutils@gnu.org; Thu, 23 Feb 2012 16:28:48 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1S0gDV-00046h-UY
	for coreutils@gnu.org; Thu, 23 Feb 2012 16:28:47 -0500
Received: from mx1.redhat.com ([209.132.183.28]:37881)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1S0gDV-00046b-Np
	for coreutils@gnu.org; Thu, 23 Feb 2012 16:28:45 -0500
Received: from int-mx09.intmail.prod.int.phx2.redhat.com
	(int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id q1NLSiVB009278
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK)
	for <coreutils@gnu.org>; Thu, 23 Feb 2012 16:28:44 -0500
Received: from [10.3.113.70] (ovpn-113-70.phx2.redhat.com [10.3.113.70])
	by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP
	id q1NLShPR023677; Thu, 23 Feb 2012 16:28:43 -0500
Message-ID: <4F46AF8A.8060101@redhat.com>
Date: Thu, 23 Feb 2012 14:28:42 -0700
From: Eric Blake <eblake@redhat.com>
Organization: Red Hat
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:10.0.1) Gecko/20120209 Thunderbird/10.0.1
MIME-Version: 1.0
To: harald@redhat.com
Subject: Re: [PATCH v2] ln: add the --relative option
References: <1330023804-20695-1-git-send-email-harald@redhat.com>
	<1330026627-21526-1-git-send-email-harald@redhat.com>
In-Reply-To: <1330026627-21526-1-git-send-email-harald@redhat.com>
X-Enigmail-Version: 1.3.5
OpenPGP: url=http://people.redhat.com/eblake/eblake.gpg
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature";
	boundary="------------enig43EC85BBFDD12E8139E288BA"
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.132.183.28
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 23 Feb 2012 21:28:48 -0000

This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--------------enig43EC85BBFDD12E8139E288BA
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

On 02/23/2012 12:50 PM, harald@redhat.com wrote:
> From: Harald Hoyer <harald@redhat.com>
>=20
> With the "--relative --symbolic" options, ln computes the relative
> symbolic link for the user.
>=20
> So, ln works just as cp, but creates relative symbolic links instead
> of copying the file.

In general, I like the idea.  I will point out that we now have realpath
that also can compute relative locations to an arbitrary starting point,
so we probably ought to share the code between the two programs rather
than rewriting it.

--=20
Eric Blake   eblake@redhat.com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


--------------enig43EC85BBFDD12E8139E288BA
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)
Comment: Public key at http://people.redhat.com/eblake/eblake.gpg
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQEcBAEBCAAGBQJPRq+LAAoJEKeha0olJ0Nqn4oH/2alH4wC1iF1WciU3zA5wnCz
oCFm16X0a+CUWa34d9xRVBZFYv/Wm8MM9UxVvLlmsIVbsPYu15RPKYxUiNzL8J6t
rpOUDtsfwz37921wWcIa+r9fq13zBQ3B2kh5VBaQSX++/N9HpPAJmKidWhy5VMdb
40JJ6FCLeLJAmFlU8L/xb8TaWGOl4OraD/ip275PFKw+W97OpeeZoe4BZfgMqHd0
r7/GwhL6W4p9oVWwnBGZPMp6ba7xjffpmcNk9vmVXYAw5iaWYamTg/Ukt8RHKWWK
WFWfYo4QyJ9P4DzNbqoDZsbwLadc9L2SdRfltBtRN5sxdxYuDlbyuySxgO64+H0=
=oCqu
-----END PGP SIGNATURE-----

--------------enig43EC85BBFDD12E8139E288BA--


From MAILER-DAEMON Thu Feb 23 17:12:59 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1S0guJ-0001Am-UG
	for mharc-coreutils@gnu.org; Thu, 23 Feb 2012 17:12:59 -0500
Received: from eggs.gnu.org ([140.186.70.92]:35292)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <bob@proulx.com>) id 1S0guG-0000zz-E7
	for coreutils@gnu.org; Thu, 23 Feb 2012 17:12:57 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <bob@proulx.com>) id 1S0guF-0003jf-6i
	for coreutils@gnu.org; Thu, 23 Feb 2012 17:12:56 -0500
Received: from joseki.proulx.com ([216.17.153.58]:51373)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <bob@proulx.com>) id 1S0guF-0003iL-1H
	for coreutils@gnu.org; Thu, 23 Feb 2012 17:12:55 -0500
Received: from hysteria.proulx.com (hysteria.proulx.com [192.168.230.119])
	by joseki.proulx.com (Postfix) with ESMTP id D5DAD211D8
	for <coreutils@gnu.org>; Thu, 23 Feb 2012 15:12:52 -0700 (MST)
Received: by hysteria.proulx.com (Postfix, from userid 1000)
	id BADAD2DCDF; Thu, 23 Feb 2012 15:12:52 -0700 (MST)
Date: Thu, 23 Feb 2012 15:12:52 -0700
From: Bob Proulx <bob@proulx.com>
To: coreutils@gnu.org
Subject: Re: bug#8736: chmod -p --parents
Message-ID: <20120223221252.GA3440@hysteria.proulx.com>
Mail-Followup-To: coreutils@gnu.org
References: <878vju3096.fsf@Apollo.jerryland.fr>
	<8739a1u7yh.fsf@rho.meyering.net>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <8739a1u7yh.fsf@rho.meyering.net>
User-Agent: Mutt/1.5.21 (2010-09-15)
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 2)
X-Received-From: 216.17.153.58
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 23 Feb 2012 22:12:57 -0000

Jim Meyering wrote:
> Here's a little bash/zsh script that does it:

Much of the time find can do this nicely.  Of course this may do more
than the posted script but often the difference isn't important in the
provided context of creating new directories and installing new files
into those new directories.

  find somedir -exec chmod g+w {} +

I know that Jim is very much aware of this but I decided to say it
out loud anyway since this is quite a bit simpler than the script.

Bob


From MAILER-DAEMON Thu Feb 23 19:12:47 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1S0imF-00065d-5P
	for mharc-coreutils@gnu.org; Thu, 23 Feb 2012 19:12:47 -0500
Received: from eggs.gnu.org ([140.186.70.92]:45222)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1S0imC-00065S-Dd
	for coreutils@gnu.org; Thu, 23 Feb 2012 19:12:45 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1S0imB-0002If-89
	for coreutils@gnu.org; Thu, 23 Feb 2012 19:12:44 -0500
Received: from mail3.vodafone.ie ([213.233.128.45]:46081)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1S0imB-0002IM-3g
	for coreutils@gnu.org; Thu, 23 Feb 2012 19:12:43 -0500
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: AuoBACzVRk9tTnDW/2dsb2JhbAAMOIU0qzuEXwEBAQQjDwFGEAsNAQcDAgIFFgsCAgkDAgECAUUGDQEFAgEBtQeKEoEvi0ANKw4EBQoDAwcDAgUBEQEMBQMChR1jD4IXgRYEmzeMbg
Received: from unknown (HELO [192.168.1.79]) ([109.78.112.214])
	by mail3.vodafone.ie with ESMTP; 24 Feb 2012 00:12:40 +0000
Message-ID: <4F46D5F6.4060505@draigBrady.com>
Date: Fri, 24 Feb 2012 00:12:38 +0000
From: =?UTF-8?B?UMOhZHJhaWcgQnJhZHk=?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:6.0) Gecko/20110816 Thunderbird/6.0
MIME-Version: 1.0
To: Eric Blake <eblake@redhat.com>
Subject: Re: [PATCH v2] ln: add the --relative option
References: <1330023804-20695-1-git-send-email-harald@redhat.com>
	<1330026627-21526-1-git-send-email-harald@redhat.com>
	<4F46AF8A.8060101@redhat.com>
In-Reply-To: <4F46AF8A.8060101@redhat.com>
X-Enigmail-Version: 1.3.2
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 213.233.128.45
Cc: coreutils@gnu.org, harald@redhat.com
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 24 Feb 2012 00:12:45 -0000

On 02/23/2012 09:28 PM, Eric Blake wrote:
> On 02/23/2012 12:50 PM, harald@redhat.com wrote:
>> From: Harald Hoyer <harald@redhat.com>
>>
>> With the "--relative --symbolic" options, ln computes the relative
>> symbolic link for the user.
>>
>> So, ln works just as cp, but creates relative symbolic links instead
>> of copying the file.
> 
> In general, I like the idea.  I will point out that we now have realpath
> that also can compute relative locations to an arbitrary starting point,
> so we probably ought to share the code between the two programs rather
> than rewriting it.

I like the functionality too.
As Eric points out, the new `realpath` util has the
relative path generation logic in it, which should be reused.
To demonstrate the equivalence you could implement this
functionality modulo option handling using `realpath` like:

ln--relative() {
  target="$1"; link_name="$2"
  rtarget="$(realpath -m "$target" --relative-to "$(dirname "$link_name")")"
  ln -s -v "$rtarget" "$link_name"
}

Now given the simplicity of the above, there is the argument
that new options within ln are not needed. We've discussed
on and off the idea of a contrib/ folder in coreutils to
provide generally useful but higher level commands like this.
Worth considering at least.

cheers,
PÃ¡draig.


From MAILER-DAEMON Fri Feb 24 02:59:14 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1S0q3e-0002Qb-Q7
	for mharc-coreutils@gnu.org; Fri, 24 Feb 2012 02:59:14 -0500
Received: from eggs.gnu.org ([140.186.70.92]:34748)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <bernhard.voelker@siemens-enterprise.com>)
	id 1S0q3Z-0002OH-Ep
	for coreutils@gnu.org; Fri, 24 Feb 2012 02:59:13 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <bernhard.voelker@siemens-enterprise.com>)
	id 1S0q3S-0001d8-Lz
	for coreutils@gnu.org; Fri, 24 Feb 2012 02:59:08 -0500
Received: from senmx12-mx.siemens-enterprise.com ([62.134.46.10]:51624)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <bernhard.voelker@siemens-enterprise.com>)
	id 1S0q3S-0001c7-HV
	for coreutils@gnu.org; Fri, 24 Feb 2012 02:59:02 -0500
Received: from MCHP063A.global-ad.net (unknown [172.29.37.61])
	by senmx12-mx.siemens-enterprise.com (Server) with ESMTP id
	DC42D23F0468; Fri, 24 Feb 2012 08:58:56 +0100 (CET)
Received: from MCHP058A.global-ad.net ([172.29.37.55]) by
	MCHP063A.global-ad.net ([172.29.37.61]) with mapi;
	Fri, 24 Feb 2012 08:58:57 +0100
From: "Voelker, Bernhard" <bernhard.voelker@siemens-enterprise.com>
To: "harald@redhat.com" <harald@redhat.com>, "coreutils@gnu.org"
	<coreutils@gnu.org>
Date: Fri, 24 Feb 2012 08:58:56 +0100
Subject: RE: [PATCH v2] ln: add the --relative option
Thread-Topic: [PATCH v2] ln: add the --relative option
Thread-Index: AczyZwizMSBhroYkTOKb6Yu9E3CCRAAYc7jg
Message-ID: <7856072A9D04C24B82DFE2B1112FE38A0F0AFACDAD@MCHP058A.global-ad.net>
References: <1330023804-20695-1-git-send-email-harald@redhat.com>
	<1330026627-21526-1-git-send-email-harald@redhat.com>
In-Reply-To: <1330026627-21526-1-git-send-email-harald@redhat.com>
Accept-Language: de-DE, en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
acceptlanguage: de-DE, en-US
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
X-Received-From: 62.134.46.10
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 24 Feb 2012 07:59:13 -0000

harald@redhat.com wrote:

> With the "--relative --symbolic" options, ln computes the relative
> symbolic link for the user.

I also like this idea.

> $ ln -s -v --relative usr/bin/foo usr/lib/foo/link-to-foo
> 'usr/lib/foo/link-to-foo' -> 'foo'
>=20
> $ tree ./
> ./
> `-- usr
>     |-- bin
>     |   `-- foo -> ../lib/foo/foo
>     `-- lib
>         `-- foo
>             |-- link-to-foo -> foo
>             `-- foo
>=20
> 4 directories, 3 files

In this case, I am be a bit surprised that ln has created
a symlink to the resolved target ('foo') instead of to what
the user told us to do: '../../../bin/foo'.
I'd have expected this only with an additional "resolve" or
"canonicalize" option.

> +  if (relative && !symbolic_link)
> +    {
> +        error (EXIT_FAILURE, 0,
> +               _("cannot do --relative without --symbolic"));
> +    }

Why restrict this to symlinks - it could also be useful for
hardlinks, right?

Have a nice day,
Berny=


From MAILER-DAEMON Fri Feb 24 03:04:35 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1S0q8p-0002vK-Oa
	for mharc-coreutils@gnu.org; Fri, 24 Feb 2012 03:04:35 -0500
Received: from eggs.gnu.org ([140.186.70.92]:60277)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1S0q8i-0002rx-3u
	for coreutils@gnu.org; Fri, 24 Feb 2012 03:04:33 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1S0q8c-0002Xh-Bc
	for coreutils@gnu.org; Fri, 24 Feb 2012 03:04:28 -0500
Received: from mx.meyering.net ([88.168.87.75]:35321)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1S0q8c-0002Xb-5T
	for coreutils@gnu.org; Fri, 24 Feb 2012 03:04:22 -0500
Received: from rho.meyering.net (localhost.localdomain [127.0.0.1])
	by rho.meyering.net (Acme Bit-Twister) with ESMTP id 1B09560052
	for <coreutils@gnu.org>; Fri, 24 Feb 2012 09:04:21 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: coreutils@gnu.org
Subject: Re: bug#8736: chmod -p --parents
In-Reply-To: <20120223221252.GA3440@hysteria.proulx.com> (Bob Proulx's message
	of "Thu, 23 Feb 2012 15:12:52 -0700")
References: <878vju3096.fsf@Apollo.jerryland.fr>
	<8739a1u7yh.fsf@rho.meyering.net>
	<20120223221252.GA3440@hysteria.proulx.com>
Date: Fri, 24 Feb 2012 09:04:21 +0100
Message-ID: <87zkc8odx6.fsf@rho.meyering.net>
Lines: 15
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 88.168.87.75
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 24 Feb 2012 08:04:33 -0000

Bob Proulx wrote:
> Jim Meyering wrote:
>> Here's a little bash/zsh script that does it:
>
> Much of the time find can do this nicely.  Of course this may do more
> than the posted script but often the difference isn't important in the
> provided context of creating new directories and installing new files
> into those new directories.
>
>   find somedir -exec chmod g+w {} +
>
> I know that Jim is very much aware of this but I decided to say it
> out loud anyway since this is quite a bit simpler than the script.

Good point.  Thanks.


From MAILER-DAEMON Fri Feb 24 06:11:38 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1S0t3q-0008TO-0g
	for mharc-coreutils@gnu.org; Fri, 24 Feb 2012 06:11:38 -0500
Received: from eggs.gnu.org ([140.186.70.92]:57084)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1S0t3i-0008T4-F1
	for coreutils@gnu.org; Fri, 24 Feb 2012 06:11:36 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1S0t3b-0002kZ-DP
	for coreutils@gnu.org; Fri, 24 Feb 2012 06:11:28 -0500
Received: from mail3.vodafone.ie ([213.233.128.45]:21689)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1S0t3b-0002kE-7c
	for coreutils@gnu.org; Fri, 24 Feb 2012 06:11:23 -0500
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: ApoBADxvR09tTnDW/2dsb2JhbAAMNrEEhGEBAQEEJwsBRhALDQsJFg8JAwIBAgFFBg0BBQIBAb9FjQQVGAcLAw8NAgYCDwEMBQMChR1jD4MtBJs6jHA
Received: from unknown (HELO [192.168.1.79]) ([109.78.112.214])
	by mail3.vodafone.ie with ESMTP; 24 Feb 2012 11:11:21 +0000
Message-ID: <4F477058.9040303@draigBrady.com>
Date: Fri, 24 Feb 2012 11:11:20 +0000
From: =?windows-1252?Q?P=E1draig_Brady?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:6.0) Gecko/20110816 Thunderbird/6.0
MIME-Version: 1.0
To: "Voelker, Bernhard" <bernhard.voelker@siemens-enterprise.com>
Subject: Re: [PATCH v2] ln: add the --relative option
References: <1330023804-20695-1-git-send-email-harald@redhat.com>
	<1330026627-21526-1-git-send-email-harald@redhat.com>
	<7856072A9D04C24B82DFE2B1112FE38A0F0AFACDAD@MCHP058A.global-ad.net>
In-Reply-To: <7856072A9D04C24B82DFE2B1112FE38A0F0AFACDAD@MCHP058A.global-ad.net>
X-Enigmail-Version: 1.3.2
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: 8bit
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 213.233.128.45
Cc: "coreutils@gnu.org" <coreutils@gnu.org>,
	"harald@redhat.com" <harald@redhat.com>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 24 Feb 2012 11:11:36 -0000

On 02/24/2012 07:58 AM, Voelker, Bernhard wrote:
> harald@redhat.com wrote:
> 
>> With the "--relative --symbolic" options, ln computes the relative
>> symbolic link for the user.
> 
> I also like this idea.
> 
>> $ ln -s -v --relative usr/bin/foo usr/lib/foo/link-to-foo
>> 'usr/lib/foo/link-to-foo' -> 'foo'
>>
>> $ tree ./
>> ./
>> `-- usr
>>     |-- bin
>>     |   `-- foo -> ../lib/foo/foo
>>     `-- lib
>>         `-- foo
>>             |-- link-to-foo -> foo
>>             `-- foo
>>
>> 4 directories, 3 files
> 
> In this case, I am be a bit surprised that ln has created
> a symlink to the resolved target ('foo') instead of to what
> the user told us to do: '../../../bin/foo'.
> I'd have expected this only with an additional "resolve" or
> "canonicalize" option.

Yep a valid point. Extending my previous function a little...

ln--relative() {
  test "$1" = --no-symlinks && { nosym=$1; shift; }
  target="$1"; link_name="$2"
  rtarget="$(realpath $nosym -m "$target" --relative-to "$(dirname "$link_name")")"
  ln -s -v "$rtarget" "$link_name"
}

# ln--relative --no-symlinks usr/bin/foo usr/lib/foo/link-to-foo
‘usr/lib/foo/link-to-foo’ -> ‘../../bin/foo’

Now I am leaning towards adding the --relative option to ln as it's so useful.
However cases like this show the benefit of the more functional approach.
Do we add the --no-symlinks option to ln, probably not.
Perhaps we could key this behavior off the existing -P option of ln.
That's not ideal though.

>> +  if (relative && !symbolic_link)
>> +    {
>> +        error (EXIT_FAILURE, 0,
>> +               _("cannot do --relative without --symbolic"));
>> +    }
> 
> Why restrict this to symlinks - it could also be useful for
> hardlinks, right?

Well hardlinks reference inodes and so are independent
of any naming, whether relative or absolute etc.

cheers,
Pádraig.


From MAILER-DAEMON Fri Feb 24 07:57:34 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1S0uiM-0000Qq-N9
	for mharc-coreutils@gnu.org; Fri, 24 Feb 2012 07:57:34 -0500
Received: from eggs.gnu.org ([140.186.70.92]:60397)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1S0uiJ-0000Pi-3Z
	for coreutils@gnu.org; Fri, 24 Feb 2012 07:57:33 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1S0uiE-0004Ks-2U
	for coreutils@gnu.org; Fri, 24 Feb 2012 07:57:29 -0500
Received: from moutng.kundenserver.de ([212.227.126.187]:51747)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1S0uiD-0004KD-Q9
	for coreutils@gnu.org; Fri, 24 Feb 2012 07:57:26 -0500
Received: from [192.168.2.108] (p4FF736E3.dip.t-dialin.net [79.247.54.227])
	by mrelayeu.kundenserver.de (node=mrbap3) with ESMTP (Nemesis)
	id 0M9XvL-1SA1FE1mlL-00C8h5; Fri, 24 Feb 2012 13:57:15 +0100
Message-ID: <4F47892A.4050400@bernhard-voelker.de>
Date: Fri, 24 Feb 2012 13:57:14 +0100
From: Bernhard Voelker <mail@bernhard-voelker.de>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:10.0.1) Gecko/20120208 Thunderbird/10.0.1
MIME-Version: 1.0
To: =?windows-1252?Q?P=E1draig_Brady?= <P@draigBrady.com>
Subject: Re: [PATCH v2] ln: add the --relative option
References: <1330023804-20695-1-git-send-email-harald@redhat.com>
	<1330026627-21526-1-git-send-email-harald@redhat.com>
	<7856072A9D04C24B82DFE2B1112FE38A0F0AFACDAD@MCHP058A.global-ad.net>
	<4F477058.9040303@draigBrady.com>
In-Reply-To: <4F477058.9040303@draigBrady.com>
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: 8bit
X-Provags-ID: V02:K0:OGmn90ZpCGOJ17PSdJf/FvabqVOyPJovpGlzLevbaYS
	QRN5EJSz+X4NPLp3tee7DET3sCQv2vtfvS6FbepRC2WGcOHRY+
	xsc4b2dmKMi/GV5fIKEyRBJ+9zN0JQWkNQLpok+gpS5Ks91t0K
	ztMc8Pm6Ch9lKVsko0i2+yFNMCLUhIZ/6lRfOS2nlUSv7+im5m
	o8VviWa+z9FZBB5aHKYd5G6UD3SWGhNzUhF60Zq27okfcio0xM
	OSwAg3rZNXWicrOAWkevmvCLeCncww8Ns/Tv2QEaXw8+7o8JP3
	XMJy8e0ivxv5+eQ3gIIxW40s+SnBpFrzWzGIFFP7nuEynXZZwL
	cbkz89Vi5MtvuAiQ6FwgvXr8FHe0C1Ny+5l3ZVabz
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 212.227.126.187
Cc: "harald@redhat.com" <harald@redhat.com>,
	"coreutils@gnu.org" <coreutils@gnu.org>, "Voelker,
	Bernhard" <bernhard.voelker@siemens-enterprise.com>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 24 Feb 2012 12:57:33 -0000

On 02/24/2012 12:11 PM, Pádraig Brady wrote:
> > > +  if (relative && !symbolic_link)
> > > +    {
> > > +        error (EXIT_FAILURE, 0,
> > > +               _("cannot do --relative without --symbolic"));
> > > +    }
>
> > 
> > Why restrict this to symlinks - it could also be useful for
> > hardlinks, right?
>
> Well hardlinks reference inodes and so are independent
> of any naming, whether relative or absolute etc.

Correct, but I think the whole point about --relative is this:
without that, the user has to think about how to get from
the link target to the link, while with --relative, [s]he
has to think from PWD to both names, i.e. one can use e.g.
bash-completion to get both path names.
And this would also be nice for hardlinks.

Have a nice day,
Berny


From MAILER-DAEMON Fri Feb 24 08:04:56 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1S0upU-0001a6-3H
	for mharc-coreutils@gnu.org; Fri, 24 Feb 2012 08:04:56 -0500
Received: from eggs.gnu.org ([140.186.70.92]:54267)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1S0upN-0001XG-HW
	for coreutils@gnu.org; Fri, 24 Feb 2012 08:04:53 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1S0upG-000612-Kx
	for coreutils@gnu.org; Fri, 24 Feb 2012 08:04:48 -0500
Received: from mx1.redhat.com ([209.132.183.28]:54242)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eblake@redhat.com>) id 1S0upG-00060q-Ca
	for coreutils@gnu.org; Fri, 24 Feb 2012 08:04:42 -0500
Received: from int-mx01.intmail.prod.int.phx2.redhat.com
	(int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id q1OD4dfC002700
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK);
	Fri, 24 Feb 2012 08:04:39 -0500
Received: from [10.3.113.70] (ovpn-113-70.phx2.redhat.com [10.3.113.70])
	by int-mx01.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP
	id q1OD4cn4018603; Fri, 24 Feb 2012 08:04:39 -0500
Message-ID: <4F478AE6.3090805@redhat.com>
Date: Fri, 24 Feb 2012 06:04:38 -0700
From: Eric Blake <eblake@redhat.com>
Organization: Red Hat
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:10.0.1) Gecko/20120209 Thunderbird/10.0.1
MIME-Version: 1.0
To: =?windows-1252?Q?P=E1draig_Brady?= <P@draigBrady.com>
Subject: Re: [PATCH v2] ln: add the --relative option
References: <1330023804-20695-1-git-send-email-harald@redhat.com>
	<1330026627-21526-1-git-send-email-harald@redhat.com>
	<7856072A9D04C24B82DFE2B1112FE38A0F0AFACDAD@MCHP058A.global-ad.net>
	<4F477058.9040303@draigBrady.com>
In-Reply-To: <4F477058.9040303@draigBrady.com>
X-Enigmail-Version: 1.3.5
OpenPGP: url=http://people.redhat.com/eblake/eblake.gpg
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature";
	boundary="------------enig3F182EFFB9914172BC8D36F8"
X-Scanned-By: MIMEDefang 2.67 on 10.5.11.11
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 209.132.183.28
Cc: "harald@redhat.com" <harald@redhat.com>,
	"coreutils@gnu.org" <coreutils@gnu.org>, "Voelker,
	Bernhard" <bernhard.voelker@siemens-enterprise.com>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 24 Feb 2012 13:04:53 -0000

This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--------------enig3F182EFFB9914172BC8D36F8
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: quoted-printable

On 02/24/2012 04:11 AM, P=E1draig Brady wrote:
>>
>> Why restrict this to symlinks - it could also be useful for
>> hardlinks, right?
>=20
> Well hardlinks reference inodes and so are independent
> of any naming, whether relative or absolute etc.

Furthermore,

ln ../a b/c

states that ../a must exist, and that b/c now names the same inode, while=


ln -s ../a b/c

states that b/c is a symlink containing the contents ../a, which means
b/c resolves to b/../a, which is (usually) a different location than
=2E./a.  That is, the difference stems from the fact that 'ln' must
resolve the target, and once the target is resolved, no name is stored
in the destination; while 'ln -s' must not resolve the target, and the
target becomes relative to the destination, not the current starting
point.  Therefore, having the ability to create a relative symlink to an
existing file but residing in a directory that you are not currently in
becomes tricky to think about, so having ln add options to make the task
easier seems worthwhile.

I also wonder if it can go both directions; not only can I envision

ln -s --relative /abs/path/to/a b/c

as a way to create b/c as the minimal relative path in relation to b,
rather than as an absolute path, but it might also make sense to have

ln -s --absolute ../a b/c

which says to make b/c a symlink to the absolute path of '../a' as
resolved with respect to the current directory, rather than to the
normal semantics of not resolving ../a and instead installing the
symlink relative to the destination directory b.

Or looking at it slightly differently, maybe the thing to do is propose
two orthogonal switches - one switch that says whether the source name
is to be treated relative to the current working directory rather than
as a raw string that will be applied relative to the target directory,
and one switch that says whether the symlink being installed is relative
or absolute, regardless of whether the command line gave it in a
different form.

Certainly some food for thought.

--=20
Eric Blake   eblake@redhat.com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


--------------enig3F182EFFB9914172BC8D36F8
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)
Comment: Public key at http://people.redhat.com/eblake/eblake.gpg
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQEcBAEBCAAGBQJPR4rmAAoJEKeha0olJ0Nqz14H/jKYWEtW9u30kitMHVFzSYmZ
HU8GF8GGiDH+m5+0byPWzBuJad3/IxCRf4ZyaVrGdJ+84oFTZgCQNiDNt4UYp4e5
LDY1OPEO25hfZw3+uh1JnuTAFedTdY1zCQsorlt48ZxlCmzogBmtDe28/lB26zAI
vZuYdwqc3e2VUYLJ3jShI5zAIEXIks0X5CQO/8JnLmpH2k1FSTFyErBU6YrngBJR
NQVrdGDjMpZpgyzil+5t199uu0qwb6XeckhJuFU9dqo4z7IRop/RBj3T2WwuEl4L
YNEz3DkeIsQObJs6FBkn5hlzgeiipGWeZcHzGhCVCN2h0NKQln6yoCI31NsiRJs=
=PZd/
-----END PGP SIGNATURE-----

--------------enig3F182EFFB9914172BC8D36F8--


From MAILER-DAEMON Fri Feb 24 17:08:01 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1S13J3-0008BG-AX
	for mharc-coreutils@gnu.org; Fri, 24 Feb 2012 17:08:01 -0500
Received: from eggs.gnu.org ([208.118.235.92]:41140)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1S13J0-0008Ah-9k
	for coreutils@gnu.org; Fri, 24 Feb 2012 17:07:59 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1S13Iy-00053c-UQ
	for coreutils@gnu.org; Fri, 24 Feb 2012 17:07:58 -0500
Received: from mail-ww0-f49.google.com ([74.125.82.49]:35089)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1S13Iy-00053Q-N0
	for coreutils@gnu.org; Fri, 24 Feb 2012 17:07:56 -0500
Received: by wgbdt13 with SMTP id dt13so1961347wgb.30
	for <coreutils@gnu.org>; Fri, 24 Feb 2012 14:07:55 -0800 (PST)
Received-SPF: pass (google.com: domain of jeremy.compostella@gmail.com
	designates 10.181.11.227 as permitted sender)
	client-ip=10.181.11.227; 
Authentication-Results: mr.google.com; spf=pass (google.com: domain of
	jeremy.compostella@gmail.com designates 10.181.11.227 as
	permitted sender) smtp.mail=jeremy.compostella@gmail.com;
	dkim=pass header.i=jeremy.compostella@gmail.com
Received: from mr.google.com ([10.181.11.227])
	by 10.181.11.227 with SMTP id el3mr8721077wid.18.1330121275399
	(num_hops = 1); Fri, 24 Feb 2012 14:07:55 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma;
	h=from:to:subject:date:message-id:mime-version:content-type
	:content-transfer-encoding;
	bh=tRwyS4K/YBB9LGIVdmIEWnINwCTfTTZQ+SGetV/kH2M=;
	b=fOYFrZX+D7BARGbkyPoHvBKVZMewYJ2ptsrRoj0Y/5UcrwNWW/hAK/uZ+piE3wZXwV
	hni09MH3B5qy4/7kqIh6eCFyTpir2jB1/xyZLkZsJ9hyLeseeSA4pDE8X7m2uQieB9El
	ZjHlVlxwUTIeOAnPwtWZ9eJk2BbO6My/uLsjc=
Received: by 10.181.11.227 with SMTP id el3mr7015200wid.18.1330121275358;
	Fri, 24 Feb 2012 14:07:55 -0800 (PST)
Received: from Apollo.jerryland.fr ([85.69.251.50])
	by mx.google.com with ESMTPS id q2sm6179404wiy.7.2012.02.24.14.07.53
	(version=TLSv1/SSLv3 cipher=OTHER);
	Fri, 24 Feb 2012 14:07:54 -0800 (PST)
From: =?utf-8?Q?J=C3=A9r=C3=A9my_Compostella?= <jeremy.compostella@gmail.com>
To: coreutils@gnu.org
Subject: split: support unlimited number of split files
Date: Fri, 24 Feb 2012 23:08:18 +0100
Message-ID: <87399z3mwd.fsf@Apollo.jerryland.fr>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 74.125.82.49
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 24 Feb 2012 22:07:59 -0000

All,

I'm interesting in implementing this feature. In fact, I already made a
quick implementation to play with.

I refer to the original thread : "split behavior"
http://lists.gnu.org/archive/html/bug-coreutils/2009-09/msg00217.html

To summarise it (quick version), in the past the split command provided
this unlimited number of split files as its default behavior. But it did
not conform to POSIX, so it has been removed (see
http://git.savannah.gnu.org/gitweb/?p=3Dcoreutils.git;a=3Dcommit;h=3D65cbf7=
d1).

This old behavior was:
$ cat /var/log/messages | split -2 - /tmp/x.
x.aa
x.ab
...
x.yz
x.zaaa
x.zaab
...
x.zyzz
x.zzaaaa
x.zzaaab

But, others in the "split behavior" thread propose something like:
x.aa
...
x.zz
x.zzaa
...
x.zzzz
x.zzzzaa

These two possibilities deserves the same goal, split files order, once
alphabetically sorted, is the correct order.

However, the second possibility does not satisfy me since it will make the
use of the --additional-suffix option break this:
$ cat /var/log/messages | split --additional-suffix=3D.txt -2 - /tmp/x. && =
ls /tmp/x.* | sort
x.aa.txt
...
x.zy.txt
x.zzaa.txt
...
x.zztw.txt
x.zz.txt      <---- :(
x.zztx.txt
...

Therefore, my opinion is : the old behavior is more adapted to the
current split option set.

In the "split behavior" thread it was proposed to look at the
POSIXLY_CORRECT environment variable to activate or not the unlimited
split files behavior. But, I think it's dangerous. Indeed, it breaks the
usual files list: x.aa ... x.zz ... vs. x.aa ... x.yz x.zaa .. (the x.zz
file does not exist anymore). User may be surprised and older scripts
may failed.

Maybe adding a new option or a new argument would be fine, I was
thinking to the following:
* --unlimited-suffixes
* --suffix-length=3Dunlimited or --suffix-length=3Dauto

With this new option (or argument), user would keep the ability to
select the start suffix length. For example:
$ cat /var/log/messages | split --suffix-length=3Dauto --suffix-length 3 -2=
 - /tmp/x.
x.aaa  <--- start with suffix length =3D 3
x.aab
...
x.yzz
x.zaaaa
x.zaaab
...
x.zyzzz
x.zzaaaaa
x.zzaaaab

Cheers,

J=C3=A9r=C3=A9my


From MAILER-DAEMON Fri Feb 24 18:12:39 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1S14Jb-0006Pa-Dv
	for mharc-coreutils@gnu.org; Fri, 24 Feb 2012 18:12:39 -0500
Received: from eggs.gnu.org ([208.118.235.92]:37973)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1S14JW-00069v-4T
	for coreutils@gnu.org; Fri, 24 Feb 2012 18:12:35 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1S14JU-00065F-9B
	for coreutils@gnu.org; Fri, 24 Feb 2012 18:12:33 -0500
Received: from mail3.vodafone.ie ([213.233.128.45]:44632)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1S14JU-00064K-3H
	for coreutils@gnu.org; Fri, 24 Feb 2012 18:12:32 -0500
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: ApMBAKUYSE9tTnDW/2dsb2JhbAAMOIRmVLBhAQEBAwEBAgkXDwFGBQsJAg0LAgIFIQICDwIXLwYNAQcBAReHZhCtGIoGgS+IRYMXBg8IEA4EBQoDAwcDAgUBEQEMCoUdFi0DAxaCKoEWBJMTiCeMcIFU
Received: from unknown (HELO [192.168.1.79]) ([109.78.112.214])
	by mail3.vodafone.ie with ESMTP; 24 Feb 2012 23:12:30 +0000
Message-ID: <4F48195D.60709@draigBrady.com>
Date: Fri, 24 Feb 2012 23:12:29 +0000
From: =?UTF-8?B?UMOhZHJhaWcgQnJhZHk=?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:6.0) Gecko/20110816 Thunderbird/6.0
MIME-Version: 1.0
To: =?UTF-8?B?SsOpcsOpbXkgQ29tcG9zdGVsbGE=?=
 <jeremy.compostella@gmail.com>
Subject: Re: split: support unlimited number of split files
References: <87399z3mwd.fsf@Apollo.jerryland.fr>
In-Reply-To: <87399z3mwd.fsf@Apollo.jerryland.fr>
X-Enigmail-Version: 1.3.2
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 213.233.128.45
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 24 Feb 2012 23:12:36 -0000

On 02/24/2012 10:08 PM, JÃ©rÃ©my Compostella wrote:
> All,
> 
> I'm interesting in implementing this feature. In fact, I already made a
> quick implementation to play with.
> 
> I refer to the original thread : "split behavior"
> http://lists.gnu.org/archive/html/bug-coreutils/2009-09/msg00217.html
> 
> To summarise it (quick version), in the past the split command provided
> this unlimited number of split files as its default behavior. But it did
> not conform to POSIX, so it has been removed (see
> http://git.savannah.gnu.org/gitweb/?p=coreutils.git;a=commit;h=65cbf7d1).


Just to consider POSIX again, it is fairly explicit:

"By default, the names of the output files shall be 'x' , followed by a two-character
suffix from the character set as described above, starting with "aa" , "ab" , "ac" ,
and so on, and continuing until the suffix "zz" , for a maximum of 676 files."

However I think it's incorrect to impose the arbitrary limit. Note the spec also says:

"The -a option was added to overcome the limitation of being able to create only 676 files."

So there doesn't seem to be an intention to limit the number of output files.
I think it's just that alternative solutions were not considered.
Also with numeric suffixes the limit is only 100 files.

> This old behavior was:
> $ cat /var/log/messages | split -2 - /tmp/x.
> x.aa
> x.ab
> ...
> x.yz
> x.zaaa
> x.zaab
> ...
> x.zyzz
> x.zzaaaa
> x.zzaaab
> 
> But, others in the "split behavior" thread propose something like:
> x.aa
> ...
> x.zz
> x.zzaa
> ...
> x.zzzz
> x.zzzzaa
> 
> These two possibilities deserves the same goal, split files order, once
> alphabetically sorted, is the correct order.
> 
> However, the second possibility does not satisfy me since it will make the
> use of the --additional-suffix option break this:
> $ cat /var/log/messages | split --additional-suffix=.txt -2 - /tmp/x. && ls /tmp/x.* | sort
> x.aa.txt
> ...
> x.zy.txt
> x.zzaa.txt
> ...
> x.zztw.txt
> x.zz.txt      <---- :(
> x.zztx.txt
> ...
> 
> Therefore, my opinion is : the old behavior is more adapted to the
> current split option set.

Good. That's what I'd prefer anyway so as to be compatible
with old data sets. Note '.' sorts before digits (-d) too,
so there should be no ordering issues with --additional-suffix=... either.

> In the "split behavior" thread it was proposed to look at the
> POSIXLY_CORRECT environment variable to activate or not the unlimited
> split files behavior. But, I think it's dangerous. Indeed, it breaks the
> usual files list: x.aa ... x.zz ... vs. x.aa ... x.yz x.zaa .. (the x.zz
> file does not exist anymore). User may be surprised and older scripts
> may failed.

We could key the new behavior on POSIXLY_CORRECT, but there is no need IMHO.
Using POSIXLY_CORRECT is not desired and only used as a very last resort.

> Maybe adding a new option or a new argument would be fine, I was
> thinking to the following:
> * --unlimited-suffixes
> * --suffix-length=unlimited or --suffix-length=auto

If we were to add an option --suffix-length=auto is the best IMHO.
But I don't think we even need that. Just do it by default.

> With this new option (or argument), user would keep the ability to
> select the start suffix length. For example:
> $ cat /var/log/messages | split --suffix-length=auto --suffix-length 3 -2 - /tmp/x.
> x.aaa  <--- start with suffix length = 3

No need for that functionality I think.

cheers,
PÃ¡draig.


From MAILER-DAEMON Fri Feb 24 18:57:42 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1S151C-0003rt-BT
	for mharc-coreutils@gnu.org; Fri, 24 Feb 2012 18:57:42 -0500
Received: from eggs.gnu.org ([208.118.235.92]:52812)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1S151A-0003qs-G6
	for coreutils@gnu.org; Fri, 24 Feb 2012 18:57:41 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1S1519-0004N8-7R
	for coreutils@gnu.org; Fri, 24 Feb 2012 18:57:40 -0500
Received: from mail3.vodafone.ie ([213.233.128.45]:22349)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1S1519-0004N3-1o
	for coreutils@gnu.org; Fri, 24 Feb 2012 18:57:39 -0500
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: AqwJACUjSE9tTnDW/2dsb2JhbAAMOIU6rFwBAwGEKg8Bbg0CBRYLAgsDAgECAT8MDQgBAQWICK0digeBL4tQOQ4EBQoDAwcDAgUBEQELAQEBAQIBAgIqgkuCKDcMBgiCOIEWBJs6jHA
Received: from unknown (HELO [192.168.1.79]) ([109.78.112.214])
	by mail3.vodafone.ie with ESMTP; 24 Feb 2012 23:57:36 +0000
Message-ID: <4F4823F0.9090400@draigBrady.com>
Date: Fri, 24 Feb 2012 23:57:36 +0000
From: =?UTF-8?B?UMOhZHJhaWcgQnJhZHk=?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:6.0) Gecko/20110816 Thunderbird/6.0
MIME-Version: 1.0
To: Coreutils <coreutils@gnu.org>
Subject: unix core command index
X-Enigmail-Version: 1.3.2
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 213.233.128.45
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 24 Feb 2012 23:57:41 -0000

I look this stuff up all the time,
so I decided to create an index.
Hopefully others find it useful:

http://www.pixelbeat.org/docs/unix_commands/

cheers,
PÃ¡draig.


From MAILER-DAEMON Sun Feb 26 03:42:09 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1S1ZgH-00048Z-TY
	for mharc-coreutils@gnu.org; Sun, 26 Feb 2012 03:42:09 -0500
Received: from eggs.gnu.org ([208.118.235.92]:59709)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1S1ZgB-00047c-55
	for coreutils@gnu.org; Sun, 26 Feb 2012 03:42:08 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1S1Zg6-0003tH-1Z
	for coreutils@gnu.org; Sun, 26 Feb 2012 03:42:02 -0500
Received: from mx.meyering.net ([88.168.87.75]:44478)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1S1Zg5-0003tB-Me
	for coreutils@gnu.org; Sun, 26 Feb 2012 03:41:57 -0500
Received: from rho.meyering.net (localhost.localdomain [127.0.0.1])
	by rho.meyering.net (Acme Bit-Twister) with ESMTP id 880A860822;
	Sun, 26 Feb 2012 09:41:56 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: Bernhard Voelker <mail@bernhard-voelker.de>
Subject: Re: "ls -l": Avoid unnecessary getxattr() overhead
In-Reply-To: <4F420517.2050804@bernhard-voelker.de> (Bernhard Voelker's
	message of "Mon, 20 Feb 2012 09:32:23 +0100")
References: <87pqds3snt.fsf@Apollo.jerryland.fr>
	<4F30631A.2070009@itwm.fraunhofer.de>
	<87y5scfdiq.fsf@rho.meyering.net> <4F33B5D7.3010909@draigBrady.com>
	<87mx8sf5w0.fsf@rho.meyering.net>
	<4F3437CE.4050308@itwm.fraunhofer.de>
	<4F346CC7.4030608@draigBrady.com> <87r4y3dry0.fsf@rho.meyering.net>
	<4F3710CA.8020402@itwm.fraunhofer.de>
	<8739aao9e2.fsf@rho.meyering.net>
	<4F3E12F9.6080102@bernhard-voelker.de>
	<8762f5lpct.fsf@rho.meyering.net> <4F3E5BBD.80809@redhat.com>
	<874nupk1cx.fsf@rho.meyering.net> <4F3E5E8E.2040802@redhat.com>
	<87lio1ij8b.fsf@rho.meyering.net>
	<4F3E7B1E.3090300@bernhard-voelker.de>
	<8739a9ig96.fsf@rho.meyering.net>
	<4F3EF935.2070700@bernhard-voelker.de>
	<878vk0gujb.fsf@rho.meyering.net>
	<4F420517.2050804@bernhard-voelker.de>
Date: Sun, 26 Feb 2012 09:41:56 +0100
Message-ID: <87ehtix9yj.fsf@rho.meyering.net>
Lines: 71
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 88.168.87.75
Cc: Bernd Schubert <bernd.schubert@itwm.fraunhofer.de>,
	Sven Breuner <sven.breuner@itwm.fraunhofer.de>,
	=?iso-8859-1?Q?J=E9r=E9my?= Compostella <jeremy.compostella@gmail.com>,
	"Daniel P. Berrange" <berrange@redhat.com>, coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 26 Feb 2012 08:42:08 -0000

Bernhard Voelker wrote:

> On 02/18/2012 02:06 PM, Jim Meyering wrote:
>>> > See the attached getxattr-speedup.log.gz - I added a "cat k.c" so that
>>> > you can check that I got your patch right.
>>> > It seems that GCC doesn't like our "__attribute__((destructor)) void p()".
>>> > It basically works in a standalone program, but not with LD_PRELOAD.
>>> > I didn't find much about this. Can this be a 64-bit issue?
>>
>> Is that an open-suse system?
>> I tried on an opensuse system with 2.6.37.6-0.11-desktop,
>> and since there were no getxattr calls at all, no wrapper
>> would run, and hence, your atexit call would never be reached.
>> I've added this comment at the top:
>
> Yes, it is a OpenSuSE 12.1 with kernel 3.1.9-1.4-desktop.
>
>>   # This test is skipped on systems that lack LD_PRELOAD support; that's fine.
>
> It doesn't lack LD_PRELOAD support, it simply doesn't call the destructor.

That comment was not intended to describe your case,
but rather any system that lacks LD_PRELOAD support.

>> + # Similarly, on a system that lacks getxattr altogether, skipping it is fine.
>>
>> and this new skip_ use below:
>>
>> test -f x || skip_ "internal test failure: maybe LD_PRELOAD doesn't work?"
>>
>> Which may mean this test will simply be skipped on your system.
>> But that is fine.
>
> Maybe, but why is LD_PRELOAD honoring the "__attribute__((destructor))",
> and on some systems not?

Good question.
There must be some OpenSuSE 12.1 users willing to investigate.
Filing a bug report should get their attention.

>> +ld -G k.o -lc -o k.so || framework_failure_ 'failed to invoke ld -G ...'
>
> I just added "-lc" for pulling in atexit(). It's not needed in your version.

Thanks.

>From 641143a0cc613a503b80457b3c64dc46c9864980 Mon Sep 17 00:00:00 2001
From: Bernhard Voelker <mail@bernhard-voelker.de>
Date: Fri, 24 Feb 2012 12:22:20 +0100
Subject: [PATCH] tests: remove unnecessary use of -lc

* tests/ls/getxattr-speedup: Remove unneeded use of -lc.
---
 tests/ls/getxattr-speedup |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/ls/getxattr-speedup b/tests/ls/getxattr-speedup
index a35dd69..d32e24a 100755
--- a/tests/ls/getxattr-speedup
+++ b/tests/ls/getxattr-speedup
@@ -48,7 +48,7 @@ EOF

 # Then compile/link it:
 $CC -fPIC -O2 -c k.c || framework_failure_ 'failed to compile with -fPIC'
-ld -G k.o -lc -o k.so || framework_failure_ 'failed to invoke ld -G ...'
+ld -G k.o -o k.so || framework_failure_ 'failed to invoke ld -G ...'

 # Create a few files:
 seq 20 | xargs touch || framework_failure_
--
1.7.9.2.263.g9be8b7


From MAILER-DAEMON Sun Feb 26 04:20:57 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1S1aHp-0006pC-KB
	for mharc-coreutils@gnu.org; Sun, 26 Feb 2012 04:20:57 -0500
Received: from eggs.gnu.org ([208.118.235.92]:50348)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1S1aHi-0006p1-8x
	for coreutils@gnu.org; Sun, 26 Feb 2012 04:20:55 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1S1aHc-00087S-BJ
	for coreutils@gnu.org; Sun, 26 Feb 2012 04:20:50 -0500
Received: from mx.meyering.net ([88.168.87.75]:44584)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1S1aHc-00087M-48
	for coreutils@gnu.org; Sun, 26 Feb 2012 04:20:44 -0500
Received: from rho.meyering.net (localhost.localdomain [127.0.0.1])
	by rho.meyering.net (Acme Bit-Twister) with ESMTP id 634D76103E
	for <coreutils@gnu.org>; Sun, 26 Feb 2012 10:20:43 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: coreutils@gnu.org
Subject: [PATCH] tests: fix help-version on cygwin, where $EXEEXT is nonempty
Date: Sun, 26 Feb 2012 10:20:43 +0100
Message-ID: <878vjqx85w.fsf@rho.meyering.net>
Lines: 39
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 88.168.87.75
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 26 Feb 2012 09:20:56 -0000

FYI, this changes comes thanks to the fact that the nixos/hydra
autobuilder builds cppi on cygwin, which first highlighted this bug.


>From 72addfcac9c109e3bde5f6a398d39f7a188ce77c Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering@redhat.com>
Date: Sun, 19 Feb 2012 18:32:54 +0100
Subject: [PATCH] tests: fix help-version on cygwin, where $EXEEXT is nonempty

* tests/misc/help-version: Remove the $EXEEXT suffix from each
program name before comparing with a literal.
---
 tests/misc/help-version |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/tests/misc/help-version b/tests/misc/help-version
index c89d118..7416062 100755
--- a/tests/misc/help-version
+++ b/tests/misc/help-version
@@ -100,7 +100,7 @@ for lang in C fr da; do
       env $i --help    >/dev/full 2>/dev/null && fail=1
       env $i --version >/dev/full 2>/dev/null && fail=1
       status=$?
-      test $i = [ && prog=lbracket || prog=$i
+      test $i = [ && prog=lbracket || prog=$(echo $i|sed "s/$EXEEXT$//")
       eval "expected=\$expected_failure_status_$prog"
       test x$expected = x && expected=1
       if test $status = $expected; then
@@ -265,7 +265,7 @@ for i in $built_programs; do
   cp $tmp_in $tmp_in2
   mkdir $tmp_dir
   # echo ================== $i
-  test $i = [ && prog=lbracket || prog=$i
+  test $i = [ && prog=lbracket || prog=$(echo $i|sed "s/$EXEEXT$//")
   if type ${prog}_setup > /dev/null 2>&1; then
     ${prog}_setup
   else
--
1.7.9.2.263.g9be8b7


From MAILER-DAEMON Sun Feb 26 09:12:34 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1S1eq2-0005ZJ-SL
	for mharc-coreutils@gnu.org; Sun, 26 Feb 2012 09:12:34 -0500
Received: from eggs.gnu.org ([208.118.235.92]:40000)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1S1ept-0005WQ-Lb
	for coreutils@gnu.org; Sun, 26 Feb 2012 09:12:32 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1S1epm-00019V-FW
	for coreutils@gnu.org; Sun, 26 Feb 2012 09:12:25 -0500
Received: from mail-ww0-f49.google.com ([74.125.82.49]:42361)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jeremy.compostella@gmail.com>) id 1S1epm-00019P-7s
	for coreutils@gnu.org; Sun, 26 Feb 2012 09:12:18 -0500
Received: by wgbdt12 with SMTP id dt12so49400wgb.30
	for <coreutils@gnu.org>; Sun, 26 Feb 2012 06:12:17 -0800 (PST)
Received-SPF: pass (google.com: domain of jeremy.compostella@gmail.com
	designates 10.216.132.94 as permitted sender)
	client-ip=10.216.132.94; 
Authentication-Results: mr.google.com; spf=pass (google.com: domain of
	jeremy.compostella@gmail.com designates 10.216.132.94 as
	permitted sender) smtp.mail=jeremy.compostella@gmail.com;
	dkim=pass header.i=jeremy.compostella@gmail.com
Received: from mr.google.com ([10.216.132.94])
	by 10.216.132.94 with SMTP id n72mr5110495wei.4.1330265537018 (num_hops
	= 1); Sun, 26 Feb 2012 06:12:17 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma;
	h=from:to:subject:date:message-id:mime-version:content-type;
	bh=H7g4/Sby1UayEIvTCRHgTWRjNxZKm4nktTpicLwmX6I=;
	b=U9AWwAyWCmgU7GITd7juKju5SzLTn9vJaTzn2sgcmV2bMe9FXs6hY1hMaQx3lTNVSD
	p23k9CBRA9NGo6oDDWdFg5VBUyh9KosPfuXXqL6Ma0HoeqrUbAKNXzMSYoASrNng+p1+
	ryOiZELmwH5RAQOQNko6jueumr8rgq4Xoi3I8=
Received: by 10.216.132.94 with SMTP id n72mr4077560wei.4.1330265536973;
	Sun, 26 Feb 2012 06:12:16 -0800 (PST)
Received: from Apollo.jerryland.fr ([85.69.251.50])
	by mx.google.com with ESMTPS id s8sm15607410wiz.8.2012.02.26.06.12.15
	(version=TLSv1/SSLv3 cipher=OTHER);
	Sun, 26 Feb 2012 06:12:15 -0800 (PST)
From: =?utf-8?Q?J=C3=A9r=C3=A9my_Compostella?= <jeremy.compostella@gmail.com>
To: coreutils@gnu.org
Subject: [PATCH] doc: fix a wrong option reference in split invocation
Date: Sun, 26 Feb 2012 15:12:41 +0100
Message-ID: <87zkc51y5i.fsf@Apollo.jerryland.fr>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="=-=-="
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 74.125.82.49
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 26 Feb 2012 14:12:32 -0000

--=-=-=
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

All,

I think there is a mistake in the output file names format description
of the split invocation section. Indeed, there is a reference to a
nonexistent option (-r). I think the right option is --number with r/n
(round-robin distribution) as argument I.E. -nr/n.

I propose the attached patch to fix it.

Cheers,

J=C3=A9r=C3=A9my
---
Sent from my GNU/Emacs


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=0001-doc-fix-a-wrong-option-reference-in-split-invocation.patch
Content-Description: doc: fix a wrong option reference in split invocation
 patch

>From ab45578439f34f0f1651e68a58afb2df6c6dd0ea Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?J=C3=A9r=C3=A9my=20Compostella?= <jeremy.compostella@gmail.com>
Date: Sun, 26 Feb 2012 15:05:36 +0100
Subject: [PATCH] doc: fix a wrong option reference in split invocation

* doc/coreutils.texi (split invocation): replace `-r' reference with
`-nr/n'.
---
 doc/coreutils.texi |    5 +++--
 1 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/doc/coreutils.texi b/doc/coreutils.texi
index cc300a8..414626d 100644
--- a/doc/coreutils.texi
+++ b/doc/coreutils.texi
@@ -2990,8 +2990,9 @@ The output files' names consist of @var{prefix} (@samp{x} by default)
 followed by a group of characters (@samp{aa}, @samp{ab}, @dots{} by
 default), such that concatenating the output files in traditional
 sorted order by file name produces the original input file (except
-@option{-r}).  If the output file names are exhausted, @command{split}
-reports an error without deleting the output files that it did create.
+@option{-nr/@var{n}}).  If the output file names are exhausted,
+@command{split} reports an error without deleting the output files
+that it did create.
 
 The program accepts the following options.  Also see @ref{Common options}.
 
-- 
1.7.2.5


--=-=-=--


From MAILER-DAEMON Sun Feb 26 17:56:33 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1S1n17-00047p-Qv
	for mharc-coreutils@gnu.org; Sun, 26 Feb 2012 17:56:33 -0500
Received: from eggs.gnu.org ([208.118.235.92]:36721)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <sven.breuner@itwm.fraunhofer.de>) id 1S1n14-000460-LW
	for coreutils@gnu.org; Sun, 26 Feb 2012 17:56:31 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <sven.breuner@itwm.fraunhofer.de>) id 1S1n13-0008Mk-Ms
	for coreutils@gnu.org; Sun, 26 Feb 2012 17:56:30 -0500
Received: from mailgw1.uni-kl.de ([131.246.120.220]:47364)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <sven.breuner@itwm.fraunhofer.de>) id 1S1n13-0008Me-Eu
	for coreutils@gnu.org; Sun, 26 Feb 2012 17:56:29 -0500
Received: from itwm2.itwm.fhg.de (itwm2.itwm.fhg.de [131.246.191.3])
	by mailgw1.uni-kl.de (8.14.3/8.14.3/Debian-5+lenny1) with ESMTP id
	q1QMuPAW013459
	(version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=NOT)
	for <coreutils@gnu.org>; Sun, 26 Feb 2012 23:56:25 +0100
Received: from mail2.itwm.fhg.de ([131.246.191.79]:57985)
	by itwm2.itwm.fhg.de with esmtps (TLSv1:DES-CBC3-SHA:168)
	(/C=DE/ST=Rheinland-Pfalz/L=Kaiserslautern/O=Fraunhofer
	ITWM/OU=SLG/CN=mail2.itwm.fhg.de)(verified=1) (Exim 4.74 #1)
	id 1S1n0x-0007mg-0O; Sun, 26 Feb 2012 23:56:23 +0100
Message-ID: <4F4AB896.8080907@itwm.fraunhofer.de>
Date: Sun, 26 Feb 2012 23:56:22 +0100
From: Sven Breuner <sven.breuner@itwm.fraunhofer.de>
User-Agent: Mozilla/5.0 (X11; Linux i686 on x86_64;
	rv:10.0.2) Gecko/20120216 Thunderbird/10.0.2
MIME-Version: 1.0
To: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
Subject: Re: "ls -l": Avoid unnecessary getxattr() overhead
References: <87pqds3snt.fsf@Apollo.jerryland.fr>
	<4F30206B.1020802@itwm.fraunhofer.de>
	<CA+TATJHorgsuY7pxmpxxAeWUHRP5MfFfzQP05o=1H6C4ttswdA@mail.gmail.com>
	<4F3032BC.50702@draigBrady.com>
	<4F30631A.2070009@itwm.fraunhofer.de>
	<87y5scfdiq.fsf@rho.meyering.net> <4F33B5D7.3010909@draigBrady.com>
	<87mx8sf5w0.fsf@rho.meyering.net>
	<4F3437CE.4050308@itwm.fraunhofer.de>
	<4F346CC7.4030608@draigBrady.com> <87r4y3dry0.fsf@rho.meyering.net>
	<4F3710CA.8020402@itwm.fraunhofer.de>
	<8739aao9e2.fsf@rho.meyering.net>
	<4F3E12F9.6080102@bernhard-voelker.de>
	<8762f5lpct.fsf@rho.meyering.net>
	<4F3E35C9.7050707@bernhard-voelker.de>
	<87mx8hk6bm.fsf@rho.meyering.net>
	<4F4069D3.5000806@itwm.fraunhofer.de>
	<87pqdbcia0.fsf@rho.meyering.net> <4F40D061.3000206@draigBrady.com>
	<871upqdagt.fsf@rho.meyering.net>
	<4F41762E.1030801@itwm.fraunhofer.de>
	<4F418BAE.20703@draigBrady.com>
	<4F41AA9D.2060007@itwm.fraunhofer.de>
	<4F42267F.60205@draigBrady.com>
In-Reply-To: <4F42267F.60205@draigBrady.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
X-ITWM-CharSet: ISO-8859-1
X-ITWM-Scanned-By: mail2.itwm.fhg.de
Content-Transfer-Encoding: quoted-printable
X-MIME-Autoconverted: from 8bit to quoted-printable by mailgw1.uni-kl.de id
	q1QMuPAW013459
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3)
X-Received-From: 131.246.120.220
Cc: "Daniel P. Berrange" <berrange@redhat.com>,
	Bernhard Voelker <mail@bernhard-voelker.de>,
	Bernd Schubert <bernd.schubert@itwm.fraunhofer.de>, coreutils@gnu.org,
	=?ISO-8859-1?Q?J=E9r?= =?ISO-8859-1?Q?=E9my_Compostella?=
	<jeremy.compostella@gmail.com>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 26 Feb 2012 22:56:31 -0000

P=E1draig Brady wrote on 02/20/2012 11:54 AM:
> On 02/20/2012 02:06 AM, Sven Breuner wrote:
>> P=E1draig Brady wrote on 02/20/2012 12:54 AM:
>>> I just booted RHEL 6.2 with selinux=3D0 on the kernel command line,
>>> so as I could test new files without a context created, and...
>>>
>>> getxattr("security.selinux") on new files on tmpfs, returns ENOTSUP
>>>  [...]
>>
>> Seems to be inconsistent. Some of our RHEL 6.x systems return "unlable=
d" on a tmpfs:
>>  [...]
>
> I thought the difference might be selinux=3D0 boot param,
> versus selinux=3Ddisabled in /etc/selinux/config, but that's not signif=
icant.

You were right, P=E1draig. Booting the machines that previously returned=20
"unlabled" with "selinux=3D0" makes them return EOPNOTSUPP.

Best regards,
Sven


From MAILER-DAEMON Sun Feb 26 18:17:15 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1S1nL9-0007vu-CU
	for mharc-coreutils@gnu.org; Sun, 26 Feb 2012 18:17:15 -0500
Received: from eggs.gnu.org ([208.118.235.92]:45276)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1S1nL7-0007uU-Ew
	for coreutils@gnu.org; Sun, 26 Feb 2012 18:17:14 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1S1nL6-00031m-7P
	for coreutils@gnu.org; Sun, 26 Feb 2012 18:17:13 -0500
Received: from mail2.vodafone.ie ([213.233.128.44]:29717)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1S1nL6-00031g-1Z
	for coreutils@gnu.org; Sun, 26 Feb 2012 18:17:12 -0500
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: AhsDAOu8Sk9tThHq/2dsb2JhbAAMNrAsgX0DhAEBAQEEMgFGEAsNCwkWDwkDAgECAUUGDQEHAQEXvkSLFIICLAsGBAMEAwgECgcPAQoGAwKFCAMHhBYEmzyMbw
Received: from unknown (HELO [192.168.1.79]) ([109.78.17.234])
	by mail2.vodafone.ie with ESMTP; 26 Feb 2012 23:17:08 +0000
Message-ID: <4F4ABD74.5010601@draigBrady.com>
Date: Sun, 26 Feb 2012 23:17:08 +0000
From: =?ISO-8859-1?Q?P=E1draig_Brady?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:6.0) Gecko/20110816 Thunderbird/6.0
MIME-Version: 1.0
To: Sven Breuner <sven.breuner@itwm.fraunhofer.de>
Subject: Re: "ls -l": Avoid unnecessary getxattr() overhead
References: <87pqds3snt.fsf@Apollo.jerryland.fr>
	<4F30206B.1020802@itwm.fraunhofer.de>
	<CA+TATJHorgsuY7pxmpxxAeWUHRP5MfFfzQP05o=1H6C4ttswdA@mail.gmail.com>
	<4F3032BC.50702@draigBrady.com>
	<4F30631A.2070009@itwm.fraunhofer.de>
	<87y5scfdiq.fsf@rho.meyering.net> <4F33B5D7.3010909@draigBrady.com>
	<87mx8sf5w0.fsf@rho.meyering.net>
	<4F3437CE.4050308@itwm.fraunhofer.de>
	<4F346CC7.4030608@draigBrady.com> <87r4y3dry0.fsf@rho.meyering.net>
	<4F3710CA.8020402@itwm.fraunhofer.de>
	<8739aao9e2.fsf@rho.meyering.net>
	<4F3E12F9.6080102@bernhard-voelker.de>
	<8762f5lpct.fsf@rho.meyering.net>
	<4F3E35C9.7050707@bernhard-voelker.de>
	<87mx8hk6bm.fsf@rho.meyering.net>
	<4F4069D3.5000806@itwm.fraunhofer.de>
	<87pqdbcia0.fsf@rho.meyering.net> <4F40D061.3000206@draigBrady.com>
	<871upqdagt.fsf@rho.meyering.net>
	<4F41762E.1030801@itwm.fraunhofer.de>
	<4F418BAE.20703@draigBrady.com>
	<4F41AA9D.2060007@itwm.fraunhofer.de>
	<4F42267F.60205@draigBrady.com>
	<4F4AB896.8080907@itwm.fraunhofer.de>
In-Reply-To: <4F4AB896.8080907@itwm.fraunhofer.de>
X-Enigmail-Version: 1.3.2
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 213.233.128.44
Cc: "Daniel P. Berrange" <berrange@redhat.com>,
	Bernhard Voelker <mail@bernhard-voelker.de>,
	Bernd Schubert <bernd.schubert@itwm.fraunhofer.de>, coreutils@gnu.org,
	=?ISO-8859-1?Q?J=E9r?= =?ISO-8859-1?Q?=E9my_Compostella?=
	<jeremy.compostella@gmail.com>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sun, 26 Feb 2012 23:17:14 -0000

On 02/26/2012 10:56 PM, Sven Breuner wrote:
> Pádraig Brady wrote on 02/20/2012 11:54 AM:
>> On 02/20/2012 02:06 AM, Sven Breuner wrote:
>>> Pádraig Brady wrote on 02/20/2012 12:54 AM:
>>>> I just booted RHEL 6.2 with selinux=0 on the kernel command line,
>>>> so as I could test new files without a context created, and...
>>>>
>>>> getxattr("security.selinux") on new files on tmpfs, returns ENOTSUP
>>>>  [...]
>>>
>>> Seems to be inconsistent. Some of our RHEL 6.x systems return "unlabled" on a tmpfs:
>>>  [...]
>>
>> I thought the difference might be selinux=0 boot param,
>> versus selinux=disabled in /etc/selinux/config, but that's not significant.
> 
> You were right, Pádraig. Booting the machines that previously returned "unlabled" with "selinux=0" makes them return EOPNOTSUPP.

Thanks for confirming that.
My intuition was right, but I must have messed up something.
Or maybe it's dependent on your kernel ver.
Anyway you now have the solution to avoid all extraneous getxattr calls.

cheers,
Pádraig.


From MAILER-DAEMON Sun Feb 26 20:02:01 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1S1oyX-0001bV-MA
	for mharc-coreutils@gnu.org; Sun, 26 Feb 2012 20:02:01 -0500
Received: from eggs.gnu.org ([208.118.235.92]:36672)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1S1oyV-0001b2-0E
	for coreutils@gnu.org; Sun, 26 Feb 2012 20:01:59 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1S1oyU-0007ly-0E
	for coreutils@gnu.org; Sun, 26 Feb 2012 20:01:58 -0500
Received: from mail2.vodafone.ie ([213.233.128.44]:24148)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <P@draigBrady.com>) id 1S1oyS-0007kl-Tw
	for coreutils@gnu.org; Sun, 26 Feb 2012 20:01:57 -0500
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: Ao0HAGrVSk9tThHq/2dsb2JhbAAMNoRlVKp2gXkEA4QBAQEBBAwXDwFGEAkCDQsCAgUWCwICCQMCAQIBRQYNAQcBAbVEiSuBL4llgi4LBgQHAwEEAwUJBw8BCgMBAgECAoIDgwUDBwkFAwYHAwQEgleBFgSbPIxv
Received: from unknown (HELO [192.168.1.79]) ([109.78.17.234])
	by mail2.vodafone.ie with ESMTP; 27 Feb 2012 01:01:55 +0000
Message-ID: <4F4AD603.3010902@draigBrady.com>
Date: Mon, 27 Feb 2012 01:01:55 +0000
From: =?UTF-8?B?UMOhZHJhaWcgQnJhZHk=?= <P@draigBrady.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:6.0) Gecko/20110816 Thunderbird/6.0
MIME-Version: 1.0
To: =?UTF-8?B?SsOpcsOpbXkgQ29tcG9zdGVsbGE=?=
 <jeremy.compostella@gmail.com>
Subject: Re: [PATCH] doc: fix a wrong option reference in split invocation
References: <87zkc51y5i.fsf@Apollo.jerryland.fr>
In-Reply-To: <87zkc51y5i.fsf@Apollo.jerryland.fr>
X-Enigmail-Version: 1.3.2
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 213.233.128.44
Cc: coreutils@gnu.org
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 27 Feb 2012 01:01:59 -0000

On 02/26/2012 02:12 PM, JÃ©rÃ©my Compostella wrote:
> All,
> 
> I think there is a mistake in the output file names format description
> of the split invocation section. Indeed, there is a reference to a
> nonexistent option (-r). I think the right option is --number with r/n
> (round-robin distribution) as argument I.E. -nr/n.
> 
> I propose the attached patch to fix it.

pushed.

cheers,
PÃ¡draig.


From MAILER-DAEMON Mon Feb 27 15:16:03 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1S26zL-0005dp-TJ
	for mharc-coreutils@gnu.org; Mon, 27 Feb 2012 15:16:03 -0500
Received: from eggs.gnu.org ([208.118.235.92]:34338)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1S26zG-0005dh-6H
	for coreutils@gnu.org; Mon, 27 Feb 2012 15:16:02 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1S26zE-00020m-CL
	for coreutils@gnu.org; Mon, 27 Feb 2012 15:15:58 -0500
Received: from moutng.kundenserver.de ([212.227.17.10]:60875)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1S26zE-00020Y-2f
	for coreutils@gnu.org; Mon, 27 Feb 2012 15:15:56 -0500
Received: from [192.168.2.108] (p4FF749F6.dip.t-dialin.net [79.247.73.246])
	by mrelayeu.kundenserver.de (node=mreu4) with ESMTP (Nemesis)
	id 0Lc2Qz-1Skl7j1NLx-00k9tK; Mon, 27 Feb 2012 21:15:43 +0100
Message-ID: <4F4BE46D.6060100@bernhard-voelker.de>
Date: Mon, 27 Feb 2012 21:15:41 +0100
From: Bernhard Voelker <mail@bernhard-voelker.de>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:10.0.2) Gecko/20120215 Thunderbird/10.0.2
MIME-Version: 1.0
To: Jim Meyering <jim@meyering.net>
Subject: Re: "ls -l": Avoid unnecessary getxattr() overhead
References: <87pqds3snt.fsf@Apollo.jerryland.fr>
	<4F30631A.2070009@itwm.fraunhofer.de>
	<87y5scfdiq.fsf@rho.meyering.net> <4F33B5D7.3010909@draigBrady.com>
	<87mx8sf5w0.fsf@rho.meyering.net>
	<4F3437CE.4050308@itwm.fraunhofer.de>
	<4F346CC7.4030608@draigBrady.com> <87r4y3dry0.fsf@rho.meyering.net>
	<4F3710CA.8020402@itwm.fraunhofer.de>
	<8739aao9e2.fsf@rho.meyering.net>
	<4F3E12F9.6080102@bernhard-voelker.de>
	<8762f5lpct.fsf@rho.meyering.net> <4F3E5BBD.80809@redhat.com>
	<874nupk1cx.fsf@rho.meyering.net> <4F3E5E8E.2040802@redhat.com>
	<87lio1ij8b.fsf@rho.meyering.net>
	<4F3E7B1E.3090300@bernhard-voelker.de>
	<8739a9ig96.fsf@rho.meyering.net>
	<4F3EF935.2070700@bernhard-voelker.de>
	<878vk0gujb.fsf@rho.meyering.net>
	<4F420517.2050804@bernhard-voelker.de>
	<87ehtix9yj.fsf@rho.meyering.net>
In-Reply-To: <87ehtix9yj.fsf@rho.meyering.net>
Content-Type: multipart/mixed; boundary="------------080807000602000804080909"
X-Provags-ID: V02:K0:LKKC51y9A/U3vlk9e+fJ1BK+bRwtmWW+BdEMS123W4O
	vu3bLfqJgByH6ICmKu5YQGCBxdbm4MHoxzvgAy6qtNOPnR5bgO
	HbwN8Fd4vhjjGrGBs0rwQEAj0vR7SbhtbTUH3HY7dzhmFg86tg
	KEgxNys13ezbjLn2F0DCyxLgtIdACp73Iu046FRrE3Tr/T+OAK
	QRMrgAu0HE/gQVKmNgIGHqN12gBoBekx5aO6D3mn4SKQgS78op
	rLtNhhH1OLo50vrUaa5F5hpDcjZJ8HRZs8SkslPgrWY7oCe7Yb
	aH0T7GCqkEOWd65e66qNZAtmEK8ZU5f4B5ZQfM6RRyd99nQPJf
	AVl0YKWzAGL7BC7WOWRqvSOT8p8p70/ELCDr1hS4t
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 212.227.17.10
Cc: "Daniel P. Berrange" <berrange@redhat.com>,
	=?ISO-8859-1?Q?Cristian_Rodr=EDguez?= <crrodriguez@opensuse.org>,
	Bernd Schubert <bernd.schubert@itwm.fraunhofer.de>, coreutils@gnu.org,
	Sven Breuner <sven.breuner@itwm.fraunhofer.de>,
	=?ISO-8859-1?Q?J=E9r=E9my_Compostella?= <jeremy.compostella@gmail.com>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 27 Feb 2012 20:16:02 -0000

This is a multi-part message in MIME format.
--------------080807000602000804080909
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit

On 02/26/2012 09:41 AM, Jim Meyering wrote:
>> Maybe, but why is LD_PRELOAD honoring the "__attribute__((destructor))",
>> and on some systems not?
>
> Good question.
> There must be some OpenSuSE 12.1 users willing to investigate.
> Filing a bug report should get their attention.

I got an answer on the opensuse mailing list:
(http://lists.opensuse.org/opensuse/2012-02/msg01342.html)

It's a problem when linking with LD directly:

  Cristian Rodríguez wrote:
  > Bernhard Voelker wrote:
  > > This works here, too:
  > >
  > >    gcc -shared -fPIC -O2 k.c -o k.so
  > >
  > > Can you confirm the failure when compiling + linking
  > > is done in 2 separate steps?
  >
  > Well, I expected that to fail so didnt bothered.. You know..
  > distributions have a crapload of broken makefiles that link with LD
  > directly. it fails in subtle ways when you do so, when using GCC, it
  > passes the right options to the linker so we avoid nasty surprises...

I've created the attached patch to make the test pass.

Have a nice day,
Berny

--------------080807000602000804080909
Content-Type: text/x-patch;
 name="ls-getxattr.patch"
Content-Transfer-Encoding: 8bit
Content-Disposition: attachment;
 filename="ls-getxattr.patch"

>From fe17ee2b3b322689dde043de08dab4d6b44c58e0 Mon Sep 17 00:00:00 2001
From: Bernhard Voelker <mail@bernhard-voelker.de>
Date: Mon, 27 Feb 2012 21:13:10 +0100
Subject: [PATCH] tests: compile and link shared object with GCC to make
 LD_PRELOAD work
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

* tests/ls/getxattr-speedup: Compile and link in one step with $CC.
If the shared object file is created by ld (binutils), then the
destructor print_call_count() may not run (seen on OpenSuSE 12.1).
See http://lists.opensuse.org/opensuse/2012-02/msg01342.html
Thanks to Cristian RodrÃ­guez.
---
 tests/ls/getxattr-speedup |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/tests/ls/getxattr-speedup b/tests/ls/getxattr-speedup
index d32e24a..812a03b 100755
--- a/tests/ls/getxattr-speedup
+++ b/tests/ls/getxattr-speedup
@@ -47,8 +47,8 @@ ssize_t lgetxattr(const char *path, const char *name, void *value, size_t size)
 EOF
 
 # Then compile/link it:
-$CC -fPIC -O2 -c k.c || framework_failure_ 'failed to compile with -fPIC'
-ld -G k.o -o k.so || framework_failure_ 'failed to invoke ld -G ...'
+$CC -shared -fPIC -O2 k.c -o k.so ||
+  framework_failure_ 'failed to compile with -shared -fPIC'
 
 # Create a few files:
 seq 20 | xargs touch || framework_failure_
-- 
1.7.7


--------------080807000602000804080909--


From MAILER-DAEMON Mon Feb 27 16:03:38 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1S27jO-0003qZ-NA
	for mharc-coreutils@gnu.org; Mon, 27 Feb 2012 16:03:38 -0500
Received: from eggs.gnu.org ([208.118.235.92]:44849)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1S27jK-0003nc-Vp
	for coreutils@gnu.org; Mon, 27 Feb 2012 16:03:36 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1S27jF-0002RW-Hi
	for coreutils@gnu.org; Mon, 27 Feb 2012 16:03:34 -0500
Received: from mx.meyering.net ([88.168.87.75]:49888)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <jim@meyering.net>) id 1S27jF-0002R6-9Y
	for coreutils@gnu.org; Mon, 27 Feb 2012 16:03:29 -0500
Received: from rho.meyering.net (localhost.localdomain [127.0.0.1])
	by rho.meyering.net (Acme Bit-Twister) with ESMTP id 9AF86602A0;
	Mon, 27 Feb 2012 22:03:26 +0100 (CET)
From: Jim Meyering <jim@meyering.net>
To: Bernhard Voelker <mail@bernhard-voelker.de>
Subject: Re: "ls -l": Avoid unnecessary getxattr() overhead
In-Reply-To: <4F4BE46D.6060100@bernhard-voelker.de> (Bernhard Voelker's
	message of "Mon, 27 Feb 2012 21:15:41 +0100")
References: <87pqds3snt.fsf@Apollo.jerryland.fr>
	<4F33B5D7.3010909@draigBrady.com> <87mx8sf5w0.fsf@rho.meyering.net>
	<4F3437CE.4050308@itwm.fraunhofer.de>
	<4F346CC7.4030608@draigBrady.com> <87r4y3dry0.fsf@rho.meyering.net>
	<4F3710CA.8020402@itwm.fraunhofer.de>
	<8739aao9e2.fsf@rho.meyering.net>
	<4F3E12F9.6080102@bernhard-voelker.de>
	<8762f5lpct.fsf@rho.meyering.net> <4F3E5BBD.80809@redhat.com>
	<874nupk1cx.fsf@rho.meyering.net> <4F3E5E8E.2040802@redhat.com>
	<87lio1ij8b.fsf@rho.meyering.net>
	<4F3E7B1E.3090300@bernhard-voelker.de>
	<8739a9ig96.fsf@rho.meyering.net>
	<4F3EF935.2070700@bernhard-voelker.de>
	<878vk0gujb.fsf@rho.meyering.net>
	<4F420517.2050804@bernhard-voelker.de>
	<87ehtix9yj.fsf@rho.meyering.net>
	<4F4BE46D.6060100@bernhard-voelker.de>
Date: Mon, 27 Feb 2012 22:03:26 +0100
Message-ID: <874nucq99d.fsf@rho.meyering.net>
Lines: 39
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 88.168.87.75
Cc: "Daniel P. Berrange" <berrange@redhat.com>,
	Cristian =?iso-8859-1?Q?Rodr=EDguez?= <crrodriguez@opensuse.org>,
	Bernd Schubert <bernd.schubert@itwm.fraunhofer.de>, coreutils@gnu.org,
	Sven Breuner <sven.breuner@itwm.fraunhofer.de>,
	=?iso-8859-1?Q?J=E9r=E9my?= Compostella <jeremy.compostella@gmail.com>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 27 Feb 2012 21:03:36 -0000

Bernhard Voelker wrote:
> On 02/26/2012 09:41 AM, Jim Meyering wrote:
...
> I've created the attached patch to make the test pass.
...
> Subject: [PATCH] tests: compile and link shared object with GCC to make
>  LD_PRELOAD work

Thanks I tweaked the title: s/G/$/
...

...
>  # Then compile/link it:
> -$CC -fPIC -O2 -c k.c || framework_failure_ 'failed to compile with -fPIC'
> -ld -G k.o -o k.so || framework_failure_ 'failed to invoke ld -G ...'
> +$CC -shared -fPIC -O2 k.c -o k.so ||
> +  framework_failure_ 'failed to compile with -shared -fPIC'
>
>  # Create a few files:
>  seq 20 | xargs touch || framework_failure_

And adjusted the above so that the operator is at the
beginning of the line:

diff --git a/tests/ls/getxattr-speedup b/tests/ls/getxattr-speedup
index 812a03b..967f7c9 100755
--- a/tests/ls/getxattr-speedup
+++ b/tests/ls/getxattr-speedup
@@ -47,8 +47,8 @@ ssize_t lgetxattr(const char *path, const char *name, void *value, size_t size)
 EOF

 # Then compile/link it:
-$CC -shared -fPIC -O2 k.c -o k.so ||
-  framework_failure_ 'failed to compile with -shared -fPIC'
+$CC -shared -fPIC -O2 k.c -o k.so \
+  || framework_failure_ 'failed to compile with -shared -fPIC'

 # Create a few files:
 seq 20 | xargs touch || framework_failure_


From MAILER-DAEMON Tue Feb 28 08:11:27 2012
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1S2Mpz-0002cI-TH
	for mharc-coreutils@gnu.org; Tue, 28 Feb 2012 08:11:27 -0500
Received: from eggs.gnu.org ([208.118.235.92]:53767)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1S2Mps-0002au-FP
	for coreutils@gnu.org; Tue, 28 Feb 2012 08:11:26 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1S2Mpn-0005Wf-G7
	for coreutils@gnu.org; Tue, 28 Feb 2012 08:11:20 -0500
Received: from moutng.kundenserver.de ([212.227.126.171]:58027)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <mail@bernhard-voelker.de>) id 1S2Mpn-0005WX-5v
	for coreutils@gnu.org; Tue, 28 Feb 2012 08:11:15 -0500
Received: from oxltgw03.schlund.de (oxltgw03.schlund.de [172.19.158.41])
	by mrelayeu.kundenserver.de (node=mreu1) with ESMTP (Nemesis)
	id 0LiGhD-1SpT5i2JQh-00nMVU; Tue, 28 Feb 2012 14:11:09 +0100
Date: Tue, 28 Feb 2012 14:11:09 +0100 (CET)
From: "mail@bernhard-voelker.de" <mail@bernhard-voelker.de>
To: Jim Meyering <jim@meyering.net>
Message-ID: <1053730353.513994.1330434669611.JavaMail.open-xchange@email.1und1.de>
In-Reply-To: <874nucq99d.fsf@rho.meyering.net>
References: <87pqds3snt.fsf@Apollo.jerryland.fr>
	<4F33B5D7.3010909@draigBrady.com> <87mx8sf5w0.fsf@rho.meyering.net>
	<4F3437CE.4050308@itwm.fraunhofer.de>
	<4F346CC7.4030608@draigBrady.com> <87r4y3dry0.fsf@rho.meyering.net>
	<4F3710CA.8020402@itwm.fraunhofer.de>
	<8739aao9e2.fsf@rho.meyering.net>
	<4F3E12F9.6080102@bernhard-voelker.de>
	<8762f5lpct.fsf@rho.meyering.net> <4F3E5BBD.80809@redhat.com>
	<874nupk1cx.fsf@rho.meyering.net> <4F3E5E8E.2040802@redhat.com>
	<87lio1ij8b.fsf@rho.meyering.net>
	<4F3E7B1E.3090300@bernhard-voelker.de>
	<8739a9ig96.fsf@rho.meyering.net>
	<4F3EF935.2070700@bernhard-voelker.de>
	<878vk0gujb.fsf@rho.meyering.net>
	<4F420517.2050804@bernhard-voelker.de>
	<87ehtix9yj.fsf@rho.meyering.net>
	<4F4BE46D.6060100@bernhard-voelker.de>
	<874nucq99d.fsf@rho.meyering.net>
Subject: Re: "ls -l": Avoid unnecessary getxattr() overhead
MIME-Version: 1.0
Content-Type: multipart/alternative; 
	boundary="----=_Part_513993_1194264713.1330434669540"
X-Priority: 3
Importance: Medium
X-Mailer: Open-Xchange Mailer v6.20.0-Rev36
X-Provags-ID: V02:K0:reVJgyc2BxEwTXdk+Y2mhJeaYQX5S7KiWtO0FRngQKm
	CuHsTJ5JLiaqdw68+isJq7MIo8/SKgtDzHpfrPt+3epuCBheGS
	Oz73RhEm6QElxmyba0utXb9ji/Ps7iId5KIV5CGWitDkv8FWAc
	Uojj+p/zBunihbfnXdw7cKCw63S6Z+k6DpcXDZVmMDbK0z5D2I
	XwzeMSfIwGH4x/kGzwbkm+XGKMo7Ixtsnfcs9659J1NfJY4rW+
	UTN8bp500/b6txJ/dE/Ukn0AAekALi0HkkVGg5H1g9zOmlOBBr
	Dyug0MYD/76ubd8iwzJarvldC8ZsGLGuiIcKWN1U+Fmy5CpjTS
	0O7H+1CmZ/1stiiVPUBE3u8hoGyU9fctZyc5Bdz/t+7Xu7lMJK
	xrt1siarATHGrz8rtj/VrAEEu8h/eVw/7pNyblElQdH7iX6S5d 2wqiI
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 212.227.126.171
Cc: "Daniel P. Berrange" <berrange@redhat.com>,
	=?UTF-8?Q?=22Cristian_Rodr=C3=ADguez=22?= <crrodriguez@opensuse.org>,
	Bernd Schubert <bernd.schubert@itwm.fraunhofer.de>, coreutils@gnu.org,
	Sven Breuner <sven.breuner@itwm.fraunhofer.de>,
	=?UTF-8?Q?=22J=C3=A9r=C3=A9my_Compostella=22?=
	<jeremy.compostella@gmail.com>
X-BeenThere: coreutils@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
Reply-To: "mail@bernhard-voelker.de" <mail@bernhard-voelker.de>
List-Id: GNU coreutils General Discussion <coreutils.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/coreutils>
List-Post: <mailto:coreutils@gnu.org>
List-Help: <mailto:coreutils-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/coreutils>,
	<mailto:coreutils-request@gnu.org?subject=subscribe>
X-List-Received-Date: Tue, 28 Feb 2012 13:11:26 -0000

------=_Part_513993_1194264713.1330434669540
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On February 27, 2012 at 10:03 PM Jim Meyering <jim@meyering.net> wrote:

> Bernhard Voelker wrote:
> > On 02/26/2012 09:41 AM, Jim Meyering wrote:
> ...
> > I've created the attached patch to make the test pass.
> ...
> > Subject: [PATCH] tests: compile and link shared object with GCC to make
> >  LD_PRELOAD work
> 
> Thanks I tweaked

...



Thanks, looks fine.



Have a nice day,

Berny




------=_Part_513993_1194264713.1330434669540
MIME-Version: 1.0
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: 7bit

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml">
 <head>
    <meta content="text/html; charset=UTF-8" http-equiv="Content-Type"/>
 </head>
 <body>
  <p>
   <span>On February 27, 2012 at 10:03 PM Jim Meyering &#60;jim@meyering.net&#62; wrote:</span>
   <br/>
   <br/>
   <span>&#62; Bernhard Voelker wrote:</span>
   <br/>
   <span>&#62; &#62; On 02/26/2012 09:41 AM, Jim Meyering wrote:</span>
   <br/>
   <span>&#62; ...</span>
   <br/>
   <span>&#62; &#62; I&#39;ve created the attached patch to make the test pass.</span>
   <br/>
   <span>&#62; ...</span>
   <br/>
   <span>&#62; &#62; Subject: [PATCH] tests: compile and link shared object with GCC to make</span>
   <br/>
   <span>&#62; &#62;&#160; LD_PRELOAD work</span>
   <br/>
   <span>&#62;&#160;</span>
   <br/>
   <span>&#62; Thanks I tweaked</span>
  </p>
  <p>
   <span>...</span>
  </p>
  <p>
   <span>&#160;</span>
  </p>
  <p>
   <span>Thanks, looks fine.</span>
  </p>
  <p>
   <span>&#160;</span>
  </p>
  <p>
   <span>Have a nice day,</span>
  </p>
  <p>
   <span>Berny</span>
  </p>
  <p>
   <span>&#160;</span>
  </p>
  <p>
   <span>&#160;</span>
  </p>
 </body>
</html>
------=_Part_513993_1194264713.1330434669540--


