From MAILER-DAEMON Wed Jun 13 00:38:17 2007
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1HyKcb-0004jP-J0
	for mharc-bug-gnu-pspp@gnu.org; Wed, 13 Jun 2007 00:38:17 -0400
Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43)
	id 1HyKca-0004jK-P9
	for bug-gnu-pspp@gnu.org; Wed, 13 Jun 2007 00:38:16 -0400
Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43)
	id 1HyKcZ-0004ht-Jo
	for bug-gnu-pspp@gnu.org; Wed, 13 Jun 2007 00:38:16 -0400
Received: from [199.232.76.173] (helo=monty-python.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1HyKcZ-0004hq-GN
	for bug-gnu-pspp@gnu.org; Wed, 13 Jun 2007 00:38:15 -0400
Received: from savannah.gnu.org ([199.232.41.3] helo=sv.gnu.org)
	by monty-python.gnu.org with esmtps (TLS-1.0:RSA_AES_256_CBC_SHA1:32)
	(Exim 4.60) (envelope-from <www-data@savannah.gnu.org>)
	id 1HyKcY-0004zj-9X; Wed, 13 Jun 2007 00:38:14 -0400
Received: from savannah.gnu.org ([199.232.41.3] helo=frontend)
	by sv.gnu.org with esmtp (Exim 4.63)
	(envelope-from <www-data@savannah.gnu.org>)
	id 1HyKcX-0001mh-Jj; Wed, 13 Jun 2007 04:38:13 +0000
Received: from www-data by frontend with local (Exim 4.63)
	(envelope-from <www-data@savannah.gnu.org>)
	id 1HyKcX-0001me-BN; Wed, 13 Jun 2007 04:38:13 +0000
To: Ben Pfaff <blp@gnu.org>, John Darrington <john@darrington.wattle.id.au>,
	bug-gnu-pspp@gnu.org
From: Ben Pfaff <INVALID.NOREPLY@gnu.org>
X-Savane-Server: savannah.gnu.org:443 [199.232.41.3]
MIME-Version: 1.0
Content-Type: text/plain;charset=UTF-8
X-Savane-Project: pspp
X-Savane-Tracker: bugs
X-Savane-Item-ID: 18559
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US;
	rv:1.8.1.1) Gecko/20061205 Iceweasel/2.0.0.1
	(Debian-2.0.0.1+dfsg-1)
X-Apparently-From: 76.103.81.97 (Savane authenticated user blp)
Message-Id: <20070612-213812.sv245.41050@savannah.gnu.org>
References: <20061219-134826.sv245.16054@savannah.gnu.org>
	<20061219-142645.sv245.66789@savannah.gnu.org>
	<20061220-083404.sv5461.42682@savannah.gnu.org>
	<20070203-220311.sv245.12339@savannah.gnu.org>
In-Reply-To: <20070203-220311.sv245.12339@savannah.gnu.org>
Date: Wed, 13 Jun 2007 04:38:13 +0000
X-detected-kernel: Linux 2.6 (newer, 3)
Cc: 
Subject: PSPP-BUG: [bug #18559] Flexifile needs proper implementation
X-BeenThere: bug-gnu-pspp@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Bug reports for GNU PSPP <bug-gnu-pspp.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/bug-gnu-pspp>,
	<mailto:bug-gnu-pspp-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/pipermail/bug-gnu-pspp>
List-Post: <mailto:bug-gnu-pspp@gnu.org>
List-Help: <mailto:bug-gnu-pspp-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/bug-gnu-pspp>,
	<mailto:bug-gnu-pspp-request@gnu.org?subject=subscribe>
X-List-Received-Date: Wed, 13 Jun 2007 04:38:16 -0000


Update of bug #18559 (project pspp):

                  Status:             In Progress => Fixed                  
             Open/Closed:                    Open => Closed                 

    _______________________________________________________

Follow-up Comment #4:

I fixed this!

    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?18559>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/




From MAILER-DAEMON Wed Jun 13 00:41:56 2007
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1HyKg7-00071f-TB
	for mharc-bug-gnu-pspp@gnu.org; Wed, 13 Jun 2007 00:41:55 -0400
Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43)
	id 1HyKg6-00071Z-F3
	for bug-gnu-pspp@gnu.org; Wed, 13 Jun 2007 00:41:54 -0400
Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43)
	id 1HyKg6-00071N-1g
	for bug-gnu-pspp@gnu.org; Wed, 13 Jun 2007 00:41:54 -0400
Received: from [199.232.76.173] (helo=monty-python.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1HyKg5-00071K-Sa
	for bug-gnu-pspp@gnu.org; Wed, 13 Jun 2007 00:41:53 -0400
Received: from savannah.gnu.org ([199.232.41.3] helo=sv.gnu.org)
	by monty-python.gnu.org with esmtps (TLS-1.0:RSA_AES_256_CBC_SHA1:32)
	(Exim 4.60) (envelope-from <www-data@savannah.gnu.org>)
	id 1HyKg4-0005H0-Kk; Wed, 13 Jun 2007 00:41:52 -0400
Received: from savannah.gnu.org ([199.232.41.3] helo=frontend)
	by sv.gnu.org with esmtp (Exim 4.63)
	(envelope-from <www-data@savannah.gnu.org>)
	id 1HyKg3-0001xW-UX; Wed, 13 Jun 2007 04:41:52 +0000
Received: from www-data by frontend with local (Exim 4.63)
	(envelope-from <www-data@savannah.gnu.org>)
	id 1HyKg3-0001xT-Ny; Wed, 13 Jun 2007 04:41:51 +0000
To: Ben Pfaff <blp@gnu.org>, bug-gnu-pspp@gnu.org
From: Ben Pfaff <INVALID.NOREPLY@gnu.org>
X-Savane-Server: savannah.gnu.org:443 [199.232.41.3]
MIME-Version: 1.0
Content-Type: text/plain;charset=UTF-8
X-Savane-Project: pspp
X-Savane-Tracker: bugs
X-Savane-Item-ID: 20161
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US;
	rv:1.8.1.1) Gecko/20061205 Iceweasel/2.0.0.1
	(Debian-2.0.0.1+dfsg-1)
X-Apparently-From: 76.103.81.97 (Savane authenticated user blp)
Message-Id: <20070612-214151.sv245.90941@savannah.gnu.org>
References: 
In-Reply-To: 
Date: Wed, 13 Jun 2007 04:41:51 +0000
X-detected-kernel: Linux 2.6 (newer, 3)
Cc: 
Subject: PSPP-BUG: [bug #20161] support automatic (de)compression for
	system/portable file I/O
X-BeenThere: bug-gnu-pspp@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Bug reports for GNU PSPP <bug-gnu-pspp.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/bug-gnu-pspp>,
	<mailto:bug-gnu-pspp-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/pipermail/bug-gnu-pspp>
List-Post: <mailto:bug-gnu-pspp@gnu.org>
List-Help: <mailto:bug-gnu-pspp-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/bug-gnu-pspp>,
	<mailto:bug-gnu-pspp-request@gnu.org?subject=subscribe>
X-List-Received-Date: Wed, 13 Jun 2007 04:41:54 -0000


URL:
  <http://savannah.gnu.org/bugs/?20161>

                 Summary: support automatic (de)compression for
system/portable file I/O
                 Project: PSPP
            Submitted by: blp
            Submitted on: Tuesday 06/12/07 at 21:41
                Category: System/Portable File I/O
                Severity: 1 - Wishlist
                  Status: None
             Assigned to: None
             Open/Closed: Open
         Discussion Lock: Any
                 Release: None

    _______________________________________________________

Details:

We could easily support automatic compression and decompression for system
file I/O, based on file name extensions (.gz .bz2 etc.) or on file magic
numbers.  This would be a much better alternative to the "compression" that
SPSS system files support.




    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?20161>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/




From MAILER-DAEMON Fri Jun 15 03:46:50 2007
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1Hz6WA-0003CY-Dv
	for mharc-bug-gnu-pspp@gnu.org; Fri, 15 Jun 2007 03:46:50 -0400
Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43)
	id 1Hz6W9-0003CT-KT
	for bug-gnu-pspp@gnu.org; Fri, 15 Jun 2007 03:46:49 -0400
Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43)
	id 1Hz6W7-0003CH-AF
	for bug-gnu-pspp@gnu.org; Fri, 15 Jun 2007 03:46:48 -0400
Received: from [199.232.76.173] (helo=monty-python.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1Hz6W7-0003CE-51
	for bug-gnu-pspp@gnu.org; Fri, 15 Jun 2007 03:46:47 -0400
Received: from savannah.gnu.org ([199.232.41.3] helo=sv.gnu.org)
	by monty-python.gnu.org with esmtps (TLS-1.0:RSA_AES_256_CBC_SHA1:32)
	(Exim 4.60) (envelope-from <www-data@savannah.gnu.org>)
	id 1Hz6W4-0003mZ-7t; Fri, 15 Jun 2007 03:46:44 -0400
Received: from savannah.gnu.org ([199.232.41.3] helo=frontend)
	by sv.gnu.org with esmtp (Exim 4.63)
	(envelope-from <www-data@savannah.gnu.org>)
	id 1Hz6W2-00085C-Ad; Fri, 15 Jun 2007 07:46:42 +0000
Received: from www-data by frontend with local (Exim 4.63)
	(envelope-from <www-data@savannah.gnu.org>)
	id 1Hz6W1-000859-QF; Fri, 15 Jun 2007 07:46:42 +0000
To: Ben Pfaff <blp@gnu.org>, John Darrington <john@darrington.wattle.id.au>,
	bug-gnu-pspp@gnu.org
From: John Darrington <INVALID.NOREPLY@gnu.org>
X-Savane-Server: savannah.gnu.org:443 [199.232.41.3]
MIME-Version: 1.0
Content-Type: text/plain;charset=UTF-8
X-Savane-Project: pspp
X-Savane-Tracker: bugs
X-Savane-Item-ID: 20161
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US;
	rv:1.7.8) Gecko/20070113 Debian/1.7.8-1sarge10
X-Apparently-From: 203.161.82.55 (Savane authenticated user jmd)
Message-Id: <20070615-154641.sv5461.87195@savannah.gnu.org>
References: <20070612-214151.sv245.90941@savannah.gnu.org>
In-Reply-To: <20070612-214151.sv245.90941@savannah.gnu.org>
Date: Fri, 15 Jun 2007 07:46:41 +0000
X-detected-kernel: Linux 2.6 (newer, 3)
Cc: 
Subject: PSPP-BUG: [bug #20161] support automatic (de)compression for
	system/portable file I/O
X-BeenThere: bug-gnu-pspp@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Bug reports for GNU PSPP <bug-gnu-pspp.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/bug-gnu-pspp>,
	<mailto:bug-gnu-pspp-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/pipermail/bug-gnu-pspp>
List-Post: <mailto:bug-gnu-pspp@gnu.org>
List-Help: <mailto:bug-gnu-pspp-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/bug-gnu-pspp>,
	<mailto:bug-gnu-pspp-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 15 Jun 2007 07:46:49 -0000


Follow-up Comment #1, bug #20161 (project pspp):

It would be "better" in the sense that it would achieve a greater compression
ratio, but it would also mean that the files would no longer be serializeable.
 So reading/writing large ones would be significantly slower.

    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?20161>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/




From MAILER-DAEMON Fri Jun 15 10:25:29 2007
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1HzCjx-0000bQ-23
	for mharc-bug-gnu-pspp@gnu.org; Fri, 15 Jun 2007 10:25:29 -0400
Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43)
	id 1HzCjv-0000bB-2u
	for bug-gnu-pspp@gnu.org; Fri, 15 Jun 2007 10:25:27 -0400
Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43)
	id 1HzCjt-0000ay-8U
	for bug-gnu-pspp@gnu.org; Fri, 15 Jun 2007 10:25:25 -0400
Received: from [199.232.76.173] (helo=monty-python.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1HzCjt-0000av-69
	for bug-gnu-pspp@gnu.org; Fri, 15 Jun 2007 10:25:25 -0400
Received: from savannah.gnu.org ([199.232.41.3] helo=sv.gnu.org)
	by monty-python.gnu.org with esmtps (TLS-1.0:RSA_AES_256_CBC_SHA1:32)
	(Exim 4.60) (envelope-from <www-data@savannah.gnu.org>)
	id 1HzCjr-0005UR-W7; Fri, 15 Jun 2007 10:25:24 -0400
Received: from savannah.gnu.org ([199.232.41.3] helo=frontend)
	by sv.gnu.org with esmtp (Exim 4.63)
	(envelope-from <www-data@savannah.gnu.org>)
	id 1HzCjq-00064y-7H; Fri, 15 Jun 2007 14:25:22 +0000
Received: from www-data by frontend with local (Exim 4.63)
	(envelope-from <www-data@savannah.gnu.org>)
	id 1HzCjp-00064u-To; Fri, 15 Jun 2007 14:25:22 +0000
To: Ben Pfaff <blp@gnu.org>, John Darrington <john@darrington.wattle.id.au>,
	bug-gnu-pspp@gnu.org
From: Ben Pfaff <INVALID.NOREPLY@gnu.org>
X-Savane-Server: savannah.gnu.org:443 [199.232.41.3]
MIME-Version: 1.0
Content-Type: text/plain;charset=UTF-8
X-Savane-Project: pspp
X-Savane-Tracker: bugs
X-Savane-Item-ID: 20161
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US;
	rv:1.8.1.1) Gecko/20061205 Iceweasel/2.0.0.1
	(Debian-2.0.0.1+dfsg-1)
X-Apparently-From: 76.103.81.97 (Savane authenticated user blp)
Message-Id: <20070615-072521.sv245.19378@savannah.gnu.org>
References: <20070612-214151.sv245.90941@savannah.gnu.org>
	<20070615-154641.sv5461.87195@savannah.gnu.org>
In-Reply-To: <20070615-154641.sv5461.87195@savannah.gnu.org>
Date: Fri, 15 Jun 2007 14:25:21 +0000
X-detected-kernel: Linux 2.6 (newer, 3)
Cc: 
Subject: PSPP-BUG: [bug #20161] support automatic (de)compression for
	system/portable file I/O
X-BeenThere: bug-gnu-pspp@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Bug reports for GNU PSPP <bug-gnu-pspp.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/bug-gnu-pspp>,
	<mailto:bug-gnu-pspp-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/pipermail/bug-gnu-pspp>
List-Post: <mailto:bug-gnu-pspp@gnu.org>
List-Help: <mailto:bug-gnu-pspp-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/bug-gnu-pspp>,
	<mailto:bug-gnu-pspp-request@gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 15 Jun 2007 14:25:27 -0000


Follow-up Comment #2, bug #20161 (project pspp):

Not sure what you mean by "no longer serializable".  We currently read system
and portable files only sequentially, and with one small exception the same is
true for writing system and portable files.  I was planning to implement this
feature just by piping data to/from a gzip or bzip2 process.  Can you think
of why this wouldn't work?

    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?20161>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/




From MAILER-DAEMON Fri Jun 15 21:15:53 2007
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1HzMtN-0001eu-7r
	for mharc-bug-gnu-pspp@gnu.org; Fri, 15 Jun 2007 21:15:53 -0400
Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43)
	id 1HzMtL-0001cE-ID
	for bug-gnu-pspp@gnu.org; Fri, 15 Jun 2007 21:15:51 -0400
Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43)
	id 1HzMtL-0001bj-1t
	for bug-gnu-pspp@gnu.org; Fri, 15 Jun 2007 21:15:51 -0400
Received: from [199.232.76.173] (helo=monty-python.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1HzMtK-0001bb-U1
	for bug-gnu-pspp@gnu.org; Fri, 15 Jun 2007 21:15:50 -0400
Received: from savannah.gnu.org ([199.232.41.3] helo=sv.gnu.org)
	by monty-python.gnu.org with esmtps (TLS-1.0:RSA_AES_256_CBC_SHA1:32)
	(Exim 4.60) (envelope-from <www-data@savannah.gnu.org>)
	id 1HzMtJ-0007PF-MO; Fri, 15 Jun 2007 21:15:49 -0400
Received: from savannah.gnu.org ([199.232.41.3] helo=frontend)
	by sv.gnu.org with esmtp (Exim 4.63)
	(envelope-from <www-data@savannah.gnu.org>)
	id 1HzMtI-0006Dg-Hf; Sat, 16 Jun 2007 01:15:48 +0000
Received: from www-data by frontend with local (Exim 4.63)
	(envelope-from <www-data@savannah.gnu.org>)
	id 1HzMtI-0006Dc-5O; Sat, 16 Jun 2007 01:15:48 +0000
To: Ben Pfaff <blp@gnu.org>, John Darrington <john@darrington.wattle.id.au>,
	bug-gnu-pspp@gnu.org
From: John Darrington <INVALID.NOREPLY@gnu.org>
X-Savane-Server: savannah.gnu.org:443 [199.232.41.3]
MIME-Version: 1.0
Content-Type: text/plain;charset=UTF-8
X-Savane-Project: pspp
X-Savane-Tracker: bugs
X-Savane-Item-ID: 20161
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US;
	rv:1.8.1.3) Gecko/20070310 Iceweasel/2.0.0.3 (Debian-2.0.0.3-1)
X-Apparently-From: 203.161.82.55 (Savane authenticated user jmd)
Message-Id: <20070616-091547.sv5461.85814@savannah.gnu.org>
References: <20070612-214151.sv245.90941@savannah.gnu.org>
	<20070615-154641.sv5461.87195@savannah.gnu.org>
	<20070615-072521.sv245.19378@savannah.gnu.org>
In-Reply-To: <20070615-072521.sv245.19378@savannah.gnu.org>
Date: Sat, 16 Jun 2007 01:15:48 +0000
X-detected-kernel: Linux 2.6 (newer, 3)
Cc: 
Subject: PSPP-BUG: [bug #20161] support automatic (de)compression for
	system/portable file I/O
X-BeenThere: bug-gnu-pspp@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Bug reports for GNU PSPP <bug-gnu-pspp.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/bug-gnu-pspp>,
	<mailto:bug-gnu-pspp-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/pipermail/bug-gnu-pspp>
List-Post: <mailto:bug-gnu-pspp@gnu.org>
List-Help: <mailto:bug-gnu-pspp-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/bug-gnu-pspp>,
	<mailto:bug-gnu-pspp-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sat, 16 Jun 2007 01:15:52 -0000


Follow-up Comment #3, bug #20161 (project pspp):

I'm not saying that it wouldn't work, or that it wouldn't be a good idea (so
long as gzip compression is disabled by default).

I'm saying that if you pipe through gzip (or any compression tool), then it
produces nothing until it's input is closed.  Although "you" (pspp) will be
reading cases sequentially, the gzip process will have to store them all in
memory until after the very last one.  So some of the advantages of the
current implementation will be lost.  It'll mean for instance that
reading/writing will be considerably slower, and the maximum possible number
of cases will be smaller.

Just so long as users realize this, I don't think there will be a problem.

    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?20161>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/




From MAILER-DAEMON Fri Jun 15 21:22:37 2007
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1HzMzt-0003G1-5h
	for mharc-bug-gnu-pspp@gnu.org; Fri, 15 Jun 2007 21:22:37 -0400
Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43)
	id 1HzMzr-0003Fw-5c
	for bug-gnu-pspp@gnu.org; Fri, 15 Jun 2007 21:22:35 -0400
Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43)
	id 1HzMzp-0003Fk-NU
	for bug-gnu-pspp@gnu.org; Fri, 15 Jun 2007 21:22:33 -0400
Received: from [199.232.76.173] (helo=monty-python.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1HzMzp-0003Fh-IK
	for bug-gnu-pspp@gnu.org; Fri, 15 Jun 2007 21:22:33 -0400
Received: from savannah.gnu.org ([199.232.41.3] helo=sv.gnu.org)
	by monty-python.gnu.org with esmtps (TLS-1.0:RSA_AES_256_CBC_SHA1:32)
	(Exim 4.60) (envelope-from <www-data@savannah.gnu.org>)
	id 1HzMzo-00087R-Fj; Fri, 15 Jun 2007 21:22:32 -0400
Received: from savannah.gnu.org ([199.232.41.3] helo=frontend)
	by sv.gnu.org with esmtp (Exim 4.63)
	(envelope-from <www-data@savannah.gnu.org>)
	id 1HzMzn-0006js-PM; Sat, 16 Jun 2007 01:22:31 +0000
Received: from www-data by frontend with local (Exim 4.63)
	(envelope-from <www-data@savannah.gnu.org>)
	id 1HzMzm-0006jj-VZ; Sat, 16 Jun 2007 01:22:31 +0000
To: Ben Pfaff <blp@gnu.org>, John Darrington <john@darrington.wattle.id.au>,
	bug-gnu-pspp@gnu.org
From: Ben Pfaff <INVALID.NOREPLY@gnu.org>
X-Savane-Server: savannah.gnu.org:443 [199.232.41.3]
MIME-Version: 1.0
Content-Type: text/plain;charset=UTF-8
X-Savane-Project: pspp
X-Savane-Tracker: bugs
X-Savane-Item-ID: 20161
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US;
	rv:1.8.1.1) Gecko/20061205 Iceweasel/2.0.0.1
	(Debian-2.0.0.1+dfsg-1)
X-Apparently-From: 76.103.81.97 (Savane authenticated user blp)
Message-Id: <20070615-182230.sv245.73931@savannah.gnu.org>
References: <20070612-214151.sv245.90941@savannah.gnu.org>
	<20070615-154641.sv5461.87195@savannah.gnu.org>
	<20070615-072521.sv245.19378@savannah.gnu.org>
	<20070616-091547.sv5461.85814@savannah.gnu.org>
In-Reply-To: <20070616-091547.sv5461.85814@savannah.gnu.org>
Date: Sat, 16 Jun 2007 01:22:30 +0000
X-detected-kernel: Linux 2.6 (newer, 3)
Cc: 
Subject: PSPP-BUG: [bug #20161] support automatic (de)compression for
	system/portable file I/O
X-BeenThere: bug-gnu-pspp@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Bug reports for GNU PSPP <bug-gnu-pspp.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/bug-gnu-pspp>,
	<mailto:bug-gnu-pspp-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/pipermail/bug-gnu-pspp>
List-Post: <mailto:bug-gnu-pspp@gnu.org>
List-Help: <mailto:bug-gnu-pspp-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/bug-gnu-pspp>,
	<mailto:bug-gnu-pspp-request@gnu.org?subject=subscribe>
X-List-Received-Date: Sat, 16 Jun 2007 01:22:35 -0000


Follow-up Comment #4, bug #20161 (project pspp):

>I'm saying that if you pipe through gzip (or any compression 
>tool), then it produces nothing until it's input is closed. 
>Although "you" (pspp) will be reading cases sequentially, the gzip
>process will have to store them all in memory until after the 
>very last one. 

That's not at all true.  You can demonstrate this easily: run "gzip <
/dev/urandom > /dev/null" in one terminal, and "top" in another.  gzip will
use as much CPU as it can get, but its memory consumption is O(1).

>It'll mean for instance that reading/writing will be considerably
>slower, 

Possibly.  I bet it wouldn't be any slower on any mid-range to high-end
computer you buy off-the-shelf this year, though, because all of those are
multi-core.

>and the maximum possible number of cases will be smaller. 

I doubt it.

    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?20161>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/




From MAILER-DAEMON Mon Jun 18 09:19:32 2007
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1I0H8m-0007rd-FT
	for mharc-bug-gnu-pspp@gnu.org; Mon, 18 Jun 2007 09:19:32 -0400
Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43)
	id 1I0H8k-0007lG-Da
	for bug-gnu-pspp@gnu.org; Mon, 18 Jun 2007 09:19:30 -0400
Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43)
	id 1I0H8i-0007gr-Sh
	for bug-gnu-pspp@gnu.org; Mon, 18 Jun 2007 09:19:29 -0400
Received: from [199.232.76.173] (helo=monty-python.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1I0H8i-0007gK-MZ
	for bug-gnu-pspp@gnu.org; Mon, 18 Jun 2007 09:19:28 -0400
Received: from mhw.ulib.iupui.edu ([134.68.171.23])
	by monty-python.gnu.org with esmtp (Exim 4.60)
	(envelope-from <mwood@IUPUI.Edu>) id 1I0H8i-0004ux-Cd
	for bug-gnu-pspp@gnu.org; Mon, 18 Jun 2007 09:19:28 -0400
Received: from mwood by mhw.ulib.iupui.edu with local (Exim 4.67)
	(envelope-from <mwood@IUPUI.Edu>) id 1I0H8a-0004ph-JI
	for bug-gnu-pspp@gnu.org; Mon, 18 Jun 2007 09:19:20 -0400
Date: Mon, 18 Jun 2007 09:19:20 -0400
From: "Mark H. Wood" <mwood@IUPUI.Edu>
To: bug-gnu-pspp@gnu.org
Subject: Re: PSPP-BUG: [bug #20161] support automatic (de)compression for
	system/portable file I/O
Message-ID: <20070618131920.GB17388@IUPUI.Edu>
Mail-Followup-To: bug-gnu-pspp@gnu.org
References: <20070612-214151.sv245.90941@savannah.gnu.org>
	<20070615-154641.sv5461.87195@savannah.gnu.org>
	<20070615-072521.sv245.19378@savannah.gnu.org>
	<20070616-091547.sv5461.85814@savannah.gnu.org>
	<20070615-182230.sv245.73931@savannah.gnu.org>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha1;
	protocol="application/pgp-signature"; boundary="2B/JsCI69OhZNC5r"
Content-Disposition: inline
In-Reply-To: <20070615-182230.sv245.73931@savannah.gnu.org>
User-Agent: Mutt/1.5.13 (2006-08-11)
Sender: "Mark H. Wood,UL 0115A,+1 317 274 0749," <mwood@mhw.ulib.iupui.edu>
X-detected-kernel: Linux 2.6 (newer, 3)
X-BeenThere: bug-gnu-pspp@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Bug reports for GNU PSPP <bug-gnu-pspp.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/bug-gnu-pspp>,
	<mailto:bug-gnu-pspp-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/pipermail/bug-gnu-pspp>
List-Post: <mailto:bug-gnu-pspp@gnu.org>
List-Help: <mailto:bug-gnu-pspp-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/bug-gnu-pspp>,
	<mailto:bug-gnu-pspp-request@gnu.org?subject=subscribe>
X-List-Received-Date: Mon, 18 Jun 2007 13:19:30 -0000


--2B/JsCI69OhZNC5r
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

Why not look at the algorithms?

http://www.gzip.org/algorithm.txt

'deflate' does seem to buffer up the output stream for a while, but it
can flush those buffers when it determines that a fresh dictionary
should be started.  Small streams might be completely buffered, but it
becomes increasingly likely with increasing length that the output
stream will be blocked rather than retained complete.

GNU 'tar' can be instructed to pass the archive through 'compress' or
(more usually, these days) 'gzip', so there is some experience there
to draw on.

There's some discussion of bzip2 at Wikipedia:

http://en.wikipedia.org/wiki/Bzip2

It looks like bzip2 doesn't necessarily buffer the whole stream
either.  Again, recent GNU 'tar' can pipe through bzip2 so there is
some experience with real-world behavior of this model.

--=20
Mark H. Wood, Lead System Programmer   mwood@IUPUI.Edu
Typically when a software vendor says that a product is "intuitive" he
means the exact opposite.


--2B/JsCI69OhZNC5r
Content-Type: application/pgp-signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (GNU/Linux)

iD8DBQFGdoZYs/NR4JuTKG8RAjHoAKCqG2Lqkb3hkeyMsOPeIzOSS1vVpwCfbd9S
VRywxsoHm9CVgFBBk6fea2o=
=GmSS
-----END PGP SIGNATURE-----

--2B/JsCI69OhZNC5r--



From MAILER-DAEMON Wed Jun 27 20:21:24 2007
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1I3hlE-0001n6-66
	for mharc-bug-gnu-pspp@gnu.org; Wed, 27 Jun 2007 20:21:24 -0400
Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43)
	id 1I3hlC-0001mr-F4
	for bug-gnu-pspp@gnu.org; Wed, 27 Jun 2007 20:21:22 -0400
Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43)
	id 1I3hlC-0001mL-0U
	for bug-gnu-pspp@gnu.org; Wed, 27 Jun 2007 20:21:22 -0400
Received: from [199.232.76.173] (helo=monty-python.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1I3hlB-0001mI-Q6
	for bug-gnu-pspp@gnu.org; Wed, 27 Jun 2007 20:21:21 -0400
Received: from savannah.gnu.org ([199.232.41.3] helo=sv.gnu.org)
	by monty-python.gnu.org with esmtps (TLS-1.0:RSA_AES_256_CBC_SHA1:32)
	(Exim 4.60) (envelope-from <www-data@savannah.gnu.org>)
	id 1I3hlB-0003tW-Ev
	for bug-gnu-pspp@gnu.org; Wed, 27 Jun 2007 20:21:21 -0400
Received: from savannah.gnu.org ([199.232.41.3] helo=frontend)
	by sv.gnu.org with esmtp (Exim 4.63)
	(envelope-from <www-data@savannah.gnu.org>)
	id 1I3hl9-0000Sx-TH; Thu, 28 Jun 2007 00:21:20 +0000
Received: from www-data by frontend with local (Exim 4.63)
	(envelope-from <www-data@savannah.gnu.org>)
	id 1I3hl9-0000Su-GN; Thu, 28 Jun 2007 00:21:19 +0000
To: John Darrington <john@darrington.wattle.id.au>, bug-gnu-pspp@gnu.org
From: John Darrington <INVALID.NOREPLY@gnu.org>
X-Savane-Server: savannah.gnu.org:443 [199.232.41.3]
MIME-Version: 1.0
Content-Type: text/plain;charset=UTF-8
X-Savane-Project: pspp
X-Savane-Tracker: bugs
X-Savane-Item-ID: 20285
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US;
	rv:1.8.0.12) Gecko/20070510 Iceape/1.0.9 (Debian-1.0.9-0etch1)
X-Apparently-From: 203.161.82.55 (Savane authenticated user jmd)
Message-Id: <20070628-082118.sv5461.58441@savannah.gnu.org>
References: 
In-Reply-To: 
Date: Thu, 28 Jun 2007 00:21:19 +0000
X-detected-kernel: Linux 2.6 (newer, 3)
Cc: 
Subject: PSPP-BUG: [bug #20285] SAVE following INPUT PROGRAM crashes.
X-BeenThere: bug-gnu-pspp@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Bug reports for GNU PSPP <bug-gnu-pspp.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/bug-gnu-pspp>,
	<mailto:bug-gnu-pspp-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/pipermail/bug-gnu-pspp>
List-Post: <mailto:bug-gnu-pspp@gnu.org>
List-Help: <mailto:bug-gnu-pspp-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/bug-gnu-pspp>,
	<mailto:bug-gnu-pspp-request@gnu.org?subject=subscribe>
X-List-Received-Date: Thu, 28 Jun 2007 00:21:22 -0000


URL:
  <http://savannah.gnu.org/bugs/?20285>

                 Summary: SAVE following INPUT PROGRAM crashes.
                 Project: PSPP
            Submitted by: jmd
            Submitted on: Thursday 06/28/07 at 08:21
                Category: Other
                Severity: 7 - Major
                  Status: None
             Assigned to: None
             Open/Closed: Open
         Discussion Lock: Any
                 Release: None

    _______________________________________________________

Details:

This bug seems to have crept in with the simpler-proc merge.

INPUT PROGRAM.
        LOOP #I = 1 TO 55000.
                COMPUTE X = #I.
        END CASE.
        END LOOP.
END FILE.
END INPUT PROGRAM.

SAVE OUTFILE='enormous.sav'.


adding EXECUTE before the SAVE seems to work around the problem.




    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?20285>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/




