From MAILER-DAEMON Wed Aug 11 22:38:15 2004
Received: from mailman by lists.gnu.org with archive (Exim 4.33)
	id 1Bv5Tr-00069e-Oz
	for mharc-cfengine-develop@gnu.org; Wed, 11 Aug 2004 22:38:15 -0400
Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.33)
	id 1Bv5Tq-00069P-5N
	for cfengine-develop@nongnu.org; Wed, 11 Aug 2004 22:38:14 -0400
Received: from exim by lists.gnu.org with spam-scanned (Exim 4.33)
	id 1Bv5Tn-00068Y-TH
	for cfengine-develop@nongnu.org; Wed, 11 Aug 2004 22:38:13 -0400
Received: from [199.232.76.173] (helo=monty-python.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.33) id 1Bv5Tn-00068V-Op
	for cfengine-develop@nongnu.org; Wed, 11 Aug 2004 22:38:11 -0400
Received: from [66.93.249.120] (helo=kirby.madstop.com)
	by monty-python.gnu.org with esmtp (Exim 4.34) id 1Bv5Pt-0007rh-UV
	for cfengine-develop@nongnu.org; Wed, 11 Aug 2004 22:34:10 -0400
Received: by kirby.madstop.com (Postfix, from userid 100)
	id 3DF0F434E3; Wed, 11 Aug 2004 21:34:07 -0500 (CDT)
Received: from localhost (localhost [127.0.0.1])
	by kirby.madstop.com (Postfix) with ESMTP id 376534CE80
	for <cfengine-develop@nongnu.org>; Wed, 11 Aug 2004 21:34:07 -0500 (CDT)
Date: Wed, 11 Aug 2004 21:34:07 -0500 (CDT)
From: "Luke A. Kanies" <luke@madstop.com>
To: Cfengine Developers <cfengine-develop@nongnu.org>
Message-ID: <Pine.GSO.4.60.0408112133190.11270@kirby.madstop.com>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed
Subject: [Cfengine-develop] caching checksums with cfservd
X-BeenThere: cfengine-develop@nongnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Cfengine developers <cfengine-develop.nongnu.org>
List-Unsubscribe: <http://lists.nongnu.org/mailman/listinfo/cfengine-develop>, 
	<mailto:cfengine-develop-request@nongnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/pipermail/cfengine-develop>
List-Post: <mailto:cfengine-develop@nongnu.org>
List-Help: <mailto:cfengine-develop-request@nongnu.org?subject=help>
List-Subscribe: <http://lists.nongnu.org/mailman/listinfo/cfengine-develop>,
	<mailto:cfengine-develop-request@nongnu.org?subject=subscribe>
X-List-Received-Date: Thu, 12 Aug 2004 02:38:14 -0000

Hi,

I'm getting to the point where I'm doing a lot of data transfer with 
cfservd, and I'd like to stop having cfservd force a checksum on every 
file transfer (as seen in cfservd.c where ChecksumChanged is passed 'true' 
as the value of 'refresh').

Would there be some problem if I made this change?  I know that I would 
need some kind of aging mechanism, but even if I could cache sums for five 
minutes I would see a performance benefit, because I'm looking at having 
hundreds of machines connect and get checksums for hundreds of files, 
possible up to hundreds of MB of data, and in the space of about half an 
hour.  If I could cache those sums, I would be much much much happier.

Any idea how hard it would be to make this work?  Maybe force a stat if 
we're running inside cfservd, and only force a sum if the stat shows a 
difference?

Luke

-- 
If a `religion' is defined to be a system of ideas that contains
unprovable statements, then Godel taught us that mathematics is not
only a religion, it is the only religion that can prove itself to be
one. 					-- John Barrow
---------------------------------------------------------------------
Luke Kanies | http://abstractive.org | http://reductiveconsulting.com



From MAILER-DAEMON Thu Aug 12 01:33:13 2004
Received: from mailman by lists.gnu.org with archive (Exim 4.33)
	id 1Bv8DB-00010D-8v
	for mharc-cfengine-develop@gnu.org; Thu, 12 Aug 2004 01:33:13 -0400
Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.33)
	id 1Bv8DA-000107-Eh
	for cfengine-develop@nongnu.org; Thu, 12 Aug 2004 01:33:12 -0400
Received: from exim by lists.gnu.org with spam-scanned (Exim 4.33)
	id 1Bv8D8-0000zv-3Z
	for cfengine-develop@nongnu.org; Thu, 12 Aug 2004 01:33:12 -0400
Received: from [199.232.76.173] (helo=monty-python.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.33) id 1Bv8D7-0000zs-W4
	for cfengine-develop@nongnu.org; Thu, 12 Aug 2004 01:33:10 -0400
Received: from [128.39.89.10] (helo=nexus.iu.hio.no)
	by monty-python.gnu.org with esmtp (Exim 4.34) id 1Bv89D-0005vG-63
	for cfengine-develop@nongnu.org; Thu, 12 Aug 2004 01:29:07 -0400
Received: from iu.hio.no (nexus.iu.hio.no [128.39.89.10])
	by nexus.iu.hio.no (8.13.0/8.13.0) with ESMTP id i7C5SxJQ018066;
	Thu, 12 Aug 2004 07:29:03 +0200 (MEST)
Message-Id: <200408120529.i7C5SxJQ018066@nexus.iu.hio.no>
Date: Thu, 12 Aug 2004 07:28:59 +0200 (MEST)
From: Mark.Burgess@iu.hio.no
Subject: Re: [Cfengine-develop] caching checksums with cfservd
To: luke@madstop.com
In-Reply-To: <Pine.GSO.4.60.0408112133190.11270@kirby.madstop.com>
MIME-Version: 1.0
Content-Type: TEXT/plain; charset=us-ascii
X-SpamAssassin-Score: 0.339 NO_REAL_NAME
X-Scanned-By: MIMEDefang 2.41
Cc: cfengine-develop@nongnu.org
X-BeenThere: cfengine-develop@nongnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Cfengine developers <cfengine-develop.nongnu.org>
List-Unsubscribe: <http://lists.nongnu.org/mailman/listinfo/cfengine-develop>, 
	<mailto:cfengine-develop-request@nongnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/pipermail/cfengine-develop>
List-Post: <mailto:cfengine-develop@nongnu.org>
List-Help: <mailto:cfengine-develop-request@nongnu.org?subject=help>
List-Subscribe: <http://lists.nongnu.org/mailman/listinfo/cfengine-develop>,
	<mailto:cfengine-develop-request@nongnu.org?subject=subscribe>
X-List-Received-Date: Thu, 12 Aug 2004 05:33:12 -0000



On 11 Aug, Luke A. Kanies wrote:
> Hi,
> 
> I'm getting to the point where I'm doing a lot of data transfer with 
> cfservd, and I'd like to stop having cfservd force a checksum on every 
> file transfer (as seen in cfservd.c where ChecksumChanged is passed 'true' 
> as the value of 'refresh').
> 
> Would there be some problem if I made this change?  I know that I would 
> need some kind of aging mechanism, but even if I could cache sums for five 
> minutes I would see a performance benefit, because I'm looking at having 
> hundreds of machines connect and get checksums for hundreds of files, 
> possible up to hundreds of MB of data, and in the space of about half an 
> hour.  If I could cache those sums, I would be much much much happier.
> 
> Any idea how hard it would be to make this work?  Maybe force a stat if 
> we're running inside cfservd, and only force a sum if the stat shows a 
> difference?
> 
> Luke
> 

Luke. Originally the checksums were cached, as you observe in the variable
CHECKSUMUPDATES. I had tonnes of trouble with berkeley db on this
and got rid of it for stability. However, I always intended to come
back to it, but never have... so yes I would be very interested
in fixing this. Thanks for pointing it out.

Stating is, of course, a costly operation too for high performance.
It might be worthwhile having an optimizer that works like locks,
so that updates cannot happen more often than, say, once every half hour.

These things tend to get away from me.

M

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Work: +47 22453272            Email:  Mark.Burgess@iu.hio.no
Fax : +47 22453205            WWW  :  http://www.iu.hio.no/~mark
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~




From MAILER-DAEMON Thu Aug 12 01:41:47 2004
Received: from mailman by lists.gnu.org with archive (Exim 4.33)
	id 1Bv8LT-0003Vf-LS
	for mharc-cfengine-develop@gnu.org; Thu, 12 Aug 2004 01:41:47 -0400
Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.33)
	id 1Bv8LS-0003U4-6V
	for cfengine-develop@nongnu.org; Thu, 12 Aug 2004 01:41:46 -0400
Received: from exim by lists.gnu.org with spam-scanned (Exim 4.33)
	id 1Bv8LQ-0003RU-Hy
	for cfengine-develop@nongnu.org; Thu, 12 Aug 2004 01:41:45 -0400
Received: from [199.232.76.173] (helo=monty-python.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.33) id 1Bv8LQ-0003RF-Dj
	for cfengine-develop@nongnu.org; Thu, 12 Aug 2004 01:41:44 -0400
Received: from [64.152.73.80] (helo=leftcoast.thebackrow.net)
	by monty-python.gnu.org with esmtp (TLSv1:DES-CBC3-SHA:168)
	(Exim 4.34) id 1Bv8H9-000728-12
	for cfengine-develop@nongnu.org; Thu, 12 Aug 2004 01:37:19 -0400
Received: from harpo by leftcoast.thebackrow.net with local (Microsoft
	Exchange Internet Mail Service 5.5.2653.13) ;
	Wed, 11 Aug 2004 22:37:10 -0700
Date: Wed, 11 Aug 2004 22:37:10 -0700
To: Mark.Burgess@iu.hio.no
Subject: Re: [Cfengine-develop] caching checksums with cfservd
Message-ID: <20040812053710.GC2099@thebackrow.net>
References: <Pine.GSO.4.60.0408112133190.11270@kirby.madstop.com>
	<200408120529.i7C5SxJQ018066@nexus.iu.hio.no>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <200408120529.i7C5SxJQ018066@nexus.iu.hio.no>
User-Agent: Mutt/1.5.6+20040722i
From: Will Lowe <harpo@thebackrow.net>
Cc: cfengine-develop@nongnu.org
X-BeenThere: cfengine-develop@nongnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Cfengine developers <cfengine-develop.nongnu.org>
List-Unsubscribe: <http://lists.nongnu.org/mailman/listinfo/cfengine-develop>, 
	<mailto:cfengine-develop-request@nongnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/pipermail/cfengine-develop>
List-Post: <mailto:cfengine-develop@nongnu.org>
List-Help: <mailto:cfengine-develop-request@nongnu.org?subject=help>
List-Subscribe: <http://lists.nongnu.org/mailman/listinfo/cfengine-develop>,
	<mailto:cfengine-develop-request@nongnu.org?subject=subscribe>
X-List-Received-Date: Thu, 12 Aug 2004 05:41:46 -0000

> so that updates cannot happen more often than, say, once every half hour.

In my environment we keep the entire cfservd file tree in CVS and make
changes by changing CVS and then exporting the tree.  Since I know
when the files change (only when the tree is reexported) I'd like
cfservd to cache them *forever* -- basically until I tell it to stop
caching them by restarting it or passing it some special option or
something (easy to add to teh end of my export-the-cvs-tree script).

-- 
					thanks,
		
					Will



From MAILER-DAEMON Thu Aug 12 01:44:44 2004
Received: from mailman by lists.gnu.org with archive (Exim 4.33)
	id 1Bv8OJ-0004Hk-UE
	for mharc-cfengine-develop@gnu.org; Thu, 12 Aug 2004 01:44:44 -0400
Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.33)
	id 1Bv8OH-0004Hf-N7
	for cfengine-develop@nongnu.org; Thu, 12 Aug 2004 01:44:41 -0400
Received: from exim by lists.gnu.org with spam-scanned (Exim 4.33)
	id 1Bv8OG-0004HT-88
	for cfengine-develop@nongnu.org; Thu, 12 Aug 2004 01:44:41 -0400
Received: from [199.232.76.173] (helo=monty-python.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.33) id 1Bv8OF-0004HQ-VS
	for cfengine-develop@nongnu.org; Thu, 12 Aug 2004 01:44:40 -0400
Received: from [128.39.89.10] (helo=nexus.iu.hio.no)
	by monty-python.gnu.org with esmtp (Exim 4.34) id 1Bv8K5-0007R9-L4
	for cfengine-develop@nongnu.org; Thu, 12 Aug 2004 01:40:22 -0400
Received: from iu.hio.no (nexus.iu.hio.no [128.39.89.10])
	by nexus.iu.hio.no (8.13.0/8.13.0) with ESMTP id i7C5eEh7019274;
	Thu, 12 Aug 2004 07:40:18 +0200 (MEST)
Message-Id: <200408120540.i7C5eEh7019274@nexus.iu.hio.no>
Date: Thu, 12 Aug 2004 07:40:14 +0200 (MEST)
From: Mark.Burgess@iu.hio.no
Subject: Re: [Cfengine-develop] caching checksums with cfservd
To: harpo@thebackrow.net
In-Reply-To: <20040812053710.GC2099@thebackrow.net>
MIME-Version: 1.0
Content-Type: TEXT/plain; charset=us-ascii
X-SpamAssassin-Score: 0.339 NO_REAL_NAME
X-Scanned-By: MIMEDefang 2.41
Cc: cfengine-develop@nongnu.org, Mark.Burgess@iu.hio.no
X-BeenThere: cfengine-develop@nongnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Cfengine developers <cfengine-develop.nongnu.org>
List-Unsubscribe: <http://lists.nongnu.org/mailman/listinfo/cfengine-develop>, 
	<mailto:cfengine-develop-request@nongnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/pipermail/cfengine-develop>
List-Post: <mailto:cfengine-develop@nongnu.org>
List-Help: <mailto:cfengine-develop-request@nongnu.org?subject=help>
List-Subscribe: <http://lists.nongnu.org/mailman/listinfo/cfengine-develop>,
	<mailto:cfengine-develop-request@nongnu.org?subject=subscribe>
X-List-Received-Date: Thu, 12 Aug 2004 05:44:41 -0000



On 11 Aug, Will Lowe wrote:
>> so that updates cannot happen more often than, say, once every half hour.
> 
> In my environment we keep the entire cfservd file tree in CVS and make
> changes by changing CVS and then exporting the tree.  Since I know
> when the files change (only when the tree is reexported) I'd like
> cfservd to cache them *forever* -- basically until I tell it to stop
> caching them by restarting it or passing it some special option or
> something (easy to add to teh end of my export-the-cvs-tree script).
> 


As i said to Luke offline (I thought no one was using this list anymore)
I shall look into this. I have some time at the moment.

M

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Work: +47 22453272            Email:  Mark.Burgess@iu.hio.no
Fax : +47 22453205            WWW  :  http://www.iu.hio.no/~mark
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~




From MAILER-DAEMON Fri Aug 13 03:36:08 2004
Received: from mailman by lists.gnu.org with archive (Exim 4.33)
	id 1BvWbg-00005V-4O
	for mharc-cfengine-develop@gnu.org; Fri, 13 Aug 2004 03:36:08 -0400
Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.33)
	id 1BvWbY-00005O-QC
	for cfengine-develop@nongnu.org; Fri, 13 Aug 2004 03:36:00 -0400
Received: from exim by lists.gnu.org with spam-scanned (Exim 4.33)
	id 1BvWbX-000050-53
	for cfengine-develop@nongnu.org; Fri, 13 Aug 2004 03:36:00 -0400
Received: from [199.232.76.173] (helo=monty-python.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.33) id 1BvWbX-0008WV-2E
	for cfengine-develop@nongnu.org; Fri, 13 Aug 2004 03:35:59 -0400
Received: from [128.39.89.10] (helo=nexus.iu.hio.no)
	by monty-python.gnu.org with esmtp (Exim 4.34) id 1BvWXK-0007y2-As
	for cfengine-develop@nongnu.org; Fri, 13 Aug 2004 03:31:38 -0400
Received: from iu.hio.no (nexus.iu.hio.no [128.39.89.10])
	by nexus.iu.hio.no (8.13.0/8.13.0) with ESMTP id i7D7VTSx021702;
	Fri, 13 Aug 2004 09:31:34 +0200 (MEST)
Message-Id: <200408130731.i7D7VTSx021702@nexus.iu.hio.no>
Date: Fri, 13 Aug 2004 09:31:29 +0200 (MEST)
From: Mark.Burgess@iu.hio.no
Subject: Re: [Cfengine-develop] caching checksums with cfservd
To: luke@madstop.com
In-Reply-To: <Pine.GSO.4.60.0408112133190.11270@kirby.madstop.com>
MIME-Version: 1.0
Content-Type: TEXT/plain; charset=us-ascii
X-SpamAssassin-Score: 0.339 NO_REAL_NAME
X-Scanned-By: MIMEDefang 2.41
Cc: cfengine-develop@nongnu.org
X-BeenThere: cfengine-develop@nongnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Cfengine developers <cfengine-develop.nongnu.org>
List-Unsubscribe: <http://lists.nongnu.org/mailman/listinfo/cfengine-develop>, 
	<mailto:cfengine-develop-request@nongnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/pipermail/cfengine-develop>
List-Post: <mailto:cfengine-develop@nongnu.org>
List-Help: <mailto:cfengine-develop-request@nongnu.org?subject=help>
List-Subscribe: <http://lists.nongnu.org/mailman/listinfo/cfengine-develop>,
	<mailto:cfengine-develop-request@nongnu.org?subject=subscribe>
X-List-Received-Date: Fri, 13 Aug 2004 07:36:01 -0000



Hi - I've looked at this now and see that checksums are indeed cachced but
that there is a lot of checking to see if they need to be recomputed.

Solving this problem is a little complicated since it is server side and
must serve many clients. I think my feeling was originally that not many
comparisons will be made by checksum (mostly by date) to this would not
be too heavy. Probably that is not true in our paranoid age :)


> I'm getting to the point where I'm doing a lot of data transfer with 
> cfservd, and I'd like to stop having cfservd force a checksum on every 
> file transfer (as seen in cfservd.c where ChecksumChanged is passed 'true' 
> as the value of 'refresh').


True seems only to say that one should perform the worst case check, namely
that the file we are looking at is newer than the latest change to the
database (not necessarily to that file though!!)

 
> Would there be some problem if I made this change?  I know that I would 
> need some kind of aging mechanism, but even if I could cache sums for five 
> minutes I would see a performance benefit, because I'm looking at having 
> hundreds of machines connect and get checksums for hundreds of files, 
> possible up to hundreds of MB of data, and in the space of about half an 
> hour.  If I could cache those sums, I would be much much much happier.

I'm, not sure what change you are suggesting. Set this to false? That
would only prevent updates altogether.

One possibility would be to update the database periodically instead for
all files. That does not sound like a great solution to me.

Another would be to store the timestamp of the checksum in the database (change
the format of the db) so that more specific updates can be made. (This would
also be more correct with regard to update policy).   
 
> Any idea how hard it would be to make this work?  Maybe force a stat if 
> we're running inside cfservd, and only force a sum if the stat shows a 
> difference?
> 
> Luke
> 

What do you think?

M

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Work: +47 22453272            Email:  Mark.Burgess@iu.hio.no
Fax : +47 22453205            WWW  :  http://www.iu.hio.no/~mark
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~




From MAILER-DAEMON Fri Aug 13 03:53:34 2004
Received: from mailman by lists.gnu.org with archive (Exim 4.33)
	id 1BvWsY-0004mf-IB
	for mharc-cfengine-develop@gnu.org; Fri, 13 Aug 2004 03:53:34 -0400
Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.33)
	id 1BvWsW-0004lt-0h
	for cfengine-develop@nongnu.org; Fri, 13 Aug 2004 03:53:32 -0400
Received: from exim by lists.gnu.org with spam-scanned (Exim 4.33)
	id 1BvWsT-0004la-RF
	for cfengine-develop@nongnu.org; Fri, 13 Aug 2004 03:53:31 -0400
Received: from [199.232.76.173] (helo=monty-python.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.33) id 1BvWsT-0004lQ-Lr
	for cfengine-develop@nongnu.org; Fri, 13 Aug 2004 03:53:29 -0400
Received: from [64.152.73.80] (helo=leftcoast.thebackrow.net)
	by monty-python.gnu.org with esmtp (TLSv1:DES-CBC3-SHA:168)
	(Exim 4.34) id 1BvWo3-0001lR-Ga
	for cfengine-develop@nongnu.org; Fri, 13 Aug 2004 03:48:55 -0400
Received: from harpo by leftcoast.thebackrow.net with local (Microsoft
	Exchange Internet Mail Service 5.5.2653.13) ;
	Fri, 13 Aug 2004 00:48:47 -0700
Date: Fri, 13 Aug 2004 00:48:47 -0700
To: Mark.Burgess@iu.hio.no
Subject: Re: [Cfengine-develop] caching checksums with cfservd
Message-ID: <20040813074847.GA27798@thebackrow.net>
References: <Pine.GSO.4.60.0408112133190.11270@kirby.madstop.com>
	<200408130731.i7D7VTSx021702@nexus.iu.hio.no>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <200408130731.i7D7VTSx021702@nexus.iu.hio.no>
User-Agent: Mutt/1.5.6+20040722i
From: Will Lowe <harpo@thebackrow.net>
Cc: cfengine-develop@nongnu.org
X-BeenThere: cfengine-develop@nongnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Cfengine developers <cfengine-develop.nongnu.org>
List-Unsubscribe: <http://lists.nongnu.org/mailman/listinfo/cfengine-develop>, 
	<mailto:cfengine-develop-request@nongnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/pipermail/cfengine-develop>
List-Post: <mailto:cfengine-develop@nongnu.org>
List-Help: <mailto:cfengine-develop-request@nongnu.org?subject=help>
List-Subscribe: <http://lists.nongnu.org/mailman/listinfo/cfengine-develop>,
	<mailto:cfengine-develop-request@nongnu.org?subject=subscribe>
X-List-Received-Date: Fri, 13 Aug 2004 07:53:32 -0000

> One possibility would be to update the database periodically instead
> for all files. That does not sound like a great solution to me.

In my environment it'd be great if cvservd stored them until it
received SIGUSR1 (or some other generally-underused signal) or until
it restarted, at which point it would force checksums to be
regenerated the next time it looked at the file.  That way I can
"killall -USR1 cfservd" whenever I make changes.

Of course this only works in an environment where the admin is willing
to notify cfservd when things have changed -- but I'd actually prefer
it that way in mine.

-- 
					thanks,
		
					Will



From MAILER-DAEMON Sat Aug 21 00:19:24 2004
Received: from mailman by lists.gnu.org with archive (Exim 4.33)
	id 1ByNLg-0004hp-Ip
	for mharc-cfengine-develop@gnu.org; Sat, 21 Aug 2004 00:19:24 -0400
Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.33)
	id 1ByNLf-0004hg-7J
	for cfengine-develop@nongnu.org; Sat, 21 Aug 2004 00:19:23 -0400
Received: from exim by lists.gnu.org with spam-scanned (Exim 4.33)
	id 1ByNLe-0004hU-FQ
	for cfengine-develop@nongnu.org; Sat, 21 Aug 2004 00:19:22 -0400
Received: from [199.232.76.173] (helo=monty-python.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.33) id 1ByNLe-0004hR-Cv
	for cfengine-develop@nongnu.org; Sat, 21 Aug 2004 00:19:22 -0400
Received: from [66.93.249.120] (helo=kirby.madstop.com)
	by monty-python.gnu.org with esmtp (Exim 4.34) id 1ByNHB-0006zr-Li
	for cfengine-develop@nongnu.org; Sat, 21 Aug 2004 00:14:46 -0400
Received: by kirby.madstop.com (Postfix, from userid 100)
	id B5E4E434E7; Fri, 20 Aug 2004 23:14:44 -0500 (CDT)
Received: from localhost (localhost [127.0.0.1])
	by kirby.madstop.com (Postfix) with ESMTP id B43664CE80;
	Fri, 20 Aug 2004 23:14:44 -0500 (CDT)
Date: Fri, 20 Aug 2004 23:14:44 -0500 (CDT)
From: "Luke A. Kanies" <luke@madstop.com>
To: Cfengine Developers <cfengine-develop@nongnu.org>
Message-ID: <Pine.GSO.4.60.0408202301060.16235@kirby.madstop.com>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed
Cc: Mark.Burgess@iu.hio.no
Subject: [Cfengine-develop] Kind of a bug, maybe
X-BeenThere: cfengine-develop@nongnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Cfengine developers <cfengine-develop.nongnu.org>
List-Unsubscribe: <http://lists.nongnu.org/mailman/listinfo/cfengine-develop>, 
	<mailto:cfengine-develop-request@nongnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/pipermail/cfengine-develop>
List-Post: <mailto:cfengine-develop@nongnu.org>
List-Help: <mailto:cfengine-develop-request@nongnu.org?subject=help>
List-Subscribe: <http://lists.nongnu.org/mailman/listinfo/cfengine-develop>,
	<mailto:cfengine-develop-request@nongnu.org?subject=subscribe>
X-List-Received-Date: Sat, 21 Aug 2004 04:19:23 -0000

Hi Mark,

I kind of found a bug in cfengine today.  Well, I know I found one and I 
kind of found another.

First, the known bug:

Cfengine should probably never move a file that already ends in '.cfsaved' 
to <filename>.cfsaved.  I've had multiple occasions where I've had files 
named something like file.cfsaved.cfsaved.cfsaved, and it can be quite 
disastrous sometimes.  If you don't know of a simple way to fix this, I 
can probably provide a patch.

Now to the kind-of bug:

Run the following shell script:

#!/bin/sh

bd=/tmp/cftest
mkdir $bd

mkdir -p $bd/orig/a
touch $bd/orig/a/b
mkdir $bd/new
ln -s $bd/orig/a $bd/new

--------------------------------
You now have a simple directory structure in /tmp.  Now run the following 
simple cfengine script (I highly recommend -v):

control:
     actionsequence = ( links )
     bd = ( /tmp/cftest )

links:
     ${bd}/new +>! ${bd}/orig r=inf

-----------------------------------

You now have a very destroyed directory.  This error is the first clue:

cfengine:: Moving /tmp/cftest/new/a/b to /tmp/cftest/new/a/b.cfsaved
cfengine:: INFO: /tmp/cftest/new/a is a symbolic link, not a true 
directory!
cfengine:: Linking files /tmp/cftest/new/a/b -> /tmp/cftest/orig/a/b

>From there, you get a lot of badness.  The whole problem is because of the 
link done in the shell script, cfengine is actually linking from a file 
directly back to itself, just with a different filename, because 
/tmp/cftest/orig/a/b is the exact same file as /tmp/cftest/new/a/b.

I call this "kind of a bug" because I don't know the best solution.  My 
initial reaction is that cfengine should get rid of the soft link, create 
a directory, and go on its merry way.  That quite possibly is a very bad 
thing, though.

It seems like the best response is just make sure that a file is never 
linked to itself; if the file you're linking to exists then stat it to 
make sure it is not the same file.

And, of course, this is a very handy demonstration of the chaos that 
results from saving files that have already been saved via cfengine:

cfengine:: Can't rename 
/tmp/cftest/new/a/b_1093061372_Fri_Aug_20_23_09_32_2004_.cfsaved_1093061372_Fri_Aug_20_23_09_32_2004_.cfsaved_1093061372_Fri_Aug_20_23_09_32_2004_.cfsaved_1093061372_Fri_Aug_20_23_09_32_2004_.cfsaved_1093061372_Fri_Aug_20_23_09_32_2004_.cfsaved 
to 
/tmp/cftest/new/a/b_1093061372_Fri_Aug_20_23_09_32_2004_.cfsaved_1093061372_Fri_Aug_20_23_09_32_2004_.cfsaved_1093061372_Fri_Aug_20_23_09_32_2004_.cfsaved_1093061372_Fri_Aug_20_23_09_32_2004_.cfsaved_1093061372_Fri_Aug_20_23_09_32_2004_.cfsaved_1093061372_Fri_Aug_20_23_09_32_2004_.cfsaved
cfengine:: rename: File name too long

I don't really know what the resolution is, but someone needs to be done; 
this quite destroyed my (fortunately small) network, and if I had shipped 
this file off to my clients before this happened to me... I shudder quite 
a bit.

What do you think?

Luke

-- 
Nature and nature's laws lay hid in night,
God said, "Let Newton be," and all was light.

It did not last; the devil howling "Ho!
Let Einstein be!" restored the status quo.
---------------------------------------------------------------------
Luke Kanies | http://abstractive.org | http://reductiveconsulting.com



From MAILER-DAEMON Tue Aug 24 13:00:34 2004
Received: from mailman by lists.gnu.org with archive (Exim 4.33)
	id 1Bzeew-0007nk-6g
	for mharc-cfengine-develop@gnu.org; Tue, 24 Aug 2004 13:00:34 -0400
Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.33)
	id 1Bzeeu-0007ne-TK
	for cfengine-develop@nongnu.org; Tue, 24 Aug 2004 13:00:33 -0400
Received: from exim by lists.gnu.org with spam-scanned (Exim 4.33)
	id 1Bzeet-0007nR-5H
	for cfengine-develop@nongnu.org; Tue, 24 Aug 2004 13:00:32 -0400
Received: from [199.232.76.173] (helo=monty-python.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.33) id 1Bzeet-0007nO-26
	for cfengine-develop@nongnu.org; Tue, 24 Aug 2004 13:00:31 -0400
Received: from [208.252.163.7] (helo=trdlnk.com)
	by monty-python.gnu.org with esmtp (Exim 4.34) id 1BzeaL-0008Av-VR
	for cfengine-develop@nongnu.org; Tue, 24 Aug 2004 12:55:50 -0400
Received: from mulva (mulva [10.10.1.45])
	by trdlnk.com (8.12.10+Sun/8.12.9) with ESMTP id i7OGtmGp013584
	for <cfengine-develop@nongnu.org>; Tue, 24 Aug 2004 11:55:49 -0500 (CDT)
From: Chip Seraphine <chip@trdlnk.com>
Organization: Tradelink LLC
To: cfengine-develop@nongnu.org
Date: Tue, 24 Aug 2004 11:55:48 -0500
User-Agent: KMail/1.5.4
MIME-Version: 1.0
Content-Disposition: inline
Content-Type: text/plain;
  charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
Message-Id: <200408241155.48889.chip@trdlnk.com>
Subject: [Cfengine-develop] Patch: opt_zone_info defined during -z runs
X-BeenThere: cfengine-develop@nongnu.org
X-Mailman-Version: 2.1.5
Precedence: list
Reply-To: chip@trdlnk.com
List-Id: Cfengine developers <cfengine-develop.nongnu.org>
List-Unsubscribe: <http://lists.nongnu.org/mailman/listinfo/cfengine-develop>, 
	<mailto:cfengine-develop-request@nongnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/pipermail/cfengine-develop>
List-Post: <mailto:cfengine-develop@nongnu.org>
List-Help: <mailto:cfengine-develop-request@nongnu.org?subject=help>
List-Subscribe: <http://lists.nongnu.org/mailman/listinfo/cfengine-develop>,
	<mailto:cfengine-develop-request@nongnu.org?subject=subscribe>
X-List-Received-Date: Tue, 24 Aug 2004 17:00:33 -0000

[Should unsolicited, non-bugfix patches be sent to this list instead of -help?  
This seems more appropriate, but -help could generate more comments.]


When cfexecd runs cfagent with a "-z" flag to get scheduling information, I 
would like to minimize the amount of code that executes (largely in the 
interest of not consuming more CPU than absolutely needed).  I propose the 
following simple patch in order to accomplish this:


--- cfagent.c.orig      2004-08-24 11:47:23.084732000 -0500
+++ cfagent.c   2004-08-24 11:47:29.634963000 -0500
@@ -1631,6 +1631,7 @@
       case 'z': PRMAILSERVER = true;
           IGNORELOCK = true;
           PARSEONLY = true;
+          AddClassToHeap("opt_zone_info");
           break;

       case 'Z': strncpy(METHODMD5,optarg,CF_BUFSIZE-1);




All it does is set a classed called "opt_zone_info" which can be used in the 
script logic to avoid running any unneeded tasks.

-- 

Chip Seraphine
Unix Administrator
TradeLink, LLC
312-264-2048
chip@trdlnk.com




From MAILER-DAEMON Thu Aug 26 17:11:44 2004
Received: from mailman by lists.gnu.org with archive (Exim 4.33)
	id 1C0RX6-0000UC-Ls
	for mharc-cfengine-develop@gnu.org; Thu, 26 Aug 2004 17:11:44 -0400
Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.33)
	id 1C0RX4-0000RJ-4R
	for cfengine-develop@nongnu.org; Thu, 26 Aug 2004 17:11:42 -0400
Received: from exim by lists.gnu.org with spam-scanned (Exim 4.33)
	id 1C0RX2-0000PH-Gx
	for cfengine-develop@nongnu.org; Thu, 26 Aug 2004 17:11:41 -0400
Received: from [199.232.76.173] (helo=monty-python.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.33)
	id 1C0RX2-0000PB-E7; Thu, 26 Aug 2004 17:11:40 -0400
Received: from [128.39.89.10] (helo=nexus.iu.hio.no)
	by monty-python.gnu.org with esmtp (Exim 4.34)
	id 1C0RSD-0001HJ-8B; Thu, 26 Aug 2004 17:06:41 -0400
Received: from iu.hio.no (nexus.iu.hio.no [128.39.89.10])
	by nexus.iu.hio.no (8.13.0/8.13.0) with ESMTP id i7QL6cVX022731;
	Thu, 26 Aug 2004 23:06:43 +0200 (MEST)
Message-Id: <200408262106.i7QL6cVX022731@nexus.iu.hio.no>
Date: Thu, 26 Aug 2004 23:06:38 +0200 (MEST)
From: Mark.Burgess@iu.hio.no
To: help-cfengine@gnu.org,
	CFengine development team <cfengine-develop@nongnu.org>,
	bug-cfengine@gnu.org
MIME-Version: 1.0
Content-Type: TEXT/plain; charset=us-ascii
X-SpamAssassin-Score: 0.339 NO_REAL_NAME
X-Scanned-By: MIMEDefang 2.41
Cc: 
Subject: [Cfengine-develop] Tutorials etc at LISA
X-BeenThere: cfengine-develop@nongnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Cfengine developers <cfengine-develop.nongnu.org>
List-Unsubscribe: <http://lists.nongnu.org/mailman/listinfo/cfengine-develop>, 
	<mailto:cfengine-develop-request@nongnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/pipermail/cfengine-develop>
List-Post: <mailto:cfengine-develop@nongnu.org>
List-Help: <mailto:cfengine-develop-request@nongnu.org?subject=help>
List-Subscribe: <http://lists.nongnu.org/mailman/listinfo/cfengine-develop>,
	<mailto:cfengine-develop-request@nongnu.org?subject=subscribe>
X-List-Received-Date: Thu, 26 Aug 2004 21:11:42 -0000


I shall be giving introductory and "advanced" tutorials
on cfengine at LISA'04 for anyone who is interested.
The advanced tutorial will go into how to use the newer
features, structural stuff (methods) and organize
files etc. (http://www.usenix.org/lisa04)

Feel free to send me requests for special topics you would
like covered. There is limited time (half-day tutorials)
but I shall consider all requests.

Is there also a need for a developer's BoF? 
To discuss future development/other issues?
Let me know who is interested, to see how big an
event we'll need.

M

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Work: +47 22453272            Email:  Mark.Burgess@iu.hio.no
Fax : +47 22453205            WWW  :  http://www.iu.hio.no/~mark
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~




