From MAILER-DAEMON Sun Apr 03 04:55:42 2005
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1DI0tP-00014m-GC
	for mharc-dms-commit@gnu.org; Sun, 03 Apr 2005 04:55:40 -0400
Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43)
	id 1DHmVk-00015Q-G4
	for dms-commit@nongnu.org; Sat, 02 Apr 2005 12:34:16 -0500
Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43)
	id 1DHmVY-000101-JK
	for dms-commit@nongnu.org; Sat, 02 Apr 2005 12:34:05 -0500
Received: from [199.232.76.173] (helo=monty-python.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1DHmVX-0000wk-RW
	for dms-commit@nongnu.org; Sat, 02 Apr 2005 12:34:03 -0500
Received: from [199.232.41.67] (helo=mx20.gnu.org)
	by monty-python.gnu.org with esmtp (TLS-1.0:RSA_ARCFOUR_SHA:16)
	(Exim 4.34) id 1DHmHj-00067b-P7
	for dms-commit@nongnu.org; Sat, 02 Apr 2005 12:19:47 -0500
Received: from [199.232.41.3] (helo=savannah.gnu.org)
	by mx20.gnu.org with esmtp (Exim 4.34) id 1DHmDK-0008Vq-8V
	for dms-commit@nongnu.org; Sat, 02 Apr 2005 12:15:14 -0500
Received: from savannah.gnu.org ([127.0.0.1])
	by savannah.gnu.org with smtp (Exim 3.35 #1 (Debian))
	id 1DHmD8-0005fg-00
	for <dms-commit@nongnu.org>; Sat, 02 Apr 2005 12:15:02 -0500
To: dms-commit@nongnu.org
From: Julien Barbot <julien.barbot@laposte.net>
Message-Id: <E1DHmD8-0005fg-00@savannah.gnu.org>
Date: Sat, 02 Apr 2005 12:15:02 -0500
X-Mailman-Approved-At: Sun, 03 Apr 2005 04:55:19 -0400
Subject: [Dms-commit] Changes to dms/src/CompilerDaemon_mod.py [refactoring]
X-BeenThere: dms-commit@nongnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: dms-commit.nongnu.org
List-Unsubscribe: <http://lists.nongnu.org/mailman/listinfo/dms-commit>,
	<mailto:dms-commit-request@nongnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/pipermail/dms-commit>
List-Post: <mailto:dms-commit@nongnu.org>
List-Help: <mailto:dms-commit-request@nongnu.org?subject=help>
List-Subscribe: <http://lists.nongnu.org/mailman/listinfo/dms-commit>,
	<mailto:dms-commit-request@nongnu.org?subject=subscribe>
X-List-Received-Date: Sat, 02 Apr 2005 17:34:20 -0000
X-List-Received-Date: Sat, 02 Apr 2005 17:34:20 -0000

Index: dms/src/CompilerDaemon_mod.py
diff -u dms/src/CompilerDaemon_mod.py:1.14.2.4 dms/src/CompilerDaemon_mod.py:1.14.2.5
--- dms/src/CompilerDaemon_mod.py:1.14.2.4	Sun Mar 27 20:45:13 2005
+++ dms/src/CompilerDaemon_mod.py	Sat Apr  2 17:15:01 2005
@@ -14,7 +14,6 @@
 import os
 from Daemon import Daemon
 import syslog
-import commands
 import sys
 import CompilerDaemon_mod
 import string



From MAILER-DAEMON Sun Apr 03 04:55:45 2005
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1DI0tR-00014u-Do
	for mharc-dms-commit@gnu.org; Sun, 03 Apr 2005 04:55:42 -0400
Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43)
	id 1DHmVo-00016a-Es
	for dms-commit@nongnu.org; Sat, 02 Apr 2005 12:34:20 -0500
Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43)
	id 1DHmVg-00012y-V9
	for dms-commit@nongnu.org; Sat, 02 Apr 2005 12:34:15 -0500
Received: from [199.232.76.173] (helo=monty-python.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1DHmVc-0000wk-QR
	for dms-commit@nongnu.org; Sat, 02 Apr 2005 12:34:08 -0500
Received: from [199.232.41.3] (helo=savannah.gnu.org)
	by monty-python.gnu.org with esmtp (Exim 4.34) id 1DHmDL-0005mx-6K
	for dms-commit@nongnu.org; Sat, 02 Apr 2005 12:15:15 -0500
Received: from savannah.gnu.org ([127.0.0.1])
	by savannah.gnu.org with smtp (Exim 3.35 #1 (Debian))
	id 1DHmD9-0005fr-00
	for <dms-commit@nongnu.org>; Sat, 02 Apr 2005 12:15:03 -0500
To: dms-commit@nongnu.org
From: Julien Barbot <julien.barbot@laposte.net>
Message-Id: <E1DHmD9-0005fr-00@savannah.gnu.org>
Date: Sat, 02 Apr 2005 12:15:03 -0500
X-Mailman-Approved-At: Sun, 03 Apr 2005 04:55:22 -0400
Subject: [Dms-commit] dms/src Compiler.py CompilerDaemon_mod.py Remot...
	[refactoring]
X-BeenThere: dms-commit@nongnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: dms-commit.nongnu.org
List-Unsubscribe: <http://lists.nongnu.org/mailman/listinfo/dms-commit>,
	<mailto:dms-commit-request@nongnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/pipermail/dms-commit>
List-Post: <mailto:dms-commit@nongnu.org>
List-Help: <mailto:dms-commit-request@nongnu.org?subject=help>
List-Subscribe: <http://lists.nongnu.org/mailman/listinfo/dms-commit>,
	<mailto:dms-commit-request@nongnu.org?subject=subscribe>
X-List-Received-Date: Sat, 02 Apr 2005 17:34:32 -0000

CVSROOT:	/cvsroot/dms
Module name:	dms
Branch: 	refactoring
Changes by:	Julien Barbot <klyr@savannah.gnu.org>	05/04/02 17:15:01

Modified files:
	src            : Compiler.py CompilerDaemon_mod.py RemoteJob.py 

Log message:
	- NamedTemporaryFile is now used instead of mktemp

CVSWeb URLs:
http://savannah.gnu.org/cgi-bin/viewcvs/dms/dms/src/Compiler.py.diff?only_with_tag=refactoring&tr1=1.20.2.5&tr2=1.20.2.6&r1=text&r2=text
http://savannah.gnu.org/cgi-bin/viewcvs/dms/dms/src/CompilerDaemon_mod.py.diff?only_with_tag=refactoring&tr1=1.14.2.4&tr2=1.14.2.5&r1=text&r2=text
http://savannah.gnu.org/cgi-bin/viewcvs/dms/dms/src/RemoteJob.py.diff?only_with_tag=refactoring&tr1=1.10.2.3&tr2=1.10.2.4&r1=text&r2=text




From MAILER-DAEMON Sun Apr 03 04:55:56 2005
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1DI0tV-00015J-32
	for mharc-dms-commit@gnu.org; Sun, 03 Apr 2005 04:55:46 -0400
Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43)
	id 1DHmVo-00016b-Bw
	for dms-commit@nongnu.org; Sat, 02 Apr 2005 12:34:20 -0500
Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43)
	id 1DHmVd-00012p-AF
	for dms-commit@nongnu.org; Sat, 02 Apr 2005 12:34:14 -0500
Received: from [199.232.76.173] (helo=monty-python.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1DHmVc-0000wk-K5
	for dms-commit@nongnu.org; Sat, 02 Apr 2005 12:34:08 -0500
Received: from [199.232.41.3] (helo=savannah.gnu.org)
	by monty-python.gnu.org with esmtp (Exim 4.34) id 1DHmDL-0005mx-FB
	for dms-commit@nongnu.org; Sat, 02 Apr 2005 12:15:15 -0500
Received: from savannah.gnu.org ([127.0.0.1])
	by savannah.gnu.org with smtp (Exim 3.35 #1 (Debian))
	id 1DHmD8-0005fm-00
	for <dms-commit@nongnu.org>; Sat, 02 Apr 2005 12:15:03 -0500
To: dms-commit@nongnu.org
From: Julien Barbot <julien.barbot@laposte.net>
Message-Id: <E1DHmD8-0005fm-00@savannah.gnu.org>
Date: Sat, 02 Apr 2005 12:15:03 -0500
X-Mailman-Approved-At: Sun, 03 Apr 2005 04:55:23 -0400
Subject: [Dms-commit] Changes to dms/src/RemoteJob.py [refactoring]
X-BeenThere: dms-commit@nongnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: dms-commit.nongnu.org
List-Unsubscribe: <http://lists.nongnu.org/mailman/listinfo/dms-commit>,
	<mailto:dms-commit-request@nongnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/pipermail/dms-commit>
List-Post: <mailto:dms-commit@nongnu.org>
List-Help: <mailto:dms-commit-request@nongnu.org?subject=help>
List-Subscribe: <http://lists.nongnu.org/mailman/listinfo/dms-commit>,
	<mailto:dms-commit-request@nongnu.org?subject=subscribe>
X-List-Received-Date: Sat, 02 Apr 2005 17:34:33 -0000

Index: dms/src/RemoteJob.py
diff -u dms/src/RemoteJob.py:1.10.2.3 dms/src/RemoteJob.py:1.10.2.4
--- dms/src/RemoteJob.py:1.10.2.3	Sun Mar 27 21:01:07 2005
+++ dms/src/RemoteJob.py	Sat Apr  2 17:15:01 2005
@@ -28,7 +28,6 @@
     """
     # a lock that is shared by all RemoteJob instances
     # in order to prevent the creation of same temporary file names
-    tmp_file_creation_lock = threading.Lock()
     nb_job = 0
 
     def __init__(self, socket, scheduler_address, scheduer_port):
@@ -104,39 +103,43 @@
         # send stderr messages
         ProtocolUtils.send_data(self.__client_socket, msg_stderr)
   
-    def send_result_files_back_to_client(\
-        self,\
-        input_files,\
-        output_files):
+    def send_result_files_back_to_client(self,
+                                         output_filenames,
+                                         output_files):
         """Send the result (output files and output messages) to the client,
         and replace the temporary input file names in the various messages by
         the original ones."""
 
-        #clean temporary input files created
-        for tmp_input_file in input_files.values():
-            syslog.syslog(syslog.LOG_DEBUG | syslog.LOG_DAEMON, "Removing tem file : "\
-                          + tmp_input_file)
-            os.unlink(tmp_input_file)
         # FIXME: don't forget to send stdout and stderr
         
         # then send output files back to the client
-        for original_output_file_name in output_files.keys():
+        for (original_output_file_name, tmp_output_file_name) in output_filenames.items():
             self.__client_socket.send(Protocol.FILE_COMMAND)
-
+            
             ProtocolUtils.send_data(self.__client_socket, original_output_file_name)
-
             # send the file
-            tmp_output_file = open(\
-                output_files[original_output_file_name], 'rb')
+            # If this not a NamedTemporaryFile
+            if original_output_file_name not in output_files.keys():
+                tmp_output_file = open(tmp_output_file_name, 'rb')
+
+            else:
+                tmp_output_file = output_files[original_output_file_name]
+
             if __debug__:
-                syslog.syslog(syslog.LOG_DEBUG | syslog.LOG_DAEMON, "Sending file : "\
-                              + output_files[original_output_file_name])
-            ProtocolUtils.send_data(self.__client_socket, tmp_output_file.read())                 
-            tmp_output_file.close()
-            os.unlink(output_files[original_output_file_name])
+                syslog.syslog(syslog.LOG_DEBUG | syslog.LOG_DAEMON, "Sending file : %s" %
+                              tmp_output_file_name)
+
+            tmp_output_file.seek(0)
+            ProtocolUtils.send_data(self.__client_socket, tmp_output_file.read())
+
+            # If this not a NamedTemporaryFile
+            if original_output_file_name not in output_files.keys():
+                tmp_output_file.close()
+                os.unlink(tmp_output_file_name)
+
             if __debug__:
-                syslog.syslog(syslog.LOG_DEBUG | syslog.LOG_DAEMON, "File sent : "\
-                              + output_files[original_output_file_name])
+                syslog.syslog(syslog.LOG_DEBUG | syslog.LOG_DAEMON, "File sent : %s"
+                              % tmp_output_file_name)
         
     def compiler_job(self):
         """This method is called when a remote client
@@ -156,50 +159,46 @@
         compiler_command_line = ProtocolUtils.recv_data(self.__client_socket)
         if __debug__:
             syslog.syslog(syslog.LOG_DEBUG | syslog.LOG_DAEMON,
-                          "Compilation command line received : "\
-                          + compiler_command_line)
+                          "Compilation command line received : %s"
+                          % compiler_command_line)
         compiler_command = CompilerCommandFactory.build_compiler_instance\
                            (compiler_command_line.split())
 
 
         # get the content of the input files used in the command line we
         # have just received
+        input_temp_filenames = {}
+        # keep a reference on temporary files to prevent it to be garbage collected
         input_temp_files = {}
         output_temp_files = {}
+        output_temp_filenames = {}
         command = self.__client_socket.recv(Protocol.COMMAND_TYPE_SIZE)
                                             
         while command == Protocol.FILE_COMMAND:
 
             file_name = ProtocolUtils.recv_data(self.__client_socket)
 
-            # FIXME: do we need to create the file inside the
-            # critical section ?
-            RemoteJob.tmp_file_creation_lock.acquire()
-            if __debug__:
-                syslog.syslog(syslog.LOG_DEBUG | syslog.LOG_DAEMON,\
-                              "Creating temporary file.")
-            tmp_file_name = tempfile.mktemp(\
-                FileUtils.get_file_extension(file_name))
-            tmp_file = open(tmp_file_name, 'w')
-            RemoteJob.tmp_file_creation_lock.release()
+            tmp_file = tempfile.NamedTemporaryFile("w+b", -1, FileUtils.get_file_extension(file_name))
+
             if __debug__:
                 syslog.syslog(syslog.LOG_DEBUG | syslog.LOG_DAEMON,\
-                              "Temporary file created.")
+                              "Temporary file created: %s." % tmp_file.name)
 
-            input_temp_files[file_name] = tmp_file_name
+            input_temp_filenames[file_name] = tmp_file.name
+            input_temp_files[file_name] = tmp_file
+            
             tmp_file.write(ProtocolUtils.recv_data(self.__client_socket))
             tmp_file.flush()
-            tmp_file.close()
+
             command = self.__client_socket.recv(Protocol.COMMAND_TYPE_SIZE)
-                                                
 
         # replace original input files in the command line by the
         # temporary ones
-        compiler_command.replace_input_files(input_temp_files)
+        compiler_command.replace_input_files(input_temp_filenames)
         if __debug__:
             syslog.syslog(syslog.LOG_DEBUG | syslog.LOG_DAEMON, \
-                          "New compilation command line :"\
-                          + " ".join(compiler_command.get_command_args()))
+                          "New compilation command line : %s"\
+                           % " ".join(compiler_command.get_command_args()))
 
 
         # FIXME We should not use "-o" here, this is compiler dependant
@@ -211,20 +210,18 @@
                 index_output = compiler_command.get_command_args().index("-o") \
                                + 1
                 
-                # FIXME: do we need to create the file inside the
-                # critical section ?
-                RemoteJob.tmp_file_creation_lock.acquire()
-                tmp_output_file = tempfile.mktemp()
-                tmp_output_file_hdl = open(tmp_output_file, 'w')
-                tmp_output_file_hdl.close()
-                RemoteJob.tmp_file_creation_lock.release()
+                tmp_output_file = tempfile.NamedTemporaryFile()
+
                 # associate the output tmp file with the original one
                 output_file_name = compiler_command.get_command_args()\
                                   [index_output]
+
+                output_temp_filenames[output_file_name] = tmp_output_file.name
                 output_temp_files[output_file_name] = tmp_output_file
+                
                 # replace the output file name in the command line
                 compiler_command.get_command_args(\
-                    )[index_output] = tmp_output_file
+                    )[index_output] = tmp_output_file.name
             except IndexError:
                 # if there is no file name after the -o option,
                 # it means that the command line is wrong, but we
@@ -236,22 +233,25 @@
             if __debug__:
                 syslog.syslog(syslog.LOG_DEBUG | syslog.LOG_DAEMON, \
                               "No -o option in command line.")
-            for original_input_file_name in input_temp_files.keys():
+            for original_input_file_name in input_temp_filenames.keys():
                 stop_step = compiler_command.get_stop_step()
+
                 orig_output_file_name = compiler_command.\
                                         get_output_file_name_for_step(\
                     original_input_file_name,\
                     stop_step)
-                output_temp_files[\
+
+                # FIXME: Here there is a problem ... we do not give a temp filename
+                output_temp_filenames[\
                     orig_output_file_name] = compiler_command.\
-                    get_output_file_name_for_step(\
-                    input_temp_files[original_input_file_name],\
+                    get_output_file_name_for_step(
+                    input_temp_filenames[original_input_file_name],
                     stop_step)
                 
                 if __debug__:
                     syslog.syslog(syslog.LOG_DEBUG | syslog.LOG_DAEMON, \
-                                  "File to return to the client : "\
-                                  + output_temp_files[orig_output_file_name])
+                                  "File to return to the client : %s" %
+                                  output_temp_filenames[orig_output_file_name])
                 
         # execute the command in a subshell and get the stdout and stderr output
         if __debug__:
@@ -271,8 +271,9 @@
         proc.stdout.close()
         exit_code = proc.wait()
 
-        self.send_output_messages(msg_stdout, msg_stderr, input_temp_files,\
-                                  output_temp_files)
+        self.send_output_messages(msg_stdout, msg_stderr,
+                                  input_temp_filenames,
+                                  output_temp_filenames)
 
         if os.WIFEXITED(exit_code):
             exit_code = os.WEXITSTATUS(exit_code)
@@ -283,7 +284,7 @@
                           "Exit code : " + str(exit_code))
         if exit_code == 0:
             # send the result (output files and output messages) to the client
-            self.send_result_files_back_to_client(input_temp_files,\
+            self.send_result_files_back_to_client(output_temp_filenames,
                                                   output_temp_files)
             if __debug__:
                 syslog.syslog(syslog.LOG_DEBUG | syslog.LOG_DAEMON,\



From MAILER-DAEMON Sun Apr 03 04:56:00 2005
Received: from mailman by lists.gnu.org with archive (Exim 4.43)
	id 1DI0tf-00015j-MD
	for mharc-dms-commit@gnu.org; Sun, 03 Apr 2005 04:55:57 -0400
Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43)
	id 1DHmVm-00015c-0Z
	for dms-commit@nongnu.org; Sat, 02 Apr 2005 12:34:18 -0500
Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43)
	id 1DHmVf-00013c-A1
	for dms-commit@nongnu.org; Sat, 02 Apr 2005 12:34:11 -0500
Received: from [199.232.76.173] (helo=monty-python.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1DHmVc-0000wk-U3
	for dms-commit@nongnu.org; Sat, 02 Apr 2005 12:34:09 -0500
Received: from [199.232.41.3] (helo=savannah.gnu.org)
	by monty-python.gnu.org with esmtp (Exim 4.34) id 1DHmDK-0005mx-V4
	for dms-commit@nongnu.org; Sat, 02 Apr 2005 12:15:15 -0500
Received: from savannah.gnu.org ([127.0.0.1])
	by savannah.gnu.org with smtp (Exim 3.35 #1 (Debian))
	id 1DHmD7-0005fX-00
	for <dms-commit@nongnu.org>; Sat, 02 Apr 2005 12:15:01 -0500
To: dms-commit@nongnu.org
From: Julien Barbot <julien.barbot@laposte.net>
Message-Id: <E1DHmD7-0005fX-00@savannah.gnu.org>
Date: Sat, 02 Apr 2005 12:15:01 -0500
X-Mailman-Approved-At: Sun, 03 Apr 2005 04:55:24 -0400
Subject: [Dms-commit] Changes to dms/src/Compiler.py [refactoring]
X-BeenThere: dms-commit@nongnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: dms-commit.nongnu.org
List-Unsubscribe: <http://lists.nongnu.org/mailman/listinfo/dms-commit>,
	<mailto:dms-commit-request@nongnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/pipermail/dms-commit>
List-Post: <mailto:dms-commit@nongnu.org>
List-Help: <mailto:dms-commit-request@nongnu.org?subject=help>
List-Subscribe: <http://lists.nongnu.org/mailman/listinfo/dms-commit>,
	<mailto:dms-commit-request@nongnu.org?subject=subscribe>
X-List-Received-Date: Sat, 02 Apr 2005 17:34:48 -0000

Index: dms/src/Compiler.py
diff -u dms/src/Compiler.py:1.20.2.5 dms/src/Compiler.py:1.20.2.6
--- dms/src/Compiler.py:1.20.2.5	Sun Mar 27 20:30:29 2005
+++ dms/src/Compiler.py	Sat Apr  2 17:15:01 2005
@@ -1,4 +1,4 @@
-#!/usr/bin/python -O
+#!/usr/bin/python
 
 __author__ = "Julien Gilli <jgilli@nerim.fr>"
 __version__ = "0.0"



