From schnetter at cct.lsu.edu  Wed Sep  2 09:48:49 2009
From: schnetter at cct.lsu.edu (Erik Schnetter)
Date: Wed, 2 Sep 2009 09:48:49 -0500
Subject: [Developers] CactusEOS/EOS_Ideal_Fluid
In-Reply-To: <20090831154922.GR18118@numrel07.cct.lsu.edu>
References: <20090831154922.GR18118@numrel07.cct.lsu.edu>
Message-ID: <0432ED8B-4A8A-4F97-AC20-5B4512DBC56D@cct.lsu.edu>

On Aug 31, 2009, at 10:49 AM, Frank Loeffler wrote:

> Hi,
>
> are there objections for making the parameter eos_ideal_fluid_gamma in
> CactusEOS/EOS_Ideal_Fluid restricted from now private? This way other
> thorns can use this parameter.


This is a good idea.

-erik

-- 
Erik Schnetter <schnetter at cct.lsu.edu>   http://www.cct.lsu.edu/~eschnett/

My email is as private as my paper mail.  I therefore support encrypting
and signing email messages.  Get my PGP key from <http://pgp.mit.edu>.



-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 203 bytes
Desc: This is a digitally signed message part
Url : http://www.cactuscode.org/pipermail/developers/attachments/20090902/596a2533/attachment.bin 

From knarf at cct.lsu.edu  Thu Sep 10 14:54:27 2009
From: knarf at cct.lsu.edu (Frank Loeffler)
Date: Thu, 10 Sep 2009 14:54:27 -0500
Subject: [Developers] improve parser for parameter files
Message-ID: <20090910195426.GA11625@numrel07.cct.lsu.edu>

This patch improves the parser used by Cactus to prepocess parameter
files.

The main changes are:

- when reading a parameter file do not parse the file while reading,
  but read it first into a buffer, preprocess that and parse the
  buffer after that
- replace the code which changes $parfile into the parameter file
  name to use this buffer preprocessing
- replace $ENV_ "defines" by environment variable values

Frank


Index: util/ParseFile.c
===================================================================
RCS file: /cactusdevcvs/Cactus/src/util/ParseFile.c,v
retrieving revision 1.26
diff -d -u -r1.26 ParseFile.c
--- util/ParseFile.c	31 Aug 2006 16:54:40 -0000	1.26
+++ util/ParseFile.c	10 Sep 2009 19:49:49 -0000
@@ -7,7 +7,6 @@
               to a user-supplied subroutine.
               Currently taken from the old cactus ones and slightly modifed.
    @enddesc
-   @version   $Id: ParseFile.c,v 1.26 2006/08/31 16:54:40 tradke Exp $
  @@*/
 
 /*#define DEBUG*/
@@ -15,6 +14,9 @@
 #include <stdio.h>
 #include <string.h>
 #include <stdlib.h>
+#include <string.h>
+
+#include <cctk.h>
 
 #include "cctk_Flesh.h"
 #include "cctk_CommandLine.h"
@@ -38,6 +40,11 @@
 
 static void CheckBuf(int, int);
 static void removeSpaces(char *stripMe);
+static char *ReadFile(FILE *file, unsigned long *filesize);
+static char *ParseDefines(char *buffer, unsigned long *buffersize);
+int ParseBuffer(char *buffer,
+                int (*set_function)(const char *, const char *, int),
+                tFleshConfig *ConfigData);
 
 /********************************************************************
  ********************* Other Routine Prototypes *********************
@@ -63,7 +70,7 @@
 
  /*@@
    @routine ParseFile
-   @author Paul Walker
+   @author Paul Walker, Frank Loeffler
    @desc
    This routine actually parses the parameter file. The
    syntax we allow is
@@ -83,6 +90,7 @@
    @hdesc Moved to CCTK.
           Changed to pass data to arbitrary function.
           Changed to take a file descriptor rather than a filename.
+          Use a buffer to parse which can be preprocessed before main parsing
    @endhistory
    @var     ifp
    @vdesc   The filestream to parse
@@ -111,10 +119,428 @@
    0 - success
    @endreturndesc
 @@*/
+
 int ParseFile(FILE *ifp,
               int (*set_function)(const char *, const char *, int),
               tFleshConfig *ConfigData)
 {
+  int retval;
+  unsigned long buffersize;
+  char *buffer = ReadFile(ifp, &buffersize);
+  if (!buffer)
+    return 1;
+  buffer = ParseDefines(buffer, &buffersize);
+  retval = ParseBuffer(buffer, set_function, ConfigData);
+  free(buffer);
+  return retval;
+}
+
+/********************************************************************
+ *********************     Local Routines   *************************
+ ********************************************************************/
+
+ /*@@
+   @routine CheckBuf
+   @author Paul Walker
+   @desc
+   A simple description and warning message in case of
+   a fixed parse buffer overflow.
+   @enddesc
+   @calls
+   @calledby
+   @history
+
+   @endhistory
+   @var     p
+   @vdesc   buffer location
+   @vtype   int
+   @vio     in
+   @vcomment
+
+   @endvar
+   @var     l
+   @vdesc   Line number
+   @vtype   int
+   @vio     in
+   @vcomment
+
+   @endvar
+
+@@*/
+
+static void CheckBuf(int p, int l)
+{
+  if (p >= BUF_SZ)
+  {
+    fprintf(stderr,"WARNING: Parser buffer overflow on line %d\n",
+            l);
+    fprintf(stderr,"This indicates either an incorrect parm file or\n");
+    fprintf(stderr,"the need to recompile " __FILE__ " with a bigger\n");
+    fprintf(stderr,"BUF_SZ parm.\n");
+
+    assert(0);
+    exit(1);
+  }
+}
+
+
+ /*@@
+   @routine removeSpaces
+   @author Paul Walker
+   @desc
+   removes the spaces from a char * <b>in place</b>. Beware
+   that this function will change the input value and if you
+   want to keep a copy you have to do so yourself!
+   @enddesc
+   @calls
+   @calledby
+   @history
+
+   @endvar
+   @var     stripMe
+   @vdesc   String to strip
+   @vtype   char *
+   @vio     inout
+   @vcomment
+
+   @endvar
+
+@@*/
+static void removeSpaces(char *stripMe)
+{
+  char *s;
+  unsigned int i,j;
+  s = (char *)malloc((strlen(stripMe)+2)*sizeof(char));
+
+  if(s)
+  {
+    strcpy(s,stripMe);
+    for (i=0,j=0;i<strlen(s);i++)
+    {
+      if (s[i] != ' ' && s[i] != '\t' && s[i] != '\n' && s[i] != '\r')
+      {
+        stripMe[j++] = s[i];
+      }
+      stripMe[j] = '\0';
+    }
+  }
+
+  free(s);
+}
+
+
+/* #define TEST_ParseFile */
+
+#ifdef TEST_ParseFile
+
+int parameter_printer(const char *param, const char *val)
+{
+  printf("Parameter %s has value %s\n", param, val);
+
+  return 0;
+}
+
+int main(int argc, char *argv[])
+{
+  int retval;
+  FILE *parameter_file;
+
+  if(argc > 1)
+  {
+    parameter_file = fopen(argv[1], "r");
+    if(parameter_file)
+    {
+      ParseFile(parameter_file, parameter_printer);
+      fclose(parameter_file);
+      retval = 0;
+    }
+    else
+    {
+      retval=2;
+    }
+  }
+  else
+  {
+    printf("Usage: %s <filename>\n", argv[0]);
+    retval = 1;
+  };
+
+  return 0;
+}
+
+#endif
+
+
+ /*@@
+   @routine ReadFile
+   @author Frank Loeffler
+   @desc
+   This routine reads a file into a buffer
+   @enddesc
+   @history
+   @var     file
+   @vdesc   The filestream to read
+   @vtype   FILE *
+   @vio     in
+   @vcomment
+
+   @endvar
+   @var     filesize
+   @vdesc   The size of the file
+   @vtype   *unsigned long
+   @vio     out
+   @vcomment
+
+   @endvar
+
+   @returntype char *
+   @returndesc
+   NULL - failure
+   !NULL allocated buffer
+   @endreturndesc
+@@*/
+static char *ReadFile(FILE *file, unsigned long *filesize)
+{
+  char *buffer;
+
+  if (!file)
+  {
+    fprintf(stderr, "Could not use file for reading.\n");
+    return NULL;
+  }
+  /* Get the file size */
+  fseek(file, 0, SEEK_END);
+  *filesize = ftell(file);
+  fseek(file, 0, SEEK_SET);
+  /* Allocate buffer */
+  buffer = (char *)malloc(*filesize+1);
+  if (!buffer)
+  {
+    fprintf(stderr, "Could not allocate memory.\n");
+    return NULL;
+  }
+  /* Read file into buffer and return */
+  fread(buffer, *filesize, 1, file);
+  /* Protect buffer for string operations */
+  buffer[*filesize] = '\0';
+  return buffer;
+}
+
+ /*@@
+   @routine ParseDefines
+   @author Frank Loeffler
+   @desc
+   This routine parses a buffer for defines of the form \$[^ \n] and
+   replaces them with some predefined defines, returning the new buffer,
+   which might be the same as the old buffer. Otherwise the old buffer
+   gets freed by this function. In case of parsing errors, the whole
+   parsing is stopped and the old buffer is returned
+   (FIXME: Should we return NULL?)
+   @enddesc
+   @history
+   @var     buffer
+   @vdesc   The buffer to parse
+   @vtype   char *
+   @vio     in
+   @vcomment
+
+   @endvar
+   @var     buffersize
+   @vdesc   The size of the buffer
+   @vtype   *unsigned long
+   @vio     out
+   @vcomment
+
+   @endvar
+
+   @returntype char *
+   @returndesc
+   !NULL - new buffer, might be == buffer
+   @endreturndesc
+@@*/
+static char *ParseDefines(char *buffer, unsigned long *buffersize)
+{
+  /* define name */
+  char define[1024];
+  /* Position in define name */
+  char defpos = 0;
+  /* Current position in buffer */
+  size_t pos = 0;
+  /* Character at current position */
+  char c;
+  /* Position of start of found define */
+  size_t def_start = 0;
+  /* Flag to indicate if we are inside a definition name */
+  int indef = 0;
+  if (!buffer)
+    return buffer;
+  /* Main loop over the buffer */
+  while (((c=buffer[pos]) != '\0') && (pos < *buffersize) )
+  {
+    /* Check if we found a define */
+    if (c == '$')
+    {
+      /* Mark the state and location and go on */
+      indef = 1;
+      def_start = pos;
+    }
+    /* If we are inside a define */
+    else if (indef)
+    {
+      /* Look for the end of the define name */
+      if (c == ' ' || c == '/' || c == '\0' || c == '\n')
+      {
+        /* make the define name a proper string */
+        define[defpos] = '\0';
+        /* this holds the value of the define to be filled in */
+        char *value = NULL;
+        /* a new buffer for the whole thing */
+        char * new_buffer;
+        unsigned long new_size;
+        /* Check for different kinds of defines */
+        /* Environment variables */
+        if (strncmp(define, "ENV_", 4) == 0)
+        {
+          value = getenv(define+4);
+          if (!value)
+          {
+            CCTK_VWarn(CCTK_WARN_ALERT, __LINE__, __FILE__, CCTK_THORNSTRING,
+                       "No environment variable %s found\n", define+4);
+            /* TODO: Should we abort here? (return NULL) */
+          }
+        }
+        /* Parameter file name */
+        else if (strcmp(define, "parfile") == 0)
+        {
+          char path[500];
+          CCTK_ParameterFilename(500, path);
+          value = strrchr (path, '/');
+          if (value == NULL)
+          {
+            value = path;
+          }
+          else
+          {
+            value++;
+          }
+          /* skip the parameter file extension */
+          if (strcmp (value + strlen (value) - 4, ".par") == 0)
+          {
+            value[strlen (value) - 4] = '\0';
+          }
+        }
+        /* Else: unknown define - or no define at all: ignore */
+        if (value)
+        {
+          /* Replace the old buffer with the new, copying in all the data */
+          new_size = *buffersize - strlen(define) + strlen(value) + 1;
+          if (new_size < def_start)
+          {
+            CCTK_WARN(0, "Something is wrong with me, HELP!");
+            return buffer;
+          }
+          new_buffer = (char *)malloc(new_size);
+          if (!new_buffer)
+          {
+            CCTK_WARN(0, "I am out of memory and give up parsing for defines.");
+            return buffer;
+          }
+          new_buffer[0]='\0';
+          strncat(new_buffer, buffer, def_start);
+          strncat(new_buffer, value, new_size-1);
+          strncat(new_buffer, buffer+pos, new_size-1);
+          new_buffer[new_size-1] = '\0';
+          /* free the old buffer */
+          free(buffer);
+          /* Start further processing of new buffer */
+          pos = def_start + strlen(define);
+          buffer = new_buffer;
+          *buffersize = new_size;
+        }
+        /* General define cleanup */
+        indef = 0;
+        defpos = 0;
+        define[0] = '\0';
+        def_start = 0;
+      }
+      else if (defpos > 1023)
+      {
+        /* We only print this warning and ignore the possible define problem.
+         * It might not be a define after all but a valid, long parameter
+         * value containing a '$'.*/
+        define[1023] = '\0';
+        CCTK_VWarn(CCTK_WARN_PICKY, __LINE__, __FILE__, CCTK_THORNSTRING,
+                   "Possible define too long: %s", define);
+        indef = 0;
+        defpos = 0;
+        define[0] = '\0';
+        def_start = 0;
+      }
+      else
+        define[defpos++] = c;
+    }
+    pos++;
+  }
+  return buffer;
+}
+
+ /*@@
+   @routine ParseBuffer
+   @author Paul Walker
+   @desc
+   This routine actually parses the parameter file. The
+   syntax we allow is
+   <ul>
+     <li>a = b
+         <li>a,b,c = d,e,f
+         <li># rest of the line is ignored
+         <li>x = "string value"
+   </ul>
+   So it is easy to parse
+   <p>
+   We go through the file looking for stuff and then set
+   it in the global database using calls to the passed in set_function.
+   @enddesc
+   @history
+   @hdate Tue Jan 12 16:41:36 1999 @hauthor Tom Goodale
+   @hdesc Moved to CCTK.
+          Changed to pass data to arbitrary function.
+          Changed to take a file descriptor rather than a filename.
+          Use a buffer to parse which can be preprocessed before main parsing
+   @endhistory
+   @var     buffer
+   @vdesc   The buffer to parse
+   @vtype   char *
+   @vio     in
+   @vcomment
+
+   @endvar
+   @var     set_function
+   @vdesc   The function to call to set the value of a parameter
+   @vtype   int (*)(const char *, const char *)
+   @vio     in
+   @vcomment
+
+   @endvar
+   @var     ConfigData
+   @vdesc   Flesh configuration data
+   @vtype   tFleshConfig *
+   @vio     in
+   @vcomment
+
+   @endvar
+
+   @returntype int
+   @returndesc
+   0 - success
+   @endreturndesc
+@@*/
+int ParseBuffer(char *buffer,
+              int (*set_function)(const char *, const char *, int),
+              tFleshConfig *ConfigData)
+{
+  /* position in buffer */
+  unsigned int pos = 0;
   /* Buffers for parsing from the file */
   char *tokens, *value;
   char *subtoken, *subvalue;
@@ -122,10 +548,8 @@
   int ntokens;
   /* Status flags */
   int intoken, inval;
-  /* Current char. Have to make it an int so we can compare with
-     EOF. See man 3 fgetc
-   */
-  int c;
+  /* Current char */
+  char c;
   int num_errors; /* number of errors in file parsing */
 
   num_errors = 0;
@@ -141,7 +565,7 @@
 
   intoken = 0; inval = 0;
 
-  while ((c=fgetc(ifp)) != EOF)
+  while ((c=buffer[pos++]) != '\0')
   {
 #ifdef DEBUG
     printf("%c",c);
@@ -150,7 +574,7 @@
     while (c == '#' || c == '!' )
     {
       /* Comment line.  So forget rest of line */
-      while ((c=fgetc(ifp)) != '\n' && c != '\r' && c != EOF)
+      while ((c=buffer[pos++]) != '\n' && c != '\r' && c != '\0')
       {
 #ifdef DEBUG
         printf("%c",c);
@@ -160,7 +584,7 @@
       {
         lineno++;
       }
-      c = fgetc(ifp);
+      c = buffer[pos++];
 #ifdef DEBUG
       printf("%c",c);
 #endif
@@ -220,7 +644,7 @@
          * and check if the value is a string or not.
          * This parser DOES strip quotes off of the strings.
          */
-        while ((c = fgetc(ifp)) == ' ' || c == '\n' || c == '\r' || c == '\t')
+        while ((c = buffer[pos++]) == ' ' || c == '\n' || c == '\r' || c == '\t')
         {
 #ifdef DEBUG
           printf("%c",c);
@@ -251,7 +675,7 @@
             free (tokens);
             return 1;
           }
-          while ((c = fgetc(ifp)) != '"')
+          while ((c = buffer[pos++]) != '"')
           {
 #ifdef DEBUG
             printf("%c",c);
@@ -265,7 +689,7 @@
 #endif
               lineno++;
             }
-            else if (c == EOF)
+            else if (c == '\0')
             {
               break;
             }
@@ -274,9 +698,9 @@
           if (c == '"')
           {
             int is_comment = 0;
-            while ((c = fgetc(ifp)) != '\n')
+            while ((c = buffer[pos++]) != '\n')
             {
-              if (c == EOF)
+              if (c == '\0')
               {
                 break;
               }
@@ -295,7 +719,7 @@
                 return 1;
               }
             }
-            if (c == '\n' || c == EOF)
+            if (c == '\n' || c == '\0')
             {
               if (c == '\n') lineno++;
               /* mark it as properly quoted string value */
@@ -319,39 +743,6 @@
 #endif
           set_function(tokens,value, lineno);
         }
-        else if (c == '$')
-        {
-          /* We got a define */
-          /* FIXME: Assume it is a parameter file for now */
-          char path[500];
-          char *parfile;
-
-          CCTK_ParameterFilename(500,path);
-          parfile = strrchr (path, '/');
-          if (parfile == NULL)
-          {
-            parfile = path;
-          }
-          else
-          {
-            parfile++;
-          }
-          /* skip the parameter file extension */
-          if (strcmp (parfile + strlen (parfile) - 4, ".par") == 0)
-          {
-            parfile[strlen (parfile) - 4] = 0;
-          }
-
-          /* ignore everything else on the line */
-          while (!(c==' ' || c=='\t' || c == '\n' || c == '\r' || c == EOF))
-          {
-            c = fgetc(ifp);
-#ifdef DEBUG
-            printf("%c",c);
-#endif
-          }
-          set_function(tokens,parfile,lineno);
-        }
         else
         {
 
@@ -362,15 +753,15 @@
             /* Simple case. We have an int
                or a double which contain no
                spaces! */
-            c = fgetc(ifp);
+            c = buffer[pos++];
 #ifdef DEBUG
             printf("%c",c);
 #endif
-            while (!(c==' ' || c=='\t' || c == '\n' || c == '\r' || c == EOF))
+            while (!(c==' ' || c=='\t' || c == '\n' || c == '\r' || c == '\0'))
             {
               value[p++] = (char)c;
               CheckBuf(p,lineno);
-              c = fgetc(ifp);
+              c = buffer[pos++];
 #ifdef DEBUG
               printf("%c",c);
 #endif
@@ -402,11 +793,11 @@
                ntokens-1 commas, stripping spaces, and
                make a nice little string.
                */
-            c = fgetc(ifp);
+            c = buffer[pos++];
 #ifdef DEBUG
             printf("%c",c);
 #endif
-            while (ncommas < ntokens-1 && c != EOF)
+            while (ncommas < ntokens-1 && c != '\0')
             {
               if (!(c == ' ' || c == '\t' || c == '\n' || c == '\r'))
               {
@@ -414,7 +805,7 @@
                 CheckBuf(pp,lineno);
               }
               if (c == ',') ncommas ++;
-              c = fgetc(ifp);
+              c = buffer[pos++];
 #ifdef DEBUG
               printf("%c",c);
 #endif
@@ -422,7 +813,7 @@
             if (c == ' ' || c == '\t')
             {
               /* Great now strip out the spaces */
-              while((c = fgetc(ifp)) == ' ' || c=='\t' || c == '\n' || c == '\r')
+              while((c = buffer[pos++]) == ' ' || c=='\t' || c == '\n' || c == '\r')
               {
 #ifdef DEBUG
                 printf("%c",c);
@@ -441,15 +832,15 @@
             value[pp++] = (char)c;
             CheckBuf(p,lineno);
 
-            c = fgetc(ifp);
+            c = buffer[pos++];
 #ifdef DEBUG
             printf("%c",c);
 #endif
-            while (c != ' ' && c != '\t' && c != '\n' && c != '\r' && c != EOF)
+            while (c != ' ' && c != '\t' && c != '\n' && c != '\r' && c != '\0')
             {
               value[pp++] = (char)c;
               CheckBuf(pp,lineno);
-              c = fgetc(ifp);
+              c = buffer[pos++];
 #ifdef DEBUG
               printf("%c",c);
 #endif
@@ -525,137 +916,3 @@
   return num_errors;
 }
 
-/********************************************************************
- *********************     Local Routines   *************************
- ********************************************************************/
-
- /*@@
-   @routine CheckBuf
-   @author Paul Walker
-   @desc
-   A simple description and warning message in case of
-   a fixed parse buffer overflow.
-   @enddesc
-   @calls
-   @calledby
-   @history
-
-   @endhistory
-   @var     p
-   @vdesc   buffer location
-   @vtype   int
-   @vio     in
-   @vcomment
-
-   @endvar
-   @var     l
-   @vdesc   Line number
-   @vtype   int
-   @vio     in
-   @vcomment
-
-   @endvar
-
-@@*/
-
-static void CheckBuf(int p, int l)
-{
-  if (p >= BUF_SZ)
-  {
-    fprintf(stderr,"WARNING: Parser buffer overflow on line %d\n",
-            l);
-    fprintf(stderr,"This indicates either an incorrect parm file or\n");
-    fprintf(stderr,"the need to recompile " __FILE__ " with a bigger\n");
-    fprintf(stderr,"BUF_SZ parm.\n");
-
-    assert(0);
-    exit(1);
-  }
-}
-
-
- /*@@
-   @routine removeSpaces
-   @author Paul Walker
-   @desc
-   removes the spaces from a char * <b>in place</b>. Beware
-   that this function will change the input value and if you
-   want to keep a copy you have to do so yourself!
-   @enddesc
-   @calls
-   @calledby
-   @history
-
-   @endvar
-   @var     stripMe
-   @vdesc   String to strip
-   @vtype   char *
-   @vio     inout
-   @vcomment
-
-   @endvar
-
-@@*/
-static void removeSpaces(char *stripMe)
-{
-  char *s;
-  unsigned int i,j;
-  s = (char *)malloc((strlen(stripMe)+2)*sizeof(char));
-
-  if(s)
-  {
-    strcpy(s,stripMe);
-    for (i=0,j=0;i<strlen(s);i++)
-    {
-      if (s[i] != ' ' && s[i] != '\t' && s[i] != '\n' && s[i] != '\r')
-      {
-        stripMe[j++] = s[i];
-      }
-      stripMe[j] = '\0';
-    }
-  }
-
-  free(s);
-}
-
-
-/* #define TEST_ParseFile */
-
-#ifdef TEST_ParseFile
-
-int parameter_printer(const char *param, const char *val)
-{
-  printf("Parameter %s has value %s\n", param, val);
-
-  return 0;
-}
-
-int main(int argc, char *argv[])
-{
-  int retval;
-  FILE *parameter_file;
-
-  if(argc > 1)
-  {
-    parameter_file = fopen(argv[1], "r");
-    if(parameter_file)
-    {
-      ParseFile(parameter_file, parameter_printer);
-      fclose(parameter_file);
-      retval = 0;
-    }
-    else
-    {
-      retval=2;
-    }
-  }
-  else
-  {
-    printf("Usage: %s <filename>\n", argv[0]);
-    retval = 1;
-  };
-
-  return 0;
-}
-
-#endif


--- StripMime Report -- processed MIME parts ---
multipart/mixed
  text/plain (text body -- kept)
  text/plain (text body -- kept)
---

From schnetter at cct.lsu.edu  Thu Sep 10 17:53:44 2009
From: schnetter at cct.lsu.edu (Erik Schnetter)
Date: Fri, 11 Sep 2009 06:53:44 +0800
Subject: [Developers] [Patches] improve parser for parameter files
In-Reply-To: <20090910195426.GA11625@numrel07.cct.lsu.edu>
References: <20090910195426.GA11625@numrel07.cct.lsu.edu>
Message-ID: <8B2F1374-1B31-45BF-82E5-B3F27970212E@cct.lsu.edu>

On Sep 11, 2009, at 3:54 , Frank Loeffler wrote:

> This patch improves the parser used by Cactus to prepocess parameter
> files.
>
> The main changes are:
>
> - when reading a parameter file do not parse the file while reading,
>  but read it first into a buffer, preprocess that and parse the
>  buffer after that
> - replace the code which changes $parfile into the parameter file
>  name to use this buffer preprocessing
> - replace $ENV_ "defines" by environment variable values


I like the idea of being able to use environment variables in  
parameter files.

The code that processes the buffer is not very efficient.  It probably  
doesn't matter in practice, but it would be more efficient to allocate  
a much larger buffer in the beginning instead of extending it  
piecewise.  Extending it piecewise is an O(N^2) algorithm, doubling  
the buffer size each time it is too small has a complexity of O(N).

The syntax $ENV_* is somewhat arbitrary.  I would prefer a more  
standard syntax, as found e.g. in bash or in perl.  Bash syntax would  
be $var or ${var}, perl syntax would be $ENV{'var'}.  Maybe we should  
use $var, which has the same syntax as $parfile?  In this way,  
Cactus's $parfile would be just a special variable.  If we  
additionally allow ${var}, people could also use $parfile to construct  
file name, especially if we also introduce $pardir, and modify the  
exact definition of $parfile ? the current parsing code for $parfile  
is very naive.

-erik

-- 
Erik Schnetter <schnetter at cct.lsu.edu>   http://www.cct.lsu.edu/~eschnett/

My email is as private as my paper mail.  I therefore support encrypting
and signing email messages.  Get my PGP key from <http://pgp.mit.edu>.



-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 203 bytes
Desc: This is a digitally signed message part
Url : http://www.cactuscode.org/pipermail/developers/attachments/20090911/bf9c49f3/attachment.bin 

From baiotti at yukawa.kyoto-u.ac.jp  Thu Sep 10 20:40:34 2009
From: baiotti at yukawa.kyoto-u.ac.jp (Baiotti Luca)
Date: Fri, 11 Sep 2009 10:40:34 +0900
Subject: [Developers] improve parser for parameter files
In-Reply-To: <20090910195426.GA11625@numrel07.cct.lsu.edu>
References: <20090910195426.GA11625@numrel07.cct.lsu.edu>
Message-ID: <4AA9AA92.4040601@yukawa.kyoto-u.ac.jp>

Hello,

I would have a feature request about the parameter-file checks (so it is not 
strictly related to the patch, but I would like to ask anyway).

When an error is detected in the parfile at runtime, each processor produces a 
warning. Since there may be hundreds of processors and since each processor parses 
and checks the same parfile, I find it useless and confusing that the error 
message is repeated one time per processor. I would like the error message 
originated from errors in the parfile to be output only once (from the headnode?).


Best,

Luca



Frank Loeffler wrote:
> This patch improves the parser used by Cactus to prepocess parameter
> files.
> 
> The main changes are:
> 
> - when reading a parameter file do not parse the file while reading,
>   but read it first into a buffer, preprocess that and parse the
>   buffer after that
> - replace the code which changes $parfile into the parameter file
>   name to use this buffer preprocessing
> - replace $ENV_ "defines" by environment variable values
> 
> Frank
> 
> 
> Index: util/ParseFile.c
> ===================================================================
> RCS file: /cactusdevcvs/Cactus/src/util/ParseFile.c,v
> retrieving revision 1.26
> diff -d -u -r1.26 ParseFile.c
> --- util/ParseFile.c	31 Aug 2006 16:54:40 -0000	1.26
> +++ util/ParseFile.c	10 Sep 2009 19:49:49 -0000
> @@ -7,7 +7,6 @@
>                to a user-supplied subroutine.
>                Currently taken from the old cactus ones and slightly modifed.
>     @enddesc
> -   @version   $Id: ParseFile.c,v 1.26 2006/08/31 16:54:40 tradke Exp $
>   @@*/
>  
>  /*#define DEBUG*/
> @@ -15,6 +14,9 @@
>  #include <stdio.h>
>  #include <string.h>
>  #include <stdlib.h>
> +#include <string.h>
> +
> +#include <cctk.h>
>  
>  #include "cctk_Flesh.h"
>  #include "cctk_CommandLine.h"
> @@ -38,6 +40,11 @@
>  
>  static void CheckBuf(int, int);
>  static void removeSpaces(char *stripMe);
> +static char *ReadFile(FILE *file, unsigned long *filesize);
> +static char *ParseDefines(char *buffer, unsigned long *buffersize);
> +int ParseBuffer(char *buffer,
> +                int (*set_function)(const char *, const char *, int),
> +                tFleshConfig *ConfigData);
>  
>  /********************************************************************
>   ********************* Other Routine Prototypes *********************
> @@ -63,7 +70,7 @@
>  
>   /*@@
>     @routine ParseFile
> -   @author Paul Walker
> +   @author Paul Walker, Frank Loeffler
>     @desc
>     This routine actually parses the parameter file. The
>     syntax we allow is
> @@ -83,6 +90,7 @@
>     @hdesc Moved to CCTK.
>            Changed to pass data to arbitrary function.
>            Changed to take a file descriptor rather than a filename.
> +          Use a buffer to parse which can be preprocessed before main parsing
>     @endhistory
>     @var     ifp
>     @vdesc   The filestream to parse
> @@ -111,10 +119,428 @@
>     0 - success
>     @endreturndesc
>  @@*/
> +
>  int ParseFile(FILE *ifp,
>                int (*set_function)(const char *, const char *, int),
>                tFleshConfig *ConfigData)
>  {
> +  int retval;
> +  unsigned long buffersize;
> +  char *buffer = ReadFile(ifp, &buffersize);
> +  if (!buffer)
> +    return 1;
> +  buffer = ParseDefines(buffer, &buffersize);
> +  retval = ParseBuffer(buffer, set_function, ConfigData);
> +  free(buffer);
> +  return retval;
> +}
> +
> +/********************************************************************
> + *********************     Local Routines   *************************
> + ********************************************************************/
> +
> + /*@@
> +   @routine CheckBuf
> +   @author Paul Walker
> +   @desc
> +   A simple description and warning message in case of
> +   a fixed parse buffer overflow.
> +   @enddesc
> +   @calls
> +   @calledby
> +   @history
> +
> +   @endhistory
> +   @var     p
> +   @vdesc   buffer location
> +   @vtype   int
> +   @vio     in
> +   @vcomment
> +
> +   @endvar
> +   @var     l
> +   @vdesc   Line number
> +   @vtype   int
> +   @vio     in
> +   @vcomment
> +
> +   @endvar
> +
> +@@*/
> +
> +static void CheckBuf(int p, int l)
> +{
> +  if (p >= BUF_SZ)
> +  {
> +    fprintf(stderr,"WARNING: Parser buffer overflow on line %d\n",
> +            l);
> +    fprintf(stderr,"This indicates either an incorrect parm file or\n");
> +    fprintf(stderr,"the need to recompile " __FILE__ " with a bigger\n");
> +    fprintf(stderr,"BUF_SZ parm.\n");
> +
> +    assert(0);
> +    exit(1);
> +  }
> +}
> +
> +
> + /*@@
> +   @routine removeSpaces
> +   @author Paul Walker
> +   @desc
> +   removes the spaces from a char * <b>in place</b>. Beware
> +   that this function will change the input value and if you
> +   want to keep a copy you have to do so yourself!
> +   @enddesc
> +   @calls
> +   @calledby
> +   @history
> +
> +   @endvar
> +   @var     stripMe
> +   @vdesc   String to strip
> +   @vtype   char *
> +   @vio     inout
> +   @vcomment
> +
> +   @endvar
> +
> +@@*/
> +static void removeSpaces(char *stripMe)
> +{
> +  char *s;
> +  unsigned int i,j;
> +  s = (char *)malloc((strlen(stripMe)+2)*sizeof(char));
> +
> +  if(s)
> +  {
> +    strcpy(s,stripMe);
> +    for (i=0,j=0;i<strlen(s);i++)
> +    {
> +      if (s[i] != ' ' && s[i] != '\t' && s[i] != '\n' && s[i] != '\r')
> +      {
> +        stripMe[j++] = s[i];
> +      }
> +      stripMe[j] = '\0';
> +    }
> +  }
> +
> +  free(s);
> +}
> +
> +
> +/* #define TEST_ParseFile */
> +
> +#ifdef TEST_ParseFile
> +
> +int parameter_printer(const char *param, const char *val)
> +{
> +  printf("Parameter %s has value %s\n", param, val);
> +
> +  return 0;
> +}
> +
> +int main(int argc, char *argv[])
> +{
> +  int retval;
> +  FILE *parameter_file;
> +
> +  if(argc > 1)
> +  {
> +    parameter_file = fopen(argv[1], "r");
> +    if(parameter_file)
> +    {
> +      ParseFile(parameter_file, parameter_printer);
> +      fclose(parameter_file);
> +      retval = 0;
> +    }
> +    else
> +    {
> +      retval=2;
> +    }
> +  }
> +  else
> +  {
> +    printf("Usage: %s <filename>\n", argv[0]);
> +    retval = 1;
> +  };
> +
> +  return 0;
> +}
> +
> +#endif
> +
> +
> + /*@@
> +   @routine ReadFile
> +   @author Frank Loeffler
> +   @desc
> +   This routine reads a file into a buffer
> +   @enddesc
> +   @history
> +   @var     file
> +   @vdesc   The filestream to read
> +   @vtype   FILE *
> +   @vio     in
> +   @vcomment
> +
> +   @endvar
> +   @var     filesize
> +   @vdesc   The size of the file
> +   @vtype   *unsigned long
> +   @vio     out
> +   @vcomment
> +
> +   @endvar
> +
> +   @returntype char *
> +   @returndesc
> +   NULL - failure
> +   !NULL allocated buffer
> +   @endreturndesc
> +@@*/
> +static char *ReadFile(FILE *file, unsigned long *filesize)
> +{
> +  char *buffer;
> +
> +  if (!file)
> +  {
> +    fprintf(stderr, "Could not use file for reading.\n");
> +    return NULL;
> +  }
> +  /* Get the file size */
> +  fseek(file, 0, SEEK_END);
> +  *filesize = ftell(file);
> +  fseek(file, 0, SEEK_SET);
> +  /* Allocate buffer */
> +  buffer = (char *)malloc(*filesize+1);
> +  if (!buffer)
> +  {
> +    fprintf(stderr, "Could not allocate memory.\n");
> +    return NULL;
> +  }
> +  /* Read file into buffer and return */
> +  fread(buffer, *filesize, 1, file);
> +  /* Protect buffer for string operations */
> +  buffer[*filesize] = '\0';
> +  return buffer;
> +}
> +
> + /*@@
> +   @routine ParseDefines
> +   @author Frank Loeffler
> +   @desc
> +   This routine parses a buffer for defines of the form \$[^ \n] and
> +   replaces them with some predefined defines, returning the new buffer,
> +   which might be the same as the old buffer. Otherwise the old buffer
> +   gets freed by this function. In case of parsing errors, the whole
> +   parsing is stopped and the old buffer is returned
> +   (FIXME: Should we return NULL?)
> +   @enddesc
> +   @history
> +   @var     buffer
> +   @vdesc   The buffer to parse
> +   @vtype   char *
> +   @vio     in
> +   @vcomment
> +
> +   @endvar
> +   @var     buffersize
> +   @vdesc   The size of the buffer
> +   @vtype   *unsigned long
> +   @vio     out
> +   @vcomment
> +
> +   @endvar
> +
> +   @returntype char *
> +   @returndesc
> +   !NULL - new buffer, might be == buffer
> +   @endreturndesc
> +@@*/
> +static char *ParseDefines(char *buffer, unsigned long *buffersize)
> +{
> +  /* define name */
> +  char define[1024];
> +  /* Position in define name */
> +  char defpos = 0;
> +  /* Current position in buffer */
> +  size_t pos = 0;
> +  /* Character at current position */
> +  char c;
> +  /* Position of start of found define */
> +  size_t def_start = 0;
> +  /* Flag to indicate if we are inside a definition name */
> +  int indef = 0;
> +  if (!buffer)
> +    return buffer;
> +  /* Main loop over the buffer */
> +  while (((c=buffer[pos]) != '\0') && (pos < *buffersize) )
> +  {
> +    /* Check if we found a define */
> +    if (c == '$')
> +    {
> +      /* Mark the state and location and go on */
> +      indef = 1;
> +      def_start = pos;
> +    }
> +    /* If we are inside a define */
> +    else if (indef)
> +    {
> +      /* Look for the end of the define name */
> +      if (c == ' ' || c == '/' || c == '\0' || c == '\n')
> +      {
> +        /* make the define name a proper string */
> +        define[defpos] = '\0';
> +        /* this holds the value of the define to be filled in */
> +        char *value = NULL;
> +        /* a new buffer for the whole thing */
> +        char * new_buffer;
> +        unsigned long new_size;
> +        /* Check for different kinds of defines */
> +        /* Environment variables */
> +        if (strncmp(define, "ENV_", 4) == 0)
> +        {
> +          value = getenv(define+4);
> +          if (!value)
> +          {
> +            CCTK_VWarn(CCTK_WARN_ALERT, __LINE__, __FILE__, CCTK_THORNSTRING,
> +                       "No environment variable %s found\n", define+4);
> +            /* TODO: Should we abort here? (return NULL) */
> +          }
> +        }
> +        /* Parameter file name */
> +        else if (strcmp(define, "parfile") == 0)
> +        {
> +          char path[500];
> +          CCTK_ParameterFilename(500, path);
> +          value = strrchr (path, '/');
> +          if (value == NULL)
> +          {
> +            value = path;
> +          }
> +          else
> +          {
> +            value++;
> +          }
> +          /* skip the parameter file extension */
> +          if (strcmp (value + strlen (value) - 4, ".par") == 0)
> +          {
> +            value[strlen (value) - 4] = '\0';
> +          }
> +        }
> +        /* Else: unknown define - or no define at all: ignore */
> +        if (value)
> +        {
> +          /* Replace the old buffer with the new, copying in all the data */
> +          new_size = *buffersize - strlen(define) + strlen(value) + 1;
> +          if (new_size < def_start)
> +          {
> +            CCTK_WARN(0, "Something is wrong with me, HELP!");
> +            return buffer;
> +          }
> +          new_buffer = (char *)malloc(new_size);
> +          if (!new_buffer)
> +          {
> +            CCTK_WARN(0, "I am out of memory and give up parsing for defines.");
> +            return buffer;
> +          }
> +          new_buffer[0]='\0';
> +          strncat(new_buffer, buffer, def_start);
> +          strncat(new_buffer, value, new_size-1);
> +          strncat(new_buffer, buffer+pos, new_size-1);
> +          new_buffer[new_size-1] = '\0';
> +          /* free the old buffer */
> +          free(buffer);
> +          /* Start further processing of new buffer */
> +          pos = def_start + strlen(define);
> +          buffer = new_buffer;
> +          *buffersize = new_size;
> +        }
> +        /* General define cleanup */
> +        indef = 0;
> +        defpos = 0;
> +        define[0] = '\0';
> +        def_start = 0;
> +      }
> +      else if (defpos > 1023)
> +      {
> +        /* We only print this warning and ignore the possible define problem.
> +         * It might not be a define after all but a valid, long parameter
> +         * value containing a '$'.*/
> +        define[1023] = '\0';
> +        CCTK_VWarn(CCTK_WARN_PICKY, __LINE__, __FILE__, CCTK_THORNSTRING,
> +                   "Possible define too long: %s", define);
> +        indef = 0;
> +        defpos = 0;
> +        define[0] = '\0';
> +        def_start = 0;
> +      }
> +      else
> +        define[defpos++] = c;
> +    }
> +    pos++;
> +  }
> +  return buffer;
> +}
> +
> + /*@@
> +   @routine ParseBuffer
> +   @author Paul Walker
> +   @desc
> +   This routine actually parses the parameter file. The
> +   syntax we allow is
> +   <ul>
> +     <li>a = b
> +         <li>a,b,c = d,e,f
> +         <li># rest of the line is ignored
> +         <li>x = "string value"
> +   </ul>
> +   So it is easy to parse
> +   <p>
> +   We go through the file looking for stuff and then set
> +   it in the global database using calls to the passed in set_function.
> +   @enddesc
> +   @history
> +   @hdate Tue Jan 12 16:41:36 1999 @hauthor Tom Goodale
> +   @hdesc Moved to CCTK.
> +          Changed to pass data to arbitrary function.
> +          Changed to take a file descriptor rather than a filename.
> +          Use a buffer to parse which can be preprocessed before main parsing
> +   @endhistory
> +   @var     buffer
> +   @vdesc   The buffer to parse
> +   @vtype   char *
> +   @vio     in
> +   @vcomment
> +
> +   @endvar
> +   @var     set_function
> +   @vdesc   The function to call to set the value of a parameter
> +   @vtype   int (*)(const char *, const char *)
> +   @vio     in
> +   @vcomment
> +
> +   @endvar
> +   @var     ConfigData
> +   @vdesc   Flesh configuration data
> +   @vtype   tFleshConfig *
> +   @vio     in
> +   @vcomment
> +
> +   @endvar
> +
> +   @returntype int
> +   @returndesc
> +   0 - success
> +   @endreturndesc
> +@@*/
> +int ParseBuffer(char *buffer,
> +              int (*set_function)(const char *, const char *, int),
> +              tFleshConfig *ConfigData)
> +{
> +  /* position in buffer */
> +  unsigned int pos = 0;
>    /* Buffers for parsing from the file */
>    char *tokens, *value;
>    char *subtoken, *subvalue;
> @@ -122,10 +548,8 @@
>    int ntokens;
>    /* Status flags */
>    int intoken, inval;
> -  /* Current char. Have to make it an int so we can compare with
> -     EOF. See man 3 fgetc
> -   */
> -  int c;
> +  /* Current char */
> +  char c;
>    int num_errors; /* number of errors in file parsing */
>  
>    num_errors = 0;
> @@ -141,7 +565,7 @@
>  
>    intoken = 0; inval = 0;
>  
> -  while ((c=fgetc(ifp)) != EOF)
> +  while ((c=buffer[pos++]) != '\0')
>    {
>  #ifdef DEBUG
>      printf("%c",c);
> @@ -150,7 +574,7 @@
>      while (c == '#' || c == '!' )
>      {
>        /* Comment line.  So forget rest of line */
> -      while ((c=fgetc(ifp)) != '\n' && c != '\r' && c != EOF)
> +      while ((c=buffer[pos++]) != '\n' && c != '\r' && c != '\0')
>        {
>  #ifdef DEBUG
>          printf("%c",c);
> @@ -160,7 +584,7 @@
>        {
>          lineno++;
>        }
> -      c = fgetc(ifp);
> +      c = buffer[pos++];
>  #ifdef DEBUG
>        printf("%c",c);
>  #endif
> @@ -220,7 +644,7 @@
>           * and check if the value is a string or not.
>           * This parser DOES strip quotes off of the strings.
>           */
> -        while ((c = fgetc(ifp)) == ' ' || c == '\n' || c == '\r' || c == '\t')
> +        while ((c = buffer[pos++]) == ' ' || c == '\n' || c == '\r' || c == '\t')
>          {
>  #ifdef DEBUG
>            printf("%c",c);
> @@ -251,7 +675,7 @@
>              free (tokens);
>              return 1;
>            }
> -          while ((c = fgetc(ifp)) != '"')
> +          while ((c = buffer[pos++]) != '"')
>            {
>  #ifdef DEBUG
>              printf("%c",c);
> @@ -265,7 +689,7 @@
>  #endif
>                lineno++;
>              }
> -            else if (c == EOF)
> +            else if (c == '\0')
>              {
>                break;
>              }
> @@ -274,9 +698,9 @@
>            if (c == '"')
>            {
>              int is_comment = 0;
> -            while ((c = fgetc(ifp)) != '\n')
> +            while ((c = buffer[pos++]) != '\n')
>              {
> -              if (c == EOF)
> +              if (c == '\0')
>                {
>                  break;
>                }
> @@ -295,7 +719,7 @@
>                  return 1;
>                }
>              }
> -            if (c == '\n' || c == EOF)
> +            if (c == '\n' || c == '\0')
>              {
>                if (c == '\n') lineno++;
>                /* mark it as properly quoted string value */
> @@ -319,39 +743,6 @@
>  #endif
>            set_function(tokens,value, lineno);
>          }
> -        else if (c == '$')
> -        {
> -          /* We got a define */
> -          /* FIXME: Assume it is a parameter file for now */
> -          char path[500];
> -          char *parfile;
> -
> -          CCTK_ParameterFilename(500,path);
> -          parfile = strrchr (path, '/');
> -          if (parfile == NULL)
> -          {
> -            parfile = path;
> -          }
> -          else
> -          {
> -            parfile++;
> -          }
> -          /* skip the parameter file extension */
> -          if (strcmp (parfile + strlen (parfile) - 4, ".par") == 0)
> -          {
> -            parfile[strlen (parfile) - 4] = 0;
> -          }
> -
> -          /* ignore everything else on the line */
> -          while (!(c==' ' || c=='\t' || c == '\n' || c == '\r' || c == EOF))
> -          {
> -            c = fgetc(ifp);
> -#ifdef DEBUG
> -            printf("%c",c);
> -#endif
> -          }
> -          set_function(tokens,parfile,lineno);
> -        }
>          else
>          {
>  
> @@ -362,15 +753,15 @@
>              /* Simple case. We have an int
>                 or a double which contain no
>                 spaces! */
> -            c = fgetc(ifp);
> +            c = buffer[pos++];
>  #ifdef DEBUG
>              printf("%c",c);
>  #endif
> -            while (!(c==' ' || c=='\t' || c == '\n' || c == '\r' || c == EOF))
> +            while (!(c==' ' || c=='\t' || c == '\n' || c == '\r' || c == '\0'))
>              {
>                value[p++] = (char)c;
>                CheckBuf(p,lineno);
> -              c = fgetc(ifp);
> +              c = buffer[pos++];
>  #ifdef DEBUG
>                printf("%c",c);
>  #endif
> @@ -402,11 +793,11 @@
>                 ntokens-1 commas, stripping spaces, and
>                 make a nice little string.
>                 */
> -            c = fgetc(ifp);
> +            c = buffer[pos++];
>  #ifdef DEBUG
>              printf("%c",c);
>  #endif
> -            while (ncommas < ntokens-1 && c != EOF)
> +            while (ncommas < ntokens-1 && c != '\0')
>              {
>                if (!(c == ' ' || c == '\t' || c == '\n' || c == '\r'))
>                {
> @@ -414,7 +805,7 @@
>                  CheckBuf(pp,lineno);
>                }
>                if (c == ',') ncommas ++;
> -              c = fgetc(ifp);
> +              c = buffer[pos++];
>  #ifdef DEBUG
>                printf("%c",c);
>  #endif
> @@ -422,7 +813,7 @@
>              if (c == ' ' || c == '\t')
>              {
>                /* Great now strip out the spaces */
> -              while((c = fgetc(ifp)) == ' ' || c=='\t' || c == '\n' || c == '\r')
> +              while((c = buffer[pos++]) == ' ' || c=='\t' || c == '\n' || c == '\r')
>                {
>  #ifdef DEBUG
>                  printf("%c",c);
> @@ -441,15 +832,15 @@
>              value[pp++] = (char)c;
>              CheckBuf(p,lineno);
>  
> -            c = fgetc(ifp);
> +            c = buffer[pos++];
>  #ifdef DEBUG
>              printf("%c",c);
>  #endif
> -            while (c != ' ' && c != '\t' && c != '\n' && c != '\r' && c != EOF)
> +            while (c != ' ' && c != '\t' && c != '\n' && c != '\r' && c != '\0')
>              {
>                value[pp++] = (char)c;
>                CheckBuf(pp,lineno);
> -              c = fgetc(ifp);
> +              c = buffer[pos++];
>  #ifdef DEBUG
>                printf("%c",c);
>  #endif
> @@ -525,137 +916,3 @@
>    return num_errors;
>  }
>  
> -/********************************************************************
> - *********************     Local Routines   *************************
> - ********************************************************************/
> -
> - /*@@
> -   @routine CheckBuf
> -   @author Paul Walker
> -   @desc
> -   A simple description and warning message in case of
> -   a fixed parse buffer overflow.
> -   @enddesc
> -   @calls
> -   @calledby
> -   @history
> -
> -   @endhistory
> -   @var     p
> -   @vdesc   buffer location
> -   @vtype   int
> -   @vio     in
> -   @vcomment
> -
> -   @endvar
> -   @var     l
> -   @vdesc   Line number
> -   @vtype   int
> -   @vio     in
> -   @vcomment
> -
> -   @endvar
> -
> -@@*/
> -
> -static void CheckBuf(int p, int l)
> -{
> -  if (p >= BUF_SZ)
> -  {
> -    fprintf(stderr,"WARNING: Parser buffer overflow on line %d\n",
> -            l);
> -    fprintf(stderr,"This indicates either an incorrect parm file or\n");
> -    fprintf(stderr,"the need to recompile " __FILE__ " with a bigger\n");
> -    fprintf(stderr,"BUF_SZ parm.\n");
> -
> -    assert(0);
> -    exit(1);
> -  }
> -}
> -
> -
> - /*@@
> -   @routine removeSpaces
> -   @author Paul Walker
> -   @desc
> -   removes the spaces from a char * <b>in place</b>. Beware
> -   that this function will change the input value and if you
> -   want to keep a copy you have to do so yourself!
> -   @enddesc
> -   @calls
> -   @calledby
> -   @history
> -
> -   @endvar
> -   @var     stripMe
> -   @vdesc   String to strip
> -   @vtype   char *
> -   @vio     inout
> -   @vcomment
> -
> -   @endvar
> -
> -@@*/
> -static void removeSpaces(char *stripMe)
> -{
> -  char *s;
> -  unsigned int i,j;
> -  s = (char *)malloc((strlen(stripMe)+2)*sizeof(char));
> -
> -  if(s)
> -  {
> -    strcpy(s,stripMe);
> -    for (i=0,j=0;i<strlen(s);i++)
> -    {
> -      if (s[i] != ' ' && s[i] != '\t' && s[i] != '\n' && s[i] != '\r')
> -      {
> -        stripMe[j++] = s[i];
> -      }
> -      stripMe[j] = '\0';
> -    }
> -  }
> -
> -  free(s);
> -}
> -
> -
> -/* #define TEST_ParseFile */
> -
> -#ifdef TEST_ParseFile
> -
> -int parameter_printer(const char *param, const char *val)
> -{
> -  printf("Parameter %s has value %s\n", param, val);
> -
> -  return 0;
> -}
> -
> -int main(int argc, char *argv[])
> -{
> -  int retval;
> -  FILE *parameter_file;
> -
> -  if(argc > 1)
> -  {
> -    parameter_file = fopen(argv[1], "r");
> -    if(parameter_file)
> -    {
> -      ParseFile(parameter_file, parameter_printer);
> -      fclose(parameter_file);
> -      retval = 0;
> -    }
> -    else
> -    {
> -      retval=2;
> -    }
> -  }
> -  else
> -  {
> -    printf("Usage: %s <filename>\n", argv[0]);
> -    retval = 1;
> -  };
> -
> -  return 0;
> -}
> -
> -#endif
> 
> 
> --- StripMime Report -- processed MIME parts ---
> multipart/mixed
>   text/plain (text body -- kept)
>   text/plain (text body -- kept)
> ---
> _______________________________________________
> Developers mailing list
> Developers at cactuscode.org
> http://www.cactuscode.org/mailman/listinfo/developers
> 

From knarf at cct.lsu.edu  Thu Sep 10 20:50:43 2009
From: knarf at cct.lsu.edu (Frank Loeffler)
Date: Thu, 10 Sep 2009 20:50:43 -0500
Subject: [Developers] [Patches] improve parser for parameter files
In-Reply-To: <8B2F1374-1B31-45BF-82E5-B3F27970212E@cct.lsu.edu>
References: <20090910195426.GA11625@numrel07.cct.lsu.edu>
	<8B2F1374-1B31-45BF-82E5-B3F27970212E@cct.lsu.edu>
Message-ID: <20090911015042.GC11625@numrel07.cct.lsu.edu>

Hi,

On Fri, Sep 11, 2009 at 06:53:44AM +0800, Erik Schnetter wrote:
> The code that processes the buffer is not very efficient.  It
> probably doesn't matter in practice, but it would be more efficient
> to allocate a much larger buffer in the beginning instead of
> extending it piecewise.  Extending it piecewise is an O(N^2)
> algorithm, doubling the buffer size each time it is too small has a
> complexity of O(N).

That N would be the number of replacements made. I assume that N would
always be quite small, but in principle you are right.

> The syntax $ENV_* is somewhat arbitrary.  I would prefer a more
> standard syntax, as found e.g. in bash or in perl.  Bash syntax
> would be $var or ${var}, perl syntax would be $ENV{'var'}.

Of those two I like the perl syntax more because it leaves the
possibility of other defines than environment variables open, without
limiting them later. We should also pick something which does not
collide with other valid uses of '$' so often, e.g. in parameters
containing regular expressions and the like.

> If we
> additionally allow ${var}, people could also use $parfile to
> construct file name, especially if we also introduce $pardir, and
> modify the exact definition of $parfile ? the current parsing code
> for $parfile is very naive.

I am not sure if I understand you here. $parfile is currently replaced
by the name of the parameter file. The path to it and the extension are
chopped. E.g. /work/test.par would restult as $parfile->test. This
can already be used in a parameter file to e.g. point to
/scratch/$parfile.

I copied that code from the old version to leave the meaning of $parfile
as it currently is. The whole parsing code does look a bit like a hack
but it is a bit difficult to do that better with only plain C.

As for the syntax of the environment variables ($ENV_* at the moment), we
could certainly change that to either $ENV{*} or $ENV{'*'}. Pick your
favorite.

Frank


From schnetter at cct.lsu.edu  Thu Sep 10 21:15:35 2009
From: schnetter at cct.lsu.edu (Erik Schnetter)
Date: Fri, 11 Sep 2009 10:15:35 +0800
Subject: [Developers] improve parser for parameter files
In-Reply-To: <4AA9AA92.4040601@yukawa.kyoto-u.ac.jp>
References: <20090910195426.GA11625@numrel07.cct.lsu.edu>
	<4AA9AA92.4040601@yukawa.kyoto-u.ac.jp>
Message-ID: <BC4AA0E5-3479-4C3A-BA0F-7441BC8D08D3@cct.lsu.edu>

On Sep 11, 2009, at 9:40 , Baiotti Luca wrote:

> Hello,
>
> I would have a feature request about the parameter-file checks (so  
> it is not
> strictly related to the patch, but I would like to ask anyway).
>
> When an error is detected in the parfile at runtime, each processor  
> produces a
> warning. Since there may be hundreds of processors and since each  
> processor parses
> and checks the same parfile, I find it useless and confusing that  
> the error
> message is repeated one time per processor. I would like the error  
> message
> originated from errors in the parfile to be output only once (from  
> the headnode?).


Luca

What Cactus typically tries to do is to separate information on stdout  
and stderr.  By default, stdout contains messages and warnings from  
the root process only, whereas stderr contains only the warnings but  
from all processes.  In principle, looking at the last few lines of  
stdout should give you what you ask for.

However, for some strange reason, errors about parameters are only  
output to stderr, forcing you to look at stderr.  Only the final error  
message (telling you that the run is about to abort) is output to  
stdout.  This should clearly be corrected.

-erik

-- 
Erik Schnetter <schnetter at cct.lsu.edu>   http://www.cct.lsu.edu/~eschnett/

My email is as private as my paper mail.  I therefore support encrypting
and signing email messages.  Get my PGP key from <http://pgp.mit.edu>.



-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 203 bytes
Desc: This is a digitally signed message part
Url : http://www.cactuscode.org/pipermail/developers/attachments/20090911/f9800671/attachment.bin 

From schnetter at cct.lsu.edu  Thu Sep 10 21:21:55 2009
From: schnetter at cct.lsu.edu (Erik Schnetter)
Date: Fri, 11 Sep 2009 10:21:55 +0800
Subject: [Developers] Output parameter warnings to stdout as well
Message-ID: <842E38A7-FC1E-4A10-B559-0A768FC8B577@cct.lsu.edu>

The enclosed patch outputs parameter warnings and parameter errors to  
stdout as well.  Different from regular warnings, they are currently  
only output to stderr.  This is inconvenient, since stderr collects  
warnings from all processors, and thus contains many repetitions of  
the same error messages.

The patch also cleans up the code a bit.  Instead of implementing  
parameter warnings twice in CCTK_ParamWarn and CCTK_VParamWarn, the  
function CCTK_ParamWarn now calls CCTK_VParamWarn.  This removes the  
need for the function print_bold_stderr.

Furthermore it removes an unnecessary call to va_copy, and adds  
missing calls to va_end (avoiding potential memory leaks).

-erik

-- 
Erik Schnetter <schnetter at cct.lsu.edu>   http://www.cct.lsu.edu/~eschnett/

My email is as private as my paper mail.  I therefore support encrypting
and signing email messages.  Get my PGP key from <http://pgp.mit.edu>.




--- StripMime Report -- processed MIME parts ---
multipart/mixed
  text/plain (text body -- kept)
  application/octet-stream
---

From bgiacoma at aei.mpg.de  Fri Sep 11 04:12:17 2009
From: bgiacoma at aei.mpg.de (Bruno Giacomazzo)
Date: Fri, 11 Sep 2009 11:12:17 +0200 (CEST)
Subject: [Developers] Output parameter warnings to stdout as well
In-Reply-To: <842E38A7-FC1E-4A10-B559-0A768FC8B577@cct.lsu.edu>
References: <842E38A7-FC1E-4A10-B559-0A768FC8B577@cct.lsu.edu>
Message-ID: <Pine.LNX.4.64.0909111108170.10750@duo01.aei.mpg.de>

Erik,
 	both me and Luca save stdout and stderr in the same file. I do 
this so that I'm sure of when (i.e. at which iteration/point of the run) a 
warning message is generated since the WARNING messages don't contain any 
information about the iteration at which they were generated. In this case 
it seems to me that your patch would not solve the problem (at least not 
for me and Luca). Is it correct?

Thanks,
Bruno

On Fri, 11 Sep 2009, Erik Schnetter wrote:

> The enclosed patch outputs parameter warnings and parameter errors to
> stdout as well.  Different from regular warnings, they are currently
> only output to stderr.  This is inconvenient, since stderr collects
> warnings from all processors, and thus contains many repetitions of
> the same error messages.
>
> The patch also cleans up the code a bit.  Instead of implementing
> parameter warnings twice in CCTK_ParamWarn and CCTK_VParamWarn, the
> function CCTK_ParamWarn now calls CCTK_VParamWarn.  This removes the
> need for the function print_bold_stderr.
>
> Furthermore it removes an unnecessary call to va_copy, and adds
> missing calls to va_end (avoiding potential memory leaks).
>
> -erik
>
>

-- 
Dr. Bruno Giacomazzo
Max Planck Institute for Gravitational Physics
Albert Einstein Institute
Am Muehlenberg 1
D-14476 Potsdam
Germany

Tel.  : +49 331 567 7183
Fax   : +49 331 567 7298
cell. : +49 173 826 4488
email : bgiacoma at aei.mpg.de

-------------------------------------------------
There are only 10 types of people in the world:
Those who understand binary, and those who don't
-------------------------------------------------

From bgiacoma at aei.mpg.de  Fri Sep 11 04:12:17 2009
From: bgiacoma at aei.mpg.de (Bruno Giacomazzo)
Date: Fri, 11 Sep 2009 11:12:17 +0200 (CEST)
Subject: [Developers] Output parameter warnings to stdout as well
In-Reply-To: <842E38A7-FC1E-4A10-B559-0A768FC8B577@cct.lsu.edu>
References: <842E38A7-FC1E-4A10-B559-0A768FC8B577@cct.lsu.edu>
Message-ID: <Pine.LNX.4.64.0909111108170.10750@duo01.aei.mpg.de>

Erik,
 	both me and Luca save stdout and stderr in the same file. I do 
this so that I'm sure of when (i.e. at which iteration/point of the run) a 
warning message is generated since the WARNING messages don't contain any 
information about the iteration at which they were generated. In this case 
it seems to me that your patch would not solve the problem (at least not 
for me and Luca). Is it correct?

Thanks,
Bruno

On Fri, 11 Sep 2009, Erik Schnetter wrote:

> The enclosed patch outputs parameter warnings and parameter errors to
> stdout as well.  Different from regular warnings, they are currently
> only output to stderr.  This is inconvenient, since stderr collects
> warnings from all processors, and thus contains many repetitions of
> the same error messages.
>
> The patch also cleans up the code a bit.  Instead of implementing
> parameter warnings twice in CCTK_ParamWarn and CCTK_VParamWarn, the
> function CCTK_ParamWarn now calls CCTK_VParamWarn.  This removes the
> need for the function print_bold_stderr.
>
> Furthermore it removes an unnecessary call to va_copy, and adds
> missing calls to va_end (avoiding potential memory leaks).
>
> -erik
>
>

-- 
Dr. Bruno Giacomazzo
Max Planck Institute for Gravitational Physics
Albert Einstein Institute
Am Muehlenberg 1
D-14476 Potsdam
Germany

Tel.  : +49 331 567 7183
Fax   : +49 331 567 7298
cell. : +49 173 826 4488
email : bgiacoma at aei.mpg.de

-------------------------------------------------
There are only 10 types of people in the world:
Those who understand binary, and those who don't
-------------------------------------------------

From knarf at cct.lsu.edu  Fri Sep 11 08:46:35 2009
From: knarf at cct.lsu.edu (Frank Loeffler)
Date: Fri, 11 Sep 2009 08:46:35 -0500
Subject: [Developers] Output parameter warnings to stdout as well
In-Reply-To: <842E38A7-FC1E-4A10-B559-0A768FC8B577@cct.lsu.edu>
References: <842E38A7-FC1E-4A10-B559-0A768FC8B577@cct.lsu.edu>
Message-ID: <20090911134634.GH11625@numrel07.cct.lsu.edu>

Hi,

On Fri, Sep 11, 2009 at 10:21:55AM +0800, Erik Schnetter wrote:
> The enclosed patch outputs parameter warnings and parameter errors to  
> stdout as well.  Different from regular warnings, they are currently  
> only output to stderr.  This is inconvenient, since stderr collects  
> warnings from all processors, and thus contains many repetitions of  
> the same error messages.

This creates another inconvenience: when simply run interactivly, there
is now one more warning output, in addition to all the others from
stderr. Even when running on with one process a warning is now printed
twice. I don't think this is optimal.

I wonder if it wouldn't be better to only emit warnings for parameter
file settings once, on the root processor, and only print them to
stderr. Each process parses the same parameter file and should come to
the same warnings anyway.

The issue with multiple messages from the parameter file parsing
wouldn't be solved by that directly however, because this code calls
fprintf on stderr directly instead of CCTK_ParamWarn. This could and
should be changed, but I would prefer to postpone this until the other
patch for that file has been discussed and commited, hopefully only a
couple of days.

> The patch also cleans up the code a bit.

That is very much appreciated.

Frank


From knarf at cct.lsu.edu  Fri Sep 11 08:46:35 2009
From: knarf at cct.lsu.edu (Frank Loeffler)
Date: Fri, 11 Sep 2009 08:46:35 -0500
Subject: [Developers] Output parameter warnings to stdout as well
In-Reply-To: <842E38A7-FC1E-4A10-B559-0A768FC8B577@cct.lsu.edu>
References: <842E38A7-FC1E-4A10-B559-0A768FC8B577@cct.lsu.edu>
Message-ID: <20090911134634.GH11625@numrel07.cct.lsu.edu>

Hi,

On Fri, Sep 11, 2009 at 10:21:55AM +0800, Erik Schnetter wrote:
> The enclosed patch outputs parameter warnings and parameter errors to  
> stdout as well.  Different from regular warnings, they are currently  
> only output to stderr.  This is inconvenient, since stderr collects  
> warnings from all processors, and thus contains many repetitions of  
> the same error messages.

This creates another inconvenience: when simply run interactivly, there
is now one more warning output, in addition to all the others from
stderr. Even when running on with one process a warning is now printed
twice. I don't think this is optimal.

I wonder if it wouldn't be better to only emit warnings for parameter
file settings once, on the root processor, and only print them to
stderr. Each process parses the same parameter file and should come to
the same warnings anyway.

The issue with multiple messages from the parameter file parsing
wouldn't be solved by that directly however, because this code calls
fprintf on stderr directly instead of CCTK_ParamWarn. This could and
should be changed, but I would prefer to postpone this until the other
patch for that file has been discussed and commited, hopefully only a
couple of days.

> The patch also cleans up the code a bit.

That is very much appreciated.

Frank


From ian.hinder at aei.mpg.de  Fri Sep 11 11:00:04 2009
From: ian.hinder at aei.mpg.de (Ian Hinder)
Date: Fri, 11 Sep 2009 18:00:04 +0200
Subject: [Developers] [Patches] Output parameter warnings to stdout as
	well
In-Reply-To: <20090911134634.GH11625@numrel07.cct.lsu.edu>
References: <842E38A7-FC1E-4A10-B559-0A768FC8B577@cct.lsu.edu>
	<20090911134634.GH11625@numrel07.cct.lsu.edu>
Message-ID: <2A4819D9-FB76-49DF-A50D-38188252638B@aei.mpg.de>


On 11 Sep 2009, at 15:46, Frank Loeffler wrote:

> Hi,
>
> On Fri, Sep 11, 2009 at 10:21:55AM +0800, Erik Schnetter wrote:
>> The enclosed patch outputs parameter warnings and parameter errors to
>> stdout as well.  Different from regular warnings, they are currently
>> only output to stderr.  This is inconvenient, since stderr collects
>> warnings from all processors, and thus contains many repetitions of
>> the same error messages.
>
> This creates another inconvenience: when simply run interactivly,  
> there
> is now one more warning output, in addition to all the others from
> stderr. Even when running on with one process a warning is now printed
> twice. I don't think this is optimal.
>
> I wonder if it wouldn't be better to only emit warnings for parameter
> file settings once, on the root processor, and only print them to
> stderr. Each process parses the same parameter file and should come to
> the same warnings anyway.

Why are we printing errors and warnings to standard error anyway?  I  
think it causes more trouble than it is worth.  Can you think of a  
good reason why we don't output everything to standard output?

The reason for having two separate streams is related to Unix programs  
whose output is machine-readable data which may be passed to another  
program through a pipe.  In that situation, it would be bad for that  
data to be corrupted with error or warning messages.  Since Cactus  
output is freeform, this does not apply here.

On supercomputers, stdout and stderr are usually sent to two separate  
files.  When checking to see the status of a simulation, first you  
have to check stdout, then go and check stderr to find the actual  
reason for the failure.  If everything was in stdout this would not be  
necessary.  The split also makes it harder to get a complete picture  
if you ware watching the output of a simulation in realtime (tail -f  
outfile) as the errors and warnings don't appear in the output file.

> The issue with multiple messages from the parameter file parsing
> wouldn't be solved by that directly however, because this code calls
> fprintf on stderr directly instead of CCTK_ParamWarn. This could and
> should be changed, but I would prefer to postpone this until the other
> patch for that file has been discussed and commited, hopefully only a
> couple of days.
>
>> The patch also cleans up the code a bit.
>
> That is very much appreciated.
>
> Frank
>
> _______________________________________________
> Patches mailing list
> Patches at cactuscode.org
> http://www.cactuscode.org/mailman/listinfo/patches

-- 
Ian Hinder
ian.hinder at aei.mpg.de


From schnetter at cct.lsu.edu  Tue Sep 15 16:40:59 2009
From: schnetter at cct.lsu.edu (Erik Schnetter)
Date: Tue, 15 Sep 2009 17:40:59 -0400
Subject: [Developers] [Patches] Output parameter warnings to stdout as
	well
In-Reply-To: <2A4819D9-FB76-49DF-A50D-38188252638B@aei.mpg.de>
References: <842E38A7-FC1E-4A10-B559-0A768FC8B577@cct.lsu.edu>
	<20090911134634.GH11625@numrel07.cct.lsu.edu>
	<2A4819D9-FB76-49DF-A50D-38188252638B@aei.mpg.de>
Message-ID: <F02155ED-E199-4A34-AED8-77FC06293048@cct.lsu.edu>

On Sep 11, 2009, at 12:00 , Ian Hinder wrote:

> On 11 Sep 2009, at 15:46, Frank Loeffler wrote:
>
>> Hi,
>>
>> On Fri, Sep 11, 2009 at 10:21:55AM +0800, Erik Schnetter wrote:
>>> The enclosed patch outputs parameter warnings and parameter errors  
>>> to
>>> stdout as well.  Different from regular warnings, they are currently
>>> only output to stderr.  This is inconvenient, since stderr collects
>>> warnings from all processors, and thus contains many repetitions of
>>> the same error messages.
>>
>> This creates another inconvenience: when simply run interactivly,
>> there
>> is now one more warning output, in addition to all the others from
>> stderr. Even when running on with one process a warning is now  
>> printed
>> twice. I don't think this is optimal.
>>
>> I wonder if it wouldn't be better to only emit warnings for parameter
>> file settings once, on the root processor, and only print them to
>> stderr. Each process parses the same parameter file and should come  
>> to
>> the same warnings anyway.
>
> Why are we printing errors and warnings to standard error anyway?  I
> think it causes more trouble than it is worth.  Can you think of a
> good reason why we don't output everything to standard output?

The original idea was that everything should go to stdout, while  
important warnings and errors should also go to stderr.  stdout is  
often quite noise, and if something goes seriously wrong, then a  
glance at stderr should give a first idea of where the problem lies.   
Additionally, since stdout captures only output from the root process,  
it is important to see the important warnings and errors from the  
other processes.  We can certainly play with what should be output  
into what file, with the caveat that outputting everything to files is  
too expensive when running at >1000 processes.

> The reason for having two separate streams is related to Unix programs
> whose output is machine-readable data which may be passed to another
> program through a pipe.  In that situation, it would be bad for that
> data to be corrupted with error or warning messages.  Since Cactus
> output is freeform, this does not apply here.
>
> On supercomputers, stdout and stderr are usually sent to two separate
> files.  When checking to see the status of a simulation, first you
> have to check stdout, then go and check stderr to find the actual
> reason for the failure.  If everything was in stdout this would not be
> necessary.  The split also makes it harder to get a complete picture
> if you ware watching the output of a simulation in realtime (tail -f
> outfile) as the errors and warnings don't appear in the output file.

I agree that everything should go to stdout.  I believe that the  
parameter warnings are the only case where something is only output to  
stderr.

-erik

-- 
Erik Schnetter <schnetter at cct.lsu.edu>   http://www.cct.lsu.edu/~eschnett/

My email is as private as my paper mail.  I therefore support encrypting
and signing email messages.  Get my PGP key from <http://pgp.mit.edu>.



-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 203 bytes
Desc: This is a digitally signed message part
Url : http://www.cactuscode.org/pipermail/developers/attachments/20090915/5ca74703/attachment.bin 

From schnetter at cct.lsu.edu  Tue Sep 15 17:49:19 2009
From: schnetter at cct.lsu.edu (Erik Schnetter)
Date: Tue, 15 Sep 2009 18:49:19 -0400
Subject: [Developers] Output parameter warnings to stdout as well
In-Reply-To: <Pine.LNX.4.64.0909111108170.10750@duo01.aei.mpg.de>
References: <842E38A7-FC1E-4A10-B559-0A768FC8B577@cct.lsu.edu>
	<Pine.LNX.4.64.0909111108170.10750@duo01.aei.mpg.de>
Message-ID: <9F16CF0D-C255-4147-AF9C-A4D2B1B819ED@cct.lsu.edu>

Yes, that is correct.  However, you would not need to save stdout and  
stderr to the same file any more, since stdout alone would already  
contain all errors from the root processor.

However, if you expect errors that are output only from one of the  
processors, and you want to have these in the context of stdout of the  
root processor, then you are still out of luck.

You are essentially looking for a way to see all output from all  
processors, but have annoying duplicates removed.  This is currently  
not possible in Cactus.  I wrote some code for this many years ago  
that worked surprisingly well.  All output was sent to the root  
process via MPI, and the root process then combined messages as  
necessary to indicate which message came from what process.

The only danger with this approach is that a fatal error may terminate  
the application and thus lose some of the messages.  This, however, is  
also currently a problem.  Let me see whether I can revive that code.   
I don't know how efficient it is ? but sending messages via MPI can  
hardly be less efficient than writing them to a file.  If this works,  
then stdout would contain the "nicely combined" output from all  
processes, while stderr would still show the duplicated output for  
debugging.

-erik

On Sep 11, 2009, at 5:12 , Bruno Giacomazzo wrote:

> Erik,
> 	both me and Luca save stdout and stderr in the same file. I do
> this so that I'm sure of when (i.e. at which iteration/point of the  
> run) a
> warning message is generated since the WARNING messages don't  
> contain any
> information about the iteration at which they were generated. In  
> this case
> it seems to me that your patch would not solve the problem (at least  
> not
> for me and Luca). Is it correct?
>
> Thanks,
> Bruno
>
> On Fri, 11 Sep 2009, Erik Schnetter wrote:
>
>> The enclosed patch outputs parameter warnings and parameter errors to
>> stdout as well.  Different from regular warnings, they are currently
>> only output to stderr.  This is inconvenient, since stderr collects
>> warnings from all processors, and thus contains many repetitions of
>> the same error messages.
>>
>> The patch also cleans up the code a bit.  Instead of implementing
>> parameter warnings twice in CCTK_ParamWarn and CCTK_VParamWarn, the
>> function CCTK_ParamWarn now calls CCTK_VParamWarn.  This removes the
>> need for the function print_bold_stderr.
>>
>> Furthermore it removes an unnecessary call to va_copy, and adds
>> missing calls to va_end (avoiding potential memory leaks).
>>
>> -erik
>>
>>
>
> -- 
> Dr. Bruno Giacomazzo
> Max Planck Institute for Gravitational Physics
> Albert Einstein Institute
> Am Muehlenberg 1
> D-14476 Potsdam
> Germany
>
> Tel.  : +49 331 567 7183
> Fax   : +49 331 567 7298
> cell. : +49 173 826 4488
> email : bgiacoma at aei.mpg.de
>
> -------------------------------------------------
> There are only 10 types of people in the world:
> Those who understand binary, and those who don't
> -------------------------------------------------
> _______________________________________________
> Developers mailing list
> Developers at cactuscode.org
> http://www.cactuscode.org/mailman/listinfo/developers


-- 
Erik Schnetter <schnetter at cct.lsu.edu>   http://www.cct.lsu.edu/~eschnett/

My email is as private as my paper mail.  I therefore support encrypting
and signing email messages.  Get my PGP key from <http://pgp.mit.edu>.



-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 203 bytes
Desc: This is a digitally signed message part
Url : http://www.cactuscode.org/pipermail/developers/attachments/20090915/e48ba771/attachment-0001.bin 

From knarf at cct.lsu.edu  Thu Sep 17 20:51:05 2009
From: knarf at cct.lsu.edu (Frank Loeffler)
Date: Thu, 17 Sep 2009 20:51:05 -0500
Subject: [Developers] improve parser for parameter files (2)
In-Reply-To: <20090911015042.GC11625@numrel07.cct.lsu.edu>
References: <20090910195426.GA11625@numrel07.cct.lsu.edu>
	<8B2F1374-1B31-45BF-82E5-B3F27970212E@cct.lsu.edu>
	<20090911015042.GC11625@numrel07.cct.lsu.edu>
Message-ID: <20090918015104.GG5912@numrel07.cct.lsu.edu>

Hi,

On Thu, Sep 10, 2009 at 08:50:43PM -0500, Frank Loeffler wrote:
> As for the syntax of the environment variables ($ENV_* at the moment), we
> could certainly change that to either $ENV{*} or $ENV{'*'}. Pick your
> favorite.

Since nobody else did, I picked one:


This patch improves the parser used by Cactus to prepocess parameter
files.

The main changes are:

- when reading a parameter file do not parse the file while reading,
  but read it first into a buffer, preprocess that and parse the
  buffer after that
- replace the code which changes $parfile into the parameter file
  name to use this buffer preprocessing
- replace $ENV{'*'} "defines" by environment variable values (*),
  e.g. $ENV{'HOME'} with $HOME.

Frank

Index: ParseFile.c
=========================
=========================
=================
RCS file: /cactusdevcvs/Cactus/src/util/ParseFile.c,v
retrieving revision 1.26
diff -u -r1.26 ParseFile.c
--- ParseFile.c	31 Aug 2006 16:54:40 -0000	1.26
+++ ParseFile.c	18 Sep 2009 01:43:53 -0000
@@ -7,7 +7,6 @@
               to a user-supplied subroutine.
               Currently taken from the old cactus ones and slightly modife
d.
    @enddesc
-   @version   $Id: ParseFile.c,v 1.26 2006/08/31 16:54:40 tradke Exp $
  @@*/
 
 /*#define DEBUG*/
@@ -15,6 +14,9 @@
 #include <stdio.h>
 #include <string.h>
 #include <stdlib.h>
+#include <string.h>
+
+#include <cctk.h>
 
 #include "cctk_Flesh.h"
 #include "cctk_CommandLine.h"
@@ -38,6 +40,11 @@
 
 static void CheckBuf(int, int);
 static void removeSpaces(char *stripMe);
+static char *ReadFile(FILE *file, unsigned long *filesize);
+static char *ParseDefines(char *buffer, unsigned long *buffersize);
+int ParseBuffer(char *buffer,
+                int (*set_function)(const char *, const char *, int),
+                tFleshConfig *ConfigData);
 
 /********************************************************************
  ********************* Other Routine Prototypes *********************
@@ -63,7 +70,7 @@
 
  /*@@
    @routine ParseFile
-   @author Paul Walker
+   @author Paul Walker, Frank Loeffler
    @desc
    This routine actually parses the parameter file. The
    syntax we allow is
@@ -83,6 +90,7 @@
    @hdesc Moved to CCTK.
           Changed to pass data to arbitrary function.
           Changed to take a file descriptor rather than a filename.
+          Use a buffer to parse which can be preprocessed before main pars
ing
    @endhistory
    @var     ifp
    @vdesc   The filestream to parse
@@ -111,10 +119,434 @@
    0 - success
    @endreturndesc
 @@*/
+
 int ParseFile(FILE *ifp,
               int (*set_function)(const char *, const char *, int),
               tFleshConfig *ConfigData)
 {
+  int retval;
+  unsigned long buffersize;
+  char *buffer = ReadFile(ifp, &buffersize);
+  if (!buffer)
+    return 1;
+  buffer = ParseDefines(buffer, &buffersize);
+  retval = ParseBuffer(buffer, set_function, ConfigData);
+  free(buffer);
+  return retval;
+}
+
+/********************************************************************
+ *********************     Local Routines   *************************
+ ********************************************************************/
+
+ /*@@
+   @routine CheckBuf
+   @author Paul Walker
+   @desc
+   A simple description and warning message in case of
+   a fixed parse buffer overflow.
+   @enddesc
+   @calls
+   @calledby
+   @history
+
+   @endhistory
+   @var     p
+   @vdesc   buffer location
+   @vtype   int
+   @vio     in
+   @vcomment
+
+   @endvar
+   @var     l
+   @vdesc   Line number
+   @vtype   int
+   @vio     in
+   @vcomment
+
+   @endvar
+
+@@*/
+
+static void CheckBuf(int p, int l)
+{
+  if (p >= BUF_SZ)
+  {
+    fprintf(stderr,"WARNING: Parser buffer overflow on line %d\n",
+            l);
+    fprintf(stderr,"This indicates either an incorrect parm file or\n");
+    fprintf(stderr,"the need to recompile " __FILE__ " with a bigger\n");
+    fprintf(stderr,"BUF_SZ parm.\n");
+
+    assert(0);
+    exit(1);
+  }
+}
+
+
+ /*@@
+   @routine removeSpaces
+   @author Paul Walker
+   @desc
+   removes the spaces from a char * <b>in place</b>. Beware
+   that this function will change the input value and if you
+   want to keep a copy you have to do so yourself!
+   @enddesc
+   @calls
+   @calledby
+   @history
+
+   @endvar
+   @var     stripMe
+   @vdesc   String to strip
+   @vtype   char *
+   @vio     inout
+   @vcomment
+
+   @endvar
+
+@@*/
+static void removeSpaces(char *stripMe)
+{
+  char *s;
+  unsigned int i,j;
+  s = (char *)malloc((strlen(stripMe)+2)*sizeof(char));
+
+  if(s)
+  {
+    strcpy(s,stripMe);
+    for (i=0,j=0;i<strlen(s);i++)
+    {
+      if (s[i] != ' ' && s[i] != '\t' && s[i] != '\n' && s[i] != '
\r')
+      {
+        stripMe[j++] = s[i];
+      }
+      stripMe[j] = '\0';
+    }
+  }
+
+  free(s);
+}
+
+
+/* #define TEST_ParseFile */
+
+#ifdef TEST_ParseFile
+
+int parameter_printer(const char *param, const char *val)
+{
+  printf("Parameter %s has value %s\n", param, val);
+
+  return 0;
+}
+
+int main(int argc, char *argv[])
+{
+  int retval;
+  FILE *parameter_file;
+
+  if(argc > 1)
+  {
+    parameter_file = fopen(argv[1], "r");
+    if(parameter_file)
+    {
+      ParseFile(parameter_file, parameter_printer);
+      fclose(parameter_file);
+      retval = 0;
+    }
+    else
+    {
+      retval=2;
+    }
+  }
+  else
+  {
+    printf("Usage: %s <filename>\n", argv[0]);
+    retval = 1;
+  };
+
+  return 0;
+}
+
+#endif
+
+
+ /*@@
+   @routine ReadFile
+   @author Frank Loeffler
+   @desc
+   This routine reads a file into a buffer
+   @enddesc
+   @history
+   @var     file
+   @vdesc   The filestream to read
+   @vtype   FILE *
+   @vio     in
+   @vcomment
+
+   @endvar
+   @var     filesize
+   @vdesc   The size of the file
+   @vtype   *unsigned long
+   @vio     out
+   @vcomment
+
+   @endvar
+
+   @returntype char *
+   @returndesc
+   NULL - failure
+   !NULL allocated buffer
+   @endreturndesc
+@@*/
+static char *ReadFile(FILE *file, unsigned long *filesize)
+{
+  char *buffer;
+
+  if (!file)
+  {
+    fprintf(stderr, "Could not use file for reading.\n");
+    return NULL;
+  }
+  /* Get the file size */
+  fseek(file, 0, SEEK_END);
+  *filesize = ftell(file);
+  fseek(file, 0, SEEK_SET);
+  /* Allocate buffer */
+  buffer = (char *)malloc(*filesize+1);
+  if (!buffer)
+  {
+    fprintf(stderr, "Could not allocate memory.\n");
+    return NULL;
+  }
+  /* Read file into buffer and return */
+  fread(buffer, *filesize, 1, file);
+  /* Protect buffer for string operations */
+  buffer[*filesize] = '\0';
+  return buffer;
+}
+
+ /*@@
+   @routine ParseDefines
+   @author Frank Loeffler
+   @desc
+   This routine parses a buffer for defines of the form \$[^ \n] and
+   replaces them with some predefined defines, returning the new buffer,
+   which might be the same as the old buffer. Otherwise the old buffer
+   gets freed by this function. In case of parsing errors, the whole
+   parsing is stopped and the old buffer is returned
+   (FIXME: Should we return NULL?)
+   @enddesc
+   @history
+   @var     buffer
+   @vdesc   The buffer to parse
+   @vtype   char *
+   @vio     in
+   @vcomment
+
+   @endvar
+   @var     buffersize
+   @vdesc   The size of the buffer
+   @vtype   *unsigned long
+   @vio     out
+   @vcomment
+
+   @endvar
+
+   @returntype char *
+   @returndesc
+   !NULL - new buffer, might be == buffer
+   @endreturndesc
+@@*/
+static char *ParseDefines(char *buffer, unsigned long *buffersize)
+{
+  /* define name */
+  char define[1024];
+  /* Position in define name */
+  char defpos = 0;
+  /* Current position in buffer */
+  size_t pos = 0;
+  /* Character at current position */
+  char c;
+  /* Position of start of found define */
+  size_t def_start = 0;
+  /* Flag to indicate if we are inside a definition name */
+  int indef = 0;
+  if (!buffer)
+    return buffer;
+  /* Main loop over the buffer */
+  while (((c=buffer[pos]) != '\0') && (pos < *buffersize) )
+  {
+    /* Check if we found a define */
+    if (c == '$')
+    {
+      /* Mark the state and location and go on */
+      indef = 1;
+      def_start = pos;
+    }
+    /* If we are inside a define */
+    else if (indef)
+    {
+      /* Look for the end of the define name */
+      if (c == '}' || c == '\0' || c == '\n')
+      {
+        /* make the define name a proper string */
+        define[defpos] = '\0';
+        /* this holds the value of the define to be filled in */
+        char *value = NULL;
+        /* a new buffer for the whole thing */
+        char * new_buffer;
+        unsigned long new_size;
+        /* Check for different kinds of defines */
+        /* Environment variables */
+        if ((defpos > 6) &&
+            (strncmp(define, "ENV{'", 5) == 0) &&
+            (define[defpos-1] == '\''))
+        {
+          /* overwrite the trailing ' */
+          define[defpos-1] = '\0';
+          value = getenv(define+5);
+          if (!value)
+          {
+            CCTK_VWarn(CCTK_WARN_ALERT, __LINE__, __FILE__, CCTK_THORNSTRI
NG,
+                       "No environment variable %s found\n", define+5);
+            /* TODO: Should we abort here? (return NULL) */
+          }
+          /* increase pos to jump over the trailing } */
+          if (pos+1 < *buffersize) { pos++; }
+        }
+        /* Parameter file name ? */
+        else if (strcmp(define, "parfile") == 0)
+        {
+          char path[500];
+          CCTK_ParameterFilename(500, path);
+          value = strrchr (path, '/');
+          if (value == NULL)
+          {
+            value = path;
+          }
+          else
+          {
+            value++;
+          }
+          /* skip the parameter file extension */
+          if (strcmp (value + strlen (value) - 4, ".par") == 0)
+          {
+            value[strlen (value) - 4] = '\0';
+          }
+        }
+        /* Else: unknown define - or no define at all: ignore */
+        if (value)
+        {
+          /* Replace the old buffer with the new, copying in all the data 
*/
+          new_size = *buffersize - strlen(define) + strlen(value) + 1;
+          if (new_size < def_start)
+          {
+            CCTK_WARN(0, "Something is wrong with me, HELP!");
+            return buffer;
+          }
+          new_buffer = (char *)malloc(new_size);
+          if (!new_buffer)
+          {
+            CCTK_WARN(0, "I am out of memory and give up parsing for defin
es.");
+            return buffer;
+          }
+          new_buffer[0]='\0';
+          strncat(new_buffer, buffer, def_start);
+          strncat(new_buffer, value, new_size-1);
+          strncat(new_buffer, buffer+pos, new_size-1);
+          new_buffer[new_size-1] = '\0';
+          /* free the old buffer */
+          free(buffer);
+          /* Start further processing of new buffer */
+          pos = def_start + strlen(define);
+          buffer = new_buffer;
+          *buffersize = new_size;
+        }
+        /* General define cleanup */
+        indef = 0;
+        defpos = 0;
+        define[0] = '\0';
+        def_start = 0;
+      }
+      else if (defpos > 1023)
+      {
+        /* We only print this warning and ignore the possible define probl
em.
+         * It might not be a define after all but a valid, long parameter
+         * value containing a '$'.*/
+        define[1023] = '\0';
+        CCTK_VWarn(CCTK_WARN_PICKY, __LINE__, __FILE__, CCTK_THORNSTRING,
+                   "Possible define too long: %s", define);
+        indef = 0;
+        defpos = 0;
+        define[0] = '\0';
+        def_start = 0;
+      }
+      else
+        define[defpos++] = c;
+    }
+    pos++;
+  }
+  return buffer;
+}
+
+ /*@@
+   @routine ParseBuffer
+   @author Paul Walker
+   @desc
+   This routine actually parses the parameter file. The
+   syntax we allow is
+   <ul>
+     <li>a = b
+         <li>a,b,c = d,e,f
+         <li># rest of the line is ignored
+         <li>x = "string value"
+   </ul>
+   So it is easy to parse
+   <p>
+   We go through the file looking for stuff and then set
+   it in the global database using calls to the passed in set_function.
+   @enddesc
+   @history
+   @hdate Tue Jan 12 16:41:36 1999 @hauthor Tom Goodale
+   @hdesc Moved to CCTK.
+          Changed to pass data to arbitrary function.
+          Changed to take a file descriptor rather than a filename.
+          Use a buffer to parse which can be preprocessed before main pars
ing
+   @endhistory
+   @var     buffer
+   @vdesc   The buffer to parse
+   @vtype   char *
+   @vio     in
+   @vcomment
+
+   @endvar
+   @var     set_function
+   @vdesc   The function to call to set the value of a parameter
+   @vtype   int (*)(const char *, const char *)
+   @vio     in
+   @vcomment
+
+   @endvar
+   @var     ConfigData
+   @vdesc   Flesh configuration data
+   @vtype   tFleshConfig *
+   @vio     in
+   @vcomment
+
+   @endvar
+
+   @returntype int
+   @returndesc
+   0 - success
+   @endreturndesc
+@@*/
+int ParseBuffer(char *buffer,
+              int (*set_function)(const char *, const char *, int),
+              tFleshConfig *ConfigData)
+{
+  /* position in buffer */
+  unsigned int pos = 0;
   /* Buffers for parsing from the file */
   char *tokens, *value;
   char *subtoken, *subvalue;
@@ -122,10 +554,8 @@
   int ntokens;
   /* Status flags */
   int intoken, inval;
-  /* Current char. Have to make it an int so we can compare with
-     EOF. See man 3 fgetc
-   */
-  int c;
+  /* Current char */
+  char c;
   int num_errors; /* number of errors in file parsing */
 
   num_errors = 0;
@@ -141,7 +571,7 @@
 
   intoken = 0; inval = 0;
 
-  while ((c=fgetc(ifp)) != EOF)
+  while ((c=buffer[pos++]) != '\0')
   {
 #ifdef DEBUG
     printf("%c",c);
@@ -150,7 +580,7 @@
     while (c == '#' || c == '!' )
     {
       /* Comment line.  So forget rest of line */
-      while ((c=fgetc(ifp)) != '\n' && c != '\r' && c != EOF)
+      while ((c=buffer[pos++]) != '\n' && c != '\r' && c != '\0')
       {
 #ifdef DEBUG
         printf("%c",c);
@@ -160,7 +590,7 @@
       {
         lineno++;
       }
-      c = fgetc(ifp);
+      c = buffer[pos++];
 #ifdef DEBUG
       printf("%c",c);
 #endif
@@ -220,7 +650,7 @@
          * and check if the value is a string or not.
          * This parser DOES strip quotes off of the strings.
          */
-        while ((c = fgetc(ifp)) == ' ' || c == '\n' || c == 
'\r' || c == '\t')
+        while ((c = buffer[pos++]) == ' ' || c == '\n' || c =
= '\r' || c == '\t')
         {
 #ifdef DEBUG
           printf("%c",c);
@@ -251,7 +681,7 @@
             free (tokens);
             return 1;
           }
-          while ((c = fgetc(ifp)) != '"')
+          while ((c = buffer[pos++]) != '"')
           {
 #ifdef DEBUG
             printf("%c",c);
@@ -265,7 +695,7 @@
 #endif
               lineno++;
             }
-            else if (c == EOF)
+            else if (c == '\0')
             {
               break;
             }
@@ -274,9 +704,9 @@
           if (c == '"')
           {
             int is_comment = 0;
-            while ((c = fgetc(ifp)) != '\n')
+            while ((c = buffer[pos++]) != '\n')
             {
-              if (c == EOF)
+              if (c == '\0')
               {
                 break;
               }
@@ -295,7 +725,7 @@
                 return 1;
               }
             }
-            if (c == '\n' || c == EOF)
+            if (c == '\n' || c == '\0')
             {
               if (c == '\n') lineno++;
               /* mark it as properly quoted string value */
@@ -319,39 +749,6 @@
 #endif
           set_function(tokens,value, lineno);
         }
-        else if (c == '$')
-        {
-          /* We got a define */
-          /* FIXME: Assume it is a parameter file for now */
-          char path[500];
-          char *parfile;
-
-          CCTK_ParameterFilename(500,path);
-          parfile = strrchr (path, '/');
-          if (parfile == NULL)
-          {
-            parfile = path;
-          }
-          else
-          {
-            parfile++;
-          }
-          /* skip the parameter file extension */
-          if (strcmp (parfile + strlen (parfile) - 4, ".par") == 0)
-          {
-            parfile[strlen (parfile) - 4] = 0;
-          }
-
-          /* ignore everything else on the line */
-          while (!(c==' ' || c=='\t' || c == '\n' || c == 
'\r' || c == EOF))
-          {
-            c = fgetc(ifp);
-#ifdef DEBUG
-            printf("%c",c);
-#endif
-          }
-          set_function(tokens,parfile,lineno);
-        }
         else
         {
 
@@ -362,15 +759,15 @@
             /* Simple case. We have an int
                or a double which contain no
                spaces! */
-            c = fgetc(ifp);
+            c = buffer[pos++];
 #ifdef DEBUG
             printf("%c",c);
 #endif
-            while (!(c==' ' || c=='\t' || c == '\n' || c =
= '\r' || c == EOF))
+            while (!(c==' ' || c=='\t' || c == '\n' || c =
= '\r' || c == '\0'))
             {
               value[p++] = (char)c;
               CheckBuf(p,lineno);
-              c = fgetc(ifp);
+              c = buffer[pos++];
 #ifdef DEBUG
               printf("%c",c);
 #endif
@@ -402,11 +799,11 @@
                ntokens-1 commas, stripping spaces, and
                make a nice little string.
                */
-            c = fgetc(ifp);
+            c = buffer[pos++];
 #ifdef DEBUG
             printf("%c",c);
 #endif
-            while (ncommas < ntokens-1 && c != EOF)
+            while (ncommas < ntokens-1 && c != '\0')
             {
               if (!(c == ' ' || c == '\t' || c == '\n' || c 
== '\r'))
               {
@@ -414,7 +811,7 @@
                 CheckBuf(pp,lineno);
               }
               if (c == ',') ncommas ++;
-              c = fgetc(ifp);
+              c = buffer[pos++];
 #ifdef DEBUG
               printf("%c",c);
 #endif
@@ -422,7 +819,7 @@
             if (c == ' ' || c == '\t')
             {
               /* Great now strip out the spaces */
-              while((c = fgetc(ifp)) == ' ' || c=='\t' || c =
= '\n' || c == '\r')
+              while((c = buffer[pos++]) == ' ' || c=='\t' || c 
== '\n' || c == '\r')
               {
 #ifdef DEBUG
                 printf("%c",c);
@@ -441,15 +838,15 @@
             value[pp++] = (char)c;
             CheckBuf(p,lineno);
 
-            c = fgetc(ifp);
+            c = buffer[pos++];
 #ifdef DEBUG
             printf("%c",c);
 #endif
-            while (c != ' ' && c != '\t' && c != '\n' && c != '\r'
 && c != EOF)
+            while (c != ' ' && c != '\t' && c != '\n' && c != '\r'
 && c != '\0')
             {
               value[pp++] = (char)c;
               CheckBuf(pp,lineno);
-              c = fgetc(ifp);
+              c = buffer[pos++];
 #ifdef DEBUG
               printf("%c",c);
 #endif
@@ -525,137 +922,3 @@
   return num_errors;
 }
 
-/********************************************************************
- *********************     Local Routines   *************************
- ********************************************************************/
-
- /*@@
-   @routine CheckBuf
-   @author Paul Walker
-   @desc
-   A simple description and warning message in case of
-   a fixed parse buffer overflow.
-   @enddesc
-   @calls
-   @calledby
-   @history
-
-   @endhistory
-   @var     p
-   @vdesc   buffer location
-   @vtype   int
-   @vio     in
-   @vcomment
-
-   @endvar
-   @var     l
-   @vdesc   Line number
-   @vtype   int
-   @vio     in
-   @vcomment
-
-   @endvar
-
-@@*/
-
-static void CheckBuf(int p, int l)
-{
-  if (p >= BUF_SZ)
-  {
-    fprintf(stderr,"WARNING: Parser buffer overflow on line %d\n",
-            l);
-    fprintf(stderr,"This indicates either an incorrect parm file or\n");
-    fprintf(stderr,"the need to recompile " __FILE__ " with a bigger\n");
-    fprintf(stderr,"BUF_SZ parm.\n");
-
-    assert(0);
-    exit(1);
-  }
-}
-
-
- /*@@
-   @routine removeSpaces
-   @author Paul Walker
-   @desc
-   removes the spaces from a char * <b>in place</b>. Beware
-   that this function will change the input value and if you
-   want to keep a copy you have to do so yourself!
-   @enddesc
-   @calls
-   @calledby
-   @history
-
-   @endvar
-   @var     stripMe
-   @vdesc   String to strip
-   @vtype   char *
-   @vio     inout
-   @vcomment
-
-   @endvar
-
-@@*/
-static void removeSpaces(char *stripMe)
-{
-  char *s;
-  unsigned int i,j;
-  s = (char *)malloc((strlen(stripMe)+2)*sizeof(char));
-
-  if(s)
-  {
-    strcpy(s,stripMe);
-    for (i=0,j=0;i<strlen(s);i++)
-    {
-      if (s[i] != ' ' && s[i] != '\t' && s[i] != '\n' && s[i] != '
\r')
-      {
-        stripMe[j++] = s[i];
-      }
-      stripMe[j] = '\0';
-    }
-  }
-
-  free(s);
-}
-
-
-/* #define TEST_ParseFile */
-
-#ifdef TEST_ParseFile
-
-int parameter_printer(const char *param, const char *val)
-{
-  printf("Parameter %s has value %s\n", param, val);
-
-  return 0;
-}
-
-int main(int argc, char *argv[])
-{
-  int retval;
-  FILE *parameter_file;
-
-  if(argc > 1)
-  {
-    parameter_file = fopen(argv[1], "r");
-    if(parameter_file)
-    {
-      ParseFile(parameter_file, parameter_printer);
-      fclose(parameter_file);
-      retval = 0;
-    }
-    else
-    {
-      retval=2;
-    }
-  }
-  else
-  {
-    printf("Usage: %s <filename>\n", argv[0]);
-    retval = 1;
-  };
-
-  return 0;
-}
-
-#endif


--- StripMime Report -- processed MIME parts ---
multipart/signed
  multipart/mixed
    text/plain (text body -- kept)
    text/plain (text body -- kept)
  application/pgp-signature
---

From schnetter at cct.lsu.edu  Thu Sep 17 23:21:27 2009
From: schnetter at cct.lsu.edu (Erik Schnetter)
Date: Thu, 17 Sep 2009 23:21:27 -0500
Subject: [Developers] New parameter HydroBase::evolution_method?
Message-ID: <BAD38C6D-8837-4F91-8F67-B34FAE6F5BC6@cct.lsu.edu>

Thorn ADMBase has a parameter evolution_method that decides which  
spacetime evolution method is used.  For example, thorn BSSN_MoL  
extends this parameter by "adm_bssn", and McLachlan by "mclachlan".   
The default setting is "static", meaning that the spacetime is not  
evolved.

There is no corresponding parameter in HydroBase.  I suggest to add  
HydroBase::evolution_method, with a default setting of "static".   
Whisky would extend this parameter by "whisky".  This would allow a  
very natural way to allow Cowling and anti-Cowling simulations,  
without requiring BSSN_MoL or Whisky to be active if they are not used.

This would replace the current Whisky::whisky_evolve.

-erik

-- 
Erik Schnetter <schnetter at cct.lsu.edu>   http://www.cct.lsu.edu/~eschnett/

My email is as private as my paper mail.  I therefore support encrypting
and signing email messages.  Get my PGP key from <http://pgp.mit.edu>.



-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 203 bytes
Desc: This is a digitally signed message part
Url : http://www.cactuscode.org/pipermail/developers/attachments/20090917/7252c4c9/attachment.bin 

From i.hawke at soton.ac.uk  Fri Sep 18 03:04:21 2009
From: i.hawke at soton.ac.uk (I.Hawke)
Date: Fri, 18 Sep 2009 09:04:21 +0100
Subject: [Developers] New parameter HydroBase::evolution_method?
In-Reply-To: <BAD38C6D-8837-4F91-8F67-B34FAE6F5BC6@cct.lsu.edu>
References: <BAD38C6D-8837-4F91-8F67-B34FAE6F5BC6@cct.lsu.edu>
Message-ID: <4AB33F05.8080409@soton.ac.uk>

Erik Schnetter wrote:
> Thorn ADMBase has a parameter evolution_method that decides which  
> spacetime evolution method is used.  For example, thorn BSSN_MoL  
> extends this parameter by "adm_bssn", and McLachlan by "mclachlan".   
> The default setting is "static", meaning that the spacetime is not  
> evolved.
>
> There is no corresponding parameter in HydroBase.  I suggest to add  
> HydroBase::evolution_method, with a default setting of "static".   
> Whisky would extend this parameter by "whisky".  This would allow a  
> very natural way to allow Cowling and anti-Cowling simulations,  
> without requiring BSSN_MoL or Whisky to be active if they are not used.
>
> This would replace the current Whisky::whisky_evolve.
This makes sense to me.

Ian



From knarf at cct.lsu.edu  Fri Sep 18 08:15:51 2009
From: knarf at cct.lsu.edu (Frank Loeffler)
Date: Fri, 18 Sep 2009 08:15:51 -0500
Subject: [Developers] New parameter HydroBase::evolution_method?
In-Reply-To: <BAD38C6D-8837-4F91-8F67-B34FAE6F5BC6@cct.lsu.edu>
References: <BAD38C6D-8837-4F91-8F67-B34FAE6F5BC6@cct.lsu.edu>
Message-ID: <20090918131550.GI5912@numrel07.cct.lsu.edu>

Hi,

On Thu, Sep 17, 2009 at 11:21:27PM -0500, Erik Schnetter wrote:
> I suggest to add
> HydroBase::evolution_method, with a default setting of "static".

I also think this would be a good idea.

Frank

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
Url : http://www.cactuscode.org/pipermail/developers/attachments/20090918/c52d5e8c/attachment-0001.bin 

From knarf at cct.lsu.edu  Fri Sep 18 09:07:24 2009
From: knarf at cct.lsu.edu (Frank Loeffler)
Date: Fri, 18 Sep 2009 09:07:24 -0500
Subject: [Developers] New parameter HydroBase::evolution_method?
In-Reply-To: <189219.261253282354802.OPEN-XCHANGE.WebMail.tomcat@OX1.aei.mpg.de>
References: <20090918131113.GH5912@numrel07.cct.lsu.edu>
	<189219.261253282354802.OPEN-XCHANGE.WebMail.tomcat@OX1.aei.mpg.de>
Message-ID: <20090918140723.GJ5912@numrel07.cct.lsu.edu>

Hi,

On Fri, Sep 18, 2009 at 03:59:14PM +0200, Aaryn Tonita wrote:
> Won't something still need to set the stress energy tensor for the
> anti-cowling type of simulation to run? Does HydroBase or some other
> thorn already do that? If the hydro is not evolved, all that is really
> necessary is the stress energy tensor which would need to be set by some
> sort of initial data routine and preferably there would be no need to
> store any hydro variables after this.

You raise an interesting point. Since the primitive variables are now
defined in HydroBase, wouldn't it make sense to let HydroBase also set
the energy momentum tensor, at least the parts which it can set from its
variables? There could be an option to turn that off if there is
interest.
 
> In that case, wouldn't one simply want to turn hydrobase off and use
> Tmunubase instead?

In principle you could. However, most of the time you probably want to
have some kind of initial data and all the initial data thorns rely
on HydroBase. What one could do is to turn the storage for the HydroBase
variables off after the energy momentum tensor was calculated. But I
don't think it would be a big advantage in terms of memory, still it
might be worth a try if someone cares.

Frank

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
Url : http://www.cactuscode.org/pipermail/developers/attachments/20090918/b3b33f29/attachment.bin 

From knarf at cct.lsu.edu  Fri Sep 18 09:11:28 2009
From: knarf at cct.lsu.edu (Frank Loeffler)
Date: Fri, 18 Sep 2009 09:11:28 -0500
Subject: [Developers] [Whisky-developers] Re: New parameter
 HydroBase::evolution_method?
In-Reply-To: <20090918140723.GJ5912@numrel07.cct.lsu.edu>
References: <20090918131113.GH5912@numrel07.cct.lsu.edu>
	<189219.261253282354802.OPEN-XCHANGE.WebMail.tomcat@OX1.aei.mpg.de>
	<20090918140723.GJ5912@numrel07.cct.lsu.edu>
Message-ID: <20090918141127.GK5912@numrel07.cct.lsu.edu>

Hi,

On Fri, Sep 18, 2009 at 09:07:24AM -0500, Frank Loeffler wrote:
> You raise an interesting point. Since the primitive variables are now
> defined in HydroBase, wouldn't it make sense to let HydroBase also set
> the energy momentum tensor, at least the parts which it can set from its
> variables?

As an argument against that: HydroBase does not depend on ADMBase or
TmunuBase at the moment. Setting Tmunu would introduce that dependency.

Frank

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
Url : http://www.cactuscode.org/pipermail/developers/attachments/20090918/5ff69ea3/attachment.bin 

From roland.haas at physics.gatech.edu  Fri Sep 18 09:29:55 2009
From: roland.haas at physics.gatech.edu (Roland Haas)
Date: Fri, 18 Sep 2009 10:29:55 -0400
Subject: [Developers] New parameter HydroBase::evolution_method?
In-Reply-To: <20090918140723.GJ5912@numrel07.cct.lsu.edu>
References: <20090918131113.GH5912@numrel07.cct.lsu.edu>	<189219.261253282354802.OPEN-XCHANGE.WebMail.tomcat@OX1.aei.mpg.de>
	<20090918140723.GJ5912@numrel07.cct.lsu.edu>
Message-ID: <4AB39963.3090800@mail.gatech.edu>

Hello all,

> You raise an interesting point. Since the primitive variables are now
> defined in HydroBase, wouldn't it make sense to let HydroBase also set
> the energy momentum tensor, at least the parts which it can set from its
> variables? There could be an option to turn that off if there is
> interest.
Another argument against this is that there might be no agreement as to 
the meaning of HydroBase's variables, in particular I have seen at least 
two different definitions of the primitive velocities (eg. the Valencia 
and the older Wilson one), which influences how the stress-energy is 
constructed from the primitives. (Unless one can make the policy 
decision that vel[] in HydroBase is defined as in the Valencia formulation).

Yours,
Roland

From bgiacoma at aei.mpg.de  Fri Sep 18 09:30:27 2009
From: bgiacoma at aei.mpg.de (Bruno Giacomazzo)
Date: Fri, 18 Sep 2009 16:30:27 +0200 (CEST)
Subject: [Developers] [Whisky-developers] Re: New parameter
 HydroBase::evolution_method?
In-Reply-To: <20090918131550.GI5912@numrel07.cct.lsu.edu>
References: <BAD38C6D-8837-4F91-8F67-B34FAE6F5BC6@cct.lsu.edu>
	<20090918131550.GI5912@numrel07.cct.lsu.edu>
Message-ID: <Pine.LNX.4.64.0909181629530.32654@duo01.aei.mpg.de>

Hi,

> On Thu, Sep 17, 2009 at 11:21:27PM -0500, Erik Schnetter wrote:
>> I suggest to add
>> HydroBase::evolution_method, with a default setting of "static".

 	it seems OK to me.

Cheers,
Bruno

-- 
Dr. Bruno Giacomazzo
Max Planck Institute for Gravitational Physics
Albert Einstein Institute
Am Muehlenberg 1
D-14476 Potsdam
Germany

Tel.  : +49 331 567 7183
Fax   : +49 331 567 7298
cell. : +49 173 826 4488
email : bgiacoma at aei.mpg.de

-------------------------------------------------
There are only 10 types of people in the world:
Those who understand binary, and those who don't
-------------------------------------------------

From bgiacoma at aei.mpg.de  Fri Sep 18 09:35:14 2009
From: bgiacoma at aei.mpg.de (Bruno Giacomazzo)
Date: Fri, 18 Sep 2009 16:35:14 +0200 (CEST)
Subject: [Developers] New parameter HydroBase::evolution_method?
In-Reply-To: <4AB39963.3090800@mail.gatech.edu>
References: <20090918131113.GH5912@numrel07.cct.lsu.edu>
	<189219.261253282354802.OPEN-XCHANGE.WebMail.tomcat@OX1.aei.mpg.de>
	<20090918140723.GJ5912@numrel07.cct.lsu.edu>
	<4AB39963.3090800@mail.gatech.edu>
Message-ID: <Pine.LNX.4.64.0909181630570.32654@duo01.aei.mpg.de>

Hi,
 	I'm also against the idea of setting the energy momentum tensor 
inside hydrobase. This is something that should be done by the evolution 
code for the matter.

 	Codes that use different formulations (as Roland pointed out) or 
don't do "simply" GRHD would have to redefine the tensor internally and 
I'm afraid this could lead to potential problems/conflicts.

Cheers,
Bruno

On Fri, 18 Sep 2009, Roland Haas wrote:
>> You raise an interesting point. Since the primitive variables are now
>> defined in HydroBase, wouldn't it make sense to let HydroBase also set
>> the energy momentum tensor, at least the parts which it can set from its
>> variables? There could be an option to turn that off if there is
>> interest.
> Another argument against this is that there might be no agreement as to
> the meaning of HydroBase's variables, in particular I have seen at least
> two different definitions of the primitive velocities (eg. the Valencia
> and the older Wilson one), which influences how the stress-energy is
> constructed from the primitives. (Unless one can make the policy
> decision that vel[] in HydroBase is defined as in the Valencia formulation).
>
> Yours,
> Roland
> _______________________________________________
> Developers mailing list
> Developers at cactuscode.org
> http://www.cactuscode.org/mailman/listinfo/developers
>

-- 
Dr. Bruno Giacomazzo
Max Planck Institute for Gravitational Physics
Albert Einstein Institute
Am Muehlenberg 1
D-14476 Potsdam
Germany

Tel.  : +49 331 567 7183
Fax   : +49 331 567 7298
cell. : +49 173 826 4488
email : bgiacoma at aei.mpg.de

-------------------------------------------------
There are only 10 types of people in the world:
Those who understand binary, and those who don't
-------------------------------------------------

From I.Hawke at soton.ac.uk  Fri Sep 18 10:50:36 2009
From: I.Hawke at soton.ac.uk (Ian Hawke)
Date: Fri, 18 Sep 2009 16:50:36 +0100
Subject: [Developers] [Whisky-developers] Re: New parameter
	HydroBase::evolution_method?
In-Reply-To: <31534957.311253284887926.OPEN-XCHANGE.WebMail.tomcat@OX1.aei.mpg.de>
References: <20090918131113.GH5912@numrel07.cct.lsu.edu>	<189219.261253282354802.OPEN-XCHANGE.WebMail.tomcat@OX1.aei.mpg.de>	<20090918140723.GJ5912@numrel07.cct.lsu.edu>	<20090918141127.GK5912@numrel07.cct.lsu.edu>
	<31534957.311253284887926.OPEN-XCHANGE.WebMail.tomcat@OX1.aei.mpg.de>
Message-ID: <4AB3AC4C.8040804@soton.ac.uk>

An additional point that I should have thought of earlier. At present
Whisky acts as a central place to define the current EOS in use. Some ID
thorns look at Whisky parameters in order to know what they should be
doing. This should also be disentangled in order to set all variables
necessary for the stress energy.

Ian

Aaryn Tonita wrote:
> I wouldn't want to see that dependency either but I can easily write a
> TOV solver which does not depend on whisky, hydrobase or any other fluid
> thorn and set the correct value of stress energy tensor grid functions.
> Then any given evolution code should be able to read the static values
> of the stress energy tensor.
>
> Simply put, this strikes me as a parameter which would simply duplicate
> the line
>
> ActiveThorns = "Whisky"
>
> or whatever other fluid evolution thorn might be activated.
>
> In regard to that, currently Whisky_Analysis (I believe) depends on some
> scheduling provided by bssn_mol which necessitates activating bssn_mol
> even if I want to evolve with ctgamma. Therefore, there is a need, in
> bssn_mol, because of whisky_analysis to be able to disable bssn_mol from
> evolving the metrics while it is activated. This kind of dependency
> should not be necessary, nor encouraged, in my opinion if we want code
> to be insulated well from one another.
>
> Other than supporting this type of dependency, I do not even really see
> the point of having it in ADMBase, unless there is some functionality in
> ADMBase that I am unaware of that is switched off when the value is not
> static. However, given that this type of dependency can exist, it might
> be a good idea to have said parameter.
>
> Cheers,
>
> Aaryn
>
> On Sep 18, 2009 04:11 PM, Frank Loeffler <knarf at cct.lsu.edu> wrote:
>
>   
>> Hi,
>>
>> On Fri, Sep 18, 2009 at 09:07:24AM -0500, Frank Loeffler wrote:
>>     
>>> You raise an interesting point. Since the primitive variables are now
>>> defined in HydroBase, wouldn't it make sense to let HydroBase also set
>>> the energy momentum tensor, at least the parts which it can set from
>>> its
>>> variables?
>>>       
>> As an argument against that: HydroBase does not depend on ADMBase or
>> TmunuBase at the moment. Setting Tmunu would introduce that dependency.
>>
>> Frank
>>
>>     
>
> _______________________________________________
> Whisky-developers mailing list
> Whisky-developers at aei.mpg.de
> http://lists.aei.mpg.de/cgi-bin/mailman/listinfo/whisky-developers
>   




From schnetter at cct.lsu.edu  Fri Sep 18 13:36:13 2009
From: schnetter at cct.lsu.edu (Erik Schnetter)
Date: Fri, 18 Sep 2009 13:36:13 -0500
Subject: [Developers] New parameter HydroBase::evolution_method?
In-Reply-To: <4AB39963.3090800@mail.gatech.edu>
References: <20090918131113.GH5912@numrel07.cct.lsu.edu>	<189219.261253282354802.OPEN-XCHANGE.WebMail.tomcat@OX1.aei.mpg.de>
	<20090918140723.GJ5912@numrel07.cct.lsu.edu>
	<4AB39963.3090800@mail.gatech.edu>
Message-ID: <7F7A2845-A120-448A-8075-61DAAB87C0BC@cct.lsu.edu>

On Sep 18, 2009, at 9:29 , Roland Haas wrote:

> Hello all,
>
>> You raise an interesting point. Since the primitive variables are now
>> defined in HydroBase, wouldn't it make sense to let HydroBase also  
>> set
>> the energy momentum tensor, at least the parts which it can set  
>> from its
>> variables? There could be an option to turn that off if there is
>> interest.
> Another argument against this is that there might be no agreement as  
> to
> the meaning of HydroBase's variables, in particular I have seen at  
> least
> two different definitions of the primitive velocities (eg. the  
> Valencia
> and the older Wilson one), which influences how the stress-energy is
> constructed from the primitives. (Unless one can make the policy
> decision that vel[] in HydroBase is defined as in the Valencia  
> formulation).


Yes, this is the case.  HydroBase is not useful unless it's variables  
are well and explicitly defined.  If someone wants to use another  
formulation, then the variables could (a) be rather easily converted,  
or (b) we could introduce a state variable in HydroBase that defines  
the current meaning, similar to the conformal factor introduced by  
CactusEinstein/StaticConformal.

-erik

-- 
Erik Schnetter <schnetter at cct.lsu.edu>   http://www.cct.lsu.edu/~eschnett/

My email is as private as my paper mail.  I therefore support encrypting
and signing email messages.  Get my PGP key from <http://pgp.mit.edu>.



-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 203 bytes
Desc: This is a digitally signed message part
Url : http://www.cactuscode.org/pipermail/developers/attachments/20090918/b5ea6825/attachment.bin 

From schnetter at cct.lsu.edu  Fri Sep 18 13:36:24 2009
From: schnetter at cct.lsu.edu (Erik Schnetter)
Date: Fri, 18 Sep 2009 13:36:24 -0500
Subject: [Developers] [Whisky-developers] Re: New parameter
	HydroBase::evolution_method?
In-Reply-To: <20090918140723.GJ5912@numrel07.cct.lsu.edu>
References: <20090918131113.GH5912@numrel07.cct.lsu.edu>
	<189219.261253282354802.OPEN-XCHANGE.WebMail.tomcat@OX1.aei.mpg.de>
	<20090918140723.GJ5912@numrel07.cct.lsu.edu>
Message-ID: <2CD84BFC-0094-419F-8B6C-573016156E58@cct.lsu.edu>

On Sep 18, 2009, at 9:07 , Frank Loeffler wrote:

> Hi,
>
> On Fri, Sep 18, 2009 at 03:59:14PM +0200, Aaryn Tonita wrote:
>> Won't something still need to set the stress energy tensor for the
>> anti-cowling type of simulation to run? Does HydroBase or some other
>> thorn already do that? If the hydro is not evolved, all that is  
>> really
>> necessary is the stress energy tensor which would need to be set by  
>> some
>> sort of initial data routine and preferably there would be no need to
>> store any hydro variables after this.
>
> You raise an interesting point. Since the primitive variables are now
> defined in HydroBase, wouldn't it make sense to let HydroBase also set
> the energy momentum tensor, at least the parts which it can set from  
> its
> variables? There could be an option to turn that off if there is
> interest.
>
>> In that case, wouldn't one simply want to turn hydrobase off and use
>> Tmunubase instead?
>
> In principle you could. However, most of the time you probably want to
> have some kind of initial data and all the initial data thorns rely
> on HydroBase. What one could do is to turn the storage for the  
> HydroBase
> variables off after the energy momentum tensor was calculated. But I
> don't think it would be a big advantage in terms of memory, still it
> might be worth a try if someone cares.


Thorn AEIThorns/Exact provides initial spacetime data that have a non- 
zero Tmunu, but does not involve any hydrodynamics.  (Some of these  
initial data are for radiation energy, or even more strange forms of  
energy.)  It does not use HydroBase at all.

However, e.g. neutron star initial data may (should) rely on HydroBase.

-erik

-- 
Erik Schnetter <schnetter at cct.lsu.edu>   http://www.cct.lsu.edu/~eschnett/

My email is as private as my paper mail.  I therefore support encrypting
and signing email messages.  Get my PGP key from <http://pgp.mit.edu>.



-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 203 bytes
Desc: This is a digitally signed message part
Url : http://www.cactuscode.org/pipermail/developers/attachments/20090918/b8e4cec4/attachment.bin 

From schnetter at cct.lsu.edu  Mon Sep 21 11:15:05 2009
From: schnetter at cct.lsu.edu (Erik Schnetter)
Date: Mon, 21 Sep 2009 11:15:05 -0500
Subject: [Developers] Executing routines on a single processor only
Message-ID: <AACB9E67-FEC8-43F0-ACC0-CAD3372DCCDD@cct.lsu.edu>

It happens from time to time that one needs to execute certain  
routines only on a single MPI process.  While it is possible to add an  
"if (CCTK_MyProc(cctkGH)==0)", it would be nicer to declare this  
explicitly to the Cactus scheduler.

OpenMP faces a similar problem for multi-threading.  OpenMP offers  
three different mechanisms:

#pragma omp single
	Execute a region of code only by a single thread (can be any thread)
#pragma omp master
	Execute a region of code only by the master thread
#pragma omp critital
	Execute a region of code by all threads, by one thread at a time

Corresponding to this, we could offer the following options in the  
schedule:

	proc=single
		Is executed by a single process only (can be any process, e.g.  
creating a new directory)
	proc=root
		Is executed on the root process only (always the root process, e.g.  
serving web pages)
	proc=critical
		Is executed by each process, one at a time (e.g. appending to an  
ASCII file)

OpenMP has in addition a number of features that decide which threads  
can run at the same time, and whether threads have to wait for other  
threads.  I'm ignoring these here, so that the Cactus schedule is  
still traversed sequentially (as it is now).

-erik

-- 
Erik Schnetter <schnetter at cct.lsu.edu>   http://www.cct.lsu.edu/~eschnett/

My email is as private as my paper mail.  I therefore support encrypting
and signing email messages.  Get my PGP key from <http://pgp.mit.edu>.



-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 203 bytes
Desc: This is a digitally signed message part
Url : http://www.cactuscode.org/pipermail/developers/attachments/20090921/f88b1202/attachment-0001.bin 

From schnetter at cct.lsu.edu  Tue Sep 22 14:15:56 2009
From: schnetter at cct.lsu.edu (Erik Schnetter)
Date: Tue, 22 Sep 2009 14:15:56 -0500
Subject: [Developers] Support AMR in IOJpeg
Message-ID: <ECEAAF13-61E6-4A5A-AC28-9B70B244394E@cct.lsu.edu>

The enclosed patch supports AMR with IOJpeg.

It introduces a new keyword parameter "gridpoints" with possible  
values "hyperslab" and "interpolate".  Hyperslabbing works only on  
uniform grids, whereas interpolation also works with AMR.  Additional  
parameters select where in the domain the points should be interpolated.

-erik

-- 
Erik Schnetter <schnetter at cct.lsu.edu>   http://www.cct.lsu.edu/~eschnett/

My email is as private as my paper mail.  I therefore support encrypting
and signing email messages.  Get my PGP key from <http://pgp.mit.edu>.




--- StripMime Report -- processed MIME parts ---
multipart/mixed
  text/plain (text body -- kept)
  application/octet-stream
---

From schnetter at cct.lsu.edu  Fri Sep 25 10:04:29 2009
From: schnetter at cct.lsu.edu (Erik Schnetter)
Date: Fri, 25 Sep 2009 10:04:29 -0500
Subject: [Developers] CactusEinstein/TmunuBase
Message-ID: <33850118-B9BF-4BFF-9A55-C523C7609DAA@cct.lsu.edu>

I propose to move the thorn TmunuBase to CactusEinstein.  In addition  
to the version currently available from AEIThorns, I propose to add a  
new parameter support_old_CalcTmunu_mechanism (defaulting to yes).   
This parameter can be used to disable support for the CalcTmunu.inc  
mechanism, reducing storage and potentially increasing efficiency.

-erik

-- 
Erik Schnetter <schnetter at cct.lsu.edu>   http://www.cct.lsu.edu/~eschnett/

My email is as private as my paper mail.  I therefore support encrypting
and signing email messages.  Get my PGP key from <http://pgp.mit.edu>.




--- StripMime Report -- processed MIME parts ---
multipart/mixed
  text/plain (text body -- kept)
  application/x-gzip
---

From knarf at cct.lsu.edu  Fri Sep 25 10:09:01 2009
From: knarf at cct.lsu.edu (Frank Loeffler)
Date: Fri, 25 Sep 2009 10:09:01 -0500
Subject: [Developers] CactusEinstein/TmunuBase
In-Reply-To: <33850118-B9BF-4BFF-9A55-C523C7609DAA@cct.lsu.edu>
References: <33850118-B9BF-4BFF-9A55-C523C7609DAA@cct.lsu.edu>
Message-ID: <20090925150900.GK2142@numrel07.cct.lsu.edu>

On Fri, Sep 25, 2009 at 10:04:29AM -0500, Erik Schnetter wrote:
> I propose to move the thorn TmunuBase to CactusEinstein.  In addition  
> to the version currently available from AEIThorns, I propose to add a  
> new parameter support_old_CalcTmunu_mechanism (defaulting to yes).   
> This parameter can be used to disable support for the CalcTmunu.inc  
> mechanism, reducing storage and potentially increasing efficiency.

I agree to both ideas.

Frank

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
Url : http://www.cactuscode.org/pipermail/developers/attachments/20090925/4accd2fe/attachment.bin 

From denis.pollney at aei.mpg.de  Fri Sep 25 10:48:34 2009
From: denis.pollney at aei.mpg.de (Denis Pollney)
Date: Fri, 25 Sep 2009 17:48:34 +0200
Subject: [Developers] CactusEinstein/TmunuBase
In-Reply-To: <20090925150900.GK2142@numrel07.cct.lsu.edu>
References: <33850118-B9BF-4BFF-9A55-C523C7609DAA@cct.lsu.edu>
	<20090925150900.GK2142@numrel07.cct.lsu.edu>
Message-ID: <1253893714.4386.0.camel@dp-laptop>

On Fri, 2009-09-25 at 10:09 -0500, Frank Loeffler wrote:
> On Fri, Sep 25, 2009 at 10:04:29AM -0500, Erik Schnetter wrote:
> > I propose to move the thorn TmunuBase to CactusEinstein.  In addition  
> > to the version currently available from AEIThorns, I propose to add a  
> > new parameter support_old_CalcTmunu_mechanism (defaulting to yes).   
> > This parameter can be used to disable support for the CalcTmunu.inc  
> > mechanism, reducing storage and potentially increasing efficiency.
> 
> I agree to both ideas.

Hi, I also think they sound like good ideas.
Denis.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part
Url : http://www.cactuscode.org/pipermail/developers/attachments/20090925/a20957ab/attachment.bin 

From baiotti at yukawa.kyoto-u.ac.jp  Sun Sep 27 03:31:32 2009
From: baiotti at yukawa.kyoto-u.ac.jp (Baiotti Luca)
Date: Sun, 27 Sep 2009 17:31:32 +0900
Subject: [Developers] CactusEinstein/TmunuBase
In-Reply-To: <1253893714.4386.0.camel@dp-laptop>
References: <33850118-B9BF-4BFF-9A55-C523C7609DAA@cct.lsu.edu>	<20090925150900.GK2142@numrel07.cct.lsu.edu>
	<1253893714.4386.0.camel@dp-laptop>
Message-ID: <4ABF22E4.9010707@yukawa.kyoto-u.ac.jp>

Denis Pollney wrote:
> On Fri, 2009-09-25 at 10:09 -0500, Frank Loeffler wrote:
>> On Fri, Sep 25, 2009 at 10:04:29AM -0500, Erik Schnetter wrote:
>>> I propose to move the thorn TmunuBase to CactusEinstein.  In addition  
>>> to the version currently available from AEIThorns, I propose to add a  
>>> new parameter support_old_CalcTmunu_mechanism (defaulting to yes).   
>>> This parameter can be used to disable support for the CalcTmunu.inc  
>>> mechanism, reducing storage and potentially increasing efficiency.
>> I agree to both ideas.
> 
> Hi, I also think they sound like good ideas.

OK for me as well.

Luca

From knarf at cct.lsu.edu  Mon Sep 28 09:11:55 2009
From: knarf at cct.lsu.edu (Frank Loeffler)
Date: Mon, 28 Sep 2009 09:11:55 -0500
Subject: [Developers] check available timelevels for static boundary
	condition
Message-ID: <20090928141154.GB2142@numrel07.cct.lsu.edu>

Hi,

this patch checks the available timelevels for a variable before it
attempts a memcpy.

In the case of failure, this changes a segfault to an informative
message, which makes debugging this case much easier.

In case of success, it adds a test every time the static boundary
condition is called, for each variable. I am not sure about the overhead
this creates (a call to CCTK_ActiveTimeLevelsVI).

Frank


Index: StaticBoundary.c
===================================================================
RCS file: /cactusdevcvs/CactusBase/Boundary/src/StaticBoundary.c,v
retrieving revision 1.20
diff -u -r1.20 StaticBoundary.c
--- StaticBoundary.c	5 Oct 2006 16:56:28 -0000	1.20
+++ StaticBoundary.c	28 Sep 2009 14:08:27 -0000
@@ -1018,6 +1018,13 @@
        var < first_var + num_vars;
        var++)
   {
+    if (CCTK_ActiveTimeLevelsVI(GH, var) < 2)
+    {
+      CCTK_VWarn(0, __LINE__, __FILE__, CCTK_THORNSTRING,
+                 "Static Boundary condition needs at least two timelevels "
+                 "active, but %s only has %d.",
+                 CCTK_VarName(var), CCTK_ActiveTimeLevelsVI(GH, var));
+    }
     /* Apply condition if:
        + boundary is an outer boundary
        + have enough grid points


--- StripMime Report -- processed MIME parts ---
multipart/signed
  multipart/mixed
    text/plain (text body -- kept)
    text/plain (text body -- kept)
  application/pgp-signature
---

From schnetter at cct.lsu.edu  Mon Sep 28 11:18:51 2009
From: schnetter at cct.lsu.edu (Erik Schnetter)
Date: Mon, 28 Sep 2009 11:18:51 -0500
Subject: [Developers] [Patches] check available timelevels for static
	boundary condition
In-Reply-To: <20090928141154.GB2142@numrel07.cct.lsu.edu>
References: <20090928141154.GB2142@numrel07.cct.lsu.edu>
Message-ID: <D4CA4165-934C-47F2-8FF5-B044512CB5EA@cct.lsu.edu>

On Sep 28, 2009, at 9:11 , Frank Loeffler wrote:

> Hi,
>
> this patch checks the available timelevels for a variable before it
> attempts a memcpy.
>
> In the case of failure, this changes a segfault to an informative
> message, which makes debugging this case much easier.
>
> In case of success, it adds a test every time the static boundary
> condition is called, for each variable. I am not sure about the  
> overhead
> this creates (a call to CCTK_ActiveTimeLevelsVI).


The overhead should be negligible.

I would output the full variable name (including the group name) in  
the error message.

Yes, please apply.

-erik

-- 
Erik Schnetter <schnetter at cct.lsu.edu>   http://www.cct.lsu.edu/~eschnett/

My email is as private as my paper mail.  I therefore support encrypting
and signing email messages.  Get my PGP key from <http://pgp.mit.edu>.




From schnetter at cct.lsu.edu  Mon Sep 28 13:24:47 2009
From: schnetter at cct.lsu.edu (Erik Schnetter)
Date: Mon, 28 Sep 2009 13:24:47 -0500
Subject: [Developers] CactusEinstein/TmunuBase
In-Reply-To: <4ABF22E4.9010707@yukawa.kyoto-u.ac.jp>
References: <33850118-B9BF-4BFF-9A55-C523C7609DAA@cct.lsu.edu>	<20090925150900.GK2142@numrel07.cct.lsu.edu>
	<1253893714.4386.0.camel@dp-laptop>
	<4ABF22E4.9010707@yukawa.kyoto-u.ac.jp>
Message-ID: <5B7B24DE-2AB3-4B2A-9C56-5D6D9EC8D511@cct.lsu.edu>

On Sep 27, 2009, at 3:31 , Baiotti Luca wrote:

> Denis Pollney wrote:
>> On Fri, 2009-09-25 at 10:09 -0500, Frank Loeffler wrote:
>>> On Fri, Sep 25, 2009 at 10:04:29AM -0500, Erik Schnetter wrote:
>>>> I propose to move the thorn TmunuBase to CactusEinstein.  In  
>>>> addition
>>>> to the version currently available from AEIThorns, I propose to  
>>>> add a
>>>> new parameter support_old_CalcTmunu_mechanism (defaulting to yes).
>>>> This parameter can be used to disable support for the CalcTmunu.inc
>>>> mechanism, reducing storage and potentially increasing efficiency.
>>> I agree to both ideas.
>>
>> Hi, I also think they sound like good ideas.
>
> OK for me as well.


I have moved thorn AEIThorns/TmunuBase to CactusEinstein/TmumuBase.   
(The stable version of Cactus remains unchanged; this requires the  
development version of Cactus.)  Please make no further modifications  
to AEIThorns/TmunuBase.

TmunuBase has now one new parameter "support_old_CalcTmunu_mechanism",  
defaulting to yes.  If this parameter is set to no, then TmunuBase is  
not backwards compatible with the old CalcTmunu.inc mechanism any  
more.  This saves some memory and computation time, apparently even if  
not thorn using the CalcTmunu.inc is active.

-erik

-- 
Erik Schnetter <schnetter at cct.lsu.edu>   http://www.cct.lsu.edu/~eschnett/

My email is as private as my paper mail.  I therefore support encrypting
and signing email messages.  Get my PGP key from <http://pgp.mit.edu>.