From knarf at cct.lsu.edu Thu Sep 10 14:54:27 2009 From: knarf at cct.lsu.edu (Frank Loeffler) Date: Thu, 10 Sep 2009 14:54:27 -0500 Subject: [Patches] improve parser for parameter files Message-ID: <20090910195426.GA11625@numrel07.cct.lsu.edu> This patch improves the parser used by Cactus to prepocess parameter files. The main changes are: - when reading a parameter file do not parse the file while reading, but read it first into a buffer, preprocess that and parse the buffer after that - replace the code which changes $parfile into the parameter file name to use this buffer preprocessing - replace $ENV_ "defines" by environment variable values Frank -------------- next part -------------- Index: util/ParseFile.c =================================================================== RCS file: /cactusdevcvs/Cactus/src/util/ParseFile.c,v retrieving revision 1.26 diff -d -u -r1.26 ParseFile.c --- util/ParseFile.c 31 Aug 2006 16:54:40 -0000 1.26 +++ util/ParseFile.c 10 Sep 2009 19:49:49 -0000 @@ -7,7 +7,6 @@ to a user-supplied subroutine. Currently taken from the old cactus ones and slightly modifed. @enddesc - @version $Id: ParseFile.c,v 1.26 2006/08/31 16:54:40 tradke Exp $ @@*/ /*#define DEBUG*/ @@ -15,6 +14,9 @@ #include #include #include +#include + +#include #include "cctk_Flesh.h" #include "cctk_CommandLine.h" @@ -38,6 +40,11 @@ static void CheckBuf(int, int); static void removeSpaces(char *stripMe); +static char *ReadFile(FILE *file, unsigned long *filesize); +static char *ParseDefines(char *buffer, unsigned long *buffersize); +int ParseBuffer(char *buffer, + int (*set_function)(const char *, const char *, int), + tFleshConfig *ConfigData); /******************************************************************** ********************* Other Routine Prototypes ********************* @@ -63,7 +70,7 @@ /*@@ @routine ParseFile - @author Paul Walker + @author Paul Walker, Frank Loeffler @desc This routine actually parses the parameter file. The syntax we allow is @@ -83,6 +90,7 @@ @hdesc Moved to CCTK. Changed to pass data to arbitrary function. Changed to take a file descriptor rather than a filename. + Use a buffer to parse which can be preprocessed before main parsing @endhistory @var ifp @vdesc The filestream to parse @@ -111,10 +119,428 @@ 0 - success @endreturndesc @@*/ + int ParseFile(FILE *ifp, int (*set_function)(const char *, const char *, int), tFleshConfig *ConfigData) { + int retval; + unsigned long buffersize; + char *buffer = ReadFile(ifp, &buffersize); + if (!buffer) + return 1; + buffer = ParseDefines(buffer, &buffersize); + retval = ParseBuffer(buffer, set_function, ConfigData); + free(buffer); + return retval; +} + +/******************************************************************** + ********************* Local Routines ************************* + ********************************************************************/ + + /*@@ + @routine CheckBuf + @author Paul Walker + @desc + A simple description and warning message in case of + a fixed parse buffer overflow. + @enddesc + @calls + @calledby + @history + + @endhistory + @var p + @vdesc buffer location + @vtype int + @vio in + @vcomment + + @endvar + @var l + @vdesc Line number + @vtype int + @vio in + @vcomment + + @endvar + +@@*/ + +static void CheckBuf(int p, int l) +{ + if (p >= BUF_SZ) + { + fprintf(stderr,"WARNING: Parser buffer overflow on line %d\n", + l); + fprintf(stderr,"This indicates either an incorrect parm file or\n"); + fprintf(stderr,"the need to recompile " __FILE__ " with a bigger\n"); + fprintf(stderr,"BUF_SZ parm.\n"); + + assert(0); + exit(1); + } +} + + + /*@@ + @routine removeSpaces + @author Paul Walker + @desc + removes the spaces from a char * in place. Beware + that this function will change the input value and if you + want to keep a copy you have to do so yourself! + @enddesc + @calls + @calledby + @history + + @endvar + @var stripMe + @vdesc String to strip + @vtype char * + @vio inout + @vcomment + + @endvar + +@@*/ +static void removeSpaces(char *stripMe) +{ + char *s; + unsigned int i,j; + s = (char *)malloc((strlen(stripMe)+2)*sizeof(char)); + + if(s) + { + strcpy(s,stripMe); + for (i=0,j=0;i 1) + { + parameter_file = fopen(argv[1], "r"); + if(parameter_file) + { + ParseFile(parameter_file, parameter_printer); + fclose(parameter_file); + retval = 0; + } + else + { + retval=2; + } + } + else + { + printf("Usage: %s \n", argv[0]); + retval = 1; + }; + + return 0; +} + +#endif + + + /*@@ + @routine ReadFile + @author Frank Loeffler + @desc + This routine reads a file into a buffer + @enddesc + @history + @var file + @vdesc The filestream to read + @vtype FILE * + @vio in + @vcomment + + @endvar + @var filesize + @vdesc The size of the file + @vtype *unsigned long + @vio out + @vcomment + + @endvar + + @returntype char * + @returndesc + NULL - failure + !NULL allocated buffer + @endreturndesc +@@*/ +static char *ReadFile(FILE *file, unsigned long *filesize) +{ + char *buffer; + + if (!file) + { + fprintf(stderr, "Could not use file for reading.\n"); + return NULL; + } + /* Get the file size */ + fseek(file, 0, SEEK_END); + *filesize = ftell(file); + fseek(file, 0, SEEK_SET); + /* Allocate buffer */ + buffer = (char *)malloc(*filesize+1); + if (!buffer) + { + fprintf(stderr, "Could not allocate memory.\n"); + return NULL; + } + /* Read file into buffer and return */ + fread(buffer, *filesize, 1, file); + /* Protect buffer for string operations */ + buffer[*filesize] = '\0'; + return buffer; +} + + /*@@ + @routine ParseDefines + @author Frank Loeffler + @desc + This routine parses a buffer for defines of the form \$[^ \n] and + replaces them with some predefined defines, returning the new buffer, + which might be the same as the old buffer. Otherwise the old buffer + gets freed by this function. In case of parsing errors, the whole + parsing is stopped and the old buffer is returned + (FIXME: Should we return NULL?) + @enddesc + @history + @var buffer + @vdesc The buffer to parse + @vtype char * + @vio in + @vcomment + + @endvar + @var buffersize + @vdesc The size of the buffer + @vtype *unsigned long + @vio out + @vcomment + + @endvar + + @returntype char * + @returndesc + !NULL - new buffer, might be == buffer + @endreturndesc +@@*/ +static char *ParseDefines(char *buffer, unsigned long *buffersize) +{ + /* define name */ + char define[1024]; + /* Position in define name */ + char defpos = 0; + /* Current position in buffer */ + size_t pos = 0; + /* Character at current position */ + char c; + /* Position of start of found define */ + size_t def_start = 0; + /* Flag to indicate if we are inside a definition name */ + int indef = 0; + if (!buffer) + return buffer; + /* Main loop over the buffer */ + while (((c=buffer[pos]) != '\0') && (pos < *buffersize) ) + { + /* Check if we found a define */ + if (c == '$') + { + /* Mark the state and location and go on */ + indef = 1; + def_start = pos; + } + /* If we are inside a define */ + else if (indef) + { + /* Look for the end of the define name */ + if (c == ' ' || c == '/' || c == '\0' || c == '\n') + { + /* make the define name a proper string */ + define[defpos] = '\0'; + /* this holds the value of the define to be filled in */ + char *value = NULL; + /* a new buffer for the whole thing */ + char * new_buffer; + unsigned long new_size; + /* Check for different kinds of defines */ + /* Environment variables */ + if (strncmp(define, "ENV_", 4) == 0) + { + value = getenv(define+4); + if (!value) + { + CCTK_VWarn(CCTK_WARN_ALERT, __LINE__, __FILE__, CCTK_THORNSTRING, + "No environment variable %s found\n", define+4); + /* TODO: Should we abort here? (return NULL) */ + } + } + /* Parameter file name */ + else if (strcmp(define, "parfile") == 0) + { + char path[500]; + CCTK_ParameterFilename(500, path); + value = strrchr (path, '/'); + if (value == NULL) + { + value = path; + } + else + { + value++; + } + /* skip the parameter file extension */ + if (strcmp (value + strlen (value) - 4, ".par") == 0) + { + value[strlen (value) - 4] = '\0'; + } + } + /* Else: unknown define - or no define at all: ignore */ + if (value) + { + /* Replace the old buffer with the new, copying in all the data */ + new_size = *buffersize - strlen(define) + strlen(value) + 1; + if (new_size < def_start) + { + CCTK_WARN(0, "Something is wrong with me, HELP!"); + return buffer; + } + new_buffer = (char *)malloc(new_size); + if (!new_buffer) + { + CCTK_WARN(0, "I am out of memory and give up parsing for defines."); + return buffer; + } + new_buffer[0]='\0'; + strncat(new_buffer, buffer, def_start); + strncat(new_buffer, value, new_size-1); + strncat(new_buffer, buffer+pos, new_size-1); + new_buffer[new_size-1] = '\0'; + /* free the old buffer */ + free(buffer); + /* Start further processing of new buffer */ + pos = def_start + strlen(define); + buffer = new_buffer; + *buffersize = new_size; + } + /* General define cleanup */ + indef = 0; + defpos = 0; + define[0] = '\0'; + def_start = 0; + } + else if (defpos > 1023) + { + /* We only print this warning and ignore the possible define problem. + * It might not be a define after all but a valid, long parameter + * value containing a '$'.*/ + define[1023] = '\0'; + CCTK_VWarn(CCTK_WARN_PICKY, __LINE__, __FILE__, CCTK_THORNSTRING, + "Possible define too long: %s", define); + indef = 0; + defpos = 0; + define[0] = '\0'; + def_start = 0; + } + else + define[defpos++] = c; + } + pos++; + } + return buffer; +} + + /*@@ + @routine ParseBuffer + @author Paul Walker + @desc + This routine actually parses the parameter file. The + syntax we allow is +
    +
  • a = b +
  • a,b,c = d,e,f +
  • # rest of the line is ignored +
  • x = "string value" +
+ So it is easy to parse +

+ We go through the file looking for stuff and then set + it in the global database using calls to the passed in set_function. + @enddesc + @history + @hdate Tue Jan 12 16:41:36 1999 @hauthor Tom Goodale + @hdesc Moved to CCTK. + Changed to pass data to arbitrary function. + Changed to take a file descriptor rather than a filename. + Use a buffer to parse which can be preprocessed before main parsing + @endhistory + @var buffer + @vdesc The buffer to parse + @vtype char * + @vio in + @vcomment + + @endvar + @var set_function + @vdesc The function to call to set the value of a parameter + @vtype int (*)(const char *, const char *) + @vio in + @vcomment + + @endvar + @var ConfigData + @vdesc Flesh configuration data + @vtype tFleshConfig * + @vio in + @vcomment + + @endvar + + @returntype int + @returndesc + 0 - success + @endreturndesc +@@*/ +int ParseBuffer(char *buffer, + int (*set_function)(const char *, const char *, int), + tFleshConfig *ConfigData) +{ + /* position in buffer */ + unsigned int pos = 0; /* Buffers for parsing from the file */ char *tokens, *value; char *subtoken, *subvalue; @@ -122,10 +548,8 @@ int ntokens; /* Status flags */ int intoken, inval; - /* Current char. Have to make it an int so we can compare with - EOF. See man 3 fgetc - */ - int c; + /* Current char */ + char c; int num_errors; /* number of errors in file parsing */ num_errors = 0; @@ -141,7 +565,7 @@ intoken = 0; inval = 0; - while ((c=fgetc(ifp)) != EOF) + while ((c=buffer[pos++]) != '\0') { #ifdef DEBUG printf("%c",c); @@ -150,7 +574,7 @@ while (c == '#' || c == '!' ) { /* Comment line. So forget rest of line */ - while ((c=fgetc(ifp)) != '\n' && c != '\r' && c != EOF) + while ((c=buffer[pos++]) != '\n' && c != '\r' && c != '\0') { #ifdef DEBUG printf("%c",c); @@ -160,7 +584,7 @@ { lineno++; } - c = fgetc(ifp); + c = buffer[pos++]; #ifdef DEBUG printf("%c",c); #endif @@ -220,7 +644,7 @@ * and check if the value is a string or not. * This parser DOES strip quotes off of the strings. */ - while ((c = fgetc(ifp)) == ' ' || c == '\n' || c == '\r' || c == '\t') + while ((c = buffer[pos++]) == ' ' || c == '\n' || c == '\r' || c == '\t') { #ifdef DEBUG printf("%c",c); @@ -251,7 +675,7 @@ free (tokens); return 1; } - while ((c = fgetc(ifp)) != '"') + while ((c = buffer[pos++]) != '"') { #ifdef DEBUG printf("%c",c); @@ -265,7 +689,7 @@ #endif lineno++; } - else if (c == EOF) + else if (c == '\0') { break; } @@ -274,9 +698,9 @@ if (c == '"') { int is_comment = 0; - while ((c = fgetc(ifp)) != '\n') + while ((c = buffer[pos++]) != '\n') { - if (c == EOF) + if (c == '\0') { break; } @@ -295,7 +719,7 @@ return 1; } } - if (c == '\n' || c == EOF) + if (c == '\n' || c == '\0') { if (c == '\n') lineno++; /* mark it as properly quoted string value */ @@ -319,39 +743,6 @@ #endif set_function(tokens,value, lineno); } - else if (c == '$') - { - /* We got a define */ - /* FIXME: Assume it is a parameter file for now */ - char path[500]; - char *parfile; - - CCTK_ParameterFilename(500,path); - parfile = strrchr (path, '/'); - if (parfile == NULL) - { - parfile = path; - } - else - { - parfile++; - } - /* skip the parameter file extension */ - if (strcmp (parfile + strlen (parfile) - 4, ".par") == 0) - { - parfile[strlen (parfile) - 4] = 0; - } - - /* ignore everything else on the line */ - while (!(c==' ' || c=='\t' || c == '\n' || c == '\r' || c == EOF)) - { - c = fgetc(ifp); -#ifdef DEBUG - printf("%c",c); -#endif - } - set_function(tokens,parfile,lineno); - } else { @@ -362,15 +753,15 @@ /* Simple case. We have an int or a double which contain no spaces! */ - c = fgetc(ifp); + c = buffer[pos++]; #ifdef DEBUG printf("%c",c); #endif - while (!(c==' ' || c=='\t' || c == '\n' || c == '\r' || c == EOF)) + while (!(c==' ' || c=='\t' || c == '\n' || c == '\r' || c == '\0')) { value[p++] = (char)c; CheckBuf(p,lineno); - c = fgetc(ifp); + c = buffer[pos++]; #ifdef DEBUG printf("%c",c); #endif @@ -402,11 +793,11 @@ ntokens-1 commas, stripping spaces, and make a nice little string. */ - c = fgetc(ifp); + c = buffer[pos++]; #ifdef DEBUG printf("%c",c); #endif - while (ncommas < ntokens-1 && c != EOF) + while (ncommas < ntokens-1 && c != '\0') { if (!(c == ' ' || c == '\t' || c == '\n' || c == '\r')) { @@ -414,7 +805,7 @@ CheckBuf(pp,lineno); } if (c == ',') ncommas ++; - c = fgetc(ifp); + c = buffer[pos++]; #ifdef DEBUG printf("%c",c); #endif @@ -422,7 +813,7 @@ if (c == ' ' || c == '\t') { /* Great now strip out the spaces */ - while((c = fgetc(ifp)) == ' ' || c=='\t' || c == '\n' || c == '\r') + while((c = buffer[pos++]) == ' ' || c=='\t' || c == '\n' || c == '\r') { #ifdef DEBUG printf("%c",c); @@ -441,15 +832,15 @@ value[pp++] = (char)c; CheckBuf(p,lineno); - c = fgetc(ifp); + c = buffer[pos++]; #ifdef DEBUG printf("%c",c); #endif - while (c != ' ' && c != '\t' && c != '\n' && c != '\r' && c != EOF) + while (c != ' ' && c != '\t' && c != '\n' && c != '\r' && c != '\0') { value[pp++] = (char)c; CheckBuf(pp,lineno); - c = fgetc(ifp); + c = buffer[pos++]; #ifdef DEBUG printf("%c",c); #endif @@ -525,137 +916,3 @@ return num_errors; } -/******************************************************************** - ********************* Local Routines ************************* - ********************************************************************/ - - /*@@ - @routine CheckBuf - @author Paul Walker - @desc - A simple description and warning message in case of - a fixed parse buffer overflow. - @enddesc - @calls - @calledby - @history - - @endhistory - @var p - @vdesc buffer location - @vtype int - @vio in - @vcomment - - @endvar - @var l - @vdesc Line number - @vtype int - @vio in - @vcomment - - @endvar - -@@*/ - -static void CheckBuf(int p, int l) -{ - if (p >= BUF_SZ) - { - fprintf(stderr,"WARNING: Parser buffer overflow on line %d\n", - l); - fprintf(stderr,"This indicates either an incorrect parm file or\n"); - fprintf(stderr,"the need to recompile " __FILE__ " with a bigger\n"); - fprintf(stderr,"BUF_SZ parm.\n"); - - assert(0); - exit(1); - } -} - - - /*@@ - @routine removeSpaces - @author Paul Walker - @desc - removes the spaces from a char * in place. Beware - that this function will change the input value and if you - want to keep a copy you have to do so yourself! - @enddesc - @calls - @calledby - @history - - @endvar - @var stripMe - @vdesc String to strip - @vtype char * - @vio inout - @vcomment - - @endvar - -@@*/ -static void removeSpaces(char *stripMe) -{ - char *s; - unsigned int i,j; - s = (char *)malloc((strlen(stripMe)+2)*sizeof(char)); - - if(s) - { - strcpy(s,stripMe); - for (i=0,j=0;i 1) - { - parameter_file = fopen(argv[1], "r"); - if(parameter_file) - { - ParseFile(parameter_file, parameter_printer); - fclose(parameter_file); - retval = 0; - } - else - { - retval=2; - } - } - else - { - printf("Usage: %s \n", argv[0]); - retval = 1; - }; - - return 0; -} - -#endif From schnetter at cct.lsu.edu Thu Sep 10 21:21:55 2009 From: schnetter at cct.lsu.edu (Erik Schnetter) Date: Fri, 11 Sep 2009 10:21:55 +0800 Subject: [Patches] Output parameter warnings to stdout as well Message-ID: <842E38A7-FC1E-4A10-B559-0A768FC8B577@cct.lsu.edu> The enclosed patch outputs parameter warnings and parameter errors to stdout as well. Different from regular warnings, they are currently only output to stderr. This is inconvenient, since stderr collects warnings from all processors, and thus contains many repetitions of the same error messages. The patch also cleans up the code a bit. Instead of implementing parameter warnings twice in CCTK_ParamWarn and CCTK_VParamWarn, the function CCTK_ParamWarn now calls CCTK_VParamWarn. This removes the need for the function print_bold_stderr. Furthermore it removes an unnecessary call to va_copy, and adds missing calls to va_end (avoiding potential memory leaks). -erik -- Erik Schnetter http://www.cct.lsu.edu/~eschnett/ My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from . -------------- next part -------------- A non-text attachment was scrubbed... Name: WarnLevel.c.diff Type: application/octet-stream Size: 4627 bytes Desc: not available Url : http://www.cactuscode.org/pipermail/patches/attachments/20090911/e66a4f43/attachment.obj From knarf at cct.lsu.edu Fri Sep 11 08:46:35 2009 From: knarf at cct.lsu.edu (Frank Loeffler) Date: Fri, 11 Sep 2009 08:46:35 -0500 Subject: [Patches] [Developers] Output parameter warnings to stdout as well In-Reply-To: <842E38A7-FC1E-4A10-B559-0A768FC8B577@cct.lsu.edu> References: <842E38A7-FC1E-4A10-B559-0A768FC8B577@cct.lsu.edu> Message-ID: <20090911134634.GH11625@numrel07.cct.lsu.edu> Hi, On Fri, Sep 11, 2009 at 10:21:55AM +0800, Erik Schnetter wrote: > The enclosed patch outputs parameter warnings and parameter errors to > stdout as well. Different from regular warnings, they are currently > only output to stderr. This is inconvenient, since stderr collects > warnings from all processors, and thus contains many repetitions of > the same error messages. This creates another inconvenience: when simply run interactivly, there is now one more warning output, in addition to all the others from stderr. Even when running on with one process a warning is now printed twice. I don't think this is optimal. I wonder if it wouldn't be better to only emit warnings for parameter file settings once, on the root processor, and only print them to stderr. Each process parses the same parameter file and should come to the same warnings anyway. The issue with multiple messages from the parameter file parsing wouldn't be solved by that directly however, because this code calls fprintf on stderr directly instead of CCTK_ParamWarn. This could and should be changed, but I would prefer to postpone this until the other patch for that file has been discussed and commited, hopefully only a couple of days. > The patch also cleans up the code a bit. That is very much appreciated. Frank From knarf at cct.lsu.edu Thu Sep 17 20:51:05 2009 From: knarf at cct.lsu.edu (Frank Loeffler) Date: Thu, 17 Sep 2009 20:51:05 -0500 Subject: [Patches] improve parser for parameter files (2) In-Reply-To: <20090911015042.GC11625@numrel07.cct.lsu.edu> References: <20090910195426.GA11625@numrel07.cct.lsu.edu> <8B2F1374-1B31-45BF-82E5-B3F27970212E@cct.lsu.edu> <20090911015042.GC11625@numrel07.cct.lsu.edu> Message-ID: <20090918015104.GG5912@numrel07.cct.lsu.edu> Hi, On Thu, Sep 10, 2009 at 08:50:43PM -0500, Frank Loeffler wrote: > As for the syntax of the environment variables ($ENV_* at the moment), we > could certainly change that to either $ENV{*} or $ENV{'*'}. Pick your > favorite. Since nobody else did, I picked one: This patch improves the parser used by Cactus to prepocess parameter files. The main changes are: - when reading a parameter file do not parse the file while reading, but read it first into a buffer, preprocess that and parse the buffer after that - replace the code which changes $parfile into the parameter file name to use this buffer preprocessing - replace $ENV{'*'} "defines" by environment variable values (*), e.g. $ENV{'HOME'} with $HOME. Frank -------------- next part -------------- Index: ParseFile.c =================================================================== RCS file: /cactusdevcvs/Cactus/src/util/ParseFile.c,v retrieving revision 1.26 diff -u -r1.26 ParseFile.c --- ParseFile.c 31 Aug 2006 16:54:40 -0000 1.26 +++ ParseFile.c 18 Sep 2009 01:43:53 -0000 @@ -7,7 +7,6 @@ to a user-supplied subroutine. Currently taken from the old cactus ones and slightly modifed. @enddesc - @version $Id: ParseFile.c,v 1.26 2006/08/31 16:54:40 tradke Exp $ @@*/ /*#define DEBUG*/ @@ -15,6 +14,9 @@ #include #include #include +#include + +#include #include "cctk_Flesh.h" #include "cctk_CommandLine.h" @@ -38,6 +40,11 @@ static void CheckBuf(int, int); static void removeSpaces(char *stripMe); +static char *ReadFile(FILE *file, unsigned long *filesize); +static char *ParseDefines(char *buffer, unsigned long *buffersize); +int ParseBuffer(char *buffer, + int (*set_function)(const char *, const char *, int), + tFleshConfig *ConfigData); /******************************************************************** ********************* Other Routine Prototypes ********************* @@ -63,7 +70,7 @@ /*@@ @routine ParseFile - @author Paul Walker + @author Paul Walker, Frank Loeffler @desc This routine actually parses the parameter file. The syntax we allow is @@ -83,6 +90,7 @@ @hdesc Moved to CCTK. Changed to pass data to arbitrary function. Changed to take a file descriptor rather than a filename. + Use a buffer to parse which can be preprocessed before main parsing @endhistory @var ifp @vdesc The filestream to parse @@ -111,10 +119,434 @@ 0 - success @endreturndesc @@*/ + int ParseFile(FILE *ifp, int (*set_function)(const char *, const char *, int), tFleshConfig *ConfigData) { + int retval; + unsigned long buffersize; + char *buffer = ReadFile(ifp, &buffersize); + if (!buffer) + return 1; + buffer = ParseDefines(buffer, &buffersize); + retval = ParseBuffer(buffer, set_function, ConfigData); + free(buffer); + return retval; +} + +/******************************************************************** + ********************* Local Routines ************************* + ********************************************************************/ + + /*@@ + @routine CheckBuf + @author Paul Walker + @desc + A simple description and warning message in case of + a fixed parse buffer overflow. + @enddesc + @calls + @calledby + @history + + @endhistory + @var p + @vdesc buffer location + @vtype int + @vio in + @vcomment + + @endvar + @var l + @vdesc Line number + @vtype int + @vio in + @vcomment + + @endvar + +@@*/ + +static void CheckBuf(int p, int l) +{ + if (p >= BUF_SZ) + { + fprintf(stderr,"WARNING: Parser buffer overflow on line %d\n", + l); + fprintf(stderr,"This indicates either an incorrect parm file or\n"); + fprintf(stderr,"the need to recompile " __FILE__ " with a bigger\n"); + fprintf(stderr,"BUF_SZ parm.\n"); + + assert(0); + exit(1); + } +} + + + /*@@ + @routine removeSpaces + @author Paul Walker + @desc + removes the spaces from a char * in place. Beware + that this function will change the input value and if you + want to keep a copy you have to do so yourself! + @enddesc + @calls + @calledby + @history + + @endvar + @var stripMe + @vdesc String to strip + @vtype char * + @vio inout + @vcomment + + @endvar + +@@*/ +static void removeSpaces(char *stripMe) +{ + char *s; + unsigned int i,j; + s = (char *)malloc((strlen(stripMe)+2)*sizeof(char)); + + if(s) + { + strcpy(s,stripMe); + for (i=0,j=0;i 1) + { + parameter_file = fopen(argv[1], "r"); + if(parameter_file) + { + ParseFile(parameter_file, parameter_printer); + fclose(parameter_file); + retval = 0; + } + else + { + retval=2; + } + } + else + { + printf("Usage: %s \n", argv[0]); + retval = 1; + }; + + return 0; +} + +#endif + + + /*@@ + @routine ReadFile + @author Frank Loeffler + @desc + This routine reads a file into a buffer + @enddesc + @history + @var file + @vdesc The filestream to read + @vtype FILE * + @vio in + @vcomment + + @endvar + @var filesize + @vdesc The size of the file + @vtype *unsigned long + @vio out + @vcomment + + @endvar + + @returntype char * + @returndesc + NULL - failure + !NULL allocated buffer + @endreturndesc +@@*/ +static char *ReadFile(FILE *file, unsigned long *filesize) +{ + char *buffer; + + if (!file) + { + fprintf(stderr, "Could not use file for reading.\n"); + return NULL; + } + /* Get the file size */ + fseek(file, 0, SEEK_END); + *filesize = ftell(file); + fseek(file, 0, SEEK_SET); + /* Allocate buffer */ + buffer = (char *)malloc(*filesize+1); + if (!buffer) + { + fprintf(stderr, "Could not allocate memory.\n"); + return NULL; + } + /* Read file into buffer and return */ + fread(buffer, *filesize, 1, file); + /* Protect buffer for string operations */ + buffer[*filesize] = '\0'; + return buffer; +} + + /*@@ + @routine ParseDefines + @author Frank Loeffler + @desc + This routine parses a buffer for defines of the form \$[^ \n] and + replaces them with some predefined defines, returning the new buffer, + which might be the same as the old buffer. Otherwise the old buffer + gets freed by this function. In case of parsing errors, the whole + parsing is stopped and the old buffer is returned + (FIXME: Should we return NULL?) + @enddesc + @history + @var buffer + @vdesc The buffer to parse + @vtype char * + @vio in + @vcomment + + @endvar + @var buffersize + @vdesc The size of the buffer + @vtype *unsigned long + @vio out + @vcomment + + @endvar + + @returntype char * + @returndesc + !NULL - new buffer, might be == buffer + @endreturndesc +@@*/ +static char *ParseDefines(char *buffer, unsigned long *buffersize) +{ + /* define name */ + char define[1024]; + /* Position in define name */ + char defpos = 0; + /* Current position in buffer */ + size_t pos = 0; + /* Character at current position */ + char c; + /* Position of start of found define */ + size_t def_start = 0; + /* Flag to indicate if we are inside a definition name */ + int indef = 0; + if (!buffer) + return buffer; + /* Main loop over the buffer */ + while (((c=buffer[pos]) != '\0') && (pos < *buffersize) ) + { + /* Check if we found a define */ + if (c == '$') + { + /* Mark the state and location and go on */ + indef = 1; + def_start = pos; + } + /* If we are inside a define */ + else if (indef) + { + /* Look for the end of the define name */ + if (c == '}' || c == '\0' || c == '\n') + { + /* make the define name a proper string */ + define[defpos] = '\0'; + /* this holds the value of the define to be filled in */ + char *value = NULL; + /* a new buffer for the whole thing */ + char * new_buffer; + unsigned long new_size; + /* Check for different kinds of defines */ + /* Environment variables */ + if ((defpos > 6) && + (strncmp(define, "ENV{'", 5) == 0) && + (define[defpos-1] == '\'')) + { + /* overwrite the trailing ' */ + define[defpos-1] = '\0'; + value = getenv(define+5); + if (!value) + { + CCTK_VWarn(CCTK_WARN_ALERT, __LINE__, __FILE__, CCTK_THORNSTRING, + "No environment variable %s found\n", define+5); + /* TODO: Should we abort here? (return NULL) */ + } + /* increase pos to jump over the trailing } */ + if (pos+1 < *buffersize) { pos++; } + } + /* Parameter file name ? */ + else if (strcmp(define, "parfile") == 0) + { + char path[500]; + CCTK_ParameterFilename(500, path); + value = strrchr (path, '/'); + if (value == NULL) + { + value = path; + } + else + { + value++; + } + /* skip the parameter file extension */ + if (strcmp (value + strlen (value) - 4, ".par") == 0) + { + value[strlen (value) - 4] = '\0'; + } + } + /* Else: unknown define - or no define at all: ignore */ + if (value) + { + /* Replace the old buffer with the new, copying in all the data */ + new_size = *buffersize - strlen(define) + strlen(value) + 1; + if (new_size < def_start) + { + CCTK_WARN(0, "Something is wrong with me, HELP!"); + return buffer; + } + new_buffer = (char *)malloc(new_size); + if (!new_buffer) + { + CCTK_WARN(0, "I am out of memory and give up parsing for defines."); + return buffer; + } + new_buffer[0]='\0'; + strncat(new_buffer, buffer, def_start); + strncat(new_buffer, value, new_size-1); + strncat(new_buffer, buffer+pos, new_size-1); + new_buffer[new_size-1] = '\0'; + /* free the old buffer */ + free(buffer); + /* Start further processing of new buffer */ + pos = def_start + strlen(define); + buffer = new_buffer; + *buffersize = new_size; + } + /* General define cleanup */ + indef = 0; + defpos = 0; + define[0] = '\0'; + def_start = 0; + } + else if (defpos > 1023) + { + /* We only print this warning and ignore the possible define problem. + * It might not be a define after all but a valid, long parameter + * value containing a '$'.*/ + define[1023] = '\0'; + CCTK_VWarn(CCTK_WARN_PICKY, __LINE__, __FILE__, CCTK_THORNSTRING, + "Possible define too long: %s", define); + indef = 0; + defpos = 0; + define[0] = '\0'; + def_start = 0; + } + else + define[defpos++] = c; + } + pos++; + } + return buffer; +} + + /*@@ + @routine ParseBuffer + @author Paul Walker + @desc + This routine actually parses the parameter file. The + syntax we allow is +

    +
  • a = b +
  • a,b,c = d,e,f +
  • # rest of the line is ignored +
  • x = "string value" +
+ So it is easy to parse +

+ We go through the file looking for stuff and then set + it in the global database using calls to the passed in set_function. + @enddesc + @history + @hdate Tue Jan 12 16:41:36 1999 @hauthor Tom Goodale + @hdesc Moved to CCTK. + Changed to pass data to arbitrary function. + Changed to take a file descriptor rather than a filename. + Use a buffer to parse which can be preprocessed before main parsing + @endhistory + @var buffer + @vdesc The buffer to parse + @vtype char * + @vio in + @vcomment + + @endvar + @var set_function + @vdesc The function to call to set the value of a parameter + @vtype int (*)(const char *, const char *) + @vio in + @vcomment + + @endvar + @var ConfigData + @vdesc Flesh configuration data + @vtype tFleshConfig * + @vio in + @vcomment + + @endvar + + @returntype int + @returndesc + 0 - success + @endreturndesc +@@*/ +int ParseBuffer(char *buffer, + int (*set_function)(const char *, const char *, int), + tFleshConfig *ConfigData) +{ + /* position in buffer */ + unsigned int pos = 0; /* Buffers for parsing from the file */ char *tokens, *value; char *subtoken, *subvalue; @@ -122,10 +554,8 @@ int ntokens; /* Status flags */ int intoken, inval; - /* Current char. Have to make it an int so we can compare with - EOF. See man 3 fgetc - */ - int c; + /* Current char */ + char c; int num_errors; /* number of errors in file parsing */ num_errors = 0; @@ -141,7 +571,7 @@ intoken = 0; inval = 0; - while ((c=fgetc(ifp)) != EOF) + while ((c=buffer[pos++]) != '\0') { #ifdef DEBUG printf("%c",c); @@ -150,7 +580,7 @@ while (c == '#' || c == '!' ) { /* Comment line. So forget rest of line */ - while ((c=fgetc(ifp)) != '\n' && c != '\r' && c != EOF) + while ((c=buffer[pos++]) != '\n' && c != '\r' && c != '\0') { #ifdef DEBUG printf("%c",c); @@ -160,7 +590,7 @@ { lineno++; } - c = fgetc(ifp); + c = buffer[pos++]; #ifdef DEBUG printf("%c",c); #endif @@ -220,7 +650,7 @@ * and check if the value is a string or not. * This parser DOES strip quotes off of the strings. */ - while ((c = fgetc(ifp)) == ' ' || c == '\n' || c == '\r' || c == '\t') + while ((c = buffer[pos++]) == ' ' || c == '\n' || c == '\r' || c == '\t') { #ifdef DEBUG printf("%c",c); @@ -251,7 +681,7 @@ free (tokens); return 1; } - while ((c = fgetc(ifp)) != '"') + while ((c = buffer[pos++]) != '"') { #ifdef DEBUG printf("%c",c); @@ -265,7 +695,7 @@ #endif lineno++; } - else if (c == EOF) + else if (c == '\0') { break; } @@ -274,9 +704,9 @@ if (c == '"') { int is_comment = 0; - while ((c = fgetc(ifp)) != '\n') + while ((c = buffer[pos++]) != '\n') { - if (c == EOF) + if (c == '\0') { break; } @@ -295,7 +725,7 @@ return 1; } } - if (c == '\n' || c == EOF) + if (c == '\n' || c == '\0') { if (c == '\n') lineno++; /* mark it as properly quoted string value */ @@ -319,39 +749,6 @@ #endif set_function(tokens,value, lineno); } - else if (c == '$') - { - /* We got a define */ - /* FIXME: Assume it is a parameter file for now */ - char path[500]; - char *parfile; - - CCTK_ParameterFilename(500,path); - parfile = strrchr (path, '/'); - if (parfile == NULL) - { - parfile = path; - } - else - { - parfile++; - } - /* skip the parameter file extension */ - if (strcmp (parfile + strlen (parfile) - 4, ".par") == 0) - { - parfile[strlen (parfile) - 4] = 0; - } - - /* ignore everything else on the line */ - while (!(c==' ' || c=='\t' || c == '\n' || c == '\r' || c == EOF)) - { - c = fgetc(ifp); -#ifdef DEBUG - printf("%c",c); -#endif - } - set_function(tokens,parfile,lineno); - } else { @@ -362,15 +759,15 @@ /* Simple case. We have an int or a double which contain no spaces! */ - c = fgetc(ifp); + c = buffer[pos++]; #ifdef DEBUG printf("%c",c); #endif - while (!(c==' ' || c=='\t' || c == '\n' || c == '\r' || c == EOF)) + while (!(c==' ' || c=='\t' || c == '\n' || c == '\r' || c == '\0')) { value[p++] = (char)c; CheckBuf(p,lineno); - c = fgetc(ifp); + c = buffer[pos++]; #ifdef DEBUG printf("%c",c); #endif @@ -402,11 +799,11 @@ ntokens-1 commas, stripping spaces, and make a nice little string. */ - c = fgetc(ifp); + c = buffer[pos++]; #ifdef DEBUG printf("%c",c); #endif - while (ncommas < ntokens-1 && c != EOF) + while (ncommas < ntokens-1 && c != '\0') { if (!(c == ' ' || c == '\t' || c == '\n' || c == '\r')) { @@ -414,7 +811,7 @@ CheckBuf(pp,lineno); } if (c == ',') ncommas ++; - c = fgetc(ifp); + c = buffer[pos++]; #ifdef DEBUG printf("%c",c); #endif @@ -422,7 +819,7 @@ if (c == ' ' || c == '\t') { /* Great now strip out the spaces */ - while((c = fgetc(ifp)) == ' ' || c=='\t' || c == '\n' || c == '\r') + while((c = buffer[pos++]) == ' ' || c=='\t' || c == '\n' || c == '\r') { #ifdef DEBUG printf("%c",c); @@ -441,15 +838,15 @@ value[pp++] = (char)c; CheckBuf(p,lineno); - c = fgetc(ifp); + c = buffer[pos++]; #ifdef DEBUG printf("%c",c); #endif - while (c != ' ' && c != '\t' && c != '\n' && c != '\r' && c != EOF) + while (c != ' ' && c != '\t' && c != '\n' && c != '\r' && c != '\0') { value[pp++] = (char)c; CheckBuf(pp,lineno); - c = fgetc(ifp); + c = buffer[pos++]; #ifdef DEBUG printf("%c",c); #endif @@ -525,137 +922,3 @@ return num_errors; } -/******************************************************************** - ********************* Local Routines ************************* - ********************************************************************/ - - /*@@ - @routine CheckBuf - @author Paul Walker - @desc - A simple description and warning message in case of - a fixed parse buffer overflow. - @enddesc - @calls - @calledby - @history - - @endhistory - @var p - @vdesc buffer location - @vtype int - @vio in - @vcomment - - @endvar - @var l - @vdesc Line number - @vtype int - @vio in - @vcomment - - @endvar - -@@*/ - -static void CheckBuf(int p, int l) -{ - if (p >= BUF_SZ) - { - fprintf(stderr,"WARNING: Parser buffer overflow on line %d\n", - l); - fprintf(stderr,"This indicates either an incorrect parm file or\n"); - fprintf(stderr,"the need to recompile " __FILE__ " with a bigger\n"); - fprintf(stderr,"BUF_SZ parm.\n"); - - assert(0); - exit(1); - } -} - - - /*@@ - @routine removeSpaces - @author Paul Walker - @desc - removes the spaces from a char * in place. Beware - that this function will change the input value and if you - want to keep a copy you have to do so yourself! - @enddesc - @calls - @calledby - @history - - @endvar - @var stripMe - @vdesc String to strip - @vtype char * - @vio inout - @vcomment - - @endvar - -@@*/ -static void removeSpaces(char *stripMe) -{ - char *s; - unsigned int i,j; - s = (char *)malloc((strlen(stripMe)+2)*sizeof(char)); - - if(s) - { - strcpy(s,stripMe); - for (i=0,j=0;i 1) - { - parameter_file = fopen(argv[1], "r"); - if(parameter_file) - { - ParseFile(parameter_file, parameter_printer); - fclose(parameter_file); - retval = 0; - } - else - { - retval=2; - } - } - else - { - printf("Usage: %s \n", argv[0]); - retval = 1; - }; - - return 0; -} - -#endif -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: not available Url : http://www.cactuscode.org/pipermail/patches/attachments/20090917/4094cc95/attachment-0001.bin From schnetter at cct.lsu.edu Tue Sep 22 14:15:56 2009 From: schnetter at cct.lsu.edu (Erik Schnetter) Date: Tue, 22 Sep 2009 14:15:56 -0500 Subject: [Patches] Support AMR in IOJpeg Message-ID: The enclosed patch supports AMR with IOJpeg. It introduces a new keyword parameter "gridpoints" with possible values "hyperslab" and "interpolate". Hyperslabbing works only on uniform grids, whereas interpolation also works with AMR. Additional parameters select where in the domain the points should be interpolated. -erik -- Erik Schnetter http://www.cct.lsu.edu/~eschnett/ My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from . -------------- next part -------------- A non-text attachment was scrubbed... Name: IOJpeg.diff Type: application/octet-stream Size: 18448 bytes Desc: not available Url : http://www.cactuscode.org/pipermail/patches/attachments/20090922/ee0ee687/attachment.obj From schnetter at cct.lsu.edu Fri Sep 25 10:04:29 2009 From: schnetter at cct.lsu.edu (Erik Schnetter) Date: Fri, 25 Sep 2009 10:04:29 -0500 Subject: [Patches] CactusEinstein/TmunuBase Message-ID: <33850118-B9BF-4BFF-9A55-C523C7609DAA@cct.lsu.edu> I propose to move the thorn TmunuBase to CactusEinstein. In addition to the version currently available from AEIThorns, I propose to add a new parameter support_old_CalcTmunu_mechanism (defaulting to yes). This parameter can be used to disable support for the CalcTmunu.inc mechanism, reducing storage and potentially increasing efficiency. -erik -- Erik Schnetter http://www.cct.lsu.edu/~eschnett/ My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from . -------------- next part -------------- A non-text attachment was scrubbed... Name: TmunuBase.tar.gz Type: application/x-gzip Size: 13104 bytes Desc: not available Url : http://www.cactuscode.org/pipermail/patches/attachments/20090925/087370b2/attachment-0001.gz From knarf at cct.lsu.edu Mon Sep 28 09:11:55 2009 From: knarf at cct.lsu.edu (Frank Loeffler) Date: Mon, 28 Sep 2009 09:11:55 -0500 Subject: [Patches] check available timelevels for static boundary condition Message-ID: <20090928141154.GB2142@numrel07.cct.lsu.edu> Hi, this patch checks the available timelevels for a variable before it attempts a memcpy. In the case of failure, this changes a segfault to an informative message, which makes debugging this case much easier. In case of success, it adds a test every time the static boundary condition is called, for each variable. I am not sure about the overhead this creates (a call to CCTK_ActiveTimeLevelsVI). Frank -------------- next part -------------- Index: StaticBoundary.c =================================================================== RCS file: /cactusdevcvs/CactusBase/Boundary/src/StaticBoundary.c,v retrieving revision 1.20 diff -u -r1.20 StaticBoundary.c --- StaticBoundary.c 5 Oct 2006 16:56:28 -0000 1.20 +++ StaticBoundary.c 28 Sep 2009 14:08:27 -0000 @@ -1018,6 +1018,13 @@ var < first_var + num_vars; var++) { + if (CCTK_ActiveTimeLevelsVI(GH, var) < 2) + { + CCTK_VWarn(0, __LINE__, __FILE__, CCTK_THORNSTRING, + "Static Boundary condition needs at least two timelevels " + "active, but %s only has %d.", + CCTK_VarName(var), CCTK_ActiveTimeLevelsVI(GH, var)); + } /* Apply condition if: + boundary is an outer boundary + have enough grid points -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: not available Url : http://www.cactuscode.org/pipermail/patches/attachments/20090928/331a1b66/attachment.bin From bgiacoma at aei.mpg.de Fri Sep 11 04:18:11 2009 From: bgiacoma at aei.mpg.de (Bruno Giacomazzo) Date: Fri, 11 Sep 2009 09:18:11 -0000 Subject: [Patches] [Developers] Output parameter warnings to stdout as well In-Reply-To: <842E38A7-FC1E-4A10-B559-0A768FC8B577@cct.lsu.edu> References: <842E38A7-FC1E-4A10-B559-0A768FC8B577@cct.lsu.edu> Message-ID: Erik, both me and Luca save stdout and stderr in the same file. I do this so that I'm sure of when (i.e. at which iteration/point of the run) a warning message is generated since the WARNING messages don't contain any information about the iteration at which they were generated. In this case it seems to me that your patch would not solve the problem (at least not for me and Luca). Is it correct? Thanks, Bruno On Fri, 11 Sep 2009, Erik Schnetter wrote: > The enclosed patch outputs parameter warnings and parameter errors to > stdout as well. Different from regular warnings, they are currently > only output to stderr. This is inconvenient, since stderr collects > warnings from all processors, and thus contains many repetitions of > the same error messages. > > The patch also cleans up the code a bit. Instead of implementing > parameter warnings twice in CCTK_ParamWarn and CCTK_VParamWarn, the > function CCTK_ParamWarn now calls CCTK_VParamWarn. This removes the > need for the function print_bold_stderr. > > Furthermore it removes an unnecessary call to va_copy, and adds > missing calls to va_end (avoiding potential memory leaks). > > -erik > > -- Dr. Bruno Giacomazzo Max Planck Institute for Gravitational Physics Albert Einstein Institute Am Muehlenberg 1 D-14476 Potsdam Germany Tel. : +49 331 567 7183 Fax : +49 331 567 7298 cell. : +49 173 826 4488 email : bgiacoma at aei.mpg.de ------------------------------------------------- There are only 10 types of people in the world: Those who understand binary, and those who don't -------------------------------------------------