From schnetter at cct.lsu.edu  Wed Nov  4 16:22:09 2009
From: schnetter at cct.lsu.edu (Erik Schnetter)
Date: Wed, 4 Nov 2009 16:22:09 -0600
Subject: [Developers] Detect certain GNU attributes
Message-ID: <9CAA4657-B5E5-48B5-9E72-D510E4CDEE04@cct.lsu.edu>

Detect whether the compiler supports the attributes

	const
	pure
	unused
	cold
	hot

These attributes can be used to annotate declarations of variables,  
functions, and member functions to avoid warnings or enable additional  
optimisations.

Detect whether the _Pragma directive is recognised.  _Pragma can be  
used instead of #pragma, but works also inside macros.

Update the detection of the static, inline, and restrict qualifiers to  
catch certain compiler errors, and avoids using these attributes with  
these compilers.

-erik

-- 
Erik Schnetter <schnetter at cct.lsu.edu>   http://www.cct.lsu.edu/~eschnett/




--- StripMime Report -- processed MIME parts ---
multipart/mixed
  text/plain (text body -- kept)
  application/octet-stream
---

From dprideout at gmail.com  Wed Nov  4 19:22:48 2009
From: dprideout at gmail.com (David Rideout)
Date: Wed, 4 Nov 2009 20:22:48 -0500
Subject: [Developers] controlling the size of grid arrays
Message-ID: <1ce81abb0911041722w4b7aa47bka55442d2e5822d3e@mail.gmail.com>

I would like to declare a grid array whose size is computed by the
activated thorns.  (i.e. its size is a function of the active thorns
list).  I tried setting its size to be given by a dummy parameter in
the interface.ccl, and then steer that parameter's value during
STARTUP (via CCTK_ParameterSet()).  I find that the size is allocated
based on the default value of the dummy parameter, rather than the
steered value.  Is there some way to make this work?

Thanks,
David

From schnetter at cct.lsu.edu  Wed Nov  4 20:23:48 2009
From: schnetter at cct.lsu.edu (Erik Schnetter)
Date: Wed, 4 Nov 2009 20:23:48 -0600
Subject: [Developers] LoopControl
Message-ID: <17021ADC-0711-4894-896F-BA14E4D8AA13@cct.lsu.edu>

I'm thinking of generalising the LoopControl thorn of Carpet, and to  
propose it as generic looping mechanism to be included into the Cactus  
flesh.  The flesh would contain than a simple, straightforward  
implementation, and other thorns could contain various optimisations,  
e.g. loop tiling, as currently implemented by Carpet's LoopControl.

I attach below a header file defining macros for loops.  There are  
macros for 1D, 2D, 3D, and 4D loops, corresponding to Cactus'  
CCTK_GFINDEX* macros.  Each macro comes in four variants:

	CCTK_LOOP3: a generic 3D loop

	CCTK_LOOP3_ALL: loop over all grid points of a grid function

	CCTK_LOOP3_INTERIOR: loop over the interior points, excluding ghost,  
symmetry, and boundary points

	CCTK_LOOP3_BOUNDARY: loop over all (outer) boundary points

The attached header file contains a straightforward implementation via  
macros that expand to corresponding for statements.

There are also macros for Fortran, but these are much less elegant  
since Fortran does not support block-scope local variables.  In  
addition, Fortran requires declaring certain variables as private when  
OpenMP is used, which adds another complication.

Please comment.

-erik

-- 
Erik Schnetter <schnetter at cct.lsu.edu>   http://www.cct.lsu.edu/~eschnett/


-------------- next part --------------
A non-text attachment was scrubbed...
Name: cctk_Loop.h
Type: application/octet-stream
Size: 25157 bytes
Desc: not available
Url : http://www.cactuscode.org/pipermail/developers/attachments/20091104/c016787e/attachment-0001.obj 

From schnetter at cct.lsu.edu  Wed Nov  4 21:06:24 2009
From: schnetter at cct.lsu.edu (Erik Schnetter)
Date: Wed, 4 Nov 2009 21:06:24 -0600
Subject: [Developers] controlling the size of grid arrays
In-Reply-To: <1ce81abb0911041722w4b7aa47bka55442d2e5822d3e@mail.gmail.com>
References: <1ce81abb0911041722w4b7aa47bka55442d2e5822d3e@mail.gmail.com>
Message-ID: <DE9DE828-A153-4F43-ACD6-6B388D15AE43@cct.lsu.edu>

On Nov 4, 2009, at 19:22 , David Rideout wrote:

> I would like to declare a grid array whose size is computed by the
> activated thorns.  (i.e. its size is a function of the active thorns
> list).  I tried setting its size to be given by a dummy parameter in
> the interface.ccl, and then steer that parameter's value during
> STARTUP (via CCTK_ParameterSet()).  I find that the size is allocated
> based on the default value of the dummy parameter, rather than the
> steered value.  Is there some way to make this work?


There is a different mechanism that you can use instead of changing  
the parameter manually.  This mechanism is used e.g. by MoL to  
allocate scratch space for all evolved variables, where the number of  
evolved variables depends on which thorns are active.  The mechanism  
is called "accumulator parameters".

Basically, the thorn declaring the grid array declares a parameter  
with a certain default value.  For example, MoL declares a parameter

CCTK_INT MoL_Num_Evolved_Vars "The maximum number of variables to be  
evolved by MoL" ACCUMULATOR = (x+y)

The "accumulator" keyword means that this parameter cannot be set  
directly; instead, other thorns can accumulate onto (in this case: add  
onto) this parameter.  The thorn CactusExamples/WaveMoL declares

shares: MethodOfLines
USES CCTK_INT MoL_Num_Evolved_Vars

restricted:
CCTK_INT WaveMoL_MaxNumEvolvedVars "The maximum number of evolved  
variables used by WaveMoL" ACCUMULATOR- 
BASE=MethodofLines::MoL_Num_Evolved_Vars

where the value of the new parameter WaveMoL_MaxNumEvolvedVars is  
added to MoL's parameter.  The new parameter WaveMoL_MaxNumEvolvedVars  
has a default value, and can also be set in the parameter file if  
necessary.

-erik

-- 
Erik Schnetter <schnetter at cct.lsu.edu>   http://www.cct.lsu.edu/~eschnett/




From schnetter at cct.lsu.edu  Wed Nov  4 21:46:54 2009
From: schnetter at cct.lsu.edu (Erik Schnetter)
Date: Wed, 4 Nov 2009 21:46:54 -0600
Subject: [Developers] Add new modes global_early and global_late
Message-ID: <2233E246-0708-4F1A-9253-A46D76E36FDE@cct.lsu.edu>

Add flesh support for new modes global_early and global_late, as well  
as meta_early and meta_late.  These modes are parsed by the flesh, and  
are passed to the driver; support in the driver has to be provided  
independently

Output the mode in the schedule tree.

-erik

-- 
Erik Schnetter <schnetter at cct.lsu.edu>   http://www.cct.lsu.edu/~eschnett/




--- StripMime Report -- processed MIME parts ---
multipart/mixed
  text/plain (text body -- kept)
  application/octet-stream
---

From dprideout at gmail.com  Thu Nov  5 10:47:34 2009
From: dprideout at gmail.com (David Rideout)
Date: Thu, 5 Nov 2009 11:47:34 -0500
Subject: [Developers] controlling the size of grid arrays
In-Reply-To: <DE9DE828-A153-4F43-ACD6-6B388D15AE43@cct.lsu.edu>
References: <1ce81abb0911041722w4b7aa47bka55442d2e5822d3e@mail.gmail.com> 
	<DE9DE828-A153-4F43-ACD6-6B388D15AE43@cct.lsu.edu>
Message-ID: <1ce81abb0911050847y7fb3c2dbp845e1cb85dcb1839@mail.gmail.com>

Thanks.

Can the accumulator parameter's value end up being the maximum of the
parameters which contribute to it?  e.g.

int Nmax accumulator=max(x,y)

Unfortunately the values of y which contribute to it is not given
directly by parameter values, but is a function of other parameters
which involves square roots.  Is there some way to handle this?


Is there any design requirement which is preventing us from having a
much more flexible specification of the sizes of Cactus grid
variables?  e.g. I have always thought it strange that the size of
grid functions (and number of ghost zones!) is specified directly by
parameters owned by the driver.  In general the size of grid variables
should be computed by thorns, not given directly by parameters.  In
fact I set PUGH's global_nx manually during STARTUP with

ierr = CCTK_ParameterSet("global_nx", "PUGH", GFsizest);

but this same trick is not working for grid arrays.  Also it is driver
dependent which is ugly.  How does one specify the size of grid
functions in Carpet?

Thanks,
David


On Wed, Nov 4, 2009 at 10:06 PM, Erik Schnetter <schnetter at cct.lsu.edu> wrote:
> On Nov 4, 2009, at 19:22 , David Rideout wrote:
>
>> I would like to declare a grid array whose size is computed by the
>> activated thorns.  (i.e. its size is a function of the active thorns
>> list).  I tried setting its size to be given by a dummy parameter in
>> the interface.ccl, and then steer that parameter's value during
>> STARTUP (via CCTK_ParameterSet()).  I find that the size is allocated
>> based on the default value of the dummy parameter, rather than the
>> steered value.  Is there some way to make this work?
>
>
> There is a different mechanism that you can use instead of changing
> the parameter manually.  This mechanism is used e.g. by MoL to
> allocate scratch space for all evolved variables, where the number of
> evolved variables depends on which thorns are active.  The mechanism
> is called "accumulator parameters".
>
> Basically, the thorn declaring the grid array declares a parameter
> with a certain default value.  For example, MoL declares a parameter
>
> CCTK_INT MoL_Num_Evolved_Vars "The maximum number of variables to be
> evolved by MoL" ACCUMULATOR = (x+y)
>
> The "accumulator" keyword means that this parameter cannot be set
> directly; instead, other thorns can accumulate onto (in this case: add
> onto) this parameter.  The thorn CactusExamples/WaveMoL declares
>
> shares: MethodOfLines
> USES CCTK_INT MoL_Num_Evolved_Vars
>
> restricted:
> CCTK_INT WaveMoL_MaxNumEvolvedVars "The maximum number of evolved
> variables used by WaveMoL" ACCUMULATOR-
> BASE=MethodofLines::MoL_Num_Evolved_Vars
>
> where the value of the new parameter WaveMoL_MaxNumEvolvedVars is
> added to MoL's parameter.  The new parameter WaveMoL_MaxNumEvolvedVars
> has a default value, and can also be set in the parameter file if
> necessary.
>
> -erik
>
> --
> Erik Schnetter <schnetter at cct.lsu.edu>   http://www.cct.lsu.edu/~eschnett/
>
>
>
> _______________________________________________
> Developers mailing list
> Developers at cactuscode.org
> http://www.cactuscode.org/mailman/listinfo/developers
>

From schnetter at cct.lsu.edu  Thu Nov  5 12:52:16 2009
From: schnetter at cct.lsu.edu (Erik Schnetter)
Date: Thu, 5 Nov 2009 12:52:16 -0600
Subject: [Developers] controlling the size of grid arrays
In-Reply-To: <1ce81abb0911050847y7fb3c2dbp845e1cb85dcb1839@mail.gmail.com>
References: <1ce81abb0911041722w4b7aa47bka55442d2e5822d3e@mail.gmail.com>
	<DE9DE828-A153-4F43-ACD6-6B388D15AE43@cct.lsu.edu>
	<1ce81abb0911050847y7fb3c2dbp845e1cb85dcb1839@mail.gmail.com>
Message-ID: <3800E924-295B-4CC5-9A2B-0FEA3EC11099@cct.lsu.edu>

On Nov 5, 2009, at 10:47 , David Rideout wrote:

> Thanks.
>
> Can the accumulator parameter's value end up being the maximum of the
> parameters which contribute to it?  e.g.
>
> int Nmax accumulator=max(x,y)
>
> Unfortunately the values of y which contribute to it is not given
> directly by parameter values, but is a function of other parameters
> which involves square roots.  Is there some way to handle this?
>
>
> Is there any design requirement which is preventing us from having a
> much more flexible specification of the sizes of Cactus grid
> variables?  e.g. I have always thought it strange that the size of
> grid functions (and number of ghost zones!) is specified directly by
> parameters owned by the driver.  In general the size of grid variables
> should be computed by thorns, not given directly by parameters.  In
> fact I set PUGH's global_nx manually during STARTUP with
>
> ierr = CCTK_ParameterSet("global_nx", "PUGH", GFsizest);
>
> but this same trick is not working for grid arrays.  Also it is driver
> dependent which is ugly.  How does one specify the size of grid
> functions in Carpet?


Both Carpet and PUGH determine the sizes of grid functions on their  
own.  You can set Carpet::global_nx, but there are also other  
methods.  And PUGH also supports setting PUGH::local_nx, for example.

Instead of parameters we could also allow using aliased functions to  
set grid array sizes.  These would be called in the beginning of the  
simulation and would take no parameters.

The driver obtains the size of grid arrays via a call to  
CCTK_GroupSizesI.  This function could be updated, or there could be a  
driver flag to call another function.  (All other places that need to  
know the size of the group query the driver, so that there won't be an  
inconsistency.)

-erik

-- 
Erik Schnetter <schnetter at cct.lsu.edu>   http://www.cct.lsu.edu/~eschnett/




From dprideout at gmail.com  Thu Nov  5 13:42:53 2009
From: dprideout at gmail.com (David Rideout)
Date: Thu, 5 Nov 2009 14:42:53 -0500
Subject: [Developers] controlling the size of grid arrays
In-Reply-To: <3800E924-295B-4CC5-9A2B-0FEA3EC11099@cct.lsu.edu>
References: <1ce81abb0911041722w4b7aa47bka55442d2e5822d3e@mail.gmail.com> 
	<DE9DE828-A153-4F43-ACD6-6B388D15AE43@cct.lsu.edu>
	<1ce81abb0911050847y7fb3c2dbp845e1cb85dcb1839@mail.gmail.com> 
	<3800E924-295B-4CC5-9A2B-0FEA3EC11099@cct.lsu.edu>
Message-ID: <1ce81abb0911051142v125dd6f9pb535f0dfbc6d10b3@mail.gmail.com>

Ok thanks.  I would be in favor of an API which allows thorns to
specify the sizes of grid variables by defining an aliased function or
so.  Is anyone else interested in this?

Thanks,
David

On Thu, Nov 5, 2009 at 1:52 PM, Erik Schnetter <schnetter at cct.lsu.edu> wrote:
> On Nov 5, 2009, at 10:47 , David Rideout wrote:
>
>> Thanks.
>>
>> Can the accumulator parameter's value end up being the maximum of the
>> parameters which contribute to it?  e.g.
>>
>> int Nmax accumulator=max(x,y)
>>
>> Unfortunately the values of y which contribute to it is not given
>> directly by parameter values, but is a function of other parameters
>> which involves square roots.  Is there some way to handle this?
>>
>>
>> Is there any design requirement which is preventing us from having a
>> much more flexible specification of the sizes of Cactus grid
>> variables?  e.g. I have always thought it strange that the size of
>> grid functions (and number of ghost zones!) is specified directly by
>> parameters owned by the driver.  In general the size of grid variables
>> should be computed by thorns, not given directly by parameters.  In
>> fact I set PUGH's global_nx manually during STARTUP with
>>
>> ierr = CCTK_ParameterSet("global_nx", "PUGH", GFsizest);
>>
>> but this same trick is not working for grid arrays.  Also it is driver
>> dependent which is ugly.  How does one specify the size of grid
>> functions in Carpet?
>
>
> Both Carpet and PUGH determine the sizes of grid functions on their
> own.  You can set Carpet::global_nx, but there are also other
> methods.  And PUGH also supports setting PUGH::local_nx, for example.
>
> Instead of parameters we could also allow using aliased functions to
> set grid array sizes.  These would be called in the beginning of the
> simulation and would take no parameters.
>
> The driver obtains the size of grid arrays via a call to
> CCTK_GroupSizesI.  This function could be updated, or there could be a
> driver flag to call another function.  (All other places that need to
> know the size of the group query the driver, so that there won't be an
> inconsistency.)
>
> -erik
>
> --
> Erik Schnetter <schnetter at cct.lsu.edu>   http://www.cct.lsu.edu/~eschnett/
>
>
>
> _______________________________________________
> Developers mailing list
> Developers at cactuscode.org
> http://www.cactuscode.org/mailman/listinfo/developers
>

From knarf at cct.lsu.edu  Thu Nov  5 15:34:12 2009
From: knarf at cct.lsu.edu (Frank Loeffler)
Date: Thu, 5 Nov 2009 15:34:12 -0600
Subject: [Developers] Detect certain GNU attributes
In-Reply-To: <9CAA4657-B5E5-48B5-9E72-D510E4CDEE04@cct.lsu.edu>
References: <9CAA4657-B5E5-48B5-9E72-D510E4CDEE04@cct.lsu.edu>
Message-ID: <20091105213411.GB22099@numrel07.cct.lsu.edu>

On Wed, Nov 04, 2009 at 04:22:09PM -0600, Erik Schnetter wrote:
> Detect whether the compiler supports the attributes

I only had a look at the patch itself so far (which means I didn't test
it), and it looks good so far.

However, the patch also touches other (minor) things which are not
mentioned in the description and which should ideally me comitted in a
separate patch (slitely changing config definitions, changing
intendation, removing depreciated BOOL definitions...).

Frank

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
Url : http://www.cactuscode.org/pipermail/developers/attachments/20091105/a3fa9ba3/attachment.bin 

From dprideout at gmail.com  Thu Nov 12 14:02:24 2009
From: dprideout at gmail.com (David Rideout)
Date: Thu, 12 Nov 2009 15:02:24 -0500
Subject: [Developers] The flesh evaluates parameter expressions for the size
	of grid arrays too early
Message-ID: <1ce81abb0911121202l6cfff2c0oee7827d2504b59ec@mail.gmail.com>

The flesh evaluates parameter expressions for the size of grid arrays
when the thorn is activated.  This disallows the possibility of
steering the parameters before the driver allocates memory for such
grid variables.  The enclosed patch evaluates parameter expressions
for the size of grid arrays within the call to CCTK_GroupSizesI(), so
that it takes into account the most current values for the parameters.

May I commit this?

Thanks,
David


--- StripMime Report -- processed MIME parts ---
multipart/mixed
  text/plain (text body -- kept)
  application/octet-stream
---

From schnetter at cct.lsu.edu  Thu Nov 12 15:45:23 2009
From: schnetter at cct.lsu.edu (Erik Schnetter)
Date: Thu, 12 Nov 2009 15:45:23 -0600
Subject: [Developers] [Patches] The flesh evaluates parameter
	expressions for the size of grid arrays too early
In-Reply-To: <1ce81abb0911121202l6cfff2c0oee7827d2504b59ec@mail.gmail.com>
References: <1ce81abb0911121202l6cfff2c0oee7827d2504b59ec@mail.gmail.com>
Message-ID: <5FD97318-B8C0-47FA-88C4-BEBEDEF6A498@cct.lsu.edu>

On Nov 12, 2009, at 14:02 , David Rideout wrote:

> The flesh evaluates parameter expressions for the size of grid arrays
> when the thorn is activated.  This disallows the possibility of
> steering the parameters before the driver allocates memory for such
> grid variables.  The enclosed patch evaluates parameter expressions
> for the size of grid arrays within the call to CCTK_GroupSizesI(), so
> that it takes into account the most current values for the parameters.

This is an intriguingly simple change to solve your problem (having grid array sizes be arbitrarily complex expressions instead of parameter values).  It is simpler than one of the alternative we discussed, which would be to allow calling aliased functions in these expressions.

I'm concerned about several things.  The first is that, if array sizes are evaluated at a later time, then maybe other quantities should also be evaluated at a later time.  What about e.g. the number of vector elements in a grid variable group?

Another is that this probably requires the parameter to be steerable, so that it can be set before the driver starts up.  However, being truly steerable is not a good idea, since the parameter should then remain constant during evolution, since the size of the grid array will never be recalculated.

A third issue is with checkpointing and recovery.  I assume that the parameter value will not be set in the parameter file (since it will be calculated during startup).  That means that the parameter value will be checkpointed, and will also be recovered, which ensures that it then has the same value, which is necessary.  There should be some kind of check to ensure that the parameter is not set in a parameter file, and that it is not re-calculated differently upon recovery.

Should there maybe a new parameter type "calculated", in addition to "steerable"?  Calculated parameters could not be set in the parameter file.

Can you give an example of a function calculating a parameter?  How complex are these function in your case?  You mentioned earlier that you would like to calculate a maximum -- if this is all that you need, then extending accumulator parameters would be much simpler.

-erik

-- 
Erik Schnetter <schnetter at cct.lsu.edu>   http://www.cct.lsu.edu/~eschnett/





From knarf at cct.lsu.edu  Thu Nov 12 22:03:28 2009
From: knarf at cct.lsu.edu (Frank Loeffler)
Date: Thu, 12 Nov 2009 22:03:28 -0600
Subject: [Developers] [Patches] The flesh evaluates parameter
 expressions for the size of grid arrays too early
In-Reply-To: <5FD97318-B8C0-47FA-88C4-BEBEDEF6A498@cct.lsu.edu>
References: <1ce81abb0911121202l6cfff2c0oee7827d2504b59ec@mail.gmail.com>
	<5FD97318-B8C0-47FA-88C4-BEBEDEF6A498@cct.lsu.edu>
Message-ID: <20091113040328.GD1585@numrel07.cct.lsu.edu>

I didn't yet have a close look at the patch and should think more about
the whole problem, but the following came to my mind:

On Thu, Nov 12, 2009 at 03:45:23PM -0600, Erik Schnetter wrote:
> Should there maybe a new parameter type "calculated", in addition to "steerable"?  Calculated parameters could not be set in the parameter file.

If a parameter cannot be set in a parameter file, it should probably not
be called a parameter, but a simple scalar.

Frank

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
Url : http://www.cactuscode.org/pipermail/developers/attachments/20091112/1de88d7c/attachment.bin 

From dprideout at gmail.com  Wed Nov 18 08:47:23 2009
From: dprideout at gmail.com (David Rideout)
Date: Wed, 18 Nov 2009 09:47:23 -0500
Subject: [Developers] [Patches] The flesh evaluates parameter
	expressions for the size of grid arrays too early
In-Reply-To: <5FD97318-B8C0-47FA-88C4-BEBEDEF6A498@cct.lsu.edu>
References: <1ce81abb0911121202l6cfff2c0oee7827d2504b59ec@mail.gmail.com> 
	<5FD97318-B8C0-47FA-88C4-BEBEDEF6A498@cct.lsu.edu>
Message-ID: <1ce81abb0911180647j2a568493j51506bb3d86322a6@mail.gmail.com>

In general I would like to have more flexibility in the specification
of sizes of Cactus variables.  This is something that can be improved
in a future version.
In the mean time I have found a somewhat convenient workaround, which
is to steer parameters which control the size of grid variables during
STARTUP.  (To make it work I sandwich calls to CCTK_ParameterSet()
between
CCTKi_SetParameterSetMask (PARAMETER_RECOVERY_PRE);
...
CCTKi_SetParameterSetMask (PARAMETER_RECOVERY_POST);
)  It would be convenient to have the size of a vector group also
evaluated during STARTUP, instead of at thorn activation.

Perhaps we could add another steerability mode, for such parameters?

I don't see a problem with checkpoint/recovery.  Thorns which use such
parameters are free to check their value during recovery if they
choose.

Extending accumulator parameters is a possibility, but it will be
messy, as it will force each contributing thorn to define yet another
dummy parameter.  Also it has the limitation that the contribution to
the accumulator parameter value must be a simple (closed form)
expression of parameters.  I can imagine the contribution having a
different functional form for different input parameter values.


My current situation is that I have a data structure whose size is a
random variable (its size changes every iteration).  Since this is
difficult to handle in Cactus, I want some upper bound on its size.
So I have each thorn declare an upper bound on this size to my 'base
thorn' (via an aliased function call), and that thorn then sets an
'Nmax' parameter with the supremum of all these bounds.

Here is a current function which declares a bound:

void CFlatSprinkle_Startup(void) {
  DECLARE_CCTK_PARAMETERS;

  /* Since a Poisson distribution is essentially a Gaussian with variance
     'size', I choose violation of this bound to be a 6 sigma event. */
  CausetBase_RegisterGFSize(size + 6*sqrt(size));
}

Unfortunately I am finding this bound inadequate for small values of
'size', hence my concern above that one will want to use non-closed
form expressions of parameters.

Thanks,
David


On Thu, Nov 12, 2009 at 4:45 PM, Erik Schnetter <schnetter at cct.lsu.edu> wrote:
> On Nov 12, 2009, at 14:02 , David Rideout wrote:
>
>> The flesh evaluates parameter expressions for the size of grid arrays
>> when the thorn is activated.  This disallows the possibility of
>> steering the parameters before the driver allocates memory for such
>> grid variables.  The enclosed patch evaluates parameter expressions
>> for the size of grid arrays within the call to CCTK_GroupSizesI(), so
>> that it takes into account the most current values for the parameters.
>
> This is an intriguingly simple change to solve your problem (having grid array sizes be arbitrarily complex expressions instead of parameter values).  It is simpler than one of the alternative we discussed, which would be to allow calling aliased functions in these expressions.
>
> I'm concerned about several things.  The first is that, if array sizes are evaluated at a later time, then maybe other quantities should also be evaluated at a later time.  What about e.g. the number of vector elements in a grid variable group?
>
> Another is that this probably requires the parameter to be steerable, so that it can be set before the driver starts up.  However, being truly steerable is not a good idea, since the parameter should then remain constant during evolution, since the size of the grid array will never be recalculated.
>
> A third issue is with checkpointing and recovery.  I assume that the parameter value will not be set in the parameter file (since it will be calculated during startup).  That means that the parameter value will be checkpointed, and will also be recovered, which ensures that it then has the same value, which is necessary.  There should be some kind of check to ensure that the parameter is not set in a parameter file, and that it is not re-calculated differently upon recovery.
>
> Should there maybe a new parameter type "calculated", in addition to "steerable"?  Calculated parameters could not be set in the parameter file.
>
> Can you give an example of a function calculating a parameter?  How complex are these function in your case?  You mentioned earlier that you would like to calculate a maximum -- if this is all that you need, then extending accumulator parameters would be much simpler.
>
> -erik
>
> --
> Erik Schnetter <schnetter at cct.lsu.edu>   http://www.cct.lsu.edu/~eschnett/


From schnetter at cct.lsu.edu  Tue Nov 24 12:36:05 2009
From: schnetter at cct.lsu.edu (Erik Schnetter)
Date: Tue, 24 Nov 2009 12:36:05 -0600
Subject: [Developers] Add magnetic fields to HydroBase
Message-ID: <36984CB8-B065-4A8B-A0E6-B371C6706A68@cct.lsu.edu>

I suggest to add magnetic fields to HydroBase, using the same conventions and units as in Whisky.

Similar to the three-velocity, the magnetic field should be a vector group.  A single-letter name such as "B" is a bad idea.  I would suggest "Bmag[3]" as variable name; Frank L?ffler prefers "EM_B[3]".  Comments?

-erik

-- 
Erik Schnetter <schnetter at cct.lsu.edu>   http://www.cct.lsu.edu/~eschnett/





From dprideout at gmail.com  Tue Nov 24 12:41:24 2009
From: dprideout at gmail.com (David Rideout)
Date: Tue, 24 Nov 2009 13:41:24 -0500
Subject: [Developers] Add magnetic fields to HydroBase
In-Reply-To: <36984CB8-B065-4A8B-A0E6-B371C6706A68@cct.lsu.edu>
References: <36984CB8-B065-4A8B-A0E6-B371C6706A68@cct.lsu.edu>
Message-ID: <1ce81abb0911241041q3ae95260q20dd6056edbc7949@mail.gmail.com>

F_B?  FB?

On Tue, Nov 24, 2009 at 1:36 PM, Erik Schnetter <schnetter at cct.lsu.edu> wrote:
> I suggest to add magnetic fields to HydroBase, using the same conventions and units as in Whisky.
>
> Similar to the three-velocity, the magnetic field should be a vector group.  A single-letter name such as "B" is a bad idea.  I would suggest "Bmag[3]" as variable name; Frank L?ffler prefers "EM_B[3]".  Comments?
>
> -erik
>
> --
> Erik Schnetter <schnetter at cct.lsu.edu>   http://www.cct.lsu.edu/~eschnett/
>
>
>
>
> _______________________________________________
> Developers mailing list
> Developers at cactuscode.org
> http://www.cactuscode.org/mailman/listinfo/developers
>


From atonita at aei.mpg.de  Tue Nov 24 12:42:14 2009
From: atonita at aei.mpg.de (Aaryn Tonita)
Date: Tue, 24 Nov 2009 19:42:14 +0100
Subject: [Developers] Add magnetic fields to HydroBase
In-Reply-To: <36984CB8-B065-4A8B-A0E6-B371C6706A68@cct.lsu.edu>
References: <36984CB8-B065-4A8B-A0E6-B371C6706A68@cct.lsu.edu>
Message-ID: <003d01ca6d35$d7889ff0$8699dfd0$@mpg.de>

Well, if you want to do ideal MHD then there is both the Eulerian magnetic
field and the fluid magnetic field to worry about. So maybe you want eulB[3]
and lagB[3]. Then again, maybe you want to allow for something more general
than mhd.

-----Original Message-----
From: developers-bounces at cactuscode.org
[mailto:developers-bounces at cactuscode.org] On Behalf Of Erik Schnetter
Sent: Tuesday, November 24, 2009 7:36 PM
To: Cactus Developers
Subject: [Developers] Add magnetic fields to HydroBase

I suggest to add magnetic fields to HydroBase, using the same conventions
and units as in Whisky.

Similar to the three-velocity, the magnetic field should be a vector group.
A single-letter name such as "B" is a bad idea.  I would suggest "Bmag[3]"
as variable name; Frank L?ffler prefers "EM_B[3]".  Comments?

-erik

-- 
Erik Schnetter <schnetter at cct.lsu.edu>   http://www.cct.lsu.edu/~eschnett/




_______________________________________________
Developers mailing list
Developers at cactuscode.org
http://www.cactuscode.org/mailman/listinfo/developers



From bgiacoma at astro.umd.edu  Tue Nov 24 13:45:38 2009
From: bgiacoma at astro.umd.edu (Bruno Giacomazzo)
Date: Tue, 24 Nov 2009 14:45:38 -0500
Subject: [Developers] Add magnetic fields to HydroBase
In-Reply-To: <36984CB8-B065-4A8B-A0E6-B371C6706A68@cct.lsu.edu>
References: <36984CB8-B065-4A8B-A0E6-B371C6706A68@cct.lsu.edu>
Message-ID: <1B3B6ABE-1084-43DE-9449-63B8CFB48D42@astro.umd.edu>

On Nov 24, 2009, at 1:36 PM, Erik Schnetter wrote:

> I suggest to add magnetic fields to HydroBase, using the same  
> conventions and units as in Whisky.
>
> Similar to the three-velocity, the magnetic field should be a  
> vector group.  A single-letter name such as "B" is a bad idea.  I  
> would suggest "Bmag[3]" as variable name; Frank L?ffler prefers  
> "EM_B[3]".  Comments?
>

	For those who don't know, the field defined in HydroBase would be  
the magnetic field B^i as defined in Anton et al. 2006 ("Numerical 3 
+1 General Relativistic Magnetohydrodynamics: A Local Characteristic  
Approach"), i.e. the one measured by an Eulerian observer in analogy  
with the definition of the velocity (which is also measured by an  
Eulerian observer).

	I imagine Frank is thinking also of adding the electric field in the  
future and in that case one would have EM_B and EM_E. Another  
possibility could be Bvec and Evec. Also the electric field should be  
considered as the one measured by an Eulerian observer to keep  
everything consistent. In ideal MHD the electric field is not used  
since it is a simple function of the magnetic field and of the  
velocity, so when doing storage of these quantities you should keep  
them separate and have two different parameters to switch on the  
magnetic and the electric field components.

Cheers,
Bruno

Dr. Bruno Giacomazzo
Department of Astronomy
University of Maryland
College Park
MD 20742-2421
USA

Tel.  : +1 301 405 0676
Fax  : +1 301 314 9067
Cell. : +1 443 535 5846
email : bgiacoma at astro.umd.edu

----------------------------------------------------------------------
There are only 10 types of people in the world:
Those who understand binary, and those who don't
----------------------------------------------------------------------

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.cactuscode.org/pipermail/developers/attachments/20091124/09829df7/attachment.html 

From knarf at cct.lsu.edu  Tue Nov 24 13:51:03 2009
From: knarf at cct.lsu.edu (Frank Loeffler)
Date: Tue, 24 Nov 2009 13:51:03 -0600
Subject: [Developers] Add magnetic fields to HydroBase
In-Reply-To: <1B3B6ABE-1084-43DE-9449-63B8CFB48D42@astro.umd.edu>
References: <36984CB8-B065-4A8B-A0E6-B371C6706A68@cct.lsu.edu>
	<1B3B6ABE-1084-43DE-9449-63B8CFB48D42@astro.umd.edu>
Message-ID: <20091124195102.GO26917@numrel07.cct.lsu.edu>

On Tue, Nov 24, 2009 at 02:45:38PM -0500, Bruno Giacomazzo wrote:
> 	I imagine Frank is thinking also of adding the electric field in
> the future and in that case one would have EM_B and EM_E. Another
> possibility could be Bvec and Evec.

That is right and I now think that your suggestion is even better than
EM_?, but would start with E or B:

  B_vec[3] E_vec[3]

or when taking eulerian and lagrangian also into account from the
beginning (should we?):

  B_lag[3] E_lag[3] B_eul[3] E_eul[3]

Frank

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
Url : http://www.cactuscode.org/pipermail/developers/attachments/20091124/42128b45/attachment.bin 

From bgiacoma at astro.umd.edu  Tue Nov 24 14:04:52 2009
From: bgiacoma at astro.umd.edu (Bruno Giacomazzo)
Date: Tue, 24 Nov 2009 15:04:52 -0500
Subject: [Developers] Add magnetic fields to HydroBase
In-Reply-To: <20091124195102.GO26917@numrel07.cct.lsu.edu>
References: <36984CB8-B065-4A8B-A0E6-B371C6706A68@cct.lsu.edu>
	<1B3B6ABE-1084-43DE-9449-63B8CFB48D42@astro.umd.edu>
	<20091124195102.GO26917@numrel07.cct.lsu.edu>
Message-ID: <3FD9CFFF-C60D-431C-BBA7-685A298FC8B7@astro.umd.edu>

Frank,

On Nov 24, 2009, at 2:51 PM, Frank Loeffler wrote:

> On Tue, Nov 24, 2009 at 02:45:38PM -0500, Bruno Giacomazzo wrote:
>> 	I imagine Frank is thinking also of adding the electric field in
>> the future and in that case one would have EM_B and EM_E. Another
>> possibility could be Bvec and Evec.
>
> That is right and I now think that your suggestion is even better than
> EM_?, but would start with E or B:
>
>   B_vec[3] E_vec[3]
>

	this sounds OK for me

> or when taking eulerian and lagrangian also into account from the
> beginning (should we?):
>
>   B_lag[3] E_lag[3] B_eul[3] E_eul[3]


	I think we should stick to one definition as it was done for the  
velocity field. Having too many variables would make things too  
complicated I think. Moreover I think everybody in the GR community  
working with MHD and/or Maxwell equations in vacuum is using the  
Eulerian definition.

Cheers,
Bruno

Dr. Bruno Giacomazzo
Department of Astronomy
University of Maryland
College Park
MD 20742-2421
USA

Tel.  : +1 301 405 0676
Fax  : +1 301 314 9067
Cell. : +1 443 535 5846
email : bgiacoma at astro.umd.edu

----------------------------------------------------------------------
There are only 10 types of people in the world:
Those who understand binary, and those who don't
----------------------------------------------------------------------

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.cactuscode.org/pipermail/developers/attachments/20091124/19b202a5/attachment.html 

From knarf at cct.lsu.edu  Tue Nov 24 14:15:21 2009
From: knarf at cct.lsu.edu (Frank Loeffler)
Date: Tue, 24 Nov 2009 14:15:21 -0600
Subject: [Developers] Add magnetic fields to HydroBase
In-Reply-To: <3FD9CFFF-C60D-431C-BBA7-685A298FC8B7@astro.umd.edu>
References: <36984CB8-B065-4A8B-A0E6-B371C6706A68@cct.lsu.edu>
	<1B3B6ABE-1084-43DE-9449-63B8CFB48D42@astro.umd.edu>
	<20091124195102.GO26917@numrel07.cct.lsu.edu>
	<3FD9CFFF-C60D-431C-BBA7-685A298FC8B7@astro.umd.edu>
Message-ID: <20091124201520.GR26917@numrel07.cct.lsu.edu>

Hi,

On Tue, Nov 24, 2009 at 03:04:52PM -0500, Bruno Giacomazzo wrote:
> 	I think we should stick to one definition as it was done for the
> velocity field. Having too many variables would make things too
> complicated I think. Moreover I think everybody in the GR community
> working with MHD and/or Maxwell equations in vacuum is using the
> Eulerian definition.

We could start using B_eul[3] and E_eul[3] and add the other two later,
or add them already now, but disable them by default using a parameter.

The important question is if we could possibly need both definitions in
the forseable future. If we answer here "yes" or even "maybe", we should
think about this now.

I would answer "maybe" and would only define B_eul[3] and E_eul[3] for
the moment. This avoids confusion in the future, but also does not add
overhead which would not be used at the moment. Adding the other two
variables would then be easy and should be done when there is need.

Frank

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
Url : http://www.cactuscode.org/pipermail/developers/attachments/20091124/1c78e2da/attachment.bin 

From bgiacoma at astro.umd.edu  Tue Nov 24 14:50:16 2009
From: bgiacoma at astro.umd.edu (Bruno Giacomazzo)
Date: Tue, 24 Nov 2009 15:50:16 -0500
Subject: [Developers] Add magnetic fields to HydroBase
In-Reply-To: <20091124201520.GR26917@numrel07.cct.lsu.edu>
References: <36984CB8-B065-4A8B-A0E6-B371C6706A68@cct.lsu.edu>
	<1B3B6ABE-1084-43DE-9449-63B8CFB48D42@astro.umd.edu>
	<20091124195102.GO26917@numrel07.cct.lsu.edu>
	<3FD9CFFF-C60D-431C-BBA7-685A298FC8B7@astro.umd.edu>
	<20091124201520.GR26917@numrel07.cct.lsu.edu>
Message-ID: <E9BDEF46-0D0C-4D08-A200-B9AE1FE5511C@astro.umd.edu>

Frank,
	note that the Eulerian definition gives you a purely spatial vector  
for the magnetic field (B^i) while the lagrangian definition (usually  
denoted with b^\mu as opposed to B^i) is a four-vector.

	So I think it doesn't have much sense to store only the spatial  
components of b^\mu. Moreover there are simple relations to pass from  
one to the other. So even if somebody would use the lagrangian as the  
primitive variable for some reason (i don't know anybody doing that  
both in special and general relativity), then he/she should just take  
care of computing the Eulerian one in order that external analysis  
routines (for example) could compute what they need.

	Anyway, if you want instead to include also the lagrangian  
definition in HydroBase then you should define

	B_eul[3] E_eul[3]
	B_lag[4] E_lag[4]

Cheers,
Bruno

On Nov 24, 2009, at 3:15 PM, Frank Loeffler wrote:

> Hi,
>
> On Tue, Nov 24, 2009 at 03:04:52PM -0500, Bruno Giacomazzo wrote:
>> 	I think we should stick to one definition as it was done for the
>> velocity field. Having too many variables would make things too
>> complicated I think. Moreover I think everybody in the GR community
>> working with MHD and/or Maxwell equations in vacuum is using the
>> Eulerian definition.
>
> We could start using B_eul[3] and E_eul[3] and add the other two  
> later,
> or add them already now, but disable them by default using a  
> parameter.
>
> The important question is if we could possibly need both  
> definitions in
> the forseable future. If we answer here "yes" or even "maybe", we  
> should
> think about this now.
>
> I would answer "maybe" and would only define B_eul[3] and E_eul[3] for
> the moment. This avoids confusion in the future, but also does not add
> overhead which would not be used at the moment. Adding the other two
> variables would then be easy and should be done when there is need.
>
> Frank
>
> _______________________________________________
> Developers mailing list
> Developers at cactuscode.org
> http://www.cactuscode.org/mailman/listinfo/developers

Dr. Bruno Giacomazzo
Department of Astronomy
University of Maryland
College Park
MD 20742-2421
USA

Tel.  : +1 301 405 0676
Fax  : +1 301 314 9067
Cell. : +1 443 535 5846
email : bgiacoma at astro.umd.edu

----------------------------------------------------------------------
There are only 10 types of people in the world:
Those who understand binary, and those who don't
----------------------------------------------------------------------

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.cactuscode.org/pipermail/developers/attachments/20091124/1af81f78/attachment-0001.html 

From knarf at cct.lsu.edu  Tue Nov 24 15:19:13 2009
From: knarf at cct.lsu.edu (Frank Loeffler)
Date: Tue, 24 Nov 2009 15:19:13 -0600
Subject: [Developers] Add magnetic fields to HydroBase
In-Reply-To: <E9BDEF46-0D0C-4D08-A200-B9AE1FE5511C@astro.umd.edu>
References: <36984CB8-B065-4A8B-A0E6-B371C6706A68@cct.lsu.edu>
	<1B3B6ABE-1084-43DE-9449-63B8CFB48D42@astro.umd.edu>
	<20091124195102.GO26917@numrel07.cct.lsu.edu>
	<3FD9CFFF-C60D-431C-BBA7-685A298FC8B7@astro.umd.edu>
	<20091124201520.GR26917@numrel07.cct.lsu.edu>
	<E9BDEF46-0D0C-4D08-A200-B9AE1FE5511C@astro.umd.edu>
Message-ID: <20091124211912.GT26917@numrel07.cct.lsu.edu>

Hi,

On Tue, Nov 24, 2009 at 03:50:16PM -0500, Bruno Giacomazzo wrote:
> 	Anyway, if you want instead to include also the lagrangian
> definition in HydroBase then you should define
> 
> 	B_eul[3] E_eul[3]
> 	B_lag[4] E_lag[4]

I suggest to define the first two in HydroBase and comment in the
documentation that the latter two are (at the moment) not defined
because effectivly unused, but could be added in the future. Even if
that would not happen, someone using them could define them in their
own thorn and then have a consistent name scheme.

Frank

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
Url : http://www.cactuscode.org/pipermail/developers/attachments/20091124/697e56e6/attachment.bin 

From schnetter at cct.lsu.edu  Tue Nov 24 15:55:20 2009
From: schnetter at cct.lsu.edu (Erik Schnetter)
Date: Tue, 24 Nov 2009 15:55:20 -0600
Subject: [Developers] Add magnetic fields to HydroBase
In-Reply-To: <20091124211912.GT26917@numrel07.cct.lsu.edu>
References: <36984CB8-B065-4A8B-A0E6-B371C6706A68@cct.lsu.edu>
	<1B3B6ABE-1084-43DE-9449-63B8CFB48D42@astro.umd.edu>
	<20091124195102.GO26917@numrel07.cct.lsu.edu>
	<3FD9CFFF-C60D-431C-BBA7-685A298FC8B7@astro.umd.edu>
	<20091124201520.GR26917@numrel07.cct.lsu.edu>
	<E9BDEF46-0D0C-4D08-A200-B9AE1FE5511C@astro.umd.edu>
	<20091124211912.GT26917@numrel07.cct.lsu.edu>
Message-ID: <029089E6-D2D1-4F89-97EC-BDBF172C4806@cct.lsu.edu>

On Nov 24, 2009, at 15:19 , Frank Loeffler wrote:

> Hi,
> 
> On Tue, Nov 24, 2009 at 03:50:16PM -0500, Bruno Giacomazzo wrote:
>> 	Anyway, if you want instead to include also the lagrangian
>> definition in HydroBase then you should define
>> 
>> 	B_eul[3] E_eul[3]
>> 	B_lag[4] E_lag[4]
> 
> I suggest to define the first two in HydroBase and comment in the
> documentation that the latter two are (at the moment) not defined
> because effectivly unused, but could be added in the future. Even if
> that would not happen, someone using them could define them in their
> own thorn and then have a consistent name scheme.


Same as with the three-velocity we should just define one simple quantity.  Given that I don't like underscores, I then propose to use "Bvec".

-erik

-- 
Erik Schnetter <schnetter at cct.lsu.edu>   http://www.cct.lsu.edu/~eschnett/





From dprideout at gmail.com  Thu Nov 26 11:41:10 2009
From: dprideout at gmail.com (David Rideout)
Date: Thu, 26 Nov 2009 12:41:10 -0500
Subject: [Developers] trouble profiling Fortran routines in Cactus
Message-ID: <1ce81abb0911260941ja66f01crb2096938aabea62f@mail.gmail.com>

Hi all,

I have a scheduled routine Propagator() in C, which calls

CCTK_FCALL CCTK_FNAME(Propagator_Feynman)(Delta, N, &nevals, evals, evectors);

When I profile the code, all time spent in
CCTK_FNAME(Propagator_Feynman) seems to be assigned to Propagator(),
and gprof claims that exactly zero time is spent in
CCTK_FNAME(Propagator_Feynman).  It is rock bottom at the end of the
list of most time consuming routines, which I know is not the case.
Can anyone explain why this is happening?

Thanks,
David

From schnetter at cct.lsu.edu  Thu Nov 26 12:21:12 2009
From: schnetter at cct.lsu.edu (Erik Schnetter)
Date: Thu, 26 Nov 2009 12:21:12 -0600
Subject: [Developers] trouble profiling Fortran routines in Cactus
In-Reply-To: <1ce81abb0911260941ja66f01crb2096938aabea62f@mail.gmail.com>
References: <1ce81abb0911260941ja66f01crb2096938aabea62f@mail.gmail.com>
Message-ID: <9CCE6482-DBB2-4DDB-8FFD-BA4D353ADBFA@cct.lsu.edu>

On Nov 26, 2009, at 11:41 , David Rideout wrote:

> Hi all,
> 
> I have a scheduled routine Propagator() in C, which calls
> 
> CCTK_FCALL CCTK_FNAME(Propagator_Feynman)(Delta, N, &nevals, evals, evectors);
> 
> When I profile the code, all time spent in
> CCTK_FNAME(Propagator_Feynman) seems to be assigned to Propagator(),
> and gprof claims that exactly zero time is spent in
> CCTK_FNAME(Propagator_Feynman).  It is rock bottom at the end of the
> list of most time consuming routines, which I know is not the case.
> Can anyone explain why this is happening?


Offhand I cannot.

How long is the overall evolution?  Have all routines been compiles with profiling flags?  Is the call graph between these two routines correct?  How often are these routines executed?

-erik

-- 
Erik Schnetter <schnetter at cct.lsu.edu>   http://www.cct.lsu.edu/~eschnett/





From dprideout at gmail.com  Thu Nov 26 15:15:44 2009
From: dprideout at gmail.com (David Rideout)
Date: Thu, 26 Nov 2009 16:15:44 -0500
Subject: [Developers] trouble profiling Fortran routines in Cactus
In-Reply-To: <9CCE6482-DBB2-4DDB-8FFD-BA4D353ADBFA@cct.lsu.edu>
References: <1ce81abb0911260941ja66f01crb2096938aabea62f@mail.gmail.com> 
	<9CCE6482-DBB2-4DDB-8FFD-BA4D353ADBFA@cct.lsu.edu>
Message-ID: <1ce81abb0911261315i6511c873s2786491891e09f70@mail.gmail.com>

On Thu, Nov 26, 2009 at 1:21 PM, Erik Schnetter <schnetter at cct.lsu.edu> wrote:
> On Nov 26, 2009, at 11:41 , David Rideout wrote:
>
>> Hi all,
>>
>> I have a scheduled routine Propagator() in C, which calls
>>
>> CCTK_FCALL CCTK_FNAME(Propagator_Feynman)(Delta, N, &nevals, evals, evectors);
>>
>> When I profile the code, all time spent in
>> CCTK_FNAME(Propagator_Feynman) seems to be assigned to Propagator(),
>> and gprof claims that exactly zero time is spent in
>> CCTK_FNAME(Propagator_Feynman).  It is rock bottom at the end of the
>> list of most time consuming routines, which I know is not the case.
>> Can anyone explain why this is happening?
>
>
> Offhand I cannot.
>
> How long is the overall evolution?  Have all routines been compiles with profiling flags?  Is the call graph between these two routines correct?  How often are these routines executed?

I have tried various times, from 1.5 sec to 1.5 minutes.  All give
zero time spent in CCTK_FNAME(Propagator_Feynman).
All routines have been compiled with '-pg'.  I am using the gnu f95 compiler.
The call graph appears to be correct, including the 'C children' of
CCTK_FNAME(Propagator_Feynman).
CCTK_FNAME(Propagator_Feynman) is called only once.

Thanks,
David

From schnetter at cct.lsu.edu  Fri Nov 27 19:44:31 2009
From: schnetter at cct.lsu.edu (Erik Schnetter)
Date: Fri, 27 Nov 2009 19:44:31 -0600
Subject: [Developers] trouble profiling Fortran routines in Cactus
In-Reply-To: <1ce81abb0911261315i6511c873s2786491891e09f70@mail.gmail.com>
References: <1ce81abb0911260941ja66f01crb2096938aabea62f@mail.gmail.com>
	<9CCE6482-DBB2-4DDB-8FFD-BA4D353ADBFA@cct.lsu.edu>
	<1ce81abb0911261315i6511c873s2786491891e09f70@mail.gmail.com>
Message-ID: <23D8308E-BFC7-4477-8F00-D7C4F9FF57AE@cct.lsu.edu>

On Nov 26, 2009, at 15:15 , David Rideout wrote:

> On Thu, Nov 26, 2009 at 1:21 PM, Erik Schnetter <schnetter at cct.lsu.edu> wrote:
>> On Nov 26, 2009, at 11:41 , David Rideout wrote:
>> 
>>> Hi all,
>>> 
>>> I have a scheduled routine Propagator() in C, which calls
>>> 
>>> CCTK_FCALL CCTK_FNAME(Propagator_Feynman)(Delta, N, &nevals, evals, evectors);
>>> 
>>> When I profile the code, all time spent in
>>> CCTK_FNAME(Propagator_Feynman) seems to be assigned to Propagator(),
>>> and gprof claims that exactly zero time is spent in
>>> CCTK_FNAME(Propagator_Feynman).  It is rock bottom at the end of the
>>> list of most time consuming routines, which I know is not the case.
>>> Can anyone explain why this is happening?
>> 
>> 
>> Offhand I cannot.
>> 
>> How long is the overall evolution?  Have all routines been compiles with profiling flags?  Is the call graph between these two routines correct?  How often are these routines executed?
> 
> I have tried various times, from 1.5 sec to 1.5 minutes.  All give
> zero time spent in CCTK_FNAME(Propagator_Feynman).
> All routines have been compiled with '-pg'.  I am using the gnu f95 compiler.
> The call graph appears to be correct, including the 'C children' of
> CCTK_FNAME(Propagator_Feynman).
> CCTK_FNAME(Propagator_Feynman) is called only once.


You could try using a different compiler, or a different system...

-erik

-- 
Erik Schnetter <schnetter at cct.lsu.edu>   http://www.cct.lsu.edu/~eschnett/