From tradke at aei.mpg.de  Mon Apr  6 11:56:43 2009
From: tradke at aei.mpg.de (Thomas Radke)
Date: Mon, 06 Apr 2009 18:56:43 +0200
Subject: [Developers] more accurate Cactus timer reports
Message-ID: <49DA344B.70001@aei.mpg.de>

The current Cactus scheduler attaches a timer to each scheduled routine
to measure the time spent in it. A routine's execution time is
accumulated regardless of the schedule bin it has been called in.
Routines scheduled more than once in a given schedule bin, or scheduled
in more than one schedule bin, are then accounted for in
a timer report multiple times.

The attached patch associates a separate timer for each routine called
in a given schedule bin so its execution times in different bins can be
distinguished. For multiple calls within the same bin, the accumulated 
time is reported only once. Now the total sums of all routines / 
schedule bins actually add up to the total time of the simulation.

-- 
Cheers, Thomas.



--- StripMime Report -- processed MIME parts ---
multipart/mixed
  text/plain (text body -- kept)
  text/x-patch
---

From tradke at aei.mpg.de  Thu Apr 16 06:30:37 2009
From: tradke at aei.mpg.de (Thomas Radke)
Date: Thu, 16 Apr 2009 20:30:37 +0900
Subject: [Developers] CactusBase/IOUtil: parse output reductions lists for
	individual variables
Message-ID: <49E716DD.5070205@aei.mpg.de>

Hi,

the attached patch for routine IOUtil_ParseVarsForOutput() parses an 
options string for a list of reductions to be used in scalar output methods.

With the corresponding patch in Carpet/CarpetIOScalar it will then be 
possible to specify separate reductions for individual variables.

-- 
Cheers, Thomas.


--- StripMime Report -- processed MIME parts ---
multipart/mixed
  text/plain (text body -- kept)
  text/x-patch
---

From schnetter at cct.lsu.edu  Fri Apr 17 11:35:33 2009
From: schnetter at cct.lsu.edu (Erik Schnetter)
Date: Fri, 17 Apr 2009 11:35:33 -0500
Subject: [Developers] [Patches] CactusBase/IOUtil: parse output
	reductions lists for individual variables
In-Reply-To: <49E716DD.5070205@aei.mpg.de>
References: <49E716DD.5070205@aei.mpg.de>
Message-ID: <D7041620-12B1-4C21-BFDA-5208C1229B38@cct.lsu.edu>

On Apr 16, 2009, at 06:30:37, Thomas Radke wrote:

> Hi,
>
> the attached patch for routine IOUtil_ParseVarsForOutput() parses an  
> options string for a list of reductions to be used in scalar output  
> methods.
>
> With the corresponding patch in Carpet/CarpetIOScalar it will then  
> be possible to specify separate reductions for individual variables.


Does CactusBase/IOBasic support separate (scalar)reductions for  
different variables?

One can specify e.g. different output intervals for different  
variables.  Would the different reductions be specified in the same way?

The patch looks good.

-erik

-- 
Erik Schnetter <schnetter at cct.lsu.edu>   http://www.cct.lsu.edu/~eschnett/

My email is as private as my paper mail.  I therefore support encrypting
and signing email messages.  Get my PGP key from <http://pgp.mit.edu>.



-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 194 bytes
Desc: This is a digitally signed message part
Url : http://www.cactuscode.org/pipermail/developers/attachments/20090417/3539924c/attachment.bin 

From schnetter at cct.lsu.edu  Fri Apr 17 17:28:19 2009
From: schnetter at cct.lsu.edu (Erik Schnetter)
Date: Fri, 17 Apr 2009 17:28:19 -0500
Subject: [Developers] Jpeg output from AMR simulations
Message-ID: <47C50FAE-CAA0-4C2B-8E2D-2E1816FD6B63@cct.lsu.edu>

The enclosed patch adds capabilities for jpeg output from AMR  
simulation.  In addition to hyperslabbing, IOJpeg can also use  
interpolation to access simulation data.  The region and resolution  
from which data are taken can be freely chosen.  The values can also  
be multiplied with the radius r, since this is often useful for  
extracted waves.

-erik

-- 
Erik Schnetter <schnetter at cct.lsu.edu>   http://www.cct.lsu.edu/~eschnett/

My email is as private as my paper mail.  I therefore support encrypting
and signing email messages.  Get my PGP key from <http://pgp.mit.edu>.




--- StripMime Report -- processed MIME parts ---
multipart/mixed
  text/plain (text body -- kept)
  application/octet-stream
---

From schnetter at cct.lsu.edu  Fri Apr 17 17:41:34 2009
From: schnetter at cct.lsu.edu (Erik Schnetter)
Date: Fri, 17 Apr 2009 17:41:34 -0500
Subject: [Developers] [Patches] Jpeg output from AMR simulations
In-Reply-To: <47C50FAE-CAA0-4C2B-8E2D-2E1816FD6B63@cct.lsu.edu>
References: <47C50FAE-CAA0-4C2B-8E2D-2E1816FD6B63@cct.lsu.edu>
Message-ID: <75553386-0AE6-4571-A93E-27A0E75CA9BC@cct.lsu.edu>

On Apr 17, 2009, at 17:28:19, Erik Schnetter wrote:

> The enclosed patch adds capabilities for jpeg output from AMR  
> simulation.  In addition to hyperslabbing, IOJpeg can also use  
> interpolation to access simulation data.  The region and resolution  
> from which data are taken can be freely chosen.  The values can also  
> be multiplied with the radius r, since this is often useful for  
> extracted waves.

Attached is the wave form as extracted from a prescribed binary scalar  
source calculated with mesh refinement.

-erik

-- 
Erik Schnetter <schnetter at cct.lsu.edu>   http://www.cct.lsu.edu/~eschnett/

My email is as private as my paper mail.  I therefore support encrypting
and signing email messages.  Get my PGP key from <http://pgp.mit.edu>.


-------------- next part --------------
A non-text attachment was scrubbed...
Name: phi_xy_[35].jpeg
Type: image/jpeg
Size: 1991 bytes
Desc: not available
Url : http://www.cactuscode.org/pipermail/developers/attachments/20090417/af8fea3f/attachment.jpeg 
-------------- next part --------------



-- 
Erik Schnetter <schnetter at cct.lsu.edu>   http://www.cct.lsu.edu/~eschnett/

My email is as private as my paper mail.  I therefore support encrypting
and signing email messages.  Get my PGP key from <http://pgp.mit.edu>.



-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 194 bytes
Desc: This is a digitally signed message part
Url : http://www.cactuscode.org/pipermail/developers/attachments/20090417/af8fea3f/attachment.bin 

From tradke at aei.mpg.de  Fri Apr 17 20:48:50 2009
From: tradke at aei.mpg.de (Thomas Radke)
Date: Sat, 18 Apr 2009 10:48:50 +0900
Subject: [Developers] [Patches] CactusBase/IOUtil: parse
 output	reductions lists for individual variables
In-Reply-To: <D7041620-12B1-4C21-BFDA-5208C1229B38@cct.lsu.edu>
References: <49E716DD.5070205@aei.mpg.de>
	<D7041620-12B1-4C21-BFDA-5208C1229B38@cct.lsu.edu>
Message-ID: <49E93182.3080209@aei.mpg.de>

Erik Schnetter wrote:
> On Apr 16, 2009, at 06:30:37, Thomas Radke wrote:
> 
>> Hi,
>>
>> the attached patch for routine IOUtil_ParseVarsForOutput() parses an 
>> options string for a list of reductions to be used in scalar output 
>> methods.
>>
>> With the corresponding patch in Carpet/CarpetIOScalar it will then be 
>> possible to specify separate reductions for individual variables.
> 
> 
> Does CactusBase/IOBasic support separate (scalar)reductions for 
> different variables?

Yes, CactusBase/IOBasic supports separate reductions for different 
variables already. It implements its own options string parsing code 
though and does not use IOUtil_ParseVarsForOutput(), so I'd rather not 
touch it without necessity.
Looking through the code of CactusBase/IOBasic I found that it uses a 
slightly different keyword in the options string: 'reductions' instead 
of 'out_reductions'. For compatibility reasons I would change the 
CactusBase/IOUtil patch to also use the former keyword.

> One can specify e.g. different output intervals for different 
> variables.  Would the different reductions be specified in the same way?

Yes, precisely. Here is an example from the CactusBase/IOBasic thorn 
documentation:

   IOBasic::outInfo_vars       = "grid::r
                                  wavetoy::phi{reductions = 'norm2'}
                                  mythorn::complex"
   IOBasic::outInfo_reductions = "minimum maximum"

-- 
Cheers, Thomas.

From tradke at aei.mpg.de  Fri Apr 17 21:10:41 2009
From: tradke at aei.mpg.de (Thomas Radke)
Date: Sat, 18 Apr 2009 11:10:41 +0900
Subject: [Developers] more accurate Cactus timer reports
In-Reply-To: <49DA344B.70001@aei.mpg.de>
References: <49DA344B.70001@aei.mpg.de>
Message-ID: <49E936A1.7090309@aei.mpg.de>

http://www.cactuscode.org/old/pipermail/patches/2009-April/000258.html

Ok to apply ?

-- 
Cheers, Thomas.

From tradke at aei.mpg.de  Fri Apr 17 22:09:05 2009
From: tradke at aei.mpg.de (Thomas Radke)
Date: Sat, 18 Apr 2009 12:09:05 +0900
Subject: [Developers] Jpeg output from AMR simulations
In-Reply-To: <47C50FAE-CAA0-4C2B-8E2D-2E1816FD6B63@cct.lsu.edu>
References: <47C50FAE-CAA0-4C2B-8E2D-2E1816FD6B63@cct.lsu.edu>
Message-ID: <49E94451.1080108@aei.mpg.de>

Erik Schnetter wrote:
> The enclosed patch adds capabilities for jpeg output from AMR  
> simulation.  In addition to hyperslabbing, IOJpeg can also use  
> interpolation to access simulation data.  The region and resolution  
> from which data are taken can be freely chosen.  The values can also  
> be multiplied with the radius r, since this is often useful for  
> extracted waves.

Hi Erik,

one small addition to your patch: I had to '#include "cctk_Arguments.h"' 
in ChooseOutput.c in order to make it compile.

When testing the patch, I found it difficult to find good values for the 
origin/spacing/npoints parameters. It would be good if you could provide 
an example parfile to demonstrate jpeg output from AMR simulations, eg. 
the one with which you create the nice 2D plot attached to 
http://www.cactuscode.org/old/pipermail/developers/2009-April/005706.html.
The Carpet parfile that's already there (par/CarpetIOJpeg.par) can 
possibly be removed. I get an assertion failure (probably from 
CarpetSlab) when running it .

-- 
Cheers, Thomas.

From schnetter at cct.lsu.edu  Fri Apr 17 22:12:48 2009
From: schnetter at cct.lsu.edu (Erik Schnetter)
Date: Fri, 17 Apr 2009 22:12:48 -0500
Subject: [Developers] Jpeg output from AMR simulations
In-Reply-To: <49E94451.1080108@aei.mpg.de>
References: <47C50FAE-CAA0-4C2B-8E2D-2E1816FD6B63@cct.lsu.edu>
	<49E94451.1080108@aei.mpg.de>
Message-ID: <D75A60F9-4602-46F1-8E03-412EC52CC67B@cct.lsu.edu>

On Apr 17, 2009, at 22:09:05, Thomas Radke wrote:

> Erik Schnetter wrote:
>> The enclosed patch adds capabilities for jpeg output from AMR
>> simulation.  In addition to hyperslabbing, IOJpeg can also use
>> interpolation to access simulation data.  The region and resolution
>> from which data are taken can be freely chosen.  The values can also
>> be multiplied with the radius r, since this is often useful for
>> extracted waves.
>
> Hi Erik,
>
> one small addition to your patch: I had to '#include  
> "cctk_Arguments.h"'
> in ChooseOutput.c in order to make it compile.
>
> When testing the patch, I found it difficult to find good values for  
> the
> origin/spacing/npoints parameters. It would be good if you could  
> provide
> an example parfile to demonstrate jpeg output from AMR simulations,  
> eg.
> the one with which you create the nice 2D plot attached to
> http://www.cactuscode.org/old/pipermail/developers/2009-April/005706.html 
> .
> The Carpet parfile that's already there (par/CarpetIOJpeg.par) can
> possibly be removed. I get an assertion failure (probably from
> CarpetSlab) when running it .


Yes, the "cctk_Arguments.h" is necessary.  I don't understand why I  
didn't see an error.

I attach an example thorn list and example parameter file.

-erik

-- 
Erik Schnetter <schnetter at cct.lsu.edu>   http://www.cct.lsu.edu/~eschnett/

My email is as private as my paper mail.  I therefore support encrypting
and signing email messages.  Get my PGP key from <http://pgp.mit.edu>.


-------------- next part --------------
A non-text attachment was scrubbed...
Name: jpeg_amr.th
Type: application/octet-stream
Size: 535 bytes
Desc: not available
Url : http://www.cactuscode.org/pipermail/developers/attachments/20090417/3b5f7eaa/attachment-0002.obj 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: jpeg_amr.par
Type: application/octet-stream
Size: 2369 bytes
Desc: not available
Url : http://www.cactuscode.org/pipermail/developers/attachments/20090417/3b5f7eaa/attachment-0003.obj 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 194 bytes
Desc: This is a digitally signed message part
Url : http://www.cactuscode.org/pipermail/developers/attachments/20090417/3b5f7eaa/attachment-0001.bin 

From tradke at aei.mpg.de  Sat Apr 18 05:29:38 2009
From: tradke at aei.mpg.de (Thomas Radke)
Date: Sat, 18 Apr 2009 19:29:38 +0900
Subject: [Developers] Jpeg output from AMR simulations
In-Reply-To: <D75A60F9-4602-46F1-8E03-412EC52CC67B@cct.lsu.edu>
References: <47C50FAE-CAA0-4C2B-8E2D-2E1816FD6B63@cct.lsu.edu>	<49E94451.1080108@aei.mpg.de>
	<D75A60F9-4602-46F1-8E03-412EC52CC67B@cct.lsu.edu>
Message-ID: <49E9AB92.4040706@aei.mpg.de>

Erik Schnetter wrote:
> On Apr 17, 2009, at 22:09:05, Thomas Radke wrote:
> 
>> Erik Schnetter wrote:
>>> The enclosed patch adds capabilities for jpeg output from AMR
>>> simulation.  In addition to hyperslabbing, IOJpeg can also use
>>> interpolation to access simulation data.  The region and resolution
>>> from which data are taken can be freely chosen.  The values can also
>>> be multiplied with the radius r, since this is often useful for
>>> extracted waves.
>>
>> Hi Erik,
>>
>> one small addition to your patch: I had to '#include "cctk_Arguments.h"'
>> in ChooseOutput.c in order to make it compile.
>>
>> When testing the patch, I found it difficult to find good values for the
>> origin/spacing/npoints parameters. It would be good if you could provide
>> an example parfile to demonstrate jpeg output from AMR simulations, eg.
>> the one with which you create the nice 2D plot attached to
>> http://www.cactuscode.org/old/pipermail/developers/2009-April/005706.html. 
>>
>> The Carpet parfile that's already there (par/CarpetIOJpeg.par) can
>> possibly be removed. I get an assertion failure (probably from
>> CarpetSlab) when running it .
> 
> 
> Yes, the "cctk_Arguments.h" is necessary.  I don't understand why I 
> didn't see an error.
> 
> I attach an example thorn list and example parameter file.

Thanks ! I was also missing the latest patch which added an option to 
force interpolation in global mode - without it I got lots of errors 
from the local interpolator.

Still, when I run your example parfile it seems nothing is actually 
evolved, only when I take out AMR (set reflevels to 1). Do I need 
something else ?

-- 
Cheers, Thomas.

From schnetter at cct.lsu.edu  Sat Apr 18 08:49:59 2009
From: schnetter at cct.lsu.edu (Erik Schnetter)
Date: Sat, 18 Apr 2009 08:49:59 -0500
Subject: [Developers] Jpeg output from AMR simulations
In-Reply-To: <49E9AB92.4040706@aei.mpg.de>
References: <47C50FAE-CAA0-4C2B-8E2D-2E1816FD6B63@cct.lsu.edu>	<49E94451.1080108@aei.mpg.de>
	<D75A60F9-4602-46F1-8E03-412EC52CC67B@cct.lsu.edu>
	<49E9AB92.4040706@aei.mpg.de>
Message-ID: <3C06FBC5-AAC3-4165-8FC4-366864F2A767@cct.lsu.edu>

On Apr 18, 2009, at 05:29:38, Thomas Radke wrote:

> Erik Schnetter wrote:
>> On Apr 17, 2009, at 22:09:05, Thomas Radke wrote:
>>
>>> Erik Schnetter wrote:
>>>> The enclosed patch adds capabilities for jpeg output from AMR
>>>> simulation.  In addition to hyperslabbing, IOJpeg can also use
>>>> interpolation to access simulation data.  The region and resolution
>>>> from which data are taken can be freely chosen.  The values can  
>>>> also
>>>> be multiplied with the radius r, since this is often useful for
>>>> extracted waves.
>>>
>>> Hi Erik,
>>>
>>> one small addition to your patch: I had to '#include  
>>> "cctk_Arguments.h"'
>>> in ChooseOutput.c in order to make it compile.
>>>
>>> When testing the patch, I found it difficult to find good values  
>>> for the
>>> origin/spacing/npoints parameters. It would be good if you could  
>>> provide
>>> an example parfile to demonstrate jpeg output from AMR  
>>> simulations, eg.
>>> the one with which you create the nice 2D plot attached to
>>> http://www.cactuscode.org/old/pipermail/developers/2009-April/005706.html 
>>> .
>>>
>>> The Carpet parfile that's already there (par/CarpetIOJpeg.par) can
>>> possibly be removed. I get an assertion failure (probably from
>>> CarpetSlab) when running it .
>>
>>
>> Yes, the "cctk_Arguments.h" is necessary.  I don't understand why I
>> didn't see an error.
>>
>> I attach an example thorn list and example parameter file.
>
> Thanks ! I was also missing the latest patch which added an option to
> force interpolation in global mode - without it I got lots of errors
> from the local interpolator.
>
> Still, when I run your example parfile it seems nothing is actually
> evolved, only when I take out AMR (set reflevels to 1). Do I need
> something else ?


Okay, there is another patch I forgot.  I encountered an error in  
WaveBinarySource where the grid point indices were calculated wrong.   
I will commit this as well.

-erik

-- 
Erik Schnetter <schnetter at cct.lsu.edu>   http://www.cct.lsu.edu/~eschnett/

My email is as private as my paper mail.  I therefore support encrypting
and signing email messages.  Get my PGP key from <http://pgp.mit.edu>.



-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 194 bytes
Desc: This is a digitally signed message part
Url : http://www.cactuscode.org/pipermail/developers/attachments/20090418/7edc55f0/attachment.bin 

From schnetter at cct.lsu.edu  Mon Apr 20 13:44:07 2009
From: schnetter at cct.lsu.edu (Erik Schnetter)
Date: Mon, 20 Apr 2009 13:44:07 -0500
Subject: [Developers] Timer Output in human-readable and CSV format
Message-ID: <9B4EC688-21E0-4B7D-8ACC-A49CC056ABC6@cct.lsu.edu>

The enclosed patch adds a new parameter output_all_timers_readable to  
thorn TimerReport.  It outputs the same information as with  
output_all_timers, but creates two files that are meant to be read by  
a human and a spreadsheet, respectively.  The human readable file uses  
fewer columns, the spreadsheet file uses the CSV (comma separated  
values) format.

-erik

-- 
Erik Schnetter <schnetter at cct.lsu.edu>   http://www.cct.lsu.edu/~eschnett/

My email is as private as my paper mail.  I therefore support encrypting
and signing email messages.  Get my PGP key from <http://pgp.mit.edu>.




--- StripMime Report -- processed MIME parts ---
multipart/mixed
  text/plain (text body -- kept)
  application/octet-stream
---

From tradke at aei.mpg.de  Tue Apr 21 05:20:05 2009
From: tradke at aei.mpg.de (Thomas Radke)
Date: Tue, 21 Apr 2009 19:20:05 +0900
Subject: [Developers] Timer Output in human-readable and CSV format
In-Reply-To: <9B4EC688-21E0-4B7D-8ACC-A49CC056ABC6@cct.lsu.edu>
References: <9B4EC688-21E0-4B7D-8ACC-A49CC056ABC6@cct.lsu.edu>
Message-ID: <49ED9DD5.9030903@aei.mpg.de>

Erik Schnetter wrote:
> The enclosed patch adds a new parameter output_all_timers_readable to  
> thorn TimerReport.  It outputs the same information as with  
> output_all_timers, but creates two files that are meant to be read by  
> a human and a spreadsheet, respectively.  The human readable file uses  
> fewer columns, the spreadsheet file uses the CSV (comma separated  
> values) format.

Hi Erik,

the patch looks good, apart from a small typo in Output.c:423 where you 
probably want to write into file_csv.

-- 
Cheers, Thomas.

From schnetter at cct.lsu.edu  Wed Apr 22 12:40:49 2009
From: schnetter at cct.lsu.edu (Erik Schnetter)
Date: Wed, 22 Apr 2009 12:40:49 -0500
Subject: [Developers] Distributing external libraries?
Message-ID: <FB56E9E9-D0BD-4E9C-B913-747A8D1B6154@cct.lsu.edu>

Cactus applications often require external libraries, such as HDF5 or  
GSL, that may not be installed on certain systems, or may be installed  
without some required options.  Building a Cactus application on a  
certain system then requires first configuring and building these  
libraries, which is somewhat inconvenient and could be automated.

Many software packages distribute required external libraries together  
with their software.  We could do something similar:

- Create thorns that wrap e.g. HDF5, i.e., contain a certain (known  
good) version of HDF5, plus some Cactus configure/make magic to  
configure and build the library.  This thorn could e.g. be called  
"HDF5_source" (better suggestions appreciated), and would need to have  
an appropriated licence.
- These thorns would provide the corresponding capability, e.g.  
"HDF5", and could be used instead of the current CactusExternal/HDF5,  
which expects an external HDF5 library.

This scheme works since the thorn providing HDF5 is not mentioned in  
the parameter file, so that replacing HDF5 by HDF5_source would not  
affect the parameter file.  That means that parameter files remain  
machine independent.

Does anybody see problems with this scheme?

Of course, this would require us to update the thorn HDF5_source every  
so often when we want to switch to a new version, but this would still  
be better than manually re-installing (or asking the administrators to  
install) this new version on all the systems we use.

-erik

-- 
Erik Schnetter <schnetter at cct.lsu.edu>   http://www.cct.lsu.edu/~eschnett/

My email is as private as my paper mail.  I therefore support encrypting
and signing email messages.  Get my PGP key from <http://pgp.mit.edu>.



-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 194 bytes
Desc: This is a digitally signed message part
Url : http://www.cactuscode.org/pipermail/developers/attachments/20090422/98bc019b/attachment.bin 

From knarf at cct.lsu.edu  Wed Apr 22 13:09:20 2009
From: knarf at cct.lsu.edu (Frank Loeffler)
Date: Wed, 22 Apr 2009 13:09:20 -0500
Subject: [Developers] Distributing external libraries?
In-Reply-To: <FB56E9E9-D0BD-4E9C-B913-747A8D1B6154@cct.lsu.edu>
References: <FB56E9E9-D0BD-4E9C-B913-747A8D1B6154@cct.lsu.edu>
Message-ID: <20090422180920.GN26466@numrel07.cct.lsu.edu>

Hi,

On Wed, Apr 22, 2009 at 12:40:49PM -0500, Erik Schnetter wrote:
> Does anybody see problems with this scheme?

That depends on the library in question. Things like libjpg or libpng
should be fine. hdf5 should also be ok for serial operation. Parallel
versions might be too tightly coupled to the mpi counterpart on the
system, but that is just an expectation. However, we currently use
the serial version anyway.
Libraries like lapack which can be heavily optimized for a specific
machine and should probably stay in the hands of the system admins.

Frank


From tradke at aei.mpg.de  Wed Apr 22 20:52:30 2009
From: tradke at aei.mpg.de (Thomas Radke)
Date: Thu, 23 Apr 2009 10:52:30 +0900
Subject: [Developers] Distributing external libraries?
In-Reply-To: <FB56E9E9-D0BD-4E9C-B913-747A8D1B6154@cct.lsu.edu>
References: <FB56E9E9-D0BD-4E9C-B913-747A8D1B6154@cct.lsu.edu>
Message-ID: <49EFC9DE.2010305@aei.mpg.de>

Erik Schnetter wrote:
> Cactus applications often require external libraries, such as HDF5 or 
> GSL, that may not be installed on certain systems, or may be installed 
> without some required options.  Building a Cactus application on a 
> certain system then requires first configuring and building these 
> libraries, which is somewhat inconvenient and could be automated.
> 
> Many software packages distribute required external libraries together 
> with their software.  We could do something similar:
> 
> - Create thorns that wrap e.g. HDF5, i.e., contain a certain (known 
> good) version of HDF5, plus some Cactus configure/make magic to 
> configure and build the library.  This thorn could e.g. be called 
> "HDF5_source" (better suggestions appreciated), and would need to have 
> an appropriated licence.
> - These thorns would provide the corresponding capability, e.g. "HDF5", 
> and could be used instead of the current CactusExternal/HDF5, which 
> expects an external HDF5 library.
> 
> This scheme works since the thorn providing HDF5 is not mentioned in the 
> parameter file, so that replacing HDF5 by HDF5_source would not affect 
> the parameter file.  That means that parameter files remain machine 
> independent.
> 
> Does anybody see problems with this scheme?

Hi Erik,

I'd be in favor of changing the arrangement name and keep the thorn name 
(eg. to have CactusIO/HDF5 rather than <some_arrangement>/HDF5_source in 
the thornlist file). The simple reason being that I prefer to also 
activate capability providing thorns in parfiles (even though it's not 
necessary) but then I can use the MakeThornList script to automatically 
create a complete minimal ThornList file from that parfile.

> Of course, this would require us to update the thorn HDF5_source every 
> so often when we want to switch to a new version, but this would still 
> be better than manually re-installing (or asking the administrators to 
> install) this new version on all the systems we use.

Sure, let's try distributing the GSL and/or HDF5 library with Cactus, 
and see how well it works. We could then also incorporate BLAS and 
LAPACK just for convenience; although Frank is right, of course, that 
one should use an external specially optimised package which comes 
bundled with the compiler.

-- 
Cheers, Thomas.

From schnetter at cct.lsu.edu  Wed Apr 22 22:52:50 2009
From: schnetter at cct.lsu.edu (Erik Schnetter)
Date: Wed, 22 Apr 2009 22:52:50 -0500
Subject: [Developers] Distributing external libraries?
In-Reply-To: <49EFC9DE.2010305@aei.mpg.de>
References: <FB56E9E9-D0BD-4E9C-B913-747A8D1B6154@cct.lsu.edu>
	<49EFC9DE.2010305@aei.mpg.de>
Message-ID: <6C353DA6-AC1C-4DFB-8AE4-1DBC72F19550@cct.lsu.edu>

On Apr 22, 2009, at 20:52:30, Thomas Radke wrote:

> Erik Schnetter wrote:
>> Cactus applications often require external libraries, such as HDF5 or
>> GSL, that may not be installed on certain systems, or may be  
>> installed
>> without some required options.  Building a Cactus application on a
>> certain system then requires first configuring and building these
>> libraries, which is somewhat inconvenient and could be automated.
>>
>> Many software packages distribute required external libraries  
>> together
>> with their software.  We could do something similar:
>>
>> - Create thorns that wrap e.g. HDF5, i.e., contain a certain (known
>> good) version of HDF5, plus some Cactus configure/make magic to
>> configure and build the library.  This thorn could e.g. be called
>> "HDF5_source" (better suggestions appreciated), and would need to  
>> have
>> an appropriated licence.
>> - These thorns would provide the corresponding capability, e.g.  
>> "HDF5",
>> and could be used instead of the current CactusExternal/HDF5, which
>> expects an external HDF5 library.
>>
>> This scheme works since the thorn providing HDF5 is not mentioned  
>> in the
>> parameter file, so that replacing HDF5 by HDF5_source would not  
>> affect
>> the parameter file.  That means that parameter files remain machine
>> independent.
>>
>> Does anybody see problems with this scheme?
>
> Hi Erik,
>
> I'd be in favor of changing the arrangement name and keep the thorn  
> name
> (eg. to have CactusIO/HDF5 rather than <some_arrangement>/ 
> HDF5_source in
> the thornlist file). The simple reason being that I prefer to also
> activate capability providing thorns in parfiles (even though it's not
> necessary) but then I can use the MakeThornList script to  
> automatically
> create a complete minimal ThornList file from that parfile.

Okay.  This would be the first time that we have two different thorns  
with the same name.  However, this will also be the first time that we  
have two thorns that cannot (by their construction) compiled at the  
same time.

I suggest a new arrangement for this, maybe CactusLibraries.  CactusIO  
won't work for GSL.

-erik

-- 
Erik Schnetter <schnetter at cct.lsu.edu>   http://www.cct.lsu.edu/~eschnett/

My email is as private as my paper mail.  I therefore support encrypting
and signing email messages.  Get my PGP key from <http://pgp.mit.edu>.



-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 194 bytes
Desc: This is a digitally signed message part
Url : http://www.cactuscode.org/pipermail/developers/attachments/20090422/df7962c9/attachment.bin 

From roland.haas at physics.gatech.edu  Wed Apr 22 23:09:58 2009
From: roland.haas at physics.gatech.edu (Roland Haas)
Date: Thu, 23 Apr 2009 00:09:58 -0400
Subject: [Developers] Distributing external libraries?
In-Reply-To: <6C353DA6-AC1C-4DFB-8AE4-1DBC72F19550@cct.lsu.edu>
References: <FB56E9E9-D0BD-4E9C-B913-747A8D1B6154@cct.lsu.edu>	<49EFC9DE.2010305@aei.mpg.de>
	<6C353DA6-AC1C-4DFB-8AE4-1DBC72F19550@cct.lsu.edu>
Message-ID: <49EFEA16.5090506@mail.gatech.edu>

Hello all,

> I suggest a new arrangement for this, maybe CactusLibraries.  CactusIO 
> won't work for GSL.
For HDF5 (and possibly others, maybe GSL and anything that has a 
XXX_VERSION defined) it would also be important to make sure that the 
include files used are the ones shipped with the library and not the 
ones possibly installed on the system (Or will happen automatically 
since thorn includes might always be sorted before the system ones?).

Yours,
Roland

From tradke at aei.mpg.de  Sat Apr 25 11:31:41 2009
From: tradke at aei.mpg.de (Thomas Radke)
Date: Sat, 25 Apr 2009 18:31:41 +0200
Subject: [Developers] Distributing external libraries?
In-Reply-To: <6C353DA6-AC1C-4DFB-8AE4-1DBC72F19550@cct.lsu.edu>
References: <FB56E9E9-D0BD-4E9C-B913-747A8D1B6154@cct.lsu.edu>	<49EFC9DE.2010305@aei.mpg.de>
	<6C353DA6-AC1C-4DFB-8AE4-1DBC72F19550@cct.lsu.edu>
Message-ID: <49F33AED.7000005@aei.mpg.de>

Erik Schnetter wrote:
> On Apr 22, 2009, at 20:52:30, Thomas Radke wrote:
> 
>> Erik Schnetter wrote:
>>> Cactus applications often require external libraries, such as HDF5 or
>>> GSL, that may not be installed on certain systems, or may be installed
>>> without some required options.  Building a Cactus application on a
>>> certain system then requires first configuring and building these
>>> libraries, which is somewhat inconvenient and could be automated.
>>>
>>> Many software packages distribute required external libraries together
>>> with their software.  We could do something similar:
>>>
>>> - Create thorns that wrap e.g. HDF5, i.e., contain a certain (known
>>> good) version of HDF5, plus some Cactus configure/make magic to
>>> configure and build the library.  This thorn could e.g. be called
>>> "HDF5_source" (better suggestions appreciated), and would need to have
>>> an appropriated licence.
>>> - These thorns would provide the corresponding capability, e.g. "HDF5",
>>> and could be used instead of the current CactusExternal/HDF5, which
>>> expects an external HDF5 library.
>>>
>>> This scheme works since the thorn providing HDF5 is not mentioned in the
>>> parameter file, so that replacing HDF5 by HDF5_source would not affect
>>> the parameter file.  That means that parameter files remain machine
>>> independent.
>>>
>>> Does anybody see problems with this scheme?
>>
>> Hi Erik,
>>
>> I'd be in favor of changing the arrangement name and keep the thorn name
>> (eg. to have CactusIO/HDF5 rather than <some_arrangement>/HDF5_source in
>> the thornlist file). The simple reason being that I prefer to also
>> activate capability providing thorns in parfiles (even though it's not
>> necessary) but then I can use the MakeThornList script to automatically
>> create a complete minimal ThornList file from that parfile.
> 
> Okay.  This would be the first time that we have two different thorns 
> with the same name.  However, this will also be the first time that we 
> have two thorns that cannot (by their construction) compiled at the same 
> time.
> 
> I suggest a new arrangement for this, maybe CactusLibraries.  CactusIO 
> won't work for GSL.

Sure, CactusLibraries sounds good. Or CactusDistribLibraries but this is 
rather long.

-- 
Cheers, Thomas.