[Developers] Test suite criteria

Steve White steve.white at aei.mpg.de
Fri Jul 14 13:24:56 CDT 2006


Erik,

I don't think this addresses a fundamental problem with the relative
error calculation in the test suites.  Namely, when I looked at some of
the failure messages, I saw very large relative errors in data that looked
very good. 

Also, some files are reporting stuff like:
	"ERROR: How did I get here, maximum difference is 2 and maximum value if
0 for ADM_BS_K.minimum.asc"

This is from 
	lib/sbin/RunTestUtils.pl

Beside the typo, something is wrong in the calculation:  It's failing
because the max absolute value is zero, when it shouldn't be.


On 13.07.06, Erik Schnetter wrote:
> The Cactus test suites check for differences using both the absolute  
> and the relative differences.  A test is marked as failing if either  
> of the differences is too large.  There was one case where the  
> relative error was 1e-15, but the absolute error 1e-12 (since the  
> value itself was about 1000), and thus the test was marked as  
> failing.  However, things don't get much better than a relative  
> difference of 1e-15.
> 
> I suggest to change the criteria from "either" to "both".  That is, a  
> test would fail only if both the absolute and relative differences  
> are too large.
> 
> Comments?
> 
> -erik
> 
> -- 
> Erik Schnetter <schnetter at cct.lsu.edu>
> 
> My email is as private as my paper mail.  I therefore support encrypting
> and signing email messages.  Get my PGP key from www.keyserver.net.
> 
> 
> 



> _______________________________________________
> Developers mailing list
> Developers at cactuscode.org
> http://www.cactuscode.org/mailman/listinfo/developers


-- 
Steve White : Programmer
Max-Planck-Institut für Gravitationsphysik      Albert-Einstein-Institut
Am Mühlenberg 1, D-14476 Golm, Germany                  +49-331-567-7625



More information about the Developers mailing list