[Users] wishlist: GetComponents --avoid-useless-downloads --propose-system-installs

Boud Roukema boud at astro.uni.torun.pl
Tue Sep 25 08:14:32 CDT 2018


Dear Cactus developers/users,

I could not find an appropiate place for proposing wishlist items
for Cactus - please tell me if there is such a place, or add this
as a new wishlist item.

WISHLIST: GetComponents --avoid-useless-downloads --propose-system-installs

AIM: reduce download volume by about 40 percent or so; reduce compile time;
increase likelihood of having wider-software-community-verified versions of
external libraries

DESCRIPTION:
In my recent (Sept 2018) download of Cactus, out of about 500 Mb total
download, I have about 200Mb of downloaded external library source
packages, most of which either (i) are superfluous because they already
exist in my OS (Debian GNU/Linux), or (ii) *should* be superfluous,
since they can more safely be installed using the
apt/apt-get/aptitude/dpkg package management system. This proposal
is to add an OS-dependent option to GetComponents that (i) checks if the
OS is a debian derivative and if yes, then instead of downloading the
source packages, extracts necessary versioning metadata from the OS,
inserts this in the appropriate place in the Cactus system for defining
-I include and -L library shell or makefile variables as appropriate;
and/or (ii) checks if the OS is a debian derivative and if yes, proposes
a list of system packages to install as root user that would avoid
having to download and compile these. Proposed names for the options
are (i) --avoid-useless-downloads and (ii) --propose-system-installs.

DETAILS:

Superfluous downloads on my system:

du -sh Cactus/arrangements/
200M    Cactus/arrangements/

ls -l $(find ./Cactus/arrangements/ExternalLibraries -name "*\.t*gz") | awk 
'{b+=$5} END {print "b=",b," Mb=",b/1024/1024}'

b= 101248488  Mb= 96.5581

du -s ./Cactus/arrangements/ExternalLibraries/*/.svn/pristine/*  | awk '{b+=$1} 
END {print " Mb=",b/1024}'

Mb= 98.5

So it seems to me that 200 Mb (out of 500Mb) total download should have been 
avoided -
this gets to proposal (i). The compilation of some of these is for proposal 
(ii) - users
could avoid compiling packages that are better matched to their system using 
their
native package manager.

[The download was probably, but not certainly, done as per
https://docs.einsteintoolkit.org/et-docs/Simplified_Tutorial_for_New_Users
i.e.

curl -O -L 
https://raw.githubusercontent.com/gridaphobe/CRL/ET_2018_09/GetComponents

chmod u+x GetComponents

./GetComponents 
https://bitbucket.org/einsteintoolkit/manifest/raw/ET_2018_09/einsteintoolkit.th
]


COMMENTS:

Several of the packages provided by the above install are older than the Debian 
stable
versions, so a counterargument of "but the recommended Cactus source packages 
are
more up-to-date" does not apply, at least in this case. Debian stable versions 
of
packages are quite conservative in terms of security support and bug fixing.


(i) Politer names for --avoid-useless-downloads could be:

--avoid-redundant-downloads

--avoid-superfluous-downloads # but people are likely to misspell this


(ii) My idea for --propose-system-installs is *not* to try to do the
install, since asking a user to give the root or sudo password seems
to me unwise, given the whole idea of by default installing Cactus/ET
in user space. I'm only proposing that the perl script do something like
output to the user:

# should find something like "libfftw3-dev liblapack-dev libopenmpi-dev 
openssl";
system_installable = ... # search in Cactus/arrangements/ExternalLibraries/

if(length($system_installable) > 0){
    print("Before doing a full download, you may wish to install the following\n
       packages instead of compiling from source, using");
    print(if $OS == debian_derivative ( "aptitude or
       apt-get" else "your package manager:\n"));
    print("$system_installable");
};


I don't really know perl - the above syntax is just a quick guess.

For the same reasons, I'm probably not the best person to propose a patch.
Cactus+ET have become much easier to install since several years ago -
these two options could make them even more attractive to newbie users.

Cheers
Boud


More information about the Users mailing list