diff --git a/README.rst b/README.rst new file mode 100644 index 0000000000000000000000000000000000000000..c29a590af31b643b3ae901c564f33e9789b51a94 --- /dev/null +++ b/README.rst @@ -0,0 +1,173 @@ +MPSD Software manager +===================== + +.. contents:: + +This repository provides the ``mpsd-software`` tool which is used to install +package sets and toolchains on the `MPSD HPC cluster +<https://computational-science.mpsd.mpg.de/docs/mpsd-hpc.html>`__. + +It can also be used to install the software on other machines, such as Linux +laptops and desktops. This can be useful to work - on a local machine - with +the same software environment, for example to debug a problem. + +Note that this tool is under development, and the recommended way to install and +use as well as the user interface may change. This document will be kept +up-to-date in any case. + +Quick start +----------- + +To install, for example, the ``foss2022a-serial`` toolchain: + +1. Install this mpsd-software-manager Python package. The recommended way is to + use ``pipx`` to that this tool is available independent from the use of any + other python environments:: + + $ pipx install git+https://gitlab.gwdg.de/mpsd-cs/mpsd-software-manager + + +2. Navigate to the location in your file system where you would like to store + the compiled software. (Once compiled, the location cannot be changed.) For + example:: + + $ cd /home/user/mpsd-software + + +3. Initiate the installation at this location using:: + + $ mpsd-software init + + (This creates a hidden file ``.mpsd-software-root`` to tag the location for + as the root of the installation. All compiled files, logs etc are written in + or below this subdirectory.) + +4. From the same directory, run the command to install the ``foss2022a-serial`` + toolchain:: + + $ mpsd-software install dev-23a foss2022a-serial + + This will take some time (up to several hours depending on hardware). + +5. To see the installation status, and the required ``module use`` command line + to activate the created modules, try the ``status`` command:: + + $ mpsd-software status dev-23a + + Installed toolchains (dev-23a): + + - cascadelake + foss2022a-serial + [module use /home/user/mpsd-software/dev-23a/cascadelake/lmod/Core] + + + +Documentation +------------- + +More detailed documentation that goes beyond the `Quick Start`_ section. + +Package sets and toolchains +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +- Package sets are a combination of particular versions of multiple software + packages (such as anaconda3, or gcc and fftw). In the way the SSU + Computational Science provides software on the MPSD HPC cluster, and for the + Octopus continuous integration services, these package sets are compiled + together (using Spack). + +- Toolchains are a particular type of package sets: + + - the choice of software packages (typically a compiler and scientific + computing libraries) and their versions follows the Easybuild toolchains + (such as the `FOSS toolchains + <https://docs.easybuild.io/common-toolchains/#common_toolchains_foss>`__). + + - all packages grouped together in a toolchain can be loaded together using + the ``module load`` command. + + Example: the ``foss2022a-serial`` tool chain provides (in spack notation):: + + - gcc@11.3.0 + - binutils@2.38+headers+ld + - fftw@3.3.10+openmp~~mpi + - openblas@0.3.20 + + - in addition to the Easybuild-driven choice of packages, there are + additional packages included in each package which support the build of + Octopus within these toolchains. For ``foss2022a-serial`` these packages + include:: + + - libxc@5.2.3 # octopus-dependencies: + - gsl@2.7.1 + - sparskit@develop # 2021.06.01 + - nlopt@2.7.0 + - libgd@2.2.4 # 2.3.1 + - libvdwxc@0.4.0~~mpi + - nfft@3.2.4 + - berkeleygw@2.1~~mpi~scalapack + - python@3.9.5 + - cgal@5.0.3 # 5.2 + - hdf5@1.12.2~mpi + - etsf-io@1.0.4 + + +MPSD software releases +~~~~~~~~~~~~~~~~~~~~~~ + +As `explained in the MPSD HPC documentation <https://computational-science.mpsd.mpg.de/docs/mpsd-hpc.html#software>`__, we +label software releases available on the HPC using a naming scheme of the year +(such as ``23``) and a letter starting from ``a``. There is an exception that +the first available software version is ``dev-23a`` (starting with ``dev-`` to +indicate this was a development prototype). + +At the moment (June 2023), there is only one release (that is ``dev-23a``). + +For each MPSD software release, multiple toolchains and package sets are available:: + + $> mpsd-software available dev-23a + + MPSD software release dev-23a, AVAILABLE for installation are + Toolchains: + foss2021a-cuda-mpi + foss2021a-mpi + foss2021a-serial + foss2022a-cuda-mpi + foss2022a-mpi + foss2022a-serial + Package sets: + global (octopus@12.1, octopus@12.1) + global_generic (anaconda3@2022.10) + + +Prerequisites +~~~~~~~~~~~~~ + +What needs to be installed for the installation to succeed? + +The ``mpsd-software-manager`` python package. + +- This needs a recent Python (3.9 or later). +- Install via pip or pipx. + + Pipx commands are: + + - to install: ``pipx install git+https://gitlab.gwdg.de/mpsd-cs/mpsd-software-manager`` + - to update: ``pipx upgrade mpsd-software-manager`` + - to uninstall: ``pipx uninstall mpsd-software-manager`` + +- Requirements to be able to run `spack <spack.readthedocs.io>`__ + + - Please check https://spack.readthedocs.io/en/latest/getting_started.html#system-prerequisites + + +The compilation of the ``*-mpi`` toolchains needs linux headers installed. (TODO Add more detail.) + +Working example +~~~~~~~~~~~~~~~ + +We have a continuous integration example at XXX, that shows the complete +compilation cycle (including compilation of Octopus). (TODO) + + +