README.rst

@@ -4,8 +4,8 @@ MPSD Software manager
 .. contents::
 
 This repository provides the ``mpsd-software`` tool which is used to install
-package sets and toolchains on the `MPSD HPC cluster
-<https://computational-science.mpsd.mpg.de/docs/mpsd-hpc.html>`__.
+package sets and toolchains on the
+`MPSD HPC cluster <https://computational-science.mpsd.mpg.de/docs/mpsd-hpc.html>`__.
 
 It can also be used to install the software on other machines, such as Linux
 laptops and desktops. This can be useful to work - on a local machine - with
@@ -28,21 +28,23 @@ To install, for example, the ``foss2022a-serial`` toolchain:
 
 
 2. Navigate to the location in your file system where you would like to store
-   the compiled software. (Once compiled, the location cannot be changed.) For
-   example::
+   your "MPSD software instance" that contains the compiled software. Once
+   compiled, the location cannot be changed. For example::
 
      $ cd /home/user/mpsd-software
 
+   Future calls of the `mpsd-software` command need to be executed from this
+   "mpsd-software-root" directory.
 
 .. comment:
     3. Initiate the installation at this location using::
-    
+
          $ mpsd-software init
-    
+
        (This creates a hidden file ``.mpsd-software-root`` to tag the location for
        as the root of the installation. All compiled files, logs etc are written in
        or below this subdirectory.)
-    
+
 3. From the same directory, run the command to install the ``foss2022a-serial``
    toolchain::
 
@@ -61,6 +63,17 @@ To install, for example, the ``foss2022a-serial`` toolchain:
          foss2022a-serial
          [module use /home/user/mpsd-software/dev-23a/cascadelake/lmod/Core]
 
+5. To compile Octopus, source the provided configure script, for example ``foss2022a-serial-config.sh``, as
+   `explained here <https://computational-science.mpsd.mpg.de/docs/mpsd-hpc.html#loading-a-toolchain-to-compile-octopus>`__).
+   The configure scripts are located in ``dev-23a/spack-environments/octopus``::
+
+     $ ls -1 dev-23a/spack-environments/octopus
+
+     foss2021a-cuda-mpi-config.sh
+     foss2021a-mpi-config.sh
+     foss2021a-serial-config.sh
+     foss2022a-cuda-mpi-config.sh
+     foss2022a-mpi-config.sh
 
 
 Documentation
@@ -161,12 +174,15 @@ The ``mpsd-software-manager`` python package.
 
   - Please check https://spack.readthedocs.io/en/latest/getting_started.html#system-prerequisites
 
+- The installation is only expected to work for x86 architectures at the moment.
+
+- The installation is only expected to work on Linux at the moment (i.e. not on OSX).
 
 Requirements for particular toolchains and package sets
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 - ``foss*-serial`` should compile with the dependencies outlined above
-- ``foss*-mpi`` currently needs linux header files installed (to compile the ``kmem`` package)
+- ``foss*-mpi`` currently needs linux header files installed (to compile the ``knem`` package)
 - ``foss*-cuda-mpi`` (proably as `*-mpi, needs testing TODO)
 
 
@@ -174,10 +190,62 @@ Working example
 ~~~~~~~~~~~~~~~
 
 There is an
-[example](https://github.com/mpsd-computational-science/octopus-with-mpsd-software)
+`example <https://github.com/mpsd-computational-science/octopus-with-mpsd-software>`__
 compilation that shows the complete compilation cycle (including compilation of
 Octopus) using the ``foss2022a-serial`` toolchain. 
 
 
 
+Frequently asked questions
+--------------------------
+
+- Can I install the ``mpsd-software-manager`` package in a python virtual environment?
+
+  Yes. ``pipx`` is probably more convenient, but you can create your own Pyton
+  virtual environment and install the ``mpsd-software-manager`` in that as a
+  regular python package::
+
+    python3 -m venv venv
+    . venv/bin/activate
+    pip install git+https://gitlab.gwdg.de/mpsd-cs/mpsd-software-manager
+
+  You just need to activate that python virtual environment before being able to
+  use the tool.
+
+- Does the command write anything outside the mpsd-software-root directory?
+
+  No. All changes to disk take place in and below the mpsd-software-root
+  directory (which is the one in which the ``mpsd-software`` command is called).
+
+- How can I uninstall the mpsd-software?
+
+  For now, the easiest is to delete the ``mpsd-software-root`` directory. You
+  can probably delete just a release subdirectory (such as ``dev-23a``) if you
+  have multiple release subdirectories installed and you only want to delete
+  one. (Untested.)
+
+- How long does the compilation take?
+
+  This depends on the hardware. A few hours are typical per toolchain. If a
+  second toolchain is compiled in the same MPSD software instance is likely to
+  be faster, in particular if the same compiler is used (and thus the compiler
+  does not need to be re-compiled for the second toolchain).
+
+- How much disk storage do I need?
+
+  A toolchain needs of the order of 5GB on disk. The second or third toolchain
+  (in the same MPSD software instance) will use less additional space, as
+  libraries and tools are re-used where possible.
+
+- Can I have more than one MPSD software instance?
+
+  Yes.
+
+  We call "MPSD software instance" all the compiled software that is stored
+  in and below a "mpsd-software-root" directory (see instructions above).
+
+  It is possible to install multiple MPSD software instances on the same
+  computer (just in different (not nested) directories. This makes it possible
+  to experiment with toolchains etc.
+
 

src/mpsd_software_manager/mpsd_software.py

@@ -2,7 +2,7 @@
 
 """mpsd-software: tool for installation of software as on MPSD HPC."""
 
-__version__ = "2023.6.16"
+__version__ = "2023.6.19"
 
 import argparse
 import datetime
@@ -393,7 +393,9 @@ def get_available_package_sets(mpsd_release: str) -> List[str]:
 
     # find package_sets by cloning repository and checking out right branch
     clone_repo(
-        tmp_dir_path, config_vars["spack_environments_repo"], branch=mpsd_release
+        tmp_dir_path,
+        config_vars["spack_environments_repo"],
+        branch=f"releases/{mpsd_release}",
     )
     # look for directories defining the package_sets
 
@@ -802,7 +804,7 @@ def prepare_environment(mpsd_release: str, root_dir: Path) -> List[str]:
     else:
         repo_url = config_vars["spack_environments_repo"]
         logging.info(f"cloning repository {repo_path} from {repo_url}")
-        clone_repo(repo_path, repo_url, branch=mpsd_release)
+        clone_repo(repo_path, repo_url, branch=f"releases/{mpsd_release}")
 
     logging.getLogger("print").info(
         f"Release {mpsd_release} is prepared in {release_base_dir}"

tests/test_mpsd_software.py

@@ -146,7 +146,7 @@ def test_prepare_environment(tmp_path):
         capture_output=True,
     )
     git_branch_stdout = git_branch_output_raw.stdout.decode("utf-8")
-    assert f"* {mpsd_release_to_test}" in git_branch_stdout
+    assert f"* releases/{mpsd_release_to_test}" in git_branch_stdout
 
     # check that result is a list and contains atleast ['global','foss2021a-mpi']
     assert isinstance(result, list)
@@ -185,7 +185,7 @@ def test_record_script_execution_summary(tmp_path):
     # Check that the log file has "Spack environments branch: dev-23a " in the last line
     with open(root_dir / cmd_log_file, "r") as f:
         last_line = f.readlines()[-1]
-        assert "Spack environments branch: dev-23a " in last_line
+        assert "Spack environments branch: releases/dev-23a" in last_line
 
 
 def test_install_environment_wrong_package_set(tmp_path):