Apptainer support

.pre-commit-config.yaml

@@ -11,7 +11,7 @@ repos:
       - id: trailing-whitespace
         args: [--markdown-linebreak-ext=md]
   - repo: https://github.com/adrienverge/yamllint
-    rev: "v1.26.0"
+    rev: "v1.29.0"
     hooks:
       - id: yamllint
   - repo: https://github.com/asottile/setup-cfg-fmt
@@ -23,7 +23,7 @@ repos:
     hooks:
       - id: black-jupyter
   - repo: https://github.com/PyCQA/isort
-    rev: "5.9.3"
+    rev: 5.12.0
     hooks:
       - id: isort
   # TODO renable when errors are fixed/ignored
@@ -78,7 +78,7 @@ repos:
     rev: 1.1.0
     hooks:
       - id: nbqa-isort
-        additional_dependencies: [isort==5.9.3]
+        additional_dependencies: [isort==5.11.2]
       - id: nbqa-mypy
         additional_dependencies: [mypy==0.910, types-python-dateutil]
       # TODO renable when errors are fixed/ignored

CHANGELOG.md

@@ -6,6 +6,14 @@ Formatted as described on [https://keepachangelog.com](https://keepachangelog.co
 
 ## [Unreleased]
 
+### Added
+
+- Apptainer support ([#290](https://github.com/eWaterCycle/ewatercycle/issues/290))
+
+### Deprecated
+
+- Singularity support
+
 ## [1.4.1] (2022-12-20)
 
 ### Fixed

docs/adding_models.rst

@@ -50,7 +50,7 @@ Package the model together with grpc4bmi server in a docker container
 
 In eWaterCycle models are stored in Docker container images, which can be shared
 through DockerHub. Because Docker is not always available on compute clusters,
-we also create Singularity images. Besides the model code, the container image
+we also create Apptainer images. Besides the model code, the container image
 should install grpc4bmi server as an entrypoint to enable communication with the
 model from outside of the container. We use standardized image names including a
 unique version number for the model. See the section on :ref:`versions<New
@@ -64,9 +64,9 @@ Concretely, these are the steps you should follow:
   https://grpc4bmi.readthedocs.io/en/latest/container/building.html
 * Host Docker container image on `Docker Hub
   <https://hub.docker.com/u/ewatercycle>`_
-* Create Singularity image from Docker with ``singularity build
+* Create Apptainer image from Docker with ``apptainer build
   ./ewatercycle-<model>-grpc4bmi_<version>.sif
-  docker://ewatercycle/<model>-grpc4bmi:<version>``
+  docker://ewatercycle/<model>-grpc4bmi:<version>``.
 
 .. _Make recipe:
 
@@ -162,7 +162,7 @@ To make sure that your model will be available on a new or existing platform,
 you need to make sure that:
 
 * The latest version of eWaterCycle is installed on that platform
-* The singularity image is available on that platform
+* The Apptainer image is available on that platform
 * The example parameter set is available on that platform
 
 Typically these steps should be performed by platform developers and
@@ -171,7 +171,7 @@ maintainers.
 For SURF infrastructure specifically, this requires to the following changes.
 
 * Install version/branch of eWaterCycle Python package with new model version on any running virtual machines
-* Add Singularity image to storage. In our case, we use a dCache folder ``ewcdcache:/singularity-images/<model>-grpc4bmi_<version>.sif``
+* Add Apptainer image to storage. In our case, we use a dCache folder ``ewcdcache:/apptainer-images/<model>-grpc4bmi_<version>.sif``
 * Add container image to infrastructure repository
 
   * data preparation scripts_
@@ -185,7 +185,7 @@ Adding a new version of a model
 ===============================
 
 A model can have different versions. A model version in the eWaterCycle Python
-package corresponds to the tag of Docker image and the version in a Singularity
+package corresponds to the tag of Docker image and the version in a Apptainer
 container image filename. The version of the container image should preferably
 be one of release versions of the model code. Alternatively the version could be
 the name of a feature branch or a date.

docs/examples/hype.ipynb

@@ -134,7 +134,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Calling `setup()` will start up a docker or singularity container. Be careful with calling it multiple times!"
+    "Calling `setup()` will start up a container. Be careful with calling it multiple times!"
    ]
   },
   {
@@ -386,7 +386,7 @@
    "source": [
     "## Cleaning up\n",
     "\n",
-    "Models usually perform some \"wrap up tasks\" at the end of a model run, such as writing the last outputs to disk and releasing memory. In the case of eWaterCycle, another important teardown task is destroying the docker or singularity container in which the model was running. This can free up a lot of resources on your system. Therefore it is good practice to always call `finalize()` when you're done with an experiment."
+    "Models usually perform some \"wrap up tasks\" at the end of a model run, such as writing the last outputs to disk and releasing memory. In the case of eWaterCycle, another important teardown task is destroying the container in which the model was running. This can free up a lot of resources on your system. Therefore it is good practice to always call `finalize()` when you're done with an experiment."
    ]
   },
   {

docs/examples/lisflood.ipynb

@@ -283,7 +283,7 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Running /mnt/data/singularity-images/ewatercycle-lisflood-grpc4bmi_20.10.sif singularity container on port 41619\n",
+      "Running /mnt/data/apptainer-images/ewatercycle-lisflood-grpc4bmi_20.10.sif apptainer container on port 41619\n",
       "/home/vagrant/ewatercycle/docs/examples/ewatercycle_output/lisflood_20210930_093520/lisflood_setting.xml\n",
       "/home/vagrant/ewatercycle/docs/examples/ewatercycle_output/lisflood_20210930_093520\n"
      ]

docs/examples/pcrglobwb.ipynb

@@ -279,7 +279,7 @@
   {
    "cell_type": "markdown",
    "source": [
-    "Calling `setup()` will start up a docker or singularity container. Be careful with calling it multiple times!"
+    "Calling `setup()` will start up a container. Be careful with calling it multiple times!"
    ],
    "metadata": {}
   },
@@ -297,7 +297,7 @@
      "output_type": "stream",
      "name": "stdout",
      "text": [
-      "Running /home/peter/ewatercycle/ewatercycle/ewatercycle-pcrg-grpc4bmi-setters.sif singularity container on port 50639\n"
+      "Running /home/peter/ewatercycle/ewatercycle/ewatercycle-pcrg-grpc4bmi-setters.sif apptainer container on port 50639\n"
      ]
     },
     {
@@ -909,7 +909,7 @@
    "source": [
     "## Cleaning up\n",
     "\n",
-    "Models usually perform some \"wrap up tasks\" at the end of a model run, such as writing the last outputs to disk and releasing memory. In the case of eWaterCycle, another important teardown task is destroying the docker or singularity container in which the model was running. This can free up a lot of resources on your system. Therefore it is good practice to always call `finalize()` when you're done with an experiment."
+    "Models usually perform some \"wrap up tasks\" at the end of a model run, such as writing the last outputs to disk and releasing memory. In the case of eWaterCycle, another important teardown task is destroying the container in which the model was running. This can free up a lot of resources on your system. Therefore it is good practice to always call `finalize()` when you're done with an experiment."
    ],
    "metadata": {}
   },

docs/hpc_to_cluster.rst

@@ -108,26 +108,28 @@ Install eWatercycle package:
 
 - ``pip install ewatercycle``
 
-**4. Create Singularity Container**
+**4. Create Apptainer container**
 
-On Snellius, Docker requires root access and can therefore not be used. Singularity is similar to, and integrates well with Docker.         It also requires root access, but it is pre-installed on the compute nodes on Snellius.
+On Snellius, Docker requires root access and can therefore not be used.
+Apptainer is similar to, and integrates well with Docker.
+It does not require root access, and is pre-installed on the compute nodes on Snellius.
 
-The first step to run the model on a compute node is thus to use singularity to create a Singularity image (``.sif`` file) based on the Docker image. This is done with (note the ``srun`` command to access the compute node):
+The first step to run the model on a compute node is thus to use apptainer to create a Apptainer image (``.sif`` file) based on the Docker image. This is done with (note the ``srun`` command to access the compute node):
 
-- ``srun -N 1 -t 40 -p short singularity build --disable-cache ewatercycle-wflow-grpc4bmi.sif docker://ewatercycle/wflow-grpc4bmi:latest``
+- ``srun -N 1 -t 40 -p short apptainer build --disable-cache ewatercycle-wflow-grpc4bmi.sif docker://ewatercycle/wflow-grpc4bmi:latest``
 
 This is an example for the wflow_sbm model, change to the correct Docker container:
 
 -  ``docker://ewatercycle/{model}-grpc4bmi:{version}``
 
-**5. Adjust code to run Singularity container**
+**5. Adjust code to run Apptainer container**
 
-Code should be adjusted to run Singularity instead of Docker following:
+Code should be adjusted to run Apptainer instead of Docker following:
 ::
 
-    from grpc4bmi.bmi_client_singularity import BmiClientSingularity
+    from grpc4bmi.bmi_client_apptainer import BmiClientApptainer
 
-    model = BmiClientSingularity(image='ewatercycle-wflow-grpc4bmi.sif', input_dirs=[input_dir], work_dir=work_dir)
+    model = BmiClientApptainer(image='ewatercycle-wflow-grpc4bmi.sif', input_dirs=[input_dir], work_dir=work_dir)
     ...
 
 **6. Adjust code to use Scratch directory**

docs/system_setup.rst

@@ -133,17 +133,23 @@ Install container engine
 ------------------------
 
 In eWaterCycle package, the hydrological models are run in containers
-with engines like `Singularity <https://singularity.lbl.gov/>`__ or
-`Docker <https://www.docker.com/>`__. At least Singularity or Docker
+with engines like `Apptainer <https://apptainer.org/>`__ or
+`Docker <https://www.docker.com/>`__. At least Apptainer or Docker
 should be installed.
 
+.. note::
+
+    Apptainer is the open source fork of `Singularity <https://sylabs.io/singularity/>`__.
+    In the eWaterCycle project we prefer to use Apptainer over Singularity.
+    Apptainer uses the same image format as Singularity.
+
 Installing a container engine requires root permission on the machine.
 
-Singularity
-~~~~~~~~~~~
+Apptainer
+~~~~~~~~~
 
-Install Singularity using
-`instructions <https://singularity.hpcng.org/user-docs/master/quick_start.html>`__.
+Install Apptainer using
+`instructions <https://apptainer.org/docs/user/main/quick_start.html>`__.
 
 Docker
 ~~~~~~
@@ -168,9 +174,9 @@ The configuration can be set in Python with
     import ewatercycle
     import ewatercycle.parameter_sets
     # Which container engine is used to run the hydrological models
-    ewatercycle.CFG['container_engine'] = 'singularity'  # or 'docker'
-    # If container_engine==singularity then where can the singularity images files (*.sif) be found.
-    ewatercycle.CFG['singularity_dir'] = './singularity-images'
+    ewatercycle.CFG['container_engine'] = 'apptainer'   # or 'docker'
+    # If container_engine==apptainer then where can the Apptainer images files (*.sif) be found.
+    ewatercycle.CFG['apptainer_dir'] = './apptainer-images'
     # Directory in which output of model runs is stored. Each model run will generate a sub directory inside output_dir
     ewatercycle.CFG['output_dir'] = './'
     # Where can GRDC observation files (<station identifier>_Q_Day.Cmd.txt) be found.
@@ -206,8 +212,8 @@ SURF <https://servicedesk.surfsara.nl/wiki/display/WIKI/Snellius>`_:
 
 .. code:: yaml
 
-   container_engine: singularity
-   singularity_dir: /projects/0/wtrcycle/singularity-images
+   container_engine: apptainer
+   apptainer_dir: /projects/0/wtrcycle/apptainer-images
    output_dir: /scratch-shared/ewatercycle
    grdc_location:  /projects/0/wtrcycle/GRDC/GRDC_GCOSGTN-H_27_03_2019
    parameterset_dir: /projects/0/wtrcycle/parameter-sets
@@ -229,8 +235,8 @@ Cloud:
 
 .. code:: yaml
 
-   container_engine: singularity
-   singularity_dir: /mnt/data/singularity-images
+   container_engine: apptainer
+   apptainer_dir: /mnt/data/apptainer-images
    output_dir: /scratch
    grdc_location: /mnt/data/GRDC
    parameterset_dir: /mnt/data/parameter-sets
@@ -259,22 +265,22 @@ Docker images will be downloaded with ``docker pull``:
     docker pull ewatercycle/wflow-grpc4bmi:2020.1.3
     docker pull ewatercycle/hype-grpc4bmi:feb2021
 
-Singularity
-~~~~~~~~~~~
+Apptainer
+~~~~~~~~~
 
-Singularity images should be stored in configured directory
-(``ewatercycle.CFG['singularity_dir']``) and can build from Docker with:
+Apptainer images should be stored in configured directory
+(``ewatercycle.CFG['apptainer_dir']``) and can build from Docker with:
 
 .. code:: shell
 
-    cd {ewatercycle.CFG['singularity_dir']}
-    singularity build ewatercycle-lisflood-grpc4bmi_20.10.sif docker://ewatercycle/lisflood-grpc4bmi:20.10
-    singularity build ewatercycle-marrmot-grpc4bmi_2020.11.sif docker://ewatercycle/marrmot-grpc4bmi:2020.11
-    singularity build ewatercycle-pcrg-grpc4bmi_setters.sif docker://ewatercycle/pcrg-grpc4bmi:setters
-    singularity build ewatercycle-wflow-grpc4bmi_2020.1.1.sif docker://ewatercycle/wflow-grpc4bmi:2020.1.1
-    singularity build ewatercycle-wflow-grpc4bmi_2020.1.2.sif docker://ewatercycle/wflow-grpc4bmi:2020.1.2
-    singularity build ewatercycle-wflow-grpc4bmi_2020.1.3.sif docker://ewatercycle/wflow-grpc4bmi:2020.1.3
-    singularity build ewatercycle-hype-grpc4bmi_feb2021.sif docker://ewatercycle/hype-grpc4bmi:feb2021
+    cd {ewatercycle.CFG['apptainer_dir']}
+    apptainer build ewatercycle-lisflood-grpc4bmi_20.10.sif docker://ewatercycle/lisflood-grpc4bmi:20.10
+    apptainer build ewatercycle-marrmot-grpc4bmi_2020.11.sif docker://ewatercycle/marrmot-grpc4bmi:2020.11
+    apptainer build ewatercycle-pcrg-grpc4bmi_setters.sif docker://ewatercycle/pcrg-grpc4bmi:setters
+    apptainer build ewatercycle-wflow-grpc4bmi_2020.1.1.sif docker://ewatercycle/wflow-grpc4bmi:2020.1.1
+    apptainer build ewatercycle-wflow-grpc4bmi_2020.1.2.sif docker://ewatercycle/wflow-grpc4bmi:2020.1.2
+    apptainer build ewatercycle-wflow-grpc4bmi_2020.1.3.sif docker://ewatercycle/wflow-grpc4bmi:2020.1.3
+    apptainer build ewatercycle-hype-grpc4bmi_feb2021.sif docker://ewatercycle/hype-grpc4bmi:feb2021
     cd -
 
 Download example parameter sets
@@ -337,7 +343,7 @@ configuration file.
         supported_model_versions: !!set {2020.1.1: null}
         target_model: wflow
     parameterset_dir: /home/verhoes/git/eWaterCycle/ewatercycle/docs/examples/parameter-sets
-    singularity_dir: None
+    apptainer_dir: None
 
 
 .. code:: ipython3

docs/user_guide.ipynb

@@ -55,7 +55,7 @@
     {
      "data": {
       "text/plain": [
-       "Config({'container_engine': 'singularity',\n",
+       "Config({'container_engine': 'apptainer',\n",
        "        'ewatercycle_config': PosixPath('/home/fakhereh/.config/ewatercycle/ewatercycle.yaml'),\n",
        "        'grdc_location': PosixPath('/projects/0/wtrcycle/GRDC/GRDC_GCOSGTN-H_27_03_2019'),\n",
        "        'output_dir': PosixPath('/scratch-shared/ewatercycle'),\n",
@@ -145,7 +145,7 @@
        "                                                                               '2020.1.2'},\n",
        "                                                  'target_model': 'wflow'}},\n",
        "        'parameterset_dir': PosixPath('/projects/0/wtrcycle/parameter-sets'),\n",
-       "        'singularity_dir': PosixPath('/projects/0/wtrcycle/singularity-images')})"
+       "        'apptainer_dir': PosixPath('/projects/0/wtrcycle/apptainer-images')})"
       ]
      },
      "execution_count": 2,
@@ -745,7 +745,7 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Running /projects/0/wtrcycle/singularity-images/ewatercycle-wflow-grpc4bmi_2020.1.2.sif singularity container on port 35805\n"
+      "Running /projects/0/wtrcycle/apptainer-images/ewatercycle-wflow-grpc4bmi_2020.1.2.sif apptainer container on port 35805\n"
      ]
     }
    ],

setup.cfg

@@ -40,7 +40,7 @@ install_requires =
     basic_modeling_interface
     cftime
     esmvaltool>=2.4.0
-    grpc4bmi>=0.2.12,<0.3
+    grpc4bmi>=0.2.16,<0.3
     grpcio
     hydrostats
     matplotlib>=3.5.0

src/ewatercycle/config/__init__.py

@@ -12,9 +12,8 @@
     Config({'container_engine': None,
             'grdc_location': None,
             'output_dir': None,
-            'singularity_dir': None,
-            'wflow.docker_image': None,
-            'wflow.singularity_image': None})
+            'apptainer_dir': None,
+            })
 
 By default all values are initialized as ``None``.
 
@@ -75,13 +74,11 @@
 .. code-block:: yaml
 
     grdc_location: /data/grdc
-    container_engine: singularity
-    singularity_dir: /data/singularity-images
+    container_engine: apptainer
+    apptainer_dir: /data/apptainer-images
     output_dir: /scratch
-    # Created with cd  /data/singularity-images &&
-    # singularity pull docker://ewatercycle/wflow-grpc4bmi:2020.1.1
-    wflow.singularity_images: wflow-grpc4bmi_2020.1.1.sif
-    wflow.docker_images: ewatercycle/wflow-grpc4bmi:2020.1.1
+    # Created with cd  /data/apptainer-images &&
+    # apptainer pull docker://ewatercycle/wflow-grpc4bmi:2020.1.1
 """
 
 from ._config_object import CFG, DEFAULT_CONFIG, SYSTEM_CONFIG, USER_HOME_CONFIG, Config

src/ewatercycle/config/_config_object.py

@@ -82,7 +82,7 @@ def _save_to_stream(self, stream: TextIO):
         cp.pop("ewatercycle_config", None)
 
         cp["grdc_location"] = str(cp["grdc_location"])
-        cp["singularity_dir"] = str(cp["singularity_dir"])
+        cp["apptainer_dir"] = str(cp["apptainer_dir"])
         cp["output_dir"] = str(cp["output_dir"])
         cp["parameterset_dir"] = str(cp["parameterset_dir"])
 

src/ewatercycle/config/_lisflood_versions.py

@@ -6,7 +6,7 @@
 version_images = {
     "20.10": {
         "docker": "ewatercycle/lisflood-grpc4bmi:20.10",
-        "singularity": "ewatercycle-lisflood-grpc4bmi_20.10.sif",
+        "apptainer": "ewatercycle-lisflood-grpc4bmi_20.10.sif",
     }
 }
 
@@ -15,5 +15,5 @@ def get_docker_image(version):
     return version_images[version]["docker"]
 
 
-def get_singularity_image(version, singularity_dir: Path):
-    return singularity_dir / version_images[version]["singularity"]
+def get_apptainer_image(version, apptainer_dir: Path):
+    return apptainer_dir / version_images[version]["apptainer"]

src/ewatercycle/config/_validators.py

@@ -135,6 +135,10 @@ def validate_path(value, allow_none=False):
 _validators = {
     "grdc_location": validate_path_or_none,
     "container_engine": validate_string_or_none,
+    "apptainer_dir": validate_path_or_none,
+    # TODO mark as deprecated,
+    # TODO use if apptainer_dir not set,
+    # TODO make key optional
     "singularity_dir": validate_path_or_none,
     "output_dir": validate_path_or_none,
     "parameterset_dir": validate_path_or_none,