require docker compose v2 support (#471)

.github/workflows/tests-docker.yml

@@ -22,7 +22,7 @@ jobs:
     - name: display Docker and Python versions 📦
       run: |
         docker version
-        docker-compose version
+        docker compose version
         python3 -V
     - name: setup wis2box configuration
       run: |
@@ -92,5 +92,5 @@ jobs:
     - name: failed tests 🚩
       if: ${{ failure() }}
       run: |
-        docker-compose --file docker-compose.yml --file docker-compose.override.yml --file docker-compose.monitoring.yml --env-file dev.env --project-name wis2box_project ps
-        docker-compose --file docker-compose.yml --file docker-compose.override.yml --file docker-compose.monitoring.yml --env-file dev.env --project-name wis2box_project logs
+        docker compose --file docker-compose.yml --file docker-compose.override.yml --file docker-compose.monitoring.yml --env-file dev.env --project-name wis2box_project ps
+        docker compose --file docker-compose.yml --file docker-compose.override.yml --file docker-compose.monitoring.yml --env-file dev.env --project-name wis2box_project logs

docs/source/community/troubleshooting.rst

@@ -5,36 +5,6 @@ Troubleshooting
 
 This page lists several commonly seen issues and how to address them.
 
-'./docker-compose.yml' is invalid
-----------------------------------------
-
-When starting wis2box you see the errors:
-
-.. code-block:: bash
-
-    ERROR: The Compose file './docker-compose.yml' is invalid because:
-    Unsupported config option for volumes: 'auth-data'
-    Unsupported config option for services: 'wis2box-auth'
-
-check the version of docker-compose you are running with:
-
-.. code-block:: bash
-
-    docker-compose --version
-
-if not 1.29.2 you can install this using the following docker-compose :
-
-.. code-block:: bash
-
-    # download docker-compose 1.29.2
-    sudo curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
-    # set executable
-    sudo chmod +x /usr/local/bin/docker-compose
-    # remove current version
-    sudo rm /usr/bin/docker-compose
-    # set link to downloaded version
-    sudo ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose
-
 
 OSError: Missing data mappings
 ------------------------------
@@ -100,7 +70,7 @@ The Access Key Id you provided does not exist in our records
 ------------------------------------------------------------
 
 If you see this error when uploading data to the wis2box-incoming storage, you have provided the wrong username and/or password to access MinIO.
-Check the values for ``WIS2BOX_BROKER_USERNAME`` and ``WIS2BOX_BROKER_PASSWORD`` you have provided in your ``dev.env`` file. 
+Check the values for ``WIS2BOX_STORAGE_USERNAME`` and ``WIS2BOX_STORAGE_PASSWORD`` you have provided in your ``dev.env`` file. 
 The default username/password for MinIO is ``minio/minio123``.
 
 Topic Hierarchy validation error: No plugins for ... in data mappings
@@ -132,7 +102,8 @@ Please check all services are Running using the following command:
 
 Possible issues are:
 
-- port 80 is already in use, the nginx-service will fail to start if there is already a web-server running on your instance
+- The directory defined by WIS2BOX_HOST_DATADIR does not contain the file 'data-mappings.yml' or the file is invalid
+- The directory defined by WIS2BOX_HOST_DATADIR does not contain the file 'metastation/station/station_list.csv' or the file is invalid
 - WIS2BOX_STORAGE_PASSWORD is too short, minio will fail to start if you specify a WIS2BOX_STORAGE_PASSWORD of less than 8 characters
 
 wisbox-UI is empty
@@ -152,5 +123,5 @@ And repeat the commands for adding your dataset and publishing your metadata, to
 .. code-block:: bash
 
   python3 wis2box-ctl.py login
-  wis2box data add-collection ${WIS2BOX_HOST_DATADIR}/surface-weather-observations.yml
-  wis2box metadata discovery publish ${WIS2BOX_HOST_DATADIR}/surface-weather-observations.yml
+  wis2box data add-collection ${WIS2BOX_DATADIR}/surface-weather-observations.yml
+  wis2box metadata discovery publish ${WIS2BOX_DATADIR}/surface-weather-observations.yml

docs/source/user/data-ingest.rst

@@ -120,7 +120,7 @@ Then start the ``wis2box-ftp`` service with the following command:
 
 .. code-block:: bash
 
-    docker-compose -f docker-compose.wis2box-ftp.yml --env-file ftp.env up -d
+    docker compose -f docker-compose.wis2box-ftp.yml --env-file ftp.env up -d
 
 When using the wis2box-ftp service to ingest data, please note that the topic is determined by the directory structure in which the data arrives.
 
@@ -154,7 +154,7 @@ You then you can activate the optional 'wis2box-data-subscriber' service as foll
 
 .. code-block:: bash
 
-    docker-compose -f docker-compose.data-subscriber.yml --env-file dev.env up -d
+    docker compose -f docker-compose.data-subscriber.yml --env-file dev.env up -d
 
 See the GitHub `wis2box-data-subscriber`_ repository for more information on this service.
 

docs/source/user/downloading-data.rst

@@ -49,13 +49,13 @@ To start a continuous subscribe and download process, run the ``wis2box-subscrib
 
 .. code-block:: bash
 
-   docker-compose -f docker.subscribe-download.yml up -d --build
+   docker compose -f docker.subscribe-download.yml up -d --build
 
 To stop the subscribe and download process, run the following command:
 
 .. code-block:: bash
 
-   docker-compose -f docker.subscribe-download.yml down
+   docker compose -f docker.subscribe-download.yml down
 
 
 Running pywis-pubsub interactively

docs/source/user/getting-started.rst

@@ -37,7 +37,7 @@ wis2box requires the following prior to installation:
 
    Python,3.8 or higher
    Docker Engine, 20.10.14 or higher
-   Docker Compose, 1.29.2
+   Docker Compose, 2.0 or higher
 
 The following commands be used to setup the required software on Ubuntu 20.04 LTS:
 
@@ -48,17 +48,17 @@ The following commands be used to setup the required software on Ubuntu 20.04 LT
     sudo chmod a+r /etc/apt/keyrings/docker.gpg
     sudo echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
     sudo apt-get -y update
-    sudo apt-get install -y docker-ce unzip python3-pip
+    sudo apt-get install -y docker-ce docker-compose-plugin unzip python3-pip
     sudo pip3 install pip --upgrade
     sudo pip3 install pyopenssl --upgrade
-    sudo pip3 install docker-compose==1.29.2 requests==2.26.0 urllib3==1.26.0
+    sudo pip3 install requests==2.26.0 urllib3==1.26.0
 
 The following commands can be used to inspect the available versions of Python, Docker and Docker Compose on your system:
 
 .. code-block:: bash
 
     docker version
-    docker-compose version
+    docker compose version
     python3 -V
 
 Once you have verified these requirements, go to :ref:`setup` for a step-by-step guide to install and configure your wis2box.

wis2box-ctl.py

@@ -24,6 +24,11 @@
 import os
 import subprocess
 
+if subprocess.call(['docker', 'compose'], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL) > 0:
+    DOCKER_COMPOSE_COMMAND = 'docker-compose'
+else:
+    DOCKER_COMPOSE_COMMAND = 'docker compose'
+
 DOCKER_COMPOSE_ARGS = """
     --file docker-compose.yml
     --file docker-compose.override.yml
@@ -152,22 +157,22 @@ def make(args) -> None:
     container = "wis2box-management" if not args.args else ' '.join(args.args)
 
     if args.command == "config":
-        run(args, split(f'docker-compose {docker_compose_args} config'))
+        run(args, split(f'{DOCKER_COMPOSE_COMMAND} {docker_compose_args} config'))
     elif args.command == "build":
         run(args, split(
-            f'docker-compose {docker_compose_args} build {containers}'))
+            f'{DOCKER_COMPOSE_COMMAND} {docker_compose_args} build {containers}'))
     elif args.command in ["up", "start", "start-dev"]:
         run(args, split(
             'docker plugin install grafana/loki-docker-driver:latest --alias loki --grant-all-permissions > /dev/null 2>&1'))
         run(args, split(
             'docker plugin enable loki'))
         if containers:
-            run(args, split(f"docker-compose {docker_compose_args} start {containers}"))
+            run(args, split(f"{DOCKER_COMPOSE_COMMAND} {docker_compose_args} start {containers}"))
         else:
             if args.command == 'start-dev':
-                run(args, split(f'docker-compose {docker_compose_args} --file docker-compose.dev.yml up'))
+                run(args, split(f'{DOCKER_COMPOSE_COMMAND} {docker_compose_args} --file docker-compose.dev.yml up'))
             else:
-                run(args, split(f'docker-compose {docker_compose_args} up -d'))
+                run(args, split(f'{DOCKER_COMPOSE_COMMAND} {docker_compose_args} up -d'))
     elif args.command == "execute":
         run(args, ['docker', 'exec', '-i', 'wis2box-management', 'sh', '-c', containers])
     elif args.command == "login":
@@ -176,15 +181,15 @@ def make(args) -> None:
         run(args, split(f'docker exec -u -0 -it {container} /bin/bash'))
     elif args.command == "logs":
         run(args, split(
-            f'docker-compose {docker_compose_args} logs --follow {containers}'))
+            f'{DOCKER_COMPOSE_COMMAND} {docker_compose_args} logs --follow {containers}'))
     elif args.command in ["stop", "down"]:
         if containers:
-            run(args, split(f"docker-compose {docker_compose_args} {containers}"))
+            run(args, split(f"{DOCKER_COMPOSE_COMMAND} {docker_compose_args} {containers}"))
         else:
             run(args, split(
-                f'docker-compose {docker_compose_args} down --remove-orphans {containers}'))
+                f'{DOCKER_COMPOSE_COMMAND} {docker_compose_args} down --remove-orphans {containers}'))
     elif args.command == "update":
-        run(args, split(f'docker-compose {docker_compose_args} pull'))
+        run(args, split(f'{DOCKER_COMPOSE_COMMAND} {docker_compose_args} pull'))
     elif args.command == "prune":
         run(args, split('docker builder prune -f'))
         run(args, split('docker container prune -f'))
@@ -198,17 +203,17 @@ def make(args) -> None:
     elif args.command == "restart":
         if containers:
             run(args, split(
-                f'docker-compose {docker_compose_args} stop {containers}'))
+                f'{DOCKER_COMPOSE_COMMAND} {docker_compose_args} stop {containers}'))
             run(args, split(
-                f'docker-compose {docker_compose_args} start {containers}'))
+                f'{DOCKER_COMPOSE_COMMAND} {docker_compose_args} start {containers}'))
         else:
             run(args, split(
-                f'docker-compose {docker_compose_args} down --remove-orphans'))
+                f'{DOCKER_COMPOSE_COMMAND} {docker_compose_args} down --remove-orphans'))
             run(args, split(
-                f'docker-compose {docker_compose_args} up -d'))
+                f'{DOCKER_COMPOSE_COMMAND} {docker_compose_args} up -d'))
     elif args.command == "status":
         run(args, split(
-            f'docker-compose {docker_compose_args} ps {containers}'))
+            f'{DOCKER_COMPOSE_COMMAND} {docker_compose_args} ps {containers}'))
     elif args.command == "lint":
         files = walk_path(".")
         run(args, ('python3', '-m', 'flake8', *files))