Improve Docker images usability (#249)

* Address import error encountered in annotation.ipynb

When trying to run `sc.pp.highly_variable_genes()` in
docs/tutorials/annotation.ipynb the following ImportError is encountered:

ImportError: Please install skmisc package via `pip install --user scikit-misc

Looking at the pyproject.toml for the `scanpy` project it looks like
the `scikit-misc` package can be co-installed by specifying
`scanpy[skmisc]` (see:
https://github.com/scverse/scanpy/blob/a956fa781a8406fc5030093356c0372456d9184b/pyproject.toml#L142)

* Make recommend-interactive docker image even more user friendly

User friendliness improvements:
- Added short guide on how to make use of Nvidia GPU with Docker image
- Updated recommend-interactive image and example command so that
  downloaded datasets can be shared with local machine (and vice versa)

- This commit also bumps the default `SnapATAC2` version that will
  be built when following the README to v2.6.0.

* Add guide sections for running SnapATAC2 Docker image on other platforms

* Fix formatting error in Docker README.md

docker/README.md

@@ -30,26 +30,60 @@ This guide assumes you are building with `docker build` terminal commands
 
 2. Then run the build command:
     - For the default flavor of snapatac2 run:
-        `DOCKER_BUILDKIT=1 docker build --platform linux/amd64 --tag snapatac2:v2.5.1-default-py3.11 .`
+        `DOCKER_BUILDKIT=1 docker build --platform linux/amd64 --tag snapatac2:v2.6.0-default-py3.11 .`
     - For the recommend-interactive flavor of snapatac2 run:
-        `DOCKER_BUILDKIT=1 docker build --platform linux/amd64 --tag snapatac2:v2.5.1-recommend-interactive-py3.11 .`
+        `DOCKER_BUILDKIT=1 docker build --platform linux/amd64 --tag snapatac2:v2.6.0-recommend-interactive-py3.11 .`
 
 > [!NOTE]
 > You can also provide `BASE_PYTHON_IMAGE` and `SNAP_ATAC_VERSION` build args to customize the image that gets built.
-> As an example, if you want an image with python 3.1.2 and SnapATAC2 v2.5.0 you could run a build command like:
-> `DOCKER_BUILDKIT=1 docker build --platform linux/amd64 --build-arg BASE_PYTHON_IMAGE=python:3.12-slim --build-arg SNAP_ATAC_VERSION=v2.5.0 --tag snapatac2:v2.5.0-default-py3.12 .`
+> As an example, if you want an image with python 3.1.2 and a different version of SnapATAC2 (e.g. v2.5.1) you could run a build command like:
+> `DOCKER_BUILDKIT=1 docker build --platform linux/amd64 --build-arg BASE_PYTHON_IMAGE=python:3.12-slim --build-arg SNAP_ATAC_VERSION=v2.5.1 --tag snapatac2:v2.5.1-default-py3.12 .`
 
 > [!WARNING]
 > Depending on the version of BASE_PYTHON_IMAGE and SNAP_ATAC_VERSION, the
 > resulting images are *NOT* guaranteed to be well-tested or even functional!
 
-### Run Instructions for `snapatac2:v2.5.1-recommend-interactive-py3.11` Image
+## Running SnapATAC2 Docker Images
 
-Once the image has been built, you can run it with:
+### Run Instructions for `snapatac2:v2.6.0-recommend-interactive-py3.11` Image on Linux/MacOS (amd64)
 
-`docker run --interactive --tty --rm --publish 8888:8888 --volume <path_to_notebooks_on_your_local_machine>:/home/jupyter/notebooks snapatac2:v2.5.1-recommend-interactive-py3.11`
+> [!WARNING]
+> If you want the recommended image to make use of CUDA (GPU) functionality, you will need to separately install the Nvidia container toolkit.
+> [Official instructions can be found here (don't accidentally skip the Configuration section either!!)](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html).
+> When invoking the `docker run` command you will also need to provide the `--gpus` arg.
+> You can read more about how to do this in the [official Docker documentation](https://docs.docker.com/config/containers/resource_constraints/#gpu).
+
+1. Once the image has been built, you can run it with:
 
-You can then navigate in your browser to the `http://127.0.0.1:8888/lab?token=<jupyter-lab-token>` link to access Jupyter Lab
+```docker run --interactive --tty --rm --env LOCAL_USER_ID=`id -u $USER` --publish 8888:8888 --volume <path_to_local_machine_notebooks>:/notebooks --volume <path_where_you_want_data_saved>:/data snapatac2:v2.6.0-recommend-interactive-py3.11```
+
+2. You can then navigate in your browser to the `http://127.0.0.1:8888/lab?token=<jupyter-lab-token>` link to access Jupyter Lab
 
 > [!NOTE]
 > You can learn more about the `docker run` command options from the [official Docker documentation](https://docs.docker.com/engine/reference/commandline/run/#usage)
+
+### Run Instructions for `snapatac2:v2.6.0-recommend-interactive-py3.11` Image on MacOS (arm64) [UNTESTED, EXPERIMENTAL]
+
+> [!WARNING]
+> This section is COMPLETELY UNTESTED so no guarantees that it will work at all
+
+1. Similar to the above except you should add `--platform linux/amd64` to the `docker run` command like so:
+
+```docker run --platform linux/amd64 --interactive --tty --rm --env LOCAL_USER_ID=`id -u $USER` --publish 8888:8888 --volume <path_to_local_machine_notebooks>:/notebooks --volume <path_where_you_want_data_saved>:/data snapatac2:v2.6.0-recommend-interactive-py3.11```
+
+### Run Instructions for `snapatac2:v2.6.0-recommend-interactive-py3.11` Image on Windows [EXPERIMENTAL]
+
+1. Install Docker Desktop for Windows: https://docs.docker.com/desktop/install/windows-install/
+2. Start up Docker Desktop for Windows
+3. Look for the snapatac2 image that you want to run and `pull` it
+4. Go back to the main images menu and 'run' the image
+
+<img src="docker-windows-tutorial-0.png" width=25% height=25%>
+5. Before clicking `run` open up the `optional settings` and fill in the following:
+
+<img src="docker-windows-tutorial-1.png" width=25% height=25%>
+6. A new container should be spun up and you should see in the `logs` section the following:
+
+<img src="docker-windows-tutorial-2.png" width=25% height=25%>
+7. Pasting the url in the `logs` section into your browser should let you access Jupyter Lab
+

docker/default/Dockerfile

@@ -1,7 +1,7 @@
-# Example build command: `DOCKER_BUILDKIT=1 docker build --tag snapatac2:2.5.1-default-py3.11 .`
+# Example build command: `DOCKER_BUILDKIT=1 docker build --tag snapatac2:2.6.0-default-py3.11 .`
 # Use a 2-step build so our final image doesn't include bulky compile tools
 ARG BASE_PYTHON_IMAGE=python:3.11-slim
-ARG SNAP_ATAC_VERSION=v2.5.1
+ARG SNAP_ATAC_VERSION=v2.6.0
 FROM ${BASE_PYTHON_IMAGE} AS builder-image
 
 # https://docs.docker.com/engine/reference/builder/#understand-how-arg-and-from-interact

docker/recommend-interactive/Dockerfile

@@ -1,7 +1,7 @@
-# Example build command: `DOCKER_BUILDKIT=1 docker build --tag snapatac2:2.5.1-recommend-interactive-py3.11 .`
+# Example build command: `DOCKER_BUILDKIT=1 docker build --tag snapatac2:2.6.0-recommend-interactive-py3.11 .`
 # Use a 2-step build so our final image doesn't include bulky compile tools
 ARG BASE_PYTHON_IMAGE=python:3.11-slim
-ARG SNAP_ATAC_VERSION=v2.5.1
+ARG SNAP_ATAC_VERSION=v2.6.0
 FROM ${BASE_PYTHON_IMAGE} AS builder-image
 
 # https://docs.docker.com/engine/reference/builder/#understand-how-arg-and-from-interact
@@ -32,16 +32,31 @@ FROM ${BASE_PYTHON_IMAGE}
 # https://docs.docker.com/engine/reference/builder/#scope
 ARG SNAP_ATAC_VERSION
 
+# Install gosu to gracefully step-down from root in a Docker context
+# https://github.com/tianon/gosu/tree/master
+RUN set -eux; \
+	apt-get update; \
+	apt-get install -y gosu; \
+	rm -rf /var/lib/apt/lists/*; \
+    # verify that the binary works
+	gosu nobody true
+
 # Mount our first stage builder-image *temporarily* and install from the compiled .whl files
 RUN --mount=type=bind,from=builder-image,source=/python-wheel-dir,target=/python-wheel-dir \
     python3 -m pip install \
         --no-index --no-cache-dir --find-links=/python-wheel-dir \
         "snapatac2[recommend]==${SNAP_ATAC_VERSION}" \
         jupyterlab
 
-RUN useradd --create-home --shell /bin/bash jupyter
-USER jupyter
-EXPOSE 8888
+# Use entrypoint that helps us step-down from root
+COPY entrypoint.sh /usr/local/bin/entrypoint.sh
+RUN chmod +x /usr/local/bin/entrypoint.sh
 
-WORKDIR /home/jupyter/notebooks
-ENTRYPOINT ["jupyter", "lab", "--notebook-dir=/home/jupyter/notebooks", "--ip=0.0.0.0", "--port=8888", "--no-browser"]
+# Setup SnapATAC2 data download directory
+RUN mkdir -p /data
+ENV SNAP_DATA_DIR=/data
+
+WORKDIR /notebooks
+ENTRYPOINT ["/usr/local/bin/entrypoint.sh"]
+EXPOSE 8888
+CMD ["jupyter", "lab", "--notebook-dir=/notebooks", "--ip=0.0.0.0", "--port=8888", "--no-browser"]

docker/recommend-interactive/entrypoint.sh

@@ -0,0 +1,14 @@
+#!/bin/bash
+# Adapted from: https://denibertovic.com/posts/handling-permissions-with-docker-volumes/
+
+# Add local user
+# Either use the LOCAL_USER_ID if passed in at runtime or
+# fallback
+
+USER_ID=${LOCAL_USER_ID:-9001}
+
+echo "Starting with UID : $USER_ID"
+useradd --shell /bin/bash -u $USER_ID -o -c "" -m user
+export HOME=/home/user
+
+exec gosu user "$@"

snapatac2-python/pyproject.toml

@@ -57,6 +57,6 @@ Changelog = "https://kzhang.org/SnapATAC2/version/dev/changelog.html"
 
 [project.optional-dependencies]
 extra = ["scanorama>=1.7.3", "harmonypy>=0.0.9", "xgboost>=1.4", "umap-learn>=0.5.0"]
-recommend = ["scanpy>=1.9", "scvi-tools>=1.0"]
+recommend = ["scanpy[skmisc]>=1.9", "scvi-tools>=1.0"]
 all = ["snapatac2[extra]", "snapatac2[recommend]"]
 test = ["pytest", "hypothesis==6.72.4"]