Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

V1.6 cuda 12.0 ubuntu 22.04 #137

Merged
merged 6 commits into from
Jan 24, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .build/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -434,6 +434,13 @@ RUN pip install --no-cache-dir nvidia-pyindex && \

# reinstall nvcc with cuda-nvcc to install ptax
USER $NB_UID
# These need to be two separate pip install commands, otherwise it will throw an error
# attempting to resolve the nvidia-cuda-nvcc package at the same time as nvidia-pyindex
RUN pip install --no-cache-dir nvidia-pyindex && \
pip install --no-cache-dir nvidia-cuda-nvcc && \
fix-permissions "${CONDA_DIR}" && \
fix-permissions "/home/${NB_USER}"

RUN mamba install -c nvidia cuda-nvcc=12.0.140 -y && \
mamba clean --all -f -y && \
fix-permissions $CONDA_DIR && \
Expand Down
69 changes: 37 additions & 32 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,40 +85,42 @@ for creating and maintaining a robust Python, R, and Julia toolstack for Data Sc
You can optionally set a password in [http://localhost:8848/login](http://localhost:8848/login) or as described [below](#change-the-password) (former default password `gpu-jupyter`).
Additionally, data within the host's `data` directory is shared with the container.

Note that the following images of GPU-Jupyter are available on [Dockerhub](https://hub.docker.com/r/cschranz/gpu-jupyter):
The following images of GPU-Jupyter are available on [Dockerhub](https://hub.docker.com/r/cschranz/gpu-jupyter):
- `v1.6_cuda-12.0_ubuntu-22.04` (full image)
- `v1.6_cuda-12.0_ubuntu-22.04_python-only` (only with a python interpreter and without Julia and R)
- `v1.6_cuda-12.0_ubuntu-22.04_slim` (only with a python interpreter and without additional packages)
- `v1.6_cuda-11.8_ubuntu-22.04` (full image)
- `v1.6_cuda-11.8_ubuntu-22.04_python-only` (only with a python interpreter and without Julia and R)
- `v1.6_cuda-11.8_ubuntu-22.04_slim` (only with a python interpreter and without additional packages)
- `v1.6_cuda-11.6_ubuntu-20.04` (full image)
- `v1.6_cuda-11.6_ubuntu-20.04_python-only` (only with a python interpreter and without Julia and R)
- `v1.6_cuda-11.6_ubuntu-20.04_slim` (only with a python interpreter and without additional packages)
- `v1.5_cuda-12.0_ubuntu-22.04` (full image)
- `v1.5_cuda-12.0_ubuntu-22.04_python-only` (only with a python interpreter and without Julia and R)
- `v1.5_cuda-12.0_ubuntu-22.04_slim` (only with a python interpreter and without additional packages)
- `v1.5_cuda-11.8_ubuntu-22.04` (full image)
- `v1.5_cuda-11.8_ubuntu-22.04_python-only` (only with a python interpreter and without Julia and R)
- `v1.5_cuda-11.8_ubuntu-22.04_slim` (only with a python interpreter and without additional packages)
- `v1.5_cuda-11.6_ubuntu-20.04` (full image)
- `v1.5_cuda-11.6_ubuntu-20.04_python-only` (only with a python interpreter and without Julia and R)
- `v1.5_cuda-11.6_ubuntu-20.04_slim` (only with a python interpreter and without additional packages)
- `v1.4_cuda-11.6_ubuntu-20.04` (full image)
- `v1.4_cuda-11.6_ubuntu-20.04_python-only` (only with a python interpreter and without Julia and R)
- `v1.4_cuda-11.6_ubuntu-20.04_slim` (only with a python interpreter and without additional packages)
- `v1.4_cuda-11.2_ubuntu-20.04` (full image)
- `v1.4_cuda-11.2_ubuntu-20.04_python-only` (only with a python interpreter and without Julia and R)
- `v1.4_cuda-11.2_ubuntu-20.04_slim` (only with a python interpreter and without additional packages)
- `v1.4_cuda-11.0_ubuntu-20.04` (full image)
- `v1.4_cuda-11.0_ubuntu-20.04_python-only` (only with a python interpreter and without Julia and R)
- `v1.4_cuda-11.0_ubuntu-20.04_slim` (only with a python interpreter and without additional packages)
- `v1.4_cuda-11.0_ubuntu-18.04` (full image)
- `v1.4_cuda-11.0_ubuntu-18.04_python-only` (only with a python interpreter and without Julia and R)
- `v1.4_cuda-11.0_ubuntu-18.04_slim` (only with a python interpreter and without additional packages)
- `v1.4_cuda-10.1_ubuntu-18.04` (full image)
- `v1.4_cuda-10.1_ubuntu-18.04_python-only` (only with a python interpreter and without Julia and R)
- `v1.4_cuda-10.1_ubuntu-18.04_slim` (only with a python interpreter and without additional packages)

<details>
<summary><font color=blue> Older images</font></summary>

- `v1.5_cuda-12.0_ubuntu-22.04` (full image)
- `v1.5_cuda-12.0_ubuntu-22.04_python-only` (only with a python interpreter and without Julia and R)
- `v1.5_cuda-12.0_ubuntu-22.04_slim` (only with a python interpreter and without additional packages)
- `v1.5_cuda-11.8_ubuntu-22.04` (full image)
- `v1.5_cuda-11.8_ubuntu-22.04_python-only` (only with a python interpreter and without Julia and R)
- `v1.5_cuda-11.8_ubuntu-22.04_slim` (only with a python interpreter and without additional packages)
- `v1.5_cuda-11.6_ubuntu-20.04` (full image)
- `v1.5_cuda-11.6_ubuntu-20.04_python-only` (only with a python interpreter and without Julia and R)
- `v1.5_cuda-11.6_ubuntu-20.04_slim` (only with a python interpreter and without additional packages)
- `v1.4_cuda-11.6_ubuntu-20.04` (full image)
- `v1.4_cuda-11.6_ubuntu-20.04_python-only` (only with a python interpreter and without Julia and R)
- `v1.4_cuda-11.6_ubuntu-20.04_slim` (only with a python interpreter and without additional packages)
- `v1.4_cuda-11.2_ubuntu-20.04` (full image)
- `v1.4_cuda-11.2_ubuntu-20.04_python-only` (only with a python interpreter and without Julia and R)
- `v1.4_cuda-11.2_ubuntu-20.04_slim` (only with a python interpreter and without additional packages)
- `v1.4_cuda-11.0_ubuntu-20.04` (full image)
- `v1.4_cuda-11.0_ubuntu-20.04_python-only` (only with a python interpreter and without Julia and R)
- `v1.4_cuda-11.0_ubuntu-20.04_slim` (only with a python interpreter and without additional packages)
- `v1.4_cuda-11.0_ubuntu-18.04` (full image)
- `v1.4_cuda-11.0_ubuntu-18.04_python-only` (only with a python interpreter and without Julia and R)
- `v1.4_cuda-11.0_ubuntu-18.04_slim` (only with a python interpreter and without additional packages)
- `v1.4_cuda-10.1_ubuntu-18.04` (full image)
- `v1.4_cuda-10.1_ubuntu-18.04_python-only` (only with a python interpreter and without Julia and R)
- `v1.4_cuda-10.1_ubuntu-18.04_slim` (only with a python interpreter and without additional packages)
</details>

The version, e.g. `v1.6`, declares the version of the generator setup.
The Cuda version, e.g. `cuda-12.0`, must match the CUDA driver version and be supported by the GPU libraries.
Expand All @@ -144,7 +146,7 @@ The following commands will start *GPU-Jupyter* on [localhost:8848](http://local
git clone https://github.com/iot-salzburg/gpu-jupyter.git
cd gpu-jupyter
git branch # Check for extisting branches
git checkout v1.6_cuda-12.0_ubuntu-22.04 # select or create a new version
git checkout v1.6_cuda-11.8_ubuntu-22.04 # select or create a new version
# generate the Dockerfile with python and without Julia and R (see options: --help)
./generate-Dockerfile.sh --python-only
docker build -t gpu-jupyter .build/ # will take a while
Expand Down Expand Up @@ -232,9 +234,9 @@ Some useful packages are suggested in the [Extension docs](https://jupyterlab.re
If you notice that an important package is missing in the default stack, please let us know so we can update it.


### Authentification
### Authorization

### Set a Static Token
#### Set a Static Token

Jupyter by default regenerates a new token on each new start.
GPU-Jupyter provides the environment variable `JUPYTER_TOKEN` to set a customized static token.
Expand All @@ -243,6 +245,7 @@ It is suggested to use a long token such as a UUID:

```bash
export JUPYTER_TOKEN=$(uuidgen)
echo $JUPYTER_TOKEN
```

- For Docker add the parameter `-e JUPYTER_TOKEN=${JUPYTER_TOKEN}`, e.g.:
Expand All @@ -254,7 +257,9 @@ export JUPYTER_TOKEN=$(uuidgen)
- In `docker-compose.yml`, the environment variable can be set under
`ENVIRONMENT`:

`NB_GID: ${JUPYTER_GID:-1000}`
`JUPYTER_TOKEN: ${JUPYTER_TOKEN}`

Please note that environment variables might not be accessable for all docker-compose versions. Consider to setting `JUPYTER_TOKEN` in a separate `.env`-file and using `env_file` in docker-compose or a hard-coded token.

The static token can be requested using `docker exec`:
```bash
Expand Down
7 changes: 7 additions & 0 deletions custom/gpulibs.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,13 @@ RUN pip install --no-cache-dir nvidia-pyindex && \

# reinstall nvcc with cuda-nvcc to install ptax
USER $NB_UID
# These need to be two separate pip install commands, otherwise it will throw an error
# attempting to resolve the nvidia-cuda-nvcc package at the same time as nvidia-pyindex
RUN pip install --no-cache-dir nvidia-pyindex && \
pip install --no-cache-dir nvidia-cuda-nvcc && \
fix-permissions "${CONDA_DIR}" && \
fix-permissions "/home/${NB_USER}"

RUN mamba install -c nvidia cuda-nvcc=12.0.140 -y && \
mamba clean --all -f -y && \
fix-permissions $CONDA_DIR && \
Expand Down
2 changes: 1 addition & 1 deletion generate-Dockerfile.sh
Original file line number Diff line number Diff line change
Expand Up @@ -175,7 +175,7 @@ fi
cp -r extra/Getting_Started data
chmod -R 755 data/

# set static token
# set static token (optional if set)
# copy jupyter server config token addendum to .build
cp custom/jupyter_server_config_token_addendum.py .build/
# append token config into the jupyter server config
Expand Down
Loading