From 069344b15f52ab934d489075ca17c1c5c3d61a02 Mon Sep 17 00:00:00 2001 From: w-lampaert <85161373+w-lampaert@users.noreply.github.com> Date: Wed, 9 Aug 2023 13:40:44 +0200 Subject: [PATCH 1/7] Update openondemand.rst Add newest features, minor changes --- source/leuven/services/openondemand.rst | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/source/leuven/services/openondemand.rst b/source/leuven/services/openondemand.rst index b933fb9b..521f8f49 100644 --- a/source/leuven/services/openondemand.rst +++ b/source/leuven/services/openondemand.rst @@ -14,7 +14,7 @@ Open OnDemand is available for the Tier-2 Genius and wICE clusters. You can use this interface by navigating to the `KU Leuven Open OnDemand page`_. You can log in using your KU Leuven or VSC credentials. -Access is only granted to VSC3 users for the moment. +Access is only granted to VSC3 users at the moment. Once logged in, you'll notice you are connected with your VSC account. Use @@ -27,8 +27,8 @@ The KU Leuven Open OnDemand page provides a range of functions: - Opening a shell on one of the login nodes - Using interactive apps -All of these functionalities can be used by accessing them through the tabs at the top of the page. In the following, we will describe the different parts in some more -detail. +All of these functionalities can be used by accessing them through the tabs at the top of the page. In the following, we will describe the different parts in +some more detail. Files ----- @@ -112,7 +112,7 @@ The functioning of creating jobs is a bit similar to how you create new template Whatever method you choose, you will create a new folder for each job, this time located at ``$VSC_DATA/ondemand/data/projects/default/``. The job folders will be numbered in the order you have created them. -**Do not change this folder name as long as you plan on using it from the job menus, as this will break the linking here.** +**Do not change this folder name as long as you plan on using it from the job menus, as this will break the linking.** When removing a job, the folder will be deleted as well. To create a job, press the 'New Job' button and choose the option that best suits @@ -163,11 +163,13 @@ resources might result in a long queue time. Between all the apps, most of these explained in the specific paragraph about the app. A general overview of the others can be found here: - Account: the credit account you want to deduct the credits from. The accounts associated with your VSC number will be displayed in a dropdown. -- Partition: you can choose any of the existing partitions. We recommend using the ``interactive`` partition for most interactive work. +- Partition: you can choose any of the existing partitions. We recommend using the ``interactive`` partition for most interactive work on wICE. Be aware that this partition is not available on Genius. There it is recommended to just request the regular ``batch`` partition. - Numbers of hours: your walltime (min 1h). - Number of cores: the amount of cores per node. This defaults to 1. - Required memory per core in megabytes. This defaults to 3400 MB. - Number of GPUs. If you request a GPU of the `gpu` partition you will get a full A100 GPU. For the `interactive` partition, every GPU is a virtual GPU slice of the available A100 GPUs. One GPU is the same as 1/7th of a A100 GPU. The default is 0. You can specify the type of GPU as well: [Type]: (e.g. A100:2). You can also just request a number of GPUs as . Then you will be appointed the first available GPU types. In practice, both methods are the same for now. This might change if we would decide to add extra GPU types. +- Reservation: if you are part of a reservation, you can also use these nodes with Open Ondemand by specifying your reservation name here. +- Pre-run scriptlet: this allows you to add bash commands to your job before launching the app. This can be used for example for loading extra modules that you need within the app. **Be aware that this feature is still somewhat experimental, and its functionality also depends on the app you are running (mainly RStudio Server has some issues here). If you would like to use this feature, but you run into problems, please contact our helpdesk.** Once you've selected all your resources, just press 'Launch' and your job will be queued. In the next part, you find an overview of the currently supported apps. @@ -176,7 +178,7 @@ Interactive shell ================= This app will launch a shell on (one of the) requested nodes, allowing you to use these compute resources from within a Linux terminal. This is different -than the shell you get in the "Clusters" menu, as this is a login shell. +than the shell you get in the "Clusters" menu, which directs you towards one of the login nodes. Jupyter Lab ================ From 13dd25b641880ad1a0fc37fae50bedb075275ce3 Mon Sep 17 00:00:00 2001 From: w-lampaert <85161373+w-lampaert@users.noreply.github.com> Date: Wed, 9 Aug 2023 14:36:53 +0200 Subject: [PATCH 2/7] Add more detailed resource description --- source/leuven/services/openondemand.rst | 30 ++++++++++++++++++++++--- 1 file changed, 27 insertions(+), 3 deletions(-) diff --git a/source/leuven/services/openondemand.rst b/source/leuven/services/openondemand.rst index 521f8f49..45b00f76 100644 --- a/source/leuven/services/openondemand.rst +++ b/source/leuven/services/openondemand.rst @@ -160,19 +160,43 @@ app will be running. To launch any of the interactive apps, you need to fill in the resources form. Be aware that you will end up in a regular queue, so requesting a large amount of resources might result in a long queue time. Between all the apps, most of these options are the same. Some apps require specific information. These will be -explained in the specific paragraph about the app. A general overview of the others can be found here: +explained in the specific paragraph about the app. A general overview of the others can be found below. A more detailed guide on how to choose your resources +is available in the next chapter. - Account: the credit account you want to deduct the credits from. The accounts associated with your VSC number will be displayed in a dropdown. -- Partition: you can choose any of the existing partitions. We recommend using the ``interactive`` partition for most interactive work on wICE. Be aware that this partition is not available on Genius. There it is recommended to just request the regular ``batch`` partition. +- Partition: you can choose any of the existing partitions on both clusters. We recommend using the ``interactive`` partition for most interactive work on wICE. Be aware that this partition is not available on Genius. There it is recommended to just request the regular `batch` partition. - Numbers of hours: your walltime (min 1h). - Number of cores: the amount of cores per node. This defaults to 1. - Required memory per core in megabytes. This defaults to 3400 MB. -- Number of GPUs. If you request a GPU of the `gpu` partition you will get a full A100 GPU. For the `interactive` partition, every GPU is a virtual GPU slice of the available A100 GPUs. One GPU is the same as 1/7th of a A100 GPU. The default is 0. You can specify the type of GPU as well: [Type]: (e.g. A100:2). You can also just request a number of GPUs as . Then you will be appointed the first available GPU types. In practice, both methods are the same for now. This might change if we would decide to add extra GPU types. +- Number of GPUs. If you request a GPU of the `gpu` partition you will get a full A100 GPU. For the `interactive` partition, every GPU is a virtual GPU slice of the available A100 GPUs. One GPU is the same as 1/7th of a A100 GPU. The default is 0. You can specify the type of GPU as well: [Type]: (e.g. A100:2). You can also just request a number of GPUs as . Then you will be appointed the first available GPU types. In practice, both methods are the same for now. This might change if we would decide to add extra GPU types. **The interactive partition only allows you to request max 1 GPU (slice) though.** - Reservation: if you are part of a reservation, you can also use these nodes with Open Ondemand by specifying your reservation name here. - Pre-run scriptlet: this allows you to add bash commands to your job before launching the app. This can be used for example for loading extra modules that you need within the app. **Be aware that this feature is still somewhat experimental, and its functionality also depends on the app you are running (mainly RStudio Server has some issues here). If you would like to use this feature, but you run into problems, please contact our helpdesk.** Once you've selected all your resources, just press 'Launch' and your job will be queued. In the next part, you find an overview of the currently supported apps. +Choosing your resources +======================= + +Choosing the correct resources for your interactive session is mostly the same as selecting them when launching regular batch jobs. For this reason we strongly +recommend you to have a look at how to specify your resources both on `Genius `_ and `wICE `_. + +As mentioned above, in most cases we recommend using the `interactive` partition on wICE for your interactive apps. This partition is meant for lighter work, like +visualisations, testing and pre- and postprocessing. Using this partition is also free, mainly to encourage you to request these resources for such work, instead +of using any of the other partitions. There are however some limitations on the amount of resources you can request here: + +- Max 1 node +- Max 8 cores +- Max 1 virtual GPU slice +- Max 16h of walltime + +This is put in place to ensure that these resources are kept for their original purpose, namely the interactive work. + +If for some reason some of these limitations are too strict for you, or you need resources that are not available on the interactive nodes (e.g. a full GPU, big +memory nodes), you can always request nodes from another partition. Remember however that these interactive apps are not meant for running full jobs. If you +indeed need multiple nodes or full GPUs to test your code/program, go ahead and request the resources for your interactive app. In the case that you have passed +the testing phase and you want to start conducting experiments, we recommend that you make the switch to batch jobs instead, as they will not require +your presence to start your code. + .. _interactive_shell: Interactive shell ================= From 37d646cc802b71ed07361b304253dcc9e519cfd2 Mon Sep 17 00:00:00 2001 From: w-lampaert <85161373+w-lampaert@users.noreply.github.com> Date: Wed, 9 Aug 2023 14:52:55 +0200 Subject: [PATCH 3/7] Minor visual changes --- source/leuven/services/openondemand.rst | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/source/leuven/services/openondemand.rst b/source/leuven/services/openondemand.rst index 45b00f76..6fc90965 100644 --- a/source/leuven/services/openondemand.rst +++ b/source/leuven/services/openondemand.rst @@ -78,9 +78,9 @@ To enter the Templates menu, you can click on 'Templates' at the top once you ar Job'-'From Template'. Once in this menu, you should see a table with 3 System Templates. The resources that are requested in these scripts are the default settings. The templates: -- CPU job template: a template for jobs on the thin nodes (the default `batch` partition). This is also the default template (which you will get when clicking 'From Default Template' under the 'New Job' button in the 'Jobs' menu). -- GPU job template: a template for jobs with GPU resources (`gpu` partition) -- Big memory CPU jobs: a template for jobs with large memory requirements (`bigmem` partition) +- CPU job template: a template for jobs on the thin nodes (the default ``batch`` partition). This is also the default template (which you will get when clicking 'From Default Template' under the 'New Job' button in the 'Jobs' menu). +- GPU job template: a template for jobs with GPU resources (``gpu`` partition) +- Big memory CPU jobs: a template for jobs with large memory requirements (``bigmem`` partition) You can create your own templates from scratch or by copying one of the existing templates. In both cases you will be redirected to a page where you can provide a @@ -164,11 +164,11 @@ explained in the specific paragraph about the app. A general overview of the oth is available in the next chapter. - Account: the credit account you want to deduct the credits from. The accounts associated with your VSC number will be displayed in a dropdown. -- Partition: you can choose any of the existing partitions on both clusters. We recommend using the ``interactive`` partition for most interactive work on wICE. Be aware that this partition is not available on Genius. There it is recommended to just request the regular `batch` partition. +- Partition: you can choose any of the existing partitions on both clusters. We recommend using the ``interactive`` partition for most interactive work on wICE. Be aware that this partition is not available on Genius. There it is recommended to just request the regular ``batch` partition. - Numbers of hours: your walltime (min 1h). - Number of cores: the amount of cores per node. This defaults to 1. - Required memory per core in megabytes. This defaults to 3400 MB. -- Number of GPUs. If you request a GPU of the `gpu` partition you will get a full A100 GPU. For the `interactive` partition, every GPU is a virtual GPU slice of the available A100 GPUs. One GPU is the same as 1/7th of a A100 GPU. The default is 0. You can specify the type of GPU as well: [Type]: (e.g. A100:2). You can also just request a number of GPUs as . Then you will be appointed the first available GPU types. In practice, both methods are the same for now. This might change if we would decide to add extra GPU types. **The interactive partition only allows you to request max 1 GPU (slice) though.** +- Number of GPUs. If you request a GPU of the ``gpu`` partition you will get a full A100 GPU. For the ``interactive`` partition, every GPU is a virtual GPU slice of the available A100 GPUs. One GPU is the same as 1/7th of a A100 GPU. The default is 0. You can specify the type of GPU as well: [Type]: (e.g. A100:2). You can also just request a number of GPUs as . Then you will be appointed the first available GPU types. In practice, both methods are the same for now. This might change if we would decide to add extra GPU types. **The interactive partition only allows you to request max 1 GPU (slice) though.** - Reservation: if you are part of a reservation, you can also use these nodes with Open Ondemand by specifying your reservation name here. - Pre-run scriptlet: this allows you to add bash commands to your job before launching the app. This can be used for example for loading extra modules that you need within the app. **Be aware that this feature is still somewhat experimental, and its functionality also depends on the app you are running (mainly RStudio Server has some issues here). If you would like to use this feature, but you run into problems, please contact our helpdesk.** @@ -180,7 +180,7 @@ Choosing your resources Choosing the correct resources for your interactive session is mostly the same as selecting them when launching regular batch jobs. For this reason we strongly recommend you to have a look at how to specify your resources both on `Genius `_ and `wICE `_. -As mentioned above, in most cases we recommend using the `interactive` partition on wICE for your interactive apps. This partition is meant for lighter work, like +As mentioned above, in most cases we recommend using the ``interactive`` partition on wICE for your interactive apps. This partition is meant for lighter work, like visualisations, testing and pre- and postprocessing. Using this partition is also free, mainly to encourage you to request these resources for such work, instead of using any of the other partitions. There are however some limitations on the amount of resources you can request here: From db29dcec4549acead58f99064fc0117f8392c31d Mon Sep 17 00:00:00 2001 From: w-lampaert <85161373+w-lampaert@users.noreply.github.com> Date: Wed, 9 Aug 2023 14:54:06 +0200 Subject: [PATCH 4/7] Typo --- source/leuven/services/openondemand.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/leuven/services/openondemand.rst b/source/leuven/services/openondemand.rst index 6fc90965..f42511d0 100644 --- a/source/leuven/services/openondemand.rst +++ b/source/leuven/services/openondemand.rst @@ -164,7 +164,7 @@ explained in the specific paragraph about the app. A general overview of the oth is available in the next chapter. - Account: the credit account you want to deduct the credits from. The accounts associated with your VSC number will be displayed in a dropdown. -- Partition: you can choose any of the existing partitions on both clusters. We recommend using the ``interactive`` partition for most interactive work on wICE. Be aware that this partition is not available on Genius. There it is recommended to just request the regular ``batch` partition. +- Partition: you can choose any of the existing partitions on both clusters. We recommend using the ``interactive`` partition for most interactive work on wICE. Be aware that this partition is not available on Genius. There it is recommended to just request the regular ``batch`` partition. - Numbers of hours: your walltime (min 1h). - Number of cores: the amount of cores per node. This defaults to 1. - Required memory per core in megabytes. This defaults to 3400 MB. From eb2f27f1e5ddfbb409a8ef88fbfbc3b2d7987a88 Mon Sep 17 00:00:00 2001 From: w-lampaert <85161373+w-lampaert@users.noreply.github.com> Date: Wed, 9 Aug 2023 14:56:09 +0200 Subject: [PATCH 5/7] Update openondemand.rst --- source/leuven/services/openondemand.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/leuven/services/openondemand.rst b/source/leuven/services/openondemand.rst index f42511d0..444f5b42 100644 --- a/source/leuven/services/openondemand.rst +++ b/source/leuven/services/openondemand.rst @@ -172,7 +172,7 @@ is available in the next chapter. - Reservation: if you are part of a reservation, you can also use these nodes with Open Ondemand by specifying your reservation name here. - Pre-run scriptlet: this allows you to add bash commands to your job before launching the app. This can be used for example for loading extra modules that you need within the app. **Be aware that this feature is still somewhat experimental, and its functionality also depends on the app you are running (mainly RStudio Server has some issues here). If you would like to use this feature, but you run into problems, please contact our helpdesk.** -Once you've selected all your resources, just press 'Launch' and your job will be queued. In the next part, you find an overview of the currently supported apps. +Once you've selected all your resources, just press 'Launch' and your job will be queued. Choosing your resources ======================= From 9f5659f7826aea7872ab6b1475c9bf448f4f4ed1 Mon Sep 17 00:00:00 2001 From: w-lampaert <85161373+w-lampaert@users.noreply.github.com> Date: Thu, 10 Aug 2023 17:03:01 +0200 Subject: [PATCH 6/7] Update according to review --- source/leuven/services/openondemand.rst | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/source/leuven/services/openondemand.rst b/source/leuven/services/openondemand.rst index 444f5b42..226729e6 100644 --- a/source/leuven/services/openondemand.rst +++ b/source/leuven/services/openondemand.rst @@ -164,16 +164,17 @@ explained in the specific paragraph about the app. A general overview of the oth is available in the next chapter. - Account: the credit account you want to deduct the credits from. The accounts associated with your VSC number will be displayed in a dropdown. -- Partition: you can choose any of the existing partitions on both clusters. We recommend using the ``interactive`` partition for most interactive work on wICE. Be aware that this partition is not available on Genius. There it is recommended to just request the regular ``batch`` partition. +- Partition: you can choose any of the existing partitions on both clusters. We recommend using the ``interactive`` partition for most interactive work on wICE. Be aware that this partition is not available on Genius. There it is recommended to just request the regular ``batch`` partition (see the :ref:`Choosing your resources` section for more detail on hoz to choose your partition). - Numbers of hours: your walltime (min 1h). - Number of cores: the amount of cores per node. This defaults to 1. - Required memory per core in megabytes. This defaults to 3400 MB. -- Number of GPUs. If you request a GPU of the ``gpu`` partition you will get a full A100 GPU. For the ``interactive`` partition, every GPU is a virtual GPU slice of the available A100 GPUs. One GPU is the same as 1/7th of a A100 GPU. The default is 0. You can specify the type of GPU as well: [Type]: (e.g. A100:2). You can also just request a number of GPUs as . Then you will be appointed the first available GPU types. In practice, both methods are the same for now. This might change if we would decide to add extra GPU types. **The interactive partition only allows you to request max 1 GPU (slice) though.** +- Number of GPUs. If you request a GPU of the ``gpu`` partition you will get a full A100 GPU. For the ``interactive`` partition on wICE, every GPU is a virtual GPU slice of the available A100 GPUs. One GPU slice is the same as 1/7th of a A100 GPU. The default is 0. You can specify the type of GPU as well: [Type]: (e.g. A100:2). You can also just request a number of GPUs as . Then you will be appointed the first available GPU types. In practice, both methods are the same for now. This might change if we would decide to add extra GPU types. **The interactive partition only allows you to request max 1 GPU (slice) though.** - Reservation: if you are part of a reservation, you can also use these nodes with Open Ondemand by specifying your reservation name here. - Pre-run scriptlet: this allows you to add bash commands to your job before launching the app. This can be used for example for loading extra modules that you need within the app. **Be aware that this feature is still somewhat experimental, and its functionality also depends on the app you are running (mainly RStudio Server has some issues here). If you would like to use this feature, but you run into problems, please contact our helpdesk.** Once you've selected all your resources, just press 'Launch' and your job will be queued. +.. _choosing_your_resources: Choosing your resources ======================= From db0faf5a58a91c4d883235de7bb9b98247930fe3 Mon Sep 17 00:00:00 2001 From: w-lampaert <85161373+w-lampaert@users.noreply.github.com> Date: Thu, 10 Aug 2023 17:08:09 +0200 Subject: [PATCH 7/7] typo --- source/leuven/services/openondemand.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/leuven/services/openondemand.rst b/source/leuven/services/openondemand.rst index 226729e6..1ba034cd 100644 --- a/source/leuven/services/openondemand.rst +++ b/source/leuven/services/openondemand.rst @@ -164,7 +164,7 @@ explained in the specific paragraph about the app. A general overview of the oth is available in the next chapter. - Account: the credit account you want to deduct the credits from. The accounts associated with your VSC number will be displayed in a dropdown. -- Partition: you can choose any of the existing partitions on both clusters. We recommend using the ``interactive`` partition for most interactive work on wICE. Be aware that this partition is not available on Genius. There it is recommended to just request the regular ``batch`` partition (see the :ref:`Choosing your resources` section for more detail on hoz to choose your partition). +- Partition: you can choose any of the existing partitions on both clusters. We recommend using the ``interactive`` partition for most interactive work on wICE. Be aware that this partition is not available on Genius. There it is recommended to just request the regular ``batch`` partition (see the :ref:`Choosing your resources` section for more detail on how to choose your partition). - Numbers of hours: your walltime (min 1h). - Number of cores: the amount of cores per node. This defaults to 1. - Required memory per core in megabytes. This defaults to 3400 MB.