diff --git a/docs/installation_instructions.md b/docs/installation_instructions.md index 02fc7ff8..10a07ed0 100644 --- a/docs/installation_instructions.md +++ b/docs/installation_instructions.md @@ -23,7 +23,7 @@ installed there. See [Install PyStemmusScope](#install-pystemmusscope). ## On your own machine -Choose how do you want to run the model, see [Run the model](notebooks/run_model_in_notebook.ipynb). +Choose how do you want to run the model, see [Run the model](notebooks/run_model_on_different_infra.ipynb). ## Install PyStemmusScope @@ -59,7 +59,7 @@ jupyter lab JupyterLab will open automatically in your browser. Now, you can run the notebook -[run_model_in_notebook.ipynb](https://github.com/EcoExtreML/STEMMUS_SCOPE_Processing/blob/main/docs/notebooks/run_model_in_notebook.ipynb). +[run_model_on_different_infra.ipynb](https://github.com/EcoExtreML/STEMMUS_SCOPE_Processing/blob/main/docs/notebooks/run_model_on_different_infra.ipynb). ## Install MATLAB Runtime diff --git a/docs/notebooks/run_model_in_notebook.ipynb b/docs/notebooks/run_model_on_different_infra.ipynb similarity index 92% rename from docs/notebooks/run_model_in_notebook.ipynb rename to docs/notebooks/run_model_on_different_infra.ipynb index aaca5d7d..cf7c5402 100644 --- a/docs/notebooks/run_model_in_notebook.ipynb +++ b/docs/notebooks/run_model_on_different_infra.ipynb @@ -5,9 +5,9 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# Running the STEMMUS_SCOPE model\n", - "## Which options are available?\n", - "This notebook shows steps to run the STEMMUS_SCOPE model, including preprocessing and postprocessing. STEMMUS_SCOPE files are located in a **private** repository https://github.com/EcoExtreML/STEMMUS_SCOPE. To clone the repository locally and access the files, make sure you have right access to the repository. Then specify the path to a config file e.g.`config_file_template.txt`, executable file `STEMMUS_SCOPE` or source code `STEMMUS_SCOPE/src` in the cells below. There are already config files for users on Snellius and CRIB, see `config_file_snellius.txt`, `config_file_crib.txt`. Choose how do you want to run the model:\n", + "# Running the STEMMUS_SCOPE model on a system\n", + "\n", + "This notebook shows steps to run the STEMMUS_SCOPE model, including preprocessing and postprocessing. STEMMUS_SCOPE files are located in a repository https://github.com/EcoExtreML/STEMMUS_SCOPE. Then specify the path to a config file e.g.`config_file_template.txt`, executable file `STEMMUS_SCOPE` or source code `STEMMUS_SCOPE/src` in the cells below. There are already config files for users on Snellius and CRIB, see `config_file_snellius.txt`, `config_file_crib.txt`. Depending on your system (OS, Matlab availability, etc.), there are a few ways you can run the model. Choose how do you want to run the model:\n", "\n", "\n", "[**Executable file**](#1-using-executable-file)\n", @@ -229,7 +229,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Perform in a terminal in the Remote Desktop (of Crib Platform), and check the output (should be similar to :1.0 ). Then execute the following line accordingly." + "  If you are on CRIB, to use Jupyter interface and MATLAB license manager at the same time, first perform in a terminal in the Remote Desktop of Crib Platform, and check the output which should be similar to :1.0 . Then execute the following line accordingly." ] }, { @@ -238,6 +238,7 @@ "metadata": {}, "outputs": [], "source": [ + "# For CRIB users\n", "%env DISPLAY=:1.0" ] }, @@ -398,7 +399,7 @@ ], "metadata": { "kernelspec": { - "display_name": "pyss_octave", + "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, @@ -412,7 +413,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.9.13 | packaged by conda-forge | (main, May 27 2022, 16:50:36) [MSC v.1929 64 bit (AMD64)]" + "version": "3.10.13" }, "vscode": { "interpreter": { diff --git a/docs/notebooks/run_model_with_different_dataset.ipynb b/docs/notebooks/run_model_with_different_dataset.ipynb new file mode 100644 index 00000000..b5202e5f --- /dev/null +++ b/docs/notebooks/run_model_with_different_dataset.ipynb @@ -0,0 +1,364 @@ +{ + "cells": [ + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Running the STEMMUS_SCOPE model with different dataset\n", + "\n", + "This notebook shows how to run the STEMMUS_SCOPE model uisng the `Executable` file, including preprocessing and postprocessing. For postprocesisng, the file `required_netcdf_variables.csv` can be found at https://github.com/EcoExtreML/STEMMUS_SCOPE/blob/main/utils/csv_to_nc/required_netcdf_variables.csv.\n", + "\n", + "See the [Running the STEMMUS_SCOPE model on a system](./run_model_on_different_infra.ipynb) for different ways e.g. Executable file, Matlab, Octave.\n", + "The `STEMMUS_SCOPE` executable file is located in a repository https://github.com/EcoExtreML/STEMMUS_SCOPE/blob/main/run_model_on_snellius/exe/STEMMUS_SCOPE.\n", + "\n", + "## Which datasets are available?\n", + "The config file shows how to set the paths to different data needed by the model. Check the data source on Snelius and CRIB and modify the config file accordingly:\n", + "\n", + "#### On CRIB\n", + "- Site data:\n", + " - path to data: `/data/shared/EcoExtreML/STEMMUS_SCOPEv1.0.0/input`,\n", + " - config file template: https://github.com/EcoExtreML/STEMMUS_SCOPE/blob/main/config_file_crib.txt\n", + "- Global data:\n", + " - path to data `/data/shared/EcoExtreML/STEMMUS_SCOPEv1.0.0/global_data`,\n", + " - config file template: https://github.com/EcoExtreML/STEMMUS_SCOPE/blob/main/run_model_on_snellius/config_file_crib_global.txt\n", + "\n", + "#### On Snellius\n", + "- Site data:\n", + " - path to data: `/projects/0/ttse0619/`,\n", + " - config file template: https://github.com/EcoExtreML/STEMMUS_SCOPE/blob/main/run_model_on_snellius/config_file_snellius.txt\n", + "- Global data:\n", + " - path to data: `/projects/0/ttse0619/global_data`,\n", + " - config file template: https://github.com/EcoExtreML/STEMMUS_SCOPE/blob/main/run_model_on_snellius/config_file_snellius_global.txt\n", + "\n", + "Choose how do you want to run the model:\n", + "\n", + "[**Using site data**](#using-site-data)\n", + "\n", + "[**Using global data**](#using-global-data)\n", + "\n" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Using site data" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [], + "source": [ + "import os\n", + "from PyStemmusScope import StemmusScope\n", + "from PyStemmusScope import save" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [], + "source": [ + "# user must provide the correct path\n", + "path_to_config_file = \"./my_config_template.txt\"\n", + "path_to_exe_file = \"./STEMMUS_SCOPE\"" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "If you run the model on your own machine, make sure `LD_LIBRARY_PATH` is set correctly. To do this, uncomment the cell below and run it:" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "/usr/local/MATLAB/MATLAB_Runtime/R2023a/runtime/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/R2023a/bin/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/R2023a/sys/os/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/R2023a/extern/bin/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/R2023a/sys/opengl/lib/glnxa64\n" + ] + } + ], + "source": [ + "# # Set LD_LIBRARY_PATH\n", + "# matlab_path = !whereis MATLAB\n", + "# matlab_path = matlab_path.s.split(\": \")[1]\n", + "# os.environ['LD_LIBRARY_PATH'] = (\n", + "# f\"{matlab_path}/MATLAB_Runtime/R2023a/runtime/glnxa64:\"\n", + "# f\"{matlab_path}/MATLAB_Runtime/R2023a/bin/glnxa64:\"\n", + "# f\"{matlab_path}/MATLAB_Runtime/R2023a/sys/os/glnxa64:\"\n", + "# f\"{matlab_path}/MATLAB_Runtime/R2023a/extern/bin/glnxa64:\"\n", + "# f\"{matlab_path}/MATLAB_Runtime/R2023a/sys/opengl/lib/glnxa64\")\n", + "# print(os.environ['LD_LIBRARY_PATH'])" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [], + "source": [ + "# create an an instance of the model\n", + "model = StemmusScope(config_file=path_to_config_file, model_src_path=path_to_exe_file)" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "New config file /home/sarah/temp/ecoextreml/test/input/ZA-Kru_2024-01-24-1125/ZA-Kru_2024-01-24-1125_config.txt\n", + "Model input dir /home/sarah/temp/ecoextreml/test/input/ZA-Kru_2024-01-24-1125/\n", + "Model output dir /home/sarah/temp/ecoextreml/test/output/ZA-Kru_2024-01-24-1125/\n" + ] + } + ], + "source": [ + "# setup the model\n", + "# here you can change the location and start & end time\n", + "config_path = model.setup(\n", + " Location=\"ZA-Kru\",\n", + " StartTime=\"2001-01-01T00:00\",\n", + " EndTime=\"2001-01-02T01:30\",\n", + ")\n", + "\n", + "# new config file genertaed to run the model \n", + "print(f\"New config file {config_path}\")\n", + "\n", + "# see input and output paths generated by the model\n", + "print(f'Model input dir {model.config[\"InputPath\"]}')\n", + "print(f'Model output dir {model.config[\"OutputPath\"]}')" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'Reading config from /home/sarah/temp/ecoextreml/test/input/ZA-Kru_2024-01-24-1125/ZA-Kru_2024-01-24-1125_config.txt\\nFinished model initialization\\nThe calculations start now\\nFinalizing STEMMUS_SCOPE\\n'" + ] + }, + "execution_count": 6, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# run the model\n", + "result = model.run()\n", + "result" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "/home/sarah/temp/ecoextreml/test/output/ZA-Kru_2024-01-24-1125/ZA-Kru_2024-01-24-1125_STEMMUS_SCOPE.nc\n" + ] + } + ], + "source": [ + "# save output in netcdf format\n", + "required_netcdf_variables = \"./required_netcdf_variables.csv\"\n", + "nc_file_name = save.to_netcdf(config_path, required_netcdf_variables)\n", + "print(nc_file_name)" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Using global data" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [], + "source": [ + "import os\n", + "from PyStemmusScope import StemmusScope\n", + "from PyStemmusScope import save" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [], + "source": [ + "# user must provide the correct path\n", + "path_to_config_file = \"./my_config_template_global.txt\"\n", + "path_to_exe_file = \"./STEMMUS_SCOPE\"" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "If you run the model on your own machine, make sure `LD_LIBRARY_PATH` is set correctly. To do this, uncomment the cell below and run it:" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "/usr/local/MATLAB/MATLAB_Runtime/R2023a/runtime/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/R2023a/bin/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/R2023a/sys/os/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/R2023a/extern/bin/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/R2023a/sys/opengl/lib/glnxa64\n" + ] + } + ], + "source": [ + "# # Set LD_LIBRARY_PATH\n", + "# matlab_path = !whereis MATLAB\n", + "# matlab_path = matlab_path.s.split(\": \")[1]\n", + "# os.environ['LD_LIBRARY_PATH'] = (\n", + "# f\"{matlab_path}/MATLAB_Runtime/R2023a/runtime/glnxa64:\"\n", + "# f\"{matlab_path}/MATLAB_Runtime/R2023a/bin/glnxa64:\"\n", + "# f\"{matlab_path}/MATLAB_Runtime/R2023a/sys/os/glnxa64:\"\n", + "# f\"{matlab_path}/MATLAB_Runtime/R2023a/extern/bin/glnxa64:\"\n", + "# f\"{matlab_path}/MATLAB_Runtime/R2023a/sys/opengl/lib/glnxa64\")\n", + "# print(os.environ['LD_LIBRARY_PATH'])" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": {}, + "outputs": [], + "source": [ + "# create an an instance of the model\n", + "model = StemmusScope(config_file=path_to_config_file, model_src_path=path_to_exe_file)" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "/home/sarah/miniconda3/envs/pystemmusscope/lib/python3.10/site-packages/dask/array/core.py:4836: PerformanceWarning: Increasing number of chunks by factor of 16\n", + " result = blockwise(\n", + "/home/sarah/miniconda3/envs/pystemmusscope/lib/python3.10/site-packages/dask/array/core.py:4836: PerformanceWarning: Increasing number of chunks by factor of 16\n", + " result = blockwise(\n", + "/home/sarah/miniconda3/envs/pystemmusscope/lib/python3.10/site-packages/dask/array/core.py:4836: PerformanceWarning: Increasing number of chunks by factor of 16\n", + " result = blockwise(\n", + "/home/sarah/miniconda3/envs/pystemmusscope/lib/python3.10/site-packages/dask/array/core.py:4836: PerformanceWarning: Increasing number of chunks by factor of 16\n", + " result = blockwise(\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "New config file /home/sarah/temp/ecoextreml/test/input/global_N61-847_E24-295_2024-02-02-1620/global_2024-02-02-1620_config.txt\n", + "Model input dir /home/sarah/temp/ecoextreml/test/input/global_N61-847_E24-295_2024-02-02-1620/\n", + "Model output dir /home/sarah/temp/ecoextreml/test/output/global_N61-847_E24-295_2024-02-02-1620/\n" + ] + } + ], + "source": [ + "# setup the model\n", + "config_path = model.setup(\n", + " Location=\"(61.8474, 24.2948)\",\n", + " StartTime=\"2014-01-10T00:00\",\n", + " EndTime=\"2014-01-15T00:00\",\n", + ")\n", + "\n", + "# new config file genertaed to run the model \n", + "print(f\"New config file {config_path}\")\n", + "\n", + "# see input and output paths generated by the model\n", + "print(f'Model input dir {model.config[\"InputPath\"]}')\n", + "print(f'Model output dir {model.config[\"OutputPath\"]}')" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# run the model\n", + "result = model.run()\n", + "result" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# save output in netcdf format\n", + "required_netcdf_variables = \"./required_netcdf_variables.csv\"\n", + "nc_file_name = save.to_netcdf(config_path, required_netcdf_variables)\n", + "print(nc_file_name)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.13" + }, + "vscode": { + "interpreter": { + "hash": "281eb038f5f4b6dc08cc25b19e3f510bd63483fa656fb888636d147f45c9c6d5" + } + } + }, + "nbformat": 4, + "nbformat_minor": 4 +} diff --git a/docs/notebooks/run_model_with_different_dataset_on_CRIB.ipynb b/docs/notebooks/run_model_with_different_dataset_on_CRIB.ipynb new file mode 100644 index 00000000..a668edb9 --- /dev/null +++ b/docs/notebooks/run_model_with_different_dataset_on_CRIB.ipynb @@ -0,0 +1,324 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Running the STEMMUS_SCOPE model with different dataset on CRIB using MATLAB\n", + "\n", + "This notebook shows how to run the STEMMUS_SCOPE model uisng source code, including preprocessing and postprocessing. For postprocesisng, the file required_netcdf_variables.csv can be found at https://github.com/EcoExtreML/STEMMUS_SCOPE/blob/main/utils/csv_to_nc/required_netcdf_variables.csv.\n", + "\n", + "See the [Running the STEMMUS_SCOPE model on a system](./run_model_on_different_infra.ipynb) for different ways e.g. Executable file, Matlab, Octave.\n", + "\n", + "## Which datasets are available?\n", + "The config file shows how to set the paths to different data needed by the model. Check the data source on Snelius and CRIB and modify the config file accordingly:\n", + "\n", + "#### On CRIB\n", + "- Site data:\n", + " - path to data: `/data/shared/EcoExtreML/STEMMUS_SCOPEv1.0.0/input`,\n", + " - config file template: https://github.com/EcoExtreML/STEMMUS_SCOPE/blob/main/config_file_crib.txt\n", + "- Global data:\n", + " - path to data `/data/shared/EcoExtreML/STEMMUS_SCOPEv1.0.0/global_data`,\n", + " - config file template: https://github.com/EcoExtreML/STEMMUS_SCOPE/blob/main/config_file_crib_global.txt\n", + "\n", + "#### On Snellius\n", + "- Site data:\n", + " - path to data: `/projects/0/ttse0619/`,\n", + " - config file template: https://github.com/EcoExtreML/STEMMUS_SCOPE/blob/main/run_model_on_snellius/config_file_snellius.txt\n", + "- Global data:\n", + " - path to data: `/projects/0/ttse0619/global_data`,\n", + " - config file template: https://github.com/EcoExtreML/STEMMUS_SCOPE/blob/main/run_model_on_snellius/config_file_snellius_global.txt\n", + "\n", + "Choose how do you want to run the model with MATLAB on CRIB:\n", + "\n", + "[**Using site data**](#1-using-site-data)\n", + "\n", + "[**Using global data**](#2-using-global-data)\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 1. Using site data" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [], + "source": [ + "import os\n", + "from PyStemmusScope import StemmusScope\n", + "from PyStemmusScope import save" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [], + "source": [ + "# user must provide the correct path\n", + "path_to_config_file = \"./config_file_crib.txt\"\n", + "path_to_model_src = \"/home/jovyan/private/Sarah/STEMMUS_SCOPE/src\"" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [], + "source": [ + "# create an an instance of the model\n", + "model = StemmusScope(config_file=path_to_config_file, model_src_path=path_to_model_src, interpreter=\"Matlab\")" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "New config file /data/private/test_run/input/ZA-Kru_2024-01-29-1142/ZA-Kru_2024-01-29-1142_config.txt\n", + "Model input dir /data/private/test_run/input/ZA-Kru_2024-01-29-1142/\n", + "Model output dir /data/private/test_run/output/ZA-Kru_2024-01-29-1142/\n" + ] + } + ], + "source": [ + "# setup the model\n", + "# here you can change the location and start & end time\n", + "config_path = model.setup(\n", + " Location=\"ZA-Kru\",\n", + " StartTime=\"2001-01-01T00:00\",\n", + " EndTime=\"2001-01-02T01:30\",\n", + ")\n", + "\n", + "# new config file genertaed to run the model \n", + "print(f\"New config file {config_path}\")\n", + "\n", + "# see input and output paths generated by the model\n", + "print(f'Model input dir {model.config[\"InputPath\"]}')\n", + "print(f'Model output dir {model.config[\"OutputPath\"]}')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "  If you are on CRIB, to use Jupyter interface and MATLAB license manager at the same time, first perform in a terminal in the Remote Desktop of Crib Platform, and check the output which should be similar to :1.0 . Then execute the following line accordingly." + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "env: DISPLAY=:1.0\n" + ] + } + ], + "source": [ + "# For CRIB users\n", + "%env DISPLAY=:1.0" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'\\n < M A T L A B (R) >\\n Copyright 1984-2021 The MathWorks, Inc.\\n R2021a (9.10.0.1602886) 64-bit (glnxa64)\\n February 17, 2021\\n\\n \\nTo get started, type doc.\\nFor product information, visit www.mathworks.com.\\n \\nReading config from /data/private/test_run/input/ZA-Kru_2024-01-29-1142/ZA-Kru_2024-01-29-1142_config.txt\\nFinished model initialization\\nThe calculations start now\\nFinalizing STEMMUS_SCOPE\\n'" + ] + }, + "execution_count": 8, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# run the model\n", + "result = model.run()\n", + "result" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "/data/private/test_run/output/ZA-Kru_2024-01-29-1142/ZA-Kru_2024-01-29-1142_STEMMUS_SCOPE.nc\n" + ] + } + ], + "source": [ + "# save output in netcdf format\n", + "required_netcdf_variables = \"/home/jovyan/private/Sarah/STEMMUS_SCOPE/utils/csv_to_nc/required_netcdf_variables.csv\"\n", + "nc_file_name = save.to_netcdf(config_path, required_netcdf_variables)\n", + "print(nc_file_name)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 2. Using global data" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [], + "source": [ + "import os\n", + "from PyStemmusScope import StemmusScope\n", + "from PyStemmusScope import save" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [], + "source": [ + "# user must provide the correct path\n", + "path_to_config_file = \"./config_file_crib_global.txt\"\n", + "path_to_model_src = \"/home/jovyan/private/Sarah/STEMMUS_SCOPE/src\"" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [], + "source": [ + "# create an an instance of the model\n", + "model = StemmusScope(config_file=path_to_config_file, model_src_path=path_to_model_src, interpreter=\"Matlab\")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# setup the model\n", + "config_path = model.setup(\n", + " Location=\"(61.8474, 24.2948)\",\n", + " StartTime=\"2014-01-01T00:00\",\n", + " EndTime=\"2014-01-15T00:00\",\n", + ")\n", + "\n", + "# new config file genertaed to run the model \n", + "print(f\"New config file {config_path}\")\n", + "\n", + "# see input and output paths generated by the model\n", + "print(f'Model input dir {model.config[\"InputPath\"]}')\n", + "print(f'Model output dir {model.config[\"OutputPath\"]}')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "  If you are on CRIB, to use Jupyter interface and MATLAB license manager at the same time, first perform in a terminal in the Remote Desktop of Crib Platform, and check the output which should be similar to :1.0 . Then execute the following line accordingly." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "env: DISPLAY=:1.0\n" + ] + } + ], + "source": [ + "# for CRIB users\n", + "%env DISPLAY=:1.0" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "b'MATLAB is selecting SOFTWARE OPENGL rendering.\\nOpening log file: /home/alidoost/java.log.37351\\n\\n < M A T L A B (R) >\\n Copyright 1984-2021 The MathWorks, Inc.\\n R2021a Update 3 (9.10.0.1684407) 64-bit (glnxa64)\\n May 27, 2021\\n\\n \\nTo get started, type doc.\\nFor product information, visit www.mathworks.com.\\n \\nReading config from /gpfs/home2/alidoost/test_matlab/input/ZA-Kru_2022-10-27-1651/ZA-Kru_2022-10-27-1651_config.txt\\nThe calculations start now\\nThe calculations end now\\n'" + ] + }, + "execution_count": 7, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# run the model\n", + "result = model.run()\n", + "result" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# save output in netcdf format\n", + "required_netcdf_variables = \"/home/jovyan/private/Sarah/STEMMUS_SCOPE/utils/csv_to_nc/required_netcdf_variables.csv\"\n", + "nc_file_name = save.to_netcdf(config_path, required_netcdf_variables)\n", + "print(nc_file_name)" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.13" + }, + "vscode": { + "interpreter": { + "hash": "281eb038f5f4b6dc08cc25b19e3f510bd63483fa656fb888636d147f45c9c6d5" + } + } + }, + "nbformat": 4, + "nbformat_minor": 4 +} diff --git a/mkdocs.yml b/mkdocs.yml index fad6f505..b17d44a1 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -5,7 +5,11 @@ repo_name: PyStemmusScope nav: - Getting started: index.md - Installation: installation_instructions.md - - Notebook page: notebooks/run_model_in_notebook.ipynb + - Run the model: + - Different ways to run the model: notebooks/run_model_on_different_infra.ipynb + - Site and global datasets with the executable file: notebooks/run_model_with_different_dataset.ipynb + - Site and global datasets on CRIB: notebooks/run_model_with_different_dataset_on_CRIB.ipynb + - BMI: - "BMI instructions": bmi.md - "BMI demonstration": notebooks/BMI_demo.ipynb