docs/notebooks/run_model_on_different_infra.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Running the STEMMUS_SCOPE model on a system\n",
    "\n",
    "This notebook shows steps to run the STEMMUS_SCOPE model, including preprocessing and postprocessing. STEMMUS_SCOPE files are located in a repository https://github.com/EcoExtreML/STEMMUS_SCOPE. Then specify the path to a config file e.g.`config_file_template.txt`, executable file `STEMMUS_SCOPE` or source code `STEMMUS_SCOPE/src` in the cells below. There are already config files for users on Snellius and CRIB, see `config_file_snellius.txt`, `config_file_crib.txt`. Depending on your system (OS, Matlab availability, etc.), there are a few ways you can run the model. Choose how do you want to run the model:\n",
    "\n",
    "\n",
    "[**Executable file**](#1-using-executable-file)\n",
    "\n",
    "&emsp; If MATLAB Runtime is installed on a Unix-like system, you can run STEMMUS_SCOPE using the executable file, for example on Snellius or WSL.\n",
    "\n",
    "[**Matlab**](#2-using-matlab)\n",
    "\n",
    "&emsp; If Matlab is installed, you can run STEMMUS_SCOPE from the source code, for example on Snellius or CRIB.\n",
    "\n",
    "[**Octave**](#3-using-octave)\n",
    "\n",
    "&emsp; If latest Octave including required packages is installed, you can run STEMMUS_SCOPE from the source code, for example on CRIB or your own machine."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Using executable file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "from PyStemmusScope import StemmusScope\n",
    "from PyStemmusScope import save"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "# user must provide the correct path\n",
    "path_to_config_file = \"~/my_config_template.txt\"\n",
    "path_to_exe_file = \"~/STEMMUS_SCOPE/exe/STEMMUS_SCOPE\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you run the model on your own machine, make sure `LD_LIBRARY_PATH` is set correctly. To do this, uncomment the cell below and run it:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "# # Set LD_LIBRARY_PATH\n",
    "# matlab_path = !whereis MATLAB\n",
    "# matlab_path = matlab_path.s.split(\": \")[1]\n",
    "# os.environ['LD_LIBRARY_PATH'] = (\n",
    "#     f\"{matlab_path}/MATLAB_Runtime/v910/runtime/glnxa64:\"\n",
    "#     f\"{matlab_path}/MATLAB_Runtime/v910/bin/glnxa64:\"\n",
    "#     f\"{matlab_path}/MATLAB_Runtime/v910/sys/os/glnxa64:\"\n",
    "#     f\"{matlab_path}/MATLAB_Runtime/v910/extern/bin/glnxa64:\"\n",
    "#     f\"{matlab_path}/MATLAB_Runtime/v910/sys/opengl/lib/glnxa64\")\n",
    "# print(os.environ['LD_LIBRARY_PATH'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create an an instance of the model\n",
    "model = StemmusScope(config_file=path_to_config_file, model_src_path=path_to_exe_file)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "New config file /gpfs/home2/alidoost/test_matlab/input/ZA-Kru_2022-10-27-1642/ZA-Kru_2022-10-27-1642_config.txt\n",
      "Model input dir /gpfs/home2/alidoost/test_matlab/input/ZA-Kru_2022-10-27-1642/\n",
      "Model output dir /gpfs/home2/alidoost/test_matlab/output/ZA-Kru_2022-10-27-1642/\n"
     ]
    }
   ],
   "source": [
    "# setup the model\n",
    "# here you can change the location and start & end time\n",
    "config_path = model.setup(\n",
    "    Location=\"ZA-Kru\",\n",
    "    StartTime=\"2001-01-01T00:00\",\n",
    "    EndTime=\"2001-01-02T01:30\",\n",
    ")\n",
    "\n",
    "# new config file genertaed to run the model \n",
    "print(f\"New config file {config_path}\")\n",
    "\n",
    "# see input and output paths generated by the model\n",
    "print(f'Model input dir {model.config[\"InputPath\"]}')\n",
    "print(f'Model output dir {model.config[\"OutputPath\"]}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "b'Opening log file:  /gpfs/home2/alidoost/test_matlab/input/ZA-Kru_2022-10-27-1642//java.log.5036\\nReading config from /gpfs/home2/alidoost/test_matlab/input/ZA-Kru_2022-10-27-1642/ZA-Kru_2022-10-27-1642_config.txt\\nThe calculations start now\\nThe calculations end now\\n'"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# run the model\n",
    "result = model.run()\n",
    "result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "# save output in netcdf format\n",
    "required_netcdf_variables = \"~/STEMMUS_SCOPE/utils/csv_to_nc/required_netcdf_variables.csv\"\n",
    "nc_file_name = save.to_netcdf(config_path, required_netcdf_variables)\n",
    "print(nc_file_name)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Using Matlab"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "from PyStemmusScope import StemmusScope\n",
    "from PyStemmusScope import save"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# user must provide the correct path\n",
    "path_to_config_file = \"~/my_config_template.txt\"\n",
    "path_to_model_src = \"~/STEMMUS_SCOPE/src\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create an an instance of the model\n",
    "model = StemmusScope(config_file=path_to_config_file, model_src_path=path_to_model_src, interpreter=\"Matlab\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "New config file /gpfs/home2/alidoost/test_matlab/input/ZA-Kru_2022-10-27-1651/ZA-Kru_2022-10-27-1651_config.txt\n",
      "Model input dir /gpfs/home2/alidoost/test_matlab/input/ZA-Kru_2022-10-27-1651/\n",
      "Model output dir /gpfs/home2/alidoost/test_matlab/output/ZA-Kru_2022-10-27-1651/\n"
     ]
    }
   ],
   "source": [
    "# setup the model\n",
    "config_path = model.setup(\n",
    "    Location=\"ZA-Kru\",\n",
    "    StartTime=\"2001-01-01T00:00\",\n",
    "    EndTime=\"2001-01-02T01:30\",\n",
    ")\n",
    "\n",
    "# new config file genertaed to run the model \n",
    "print(f\"New config file {config_path}\")\n",
    "\n",
    "# see input and output paths generated by the model\n",
    "print(f'Model input dir {model.config[\"InputPath\"]}')\n",
    "print(f'Model output dir {model.config[\"OutputPath\"]}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "&emsp; If you are on CRIB, to use Jupyter interface and MATLAB license manager at the same time, first perform <echo $DISPLAY> in a terminal in the Remote Desktop of Crib Platform, and check the output which should be similar to :1.0 . Then execute the following line accordingly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# For CRIB users\n",
    "%env DISPLAY=:1.0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "b'MATLAB is selecting SOFTWARE OPENGL rendering.\\nOpening log file:  /home/alidoost/java.log.37351\\n\\n                            < M A T L A B (R) >\\n                  Copyright 1984-2021 The MathWorks, Inc.\\n             R2021a Update 3 (9.10.0.1684407) 64-bit (glnxa64)\\n                                May 27, 2021\\n\\n \\nTo get started, type doc.\\nFor product information, visit www.mathworks.com.\\n \\nReading config from /gpfs/home2/alidoost/test_matlab/input/ZA-Kru_2022-10-27-1651/ZA-Kru_2022-10-27-1651_config.txt\\nThe calculations start now\\nThe calculations end now\\n'"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# run the model\n",
    "result = model.run()\n",
    "result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# save output in netcdf format\n",
    "required_netcdf_variables = \"~/STEMMUS_SCOPE/utils/csv_to_nc/required_netcdf_variables.csv\"\n",
    "nc_file_name = save.to_netcdf(config_path, required_netcdf_variables)\n",
    "print(nc_file_name)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Using Octave"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "from PyStemmusScope import StemmusScope\n",
    "from PyStemmusScope import save"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# user must provide the correct path\n",
    "path_to_config_file = \"~/my_config_template.txt\"\n",
    "path_to_model_src = \"~/STEMMUS_SCOPE/src\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create an an instance of the model\n",
    "model = StemmusScope(config_file=path_to_config_file, model_src_path=path_to_model_src, interpreter=\"Octave\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "New config file /data/private/Sarah/test_run/input/ZA-Kru_2022-10-28-1159/ZA-Kru_2022-10-28-1159_config.txt\n",
      "Model input dir /data/private/Sarah/test_run/input/ZA-Kru_2022-10-28-1159/\n",
      "Model output dir /data/private/Sarah/test_run/output/ZA-Kru_2022-10-28-1159/\n"
     ]
    }
   ],
   "source": [
    "# setup the model\n",
    "# here you can change the location and start & end time\n",
    "config_path = model.setup(\n",
    "    Location=\"ZA-Kru\",\n",
    "    StartTime=\"2001-01-01T00:00\",\n",
    "    EndTime=\"2001-01-02T01:30\",\n",
    ")\n",
    "\n",
    "# new config file genertaed to run the model \n",
    "print(f\"New config file {config_path}\")\n",
    "\n",
    "# see input and output paths generated by the model\n",
    "print(f'Model input dir {model.config[\"InputPath\"]}')\n",
    "print(f'Model output dir {model.config[\"OutputPath\"]}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Reading config from /data/private/Sarah/test_run/input/ZA-Kru_2022-10-28-1159/ZA-Kru_2022-10-28-1159_config.txt\n",
      "The calculations start now\n",
      "The calculations end now\n"
     ]
    }
   ],
   "source": [
    "# run the model\n",
    "result = model.run()\n",
    "result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/data/private/Sarah/test_run/output/ZA-Kru_2022-10-28-1159/ZA-Kru_2022-10-28-1159_STEMMUS_SCOPE.nc\n"
     ]
    }
   ],
   "source": [
    "# save output in netcdf format\n",
    "required_netcdf_variables = \"~/STEMMUS_SCOPE/utils/csv_to_nc/required_netcdf_variables.csv\"\n",
    "nc_file_name = save.to_netcdf(config_path, required_netcdf_variables)\n",
    "print(nc_file_name)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.13"
  },
  "vscode": {
   "interpreter": {
    "hash": "281eb038f5f4b6dc08cc25b19e3f510bd63483fa656fb888636d147f45c9c6d5"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}