Model is not running -> issue in the SAVEhh variable

[MostafaGomaa93]

Hi all,

I am a new user of STEMMUS_SCOPE. I am trying to follow the guidelines to run the model. But I have quite a few issues. The first one is that I can't find the filesread.m in the src folder of this repo. As I understand following the how_to_run_on_Crib guidelines (https://github.com/EcoExtreML/STEMMUS_SCOPE/blob/main/docs/STEMMUS_SCOPE_on_CRIB.md), I need to open the filesread.m file and set the variable CFG to the path of the config file

Could you please give me a guide? @yijianzeng

Thanks a lot in advance
Mostafa

[SarahAlidoost]

@MostafaGomaa93 thanks for submitting the issue. The instructions is outdated since we are refactoring the stemmus_scope code. You can set the path to CFG variable in STEMMUS_SCOPE.m main script, see this line.

[MostafaGomaa93]

@SarahAlidoost, thanks for your answer.

I am already facing another path issue. I can't find the Mdata.txt file in the src code which as I understand will be used to create the forcing_global.mat file. Can you please give me a guide?

[SarahAlidoost]

I am already facing another path issue. I can't find the Mdata.txt file in the src code which as I understand will be used to create the forcing_global.mat file. Can you please give me a guide?

Mdata.txt is a part of the input data to the model rather than the source code. It should be already available in the input folder. Here are my suggestions:
1- make sure you are using the latest version of stemmus_scope
2- use pystemmusscope to prepare input data, see documentation

@Yunfei-Wang1993, @yijianzeng, @Crystal-szj and @DanyangYu since you are familiar with the packages and how to use them on CRIB, could you please help @MostafaGomaa93 to run the model?

[Crystal-szj]

Hi @MostafaGomaa93, you can use CRIB to run the latest version of STEMMUS-SCOPE. Before running, the input data can be created by PyStemmusScope, a pre-processing package for preparing the requested input data of STEMMUS-SCOPE.

1. Download the PyStemmusScope and the latest version of STEMMUS_SCOPE to your workspace.

git clone [email protected]:EcoExtreML/STEMMUS_SCOPE_Processing.git
git clone [email protected]:EcoExtreML/STEMMUS_SCOPE.git

2. Install PyStemmusScope, see here. This step can create all the requested input data of STEMMUS_SCOPE and run the model directly. In Section 2 Using Matlab -- cell 7, result = model.run() is running the STEMMUS_SCOPE via Jupyter notebook, and if you run it successfully, the result should be saved in OutputPath.

You can also try the above step on your own computer if your computer has a Linux system. Noted the WSL in the Window system can not write the output of PyStemmusScope since the WSL doesn't have the write permission.

[MostafaGomaa93]

Hi @SarahAlidoost and @yijianzeng

I followed the instructions mentioned here (https://pystemmusscope.readthedocs.io/en/v0.2.0/notebooks/run_model_in_notebook/). I tried running through the Executable file option and the Matlab option. Both didn't work. I am running on the ITC Linux668 machine.

For the run with the Executable option. I received the following error:

For the run with the Matlab option. I received the following message with no response and I left it for 2 days and nothing happened. I can see the outputs files but they are all empty:

I would appreciate your help since I have already asked many ITC colleagues and almost all of them are not using pystemmusscope. They are running the model through Matlab and Crib with former guidelines. I am still in this step from almost two weeks. I am using inputs from one of the 170 Fluxnet sites

[SarahAlidoost]

@MostafaGomaa93 There are several issues with how you are trying to run the model. Here are the list of possible solutions:
1- You are using a command python3 run_model_matlab.py in a base conda environment. This is not illustrated anywhere in the documentation! I suggest using only the options mentioned in the documentation, see section Which options are available? here. Please note the Matlab, Matlab runtime and python versions that are mentioned in the documentation.
2- Also, the running model on the ITC Linux668 has not been tested. Please see the installation options and the machines in the documentation.
3- You are using the notebook in the version v0.2.0. The latest version of pystemmuscope is 0.3.0, see the notebook here.
4- Please follow the steps in the documentation as: 1. Getting started, 2. Installation, 3. Running the STEMMUS_SCOPE model

Hope it helps.

[MostafaGomaa93]

Is there a small test (with all required inputs) available that I can use to run the model and figure out more where are the problems?

[MostafaGomaa93]

@SarahAlidoost
can I get a copy of the test model files that are used in the pystemmusscope notebook?

[SarahAlidoost]

@SarahAlidoost can I get a copy of the test model files that are used in the pystemmusscope notebook?

yes, I just ran the model on CRIB using the Octave option explained in the notebook and copied the input and output files to shared/EcoExtreML/pystemmusscope_notebook_results

[MostafaGomaa93]

@SarahAlidoost
Thanks so much. I will try it

[MostafaGomaa93]

@SarahAlidoost

I have tried the test model in Crib (with the assistance of Bas, our ITC-WRS developer). Here are some conclusions we had

1. The model doesn't run through the Octave option. It shows the below error of missing package. We have tried to install this statistics package, but doesn't work, because it asked for Octave 7.2. and the Octave version of Crib is 7.1. and we don't have the right to upgrade Octave in Crib.
   

2. The model runs through the Matlab option but only through the terminal inside the Crib Remote Desktop, so in Crib -> we open the Remote Desktop -> open a terminal -> run the model -> it will ask to activate the Matlab licence (screenshot below) -> runs successfully.
   

3. However, when we tried to run the model again in the normal terminal of Crib (which is outside the Crib Remote Desktop), it didn't run. Bas expects that the normal terminal somehow can not connect to Matlab (we didn't receive the request to access Matlab as was the case when we ran through the terminal inside the Crib Remote Desktop). This was strange to us and we don't know why.

[SarahAlidoost]

@SarahAlidoost

I have tried the test model in Crib (with the assistance of Bas, our ITC-WRS developer). Here are some conclusions we had

1. The model doesn't run through the Octave option. It shows the below error of missing package. We have tried to install this statistics package, but doesn't work, because it asked for Octave 7.2. and the Octave version of Crib is 7.1. and we don't have the right to upgrade Octave in Crib.
   

There is an octave instruction in the STEMUUS_SCOPE repository where it explained which versions of io and statistics packages to install i.e. io-2.6.4.tar.gz and statistics-1.4.3.tar.gz

3. However, when we tried to run the model again in the normal terminal of Crib (which is outside the Crib Remote Desktop), it didn't run. Bas expects that the normal terminal somehow can not connect to Matlab (we didn't receive the request to access Matlab as was the case when we ran through the terminal inside the Crib Remote Desktop). This was strange to us and we don't know why.

I noticed the same issue about CRIB and MATLAB. The notebook with matlab and octave options is a solution to this problem. Alternatively, you can run the model on SURF Snelius or your own machine using exe file that does not need matlab license.

[MostafaGomaa93]

@SarahAlidoost. Thanks for the guideline and sorry that I am still bothering you. My main intention is to indeed run the model on my own machine (e.g. ITC Linux 668 server). I have tried to run the same test model (using the Matlab option) on the server that already runs on Crib, but I have received the below error.

When I check the source code -> go to the solveTridiagonalMatrixEquation.m file ->line 24 -> change the variable SAVEhh to a defined value (e.g. SAVEhh = -1.5) -> then, the model runs. So, I still don't know why this error is there when running on the server while it was not there when running on Crib. Any suggestion?

I am also not that familiar with the model, so I don't know from where this SAVEhh is calculated. Maybe some explanation from @Crystal-szj, @DanyangYu, @EntingTang, @yijianzeng would help (would highly appreciate it)

[SarahAlidoost]

My main intention is to indeed run the model on my own machine (e.g. ITC Linux 668 server).

I don't have access to ITC Linux 668 server. Can you tell me the version of MATLAB on the server?

I have tried to run the same test model (using the Matlab option) on the server that already runs on Crib,

Were you able to run the model on CRIB using matlab?

but I have received the below error.

There are several reasons why there is such error, see some issues

• How to solve this error "Operands to the || and && operators must be convertible to logical scalar values." ?
• Operands to the || and && operators must be convertible to logical scalar values.

Any suggestion?

1. inspect the values of SAVEhh
2. As suggested by MATLAB, use any function for checking the values of SAVEhh for example if any(isnan(SAVEhh)) || any(~isreal(SAVEhh)). If you are fixing this line of the code, please make a pull request to contribute your fixes into the model.

[MostafaGomaa93]

@SarahAlidoost
Okay, I think I found where is the problem. Indeed the any() function was needed to let the model run. So, I tried this change as you mentioned (if any(isnan(SAVEhh)) || any(~isreal(SAVEhh))) and the model runs successfully on the Linux668 (with MATLAB 2022b v9.13.0 and with input/output via pystemmusscope).

I also realized why the model was running on Crib and not on the server (before the change above). The source code on Crib is still the old one and not the refactored one. I copied the refactored source code in Crib and it didn't run, which proves that the issue was not about the machine type, the MATLAB version, or the inputs, but the issue was the slight change (any() function) needed in line 24 in the solveTridiagonalMatrixEquation.m file. I will suggest this change in a separate pull request

So, now the model runs on the Linux machine using the MATLAB option (great success hhhh). Still running through the executable is not working. I have tried to create a new exe file from the source code (with the any() function) change using the command I found in the build_stemmus_scope_exe.sh file (mcc -m ./src/STEMMUS_SCOPE_exe.m -a ./src -d ./run_model_on_snellius/exe -o STEMMUS_SCOPE -R nodisplay -R singleCompThread), but still the model doesn't run with this new exe file. maybe I am missing something!

[SarahAlidoost]

@SarahAlidoost Okay, I think I found where is the problem. Indeed the any() function was needed to let the model run. So, I tried this change as you mentioned (if any(isnan(SAVEhh)) || any(~isreal(SAVEhh))) and the model runs successfully

Did you perhaps inspect the results? are the outputs of your run the same as the outputs on crib that I shared before?

on the Linux668 (with MATLAB 2022b v9.13.0 and with input/output via pystemmusscope).

The MATLAB version on CRIB and snellius is 2021a.

I also realized why the model was running on Crib and not on the server (before the change above). The source code on Crib is still the old one and not the refactored one. I copied the refactored source code in Crib

I suggest not using the source code on shared directory on CRIB as it seems it has not been maintained. But either download the latest version from the release page or clone the STEMMUS_SCOPE github repository under your own user directory on CRIB.

and it didn't run, which proves that the issue was not about the machine type, the MATLAB version, or the inputs, but the issue was the slight change (any() function) needed in line 24 in the solveTridiagonalMatrixEquation.m file.

Please note that the latest version of the model is tested using matlab and exe on linux machines e.g. snelius and using octave on crib. So, I think since the matlab version on the ITC Linux server is different, this might be the cause of the issue.

I will suggest this change in a separate pull request

Thank you, I will review the pull request.

So, now the model runs on the Linux machine using the MATLAB option (great success hhhh). Still running through the executable is not working.

The exe file on this repository won't work on ITC linux server if the version of matlab runtime does not match matlab version 2021a , see documentation.

I have tried to create a new exe file from the source code (with the any() function) change using the command I found in the build_stemmus_scope_exe.sh file (mcc -m ./src/STEMMUS_SCOPE_exe.m -a ./src -d ./run_model_on_snellius/exe -o STEMMUS_SCOPE -R nodisplay -R singleCompThread),

If others also use ITC linux server, I suggest using a different output directory for example run_model_on_itc_linux668 and sharing it with a separate pull request. This way others can also use the same exe file.

but still the model doesn't run with this new exe file. maybe I am missing something!

What is the error? one guess is LD_LIBRARY_PATH. As explained in the notebook, when using the exe file on your own machine, make sure LD_LIBRARY_PATH is set correctly. See cell number 18 in the notebook. This shows how to set LD_LIBRARY_PATH using python. You can do the same things using bash. Perhaps the server administrator can help with this.

[SarahAlidoost]

@MostafaGomaa93 I generated a new exe file based on your fixes in #203 and merged your pull request. Could you please run the model on ITC linux serve with a matlab runtime that matches matlab version 2021a? Please set the LD_LIBRARY_PATH before running the model.

[MostafaGomaa93]

@SarahAlidoost
Yes, I just tested the new exe in the Linux server and finally, it works. So now it works with both options via Matlab and via the exe. Thanks so much. It took a while to get it but I am very happy that I can go forward (thanks again for your efforts). May I get your email? I have a question but not related to this issue.

[SarahAlidoost]

@SarahAlidoost Yes, I just tested the new exe in the Linux server and finally, it works. So now it works with both options via Matlab and via the exe. Thanks so much. It took a while to get it but I am very happy that I can go forward (thanks again for your efforts).

Thanks for checking it 👍 I close this issue and open another issue about the documentation.

May I get your email? I have a question but not related to this issue.

My email address is on my profile page.
Please note if your question is about the project please cc @yijianzeng. You can also submit it on the Discussion page. This way people with the same question can find the answer.