Octave, subprocess and oct2py

[SarahAlidoost]

When running the model using octave and subprocess as:

cfg_file = f"'ZA-Kru_2022-10-27-1721_config.txt'"
args = f'octave --no-gui --silent --eval "STEMMUS_SCOPE_octave({cfg_file});"'

result = subprocess.run(
    args,
    stdout=subprocess.PIPE, stderr=subprocess.PIPE,
    shell=True, check=True,
)

it runs the model. However, it returns an error:

---------------------------------------------------------------------------
CalledProcessError                        Traceback (most recent call last)
Cell In [12], line 1
----> 1 result = subprocess.run(
      2     args,
      3     stdout=subprocess.PIPE, stderr=subprocess.PIPE,
      4     shell=True, check=True,
      5 )
      6 #TODO handle stderr properly
      7 result.stdout

File /usr/lib/python3.8/subprocess.py:516, in run(input, capture_output, timeout, check, *popenargs, **kwargs)
    514     retcode = process.poll()
    515     if check and retcode:
--> 516         raise CalledProcessError(retcode, process.args,
    517                                  output=stdout, stderr=stderr)
    518 return CompletedProcess(process.args, retcode, stdout, stderr)

CalledProcessError: Command 'octave --no-gui --silent --eval "STEMMUS_SCOPE_octave('ZA-Kru_2022-10-27-1721_config.txt');"' returned non-zero exit status 139.

Also, when running the model using octave command line in a terminal as (tested on CRIB):

octave --no-gui --silent --eval "STEMMUS_SCOPE_octave('ZA-Kru_2022-10-27-1721/ZA-Kru_2022-10-27-1721_config.txt');"

it finishes the run but with the error :

fatal: caught signal Segmentation fault -- stopping myself...
Segmentation fault (core dumped)

We can use oct2py package instead of subprocess. However, oct2py requires octave installed on the system. This causes issues for github action workflow.

[SarahAlidoost]

exit code 139 is related to Segmentation fault, see here.

[yijianzeng]

@SarahAlidoost @BSchilperoort
When i run Octave on CRIB JNB, it returns the following error:

CalledProcessError Traceback (most recent call last)
Cell In [5], line 2
1 # run the model
----> 2 result = model.run()
3 result

File ~/.local/lib/python3.8/site-packages/PyStemmusScope/stemmus_scope.py:228, in StemmusScope.run(self)
225 args = ["octave", "--eval", eval_code, "--no-gui", "--silent"]
227 # seperate args dont work on linux!
--> 228 result = _run_sub_process(
229 args if utils.os_name() == "nt" else shlex.join(args), self.model_src
230 )
231 return result

File ~/.local/lib/python3.8/site-packages/PyStemmusScope/stemmus_scope.py:85, in _run_sub_process(args, cwd)
81 # TODO handle stderr properly
82 # when using octave, exit_code might be 139
83 # see issue STEMMUS_SCOPE_Processing/issues/46
84 if exit_code not in [0, 139]:
---> 85 raise subprocess.CalledProcessError(
86 returncode=exit_code, cmd=args, stderr=stderr, output=stdout
87 )
88 if exit_code == 139:
89 logger.warning(stderr)

CalledProcessError: Command 'octave --eval 'STEMMUS_SCOPE_exe('"'"'/data/shared/Digital-Twin-Earth-Water-Energy-Food-2023-1B/Test_YJ/input/ZA-Kru_2024-02-24-1158/ZA-Kru_2024-02-24-1158_config.txt'"'"');exit;' --no-gui --silent' returned non-zero exit status 1.

I am not sure if you also encountered this issue?

[SarahAlidoost]

I just run the notebook run_model_on_different_infra.ipynb using octave on crib successfully. If you are able to run the model using matlab in the same notebook, make sure that the correct versions of the octave packages io and statistics are installed, see using-stemmus-scope-with-gnu-octave.

If Matlab run also fails, make sure you are using the latest versions of pystemmuscope and stemmus_scope. Also, python version should be >=3.9. Above, I see that you are using python 3.8.