python embedding; conda env

[junmeiban]

Dear MET_help,

I have an issue about python embedding feature.

When I ran grid_stat and got the following error on Cheyenne (using conda env; see error log file here : /glade/work/jban/pandac/cloud_vx/bin/MET/24.CMORPH.log_err and MET_verification_grid.o8319873):

 /glade/p/ral/jntp/MET/MET_releases/11.0.0/bin/grid_stat PYTHON_NUMPY PYTHON_NUMPY /glade/work/jban/pandac/cloud_vx/static/MET/met_config/GridStatConfig_trad -outdir
glade/scratch/jban/CMORPH/
  File "/glade/p/ral/jntp/MET/MET_releases/11.0.0/share/met/wrappers/write_tmp_dataplane.py", line 36, in <module>
    spec.loader.exec_module(met_in)
  File "<frozen importlib._bootstrap_external>", line 843, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "python_script_fcst.py", line 3, in <module>
    import python_stuff
ModuleNotFoundError: No module named 'python_stuff'
ERROR  :
ERROR  : tmp_nc_dataplane() -> command "/glade/u/apps/opt/conda/envs/npl/bin/python /glade/p/ral/jntp/MET/MET_releases/11.0.0/share/met/wrappers/write_tmp_dataplane.py /tmp/tmp_met_nc_6198_0 python_script_fcst" failed ... status = 256
ERROR  :

But when I run python_script_fcst.py separately (even though it’s nonmeaningful ), it works for import “python_stuff”.

Any ideas of fixing this error are welcome. Many thanks!

[georgemccabe]

Hi @junmeiban, when you run a Python Embedding script in MET, it uses the version of Python that was installed with the MET tools by default. You can set the MET_PYTHON_EXE environment variable to the full path of another python3 binary to use that version of Python to run your script in MET. If the version of python3 in your path is the one you want to use, you can set:

export MET_PYTHON_EXE=which python3`

FYI, starting in the upcoming MET 11.1.0 release, the version of Python that is being used by the commands will be logged so it will be much easier to tell what is being used.

Please let me know if this resolves your issue or if you need additional assistance.

[jprestop]

@DanielAdriaansen /glade/p/ral/jntp/MET/METplus/miniconda/miniconda3/envs/metplus_v5.0_py3.8/bin/python is the version that MET was installed with and that METplus uses. pygrib wasn't in our list of packages to install in that environment, however, I will try to install that for the user now. i will follow up once it has completed.

[junmeiban]

It works! Thank you so much for your help @DanielAdriaansen .

[jprestop]

Since you have it working @junmeiban I have cancelled the installation of pygrib, but if you end up needing it in that location, please let us know.

[DanielAdriaansen]

No problem @junmeiban! @jprestop I think we can hold off on installing pygrib in that Python installation.

Just a note for the future, we will be improving our documentation to more strongly suggest that users provide absolute paths to their Python scripts since without that, unexpected behavior can occur as we see here. Thanks for helping us improve our documentation with this discussion!

If you could select an answer @junmeiban, we will mark this as closed.

[junmeiban]

This issue can be closed. Thanks you all!