Python embedding on Jet

[ywangwof]

On Jet, I would like to use python embedding to process FV3 write component outputs. The module intelpython/3.6.5 is required before loading met/10.0.0. However, module intelpython/3.6.5 does not contain netCDF4 module. So I specify my python as

export MET_PYTHON_EXE=/lfs4/NAGAPE/wof/Brian.Matilla/miniconda3/envs/wofs_post/bin/python3

My python script can read the FV3 file properly, but plot_data_plane failed with this message,

DEBUG 4: Reading temporary Python dataplane file: /tmp/tmp_met_nc_261140_0
Traceback (most recent call last):
  File "/contrib/met/10.0.0/share/met/wrappers/read_tmp_dataplane.py", line 11, in <module>
    import netCDF4 as nc
ModuleNotFoundError: No module named 'netCDF4'
WARNING:
WARNING: tmp_nc_dataplane() -> an error occurred importing module "read_tmp_dataplane"
WARNING:
ERROR  :
ERROR  : plot_data_plane -> trouble getting field "name="/lfs4/NAGAPE/hpc-wof1/ywang/MET2021/METplus-4.0.0_Tutorial/fv3met/read_FV3.py /lfs4/NAGAPE/hpc-wof1/ywang/EPIC2/oumap/rundir/NSSL/20210527/202105272000/fv3prd_mem0001/phyf003:00:00.nc refdmax";" from file "PYTHON_NUMPY"
ERROR  :

How to ensure "read_tmp_dataplane.py" to use the right Python executable specified by MET_PYTHON_EXE? Thanks.

[jprestop]

HI @ywangwof. It looks like the version of Python that MET was compiled with does not include the netCDF4 package necessary to run the code that you'd like to run. It seems that we'll either need to find an existing Python version on Jet that contains the netCDF4 package or create our own conda environment that contains the netCDF4 package and then recompile MET with that version of Python. I will follow up once we recompiled MET with a Python set up that includes the netCDF4 package.

EDIT: For full answer, see comments below.

[jprestop]

@ywangwof Thank you so much for letting us know that you were able to work around this problem. I will still plan to follow up here once we have recompile of MET on Jet.

[jprestop]

Hi @ywangwof. I have installed miniconda3 and have included the netCDF4 package. I installed MET v10.0.1 (our latest stable version) and compiled it using the miniconda3 installation. I have updated the METplus Training scripts on Jet to have the script load the necessary modules to use this version of MET. They are located at /lfs1/HFIP/dtc-hurr/METplus/METplus-4.0.0_Tutorial_Files. To use the script, you would need to copy the new file over your existing file and source it:

cd /lfs4/NAGAPE/hpc-wof1/ywang/MET2021/METplus-4.0.0_Tutorial
cp /lfs1/HFIP/dtc-hurr/METplus/METplus-4.0.0_Tutorial_Files/METplus-4.0.0_TutorialSetup.jet.`basename $SHELL` ./METplus-4.0.0_TutorialSetup.sh
source METplus-4.0.0_TutorialSetup.sh

I have also modified the Existing Builds page to reflect what is needed to use that version of MET and to use METplus with that version of MET for folks who may not be doing the training.

I was going to use /lfs4/NAGAPE/hpc-wof1/ywang/MET2021/METplus-4.0.0_Tutorial/fv3met/read_FV3.py for testing, however, I see the file no longer exists at this location.

Would you be able to test the new functionality by starting with a clean environment, grabbing the new script, and sourcing it?

[ywangwof]

Thanks. It works now, though there is a glitch. It seems that Jet front nodes does not have enough memory to allocate the working buffer and the error message is,

ERROR :
ERROR : Python3_Numpy::set(PyObject *)-> failed to get buffer
ERROR :

If I set export MET_PYTHON_EXE=/mnt/lfs4/HFIP/dtc-hurr/METplus/miniconda/miniconda3/bin/python, then it will go through the temporary file path and works normally. My test script is this one /lfs4/NAGAPE/hpc-wof1/ywang/MET2021/fv3met/plt_lcc_2d.sh if you are interest. Thanks.

[jprestop]

Thanks @ywangwof. I tested one case:

plot_data_plane PYTHON_NUMPY refdmax_0001_202105271700-001.ps 'name="./read_FV3.py /lfs4/NAGAPE/hpc-wof1/ywang/EPIC2/oumap/rundir/NSSL/20210527/202105271700/fv3prd_mem0001/phyf001:00:00.nc refdmax";'

and received the same result, although I am not sure there is anything on our end we can do about that. Although perhaps, the resulting message could indicate that users could try setting MET_PYTHON_EXE. I'm tagging @JohnHalleyGotway and @georgemccabe to see if they have any additional comments or thoughts.

[jprestop]

Hi @ywangwof. After talking with folks on our team, it sounds like setting MET_PYTHON_EXE is a reasonable approach to solving this problem with the Jet front nodes not having enough memory to allocate the working buffer. I am going to mark this discussion as being answered and "lock it" to prevent future posts. That's how we encourage users to create new discussions for new questions. Don't hesitate to start a new Discussion with any other questions.