Failed on generating ICS and BCS for predefined grid in 3km #752
-
Dear all experts, I tried to generate ICs and BCs for a predefined grid (GSL_HAFSV0.A_3km). But the log files for ICs and BCs show that they didn't finish. ======================================================================== But no such info was seen for Make_ICS_3km and Make_LBCS_3km. I have attached log files for Make_ICS_13km, Make_LBCS_13km, Make_ICS_3km, and Make_LBCS_3km. I have successfully finished the 13km simulation. I am wondering what files I should check to fix this. Is this something related to processors number since the NX and NY increased for different grids? Many thanks in advance, Make_ICS_13km.log |
Beta Was this translation helpful? Give feedback.
Replies: 5 comments 10 replies
-
@htan2013 Sorry you're having this issue. Is there a chance your test is running out of wallclock time? The lack of any errors indicates the process may have been cut off. Aside from that, it may have been a memory-related segfault that didn't log properly. Either way, increasing the number of processors would be a good step forward to try to solve this. As a side note, the GSL_HAFSV0 domains are untested, and there has been some discussion of removing them. I wouldn't recommend using them unless you need them for a specific reason. |
Beta Was this translation helpful? Give feedback.
-
Haochen, |
Beta Was this translation helpful? Give feedback.
-
Haochen, task_make_lbcs: |
Beta Was this translation helpful? Give feedback.
-
@htan2013, what hash of UFS_UTILS are you using? There was a known memory issue with how chgres_cube was handling regridding of the 3-D wind field (where your 3-km chgres_cube runs hang) for large domains at high resolutions. Please see ufs-community/UFS_UTILS#766 PR in UFS_UTILS and the associated issue. The current HEAD of ufs-srweather-app does not pull a new enough version of UFS_UTILS to contain this fix. Please try pulling the latest UFS_UTILS code and let me know if this solves your problem. @mkavulich, @natalie-perlin, we should update the hash of UFS_UTILS in the Externals.cfg file to the HEAD of that repo. |
Beta Was this translation helpful? Give feedback.
-
Haochen, The memory fix for the UFS_UTILS did solve the issue. Please do the following to update it in your code, considering your SRW= :
... and then proceed with your workflow tasks. [make_ics_2020030612.log](https://github.com/ufs-community/ufs-srweather-app/files/11434361/m |
Beta Was this translation helpful? Give feedback.
Haochen,
The memory fix for the UFS_UTILS did solve the issue. Please do the following to update it in your code, considering your SRW= :