The weather model is failing with mpp_domains_define.inc: At least one pe in pelist is not used by any tile in the mosaic error message

[MichaelLueken]

While updating the Short-Range Weather Application to stop using the deprecated atmos_nthreads and use ATM_omp_num_threads, about half of the comprehensive workflow end-to-end (WE2E) tests are now failing with the following error message:

FATAL from PE *: mpp_domains_define.inc: At least one pe in pelist is not used by any tile in the mosaic

This is confusing because:

• There were no issues while using atmos_nthreads.
• There appears to be different behaviors depending on the machine that the updated code is run on.
• The ufs.configure, model_configure, and input.nml files are as expected.

Since there appears to be different behavior depending on the machine (on Hercules, all of the six fundamental tests pass, but one of the tests fails on Hera with the above message), does this suggest that there's an off-by-one or similar edge case based on node size?

Any clarification on what this error message represents and the best way to begin debugging would be greatly appreciated.

[MichaelLueken]

The issue has been discovered and corrected. Closing now.