Skip to content

Version 2.0 Functional Release
Compare
Choose a tag to compare
@gdicker1 gdicker1 released this 08 Nov 23:42
· 100 commits to main since this release
release-ew2.0
7d81efb

Version 2 of the EarthWorks Modeling System extends support from Version 1 {30,60,120 km}_L32 to 15km_L58 (58-level) configurations, and introduces support for the Intel OneAPI (ifx) and NVIDIA (nvfortran) compilers. Also, for the first time, support for GPU offload in the case of PUMAS model physics running with the F2000Climo compset at 30km_L32 configuration, and is introduced as a technical demonstration.

All of these configurations should be considered functional releases, i.e. that they have not been scientifically verified.

Description of Model Configurations (Compsets)

See EarthWorks Supported Configurations in the GitHub wiki.

Testing

Tested Systems

Unless otherwise indicated, all tests were performed on NCAR’s Derecho supercomputer.

CPU-only tests used Derecho’s CPU-only nodes consisting of dual, 64-core, 3rd Gen AMD EPYC™ 7763 Milan processors with 256 GB of DDR4 memory.

CPU/GPU hybrid tests used Derecho’s GPU nodes consisting of a single 3rd Gen AMD EPYC™ 7763 Milan processors with 512 GB of DDR4 memory and 4 NVIDIA A100 GPUs.

Tested Software Stacks

Compiler Versions

  • ifort (Intel Classic compiler version 2021.8)
  • ifx (Intel OneAPI version 2023.0)
  • Nvfortran (NVHPC fortran compiler version 23.5)
  • Gnufortran (compiler version 12.2)

Libraries

  • ESMF (version 8.6)
  • PIO2 (version 2.6.2)

Testing Results

Known cross cutting NVHPC compilation issues

  • NVHPC compilers (tested nvfortran version 23.5): Compilation fails

    • Additional details: NVHPC compilers won’t build any of the V2 release compsets by default. Line 65 of hcoio_read_std_mod.F90 breaks Fortran free format line length convention.
    • Work around: Once the offending line is split into multiple, builds succeed. This has been fixed upstream and will be fixed for EarthWorks in a future release. See CAM Issue #871 for more information.

  • NVHPC compilers (tested nvfortran version 23.5): Initializing from restart fails.

    • Additional details: Any configuration (tested with QPC6, F2000Climo, Fully-Coupled) that attempts to restart from a previous run will fail in CAM subroutine dyn_init.
    • Work around: Run without restart.

Known issues by compset/compiler/resolution (CPU-only):

  • Fully-Coupled simulations:

    • Supported resolution/level combinations:

      • 120km_L32
      • 60km_L32
      • 30km_L32
      • 15km_L58

    • NVHPC compiler (nvfortran 23.5)

      • See Known cross cutting issues section above.

    • Gnu compiler (gfortran version 12.2)

      • No known issues.

    • Intel OneAPI compiler (version 2023.0): Compilation failed with internal compiler error.

      • Resolutions affected: all supported resolution/level combinations
      • Additional details: Build log files contain error #5633: **Internal compiler error: segmentation violation signal raised** Please report this error along with the circumstances in which it occurred in a Software Problem Report. Note: File and line given may not be the explicit cause of this error. compilation aborted for mpas_seaice_core_interface.F
      • Work around: use ifort (Intel classic) compiler version 2021.8 instead.

  • QPC6 (Aquaplanet):

    • Supported resolution/level combinations:

      • 120km_L32
      • 60km_L32
      • 30km_L32
      • 15km_L58

    • NVHPC compiler (nvfortran 23.5)

      • See Known cross cutting issues section above.

    • Gnu compiler (gfortran version 12.2)

      • No known issues.

    • Intel OneAPI compiler (version 2023.0):

      • No known issues.

  • F2000Climo (coupled atmosphere, land surface):

    • Supported resolution/level combinations:

      • 120km_L32
      • 60km_L32
      • 30km_L32
      • 15km_L58

    • NVHPC compiler (nvfortran 23.5)

      • See Known cross cutting issues section above.

    • Gnu compiler (gfortran version 12.2)

      • No known issues.

    • Intel OneAPI compiler (version 2023.0):

      • No known issues.

  • FHS94 (Held-Suarez) and FKESSLER (Kessler microphysics) test cases:

    • Supported resolution/level combinations:

      • 120km_L32
      • 60km_L32
      • 30km_L32
      • 15km_L32

    • NVIDIA nvhpc compiler (version 23.5):

      • No known issues.

    • Gnu compiler (gfortran version 12.2)

      • No known issues.

    • Intel OneAPI compiler (version 2023.0):

      • No known issues.

Known issues by compset/compiler/resolution (Hybrid CPU-GPU)

  • F2000Climo-GPU-PUMAS-only:

    • Supported resolution/level combinations:

      • 120km_L32
      • 60km_L32

    • NVIDIA nvhpc compiler (nvfortran version 23.5):

      • See Known cross cutting NVHPC compilation issues above.
Assets 2
Loading