add SCHISM submodule and build files (#44)

* add PaHM as submodule

* add SCHISM submodules

* Test ground for the inclusion of SCHISM into the NEMS application

* Added infrastructure for platiform macports

* update submodules

* update README.md

* update submodules (after pushing)

* update submodules after amending

* update submodules after merging / amending

* removed `schism-esmf` submodule

* update schism-esmf

* update modulefile for Orion

* updated SCHISM submodule pointer to master branch

* added cmake to some modulefiles

* reorder

* update `schism-esmf` location in `SCHISM`

* changes to modulefiles

* updated modules for Orion

* updated PaHM submodule to point to newest commit + changed name to PAHM

* updated PaHM sources to include the nuopc folder

* update modulefile to use stable parallel ESMF build

* modules are under emc-nemspara

* updated build scripts to fix compilation errors

* added nuopc/Makefile.in and modified distclean.cmake to not delete the top level Makefiles

* modified the work/cmprflags.mk file for intel

* updated the modulefiles/* files to be more general and mofified the file scripts/functions_build to include the additional supported platforms

* fixed the bug for use_defaults

* fixed conf/configure.nems* files to set the NETCDF libraries properly

* fixed orion compiling issues

* fixed MACHINE_ID variables

* switched the hera and orion environment modulefiles to theis sys state

* commended out the LICENSE line in the orion cmake file

* updated submodule

* update submodule

* fixed the missing nemsio issue

* deleted the old build scripts

* updated the modulefile for intel in orion

* updated the build and function scripts

* modified the build script to find the functions_build file; in some systems was failing to find it

* modified PAHM build scripts

* updates PaHM build scripts

* update modules

* updated submodules

* fixed the build script to allow the user to define customized names for the parallel compilers

* fixed functions_build to remove NEMS/exe/NEMS*.x if requested

* modified  component_ADCIRC.mk so distclean cleans properly the nuopc folder

* updated submodule

* Added environment file for femto/intel (#52)

* Added environment file for femto/intel

* Removed hardcoded CC/FC env variables form build script

Co-authored-by: Carsten Lemmen <[email protected]>

* updated ESMF to latest parallel build

* updated the envmodules_*.linux files to load the proper modules

* fixed the gnu configure file to include the correct compilers and compiler flags

* fixed make_nuopc.sh to set the compiler variable correctly

* minor fix of module_EARTH_GRID_COMP.F90 line 2956 to comile with gfortran

* fix module_EARTH_GRID_COMP.F90 to compile with gfortran

* fix component_ADCIRC.mk to pass the correct compiler option to make_nuopc.sh script

* updated submodules

* Support Mac gnu compiler options (#67)

* Added environment file for femto/intel

* Removed hardcoded CC/FC env variables form build script

* Removed fp-model strict from mac configure.nems.mac*.gnu files

Co-authored-by: Carsten Lemmen <[email protected]>

* updated ADCIRC to cherry-pick .gitignore organization

* make build script executable

* updated pointer for filename rename

* change name to `CoastalApp`

* updated strand configuration for CoastalApp

* Added configure.nems.femto.intel to repo

* Added configure.nems.strand.intel

* Set NETCDF variable for strand/intel

* update pointer to the latest SCHISM commit

* On `feature/schism` Added configure.nems.femto.intel to repo (#87)

* Added configure.nems.femto.intel to repo

* Added configure.nems.strand.intel

* Set NETCDF variable for strand/intel

Co-authored-by: Carsten Lemmen <[email protected]>
Co-authored-by: Zachary Burnett <[email protected]>

* added PaHM submodule

* updated pointers for PaHM and NEMS

* updated NEMS pointer

* updated pointers

* added submodule pointer to `schism-esmf` on the toplevel

* updated SCHISM-ESMF submodule

* updated SCHISM-ESMF submodule

* removed submodules

* changed SCHISM submodule to point to schism-esmf

* updated SCHISM submodule to fix URL

* Fixed netcdf c libs on mistral

* remove submodule

* moved SCHISM submodules into shared SCHISM folder in root directory

* Disable error checking on gnu/macosx

* Ignore build directory in SCHISM subdir

* Updated pointer to SCHISM/schism-esmf

* updated SCHISM and SCHISM-ESMF submodules to point to new multicore code

* Added draft envmodules files for levante (#104)

Co-authored-by: Carsten Lemmen <[email protected]>

Co-authored-by: Panagiotis Velissariou <[email protected]>
Co-authored-by: Carsten Lemmen <[email protected]>
Co-authored-by: Carsten Lemmen <[email protected]>
Co-authored-by: Carsten Lemmen <[email protected]>
Co-authored-by: zacharyburnett <[email protected]>

.gitmodules

@@ -29,3 +29,9 @@
 [submodule "BARDATA"]
 	path = BARDATA
 	url = https://github.com/noaa-ocs-modeling/BARDATA
+[submodule "SCHISM/schism"]
+	path = SCHISM/schism
+	url = https://github.com/schism-dev/schism.git
+[submodule "SCHISM/schism-esmf"]
+	path = SCHISM/schism-esmf
+	url = https://github.com/schism-dev/schism-esmf.git

README.md

@@ -4,15 +4,41 @@
 [![build](https://github.com/noaa-ocs-modeling/ADC-WW3-NWM-NEMS/workflows/build/badge.svg)](https://github.com/noaa-ocs-modeling/ADC-WW3-NWM-NEMS/actions?query=workflow%3Abuild)
 [![license](https://img.shields.io/github/license/noaa-ocs-modeling/ADC-WW3-NWM-NEMS)](https://creativecommons.org/share-your-work/public-domain/cc0)
 
-ESMF application for building a NUOPC / NEMS application coupling ADCIRC, ATMESH, WW3 / WW3DATA, and NWM.
+ESMF application for building a NUOPC / NEMS application coupling
+circulation models (ADCIRC, SCHISM) with forcing models (WW3, NWM) or
+even file forcings (ATMESH, WW3DATA).
 
-`CoastalApp` is an ESMF application developed as part of the Coastal Act coupling project to determine wind versus water percentage loss caused by a Named Storm Event. 
+`ADC-WW3-NWM-NEMS` is an ESMF application developed as part of the
+Coastal Act coupling project to determine wind versus water percentage
+loss caused by a Named Storm Event.
 
 ```bash
 git clone --recursive https://github.com/noaa-ocs-modeling/CoastalApp
 cd CoastalApp
 ```
 
+## Requirements
+
+#### installing ParMETIS for WW3
+
+Using unstructured WW3 requires an installation of ParMETIS for domain
+decomposition.
+
+1. [download the code here](http://glaros.dtc.umn.edu/gkhome/metis/parmetis/download)
+2. build ParMETIS
+    ```bash
+    module purge
+    module load intel impi
+    setenv CFLAGS -fPIC
+    make config cc=mpiicc cxx=mpiicc prefix=/path/to/your/parmetis/ | & tee config.out-rr
+    make install | & tee make-install.out-rr
+    ```
+   This adds `libparmetis.a`
+   under `/path/to/your/parmetis/lib/libparmetis.a`.
+3. set the path to ParMETIS
+    ```bash
+    setenv METIS_PATH /path/to/your/parmetis
+    ```
 ## Organization / Responsibility
 
 #### `NEMS` application implementing ESMF / NUOPC coupling
@@ -50,15 +76,39 @@ cd CoastalApp
 
 ## Compilation
 
+This section contains some generic instructions of how to build the NEMS
+application. The contents of this section will change soon.
+
+```bash
+./build.sh --help
+```
+
+The following example builds `NEMS.x` with ADCIRC, ATMESH, and WW3DATA,
+using variables / modules from the Hera environment, compiling with the
+Intel compiler, and cleaning before building.
+
 ```bash
 ./build.sh --component "ADCIRC ATMESH WW3DATA" --plat hera --compiler intel --clean -2 
 ```
 
+### Components
+
 - `--component` can be any combination of
     - `ADCIRC`
     - `ATMESH`
     - `WW3` / `WW3DATA`
     - `NWM`
+
+```bash
+./build.sh --component "ADCIRC WW3" --plat hera --compiler intel --clean -2 
+```
+
+```bash
+./build.sh --component "SCHISM" --plat orion --compiler intel --clean -2
+```
+
+### Platforms
+
 - `--plat` can be one of
     - `hera`
     - `stampede`
@@ -69,43 +119,62 @@ cd CoastalApp
     - `cheyenne`
     - `linux`
     - `macosx`
+    - `macports`
+
+For MacOS running MacPorts, use the `macports` option.
+
+```bash
+./build.sh --component "SCHISM" --plat hera --compiler intel --clean -2
+```
+
+```bash
+./build.sh --component "SCHISM" --plat orion --compiler intel --clean -2
+```
+
+### Compiler
+
 - `--compiler` can be one of
     - `intel`
     - `gnu`
     - `pgi`
+
+```bash
+./build.sh --component "SCHISM" --compiler gnu --plat macports
+```
+
+```bash
+./build.sh --component "SCHISM" --compiler intel --plat hera
+```
+
+### Clean
+
 - `--clean` is optional, and can be one of
     - ` ` (`make clean` and exit)
     - `1` (`make clean` and exit)
     - `2` (`make clobber` and exit)
     - `-1` (`make clean` and build)
     - `-2` (`make clobber` and build)
 
-#### adding a new platform / compiler to compilation script
+```bash
+./build.sh --component "SCHISM" --compiler intel --plat hera --clean
+```
 
-Environment files are stored in `modulefiles/` with the filename `envmodules_<COMPILER>.<PLATFORM>`
+```bash
+./build.sh --component "SCHISM" --compiler intel --plat hera --clean -1
+```
 
-To compile in your own system you should create a similar file, then run `build.sh` to compile.
+## Contributing
 
-## Requirements
+Feel free to fork this repository and create a pull request with
+contributions.
 
-#### installing ParMETIS for WW3
+#### adding a new platform / compiler to compilation script
 
-Using unstructured WW3 requires an installation of ParMETIS for domain decomposition.
+Environment files are stored in `modulefiles/` with the
+filename `envmodules_<COMPILER>.<PLATFORM>`
 
-1. [download the code here](http://glaros.dtc.umn.edu/gkhome/metis/parmetis/download)
-2. build ParMETIS
-    ```bash
-    module purge
-    module load intel impi
-    setenv CFLAGS -fPIC
-    make config cc=mpiicc cxx=mpiicc prefix=/path/to/your/parmetis/ | & tee config.out-rr
-    make install | & tee make-install.out-rr
-    ```
-   This adds `libparmetis.a` under `/path/to/your/parmetis/lib/libparmetis.a`.
-3. set the path to ParMETIS
-    ```bash
-    setenv METIS_PATH /path/to/your/parmetis
-    ```
+To compile in your own system you should create a similar file, then
+run `build.sh` to compile.
 
 ## Collaboration
 

SCHISM/.gitignore

@@ -0,0 +1 @@
+build

conf/configure.nems.femto.intel

@@ -0,0 +1,56 @@
+## NEMS configuration file
+##
+## Platform: Hera
+## Compiler: Intel with IntelMPI
+
+SHELL           = /bin/sh
+
+################################################################################
+## Include the common configuration parts
+include         $(TOP)/conf/configure.nems.NUOPC
+
+################################################################################
+## Other settings
+
+NETCDF_INC   = -I$(NETCDF)/include
+NETCDF_LIB   = -L$(NETCDF)/lib -lnetcdf
+
+NEMSIO_INC   = -I${LIBDIR}/incmod/nemsio
+NEMSIO_LIB   = -L${LIBDIR} -lnemsio
+SYS_LIB      =
+
+EXTLIBS      = $(NEMSIO_LIB) \
+               $(NETCDF_LIB) \
+               $(ESMF_LIB)   \
+               $(SYS_LIB) -lm
+
+EXTLIBS_POST = $(NEMSIO_LIB)  \
+               $(ESMF_LIB)    \
+               $(NETCDF_LIB)  \
+               $(SYS_LIB)
+###
+FC          = mpifort -g -qopenmp -mkl=sequential -align array32byte -lmkl_intel_lp64 -lmkl_core -lmkl_sequential -lpthread -qopenmp -convert big_endian -assume byterecl -mkl=sequential
+F77         = mpifort -g -qopenmp -mkl=sequential -align array32byte -lmkl_intel_lp64 -lmkl_core -lmkl_sequential -lpthread -qopenmp -convert big_endian -assume byterecl -mkl=sequential
+FREE         = -free
+FIXED        = -fixed
+R8           = -r8
+
+FINCS        = $(ESMF_INC) $(NEMSIO_INC) $(NETCDF_INC)
+#TRAPS        = -g -fno-inline -no-ip -traceback -ftrapuv -fpe0 -ftz -check all -check noarg_temp_created -fp-stack-check
+
+FFLAGS       = $(TRAPS) $(FINCS) -fp-model strict
+
+OPTS_NMM     = -g -fno-inline -no-ip -traceback -ftrapuv -fpe0 -ftz -check all -check noarg_temp_created -fp-stack-check $(FREE)
+
+FFLAGM_DEBUG =
+
+FFLAGS_NMM   = $(MACROS_NWM) $(OPTS_NMM) $(FFLAGS)
+
+FPP          = -fpp
+CPP          = cpp -P -traditional
+CPPFLAGS     = -DENABLE_SMP -DCHNK_RRTM=8 
+
+AR           = ar
+ARFLAGS      = -r
+
+RM           = rm

conf/configure.nems.macosx.gnu

@@ -0,0 +1,60 @@
+## NEMS configuration file
+##
+## Platform: Generic/Linux
+## Compiler: GNU with MPI   --- needs fixing
+
+SHELL           = /bin/sh
+
+################################################################################
+## Include the common configuration parts
+include         $(TOP)/conf/configure.nems.NUOPC
+
+################################################################################
+## Other settings
+#
+CC=mpicc
+CXX=mpicxx
+FC=mpifort
+
+NETCDF_INC   = -I$(NETCDF_INCDIR)
+NETCDF_LIB   = -L$(NETCDF_LIBDIR) -lnetcdf
+
+NEMSIO_INC   = -I${LIBDIR}/incmod/nemsio
+NEMSIO_LIB   = -L${LIBDIR} -lnemsio
+SYS_LIB      =
+
+EXTLIBS      = $(NEMSIO_LIB) \
+               $(NETCDF_LIB) \
+               $(ESMF_LIB)   \
+               $(SYS_LIB) -lm
+
+EXTLIBS_POST = $(NEMSIO_LIB)  \
+               $(ESMF_LIB)    \
+               $(NETCDF_LIB)  \
+               $(SYS_LIB)
+###
+FC          = mpifort -g -ffree-line-length-none -fno-range-check -fbacktrace
+F77         = mpifort -g -ffree-line-length-none -fno-range-check -fbacktrace
+FREE         = -free
+FIXED        = -fixed
+R8           = -r8
+
+FINCS        = $(ESMF_INC) $(NEMSIO_INC) $(NETCDF_INC)
+#TRAPS        = ???
+
+FFLAGS       = $(TRAPS) $(FINCS) 
+
+OPTS_NMM     = -g -ffree-line-length-none -fno-range-check -fbacktrace $(FREE)
+
+FFLAGM_DEBUG =
+
+FFLAGS_NMM   = $(MACROS_NWM) $(OPTS_NMM) $(FFLAGS)
+
+FPP          = -fpp
+CPP          = cpp -P -traditional
+CPPFLAGS     = -DENABLE_SMP -DCHNK_RRTM=8 
+
+AR           = ar
+ARFLAGS      = -r
+
+RM           = rm

conf/configure.nems.macports.gnu

@@ -0,0 +1,60 @@
+## NEMS configuration file
+##
+## Platform: Generic/Linux
+## Compiler: GNU with MPI   --- needs fixing
+
+SHELL           = /bin/sh
+
+################################################################################
+## Include the common configuration parts
+include         $(TOP)/conf/configure.nems.NUOPC
+
+################################################################################
+## Other settings
+#
+CC=mpicc
+CXX=mpicxx
+FC=mpifort
+
+NETCDF_INC   = -I$(NETCDF_INCDIR)
+NETCDF_LIB   = -L$(NETCDF_LIBDIR) -lnetcdf
+
+NEMSIO_INC   = -I${LIBDIR}/incmod/nemsio
+NEMSIO_LIB   = -L${LIBDIR} -lnemsio
+SYS_LIB      =
+
+EXTLIBS      = $(NEMSIO_LIB) \
+               $(NETCDF_LIB) \
+               $(ESMF_LIB)   \
+               $(SYS_LIB) -lm
+
+EXTLIBS_POST = $(NEMSIO_LIB)  \
+               $(ESMF_LIB)    \
+               $(NETCDF_LIB)  \
+               $(SYS_LIB)
+###
+FC          = mpifort -g -ffree-line-length-none -fno-range-check -fbacktrace -fallow-argument-mismatch
+F77         = mpifort -g -ffree-line-length-none -fno-range-check -fbacktrace
+FREE         = -free
+FIXED        = -fixed
+R8           = -r8
+
+FINCS        = $(ESMF_INC) $(NEMSIO_INC) $(NETCDF_INC)
+#TRAPS        = ???
+
+FFLAGS       = $(TRAPS) $(FINCS) 
+
+OPTS_NMM     = -g -ffree-line-length-none -fno-range-check -fbacktrace $(FREE)
+
+FFLAGM_DEBUG =
+
+FFLAGS_NMM   = $(MACROS_NWM) $(OPTS_NMM) $(FFLAGS)
+
+FPP          = -fpp
+CPP          = cpp -P -traditional
+CPPFLAGS     = -DENABLE_SMP -DCHNK_RRTM=8 
+
+AR           = ar
+ARFLAGS      = -r
+
+RM           = rm