Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable PIO with mpiuni #205

Merged
merged 16 commits into from
Jun 27, 2024
Merged

Enable PIO with mpiuni #205

merged 16 commits into from
Jun 27, 2024

Commits on Oct 18, 2023

  1. Configuration menu
    Copy the full SHA
    50af0ea View commit details
    Browse the repository at this point in the history

Commits on Oct 19, 2023

  1. Changes to mpiuni needed to build PIO

    With these changes, I can successfully build PIO. I tested a standalone
    PIO build (i.e., outside of an ESMF build) with:
    
    CC=clang FC=gfortran cmake -DPIO_USE_MPISERIAL=ON -DPIO_ENABLE_TIMING=OFF -DPIO_ENABLE_EXAMPLES=OFF -DPIO_ENABLE_FORTRAN=OFF -DMPISERIAL_PATH=/Users/sacks/esmf/esmf1/src/Infrastructure/stubs/mpiuni -DPIO_ENABLE_TESTS=OFF .
    
    make VERBOSE=1
    billsacks committed Oct 19, 2023
    Configuration menu
    Copy the full SHA
    5f7bea9 View commit details
    Browse the repository at this point in the history

Commits on Oct 20, 2023

  1. Configuration menu
    Copy the full SHA
    2242fce View commit details
    Browse the repository at this point in the history
  2. mpiuni: implement check for MPI_IN_PLACE

    According to MPI documentation, most MPI routines check sendbuf, but
    some (MPI_Scatter, MPI_Scatterv and maybe others; note that these are
    not yet implemented yet in mpiuni) check recvbuf, and some (e.g.,
    MPI_Sendrecv_replace) don't check for MPI_IN_PLACE at all. To make
    mpiuni respect the MPI standard in this respect, I have added an
    argument to MPIUNI_Memcpy saying whether to check the source (sendbuf),
    dest (recvbuf) or neither for equality with MPI_IN_PLACE.
    
    (We could probably get away with keeping things simpler by always
    checking both a and b for equality with MPI_IN_PLACE, but following the
    MPI standard in this respect seems marginally safer.)
    billsacks committed Oct 20, 2023
    Configuration menu
    Copy the full SHA
    e33eade View commit details
    Browse the repository at this point in the history

Commits on Oct 25, 2023

  1. Configuration menu
    Copy the full SHA
    9acec7f View commit details
    Browse the repository at this point in the history

Commits on Nov 21, 2023

  1. Define MPI_GROUP_EMPTY

    This is needed for the PIO build. I'm using the same definition as is
    used in mpi-serial.
    billsacks committed Nov 21, 2023
    Configuration menu
    Copy the full SHA
    c145d23 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    19e588f View commit details
    Browse the repository at this point in the history

Commits on Nov 22, 2023

  1. Fix ESMPy logic for determining if PIO is enabled

    This no longer depends on whether we're using mpiuni
    
    I thought that we might need to add some logic to handle the case where
    the user explicitly sets ESMF_PIO=OFF: I thought that would appear in
    the esmf.mk file. But it turns out that ESMF_PIO doesn't appear in the
    esmf.mk file in that case, so the logic here seems correct.
    billsacks committed Nov 22, 2023
    Configuration menu
    Copy the full SHA
    5d570d2 View commit details
    Browse the repository at this point in the history

Commits on Dec 11, 2023

  1. Implement mpiuni functions needed by PIO

    Implement MPI_Alltoallw, MPI_Type_create_indexed_block and MPI_Type_size.
    
    These are needed for a srcfield.read call in ESMPy to work. With these
    changes, the following python program appears to work correctly:
    
    import os
    import esmpy
    
    from esmpy.util.cache_data import DATA_DIR
    from esmpy.util.exceptions import DataMissing
    
    datafile = os.path.join(DATA_DIR, "so_Omon_GISS-E2.nc")
    meshfile = os.path.join(DATA_DIR, "mpas_uniform_10242_dual_counterclockwise.nc")
    
    grid = esmpy.Grid(filename=os.path.join(DATA_DIR, datafile),
    				  filetype=esmpy.FileFormat.GRIDSPEC)
    srcfield = esmpy.Field(grid, staggerloc=esmpy.StaggerLoc.CENTER, ndbounds=[33, 2])
    
    srcfield.read(filename=os.path.join(DATA_DIR, datafile),
    	      variable="so", timeslice=2)
    
    print("Worked!")
    billsacks committed Dec 11, 2023
    Configuration menu
    Copy the full SHA
    4a69073 View commit details
    Browse the repository at this point in the history

Commits on Dec 28, 2023

  1. Add mpiuni implementations needed for an ESMPy example

    - Implement MPI_Scatterv; this is needed for PIO's subset rearranger,
    which is used to read a Mesh (and is the only rearranger allowed by
    PIOc_InitDecomp_ReadOnly)
    
    - Generalize the implementation of MPI_Type_create_indexed_block to
    allow count > 1
    
    With these changes, the example in
    https://earthsystemmodeling.org/esmpy_doc/nightly/develop/html/examples.html#grid-mesh-and-field-created-from-file
    runs to completion
    billsacks committed Dec 28, 2023
    Configuration menu
    Copy the full SHA
    3b290f6 View commit details
    Browse the repository at this point in the history
  2. Remove a limitation in mpiuni's MPI_Type_create_indexed_block

    I had a check that the displacements go in increments of blocklength,
    but from some experimentation with checking the MPI_Type_size of the new
    type returned from this function from openmpi, it seems that the
    resulting type's size is independent of the displacements. So I don't
    think we need this check.
    billsacks committed Dec 28, 2023
    Configuration menu
    Copy the full SHA
    59a996d View commit details
    Browse the repository at this point in the history
  3. Add mpiuni implementation of MPI_Type_create_hvector

    and MPI_Type_hvector
    
    These are needed in PIO. I am implementing both MPI_Type_create_hvector
    and MPI_Type_hvector because PIO potentially uses the latter when it
    thinks it's using mpi-serial.
    billsacks committed Dec 28, 2023
    Configuration menu
    Copy the full SHA
    ebded92 View commit details
    Browse the repository at this point in the history

Commits on Jan 4, 2024

  1. In mpiuni's mpirun only prepend './' if needed

    mpiuni's mpirun was always prepending './' to the command, even for a
    command in your path. This prevented running python3 via mpirun (needed
    in the ESMPy testing), for example.
    
    This change only prepends './' if the given command isn't in your PATH.
    This seems to be the behavior for openmpi's mpirun.
    billsacks committed Jan 4, 2024
    Configuration menu
    Copy the full SHA
    f8cde05 View commit details
    Browse the repository at this point in the history

Commits on Jan 5, 2024

  1. Configuration menu
    Copy the full SHA
    a37f165 View commit details
    Browse the repository at this point in the history

Commits on Jun 25, 2024

  1. Configuration menu
    Copy the full SHA
    cc470b7 View commit details
    Browse the repository at this point in the history

Commits on Jun 27, 2024

  1. Configuration menu
    Copy the full SHA
    4478342 View commit details
    Browse the repository at this point in the history