Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Confirm backward compatibility for PIO reads on non-parallel files #966

Merged
merged 2 commits into from
Aug 8, 2024

Conversation

anton-seaice
Copy link
Contributor

For detailed information about submitting Pull Requests (PRs) to the CICE-Consortium,
please refer to: https://github.com/CICE-Consortium/About-Us/wiki/Resource-Index#information-for-developers

PR checklist

  • Short (1 sentence) summary of your PR:
    This change ensures backwards compatibility of attempting a parallel read of netcdf file which doesn't support parallel reads
  • Developer(s):
    @anton-seaice
  • Suggest PR reviewers from list in the column to the right.
    @apcraig @DeniseWorthen
  • Please copy the PR test results link or provide a summary of testing completed below.
    I did adhoc testing with PIO 2.5.10 to check that this will read restart files of format hdf5 and cdf1. I couldn't replicate the initial problem however (maybe I need an earlier PIO version?).
  • How much do the PR code changes differ from the unmodified code?
    • bit for bit
    • different at roundoff level
    • more substantial
  • Does this PR create or have dependencies on Icepack or any other models?
    • Yes
    • No
  • Does this PR update the Icepack submodule? If so, the Icepack submodule must point to a hash on Icepack's main branch.
    • Yes
    • No
  • Does this PR add any new test cases?
    • Yes
    • No
  • Is the documentation being updated? ("Documentation" includes information on the wiki or in the .rst files from doc/source/, which are used to create the online technical docs at https://readthedocs.org/projects/cice-consortium-cice/. A test build of the technical docs will be performed as part of the PR testing.)
    • Yes
    • No, does the documentation need to be updated at a later time?
      • Yes
      • No
  • Please document the changes in detail, including why the changes are made. This will become part of the PR commit log.

For some versions of the PIO library (<2.5.10), attempting a parallel read with filetype netcdf4 (hdf5) of a file which does not support parallel reads fails. The expected behaviour (which is fixed in latest PIO versions) is to fall back to a serial read. The change ensures support for older PIO versions.

@anton-seaice
Copy link
Contributor Author

Closes #932

@anton-seaice
Copy link
Contributor Author

I did get this to work in the end.

I built linked to parallelio 2.5.10, then made a restart in format cdf5, changed the runtype to 'continue' and format to 'hdf5', it worked with this log message:

(ice_pio_init) opening ./restart/iced.2005-01-02-07200.nc as type hdf5 failed, retrying as type cdf1

and then the run continued.

I haven't run any test suites, if you have suggestions on what to run @apcraig, then I can try.

@apcraig
Copy link
Contributor

apcraig commented Aug 8, 2024

You can run the io_suite which exercises most of the IO options. But these changes seem fine and if I understand, you were able to find a case where you could trigger the error. Maybe I'll run a quick io_suite on derecho as well just to make sure nothing shows up there. But I think we should be able to merge this week.

@apcraig apcraig self-requested a review August 8, 2024 16:32
@apcraig apcraig added the IO label Aug 8, 2024
@apcraig
Copy link
Contributor

apcraig commented Aug 8, 2024

I ran the quick_suite and io_suite on derecho with intel, cray, gnu, and nvhpc and didn't encounter any problems.

@apcraig apcraig merged commit 64b6aa3 into CICE-Consortium:main Aug 8, 2024
2 checks passed
@apcraig apcraig mentioned this pull request Aug 8, 2024
@anton-seaice anton-seaice deleted the pio_read branch August 8, 2024 23:56
NickSzapiro-NOAA pushed a commit to NOAA-EMC/CICE that referenced this pull request Sep 25, 2024
…ICE-Consortium#966)


Add ability for netcdf file reading to fall back to regular netcdf if a file is initially opened (for instance with hdf5 reading) and fails to open properly.

For some versions of the PIO library (<2.5.10), attempting a parallel read with filetype netcdf4 (hdf5) of a file which does not support parallel reads fails. The expected behaviour (which is fixed in latest PIO versions) is to fall back to a serial read. The change ensures support for older PIO versions.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants