Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

random floating point errors in ice_fct.F90 in subroutine ice_fem_fct #529

Open
suvarchal opened this issue Nov 1, 2023 · 2 comments
Open

Comments

@suvarchal
Copy link
Collaborator

floating point error occurs around:

icepplus(n)=min(1.0_WP,tmax(n)/flux)

presumably a OMP racing condition? that leads to flux=0.0 its initialized value.

@suvarchal
Copy link
Collaborator Author

my fear is either locks or ordered are enough for ensuring non-racing condition here:

fesom2/src/ice_fct.F90

Lines 682 to 692 in 231c416

if (flux>0) then
#if !defined(DISABLE_OPENACC_ATOMICS)
!$ACC ATOMIC UPDATE
#endif
icepplus(n)=icepplus(n)+flux
else
#if !defined(DISABLE_OPENACC_ATOMICS)
!$ACC ATOMIC UPDATE
#endif
icepminus(n)=icepminus(n)+flux
end if
we may have to use !$OMP ATOMIC UPDATE there?

@trackow
Copy link
Contributor

trackow commented Nov 3, 2023

Hey Suvi, just wanted to add that we now see this issue also on Levante (Rohit, not sure which branch of FESOM), and on Atos ECMWF (with various branches, e.g. the old fesom2.5-fix-cycle3 and ifs-support-bundle by Sebastian).

This makes me think this could have to do with some updates to Levante or Atos themselves, because we did not see those in the past with those branches and could run 5 years without issues like this on Levante (May) and Atos ECMWF around summer. Or could we have been this lucky?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants