Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pre-compute mappings? #70

Open
aekiss opened this issue Sep 21, 2023 · 7 comments
Open

Pre-compute mappings? #70

aekiss opened this issue Sep 21, 2023 · 7 comments
Labels
cmip7 mediator Related to the CMEPS mediator priority:low

Comments

@aekiss
Copy link
Contributor

aekiss commented Sep 21, 2023

For performance we might want to pre-compute mappings rather than auto-generating them at runtime (see ACCESS-NRI/access-om3-configs#5) - but only if auto-generating consumes a lot of walltime.

Unclear whether this is currently possible with CDEPS; might require code changes.

@anton-seaice
Copy link
Contributor

I think we should investigate this before we start doing long runs / cmip7 runs. If they are only getting calculated once after a restart and then stored for the rest of the run that might be ok, but if they are recalculated for every interpolation / regrid then we should definately pre-compute them.

@anton-seaice anton-seaice added the mediator Related to the CMEPS mediator label Feb 12, 2024
@micaeljtoliveira
Copy link
Contributor

Another important detail is the parallelisation of the mappings computation. Is it serial code? Parallel? If parallel, what is the scalability?

@aekiss
Copy link
Contributor Author

aekiss commented Feb 22, 2024

Online remapping weight generation is parallel, according to slide 5 of @mvertens CW2023 talk.

@mvertens
Copy link

@micaeljtoliveira - that is correct. Its totally parallel.

@mvertens
Copy link

@micaeljtoliveira - all IO in CMEPS is parallel as well.

@aekiss
Copy link
Contributor Author

aekiss commented Apr 9, 2024

@micaeljtoliveira do your scaling tools allow checking how well the weight generation parallelises?

@micaeljtoliveira
Copy link
Contributor

do your scaling tools allow checking how well the weight generation parallelises?

Yes, we might just need to add some profiling regions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cmip7 mediator Related to the CMEPS mediator priority:low
Projects
Status: Todo
Development

No branches or pull requests

5 participants