parallel_for equivalent in Ginkgo

[Timo-Schrader]

Hello,

we have the following (mathematical) problem:

We want to setup a RBF system matrix in order to solve for the coefficients. We furthermore want to accelerate this using CUDA (Gingko) by not only solving the linear system of equations, but also setting up the kernel matrix on the GPU.

Now our question is if there is a way in Ginkgo (similar to parallel_for in Kokkos) to fill such a matrix on the GPU, i.e., evaluate each RBF entry on the GPU side without having to write custom CUDA kernels ourselves and pass an array pointer to it. It might be the case that we just didn't find this feature yet, so please excuse me if this question is already answered.

Thank you and have a great day!

Best regards

[upsj]

We have such a framework in place already, it's only not yet exposed to the user since we are not certain it is final yet. For an example, see #938, or more specifically stencil_kernel.cpp. I'd be happy to pick this PR up again if it's of importance to you.

[davidscn]

Got it, thanks a lot for your quick replies!

[Timo-Schrader]

Hello again,

so I tried to implement a basic workflow in both Kokkos and Ginkgo and it seems to me that Ginkgo is much easier to use as a math library than Kokkos. I was also able to launch a self-written CUDA kernel with the help of Ginkgo using the experimental feature branch.

In summary, I really like this feature and would be delighted if it is possible to pick up this PR again (as suggested).
Thank you very much for your efforts!

[upsj]

This is great to hear - we will discuss how to move this forward again. If you have a minute, could I get some input on #1209 from the user side? The duplication of arguments/parameters is currently necessary because we need to transform host objects to device representations, but I was wondering how much of an issue potential parameter order/name mismatches actually are.

[Timo-Schrader]

After getting used to it/understanding how it works, I think the way it currently works is absolutely feasible for the user as long as it is well documented since "all the magic" happens in run_kernel.
However, looking at #1209 , I am not 100% sure what row_vector(x) and default_stride refer to.

Thank you very much for your efforts, much appreciated!

[Timo-Schrader]

Good afternoon,

we just wanted to report that we are getting first very promising results using Ginkgo + CUDA in preCICE when we need to solve a very large RBF system for data mapping. So if you need some further input from our side to get this really great feature of kernel dispatches into the main branch of Ginkgo just let us know!