batch bicgstab with batch csr structure problem on gpu

[AbdelhadiKara]

Hello, I am trying to solve linear systems with batch:: bicgstab and batch::csr for storage.
On cpu serial and openMp everything works. When trying to execute on gpu, it only works for a small size of matrix (until ~1200 ).
Otherwise, the iterative solver does not execute any iteration and the implicit residual=0. Since small size problems work well, there is no issue of data allocation on gpu or something else.

[AbdelhadiKara]

Replacing batch::bicgstab solver by batch::Cg fixes the execution problem.

[AbdelhadiKara]

Hello,
first thanks for this quick answer.
The hardware is NVIDIA-V100.
I attached the folder containing the matrix and its rhs. I exported it using gko::write since i had not been able to use write_raw.
gko_mat_and_rhs.zip

[pratikvn]

Thanks. I will have a look. Just to confirm, are you using the latest version of Ginkgo ?

[AbdelhadiKara]

It is a clone from develop just before 1.8.0 release. If i remember well the 1.7 version was not providing csr for batched solvers.

[pratikvn]

Can you maybe try 1.8.0 and confirm that you still face this issue ?

[AbdelhadiKara]

With the environnement updated to release 1.8.0, the issue is still present.

[pratikvn]

It seems the issue indeed might be due to a limitation of V100. It seems to be work fine on a A100.

This is Ginkgo 1.9.0 (develop)
    running with core module 1.9.0 (develop)
    the reference module is  1.9.0 (develop)
    the OpenMP    module is  not compiled
    the CUDA      module is  1.9.0 (develop)
    the HIP       module is  not compiled
    the DPCPP     module is  not compiled
Residual norm sqrt(r^T r):
 Exec: cuda
 System no. 0: residual norm = 4.17165e-11, implicit residual norm = 4.17166e-11, iterations = 122
Solver type: batch::bicgstab
Matrix size: (4099, 4099)
Num batch entries: 1
Entire solve took: 0.0568908 seconds.

Additionally, for these relatively large matrices, I think batched methods might not give any advantages, and will leave resources unused (as a run on only on thread block, so only 1024 threads will be used). It might be more beneficial to use the non-batched Bicgstab for these matrices.

But in case batched CG works, that should be the better performing method, even if it might take more iterations. You can also do a performance comparison for this case and see if gko::batch::solver::Cg is faster than gko::solver::Cg. I think the non-batched method might perform well for these large matrices. In case you have batched matrices, I think combining them into a block-diagonal and solving with a regular non-batched CG could be an option, especially as you are on a V100 and I think probably already exceed the shared memory capacity.

[AbdelhadiKara]

Thank you for these answers and useful precisions.

[blegouix]

@pratikvn Thanks for the precisions, do you know if using a batched version of a solver with a batch size=1 could impact the performance compared to using the non-batched version of the class ? Relying only on batched solvers even if batching is not required would simplify the implementations.

[pratikvn]

Yes, I think the performance will be different, though I cannot say for sure which will be better. With the non-batched solver, multiple kernels are launched (per iteration), but the resource utilization should be better. For the batched solver, only a single kernel is launched, so the kernel launch overhead should be very small, but there could possibly be resources left un-utilized.

In case you are using a stream approach (adding an asynchronous stream to the executor), batched methods might be the better option ( for single batch item solution as well), as the other concurrent streams can possibly use the rest of the unused resources.

But I would recommend that you do a quick benchmark for your case and compare the batched (with one batch item) with the non-batched solver and see if you observe any benefits.