AMGCL on AMD GPU #1
Comments
|
Hello Pierre, Yes, you can use amgcl with AMD cards using OpenCL via vexcl backend. There is an example (of using MPI/vexcl) here: https://amgcl.readthedocs.io/en/latest/tutorial/SerenaMPI.html#id2 |
|
Thanks for your answer. |
|
It should be possible. See more details on partitioning and MPI here: https://amgcl.readthedocs.io/en/latest/tutorial/poisson3DbMPI.html. There is an example of using PCG here: https://amgcl.readthedocs.io/en/latest/tutorial/NullspaceMPI.html |
|
Thank you. Can you explain me what the make_shared function does and where the sources are? Thanks |
|
Sorry, i would like to know for the distributed_matrix function. Thanks |
|
Hello, I could not compile AMGCL with VexCl, I have the impression that it does not find VexCL. Also, I don't know how to install VexCL. Could you help me with this installation. Then I would like to run the tutorials, especially the Poisson problem in parallel with VexCL on AMD graphics cards, for that I can't find the matrix poisson3Db.bin, could you tell me where I can find it? |
|
Try the following in a separate folder: After this, try to reconfigure amgcl. It should find vexcl now. |
|
Thanks, it did find vexcl when running cmake. Also, I can't find the fish3Db.bin file to test the fish3Db_mpi_vexcl_cl executable from the Fish Problem tutorial in mpi, could you tell me where I can find it? |
|
You can convert the mtx file to bin file using examples/mm2bin utility, search for 'mm2bin' on this page: https://amgcl.readthedocs.io/en/latest/tutorial/Serena.html?highlight=mm2bin#structural-problem. There is a link to download the Poisson3Db matrix here: https://amgcl.readthedocs.io/en/latest/tutorial/poisson3Db.html |
|
Hi, thanks but I have a problem with the AMGCL build: I can't link the scotch installation to AMGCL. When I run the command to build the Makefile :cmake -DCMAKE_INSTALL_PREFIX=/lus/home/pelyakime/AMGCL/scotch-v7.0.1/install -DCMAKE_BUILD_TYPE=Release .. I have the impression that it does not find the scotch library. |
|
Thank you very much for your help, I don't have any problem with libraries anymore and I can test the executables of the poisson3Db tutorial : poisson3Db and poisson3Db_mpi works without problem. But with poisson3Db_mpi_vexcl I have a segmentation fault problem. I'm on an AMD architecture with AMD graphics cards, and I hope to use OpenCL to compute on AMD GPUs (I compute on the Adastra machine in France). |
|
Looks like you don't have any GPUs in the context. What does vexcl/examples/devlist output on your system? |
|
Hello, |
|
I don't know if this has anything to do with it but when building vexcl, it doesn't find OPENCL_HPP And when I build Tests and examples there are one error : |
|
Looks like you do have some AMD GPUs. Try to replace these lines with std::cout << world.rank << ": " << ctx << std::endl; |
|
After remplace these lines, I have a new error And my output file is : |
|
So only one of your MPI processes got a GPU (do you only have one?). The context is created in Exclusive mode here: you can replace it with vex::Context ctx(vex::Filter::Count(1));but then each of your MPI processes will use the same GPU, which would not be effective (but should work). In general, it is better to start a single MPI process per GPU. |
|
Ok, thanks a lot for your explications, it works now. Indeed I had reserved only 1 gpu for the test in my slurm script. I can reserve others of course. For the simulation that I want to launch later, if everything works well, I think of using several MPI processes per GPU. Indeed, my code being hybrid MPI - GPU, only the resolution of the pseudo Poisson equation representing the resolution of the pressure in the Navier-Stokes equations is solved on the GPUs, this allows to reduce the MPI part of the code, while keeping the efficiency of the GPUs. But now, I have to test with the matrix that my code builds and the 2nd member. Thanks for your help. I will keep you informed of my progress. Have a nice day |
|
Hello, Also, my second question, could I easily use my partitioning without having to recode the MPI distribution of the matrix? Thank you very much for your help which is very precious to me. Here is the function where I integrate the resolution of AX=b in my code |
|
This is my error output |
|
Try to read this page: https://amgcl.readthedocs.io/en/latest/tutorial/poisson3DbMPI.html There I tried to explain what amgcl expects from the partitioned matrix. In short, each MPI process should contain consecutive row-wise chanks of the matrix, and the columns should have global numbering. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hello,
I would like to know if you have already used your AMGCL library on AMD graphics cards? It would be for the resolution of a pseudo Poisson equation in a fluid mechanics code (finite volume) with curvilinear structured meshes with a resolution on CPU in parallel (MPI) and on GPU (for the Poisson solver). Thank you very much for your answer, Pierre
The text was updated successfully, but these errors were encountered: