Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add CUDA/HIP RCM kernels #1503

Merged
merged 29 commits into from
Jan 19, 2024
Merged

Add CUDA/HIP RCM kernels #1503

merged 29 commits into from
Jan 19, 2024

Conversation

upsj
Copy link
Member

@upsj upsj commented Dec 9, 2023 •
edited
Loading

This adds Thrust-based kernels for RCM.

TODO:

  • Tests
  • Fix OpenMP and Reference handling of connected components (or tests?)

@upsj upsj added the 1:ST:ready-for-review This PR is ready for review label Dec 9, 2023
@upsj upsj self-assigned this Dec 9, 2023
@upsj upsj added 1:ST:need-feedback The PR is somewhat ready but feedback on a blocking topic is required before a proper review. and removed 1:ST:ready-for-review This PR is ready for review labels Dec 9, 2023
@ginkgo-bot ginkgo-bot added reg:testing This is related to testing. reg:benchmarking This is related to benchmarking. type:solver This is related to the solvers type:preconditioner This is related to the preconditioners type:matrix-format This is related to the Matrix formats type:factorization This is related to the Factorizations type:reordering This is related to the matrix(LinOp) reordering reg:helper-scripts This issue/PR is related to the helper scripts mainly concerned with development of Ginkgo. type:multigrid This is related to multigrid type:stopping-criteria This is related to the stopping criteria mod:all This touches all Ginkgo modules. labels Dec 9, 2023
@codecov Codecov
Copy link

codecov bot commented Dec 10, 2023 •
edited
Loading

Codecov Report

Attention: 11 lines in your changes are missing coverage. Please review.

Comparison is base (fb70ec0) 91.04% compared to head (69b4f78) 91.05%.
Report is 29 commits behind head on develop.

❗ Current head 69b4f78 differs from pull request most recent head 9f65902. Consider uploading reports for the commit 9f65902 to get more accurate results

Files Patch % Lines
test/reorder/rcm.cpp 95.65% 9 Missing ⚠️
core/device_hooks/common_kernels.inc.cpp 0.00% 1 Missing ⚠️
core/reorder/rcm.cpp 88.88% 1 Missing ⚠️
Additional details and impacted files 
@@            Coverage Diff            @@
##           develop    #1503    +/-   ##
=========================================
  Coverage    91.04%   91.05%            
=========================================
  Files          700      697     -3     
  Lines        56996    56773   -223     
=========================================
- Hits         51894    51692   -202     
+ Misses        5102     5081    -21

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@MarcelKoch
Copy link
Member

Could you rebase this? There seem to be a lot of unrelated changes.

@upsj
Copy link
Member Author

upsj commented Dec 11, 2023

@MarcelKoch not until #1485 is resolved

@upsj upsj force-pushed the array_access branch 2 times, most recently from 83f67c5 to f47439a Compare December 12, 2023 15:09
Base automatically changed from array_access to develop December 12, 2023 23:05
@upsj upsj marked this pull request as ready for review December 14, 2023 23:38
@upsj upsj requested a review from a team December 18, 2023 15:06
@upsj upsj added 1:ST:ready-for-review This PR is ready for review and removed 1:ST:need-feedback The PR is somewhat ready but feedback on a blocking topic is required before a proper review. labels Dec 18, 2023
MarcelKoch
MarcelKoch requested changes Dec 20, 2023
View reviewed changes
Copy link
Member

@MarcelKoch MarcelKoch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly LGTM. I left some minor comments.

One larger thing is that the cuda/hip implementations ignore the construct_inverse_permutation factory parameter. Either that should be deprecated, or added to cuda/hip.

common/cuda_hip/reorder/rcm_kernels.hpp.inc Outdated Show resolved Hide resolved
common/cuda_hip/reorder/rcm_kernels.hpp.inc Show resolved Hide resolved
common/cuda_hip/reorder/rcm_kernels.hpp.inc Show resolved Hide resolved
hip/components/memory.hip.hpp Show resolved Hide resolved
test/reorder/rcm.cpp Outdated Show resolved Hide resolved
test/reorder/rcm.cpp Outdated Show resolved Hide resolved
test/reorder/rcm.cpp Outdated Show resolved Hide resolved
test/reorder/rcm.cpp Outdated Show resolved Hide resolved
upsj and others added 27 commits January 18, 2024 21:27
@upsj
use CUDA/HIP RCM reordering
d7bd22f
@upsj
fix unsigned abs warnings
3345649
@upsj
add RCM GPU kernel
1b151f5
@upsj
tests
8e4ee0f
@upsj
add load_*_local loads for workgroup-coherent global memory atomics
1e150cf
@upsj
fix connected component search
100d220
@upsj
fix level initialization
d541d19
@upsj
fix membar for local atomics
eeb18e9
@upsj
fix min degree level initialization
69b298d
@upsj
test multiple connected components
4479266
@upsj
fix ubfs
02b71cd
@upsj
fix test
b258af7
@upsj
work around rocThrust bug
42833cf 
ROCm/rocThrust#352
@upsj
fix handling of multiple connected components in OpenMP
3e9420d
@upsj
compute and test inverse permutation
0de9baa
@upsj @MarcelKoch
review updates
8d7c35e 
Co-authored-by: Marcel Koch <[email protected]>
@upsj
remove old reordering test
c9b2f10
@upsj
reenable dpcpp test
ed75c86
@upsj
formatting
faf720e
@upsj
clearer connected component detection
9086dab
@upsj
fix minimum degree choice in OMP starting node
211aecf
@upsj
rename kernel get_permutation -> compute_permutation
78a1f5c
@upsj @yhmtsai
review updates
147c280 
Co-authored-by: Yuhsiang M. Tsai <[email protected]>
@upsj
use stable sort in RCM
8a9a7df
@upsj
add isolated vertices to test
3f62741
@upsj
add failing test case
7e0f19a
@upsj
fix test
9f65902 
We should check that one possible choice for the next contender
has a lower or equal height, not all of them.
@upsj upsj merged commit 7c50303 into develop Jan 19, 2024
13 of 15 checks passed
@upsj upsj deleted the gpu_rcm branch January 19, 2024 08:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@yhmtsai yhmtsai yhmtsai approved these changes

@MarcelKoch MarcelKoch MarcelKoch approved these changes

@pratikvn pratikvn pratikvn approved these changes

Assignees

@upsj upsj

Labels
1:ST:ready-to-merge This PR is ready to merge. mod:all This touches all Ginkgo modules. reg:benchmarking This is related to benchmarking. reg:helper-scripts This issue/PR is related to the helper scripts mainly concerned with development of Ginkgo. reg:testing This is related to testing. type:factorization This is related to the Factorizations type:matrix-format This is related to the Matrix formats type:multigrid This is related to multigrid type:preconditioner This is related to the preconditioners type:reordering This is related to the matrix(LinOp) reordering type:solver This is related to the solvers type:stopping-criteria This is related to the stopping criteria
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@upsj @MarcelKoch @pratikvn @yhmtsai @ginkgo-bot