Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use a proper all-reduce algorithm for ObjGroup #2240

Open
lifflander opened this issue Jan 17, 2024 · 0 comments · May be fixed by #2337
Open

Use a proper all-reduce algorithm for ObjGroup #2240

lifflander opened this issue Jan 17, 2024 · 0 comments · May be fixed by #2337
Assignees
@JacobDomagala
Labels
type: task
Milestone
Add new allreduce

Comments

@lifflander
Copy link
Collaborator

What Needs to be Done?

Right now, proxy.allreduce is a reduce followed by a broadcast. We can make this more efficient by combining them into a singular allreduce algorithm.
JacobDomagala added a commit that referenced this issue Mar 24, 2024
@JacobDomagala
#2240: Initial work for new allreduce
937ef48
JacobDomagala added a commit that referenced this issue Apr 4, 2024
@JacobDomagala
#2240: Initial work for new allreduce
4788932
JacobDomagala added a commit that referenced this issue Apr 4, 2024
@JacobDomagala
#2240: Semi working Rabenseifner
e3e2b5c
JacobDomagala added a commit that referenced this issue Apr 4, 2024
@JacobDomagala
#2240: Working Rabenseifner (non-commutative ops)
637a1c1
JacobDomagala added a commit that referenced this issue Apr 7, 2024
@JacobDomagala
#2240: Fix non power of 2 for new allreduce
8860acb
JacobDomagala added a commit that referenced this issue Apr 10, 2024
@JacobDomagala
#2240: Initial work for new allreduce
186fb44
JacobDomagala added a commit that referenced this issue Apr 10, 2024
@JacobDomagala
#2240: Semi working Rabenseifner
5167b0d
JacobDomagala added a commit that referenced this issue Apr 10, 2024
@JacobDomagala
#2240: Working Rabenseifner (non-commutative ops)
96a0eda
JacobDomagala added a commit that referenced this issue Apr 10, 2024
@JacobDomagala
#2240: Fix non power of 2 for new allreduce
193f6ed
JacobDomagala added a commit that referenced this issue Apr 10, 2024
@JacobDomagala
#2240: Initial work for adding recursive doubling allreduce algorithm
ce017d9
JacobDomagala added a commit that referenced this issue Apr 11, 2024
@JacobDomagala
#2240: Initial work for new allreduce
74513ce
JacobDomagala added a commit that referenced this issue Apr 11, 2024
@JacobDomagala
#2240: Semi working Rabenseifner
29514af
JacobDomagala added a commit that referenced this issue Apr 11, 2024
@JacobDomagala
#2240: Working Rabenseifner (non-commutative ops)
0072997
JacobDomagala added a commit that referenced this issue Apr 11, 2024
@JacobDomagala
#2240: Fix non power of 2 for new allreduce
a6d694e
JacobDomagala added a commit that referenced this issue Apr 11, 2024
@JacobDomagala
#2240: Initial work for adding recursive doubling allreduce algorithm
7534af9
JacobDomagala added a commit that referenced this issue Apr 11, 2024
@JacobDomagala
#2240: Make sure the order of reduce operations is correct
a3d10f6
JacobDomagala added a commit that referenced this issue Apr 15, 2024
@JacobDomagala
#2240: Initial work for new allreduce
ad79ef5
JacobDomagala added a commit that referenced this issue Apr 15, 2024
@JacobDomagala
#2240: Semi working Rabenseifner
a526f19
JacobDomagala added a commit that referenced this issue Apr 15, 2024
@JacobDomagala
#2240: Working Rabenseifner (non-commutative ops)
db6d5b1
JacobDomagala added a commit that referenced this issue Apr 15, 2024
@JacobDomagala
#2240: Fix non power of 2 for new allreduce
c647432
JacobDomagala added a commit that referenced this issue Apr 15, 2024
@JacobDomagala
#2240: Initial work for adding recursive doubling allreduce algorithm
e522593
JacobDomagala added a commit that referenced this issue Apr 15, 2024
@JacobDomagala
#2240: Make sure the order of reduce operations is correct
8306937
JacobDomagala added a commit that referenced this issue Apr 15, 2024
@JacobDomagala
#2240: Working Recursive doubling
3f8c7e8
JacobDomagala added a commit that referenced this issue Apr 15, 2024
@JacobDomagala
#2240: Working Recursive doubling
8414d43
@JacobDomagala JacobDomagala mentioned this issue Apr 15, 2024
Closed
@JacobDomagala JacobDomagala linked a pull request Apr 15, 2024 that will close this issue
#2240: Add Rabenseifner and Recursive doubling allreduce algorithms for ObjGroup #2272
Closed
JacobDomagala added a commit that referenced this issue Apr 16, 2024
@JacobDomagala
#2240: Code cleanup and make Rabenseifner work with any Op type
97a076a
JacobDomagala added a commit that referenced this issue Apr 16, 2024
@JacobDomagala
#2240: Code cleanup and make Rabenseifner work with any Op type
17cfcbd
@JacobDomagala JacobDomagala linked a pull request Sep 27, 2024 that will close this issue
Draft
JacobDomagala added a commit that referenced this issue Oct 11, 2024
@JacobDomagala
#2240: Initial work for new allreduce
e7e1a2e
JacobDomagala added a commit that referenced this issue Oct 11, 2024
@JacobDomagala
#2240: Semi working Rabenseifner
f90e5c7
JacobDomagala added a commit that referenced this issue Oct 11, 2024
@JacobDomagala
#2240: Working Rabenseifner (non-commutative ops)
0715c52
JacobDomagala added a commit that referenced this issue Oct 11, 2024
@JacobDomagala
#2240: Fix non power of 2 for new allreduce
0070e71
JacobDomagala added a commit that referenced this issue Oct 11, 2024
@JacobDomagala
#2240: Initial work for adding recursive doubling allreduce algorithm
a2ad6e6
JacobDomagala added a commit that referenced this issue Oct 11, 2024
@JacobDomagala
#2240: Make sure the order of reduce operations is correct
19df2f7
JacobDomagala added a commit that referenced this issue Oct 11, 2024
@JacobDomagala
#2240: Working Recursive doubling
831168f
JacobDomagala added a commit that referenced this issue Oct 11, 2024
@JacobDomagala
#2240: Code cleanup and make Rabenseifner work with any Op type
594fce5
JacobDomagala added a commit that referenced this issue Oct 11, 2024
@JacobDomagala
#2240: Improve accuracy of timing allreduce algorithms in allreduce.cc
b5095b6
JacobDomagala added a commit that referenced this issue Oct 11, 2024
@JacobDomagala
#2240: Add unit tests for new allreduce and cleanup code
42cf608
JacobDomagala added a commit that referenced this issue Oct 11, 2024
@JacobDomagala
#2240: DataHandler for Rabenseifner allreduce that provides common AP…
0ee7848 
…I for various data types
JacobDomagala added a commit that referenced this issue Oct 11, 2024
@JacobDomagala
#2240: Fix warnings
d70050e
JacobDomagala added a commit that referenced this issue Oct 11, 2024
@JacobDomagala
#2240: Update ObjGroup test to use custom DataHandler for Rabenseifne…
1f86e55 
…r allreduce test
JacobDomagala added a commit that referenced this issue Oct 11, 2024
@JacobDomagala
#2240: Add unit test for Rabenseifner with Kokkos::View as DataType a…
699a6d8 
…nd fix compile issues realted to using Kokkos::View for allreduce
JacobDomagala added a commit that referenced this issue Oct 11, 2024
@JacobDomagala
#2240: Move function definitions to impl.h file for Rabenseifner
5535449
JacobDomagala added a commit that referenced this issue Oct 11, 2024
@JacobDomagala
#2240: Add allreduce print category and use it in rabenseifner instea…
18d5090 
…d of fmt::print
JacobDomagala added a commit that referenced this issue Oct 11, 2024
@JacobDomagala
#2240: Provide documentation for RecursiveDoubling algorithm
9d88016
JacobDomagala added a commit that referenced this issue Oct 11, 2024
@JacobDomagala
#2240: Use vt_debug_print for RecursiveDoubling allreduce
5d79bd0
JacobDomagala added a commit that referenced this issue Oct 11, 2024
@JacobDomagala
#2240: Update allreduce perf tests to use array of payload sizes
74bea76
JacobDomagala added a commit that referenced this issue Oct 11, 2024
@JacobDomagala
#2240: Fix runtime failure in allreduce perf test
51ea74e
JacobDomagala added a commit that referenced this issue Oct 11, 2024
@JacobDomagala
#2240: Working allreduce perf test with Kokkos
7aeaa8e
JacobDomagala added a commit that referenced this issue Oct 11, 2024
@JacobDomagala
#2240: Working RecursiveDoubling with multiple allreduce in flight
593132f
JacobDomagala added a commit that referenced this issue Oct 11, 2024
@JacobDomagala
#2240: Update Rabenseifner to use ID for each allreduce and update tests
da46b94
JacobDomagala added a commit that referenced this issue Oct 11, 2024
@JacobDomagala
#2240: Fix failing unit and performance tests for multiple allreduce …
641c9dd 
…in flight
JacobDomagala added a commit that referenced this issue Oct 11, 2024
@JacobDomagala
#2240: Fix compile issues on some compilers and runtime issue with pa…
1a0879e 
…ssing proxy by reference
JacobDomagala added a commit that referenced this issue Oct 11, 2024
@JacobDomagala
#2240: Update logs
3b9d697
JacobDomagala added a commit that referenced this issue Oct 11, 2024
@JacobDomagala
#2240: Fix issues with handlers being executed and payload not being …
27438ea 
…initialized
JacobDomagala added a commit that referenced this issue Oct 11, 2024
@JacobDomagala
#2240: Add helpers and use Kokkos::View for internals of Rabenseifner…
6ba48d3 
… when user's payload is View
JacobDomagala added a commit that referenced this issue Oct 11, 2024
@JacobDomagala
#2240: Store Reducers by tuple(ProxyType, DataType, OperandType)
150b883
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@JacobDomagala JacobDomagala

Labels
type: task
Projects
None yet
Milestone
Add new allreduce
2 participants
@lifflander @JacobDomagala