Enable dynamic M grouped gemm #3444
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
This pull request was exported from Phabricator. Differential Revision: D66682886 |
✅ Deploy Preview for pytorch-fbgemm-docs ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
jwfromm
force-pushed
the
export-D66682886
branch
2 times, most recently
from
e6ea79c to
1bbc6bf
Compare
|
This pull request was exported from Phabricator. Differential Revision: D66682886 |
1 similar comment
|
This pull request was exported from Phabricator. Differential Revision: D66682886 |
jwfromm
added a commit
to jwfromm/FBGEMM
that referenced
this pull request
Dec 15, 2024
Summary: X-link: facebookresearch/FBGEMM#530 This diff adds support for true dynamic M as is found in grouped_gemm. To do so, we add a new `M_values` argument that must be provided by the user and indicates the number of non-zero M in each tensor. One nice thing about this approach is that we can now do a single kernel call to set up the gemm arguments. We make `M_values` optional as it requires fixed N and K. When N and K vary across group, we use the previous static shape approach. Reviewed By: jiawenliu64 Differential Revision: D66682886
jwfromm
force-pushed
the
export-D66682886
branch
from
1bbc6bf to
7b2dfc0
Compare
|
This pull request was exported from Phabricator. Differential Revision: D66682886 |
Summary: X-link: facebookresearch/FBGEMM#530 This diff adds support for true dynamic M as is found in grouped_gemm. To do so, we add a new `M_values` argument that must be provided by the user and indicates the number of non-zero M in each tensor. One nice thing about this approach is that we can now do a single kernel call to set up the gemm arguments. We make `M_values` optional as it requires fixed N and K. When N and K vary across group, we use the previous static shape approach. Reviewed By: jiawenliu64 Differential Revision: D66682886
jwfromm
force-pushed
the
export-D66682886
branch
from
7b2dfc0 to
a59e5f0
Compare
|
This pull request was exported from Phabricator. Differential Revision: D66682886 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
This diff adds support for true dynamic M as is found in grouped_gemm. To do so, we add a new
M_valuesargument that must be provided by the user and indicates the number of non-zero M in each tensor. One nice thing about this approach is that we can now do a single kernel call to set up the gemm arguments.We make
M_valuesoptional as it requires fixed N and K. When N and K vary across group, we use the previous static shape approach.Differential Revision: D66682886