You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Congrats for the great work!
I find another unmatch between code and paper. In picture of the paper, different scales of qkv will be sent to LiteMLA first, and then concatenated. But in code, I find they are concatenated first, and then sent to LiteMLA.
I wonder the reason for this design and how the results would be different if I sent them to LiteMLA first and then concatenated.
The text was updated successfully, but these errors were encountered:
Congrats for the great work!
I find another unmatch between code and paper. In picture of the paper, different scales of qkv will be sent to LiteMLA first, and then concatenated. But in code, I find they are concatenated first, and then sent to LiteMLA.
I wonder the reason for this design and how the results would be different if I sent them to LiteMLA first and then concatenated.
The text was updated successfully, but these errors were encountered: