Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Half precision support #1257

Open
wants to merge 62 commits into
base: develop
Choose a base branch
from
Open

Half precision support #1257

wants to merge 62 commits into from

Commits on Oct 22, 2024

  1. only can compile cuda/omp

    @yhmtsai
    yhmtsai committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    4ed36a8 View commit details
    Browse the repository at this point in the history

  2. next_precision to itself when complex only float, double add empty co… 

    …nditional
    @yhmtsai
    yhmtsai committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    ddc7c16 View commit details
    Browse the repository at this point in the history

  3. can compile with cuda/omp/ref (without test)

    @yhmtsai
    yhmtsai committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    501e6c7 View commit details
    Browse the repository at this point in the history

  4. compile for cuda/sycl/test/mpi (hip needs trick)

    @yhmtsai
    yhmtsai committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    957d29c View commit details
    Browse the repository at this point in the history

  5. hip finally

    @yhmtsai
    yhmtsai committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    a0c389c View commit details
    Browse the repository at this point in the history

  6. fix the narrow issue and atomic support

    @yhmtsai
    yhmtsai committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    fe5e491 View commit details
    Browse the repository at this point in the history

  7. fixed more error

    @yhmtsai
    yhmtsai committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    6c3c12b View commit details
    Browse the repository at this point in the history

  8. fix the op order and gdb 

    Co-authored-by: Marcel Koch <[email protected]>
    @yhmtsai @MarcelKoch
    yhmtsai and MarcelKoch committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    a0ee872 View commit details
    Browse the repository at this point in the history

  9. add the rand template not_implemented

    @yhmtsai
    yhmtsai committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    5acbf27 View commit details
    Browse the repository at this point in the history

  10. this version can compile/run complex<half> on cuda114 

    stack size can not be determined statically issue
some tests are failed
    @yhmtsai
    yhmtsai committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    b171312 View commit details
    Browse the repository at this point in the history

  11. does not work for the other executor

    @yhmtsai
    yhmtsai committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    6c17701 View commit details
    Browse the repository at this point in the history

  12. fix complex issue and sqrt issue 

    sqrt need to be global namespace to live with hip sqrt
    @yhmtsai
    yhmtsai committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    cdbf0a0 View commit details
    Browse the repository at this point in the history

  13. try fix the compilation issue from MSVC and MacOS 

    It seems to use complex version even using half only
    @yhmtsai
    yhmtsai committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    209c799 View commit details
    Browse the repository at this point in the history

  14. move the half to public and use sycl::half for dpcpp

    @yhmtsai
    yhmtsai committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    75b54fa View commit details
    Browse the repository at this point in the history

  15. limit the next precision in test and benchmark 

    next_precision<double> = float not half
    @yhmtsai
    yhmtsai committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    48ea338 View commit details
    Browse the repository at this point in the history

  16. allow disable half operation

    @yhmtsai
    yhmtsai committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    fdcc066 View commit details
    Browse the repository at this point in the history

  17. fix macro

    @yhmtsai
    yhmtsai committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    f0a8a07 View commit details
    Browse the repository at this point in the history

  18. clean and refine the code

    @yhmtsai
    yhmtsai committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    f041b4a View commit details
    Browse the repository at this point in the history

  19. move half.hpp out of type.hpp

    @yhmtsai
    yhmtsai committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    3154a04 View commit details
    Browse the repository at this point in the history

  20. enable half for testing

    @yhmtsai
    yhmtsai committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    58784ab View commit details
    Browse the repository at this point in the history

  21. __habs is added in cuda10.2 

    create_empty for its own type
    @yhmtsai
    yhmtsai committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    9b2465b View commit details
    Browse the repository at this point in the history

  22. fix nullptr and missing instantiation. 

    sycl::half has different rule in conv and full operator after 5.7
    @yhmtsai
    yhmtsai committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    e2a6c9a View commit details
    Browse the repository at this point in the history

  23. fix missing device_type and ptr_param

    @yhmtsai
    yhmtsai committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    51cf597 View commit details
    Browse the repository at this point in the history

  24. update rounding

    @yhmtsai
    yhmtsai committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    0a42796 View commit details
    Browse the repository at this point in the history

Commits on Oct 23, 2024

  1. do not use distribution with half

    @yhmtsai
    yhmtsai committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    e3b81df View commit details
    Browse the repository at this point in the history

  2. WIP fix half of failed test

    @yhmtsai
    yhmtsai committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    c9fd747 View commit details
    Browse the repository at this point in the history

  3. fix/skip half test and fix numeric_limit on device

    @yhmtsai
    yhmtsai committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    684cadb View commit details
    Browse the repository at this point in the history

  4. mkl csr does not support half

    @yhmtsai
    yhmtsai committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    60767ed View commit details
    Browse the repository at this point in the history

  5. add half to batch_vector

    @yhmtsai
    yhmtsai committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    d65255a View commit details
    Browse the repository at this point in the history

  6. fix hip thrust complex op, avoid const in nvhpc, reduce job in windows

    @yhmtsai
    yhmtsai committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    5c0454f View commit details
    Browse the repository at this point in the history

  7. fix nvc++ atomic, dpcpp half 

    jacobi need to use value_type generator
    @yhmtsai
    yhmtsai committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    da15916 View commit details
    Browse the repository at this point in the history

  8. make half test optional

    @yhmtsai
    yhmtsai committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    5f9e3ff View commit details
    Browse the repository at this point in the history

  9. nvhpc optimization/computation error workaround

    @yhmtsai
    yhmtsai committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    fe45560 View commit details
    Browse the repository at this point in the history

  10. some math func is not defined if nvhpc is for host

    @yhmtsai
    yhmtsai committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    c7f0d2a View commit details
    Browse the repository at this point in the history

  11. add half spmv benchmark (with cusparse for cuda)

    @yhmtsai
    yhmtsai committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    710e037 View commit details
    Browse the repository at this point in the history

  12. fixes batched support for half

    @MarcelKoch @yhmtsai
    MarcelKoch authored and yhmtsai committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    34845f3 View commit details
    Browse the repository at this point in the history

  13. generate PTX load/stores for half

    @MarcelKoch @yhmtsai
    MarcelKoch authored and yhmtsai committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    48afbb5 View commit details
    Browse the repository at this point in the history

  14. fix mc64 for half 

    Note: the issue is that numerical_limits<half>::infinite returns float instead of half. Maybe changing that would be a better solution
    @MarcelKoch @yhmtsai
    MarcelKoch authored and yhmtsai committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    a51f136 View commit details
    Browse the repository at this point in the history

  15. fix hip memory.hip.hpp for half

    @MarcelKoch @yhmtsai
    MarcelKoch authored and yhmtsai committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    60123dc View commit details
    Browse the repository at this point in the history

  16. WIP: can compile but three tests are still failed

    @yhmtsai
    yhmtsai committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    8f1e28f View commit details
    Browse the repository at this point in the history

  17. fix config, ambiguous namespace, and batch

    @yhmtsai
    yhmtsai committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    6dbd616 View commit details
    Browse the repository at this point in the history

  18. update format

    @yhmtsai
    yhmtsai committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    cd270e1 View commit details
    Browse the repository at this point in the history

  19. check the failed tests 

    - use Csr in residual norm for half apply support
- use higher tolerance for mc64 due to half range
- some example can not finish in half precision for mc64
- skip some test in half due to half range
- fix the half limit value

Co-authored-by: Marcel Koch <[email protected]>
    @yhmtsai @MarcelKoch
    yhmtsai and MarcelKoch committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    69d5b59 View commit details
    Browse the repository at this point in the history

  20. fix windows and icpx

    @yhmtsai
    yhmtsai committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    57fc170 View commit details
    Browse the repository at this point in the history

  21. hip does not support atomic on 16 bits

    @yhmtsai
    yhmtsai committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    18e825f View commit details
    Browse the repository at this point in the history

  22. fix batch

    @yhmtsai
    yhmtsai committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    825f76f View commit details
    Browse the repository at this point in the history

  23. add miss instantiation

    @yhmtsai
    yhmtsai committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    81d63ac View commit details
    Browse the repository at this point in the history

  24. update documentation, remove half.hpp 

    Co-authored-by: Marcel Koch <[email protected]>
Co-authored-by: Thomas Grützmacher <[email protected]>
    @yhmtsai @MarcelKoch
    3 people committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    2a6d382 View commit details
    Browse the repository at this point in the history

  25. put function in gko not std 

    Co-authored-by: Thomas Grützmacher <[email protected]>
    @yhmtsai
    yhmtsai and Thomas Grützmacher committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    8731fc3 View commit details
    Browse the repository at this point in the history

  26. fix after rebase

    @yhmtsai
    yhmtsai committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    64406f3 View commit details
    Browse the repository at this point in the history

  27. hip does not support 16bit shuffle

    @yhmtsai
    yhmtsai committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    baa95f7 View commit details
    Browse the repository at this point in the history

  28. merge two #if block

    @yhmtsai
    yhmtsai committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    4bb8093 View commit details
    Browse the repository at this point in the history

  29. do not use attributes in sqrt and abs

    @yhmtsai
    yhmtsai committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    c539398 View commit details
    Browse the repository at this point in the history

  30. make half constexpr

    @yhmtsai
    yhmtsai committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    d0e2446 View commit details
    Browse the repository at this point in the history

  31. isolate half out of device completely

    @yhmtsai
    yhmtsai committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    0d777df View commit details
    Browse the repository at this point in the history

  32. bits constexpr construct half and make numeric_limit in half

    @yhmtsai
    yhmtsai committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    56e2af8 View commit details
    Browse the repository at this point in the history

  33. refine the code and fix error without half

    @yhmtsai
    yhmtsai committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    6cc26d7 View commit details
    Browse the repository at this point in the history

  34. reduce abs/sqrt location

    @yhmtsai
    yhmtsai committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    3d15350 View commit details
    Browse the repository at this point in the history

  35. move the math function to math

    @yhmtsai
    yhmtsai committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    c4697a5 View commit details
    Browse the repository at this point in the history

  36. nohalf

    @yhmtsai
    yhmtsai committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    377432a View commit details
    Browse the repository at this point in the history

  37. cbgmres without half

    @yhmtsai
    yhmtsai committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    e806a0a View commit details
    Browse the repository at this point in the history

  38. direct without half

    @yhmtsai
    yhmtsai committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    3e49252 View commit details
    Browse the repository at this point in the history