Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

QB4W Fully Connected Operator #6673

Merged
merged 16 commits into from
Jul 26, 2024
Merged

QB4W Fully Connected Operator #6673

merged 16 commits into from
Jul 26, 2024

Commits on Jul 25, 2024

  1. Use bf16 block scales for qb4w gemm

    @GregoryComer
    GregoryComer committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    1557f7a View commit details
    Browse the repository at this point in the history

  2. Use bf16 block scales for sse and avx kernels

    @GregoryComer
    GregoryComer committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    5e7c61e View commit details
    Browse the repository at this point in the history

  3. Use bf16 block scales for qb4w mlal and neondot

    @GregoryComer
    GregoryComer committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    b06826c View commit details
    Browse the repository at this point in the history

  4. Use bf16 scales for i8mm qb4w

    @GregoryComer
    GregoryComer committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    e623f8f View commit details
    Browse the repository at this point in the history

  5. Update qb4w benchmarks for bf16 block scales

    @GregoryComer
    GregoryComer committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    942f648 View commit details
    Browse the repository at this point in the history

  6. Update amalgam files for qb4w bf16 kernel scales

    @GregoryComer
    GregoryComer committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    f65f1cc View commit details
    Browse the repository at this point in the history

  7. Update packing tests for bf16

    @GregoryComer
    GregoryComer committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    25d9126 View commit details
    Browse the repository at this point in the history

  8. Update qb4w avx512 vnni kernels for bf16 block scales

    @GregoryComer
    GregoryComer committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    e484d19 View commit details
    Browse the repository at this point in the history

  9. [QB4W] Move 1/16 to blockwise weight scales 

    before:
    fp_acc = 1/16 * (vksum * 16 + float(int_acc * 16) * scale)

after:
    fp_acc = vksum + float(int_acc * 16) * scale / 16
    @digantdesai @GregoryComer
    digantdesai authored and GregoryComer committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    718ba5d View commit details
    Browse the repository at this point in the history

  10. Update qb4w ukernels for 1/16 scale in packing

    @GregoryComer
    GregoryComer committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    6566480 View commit details
    Browse the repository at this point in the history

  11. Operator support for qd8-f32-qb4w

    @GregoryComer
    GregoryComer committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    8dda4be View commit details
    Browse the repository at this point in the history

  12. Operator support for qd8-f16-qb4w

    @GregoryComer
    GregoryComer committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    dbac1ea View commit details
    Browse the repository at this point in the history

  13. Improve operator-level qb4w test coverage

    @GregoryComer
    GregoryComer committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    d3d40e1 View commit details
    Browse the repository at this point in the history

  14. Unify operator creation for blockwise and non-blockwise qb4w fully co… 

    …nnected ops
    @GregoryComer
    GregoryComer committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    5132216 View commit details
    Browse the repository at this point in the history

  15. Update qb4w operator for bf16 block scales

    @GregoryComer
    GregoryComer committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    74b4735 View commit details
    Browse the repository at this point in the history

  16. Add qb4w gemm config entries

    @GregoryComer
    GregoryComer committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    f29493d View commit details
    Browse the repository at this point in the history