[RFC] Intx Tensor Subclasses Quantization

[vayuda]

Objective:

Implement sub byte unsigned integer quantization baselines from 1-7 to enable users to experiment with low bit quantization in pytorch.

Tracker:

• Create a UIntx Tensor Subclass per [RFC] torchao Contributor Guide #391
• Integrate with existing quant API + AQT
• Profile performance with llama2 and 3, noting metrics mentioned in The next tutorials #426
• Add support for int_x as well
• Integrate with existing uint dtypes
• Add fused kernel for unpack + dequant

Tasks

Beta Give feedback

1. Use torch.uint1 to torch.uint7 for Uintx tensor subclass #672
   CLA Signed +1

[jerryzh168]

This is great @vayuda, after Intx Tensor subclass matures we can also merge this into pytorch core, but we can keep this in torchao for a while to flesh out the extensibility stories (how to add a new op, layout, implementation branch to these Tensors) etc.

[HDCharles]

@vayuda can you link the results/PRs for some of these checked off bits?