chat: Add Q8_0 quantization type

[vsrinivas]

Describe your changes

Add support for Q8_0 quantization type.

Issue ticket number and link

Checklist before requesting a review

• I have performed a self-review of my code.
• If it is a core feature, I have added thorough tests.
• I have added thorough documentation for my code.
• I have tagged PR with relevant project labels. I acknowledge that a PR without labels may be dismissed.
• If this PR addresses a bug, I have provided both a screenshot/video of the original bug and the working solution.

Demo

Steps to Reproduce

Notes

[vsrinivas]

Ping!

[ThiloteE]

Have you tested all the supported backends with a supported model, such as llama-3-8b-instruct? Providing screenshots of a model that works with your PR would be good.

By the way, why only Q8 and not other quants, such as Q6_K, which is very close to Q8 in terms of perplexity, but much smaller in terms of GB?

Could you outline your rationale for this change too? What do you think will be the impact of this change on the userbase and on maintaining GPT4All?

[manyoso]

This can't go in until all of our backends support this quant type. It would also need to be signed. But again, this really needs much more since the current vulkan backend does not support this quant.