1-bit quantization?

[PABannier]

A team from Microsoft recently came up with 1-bit quantization drastically reducing memory footprint and token throughput. The weights are effectively encoded as a ternary bit {-1, 0, 1}. Reported results are super encouraging.

@ggerganov is it of interest for GGML? I can implement it and perform a bunch of benchmarks.

[Green-Sky]

ggerganov/llama.cpp#5761 more on this topic :)