Quantization Is a Trade-Off, Not Free Compression

Quantization stores model weights with fewer bits than a higher-precision representation. In many situations it can reduce memory use and make local deployment more practical. It can also change output quality, stability, or task performance. The effect varies by model, quantization method, runtime, and task.…