MosaicQuant: Inlier-Outlier Disaggregation for Unified 4-Bit LLM Quantization
新方法MosaicQuant通过分离内点与外点实现统一4-bit大模型量化,提升精度与效率。
arXiv:2606.15652v1 Announce Type: new Abstract: 4-bit quantization significantly reduces the memory footprint and accelerates the inference of large l…