突破量化 trade-off：统计无损新方法，兼顾模型保真与推理加速，大模型部署必读。
arXiv:2605.02404v2 Announce Type: replace Abstract: Model quantization has become essential for efficient large language model deployment, yet existin…
视觉Transformer部署提速新思路，用脆弱性引导混合精度量化，兼顾压缩率与精度的双赢方案。
arXiv:2607.28589v1 Announce Type: cross Abstract: Post-training quantization (PTQ) has emerged as an effective solution for deploying Vision Transform…
2.78万亿参数Kimi K3模型仅需29GB内存即可在笔记本上完整运行，挑战大模型本地部署极限。
Article URL: https://github.com/sqliteai/waste/ Comments URL: https://news.ycombinator.com/item?id=49106591 Points: 7 # Comments: 1
提出GPTQ-2D算法，实现三次时间复杂度的双向自适应舍入，有效提升大模型量化精度。
arXiv:2607.27042v1 Announce Type: cross Abstract: Adaptive rounding methods such as GPTQ, or equivalently Babai's nearest plane algorithm, round a rea…
提出GaugeQuant方法，利用LLM对称性在线学习量化最优基，提升大模型压缩效率。
arXiv:2607.20757v1 Announce Type: new Abstract: Transformers are known to have internal continuous symmetries that leave outputs invariant, while modi…
浏览器运行1-bit量级大模型，WebGPU加速，前沿高效新体验。
Article URL: https://huggingface.co/spaces/webml-community/bonsai-webgpu Comments URL: https://news.ycombinator.com/item?id=48936994 Points: 3 # Comme…
手机端首次跑通27B大模型，Bonsai利用量化技术让AI真正落地移动设备。
IT之家 7 月 15 日消息，@PrismML 官方账号今天（7 月 15 日）发布博文，宣布推出 Bonsai 27B 模型，基于 Qwen 3.6 27B 模型微调， 在保留 90% 智能水平的情况下，可以在 12GB 内存的 iPhone 上原生运行。 Qwen 3.6 27B 模型进一步提…
无需隐变量，三值量化LLM训练新方法，开源项目BitBop让你以极低成本体验高效训练。
Article URL: https://github.com/ValerioDolci/bitbop Comments URL: https://news.ycombinator.com/item?id=48892013 Points: 1 # Comments: 1
从Jetson Nano到模型变体，揭示边缘LLM基准测试中易忽视的变量，避免自欺欺人。
A Jetson Nano and Ollama benchmark review appeared among DEV's top articles on 2026-07-12 . Edge inference experiments are valuable because they make …
设备端AI推理新突破：自动云回退、4-bit无损量化、PyTorch模型一键转换。
Hi HN, Roman and Henry here from Cactus ( https://github.com/cactus-compute/cactus ). We just shipped the biggest upgrade to our on-device inference p…
在慢速电脑上运行744B参数的GLM-5.2模型，实测不同精度下多token预测头的性能表现。
A few days ago I found myself trying out GLM 5.2 and was really positively impressed. The capabilities and security I was getting from this LLM are si…
INT8与FP16量化全解析，教你让手机端AI模型更省电更流畅，开发者必读。
If you have ever tried to run a massive Large Language Model (LLM) or a high-resolution computer vision model directly on a mobile device, you’ve like…
量化部署大模型暗藏后门陷阱？FLipGuard精准防御QCB攻击，守护AI安全底线。
arXiv:2606.28962v1 Announce Type: cross Abstract: Model quantization is essential for the efficient deployment of Large Language Models (LLMs), but in…
校准引导的LLM压缩竟有输出空间分配开销？这篇实证研究用数据揭示量化压缩的隐性成本，值得技术玩家细读。
arXiv:2606.27785v1 Announce Type: cross Abstract: Training-free compression methods for large language models (LLMs) often use calibration data to gui…
BitNet二值化网络实现高效文本嵌入，为资源受限场景提供轻量级NLP新方案。
arXiv:2606.25674v1 Announce Type: new Abstract: LLM-based text embedders have substantially improved retrieval and semantic representation quality, bu…
残差感知新方法让大模型二值化训练更准更高效，已被ICML 2026接收。
arXiv:2602.05367v3 Announce Type: replace Abstract: Efficient deployment of large language models (LLMs) requires extreme quantization, forcing a crit…
突破性LLM压缩至低于1比特，结合草图算法与硬件友好算子，实现极致模型瘦身。
arXiv:2506.17255v2 Announce Type: replace-cross Abstract: Large language models (LLMs) require larger GPU memory size these days, necessitating effici…
加法码本搭配多比特宽度量化方案，为LLM推理加速与压缩提供新思路。
arXiv:2606.12876v1 Announce Type: cross Abstract: As large language models (LLMs) are increasingly deployed across heterogeneous hardware with varying…
用Ollama配合QAT量化，10GB显存笔记本也能跑12B的Gemma 4大模型，内存仅需6.7GB。
This stack uses Ollama with Gemma 4 QAT to run a 12B model on a 10GB VRAM laptop GPU. The latest Gemma 4 QAT checkpoints reduce memory usage and enabl…
从系统程序员视角拆解LLM推理，附真实CPU性能数据，直击量化与优化要点。
Article URL: https://blog.xiangpeng.systems/posts/how-to-llm-inference/ Comments URL: https://news.ycombinator.com/item?id=48461660 Points: 3 # Commen…