手机端首次跑通27B大模型，Bonsai利用量化技术让AI真正落地移动设备。
IT之家 7 月 15 日消息，@PrismML 官方账号今天（7 月 15 日）发布博文，宣布推出 Bonsai 27B 模型，基于 Qwen 3.6 27B 模型微调， 在保留 90% 智能水平的情况下，可以在 12GB 内存的 iPhone 上原生运行。 Qwen 3.6 27B 模型进一步提…
无需隐变量，三值量化LLM训练新方法，开源项目BitBop让你以极低成本体验高效训练。
Article URL: https://github.com/ValerioDolci/bitbop Comments URL: https://news.ycombinator.com/item?id=48892013 Points: 1 # Comments: 1
从Jetson Nano到模型变体，揭示边缘LLM基准测试中易忽视的变量，避免自欺欺人。
A Jetson Nano and Ollama benchmark review appeared among DEV's top articles on 2026-07-12 . Edge inference experiments are valuable because they make …
设备端AI推理新突破：自动云回退、4-bit无损量化、PyTorch模型一键转换。
Hi HN, Roman and Henry here from Cactus ( https://github.com/cactus-compute/cactus ). We just shipped the biggest upgrade to our on-device inference p…
在慢速电脑上运行744B参数的GLM-5.2模型，实测不同精度下多token预测头的性能表现。
A few days ago I found myself trying out GLM 5.2 and was really positively impressed. The capabilities and security I was getting from this LLM are si…
BitNet二值化网络实现高效文本嵌入，为资源受限场景提供轻量级NLP新方案。
arXiv:2606.25674v1 Announce Type: new Abstract: LLM-based text embedders have substantially improved retrieval and semantic representation quality, bu…
残差感知新方法让大模型二值化训练更准更高效，已被ICML 2026接收。
arXiv:2602.05367v3 Announce Type: replace Abstract: Efficient deployment of large language models (LLMs) requires extreme quantization, forcing a crit…
加法码本搭配多比特宽度量化方案，为LLM推理加速与压缩提供新思路。
arXiv:2606.12876v1 Announce Type: cross Abstract: As large language models (LLMs) are increasingly deployed across heterogeneous hardware with varying…
用Ollama配合QAT量化，10GB显存笔记本也能跑12B的Gemma 4大模型，内存仅需6.7GB。
This stack uses Ollama with Gemma 4 QAT to run a 12B model on a 10GB VRAM laptop GPU. The latest Gemma 4 QAT checkpoints reduce memory usage and enabl…
从系统程序员视角拆解LLM推理，附真实CPU性能数据，直击量化与优化要点。
Article URL: https://blog.xiangpeng.systems/posts/how-to-llm-inference/ Comments URL: https://news.ycombinator.com/item?id=48461660 Points: 3 # Commen…
LLM极低比特量化新突破，线性约束向量量化实现2比特数据高效训练，已被ICML 2026收录。
arXiv:2606.10531v1 Announce Type: cross Abstract: Quantization-aware training (QAT) is essential for extremely low-bit large language models (LLMs). C…
只需整数运算的视觉Transformer，快速语义分割无需浮点，部署更高效。
arXiv:2509.10334v2 Announce Type: replace-cross Abstract: Vision Transformers (ViTs) have recently achieved strong results in semantic segmentation, y…
用蒸馏与量化技术扩展Apertus大模型家族，低成本获得高性能小模型
arXiv:2605.29128v1 Announce Type: new Abstract: The wide adoption of LLMs has led to their use in great variety of applications and scenarios, such as…
MoBiQuant通过混合位数量化实现token自适应任意精度，大幅降低大模型内存与计算开销，推理更高效。
arXiv:2602.20191v2 Announce Type: replace-cross Abstract: Dynamic runtime latency and memory constraints necessitate flexible large language model (LL…
揭秘长上下文推理的内存陷阱：即便模型量化后塞入显存，注意力KV缓存也可能比模型本身更吃内存。
A raw, developer-first look at Google’s new open-weight Gemma 4 family—featuring a hands-on local Python setup, a comparison of the 2B, 9B, and 31B va…
TurboQuant号称8倍速，实测CPU端到端慢2.2倍，Qwen准确率还降17个百分点，别被合成数据骗了。
Article URL: https://deemwar-products.github.io/llama-cpu-benchmarks/ Comments URL: https://news.ycombinator.com/item?id=48212222 Points: 1 # Comments…
提出子1比特量化方法，大幅降低大语言模型存储与计算开销，兼顾效率与性能。
arXiv:2602.06694v2 Announce Type: replace Abstract: Weight-only quantization has become a standard approach for efficiently serving large language mod…
基于平坦度的理论最优量化方法，为深度学习模型压缩提供新思路
arXiv:2605.18800v1 Announce Type: new Abstract: Post-training quantization has emerged as a widely adopted technique for compressing and accelerating …
提出专家引导的后合并量化方法，利用合并权重锚定，在低资源部署中平衡模型压缩与性能。
arXiv:2605.16882v1 Announce Type: new Abstract: Low-resource deployment constraints have made model quantization essential for deploying neural networ…