1
微软升级 Win11 的 AI 框架，实验性支持 GGUF 与 ONNX 原生本地推理
IT之家 10 月 9 日消息，科技媒体 NeoWin 昨日（10 月 8 日）发布博文，报道称微软正与 llama.cpp 和 GGUF 开源社区合作，在 Windows 11 系统上升级 Windows ML， 实验性原生支持 GGUF 与 ONNX 格式模型推理，并推出 Windows ML …
2
GitHub Copilot 不再纯云端 AI 模型：Surface Laptop Ultra 本地推理吞吐量最高每秒 63 词元
IT之家 10 月 8 日消息，太平洋时间 10 月 7 日上午 10 点（北京时间 10 月 8 日凌晨 1 点）在美国旧金山举办的发布会中，微软宣布 GitHub Copilot 将在本月底前支持本地 AI 模型推理， 开发者可在云端模型与设备端模型之间自动或手动切换。 GitHub Copil…
3
Show HN: Shoehorn, a library to quantize an LLM to fit your Mac's VRAM
一键按显存容量精准量化 LLM，告别加载失败，适配你的 Mac。
I made this after seeing someone posit the idea online yesterday over lunch then spent some time refining it. So far it's pretty impressive IMO! Right…
4
Show HN: RAGless – similar to RAG, but $0 LLM API costs at runtime
类似RAG但运行时零LLM成本，开源项目，让AI应用摆脱API费用。
Article URL: https://github.com/EmilResearch/RAGless Comments URL: https://news.ycombinator.com/item?id=49296595 Points: 2 # Comments: 2
5
Show HN: Run Full Kimi K3 with 29 GB of RAM
2.78万亿参数Kimi K3模型仅需29GB内存即可在笔记本上完整运行，挑战大模型本地部署极限。
Article URL: https://github.com/sqliteai/waste/ Comments URL: https://news.ycombinator.com/item?id=49106591 Points: 7 # Comments: 1
6
零刻推出 SEi13 AI / SEi14 AI 迷你主机：搭载最高 160TOPS 算力独立 NPU，11895 元起
零刻两款迷你主机内置独立NPU算力卡，最高160TOPS，专为本地AI推理而生，11895元起售。
IT之家 7 月 29 日消息，零刻今日宣布推出零刻 SEi13 AI 与零刻 SEi14 AI 迷你主机，号称“为本地 AI 推理而生”，售价 11895 元起： 零刻 SEi13 AI：酷睿 i7-13620H、32G+1TB、24G 算力卡， 11895 元 零刻 SEi14 AI：酷睿 Ul…
7
Show HN: A curated index of open-source AI developer tools
GitHub 上精心整理的开源 AI 开发者工具索引，覆盖模型调用、本地推理、Agent 框架等全栈可替代方案，开发者必看。
Article URL: https://github.com/Sami-Uysal/awesome-open-ai-developer-tools Comments URL: https://news.ycombinator.com/item?id=49071446 Points: 1 # Com…
8
Show HN: Gemma 4 26B A4B running on an iPhone 17 Pro via model paging
2B参数模型Native运行于iPhone等个人设备，隐私智能完全掌控，高效推理数学。
I wanted to share a use case for Noema Overfit, a model-paging system available today in the Noema app. For disclosure, I founded Noema and I am part …
9
Show HN: Libargus:Low-latency local LLM runner via OpenJDK Panama FFM (Java 22)
低延迟的JVM本地LLM推理方案，利用OpenJDK Panama FFM避免传统串行化与GC开销。
Most existing approaches for running local LLM inference within the JVM ecosystem rely on spawning out-of-process daemons via REST sidecars (introduci…
10
Show HN: TensorSharp: Open-Source Local LLM Inference Engine
开源本地LLM推理引擎TensorSharp：基于C#/.NET，30秒启动GPU加速推理，支持GGUF模型
A native .NET LLM inference engine for GGUF models — with a command-line tool, a browser chat server, and Ollama- & OpenAI-compatible APIs for program…
11
Show HN: BurstyRouter: local LLM first then burst to end-to-end encrypted LLM
本地LLM优先、云端突发兜底的智能路由，端到端加密让连服务商都读不到你的AI对话。
Article URL: https://github.com/Lore-Hex/BurstyRouter Comments URL: https://news.ycombinator.com/item?id=48804448 Points: 1 # Comments: 0
12
Show HN: EdgeRunner – run GGUF models with Swift and Metal
在苹果设备上用 Swift 和 Metal 原生运行 GGUF 大模型，为 iOS/macOS 本地 AI 推理打开新大门。
Article URL: https://github.com/christopherkarani/EdgeRunner Comments URL: https://news.ycombinator.com/item?id=48795979 Points: 2 # Comments: 0
13
Show HN: Gemma 3 inference in pure C++ with Metal acceleration
纯C++实现Gemma 3推理，Metal加速，让本地大模型运行更轻快。
Article URL: https://github.com/ybubnov/metalchat Comments URL: https://news.ycombinator.com/item?id=48786298 Points: 3 # Comments: 1
14
What I learned building offline AI for Android
端侧AI落地避坑：下载流程就是转化引导，隐私靠设计而非口号，价值藏在数据不该上云的场景里
I built Orb because I wanted AI on my phone that still worked when the network disappeared. Most AI apps are simple from the app side: send the prompt…
15
You can now run Max AI models on Apple Silicon
Max AI模型终于支持Apple Silicon GPU，Mac用户本地跑大模型更方便了。
Article URL: https://forum.modular.com/t/max-models-can-now-run-on-apple-silicon-gpus/3283 Comments URL: https://news.ycombinator.com/item?id=48705585…
16
Ask HN: MacBook vs. Dedicated GPU for LLM
MacBook 跑 LLM 和独显到底差多少？HN 老哥用实战经验聊透选型门道，教你判断自己机器能带多大模型。
For those who are using llms on macbook, Want to understand how macbook is different than dedicated GPU in running those models? and how to know how m…
17
I built a roguelike whose dungeon master is an LLM running 100% in the browser
浏览器端LLM驱动的roguelike游戏，零服务器成本，架构设计与提示工程巧思
Most "AI games" phone home. Every turn is an API round-trip, every player burns your tokens, and the whole thing dies the day the bill scares you. I w…
18
Running local models is good now
本地模型性能跃升：M2 Mac实测达到前沿模型75%效果，低成本也能玩转Agentic Coding
Article URL: https://vickiboykis.com/2026/06/15/running-local-models-is-good-now/ Comments URL: https://news.ycombinator.com/item?id=48555993 Points: …
19
谷歌推出 DiffusionGemma 文本扩散模型：本地 AI 推理速度提升 4 倍
基于文本扩散的开放AI模型，本地推理速度提升4倍，高效处理文本生成任务
IT之家 6 月 11 日消息，谷歌今天（6 月 11 日）发布公告，宣布推出 DiffusionGemma，是基于文本扩散机制的开放 AI 模型， 相比较自回归模型在本地推理速度上提升了 4 倍。 IT之家注：自回归模型（Autoregressive Model）是当前主流的大语言模型架构（如 G…
20
New Apple Dev Betas: Is it possible to force LLM requests to stay on device?
苹果新测试版能否强制大模型请求只走本地？企业数据安全与MDM控制的实用探讨。
I can imagine many corporates with MDM settings wanting to say 'data on the endpoint is OK, data off the endpoint is not, notwithstanding Apple's PCC …