一键按显存容量精准量化 LLM，告别加载失败，适配你的 Mac。
I made this after seeing someone posit the idea online yesterday over lunch then spent some time refining it. So far it's pretty impressive IMO! Right…
基于llamafile/GGUF的终端编码智能体，本地运行高效又灵活
Article URL: https://github.com/kdeps/kdeps Comments URL: https://news.ycombinator.com/item?id=49029939 Points: 2 # Comments: 0
开源本地LLM推理引擎TensorSharp：基于C#/.NET，30秒启动GPU加速推理，支持GGUF模型
A native .NET LLM inference engine for GGUF models — with a command-line tool, a browser chat server, and Ollama- & OpenAI-compatible APIs for program…
开发者亲述如何在RTX 3050消费级显卡上打造完全离线、隐私优先的AI编排器，实现亚100ms推理速度，兼顾性能与隐私。
Hi, I'm Raj Patil (known online as Dream / lostxmusafir ), an AI Engineer and Full-Stack Developer. In this article, I want to break down how I develo…
在苹果设备上用 Swift 和 Metal 原生运行 GGUF 大模型，为 iOS/macOS 本地 AI 推理打开新大门。
Article URL: https://github.com/christopherkarani/EdgeRunner Comments URL: https://news.ycombinator.com/item?id=48795979 Points: 2 # Comments: 0
量化检查点常先用质量指标筛选，但论文揭示36组质量-安全配对均不一致，安全检测不可跳过
arXiv:2606.10154v1 Announce Type: new Abstract: Quantized checkpoints are often screened first with quality metrics and only later, if at all, with di…
一文讲清GGUF、GPTQ、AWQ三种主流LLM量化格式的区别，帮你选对压缩方式省显存。
Article URL: https://vettedconsumer.com/gguf-vs-gptq-vs-awq-the-plain-english-guide-to-llm-quantization-and-which-one-to-pick/ Comments URL: https://n…
用Docker Compose和MCP编排多智能体LLM，从基础到生产部署实战指南，开源可上手。
Article URL: https://github.com/PacktPublishing/Operational-AI-with-Docker Comments URL: https://news.ycombinator.com/item?id=48275638 Points: 2 # Com…
本地运行Gemma 4模型，通过JSON AST生成Clojure代码，隐私安全且自定义灵活
Article URL: https://github.com/quadracollision/llmisp Comments URL: https://news.ycombinator.com/item?id=48196707 Points: 1 # Comments: 0
8款LLM模型无需GPU在Linux上实测，CPU推理新选择关注真实可用性
Article URL: https://itsfoss.com/testing-local-llms-without-gpu/ Comments URL: https://news.ycombinator.com/item?id=48147334 Points: 2 # Comments: 0
纯 Rust 打造的本地 AI 助手，一个二进制文件集成聊天、索引与搜索。Metal 加速、GGUF 模型、ONNX 嵌入，Apple Silicon 用户的最佳本地 AI 编程搭档。
Article URL: https://github.com/Sok205/local_vibe Comments URL: https://news.ycombinator.com/item?id=48162858 Points: 2 # Comments: 0