LLM推理新招：用部分展开引导尾部路由，直击长尾生成瓶颈，性能提升值得关注。
arXiv:2608.22788v1 Announce Type: new Abstract: Large-scale rollouts have become a core component of modern LLM systems, spanning reinforcement learni…
长上下文LLM推理提速新方案，块稀疏注意力优化预填充阶段，值得关注。
arXiv:2608.19758v1 Announce Type: new Abstract: Long-context modeling is a pivotal capability for Large Language Models, yet the quadratic complexity …
千亿级大模型推理下沉到PC集群，预编译管道分片让闲置AI PC变身分布式算力池，技术方案直击成本痛点。
arXiv:2608.19147v1 Announce Type: cross Abstract: Modern Intel AI PCs ship capable integrated GPUs and NPUs with 16+ GB of unified memory, and they sp…
昇腾0 Day适配小红书新开源模型，MoE融合算子大幅提升推理吞吐，国产软硬件协同再进一步。
IT之家 8 月 16 日消息，小红书 8 月 14 日正式发布 dots3-note preview 开源大模型。华为官方昨日宣布， 昇腾 Atlas 800 A3、Atlas 900 A3 SuperPoD 超节点已完成 dots3-note preview 的全量适配 ，并基于 vLLM As…
用prefill激活值重思LLM路由选择，为多模型推理调度开辟高效新路径。
arXiv:2603.20895v3 Announce Type: replace-cross Abstract: Existing routers rely on semantic query features or handcrafted features, which often fail t…
把大模型推理拆成预填充和解码两段，用网络传KV缓存，破解长上下文延迟痛点，值得一读。
Article URL: https://blog.doubleword.ai/when-to-disaggregate Comments URL: https://news.ycombinator.com/item?id=49269470 Points: 4 # Comments: 0
用图结构增强大模型推理，破解长时序实体任务规划难题，值得一读。
arXiv:2608.07905v1 Announce Type: new Abstract: Embodied agents using LLM-based planners often struggle with physical hallucinations, poor generalizat…
LLM代理的跨历史缓存复用新方案，精确匹配策略约束，大幅提升推理效率。
arXiv:2608.08282v1 Announce Type: new Abstract: Tool-using language-model agents face constraints whose meaning changes with observations and prior ac…
大模型剪枝新招：高层注意力剪枝加缩放，推理提速不损精度，论文原文速看。
arXiv:2507.01900v3 Announce Type: replace-cross Abstract: Pruning is a highly effective approach for compressing large language models (LLMs), signifi…
把Kimi K3大模型分布到约10万台PS3主机上推理，物尽其用的分布式AI脑洞，值得一看。
I forked RAM coffers and implemented Kimi K3 on PS3, using one-layer-expert-per-node inference. Idea mine, used AI assistance to implement it. Inspire…
语音AI火爆，Fireworks却缺位？开源模型很强，推理平台优化思路才是真正瓶颈。
I started thinking over why doesn't fireworks support voice models. There are really good opensource models available now, like parakeet, kokoro, Qwen…
单卡AMD MI300X跑起DeepSeek V4 Flash，低成本部署大模型的最新实践。
Article URL: https://github.com/ryanzhou/deepseek-v4-flash-mi300x Comments URL: https://news.ycombinator.com/item?id=49166386 Points: 272 # Comments: …
破解旋转与细粒度组量化协同难题，为低比特大模型推理提速增效带来新思路
arXiv:2607.27694v1 Announce Type: cross Abstract: Low-bit quantization is essential for efficient LLM inference, and both rotation and fine-grained gr…
OpenAI推出GPT-5.6，在模型、推理与智能体工作流上全面提升效率，每美元带来更高智能价值。
GPT-5.6 improves AI efficiency across models, inference, and agentic workflows, helping deliver more useful intelligence per dollar.
KAP方法巧妙填补了大模型知识选择与运行时耗能之间的空白，让知识密集型推理更高效。
arXiv:2607.24260v1 Announce Type: new Abstract: Modern LLM systems increasingly rely on knowledge-selection processes that produce high-value structur…
新论文提出WISERouter，在预算约束下智能路由LLM请求，兼顾成本与性能。
arXiv:2607.23765v1 Announce Type: new Abstract: Large language models (LLMs) achieve impressive performance across multiple domains, but using the mos…
提出PlanE框架，对抽取式LLM的数据、调优和推理进行元规划，有望提升模型效率与泛化能力。
arXiv:2607.20470v1 Announce Type: new Abstract: Enhancing the task-specific capabilities of Large Language Models (LLMs) primarily requires substantia…
答案集编程（ASP）中的流线型技术，用约束剪枝大幅提升求解效率，AI逻辑推理新利器。
arXiv:2604.19251v2 Announce Type: replace-cross Abstract: Streamliner constraints reduce the search space of combinatorial problems by ruling out port…
首个针对AI代理优化开放式LLM推理的基准测试，填补评估空白。
arXiv:2607.20468v1 Announce Type: new Abstract: AI agents are increasingly used to automate research and development tasks, yet existing benchmarks ty…
针对LLM混合专家模型，提出3D内存异构加速器架构，突破内存瓶颈，显著提升推理效率与能效。
arXiv:2607.17074v1 Announce Type: cross Abstract: Mixture of Experts (MoE) architectures have emerged as a dominant paradigm for scaling Large Languag…