NVIDIA官方详解硬件感知的大模型设计，平衡吞吐量与延迟的Pareto前沿策略。
Article URL: https://developer.nvidia.com/blog/ai-model-co-design-hardware-friendly-llm-design/ Comments URL: https://news.ycombinator.com/item?id=488…
让LLM直接操作bash终端，轻量级快照平衡速度与安全，但潜在风险恐怖。
Recently, I was testing an autonomous AI agent in a local directory. I gave it a multi-step coding task and a terminal tool so it could run its own te…
语音代理中流式音频的延迟难题，从原理到开源解决方案，技术深度与实战兼备。
Blog: https://gokuljs.com/blogs/when-latency-becomes-audible code: https://github.com/gokuljs/GoSFU Comments URL: https://news.ycombinator.com/item?id…
AI Agent任务中LLM调用链被阶梯式中断，3-8倍延迟竟源于调度器无视中间状态
arXiv:2605.00528v2 Announce Type: replace-cross Abstract: AI agents execute tens to hundreds of chained LLM calls per task, yet GPU schedulers treat e…
客户端数据库能省掉频繁的服务器往返，让应用响应速度有质的提升，别再忽视这种模式了。
For the last five years, I've worked on web apps that follow the same pattern: build a backend, set up Postgres, wire up REST endpoints, fetch data on…
揭秘Google搜索自动补全系统如何承受每秒千万级请求与100毫秒硬性延迟
Section 1 — Problem Definition An autocomplete system predicts and suggests query completions as a user types, character by character. The goal is to …
探讨RAG系统中缓存复用安全性的新方法，提出Grounded Cache Routing机制，优化成本与延迟。
arXiv:2605.27494v1 Announce Type: cross Abstract: Modern retrieval-augmented generation(RAG) deployments increasingly rely on caching to reduce token …
针对长时LLM Agent的上下文溢出问题，提出并行压缩方法，减少数十秒推理阻塞。
arXiv:2605.23296v1 Announce Type: new Abstract: Long-horizon LLM agents accumulate growing conversation histories that eventually exceed the model's c…
实时语音AI的延迟生死线：从Go到Rust的迁移如何压缩毫秒级时延，守住250ms实时交互边界。
In building Vivik, an execution-grade telephony AI engine, we faced a brutal constraint: the human conversational loop. In psychoacoustics, a delay un…
无需微调模型，无需修改函数协议，AsyncFC 在纯执行层实现异步并发，让 LLM 解码与函数执行重叠，大幅降低端到端延迟。LLM 对未执行结果（symbolic futures）的推理能力被天然利用，开启模型-工具异步交互新范式。
arXiv:2605.15077v1 Announce Type: cross Abstract: Function calling, also known as tool use, is a core capability of modern LLM agents but is typically…