新论文提出WISERouter，在预算约束下智能路由LLM请求，兼顾成本与性能。
arXiv:2607.23765v1 Announce Type: new Abstract: Large language models (LLMs) achieve impressive performance across multiple domains, but using the mos…
突破传统KV缓存局限，专门针对Agentic LLM的推理优化，引入策略驱动编辑的新范式。
arXiv:2606.01065v1 Announce Type: cross Abstract: Modern KV cache management assumes the chatbot workload: prompts arrive once and the cache grows app…
将文本嵌入批处理速度提升至25万条/秒，比Hugging Face TEI快3倍，开源且支持生产环境调优。
Article URL: https://github.com/Artain-AI/ignite-ms Comments URL: https://news.ycombinator.com/item?id=48210818 Points: 2 # Comments: 0