从Gemini迁移到自托管LLM的真实案例，揭示API服务被替代的三大隐形推力，为生产环境部署提供参考。
A while back I wrote about my terminal-inspired portfolio and the products it indexes. Two of those products lean on a language model: the portfolio t…
20分钟搞定大模型推理部署，A10 GPU成本低至一杯拿铁价，甲骨文云实战教程来了。
Last month I needed to stand up a Llama 3 inference endpoint for an internal tool. The requirements were simple: OpenAI-compatible API, auto-scaling, …
突破2-bit量化瓶颈，统一标量与向量量化方法，实现大模型低成本部署与推理加速。
arXiv:2606.10520v1 Announce Type: new Abstract: Post-training quantization at the 2-bit level enables low-cost deployment and inference acceleration f…
深入剖析LLM服务中永不停止的“飞行中批处理”机制，解锁推理效率提升新思路
Article URL: https://joker666.github.io/blog/2026-06-02-llm-serving-in-flight-batching Comments URL: https://news.ycombinator.com/item?id=48414773 Poi…
提出请求专属测试时训练状态的批处理服务方法，助力大模型高效推理部署。
arXiv:2605.28053v1 Announce Type: new Abstract: Test-time training (TTT) adapts an LLM during generation by reading and updating request-owned state, …