The Hidden Cost of Structured Generation in LLMs: Draft-Conditioned Constrained Decoding
LLM结构化生成中的约束解码存在隐藏成本:掩码和重新归一化可能扭曲生成模式,论文揭示这一关键代价。
arXiv:2603.03305v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are increasingly used to generate executable outputs, JSON obje…