把扩散语言模型变成无损压缩器，用生成式建模重构信息论极限，一篇脑洞大开的交叉前沿研究。
arXiv:2608.11249v1 Announce Type: cross Abstract: We study the problem of lossless text compression, motivated by the rapid growth in the collection a…
用信息论新指标精准识别生成式抄袭，还能重排候选来源，直击AI时代学术诚信痛点。
arXiv:2608.03859v1 Announce Type: new Abstract: Large language models (LLMs) pose challenges to academic integrity and peer review. Yet generative pla…
从理论层面重新审视LLM对齐的奖励分布假设，揭示重尾效应对对齐边界的影响，值得关注。
arXiv:2604.10727v2 Announce Type: replace-cross Abstract: Classical information-theoretic learning bounds typically rely on KL mutual information and …
大模型上下文压缩迎来首个统一理论框架，从数学层面揭示信息保留与压缩比的最优平衡，值得关注。
arXiv:2608.01326v1 Announce Type: cross Abstract: Large Language Models (LLMs) have a bounded context window. The context window is the maximum input …
用信息论量化AI决策过程，为模型可解释性提供严谨数学框架
arXiv:2507.09092v2 Announce Type: replace-cross Abstract: With the intervention of machine vision in our crucial day to day necessities including heal…
用神经网络端到端监督训练直接学习互信息估计函数，开辟数据驱动估计新范式
arXiv:2511.18945v4 Announce Type: replace Abstract: We propose a fully data-driven approach to designing mutual information (MI) estimators. Since any…
提出用秩统计量近似f-散度的新方法，理论简洁且计算高效，为信息论和机器学习提供实用工具。
arXiv:2601.22784v2 Announce Type: replace-cross Abstract: We introduce a rank-statistic approximation of $f$-divergences that avoids explicit density-…
无穷范数下分布估计新突破，理论保证更优，信息论与统计学习交汇。
arXiv:2605.30509v1 Announce Type: cross Abstract: We present improved bounds for estimating discrete probability distributions under the $\ell_\infty$…
提出信息论定义衡量跨语言一致性，用后训练方法让模型对翻译等价提示回答更可靠。
arXiv:2603.04678v3 Announce Type: replace-cross Abstract: Language models often respond inconsistently to translation-equivalent prompts across langua…
提出思考即压缩的观点，把大模型推理能力重新定义为上下文压缩机制，视角极为新颖
arXiv:2605.28713v1 Announce Type: new Abstract: Context compression aims to shorten long context inputs with minimal information loss for LLM inferenc…
从香农信息论视角重新审视LLM，揭示模型容量与缩放定律的深层联系，ICML 2026前沿研究。
arXiv:2605.23901v1 Announce Type: cross Abstract: Existing scaling laws for Large Language Models (LLMs), predominantly monotonic power laws, fail to …
从香农信息论到科尔莫戈罗夫复杂度，揭示人工智能的能力边界与理论极限。
Article URL: https://medium.com/@vishalmisra/shannon-got-ai-this-far-kolmogorov-shows-where-it-stops-c81825f89ca0 Comments URL: https://news.ycombinat…
ICLR 2026 顶会论文：用信息论指导消除奖励模型中的归纳偏置，为强化学习对齐提供更客观的评估基础
arXiv:2512.23461v2 Announce Type: replace Abstract: Reward models (RMs) are essential in reinforcement learning from human feedback (RLHF) to align la…
揭示坐标异质性如何主导二值量化，从InfoNCE损失到召回率的理论桥接，为高效嵌入压缩提供新洞察。
arXiv:2605.17524v1 Announce Type: new Abstract: Binary quantization (BQ) compresses high-dimensional embeddings into one or two bits per coordinate, e…
从信息论看AI写作为何千篇一律，揭开RLHF导致的“注释者共识方言”真相。
Article URL: https://www.pangram.com/blog/joe-stech-information-theory-why-ai-writing-sucks Comments URL: https://news.ycombinator.com/item?id=4819646…