1
Analysis of Information Theory for Explainable AI
用信息论量化AI决策过程，为模型可解释性提供严谨数学框架
arXiv:2507.09092v2 Announce Type: replace-cross Abstract: With the intervention of machine vision in our crucial day to day necessities including heal…
2
MIST: Mutual Information Estimation Via Supervised Training
用神经网络端到端监督训练直接学习互信息估计函数，开辟数据驱动估计新范式
arXiv:2511.18945v4 Announce Type: replace Abstract: We propose a fully data-driven approach to designing mutual information (MI) estimators. Since any…
3
Approximating $f$-Divergences with Rank Statistics
提出用秩统计量近似f-散度的新方法，理论简洁且计算高效，为信息论和机器学习提供实用工具。
arXiv:2601.22784v2 Announce Type: replace-cross Abstract: We introduce a rank-statistic approximation of $f$-divergences that avoids explicit density-…
4
Improved Distribution Estimation in $\ell_\infty$
无穷范数下分布估计新突破，理论保证更优，信息论与统计学习交汇。
arXiv:2605.30509v1 Announce Type: cross Abstract: We present improved bounds for estimating discrete probability distributions under the $\ell_\infty$…
5
Post-Training Language Models for Crosslingual Consistency
提出信息论定义衡量跨语言一致性，用后训练方法让模型对翻译等价提示回答更可靠。
arXiv:2603.04678v3 Announce Type: replace-cross Abstract: Language models often respond inconsistently to translation-equivalent prompts across langua…
6
Thinking as Compression: Your Reasoning Model is Secretly a Context Compressor
提出思考即压缩的观点，把大模型推理能力重新定义为上下文压缩机制，视角极为新颖
arXiv:2605.28713v1 Announce Type: new Abstract: Context compression aims to shorten long context inputs with minimal information loss for LLM inferenc…
7
LLMs as Noisy Channels: A Shannon Perspective on Model Capacity and Scaling Laws
从香农信息论视角重新审视LLM，揭示模型容量与缩放定律的深层联系，ICML 2026前沿研究。
arXiv:2605.23901v1 Announce Type: cross Abstract: Existing scaling laws for Large Language Models (LLMs), predominantly monotonic power laws, fail to …
8
Shannon Got AI This Far. Kolmogorov Shows Where It Stops
从香农信息论到科尔莫戈罗夫复杂度，揭示人工智能的能力边界与理论极限。
Article URL: https://medium.com/@vishalmisra/shannon-got-ai-this-far-kolmogorov-shows-where-it-stops-c81825f89ca0 Comments URL: https://news.ycombinat…
9
Eliminating Inductive Bias in Reward Models with Information-Theoretic Guidance
ICLR 2026 顶会论文：用信息论指导消除奖励模型中的归纳偏置，为强化学习对齐提供更客观的评估基础
arXiv:2512.23461v2 Announce Type: replace Abstract: Reward models (RMs) are essential in reinforcement learning from human feedback (RLHF) to align la…
10
Coordinate Heterogeneity Governs Binary Quantization: From InfoNCE to Recall
揭示坐标异质性如何主导二值量化，从InfoNCE损失到召回率的理论桥接，为高效嵌入压缩提供新洞察。
arXiv:2605.17524v1 Announce Type: new Abstract: Binary quantization (BQ) compresses high-dimensional embeddings into one or two bits per coordinate, e…
11
The Information Theory Behind Why AI Writing Sucks
从信息论看AI写作为何千篇一律，揭开RLHF导致的“注释者共识方言”真相。
Article URL: https://www.pangram.com/blog/joe-stech-information-theory-why-ai-writing-sucks Comments URL: https://news.ycombinator.com/item?id=4819646…