According to arxiv.org, researchers have introduced new methods to combat “model collapse” — the degradation that occurs when large language models (LLMs) are trained on AI-generated content, a phenomenon known as “AI autophagy.” The study demonstrates that model collapse corresponds to “an increasing concentration of probability mass on a small set of tokens” and is associated with a loss of common sense reasoning.
The research identifies perplexity, a measure of model “surprise,” as a key driver of collapse. According to the paper, fine-tuning on the least surprising documents leads to more severe degeneration. The researchers propose a perplexity-based filtering strategy that prioritizes high-surprise documents during training, which “does not require distinguishing between human-authored and AI-generated content.”
In related work, arxiv.org describes the Adaptive Context Elasticizer (ACE), a “plug-and-play module” designed to manage growing trajectory lengths in AI agents. ACE maintains a “lossless message maintenance layer” and adaptively assigns historical steps as “raw, abstract, or drop” at each decision point. According to the research, ACE was tested on four agent frameworks including ReAct and DeepAgent, where it “consistently outperforms truncation and summarization baselines.”
Additionally, arxiv.org introduces the Relative Surprisal Index (RSI) for reinforcement learning with verifiable rewards, which “couples the token’s entropy with the probability of the selected token” to improve LLM reasoning capabilities.