Researchers have developed a training-free method for aligning large language models (LLMs) with human preferences, according to a paper accepted at IJCAI 2026 and published on arxiv.org. The approach, called Preference alignment of Large Language Models via Residual Steering (PaLRS), extracts lightweight steering vectors from the residual streams of LLMs using as few as one hundred preference pairs.
According to the paper, PaLRS offers an alternative to existing methods like Reinforcement Learning from Human Feedback (RLHF) and Direct Preference Optimization (DPO), which “typically require curated data and expensive optimization over billions of parameters, and eventually lead to persistent task-specific models.” Instead, PaLRS creates plug-and-play steering vectors that can be applied at inference time.
The researchers evaluated PaLRS on various small-to-medium-scale open-source LLMs, demonstrating “consistent gains on mathematical reasoning and code generation benchmarks while preserving baseline general-purpose performance,” according to arxiv.org. When compared to models aligned with DPO and SimPO, PaLRS-aligned models “perform better with great time-savings.”
The paper emphasizes that PaLRS provides “an effective, much more efficient and flexible alternative to standard preference optimization pipelines, offering a training-free, plug-and-play mechanism for alignment with minimal data.”