Four research papers published on arxiv.org on May 21, 2026, advance different aspects of large language model (LLM) development and deployment.
According to arxiv.org, researchers investigated cross-modal skill injection, examining how to transfer domain-specific expertise from LLMs to Vision-Language Models (VLMs) without additional training data. The study found that cross-modal skill injection “generally performs well in instruction-following and cross-lingual settings, yet struggles with mathematical reasoning,” and that classic approaches like TA and DARE “consistently achieve superior performance over alternative merging methods.”
A separate study accepted to ICML 2026 addressed multi-agent LLM aggregation, according to arxiv.org. Researchers introduced two algorithms called Optimal Weight (OW) and Inverse Surprising Popularity (ISP) that “leverage both first-order and second-order information” to improve upon standard majority voting. The methods “consistently outperform standard baselines” across benchmarks including UltraFeedback and MMLU.
Another arxiv.org paper systematically re-evaluated nine LoRA variants for efficient LLM fine-tuning. After extensive hyperparameter searches, researchers found that “once learning rates are properly tuned, all methods achieve similar peak performance (within 1-2%),” suggesting that “vanilla LoRA remains a competitive baseline.”
Finally, researchers introduced Formal Skill, “a runtime-native abstraction” for LLM agents that represents reusable capabilities with structured metadata and executable state machines, according to arxiv.org. The implementation, called FairyClaw, achieved “highly competitive average scores while using substantially fewer tokens” on Harness-Bench.