New Research Addresses Multi-Attribute Steering and Inference Acceleration for Large Language Models

New Research Addresses Multi-Attribute Steering and Inference Acceleration for Large Language Models

Researchers have introduced Multi-Attribute Targeted Steering (MAT-Steer), a framework designed to steer large language model behavior across multiple attributes simultaneously, according to a paper published on arxiv.org and accepted for ACL 2025. The method addresses conflicts that arise when trying to enhance multiple attributes like helpfulness while reducing toxicity.

According to the paper, MAT-Steer works by learning steering vectors that shift the model’s internal representations of undesirable outputs closer to desirable ones, while enforcing sparsity and orthogonality among vectors for different attributes to reduce conflicts. The approach operates through inference-time intervention without requiring costly parameter updates.

In evaluations on question answering tasks balancing truthfulness, bias, and toxicity, as well as generative tasks targeting helpfulness, correctness, and coherence, MAT-Steer reportedly achieved a 3% average accuracy gain across QA tasks and a 55.82% win rate against the best baseline, according to the research.

Separately, arxiv.org published research on accelerating GPU inference for LLMs with moderate sparsity levels around 50%. The work, accepted for DAC 2026, proposes a three-layer matrix storage format and SpMM kernel that achieved up to 1.64x kernel-level speedup over SpInfer and 1.41x end-to-end speedups over FlashLLM, according to the paper.

Another arxiv.org paper examined embodied multi-agent coordination through dialogue, extending the PARTNR benchmark for collaborative robotics with a natural-language communication channel.