According to arxiv.org, researchers have introduced TRACE (Trajectory-Based Safety Patch Learning), a new framework designed to restore safety alignment in fine-tuned large language models without compromising their customized capabilities.
The research addresses a critical challenge in Fine-Tuning-as-a-Service (FTaaS) platforms, where user customization can erode models’ safety alignment. According to the paper published on arxiv.org, existing parameter merging approaches face “task-safety update entanglement,” where safety patches and task-specific updates overlap in their dominant directions, making calibration difficult.
TRACE shifts focus from online merging to offline patch learning. According to arxiv.org, the framework “simulates harmful tuning trajectories to generate progressively corrupted states” and optimizes a plug-in patch to recover safety while maintaining utility across varying corrupted base states.
According to the research, TRACE achieved nearly 100% safety across six benchmarks and two models while maintaining comparable utility to undefended fine-tuned models. The paper states that TRACE “consistently dominates the safety-utility frontier.”
Separately, arxiv.org published concurrent research on model-based control and LLM alignment, arguing that “learning signals should come from world feedback rather than from internal model proxies.” The work examined outcome-supervised feedback models, finding they apply to both model-based control and RLHF approaches in LLM alignment.