New Research Examines LLM Agent Performance, Safety, and Tool-Calling Methods

New Research Examines LLM Agent Performance, Safety, and Tool-Calling Methods

Several new studies published on arxiv.org explore different aspects of large language model (LLM) agents, from benchmarking to practical applications.

According to arxiv.org, researchers introduced AgentGym2, a benchmark for evaluating LLM agents in “de-idealized real-world environments,” though specific methodology details were not provided in the available excerpt.

A separate study on arxiv.org independently replicated and extended the Natural Language Tools (NLT) framework, finding that natural language tool calling improves accuracy by 14.9 percentage points overall compared to structured tool calling (62.3% versus 47.4%). According to the research, NLT also reduces critical errors by 93% (51 versus 755 errors) and cuts token usage by 25.2%. The study evaluated 14 models across 8,560 trials, finding that “models without native tool calling, reasoning models, and smaller models gain substantially (+24.0pp to +43.1pp), while heavily optimized frontier models (GPT-5, Gemini 2.5 Pro) show smaller or reversed advantages.”

In healthcare applications, arxiv.org reported on CareConnect, a conversational agent for healthcare appointment scheduling that achieved a 91.8% task completion rate with 96.0% safety compliance and an average operational cost of $0.0324 per appointment.

Finally, arxiv.org presented the first controlled study examining how agentic scaffolding—the middleware layer between developers and LLMs—affects coding agent quality over time. The research evaluated 35 sequential releases while holding the underlying LLM constant, revealing “extreme release velocities exceeding two releases per day.”