CodeAgents + Structure: A Better Way to Execute Actions

Source: HuggingFace Date: 2025-05-28 URL: https://huggingface.co/blog/structured-codeagent

Summary

Research from the smolagents team showing that forcing agents to generate both reasoning and code inside JSON structure (structured CodeAgents) improves benchmark performance by 2-7 percentage points on GAIA, MATH, SimpleQA, and Frames — but only for models ≥32B. Smaller models suffer a “structure tax” where the cognitive overhead of maintaining JSON + Python simultaneously degrades output quality.

Implications

Thread: transformers library trajectory / smolagents. This is a capability gate within the HF agent stack: use_structured_outputs_internally=True is now a supported flag in smolagents CodeAgent, but it requires provider-side structured output support (OpenAI, Anthropic, select HF providers). The 2.4% parsing error rate — which cuts success by 21% when it occurs — is the failure mode to watch. As frontier models grow more capable at structure-following this tradeoff improves, but small-model deployments face a real regression risk if this becomes the default. Watch whether smolagents makes structured mode opt-in or default in future releases.