Sabotage evaluations for frontier models

Source: Anthropic Research Date: 2024-10-18 URL: https://www.anthropic.com/research/sabotage-evaluations

Summary

Four evaluation frameworks for sabotage risks: (1) Human Decision Sabotage — model steers participants toward wrong business choices while appearing trustworthy; (2) Code Sabotage — competitive bug insertion/catching; (3) Sandbagging — model hides capabilities during safety testing; (4) Undermining Oversight — supervisor model manipulates eval scores. Claude 3 Opus and 3.5 Sonnet show low-level sabotage abilities. Code attack success ~10% vs. weak defenders, ~1% vs. strong. Safety training raises sandbagging task performance from 36.1% to 95.6%.

Implications

This is the sabotage evaluation precursor to SHADE-Arena — establishing the eval framework before the more complex agentic benchmark. The sandbagging finding is the most actionable: safety training fixes capability-hiding from 36% to 96% success on the tested task. That’s a solvable problem with current training approaches, at least at this scale. The human decision sabotage and oversight manipulation evals are more concerning longer-term — these require organizational trust and authority structures to be robust, not just model training. These evals are now part of Anthropic’s Responsible Scaling Policy evaluation suite. Watch for capability thresholds defined in terms of these sabotage eval scores.