Advancing red teaming with people and AI

Source: OpenAI Date: 2024-11-21 URL: https://openai.com/index/advancing-red-teaming-with-people-and-ai

Summary

Summary

OpenAI published a paper and process description on hybrid red teaming — using both human red teamers and AI-assisted red teaming tools to more efficiently find safety failures in frontier models. The approach uses AI to scale and diversify the space of adversarial inputs that human red teamers would need to manually construct, covering a broader distribution of potential attacks.

Implications

Safety/alignment thread. AI-assisted red teaming is a structural response to the scaling problem in safety evaluation: frontier models have vast capability surfaces that human red teamers cannot cover manually. Using AI to generate and explore adversarial input spaces is both more efficient and potentially reveals categories of failure that humans wouldn’t think to probe. The November 2024 timing places this as OpenAI was scaling up pre-deployment evaluation processes ahead of o3 and GPT-5 releases — the methodology here likely fed directly into subsequent system card evaluations. The paper also reflects the tension between capability scaling and safety evaluation scaling — the former is outrunning the latter.