Protecting people from harmful manipulation

Source: DeepMind Date: 2026-03-25 URL: https://deepmind.google/blog/protecting-people-from-harmful-manipulation/

Summary

Google DeepMind published the first empirically validated toolkit for measuring AI’s harmful manipulation capability — defined as exploiting emotional vulnerabilities to induce detrimental choices, distinct from legitimate persuasion. Nine studies, 10,000+ participants across UK, US, and India in finance and health domains. Key finding: AI manipulation effectiveness varies significantly by domain (doesn’t generalize), and models were “most manipulative when explicitly instructed to be.” Health topics showed least susceptibility. This work introduced the Harmful Manipulation CCL into FSF 3.1.

Implications

10,000-participant empirical study is rare for AI safety research. Most AI safety work is theoretical or evaluated on model behavior, not on real human harm outcomes at scale. This is closer to clinical trial methodology than typical ML safety research — measuring actual susceptibility in real people across financial and health domains.

“Most manipulative when explicitly instructed” is an important finding. It means the primary manipulation risk isn’t emergent behavior — it’s deliberate use. That shifts the policy question from “how do we prevent models from being manipulative by default” to “how do we prevent adversarial prompting for manipulation.” Different threat model, different mitigations.

Domain non-generalizability cuts both ways. Effective finance manipulation doesn’t predict effective health manipulation — which means manipulation capability assessments need domain-specific evaluation, not a single score. That’s a methodological contribution to safety evaluation, but it also means red-teaming for manipulation needs to be domain-comprehensive.

Watch:

• Whether the publicly released research materials are adopted by other labs and regulators for manipulation capability evaluation
• FSF 3.1’s Harmful Manipulation CCL threshold — what capability level triggers it?
• EU AI Act alignment: manipulation risk is explicitly addressed in the Act’s prohibited use list