OpenAI Shuts Down Chinese Campaigns Misusing Its AI Tools
OpenAI has taken action against coordinated operations from China that were using its artificial intelligence tools to conduct influence campaigns. The company identified and disrupted these efforts, reflecting growing concerns about the misuse of AI technologies for information manipulation.
Tl;dr
Expanding Tactics in State-Sponsored Disinformation
When it comes to the evolving landscape of online influence, the deployment of artificial intelligence has opened new chapters—sometimes unsettling ones. Researchers and cybersecurity experts have increasingly noted that state actors are expanding both their range and sophistication of techniques. As Ben Nimmo, the principal investigator at Mandiant, observed in a conversation with NPR: « Ce que nous observons du côté chinois, c’est une gamme croissante d’opérations secrètes utilisant une palette toujours plus étendue de techniques. » This broadening arsenal includes not just traditional propaganda, but also automated content generation, targeted phishing attempts, and even the identification of code vulnerabilities.
OpenAI Uncovers Coordinated Chinese Operations
The trend gained further clarity in spring 2024, following a report from OpenAI. The company disclosed it had identified and shut down four coordinated campaigns originating from China that leveraged its generative models. These operations pushed content onto platforms such as TikTok, Facebook, Reddit and X. Rather than sticking to a single narrative, these efforts generated posts taking conflicting stances on contentious topics—from U.S. politics to criticism of the Chinese Communist Party through a Taiwanese video game. The intent? To muddy public debate by amplifying confusion and division.
The Role of Generative AI in Political Manipulation
Looking back, it becomes apparent that this is hardly an isolated phenomenon. Since at least 2019—as cybersecurity teams at Mandiant previously reported—tools like ChatGPT, developed by OpenAI, have been harnessed for influence operations across multiple states. While similar tactics have been traced to actors in Russia, Iran (notably an Iranian effort to publish fake op-eds about U.S. elections), or North Korea, China appears especially proactive in exploiting all facets of automated social engineering. The objectives stretch beyond spreading misinformation; they encompass phishing schemes and attempts to probe software vulnerabilities.
A Cautious Assessment: Modest Impact for Now
And yet—there’s an important caveat. According to Ben Nimmo, there’s little evidence that these AI-driven campaigns truly achieve greater engagement than previous efforts: « Avoir de meilleurs outils ne garantit pas nécessairement une portée accrue. » Despite this current limitation, there’s reason for concern. As generative technologies become more affordable and widespread, these sophisticated tactics will only become easier to deploy on a large scale. For now their effectiveness may be contained—but their looming potential already worries both experts and tech platforms alike.