OpenAI Bans Chinese Accounts Using ChatGPT for Social Media Surveillance

OpenAI has decisively banned Chinese accounts from using ChatGPT to alter codes for social media surveillance purposes.

OpenAI Takes Stand Against Social Media Surveillance

OpenAI recently shut down a Chinese user group that was leveraging ChatGPT, an AI developed by OpenAI, to create a social media monitoring tool. Dubbed “Peer Review” by OpenAI, the operation involved the group using ChatGPT to craft sales pitches for a program designed to monitor anti-Chinese sentiment across various platforms including X, Facebook, YouTube, and Instagram.

Unprecedented Actions

According to OpenAI, “This network of ChatGPT accounts operated on a schedule matching Chinese working hours, engaged our Chinese language models, and demonstrated usage patterns indicative of manual rather than automated queries”. Ben Nimmo, a lead researcher at OpenAI, noted this was the first instance of such AI tool misuse discovered by the company.

Complex Ramifications

The surveillance tool’s code appears to be largely based on an open-source version of Meta’s Llama models. The group also used ChatGPT to compile a year-end review, claiming to have crafted phishing emails for clients in China.

Still Unclear Consequences

“Assessing the impact of this activity would require input from multiple stakeholders, including operators of all open-source models that could shed light on this activity”, stated OpenAI. Additionally, OpenAI recently blocked another account that was using ChatGPT to generate social media posts criticizing Cai Xia, a Chinese political scientist and dissident living in exile in the United States.

OpenAI continues to combat the misuse of its tools, emphasizing the need for regulation in the AI field.