Police in China Arrest Man for Allegedly Lying About a Train Crash Using ChatGPT

Chinese police arrested a man Sunday in what could be one of the first precedent-setting cases of a person using ultra-popular AI chatbot ChatGPT to allegedly spread misinformation. That, or it could be a new means for state actors to discredit real news by proclaiming it’s all a lie manufactured by generative AI.

As reported by the South China Morning Post Monday, police in the Gansu province in Northern China say they arrested a man for allegedly creating fake articles about a supposed train crash that killed nine people on April 25. According to the SCMP report, police listed the man’s surname as Hong and claimed he used ChatGPT to create the “fake” story to then spread through multiple blogs online. Notably, Hong was specifically cited for “picking quarrels and provoking trouble,” a political catch-all offense used against dissidents, activists. The charge can lead to a five-to-10-year prison term. In 2013, the charge was expanded to include those who allegedly post and spread fake news or rumors online.

The article in question was allegedly posted to a few dozen accounts on Chinese blogging platform Baijiahao. Police reportedly alleged Hong bypassed Baijiahao’s restrictions on posting the same content to multiple accounts, and put several trending stories into ChatGPT to create multiple different versions of news article to generate around 15,000 clicks. Hong allegedly owns a company that operates multiple blog-type outlets. The platforms are reportedly registered in Shenzhen, a major big tech business and manufacturing hub in southern China.

China’s Cyberspace Administration enacted a new law in January restricting people from using AI to create deepfakes. In addition to video and audio recreations, the law restricts news stories created by AI save for a list of approved outlets. Last month, China issued guidelines restricting AI chatbots including ChatGPT, and experts told Gizmodo this was to help create a CCP-friendly environment around generative AI. The situation becomes even more murky, since the ruling Beijing government has been accused of covering up other major train crashes like the Zhejiang high-speed pileup in 2011 that killed at least 38 people and injured far more.

Though Chinese-based users can access ChatGPT through a VPN, Chinese firms are creating their own chatbots that will comply with the country’s stringent censorship policies. Companies like Baidu, which also owns the Baijiahao blogging platform, are already working hard on their own generative AI systems. 

Though this could be one of the first arrests under China’s new anti-AI guidelines, the political connotation of the law makes the whole arrest murky. Gansu police specifically citing the use of ChatGPT is novel, but it’s not anywhere near the first time police arrested people for allegedly spreading rumors. Back in 2013, Gansu police reportedly arrested a high school-aged boy only identified as Yang for allegedly spreading “rumors” online. The young man reportedly questioned how well police were investigating a man who died after falling from a karaoke club roof, leading to some protests.

Chatbots like ChatGPT are already pretty damn good at lying. A local politician in Australia is suing the chatbot maker OpenAI for defamation claiming the AI falsely claimed he was a convicted criminal. Still, that doesn’t mean governments aren’t seeing AI as a way to disavow legitimate reporting. Civil liberty groups working with AI have recently shared their concerns with Gizmodo that repressive governments will use the “it’s all just AI” excuse to discredit legitimate reports.

Want to know more about AI, chatbots, and the future of machine learning? Check out our full coverage of artificial intelligence, or browse our guides to The Best Free AI Art Generators, The Best ChatGPT Alternatives, and Everything We Know About OpenAI’s ChatGPT.

Chinese police arrested a man Sunday in what could be one of the first precedent-setting cases of a person using ultra-popular AI chatbot ChatGPT to allegedly spread misinformation. That, or it could be a new means for state actors to discredit real news by proclaiming it’s all a lie manufactured by generative AI.

As reported by the South China Morning Post Monday, police in the Gansu province in Northern China say they arrested a man for allegedly creating fake articles about a supposed train crash that killed nine people on April 25. According to the SCMP report, police listed the man’s surname as Hong and claimed he used ChatGPT to create the “fake” story to then spread through multiple blogs online. Notably, Hong was specifically cited for “picking quarrels and provoking trouble,” a political catch-all offense used against dissidents, activists. The charge can lead to a five-to-10-year prison term. In 2013, the charge was expanded to include those who allegedly post and spread fake news or rumors online.

The article in question was allegedly posted to a few dozen accounts on Chinese blogging platform Baijiahao. Police reportedly alleged Hong bypassed Baijiahao’s restrictions on posting the same content to multiple accounts, and put several trending stories into ChatGPT to create multiple different versions of news article to generate around 15,000 clicks. Hong allegedly owns a company that operates multiple blog-type outlets. The platforms are reportedly registered in Shenzhen, a major big tech business and manufacturing hub in southern China.

China’s Cyberspace Administration enacted a new law in January restricting people from using AI to create deepfakes. In addition to video and audio recreations, the law restricts news stories created by AI save for a list of approved outlets. Last month, China issued guidelines restricting AI chatbots including ChatGPT, and experts told Gizmodo this was to help create a CCP-friendly environment around generative AI. The situation becomes even more murky, since the ruling Beijing government has been accused of covering up other major train crashes like the Zhejiang high-speed pileup in 2011 that killed at least 38 people and injured far more.

Though Chinese-based users can access ChatGPT through a VPN, Chinese firms are creating their own chatbots that will comply with the country’s stringent censorship policies. Companies like Baidu, which also owns the Baijiahao blogging platform, are already working hard on their own generative AI systems. 

Though this could be one of the first arrests under China’s new anti-AI guidelines, the political connotation of the law makes the whole arrest murky. Gansu police specifically citing the use of ChatGPT is novel, but it’s not anywhere near the first time police arrested people for allegedly spreading rumors. Back in 2013, Gansu police reportedly arrested a high school-aged boy only identified as Yang for allegedly spreading “rumors” online. The young man reportedly questioned how well police were investigating a man who died after falling from a karaoke club roof, leading to some protests.

Chatbots like ChatGPT are already pretty damn good at lying. A local politician in Australia is suing the chatbot maker OpenAI for defamation claiming the AI falsely claimed he was a convicted criminal. Still, that doesn’t mean governments aren’t seeing AI as a way to disavow legitimate reporting. Civil liberty groups working with AI have recently shared their concerns with Gizmodo that repressive governments will use the “it’s all just AI” excuse to discredit legitimate reports.

Want to know more about AI, chatbots, and the future of machine learning? Check out our full coverage of artificial intelligence, or browse our guides to The Best Free AI Art Generators, The Best ChatGPT Alternatives, and Everything We Know About OpenAI’s ChatGPT.