China May Be Chasing Impossible Dream by Trying to Harness Internet Algorithms

The milestone marks the first systematic effort by a regulator to compel internet companies to reveal information about the technologies powering their platforms, which have shown the capacity to radically alter everything from pop culture to politics. It also puts Beijing on a path that some technology experts say few governments, if any, are equipped to handle.

The public versions of the filings explain in plain language what types of data a given algorithm uses and what it does with the data. In many instances, they provide less detail than what

Facebook

voluntarily discloses to users about how it ranks content in its news feed.

The full filings, which aren’t public, contain more-extensive descriptions of the data and algorithms, some of it considered confidential business information, people familiar with the submissions said. They also contain a self-assessment of potential security risks, according to public documentation of what the regulator asked companies to provide.

Companies submitted the information in response to a new regulation that came into effect in March that tasks regulators with cleaning up the negative effects of algorithms such as the amplification of harmful information, infringement of user privacy and abuse of gig workers. The regulation also requires algorithms to be used to promote “positive energy,” a

Xi Jinping

-era phrase for content that uplifts public opinion and favorably treats the Communist Party.

Beijing isn’t alone in seeking to restrain the power of algorithms underpinning the internet. Regulators in the U.S. and European Union are grappling with similar issues, such as how to protect teen mental health and stamp out viral misinformation.

The Chinese regulation, however, represents the most assertive attempt to police algorithms directly. Ultimately, it can be applied to any service in the country that uses algorithmic technology.

“They are doing things that no one else has tried yet, and the rest of the world can learn from what works and doesn’t work,” said

Graham Webster,

who runs the DigiChina Project, which tracks China’s digital-policy developments, at Stanford University.

One important question the effort raises, algorithm experts say, is whether direct government regulation of algorithms is practically possible.

The majority of today’s internet platform algorithms are based on a technology called machine learning, which automates decisions such as ad-targeting by learning to predict user behaviors from vast repositories of data. Unlike traditional algorithms that contain explicit rules coded by engineers, most machine-learning systems are black boxes, making it hard to decipher their logic or anticipate the consequences of their use.

Beijing’s interest in regulating algorithms started in 2020, after TikTok sought an American buyer to avoid being banned in the U.S., according to people familiar with the government’s thinking. When several bidders for the short-video platform lost interest after Chinese regulators announced new export controls on information-recommendation technology, it tipped off Beijing to the importance of algorithms, the people said.

The Cyberspace Administration of China moved swiftly to draft a new regulation on algorithmic recommendation systems, seeking in particular to understand how the country’s tech companies shape online discourse and how to curb that influence, people familiar said.

By January of 2022, the regulation was ready, and it went into force two months later—an impressive pace for a government that sometimes sits on draft legislation for years, Mr. Webster said.

The cybersecurity regulator didn’t respond to a request for comment.

China’s regulation shocked people in U.S. tech policy circles for its scope and aggressiveness, according to

Suresh Venkatasubramanian,

a computer science professor at Brown University who served as an assistant director of the White House Office of Science and Technology Policy until this month.

Some in the U.S. government were intrigued when Facebook whistleblower

Frances Haugen

argued in Congress last fall for placing limits on the social-media company’s algorithms, according to Mr. Venkatasubramanian. Regulators worried, however, that it would set a precedent for state control over the flow of information.

“Once you go down that path it’s very hard to go back,” he said.

EU regulators facing the same questions have been more forceful but have still avoided direct government reviews of algorithms.

In July, the European Parliament adopted legislation that requires the largest platforms, such as Google and Facebook, to conduct regular assessments of their systemic risks, such as whether they are spreading illegal content. The companies can choose how they address those risks, including adjusting their algorithms, but must submit to independent audits to prove their solutions actually worked.

Implementation and enforcement details in the EU law are vague, policy experts say. “It will take years and years of struggles and maybe even lawsuits” to interpret the law, said

Matthias Spielkamp,

executive director of AlgorithmWatch, a Berlin-based nonprofit research and advocacy organization.

Beijing’s approach remains vague as well. In theory, the Chinese regulation could give the government full control over the key mechanisms that orchestrate online spaces and, increasingly, offline life as well. Yet Beijing could very well trip over its own ambitions, tech experts say.

Social-media recommendation engines represent some of the most complicated algorithmic systems, with apps like Facebook and TikTok using hundreds or even thousands of algorithms to determine who sees what information.

Having detailed documentation, or even the code, for these systems isn’t enough to understand how they will affect something as broad as online discourse, according to

Cathy O’Neil,

an algorithmic auditor who works with U.S. government agencies to scrutinize company algorithms. “What’s actually important is the data that is going through the algorithm,” she said.

Even with full access to that data, which changes with each user post and interaction, a tech company’s own engineers still struggle to precisely tune the behaviors of its systems, according to Ms. O’Neil. Targeted changes like promoting more propaganda are feasible, she said, “but it is actually impossible to control what a recommendation engine does overall.”

Tech analysts and industry insiders also question whether the Cyberspace Administration, which began as a propaganda division, has the technical expertise to enforce its own rules.

Shortly after the Chinese regulation came into force, government-relations managers and algorithm engineers at ByteDance met with Cyberspace Administration officials to explain the documents they submitted, people familiar with the matter said. During one of those meetings, officials at the agency displayed little understanding of the technical details and company representatives had to rely on a mix of metaphors and simplified language to explain how the recommendation algorithm worked, one of the people said.

Companies haven’t been required to submit code or user data, the people said.

Chinese government guidelines issued last year called for multiple agencies to expand staff to supervise algorithms.

“They’re trying to build the tools, hire the people and get the technical expertise to tackle this kind of stuff,” said

Kendra Schaefer,

head of tech policy research at Beijing-based strategic advisory consulting firm Trivium China. “So enforcement of this is going to ramp up slowly over the next five to 10 years.”

—Raffaele Huang contributed to this article.

Write to Karen Hao at [email protected]

Corrections & Amplifications
China introduced a new regulation on internet companies’ algorithms. An earlier version of this article incorrectly said it was a new law. (Corrected on Aug. 31)

The milestone marks the first systematic effort by a regulator to compel internet companies to reveal information about the technologies powering their platforms, which have shown the capacity to radically alter everything from pop culture to politics. It also puts Beijing on a path that some technology experts say few governments, if any, are equipped to handle.

The public versions of the filings explain in plain language what types of data a given algorithm uses and what it does with the data. In many instances, they provide less detail than what

Facebook

voluntarily discloses to users about how it ranks content in its news feed.

The full filings, which aren’t public, contain more-extensive descriptions of the data and algorithms, some of it considered confidential business information, people familiar with the submissions said. They also contain a self-assessment of potential security risks, according to public documentation of what the regulator asked companies to provide.

Companies submitted the information in response to a new regulation that came into effect in March that tasks regulators with cleaning up the negative effects of algorithms such as the amplification of harmful information, infringement of user privacy and abuse of gig workers. The regulation also requires algorithms to be used to promote “positive energy,” a

Xi Jinping

-era phrase for content that uplifts public opinion and favorably treats the Communist Party.

Beijing isn’t alone in seeking to restrain the power of algorithms underpinning the internet. Regulators in the U.S. and European Union are grappling with similar issues, such as how to protect teen mental health and stamp out viral misinformation.

The Chinese regulation, however, represents the most assertive attempt to police algorithms directly. Ultimately, it can be applied to any service in the country that uses algorithmic technology.

“They are doing things that no one else has tried yet, and the rest of the world can learn from what works and doesn’t work,” said

Graham Webster,

who runs the DigiChina Project, which tracks China’s digital-policy developments, at Stanford University.

One important question the effort raises, algorithm experts say, is whether direct government regulation of algorithms is practically possible.

The majority of today’s internet platform algorithms are based on a technology called machine learning, which automates decisions such as ad-targeting by learning to predict user behaviors from vast repositories of data. Unlike traditional algorithms that contain explicit rules coded by engineers, most machine-learning systems are black boxes, making it hard to decipher their logic or anticipate the consequences of their use.

Beijing’s interest in regulating algorithms started in 2020, after TikTok sought an American buyer to avoid being banned in the U.S., according to people familiar with the government’s thinking. When several bidders for the short-video platform lost interest after Chinese regulators announced new export controls on information-recommendation technology, it tipped off Beijing to the importance of algorithms, the people said.

The Cyberspace Administration of China moved swiftly to draft a new regulation on algorithmic recommendation systems, seeking in particular to understand how the country’s tech companies shape online discourse and how to curb that influence, people familiar said.

By January of 2022, the regulation was ready, and it went into force two months later—an impressive pace for a government that sometimes sits on draft legislation for years, Mr. Webster said.

The cybersecurity regulator didn’t respond to a request for comment.

China’s regulation shocked people in U.S. tech policy circles for its scope and aggressiveness, according to

Suresh Venkatasubramanian,

a computer science professor at Brown University who served as an assistant director of the White House Office of Science and Technology Policy until this month.

Some in the U.S. government were intrigued when Facebook whistleblower

Frances Haugen

argued in Congress last fall for placing limits on the social-media company’s algorithms, according to Mr. Venkatasubramanian. Regulators worried, however, that it would set a precedent for state control over the flow of information.

“Once you go down that path it’s very hard to go back,” he said.

EU regulators facing the same questions have been more forceful but have still avoided direct government reviews of algorithms.

In July, the European Parliament adopted legislation that requires the largest platforms, such as Google and Facebook, to conduct regular assessments of their systemic risks, such as whether they are spreading illegal content. The companies can choose how they address those risks, including adjusting their algorithms, but must submit to independent audits to prove their solutions actually worked.

Implementation and enforcement details in the EU law are vague, policy experts say. “It will take years and years of struggles and maybe even lawsuits” to interpret the law, said

Matthias Spielkamp,

executive director of AlgorithmWatch, a Berlin-based nonprofit research and advocacy organization.

Beijing’s approach remains vague as well. In theory, the Chinese regulation could give the government full control over the key mechanisms that orchestrate online spaces and, increasingly, offline life as well. Yet Beijing could very well trip over its own ambitions, tech experts say.

Social-media recommendation engines represent some of the most complicated algorithmic systems, with apps like Facebook and TikTok using hundreds or even thousands of algorithms to determine who sees what information.

Having detailed documentation, or even the code, for these systems isn’t enough to understand how they will affect something as broad as online discourse, according to

Cathy O’Neil,

an algorithmic auditor who works with U.S. government agencies to scrutinize company algorithms. “What’s actually important is the data that is going through the algorithm,” she said.

Even with full access to that data, which changes with each user post and interaction, a tech company’s own engineers still struggle to precisely tune the behaviors of its systems, according to Ms. O’Neil. Targeted changes like promoting more propaganda are feasible, she said, “but it is actually impossible to control what a recommendation engine does overall.”

Tech analysts and industry insiders also question whether the Cyberspace Administration, which began as a propaganda division, has the technical expertise to enforce its own rules.

Shortly after the Chinese regulation came into force, government-relations managers and algorithm engineers at ByteDance met with Cyberspace Administration officials to explain the documents they submitted, people familiar with the matter said. During one of those meetings, officials at the agency displayed little understanding of the technical details and company representatives had to rely on a mix of metaphors and simplified language to explain how the recommendation algorithm worked, one of the people said.

Companies haven’t been required to submit code or user data, the people said.

Chinese government guidelines issued last year called for multiple agencies to expand staff to supervise algorithms.

“They’re trying to build the tools, hire the people and get the technical expertise to tackle this kind of stuff,” said

Kendra Schaefer,

head of tech policy research at Beijing-based strategic advisory consulting firm Trivium China. “So enforcement of this is going to ramp up slowly over the next five to 10 years.”

—Raffaele Huang contributed to this article.

Write to Karen Hao at [email protected]

Corrections & Amplifications
China introduced a new regulation on internet companies’ algorithms. An earlier version of this article incorrectly said it was a new law. (Corrected on Aug. 31)