Remember that “Black Mirror” episode where society’s hierarchy was determined by social media-style star ratings, an average of the score everyone you interacted with from day-to-day gave you? When the protagonist’s score has nosedived, she finds herself stranded on the roadside hollering at speeding cars as drivers whizzed by too put off by her one-star status to lend a hand. She becomes undesirable, almost invisible, a non-person.
Like many “Black Mirror” premises, this science fiction is distressingly close to reality and getting closer. Only as conceived, the real-world equivalent will be based on not just the perfectionist whims of one’s followers, but on the goals of an authoritarian government ruling over the world’s most populous and economically powerful nation.
Last year, Chinese courts banned 23 million would-be travelers from purchasing flight or train tickets based on their low “social credit” scores (SCS). Earlier, in 2017, anti-censorship journalist Liu Hu found himself suddenly cut off from traveling on planes, buying property or taking out loans while in the midst of seeking legal redress for a charge of “fabricating and spreading rumors.”
“There was no file, no police warrant, no official advance notification,” he told The Globe and Mail. “What’s really scary is there’s nothing you can do about it. You can report to no one. You are stuck in the middle of nowhere.”
Hu was among the first to suffer consequences owing to the Chinese Communist Party (CCP)’s most ambitious social management project to date, which aims to numerically rate all 1.35 billion Chinese citizens on a single unified scale by 2020.
First announced in 2014 and deeply rooted in party history, this mandatory and overarching social credit system still exists only in theory. There are currently more than 40 individualized opt-in systems operating in villages and big cities across China, run jointly by local governments and private data-collecting companies such as Sesame Credit (affiliated with payment platform Alipay) and Rapid Finance (affiliated with mobile messaging app WeChat), though the lines between these services and federal SCS trials are being increasingly blurred.
The systems utilize millions of data points on participants’ daily habits – collected via smartphone surveillance, facial recognition cameras, or community “watchers” – to assign them an evolving, publicly available trustworthiness rating based on evaluation by government bureaucrats and/or secret algorithms. Socially positive behaviors like picking up litter or donating to government-run charities earn a higher score and preferential treatment in booking luxury hotels or renting apartments; socially negative behaviors like counterfeiting or jaywalking can cost points and block access to high-end retailers, loan applications or public transit.
The Chinese government’s stated goal in nationalizing surveillance-based credit scores is to create a “culture of sincerity” that will “allow the trustworthy to roam everywhere under heaven while making it hard for the discredited to take a single step.” A professor of history at Dartmouth and author of “The Wobbling Pivot: China since 1800,” Pamela Crossley writes in an email that the system’s unstated goals are “conformity, surrender of privacy, and competition among citizens — as they would be in any country that applies such a system.”
As SCS schemes expand, they’re assigning hard and fast points values to more and more morally neutral behaviors – from buying diapers (that’s good) to frequent videogaming (that’s bad) – based on biased assessments of whether or not a given activity upholds state power. Limiting social mobility and amplifying existing inequalities by design, they’ve already aided in the Chinese government’s ongoing internment and public erasure of Muslim religious minorities by incentivizing citizens to report on public prayer and other Islamic practices to authorities.
Since all companies with a Chinese business license were given their own 18-digit social credit code in 2018, blacklists have also been used to crackdown on illegal social organizations including trade unions, non-profits, foreign-funded NGOs and any political groups that don’t adequately prioritize CCP values in their charters. They’ve also been used to successfully coerce international airlines and other companies into showing Taiwan as part of China on in-flight maps, adopting the CCP’s version of truth by the threat of sanction. It’s a sign the system is already fulfilling its secondary objective of strengthening China’s “discourse power,” a task extending far beyond its borders and potentially interfering with other nations’ sovereignty.
So far reactions to SCSs by those affected have ranged from indifference to outright enthusiasm, likely because they fill a genuine need for accountability shared by many developing economies like China’s, where bank accounts and credit histories are still a novelty. In a 2018 survey of 2,200 Chinese citizens, 76 percent said mutual mistrust between citizens was a problem, and 80 percent either somewhat or strongly supported commercial social credit systems, in which more than 80 percent were already enrolled.
“You tell people it’s a game of merit accumulation,” Crossley explains. “As with any game, it works so long as people believe that the rules are reliable.”
So far, the CCP has succeeded in keeping public perceptions of unfairness and the number of people severely impacted by social credit low, but one of the system’s many Catch-22s is that the most critical and victimized voices naturally become those least likely to be heard. While officials extoll the 3.5 million people and companies who’ve paid outstanding taxes or debts thanks to social credit, average citizens in-general seem willing to make the tradeoff between less privacy and greater security navigating the commercial world, especially when believing they’ll reap the glamorized benefits of social credit rather than its hidden costs.
If this isn’t starting to sound familiar, you haven’t been paying attention.
As Crossley notes, the Chinese SCS system is “derived from American practices,” with the advent of central planning and “a political index that is at most only suggested in American ideas of credit worthiness.” Between FICO ratings, social media rankings and commercial reputation sites, most Westerners have already acquiesced to being judged by many opaque, algorithmically-defined scores, so what’s to stop a concerted government-private sector effort from consolidating them into just one? We already know average Americans are more than willing to sell a maligned group of fellow citizens like drug users, welfare recipients or immigrants down the river if they think it means preferential treatment for them and theirs.
While most operate under the illusion that our surrender of personal info to private companies couldn’t possibly precede mass data collection and abuse – as with Cambridge Analytica and Russia’s role in 2016 elections – Google’s Chinese equivalent Baidu and other tech players are already kowtowing to Beijing’s social credit development, human rights violations and all. Opaque digital algorithms are increasingly replacing human judgment in determining our deservingness for everything from public housing and job openings to home nursing care visits and American citizenship.
“They condition people to respond to externally-imposed values (usually connected to conformity) and turn them into the agents of mutual discipline and surveillance,” Crossley writes of algorithmic judgment and social credit systems. “They also empower internet platforms and financial institutions in much greater degree than governments, leading to gradual displacement of public accountability by private financial objectives.”
In spite of our natural human efforts to simplify, every action exists within a personal and sociocultural context no one algorithm or other system of judgment can fully consider. We say justice is blind like it’s a good thing, but if the same scales become responsible for punishing not just major transgressions but even minor moral lapses and norm violations, we’ll be undercutting innovation and doling out a lot of punishment without regard to its human costs. Moreover, any mandatory, surveillance-based system of social ranking will inherently lessen freedom of choice by making governments and private data collectors third parties, enforcing observers, in every interaction. As on social media platforms, the knowledge of being observed will inevitably alter and homogenize both our behavior and how it’s perceived so even spontaneous acts of kindness will begin to seem performative and transactional.
To avert this Orwellian future, global societies need to become much more proactive in strengthening democratic safeguards and limiting cooperation with Chinese social credit expansion, as well as relying less on data-mining companies and credit scoring systems on an individual level.
“If we don’t do anything, then one day a corporation or a government institution will pull all the information from different data banks together and come up with a social credit score,” Gerd Gigerenzer, a director at Berlin’s Max Planck Institute for Human Development, put it bluntly to Der Tagesspiegel. “At the moment we are investing billions in digital technologies when we should be investing just as much in digital education so humans are aware what algorithms really can, and cannot, do.”
How to Maintain a Good Social Credit Score
China’s social credit systems evaluate and score citizens and companies on so many data points it can be difficult to keep track of which behaviors will earn you points, and which will cost you. To help you remain an upstanding, prosperous, and high-scoring citizen, follow our key.
Punished behaviors include:
- Traffic violations
- Taking drugs
- Not paying taxes
- Smoking on trains
- Occupying reserved seats
- Spreading rumors
- Walking dogs without a leash
- Cheating at online video games
- Associating with people with low scores
Steer clear of such behaviors and you’ll be privy to rewards like:
- High internet speeds
- Deposit-free car and housing rentals
- Special waiting areas at airports and train stations
- Free access to gym facilities
- Better foreign exchange rates
- Skipping hospital waiting lines
- Profile boosts on dating platforms