China’s All-Seeing Surveillance State Is Reading Its Citizens’ Faces
Josh Chin and Liza Lin
13-17 minutes

SHENZHEN—Gan Liping pumped her bike across a busy street, racing to beat a crossing light before it turned red. She didn’t make it. Immediately, her face popped up on two video screens above the street. “Jaywalkers will be captured using facial-recognition technology,” the screens said.

Facial-recognition technology, once a specter of dystopian science fiction, is becoming a feature of daily life in China, where authorities are using it on streets, in subway stations, at airports and at border crossings in a vast experiment in social engineering. Their goal: to influence behavior and identify lawbreakers.

Ms. Gan, 31 years old, had been caught on camera crossing illegally here once before, allowing the system to match her two images. Text displayed on the crosswalk screens identified her as a repeat offender.

“I won’t ever run a red light again,” she said.

China is rushing to deploy new technologies to monitor its people in ways that would spook many in the U.S. and the West. Unfettered by privacy concerns or public debate, Beijing’s authoritarian leaders are installing iris scanners at security checkpoints in troubled regions and using sophisticated software to monitor ramblings on social media. By 2020, the government hopes to implement a national “social credit” system that would assign every citizen a rating based on how they behave at work, in public venues and in their financial dealings.

China’s technology companies are helping lead the way, scooping up unprecedented data on people’s lives through their mobile phones and competing to develop and market surveillance systems for government use.

Facial-recognition technology is one of the most powerful new tools in the surveillance arsenal. Fueled by advances in artificial intelligence, these systems can measure key aspects of a face, such as distance between the eyes and skin tone, then cross-reference them against huge databases of photographs collected by government agencies and businesses and shared on social media.

Other countries also have begun experimenting. In the U.S., the Federal Bureau of Investigation uses the technology to help identify criminal suspects, and the Department of Homeland Security is deploying it in airports to track when foreign visitors leave the country. Its use is expected to grow world-wide as a tool for law enforcement and personal identification, and U.S. companies are among those using it in pilot programs.

China, however, stands apart in harnessing facial recognition as a cudgel to influence behavior. The Chinese Ministry of Public Security—its national police force—and other agencies called in 2015 for the creation of an “omnipresent, completely connected, always on and fully controllable” nationwide video-surveillance network as a public-safety imperative. In a policy statement, the agencies included “facial comparison” in a list of techniques to be used to improve surveillance networks.

“These security steps appear in American movies,” said Xie Yinan of Megvii Technology Inc., a Chinese tech startup that sells facial-recognition systems to private and public enterprises. “But in China, it’s actually being used in real life.”

Chinese government agencies including the public-security ministry, the central planning agency and the ministry in charge of information technology either declined to comment or didn’t return calls.

On Chongming Island near Shanghai, a new running course has been outfitted with a facial-recognition system to ensure runners don’t take shortcuts through the foliage during timed competitions, said Chen Zhixian, a manager at the company that built the track.

Jogger Chen Xiang, 42, said he was aware of the system but wondered why it was needed. “Running is an activity, and we’re just out here to have fun,” he said.

Facial-recognition cameras are being used in China for routine activities such as gaining entrance to a workplace, withdrawing cash from an ATM and unlocking a smartphone. A KFC restaurant in Beijing is scanning customer faces, then making menu suggestions based on gender and age estimates. One popular park in the capital has deployed it to fight toilet-paper theft in restrooms, using face-scanning dispensers that limit each person to one 2-foot length of paper every nine minutes.

A world where everyone can be tracked by their face wherever they go is still a long way off, and will require much better algorithms and cameras than currently exist, said Anil Jain, the head of Michigan State University’s Biometrics Research Group.

China is moving in that direction, abetted by a vast surveillance network. Industry researcher IHS Markit Ltd. estimates China has 176 million surveillance cameras in public and private hands, and it forecasts the nation will install about 450 million new ones by 2020. The U.S., by comparison, has about 50 million.

It isn’t known how many cameras in China are enabled for facial recognition, but any high-definition camera can potentially be linked to such a system.

The sprawling camera network has spawned anxiety in some quarters. One night in early May, government cameras in the coastal city of Wenzhou kept watch as dozens of people filed into a Protestant church for an emergency meeting called following the installation of the cameras near and inside the church compound the previous month.

The growing appeal of religion in China has unsettled the country’s officially atheist leadership. Three years ago, authorities began removing crosses from many places of worship in Wenzhou, and last year China’s State Administration of Religious Affairs ordered major churches, mosques and temples to be “fully covered” by surveillance cameras. Cameras were installed at the Wenzhou church holding the meeting and at others, including some trained on pews.

In an interview before the meeting, the pastor said local authorities told him the video feed went to police headquarters. “I assume the cameras have facial recognition. Why wouldn’t they?” he said. “I have Communist Party members and prominent business owners in my congregation. If they think their faces are being scanned when they walk through the door on Sunday, of course they’re going to stop coming.”

Police authorities in Wenzhou declined to comment on church surveillance.

Elsewhere in China, an outspoken government critic said in an interview he had been tracked and detained by police while traveling in southwestern China despite taking steps to cloak his whereabouts by using an anonymous SIM card in his phone and traveling on a fake ID.

Police refused to say how they found him, he said, but in previous encounters authorities told him they had facial-recognition systems. Local police once even bragged about their ability to track him, he said. “They told me that whenever I walked through certain intersections, a computer system would alert them to my location,” he said.

Facial recognition works by breaking down a face into a series of measurements and using them to create a template that can then be compared with others in a database.

Early systems could only compare two photos taken in relatively ideal conditions. The application of artificial-intelligence techniques such as deep learning, which uses software to mimic the way neurons in the brain process information, has revolutionized the technology. Algorithms can now pick out and manipulate patterns on their own, making it easier to detect and identify faces turned to the side, smiling or frowning, or weathered by age.

China has access to immense amounts of data—photos uploaded by the country’s more than 700 million internet users and a centralized image database of citizens, all of whom must have a government-issued photo ID by age 16.

This year, China set up a government-funded laboratory to push the development of facial recognition and other forms of artificial intelligence. China hopes to become a leading innovator in those technologies.

“The things we’ve been able to do in this space surprise me, and I’ve been in this business a long time,” said Lin Yuanqing, a computer scientist for internet search company Baidu Inc. and a member of the team overseeing the lab.

The Chinese startups say their best customers are local police bureaus, which are under pressure from central authorities to identify and squelch threats to social stability.

At a recent security-equipment conference in Chengdu, displays of facial-recognition systems were popular. At one booth, a promotional video by Intellifusion Technology Co. showed a police officer directing a squadron of facial-recognition drones for crowd surveillance. “That’s a little ways into the future,” said Huang Fan, a marketer for Intellifusion, the company that installed the jaywalking-detection system in Shenzhen.

The company’s current systems can track an individual’s movements inside a building through facial recognition and alert authorities if that person attempts to access restricted floors.

In May, facial-recognition systems were used at the Belt and Road Forum hosted by President Xi Jinping in Beijing to promote old Silk Road trade routes. At entrances to the event, paramilitary police stood next to face-detecting video consoles linked to cameras trained on the doors.

“It’s really advanced,” a guard said as the system snapped images of two people who had approached the screening area. In an instant, the screen pulled up their names, photos and profiles, verifying them as invited guests.

Several dozen Chinese police agencies are either testing or using facial-recognition systems, according to facial-recognition firms and state media reports.

In Chongqing, two systems identified 69 criminal suspects during the first 40 days they were in use last year, according to Xu Li, a co-founder of SenseTime Group Ltd., which provided the systems. Mr. Xu showed a letter from the local police crediting it with the detention of 14 suspects.

During the Group of 20 international summit in Hangzhou last fall, Megvii and other firms worked with local police. Surveillance cameras scanned the faces of pedestrians, which an artificial intelligence system checked against a list of criminal and terror suspects. Police were alerted each time the system found a match, leading to the detentions of more than 60 people over a month, according to tallies from the companies.

Police in Hangzhou, Chongqing and several other cities identified by companies and state media as using facial recognition didn’t respond to requests for comment.

For 33-year-old Fu Gui, the technology proved life-changing in a positive way. He was 6 years old when he was kidnapped from his village in Chongqing and sold to a family in faraway Fujian province, according to Fu Guangyou, his aunt and caretaker at the time he was abducted. Years later, he provided his photo at age 10 to a nonprofit group that reunites stolen children with their families. His aunt says she contacted the same group a few years later, submitting a photo of Mr. Fu at age 4.

Early this year, the nonprofit got access to Baidu’s facial-recognition program, which matched Mr. Fu’s photos.

“I immediately called his father,” Ms. Fu recalled. “Fu Gui’s dad didn’t even believe me. He had given up hope.” Mr. Fu, who was reunited with his family, declined to comment.

Developers of facial-recognition systems also pitch them as an alternative to keys, credit cards and ID cards.

China Merchants Bank allows customers to scan their faces instead of using bank cards to withdraw money from about 1,000 ATMs. A mobile affiliate of Ping An Bank uses facial recognition to authenticate a borrower or investor’s identity over the internet.

“We won’t need to remember another password,” said Xu Bing, a SenseTime co-founder and vice president. “All you’ll need to do to unlock your phone or log in to an account is scan your face.”

SenseTime’s Beijing showroom gives an idea of where things are heading. In the lobby, a face-detecting console estimates for visitors their age, gender, mood, attractiveness and closest celebrity resemblance, while also serving up ads based on those characteristics. The company also displayed a system it says can use camera networks to track a person’s movements around a neighborhood.

Still to come: a police car with a roof-mounted camera able to scan in all directions at once and identify wanted lawbreakers. Researchers at the University of Electronic Science and Technology of China in Sichuan province have developed a working prototype. “We’ve tested it at up to 120 kilometers per hour,” said Yin Guangqiang, head of the university’s security-technology lab.

A national facial-recognition system is still years away, but state-run media reports that Chinese police departments already are making arrests using the technology. At least five cities are using it to identify jaywalkers.

Jaywalkers in China are typically subject to small fines, but authorities in the southwestern city of Fuzhou are using facial recognition to identify offenders. Authorities have published the names of jaywalkers in local media and have said they notified the employers of certain offenders.

Jiang Hui, a young Fuzhou resident, recently rode his electric scooter through a red light at a crosswalk. He said discouraging jaywalking is reasonable. “But sending the information to your company?” he said. “What are they going to do with it?”

—Kersten Zhang in Beijing and Junya Qian in Shanghai contributed to this article.

Write to Josh Chin at josh.chin@wsj.com and Liza Lin at Liza.Lin@wsj.com

Appeared in the June 27, 2017, print edition as 'China Tracks Faces to Shape Behavior.'