How Computer Facial “De-Anonymization” Works and Whether It Can Be Avoided
Modern computer vision has given governments unprecedented capabilities in video surveillance. Authoritarian (or authoritarian-leaning) regimes seem to be making the most of these technologies. People try to fight back against de-anonymization in various ways—from laser pointers to special makeup and accessories. However, while protesters in Hong Kong wear masks and scarves, in Moscow, it’s the law enforcement officers themselves who have started wearing balaclavas en masse. Still, with the emergence of new facial recognition systems, neither approach may be effective for much longer.
SenseVideo, a surveillance system from China’s SenseTime Group, is one of the most expensive startups in computer vision. The system not only detects and classifies objects but also automatically recognizes pedestrians.
Currently, there are over 200 million cameras operating in China, and by 2020, that number was expected to reach 400 million. The Moscow government is also trying to keep up. In June 2019, Moscow’s mayor Sergei Sobyanin announced that more than 200,000 cameras across the city would soon be connected to a facial identification system. Companies IVA Cognitive and NTechLab are competing to develop it. NTechLab is well-known for its algorithm and its now-closed FindFace service.
Experiments with facial recognition in Moscow, conducted by the city’s Department of Information Technology (DIT) together with the Ministry of Internal Affairs, have already shown impressive results. According to police, since 2017, the installation of thousands of cameras near residential entrances has led to the arrest of over 90 wanted criminals. Using video monitoring and facial identification at several Moscow metro stations results in five to ten arrests each month.
Facial recognition isn’t limited to stationary cameras. Systems capable of identifying people in crowds can be mobile. For example, in China, portable recognition systems resembling Google Glass have been tested since early 2018. Similar, though less futuristic, mobile systems will soon be available to Russian police: in May 2019, Zhejiang Dahua Technology and NTechLab (part of Rostec) offered law enforcement a wearable camera with facial recognition. The camera runs Android 6.0.1, detects faces on-device, and sends cropped “portraits” with metadata to a server for identification. According to a source from Vedomosti, police testing has already begun, and competitors like VisionLabs have announced similar devices.
The deployment of such systems could have far-reaching consequences for participants in mass events. According to statistics from pilot projects by the DIT and Ministry of Internal Affairs, fixed cameras alone have helped identify and detain over 150 wanted criminals at public gatherings; with portable cameras, that number could rise significantly.
How Facial Recognition Works
Facial recognition technology consists of two distinct and complex stages: detection and actual recognition. In the first stage, a computer algorithm analyzes video footage and tries to find the area containing a face. This can be done manually if you have a photo of a specific suspect, but most systems work with continuous video streams, using simple automatic algorithms to find faces in the frame.
The main algorithm is the Viola–Jones method, created in 2001, now built into every smartphone and many cameras. It looks for areas where the arrangement of light and dark patches resembles a face. This method is simple, making it fast and resource-efficient, but also easy to fool with makeup (which changes the placement of light and dark spots), a deep hood, a hat pulled over the eyes, and other simple means.
You can test whether you can fool Viola–Jones yourself: try taking a selfie or uploading a photo to a social network—automatically detected faces are usually highlighted with a frame. Even if detection fails, remember that your face can be manually cropped from the frame and sent to the next stage—recognition.
After detecting the face area, the image is converted to grayscale, the position of the eyes is determined, and the face is cropped (the actual contour doesn’t matter for recognition, so standard “ovals” are used). This schematic image is then turned into a numerical vector—a sequence of several hundred parameters distinguishing one person’s face from another’s.
This transformation is the most complex part and is usually handled by convolutional neural networks, which minimize differences in lighting, angle, expression, and other features, reducing the image to a set of traits that are as similar as possible for different photos of the same person and as different as possible for different people.
Once this set of features is found, recognition becomes straightforward: the values are compared to vectors in a database, and the closest match is found. If the similarity exceeds a preset threshold, the system signals a match and provides the user with an identifier and a linked file.
For the system to find someone, their photos and corresponding feature vectors must already be in the database. The more photos, the more accurate the vector, and the easier it is to recognize you. The best sources for these photos are social networks, police and government databases, or company records. Betting companies (which require selfies with a passport), banks offering “pay by face” services, hairstyle and makeup apps, and apps like Msqrd or FaceApp all have huge face photo databases.
Countermeasures: Lasers and Balaclavas
If your photos have never been in any database, you can’t be identified. But this requires leaving no digital traces at all, which is nearly impossible—such as not even getting a passport. Is it possible to protect yourself from identification?
Countermeasures can be divided into two groups: those that prevent algorithms from detecting a face in video, and those that prevent correct recognition.
Protesters in Hong Kong came up with two simple, practical ways to avoid face detection. One is wearing balaclavas and face masks. Note that regular surgical masks won’t help, since key information for algorithms is in the eye area, brow ridges, eyebrows, and upper nose. Standard dark glasses don’t help either. But combining these accessories with a hood can sometimes prevent automatic detection and identification—at least with current industrial systems.
However, in Russia, this method is no longer viable: according to the law “On Assemblies, Meetings, Demonstrations, Marches and Picketing,” participants are prohibited from using “masks, disguises, or other items specifically intended to hinder identification.”
The other method is blinding camera lenses with laser pointers. In Hong Kong, protesters even use them against police, shining them in their eyes and interfering with aiming. An expert in video analytics from a Moscow firm confirmed to Meduza that even bright sunlight can confuse algorithms, and lasers can turn a normal image into a few blurry light spots on a dark background.
There’s indirect evidence of the effectiveness of simple blinding in the design of automatic vehicle classification systems: at toll road entrances, cameras are always duplicated from both sides to avoid errors from head-on sunlight at sunrise or sunset.
Countermeasures: Makeup and Accessories
Besides masks (which are banned in Russia) and laser pointers (which could be considered disobedience or even assault), there are other ways to reduce the chance of identification—methods that exist in a legal gray area. These include makeup, wigs, glasses, and bright cosmetics—what’s called disguised face identification in computer vision. Unlike standard recognition, there are no well-developed, high-precision, ready-to-deploy industrial systems for this yet. Different engineering teams test various approaches, with mixed results.
For example, a report by Russian company Videomax tested the “Face-Intellect” facial recognition software (by Axxon, a division of the UK’s ITV-Group). No fake mustaches, beards, or glasses could fool the algorithms. But wearing a large wig cut recognition accuracy in half. Combining a wig with long hair, a hat, bandages, and fake bruises reduced identification accuracy to just 51%.
Surprisingly, the most impressive result came from patriotic face paint: painting the Russian flag at a 45-degree angle and wearing a “Russia” cap prevented even face detection. The system simply saw nothing. Whether patriotic face paint counts as a disguise is unclear—the Constitutional Court allows face paint for campaigning, but not for hiding identity.
In 2017, Yandex’s Director of Technology Distribution, Grigory Bakunov, invented a more complex method. He considered how neural networks recognize facial features and suggested applying makeup as seemingly random lines and dots. This not only helped avoid identification but could trick the system into thinking it was a different person. Certain makeup patterns fooled the algorithm into identifying the face as someone else. Bakunov and his colleagues decided not to release a service based on this, citing concerns it could be misused.
Bakunov notes two weaknesses: this makeup is hard to apply correctly, and it looks so odd it may attract unwanted attention.
A similar approach was developed by Polish designers at Nova, who created face accessories to confuse identification systems. Their most popular item is a glasses-like frame with two copper circles covering the cheekbones and a copper horn above the nose. This accessory barely covers the face for humans but blocks facial recognition—at least the version Facebook used at the time.
Countermeasures: Glasses and Reflectors
One of the most promising approaches is special high-tech glasses. Some look stylish and don’t attract attention, so you won’t arouse police suspicion and can wear them at rallies. Their effectiveness is especially high against surveillance cameras with infrared illumination.
In 2015, engineers at Japan’s National Institute of Informatics tested glasses with built-in infrared LEDs, invisible to the human eye but able to blind the area around the eyes and nose for sensitive cameras. This simple trick prevented the system from even detecting a face in the frame.
Engineer and designer Scott Urban uses a similar approach. His Reflectacles Ghost and Phantom glasses have frames with many reflectors that bounce back infrared rays from many surveillance cameras (especially those for night mode), turning the face into a bright glowing spot. Another model, IRPair, uses special filters to block infrared light from reaching and illuminating the face for cameras. They also prevent iris scanning and 3D face mapping.
Countermeasures: Multi-Faced Clothing
The most extravagant way to evade “Big Brother” was proposed by clothing designers: smart textiles called HyperFace, created for the Hyphen-Labs NeuroSpeculative AfroFeminism project. HyperFace clothing is designed to overload identification algorithms by displaying patterns that mimic multiple schematic faces. Each pattern targets a specific algorithm, such as the Viola–Jones method in OpenCV or HoG/SVM (Histogram of Oriented Gradients/Support Vector Machines).
In essence, “multi-faced clothing” exploits the old psychological problem of figure and ground. A face stands out against the background, but if the whole background is faces, it’s impossible to pick out and recognize just one. Everything becomes background, and the algorithm fails.
Countermeasures: The Reverse Problem
It’s not just the state that can use facial identification against citizens. Some protesters have started using photos and videos from rallies to de-anonymize police officers who abuse their power. In response, police themselves have started wearing balaclavas, which are banned for ordinary citizens at rallies. This seems like a guaranteed way to stay anonymous. But is it really?
To answer this, we need to clarify the concept of “recognition accuracy,” which is often discussed regarding algorithms. Unfortunately, “accuracy”—the total number of false positives and negatives divided by the number of tests—says little about whether an algorithm can solve a specific task. You need to know the sample size of people among whom the face must be found (and ideally, the cost of errors and the area under the ROC curve).
In mass surveillance—like in the subway or at large events—the false positive rate must be very low (and “accuracy” very high), or the cost of responding to each alert becomes too high. If the search pool can be narrowed (e.g., by profession, gender, age), the accuracy requirements are much lower.
One of the main testing grounds for modern facial recognition is competitions like Disguised Faces in the Wild. Here, teams must identify people among images of faces altered with makeup, masks, or professional effects. There’s no special dataset for balaclavas, but for added difficulty, some incorrect answers include impersonators—so the system must pick out, say, Lady Gaga among actors imitating her.
The latest results show that with a 1% false positive rate, the best algorithms give the correct answer over 90% of the time. If the false positive rate is reduced tenfold, the result is only about 10% worse.
This isn’t enough to deploy hidden-face recognition in the subway today. But if the search pool is much smaller, current methods are likely accurate enough to find potential candidates for de-anonymization, even under makeup or masks. While there’s no ready-made service based on these algorithms yet, one could appear soon.
What to do with information obtained this way is another big question—one of ethics, not technology. The principle of ethical neutrality applies to facial recognition algorithms just as it does to encryption or weapons development.
Prepared by: @iskatel007
Private Detective / Security Analyst / Information Security Specialist.
Telegram: @iskatel007
Wickr Me: iskatel007
Eleet: AB802089
Threema: URT2S2X6
Telegram channel for business security information services (data collection, dossiers, due diligence, analytics, consulting, user de-anonymization): https://t.me/joinchat/AAAAAEpnFMcCaUf9czghzg