One of the hottest buzzwords of this decade is Facial Recognition. It’s the part of applied machine learning that can detect and identify human faces, a problem that has been notoriously difficult for computers up to now. And this has burst open a whole new world of exciting possibilities and challenges for businesses, governments, and individuals alike. If you’re a business leader and have been wondering what the fuss is all about and whether there’s some utility in this new development, we’ve got you covered. In this article, we’ll look at the history of Facial Recognition, its development, current uses, controversies, deployment, and many more facets. By the end of it, you’ll have a solid grasp of what Facial Recognition technology is all about and what its implications are for businesses. Let’s get started!
Evolution of Facial Recognition
The technology has been around for some time for all the hype and media coverage surrounding Facial Recognition. The first serious algorithmic work in detecting faces was the Viola-Jones Object Detection Framework published in 2001. Though a general-purpose framework for identifying objects within images, it was quickly applied to face detection with very good success. The main reason for this algorithm’s popularity was its speed; while the training process was excruciatingly slow, the detection process was extremely fast. As early as 2001/2004, the average desktop computer running this algorithm was able to process a 300px X 300px frame in 0.07 seconds (more here). The accuracy rates, though not comparable to what humans can achieve, were impressive at 90%. However, real progress wasn’t made until the decade 2010-2020, when Convolutional Neural Networks emerged as the best method to perform facial detection. The reason was the availability of raw processing power and gigantic system memories made available through cloud computing by Infrastructure-as-a-Service (IaaS) providers. For the first time in history, computers were consistently beating humans in recognizing faces, especially when a large number of random faces were involved.
How does Facial Recognition work?
Facial recognition is a multi-step process with several specialized sub-systems involved. Here’s what the various stages mean: Detection / Tracking: This part of the preprocessing stage is responsible for identifying and tracking faces in the given image or video file. Once this process is complete, we know for sure that there’s a face in the given input, and it can be processed further. The tracking phase is also responsible for tracking certain parts, particular features, or expressions in a face, should that be needed. Alignment: The problem of facial recognition is compounded because faces in a given image or video do not follow any guidelines. The person may be zoomed-in or out, peeking from behind a tree, or present in a side profile, making the problem of face detection even harder. This is where face alignment comes in: it tells us where in the given image/video the face lines, and what are the contours for facial features. Feature extraction: As the name suggests, during this phase of the process (we’re now in the Recognition stage), the individual features of the face, such as eyes, nose, chin, lips, etc., are extracted in the form that algorithms can use in the next stage. At this stage, the computer has collected enough complex data to tell a face apart uniquely. Feature matching/classification: In this stage, the inputs received from feature extraction are matched against the given database to deduce the person’s identity. This phase is also known as a classification because the algorithm may be needed to categorize faces instead of individually identifying them. Once this process is over, we know for sure whether the given face is part of the database we compared against or not. The final output may also contain tagging, the way we’re used to seeing on Facebook. Deployment considerations: Server-side vs. client-side Facial recognition can work both on the server and the device the user is interacting with. For example, when you upload a photo to Facebook, the algorithms are run on the server side; on the other hand, an ID system that uses your face to unlock the device must run on the client side. So, which one is better? Honestly, it’s not about which one is better. Both server-side and client-side deployments have their strengths; in practice, businesses deploy a hybrid system. The recommended practice is to train your models on the server side, where training data and processing resources are without limit. Once the models have been trained, these can be packaged and deployed on the client-side, which improves the speed of the system as well as maintains the user’s privacy. Sending everything to the server introduces a delay, which can be bad or unacceptable in certain cases. At the same time, keeping everything on the client-side will result in weaker models.
How accurate is Facial Recognition?
Accuracy is not a very well-defined term in facial recognition. The main reason is that it’s a fuzzy problem with all sorts of messed-up inputs (low light, face partially covered by hair, camera quality, etc.) and even deceptive inputs (more on this later!). As a result, the neural networks involved in facial recognition need to be tweaked for the problem at hand, limiting their scope. So, while an industrial facial recognition system might boast of 100% accuracy (which is often the case), the same system may not be even 20% accurate when asked to identify faces in a crowded photo. In one research, a particular type of facial recognition algorithm was able to achieve 98.52% accuracy, higher than the human accuracy of 97.53% achieved in the same test. In another study conducted in forensics, the combination of human judgment and algorithms yielded the best results in some cases. Bottom line — for focused, well-defined applications, facial recognition is the best tool we have.
Where is Facial Recognition being used?
Even in the short period that viable algorithms have been developed, Facial Recognition has found incredibly useful and exciting applications. Some of these are conspicuous, but some are so subtly and fundamentally woven into everyday life that we hardly pause to think about what’s underneath. Facebook is perhaps the most common example of modern facial recognition systems at work. As soon as you upload a photo, the social network is able to detect faces. While some time ago you were asked to tag friends, now Facebook is able to do so on its own. A cool new application by Facebook is the feature of informing users when photos containing their faces are uploaded by someone, even if they’ve not been tagged in those photos. Snapchat makes heavy use of face detection and recognition for many of its features, most notably, the funny filters that are such a rage. For these filters to work, the contours and features of the subject’s face need to be detected perfectly, otherwise, the overlays won’t look realistic. The same goes for Face Swap, another popular feature on Snapchat. In case you’re interested in diving deeper into Snapchat’s capabilities in facial recognition, see here. Uber has been battling privacy and safety concerns for a while now, and the newest weapon in the company’s arsenal is facial recognition. The company has rolled out a new feature where its driver-partners’ identity is verified by using their faces. The company says on its blog that after testing several facial recognition tech vendors, they settled on Microsoft Face API for its high quality. Interestingly, this real-time ID check works well in low-light conditions and is able to detect glasses. With facial recognition proving successful in the wild, it’s easy to predict that it might soon replace other identification methods at educational institutions, hospitals, libraries, etc. Retail crime prevention is a natural extension of the application of facial recognition. The retail industry loses an estimated $45 billion every year to shoplifters and other retail crimes, with very little to counter it with. Now, companies like FaceFirst are helping retailers use facial recognition to detect previous offenders and alert security officers. Police surveillance is beginning to leverage facial recognition like all the other institutions out there. For example, in the UK, the police of South Wales are using cameras fitted on vans to make surveillance of crowds easier. While this newfound superpower in the hands of the police has sparked heated public debates over individual privacy, the police believe it will help them restrict wrongdoers better. As Richard Lewis, deputy chief constable of South Wales police, told the Financial Times: Healthcare recently had an unexpected application, where facial recognition helped detect a rare genetic disorder called DiGeorge Syndrome. The DiGeorge Syndrome appears in about 1 in 6,000 children and results in deformities in several body parts. The healthcare problem, in this case, is more severe for poorer countries, which don’t have the resources to go for expensive diagnosis methods. As such, facial recognition, with an astounding accuracy of 96.6%, offers new hope for victims of DiGeorge Syndrome. In the Airlines industry, facial recognition adoption is picking up, and it’ll soon replace the conventional boarding passes. Currently, there are limited but promising results in helping identify passengers as they leave the country. In fact, the Transport Security Administration (TSA) of the US has laid out a plan for the widespread use of facial recognition-based biometrics.
Controversial uses of Facial Recognition
Technology empowers us, though its good or bad use is up to us. No doubt, then, that something as potent and radical as facial recognition is being put to use in a way that raises concern about fundamental human rights and ethics. The most prominent example of controversial uses of facial recognition is China’s enormous surveillance system that employs an estimated 200 million cameras to keep an eye on its 1.4 billion citizens. The system tracks people and evaluates their actions, constantly updating a metric called citizen score. While there’s some value in having a powerful state-controlled surveillance system (tracking debt-defaulters, for example), most see it as the arrival of the dystopian future George Orwell imagined. It’s a future where governments have unlimited power over the individual, and privacy is non-existent. The second example of the debatable use of facial recognition also comes (unsurprisingly?) from China. This time, the school system adopts facial recognition to ensure students are “attentive” during classes. The new facial recognition system, though not widespread yet, replaces ID cards, library cards, attendance systems, etc., using the student’s face for identification. But the creepy part is that this system monitors students’ attention levels, use of mobile phones, etc., and alerts the teacher when a certain threshold is crossed. While video surveillance powered by facial recognition isn’t exclusive to China—the US has been making efforts to use it to curb gun violence in schools—it’s China that seems to be taking this further than any other country.
Comparison of popular Facial Recognition APIs
When it comes to making use of Facial Recognition, what options do you have? In this section, we’ll look at what’s in common use, and how the various solutions stack up against each other. Before we begin, though: a reminder that these APIs are evolving rapidly, and you’re likely to come across blog posts saying this API lacks this feature or that feature. Don’t make your decisions based on that. Analyze your business needs first, carefully check the features offered, go for a trail, and only then make up your mind.
OpenCV
AI research is a sinkhole with no bottom. Training and perfecting a facial recognition system is hard and is best left to conglomerates with deep pockets and an army of researchers. However, if your needs are simple and you like to have full control—and of course, are ready for the upkeep of a tiny/small engineering team—OpenCV might just work for you. It’s an Open source Computer Vision library that is remarkably accurate and is available for all programming platforms. Here’s a hair-raising example of how you can spin up a face-detection system with Python and OpenCV in 25 lines of code! Now, you may come across some blogs that say OpenCV doesn’t have face recognition. Well, it’s a complete lie, and here’s proof. All in all, OpenCV can be a great choice for your business if the needs are simple and specific. Rekognition is a heavy-duty offering from one of the largest cloud providers out there — AWS. It’s a fully managed, powerful service for the AWS platform, and if you’re already making use of AWS for deployment, Rekognition is probably the best choice. Some of the mind-blowing features offered by Rekognition are:
Real-time analysis (as you upload an image or video to S3)Extensive facial analysis (gender, the color of hair, facial expression, eyes open or not, etc.)Pathing (capturing paths of identified objects in videos)Scene and activity detection (indoors/outdoors, “playing soccer”, etc.)Moderating unsafe content (nudity, for example)
The biggest plus with Rekognition is also the biggest minus — you’ll really struggle to use it with non-AWS services to the point where you’ll just have to give up.
Kairos
Kairos works both on images and videos and has all the nice features you’d expect from a modern face recognition API. It does some of the astonishing features found in Rekognition, but if you don’t need them and are already managing your data, why bother?! Kairos has an on-premise deployment for those who are paranoid about privacy and don’t even want to send data over the wire for processing, Kairos has on-premise deployment, the pricing depends on your use case and can be quite steep.
Google Cloud Vision
Google has chosen to differentiate between its face recognition services for images and videos. The image API is known as Cloud Vision, while the video-focused service is called Video Intelligence. While the image-focused service is pretty similar to what AWS has to offer, the video service has a nice feature of cataloging and searching. This will be useful for companies that have large video archives they might want to analyze or search through.
Azure Face API
With Microsoft taking their cloud offerings more seriously than the desktop ones (finally), the Azure Face API is one delightful offering. It has all the interesting features you’d expect (detection, identification, face grouping, similar faces search, emotion, etc.), and works equally well with videos. Now, this isn’t strictly related to facial recognition, but it’s worth mentioning that Azure also offers a customer computer vision service, which allows you to use your inputs and train models as per your needs. Just like Google’s service, there’s a playground available right on the home page, which makes testing the API a lot of fun! Are there significant differences among the top managed face recognition services? Not really. There’s intense competition in the field right now, and new features are being rolled out faster than pizzas. If you’re tied to a particular ecosystem already, using their own face recognition service makes sense. Otherwise, you might want to choose a different vendor if your needs are specific (controlling your own data, needing plain detection only, etc.). Anti-Facial Recognition Systems Just as some researchers have dedicated their lives to perfecting facial recognition technology, others are busy developing techniques to fool them. One such interesting development is Adversarial Glasses, which look otherwise normal to human beings but have fooled expert facial recognition systems. That said, these glasses are not available in the market yet, though the researchers say that these can be 3D-printed easily. Another interesting development was the launch of ekō glasses on Kickstarter. Although the product now stands canceled, it worked on a remarkably simple idea: plain, everyday sunglasses for $45 that simply reflected the light, causing cameras and video surveillance devices to go bonkers. Just like the cybersecurity domain, “hackers” and researchers are locking horns in facial recognition for the race to perfection. In around 2014, we saw the popularity of camouflage makeup that bestowed invisibility against facial recognition, but are not viable anymore. Will there be an AES encryption of facial recognition? Only time will tell! Is Facial Recognition for you? The kind of business that can benefit from facial recognition is that one that involves people—yes, which means every business out there! While the current uses of facial recognition seem to be championed by governments, large businesses, or tech startups, there’s no reason why your business can’t benefit from it. The possibilities are truly endless when we combine a little bit of creative thinking—greeting and identifying customers in a hotel, locating your friend in a sea of people, finding people with similar faces (maybe to be used as actors), detecting personalities for job interviews (again, we’re just letting the imagination run wild here; there may not be anything substantial in such a study), customizing banking experience when a high-value client walks in . . . There are endless ways to use facial recognition at levels small and large to make your business perform better.
Conclusion
Pretty soon facial recognition will become so widespread and so common that we won’t even notice it (like mobile phones?). The underlying technology has been nearly perfected, but in the real world, it’s not just about detecting faces—it’s about what we can do with that ability. Sounds fascinating and interested in learning more? Check out this brilliant course about Computer Vision.
