Suppose you have built a face recognition pipeline using nothing but OpenCV’s FaceDetectorYN and FaceRecognizerSF classes, with the YuNet (face detection) and SFace (face recognition) models from the OpenCV Zoo. The enrolled user is accepted, other people are rejected, and everything works as expected.<\/p>\n
Now, what happens if the thing in front of the camera is not the enrolled user, but a printed photo of the enrolled user’s face<\/strong>?<\/p>\n Short answer: it gets through<\/strong>. With an ordinary color printout, the cosine similarity clears the match threshold. Holding up a smartphone that shows the same photo works just as well.<\/p>\n We have measured this behavior many times while developing and evaluating our liveness detection technology at Swallow Incubate. The code below is a single file: download the two ONNX models from the OpenCV Zoo (face_detection_yunet_2023mar.onnx and face_recognition_sface_2021dec.onnx), and you can reproduce it with your own face. We used OpenCV 4.x (the 2023mar YuNet model requires OpenCV 4.8 or later).<\/p>\n We create a feature vector from an enrollment image, then keep matching whatever face appears in the camera feed against it.<\/p>\n The threshold of 0.363 is the default cosine-similarity threshold used in OpenCV’s face_detect sample (the sample’s L2-norm equivalent is 1.128).<\/p>\n In cases 2 and 3, simply holding the paper or the phone where the face would be produces the green MATCH box. The halftone texture of the print, and the reflections and moir\u00e9 on the phone screen, do not stop YuNet from detecting the face, and SFace extracts the enrolled person’s features without complaint.<\/p>\n This is neither an implementation mistake nor a flaw in the model.<\/p>\n Face recognition answers one question: “Is this the same person as the one enrolled?”<\/strong> A printed photo and a phone screen both carry the enrolled person’s facial features, intact<\/strong>. So SFace correctly answers “yes, this is the same person’s face.” The system works exactly as designed \u2014 and authentication is defeated anyway.<\/strong><\/p>\n “Is the enrolled person actually present, right now?” is a completely different question. The technology that answers it is called liveness detection<\/strong>, or PAD (Presentation Attack Detection)<\/strong>, and it requires its own dedicated classifier.<\/p>\n You might wonder whether anyone would really print a photo to defeat face recognition. But consider this: the attack material \u2014 a photo of the user’s face \u2014 is often published by the users themselves<\/strong> on social media and the web. The only tools needed are a printer or a phone. Print and screen attacks are therefore the cheapest attacks there are<\/strong>.<\/p>\n That is why putting plain face recognition in front of any system where impersonation pays off \u2014 time and attendance, door access, identity verification \u2014 is risky. We break down the types of attacks, and how much defense each use case needs, in the next article:<\/p>\n
\nThe experiment<\/h2>\n
\n\n
\n \nCase<\/th>\n What is in front of the camera<\/th>\n<\/tr>\n<\/thead>\n \n 1<\/td>\n The enrolled person (live)<\/td>\n<\/tr>\n \n 2<\/td>\n A printed photo<\/strong> of the enrolled person’s face<\/td>\n<\/tr>\n \n 3<\/td>\n A smartphone screen<\/strong> showing the enrolled person’s face<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n Code (C++)<\/h3>\n
#include <opencv2\/opencv.hpp>\n#include <iostream>\n\nint main()\n{\n auto detector = cv::FaceDetectorYN::create(\n "face_detection_yunet_2023mar.onnx", "", cv::Size(320, 320));\n auto recognizer = cv::FaceRecognizerSF::create(\n "face_recognition_sface_2021dec.onnx", "");\n\n \/\/ Build a feature vector from the enrollment image (a frontal photo of the user)\n cv::Mat enroll = cv::imread("enroll.jpg");\n if (enroll.empty()) {\n std::cerr << "Cannot read enroll.jpg" << std::endl;\n return -1;\n }\n detector->setInputSize(enroll.size());\n cv::Mat facesEnroll;\n detector->detect(enroll, facesEnroll);\n if (facesEnroll.rows < 1) {\n std::cerr << "No face found in the enrollment image" << std::endl;\n return -1;\n }\n cv::Mat alignedE, featEnroll;\n recognizer->alignCrop(enroll, facesEnroll.row(0), alignedE);\n recognizer->feature(alignedE, featEnroll);\n featEnroll = featEnroll.clone();\n\n \/\/ Keep matching "whatever" is in front of the camera\n cv::VideoCapture cap(0);\n if (!cap.isOpened()) {\n std::cerr << "Cannot open the camera" << std::endl;\n return -1;\n }\n cv::Mat frame;\n while (cap.read(frame)) {\n detector->setInputSize(frame.size());\n cv::Mat faces;\n detector->detect(frame, faces);\n if (faces.rows > 0) {\n cv::Mat aligned, feat;\n recognizer->alignCrop(frame, faces.row(0), aligned);\n recognizer->feature(aligned, feat);\n double score = recognizer->match(featEnroll, feat,\n cv::FaceRecognizerSF::DisType::FR_COSINE);\n bool same = score >= 0.363;\n cv::Rect box(cv::Point((int)faces.at<float>(0, 0), (int)faces.at<float>(0, 1)),\n cv::Size((int)faces.at<float>(0, 2), (int)faces.at<float>(0, 3)));\n cv::Scalar color = same ? cv::Scalar(0, 255, 0) : cv::Scalar(0, 0, 255);\n cv::rectangle(frame, box, color, 2);\n cv::putText(frame, same ? "MATCH" : "NO MATCH", cv::Point(20, 40),\n cv::FONT_HERSHEY_SIMPLEX, 1.0, color, 2);\n }\n cv::imshow("face auth", frame);\n if (cv::waitKey(1) == 'q') break;\n }\n return 0;\n}\n<\/code><\/pre>\n
\nResults<\/h2>\n
\n\n
\n \nCase<\/th>\n Result<\/th>\n<\/tr>\n<\/thead>\n \n 1. Enrolled person (live)<\/td>\n MATCH<\/strong> (as expected)<\/td>\n<\/tr>\n \n 2. Printed photo<\/td>\n MATCH<\/strong> \u2014 the score clears the 0.363 threshold<\/td>\n<\/tr>\n \n 3. Photo on a smartphone screen<\/td>\n MATCH<\/strong> \u2014 same as above<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
\nWhy does it get through? It is not a bug \u2014 the model answers a different question<\/h2>\n
Face recognition: \"Whose face is this?\" -> a photo of the user still matches (correct behavior)\nLiveness detection: \"Live person or a fake?\" -> tells photos, screens, and masks apart (a separate technology)\n<\/code><\/pre>\n
\nIs this a realistic threat?<\/h2>\n