Google researchers have developed a privacy application that can instantly detect when a stranger glances at your screen over your shoulder.
Researchers See Jung Ryu and Florian Schroff will demonstrate their shoulder-surfing warning system at next month's Neural Information Processing Systems Conference in California.
The pair haven't released a paper that explains technology, but a video demonstration suggests they've installed a lightweight machine-learning model on a Pixel smartphone that uses its front-facing camera for rapid gaze detection. When the user holds a phone up to chat or view a private video, say, on a train or other crowded place, the algorithm will detect when someone else from behind begins looking at the screen too. The video shows that the algorithm can detect a stranger's gaze within milliseconds.
The video suggests the researchers are using FaceNet, a facial-recognition neural network developed by Schroff and other Google researchers in 2015, as well as GazeNet, a gaze-estimation neural network recently described by researchers in Japan and Germany.
The two Google researchers call their application an 'electronic screen protector', which combines the two areas of computer vision research to provide enhanced privacy when using a big-screen smartphone in public spaces.
The application builds on face-authentication systems already on smartphones by using the enrolled face to distinguish between the user and a stranger, while the gaze-estimation model helps detect the stranger's gaze almost instantaneously.
The system takes just two milliseconds to detect a stranger's gaze, and 47 milliseconds to recognize a face. It also takes 115 milliseconds to detect a face in each frame. "Because of the quick, robust, and accurate gaze detection mobile model, we can now easily identify the face identity and gaze simultaneously in real time," they explain in a brief description.
"Hence, the application, an electronic screen protector, can enable its enrolled users to continue reading private and confidential contents on your mobile device, while protecting their privacy from onlookers in a crowded space such as the subway or an elevator."
Download SafeUM — communicate privately, without advertising and spam.
