This app keeps prying eyes from spying your password

Researchers have created a smartphone application to combat “shoulder-surfing”—when someone else looks over your shoulder as you enter your phone’s password or other private digits, potentially even gleaning vital financial or personal information.

Every ATM or smartphone user can attest to the discomfort of having a stranger standing close enough to observe a financial transaction—and potentially note a PIN or account number.

Nasir Memon, a professor of computer science and engineering at New York University’s Tandon School of Engineering, explains that the technology, called “IllusionPIN,” deploys a hybrid-image keyboard that appears one way to the close-up user and differently to an observer at a distance of three feet or greater.

The underlying technology blends one image of a keyboard configuration with high spatial frequency and a second, completely different, keyboard configuration with low spatial frequency. The visibility of each image is dependent on the distance from which it is viewed.

“The traditional configuration of numbers on a keypad is so familiar that it’s possible for an observer to discern a PIN or access code after several viewings of surveillance video,” says Memon.

“Our goal was to increase the resilience of PIN authentication without straining the device or compromising user experience.”

“On a device running IllusionPIN, the user—who is closest to the device—sees one configuration of numbers, but someone looking from a distance sees a completely different keypad.” IllusionPIN reconfigures the keypad for each authentication or login attempt.

The research team simulated a series of shoulder-surfing attacks on smartphone devices to test the effectiveness of IllusionPIN at various distances.

In total, they performed 84 attempted shoulder-surfing attacks on 21 participants, none of which was successful. For contrast, they also mounted 21 shoulder-surfing attacks on unprotected phones using the same distance parameters; all 21 attacks were successful.

The team also determined that IllusionPIN makes it nearly impossible to steal PIN or other authentication information using surveillance footage.

Awareness of the threat potential posed by shoulder surfing has increased significantly over the past decade, since the advent of the first smartphones.

While there are no reliable statistics on the prevalence of shoulder surfing attacks, a 2016 study conducted by Memon and Nguyen found that 73 percent of mobile device users surveyed reported that they had observed someone else’s PIN (although not necessarily with malicious intent). A 2017 study of shoulder surfing awareness presented at the ACM Conference on Human Factors in Computing Systems reported that 97 percent of those surveyed claimed awareness of a shoulder surfing incident in everyday life, and that in the majority of cases, victims were unaware that they were being observed.

“PIN authentication is popular for good reasons, namely that it is easy to use and to remember,” says Memon. “Our goal was to increase the resilience of PIN authentication without straining the device or compromising user experience.”

The research team plans to further refine IllusionPIN, with the goal of commercializing the technology.

A paper covering the technology appears in the journal IEEE Transactions on Information Forensics & Security.