If you were protecting your smartphone passcode from someone lurking over your shoulder, or from unseen security cameras, you might cover the screen as you tap in the PIN’s four or six digits.
But once you’ve unlocked the phone, perhaps you’d let down your guard, and leave the screen in full view—especially if it’s off. That would be unwise, according to researchers at two German universities.
At an upcoming conference on human-computer interactions, they will present a new study that explains how someone armed with a thermal-imaging camera would have little trouble extracting your passcode from the heat signature left on your smartphone’s screen. It even works 30 seconds after you last touched it. In a short video, two of the researchers demonstrate how easy the attack is. A guy enters a PIN to unlock his phone, then turns off the screen and puts it down on a table. He gets up to grab a cup of coffee, as an attacker quietly strides in, points a small handheld thermal camera at the phone for a moment, and walks back out.
What happens next is a little like a higher-tech version of a smudge attack, in which a snooper examines the oily residue left on a screen by a user’s finger to reconstruct the phone’s login passcode or pattern. In a 2010 paper that introduced that method, researchers from the University of Pennsylvania called smudges a form of “information leakage” that can be collected and analyzed with nothing more than a regular camera and photo-editing software.
The smudge attack was surprisingly good at decoding Android passcode patterns, those shapes that users trace on their lockscreens to get into their phones. The streaking in the residue left behind after an unlock can even show the direction the user dragged his or her finger, making imitating the pattern trivial. But for strings of numbers like an iPhone PIN, the smudge attack isn’t quite as useful: It can reveal which numbers are included in the PIN, but not what order they were tapped. That still cuts down drastically on the set of possible passcodes, but finding the real one will still take some guesswork.
This is where the thermal attack excels. Because heat decays at a known rate, a person typing in a PIN with four different digits would leave behind four heat traces of slightly different temperatures: The first digit entered would be coolest, and the last digit would be warmest. If a thermal image contains only three or two heat traces, the attacker can infer that the PIN contains at least one digit more than once. The phone’s exact PIN isn’t immediately clear in these cases, but it can be guessed in three or fewer tries. And if there’s only one heat trace, the attacker knows the PIN is just one digit repeated four times. (In 2011, researchers at the University of California in San Diego used a similar approach to guess at ATM PIN numbers.)
If the image is taken within 15 seconds of a PIN being entered, it’s accurate nearly 90 percent of the time.Deriving a PIN from a thermal image takes more than just eyeballing it. The researchers behind the technique, who are affiliated with the University of Stuttgart and Ludwig Maximilian University of Munich, developed a six-step process for extracting PINs from images.
First, a thermal camera set to capture temperatures between about 66 and 90 degrees Fahrenheit snaps a photo of the target smartphone screen. Then, software converts the color image to grayscale and applies a filter to reduce noise. Next, a two-step operation removes the background entirely, leaving only the heat traces. The main features of the heat traces are then detected and extracted: For a PIN, this will result in one to four circles. From there, the final step analyzes the relative heat of each PIN to determine the most likely order for the passcode’s digits.
The thermal attack also works on Android patterns: It can trace the finger’s path across the screen, and figure out the pattern’s direction based on the relative temperature of the beginning and end.
The technique is shockingly successful. If the thermal image is taken within 15 seconds of a PIN being entered, it’s accurate nearly 90 percent of the time. At 30 seconds, it’s about 80 percent accurate. But at 45 seconds or more, the accuracy drops to 35 percent and below.
For swiped patterns, thermal attacks can guess the right shape 100 percent of the time even 30 seconds after it’s entered—but only if the shape has no overlaps. Introducing one overlap brings the accuracy at 30 seconds down to about 17 percent, and two overlaps reduces it all the way to 0. Overlapping patterns had the opposite effect of PINs with duplicate numbers, the researchers found: Duplicate numbers made the attacker’s job easier, while complex, overlapping patterns made it nearly impossible.
For Android users, then, choosing a pattern that crosses over itself would be the most obvious way to defend against thermal attacks. More generally, any sort of tapping and swiping around that happens after unlocking a phone is enough to foil a thermal camera, since doings so adds spots of heat to the screen that can confuse the attacker.
But Yomna Abdelrahman, a Ph.D. candidate at the University of Stuttgart and one of the primary authors of the research paper, said a more complex system might be able to tell different actions apart, based only on their heat signatures. “If we have a learning algorithm, we can actually differentiate between PIN entry and usage,” she said.
The researchers proposed a few ways smartphone hardware can defend against thermal attacks. Briefly increasing screen brightness to its maximum, or triggering a short burst of CPU activity, could heat up the entire phone and make PIN detection difficult.
Some people may be predisposed to a natural defense: Cool hands make it harder to detect heat traces from PIN entry, the researchers found, because the difference in temperature between the screen’s glass and the finger is less pronounced. Hot hands, on the other hand, may prolong the window of attack.
I usually use a fingerprint reader to log into my iPhone, but when I can’t, I type in a long password that has letters, numbers, and symbols. Since it takes a bit longer to type it in than a four-digit PIN, an attacker would have less time to capture the heat traces after I finish typing—but what if I typed it quickly?
“I would guess that if you are a fast typist that means the contact time is reduced, which will influence the amount of heat transferred,” Abdelrahman told me. “Hence, the heat traces left behind will be less, so still it might be hard to infer the long PINs.”
But if I’m typing quickly, I may be exerting more pressure with each stroke, she said, which could end up increasing the intensity of the heat traces I leave behind. Maybe I’ll just keep my phone in my pocket.
