Researchers Demonstrate GAZEploit Attack on Apple Vision Pro Virtual Keyboard

Security researchers from the University of Florida and Texas Tech University have demonstrated a new attack called GAZEploit, which can track the gaze of Apple Vision Pro users and determine what they are typing on the device’s virtual keyboard.

According to the researchers, “The new attack can extract biometric data related to the eyes from the user’s avatar image to reconstruct the text being typed using gaze.”

How the Attack Works

Apple Vision Pro allows users to input data on a virtual keyboard while in Persona mode, which is used for video calls, online meetings, and similar activities. The researchers discovered that by analyzing the eye movements of a user’s virtual avatar, it is possible to determine what the person is typing on the virtual keyboard while wearing the Vision Pro headset.

As a result, an attacker could analyze the virtual avatars of victims and remotely intercept their keystrokes.

Technical Details of GAZEploit

The GAZEploit attack was carried out using a model trained on data collected from 30 people. The model used Persona avatars, EAR (eye aspect ratio), and gaze direction tracking to distinguish text input sessions from other VR activities, such as watching movies or playing games.

The direction of the user’s gaze on the virtual keyboard was then mapped to specific keys to determine potential keystrokes, taking into account the keyboard’s position in virtual space.

“By remotely capturing and analyzing video of the virtual avatar, an attacker could reconstruct the pressed keys. Notably, GAZEploit is the first known attack to use gaze information for remote keystroke interception,” the experts explained.

Vulnerability and Fix

The GAZEploit attack was assigned the identifier CVE-2024-40865. Apple addressed the vulnerability in visionOS version 1.3, released in July 2024. With this update, Persona mode is now suspended whenever the virtual keyboard is active, preventing this type of attack.