AI Bots Can Now Solve CAPTCHA as Well as Humans
Researchers have developed local bots that use specially trained image recognition models and can match humans in solving certain types of CAPTCHA. In some cases, these bots demonstrate 100% effectiveness.
Andreas Plesner, a PhD student at ETH Zurich (Swiss Federal Institute of Technology), and his colleagues published a study focused on reCAPTCHA v2, specifically its “street” version, where users must find and mark elements like bicycles, cars, road signs, crosswalks, traffic lights, and so on in images.
A few years ago, Google began moving away from this system in favor of the “invisible” reCAPTCHA v3, which analyzes user behavior and requires almost no human interaction. However, reCAPTCHA v2 is still used on millions of websites and is sometimes employed as a backup if reCAPTCHA v3 assigns a user a low score.
To create a bot capable of handling reCAPTCHA v2, the researchers used a modified open-source object recognition model called YOLO (You Only Look Once). According to them, YOLO is “well known for its ability to detect objects in real time” and “can be used on devices with limited computing power, enabling large-scale attacks.”
After training the model on 14,000 “street” images, the researchers developed a system that could determine the likelihood that an image belongs to one of 13 candidate categories in reCAPTCHA v2. They also used a separate, pre-trained YOLO model for another type of task, where CAPTCHA asks the user to identify which parts of a segmented image contain objects of a specific type. This segmentation model worked with only 9 out of 13 object categories and simply requested a new image if one of the four “unsupported” categories appeared.
Types of reCAPTCHA v2
In addition to image recognition, the researchers had to take other steps to bypass reCAPTCHA. For example, to hide multiple CAPTCHA-solving attempts from a single IP address, they used a VPN. To mimic human activity, they developed a special mouse movement model and used fake browser and cookie data taken from real sessions.
As a result, depending on the type of object being identified, the modified YOLO model learned to recognize individual CAPTCHA images with success rates ranging from 69% (for motorcycles) to 100% (for fire hydrants).
This was enough to bypass CAPTCHA every time, even if it sometimes took several attempts to solve it. Ultimately, the bot learned to solve CAPTCHA better than humans, requiring fewer tasks than a person in similar tests.
Although previous research had already attempted to use image recognition to solve reCAPTCHA, specialists had only achieved 68-71% effectiveness. According to the authors of the new study, increasing effectiveness to 100% “shows that we have officially entered an era where there is no place for CAPTCHA.”
“In a sense, a good CAPTCHA marks the boundary between the smartest machine and the least intelligent human. As machine learning models approach human capabilities, finding a good CAPTCHA becomes increasingly difficult,” the researchers conclude.