AI-Generated Phishing Emails Outperform Human-Written Messages

Cybersecurity experts have long debated whether it makes sense for cybercriminals to use machine learning to train algorithms capable of generating phishing emails. After all, mass phishing messages are simple, formulaic, and have already proven to be highly effective. However, crafting targeted phishing emails is much more challenging, and this is where natural language processing (NLP) technologies can help.

At the Black Hat and Defcon conferences held last week in Las Vegas, specialists from Singapore’s Government Technology Agency presented the results of a recent experiment. In this study, they sent two hundred of their colleagues targeted phishing emails—some written by humans and others generated using an “AI-as-a-service” platform.

Both types of messages contained links that were not malicious but allowed the researchers to track how many times recipients clicked on them. To the experimenters’ surprise, the “victims” clicked on links in the AI-generated emails more often than those written by humans. The difference in click rates was significant.

“Researchers noted that AI needs a sufficient amount of specialized knowledge. Training a truly good model costs millions of dollars. But if you use an ‘AI-as-a-service’ platform, the cost drops to just a few cents, and it’s very easy to use—just input and output text. You don’t even need to run any code, just enter the data and get the result. This lowers the entry barrier for a much larger audience and increases the number of potential targets for targeted phishing. Suddenly, every mass email can be personalized for each recipient,” said Eugene Lim, a cybersecurity expert at Singapore’s Government Technology Agency.

Using the OpenAI GPT-3 platform and other “AI-as-a-service” products focused on personality analysis, the researchers created phishing emails tailored to each recipient’s characteristics. Personality-focused machine learning aims to predict a person’s tendencies and mindset based on behavioral data. By passing this data through several services, the researchers developed a pipeline that processed and refined the emails before sending them. According to the researchers, the results were “surprisingly human-like.”

Source