Security Week

AI Now Outsmarts Humans in Spear Phishing, Analysis Shows

Kevin Townsend April 9, 2025

We knew it was coming and now it is here: AI-powered spear phishing now outperforms elite human-generated spear phishing, with the real inflection point occurring in early 2025. If we add AI’s ability to operate at vastly improved speed and scale, the outlook for AI-assisted social engineering is daunting.

Since 2023, Hoxhunt has run ongoing experiments pitting AI-generated spear phishing against expert red team generated spear phishing; and has found a 55% improvement in AI performance. In 2023, when the experiment started, AI was 31% less effective than humans. By March 2025, it was 24% more effective. (Effectiveness is measured by the number of times the spear phishes succeeded in getting the target to ‘click’.)

The 2023 results were similar to those returned by a separate study conducted by IBM’s X-Force Red, also in 2023. The IBM study found a human phish achieved a 14% click rate against an 11% click rate from the AI phish, confirming that humans were, at least then, the better phishers. Both experiments were conducted with phishes generated by prompt engineering ChatGPT, because at the time, that was the only generally available way to use AI.

IBM’s Chief People Hacker at X-Force Red, Snow Carruthers, suggested a primary reason for AI’s failure to move the needle in 2023 was its lack of emotional intelligence. “Humans understand emotions in ways that AI can only dream of. We can weave narratives that tug at the heartstrings and sound more realistic, making recipients more likely to click on a malicious link.”

But she added, “I think my biggest takeaway is to question what the future is going to look like. If we continue to improve gen-AI and make it sound more human, these phishing emails are going to be possibly devastating.”

That’s exactly what started in 2024 with the arrival of agentic AI and its ability to learn and adapt. Knowing that malicious actors would also switch to agentic AI, Hoxhunt developed its own spear-phishing agent to continue its comparative analysis. It developed an agent it calls JKR (short for Joker) – and the performance of AI-generated spear phishing quickly started to improve against human generated spear phishing.

In 2024, Hoxhunt’s experiments demonstrated that AI-generated spear phishing was closing the gap on human-generated spear-phishing. It had dropped from being 31% less efficient than human-generated spear phishing to just 10% less efficient.

But 2025 is the real shocker. “The acceleration in AI Spear Phishing Agents’ effectiveness in the 3 months between Nov. 2024 to Feb. 2025 has been eye-opening,” writes Pyry Avist, co-founder and CTO at Hoxhunt. Applying a timeline to this evolution, AI was 31% less effective than humans in 2023, only 10% less effective through 2024, but by March 2025, AI was 24% more effective than humans.

Advertisement. Scroll to continue reading.

“It’s no longer theoretical,” writes Avist. “We’ve proven that AI agents can create superior spear phishing attacks at scale. Soon, the phishing-as-a-service market will shift to mass adoption of AI Spear Phishing Agents. Once that happens, the baseline quality and effectiveness of mass phishing campaigns will rise to a level we currently equate with targeted spear phishing attacks.”