The Weaponization Of AI Is Going Mainstream

Patrick Harr is CEO of SlashNext, an authority in phishing protection across all devices.

We've seen an increase in the use of artificial intelligence (AI) for business email compromise (BEC) attacks in the last couple of years, and it’s accelerating rapidly. The bad guys have exploited AI to target the weakest link, which is human security, by creating perfect emails with highly personalized details and then launching them at scale with AI automation.

AI adoption has ramped up steadily throughout 2023 and 2024, and now we should expect AI-based attacks to go mainstream in 2025. In May 2024, the latest Verizon Data Breach Investigations Report (DBIR) downplayed the threat of AI as “less of a culprit vs challenges in large-scale vulnerability management” and more of “a concern on the horizon.” Yet that historical view over the prior year is no longer valid. We believe that next year’s DBIR will focus on the oversized impacts of AI-based attacks for phishing and social engineering.

The AI risks have grown due to 3D phishing, defined as multichannel phishing that involves some combination of voice, video and text that tricks users into giving away credentials, data or money. As ChatGPT has evolved, we've seen a real step up in the quality of the language being used for these BEC attacks. Also, the video resolution has increased tenfold, from 40 to 400 frames per second, making deepfake videos much more accurate and credible. For all these reasons, almost anyone can become a threat actor at little to no cost today. And by harnessing the power of AI in 2025, their productivity will increase by 50 times.

Attackers are already launching new attack strategies based on voice recognition engines used by banks or executive Zoom calls for wire transfers. We saw one video conference simulation in Hong Kong that successfully finagled a financial fraud worth $25 million. In early 2024, an employee of a Hong Kong-based multinational firm became suspicious upon receiving a message from the company’s U.K.-based CFO. But after first suspecting it was a phishing attempt, the worker dropped his guard based on perfectly rendered deepfake versions of work colleagues who he recognized on a video call. Yet it turned out that everyone he saw on that multiperson video conference was fake.

In April 2024, we learned that a deepfake AI-generated audio clip that contained racist and antisemitic remarks was attributed to the principal of Pikesville High School near Baltimore. The police investigation revealed that the school athletic director allegedly created the fake recording in retaliation against the principal.

Another deepfake attack was thwarted in July 2024, when an executive at Ferrari was contacted by the company's CEO via WhatsApp, requesting assistance for a major acquisition. The attacker then tried to further convince the executive through a phone call using AI deepfake technology to mimic the CEO's voice. The attack was ultimately unsuccessful when the imposter failed to answer a personal question the real CEO would have known.

Taking Steps To Contain 3D Phishing

We're now at the precipice of a real 3D phishing problem, and we should expect advanced 3D phishing incidents to become much more prevalent in 2025, not less. If companies don't incorporate AI into their security programs for messaging, email and collaboration, they could become vulnerable to some very damaging impacts.

Cybercriminals are leveraging AI-assisted coding to enable the creation of more sophisticated and elusive malware, AI-generated phishing and scam websites. These AI-created pages mimic legitimate sites, but they contain hidden threats.

Traditional identity verification methods are being undermined by this new technology. In one example, attackers are using AI to create undetectable keyloggers for developers who write software in Python. Neural networks are also being used to generate convincing fake web pages at scale, just as biometric verification systems are becoming more vulnerable to sophisticated AI attacks.

This 3D phishing trend has steadily accelerated as the attackers have sharpened their techniques. They have many new options for BEC attacks through the use of traditional ChatGPT chatbots or generative AI (GenAI) bots trained on malicious large language models (LLMs). These LLMs have intimidating names such as WormGPT, FraudGPT, DarkBard, WolfGPT, EvilGPT and DarkGPT. Such criminal services and kits are offered on the dark web as AI tools that can generate phishing emails and malicious code at scale.

Mitigating all these risks requires a multipronged approach to fight AI attacks with AI controls that can secure email, messaging and communication apps. This AI-based security approach includes implementing strong user password hygiene with multifactor authentication (MFA), along with continuous user training and security process upgrades.

One simple security solution involves the use of more physical analog processes. Companies can overcome some digital vulnerabilities through direct face-to-face communications and personal social interactions. This can be as simple as holding in-person meetings or calling the bank’s customer service line to confirm that a payment request is real. Or if a wire transfer is needed, go directly to the physical bank to make the transaction.

Another promising option to fight back against AI-based BEC attacks involves new anti-3D phishing technologies. In the near future, techniques such as blockchain for digital fingerprinting of content or live streams could provide a good way to solve the trust aspect related to this problem. Bitcoin has proven that peer-to-peer networks can scale with inherent trust to transfer large amounts of financial or informational value.

3D phishing is a security problem like we’ve never seen before, due to the widespread introduction of AI tools. Scams are an age-old problem for society, but until now, scammers never had the ability to scale up their attacks with such ease and low cost. The security industry and the world at large will need to take considerable action to combat this growing threat.

Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?