Your Fraud Stack Was Built For Humans: Now What?

Yinglian Xie is CEO and co-founder of DataVisor, a leading fraud detection company with solutions powered by transformational AI technology.

getty

​I've spent years helping financial institutions detect fraud. Until recently, the industry has been focused on one main question when it comes to verifying transactions: Is there a real human behind this request? This is usually followed by a second, arguably more important question: Is this human actually who they claim to be? But now that AI agents can complete transactions, we must ask a different question entirely: Is this request legitimate and authorized?

With this question, it doesn’t matter if the transaction is coming from a person, a bot or an agent. What matters is whether this transaction was even sanctioned in the first place, who authorized it and what boundaries it was made within. ​

The entire fraud prevention ecosystem as we know it was built on the assumption that there was a human on the other end. There’s only a finite number of people, so that was a manageable task. When the industry added a second question to verify whether each human was who they claimed to be, the verification burden multiplied. With each person having multiple digital identities across platforms and services, the math got harder, but it was still manageable. ​

AI agents represent another, more dramatic multiplication that’s already taken hold. Any single person might have 10 agents accessing their accounts and acting on their behalf at any given time. Unlike human identities, agents can be created for a task, changed and retired, sometimes all within a day, leaving us with a moving target of an indefinite number of identities to verify overnight. Current authentication infrastructure was built around a one-to-one relationship between a verified human identity and an action, but that assumption no longer holds.​

The industry is responding to this new reality with frameworks focused on agents declaring themselves as agents—so you don’t need to determine whether you’re dealing with a human or a machine. But knowing that an agent is who it says it is only solves part of the problem. The harder question is what that agent is actually allowed to do—or the bounding box that governs how far an agent can go. Think of authorization as the key that gets you in the door. The bounding box is everything you have access to and the rules about which rooms you can enter, what you can touch and when you need to ask permission before going further. Without it, saying yes to an agent is like writing a blank check.​

One senior AI research director found this out the hard way when her AI agent deleted her entire email inbox. She had authorized the agent. She had given it access. But she had no way to stop it once it started, and she eventually had to unplug the machine.

For financial institutions, the stakes are even higher. A $10,000 transfer, a bulk account change, a large-scale data action: These are not situations where you can afford to unplug the machine after the fact. The bounding box has to exist before the agent acts.​

But a bounding box isn’t enough on its own either. When a credit card is compromised, it’s canceled. Agents don’t have an equivalent. If a user's bank detects a rogue agent and blocks it, that agent may continue to push transactions elsewhere because there is no shared kill switch. And that gap raises an important question: Who is liable? Is it the user who authorized it? The developer who built it? The vendor or financial institution that didn’t detect fraud? ​

These questions require a broad, global discussion. Fraud regulation has always been regional, and that's not going to change overnight, but AI agents don't operate within borders, and the rules governing them need to catch up. The industry has a chance to get ahead of this now, before a major incident forces the issue. ​

This brings us back to one question: Should agents be required to identify themselves as agents? The case for it is simple because it would reduce fraud exposure and create an audit trail. On the other hand, bad actors won't comply. Both are fair points. But this industry has never waited for perfect enforcement to move. Know your customer (KYC) became a market standard long before it became a legal requirement. The same can happen here. If pretending to be human becomes a recognized red flag, the market can act on it before any regulator has to.​

The fraud stack was built for humans. That was the right call, until now.​​

Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?