Identity And Access Management In The Age Of AI: From Stolen To Simulated

Abhesh Kumar is the CTO of Springline Advisory.

getty

For years, enterprise security operated on a relatively simple assumption: verify someone once, then trust them for the session. Identity was built around signals people believed were dependable: a familiar voice on a phone call, a recognizable email address or the face of a colleague seen in countless meetings. These signals were never perfect, but they were reliable enough to support how organizations worked.​

That assumption is eroding rapidly.​ Deception itself is not new. Fraud and impersonation have existed for as long as commerce. What has changed is the speed, scale and cost at which convincing identity can now be fabricated. AI has dramatically lowered the barrier to generating realistic voice clones, synthetic video and highly convincing writing styles. What once required significant technical expertise and resources can now be produced quickly and cheaply.​

Identity is no longer primarily stolen. It is simulated.​

This creates a fundamental problem for enterprise systems. Many identity and access management (IAM) frameworks were designed to trust the same human signals AI can now reproduce convincingly at scale. A widely reported 2024 incident involving engineering firm Arup illustrates the challenge clearly. A Hong Kong-based employee received what appeared to be an internal request for a confidential financial transaction. Suspicious, he did not act on the email alone and instead joined a video call to confirm the request. On the call, he saw the company's CFO and several familiar colleagues. The interactions appeared normal, and he ultimately authorized 15 transfers totaling approximately $25 million. Only later did he discover that every participant on the call, except him, had been an AI-generated deepfake.​

No credentials were stolen. No systems were breached. The attack succeeded entirely through simulated identity.​

From Stealing Identity To Simulating It

Earlier generations of identity-based attacks focused on stealing something real: passwords, tokens or session cookies. That model introduced friction. Credential theft leaves traces, requires system access and creates moments of compromise that can potentially be detected. The newer model removes many of those constraints.​

Microsoft's Digital Defense Reports documented how AI-generated phishing campaigns now produce emails that convincingly mimic internal communication styles. These often reference real project contexts and replicate the tone of specific individuals. Business email compromise attacks, once easier to identify through awkward phrasing or generic language, have become materially harder to distinguish from legitimate communication.​ The shift is important because the target is no longer just the system. Increasingly, the target is human perception itself.​

Traditional IAM infrastructure was built to answer a specific question: Do these credentials belong to this user? The problem is that this question is increasingly diverging from another, more important one: Is this actually the right person? ​

​In one example, members of the Scattered Spider group reportedly identified an MGM employee through LinkedIn, contacted the helpdesk while impersonating them and obtained credential resets that enabled administrative access to MGM's Okta and Azure environments. The resulting ransomware attack caused an estimated $100 million in third-quarter losses.​

Even when technical systems largely work as intended, what can fail is the assumption that familiarity and a limited set of identity details are sufficient proof of identity.​

Enterprise Impact Extends Beyond Security

The consequences of identity failure are no longer confined to cybersecurity teams. They increasingly affect operations, legal liability, financial controls and institutional trust. Real-world examples highlight the direct financial risks of fabricated authority and how identity spoofing can become an entry point for broader operational compromise.​

At the customer layer, organizations are facing a different type of exposure. In a 2024 ruling involving Air Canada, the British Columbia Civil Resolution Tribunal held the airline responsible for inaccurate information provided by its AI chatbot regarding bereavement fare policies. The airline argued the chatbot was a separate legal entity responsible for its own statements. The tribunal rejected that position.

The broader implication is important: When AI systems communicate on behalf of a business, the business remains accountable for the outcome.​

Why The Human Layer Is A Design Problem

Almost every major incident in this space involves a human making what appeared to be a reasonable decision based on the information available to them. The instinctive response is often to emphasize more training. But training people to detect increasingly sophisticated AI-generated deception is a diminishing-return strategy. The underlying challenge is structural.​

The more important design question is this: How do organizations reduce the number of critical decisions that depend entirely on one individual making a judgment call under time pressure?​

This is where workflow architecture matters. Large financial transactions may require out-of-band verification. Credential resets and privilege escalations may require approval chains that cannot be satisfied through a single communication channel. High-risk actions may require the separation of authority across multiple systems or individuals.​

Rather than relying on any single control, organizations should think in terms of layered resilience across four areas:​

1. Identity Signals: Voice, appearance and communication style should increasingly be treated as supporting context rather than primary proof of identity. Multi-signal verification using device fingerprinting, cryptographic validation, behavioral baselines and network context is materially harder to simulate convincingly at scale.​

2. Continuous Verification: Authentication should not end once a session begins. Unusual transaction behavior, abnormal access patterns or deviations from historical norms should trigger additional scrutiny and friction.​

3. System Resilience: Organizations should assume identity compromise is possible and design accordingly. Segmentation, least-privilege access and controlled blast-radius principles help contain damage when failures occur.​

4. Access Governance: Standing access creates persistent high-value targets. Time-bound, context-aware and revocable permissions help reduce exposure and improve control over sensitive systems.​

The goal is not to eliminate human judgment. It is to design systems that better support it.​​

Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?