Why Agentic AI Needs Adaptive Governance To Scale

Bill Wong - AI Research Fellow, Info-Tech Research Group.

getty

​The AI technology landscape is undergoing a shift from simple generative AI applications to autonomous agentic AI. These are systems aware of their environment that can reason, plan and execute multistep tasks with minimal to no human intervention. Agentic AI also introduces new AI risks that traditional, static AI governance programs are ill-equipped to handle.

To navigate this new era, organizations need to adopt an adaptive AI governance program: a dynamic framework that evolves alongside the AI technology it oversees. I will highlight key activities from a seven-stage process I have used with organizations planning to implement agentic AI applications and address the risks that autonomous AI agents can introduce.

1. Contextualized AI Principles: Establishing A Charter For Autonomous Behavior

The journey begins by creating a “charter" for autonomous AI. In an agentic world, organizations must move beyond abstract principles (which usually include safety and security, privacy, explainability and transparency, fairness and bias detection, validity and reliability and accountability) to contextualized AI principles that focus on describing the "how" of implementation.

One example: The traditional fairness principle might focus on biased text or recommendations, while a contextualized agentic principle would also introduce methods to address autonomous bias amplification—where an agent might systematically deprioritize certain groups of people after a series of autonomous actions.

Contextualization translates high‑level principles into enforceable rules, such as defining “safe” operational boundaries that limit what an agent can do without human oversight. The collection of foundational contextualized AI principles forms and defines a constitution for autonomous agent behavior.

2. Organizational Structure: Introducing Digital Personas

A best practice in governing AI agents is the introduction and use of digital personas. Using this model, autonomous agents are treated as "digital employees," each with unique credentials, and each is assigned human supervisors.

This approach acknowledges the autonomy an agent can have and the need to have people or organizations held responsible for their actions. Larger organizations with large volumes of AI agents have implemented a centralized “agentic AI oversight center” to triage alerts and to manage real-time interventions.

3. AI Risk Management And Compliance: Classifying AI Agent Risk

I strongly recommend that an AI risk management program be implemented before any agentic AI applications go into production. The risk management program should also introduce a classification system to identify the risk level of an AI agent and take into consideration characteristics of the AI agent such as its business criticality/sensitivity, authority, recoverability, guardrails, access, adaptability and autonomy. The higher the risk score for any of the agent's characteristics, the greater the need for oversight and risk mitigation procedures to address agentic AI risks when identified.​

For instance, if we examine an AI agent's level of autonomy, the classification system should reflect that the greater the autonomy, the greater the potential risk, and identify agents having no autonomy to full autonomy and include intermediate levels of autonomy:

• Level 0: No autonomy.

• Level 1: Assisted. AI provides recommendations, but a human needs to approve.

• Level 2: Partial autonomy. AI can execute standard repetitive tasks, but a human needs to approve edge or uncertain cases.

• Level 3: Conditional autonomy. AI operates independently over a broader range of tasks, while humans monitor and approve only for higher-impact decisions.

• Level 4: High autonomy. AI performs complex multistep tasks; humans set the goals and review outcomes.

• Level 5: Full autonomy. AI sets the goals and performs tasks independently, and human oversight is not required.

4. Operationalizing Principles: Introducing Policies-As-Code

AI agents operate at machine speed, and traditional manual policy reviews represent a bottleneck for the organization. Adaptive AI governance implements automated policy enforcement with the introduction of policies-as-code: programs that can take remedial action via APIs and algorithmic checks automatically.

An AI risk management program would introduce controls to enforce policies that include real-time guardrails and circuit breaker policies. Circuit breaker programs can automatically interrupt an AI agent’s activities if it should exceed or violate an established risk threshold. Examples would include the creation of an unauthorized entity, attempting to transfer funds without proper authorization or deleting production databases without the necessary sign-offs.

5. Assurance Program: Performing Continuous Verification

The shift to agentic AI also shifts from selective validations performed at a point in time to continuous verification. To enable this, the agentic AI platform needs to implement the necessary monitoring and auditing ecosystem to capture immutable logs of an AI agent’s actions. This will include the reasoning steps (chain of thought) of the model, the sequence of actions performed, the tools called and the interactions the agent had in pursuing its goals.

6. Roles And Accountability: Enforcing Human-In-The-Loop

As mentioned in the earlier stage focused on organizational requirements, a human or organization needs to be identified and held accountable for each autonomous action an AI agent performs. New mechanisms, such as CIBA (Client Initiated Backchannel Authentication, introduced by the OpenID Foundation), can be introduced to ensure human oversight is preserved. A CIBA prompt routes out-of-band approval requests to a human on a trusted device and will pause the agent’s activities until an approval has been received.

7. Adaptability And Resilience: When Governance Evolves Into An Agentic Process

The final stage ensures that the AI governance program continues to adapt. The agentic AI processing model of sense-reason-act-adapt should be embedded into the governance framework as well. Thus, the adaptive AI governance program will resemble an agentic AI application as well. The adaptive AI governance program would be enabled to autonomously seek out new regulatory requirements, update internal compliance policies and deploy new safeguards across the organization’s agentic workforce in real time.

By evolving from static AI governance rules to an adaptive AI governance program, enterprises can transform governance from a restrictive operations unit into a transformation enabler and driver of value for the organization. The seven-stage process provides a roadmap for organizations that want to leverage agentic AI with the confidence that they are delivering an environment that is adaptive, resilient, trustworthy and accountable.

Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?