How Leaders Can Set The Rules Of The Road Before Scaling AI Agents

Matt Swann is the former CTO of Nubank and has served in various roles for disruptive global companies like Booking and Amazon.

getty

In enterprise AI, change is the most consistent pattern I see right now.

First, it was LLMs, then it was copilots. Over the past 10 months or so, it has been agentic AI everywhere. Now the conversation is shifting again. Companies are moving from "Do you have AI technology?” to "How does the whole organization shift to be AI-first?" That sounds simple until you get inside the operating model.

Earlier-stage companies have the advantage of building with agentic AI from the outset. Their development approach is shifting from traditional software engineering toward rapid prompting, prototyping and iteration.​​

Larger companies need all of it. They need a strategy that incorporates LLMs. They want to make developers more productive and may already have machine learning competencies in place. They also have a much larger engineering workforce that has to adapt to the change.

As companies get into more agentic development, it’s common to feel a loss of control. I think the leadership question is pretty straightforward: before AI starts moving through more workflows, how do you create enough operating discipline around it?

At Amazon, Jeff Bezos pushed the six-pager because it forced people to think deeply about the problem, the objectives, the obstacles, the solution and the expected business impact. Enterprise AI is at a similar point. The technology is moving fast, and leaders need mechanisms that force clarity before the system gets bigger.

Define the decision model early.

The question goes beyond which LLM a company is going to use. Between different models, coding tools, harnesses, MCP and teams building their own workflows, there’s a lot of noise. No two companies seem to do things the same way. So the question becomes: What are the rules of the road?

I’ve seen the same pattern across the companies I’ve worked with: the tools change, the pressure changes and the work comes back to long-term goals, operating mechanisms and culture.

I tend to start with a few basics: how decisions get made, how teams test those decisions and what success metrics they watch. Flexibility is part of that. In an environment that keeps changing, the system should be able to understand changes and change decisions as seamlessly and autonomously as possible.

That’s the starting point. Before teams scale too many different approaches, define how decisions get made.

Build oversight into the way work happens.

Agents will follow a maturity curve. The more a company builds out the ecosystem and the more trust it has in that ecosystem, the faster it can evolve. Trust has to be built into the way the system operates.

In my experience, human oversight works best when it is designed into the flow of work. The same is true for validation points and governance points. Leaders need a clear view into how the company evaluates outputs, checks decisions and ensures the integrity of the system.

This starts with an architecture that protects data and intellectual property. From there, leaders need governance that spans the entire AI stack, including LLMs, orchestration layers, MCP and the controls that determine how agents operate.​​

I tend to think about parameters and access points as part of that initial control. Things like MCP and permissions become especially important once agents can move faster and create additional exposure points.

Those operational controls are most effective when they're supported by governance at the organizational level, not treated as isolated technical decisions.​ One large travel company I worked with recognized this early and established an AI governance strategy at the board level, much as it had previously approached cybersecurity. They understood that AI represented an operating model shift, not simply another technology deployment.​

When AI is handled as another isolated tool purchase, the organization can miss the operating model changes happening around it.

Instrument cost before token spend becomes the next cloud problem.

​Over the past decade, many organizations have treated engineering capacity as one of the primary constraints on software delivery, with developer productivity and cloud infrastructure representing two of the largest technology investments. As organizations adopt agentic AI, they're adding another variable to manage: token consumption and the cost of coordinating growing numbers of AI agents.

That shifts cost management from a reactive exercise to a more proactive one. Rather than analyzing cloud spend after workloads have already run, organizations can begin evaluating agent behavior, token usage and orchestration decisions much earlier in the development lifecycle. The goal isn't just to reduce costs, but to identify inefficient patterns before they become embedded in production.

I'm already seeing teams experiment with techniques such as optimizing context windows. As agent ecosystems become more complex, organizations will likely need stronger operational controls that provide visibility into agent behavior, keep humans involved where appropriate and help teams manage spending alongside quality and risk.​

Create operating discipline that can change with AI.

Agentic development is going to be disruptive across a lot of segments. The ecosystem will mature and standards will evolve. Companies will keep testing different models, harnesses, approaches and ways of working. That is why the operating model has to stay flexible.

For me, that comes back to rules of the road, humans in the loop, validation points, governance points, success metrics, instrumentation and tooling. It also means a layered approach that looks at product, people, process, data and technology end-to-end.

AI is going to keep changing. The companies that I’d bet on are the ones building enough operating discipline to change with it.​​​

Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?