AI Agents In The Enterprise: Building A Center Of Excellence For Scalable Success

Sudhir Menon is Co-founder and Chief Product Officer at Sema4.ai.

Most technological innovations take a somewhat circuitous path to mainstream adoption, often taking years to get there. Word of mouth, successful use case deployments, employee movement across companies, LinkedIn resume updates and conference talks ensure that the rollout of these technologies is never a complete surprise to CEOs and CIOs. But the quiet release of ChatGPT with little to no fanfare changed all that and sent enterprises worldwide scrambling to figure out how to adopt this new disruptive technology as it threatens to split the world into AI haves and have-nots.

By 2028, agentic AI will be integrated into 33% of enterprise software applications, according to Gartner researchers. That’s up from 1% in 2024. AI agents have transformational potential for enterprises within the next few years, with generative AI (GenAI) providing the tools to build highly efficient, scalable, data-driven businesses. The good news is that adopting disruptive technologies can quickly level the playing field for both established and emerging companies. Additionally, even as GenAI has gained mainstream attention, AI agents have often become the preferred vehicle for realizing value from GenAI.

The bad news is that most companies will get left behind while dabbling with different technologies, indulging in resume-driven innovation, spending R&D dollars and failing to demonstrate meaningful ROI to their stakeholders. That's disappointing because a framework exists for successfully adopting disruptive technologies, including GenAI.

Organizations adopting GenAI in general—and AI agents in particular—should establish a center of excellence (CoE) to drive business transformation. The CoE can identify and support high-impact AI agent use cases across departments, creating scalable frameworks and maintaining quality control for AI-driven initiatives.

The Purpose Of A Center Of Excellence (CoE)

Most new initiatives, especially ones with transformative potential, have no shortage of takers within the organization. Streamlining knowledge sharing and ensuring that the correct set of requirements is being evaluated across the organization is critical to reducing churn and ensuring long-term success.

For example, suppose the organization has policies around how data can be shared with GenAI models. In that case, ensuring those policies are uniformly applied to any product evaluated makes sense. A CoE offers a well-tested approach to successfully adopting technologies and eliminates the “fog of war” problem where everyone is busy trying to create value and has no time to learn from the successes or foibles of other teams trying the same thing.

A CoE also ensures that energy is being spent on solving the right problems. It allows the enterprise to build expertise because technologies and technological approaches tend to shift rather quickly during the early stages of a disruptive change cycle and having continuity is very important.

Building Scalable Frameworks

Getting the first project off the ground is a major undertaking, and if done right, the organization has the opportunity to put in place the proper guardrails when it comes to security, governance, data usage and measuring outcomes (more on that later). However, the real art of the possible is building a framework for the right type of use case.

Within our early customer base, we now see a pattern emerging indicative of sustained success. This pertains to the criteria used for agent project selection. Successful efforts center around workflows that:

• Have traditionally required human expertise and training.

• Have a process that requires unpredictable scaling.

• Can measure the impact of successful outcomes.

Picking consequential use cases where it's possible to use reasoning, as well as the ability to scale on demand and measure success for continuous improvement, are the pillars of a scalable framework for successful AI adoption.

Enterprises quickly learn that business users can drive the end-to-end process of building predictable, autonomous agents. This is a definite break from the past when business users had to rely entirely on developers to automate processes. The use of GenAI throughout the process means that each subsequent agent gets built and deployed faster than the previous one.

Ensuring Alignment And Quality Control

I had a professor who would insist (rather loudly) that as an entrepreneur, one had to be guided by meaningful metrics or else you could never improve anything. For organizations just getting started with their GenAI-led transformation, nothing matters more than ensuring that those efforts are aligned with the company's strategic goals. Success with these initiatives can be the force multiplier for subsequent initiatives. Quality control for agent effectiveness involves measuring agent performance and the value created. At a more technical level, it's about ensuring that each agent has a robust evaluation framework that measures the agent's accuracy under various scenarios.

AI agents powered by GenAI and your enterprise data can transform your organization quickly. Taking a CoE approach to engaging with AI can greatly improve your chances of sustainable success over a long period of time.

Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?