From Proof Of Concept To Production: Embracing Systems Thinking

Ben Blanquera - VP - Technology and Sustainability, Rackspace Technology.

In November 2022, ChatGPT burst onto the scene, captivating the world with its impressive generative AI (GenAI) capabilities. In the years since its wide deployment, machine learning has demonstrated its impact across virtually every industry in some capacity. Yet, over two years later, many enterprises find themselves stuck in the proof-of-concept (PoC) stage, unable to scale AI solutions to full production.

According to a recent Gartner study, "At least 30% of generative AI (GenAI) projects will be abandoned after proof of concept by the end of 2025, due to poor data quality, inadequate risk controls, escalating costs or unclear business value."

Overcoming these PoC challenges requires a systems-thinking approach to effectively navigate the complexities of integrating AI into organizational workflows. Systems thinking, popularized by Peter Senge in The Fifth Discipline, emphasizes viewing organizations holistically, identifying interconnected parts and understanding how changes in one area impact the entire structure.

For AI initiatives, this means acknowledging that successful implementation isn’t isolated to a single team or tool. It spans the entire organization—impacting data, technology, people and processes. To move forward, organizations must establish a clear framework for improvement and identify the key decisions required to operationalize AI initiatives.

AI In Production Demands More Resources And Greater Alignment

While PoCs can demonstrate AI's potential, they often fall short when scaled to production. The most common reasons are clear: limited datasets during PoCs rarely reflect the complexity of real-world operations, small test groups failing to account for broader user interactions, costs, regulatory challenges and the emergence of new operational requirements. Too often, enterprises confront the reality that taking AI from PoC into production will require significantly greater resources, closer oversight and more alignment than anticipated.

To overcome these hurdles, organizations should think about AI as more than a tool—it’s a fundamental shift in how business is conducted, akin to the adoption of enterprise resource planning (ERP) systems. As with ERP, scaling AI isn’t just about technology—it’s about rethinking processes, mobilizing talent and embedding technology into the core of how work gets done. It requires checks and balances, clear ownership guidelines and an ability to interconnect systems seamlessly.

A mental model to consider is that the deployment of AI at scale is in the form of AI applications and that like all applications, there is a disciplined operating model for their development, deployment, sustaining and decommissioning.

Successfully moving AI applications into production stage depends on an operating model that addresses five interconnected systems: strategy, secure AI applications, data supply chain, AI operations and AI product development. Let’s look closer at each of these:

Strategy

First and foremost, organizations should establish and reinforce policies and accountability structures to ensure that their AI applications are ethical, compliant, explainable, transparent and aligned with their organizational goals. More so than a technical change, AI at scale is a significant organizational change that must be managed and starts with ongoing investments in AI literacy and workforce readiness. Additionally, enterprises need to consider sustainability factors across environmental, economic and social dimensions.

Security And Compliance

Any AI application begins with making critical security decisions about access controls, data classification and labeling and governance models. How does the solution align with business specific objectives and risk tolerance? Equally important is incorporating advanced security measures, including role-based access controls (RBAC) to ensure that only authorized individuals access data and that through monitoring, breaches are promptly identified and addressed.

Furthermore, the architecture must prioritize and adhere to frameworks to ensure ongoing compliance, which includes periodic assessments and compliance tracking.

Adopting An “AI-As-A-Product” Mindset

Designing AI applications also involves a multitude of critical decisions that significantly influence cost, performance and user adoption. Adopting a product-thinking approach establishes a framework that prioritizes the customer perspective, starting with the design and optimization of exceptional experiences and intuitive interfaces.

To deliver that experience, it’s important to think carefully about prompt and context management, data sources being incorporated and their modality (e.g., text, images, audio, video), model types and design patterns. Product thinking also means looking at factors such as product life cycles and feature/cost tradeoff decisions.

AI Operations Roadmap

Once the application is built to align with your PoC strategy, the focus can shift to creating an operational roadmap and ensuring reliability, uptime and monitoring.

Just as achieving fitness goals requires consistent effort and regular visits to the gym, maintaining and improving AI systems demands a long-term, strategic commitment. Are you prepared to invest in the operational processes necessary to ensure the consistency of your data pipelines, models and both large and small language models?

This commitment extends to building the monitoring and automation capabilities that guarantee reliable execution. How will you ensure that you can operate effectively and seamlessly, 24/7/365? Achieving this level of reliability requires robust infrastructure, disciplined operations and continuous oversight.

The key to scaling and sustaining these efforts is establishing repeatable, dependable processes. By embedding consistency and reliability into your AI operations, you set the foundation for long-term success, allowing your systems to adapt, evolve and deliver value over time.

Data Supply Chain

Without quality data, developing effective AI applications is impossible. Beyond implementing strong governance for both structured and unstructured data, enterprises must ask critical questions: How do you define your data quality? Is data provenance essential to ensure the accuracy and reliability of your datasets? Who owns the data that feeds the application?

Enterprises need to invest as much in their data as they do in their AI technology to drive tangible, measurable results. As the adage goes, AI is only as good as the data that feeds it.

Conclusion: The Path Forward

As we look to the future, it's clear that the most successful AI implementations will come from those organizations that can think systemically, adapt continuously and view AI not as a standalone technology, but as an integral part of their evolving business ecosystem.

Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?