How To Solve Data Readiness Bottlenecks With AI Agents

Sridhar Ratakonda, Founder of Predactica, specializes in AI/ML solutions and is passionate about leveraging AI to drive business outcomes.

As enterprises start adopting both classic AI (predictive) and generative AI to drive business outcomes, ensuring data readiness for AI applications remains a thorny issue. As per some estimates, 80% of an AI project's effort is spent on preparing data for AI applications.

Enterprises typically use outside consultants and service companies to help manually prepare the data, resulting in expensive consultative costs and delayed outcomes. In addition, these manual efforts are repetitive in nature and companies are not able to apply learnings from prior efforts.

With the emergence of AI agents, companies are realizing the benefits of applying autonomous or semi-autonomous agentic solutions to reduce the repetitive nature of data readiness.

How AI Agents Can Help Data Readiness

AI agents are designed to autonomously or semi-autonomously perform tasks that typically require human intervention. For example, these tasks could include:

• Data Quality Checks: Enterprise data typically is not clean, meaning there will be incomplete data, wrongly formatted data or data that is skewed. For delivering AI applications that have good accuracy, it is important to ensure data is sanitized and ready. AI agents can help automatically detect these issues and provide recommendations to fix them (semi-autonomous) or automatically resolve them (autonomous).

• Data Standardization: Typical enterprise data originates from several sources, each with different terms used to describe the same concept. For example, a product name from Salesforce might differ from the name used in a financial system. This poses a significant challenge for enterprises, as they often invest substantial time and effort in standardizing the data.

AI agents can use concepts like nearest neighbors to normalize this data and use feedback loops to improve it over time. Additionally, domain-specific terminology and taxonomies can be applied to train the agents, optimizing the process.

• Goal-Driven Readiness Score: Enterprise users need a metric to ensure data is ready for the AI applications. By following specific goals tied to the use case and business outcome, agents can come up with a readiness score to ensure data is ready. For example, a retailer might define a readiness goal as ensuring all SKUs are relevant to existing products. In this case, the agent can cross-check the product SKUs to ensure accuracy.

• Automated Alerts: In cases where data compliance is critical, agents can create automated alerts based on the readiness score. For example, if the readiness score falls below a certain threshold, agents can be configured to interact with reporting and ticketing systems to alert relevant users of the issue. This will help companies address data quality issues in real time.

How Businesses Can Benefit From Using AI Agents For Data Readiness

• Reduced Data Preparation Costs: By leveraging automation with AI agents, businesses can establish a repeatable process and pipeline to ensure data readiness. With a well-defined agentic pipeline, new data sources can be validated without expensive manual processes.

• Faster Turnaround Times: With on-demand AI agent availability, turnaround times can be greatly reduced, resulting in faster time to market and a competitive edge. With real-time notifications and metrics, businesses can address issues rapidly.

• Meeting Compliance: Enterprises can create business compliance-specific thresholds and metrics for data readiness. This could include checks for data privacy or readiness. Agents can review all compliance requirements to ensure the data is ready.

Lessons Learned From AI Agent Applications

I've learned some lessons based on the practical implementation of AI agents for customer data readiness use cases, which I've highlighted below.

• Understanding Data Silos: Mid- to large-sized companies typically source data across various functional groups, and this data is often inconsistent. It’s important, early in the discovery phase, to understand the differences in how the data is represented and train the agents accordingly

• Navigating Stakeholder Buy-In: In almost every company, functional groups like to keep data close to their organization and typically do not like to share it outside of that. It is critical to work across the various groups to ensure stakeholder buy-in upfront. Educate them on the importance of a consistent data-readiness vision.

• Continuous Monitoring: Data readiness work is not a one-time effort. As new data comes in and new data sources are added, data quality needs to be constantly evaluated to ensure data is always ready before it can be used downstream.

• Defining Readiness Objectives: Create measurable data readiness objectives that can be used as benchmarks for agents to take appropriate actions. These objectives should be tied to business goals.

Trends In Data Readiness And AI Agents

Data observability is becoming a big trend in the industry with more and more companies adopting AI-based applications. The evolution of AI agents to fully autonomous systems will help companies meet data readiness objectives more effectively.

One key trend in this context is the emergence of use-case-specific AI agents, optimized to address data readiness requirements not only for a particular domain but also for specific use cases. These agents can be trained with business goals in mind.

Another trend is natively integrating AI agents with business applications to realize their full power. Instead of AI agents being standalone, every business application will have an in-built agent mechanism to meet business objectives.

Takeaways

As AI agents mature and evolve into fully autonomous systems, the current reliance on human-in-the-loop processes to ensure accuracy may be minimized, if not fully eliminated. Properly defined AI agents could help businesses overcome key bottlenecks in realizing the full potential of AI. I believe companies that leverage this exciting, evolving technology will maintain a competitive advantage.

Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?