​Why Your GenAI Strategy Is Stalling, And How 'Article Scoring' Fixes It

Sagi Eliyahu is CEO of KMS Lighthouse. Leading the company's vision to disrupt the knowledge management market.

getty

Every CEO I speak with right now is trying to solve the exact same puzzle: how to move generative AI out of the sandbox and into production without destroying customer trust. The initial excitement was massive. We all saw the demos where an AI bot instantly answers a complex customer query or drafts a training manual in seconds. It felt like magic.

But the honeymoon is over.

According to a recent Gartner analysis on why GenAI projects fail, roughly half of generative AI initiatives are abandoned after the proof-of-concept stage. The primary culprits are poor data quality and a lack of risk controls.

The hard reality hitting enterprise tech leaders is that large language models (LLMs) don’t actually understand what is true. They understand patterns. If your internal databases are cluttered with conflicting policies, outdated manuals or duplicated files, your AI will simply hallucinate and generate bad answers faster than ever before. You cannot scale an intelligent system on an unintelligent foundation.

The Death Of The Passive Knowledge Base

For decades, companies treated knowledge management like a corporate attic. You threw standard operating procedures (SOPs), compliance rules and product specs into a digital repository and hoped someone would find them when needed.

GenAI completely broke that passive model.

Today, your internal knowledge isn’t just a reference guide for employees; it is the training ground and fuel for retrieval-augmented generation (RAG) systems. In a recent Forrester breakdown of GenAI use cases in knowledge workflows, principal analyst Julie Mohr explicitly points out that AI's success depends entirely on the quality of captured knowledge. If your data is messy, scattered or stale, the AI output fails.

Think about the real-world stakes. If a customer service agent relies on a bot that pulls an outdated billing policy from two years ago, you lose revenue and customer trust. If a field technician uses an AI assistant to repair complex heavy machinery, and that assistant references an internal document containing an unresolved contradiction, you face a massive safety and compliance liability.

Why Hits And Thumbs-Up Metrics Are Obsolete

Historically, knowledge managers measured content health using superficial, reactive metrics. They looked at page views, assuming that if an article got a lot of hits, it must be accurate. Or they relied on passive user feedback, hoping a stressed-out contact center agent would take three extra seconds to click a "thumbs down" icon on a broken article.

That approach is useless now.

When you scale up to thousands of moving products and shifting regulations, manual audits cannot keep up. By the time your team finishes reviewing a batch of compliance documents, the market has moved, and those documents are obsolete again.

Enterprises need to move away from implicit trust. We need an automated, objective way to verify our data before we feed it to an LLM.

The Case For Article Scoring

To fix this, you need to treat knowledge quality exactly like financial risk or search engine optimization. You need a dynamic, data-driven credit score for information.

At KMS Lighthouse, we look at this through the lens of an automated "article score." Instead of relying on human guesswork, AI-driven analytics evaluate every single piece of institutional knowledge across multiple critical friction points:

• Freshness: Is the data current, or has it sat untouched past its operational shelf-life?

• Structural Health: Are there dead links, broken references or formatting issues that will confuse an ingestion engine?

• Contradictions: Does an instruction in Document A directly violate a policy laid out in Document B?

• Clarity: Is the layout readable for both human workers and RAG systems?

• Duplication: Are there three different versions of the same policy floating around in different silos, diluting search accuracy?

By running this scoring continuously, content teams stop playing catch-up. They don’t have to manually review 20,000 articles. Instead, they open a dashboard, see which assets dropped below a quality threshold and focus their engineering and writing resources precisely where the risk is highest.

Why Clean Data Is Your Only Real Moat

When you fix the foundation, the operational wins compound quickly. Contact center agents stop second-guessing their screens. Onboarding timelines shrink because training data is verified. Most importantly, hallucinations in your customer-facing AI drop off a cliff.

The differentiator for businesses over the next few years won't be who bought the most advanced LLM license. It will be who cleaned up their room first. If you want to scale GenAI safely, stop tweaking the prompts and start scoring your source data.​

Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?