How To Address The Increase Of Brittle Tests In The AI-Coding Era

Asad Khan is the Co-Founder & CEO of TestMu AI, an AI-native unified enterprise test execution cloud platform.

getty

​Walk into any engineering organization in 2026, and you'll find a dramatic increase in code production with developers shipping faster than ever, AI assistants generating Playwright tests in seconds and 200-line end-to-end scripts written before lunch.

However, while coverage charts look healthier, the mistakes are scaling faster than the code itself. In fact, CodeRabbit recently analyzed 470 pull requests and found AI-co-authored code contains 1.7 times as many issues overall as human-written code, with logic and correctness errors 75% more common.

Because of these issues, test maintenance now consumes a meaningful share of QA capacity. Today, "developers spend an average of 38% of their week on debugging, verification and troubleshooting, or roughly two working days," according to a recent Lightrun report. ​

Many leaders may point to AI tools as the root of the problem, but I've found it's often the test architecture itself struggling under the velocity of new code. ​

One important thing to remember: Most testing failures aren't actual bugs but flakiness challenges, or tests failing inconsistently due to timing issues, environment drift and brittle UI selectors. In these cases, replacing AI coding tools may slow down the problem, but it won't fix the underlying issue.

​Why AI-Generated Code Made Brittle Tests Worse ​

The traditional brittle test fails for reasons unrelated to actual bugs, such as a developer renaming a CSS class, moving a button to a different container or updating a form field ID.​

I've worked with thousands of engineering teams, and this problem has been around for 15 years. What's new in 2026 is the velocity with AI.​​​ Many organizations that adopted AI-assisted test authoring went from zero coverage to thousands of tests in weeks.

A single UI refactor can break hundreds of tests, leading to engineers spending sprints rewriting selectors. The instinct is to blame the AI or AI-generated code without addressing the testing architecture.

Let's suppose AI generates a test that clicks a button by finding it with an ID number. This process works fine until someone redesigns the page and that ID changes. The button still exists and works for real users, but the test can't find it anymore.

On the other hand, when a button is hidden behind a pop-up or too dark to see in dark mode, the test passes, but the user can't actually click it. Your dashboard glows green while customers leave.​

The Four Pillars Of Modern Test Architecture

Solving brittle tests in 2026 generally requires moving beyond selector-based architecture entirely. Technology leaders should instead establish four strategic capabilities within their testing environments:

Layer 1: Intent-Based Testing

Rather than basing testing on confirming specific elements, the architecture should focus on user intent. One way to do this is with agentic AI, which can allow you to describe ​what you want in plain language—"Fill in the email field and click submit"—and the agent can read the page and find the right elements, whether they've been renamed, moved or restyled. The key here is to focus on behavior, not selectors.​

Layer 2: Visual-Truth Validation

Tests should run against what users actually see, not what the DOM claims is there. If the button is hidden behind an overlay, the test fails. If dark mode breaks contrast, the test catches it. This is the layer that addresses the false-positive crisis DOM-based testing has lived with for over a decade.

Layer 3: Self-Healing With Context

Testing structures must possess systemic contextual awareness. When something legitimately changes, such as the team migrating to a new design system, semantic understanding can re-resolve the intent against the new DOM.

Layer 4: Deterministic Execution​

Generative AI is nondeterministic, meaning the same input can lead to a different output. That's fatal in test execution, where you need the same test to produce the same result every time. Instead, teams can use AI during authoring and analysis time but keep execution deterministic. This can help ensure that tests produce consistent, reproducible results. ​

Escaping The Brittle-Test Cycle

The trap most teams fall into is adopting one of the four layers, usually self-healing, and assuming they've fixed the problem. However, self-healing on top of a selector-based architecture just delays the failure. The maintenance burden gets papered over until the next refactor exposes it.

When building this four-layer architecture, three principles can guide teams escaping the brittle-test cycle:

• Stop treating tests as code. Tests should describe behavior, not selectors. If your tests need updating every time a developer touches the UI, they're coupled to the wrong abstraction.

• Test what the user sees, not what the codebase says. This is the single highest-leverage change. DOM-based tests will continue to produce false positives at scale; visual-truth tests catch the bugs that actually matter.

• Pick architecture over features. Self-healing, AI authoring and visual validation only work if they're integrated. Adopting them as separate tools recreates the maintenance problem in a different layer.

When all layers work together, the system can become resilient. I've worked with teams that have adopted this architecture and cut regression cycles from 10 hours to under one hour.

The Question To Ask This Quarter

The question to ask: Is our quality infrastructure keeping pace with our development velocity?

The teams I've seen escape this trap take specific action: Stop searching for better tools and treat architecture as the real problem. This is the only path to sustainable velocity.​

Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?