AI Has Broken The Vulnerability Disclosure Model

Peter Garraghan, Founder and Chief Science Officer, Mindgard.

getty

​AI, when you boil it down, is software. Like all software, AI has vulnerabilities for bad actors to target. Unfortunately, the process of flagging vulnerabilities to AI providers is rarely simple.

It isn't necessarily easy to get in touch with providers directly, and some are more receptive to feedback than others. Perhaps most worryingly, findings related to AI model issues don't always fit neatly into traditional threat categories, giving providers the latitude to decide for themselves whether a repeatable failure should be considered a vulnerability, whether it warrants action and whether to even inform their customers.​

With an increasing number of businesses incorporating AI into their daily workflows, this is a serious concern. "Move fast and break things" may sound great to businesses pursuing innovation and disruption, but someone is always left to pick up the pieces.​

As AI usage expands, providers can't afford to continue pushing security down the priority list. It's a problem that isn't going to go away, and every provider must recognize that ignoring and downplaying AI vulnerabilities has a negative impact on the industry. By placing greater emphasis on security and embracing vulnerability research, providers can build a more responsible foundation for the future of AI.​

AI vulnerability disclosure is dangerously inconsistent​.

When vulnerability researchers identify a potential issue, they'll document it and contact the appropriate vendor. Unfortunately, it can be difficult to contact AI vendors directly, breaking the traditional and expected disclosure model.

Some have a web form, or an email address, or a request to contact a bug bounty vendor. Some have no process at all. Even when contacted successfully, responses will vary. Some vendors may respond promptly, and others not at all. The report might go to someone who doesn't understand the problem (or worse, someone who doesn't consider it a problem). When an AI vendor chooses to remain silent, there's often no way to know why.

This is the core issue with the current vulnerability disclosure model. Industry standards have yet to coalesce around a consistent set of principles regarding what qualifies as a vulnerability that needs fixing, how disclosure should work and whether customers need to be notified. While some AI providers recognize that showing the public they're always searching for and addressing potential issues is a net positive for their reputation, others might see vulnerability disclosure as a threat to their public image.

Unfortunately, some of these companies might believe there's little incentive to take action until a news reporter takes interest or a high-profile incident occurs.

How are attackers exploiting AI vulnerabilities?​

Amid all of this, AI-related security incidents are becoming increasingly common. AI is allowing attackers to conduct automatic reconnaissance, engage in sophisticated impersonation attacks, generate polymorphic malware and enhance their operations in countless ways. Attackers are also targeting AI solutions themselves with prompt injection attacks, poisoned datasets and other increasingly advanced tactics.

These attacks are also more nuanced than traditional security threats. The potential fallout isn't limited to stolen data, exposed systems or service outages; they can also create content and safety issues that don't fit neatly into existing threat categories.

While content and safety issues aren't always clearly defined, they tie directly into an AI's specific capabilities. AI can generate images, audio, video and other content. Without proper guardrails in place, these tools can be manipulated into producing disturbing content.

Providers might not always be receptive to disclosures that pertain to these issues. ​With the consistent and intense pressure they're under to ship new capabilities, deliver more innovative products and capture more market share, it's easy to understand why some of them could be tempted to downplay and ignore content and safety issues—especially when their impact is difficult to quantify and the responsibility to manage them is nebulous at best.​

This creates obvious questions around vulnerability disclosure. Is the current broken system a bug, or is it a feature? Are some AI vendors deliberately making it difficult to contact them directly, using bug bounty platforms and third-party disclosure marketplaces as a buffer to contain, delay and quietly ignore issues instead of addressing them? By creating a layer of abstraction between themselves and security researchers, are AI model providers establishing a system that allows them to avoid public scrutiny even as they allow known vulnerabilities and exposures to persist?

If this is true, security researchers are being intentionally sidelined in favor of continued growth.​

We must create a responsible and sustainable future for AI​.

The problem here isn't solely that many AI providers lack an effective process for reporting potential vulnerabilities. It's that providers could feel they're being actively incentivized to downplay vulnerabilities, even at a time when attackers are constantly looking for opportunities to exploit AI security and safety issues for their own gain.

Security researchers are searching for those same dangers. Their work plays a critical role in allowing AI model providers to stay one step ahead of attackers—but only if they're willing to treat researchers as partners rather than adversaries. Acknowledging and addressing vulnerabilities is a sign of strength and reliability.​

Vulnerability researchers want AI to succeed, but they want it to succeed safely. Fixing bugs, establishing safeguards and remediating vulnerabilities make AI solutions better, safer and more effective. By demonstrating a clear desire to identify and remediate potential dangers, providers can build trust with their customers and establish greater credibility in the market.

The current vulnerability disclosure process is broken, but it doesn’t have to be. Fixing it just requires a commitment to security and responsibility as well as a willingness to embrace a disclosure process that has served the technology industry well for many decades.​

Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?