The Importance Of Contextual Data In GenAI—And Where To Find It

Pioneer in the telecommunications industry with over 25 years of experience. Jonathan Rosenberg is the CTO and Head of AI at Five9.

To understand the new data requirements for B2B generative AI (GenAI) use cases—and the challenges many organizations may face in making that data available—business leaders need to understand what “contextual data” is and the types of contextual data needed for each GenAI use case.

GenAI tools like ChatGPT work by taking an input—a natural language description of a task, question or desired content—called a prompt—and producing an output which is its response to the prompt. For example, if you want to produce a poem, your prompt might be “write a haiku about the journey of a pebble from the ocean to the shore.” ChatGPT does a better job on this one than I could:

Tumbled by the waves, Journey carved in salt and sand, Pebble finds the shore.

This is a simple example where the prompt is quite small—just a few sentences. And the task is also quite simple—the creation of a simple piece of content.

When we consider the application of GenAI to B2B applications and use cases, the tasks we want the model to perform are far more complex, and so too are the prompts required to produce them. For example, if we want to build a chatbot that can interact with a consumer to book a flight for an airline, we need to cram a lot of data into the prompt. We require information about the available destinations and flights, as well as the available seats and seat maps. We need to provide access to flight booking policies and procedures, such as cancellations and refund policies. We are also going to need information about the consumer booking the flight—their frequent flier status, their available travel credits, whether they have a card on file. And we will want to personalize this chatbot too, requiring access to the users’ preferences, their travel history, their home airport, even the records of their past conversations with the airline.

All this data that needs to be placed into the prompt—a process done dynamically as the user uses the chatbot—is called contextual data. And in the era of GenAI, contextual data is the new gold. Enterprises seeking to build GenAI-based applications will need to make sure this data is accessible to the new breed of tools being built to create these applications.

We can break down contextual data into a set of categories:

• Instructions And Grounding Rules: This is text that describes the job to be performed (e.g., “you are a booking agent for an airline”) and rules about how to behave (e.g., “be polite, greet the customer when they start chatting”). This contextual data is highly static and is usually typed into the system at the time of setup.

• Policies And Procedures: These are the rules that define how you want the GenAI model to work, the same as you would provide to a human to do the same job you want the AI to do. This content is also static but is usually preexisting and scattered across the organization. Contact centers, for example, often have training materials for new agents containing such policies and procedures. This data must be made available in digital form so it can be brought into the GenAI model.

• Product And Services Documentation: These are facts about the products and services that are to be provided by the GenAI model. This is also static content that needs to be available in digital format, consistent and correct. Consistency is often the real challenge.

• Programmatic Business Data: This is information, accessible via APIs, that contains structured information needed for the task. For an airline, this might be the flight database that contains the list of flights, their arrival and departure times, seat availability and so on. To feed this data into the prompt as contextual data, a software platform is needed that can retrieve this information on demand and use it as contextual data.

• Programmatic User Data: This is information—also accessible via APIs—that contains structured information about the user that is relevant to the task to be performed. In the airline example, this is their frequent flier status, mileage balance and credits.

• Historical Conversations: This is a summary of past conversations or the history between the user and the business, whether they be past interactions with a chatbot or calls with a contact center agent. This contextual data is essential for hyperpersonalization—so that when a user interacts with the AI, they get the impression that “I am known.”

Business leaders seeking to build or implement GenAI for B2B applications need to understand which of these contextual data types are relevant for their application. They need to make sure the data in those sources is consistent, accurate and accessible via programmatic interfaces. And finally, they need to make use of tools that can access those relevant contextual data sources. The best way to do that is to identify some key uses where you want to apply GenAI, and using the categorization defined above, fill in the data sources you need for the use case. This task can be performed by contact center leaders or with the assistance of vendors providing GenAI based bots.

Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?