Are you fluent in prompts and embeddings? Here’s a generative AI primer for busy executives

When new technologies explode onto the scene, they often produce bursts of not only apps and use cases, but also lingo. 

With its profound potential and rapid advances, generative AI poses this vocabulary challenge on an epic scale. It’s new, powerful, complex, and developing fast enough to be strategically urgent—a superfecta that few recent technologies have managed. 

But don’t worry: we have you covered. Below, we’ll define key terms and concepts you should understand to be conversationally fluent in generative AI conversations—and to start making decisions about this exciting technology for your organization. 

Generative AI: it could change everything 

• Generative AI refers to AI that can find complex relationships in large sets of training data, called a corpus, then generalize from what they learn to create new data, including original illustrations, blog drafts, answers to questions, and more. These generalizations enable AI models to perform tasks for which they were not expressly trained (i.e., zero-shot and few-shot learning), and they let humans input prompts that the model uses to generate new outputs, from text, images, and video to music, code, and even formulas for chemical compounds.  

How it affects your business: 
Generative AI is an emerging technology, but organizations are already exploring a range of disruptive and broadly applicable use cases, including accelerating content creation and software development, improving customer service through personalized self-help interactions and chatbots, unlocking new ways to explore and analyze information, and even aiding in drug discovery. 

These examples are likely just the beginning. Use cases grow by the day as the technology continues to mature and become more accessible to developers, businesses, and governments. The machine learning techniques underlying generative AI, such as the Transformer architecture developed by Google Brain, represent large leaps over earlier AI technologies, helping them excel at discerning long-range dependencies, such as a word at the end of a text being related to one at the beginning, or how variables in protein structures relate.

This capability has flung open the gates for more advanced use cases and AI problem solving, and it’s a core reason generative text has rocketed past simple autocorrect and autocomplete tasks, now able to do things like ingest documents and answer questions about them or use existing content to create new content with the same tone. 

Thanks to these cutting-edge advances, we’re seeing an explosion of AI-powered applications that can respond to natural language instructions, create human-like outputs with unprecedented precision, and help humans solve complex problems. In time, generative AI has the potential to impact every business, and to fundamentally change how people interact with machines, information, and one another.

Models: your starting points for custom generative AI applications

• Large models, or LMs, are the engines that power generative AI. Large language models (LLMs) are a common type of LM focused on text, but increasingly, cutting-edge LMs are multimodal, able to generate not only text from text prompts, but also images from text, images from images, and more. LMs typically include billions or even hundreds of billions of parameters, which refer to the size and complexity of the model. Generally, larger models have been more capable but also more expensive to train and run. This trend is evolving, however, as smaller models become more efficient and powerful. 

• Foundation models are LMs that can be leveraged via APIs and developer platforms for downstream tasks, such as building custom generative AI applications. Some organizations develop foundation models for their internal technology platforms, while others leverage third-party options from open-source projects and cloud providers.
  

How they affect your business: 
Many software providers are infusing generative AI into their established products, making it simpler and easier to integrate the technology into your productivity workflows—generative capabilities are being added to Google Workspace, for example. AI collaborators such as Bard are also enabling new ways to get things done.   

But to get the full benefit of the technology, your organization may also want its own custom generative AI apps, whether for next-generation customer experiences or innovative internal uses. To do that, you need access to foundation models.

Building a model in-house can be time consuming, expensive, and complicated. When organizations train particularly large models, the compute costs alone can easily reach tens of millions of dollars—and that’s without addressing the data overhead and machine learning expertise involved. For these reasons, many companies and governments are exploring how to leverage third-party foundation models, such as Google’s PaLM 2.  

In fact, many organizations may need a variety of foundation models, or at least many customized variants of a core model, to accommodate the variety of AI use cases likely to develop in coming years across their teams and operations. 

For example, some sophisticated use cases may require larger models, more complex prompts, or more voluminous outputs. These use cases may involve processing more tokens, which are the LM’s smallest unit of data, such as a single word or phrase. Tokens are also a key factor in the context window, which not only defines how much prompt information the model can consider before it starts to forget things, but is also one of the variables that will inform which models you choose, how you customize them, and how expensive they’ll be to use. 

For example, if you want to create a VR game in which character dialogue is generated in real time in response to whatever the player says into a microphone, that will probably require a lot of tokens generated by an advanced foundation model, among many other things. But if your application is a retail chatbot that can help shoppers browse inventory, make purchases, or process payments, a relatively lightweight LM that outputs clear, succinct, token-efficient responses can be a powerful way to optimize cost while delivering effective, delightful experiences.

Putting foundation models to work in your business

• Prompt design, also called prompt engineering, is the process of crafting prompts that provoke desired responses from the foundation model. It can refer to how end users create prompts in generative AI apps, but it also refers to how app makers train the model to behave, setting baseline instructions before end users are ever involved. For example, developers and prompt engineers might tell the model how to break complex prompts into smaller tasks, helping it to maintain the right train of reasoning (i.e., chain-of-thought prompting), and they can configure variables like temperature, which controls whether responses prioritize accuracy or creativity. 

• Parameter-efficient tuning,  is a low-cost customization method in which the foundation model is fed examples to inform its output but not retrained. 

• Fine-tuning lets organizations deeply customize the foundation model by further training it on new data—not just giving the LM a few examples as in parameter-efficient tuning, but rather updating its training with tens of thousands of new data points, resulting in changes to the weights in the model itself. This is useful if you’re pursuing highly differentiated generative AI use cases or those whose outputs involve specialized results, like legal or medical vocabulary. 

• Reinforcement Learning from human feedback (RLHF) is a technique that lets organizations fine-tune foundation models with a reward model aligned to human feedback. 

• Embeddings represent data as vectors: a string of numbers that describe the dimensions of a given piece of data, show relationships across data, and can be processed by LMs. An image might be expressed as a vector with numbers for the color of each pixel, for example, or a vector for the word “cat” might include numbers that represent dimensions like “mammal,” “four-legged,” “domesticated,” and so on. LMs can use vectors to understand relationships among elements in the data, such as “cat” and “tiger” being similar in that they’re genetically related but different in that only the former is domesticated or commonly kept as a pet. These relationships help models output more accurate predictions and let organizations build recommendation engines, classifiers, and other sophisticated generative AI apps. 

How they affect your business: 
To make custom generative AI apps, your organization will usually need to customize the behavior of foundation models, whether that means teaching them new skills for specialized use cases or simply ensuring that generative customer service chatbots provide accurate, on-brand responses. Different levels of customization are possible, some of which can be performed by upskilled knowledge workers or developers, and some of which require machine learning expertise. 

For example, with Google Cloud’s recently-announced Generative AI support in Vertex AI update, if a team wants to prompt tune a model for marketing content creation, they can get started by simply uploading brand documents, press releases, social media posts, and other assets. Likewise, if a team wants to build internal enterprise search apps, they could do so manually with embeddings, a vector database, and a foundation model—but in many cases, they could also get going more quickly with a product like Google Cloud’s Gen App Builder, which streamlines the processes of selecting data sources to investigate and combining foundation models with semantic search capabilities. 

Customization requirements also inform which models or vendors your organization should choose. Whether a foundation model is backed by enterprise-grade platform capabilities for tuning and built-in security and privacy can greatly impact ease-of-adoption.

Start exploring

With these concepts now internalized, you're well positioned to start exploring generative AI platforms, experimenting with them, and devising strategies for your business. To help you get started, be sure to check out our series "The Prompt" to explore some of the best use cases organizations can pursue while ramping up their generative AI strategies, as well as some of the misconceptions and missteps to avoid along the way.