Ask OCTO: Preparing teams for gen AI, and easing cloud migrations

The motto of Google Cloud’s Office of the CTO is “collaborative, practical magic.” The team is made up of Google Cloud technical experts and former CTOs of leading organizations, all of whom work in the service of helping our largest and most strategic customers tackle their biggest challenges.

Now they’re here to bring that same magic to you.

Our new monthly Ask OCTO series is Google Cloud’s version of an advice column — except the relationships we're looking to improve are the ones in your tech stack. Please submit questions for future columns here.

In this, our second edition, we explore preparing teams to work with AI; approaches for consultants building gen-AI apps for customers; overcoming obstacles in a cloud migration; and picking the right AI approach for your organization.

What is the best way to prepare our teams and workers for AI? - LinkedIn Poll

John Abel, London office: The integration of AI, particularly generative AI, has demonstrated the potential to eliminate repetitive tasks and facilitate scalability through technology. However, it is widely recognized that the combination of people and technology remains the most potent force, raising the question of how to prepare teams and workers for the advent of AI. Here are some of the top considerations I've been sharing with the organizations we’ve been meeting with.

1. Clear communication: Establishing a transparent communication strategy is crucial to internal and external stakeholder groups. The use of AI needs engagement to provide the ability for stakeholders to give feedback and assist in the transition. Failure to communicate clearly can slow up the adoption and transition.
2. Human-AI collaboration: A key element of the communication strategy should focus on how humans and technology will collaborate to generate business value. Humans' capacity for "lightbulb moments" combined with AI's ability to extract novel insights from existing data can lead to remarkable outcomes.
3. Training and upskilling experts: Generative AI reaches its full potential when guided by experts most familiar with the problem or field. Developing experts, particularly in business domains, is essential. While having a technological nature, the most significant benefits of AI accrue to business users and knowledge experts. It should not be treated solely as a technical project.
4. Document AI principles: Following the example of Google's AI principles, every company should document and define its own AI principles to provide clarity on when and how AI is employed. These principles serve not only to inform customers but also to guide employees.
5. Start with internal use cases: With clear communication, principles, usage guidelines, and investment in training in place, companies need a starting point. Initiating AI implementation with an internal use case allows teams and employees to learn, utilize, and comprehend the significance of the technology without the potential risks of a public-facing experiment. Skills such as prompt engineering, which involve understanding the right questions and desired outcomes, are essential for maximizing value.

By adhering to these five simple steps, businesses can effectively prepare teams and workers for the deployment of this groundbreaking technology. A bottom-up approach empowers employees to drive even greater value. The most significant challenge for companies is not technology itself but rather overcoming biases, addressing ingrained beliefs, and identifying opportunities to enhance efficiency and market reach.

I'm an entrepreneur building software for various clients. I would like to know what kind of applications I can propose to clients who are looking to quickly and easily build AI applications, like customer support for a big supermarket, as an example?

Brett Slatkin, Los Angeles office: People might expect that adopting new technologies, such as the latest developments in AI, is the hard part of digital transformation, but the required organizational changes are often just as difficult. Thus, I'd suggest you start by building applications for your clients that enhance their existing workflows, instead of trying to disrupt them. After establishing this foundation, you'll be in a better position to partner with your clients to make larger shifts in how they operate.

Let’s take customer support as one example. Instead of trying to automate it as a first project, perhaps you could start by using generative AI tools to get a clearer picture of customer sentiment. You could compute text embeddings with open source libraries or cloud APIs, and use them to semantically cluster conversation logs. Thanks to generative AI’s unique capabilities, it's now practical for the first time to identify more complex trends. With deeper insights, leaders can create a virtuous feedback loop that increases product and service quality.

Another opportunity to enhance productivity is enabling teams to better share knowledge across their organization. There is often a wealth of internal documents and reports that could increase alignment and collaboration, but people have trouble finding and synthesizing them. You can now use open source frameworks for what’s called retrieval-augmented generation (RAG) — which grounds models in external information to improve accuracy — or organizations can use enterprise-grade cloud APIs for generative-AI-enhanced search. These tools both help improve discoverability for internal information and realize its value in a whole new way.

Whatever you do, my advice is to be diligent in deploying these emerging systems. There are unique risks to consider, different tradeoffs in total cost of ownership, new potential pitfalls, and rapidly changing customer expectations. Helping your clients navigate these questions could be a great way to build trust and unlock future business.

I'm a mid-senior level colleague in a bank, and we're on a pathway to migrate all our data into the cloud. Everything feels a lot harder and more drawn out than it should. Any thoughts on how to tackle this, or mistakes big corps have made that could be avoided?

Gemma Gariga, Sunnyvale office: Large-scale cloud migrations within the highly regulated environment of the banking industry are rarely straightforward. The understandable desire for a swift transition can often clash with the need to untangle legacy systems, address security requirements, and manage the sheer volume of data involved. The common missteps I see include setting overly ambitious timelines and milestones, underestimating the need for specialized cloud expertise, and assuming that a "lift-and-shift" strategy will work for all systems. It is also critical to avoid treating security as an afterthought and to proactively manage the impact of change on the employees.

Successful cloud migration calls for a well-orchestrated approach, always keeping the business goals as the guiding principles. Start with a carefully selected project for a non-critical workload — this allows you to identify pain points, gain experience, and build internal confidence. Apart from showing early progress, it can naturally bring different parts of the organization together to deliver on early success.

With more knowledge, a tailored cloud strategy is crucial. It is important to assess workloads individually to determine optimal cloud deployment models (public, private, hybrid). No one solution suits everything, so develop a roadmap with phased milestones that acknowledge the time needed for refactoring applications and ensuring robust data governance.

Last but not least, prioritize building strong cloud expertise — either through upskilling existing teams or partnering with specialized providers. The cloud isn't just a new home for data; it calls for a shift to cloud-native practices. It’s important to choose the right partners who not only offer competitive pricing but also demonstrate deep understanding of banking compliance, robust security protocols, and outstanding support. Remember, cloud migration is an ongoing evolution, so embrace continuous optimization, communicate relentlessly, and celebrate the business benefits along the way.

At Google Cloud, we are supporting the transformation of the financial services value chain with a data-first strategy. There are already hundreds of AI use-cases for financial services. The core of the problem is data, and the goal is to modernize backend systems, make employees more productive, enhance internal process workflows, and creating personalized data-driven experiences for customers.

The best way to prepare a company for AI adoption is going to be very different for every company. How does an organization know which approach is right for them, what the value of different models might be, and what get started with?

Adrian Otto, Sunnyvale office: This question is fundamentally about innovation where any technology — AI or otherwise — is explored. To get the most from such a pursuit, I suggest:

1. Arrange for leaders and influencers at all levels to give the program adequate resources and air cover to operate well.
2. Break down the ambition into a series of chunks, each in pursuit of a small testable hypothesis.
3. Seek rapid learning and adaptation. Schedule frequent reviews of progress so participants can read out what they are learning from their experiments. Celebrate their progress even when they try things that don't work as expected. Keep the option to pivot and try a number of different approaches to find a working solution.
4. Resist the temptation to think incrementally. Consider options that are significantly different from prevailing approaches so you can find more ambitious leapfrog innovations that go far beyond what is currently done.

You don't have to figure everything out in advance. Gain operational confidence by experimenting, and learning as you go. Using AI to solve a particular business challenge may involve using a system of cooperating components rather than a simple one-tool-for-all approach. Remember that small models execute faster, and are usually much cheaper to operate than giant ones. If you can tie some smaller, more focused models together, you may get better outcomes.

But that depends on your needs. Different workflows and use cases might benefit from a different collection of models. Consider the following table, which highlights the general benefits of various first- and third-party AI models available in Google Cloud’s Vertex AI platform:

Note that in this sample set, no one model exhibits all the different strengths. Your use cases may have multiple requirements that are best met by a unique combination of models. I’d keep an open mind about how to match them up.

Start with general purpose foundation models like the ones mentioned above first, to prove that an approach yields demonstrable value, and then seek expert advice for system optimization once you’ve achieved a material advance with a prototype.

Opening image created with MidJourney, running on Google Cloud, using the prompt: a business team carrying servers through a bank of clouds drawn in an impressionistic watercolor style, suggesting the concept of cloud migration in an illustration for a magazine.