Ask OCTO: Closing the AI skills gap

In our Ask OCTO column, experts from Google Cloud's Office of the CTO answer top questions about solutions to business and IT challenges. This series is Google Cloud’s version of an advice column — except the relationships we're looking to improve are the ones in your tech stack.

Please submit questions for future columns here.

The motto of Google Cloud’s Office of the CTO is “collaborative, practical magic.” The team is made up of Google Cloud technical experts and former CTOs of leading organizations, all of whom work in the service of helping our largest and most strategic customers tackle their biggest challenges.

In this edition, we explore expert insights from the field on what's most effective for training and equipping teams with the right skills to get the most from AI rollouts.

Antonio Gulli, Senior Director, AI, Cloud, and Search, Office of the CTO
The barrier to entry for working with AI systems, particularly generative AI, has been significantly lowered. Instead of requiring deep technical knowledge of their inner workings, users can now interact with these systems using natural language, much like driving a car or listening to the radio. You don't need to be a mechanic or an electrical engineer to utilize these technologies effectively; you simply need to understand how to operate them and achieve your desired outcome. Therefore, the focus shifts from intricate technical expertise to the ability to:

• Clearly articulate your needs and goals to the AI system. This involves framing your requests in a way that the AI can understand and effectively respond to.
• Critically evaluate the AI's output. While AI models are powerful, they can sometimes produce inaccurate or biased results. Users need to be able to assess the quality and reliability of the AI's responses.
• Adapt and refine your interaction with the AI. This involves learning how to provide feedback, adjust prompts, and iterate to achieve the desired results.

In essence, the key skills for working with AI are now more about effective communication, critical thinking, and problem-solving, rather than deep technical expertise. While you don't need to be an expert to use AI, diving deeper into the underlying technology can be beneficial for those who are curious or whose work requires a more in-depth understanding.

If you want to explore the inner workings of AI, I recommend taking a deeper dive into foundational mathematical knowledge like calculus, GPU programming, and specific AI concepts and techniques, such as deep learning. There are also many structured courses, books, and practical guides out there that can equip you with the knowledge to understand how AI systems function and how to interact with gen AI models effectively. These resources allow you to potentially contribute to their development, optimization, or application in more specialized roles.

Troy Trimble, Principal Software Engineer, Office of the CTO
In the last decade or so, the infrastructure of the tech industry has been trending towards asynchronous, disconnected systems (e.g., Hadoop, Kafka, etc.) that all collaborate together to achieve an end user’s goal. Multi-node ML training workloads are similar to synchronous High Performance Computing (HPC) workloads. Imagine 5,000 compute nodes all working in lockstep with one another — if just one machine fails, then all 5,000 nodes are fully stuck and cannot make forward progress.

Given the underlying infrastructure involved, I would group what you need to know into two categories: technical skills and general skills.

Technical Skills

• Math: Cultivate a basic understanding of what computation is doing, including linear algebra and matrix math. For a deeper understanding of the ML algorithms that help models incrementally “learn,” you’ll need a strong understanding of calculus concepts, such as partial derivatives.
• Distributed concurrency and networking: Nodes communicate with one another using well-known distributed communication patterns, including All Gather, All to All, Reduce Scatter.
• Hardware architecture: Each ML accelerator architecture and generation has its own hardware specifications. Understanding how hardware features like Floating point operations per second, RAM, and high-bandwidth memory size impact a model's performance will be important for getting the most out of AI investments and making improvements.

General Skills

• Educate yourself on all the various model sharding strategies and communication patterns to understand exactly when and why you would choose to use each for a given hardware layout and scale.
• Learn how to use tools for assessing node performance, such as our Cloud TPU profiler or NVIDIA Nsight Systems. For example, I recommend watching educational video content where people demo these tools with a well-known open-source model.

Ben McCormack, Technical Director, Office of the CTO
To work effectively with AI systems, you'll need a blend of technical and soft skills.

Technical skills

• Fundamentals of AI/ML: Cultivate a basic understanding of AI concepts, machine learning algorithms, and their applications. This includes knowledge of different AI models (e.g., supervised, unsupervised, reinforcement learning, etc.), along with their strengths and weaknesses.
• Data skills: Learn how data is collected, processed, and used in AI systems.This includes understanding how data is cleaned, transformed, and prepared for AI model training. You’ll also need the ability to analyze data, identify patterns, and draw insights relevant to AI applications.
• Programming: Gain proficiency in at least one programming language commonly used in AI, such as Python or R. It’s also helpful to know how to write code to interact with AI APIs, libraries, and frameworks.
• AI-specific tools, and technologies: Get familiar with AI platforms, libraries like TensorFlow, and frameworks. Experience with tools for data visualization, model building, and evaluation are also a plus.

Soft skills

• Critical thinking and problem-solving: Know how to analyze AI outputs, identify potential biases or limitations, and interpret results in context. This includes skills in framing problems in a way that can be addressed by AI solutions.
• Communication and collaboration: Learn how to effectively communicate with both technical and non-technical stakeholders about AI capabilities and limitations. It’s also critical to collaborate with diverse teams (e.g., data scientists and domain experts) when developing and deploying AI solutions.
• Creativity and innovation: Explore new ways to apply AI to solve problems and generate value. Adapt to the rapidly evolving AI landscape and embrace new tools and techniques.
• Ethical considerations: Be aware of the ethical implications of AI, including bias, fairness, and accountability. This includes having a strong understanding of responsible AI best practices.

Also, show, don’t tell. It’s harder to teach people in an organization about something that is very abstract and not yet connected to their daily work. In an ideal situation, a company should have a gen AI experience launched that they could use as a showcase and example for a training program.

Access to gen AI services is ubiquitous. Employees can explore the latest features via services, such as NotebookLM. We have seen customers have a lot of success encouraging people to play with gen AI capabilities before launching a formal training program. This ensures their workforce has a good background as a starting point.

Scott Penberthy, Senior Director, Applied AI, Office of the CTO
We’ll soon be managing systems far brighter than we are. AI will shift from an API and tool to a true collaborator. Given that, what role will humans have? Here, take a lesson from managing research teams. If you do your job right, you’ll ”always” hire people smarter than you, but you also need to have a good, basic understanding of how they think.

For AI, it’s particularly important to understand probabilistic and shades-of-gray thinking. AI systems strive to provide the most probabilistically correct responses and operate according to probability distributions. A great place to start is a basic course in probability.

To manage AI effectively, you should also focus on communication, creativity, and the art of asking the right questions. Work will be more about your intuitive insight, the kinds of questions you asked, and the directions you took — meta-work, if you will.

I recommend taking courses in literature, writing, art, and history and reading voraciously to find good examples of asking the right questions and focusing on the right things. For example, “Good to Great,” “Innovator’s Dilemma,” and “Atomic Habits,” are some of my favorites. You can also spend time watching TikTok, Reels, and YouTube shorts to see the art of communicating in a concise format. Tease apart how these videos grab your attention by figuring out how they work through a problem, a solution, benefits, and call to action within the span of seconds.

Diane Chaleff, Group Product Manager, Office of the CTO
It’s critical to ensure AI is addressing a true business problem. As a product manager, the top miss I see is putting the technology before the user and business need.

We still need to answer basic questions: Why is this better than the status quo? How will this delight the user? How will this help advance my business? Without these fundamental questions in place, no matter how effective the AI is at accomplishing the task at hand, no one will use the feature or product, and it may, unfortunately, deter additional AI development.

I also suggest challenging every product manager to create their roadmap based not on what AI can accomplish today, but what may be possible in the future. Allow the engineering team to say, “No sorry, this isn’t possible today,” and then revisit the conversation every few months. That way, as soon as the technology is available, the idea is already baked and ready to go!

Ashwin Ram, Sr. Director of AI, Office of the CTO
To bridge the skills gap, we must hire people with the most applicable skills, including:

• AI/ML/Data engineers who have built and operated AI-driven applications on cloud infrastructure. They will manage the AI model pipelines, model architecture search, and data ingestion and labeling processes, as well as MLOps.
• UX designers and prototypers (user experience, not just user interface) who have rapidly iterated designs and conducted usability studies using many different approaches. They will craft AI-driven experiences for users.
• App developers, including mobile, web and other interfaces, who have built experiences for a broad set of consumers across many different demographics, backgrounds, etc. They will build the experiences and ensure a consistent experience across all platforms.
• Core systems engineers who are familiar with real-time embedded systems, High Performance Computing, telecommunications, and networking. They will help ensure consistent data ingestion processes and manage cloud services, device gateways, and compute hardware for AI training, evaluation and deployment.

Jeff Sternberg, Technical Director, Office of the CTO
In addition to the skills gaps addressed by my colleagues here, rolling out a successful learning program for AI is also critical. It’s similar to any enterprise learning and development program. However, it differs in a few key ways:

• Velocity of change: AI is changing extremely fast, so learning materials will become quickly out of date. What’s the fix? Plan to evolve the learning program quickly – rather than doing a yearly refresh of materials, do this semi-annually or even quarterly.
• Inclusivity: While AI is intrinsically technical, the best outcomes will happen if the learning program is extended to every team and role in the organization, including the business functions. In OCTO, we have worked with CTOs and CEOs that are driving this change through every business unit and seeing great results in getting the entire organization speaking the same AI language.
• Multi-modality: Technical teams are used to engaging with new technology using APIs, programming languages, and similar. Gen AI communicates through text, video, audio, and more. For software engineers, that means using skills like communication through natural language (e.g. prompts), which is both new and exciting!

John Abel, Technical Director, Office of the CTO
Keep these key elements in mind when planning a successful AI training rollout and change management program:

• Secure leadership buy-in: Executive sponsorship is crucial to champion AI adoption, demonstrate its business value, and drive organizational change.
• Focus on impactful applications: Prioritize AI solutions that address significant pain points, such as automating tedious tasks (toil reduction), streamlining inefficient processes (friction reduction), or enabling scalability for growth.
• Foster collaboration and engagement: Encourage cross-functional teamwork and actively involve employees in the AI implementation process. Remember that successful AI adoption relies on both technology and people.
• Establish a clear communication strategy: Maintain consistent and transparent communication throughout the AI journey to keep everyone informed and engaged.
• Align AI with business strategy: Clearly articulate the organization's AI strategy and emphasize its connection to core business objectives and value creation.
• Provide hands-on learning opportunities: Offer employees and subject matter experts dedicated spaces and resources to experiment with AI tools and techniques, such as prompt engineering and model fine-tuning.
• Prioritize responsible AI: Embed ethical considerations and responsible AI practices throughout the process, ensuring alignment with employee and customer values and mitigating potential biases.
• Embrace diversity: Cultivate diverse teams to challenge biases, foster inclusive solutions, and ensure the AI applications cater to a broad range of needs while staying focused on desired outcomes.