Ask OCTO: Making sense of agents

In our Ask OCTO column, experts from Google Cloud's Office of the CTO answer your questions about the business and IT challenges facing you and your organization now. Think of this series as Google Cloud’s version of an advice column — except the relationships we're looking to improve are the ones in your tech stack.

The motto of Google Cloud’s Office of the CTO is “collaborative, practical magic.” The team is made up of Google Cloud technical experts and former CTOs of leading organizations, all of whom work in the service of helping our largest and most strategic customers tackle their biggest challenges.

This month, we’re chatting with Will Grannis and his team at the Office of the CTO about agents, what they are, and their importance in today’s AI landscape. 

AI agents represent one of the most exciting developments in enterprise technology today. Unlike the isolated AI tools many of us have grown familiar with, agents work together, much like human teams do. They can break down complex problems, delegate specialized tasks, and collaborate to deliver results that no single AI—or human—could achieve alone.

But as our OCTO experts will tell you, building effective agent systems isn't just about the technology. It's about understanding how to design intelligent collaboration, where to start, and how to scale responsibly. Whether you're just beginning to explore AI agents or looking to expand their role in your organization, the insights ahead will help you navigate this rapidly evolving landscape. And stick around to the end to learn about a few new agents the team launched this year at I/O. 

John Abel, Managing Director

For me, agents are representatives performing actions for your business. They can help your customers, employees, and partners by removing friction and toil from business processes.

As someone with dyslexia, the barriers that agents remove for me are incredible. Agents bring my ideas to life and assist me in writing and learning. It’s not just about business; it's a key to my personal journey.

Here are five tips for getting the most out of agents:

1. Define your why: Understanding why an agent is important makes defining the what and how easier. Once you know your why, clarify your goal: what are you trying to achieve?

2. Start small and be specific: Focus on specific tasks. Marginal gains will build a foundation and demonstrate success. Don't wait — just try, learn, and improve.

3. Combine humans and agents: Keep a human in the loop and coach the agent to achieve your defined goals.

4. Understand your data: Agents are data-driven. Know your data — what can be achieved and what is missing. 

5. Cultivate a learning mindset: See the AI opportunity. This space is changing rapidly, so embrace new ways of thinking.

I’m incredibly excited about the AI era, especially the use of agents to help bring my ideas to life by removing barriers—both by reducing friction and providing scale. 

Antonio Gulli, Distinguished Engineer

Enterprises are increasingly exploring the potential of AI agents, particularly their ability to simulate and augment human teamwork. While AI models have demonstrated powerful capabilities, they often operate in isolation, accessing the external world primarily through tools in response to specific prompts. 

AI agents, however, represent a significant evolution by enabling the coordination and orchestration of multiple specialized roles—often powered by underlying AI models—to tackle complex tasks collaboratively. This multi-agent, collaborative approach offers distinct advantages:

• Specialization and incremental improvement: Individual agents can be designed or fine-tuned for highly specialized roles (e.g., data analyst agent, report-writing agent, validation agent). This focus allows for targeted development, testing, and incremental improvement of each component's performance without needing to retrain a monolithic system.

• Task decomposition and modularity: Complex business processes or problems can be broken down into simpler, well-defined sub-tasks. Each sub-task can be assigned to a dedicated agent, allowing for independent development, robust testing with specific evaluation criteria for each step, and easier debugging.

• Emergent synergy and enhanced outcomes: Similar to effective human teams, collaboration between specialized agents can lead to emergent intelligence and superior results. Agents can cross-validate information, work in parallel, refine outputs iteratively, handle different facets of a problem concurrently, and potentially achieve a level of robustness and quality exceeding what a single model or isolated human could produce.

Frameworks like Google's Agent Development Kit (ADK), for example, are dramatically simplifying the creation of sophisticated multi-agent systems. They empower developers to define distinct agent roles, personalities, and capabilities primarily through carefully engineered prompts, while the core operational logic and interaction with external systems can often be implemented using straightforward Python code. 

A significant development is the emerging meta-capability where foundation models, like Gemini, assist developers in crafting nuanced prompts and generating the necessary Python code. This meta-synergy—AI contributing to the creation of more complex, collaborative AI systems—substantially accelerates development cycles and unlocks considerable potential. 

Easy to set up yet massively powerful, AI agents are poised to revolutionize how we think and work together across all enterprise applications. The future is promising!

Michael Zimmerman, Principal Engineer

Businesses globally are beginning to adopt agent-based workflows to help increase revenue and efficiency. These businesses often require a different approach to agents than generalist models. They need agents that understand their specific business context and custom workflows. The typical enterprise environment presents a distinct challenge for standard generalist agents. 

Google Cloud assists customers by providing advanced capabilities. These include enterprise knowledge graphs, Agent to agent protocol, native MCP support in Agent Development Kit, custom tool trajectories, and other components that incorporate specific enterprise information into the agent's thinking and reasoning processes.

To further support the use of agents in enterprises, we have released an agent-to-agent (A2A) protocol, reflecting our commitment to openness. A2A enables a network of interconnected agents in the cloud, allowing different agents to work together to carry out workflows. This helps to break down the separate structures that can arise from different departmental responsibilities or business divisions.

Diane Chaleff, Group Product Manager

Understanding what AI agents can do is one thing; successfully bringing them into your team's daily work is another. It's natural to wonder about the best way to get started and what the future might hold. Here are a few practical considerations and a look at the broader picture for AI agents:

• Encourage your employees to try using an agent as soon as possible, perhaps for a less critical task, just so they can get familiar with how it works. Think of AI agents as helpful assistants, very skilled at bringing together and making sense of large amounts of information. The idea might seem complex at first, but like many technologies, once people try an agent, they often find it straightforward.

• Empower your employees. They will often have the best ideas for how agents can speed up and enhance their daily work. While central IT has an important part to play in keeping agents secure, employees typically best understand their specific work streams and where agent integration helps complete tasks more quickly. However, Consider creating fun challenges to encourage employees to explore agents and then help them team up with technical groups to develop these ideas.

• Consider the wider market for acquiring agents. When smartphones first appeared, they came with a few basic apps. But the broader developer community was then able to use app marketplaces to create the millions of applications we see today. A similar pattern is likely for AI agents. We're just at the beginning of seeing many new agents being developed and shared.

Ben McCormack, Principal Engineer

An AI agent is a system that uses AI—such as machine learning, natural language processing, and planning algorithms—to understand its surroundings, think about what it perceives, make decisions on its own, and act to reach specific goals. These agents often learn and adjust as they go. This means an AI agent can operate more like a person, rather than just being a system that answers questions.

AI agents have distinct components and characteristics that define how they work.

The core idea: Intelligent autonomy
AI agents use AI to manage uncertainty, complexity, and learning, setting them apart from simpler programmed agents. They work to mirror thinking skills such as reasoning, problem-solving, planning, and learning. This helps them reach their goals more effectively and adapt to changing situations.

Key AI-driven characteristics

Deeper perception: AI allows agents to gain a richer understanding from inputs, including text, images, audio, and video.

• Advanced decision-making: Agents can manage unclear situations, make predictions, and work towards complex goals (like maximizing long-term benefits).

• Learning & adaptation: A core feature is that AI agents improve over time. They learn from data, feedback, or experience — for example, getting better at a game, fine-tuning a trading strategy, or adjusting to the preferences of the people using them. This also helps in detecting and managing unusual situations.

• Handling complex goals: Agents can tackle more abstract or complicated goals that need several steps of reasoning and planning.

Examples of AI agents in action
AI agents are used in many fields:

• Automating business processes: Agents can manage complicated interactions between company systems that, due to their complexity, previously needed human oversight.

• Advanced virtual assistants: These go beyond simple commands to handle detailed conversations and carry out multi-step tasks (e.g., "Find restaurants near my event tonight, check reviews, and book one for 7 PM").

• Autonomous task platforms: Tools such as Auto-GPT, CrewAI, or LangChain allow agents to independently research topics online, write and debug code, summarize information, and use various software tools to achieve a broad goal.

• Self-driving vehicles: These use intricate AI (including computer vision, sensor fusion, planning, and control theory) to understand their surroundings and navigate safely.

• Smarter game AI: Game opponents that learn from how people play and change their own strategies accordingly.

• Robotic process automation (RPA) with AI: Automating complex business operations that involve understanding documents, pulling out information, and making decisions.

• Scientific research support: AI agents can help with forming hypotheses, designing experiments, and analyzing data in areas like drug discovery or materials science.

In essence, AI agents are goal-oriented, independent systems that benefit greatly from the capacity of AI models to handle complex information, reason, learn, and adapt. This makes AI agents a distinct type of technology capable of performing tasks that once required direct human involvement.

Yingchao Huang, Software Engineer

Key strategies for successful AI agents
So, how do we build these highly successful AI agents? It’s about thoughtful design and applying proven engineering principles to create robust and intelligent solutions.

Build with smart combinations: Mixing AI and traditional software
A great way to build highly effective agents is by designing systems that combine the unique strengths of AI with the precision of traditional software components. Use AI for what it's good at (like understanding language, finding patterns, and reasoning with probabilities). Use traditional software for tasks that need exact logic, precise calculations, or dependable data handling. 

Ensure clear communication between these AI-based and software-based services by using clearly defined ways for them to talk to each other (such as APIs with specific input/output rules, like gRPC/Protobuf or REST/JSON Schema). This makes sure they work together reliably. Importantly, design the AI parts to ask for more information when details are missing or unclear and to make sure their responses meet all stated needs. This helps achieve greater accuracy and avoid incorrect assumptions.

Focus on reliability: Applying software engineering practices
We can significantly enhance agent reliability by incorporating established software engineering practices right from the start. Make sure agents can be verified through thorough testing. Use different levels of testing (unit, integration, end-to-end, and system tests) that are specifically adjusted for what agents do. 

This means checking not only if they work correctly, but also how they reason, make decisions, and deal with unclear situations. Set clear goals for success and what you expect the agent to do. Critically, build in solid backup plans. This includes smooth error handling, ways for the system to manage errors automatically, and clear ways for a person to step in or for issues to be passed to someone who can help when the agent faces a problem it can't manage on its own.

_________________________________________________________________________________________

The future of work isn't about humans versus AI—it's about humans with AI. And nowhere is this more evident than with AI agents.

At Google I/O this year, our OCTO team put these principles into action, launching three specialized agents that showcase what's possible when you combine human expertise with intelligent automation:

• Academic research agent: Accelerates discovery by helping researchers find recent publications and identify emerging research areas

• Financial advisor agent: Amplifies human advisors by providing instant access to educational finance and investment content

• Marketing agency agent: Transforms launches by suggesting domains, creating websites, developing marketing strategies, and designing brand assets—all in coordination

These aren't just proof-of-concepts; they're working examples of how agents can augment human capabilities rather than replace them. You can explore them here. 

The organizations that will thrive in the age of AI agents won't be those that simply deploy the latest technology. They'll be the ones that thoughtfully design collaboration between human intelligence and artificial intelligence, creating teams where both can do what they do best. Whether you're just starting to explore AI agents or looking to scale their impact across your organization, remember this: the most powerful agent in your toolkit will always be human curiosity, creativity, and the willingness to reimagine how work gets done. Ready to build your own collaborative future? The agents are waiting.