The composable middle: The key architectural layer for scaling pilots to projects

When companies’ enterprise AI initiatives stall between pilot and production, it’s rarely due to a lack of foundation models or industry expertise. Instead, it’s usually because of the layer between them — the composable domain components. These battle-tested skills and MCP connections, single-job atomic agents and multi-agent workflows, must be bound to industry rules, validated, and plugged into an enterprise platform harness.

Pieces of this layer already exist inside every successful deployment. What’s been missing is a clear definition of what the layer must contain, as well as how platform providers and transformation partners co-own it to unlock scale.

This is what we call the “composable middle.” The stakes of figuring this technical layer out are significant.

According to a 2026 BCG report, agentic AI is unlocking up to $200 billion in net-new value pools for technology service providers over the next five years, with more than 40% of large enterprises already scaling agentic implementations and two-thirds looking to service partners to build and operationalize priority workflows.

Yet fragmented delivery methods have kept many enterprises from capturing that windfall. RAND Corporation research notes that industry estimates of AI project failure have historically run as high as 80% when problem definitions and production complexity are underestimated, and Gartner predicts that more than 40% of agentic AI projects could be canceled by the end of 2027 due to unclear business value or inadequate risk controls.

When organizations pair governed architecture with workflow discipline, however, the returns are substantial.

In our 2026 AI ROI research surveying more than 2,400 global executives, 84% report increasing financial returns from AI, 86% say AI is driving cost-efficient growth by scaling output without proportional operating costs, and 94% of organizations deploying AI agents report measurable contributions to both cost savings and revenue.

At Google Cloud, we recognize the opportunity and are actively working with our robust partner ecosystem to build out this composable middle. Our packaged industry solutions, such as Gemini for Financial Services and Gemini for Legal, are a great example. They provide the governed industry foundation, pairing baseline skills and licensed data connectors with a governed runtime where partners plug in their own reusable domain plugins, atomic agents, multi-agent workflows, and validation suites.

Early adopters like Deutsche Bank and CME Group in financial services, and global law firms such as Cleary Gottlieb and Freshfields, are already using these foundations to compress complex research, risk analysis, and drafting workflows from days to minutes.

For both enterprises and the partner organizations looking to support them, bridging this divide remains a critical step in the success of AI, both in the short and long terms.

From our experience working across Google Cloud’s partner ecosystem, and seeing what leading transformation partners have demonstrated, the composable middle explains both why so many projects fail and identifies where the real economic value lies. Our insights dovetail with those of others — like UC Berkeley’s ”compound AI systems” — and many leading system integrators, who have converged on the same idea.

Our aim is to define what the layer must contain, how each part is validated, and where ownership sits between platform, partner, and institution. Plus, we want to demonstrate how that specification changes the fundamental economics of enterprise AI.

Why “horizontal-vertical” isn’t enough

It seems like every consulting firm and cloud provider has published some version of the horizontal-vertical framework: a horizontal platform supplies foundation models, infrastructure, and the core agent harness, while system integrators bring vertical industry depth. Together, this dynamic delivers transformation. But this framing has a critical blind spot — it treats “industry expertise” as a monolithic capability. When you decompose what actually happens inside a production agent deployment, you find that domain knowledge is actually three distinct elements, each with its own radically different economic properties.

The first is encodable knowledge that’s standardized and downloadable. For example, the healthcare, finance, and insurance industries have numerous large and highly structured industry standards governing everything from medical diagnostic codes to banking rules and policy formats. And while these frameworks are huge, they’re standardized and licensable so that any provider can build on them.

The second element is industry rule frameworks like clinical guidelines, insurance regulations, and FDA submission pathways. These are reusable templates that require some level of institutional customization.

Finally, the third element is tacit exception patterns — the hundreds of things that inevitably don’t go as planned in any workflow and live in the heads of those who have run thousands of engagements. Unfortunately, no foundation model can replace it.

The first element is a horizontal commodity foundation, and the third is irreducibly bespoke to each institution. Between them sits the second element: shared industry rulebooks and operational playbooks that are built to be reusable with institutional adaptation. This middle tier of domain knowledge is where a disproportionate share of agentic transformation’s economic value sits — and encoding it once so it can be configured across many clients is what allows partners to break linear delivery bottlenecks and scale across entire industries.

The three-layer architecture

Based on our observations, we believe that replacing binary horizontal-vertical models with three layers can unlock enterprise AI’s true economic potential. Here’s how:

Ownership across the three layers is explicitly delineated between open standards, the cloud provider, and transformation partners:

• The horizontal platform (Google Cloud owned + open standards): Fully reusable across industries, this layer comprises open interoperability plumbing — protocols like A2A and MCP and open toolkits like ADK — alongside Google Cloud’s proprietary foundation models (Gemini) and the enterprise agent harness (the Gemini platform), which provides the repeatable runtime machinery for orchestration, security and IAM, governance enforcement, observability, and evaluation execution.
• The composable middle (collaboratively built — Google Cloud baseline + partner-owned scalable IP): Modular, domain-configured components ranging from packaged plugins (battle-tested skills and MCP connections) to single-responsibility atomic agents and multi-agent workflows, paired with the domain rules, benchmarks, and compliance policies that plug into the platform’s harness. Google Cloud seeds this layer with baseline industry skills and licensed data connectors, while transformation partners build and own the high-margin, reusable domain plugins, atomic agents, and vertical validation suites that deploy across many clients.
• The irreducible vertical (delivered by partner + client): Genuinely bespoke last-mile delivery — institution-specific legacy integration, regulatory sign-off on local data, tacit exception handling, and workforce change management that elevates human teams from manual processing to higher-judgment oversight.

Instead of building monolithic agents or relying on outdated functional buckets like generic “extractors” or “generators” — primitives that foundation models now handle natively — the composable middle organizes around measurable business jobs. Depending on complexity, a composable component may be a portable plugin equipping an agent with specialized skills and data connectors, an atomic agent executing one complete job end-to-end (such as KYC verification, commercial credit underwriting, or prior authorization review), or a coordinated multi-agent workflow.

Crucially, in regulated workflows, these components are engineered for governed human-in-the-loop execution: agents automate intake, cross-system synthesis, and policy checks, while routing irreversible clinical, credit, or fiduciary decisions to accountable human professionals.

These components also compose into recurring workflow skeletons. In industries as distinct as software delivery, insurance claims, and proposal response, production workflows keep resolving into the same case structure: intake, an assessment, a proposed action, an independent check, and manual approval of the one irreversible step.

The nouns, the data, and the regulators change completely; the skeleton does not. That is what makes the composable middle reusable at the workflow level and not just the component level. It’s also why validation, governance, and exception routing can be engineered into the pattern once instead of being rebuilt from scratch for every engagement.

Why composability wins

Composability is not just an organizational framework; it reflects an emerging principle in AI research that modular, composable systems consistently outperform monolithic ones. UC Berkeley’s AI Research lab put it plainly: compound systems with multiple components, not single models, are increasingly delivering the most state-of-the-art results. And DeepMind’s AI Co-Scientist, published in Nature in 2026, showed that a team of specialized agents, each with a single responsibility, produced novel, experimentally validated scientific hypotheses. It was the most compelling demonstration yet and succeeded because the composition was right, not because any one agent was extraordinary.

At the same time, interoperability standards— such as Google’s A2A protocol, which has more than 150 partners operating under Linux Foundation governance, and Anthropic’s Model Context Protocol (MCP) — are setting the stage for exactly this type of infrastructure. The implication for leaders is clear: don’t build monolithic agents that attempt everything. Instead, build a library of composable components — domain plugins, atomic single-job agents, and multi-agent workflows — that an orchestrator can compose dynamically by task.

The content layer: What protocols leave to us

To understand why this middle layer is so critical, it helps to separate raw open-source plumbing from the governed enterprise layer. Open wire protocols like A2A and MCP standardize how agents communicate and reach tools, while open packaging specifications — including Anthropic’s Agent Skills, the Agentic AI Foundation’s AGENTS.md (used by more than 60,000 repositories), and Vercel’s Agent Plugins (published with AWS, Anysphere, GitHub, Microsoft, and OpenAI to bundle a plugin.json manifest, a skills/ folder, and mcp.json tool connections into one portable unit) — standardize how instructions and tool connections are packaged across clients.

We believe packaging content plus connections as a portable unit is the right foundation, which is why plugins are a core building block of the composable middle. By design, however, open protocols and file formats are raw plumbing, not an enterprise control plane. They specify how a single agent reads instructions or calls a tool, but they intentionally leave installation policy, permissioning, multi-agent orchestration, domain evaluation, and production governance outside the specification.

To be clear, the composable middle is not the runtime harness itself — the Gemini platform already builds the horizontal harness for agent governance, security, observability, evaluation execution, and orchestration. Rather, to take packaged plugins, atomic agents, and multi-agent workflows from individual productivity into regulated enterprise production, the composable middle supplies the five domain-specific constituents that plug into that platform harness:

• Domain bindings, the industry ontologies, structured schemas, and rule frameworks that bind a capability or workflow to a specific vertical, such as insurance claims adjudication, capital markets research, or clinical documentation.
• Validation gates that ship with the package. While the horizontal platform provides the evaluation harness, the composable middle carries the domain-specific evaluation datasets, benchmarks, and regression tests that rerun automatically with every new institutional configuration.
• Orchestration semantics, which define the domain rules for how skills, atomic agents, and multi-agent workflows compose over the platform’s orchestrator and A2A — including deterministic exception routing to human reviewers and specialized agents.
• Governance and provenance, the domain compliance policies, admissibility rules, and audit schemas that bind to the platform’s cryptographic signing, permissions, and audit-logging harness in a regulated process.
• Operational instrumentation, the domain-specific SLA targets, drift thresholds, and rollback triggers that feed into the platform’s monitoring and observability harness as data and regulations evolve.

The distinction that matters is between raw content an agent reads and a production-hardened component running inside an enterprise harness. On its own, an unvalidated skill file is only advice: an agent loads it, but nothing guarantees compliance or catches edge-case drift. When plugins, atomic agents, and multi-agent workflows carry these five domain constituents — and execute on the Gemini platform’s governed harness — validation gates rerun automatically with every configuration change, exception routes fire deterministically, and provenance is signed and audited by the platform itself.

Gemini for Financial Services and Gemini for Legal show how these pieces come together in practice. Built on the Gemini platform, both solutions ship core plugin components — more than 50 foundational skills and licensed MCP data connectors (13 market data sources including FactSet, LSEG, Moody’s, PitchBook, S&P Global, and SEC EDGAR in financial services; legal platforms such as iManage, Everlaw, RelativityOne, and Thomson Reuters HighQ in legal) — alongside Google-managed and third-party atomic and multi-step agents, all running on a governed runtime with source citations, permissions enforcement, and audit logging.

Partners then bring two distinct layers of value: first, their scalable IP in the composable middle — packaging their own domain plugins, atomic single-job agents (for workflows like KYC onboarding, commercial credit underwriting, or litigation discovery), multi-agent templates, and repeatable validation gates that can be configured across many clients — and second, the irreducible vertical work of bespoke integration, local regulatory sign-off, and change management for each specific institution.

The pattern in practice: healthcare

For this architecture, the rubber meets the road in real deployments. For example, when a leading U.S. health system deployed a Gemini-based clinical notes agent to more than 7,000 clinicians at 18 hospitals and 500-plus care sites, it was able to apply the architecture to 12 distinct medical specialties. These did not have identical systems: oncologists and cardiologists work with entirely different data types and diagnostic logic. But the fundamental elements — data processing, search, verification, and medical record integration — stay the same.

For this healthcare system and its implementation partners, the composable middle unlocked both clinical capacity and deployment velocity that a bespoke approach could never have reached. Instead of stalling after a single specialty pilot, the team reused the Gemini platform harness and adapted the composable middle to expand rapidly across all 12 specialties — reducing time-to-deployment on subsequent specialties by 40–60% while freeing clinicians from administrative documentation so they could focus on patient care.

For the partner team, reusing foundational plumbing shifted engineering bandwidth into the high-value vertical work every specialty demands: bidirectional EHR workflow integration, clinician skilling and change management, and the specialty-specific clinical validation that accounts for 30–40% of healthcare AI project investment.

We see the same measurable ROI across healthcare and other regulated industries. Highmark Health, for example, delivered $27.9 million in measurable value in a single year on the Gemini platform by scaling reusable search proxies and research protocols across 74 live use cases. In banking, HSBC’s AML AI deployment cut false-positive alert volumes by more than 60% while detecting two-to-four times more suspicious activity, allowing compliance investigators to focus on complex financial crime rather than routine alert triage. Similarly, insurance claims agents scale one composable architecture across the 50-state regulatory patchwork, pairing a reusable middle with dedicated vertical validation and human fiduciary sign-off in every state.

A structural diagnosis: Why most pilots fail

Perhaps our most consequential insight was that most AI pilots fail to reach production because they only validate the horizontal layer. A typical proof-of-concept (POC) processes 10 to 20 well-formed documents, demonstrates impressive results in a controlled environment, and gives executives the confidence to greenlight production. The agent then meets the long tail — the coordination-of-benefits dispute spanning three payer systems, the clinical note full of abbreviations from an underrepresented specialty, or the fraud pattern that appears in specific geographic corridors, to name a few.

These exceptions live in the composable middle and the irreducible vertical, the layers a POC never reaches. The gap is one of architectural completeness, not technology. Closing it requires validating each layer explicitly with functional and safety gates that regression-test with every new configuration, followed by performance, compliance, integration, and business-outcome gates that reach into the vertical.

The new economics of agentic transformation

The composable middle does not merely accelerate engineering — it fundamentally upgrades the unit economics and profit margins of technology service providers. Under the traditional linear services model, every engagement is built from scratch (~40% platform plumbing, ~35% domain logic, and ~25% client integration) and billed on time and materials (billable hours multiplied by headcount). Because delivery cost scales linearly with labor (Cost ∝ Hours × Headcount), partner gross margins are structurally capped at traditional IT services benchmarks of roughly 25–35%, and revenue growth is bottlenecked by how many scarce specialists a firm can hire.

Under the composable middle model, a partner invests upfront to build and validate a library of 15 to 20 composable domain components — plugins, atomic agents, multi-agent workflow templates, and validation suites. Once built, the marginal cost of deploying that middle-layer IP onto the second, fifth, or twentieth client approaches software economics, generating 70–80%+ gross margins on the reusable asset layer. When partners pair that reusable IP with outcome-based pricing and the bespoke Layer 3 services every enterprise still requires — data readiness, legacy integration, regulatory validation, workforce training, and ongoing managed agent operations — three economic shifts happen simultaneously:

• Blended gross margins expand by 15 to 25 percentage points, lifting partner delivery from ~30% linear labor margins toward 50–65%+ asset-backed margins as upfront IP is amortized across clients.
• Delivery throughput increases 3x to 5x per engineer, compressing deployment timelines by 40–60% and allowing existing teams to convert stalled single-department pilots into 10-to-15-workflow enterprise transformations.
• Total addressable market expands into new tiers, unlocking mid-market segments and specialized sub-verticals where six-month custom builds from zero were previously economically unviable.

Leading firms like Accenture, Deloitte, TCS, KPMG, and Infosys are already proving this model in production. BCG’s research confirms that while efficiency gains compress low-judgment routine tasks, agentic AI is net-expansionary — unlocking up to $200 billion in net-new tech services demand, with more than 55% of total investment concentrated in reusable “agentic enablers.” Google Cloud’s $750 million partner commitment, the open-source Agent Development Kit (ADK), and the A2A interoperability standard are built to help partners capture both the high-margin IP layer and the massive pull-through transformation services around it.

The composable middle has been hiding in plain sight inside every successful deployment we’ve studied. Naming it is the easy part. Specifying it, validating it gate by gate, and co-owning it across platform and partner is how the ecosystem moves from isolated pilots to compounding enterprise scale.

Building for the agentic era

The agentic enterprise is not a thing of the future — it’s being built today, in the collaboration between platform providers and the partners who bring deep domain expertise to every engagement.

Its effective implementation requires not just good technology and industry knowledge but also recognizing the composable middle as a distinct layer and being transparent about what is not yet solved. This includes the hard organizational work of data readiness, production operations, workforce transformation, and security architecture that technology alone cannot deliver.

The question for every enterprise leader is simple: are we building composable architectures that compound in value with every deployment or building from scratch every time? That question, more than any technology choice, will shape the trajectory of enterprise AI.