While building the reasoning logic with Google Cloud ADK was straightforward, the real engineering challenge lay in the integration. This integration effort cannot be understated; bridging the gap between a real-time messaging interface and a complex, multi-step RAG pipeline requires robust, production-grade middleware. To handle this, the Carrefour team invested significant engineering cycles into designing an asynchronous architecture that prevents timeouts and guarantees message delivery.

In a standard webhook architecture, a messaging platform like Google Chat expects an immediate synchronous response to prevent timeout errors. However, RAG pipelines and LLM generation can take up to several seconds to fully process. To overcome this, the Carrefour team built an asynchronous, serverless middleware layer using FastAPI on Cloud Run.

Designing and implementing this custom middleware was where the bulk of the development complexity resided. It required careful management of background worker threads, API payload formatting, and secure communication protocols with both Google Cloud and Google Chat APIs, demonstrating that the success of modern AI agents is often 90% integration and 10% model prompting.

The middleware receives the HTTP POST webhook from the Google Chat API and immediately returns a 200 OK status to the platform. Simultaneously, it triggers a background worker task using FastAPI’s BackgroundTasks utility. This background task sends the user's query to the ADK agent endpoint, which retrieves relevant context through Agent Platform Search and prompts Gemini. Once the agent generates the response, complete with formatting and sample SQL queries, the middleware uses the asynchronous Google Chat API client to push the answer back into the correct conversation thread.

This native integration has had a massive impact on daily operations. Traditional support resolution times dropped from several hours or even a full working day down to less than two minutes. Because the AI assistant resolves roughly 75% of incoming queries successfully on its own, it has saved approximately half of a full-time employee's workload (0.5 FTE).

This allows the platform engineers to step away from repetitive support duties and focus entirely on core platform development. Furthermore, because Cloud Run scales to zero on nights and weekends, the infrastructure costs are kept to a bare minimum.

The project highlighted two key architectural lessons for modern development teams. First, teams should focus sprint cycles on integration. Many spend too much time tuning prompts or selecting models. This project proved that because modern LLMs and orchestration frameworks like the ADK are already highly advanced, the bulk of your engineering resources should be dedicated to data ingestion, session persistence, and middleware connectivity.

Second, it is crucial to meet users where they already collaborate. Building a standalone web portal for an AI assistant often fails due to low user adoption. By deploying the assistant straight into the active Google Chat support rooms, Carrefour minimized change management.

Both focusing on solving real business problems and meeting the users where they are—at the cost of substantial integration effort—are the keys to success in agentic deployment. Furthermore, when the agent’s response is incomplete, human engineers can jump directly into the thread to help—which seamlessly feeds back into the historical RAG data for the next query.