How Anthos clusters on AWS delivers on the promise of multicloud
Head of Engineering, PLAID
Editor’s note: Today’s post comes from Naohiko Takemura, Head of Engineering, and Kosukex Oya, Engineer, both from Japanese customer experience platform PLAID. The company runs its platform in a multicloud environment through Anthos clusters on AWS and shares more on its experiences and best practices.
At PLAID, our mission is to maximize the value of people through data, and we are developing a range of products that focus on improving customer experience. Our core product is a customer experience platform, KARTE, that can analyze the behavior and emotions of website visitors and application users, enabling businesses to deliver relevant communications in real time. We make KARTE available as a service to functions such as human resources and industries such as real estate and finance, and run the platform in a multicloud environment to achieve high-speed response and meet availability requirements. This is where Anthos comes in.
We introduced KARTE in 2015 and updated the system configuration in line with the addition of new functions and the need to increase scale. Our multicloud configuration is optimized through Anthos clusters on AWS, which give us access to the capabilities of Google Kubernetes Engine (GKE).
KARTE runs in two groups of server instances in each cloud; one group runs the management screens used by clients, and the other provides content for visitors to our website. In Google Cloud, the management system runs in GKE and content is delivered through Compute Engine.
We initially developed and operated the core of our services on another provider and from 2016 began to transition to Google Cloud due to its strong data processing capabilities. The products that handled big data, such as Cloud Bigtable and BigQuery, were attractive because they could handle data in real time and were compatible with KARTE. Now most functions, including peripheral aspects, run in Google Cloud, because we thought if we built a system centered on these products, it would become efficient to build other parts on Google Cloud.
While we considered migrating everything to Google Cloud, we decided to leverage its strengths alongside those of our existing provider, AWS. We felt a multicloud approach could create more opportunities and deliver higher growth than a mono-cloud environment.
We completed our move to a multicloud environment in 2017 and found that by building systems with almost the same content on two cloud services to leverage the strengths of each, we could reduce costs and improve performance and availability.
However, as KARTE grew, and the content of the service increased in complexity, we began to experience new problems. The increased load on the system due to an influx of in-house engineers from 2018 onwards impacted the scalability and development speeds of our conventional monolithic architecture running in virtual machines. We opted for an approach based on microservices and containerization, excluding the components that enabled real-time analysis as these had been modernized since initially being deployed in 2016, and the management screens, as the infrastructure running these did not require crisp tuning. Our key priority was to improve the ability of our engineers to deliver quickly.
From 2019, we turned to promoting microservices that make full use of container technology. When deciding to move from a target built on virtual machines to containerization, we evaluated the ease of use of GKE and decided to build in Google Cloud. At the same time, the number of systems with strict service level obligations was increasing, so to ensure higher availability, we considered running these in a multi cloud environment. The announcement of Anthos clusters on AWS at Google Cloud Next ‘19 in San Francisco provided an answer.
We had been wondering how to achieve the equivalent smooth operation of GKE in our AWS environment, and welcomed the Anthos clusters on AWS announcement. We consulted with a Google Cloud customer engineer through an early access program and quickly gained an opportunity to work with this version of Anthos. This allowed us to provide feedback and requests for improvement, and this paved the way for us to implement the product to take advantage of its functionality and future enhancements. With Google Cloud, we have been able to continue to interact closely with the development team to understand and provide input into the product roadmap.
We are now realizing the benefits of multicloud, including faster development speeds and higher availability. For businesses in general, we recommend they take a thoughtful approach to multicloud—while for us, multicloud is a useful mechanism that enables us to provide large-scale data analysis in real time, other businesses should consider whether multicloud is right for them and if so, the role of a technology like Anthos. They should also start small before ramping up. Moving forward, we are keen to see what other products Google Cloud is creating that can help drive our business to a higher level.