Kofera relies on GCP scalability and machine learning to deliver marketing automation services
Founded in March 2016 in Indonesia, Kofera provides machine learning-based marketing automation services to clients. These services aim to help clients spend budgets more effectively and optimize their campaigns. “Our primary differentiator from our competitors is our use of machine learning end-to-end in campaign services - incorporating campaign building and campaign monitoring as well as optimization,” says Bachtiar Rifai, Founder and Chief Executive Officer, Kofera.
The business focused on e-commerce clients before expanding into agencies and small-to-medium businesses. “The challenge clients face is the range of the marketing channels available and complexity of options before them,” says Rifai. “Running effective campaigns can involve multiple teams and require sizable investment. Automating many of the campaign creation and management processes using machine learning could reduce the cost and improve the outcomes for businesses. We quickly realized we could reduce campaign building from up to two weeks to five minutes.”
Kofera started operations with six team members and a simple application. However, as the number of clients increased, data volumes increased and the company’s platform installed on bare metal servers began to experience problems. Memory resources were exhausted and CPU performance slowed. The business trialled a range of systems to help scale out its platform, such as installing a Hadoop framework to manage data and migrating data to a Cassandra NoSQL distributed database management system. However, the time and resources required to manage these systems exceeded Kofera’s business requirements. In December 2016, the business resolved to review the functionality and performance of Google Cloud Platform.
“Our team was most interested in using Google BigQuery for data analytics due to its support for our time-series data that incorporated a range of metrics and dimensions,” says Rifai. “We were also interested in migrating our applications to Google App Engine so we could dedicate our resources to code without worrying about applications.” The business was also keen to use Cloud BigTable to run its NoSQL Big Data database and Google Cloud Pub/Sub as the messaging middleware for its applications.
Kofera completed the migration to GCP in February 2017. Google Compute Engine is running the business’s Ruby on Rails web application, developing, testing and staging environments, and training and testing data with machine learning algorithms. Google App Engine runs Kofera’s code for microservices-based campaign building and optimization applications that are deployed to a container-based environment. Each application service communicates using a REST API. Google App Engine also cleans, transforms and selects data before it is ingested into the Kofera database or data warehouse.
Google Cloud Storage provides object cloud storage for the web application’s asset files, saves datafeeds uploaded by clients, archives data reports from client and serves data training for machine learning. A Cloud SQL database hosts user management, payment transaction and web application configuration and session data. Cloud Bigtable manages attribute and dimension data, notification data and geographical data, while Cloud DataStore hosts campaign builder data, application task status, application error description logs and product category information.
Rifai describes Cloud Machine Learning as an “awesome” technology for performing deep learning and Kofera uses the system for critical tasks. These include optimizing budget allocations for advertising spending, classification of keywords and bid predictions. Cloud Pub/Sub is used to stream large data from APIs to its database and warehouse, and to queue tasks and distribute them to workers.
Minimal maintenance and rapid response provide a competitive edge
The business has realized all the benefits anticipated from GCP. “We are no longer preoccupied with server maintenance and installation - we only need to drag and drop to complete processes,” says Rifai. “We have also been able to use the auto scaling features of Google App Engine and the instance groups in Google Compute Engine in combination with Google Cloud Pub/Sub to deliver a scalable application to help clients build campaigns and optimize budgets.”
Google Cloud BigQuery and Google Bigtable have also enabled Kofera to improve the speed of queries of large data sets. “We can visualize queries of 1TB of data on our dashboard in as little as one second,” says Rifai. “In our previous bare-metal server environment these queries could take up to three minutes to complete.” This improvement has translated to an improved customer experience. The vast majority (90 percent) of Kofera’s customers rate dashboard queries as considerably faster than in the previous environment.
The deployment time for new applications has fallen from months to days, while Kofera has been able to reduce its infrastructure budget by 70 percent or USD$470,000 per year - money the business has been able to allocate to improving its application and supporting growth. Kofera now has 40 employees and is on a fast track to further expansion.
Using GCP has enabled Kofera to get a step ahead of its competitors and provide a high-quality service to consumers. “We are a very small team and we don’t have huge amounts of dollars to manage infrastructure,” says Rifai. “GCP meets our cost, performance and availability requirements while scaling quickly when we need it to.”