Affable: Processing 100m events per day with data pipelines created with Google Cloud
About Affable
Affable is a Singapore-based startup that has built an AI-based influencer marketing platform. The business tracks over 1 million micro-influencers across Facebook, YouTube, Instagram, and other social media services. Its platform can provide curated lists of relevant influencers to clients, complete with demographics, interests, and brand partnerships of followers. These lists are generated through machine learning, which also allows the business to detect fake followers.
Tell us your challenge. We're here to help.
Contact usWith Google Cloud, Affable is processing data and images to recommend to clients the key micro-influencers who may be most effective in helping them promote their products and services. The business is tracking 1 million micro-influencers, processing 100 million events through its data pipelines, and using machine learning models to serve up to 20 million image requests per day.
Google Cloud results
- Saves 20 hours of development productivity per engineer per month
- Delivers projects 40% faster than if using other cloud services
- Creates recommendation engine in just two weeks
Serving 20m image requests per day via ML that would otherwise need 1K humans to process
Founded in October 2017 as a software as a service platform, Affable enables brands and marketers to discover, measure, and engage social media “micro-influencers”—presenting opportunities to promote products and services. Micro-influencers typically have a few thousand to tens of thousands of followers and operate in niches such as health, food, and fashion. Many micro-influencers engage extensively with their followers—and can impact purchasing decisions and brand value.
“We track about 1 million influencers across services such as Facebook, YouTube, and Instagram, and break down the community by age, gender, location, and interests,” says Swayam Narain, co-founder and Chief Technology Officer at Affable. The Affable platform can provide curated lists of relevant influencers to clients, complete with demographics, interests, and brand partnerships of followers. Furthermore, the platform can give clients reports into the authenticity of influencers’ audiences—enabling those clients to avoid influencers with large numbers of fake followers.
Agility key to success
Agility and responsiveness are critical to the success of the business. “We are in a brand-new industry that is evolving quickly,” says Narain. “So when, for example, a new social media service becomes popular, clients will want influencer analytics quickly. In addition, marketers will want data points on which to inform decisions about working with influencers, and if they ask us to provide a new metric, we need to migrate the relevant data over for reprocessing and make the data available quickly.”
“Google Cloud was also more developer friendly, meaning our team could get started and complete work faster. In addition, because Google Cloud was scalable and incorporated a range of managed services, we could execute our strategy of focusing our developers and data scientists on delivering value.”
—Swayam Narain, co-founder and Chief Technology Officer, AffableTo meet client demands for the fast delivery of accurate, actionable influencer data, Affable devised a technology strategy based on a lean engineering organization that used managed services extensively. This meant developers and data scientists could solve business problems rather than spend valuable time on technology “plumbing.” Affable’s founders reviewed multinational cloud providers and started running its platform on a traditional cloud service. However, the business soon saw an opportunity to use machine learning and big data analysis to create even more value for its clients.
“Inference is very important for our business, so we needed to analyze images and understand what they meant in order to properly categorize influencers as operating in fashion, sports, food, or other niche,” says Narain. “This meant using models that could analyze and learn from streams of photos and other sources of raw data.”
Google Cloud easiest to use
Affable then evaluated Google Cloud and found the platform was easiest to use. In addition, Google Cloud had compelling machine learning and big data processing, management, and analysis services. “Google Cloud was also more developer friendly, meaning our team could get started and complete work faster,” says Narain. “In addition, because the platform was scalable and incorporated a range of managed services, we could execute our strategy of focusing our developers and data scientists on delivering value.”
“Finally, Google Cloud enabled us to pay only on usage to receive a discount, whereas other cloud providers required us to pay up front before applying any discounts.”
“The Google Cloud team pointed us in the right direction when we started the implementation—specifically Google Cloud support engineers who helped us decide the best storage option for our needs.”
—Swayam Narain, co-founder and Chief Technology Officer, AffableAffable finalized its Google Cloud deployment in April 2019 architecture as use of its platform began to surge. The organization completed the implementation using internal resources with support from Google Cloud. “The Google Cloud team pointed us in the right direction when we started the implementation—specifically Google Cloud support engineers who helped us decide the best storage option for our needs,” says Narain. “Our session with them was helpful in formulating the architecture of our data platform.”
The business’s Google Cloud architecture comprises AI Platform, to take its machine learning projects from idea to production and deployment; BigQuery to provide analytics data warehousing; Cloud SQL to set up, run, and maintain its relational database; and Pub/Sub to provide scalable event ingestion and delivery.
Recommendation engine built in two weeks
The agility enabled by Google Cloud’s ease of use and highly functional services has already paid off for Affable.“With AI Platform, Pub/Sub, and Cloud SQL, we were able to build a recommendation engine—that identifies the influencers most compatible to clients’ needs—in just two weeks,” says Narain. “Pub/Sub acts as our data-streaming backbone that funnels data to the engine from various sources, including raw data from our data aggregators and inferences from machine learning models created and run on AI Platform.”
“Our engine transforms the data into a queryable format backed by a PostgreSQL open-source relational database to serve recommendation requests.”
Narain is particularly pleased by the contribution of Pub/Sub to the rapid development of the engine. “If we could not use Pub/Sub to reliably deliver messages at scale, we could not have completed the project in just two weeks,” he says. Week one entailed developing the algorithm and associated decision criteria, while the business dedicated week two to establishing processes for modifying upstream data sources to support new metrics.
Serving 20 million requests per day
Meanwhile, Affable is using AI Platform to serve up to 20 million requests per day. “We use AI Platform to understand the images in photos—for example, we can identify whether a photograph was taken in a cafe by identifying items such as coffee cups, tables, and chairs, and use these to help determine the categories to assign influencers to,” says Narain. “We have set up pipelines and trained models so they can do the job of 1,000 human beings who would otherwise be working to annotate, categorize, and explain each image.”
“Overall, our four engineers have built data pipelines that process more than 100 million events per day.”
“Overall, Google Cloud has enabled us to build a lean engineering team that maximizes developer productivity. The ability of its managed services to scale elastically allows our team to focus on delivering business value without being bogged down by the effort needed to scale and maintain data pipelines.”
—Swayam Narain, co-founder and Chief Technology Officer, AffableSaves 20 hours of development productivity per engineer
The business estimates Google Cloud’s integrated big data and machine learning services saves it at least 20 hours of development productivity per engineer per month, and enables it to deliver projects 40% faster than if it was using other cloud services.
Affable is now looking at building on its successful use of Google Cloud by adding Google Kubernetes Engine for containerization and making BigQuery the primary data warehouse of its business. “Overall, Google Cloud has enabled us to build a lean engineering team that maximizes developer productivity,” says Narain. “The ability of its managed services to scale elastically allows our team to focus on delivering business value without being bogged down by the effort needed to scale and maintain data pipelines.”
Tell us your challenge. We're here to help.
Contact us
About Affable
Affable is a Singapore-based startup that has built an AI-based influencer marketing platform. The business tracks over 1 million micro-influencers across Facebook, YouTube, Instagram, and other social media services. Its platform can provide curated lists of relevant influencers to clients, complete with demographics, interests, and brand partnerships of followers. These lists are generated through machine learning, which also allows the business to detect fake followers.