Whisper: Building the media company of the future
Authenticity is not often associated with traditional social media sites, where people are more cautious about what they post or tweet. But sometimes, people want to express themselves, share their feelings, and get support and advice without anyone knowing their identity. Enter Whisper.sh, a free mobile app that allows users to share their honest thoughts and feelings without identities in an online community without worrying about how their friends, family, or coworkers might judge them.
As it turns out, people love to share their true thoughts, feelings, and secrets—and they love reading other people’s even more. Whisper’s active user base quickly grew to more than 30 million, with an audience of 250 million people consuming Whisper content on its mobile and desktop websites, social channels, and publisher network every month. Billions of page views quickly attracted a large and growing advertising roster, and a clever idea became a major media force.
Massive growth brought revenue, but also the challenges of scaling for performance, availability, and content moderation. With its previous cloud provider, Whisper had to predict how much compute and storage it would need in the foreseeable future—and pay for those reserved instances up front. It also experienced stability issues when cloud servers rebooted or were taken offline for maintenance, sometimes impacting service availability to end users. In addition, Whisper was challenged to scale to quickly identify good and bad “whispers” and keep bad content from being displayed to users.
To scale cost effectively and gain access to powerful analytics and data processing tools, Whisper moved its production services to Google Cloud Platform. No longer constrained by inflexible payment plans or unreliable infrastructure, Whisper is free to grow its business, introduce new features to delight audiences, and use artificial intelligence to help keep content compelling and appropriate. It can take advantage of sustained use discounts with no up-front commitment required.
“We wanted a cloud provider that would help us grow instead of making us predict our usage, pay up front, and write our own scripts for high availability. Google Cloud Platform lets us focus on our core business—and we’re saving more than 50% on IT.” - Antonio Lobato, Director of Infrastructure and Operations, Whisper
Managing resources effectively
As its user base continues to grow, Whisper must keep services available, resilient, and easy to manage. Google Compute Engine offers transparent maintenance and Live Migration technology, automatically moving virtual machines (VMs) to nearby hosts while underlying host machines undergo maintenance. Google Cloud Storage optimizes the availability of live data, such as images and streaming video that is critical to the user experience.
“It’s extremely rare that we have customer-facing downtime since moving to Google Cloud Platform. We can safely assume that servers will be up when we expect them to be,” says Antonio.
Whisper’s technology relies heavily on data science microservices, which were previously time consuming and to manage and expensive to host on traditional VMs. Today, Whisper runs microservices on Google Kubernetes Engine, enabling the company to launch new services and features in hours or minutes instead of days while reclaiming 20 hours of employee productivity in an average week.
“Google Kubernetes Engine reduced our server count for data science microservices from more than 100 to less than 30. It saves us a lot of money and gives us all the benefits of containers without the management hassles. We can set up a cluster in minutes with just a few clicks,” says Antonio.
A new vision for security and access control
To ensure that users do not abuse its platform, Whisper makes strong security and user access control a necessity. With Google Cloud Platform, data at rest can be automatically encrypted, with no action required. For user identification and authorization, Whisper uses Google Cloud Identity Aware-Proxy (IAP).
“Google Cloud IAP has made our service safer and more secure, even compared to a VPN solution. It gave us a new vision for access control. We can rest assured that everybody uses two-factor authentication, and it’s almost management-free,” says Antonio.
Big data analytics in seconds
Whisper is constantly analyzing large amounts of data to better understand user behavior and advertising performance. Before moving to Google Cloud Platform, the company used a common distributed SQL engine for big data, which required its own infrastructure and sometimes took hours to return answers. Although Whisper’s previous cloud provider offered services for big data analytics and reporting, they were expensive and did not offer sufficient performance and flexibility.
Now, Whisper uses Google BigQuery to warehouse data cost effectively and analyze millions of whispers and hundreds of lead campaigns every day. As the data gets bigger, performance remains consistent.
“Queries that took 4 to 16 hours to run on our old system complete in seconds with Google BigQuery, so we get answers much faster. BigQuery has been absolutely reliable—it doesn’t crash—and it’s much more cost effective because we don’t have to run all those servers,” adds Antonio.
Zero-management data processing
Every day, Whisper imports billions of events into BigQuery using Google Cloud Dataflow, a fully managed data processing service. “The zero management aspect of Google Cloud Dataflow is extremely attractive, as it allows our operations people to write jobs as code without thinking about servers or VMs,” says Antonio. To import operational data such as users and whispers into BigQuery, Whisper uses Google Cloud Dataproc, a managed service for Apache Spark that is fast, easy to use, and low-cost.
“Google Cloud Dataproc has completely eliminated the need for us to manage Spark clusters manually. With a single click, we have a fully operating cluster that is blazing fast,” says Antonio.
Scalability, consistency, performance—and cost savings
The Whisper team is using Google Spanner, a managed relational database service, to provide consistency when counting user activity, an essential element of tracking and quality control. As Whisper grows, the database will always grow with it.
“With Google Spanner, there are no worries about how we can scale our whisper counting microservice, and there’s no management required. It allowed us to solve a difficult problem in hours that would have required weeks using a traditional database, and would not have been nearly as scalable,” says Antonio.
To audit user behavior on-the-fly with low latency and high throughput, Whisper stores data about the content users have viewed in Google Cloud Bigtable. The database is growing by more than a terabyte a day. “We use Google Bigtable very heavily in situations where we need to store massive amounts of data. It’s helping us improve how we manage ancillary services while saving us close to $1 million a year versus an off-the-shelf database,” says Antonio.
Innovating faster with G Suite
Since Whisper was founded, its staff has used G Suite, including Gmail, Google Drive, Google Sheets, Google Docs, and Google Calendar to stay productive. To conduct meetings and collaborate on engineering problems, Whisper uses Google Hangouts.
“G Suite helped us at the beginning and has been a great fit for our long-term plans as well. We’re sharing more files on Google Drive and using Hangouts more and more. With Google behind our productivity applications, security, and reliability are never concerns,” says Antonio.
What cloud computing should be
With Google Cloud Platform, Whisper can move its business at a faster pace and push the boundaries of data science to make its unique service even more compelling.
“Google Cloud Platform is what cloud computing is supposed to be. We get a convenient, on-demand system that’s incredibly cost effective,” adds Antonio.