Microba: Turning to Google Cloud ML and data infrastructure to help transform gut health
About Microba
Based in Brisbane, Microba is an Australian business that is establishing itself as a world leader in analysis of the gut microbiome. With global research indicating the gut microbiome plays a central role in health and disease, Microba is a strong contributor to the development of new pathology services, therapeutics, and diagnostics. Microba emerged from research conducted by Professors Gene Tyson and Philip Hugenholtz over the past 20 years and has been made possible by advances in computing power over the past decade.
Tell us your challenge. We're here to help.
Contact us
About Max Kelsen
Max Kelsen is an AI and machine learning consultancy that aims to provide a competitive advantage for government organizations and enterprises. The business has a team of data scientists, research analysts, and software engineers who provide solutions to clients.
With Google Cloud, Microba can access massive compute on demand and process large volumes of data to analyze the gut microbiome and its relationship to mental health, metabolic disease, and gastrointestinal disorders such as IBD. The business has cut storage costs by 20%, reduced its processing costs by 50% through the use of preemptible instances and Google Kubernetes Engine, moved workload storage into a cloud-native architecture, and reduced the time for sequence run processing from 48 hours to under six hours.
Google Cloud results
- Allows the business to spend more time researching and creating new products to improve healthcare outcomes
- Paves the way for new diagnostic tools based on better understanding of bacteria-human health relationships
- Supports seamless expansion of a database of gut microbiome data to support research, analysis, and product development
Reduces the time needed to analyze the gut microbiome from days to hours
The gut microbiome—the trillions of microorganisms that reside in each person’s gut—plays a key role in overall health and wellbeing. Scientists are increasingly identifying links between the state of the microbiome and health conditions such as inflammatory bowel disease, mental health, and metabolic health. Born from research conducted by co-founders Professor Gene Tyson and Professor Philip Hugenholtz, Queensland-based early-stage biotech company Microba aims to contribute to the development of new pathology services, therapeutics, and diagnostics based on the gut microbiome. “Our vision is to improve lives through a deeper understanding of the gut microbiome using scientific knowledge and approaches,” says Blake Wills, Chief Executive Officer of Microba.
Microba began providing metagenomic gut microbiome analysis to the public in July 2018 with the launch of a home analysis kit, Microba Insight™. “Our kit gives individuals access to their own unique gut microbiome profile and information about how the microorganisms in their gut may be influencing other areas of health,” says Wills.
The business has also developed a proprietary technology platform—the Metagenomic Analysis Platform (MAP™)—that enables its researchers to identify bacteria at very high sensitivity and has been using the platform to research bacterial involvement in gastrointestinal conditions. “Research into inflammatory bowel disease by Microba scientists has demonstrated the condition can be identified with microbiome data alone at more than 90% accuracy,” says Wills.
To further understanding of the gut microbiome, Microba is using testing services to create a continuously growing database of metagenomic microbiome samples and associated health and lifestyle data and invites customers to contribute their anonymous data to the repository. “We will then leverage this database to identify unique patterns indicative of different health conditions using advanced machine learning approaches,” says Wills. “This will allow us to identify promising therapeutic leads and develop non-invasive diagnostic tests.”
The business encourages healthcare practitioners to use the analyses in their practice and has established partnerships and research projects with businesses, academic institutions, and sporting teams in Australia and internationally.
Basing its research on metagenomics—whole genome DNA sequencing—means Microba generates large volumes of data for a business its size. “This data has large processing requirements to produce the reports delivered to customers, as well as the bespoke analytical requirements required, which are similar to a research setting,” says Wills. “We also follow a ‘burst’ pattern with our data processing—meaning there are large amounts of data to process very quickly—and then a period of several days of low volume.”
The business found these volumes and processing requirements were ideally suited to the cloud and began exploring options for moving from its existing technology infrastructure to a public cloud service.
“With Google Cloud, we have an advanced and secure data infrastructure to store, process, explore, and share large, complex datasets, with the aim being to reduce the time to undertake bioinformatic analysis, so we can better support customers.”
—Blake Wills, Chief Executive Officer, MicrobaGoogle Cloud best fit for the business
Microba’s evaluation led the business to Google Cloud. “We chose Google Cloud for a number of reasons, including the availability of technical support and Google’s interest in our organization—as well as its expertise in managing the Kubernetes system that automates the deployment, scaling, and management of containerized applications,” says Dr. David Wood, Microba’s Head of Bioinformatics Operations. “While other providers offered a Kubernetes service, Google Cloud developed the technology and Google Kubernetes Engine was easy for us to use and adopt into our workflows.”
Google Cloud’s AI and machine learning services also offered rich potential for Microba’s research division to further improve the organization’s microbiome analyses.
Scale to 5,000 cores in minutes
Microba formally decided to run its workloads on Google Cloud and worked with technology partner Max Kelsen to undertake the migration. "Providing computing at the scale and elasticity required by Microba was an exciting challenge,” says Max Kelsen, Chief Executive Officer and co-founder of Nicholas Therkelsen-Terry. “By leveraging Google Cloud's particular capabilities, including the ability to size virtual machine instances to our needs, high throughput networking, and Kubernetes, we achieved a state that ticks the technical requirements at the same time as delivering significant cost reductions."
The business now runs its bioinformatics pipelines as containerized applications with rapid scalability on Google Kubernetes Engine. Changes to the pipelines are persistently retained in Container Registry, enabling easy reproduction and continual improvement. Increases in data processing needs are easily handled. BigQuery helps mine data logs, with the organization planning to use the analytics data warehouse to review terabytes of structured data and search thousands of genomic sequences.
Cloud Storage stores terabytes of data and provides fast access to nearline and coldline data if needed, while Compute Engine provides virtual machines, including multiple high-CPU and high-RAM machines and custom machines for bespoke bioinformatics applications. The business uses Pub/Sub to queue messages.
“We needed a powerful data analytics service to produce Microba Insight™ reports for each customer,” says Wills. “With Google Cloud, we have an advanced and secure data infrastructure to store, process, explore, and share large, complex datasets, with the aim being to reduce the time to undertake bioinformatic analysis, so we can better support customers.”
The project entailed rebuilding genomic processing pipelines in Google Cloud to enable more scalable, efficient delivery of services to customers. “We set up our advanced bioinformatics production processing pipelines using Google Kubernetes Engine, and the service enabled us to scale from zero to over 5,000 cores in minutes, enabling very fast turnaround times for deep datasets,” says Wills.
“We set up our production processing pipelines using Google Kubernetes Engine, and the service enabled us to scale from zero to over 5,000 cores in minutes.”
—Blake Wills, Chief Executive Officer, MicrobaMore than 10,000 samples pushed through pipeline
“Working with a global powerhouse like Google Cloud has enabled us to combine expert scientific knowledge with high-powered analytics,” says Wills. “Since we launched Microba Insight™ in June 2018, we have pushed more than 10,000 samples through our proprietary bioinformatics pipeline.” Migrating the pipeline to Google Cloud enables the business to spend less time designing architectures and infrastructure for daily operations and more on optimizing and developing new features for research—and delivering greater value to customers.
“Our collaboration with Google Cloud and partner Max Kelsen has cut our storage costs by 20%, reduced our processing costs by 50% through the use of preemptible instances and Google Kubernetes Engine, allowed us to scale up to thousands of CPUs when processing is needed and down to zero when it is not, moved our workload storage into a cloud-native architecture, and reduced the time for sequence run processing from 48 hours to under six hours,” says Dr Wood. “The very fast internal data interconnects within Google Cloud also allow us to move large volumes of data around very quickly, enabling an architecture that we envisage will scale to many times our current data workload rate.”
With BigQuery, Microba is processing more than 300 GB of log files per month, while hardware such as tensor processing units—Google’s custom-developed application-specific integrated circuits used to accelerate machine learning workloads—enables the rapid training of machine learning models. “We have deployed AI to explore complex relationships between bacteria and human health, with the findings from this program delivering promising leads for new diagnostic tools using the gut microbiome and AI,” says Wills.
Microba relies heavily on Cloud Storage for cost-effective storage and support for its production pipelines. “We aim to build the world’s largest database of metagenomic gut microbiome samples and assisted health and lifestyle data,” says Wills. “As we grow the database, we will use Google Cloud machine learning to identify significant associations between microorganisms and genes and use these to accurately predict different disease states. We will also use AI to explore more complex relationships between bacteria and human health, delivering promising leads for new diagnostic tools.”
“With Google Cloud, an early-stage company such as Microba can have massive compute readily available when needed to scale and burst-process samples and deliver microbiome services rapidly and at low cost.”
—Blake Wills, Chief Executive Officer, MicrobaIn addition, migrating its bioinformatics pipelines to Google Cloud enables the business to spend less time designing architecture and infrastructure for daily operations and more time developing new features for research.
The new architecture also allows Microba to stand up new projects in different regions with minimal effort, while the adoption of Google Cloud enables the business to recruit talented cloud engineers with ambitions to help transform healthcare.
Microba is now considering extending its use of Google Cloud machine learning services and exploring the potential of Anthos to deliver a hybrid on-premises and cloud-based infrastructure.
“With Google Cloud, an early-stage company such as Microba can have massive compute readily available when needed to scale and burst-process samples and deliver microbiome services rapidly and at low cost,” says Wills. “As Microba scales up, so will our computing requirements, and Google Cloud will meet these with ease.
“Partnering with Google has been an enjoyable and fruitful relationship, as both organizations share foundations of engineering and technology, and Google Cloud has taken an interest in our work and tangibly contributed by supporting our platform build with expertise and know-how,” he adds. “Google Cloud and Max Kelsen provided support that proved critical to establishing our existing platform. We are extremely satisfied with the services and technology Google Cloud provided to date and look forward to continuing to use these services into the future as we grow and expand into new territories.”
Tell us your challenge. We're here to help.
Contact us
About Microba
Based in Brisbane, Microba is an Australian business that is establishing itself as a world leader in analysis of the gut microbiome. With global research indicating the gut microbiome plays a central role in health and disease, Microba is a strong contributor to the development of new pathology services, therapeutics, and diagnostics. Microba emerged from research conducted by Professors Gene Tyson and Philip Hugenholtz over the past 20 years and has been made possible by advances in computing power over the past decade.
About Max Kelsen
Max Kelsen is an AI and machine learning consultancy that aims to provide a competitive advantage for government organizations and enterprises. The business has a team of data scientists, research analysts, and software engineers who provide solutions to clients.