Google's guide to building a data-driven culture

Data tells us that a data culture matters.

The digital revolution presents every business with unprecedented opportunity and risk. Cheap and abundant online resources promise new products, new markets, and new opportunities for richer customer relations. They also threaten heated competition and perpetual disruption.

When we’re beset by change, it’s good to remember the unchanging core principles: Know your market. Focus on your customer. Perfect your offering, and be ready to adapt it to changing conditions. Seek efficiency.

In other words, get your data together and use it well. Build a workplace culture around it. That will look very different depending on the people involved. If you provide the same set of technologies and data to two different teams with the objective to innovate, or solve a hard problem, you can get two very different outcomes. Different teams will need to align on their goals and their data to start building a successful culture.

Culture is an accelerating agent in and of itself. According to McKinsey and Company’s Why data culture matters: “Culture can be a compounding problem or a compounding solution. When an organization’s data mission is detached from business strategy and core operations, it should come as no surprise that the results of analytics initiatives may fail to meet expectations. But when excitement about data analytics infuses the entire organization, it becomes a source of energy and momentum. The technology, after all, is amazing. Imagine how far it can go with a culture to match.”

And remember: Using data is nothing new. Since the dawn of commerce, people have observed facts, figured out what matters, and sought patterns to leverage. Modern statistics dates to 1749, and data-powered management has radically raised global GDP for over a century with ever-increasing sophistication. These are revolutionary, data-driven times we live in. We came to them because we used data well.

How people organize their work changes with the amount and quality of the data they have. Ancient farmers used the informal data of watching weather patterns, while industrialists patented standardized machine tools. At the dawn of the computer era, we had applied mathematics and operations research. Now we need a stronger method, one that can become widespread throughout the enterprise.

What does that look like? Let’s start with the magnitude of the opportunity. In 2002, digital storage capacity overtook total analog capacity. Since then, the compounded annual growth rate of data owned by a typical corporation has been about 60%. Not only has the amount of data increased, it now comes from a more diverse set of sources, including browsers, sensors, smartphones and mobile devices, not to mention other computers. The compound annual growth rate of change is incalculable.

Google thinks about these opportunities a lot. We were founded, after all, with a mission to organize all the world’s information, and over the years we’ve solved a number of fascinating problems around yielding insights and action from large amounts of different kinds of data—now done at blistering velocity.

We work to provide digital insights and the capability to take action to both consumers and enterprises, both in our advertising work with businesses and now, through the tools and services for data management and insights we offer at Google Cloud. We hear how our products are helping accelerate digital transformation and innovation at companies around the world, including ANZ, Mayo Clinic, Sanofi, UPS, and more. Take AirAsia as an example, who’re en route to becoming a “digital airline.” Their transformation is already helping them extract new insights, become more agile, and deliver more personalized experiences so they can stand out in their industry. “We had to become a digital airline to offer customers more, personalize customer experiences, and improve booking and ticketing,” says Nikunj Shanti, chief product officer at AirAsia. “We’ve gone from data management to data-driven decision-making.”

We’ve also learned a number of lessons on internal organization to optimize on data, as well, both in our own journey and from helping our customers solve hard problems. Some of those lessons inform this ebook on why a data culture matters. It comes down to four key pillars: operating with trust, democratizing insights, increasing business agility, and applying intelligence.

There are several striking things about organizing for data operations at scale. Advances in the technologies surrounding data means there’s more access and easier manageability. Managing and working with data at scale is hard, and poses a new challenge when compared to working with data in the past; in many cases, this is balanced by today’s better process automation and tools to make sense of the data. Of course, more access means new challenges in security, in quality, and interpretability.

Great businesses are effective because they have great processes that make great products, reflecting great understanding and care for their customers. In other words, all great businesses have great internal cultures that produce these results. People adapt with curiosity and creativity. When appropriate, they challenge the status quo and innovate based on new insights. They leverage the power of data they are entrusted with, adapting and applying processes to generate new value from data.

That’s never been more true than today, when the titanic digital shifts bring into new and sharper focus the need to get culture right around collecting and using data at scale. Getting it right early on is important, because history shows us something else: Those working toward new goals never turn down having more data, as long as it’s useful. Advances in cloud computing, data management, data analytics, and artificial intelligence technologies aren’t slowing down. Neither should any business, in its hunger to change the world. 

When businesses are looking to reinvent their data culture, or create a new one, we often hear that this handful of Google Cloud’s products is useful.

• Cloud SQL to easily migrate on-premises workloads into fully managed Google Cloud
• Cloud Spanner and Cloud Bigtable, cloud-native database services for massive scalability
• BigQuery for petabyte-scale, super-fast data warehousing
• Looker for modern BI, embedded analytics and data-driven applications
• Dataproc for big data processing using Hadoop and Spark clusters
• Dataflow to process both batch and streaming data quickly and serverlessly
• Pub/Sub to send messages to and from independent apps
• Dataprep for fast and visual preparation of data for analysis or machine learning 
• AutoML to train custom ML models without much expertise
• AI Platform so you can take ML models into deployment on-premises or on Google Cloud

Business agility is a sought-after state of being these days, as companies face always-on expectations, quickly evolving technology, and massive amounts of available data. But what is agility, and how do you achieve it? At Google Cloud, we believe it involves people, processes, and data—and the technology that lets you refine that data. It’s letting people in your organization focus on what they’re good at, not on commoditized busywork. It’s about having choice in the tools and services they have access to and having those tools and services available when they need them. It’s less jumping through hoops, and more doing, big thinking, and new ideas.

What we often hear from technologists today—and what’s pretty apparent in our modern world—is that the tools they need are data tools. Companies are awash in data and more is arriving every day. Becoming agile involves standing on the shoulders of others, trying out purpose-built tools, and then not being afraid to use them at scale. These tools include managed databases to remove mundane toil. These tools include big data analytics, tamed and streamlined so many can use the data, not just a few. They include all the collected data a business has, processed in a repeatable way so it’s consistently useful and served in visual and actionable formats. Tools that can ask questions of that data to make better decisions, and even teaching data what you’re looking for to get predictions from it.

When we talk about a concept like agility, we’re talking about attitudes, as well as the technology that a company uses. When we first started out, Google used off-the-shelf technology. As our company grew, we learned a number of important lessons along the way. Because of this scale, we were forced to react and adjust (agility). We realized that having our developers operate their own stacks was not efficient. As we developed technology at Google, we focused on simplicity, automation, and openness. We learned that when users can access fully managed, integrated technology, data takes on a different role. Now, instead of spending time searching for the right data, or tinkering with hardware or software that you need to find or analyze that data, Googlers don’t worry about lack of skills or lack of agility in operating, scaling, or managing data. We believe this kind of mindset can help users work together better and reduce or eliminate silos and bottlenecks that often exist. We’ve found this to work internally as we develop products like Gmail, which has evolved since its creation but remained simple and user-friendly, even at enterprise scales.

We hear from our customers about how they’ve adapted their technology and cultures to a new agile attitude. One global media company, for example, moved multiple data centers and 1,200 services to Google Cloud. Achieving agility was a big driver for the company’s move: higher-level services let them deliver a better product faster, and focus on what their internal teams are best at—developing music services, not managing data centers. The company worked to make the right data available to teams when they needed it, adding agility through tools for streaming data analytics and processing.

Twitter’s analytics infrastructure, built as a data lake, collects petabytes of data each day, and has to be replicated to multiple destinations to serve user needs quickly. The company decided to replicate datasets to Cloud Storage to then be able to use BigQuery, Cloud Bigtable, and Dataproc on Google Cloud, and others. Users can access the data they need with self-service, adding agility across the business.

Global financial institution HSBC migrated its on-premises data warehouse to Google Cloud to become more agile, deliver smaller workloads and build automation into processes. They were able to eliminate technical debt and build a data platform to focus on innovation, not managing infrastructure. Since migrating, HSBC has better development and testing procedures, a single source of truth in the data warehouse, and authorized views for secure data access. Users aren’t constrained in their data exploration, and customers get what they need faster.

Ultimately, an agile business is a successful one, able to respond to market demands, changing customer needs, and unforeseen challenges. Think about how your people and data are interacting now, and if there are ways they could be doing it better.

We know from research that building a data culture has real business results. The culture you create will be specific to your organization’s needs and goals, and ideally bring together people, processes, and data.

The data part, of course, is essential. We’ve talked already about setting up your technology infrastructure so that your data brings the insights you need and helps your business be more agile. It’s also important to apply intelligence to your data so it’s working for you in the moment, but also for the future. AI and ML tools that we couldn’t have imagined a decade ago are available and in use now, not just in academia or for a select few large companies. We hear from businesses using these types of advanced analytics that they’re able to make better decisions, and to stay ahead of their competitors.

In the spirit of a collaborative data culture, though, advanced analytics tools and outputs have to be easily accessible to everyone. Once you’ve established which team needs which information, you can set up users with the right data to do their work. That could be optimizing operations and business processes, conducting research and development, or answering day-to-day questions. Asking analytical questions based on real-time data opens the door to new ideas and innovation. Beyond simply knowing what last month’s sales numbers were, you can apply data to understand an individual customer’s multichannel purchasing record while they’re shopping online, resulting in real-time offers based on a blend of their history and what’s currently in their cart. 

When advanced analytics are woven into familiar tools and applications, they’re already part of daily workflows, making it easy for users to get the answers they need. It also eliminates siloes across teams when sales or marketing leaders, for example, don’t need to request reports from their counterparts on the analytics team. They’re instead able to work together on a single data set to make data-driven decisions faster. And a solid data warehouse foundation means there’s enough capacity to go around, so everyone can get the insights and reports they need.

One Google Cloud analytics customer, the American Cancer Society, uses AI Platform to identify novel patterns in digital pathology images. They trained models for AI image analysis to find cancer indicators, letting researchers more quickly identify them and in turn improve patient outcomes. They’re able to analyze images 12 times faster than before.

LG CNS also uses Google Cloud’s data analytics tools to identify defects on its product manufacturing lines. They had trained machine learning models before, but with Google Cloud have found they can train them faster and be more accurate. LG CNS has saved about $1 million per production line per year with this more efficient process.

At Google Cloud, we build our AI and ML technology into systems like BigQuery for data warehousing, and in places like Contact Center AI and Document Understanding. These types of insights shouldn’t be kept to a select few users inside of a business, or be difficult to access. We built BigQuery ML with a familiar SQL interface, so you only have to write a query to be able to use machine learning. And our built-in advanced analytics features mean you can be prepared for the next generation of smart technology.

We’re firm believers in the idea that intelligence should be embedded across processes, people, and data. When users have what they need, and the process is simple, they can take their great ideas farther.

The kinds of data we can access in our modern world can be mind-expanding, surprising, and above all, useful. The best outcome from gathering lots of data is for it to be available, whether through graphics, reports, or baked into everyday business workflows. That combination of data and human knowledge and understanding can bring new ways of looking at business plans and projects, and spark new insights and ideas.

We have a lot of technology to choose from these days, but it won’t work to move the business forward unless the people using it can access it easily. With all the data sources these days—IoT sensors, business apps, marketing data and more—it’s really easy to end up with siloes of duplicate data across an organization, with different people depending on different sets of information or duplicate sets of data.

Getting to a place where there’s a single source of truth for an organization’s data, and where everyone can access it when and how they need it, is a kind of holy grail. It involves ingesting the necessary data into the data warehouse, creating data models in enterprise BI systems like Looker that can join data from 10 to 20 SaaS apps or across public clouds, then having that source of truth accessible and available to teams without provisioning lags or request bottlenecks along the way. Data governance is essential for the right users to have the right access, too, to keep data protected through its use and lifecycle. What that looks like in practice might be a data analyst running self-serve reports to break down sales numbers, or a machine learning specialist creating models to predict revenue or create a new tool. What a data culture also looks like is team collaboration, not teams intersecting only when they need a task completed. When a data culture is running well, supported by technology, it looks like users having the time and resources they need to pursue new projects that can help move the business forward. 

To get to a place where you can trust and use your data, and ask questions of it to help you move forward, you’ll need a technology foundation that’s equipped for insights. Questions you might ask when you’re choosing data analytics technology are:

• Can you get an integrated view of your organization’s data?
• Can business users at your organization also view data they need easily?
• Can the platform ingest streaming data in real time (without going over budget)? 
• Can the platform handle data types coming in from diverse sources?

Connecting users who can produce more impactful work takes some planning. If you look at the types of stakeholders involved in your company’s AI projects, you’ll likely see that the first step is them connecting and understanding each other’s work to make the project successful. The tools and services they use have to support that connection work.

We’ve heard from plenty of customers aiming to build a more data-driven culture at the same time they’re updating technology and planning for the future. AirAsia, for example, knew that data was continuing to play a bigger and bigger role in their business decisions. Their teams analyze >6 PB of data a month, and they previously had to wait on weekly or monthly Excel reporting to get results from all that analysis. They’re using BigQuery now and its ML capabilities to allow its 1,500-member team to access data anytime and do ad hoc querying on more than two years’ worth of data in minutes. Team members access straightforward interfaces and can then drill down into granular details as needed. That’s helped reduce operational costs by 5% to 10%—quite significant in that industry—and secure data more than it was when stored on individual machines.

Our BigQuery data warehouse lets you process data in object storage, transactional databases, Sheets, and more, so you never have to duplicate data. It includes public datasets, too, so you can get even more value from your internal data. We build our products to be easy to use so that users can easily work together and share insights. We also build in capabilities that you can use now and going forward, like straightforward ML tools and other advanced analytics.

Building a data culture brings together people, processes, and data. We’ve talked about making sure it’s the right data, and that the experience of using that data is easy. Part of that idea of streamlined data collection and use is that it’s trusted. With lots of on-premises or legacy systems, data gets siloed, and copied over and over again by users till there are outdated systems in lots of different places. In addition, once datasets start growing larger, data quality can become a problem.

So one big step in achieving a data-driven culture is making it easy for users to trust data—trust that it’s the most up to date, and that it’s accurate. As part of that, there shouldn’t be any of those outlying datasets or siloed projects. The gold standard is that there’s one single source of data truth at your organization, and everyone knows what it is.

Trust is a people and processes issue that can make or break your data culture goals. It’s a bigger data technology issue, too. The more data you’re collecting and analyzing, the more security and governance matters. Confidence in your own tools and processes is a big part of a successful data-driven culture.

Data comes from a huge variety of sources, but it all requires the same baseline level of protection that includes encryption, data exfiltration, and access controls. Beyond that, you’ll likely have your own threat models and areas of focus for protecting data. Depending on your industry, you might be working on complying with specific compliance and regulation rules. Your cloud provider’s technology stack should be able to provide the security features and let you use metadata, data catalogs, and data lineage to govern data and set policies. And, to allow for team collaboration, it should be easy to set access rules and allow for safe data sharing.

Putting this into practice is essential for a successful data analytics rollout, so consider the big picture of how you’ll implement security practices and tools. At Google, BeyondCorp is our own implementation of a zero-trust security model that grew out of original zero-trust networking principles. The idea behind this is that access controls shift from the network perimeter to individual users and devices, so users can work securely from any location without a VPN. At a high level, BeyondCorp includes single sign-on, access proxy, access control engine, user inventory, device inventory, security policies, and a trust repository. BeyondCorp Remote Access is a cloud solution that offers this capability, so your users can access internal web apps safely and remotely.

One of our customers, a global financial institution, had a few essential questions as it prepared for its BigQuery migration: Is data protected, and can its lineage be traced? Is the infrastructure reliable enough for mission-critical apps? They used metadata tagging and gained a single source of truth by migrating to Google Cloud. Their data in BigQuery is always encrypted, and the native Identity and Access Management (IAM) integration means that data access levels are assured, down to column-level granularity. The organization can run risk simulations regularly and has improved security, governance, and regulatory posture using BigQuery.

A data analytics platform you trust can help safeguard against attacks or security breaches, but it also has a lot to do with building a solid data culture. If users trust the data they’re getting, they’re able to focus on innovation and new ideas. 

Will data culture be a problem or a solution for your organization? And if it’s a solution, what will that look like? How will you know it’s a success? These questions can help you get started or build on what’s already been started. Data culture will look different for different businesses in different industries. You and your stakeholders will know what’s best for your teams and organization, and the data culture you build will be the one that works best for your business.