How Databricks and Google Cloud enable Uplight to deliver energy analytics solutions efficiently

You know how the saying goes, “with great power comes great…” amounts of data. And what if that data could help lower our carbon footprint?

Uplight is on a mission to do just that. Uplight helps energy providers create a more sustainable future by using raw data and derived insights to motivate actions in their customers. These actions vary from implementing consistent behavior change to purchasing energy efficient devices and enrolling in grid-supportive programs. “We enable energy customers to make informed choices about their energy-related actions, and ensure those actions are also aligned with positive grid outcomes, ” says Micaela Christopher, Director of Data Science at Uplight. 

And with over 60 million endpoints — from smart thermostats and usage meters to electric vehicle (EV) chargers and solar arrays — Uplight sure has a lot of data to process.

Unlocking actionable analytics

As Uplight has grown, so too has the volume of data it ingests and processes and “our legacy Spark management system and siloed applications couldn’t keep pace with increasing demand for our services,” says Christopher. “We needed a faster, smarter way of extracting meaningful information from the gigabytes of raw data we import daily. We also wanted to quickly introduce new, high-value energy insights for energy providers and their customers.”

To derive the most insights from its data, Uplight chose to work with Google Cloud and Databricks. With Databricks on Google Cloud, procured through the Google Cloud Marketplace, Uplight ingests and analyzes data from utility systems, sociodemographic sources, OEM partners, weather stations, and more. Uplight can now efficiently unlock actionable analytics and develop new services that continue to enhance their solutions to help people adjust their energy use to save money and reduce environmental impacts.

"We are running 30,000 Spark jobs per month on Databricks to ingest, prepare, and analyze our data,” says Daniel Coll, Senior Data Engineer at Uplight. “We’re leveraging BigQuery, BigTable, Pub/Sub, and Google Kubernetes Engine (GKE) for peak performance in our data platform, and using Databricks ML and Vertex AI we’ve halved development time for new services and accelerated the rollout of new predictive machine learning (ML) models.”

Preventing rolling power outages

Developed with Databricks MLFlow and deployed with Vertex AI, Uplight’s predictive ML models analyze energy consumption patterns to pull out actionable insights targeted at utility customers and/or utility employees. This information, for example, helps utility companies proactively alert customers of recommended specific actions — such as enrolling in demand response programs — meant to conserve energy and prevent grid overload.

Uplight’s advanced AI models and customer programs contributed to helping California utility companies avoid rolling power outages amid a dangerous heat wave in 2022. As temperatures topped 115 degrees, demand for electricity to cool homes and businesses approached an unprecedented 52,000MW. Demand Response events were called and smart thermostats were automatically adjusted to help manage this peak load. At scale, across multiple utilities and tens of thousands of homes, the cumulative effect of these small changes cut more than 1,200MW of energy from the grid.

Uplight’s demand-response and time-of-use programs include smart thermostats, water heaters, EV chargers, and even the EVs themselves and the company has plans to expand to home battery packs and beyond. According to the Brattle Group, optimized demand side management could positively impact 200GW of load flexibility, or approximately 20% of total U.S. grid demand. 

“We look forward to continuing to collaborate with energy providers to realize our vision of a smart, connected, interactive and decarbonized grid edge,” says Christopher.

Creating a greener, more sustainable future

Uplight continuously adds new applications and systems to its platform using Databricks on Google Cloud. Although most integrations are typically completed in hours, Uplight’s data engineering team works closely with Databricks to expedite even the most complex implementations.

“Seamless integration between Databricks and the full stack of Google Cloud solutions and robust security have been central to our success,” adds Coll. “Because the data plane and clusters run within our Google Cloud account, we can easily manage identity permissions with BigQuery and quickly configure clusters to securely access protected network resources. This is a major advantage when it comes to networking and security.”

With Databricks on Google Cloud, Uplight cost-effectively scales to support new use cases and launch additional services that help utility companies — and their customers — significantly reduce energy consumption and costs. As Christopher emphasizes, energy providers need to make it easy for people to understand how much energy they use while highlighting how they can better conserve and optimize their usage. 

“Uplight empowers sustainable change with actionable energy analytics and programs that minimize carbon footprints,” concludes Christopher. “We’re proud to educate, engage, and enable people so they can take a step back and say, ‘All right, I’ve got this, I'm doing what I need to do to create a greener future.”

To learn more about how Google Cloud is helping pave the way for a more sustainable future, read other recent stories on green initiatives and research at the Google Cloud blog.