Flipping out: Modernizing a classic pinball machine with cloud connectivity

In today's cloud-centric world, we often take for granted the ease with which we can integrate our applications with a vast array of powerful cloud services. However, there are still countless legacy systems and other constrained environments where integration is far from straightforward.

We faced this challenge head-on when building Backlogged Pinball, a custom pinball game that we built as a demo for integrating cloud services in uncommon environments. Backlogged Pinball is a physical pinball machine that connects to the cloud for a variety of services — think keeping track of data about current and completed games, updating leaderboards, etc. To build it, we used a base of a commercially available programmable pinball machine so we could focus on game code and cloud integration. However, the machine's software environment was limited, running on a sandboxed version of .NET Framework 3.5, which was first released 17 years ago. Practically, this meant that we couldn't use any of the modern Google cloud SDKs available for C#, and we couldn’t install tools like gcloud to help communicate with the cloud.

There’s a catch

We knew we wanted to take advantage of the cloud for databases (for high scores, and stats from the game), logging (of game events and results), and a custom service (to change the game experience on the fly). But developing software for such a constrained environment presented a variety of challenges, which might be familiar to you:

Minimal library support: If you have full control over your stack, there’s no shortage of great libraries to help you connect to cloud services. But sometimes you don’t get to pick where your software runs. For our pinball machine, it was difficult to find compatible libraries to integrate with the cloud services we wanted. For example, we knew we wanted to insert records into a Firestore database to drive a real-time visualization of everything going on in the game. Firestore has great SDKs, but they couldn’t support anything before .NET Framework 4.6.2 (which is 8 years old). We might have been able to connect to a traditional relational database using a TCP connection, but we didn’t want to be limited in the cloud tools and services we could use. Needless to say, it’s much less practical to build a real-time web application with MySQL rather than Firestore, which is designed from the ground up to push data to the browser in real time.

Difficult deployment process: Maybe you have other limitations that make updating your on-device software difficult, but you still want to add new features and cloud integrations. As third-party developers, we had to manually install each version of our game during development using a USB stick. This kind of limitation slows down the rate at which you can test, deploy and ship new versions of your code, which is never good. It’s much easier to add new features in a modern, flexible cloud environment.

Fundamentally, we found it challenging to use modern cloud services in an uncertain legacy environment.

Flipper-ing the script

At first glance, there was no practical way to integrate all the services we wanted with the code that would run on the pinball machine. But what if there was another way? What if we turned the pinball machine itself into a service, and gave it a single minimal integration? Then we could have it send a message every time something happened in the game and sort out the results in a modern cloud environment.

We decided that Pub/Sub would be an excellent way to achieve this goal. It provided a way to get information to (and from!) the cloud with a single interface, with minimal complexity. It was just a basic HTTP POST of whatever message format we wanted.

To achieve this, we designed a custom Pub/Sub messaging system. We wrote our own lightweight Pub/Sub library for the pinball machine to handle authentication and message sending over the REST API, making it incredibly easy to post events whenever a player launched a ball, hit a target, or even pressed a flipper button. Check out a simplified version of that code on GitHub!

On the cloud side, our team used multiple Cloud Run subscribers to process these events in real time. We also used Firestore to store data and drive visualizations.

Jackpot! Cloud advantages

Pushing the complexity of integration into the cloud brought numerous advantages:

Single interface: Writing our own Pub/Sub client was no small task (authentication alone could be its own blog post!). But once it was done, it was done! Once it was working, we were able to focus on processing all the events in the cloud using whatever modern client libraries and tools we wanted.

Real-time updates: At Google Cloud Next, we helped users write their own Cloud Run services to receive pinball events, process them, and send messages back to the machine. Building and deploying these services took less than a minute, which meant you could conceivably change the game while a friend was playing it!

Rich data insights: We ended up with a fine-grained log of everything that happened in a game. This proved very helpful in troubleshooting issues during development and fine-tuning scoring based on playtesting.

Plunging forward

We’re already planning the next iteration of Backlogged Pinball with features we hadn’t originally considered. For example, we’re adding AI-powered game analysis and advice based on the player’s style. Thanks to this flexible cloud-based architecture, almost all the work will be in a modern cloud environment rather than fighting with dependencies on a legacy system. And the lessons we learned from this project are broadly applicable to any constrained environment. Whether it's an embedded system, an IoT device, or an old server running legacy software, by leveraging Pub/Sub messaging and adopting a cloud-first mindset, you can break free from the limitations of your environment and unlock the full potential of the cloud.

We’ll be showing off the latest Backlogged Pinball at KubeCon North America in November 2024. If you’re there, stop by to check it out!

^{Special thanks to Mofi Rahman, Google Cloud Advocate, for his contributions to this project and this post.}