Light the way ahead: Platform Engineering, Golden Paths, and the power of self-service

Imagine that you're a Java developer who has just joined a new company, and you're tasked with creating a small Java service. In a DevOps model, the shared responsibility between Development and Operations teams might mean that you'll not only be expected to write Java code, but also operations code like build pipelines and monitoring instrumentation. Plus, your new company uses a different cloud platform than the one you learned in your old job.

Suddenly, you're drowning in YAML files, and building that small Java service becomes a big challenge. You're faced with a slew of decisions: how should I structure my code? Which tool do I use for continuous delivery?

The reality of the DevOps model is that it often introduces cognitive overhead that is unsustainable for developers, raising the barriers to onboarding, slowing productivity, and causing burnout.

Platform Engineering is the practice of building useful abstractions and self-service infrastructure within an organization, to unify scattered tools and accelerate developer productivity. Platform Engineering aims to mitigate the cognitive overload caused by a shared-responsibility model that's gone too far, by smoothing the day-to-day developer experience.

So what kind of abstractions and self-service infrastructure would best serve developers? This brings us to the idea of a Golden Path.

What is a Golden Path?

The Cloud Native Computing Foundation defines a Golden Path as a "templated composition of well-integrated code and capabilities for rapid project development."  Simply put, it is a self-service template for common tasks.

Imagine that instead of fending for yourself amidst a sea of tools and infrastructure, you are provided with a Java Spring Boot Golden Path Template on your first day at work. This is a source code repository containing:

• A getting-started tutorial 

• Skeleton source code  

• Dependency management 

• CI/CD pipeline template 

• Cloud infrastructure-as-code template

• Kubernetes YAML files 

• Policy guardrails

• Logging and monitoring instrumentation 

• Reference documentation  

This Golden Path not only helps you onboard to the Java recommended practices within your new organization, but also provides a forkable repo template to get building fast.

Now imagine that your new company provides multiple Golden Path templates, each serving common tasks, from "React Web Frontend" to "Machine Learning with Python." No developer is left in the dark to fend for themselves; Golden Paths light the way.

Who is a Golden Path for?

Golden Paths can benefit multiple personas within your organization. They help application developers onboard and get building faster by providing a clear pathway to supported tooling— less bespoke infrastructure code to maintain later. If multiple application teams build on the same golden path, they have a common foundation that aids information-sharing, inspires the creation of shared tools, and improves agility and mobility across the org.

That same consistency makes it easier for Site Reliability Engineers (SREs) to monitor a portfolio of production services. For instance, if most services at the company use the same metrics exporters and labels, this can accelerate debugging during an outage.

Golden Paths also provide Security Administrators with a clear means to implement policy guardrails. For instance, if the Security team has new requirements on firewall rules, they can work with the Platform team to update the Golden Path templates.

Lastly, Golden Paths can help the entire organization by optimizing costs. For instance, a Golden Path for a Kubernetes-based service can create a Namespace in a GKE Autopilot cluster, instead of creating a large, dedicated GKE cluster where most capacity goes unused. This allows infrastructure-level optimizations to be defined and abstracted away.

This is an incomplete list of people who can benefit from a set of Golden Paths. From Developers and Machine Learning Engineers, to Testers and FinOps, common abstractions and easy self-service can benefit everyone.

Golden Path Principles

No two Golden Paths will look the same— that's okay! However, Golden Paths often share a common set of principles:

1. Provide a single, clear, opinionated method to accomplish a specific task.

2. Reduce the cognitive load on the target audience using abstraction and getting-started documentation.

3. Integrate with existing internal developer platforms. A Golden Path should work on top of the abstracted infrastructure managed by the platform team: for instance, if a developer creates a new service with a Golden Path, that should spawn a new software catalog entry.  Another example: if a Golden Path repo contains a Kubernetes Deployment template, that Deployment should be applied to a platform team-managed GKE cluster and namespace.

4. Light the complete path from development to production. This might include instructions for local development, a templated CI/CD pipeline, and infrastructure as code templates for staging and production. 

5. Be self-service. Any developer in the organization should be able to discover and use the Golden Paths — no ticketing system required.

6. Provide a transparent abstraction without hiding the underlying infrastructure. Under a DevOps shared responsibility model, developers will eventually need to understand the full stack of their own services to maintain, optimize, and debug them.

7. Address your organization's specific needs. Is your organization primarily dealing with "brownfield" migrations, with few new "greenfield" services? Consider building Golden Paths for supported migration paths.

8. Allow for flexibility to support the requirements of multiple audiences. For instance, a Golden Path could support a SQL database like Spanner by default, but make it easy to swap in a NoSQL database like Firestore.

9. Be optional. A developer should not be forced to adopt a Golden Path if it does not fulfill their needs.

Golden Path Practices

We've covered the who, what, and why of Golden Paths. Now let's discuss when and how. 

When to build Golden Paths?

Building and maintaining a Golden Path has a cost. You might work in a small organization with few services; in this case, the cost of building a Golden Path may exceed the benefits. We recommend creating your first Golden Paths if you have several services that look similar — think similar frameworks running on similar infrastructure.

We also recommend taking a look at the day-to-day friction of a typical developer in the org. Industry-specific regulations or complex, heterogeneous infrastructure environments are both cases where abstractions provided by a Golden Path could benefit developers and your business— even if the Golden Path is serving only a few teams. 

How to choose a Path?

Let's say your organization has fifty dev teams working across four programming languages. How can you choose what Golden Paths to build first?

To start, work with the teams to identify common patterns. Perhaps your front-end developers are using both Flutter and React. Maybe you should build two frontend web Golden Path templates: one for Flutter, and one for React. Only have time to build one? Perhaps choose the simpler architecture, or identify a team with the excitement and bandwidth to co-develop and test-drive the first Golden Path.

Who builds and maintains?

Golden Path templates are technical artifacts that need to be built, documented, and maintained. We recommend that each template is co-developed by at least two people/teams, spanning both the platform team and at least one stakeholder team. 

Why co-develop? First, to ensure that the platform team can build consistency across multiple Golden Paths— for instance, to ensure all Golden Paths in the org use the same infrastructure as code stack. Second, to ensure that the stakeholder teams have a say in what goes into the template. Involving application development teams from the beginning can increase the likelihood that the Golden Path template is adopted.

Long-term maintenance and iteration of Golden Path templates will vary depending on your organization's velocity, total number of Golden Paths supported, and the overall adoption of the Golden Path. One route could be for the platform team to own the maintenance and version control of all Golden Path templates, but accept contributions from other teams.

Overall, adopting a shared responsibility model for Golden Paths highlights the best of DevOps culture, where communication, shared learning, and high cooperation are rewarded and benefit everyone involved.

Measuring success and iterating

Let's say you've identified, designed, built, and released your first Golden Path templates. How can you determine whether they're successful? 

First, it's important to identify what success looks like in your organization. Is it improved developer satisfaction? More releases per quarter? Try looking at your internal platform as a product: what does the success of that product look like? 

Some metrics might be easy to measure: for instance, the number of new services that have adopted the golden paths. Set KPIs and quantitative goals; build dashboards to track Golden Path adoption within your organization.

Other metrics are harder to measure. How do you measure developer autonomy? Or reduced coordination costs between teams? Here, qualitative measurements like internal developer surveys can help. Establish support channels to gather feedback from the users of your Golden Paths, and don't be afraid to iterate if the first implementation misses the mark.

As your quantitative and qualitative measurements come together, use the feedback to build a roadmap and release schedule to share with dev teams. You may even want to market and promote your platform with newsletters or lunch and learn sessions to raise awareness of your latest Golden Path features.

Light the way!

This post gave a high-level introduction to Golden Paths, exploring how self-service abstractions can boost engineering velocity and collaboration across teams. Hopefully this post inspires you to think about what kind of Golden Paths you'd like to use or build within your organization.

Not sure where to start? Check out the Google Cloud Terraform Blueprints page for example bundles of Google Cloud infrastructure as code. Still wrapping your head around Platform Engineering as a whole? Check out this blog post from Google's Alex McWilliam on Platform Engineering pitfalls, and the CNCF Whitepaper on Platform Engineering.