DevOps Award winner Ford on how it grew DevOps throughout the organization

In this blog post, we’re highlighting Ford for the DevOps achievements that earned them the ‘Growing DevOps throughout your organization’ award in the 2022 DevOps Awards. To learn more about the winners and how they used DORA metrics and practices to grow their businesses, start here.

About

Ford Motor Company is an American multinational automobile manufacturer headquartered in Dearborn, Michigan, United States. It was founded by Henry Ford and incorporated on June 16, 1903. The company sells automobiles and commercial vehicles under the Ford brand and luxury cars under its Lincoln luxury brand. 

Challenges 

Ford Motor Company recognized the potential of cloud computing for cloud hosting early on, including CaaS, PaaS, and SaaS. While leveraging the cloud and hybrid environments improved our business operations, many platforms did not enforce DevOps practices.   Consequently, some teams continued with manual click-ops configuration in their environments and did not consistently adopt basic DevOps and GitOps practices.

We saw an opportunity to transform our operations by requiring all new deployments to implement DevOps practices with Infrastructure as Code (IaC) and CI/CD pipelines. 

Objectives 

To maximize the benefits of the cloud, our company had to realize the importance of adopting a "cloud first" mindset, where new applications or updated technologies would be developed or migrated to the cloud as the primary option whenever feasible. To achieve this, we recognized the need for a platform that would drive mature DevOps practices.

Ford set out to:

• Further enhance customer experience and gain better cost control
• Deliver scalability to teams without resource wastage
• Provide business-led app teams with as much direct control as possible

Solution

Our teams developed a scalable approach to managing project environments through pipelines. These pipelines not only organized the platform but also built and maintained thousands of Google Cloud projects. In collaboration with our service providers, we developed a unique solution that restricted console access to sandbox environments only, with system account restrictions that enforced CI/CD pipelines within our extensive community of app team developers.

The pipelines were divided into two components:

The first pipeline component controlled Google Cloud projects, helping to ensure that we implemented Google-recommended security practices that aligned with our internal requirements IT attestations and policies. The second pipeline component granted business teams control over their infrastructure, enabling them to dynamically provision and manage infrastructure according to their requirements.

Given that most native Google Cloud services support Terraform providers, our teams were able to test and support these providers and create test scripts that DevOps teams could utilize. 

To promote the adoption of DevOps best practices, we relied on our automated provisioning tool—the Ford Cloud Portal (FCP). The FCP enabled teams to rapidly provision hardware, facilitated the creation of app-based repositories, provided links to policy attestations and training, and even offered specialized bundles supporting specific personas, thereby accelerating deployments significantly.

The FCP established managed pipelines, where each app team had its own repository to manage infrastructure, fostering true GitOps/DevOps experiences. Through an intuitive interface, the FCP guided app teams through all the steps of the Google Cloud provisioning process, allowing even novice teams to navigate and manage both Google Cloud and infrastructure-as-code environments while reinforcing DevOps practices. Additionally, the FCP provided a list of fully-vetted cloud services on a Service Selection Screen.  This approach helped to enable teams to support and continue to develop best practices effectively.

The FCP also offered architecture-specific bundles that assisted teams in configuring specific elements based on their use cases. This flexibility allowed teams to choose whether to configure just the project, the project and infrastructure, or even entire development environments. Regardless of the elements configured, they would do so with value-added and validated development and data scientist tooling.

Results 

By mandating the use of pipelines for infrastructure provisioning and management, supported by Google Cloud Terraform services providers, our app teams further embraced the DevOps culture. We no longer had to manage and configure infrastructure through tickets or interfaces. Instead, we integrated infrastructure provisioning into our pipelines, enabling continuous deployment and testing. This helped to enhance our change control and security posture, facilitating faster roll-out and rollback of changes. As a result, our company witnessed substantial adoption of DevOps practices across its developer community, with hundreds of Ford applications now benefiting from DevOps-managed Google Cloud footprints.

The results have been evident. 

In 2022 alone, Ford exceeded our application migration targets to Google Cloud with mandated DevOps and GitOps practices.

Moreover, we observed notable improvements in the cloud related to dynamic scalability and availability, including:

• Deployment frequency: Changes to projects and infrastructure became faster, taking minutes instead of weeks, thanks to the DevOps practices

• Lead time for changes: Provision velocity in the project pipeline improved, with changes that previously took over 80 hours on a single channel now being provisioned in less than half an hour.

• Productivity: Since the shift, we have reduced manual bottlenecks associated with obtaining Google Cloud projects.

Stay tuned for the rest of the series highlighting the DevOps Award Winners and read the 2022 State of DevOps report to dive deeper into the DORA research.