FinOps from the Field: Cloud cost forecasting

Cloud computing has become an essential part of many businesses and, as a Technical Account Manager, I have seen firsthand how cloud computing can help businesses save money and improve their agility. However, one of the challenges that businesses face when moving to the cloud is forecasting their cloud costs. If you are not careful, your cloud costs can quickly spiral out of control. That is where cloud cost forecasting comes in.

Cloud cost forecasting is essentially the process of predicting your future cloud usage. Forecasting is important in order to get funding, budget or support to start a new project in the cloud. Predicting your cloud costs can help you determine if you are on track with your FinOps strategy.

For many companies though, accurate cloud forecasting is one of the most difficult things to get right. In The State of Finops survey, advanced practitioners reported variances of about +/- 5% from their predictions - whilst less advanced reported variances as +/- 20%.

Perspective from the field

When I hear the word "forecasting," I immediately think of weather predictions and, specifically, seasonality. For example, in the spring, you know that the weather will be milder than in the winter, and it is very unlikely to experience a snowstorm or a heatwave. You can predict things that are more likely to happen in spring since it is usually the rainiest season of the year.

If we apply this concept to cloud workloads, we will also discover seasons and events that will have a significant impact on cloud usage. For example, organizations may experience large spikes in demand during sales periods or year-end holidays. This will have a direct impact on consumption, which will result in additional costs. As a consequence, you will not only need to know your previous data to predict usage, but also the key factors that may influence your future consumption.

Ask yourself:

1. Does your business have seasonal peaks?
2. Do you have any upcoming new projects, marketing events, or migrations planned that may result in additional users or consumption for your product?
3. How often do you collate this data for future input into forecasting cycles?

Historical usage

The other key element in accurate forecasting is historical data. By analyzing your past usage, you can get a good idea of how your workloads are likely to change in the future. Historical data is key to determining costs in the future. This will help you to plan accordingly and avoid any unexpected costs. For example, if you see that your usage has been increasing steadily over the past few months as you are migrating your applications, you can expect it to continue to increase in the future. Additionally, if you see that your usage is fluctuating significantly, you may need to implement a more robust cost-management strategy.

Ask yourself:

1. Is my historical usage in-line with my defined seasonal peaks (if applicable?)
2. Does my historical usage line up with expected business events?
3. Did I appropriately investigate any forecast variations from my actuals? What were the results of variance analysis? Can I apply any lessons learned in my forecasting model?
4. What’s my long-term trendline look like?

By understanding your historical data, you can make informed decisions about your future costs. TIP: In Google Cloud you can keep track of your consumption by exporting Cloud Billing data to BigQuery.

Tools to predict cloud costs

Now that you know your past data, and possible future events, you can start predicting your future cloud costs. There are a number of tools practitioners use to blend these two together, for example:

• Your native cloud cost management tool (In GCP, that’s the GCP Billing console)
• Cost Estimation APIs
• Spreadsheets
• Machine learning models using BQML or Vertex-AI
• Third-party forecasting tool

The best way to forecast depends on your specific needs and budget. If you have a small budget and are comfortable using spreadsheets, you can use that method. However, if you have a larger budget and need more accurate forecasts, you may want to consider using a cloud cost management tool or a third-party forecasting tool.

All things being equal, the out-of-the-box prediction feature in the Google Cloud Billing Console is a great place to start. The console provides an estimated cost for the current month based on your usage. This forecast is a combined total of actual cost to date and the projected cost trend. It is an approximation based on historical usage of all the projects that belong to that billing account. You can also adjust this forecast by filtering your report by project, service, sku, or folder in your organization.

Cloud forecasting is more manageable for a single project than for the entire organization. It is advised to forecast each project individually and then combine the forecasts to improve accuracy. The divide-and-conquer strategy is always a good approach to start small and end up with a very comprehensive forecast.

Experience from the field

I was recently involved in a forecasting exercise for a retail customer to calculate their spending for the upcoming year. The customer did not have anything specific to predict their yearly commitment with Google Cloud for this reason we, the Google Cloud account team, worked and prepared a rough estimate of their future cloud costs.

One of the good things about this customer is that they were not new to GCP so we had several advantages to predict their costs:

• We could get information from their historical data
• We knew their cloud journey and current maturity level
• We were also very familiar with their main workloads
• And they shared with us their general plans for the following year

These were just a few advantages. On the other hand, since we didn’t know the details of each project, we couldn’t predict each project separately and combine all predictions later. As an alternative, we divided the spending and aggregated it by type of service: Compute Engine, GKE, Analytics, Networking, etc. In order to complete the estimate we followed the next steps:

1. Group all spending by service category (analytics, compute, networking, etc.)
2. Calculate the average of the current and past two months' spending for each service category
3. Calculate the average of the same period from a year ago
4. Compare the two averages to identify trends and assign each category its growth percentage
5. Factor in all upcoming plans. Apply corrections for workloads that are stable, new, or intermittent
6. Share the estimate with the customer and adapt it based on their feedback

The accuracy of our predictions can only be determined by the passage of time, there are always unforeseen factors that can affect the outcome of events. Only time will confirm how accurate our predictions are.

Final piece of advice

When forecasting your cloud costs, it's important to be realistic. Don't just assume that your costs will stay the same. Factor in changes in your cloud usage, changes in your cost drivers, and changes in the cloud market or pricing. Something that I have learned while helping my customers understand their cloud costs is that some products are used consistently, while others are used sporadically. For example, a product might be used heavily for one month and then not used at all for three months.

Lastly, cloud cost forecasting is an ongoing process that requires regular review and recalculation. Making small changes throughout the year will yield better results than waiting until the end of the year to make changes.

Now that you know how to forecast your future cloud costs, you can start tracking and monitoring your actuals versus your forecast and use variance analysis.