Content delivery with Fastly and Cloud Storage
Contributed by Google employees.
A content delivery network, or CDN, offers an efficient, cost-effective way of reducing both network I/O costs and content delivery latency for regularly accessed website assets. A CDN can be understood as group of geographically distributed caches, with each cache located in one of several global points of presence (POPs). The CDN service pulls assets from an origin server at regular intervals and distributes them to these caches. When a client requests an asset, the asset is served from the nearest cache.
Though CDNs often greatly improve the speed of content delivery, they are usually at the mercy of the public Internet; that is, when they pull content from an origin server, they have to compete with other network traffic for bandwidth. To mitigate this issue, the CDN provider Fastly has partnered with Google to provide a direct connection to Google Cloud's network edge at several North American locations. This edge connection allows Fastly users on Google Cloud to bypass the public internet during origin pulls, resulting in significant increases in pull speed.
This tutorial walks you through the process of setting up Fastly to pull assets from Cloud Storage. In addition, it describes how to configure Fastly's origin shield feature to take advantage of Fastly's direct connection to Google's network edge.
This tutorial uses Cloud Storage, a billable component of Google Cloud, and Fastly, which charges bandwidth and request fees.
You can use the pricing calculator to generate a cost estimate based on your projected usage.
- Configure Fastly to source assets from Cloud Storage
- Configure Fastly to handle cache misses using origin shield
Before you begin
This tutorial assumes that you have the following:
- A Google Cloud project with the Cloud Storage and Cloud Storage JSON APIs enabled. These APIs are enabled by default.
- A Fastly account.
- A domain name you plan to use for CDN purposes.
Configure Fastly to pull content from Cloud Storage
Create a Cloud Storage bucket
Before you can configure Fastly to pull assets from Cloud Storage, you need to create a Cloud Storage bucket and populate it with assets.
To create a Cloud Storage bucket:
- In the Cloud Storage Browser in the Cloud Console, click Create Bucket.
In the Create Bucket dialog, fill out the form as follows:
[YOUR_BUCKET_NAME](must be unique)
- Storage: Multi-Regional
Location: United States
Note: Multi-Regional buckets allow the data to be broadly replicated across the location (for example across the continental United States), and always be close to Fastly's interconnects with Google's network.
Click Create to create your bucket. You will be taken to your bucket's browser page.
Next, you need to add assets that Fastly can pull from your bucket:
- On your local machine, identify an image that you'd like to upload to your bucket.
- Upload the image to your bucket by dragging it from your local file browser and dropping it into your bucket page in the Cloud Console. After the image has finished uploading, it will appear in the bucket's list of stored objects.
- On your bucket page, go to your image's list item and click Shared Publicly to make the image externally accessible.
Create a new Fastly service
Now that you've set up a Cloud Storage bucket, set up a Fastly service and configure it to use the bucket as a point of origin:
- Go to the Fastly configuration page.
Click New Service and fill out the form as follows:
- Name: Cloud Storage Origin Test
- Server address: storage.googleapis.com:80
Note: Fastly recommends against using an apex domain.
Important: To complete the process of creating your first service, you
must set your domain's CNAME record to
process for doing so will vary by domain hosting service. To find
host-specific instructions for your hosting service, see
Add a CNAME record.
Add your Cloud Storage bucket as a backend
After creating your Fastly service, add your Cloud Storage bucket as a backend for the service:
- Click the Configure icon in the top navigation bar of the Fastly configuration page.
- Click the blue Configure button.
- Click Settings in the sidebar.
Set the default host name as follows, replacing
[YOUR_BUCKET_NAME]with the name of your Cloud Storage bucket:
Optionally, define the length of time your assets should be cached by setting a time to live (TTL) value. By default, each asset is cached for 3600 seconds (1 hour) before Fastly performs another origin pull.
Save your settings.
Click the orange Activate button to activate your new backend.
Congratulations! You now have a functioning Fastly CDN service that will periodically pull assets from your storage bucket and store them in edge caches around the world.
Note: It might take 20-30 seconds for the initial origin pull to propagate across Fastly's network.
Test your service
You can test your Fastly service by loading the image file that you added
to your bucket earlier in this tutorial. In your web browser, navigate to
the following URL, replacing
[YOUR_DOMAIN] with your domain and
[IMAGE_NAME] with your image file:
For example, if your domain is
www.example.com and the file is named
test.png, you should navigate to the following URL:
Configure an origin shield and connect it to Cloud Platform
Fastly includes a feature called origin shield that places a "shield" cache between its edge caches and your origin server. This shield cache handles all cache misses across Fastly's network; that is, if one of Fastly's caches does not have the asset that has been requested, that cache will try to pull the asset from the shield cache instead of the origin server. The cache will only pull the requested asset from the origin server if the shield cache does not contain the asset.
By using an origin shield, you can drastically reduce the number of calls back to your Cloud Storage bucket. And by using an origin shield that is connected to Google's network edge, you can increase the speed of the shield's origin pulls.
It is normally recommended to use a multi-regional Cloud Storage bucket location, but when using an origin, it is instead recommended to use a regional or dual-regional location. This ensures the data is always close to the origin, improving performance, and reducing latency on cache fills.
To configure an origin shield and connect it to Google Cloud:
- In the Fastly application, click Hosts in the sidebar.
- Under Backends, click the gear next to your storage.googleapis.com backend and click Edit.
- In the Edit Backend dialog, set Shielding to a location close to the Cloud Storage bucket.
- Click Update to commit your settings.
- Click Activate to deploy the modified service.
After you've finished the Fastly tutorial, you can clean up the resources you created on Google Cloud so you won't be billed for them in the future. The following sections describe how to delete or turn off these resources.
Deleting the project
The easiest way to eliminate billing is to delete the project you created for the tutorial. If you don't want to delete the project, delete the individual instances, as described in the next section.
Warning: Deleting a project has the following consequences:
- If you used an existing project, you'll also delete any other work you've done in the project.
- You can't reuse the project ID of a deleted project. If you created a custom project ID that you plan to use in the future, you should delete the resources inside the project instead. This ensures that URLs that use the project ID, such as an appspot.com URL, remain available.
If you are exploring multiple tutorials and quickstarts, reusing projects instead of deleting them prevents you from exceeding project quota limits.
To delete the project:
- In the Cloud Console, go to the Projects page.
- In the project list, select the project you want to delete and click Delete project. After selecting the checkbox next to the project name, click Delete project.
- In the dialog, type the project ID, and then click Shut down to delete the project.
Deleting your Fastly service
To delete your Fastly service:
- Navigate to the Fastly application in your web browser.
- Click the Configure icon in the top navigation menu.
- Select your service from the Service dropdown.
- Click the Deactivate button.
- In the modal dialog, click the Deactivate button.
- Click the Delete button.
- In the modal dialog, click the Confirm button to remove the service.
Explore Fastly's feature set and configuration options by reviewing the Fastly documentation. Fastly also provides their own tutorial on using Cloud Storage, as well information on serving private buckets.