Stay organized with collections
Save and categorize content based on your preferences.
This tutorial shows you how to train the ResNet-50 model
on a Cloud TPU device with PyTorch. You can apply the same pattern to
other TPU-optimised image classification models that use PyTorch and the
ImageNet dataset.
The model in this tutorial is based on Deep Residual Learning for Image
Recognition, which first introduces
the residual network (ResNet) architecture. The tutorial uses the 50-layer
variant, ResNet-50, and demonstrates training the model using
PyTorch/XLA.
Objectives
Prepare the dataset.
Run the training job.
Verify the output results.
Costs
In this document, you use the following billable components of Google Cloud:
Compute Engine
Cloud TPU
To generate a cost estimate based on your projected usage,
use the pricing calculator.
New Google Cloud users might be eligible for a free trial.
Before you begin
Before starting this tutorial, check that your Google Cloud project is correctly
set up.
Sign in to your Google Cloud account. If you're new to
Google Cloud,
create an account to evaluate how our products perform in
real-world scenarios. New customers also get $300 in free credits to
run, test, and deploy workloads.
In the Google Cloud console, on the project selector page,
select or create a Google Cloud project.
This walkthrough uses billable components of Google Cloud. Check the
Cloud TPU pricing page to
estimate your costs. Be sure to clean up resources you created when you've
finished with them to avoid unnecessary charges.
To avoid incurring charges to your Google Cloud account for the resources used in this
tutorial, either delete the project that contains the resources, or keep the project and
delete the individual resources.
Disconnect from the TPU VM:
(vm)$exit
Your prompt should now be username@projectname, showing you
are in the Cloud Shell.
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Hard to understand","hardToUnderstand","thumb-down"],["Incorrect information or sample code","incorrectInformationOrSampleCode","thumb-down"],["Missing the information/samples I need","missingTheInformationSamplesINeed","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2025-08-25 UTC."],[],[],null,["# Training Resnet50 on Cloud TPU with PyTorch\n\n*** ** * ** ***\n\nThis tutorial shows you how to train the ResNet-50 model\non a Cloud TPU device with PyTorch. You can apply the same pattern to\nother TPU-optimised image classification models that use PyTorch and the\nImageNet dataset.\n\nThe model in this tutorial is based on [Deep Residual Learning for Image\nRecognition](https://arxiv.org/pdf/1512.03385.pdf), which first introduces\nthe residual network (ResNet) architecture. The tutorial uses the 50-layer\nvariant, ResNet-50, and demonstrates training the model using\n[PyTorch/XLA](https://github.com/pytorch/xla).\n| **Warning:** This tutorial uses a third-party dataset. Google provides no representation, warranty, or other guarantees about the validity, or any other aspects of this dataset.\n\n\nObjectives\n----------\n\n- Prepare the dataset.\n- Run the training job.\n- Verify the output results.\n\n\nCosts\n-----\n\n\nIn this document, you use the following billable components of Google Cloud:\n\n\n- Compute Engine\n- Cloud TPU\n\n\nTo generate a cost estimate based on your projected usage,\nuse the [pricing calculator](/products/calculator). \nNew Google Cloud users might be eligible for a [free trial](/free). \n\n\u003cbr /\u003e\n\n\nBefore you begin\n----------------\n\nBefore starting this tutorial, check that your Google Cloud project is correctly\nset up.\n\n- Sign in to your Google Cloud account. If you're new to Google Cloud, [create an account](https://console.cloud.google.com/freetrial) to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.\n- In the Google Cloud console, on the project selector page,\n select or create a Google Cloud project.\n\n | **Note**: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.\n\n [Go to project selector](https://console.cloud.google.com/projectselector2/home/dashboard)\n-\n [Verify that billing is enabled for your Google Cloud project](/billing/docs/how-to/verify-billing-enabled#confirm_billing_is_enabled_on_a_project).\n\n- In the Google Cloud console, on the project selector page,\n select or create a Google Cloud project.\n\n | **Note**: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.\n\n [Go to project selector](https://console.cloud.google.com/projectselector2/home/dashboard)\n-\n [Verify that billing is enabled for your Google Cloud project](/billing/docs/how-to/verify-billing-enabled#confirm_billing_is_enabled_on_a_project).\n\n1. This walkthrough uses billable components of Google Cloud. Check the [Cloud TPU pricing page](/tpu/docs/pricing) to estimate your costs. Be sure to clean up resources you created when you've finished with them to avoid unnecessary charges.\n\n\nCreate a TPU VM\n---------------\n\n1. Open a Cloud Shell window.\n\n [Open Cloud Shell](https://console.cloud.google.com/?cloudshell=true)\n2. Create a TPU VM\n\n ```bash\n gcloud compute tpus tpu-vm create your-tpu-name \\\n --accelerator-type=v3-8 \\\n --version=tpu-ubuntu2204-base \\\n --zone=us-central1-a \\\n --project=your-project\n ```\n | **Note:** The first time you run a command in a new Cloud Shell VM, an `Authorize Cloud Shell` page is displayed. Click `Authorize` at the bottom of the page to allow `gcloud` to make Google Cloud API calls with your credentials.\n3. Connect to your TPU VM using SSH:\n\n ```bash\n gcloud compute tpus tpu-vm ssh your-tpu-name --zone=us-central1-a\n ```\n4. Install PyTorch/XLA on your TPU VM:\n\n ```bash\n (vm)$ pip install torch torch_xla[tpu] torchvision -f https://storage.googleapis.com/libtpu-releases/index.html -f https://storage.googleapis.com/libtpu-wheels/index.html\n ```\n5. Clone the [PyTorch/XLA GitHub repo](https://github.com/pytorch/xla)\n\n ```bash\n (vm)$ git clone --depth=1 https://github.com/pytorch/xla.git\n ```\n6. Run the training script with fake data\n\n ```bash\n (vm) $ PJRT_DEVICE=TPU python3 xla/test/test_train_mp_imagenet.py --fake_data --batch_size=256 --num_epochs=1\n ```\n\nClean up\n--------\n\n\nTo avoid incurring charges to your Google Cloud account for the resources used in this\ntutorial, either delete the project that contains the resources, or keep the project and\ndelete the individual resources.\n\n1. Disconnect from the TPU VM:\n\n ```bash\n (vm) $ exit\n ```\n\n Your prompt should now be `username@projectname`, showing you\n are in the Cloud Shell.\n2. Delete your TPU VM.\n\n ```bash\n $ gcloud compute tpus tpu-vm delete your-tpu-name \\\n --zone=us-central1-a\n ```\n\n\nWhat's next\n-----------\n\n- [Training diffusion models with Pytorch](/tpu/docs/tutorials/diffusion-pytorch)\n- [Troubleshooting Pytorch on TPUs](/tpu/docs/troubleshooting/trouble-pytorch)\n- [Pytorch/XLA documentation](https://pytorch.org/xla/)"]]
