从文件系统上传对象

本页面介绍如何将对象从本地文件系统上传到 Cloud Storage 存储桶。上传的对象包含要存储的数据以及所有关联元数据。如需查看概念性概览(包括如何根据文件大小选择最佳上传方法),请参阅上传和下载

如需了解如何从内存上传,请参阅从内存上传对象

所需的角色

如需获得将对象上传到存储桶所需的权限,请让您的管理员为您授予存储桶的 Storage Object User (roles/storage.objectUser) IAM 角色。此预定义角色可提供将对象上传到存储桶所需的权限。如需查看所需的确切权限,请展开所需权限部分:

所需权限

  • storage.objects.create
  • storage.objects.delete
    • 只有覆盖现有对象的上传操作需要此权限。
  • storage.objects.get
    • 仅当您计划使用 Google Cloud CLI 执行本页面上的任务时,才需要此权限。
  • storage.objects.list
    • 仅当您计划使用 Google Cloud CLI 执行本页面上的任务时,才需要此权限。如果您要使用 Google Cloud 控制台验证您已上传的对象,也需要此权限。

如果您打算使用 Google Cloud 控制台执行本页面上的任务,则还需要 storage.buckets.list 权限,Storage Object User (roles/storage.objectUser) 角色不提供此权限。如需获得此权限,请让您的管理员为您授予项目的 Storage Admin (roles/storage.admin) 角色。

您也可以使用其他预定义角色自定义角色来获得这些权限。

如需了解如何授予存储桶的角色,请参阅将 IAM 与存储桶搭配使用

将对象上传到存储桶

如需将对象上传到存储桶,请完成以下步骤:

控制台

  1. 在 Google Cloud 控制台中,进入 Cloud Storage 存储桶页面。

    进入“存储桶”

  2. 在存储桶列表中,点击要将对象上传到的存储桶的名称。

  3. 在存储桶的对象标签中,执行以下任一操作:

    • 将所需文件从桌面或文件管理器拖放到 Google Cloud 控制台的主窗格中。

    • 点击上传文件按钮,在出现的对话框中选择要上传的文件,然后点击打开

如需了解如何在 Google Cloud 控制台中获取失败的 Cloud Storage 操作的详细错误信息,请参阅问题排查

命令行

使用 gcloud storage cp 命令:

gcloud storage cp OBJECT_LOCATION gs://DESTINATION_BUCKET_NAME

其中:

  • OBJECT_LOCATION 是对象的本地路径。例如 Desktop/dog.png

  • DESTINATION_BUCKET_NAME 是对象要上传到的存储桶的名称。例如 my-bucket

如果成功,则响应类似如下示例:

Completed files 1/1 | 164.3kiB/164.3kiB

您可以使用命令标志在对象上传过程中设置固定键和自定义对象元数据

客户端库

C++

如需了解详情,请参阅 Cloud Storage C++ API 参考文档

如需向 Cloud Storage 进行身份验证,请设置应用默认凭据。 如需了解详情,请参阅为本地开发环境设置身份验证

namespace gcs = ::google::cloud::storage;
using ::google::cloud::StatusOr;
[](gcs::Client client, std::string const& file_name,
   std::string const& bucket_name, std::string const& object_name) {
  // Note that the client library automatically computes a hash on the
  // client-side to verify data integrity during transmission.
  StatusOr<gcs::ObjectMetadata> metadata = client.UploadFile(
      file_name, bucket_name, object_name, gcs::IfGenerationMatch(0));
  if (!metadata) throw std::move(metadata).status();

  std::cout << "Uploaded " << file_name << " to object " << metadata->name()
            << " in bucket " << metadata->bucket()
            << "\nFull metadata: " << *metadata << "\n";
}

C#

如需了解详情,请参阅 Cloud Storage C# API 参考文档

如需向 Cloud Storage 进行身份验证,请设置应用默认凭据。 如需了解详情,请参阅为本地开发环境设置身份验证


using Google.Cloud.Storage.V1;
using System;
using System.IO;

public class UploadFileSample
{
    public void UploadFile(
        string bucketName = "your-unique-bucket-name",
        string localPath = "my-local-path/my-file-name",
        string objectName = "my-file-name")
    {
        var storage = StorageClient.Create();
        using var fileStream = File.OpenRead(localPath);
        storage.UploadObject(bucketName, objectName, null, fileStream);
        Console.WriteLine($"Uploaded {objectName}.");
    }
}

Go

如需了解详情,请参阅 Cloud Storage Go API 参考文档

如需向 Cloud Storage 进行身份验证,请设置应用默认凭据。 如需了解详情,请参阅为本地开发环境设置身份验证

import (
	"context"
	"fmt"
	"io"
	"os"
	"time"

	"cloud.google.com/go/storage"
)

// uploadFile uploads an object.
func uploadFile(w io.Writer, bucket, object string) error {
	// bucket := "bucket-name"
	// object := "object-name"
	ctx := context.Background()
	client, err := storage.NewClient(ctx)
	if err != nil {
		return fmt.Errorf("storage.NewClient: %w", err)
	}
	defer client.Close()

	// Open local file.
	f, err := os.Open("notes.txt")
	if err != nil {
		return fmt.Errorf("os.Open: %w", err)
	}
	defer f.Close()

	ctx, cancel := context.WithTimeout(ctx, time.Second*50)
	defer cancel()

	o := client.Bucket(bucket).Object(object)

	// Optional: set a generation-match precondition to avoid potential race
	// conditions and data corruptions. The request to upload is aborted if the
	// object's generation number does not match your precondition.
	// For an object that does not yet exist, set the DoesNotExist precondition.
	o = o.If(storage.Conditions{DoesNotExist: true})
	// If the live object already exists in your bucket, set instead a
	// generation-match precondition using the live object's generation number.
	// attrs, err := o.Attrs(ctx)
	// if err != nil {
	// 	return fmt.Errorf("object.Attrs: %w", err)
	// }
	// o = o.If(storage.Conditions{GenerationMatch: attrs.Generation})

	// Upload an object with storage.Writer.
	wc := o.NewWriter(ctx)
	if _, err = io.Copy(wc, f); err != nil {
		return fmt.Errorf("io.Copy: %w", err)
	}
	if err := wc.Close(); err != nil {
		return fmt.Errorf("Writer.Close: %w", err)
	}
	fmt.Fprintf(w, "Blob %v uploaded.\n", object)
	return nil
}

Java

如需了解详情,请参阅 Cloud Storage Java API 参考文档

如需向 Cloud Storage 进行身份验证,请设置应用默认凭据。 如需了解详情,请参阅为本地开发环境设置身份验证


import com.google.cloud.storage.BlobId;
import com.google.cloud.storage.BlobInfo;
import com.google.cloud.storage.Storage;
import com.google.cloud.storage.StorageOptions;
import java.io.IOException;
import java.nio.file.Paths;

public class UploadObject {
  public static void uploadObject(
      String projectId, String bucketName, String objectName, String filePath) throws IOException {
    // The ID of your GCP project
    // String projectId = "your-project-id";

    // The ID of your GCS bucket
    // String bucketName = "your-unique-bucket-name";

    // The ID of your GCS object
    // String objectName = "your-object-name";

    // The path to your file to upload
    // String filePath = "path/to/your/file"

    Storage storage = StorageOptions.newBuilder().setProjectId(projectId).build().getService();
    BlobId blobId = BlobId.of(bucketName, objectName);
    BlobInfo blobInfo = BlobInfo.newBuilder(blobId).build();

    // Optional: set a generation-match precondition to avoid potential race
    // conditions and data corruptions. The request returns a 412 error if the
    // preconditions are not met.
    Storage.BlobWriteOption precondition;
    if (storage.get(bucketName, objectName) == null) {
      // For a target object that does not yet exist, set the DoesNotExist precondition.
      // This will cause the request to fail if the object is created before the request runs.
      precondition = Storage.BlobWriteOption.doesNotExist();
    } else {
      // If the destination already exists in your bucket, instead set a generation-match
      // precondition. This will cause the request to fail if the existing object's generation
      // changes before the request runs.
      precondition =
          Storage.BlobWriteOption.generationMatch(
              storage.get(bucketName, objectName).getGeneration());
    }
    storage.createFrom(blobInfo, Paths.get(filePath), precondition);

    System.out.println(
        "File " + filePath + " uploaded to bucket " + bucketName + " as " + objectName);
  }
}

Node.js

如需了解详情,请参阅 Cloud Storage Node.js API 参考文档

如需向 Cloud Storage 进行身份验证,请设置应用默认凭据。 如需了解详情,请参阅为本地开发环境设置身份验证

以下示例会上传单个对象:

/**
 * TODO(developer): Uncomment the following lines before running the sample.
 */
// The ID of your GCS bucket
// const bucketName = 'your-unique-bucket-name';

// The path to your file to upload
// const filePath = 'path/to/your/file';

// The new ID for your GCS file
// const destFileName = 'your-new-file-name';

// Imports the Google Cloud client library
const {Storage} = require('@google-cloud/storage');

// Creates a client
const storage = new Storage();

async function uploadFile() {
  const options = {
    destination: destFileName,
    // Optional:
    // Set a generation-match precondition to avoid potential race conditions
    // and data corruptions. The request to upload is aborted if the object's
    // generation number does not match your precondition. For a destination
    // object that does not yet exist, set the ifGenerationMatch precondition to 0
    // If the destination object already exists in your bucket, set instead a
    // generation-match precondition using its generation number.
    preconditionOpts: {ifGenerationMatch: generationMatchPrecondition},
  };

  await storage.bucket(bucketName).upload(filePath, options);
  console.log(`${filePath} uploaded to ${bucketName}`);
}

uploadFile().catch(console.error);

以下示例会同时上传多个对象:

/**
 * TODO(developer): Uncomment the following lines before running the sample.
 */
// The ID of your GCS bucket
// const bucketName = 'your-unique-bucket-name';

// The ID of the first GCS file to download
// const firstFilePath = 'your-first-file-name';

// The ID of the second GCS file to download
// const secondFilePath = 'your-second-file-name';

// Imports the Google Cloud client library
const {Storage, TransferManager} = require('@google-cloud/storage');

// Creates a client
const storage = new Storage();

// Creates a transfer manager client
const transferManager = new TransferManager(storage.bucket(bucketName));

async function uploadManyFilesWithTransferManager() {
  // Uploads the files
  await transferManager.uploadManyFiles([firstFilePath, secondFilePath]);

  for (const filePath of [firstFilePath, secondFilePath]) {
    console.log(`${filePath} uploaded to ${bucketName}.`);
  }
}

uploadManyFilesWithTransferManager().catch(console.error);

以下示例并发上传具有公共前缀的所有对象:

/**
 * TODO(developer): Uncomment the following lines before running the sample.
 */
// The ID of your GCS bucket
// const bucketName = 'your-unique-bucket-name';

// The local directory to upload
// const directoryName = 'your-directory';

// Imports the Google Cloud client library
const {Storage, TransferManager} = require('@google-cloud/storage');

// Creates a client
const storage = new Storage();

// Creates a transfer manager client
const transferManager = new TransferManager(storage.bucket(bucketName));

async function uploadDirectoryWithTransferManager() {
  // Uploads the directory
  await transferManager.uploadManyFiles(directoryName);

  console.log(`${directoryName} uploaded to ${bucketName}.`);
}

uploadDirectoryWithTransferManager().catch(console.error);

PHP

如需了解详情,请参阅 Cloud Storage PHP API 参考文档

如需向 Cloud Storage 进行身份验证,请设置应用默认凭据。 如需了解详情,请参阅为本地开发环境设置身份验证

use Google\Cloud\Storage\StorageClient;

/**
 * Upload a file.
 *
 * @param string $bucketName The name of your Cloud Storage bucket.
 *        (e.g. 'my-bucket')
 * @param string $objectName The name of your Cloud Storage object.
 *        (e.g. 'my-object')
 * @param string $source The path to the file to upload.
 *        (e.g. '/path/to/your/file')
 */
function upload_object(string $bucketName, string $objectName, string $source): void
{
    $storage = new StorageClient();
    if (!$file = fopen($source, 'r')) {
        throw new \InvalidArgumentException('Unable to open file for reading');
    }
    $bucket = $storage->bucket($bucketName);
    $object = $bucket->upload($file, [
        'name' => $objectName
    ]);
    printf('Uploaded %s to gs://%s/%s' . PHP_EOL, basename($source), $bucketName, $objectName);
}

Python

如需了解详情,请参阅 Cloud Storage Python API 参考文档

如需向 Cloud Storage 进行身份验证,请设置应用默认凭据。 如需了解详情,请参阅为本地开发环境设置身份验证

以下示例会上传单个对象:

from google.cloud import storage

def upload_blob(bucket_name, source_file_name, destination_blob_name):
    """Uploads a file to the bucket."""
    # The ID of your GCS bucket
    # bucket_name = "your-bucket-name"
    # The path to your file to upload
    # source_file_name = "local/path/to/file"
    # The ID of your GCS object
    # destination_blob_name = "storage-object-name"

    storage_client = storage.Client()
    bucket = storage_client.bucket(bucket_name)
    blob = bucket.blob(destination_blob_name)

    # Optional: set a generation-match precondition to avoid potential race conditions
    # and data corruptions. The request to upload is aborted if the object's
    # generation number does not match your precondition. For a destination
    # object that does not yet exist, set the if_generation_match precondition to 0.
    # If the destination object already exists in your bucket, set instead a
    # generation-match precondition using its generation number.
    generation_match_precondition = 0

    blob.upload_from_filename(source_file_name, if_generation_match=generation_match_precondition)

    print(
        f"File {source_file_name} uploaded to {destination_blob_name}."
    )

以下示例会同时上传多个对象:

def upload_many_blobs_with_transfer_manager(
    bucket_name, filenames, source_directory="", workers=8
):
    """Upload every file in a list to a bucket, concurrently in a process pool.

    Each blob name is derived from the filename, not including the
    `source_directory` parameter. For complete control of the blob name for each
    file (and other aspects of individual blob metadata), use
    transfer_manager.upload_many() instead.
    """

    # The ID of your GCS bucket
    # bucket_name = "your-bucket-name"

    # A list (or other iterable) of filenames to upload.
    # filenames = ["file_1.txt", "file_2.txt"]

    # The directory on your computer that is the root of all of the files in the
    # list of filenames. This string is prepended (with os.path.join()) to each
    # filename to get the full path to the file. Relative paths and absolute
    # paths are both accepted. This string is not included in the name of the
    # uploaded blob; it is only used to find the source files. An empty string
    # means "the current working directory". Note that this parameter allows
    # directory traversal (e.g. "/", "../") and is not intended for unsanitized
    # end user input.
    # source_directory=""

    # The maximum number of processes to use for the operation. The performance
    # impact of this value depends on the use case, but smaller files usually
    # benefit from a higher number of processes. Each additional process occupies
    # some CPU and memory resources until finished. Threads can be used instead
    # of processes by passing `worker_type=transfer_manager.THREAD`.
    # workers=8

    from google.cloud.storage import Client, transfer_manager

    storage_client = Client()
    bucket = storage_client.bucket(bucket_name)

    results = transfer_manager.upload_many_from_filenames(
        bucket, filenames, source_directory=source_directory, max_workers=workers
    )

    for name, result in zip(filenames, results):
        # The results list is either `None` or an exception for each filename in
        # the input list, in order.

        if isinstance(result, Exception):
            print("Failed to upload {} due to exception: {}".format(name, result))
        else:
            print("Uploaded {} to {}.".format(name, bucket.name))

以下示例并发上传具有公共前缀的所有对象:

def upload_directory_with_transfer_manager(bucket_name, source_directory, workers=8):
    """Upload every file in a directory, including all files in subdirectories.

    Each blob name is derived from the filename, not including the `directory`
    parameter itself. For complete control of the blob name for each file (and
    other aspects of individual blob metadata), use
    transfer_manager.upload_many() instead.
    """

    # The ID of your GCS bucket
    # bucket_name = "your-bucket-name"

    # The directory on your computer to upload. Files in the directory and its
    # subdirectories will be uploaded. An empty string means "the current
    # working directory".
    # source_directory=""

    # The maximum number of processes to use for the operation. The performance
    # impact of this value depends on the use case, but smaller files usually
    # benefit from a higher number of processes. Each additional process occupies
    # some CPU and memory resources until finished. Threads can be used instead
    # of processes by passing `worker_type=transfer_manager.THREAD`.
    # workers=8

    from pathlib import Path

    from google.cloud.storage import Client, transfer_manager

    storage_client = Client()
    bucket = storage_client.bucket(bucket_name)

    # Generate a list of paths (in string form) relative to the `directory`.
    # This can be done in a single list comprehension, but is expanded into
    # multiple lines here for clarity.

    # First, recursively get all files in `directory` as Path objects.
    directory_as_path_obj = Path(source_directory)
    paths = directory_as_path_obj.rglob("*")

    # Filter so the list only includes files, not directories themselves.
    file_paths = [path for path in paths if path.is_file()]

    # These paths are relative to the current working directory. Next, make them
    # relative to `directory`
    relative_paths = [path.relative_to(source_directory) for path in file_paths]

    # Finally, convert them all to strings.
    string_paths = [str(path) for path in relative_paths]

    print("Found {} files.".format(len(string_paths)))

    # Start the upload.
    results = transfer_manager.upload_many_from_filenames(
        bucket, string_paths, source_directory=source_directory, max_workers=workers
    )

    for name, result in zip(string_paths, results):
        # The results list is either `None` or an exception for each filename in
        # the input list, in order.

        if isinstance(result, Exception):
            print("Failed to upload {} due to exception: {}".format(name, result))
        else:
            print("Uploaded {} to {}.".format(name, bucket.name))

Ruby

如需了解详情,请参阅 Cloud Storage Ruby API 参考文档

如需向 Cloud Storage 进行身份验证,请设置应用默认凭据。 如需了解详情,请参阅为本地开发环境设置身份验证

def upload_file bucket_name:, local_file_path:, file_name: nil
  # The ID of your GCS bucket
  # bucket_name = "your-unique-bucket-name"

  # The path to your file to upload
  # local_file_path = "/local/path/to/file.txt"

  # The ID of your GCS object
  # file_name = "your-file-name"

  require "google/cloud/storage"

  storage = Google::Cloud::Storage.new
  bucket  = storage.bucket bucket_name, skip_lookup: true

  file = bucket.create_file local_file_path, file_name

  puts "Uploaded #{local_file_path} as #{file.name} in bucket #{bucket_name}"
end

Terraform

您可以使用 Terraform 资源上传对象。必须指定 contentsource

# Create a text object in Cloud Storage
resource "google_storage_bucket_object" "default" {
  name = "new-object"
  # Use `source` or `content`
  # source       = "/path/to/an/object"
  content      = "Data as string to be uploaded"
  content_type = "text/plain"
  bucket       = google_storage_bucket.static.id
}

REST API

JSON API

JSON API 区分了媒体上传(其中请求中仅包含对象数据)和 JSON API 分段上传(其中对象数据和对象元数据均包含在请求中)之间的区别。

媒体上传(不含对象元数据的单一请求上传)

  1. OAuth 2.0 Playground 获取授权访问令牌。将 Playground 配置为使用您自己的 OAuth 凭据。如需了解相关说明,请参阅 API 身份验证
  2. 使用 cURL,通过 POST Object 请求调用 JSON API

    curl -X POST --data-binary @OBJECT_LOCATION \
        -H "Authorization: Bearer OAUTH2_TOKEN" \
        -H "Content-Type: OBJECT_CONTENT_TYPE" \
        "https://storage./upload/storage/v1/b/BUCKET_NAME/o?uploadType=media&name=OBJECT_NAME"

    其中:

    • OBJECT_LOCATION 是对象的本地路径。例如 Desktop/dog.png
    • OAUTH2_TOKEN 是您在第 1 步中生成的访问令牌。
    • OBJECT_CONTENT_TYPE 是该对象的内容类型,例如 image/png
    • BUCKET_NAME 是对象要上传到的存储桶的名称。例如 my-bucket
    • OBJECT_NAME 是您要为对象指定的网址编码名称。例如,pets/dog.png 的网址编码为 pets%2Fdog.png

JSON API 分段上传(包含对象元数据的单一请求上传)

  1. OAuth 2.0 Playground 获取授权访问令牌。将 Playground 配置为使用您自己的 OAuth 凭据。如需了解相关说明,请参阅 API 身份验证
  2. 创建一个包含以下信息的 multipart/related 文件:

    --BOUNDARY_STRING
    Content-Type: application/json; charset=UTF-8
    
    OBJECT_METADATA
    
    --BOUNDARY_STRING
    Content-Type: OBJECT_CONTENT_TYPE
    
    OBJECT_DATA
    --BOUNDARY_STRING--

    其中:

    • BOUNDARY_STRING 是您定义的一个字符串,用于标识多部分文件的不同部分。例如 separator_string
    • OBJECT_METADATA 是您要包含的文件元数据(采用 JSON 格式)。此部分至少应包含对象的 name 特性,例如 {"name": "myObject"}
    • OBJECT_CONTENT_TYPE 是该对象的内容类型,例如 text/plain
    • OBJECT_DATA 是对象的数据。

    例如:

    --separator_string
    Content-Type: application/json; charset=UTF-8
    
    {"name":"my-document.txt"}
    
    --separator_string
    Content-Type: text/plain
    
    This is a text file.
    --separator_string--
  3. 使用 cURL,通过 POST Object 请求调用 JSON API

    curl -X POST --data-binary @MULTIPART_FILE_LOCATION \
        -H "Authorization: Bearer OAUTH2_TOKEN" \
        -H "Content-Type: multipart/related; boundary=BOUNDARY_STRING" \
        -H "Content-Length: MULTIPART_FILE_SIZE" \
        "https://storage./upload/storage/v1/b/BUCKET_NAME/o?uploadType=multipart"

    其中:

    • MULTIPART_FILE_LOCATION 是您在第 2 步中创建的多部分文件的本地路径。例如 Desktop/my-upload.multipart
    • OAUTH2_TOKEN 是您在第 1 步中生成的访问令牌。
    • BOUNDARY_STRING 是您在第 2 步中定义的边界字符串。例如 my-boundary
    • MULTIPART_FILE_SIZE 是您在第 2 步中创建的多部分文件的总大小(以字节为单位)。例如 2000000
    • BUCKET_NAME 是对象要上传到的存储桶的名称。例如 my-bucket

如果请求成功,服务器将返回 HTTP 200 OK 状态代码以及文件的元数据。

XML API

  1. OAuth 2.0 Playground 获取授权访问令牌。将 Playground 配置为使用您自己的 OAuth 凭据。如需了解相关说明,请参阅 API 身份验证
  2. 使用 cURL,通过 PUT Object 请求调用 XML API

    curl -X PUT --data-binary @OBJECT_LOCATION \
        -H "Authorization: Bearer OAUTH2_TOKEN" \
        -H "Content-Type: OBJECT_CONTENT_TYPE" \
        "https://storage./BUCKET_NAME/OBJECT_NAME"

    其中:

    • OBJECT_LOCATION 是对象的本地路径。例如 Desktop/dog.png
    • OAUTH2_TOKEN 是您在第 1 步中生成的访问令牌。
    • OBJECT_CONTENT_TYPE 是该对象的内容类型,例如 image/png
    • BUCKET_NAME 是对象要上传到的存储桶的名称。例如 my-bucket
    • OBJECT_NAME 是您要为对象指定的网址编码名称。例如,pets/dog.png 的网址编码为 pets%2Fdog.png

在请求标头中上传对象时,您可以按照上述示例设置 Content-Type 相同的方式设置其他对象元数据。使用 XML API 时,您只能在写入对象时(例如上传、复制或替换对象时)设置元数据。如需了解详情,请参阅修改对象元数据

后续步骤

自行试用

如果您是 Google Cloud 新手,请创建一个账号来评估 Cloud Storage 在实际场景中的表现。新客户还可获享 $300 赠金,用于运行、测试和部署工作负载。

免费试用 Cloud Storage