This document describes metadata that is preserved when you use Storage Transfer Service and Transfer service for on-premises data to transfer data to Cloud Storage.
Overview
Storage Transfer Service and Transfer for on-premises preserve the following metadata during a transfer to Cloud Storage:
User-created custom metadata for transfers that originate from Cloud Storage, Amazon Simple Storage Service (Amazon S3), or Microsoft Azure Blob Storage (Microsoft Azure Storage).
File size and last modified time (mtime) for transfers that originate from POSIX file systems.
Metadata fields that are not explicitly mentioned in this document are not preserved.
Object and file metadata
Each object in Cloud Storage has metadata associated with the object, stored as key:value pairs. The metadata identifies properties of the object and how the object is handled when it is accessed. For more information about object metadata in Cloud Storage, see Object metadata.
The following describes the mutability of metadata in Cloud Storage:
Editable metadata: If you have sufficient privileges, you can edit these metadata values. The following are types of editable metadata available in Cloud Storage:
Fixed-key metadata: The values of fixed-key metadata can be changed, but the keys cannot. This type of metadata typically corresponds to HTTP headers. For more information, see Fixed-key metadata.
Custom metadata: Both the values and the keys of custom metadata can be changed. For more information, see Custom metadata.
Non-editable metadata: This type of metadata cannot be directly edited. It is set when the object is created or rewritten in Cloud Storage. For more information, see Non-editable metadata.
File metadata contains information about files and directories. Examples of file metadata in POSIX file systems include mtime, file size, file modes, and file ownership.
Metadata preservation behavior
The following sections list metadata examples from different source storage systems and how Storage Transfer Service and Transfer for on-premises preserve metadata from each. For an exhaustive list of metadata, refer to the source storage system's documentation.
Amazon S3 to Cloud Storage
Metadata example | Preservation behavior |
---|---|
Amazon S3 fixed-key metadata fields, such as:
Cache-Control ,
Content-Disposition , and
Content-Type .
|
Preserved as fixed-key metadata. |
Amazon S3 user-defined metadata, named as
x-amz-meta-name:value , where
name and value are user-defined key:value pairs.
For more information, see the
User-defined object metadata
section of
Object key and metadata.
|
Preserved as a custom metadata field in destination Cloud Storage objects, which you can edit later or remove.
|
ETag |
Preserved as a custom metadata field with the key
x-goog-source-etag , which you can edit later or remove.
|
Object size. |
Preserved as size .
|
Amazon S3 access control lists (ACLs). For a complete list, see the Condition Keys section of Access Control List (ACL) Overview. | Not preserved. |
Amazon S3 object tags, defined by you as key-value pairs. For more information, see Object Tags. | Not preserved. |
Amazon S3 system-defined metadata, except for ETag and
object size. For a complete list, see the
System-defined object metadata section of
Object key and metadata. |
Not preserved.
Timestamp metadata from the source isn't preserved. Creation time,
|
Microsoft Azure Storage to Cloud Storage
Metadata example | Preservation behavior |
---|---|
Microsoft Azure Storage fixed-key metadata fields, such as:
Cache-Control ,
Content-Disposition , and
Content-Type .
|
Preserved as fixed-key metadata. |
Microsoft Azure Storage user-defined metadata, named as
x-ms-meta-name:value , where
name and value are user-defined key:value pairs.
For more information, see
Settings and retrieving properties and metadata for Blob service
resources
.
|
Preserved as a custom metadata field in destination Cloud Storage objects, which you can edit later or remove.
|
ETag
|
Preserved as a custom metadata field with the key
x-goog-source-etag , which you can edit later or remove.
|
Object size. |
Preserved as size .
|
Microsoft Azure Storage access control, specifically
x-ms-blob-public-access . For more information, see the
Response Headers
section of
Get Container ACL
.
|
Not preserved. |
Microsoft Azure Storage index tags. For more information, see Manage and find Azure Blob data with blob index tags . | Not preserved. |
Microsoft Azure Storage timestamp metadata, such as:
Last-Modified ,
x-ms-creation-time ,
x-ms-version ,
x-ms-request-server-encrypted , and
x-ms-encryption-scope .
For more information, see
Set Blob Metadata
.
|
Not preserved.
Timestamp metadata from the source isn't preserved. Creation time,
|
Transfers between Cloud Storage buckets
Metadata example | Preservation behavior |
---|---|
Cloud Storage fixed-key metadata fields, such as:
For more information, see Object metadata |
Preserved as fixed-key metadata. |
Cloud Storage user-defined metadata, named as
x-goog-meta-name:value , where
name and value are user-defined key:value pairs.
For more information, see
Custom metadata.
|
Preserved as a custom metadata field in destination Cloud Storage objects, which you can edit later or remove.
|
Object size |
Preserved as size .
|
Cloud Storage Access Control Lists (ACLs). For more information, see Predefined ACLs. | Not preserved. |
Cloud Storage non-editable metadata, such as:
temporaryHold ,
eventBasedHold ,
generation ,
etag ,
componentCount , and
storageClass .
|
Not Preserved. |
Cloud Storage timestamp metadata, such as:
timeCreated and updated .
|
Not Preserved.
Timestamp metadata from the source isn't preserved. Creation time,
|
For a list of metadata in Cloud Storage, see Objects.
URL list transfer to Cloud Storage
For more information about URL lists, see Creating a URL list.
Metadata example | Preservation behavior |
---|---|
Fixed-key metadata fields, such as:
Cache-Control ,
Content-Disposition , and
Content-Type .
|
Preserved as editable metadata. |
Content-Length and
MD5
|
Preserved as non-editable metadata.
If the source doesn't provide an
This preservation behavior is specific to |
Timestamp metadata, such as: Creation time, modified time, and other source-specific metadata. |
Not preserved.
Timestamp metadata from the source isn't preserved. Creation time,
|
POSIX file system to Cloud Storage
Metadata example | Preservation behavior |
---|---|
Modified time (mtime) and file size. |
Preserved.
mtime is preserved as custom metadata with the key
|
Folder and file permissions, user ID, group ID, file permissions, hard links, and symbolic links. |
Not preserved. The reason folder metadata is not preserved is that Storage Transfer Service and Transfer service for on-premises data don't create folder placeholder objects in Cloud Storage to represent folders. |