Using the GCS+gRPC Plugin
The GCS C++ client library includes an optional plugin to access GCS via gRPC. When using GCS from Google Compute Engine (GCE) this plugin can enable higher total throughput across large workloads that run on hundreds or thousands of VMs. Note that this feature is not GA. Consult with your account manager to be included in the early access program.
Release Notes
There are some features that the gRPC plugin does not implement or implements differently from the REST-based implementation.
- For best performance consider using Protobuf >= 23.2 and gRPC >= 1.55.0. Older versions of Protobuf requires enabling ctype=CORD workarounds at compile-time.
- To override the default endpoint you should use
EndpointOption
instead of [RestEndpointOption
](@ refgoogle::cloud::storage::RestEndpointOption
). - The default endpoint (
storage.googleapis.com
) works from any hosting environment (on-prem, GKE, GCE, other cloud providers, etc.). For best performance on GCE or GKE consider usinggoogle-c2p:///storage.googleapis.com
. - The
Fields
request parameter is passed verbatim to the backend. Some fields have different names in gRPC, and the backend does not translate them. This is a rarely used feature, and we do not anticipate the behavior change is likely to cause problems. If it does cause any issues please file a bug. - All the
storage::Client::*AccessControl()
functions are emulated. For functions that change state, the emulation uses an OCC loop. While this preserves the semantics of a single RPC (e.g. no other concurrent changes are lost), it requires making at least two RPCs. This may increase the cost of calling these functions. Our telemetry shows these functions are not used via the C++ client library, but you should keep this in mind before deciding to use these functions in new code.
Required code changes
To use GCS+gRPC plugin you need to make some changes to how your application initializes the GCS C++ client library. First, an additional header provides the initialization functions:
#include "google/cloud/storage/grpc_plugin.h"
#include "google/cloud/storage/options.h"
#include "google/cloud/common_options.h"
#include "google/cloud/options.h"
Then you initialize a google::cloud::storage::Client
using a new function:
auto client = google::cloud::storage_experimental::DefaultGrpcClient();
The google::cloud::storage::Client
object returned by this function can be used as before:
void GrpcReadWrite(std::string const& bucket_name) {
namespace gcs = ::google::cloud::storage;
auto constexpr kText = R"""(Hello World!)""";
auto client = google::cloud::storage_experimental::DefaultGrpcClient();
auto object = client.InsertObject(bucket_name, "lorem.txt", kText);
if (!object) throw std::move(object).status();
auto input = client.ReadObject(bucket_name, "lorem.txt",
gcs::Generation(object->generation()));
std::string const actual(std::istreambuf_iterator<char>{input}, {});
std::cout << "The contents read back are:\n"
<< actual
<< "\nThe received checksums are: " << input.received_hash()
<< "\nThe computed checksums are: " << input.computed_hash()
<< "\nThe original hashes are: crc32c=" << object->crc32c()
<< ",md5=" << object->md5_hash() << "\n";
}
Changing your build scripts
If you are using CMake to compile your application, then you need to change the target_link_libraries()
command to use google-cloud-cpp::experimental-storage-grpc
instead of google-cloud-cpp::storage
. For example, our quickstart program for gRPC uses:
add_executable(quickstart_grpc quickstart_grpc.cc)
target_link_libraries(quickstart_grpc google-cloud-cpp::experimental-storage-grpc)
If you are using Bazel to compile your application, then you need to change the dependencies from @google_cloud_cpp//:storage
to @google_cloud_cpp//:experimental-storage-grpc
. For example, our `grpc/quickstart uses:
cc_binary(
name = "quickstart_grpc",
srcs = [
"quickstart_grpc.cc",
],
deps = [
"@google_cloud_cpp//:experimental-storage-grpc",
],
)