Cloud Dataproc API C++ Client Library
An idiomatic C++ client library for the Cloud Dataproc API, a managed Apache Spark and Apache Hadoop service that lets you take advantage of open source data tools for batch processing, querying, streaming, and machine learning. This library allows you to manage Cloud Dataproc resources, but it does not provide APIs to run C++ applications in Cloud Dataproc.
While this library is GA, please note Google Cloud C++ client libraries do not follow Semantic Versioning.
Quickstart
The following shows the code that you'll run in the google/cloud/dataproc/quickstart/
directory, which should give you a taste of the Cloud Dataproc API C++ client library API.
#include "google/cloud/dataproc/v1/cluster_controller_client.h"
#include "google/cloud/common_options.h"
#include <iostream>
int main(int argc, char* argv[]) try {
if (argc != 3) {
std::cerr << "Usage: " << argv[0] << " project-id region\n";
return 1;
}
std::string const project_id = argv[1];
std::string const region = argv[2];
namespace dataproc = ::google::cloud::dataproc_v1;
auto client = dataproc::ClusterControllerClient(
dataproc::MakeClusterControllerConnection(region == "global" ? ""
: region));
for (auto c : client.ListClusters(project_id, region)) {
if (!c) throw std::move(c).status();
std::cout << c->cluster_name() << "\n";
}
return 0;
} catch (google::cloud::Status const& status) {
std::cerr << "google::cloud::Status thrown: " << status << "\n";
return 1;
}
Main classes
This library offers multiple *Client
classes, which are listed below. Each one of these classes exposes all the RPCs for a gRPC service
as member functions of the class. This library groups multiple gRPC services because they are part of the same product or are often used together. A typical example may be the administrative and data plane operations for a single product.
The library also has other classes that provide helpers, configuration parameters, and infrastructure to mock the *Client
classes when testing your application.
dataproc_v1::AutoscalingPolicyServiceClient
dataproc_v1::BatchControllerClient
dataproc_v1::ClusterControllerClient
dataproc_v1::JobControllerClient
dataproc_v1::WorkflowTemplateServiceClient
Retry, Backoff, and Idempotency Policies.
The library automatically retries requests that fail with transient errors, and uses exponential backoff to backoff between retries. Application developers can override the default policies.
More Information
- Error Handling - describes how the library reports errors.
- How to Override the Default Endpoint - describes how to override the default endpoint.
- How to Override the Authentication Credentials - describes how to change the authentication credentials used by the library.
- Environment Variables - describes environment variables that can configure the behavior of the library.