This page shows how to get started with the Cloud Client Libraries for the Dataproc API. However, we recommend using the older Google APIs Client Libraries if running on Google App Engine standard environment. Read more about the client libraries for Cloud APIs in Client Libraries Explained.
Dataproc Cloud Client Libraries may be in alpha or beta stage. See the library reference for details.Installing the client library
C#
For more information, see Setting Up a C# Development Environment.
Also see Google.Cloud.Dataproc.V1 InstallationGo
For more information, see Setting Up a Go Development Environment.
go get -u cloud.google.com/go/dataproc/apiv1
For more information, see Install the Cloud Client Libraries for Go.
Java
For more information, see Setting Up a Java Development Environment.
If you are using Maven, add this to your pom.xml file:<dependency> <groupId>com.google.cloud</groupId> <artifactId>google-cloud-dataproc</artifactId> <version>insert dataproc-library-version here</version> </dependency>
If you are using Gradle, add this to your dependencies:
compile group: 'com.google.cloud', name: 'google-cloud-dataproc', version: 'insert dataproc-library-version here'
Node.js
For more information, see Setting Up a Node.js Development Environment.
npm install --save @google-cloud/dataproc
PHP
For more information, see Using PHP on Google Cloud.
composer require google/cloud
Python
For more information, see Setting Up a Python Development Environment.
pip install --upgrade google-cloud-dataproc
Ruby
For more information, see Setting Up a Ruby Development Environment.
gem install google-cloud-dataproc
Setting up authentication
To run the client library, you must first set up authentication by creating a service account and setting an environment variable. Complete the following steps to set up authentication. For other ways to authenticate, see the GCP authentication documentation.
Cloud Console
-
In the Cloud Console, go to the Create service account key page.
Go to the Create Service Account Key page - From the Service account list, select New service account.
- In the Service account name field, enter a name.
From the Role list, select Project > Owner.
- Click Create. A JSON file that contains your key downloads to your computer.
Command line
You can run the following commands using the Cloud SDK on your local machine, or in Cloud Shell.
-
Create the service account. Replace NAME with a name for the service account.
gcloud iam service-accounts create NAME
-
Grant permissions to the service account. Replace PROJECT_ID with your project ID.
gcloud projects add-iam-policy-binding PROJECT_ID --member="serviceAccount:NAME@PROJECT_ID.iam.gserviceaccount.com" --role="roles/owner"
-
Generate the key file. Replace FILE_NAME with a name for the key file.
gcloud iam service-accounts keys create FILE_NAME.json --iam-account=NAME@PROJECT_ID.iam.gserviceaccount.com
Provide authentication credentials to your application code by
setting the environment variable GOOGLE_APPLICATION_CREDENTIALS
.
Replace [PATH] with the file path of the JSON file that contains your
service account key. This variable only applies to your current shell session,
so if you open a new session, set the variable again.
Linux or macOS
export GOOGLE_APPLICATION_CREDENTIALS="[PATH]"
For example:
export GOOGLE_APPLICATION_CREDENTIALS="/home/user/Downloads/my-key.json"
Windows
With PowerShell:
$env:GOOGLE_APPLICATION_CREDENTIALS="[PATH]"
For example:
$env:GOOGLE_APPLICATION_CREDENTIALS="C:\Users\username\Downloads\my-key.json"
With command prompt:
set GOOGLE_APPLICATION_CREDENTIALS=[PATH]
Using the client library
The following example shows how to use the client library.
Go
Before trying this sample, follow the Go setup instructions in the Dataproc Quickstart Using Client Libraries. For more information, see the Dataproc Go API reference documentation.
Java
Before trying this sample, follow the Java setup instructions in the Dataproc Quickstart Using Client Libraries. For more information, see the Dataproc Java API reference documentation.
Node.js
Before trying this sample, follow the Node.js setup instructions in the Dataproc Quickstart Using Client Libraries. For more information, see the Dataproc Node.js API reference documentation.
.Python
Before trying this sample, follow the Python setup instructions in the Dataproc Quickstart Using Client Libraries. For more information, see the Dataproc Python API reference documentation.