Job driver logs
See Dataproc job logs in Logging for information on configuring and viewing Dataproc on GKE Spark job driver logs.
Cloud Logging
Enable Logging to view, search, filter, and archive Dataproc on GKE logs in Cloud Logging.
You can search for Spark driver and executor logs in Cloud Logging using the following labels:
- dataproc.googleapis.com/cluster-uuid = Dataproc cluster UUID
- dataproc.googleapis.com/job-uuid = Dataproc job UUID
- dataproc.googleapis.com/role
Sample queries:
labels."k8s-pod/dataproc_googleapis_com/cluster-uuid"="Dataproc cluster UUID"
labels."k8s-pod/dataproc_googleapis_com/job-uuid"="Dataproc job UUID"
labels."k8s-pod/dataproc_googleapis_com/role"="driver"
For more information
- See Dataproc logs for information on accessing Dataproc on GKE job and cluster logs inLogging.