This topic describes how Anthos clusters on AWS integrates with Cloud Logging and shows you how to access your logs.
Before you begin
Configure the Google Cloud CLI and enable the required APIs in your Google Cloud project.
Authorize Cloud Logging / Cloud Monitoring to set up permissions for Google Cloud's operations suite.
Anthos clusters on AWS has built-in support of Cloud Logging for system components running on both the control plane and node pool nodes.
What data is collected
Anthos clusters on AWS logs the following information:
- Logs for system components on each of the control plane replica.
- Logs for system services on each of the node pool nodes.
- Optionally, logs for workloads
Data collected by the agents can be managed and deleted like any other metric and log data, as described in Cloud Logging documentation.
Anthos clusters on AWS includes a Fluentbit-based logging agent. On the control plane nodes, the logging agent runs as a system component; on the node pool nodes, it runs as a Daemonset.
Enable workload logging
To enable workload logging, you must create a new Kubernetes
1.22 or later cluster with
gcloud alpha aws clusters create.
By default, when you create a cluster Anthos clusters on AWS enables system
logging. To enable workload logging, you must use
gcloud alpha to create a
cluster with the
--logging flag. You can choose to enable logging for system
components, workloads, or both.
For example, to enable both system and workload logging on a new cluster, add the following flag to your cluster creation command:
gcloud alpha container aws clusters create CLUSTER_NAME ... --logging=SYSTEM,WORKLOAD
For more information on how to create a cluster, see Create a cluster.
Access your logs
There are several ways to access your Anthos clusters on AWS logs in Cloud Logging:
Logs Explorer – You can see your logs directly from the Logs Explorer by using the logging filters to select the Kubernetes resources, such as cluster, node, namespace, pod, or container logs.
Google Cloud CLI – Using the
gcloud logging readcommand, select the appropriate cluster, node, pod, and container logs.
Understanding your logs
These are the resource types that are specific to Anthos clusters on AWS:
|Resource type||Display name|
||Control plane component logs|
||Node Pool logs|
||Workload container logs|
When Anthos clusters on AWS writes your cluster's logs, each log entry includes the resource type. Understanding where logs appear makes it easier to find logs when you need them.
Control Plane logs
Control plane logs use the
k8s_control_plane_component resource type. These
include logs from system components running on the control plane replicas, such
System apps logs
System apps logs fall under the
k8s_container resource type. These include
logs from pods installed on the worker nodes by Anthos clusters on AWS.
Specifically, containers running in the following namespaces are included:
Finding your logs in the Cloud Logging user interface
You can view your logs using the Logs Explorer in the Cloud Logging user interface.
Using the Query Builder, you can build a query by adding query parameters
manually. For example, if you want to explore logs for system workloads, you can
start with selecting or searching for the
k8s_container resource type, and
then select the location and cluster name. You can then refine your search by
filtering the container or Pod names.
The Logs Explorer offers an additional way to build your search queries using the Logs field explorer. It shows the count of log entries, sorted by decreasing count, for the given log field. Using the Logs field explorer is particularly useful for Anthos clusters on AWS logs because it provides an easy way to select the Kubernetes values for your resources. For example, you can select logs for a specific cluster, Namespace, Pod name, and container name.
For more information, see Using the Logs Explorer.
This section includes sample queries that you can make on Logs Explorer.
Example 1: Get the logs of
kube-apiserver on control plane nodes for an
AWS cluster CLUSTER_NAME:
resource.type="k8s_control_plane_component" resource.labels.cluster_name="awsClusters/CLUSTER_NAME" resource.labels.component_name="apiserver"
Example 2: Get the logs of
kubelet on worker nodes for an AWS
resource.type="k8s_node" resource.labels.cluster_name="awsClusters/CLUSTER_NAME" log_name="projects/PROJECT_ID/logs/kubelet"
Example 3: Get the logs of the
cilium-agent container for an
AWS cluster CLUSTER_NAME:
resource.type="k8s_container" resource.labels.cluster_name="awsClusters/CLUSTER_NAME" resource.labels.container_name="cilium-agent"