An organization can create an aggregated export sink that can export log entries from all the projects, folders, and billing accounts of the organization. As an example, an organization might use this feature to export audit log entries from its projects to a central location.
Without the aggregated export feature, an export sink is limited to exporting log entries from the resource in which the sink was created: a project, organization, folder, or billing account.
If you create a sink in an organization or folder, and if you set the sink's
includeChildren parameter to True, then that sink can export log entries from
the organization or folder, plus (recursively) from any contained folders,
billing accounts, or projects. The sink's filter can choose log entries from
specific projects, specific resource types, or specific named logs.
For information about export sinks and how to create them, see Exporting Logs in the API.
Creating an aggregated export sink
You must use the Stackdriver Logging API to create sinks in folders, billing accounts, or organizations.
To create an aggregated export sink, do the following:
Choose or create the export destination. It can be in any project. After creating the sink, you must authorize the sink to write to the destination.
parentparameter to be the organization or folder in which to create the sink. The parent must be one of the following:
You must have the Logs Configuration Writer IAM role for the parent to create the sink. For more information about Stackdriver Logging IAM roles, see the Access control guide.
uniqueWriterIdentityparameter to True.
In the LogSink object in the method request body:
filterproperty with the knowledge that log entries from all your projects will be matched against the filter.
For some examples of useful filters, see Using filters with aggregated exports on this page.
Authorize the sink to write to the destination:
Get the sink's service account name from the
writerIdentityfield of the sink object.
Give that service account permission to write to your export destination.
If you do not have permission to make that change to the destination, then send the service account name to someone who can make that change for you.
For more information about granting service accounts permissions for resources, see Granting roles to service accounts.
Using filters with aggregated exports
Like any sink, your aggregated export sink contains a filter that selects individual log entries. Following are some examples of filter comparisons that are useful when using the aggregated export feature. For more details about filters, see Advanced Logs Filters. Some examples use the following notation:
:is the substring operator. Do not substitute the
...represents any additional filter comparisons.
Choosing audit logs
To export audit logs, choose one of the following sample comparisons. The first alternative chooses both Admin Activity and Data Access audit logs:
logName:"/logs/cloudaudit.googleapis.com" AND ... logName:"/logs/cloudaudit.googleapis.com%2Factivity" AND ... logName:"/logs/cloudaudit.googleapis.com%2Fdata_access" AND ...
For information about audit logs, see Cloud Audit Logging.
To export logs from specific projects, folders, or organizations, use one of the following sample comparisons:
logName:"projects/[PROJECT_ID]/logs/" AND ... logName:("projects/[PROJ_A]/logs/" OR "projects/[PROJ_B]/logs/") AND ... logName:"folders/[FOLDER_ID]/logs/" AND ... logName:"organizations/[ORGANIZATION_ID]/logs/" AND ...
To export logs from only a specific resource in a project, use multiple comparisons to specify the resource exactly:
logName:"projects/[PROJECT_ID]/logs" AND resource.type=[RESOURCE_TYPE] AND resource.labels.instance_id=[INSTANCE_ID]
For a list of resource types, see Monitored Resource List.
Sampling log entries
To export a random sample of log entries, add the
function. For example, to export only 10% of the log entries matching your
current filter, use this addition:
sample(insertId, 0.10) AND ...
For more information, see Sample function.
For more information about Stackdriver Logging filters, see Advanced log filters.