>

Installing the Notifier app

This page provides the Notifier example of the Cloud SCC application package. Notifier subscribes to the Notifications Cloud Pub/Sub Topic and sends notifications to the configured channel, like email or SMS. Notifications refers to Cloud SCC query results from other apps, such as from the Creator app.

This guide was written for tools version 3.3.0. If you're using a different version, please see the README file included with the tools version you downloaded. As of May 22, 2019, the most recent release version is 4.0.1.

Before you begin

Before you start this guide, you must complete the prerequisites and installation setup in Setting up Cloud SCC tools.

To install and run the Notifier package, you will also need the following:

  • An active GCP Organization
  • An active Cloud Billing account
  • The following Cloud Identity and Access Management (Cloud IAM) roles at the organization level:
    • Project Creator - roles/resourcemanager.projectCreator
    • Billing Account User - roles/billing.user
    • Organization Role Administrator - roles/iam.organizationRoleAdmin
    • Viewer - roles/viewer
    • Service Usage Admin - roles/serviceusage.serviceUsageAdmin
    • API Keys Admin - roles/serviceusage.apiKeysAdmin
    • Compute Storage Admin - roles/storage.admin
    • Storage Admin - roles/storage.admin
    • Storage Object Admin - roles/storage.objectAdmin
    • Pub/Sub Admin - roles/pubsub.admin
    • Cloud Functions Developer - roles/cloudfunctions.developer
    • Service Account User - roles/iam.serviceAccountUser
    • Service Management Administrator - roles/servicemanagement.admin
    • Cloud Scheduler Admin - roles/cloudscheduler.admin
    • App Engine Admin - roles/appengine.appAdmin
    • Cloud Tasks Queue Admin - roles/cloudtasks.queueAdmin
  • To use Notifier, you will need to set up the communication channels you want to use. The following services are suggested:
    • App Engine Mail: if you want to send email notifications in a testing environment and your expected use is within a limited quota.
    • SendGrid: if you want to send email notifications using a robust mail service. You must have at least a trial account to have an API KEY and an email to use as the sender.
    • Twilio: if you want to send SMS notifications. You must have at least a trial account to get an account SID, an AUTH token, and a Twilio number.
    • Jira: if you want to use an issue tracking system. You must have an account with a project created and one or more users to assign issues to.

Setting up environment variables

  1. Go to the Google Cloud Platform Console.
    Go to the GCP Console page
  2. Click Activate Cloud Shell.
  3. Run the following commands to set environment variables. Use the tools release version you downloaded during setup. This guide was written for tools version 3.3.0. For other tools versions, see the README included with the files you downloaded.

    # Release version you downloaded during setup
    export VERSION=[RELEASE_VERSION]
    
    # Directory to unzip the installation files
    export WORKING_DIR=${HOME}/scc-tools-install
    
    # Organzation ID where the script will run
    export ORGANIZATION_ID=[YOUR_ORG_ID]
    
    # Project ID to be created
    export NOTIFIER_PROJECT_ID=[YOUR_NOTIFIER_PROJECT_ID]
    
    # A valid billing account ID
    export BILLING=[YOUR_BILLING_ACCOUNT_ID]
    
    # One Compute Engine region listed in Regions and Zones:
    # https://cloud.google.com/compute/docs/regions-zones
    export GCE_REGION=[YOUR_REGION]
    
    # A selected location from the App Engine Locations:
    # https://cloud.google.com/appengine/docs/locations) list
    export GAE_LOCATION=[YOUR_LOCATION]
    
    # The service account created on the Cloud SCC-enabled project in the
    # organization. See:
    # https://cloud.google.com/security-command-center/docs/how-to-programmatic-access
    export SCC_SA_FILE=[YOUR_SERVICE_ACCOUNT]
    
    # Comma separated values for custom roles that can be added to the deployer
    # service account
    export CUSTOM_ROLES=custom.gaeAppCreator
    
  4. On the Cloud Shell menu bar, click Upload file on the More devshell settings menu.

  5. Upload the scc-notifier-${VERSION}.zip file you downloaded during the installation setup.

  6. Unzip the file you uploaded earlier by running:

    unzip -qo scc-notifier-${VERSION}.zip -d ${WORKING_DIR}
    
  7. Go to the installation working directory:

    cd ${WORKING_DIR}
    

Installing the Notifier app package

In any of the following sections, you can simulate executions of the commands by using the option --simulation.

Step 1: Creating the project

Create the project and enable billing by running:

  1. Create the project:

     gcloud projects create ${NOTIFIER_PROJECT_ID} \
       --organization ${ORGANIZATION_ID}
    
  2. Enable billing:

     gcloud beta billing projects link ${NOTIFIER_PROJECT_ID} \
       --billing-account ${BILLING}
    

Step 2: Enabling App Engine for the app

The Notifier app uses App Engine as its execution environment. To select a App Engine location and create an app to install to, run:

gcloud app create \
 --region  ${GAE_LOCATION} \
 --project ${NOTIFIER_PROJECT_ID}

Step 3: Enabling APIs

To enable the required Google APIs in the Notifier project, run:

gcloud services enable \
  securitycenter.googleapis.com \
  servicemanagement.googleapis.com \
  appengine.googleapis.com \
  cloudresourcemanager.googleapis.com \
  cloudfunctions.googleapis.com \
  --project ${NOTIFIER_PROJECT_ID}

Step 5: Deploy the application

To create the remaining infrastructure and deploy the application, run the command below. If you want to run a simulation of the execution, use --simulation instead of --no-simulation.

(cd setup; \
pipenv run python3 run_setup_notifier.py \
  --organization_id ${ORGANIZATION_ID} \
  --region ${REGION} \
  --notifier_bucket ${NOTIFIER_CF_BUCKET} \
  --notifier_appengine_version ${NOTIFIER_PROJECT_ENDPOINT} \
  --notifier_project ${NOTIFIER_PROJECT_ID} \
  --no-simulation)

Configuring Notifier

Notifier sends notifications to configured users for assets or security findings that are published to a Cloud Pub/Sub topic by the Creator app. This section explains how to configure Notifier for use.

Step 1: Creating an access token

gcloud auth application-default login

export NOTIFIER_ACCESS_TOKEN=$(gcloud beta auth application-default print-access-token)

Step 2: Getting the API endpoint

export NOTIFIER_API_ENDPOINT_URL=$(gcloud app services browse api-service \
--no-launch-browser \
--format="value(url)" \
--project ${NOTIFIER_PROJECT_ID})
export NOTIFIER_API_ENDPOINT_URL=$NOTIFIER_API_ENDPOINT_URL/_ah/api

Step 3: Stop processing messages

Each time you change Notifier configuration, you should stop processing messages first:

(cd notifier/tools; \
pipenv --python 3.5.3 ; \
pipenv install --ignore-pipfile)

(cd notifier/tools; \
pipenv run python3 load.py stop \
--api_endpoints_url $NOTIFIER_API_ENDPOINT_URL \
--access_token $NOTIFIER_ACCESS_TOKEN \
--gcpid $NOTIFIER_PROJECT_ID)

Step 4: Loading users

User data value is stored in users.txt as email, phone number, JiraId, and type. To load users, add data values in the given order and upload a users.txt file:

(cd notifier/tools; \
pipenv run python3 load.py users \
--userfile ./samples/users.txt \
--gcpid $notifier_project_id \
--api_endpoints_url $NOTIFIER_API_ENDPOINT_URL \
--access_token $NOTIFIER_ACCESS_TOKEN)

For a sample user file, see notifier/tools/samples/users.txt in the app package you downloaded. The sample file includes the following fields:

  • An email address to use for email notification from App Engine and SendGrid.
  • A cell phone number to receive SMS notification. If you want to use Twilio, make sure you register the number.
  • A user ID for the Jira integration. This is usually the user's login ID for Jira.
  • The User type, for example "developer". You can use this field to filter users from a large file and upload only a single type.

If you want to add more users, copy the sample user data and then create new lines with the other users info.

Step 5: Setting up the active notification channels

The valid notification channels values are:

  • gae_email - Uses the App Engine Mail API. It is limited to 100 email each day and is used mainly for testing and validation.
  • sms - Uses the Twilio API to send SMS notifications.
  • sendgrid - Uses the SendGrid API to send email notifications.
  • jira - Uses the Jira API to create issues based on the messages received.

To set gae_email, sms, and sendgrid as the active notification channels run:

(cd notifier/tools; \
pipenv run python3 load.py channels \
--channel 'gae_email,sms,sendgrid' \
--api_endpoints_url $NOTIFIER_API_ENDPOINT_URL \
--access_token $NOTIFIER_ACCESS_TOKEN \
--gcpid $NOTIFIER_PROJECT_ID)

If you want to validate the app, you can add a single channel, like gae_email, before adding more channels.

Step 6: Setting up notification channels configuration

To configure a notification channel, edit and upload configuration.yaml for the channels you want to use. The configuration file has the following sections:

  • SMS: for Twilio configuration, you must include the following:
  • SENDGRID: for SendGrid configuration, you must include the following:
    • API_KEY
    • FROM_EMAIL, a valid email in your organization to send from.
    • REPLYTO_EMAIL, a valid email in your organization to use as a reply to.
  • GAE_EMAIL: for App Engine email configuration, you must include a FROM_EMAIL.
    • The email must conform to the App Engine restrictions.
    • The email must be registered as a valid mail sender:
    • Go to the GCP Console App Engine Email Senders page.
      Go to the Email Senders page
    • Under Mail API Authorized Senders, click Add authorized email senders.
    • In the Add email senders window that appears, enter the email address that you want to use to send notifications.
  • JIRA: for Jira issue creation, you must include the following:
    • URL in the following format: https://jiraexample.atlassian.net
    • PROJECT_KEY, the project key for your Jira project.
    • ISSUE_TYPE_NAME, the kind of issue that you want Jira to create. This value must be one of: Bug, Epic, Story, Task.
    • USER_NAME, the name of a user who is configured to access the project.
    • API_TOKEN that will be created for the user.

All notifications also require Cloud Functions configuration in the CLOUD_FUNCTION section of configuration.yaml. A sample function is deployed with the Notifier Application. CLOUD_FUNCTION must include the following:

  • URL from the following:

    # the project created to install the application
    export NOTIFIER_PROJECT_ID=[NOTIFIER_PROJECT_ID]
    
    # one region listed in Cloud Functions Locations
    # https://cloud.google.com/functions/docs/locations
    export REGION=[CLOUD_FUNCTION_REGION]
    
    echo https://${REGION}-${NOTIFIER_PROJECT_ID}.cloudfunctions.net/notifier_notifyDefaultHttp
    
  • CLIENT_KEY from the Cloud Functions configuration file at notifier/function/default/config.json.

The sample function logs the notification received and then returns a Notification accepted. message.

After you configure communication channels and the CLOUD_FUNCTION section, upload configuration.yaml by running:

(cd notifier/tools; \
pipenv run python3 load.py config \
--config ./samples/configuration.yaml \
--gcpid $NOTIFIER_PROJECT_ID \
--access_token $NOTIFIER_ACCESS_TOKEN \
--api_endpoints_url $NOTIFIER_API_ENDPOINT_URL)

Step 7: Loading rules

A set of rules determines if a notification should be sent for the kind of resource: Asset or Finding, and by type of action upon the resource: ANY-CREATED, ANY-DELETED, ANY-ACTIVE, ANY-OPENED, ANY-RESOLVE, and ALL.

The sample file notifier/tools/samples/rules.yaml is configured to send notification in all cases. You can edit it to specify the kind of notifications you want to receive, then upload it by running:

(cd notifier/tools; \
pipenv run python3 load.py rules \
--yaml_file ./samples/rules.yaml \
--gcpid $NOTIFIER_PROJECT_ID \
--access_token $NOTIFIER_ACCESS_TOKEN \
--api_endpoints_url $NOTIFIER_API_ENDPOINT_URL)

Step 8: Loading custom message info text

You can add a custom message info text to the notification using the following script:

export MSG_TEXT1 ="This notification is confidential."
export MSG_TEXT2 ="If you received it in error please notify the system administrator."

(cd notifier/tools; \
pipenv run python3 load.py extra_info \
--notitype ASSETS \
--message "${MSG_TEXT1}${MSG_TEXT2}" \
--api_endpoints_url $NOTIFIER_API_ENDPOINT_URL \
--access_token $NOTIFIER_ACCESS_TOKEN \
--gcpid ${NOTIFIER_PROJECT_ID})

This custom message info text will be added to all the messages. You can use this format to add any custom information to the message: a disclaimer, contact information, or any other relevant info for the user who will receive the message.

Step 9: Start processing messages

After you finish configuring messages, re-start message processing by running:

(cd notifier/tools; \
pipenv run python3 load.py start \
--api_endpoints_url $NOTIFIER_API_ENDPOINT_URL \
--access_token $NOTIFIER_ACCESS_TOKEN \
--gcpid ${NOTIFIER_PROJECT_ID})

Note: The Notifier Application tracks assets and findings that have been sent from a given query so that they will not be sent again. You can change this behavior when you deploy the app by adding the parameter --notifier_hash_mecanism OFF on the call to the run_setup_notifier.py script. You can also use the GCP Console and delete the hashes.

Connecting the Notifier app to the Creator app

If you deployed the Creator application, you created a topic named publish_processing on the Creator project during the app setup process. To push Creator messages to the Notifier app, you need to create a subscription on the publish_processing topic. To run the script, you need the Cloud IAM role Pub/Sub Admin - roles/pubsub.admin - on the Creator project.

export CREATOR_PROJECT_ID=[YOUR_CREATOR_PROJECT_ID]
(cd setup; \
export NOTIFIER_PUBSUB_PATH=https://${NOTIFIER_PROJECT_ENDPOINT}-pubsub-dot-${NOTIFIER_PROJECT_ID}; \
export NOTIFIER_PUSH_ENDPOINTt=${NOTIFIER_PUBSUB_PATH}.appspot.com/_ah/push-handlers/receive_message; \
pipenv run python3 add_subscription.py \
--topic_name publish_processing \
--topic_project ${CREATOR_PROJECT_ID} \
--subscription_project ${NOTIFIER_PROJECT_ID} \
--subscription_name publishprocessing-notifier \
--push_endpoint ${NOTIFIER_PUSH_ENDPOINT})

After you run the above script, queries that use the topic publish_processing, and queries that use the default topic, will send notifications to the Notifier app.

¿Te sirvió esta página? Envíanos tu opinión:

Enviar comentarios sobre…

Cloud Security Command Center
¿Necesitas ayuda? Visita nuestra página de asistencia.