A Cloud Deploy rollout includes phases. A phase is an ordered, logical grouping of jobs to do in a rollout.
Each phase includes jobs, which are the actions to take in each phase (for
example, deploy
or verify
). And each job can have zero or more job runs.
A job run is an instance of a job. If the job hasn't run, there are no job runs.
This document describes phases, jobs, and job runs, and how to manage them.
Structure of a rollout
A rollout is a Cloud Deploy resource that associates a release with a target.
Phases
A rollout consists of one or more phases.
For a standard deployment strategy, there
is only one phase: stable
.
For a canary deployment strategy, There is a separate phase for each configured percentage. For example, if you configure a canary that deploys 25%, then 50%, then 100%, there will be three phases:
canary-25
canary-50
stable
These phase names are standard: canary-[PERCENTAGE]
for canary stages, and
stable
for the 100% phase. However, if you configure a
manual or
custom canary, you can
control the phase names.
Jobs and job runs
Each rollout phase includes one or more jobs.
For a rollout in a standard deployment
strategy, with no deployment verification enabled, there is one phase
(stable
).
For a canary rollout, there will be a phase for each part of the canary
(for example, canary-25
, canary-50
, stable
), and for each phase there is a
deploy
job. If verification is enabled, there is also a verify
job for each
phase.
A job run is an instance of a job. For example, a job run for a deploy
job is
executed, and if it succeeds, there is no further job run for that job. If it
fails, it can be retried as another job run.
Skipping phases the first time
Some deployment strategies (for example, canary) apportion traffic between the old and new versions. If you're deploying to a target for the first time, there's no old version, so we can't apportion the traffic.
For this reason, when you deploy a canary for the first time, we skip the canary
phase or phases and run the stable
phase. After that, the application is
deployed, and future canary deployments will include the canary phases.
In a real-world situation, you will usually execute a canary deployment where your application is already running, so this phase skipping will be rare.
States within a rollout
Rollouts, phases, jobs, and job runs all have states. This section describes the states for each.
Rollout states
A rollout will have one of the following states:
APPROVAL_REJECTED
The rollout required approval, but the approval was rejected.
CANCELLED
The terminal state for rollouts that have been canceled by a user.
CANCELLING
A user has canceled the rollout, but the cancellation has not finished processing.
HALTED
In a parallel deployment, if one or more child rollouts fail, but at least one child rollout succeeds, the controller rollout is HALTED if there are more phases after the current one.
You can resume a halted controller rollout by doing any of the following:
Cancel the controller rollout
Retry or ignore any failed jobs on child rollouts
IN_PROGRESS
A job run is processing.
FAILED
A job failed, and the user didn't choose to ignore the failure.
PENDING
The rollout has not begun processing. This state transitions to
IN_PROGRESS
orCANCELED
.PENDING_APPROVAL
The rollout requires approval, but has not yet been approved.
PENDING_RELEASE
The rollout is waiting for the release to be rendered.
SUCCEEDED
The rollout has finished, with no failures.
Phase states
A phase will have one of the following states:
PENDING
The phase is waiting for another phase in the rollout to finish.
IN_PROGRESS
The phase has started.
SUCCEEDED
The phase completed successfully.
FAILED
A job in the phase failed, and the user didn't choose to ignore the failure.
ABORTED
A previous phase failed.
SKIPPED
When you're running a deployment strategy, such as a canary, Cloud Deploy skips to the
stable
phase in cases where there isn't yet a running version of the application with which to split traffic. In this case, the state is set toSKIPPED
.
Job states
A job will have one of the following states:
ABORTED
If a phase fails, subsequent phases are aborted.
If a job fails, and that failure is not ignored, subsequent jobs are aborted. For example, if a phase includes a deploy job and a verify job, and the deploy job fails, the verify job is aborted.
DISABLED
Some jobs in a Phase might be disabled. For example, phases always include verify jobs, whether or not verification is enabled. If verification isn't enabled, then the verify job is set to
DISABLED
.FAILED
A job run for this job failed, and the user didn't choose to ignore the failure.
The user chose to terminate the job run for this job.
IGNORED
A job run for this job failed, and the user chose to ignore the failure.
IN_PROGRESS
A job run for this job is currently running.
PENDING
The job run for this job is waiting to begin, because another phase or job hasn't finished.
SKIPPED
When you're running a deployment strategy, such as a canary, Cloud Deploy skips to the
stable
phase in cases where there isn't yet a running version of the application with which to split traffic. In this case, the state is set toSKIPPED
on jobs within the skipped phase or phases.SUCCEEDED
The job run finished successfully, and the next job in the phase has started, or the next phase has started or is ready to start (possibly pending user input), or the rollout has finished.
Job run states
FAILED
The job run failed during execution.
IN_PROGRESS
The job run has begun, but hasn't finished.
TERMINATED
The user terminated the job run.
TERMINATING
The user terminated the job run, but it hasn't finished terminating yet.
SUCCEEDED
When a job run finishes successfully, without failing or being terminated by a user, it's put into a
SUCCEEDED
state, which
Manage your rollout
Using the Google Cloud console or the Google Cloud SDK, you can do the following with a Cloud Deploy rollout:
If you're using parallel deployment with a canary deployment strategy, see this document.
Advance a rollout
For targets configured to use a deployment strategy other than "standard," you need to advance the rollout from phase to phase.
For example, if you have a target configured to perform a simple canary deploy
with 50% and stable
(100%) phases only, you would need to advance the rollout
once, from the canary-50
phase to the stable
(100%) phase.
gcloud
gcloud deploy rollouts advance ROLLOUT_NAME \
--release=RELEASE_NAME \
--delivery-pipeline=PIPELINE_NAME \
--region=REGION
Where:
ROLLOUT_NAME
is the name of the current rollout
which you're advancing to the next phase.
RELEASE_NAME
is the name of the release that this
rollout is part of.
PIPELINE_NAME
is the name of the delivery pipeline
you're using to manage deployment of this release.
REGION
is the name of the region in which the
release was created, for example us-central1
. This is required.
See the Google Cloud SDK reference for more information about the
gcloud deploy rollouts advance
command.
Console
Click your pipeline shown in the list of delivery pipelines.
The Delivery pipeline details page shows a graphical representation of your delivery pipeline's progress.
On the Rollouts tab, under Delivery pipeline details, click the name of your rollout.
The rollout details page is shown, for that rollout.
Notice that in this example, the rollout has a
canary-50
phase and astable
phase. Your rollout might have more phases or different phases.Click Advance rollout.
The rollout is advanced to the next phase.
Cancel a rollout
You can cancel any rollout which hasn't finished. You can also cancel a failed rollout to prevent further actions on it (such as ignore or retry). The rollout must be in one of the following states:
FAILED
HALTED
IN_PROGRESS
PENDING
PENDING_APPROVAL
PENDING_RELEASE
After you cancel a rollout, that rollout is in a CANCELLING
state until all
outstanding job runs have completed. You can terminate
outstanding job runs that you don't want to wait for. Once the rollout is
CANCELLED
, it can no longer be advanced or modified.
To cancel a rollout:
gcloud
gcloud deploy rollouts cancel ROLLOUT_NAME \
--release=RELEASE_NAME \
--delivery-pipeline=PIPELINE_NAME \
--region=REGION
Where:
ROLLOUT_NAME
is the name of the current rollout
which you're advancing to the next phase.
RELEASE_NAME
is the name of the release that this
rollout is part of.
PIPELINE_NAME
is the name of the delivery pipeline
you're using to manage deployment of this release.
REGION
is the name of the region in which the
release was created, for example us-central1
. This is required.
See the Google Cloud SDK reference for more information about the
gcloud deploy rollouts cancel
command.
Console
Click your pipeline shown in the list of delivery pipelines.
The Delivery pipeline details page shows a graphical representation of your delivery pipeline's progress.
On the Rollouts tab, under Delivery pipeline details, click the name of your rollout.
The rollout details page is shown, for that rollout.
Notice that in this example, the rollout has a
canary-50
phase and astable
phase. Your rollout might have more phases or different phases.Click Cancel rollout.
The rollout is canceled.
Terminate a job run
You can end a job run that's currently in progress. You might want to do this,
for example, if a job run appears to be taking too long or not working as
expected. The job run must be IN_PROGRESS
for you to terminate it.
gcloud
gcloud deploy job-runs terminate JOB_RUN_ID \
--release=RELEASE_NAME \
--delivery-pipeline=PIPELINE_NAME \
--rollout=ROLLOUT_NAME \
--region=REGION
Where:
JOB_RUN_ID
is the (UUID) of the job run you want to
terminate. You can find the job run ID in the Google Cloud console, for
Cloud Deploy, on the rollout page:
You can also get the job runs ID using the gcloud deploy rollouts
describe
command.
RELEASE_NAME
is the name of the release that this
job run is part of.
PIPELINE_NAME
is the name of the delivery pipeline
you're using to manage deployment of this release.
ROLLOUT_NAME
is the name of the rollout this job
run is part of.
REGION
is the name of the region in which the
release was created, for example us-central1
. This is required.
See the Google Cloud SDK reference for more information about the
gcloud deploy job-runs terminate
command.
Console
Click your pipeline shown in the list of delivery pipelines.
The Delivery pipeline details page shows a graphical representation of your delivery pipeline's progress.
On the Rollouts tab, under Delivery pipeline details, click the name of your rollout.
The rollout details page is shown, for that rollout.
Notice that in this example, the rollout has a
canary-50
phase and astable
phase. Your rollout might have more phases or different phases.Under Phases, click the phase that includes the job whose job run you're terminating.
Under Job runs select the specific job run you're terminating, then click Terminate.
The job run is terminated, and the job status, as shown in the Phases table, is
Failure
.
After you terminate a job run, the job is considered failed and you can do any of the following:
- Leave it that way and disregard the failed rollout
- Retry the job
- Ignore the job and continue with the next job or phase in the rollout
Ignore a job
You can ignore a failed job and move immediately to the next job in the phase. That job might have failed for any reason, including you or someone else terminated a job run for that job.
A failed job means a failed phase and a failed rollout. However if you ignore
the failure, both the phase and the rollout can be progressed and can ultimately
have SUCCEEDED
states.
gcloud
gcloud deploy rollouts ignore-job ROLLOUT_NAME \
--release=RELEASE_NAME \
--delivery-pipeline=PIPELINE_NAME \
--job-id=JOB_ID \
--phase-id=PHASE_ID \
--region=REGION
Where:
ROLLOUT_NAME
is the name of the rollout this job
run is part of.
RELEASE_NAME
is the name of the current release
that includes this job.
PIPELINE_NAME
is the name of the delivery pipeline
you're using to manage deployment of this release.
JOB_ID
is the name of the job to ignore, for
example DEPLOY
. You can find the job name in the Phases table for the
rollout, in the Google Cloud console:
PHASE_ID
is the name of the phase that includes the
job you're ignoring.
REGION
is the name of the region in which the
release was created, for example us-central1
.
See the Google Cloud SDK reference for more information about the
gcloud deploy rollouts ignore-job
command.
Console
Click your pipeline shown in the list of delivery pipelines.
The Delivery pipeline details page shows a graphical representation of your delivery pipeline's progress.
On the Rollouts tab, under Delivery pipeline details, click the name of your rollout.
The rollout details page is shown, for that rollout.
Select the failed job to ignore.
Click the Ignore failures button.
The failed job run is ignored, and the rollout continues as if the job had succeeded. That is, if there are other jobs in the same phase, they are executed. Otherwise, the rollout is ready to advance to the next phase.
Retry a failed job
You can retry a job run that failed. The job can fail for any of the following reasons:
A job run failed to complete.
For example, there could have been a permissions failure.
A user terminated a job run from that job.
Terminating a job run results in a failed job, which you can retry.
A verification test failed.
For a verification job, a verification test failed. Even though the verification job completed correctly, one of your verification tests failed, and we propagate that back to the verification job. In this case, you would retry the job as part of debugging the failed test against your application.
To retry a failed job:
gcloud
gcloud deploy rollouts retry-job JOB_NAME \
--release=RELEASE_NAME \
--delivery-pipeline=PIPELINE_NAME \
--rollout=ROLLOUT_NAME \
--phase=PHASE_ID \
--region=REGION
Where:
JOB_NAME
is the name of the job that you're
retrying. For example, if you're retrying the verify job after a failed
verification, this would be verify
.
RELEASE_NAME
is the name of the release that this
job run is part of.
PIPELINE_NAME
is the name of the delivery pipeline
you're using to manage deployment of this release.
ROLLOUT_NAME
is the name of the rollout this job
run is part of.
PHASE_ID
is name of the phase that this job is
part of. For example, canary-50
, or stable
.
REGION
is the name of the region in which the
release was created, for example us-central1
. This is required.
See the Google Cloud SDK reference for more information about the
gcloud deploy rollouts retry-job
command.
Console
Click your pipeline shown in the list of delivery pipelines.
The Delivery pipeline details page shows a graphical representation of your delivery pipeline's progress.
On the Rollouts tab, under Delivery pipeline details, click the name of your rollout.
The rollout details page is shown, for that rollout.
Under Phases and Jobs, click the phase that includes the job you're retrying.
Select the job to retry.
Click Retry, and confirm.
The job run is executed again and the job status, as shown in the Phases table, is "in progress". If there are other jobs in the same phase, they are executed. Otherwise, the rollout is ready to advance to the next phase.
What's next
Find out more about how deployment strategies work in Cloud Deploy.
See the Cloud Deploy service architecture documentation for more information about how rollouts, phases, jobs, and job runs fit in with the rest of Cloud Deploy.