Dataproc Autoscaling cluster scales up but not down
Stay organized with collections
Save and categorize content based on your preferences.
Problem
The Datatroc Autoscaling policy is scaling the cluster up and not scaling it down. The logs do not have error messages, and the autoscale logs have the message: decision: NO_SCALE.
Environment
- Cloud Dataproc Cluster with Autoscaling policy
Solution
- Check the Autoscaling policy.
- If the scaleDownFactor value is not 1, set it to 1 to aggressively scale down the cluster.
Cause
Due to graceful decommissioning, scale down operations are significantly slower than scaling up. Setting scaleDownFactor=1.0 configures aggressive scale down, which minimizes the number of down scaling operations to achieve the appropriate cluster size.
Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 4.0 License, and code samples are licensed under the Apache 2.0 License. For details, see the Google Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.
Last updated 2024-12-12 UTC.
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Hard to understand","hardToUnderstand","thumb-down"],["Incorrect information or sample code","incorrectInformationOrSampleCode","thumb-down"],["Missing the information/samples I need","missingTheInformationSamplesINeed","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2024-12-12 UTC."],[],[]]
