Basic autoscaling configurations for YARN.
Inherits
- Object
Extended By
- Google::Protobuf::MessageExts::ClassMethods
Includes
- Google::Protobuf::MessageExts
Methods
#graceful_decommission_timeout
def graceful_decommission_timeout() -> ::Google::Protobuf::Duration
Returns
-
(::Google::Protobuf::Duration) — Required. Timeout for YARN graceful decommissioning of Node Managers.
Specifies the duration to wait for jobs to complete before forcefully
removing workers (and potentially interrupting jobs). Only applicable to
downscaling operations.
Bounds: [0s, 1d].
#graceful_decommission_timeout=
def graceful_decommission_timeout=(value) -> ::Google::Protobuf::Duration
Parameter
-
value (::Google::Protobuf::Duration) — Required. Timeout for YARN graceful decommissioning of Node Managers.
Specifies the duration to wait for jobs to complete before forcefully
removing workers (and potentially interrupting jobs). Only applicable to
downscaling operations.
Bounds: [0s, 1d].
Returns
-
(::Google::Protobuf::Duration) — Required. Timeout for YARN graceful decommissioning of Node Managers.
Specifies the duration to wait for jobs to complete before forcefully
removing workers (and potentially interrupting jobs). Only applicable to
downscaling operations.
Bounds: [0s, 1d].
#scale_down_factor
def scale_down_factor() -> ::Float
Returns
-
(::Float) — Required. Fraction of average YARN pending memory in the last cooldown period
for which to remove workers. A scale-down factor of 1 will result in
scaling down so that there is no available memory remaining after the
update (more aggressive scaling). A scale-down factor of 0 disables
removing workers, which can be beneficial for autoscaling a single job.
See How autoscaling
works
for more information.
Bounds: [0.0, 1.0].
#scale_down_factor=
def scale_down_factor=(value) -> ::Float
Parameter
-
value (::Float) — Required. Fraction of average YARN pending memory in the last cooldown period
for which to remove workers. A scale-down factor of 1 will result in
scaling down so that there is no available memory remaining after the
update (more aggressive scaling). A scale-down factor of 0 disables
removing workers, which can be beneficial for autoscaling a single job.
See How autoscaling
works
for more information.
Bounds: [0.0, 1.0].
Returns
-
(::Float) — Required. Fraction of average YARN pending memory in the last cooldown period
for which to remove workers. A scale-down factor of 1 will result in
scaling down so that there is no available memory remaining after the
update (more aggressive scaling). A scale-down factor of 0 disables
removing workers, which can be beneficial for autoscaling a single job.
See How autoscaling
works
for more information.
Bounds: [0.0, 1.0].
#scale_down_min_worker_fraction
def scale_down_min_worker_fraction() -> ::Float
Returns
-
(::Float) — Optional. Minimum scale-down threshold as a fraction of total cluster size
before scaling occurs. For example, in a 20-worker cluster, a threshold of
0.1 means the autoscaler must recommend at least a 2 worker scale-down for
the cluster to scale. A threshold of 0 means the autoscaler will scale down
on any recommended change.
Bounds: [0.0, 1.0]. Default: 0.0.
#scale_down_min_worker_fraction=
def scale_down_min_worker_fraction=(value) -> ::Float
Parameter
-
value (::Float) — Optional. Minimum scale-down threshold as a fraction of total cluster size
before scaling occurs. For example, in a 20-worker cluster, a threshold of
0.1 means the autoscaler must recommend at least a 2 worker scale-down for
the cluster to scale. A threshold of 0 means the autoscaler will scale down
on any recommended change.
Bounds: [0.0, 1.0]. Default: 0.0.
Returns
-
(::Float) — Optional. Minimum scale-down threshold as a fraction of total cluster size
before scaling occurs. For example, in a 20-worker cluster, a threshold of
0.1 means the autoscaler must recommend at least a 2 worker scale-down for
the cluster to scale. A threshold of 0 means the autoscaler will scale down
on any recommended change.
Bounds: [0.0, 1.0]. Default: 0.0.
#scale_up_factor
def scale_up_factor() -> ::Float
Returns
-
(::Float) — Required. Fraction of average YARN pending memory in the last cooldown period
for which to add workers. A scale-up factor of 1.0 will result in scaling
up so that there is no pending memory remaining after the update (more
aggressive scaling). A scale-up factor closer to 0 will result in a smaller
magnitude of scaling up (less aggressive scaling).
See How autoscaling
works
for more information.
Bounds: [0.0, 1.0].
#scale_up_factor=
def scale_up_factor=(value) -> ::Float
Parameter
-
value (::Float) — Required. Fraction of average YARN pending memory in the last cooldown period
for which to add workers. A scale-up factor of 1.0 will result in scaling
up so that there is no pending memory remaining after the update (more
aggressive scaling). A scale-up factor closer to 0 will result in a smaller
magnitude of scaling up (less aggressive scaling).
See How autoscaling
works
for more information.
Bounds: [0.0, 1.0].
Returns
-
(::Float) — Required. Fraction of average YARN pending memory in the last cooldown period
for which to add workers. A scale-up factor of 1.0 will result in scaling
up so that there is no pending memory remaining after the update (more
aggressive scaling). A scale-up factor closer to 0 will result in a smaller
magnitude of scaling up (less aggressive scaling).
See How autoscaling
works
for more information.
Bounds: [0.0, 1.0].
#scale_up_min_worker_fraction
def scale_up_min_worker_fraction() -> ::Float
Returns
-
(::Float) — Optional. Minimum scale-up threshold as a fraction of total cluster size
before scaling occurs. For example, in a 20-worker cluster, a threshold of
0.1 means the autoscaler must recommend at least a 2-worker scale-up for
the cluster to scale. A threshold of 0 means the autoscaler will scale up
on any recommended change.
Bounds: [0.0, 1.0]. Default: 0.0.
#scale_up_min_worker_fraction=
def scale_up_min_worker_fraction=(value) -> ::Float
Parameter
-
value (::Float) — Optional. Minimum scale-up threshold as a fraction of total cluster size
before scaling occurs. For example, in a 20-worker cluster, a threshold of
0.1 means the autoscaler must recommend at least a 2-worker scale-up for
the cluster to scale. A threshold of 0 means the autoscaler will scale up
on any recommended change.
Bounds: [0.0, 1.0]. Default: 0.0.
Returns
-
(::Float) — Optional. Minimum scale-up threshold as a fraction of total cluster size
before scaling occurs. For example, in a 20-worker cluster, a threshold of
0.1 means the autoscaler must recommend at least a 2-worker scale-up for
the cluster to scale. A threshold of 0 means the autoscaler will scale up
on any recommended change.
Bounds: [0.0, 1.0]. Default: 0.0.