ResourceLimiter (Cloud Bigtable HBase Client for Java 0.9.7.1 API)

com.google.cloud.bigtable.grpc.async

Class ResourceLimiter



  • public class ResourceLimiter
    extends Object
    This class limits access by RPCs to system resources
    Version:
    $Id: $Id
    Author:
    sduskis
    • Constructor Detail

      • ResourceLimiter

        public ResourceLimiter(long maxHeapSize,
                               int maxInFlightRpcs)

        Constructor for ResourceLimiter.

        Parameters:
        maxHeapSize - a long.
        maxInFlightRpcs - a int.
    • Method Detail

      • registerOperationWithHeapSize

        public long registerOperationWithHeapSize(long heapSize)
                                           throws InterruptedException
        Register an operation with the given size before sending. This call WILL BLOCK until resources are available. This method must be paired with a call to markCanBeCompleted in order to make sure resources are properly released.
        Parameters:
        heapSize - The serialized size of the RPC to be sent
        Returns:
        A unique operation id
        Throws:
        InterruptedException - if any.
      • markCanBeCompleted

        public void markCanBeCompleted(long id)
        Mark an operation id, as returned by registerOperationWithHeapSize, as complete
        Parameters:
        id - a long.
      • getMaxHeapSize

        public long getMaxHeapSize()

        Getter for the field maxHeapSize.

        Returns:
        The maximum allowed number of bytes across all across all outstanding RPCs
      • getAbsoluteMaxInFlightRpcs

        public int getAbsoluteMaxInFlightRpcs()

        Getter for the field maxInFlightRpcs.

        Returns:
        The maximum allowed number of in-flight RPCs
      • getCurrentInFlightMaxRpcs

        public int getCurrentInFlightMaxRpcs()

        Getter for the field currentInFlightMaxRpcs.

        Returns:
        The current maximum number of allowed in-flight RPCs
      • setCurrentInFlightMaxRpcs

        public void setCurrentInFlightMaxRpcs(int currentInFlightMaxRpcs)

        Setter for the field currentInFlightMaxRpcs.

      • getHeapSize

        public long getHeapSize()

        getHeapSize.

        Returns:
        The total size of all currently outstanding RPCs
      • isFull

        public boolean isFull()

        isFull.

        Returns:
        true if no more RPCs can be started, false otherwise
      • hasInflightRequests

        public boolean hasInflightRequests()

        hasInflightRequests.

        Returns:
        true if there are currently in-flight RPCs
      • throttle

        public void throttle(int bulkMutationRpcTargetMs)
        Enable an experimental feature that will throttle requests made from BulkMutation. The logic is as follows:

        • To start:
          • reduce parallelism to 25% -- The parallelism is high to begin with. This reduction should reduce the impacts of a bursty job, such as those found in Dataflow.
        • every 20 seconds:
             if (rpc_latency > threshold) {
                decrease parallelism by 10% of original maximum.
             } else if (rpc_latency < threshold && rpcsWereThrottled()) {
                increase parallelism by 5% of original maximum.
             }
           
          NOTE: increases are capped by the initial maximum. Decreases are floored at 1% of the original maximum so that there is some level of throughput.
        Parameters:
        bulkMutationRpcTargetMs - the target for latency of MutateRows requests in milliseconds.


Monitor your resources on the go

Get the Google Cloud Console app to help you manage your projects.

Send feedback about...

Cloud Bigtable Documentation