EXAMPLE - ROLLINGKTHLARGEST Functions

This example describes how to use the following rolling computational functions:

  • ROLLINGKTHLARGEST - computes the kth largest value from a rolling window of rows before and after the current row. Duplicate values are treated as having the same k values. See ROLLINGKTHLARGEST Function.
  • ROLLINGKTHLARGESTUNIQUE - computes the unique kth largest value from a rolling window of rows before and after the current row. Duplicate values are treated as having different k values. See ROLLINGKTHLARGESTUNIQUE Function.

The following dataset contains daily counts of server restarts across three servers over the preceding week. High server restart counts can indicate poor server health. In this example, you are interested in knowing for each server the rolling highest and second highest count of restarts per server over the previous week.

Source:

DateServerRestarts
2/21/18s014
2/21/18s020
2/21/18s030
2/22/18s014
2/22/18s021
2/22/18s032
2/23/18s012
2/23/18s023
2/23/18s034
2/24/18s011
2/24/18s020
2/24/18s032
2/25/18s015
2/25/18s020
2/25/18s034
2/26/18s011
2/26/18s022
2/26/18s031
2/27/18s011
2/27/18s022
2/27/18s032


Transform:

First, you want to maintain the row information as a separate column. Since data is ordered already by the Date column, you can use the following:

derive type:single value:ROWNUMBER() as:'entryId'

Use the following function to compute the rolling kth largest value of server restarts per server over the previous week. In this case, you can use the ROLLINGKTHLARGEST function, setting k=1. Uniqueness doesn't matter for calculating the highest value:

derive type: multiple value: rollingkthlargest(Restarts, 1, 6, 0) group: Server order: Server as: 'rollingkthlargest_1'

Use the following function to compute the rolling second highest value. In this case, you can use ROLLINGKTHLARGESTUNIQUE:

derive type: multiple value: rollingkthlargestunique(Restarts, 2, 6, 0) group: Server order: Server as: 'rollingKthLargestUnique_2'


Results:

entryIdDateServerRestartsrollingKthLargestUnique_2rollingkthlargest_Restarts
32/21/18s02000
62/22/18s02101
92/23/18s02313
122/24/18s02013
152/25/18s02013
182/26/18s02223
212/27/18s02223
42/21/18s03000
72/22/18s03202
102/23/18s03424
132/24/18s03224
162/25/18s03424
192/26/18s03124
222/27/18s03224
22/21/18s01444
52/22/18s01444
82/23/18s01224
112/24/18s01124
142/25/18s01545
172/26/18s01145
202/27/18s01145


Was this page helpful? Let us know how we did:

Send feedback about...

Google Cloud Dataprep Documentation