EXAMPLE - Domain Functions

This examples illustrates how you can extract component parts of a URL using the following functions:

  • DOMAIN - extracts the domain value from a URL. See DOMAIN Function.
  • SUBDOMAIN - extracts the first group after the protocol identifier and before the domain value. See SUBDOMAIN Function.
  • SUFFIX - extracts the suffix of a URL. See SUFFIX Function.

Source:

Your dataset includes the following values for URLs:

URL
www.example.com
example.com/support
http://www.example.com/products/
http://1.2.3.4
https://www.example.com/free-download
https://www.example.com/about-us/careers
www.app.example.com
www.some.app.example.com
some.app.example.com
some.example.com
example.com

Transform:

When the above data is imported into the application, the column is recognized as a URL. All values are registered as valid, even the IPv4 address.

To extract the domain and subdomain values:

derive value: DOMAIN(URL) as: 'domain_URL'

derive value: SUBDOMAIN(URL) as:'subdomain_URL'

derive value: SUFFIX(URL) as:'suffix_URL'

You can use the Cloud Dataprep pattern in the following transform to extract protocol identifiers, if present, into a new column:

extract col:URL on:`{start}%*://`

To clean this up, you might want to rename the column to protocol_URL.

To extract the path values, you can use the following regular expression:

NOTE: Regular expressions are considered a developer-level method for pattern matching. Please use them with caution. See Text Matching.

extract col: URL on: /[^*:\/\/]\/.*$/

The above transform grabs a little too much of the URL. If you rename the column to path_URL, you can use the following regular expression to clean it up:

extract col:path_URL on:/[!^\/].*$/


If you drop the path_URL column and rename the path_URL1 column to the dropped one, you should have the following results:

Results:

URLpath_URLprotocol_URLsubdomain_URLdomain_URLsuffix_URL
www.example.com wwwexamplecom
example.com/support/support examplecom
http://www.example.com/products//products/http://wwwexamplecom
http://1.2.3.4 http://
https://www.example.com/free-download/free-downloadhttps://wwwexamplecom
https://www.example.com/about-us/careers/about-us/careershttps://wwwexamplecom
www.app.example.com www.appexamplecom
www.some.app.example.com www.some.appexamplecom
some.app.example.com some.appexamplecom
some.example.com someexamplecom
example.com examplecom
Was this page helpful? Let us know how we did:

Send feedback about...

Google Cloud Dataprep Documentation