Error loading list of jobs in AI Platform console page

Problem

You are not able to see the list of jobs in any of the AI Platform Jobs console page. Although the jobs are visible through the Rest API and through the gcloud commands.

Environment

  • AI Platform

Solution

Modify a PUT request from the Models page (/ai-platform/models) and modify the path to clear the filters from the jobs page (/ai-platform/jobs).

  1. Using the developer tools from the browser, open the Network tab.
  2. Visit the /ai-platform/models page and add a filter using the console, any arbitrary filter will work.
  3. Look for AiPlatformModelTable in the network logs.
  4. Find the PUT request, right click, and click Copy as cURL. This will copy the information into the clipboard.
  5. Paste the cURL command into a text editor, replace AiPlatformModelTable with AiPlatformJobTable and after the --data-binary flag, remove everything between the %5B and %5D. Example:
    --data-binary flag: --data-binary '{"value":{"protoVal{"@type":"type.googleapis.com/google.protobuf.Struct","value":{"f":"%5B%5D"}}}}'
  6.  Execute cURL command. Upon successful POST, reload jobs page. Filters should be cleared and page should load properly.

Cause

The console page can timeout, returns an HTTP error 403, under certain conditions. The default timeout is 60 seconds which can be reached when a page size and a filter for job state is applied. It is possible that the filter was stuck in the GET response call of the console.

GET /v1/projects/${PROJECT_NAME}/jobs?filter=(%20state=RUNNING)&pageSize=50