Problem
When you click on Jupyter Notebooks Web interface in Cloud Dataproc cluster, it returns the 504 error in the interfae.
Environment
- Cloud Dataproc cluster
- Component gateway enabled for Jupyter Notebooks
Solution
Check that the Notebooks instance disk is not out of space:
- Connect to the VM instance using SSH.
- Run the following command: df -h -T /home/jupyter
- If the Use% is above 85%, you need to manually delete files from /home/jupyter. As a first step, you can empty the trash with the following command: sudo rm -rf /home/jupyter/.local/share/Trash/*
Additionally, please check the memory utilization in the Notebooks instance:
- Connect to the VM instance using SSH.
- Run the following command: free -t -h
- If the used memory is above 85% of the total, you should consider changing the machine type.
You can install Cloud Monitoring agent to monitor if there is high memory usage in their Notebooks instance. See pricing information.
Cause
This can be due to disk running out of space or heavy memory utilization of the VM instance.