Debugging connectivity

Troubleshoot Private Service Connect errors

Unable to connect to the destination database: EOF

Running a connectivity test returns the [DATABASE] unable to connect to the destination database: EOF error message.

Possible cause: The service attachment is configured incorrectly.

Things to try: Make sure that enable_proxy_protocol is set to false in the service attachment Terraform configuration file. Proxy protocol is supported only for HTTP servers such as NGINX and Apache.

When using gcloud to create the Private Service Connect setup, the proxy protocol is disabled by default.

Connection timeout, connection refused

Running the connectivity test fails or times out. This is most likely caused by misconfigured routing within the Private Service Connect setup. There can be several reasons for this.

Possible cause: There's a missing firewall rule that allows the Private Service Connect NAT CIDR range to access the Private Service Connect subnet where the bastion is located, specifically the bastion VM nic0 interface.

Things to try: Make sure your organization policy doesn't restrict internal firewall rules, such as the psc_sp_in_fw firewall rule defined in the example Terraform script for setting up the Private Service Connect producer.

Possible cause: The proxy is down. There is no listener on the supplied port, and therefore the connection hangs.

Things to try: You can try to establish an SSH connection to the bastion VM, and search for the proxy using the following command:

• netstat -tunalp | grep PORT

Analyze the responses to the command:

• If you get an empty response, the proxy is down. Try running the following command:

  sudo su; cd / and check if the Dante server is installed by running sudo dpkg -s dante-server:

  • If the proxy is installed, you get the following message:

    Status: install ok installed

  • If the proxy isn't installed, the probable issue is a missing router. Add a router and check if you can download the proxy by running apt-get install dante-server.

    To learn how to add a router, check how the psc_sp_ex_router and psc_sp_ex_router_nat resources are added in the main.tf file in Configure Private Service Connect.

• If the proxy is running and listening on the supplied port, try opening a connection to it by doing the following:

  1. Install the PostgreSQL client:

     sudo apt-get install postgresql-client.

  2. Connect to the PostgreSQL database:

     psql -h 127.0.0.1 -p PORT -U DBUSERNAME -W (you will be prompted to enter the password).

     Replace the following:

     • PORT: the database port number.
     • DBUSERNAME: the username used to connect to the PostgreSQL database.

  3. Install the telnet client:

     sudo apt-get install telnet

  4. Connect to the telnet client:

     telnet 127.0.0.1 PORT

     Replace PORT with the database port number.

  Depending on the results of the commands:

  • If the commands fail to open a connection, try looking at the proxy logs to locate the root cause. The root cause can vary depending on the AlloyDB for PostgreSQL cluster setup.

  • If the connection is opened using telnet but hangs in the native client, the probable issue is the bastion IP address routing. On your VM, type ip route in the terminal. See if you can locate a routing rule that routes connections to the AlloyDB for PostgreSQL cluster private IP address using the secondary nic (nic1, the DB_SUBNETWORK_GATEWAY IP address).

Possible cause: The service attachment doesn't accept the endpoint connection coming from Database Migration Service. The service attachment holds a list of projects which are accepted, and the Database Migration Service project isn't included in the list.

Things to try: To resolve the issue, try either of the following:

• In the Google Cloud console, go to Private Service Connect.

  Go to Private Service Connect

  On the Published services tab, accept the connection from Database Migration Service for your service attachment (if it's pending).

• Add the requesting project to the allowlisted projects on the service attachment (if it's rejected).

  • For more information about adding an allowlisted project in Terraform, see Terraform documentation.

  • For more information about adding an allowlisted project in gcloud, see gcloud reference documentation.

  If this isn't solving the issue, recreate the connection profile.

• Delete the connection profile that's associated with the private service connect connectivity, and recreate it.