Troubleshoot deploying an application

This document shows you how to resolve errors that you might encounter when deploying an application.

Prebuilt Template errors

If you run into issues with the LangchainAgent template during deployment, it might be due to one of the issues in this section.

Model Builder or Runnable Builder errors

Issue:

You receive an error message similar to the following:

AttributeError: 'LangchainAgent' object has no attribute '_model_builder'

AttributeError: 'LangchainAgent' object has no attribute '_runnable_builder'

Possible cause:

This might happen if you used a version of google-cloud-aiplatform that is later than 1.50.0 to define your LangchainAgent in your development environment, but are deploying it with a version that is earlier than 1.51.0 in requirements=. To check which version you're using in your development environment, run the following command in your terminal:

pip show google-cloud-aiplatform

Recommended solution:

If you are using a version of google-cloud-aiplatform before version 1.51.0, specify a version of google-cloud-aiplatform before version 1.51.0 in requirements= when deploying the reasoning engine.

If you are using a version of google-cloud-aiplatform after version 1.50.0, specify a version of google-cloud-aiplatform after version 1.50.0 in requirements= when deploying the reasoning engine.

Internal server errors

Issue:

You receive an error message similar to the following:

InternalServerError: 500 Revision XXX is not ready and cannot serve traffic.

Unfortunately, this is a catch-all error for any issues with the container at runtime, and the possible cause is one of many errors that might be happening.

Possible cause(s):

Dirty state on LangchainAgent. This might happen if .set_up() was called on a LangchainAgent before deploying to reasoning engine.
Inconsistent package versions. This might happen if the packages installed in the development environment are different from the packages installed in the remote environment in reasoning engine.

Recommended solution(s):

Dirty state on LangchainAgent. Instantiate a fresh instance of the LangchainAgent or remove agent.set_up() from the code before deploying to reasoning engine.
Inconsistent package specs. See the section on troubleshooting serialization errors.

Serialization errors

In general, it is important to ensure that the "local" and "remote" environments are in sync when deploying to reasoning engine. You can ensure this by specifying requirements= when deploying to reasoning engine.

If you run into issues with serialization (errors related to "pickle" or "pickling" are synonymous with "serialization" errors), it might be due to one of the issues that's described in this section.

Pydantic version

Issue:

You receive an error message similar to the following:

PicklingError: Can't pickle <cyfunction str_validator at 0x7ca030133d30>: it's
not the same object as pydantic.validators.str_validator

Possible cause:

This might happen if your pydantic package is earlier than version 2.6.4. To check which version you're using, run the following command in your terminal:

pip show pydantic

Recommended solution:

Update your package by running the following command in the terminal:

pip install pydantic --upgrade

Run the following command in your terminal to verify you're using version 2.6.4 or later:

pip show pydantic

If you're in a notebook instance (for example, a Jupyter or Colab or Workbench), you might need to restart your runtime to use the updated packages.

Cloudpickle version

Issue:

You receive an error message similar to the following:

AttributeError: Can't get attribute '_class_setstate' on <module 'cloudpickle.cloudpickle'
from '/usr/local/lib/python3.10/site-packages/cloudpickle/cloudpickle.py'>

Possible cause:

This might happen if the version of your cloudpickle package is different in your development environment and your deployment environment. To check which version you're using in development, run the following command in your terminal:

pip show cloudpickle

Recommended solution:

Deploy the same version of cloudpickle in both environments (i.e. your local development environment and the remotely deployed Reasoning Engine environment) by specifying requirements= when deploying the reasoning engine.

Internal server errors

Issue:

You receive an error message similar to the following:

InternalServerError: 500 Revision XXX is not ready and cannot serve traffic.

Possible cause:

This might happen if sys_version= is different from the development environment when deploying to reasoning engine.

Recommended solution:

Consider removing sys_version= from the input arguments when deploying to reasoning engine. If you still run into issues, file a bug report.

Cloud Storage bucket errors

If you're running into issues with the Cloud Storage staging bucket that's used at deployment time to collect and upload your agent, it might be due to one of the following issues:

Permission errors

Recommended solution:

If you want to use a pre-existing bucket: make sure the principal that's authenticated to use Vertex AI (either yourself or a service account) has Storage Admin access to the bucket, and grant permissions to the Google-managed reasoning engine service account.

Alternatively, you can specify a new bucket when deploying to reasoning engine and the reasoning engine client SDK will create the bucket with the necessary permissions.

If you still run into issues, file a bug report.

Cloud Storage bucket subdirectory isn't created

Issue:

You receive an error message similar to the following:

NotFound: 404 Can not copy from \"gs://[LOCATION]-*/reasoning_engine/reasoning_engine.pkl\" to \"gs://*/code.pkl\", check if the source object and target bucket exist.

(The 404 occurs when the system tries to copy into a folder that doesn't exist.)

Possible cause:

This is likely due to an issue with string interpolation in versions of google-cloud-aiplatform earlier than version 1.49.0. This is fixed) in later versions. To check which version of google-cloud-aiplatform you're using, run the following command in your terminal:

pip show google-cloud-aiplatform

Recommended solution:

Update your package by running the following command in the terminal:

pip install google-cloud-aiplatform --upgrade

Verify that you're using version 1.49.0 or later of google-cloud-aiplatform by running the following command in your terminal:

pip show google-cloud-aiplatform

If you're using a notebook instance (for example, Jupyter or Colab or Workbench), you might need to restart your runtime before you can use the updated packages.