Try Gemini 1.5 models, the latest multimodal models in Vertex AI, and see what you can build with up to a 2M token context window. Try Gemini 1.5 models, the latest multimodal models in Vertex AI, and see what you can build with up to a 2M token context window.

Interface PredictionServiceGrpc.AsyncService (0.3.0)

public static interface PredictionServiceGrpc.AsyncService

A service for online predictions and explanations.

Methods

directPredict(DirectPredictRequest request, StreamObserver<DirectPredictResponse> responseObserver)

public default void directPredict(DirectPredictRequest request, StreamObserver<DirectPredictResponse> responseObserver)

Perform an unary online prediction request for Vertex first-party products and frameworks.

Parameters
Name	Description
`request`	`DirectPredictRequest`
`responseObserver`	`io.grpc.stub.StreamObserver<DirectPredictResponse>`

directRawPredict(DirectRawPredictRequest request, StreamObserver<DirectRawPredictResponse> responseObserver)

public default void directRawPredict(DirectRawPredictRequest request, StreamObserver<DirectRawPredictResponse> responseObserver)

Perform an online prediction request through gRPC.

Parameters
Name	Description
`request`	`DirectRawPredictRequest`
`responseObserver`	`io.grpc.stub.StreamObserver<DirectRawPredictResponse>`

explain(ExplainRequest request, StreamObserver<ExplainResponse> responseObserver)

public default void explain(ExplainRequest request, StreamObserver<ExplainResponse> responseObserver)

Perform an online explanation. If deployed_model_id is specified, the corresponding DeployModel must have explanation_spec populated. If deployed_model_id is not specified, all DeployedModels must have explanation_spec populated.

Parameters
Name	Description
`request`	`ExplainRequest`
`responseObserver`	`io.grpc.stub.StreamObserver<ExplainResponse>`

predict(PredictRequest request, StreamObserver<PredictResponse> responseObserver)

public default void predict(PredictRequest request, StreamObserver<PredictResponse> responseObserver)

Perform an online prediction.

Parameters
Name	Description
`request`	`PredictRequest`
`responseObserver`	`io.grpc.stub.StreamObserver<PredictResponse>`

rawPredict(RawPredictRequest request, StreamObserver<HttpBody> responseObserver)

public default void rawPredict(RawPredictRequest request, StreamObserver<HttpBody> responseObserver)

Perform an online prediction with an arbitrary HTTP payload. The response includes the following HTTP headers:

X-Vertex-AI-Endpoint-Id: ID of the Endpoint that served this prediction.
X-Vertex-AI-Deployed-Model-Id: ID of the Endpoint's DeployedModel that served this prediction.

Parameters
Name	Description
`request`	`RawPredictRequest`
`responseObserver`	`io.grpc.stub.StreamObserver<com.google.api.HttpBody>`

serverStreamingPredict(StreamingPredictRequest request, StreamObserver<StreamingPredictResponse> responseObserver)

public default void serverStreamingPredict(StreamingPredictRequest request, StreamObserver<StreamingPredictResponse> responseObserver)

Perform a server-side streaming online prediction request for Vertex LLM streaming.

Parameters
Name	Description
`request`	`StreamingPredictRequest`
`responseObserver`	`io.grpc.stub.StreamObserver<StreamingPredictResponse>`

streamGenerateContent(GenerateContentRequest request, StreamObserver<GenerateContentResponse> responseObserver)

public default void streamGenerateContent(GenerateContentRequest request, StreamObserver<GenerateContentResponse> responseObserver)

Generate content with multimodal inputs with streaming support.

Parameters
Name	Description
`request`	`GenerateContentRequest`
`responseObserver`	`io.grpc.stub.StreamObserver<GenerateContentResponse>`

streamingPredict(StreamObserver<StreamingPredictResponse> responseObserver)

public default StreamObserver<StreamingPredictRequest> streamingPredict(StreamObserver<StreamingPredictResponse> responseObserver)

Perform a streaming online prediction request for Vertex first-party products and frameworks.

Parameter
Name	Description
`responseObserver`	`io.grpc.stub.StreamObserver<StreamingPredictResponse>`

Returns
Type	Description
`io.grpc.stub.StreamObserver<StreamingPredictRequest>`

streamingRawPredict(StreamObserver<StreamingRawPredictResponse> responseObserver)

public default StreamObserver<StreamingRawPredictRequest> streamingRawPredict(StreamObserver<StreamingRawPredictResponse> responseObserver)

Perform a streaming online prediction request through gRPC.

Parameter
Name	Description
`responseObserver`	`io.grpc.stub.StreamObserver<StreamingRawPredictResponse>`

Returns
Type	Description
`io.grpc.stub.StreamObserver<StreamingRawPredictRequest>`