Interface PredictionServiceGrpc.AsyncService (3.56.0)

Version latest

• 3.56.0 (latest)
• 3.55.0
• 3.54.0
• 3.53.0
• 3.52.0
• 3.50.0
• 3.49.0
• 3.48.0
• 3.47.0
• 3.46.0
• 3.45.0
• 3.44.0
• 3.43.0
• 3.42.0
• 3.41.0
• 3.40.0
• 3.38.0
• 3.37.0
• 3.36.0
• 3.35.0
• 3.34.0
• 3.33.0
• 3.32.0
• 3.31.0
• 3.30.0
• 3.29.0
• 3.28.0
• 3.25.0
• 3.24.0
• 3.23.0
• 3.22.0
• 3.21.0
• 3.20.0
• 3.19.0
• 3.18.0
• 3.17.0
• 3.16.0
• 3.15.0
• 3.14.0
• 3.13.0
• 3.12.0
• 3.11.0
• 3.10.0
• 3.9.0
• 3.8.0
• 3.7.0
• 3.6.0
• 3.5.0
• 3.4.2
• 3.3.0
• 3.2.0
• 3.0.0
• 2.9.8
• 2.8.9
• 2.7.4
• 2.5.3
• 2.4.0

public static interface PredictionServiceGrpc.AsyncService

A service for online predictions and explanations.

Methods

chatCompletions(ChatCompletionsRequest request, StreamObserver<HttpBody> responseObserver)

public default void chatCompletions(ChatCompletionsRequest request, StreamObserver<HttpBody> responseObserver)

Exposes an OpenAI-compatible endpoint for chat completions.

Parameters
Name	Description
request	ChatCompletionsRequest
responseObserver	io.grpc.stub.StreamObserver<com.google.api.HttpBody>

countTokens(CountTokensRequest request, StreamObserver<CountTokensResponse> responseObserver)

public default void countTokens(CountTokensRequest request, StreamObserver<CountTokensResponse> responseObserver)

Perform a token counting.

Parameters
Name	Description
request	CountTokensRequest
responseObserver	io.grpc.stub.StreamObserver<CountTokensResponse>

directPredict(DirectPredictRequest request, StreamObserver<DirectPredictResponse> responseObserver)

public default void directPredict(DirectPredictRequest request, StreamObserver<DirectPredictResponse> responseObserver)

Perform an unary online prediction request to a gRPC model server for Vertex first-party products and frameworks.

Parameters
Name	Description
request	DirectPredictRequest
responseObserver	io.grpc.stub.StreamObserver<DirectPredictResponse>

directRawPredict(DirectRawPredictRequest request, StreamObserver<DirectRawPredictResponse> responseObserver)

public default void directRawPredict(DirectRawPredictRequest request, StreamObserver<DirectRawPredictResponse> responseObserver)

Perform an unary online prediction request to a gRPC model server for custom containers.

Parameters
Name	Description
request	DirectRawPredictRequest
responseObserver	io.grpc.stub.StreamObserver<DirectRawPredictResponse>

explain(ExplainRequest request, StreamObserver<ExplainResponse> responseObserver)

public default void explain(ExplainRequest request, StreamObserver<ExplainResponse> responseObserver)

Perform an online explanation. If deployed_model_id is specified, the corresponding DeployModel must have explanation_spec populated. If deployed_model_id is not specified, all DeployedModels must have explanation_spec populated.

Parameters
Name	Description
request	ExplainRequest
responseObserver	io.grpc.stub.StreamObserver<ExplainResponse>

generateContent(GenerateContentRequest request, StreamObserver<GenerateContentResponse> responseObserver)

public default void generateContent(GenerateContentRequest request, StreamObserver<GenerateContentResponse> responseObserver)

Generate content with multimodal inputs.

Parameters
Name	Description
request	GenerateContentRequest
responseObserver	io.grpc.stub.StreamObserver<GenerateContentResponse>

predict(PredictRequest request, StreamObserver<PredictResponse> responseObserver)

public default void predict(PredictRequest request, StreamObserver<PredictResponse> responseObserver)

Perform an online prediction.

Parameters
Name	Description
request	PredictRequest
responseObserver	io.grpc.stub.StreamObserver<PredictResponse>

rawPredict(RawPredictRequest request, StreamObserver<HttpBody> responseObserver)

public default void rawPredict(RawPredictRequest request, StreamObserver<HttpBody> responseObserver)

Perform an online prediction with an arbitrary HTTP payload. The response includes the following HTTP headers:

• X-Vertex-AI-Endpoint-Id: ID of the Endpoint that served this prediction.
• X-Vertex-AI-Deployed-Model-Id: ID of the Endpoint's DeployedModel that served this prediction.

Parameters
Name	Description
request	RawPredictRequest
responseObserver	io.grpc.stub.StreamObserver<com.google.api.HttpBody>

serverStreamingPredict(StreamingPredictRequest request, StreamObserver<StreamingPredictResponse> responseObserver)

public default void serverStreamingPredict(StreamingPredictRequest request, StreamObserver<StreamingPredictResponse> responseObserver)

Perform a server-side streaming online prediction request for Vertex LLM streaming.

Parameters
Name	Description
request	StreamingPredictRequest
responseObserver	io.grpc.stub.StreamObserver<StreamingPredictResponse>

streamDirectPredict(StreamObserver<StreamDirectPredictResponse> responseObserver)

public default StreamObserver<StreamDirectPredictRequest> streamDirectPredict(StreamObserver<StreamDirectPredictResponse> responseObserver)

Perform a streaming online prediction request to a gRPC model server for Vertex first-party products and frameworks.

Parameter
Name	Description
responseObserver	io.grpc.stub.StreamObserver<StreamDirectPredictResponse>

Returns
Type	Description
io.grpc.stub.StreamObserver<StreamDirectPredictRequest>

streamDirectRawPredict(StreamObserver<StreamDirectRawPredictResponse> responseObserver)

public default StreamObserver<StreamDirectRawPredictRequest> streamDirectRawPredict(StreamObserver<StreamDirectRawPredictResponse> responseObserver)

Perform a streaming online prediction request to a gRPC model server for custom containers.

Parameter
Name	Description
responseObserver	io.grpc.stub.StreamObserver<StreamDirectRawPredictResponse>

Returns
Type	Description
io.grpc.stub.StreamObserver<StreamDirectRawPredictRequest>

streamGenerateContent(GenerateContentRequest request, StreamObserver<GenerateContentResponse> responseObserver)

public default void streamGenerateContent(GenerateContentRequest request, StreamObserver<GenerateContentResponse> responseObserver)

Generate content with multimodal inputs with streaming support.

Parameters
Name	Description
request	GenerateContentRequest
responseObserver	io.grpc.stub.StreamObserver<GenerateContentResponse>

streamRawPredict(StreamRawPredictRequest request, StreamObserver<HttpBody> responseObserver)

public default void streamRawPredict(StreamRawPredictRequest request, StreamObserver<HttpBody> responseObserver)

Perform a streaming online prediction with an arbitrary HTTP payload.

Parameters
Name	Description
request	StreamRawPredictRequest
responseObserver	io.grpc.stub.StreamObserver<com.google.api.HttpBody>

streamingPredict(StreamObserver<StreamingPredictResponse> responseObserver)

public default StreamObserver<StreamingPredictRequest> streamingPredict(StreamObserver<StreamingPredictResponse> responseObserver)

Perform a streaming online prediction request for Vertex first-party products and frameworks.

Parameter
Name	Description
responseObserver	io.grpc.stub.StreamObserver<StreamingPredictResponse>

Returns
Type	Description
io.grpc.stub.StreamObserver<StreamingPredictRequest>

streamingRawPredict(StreamObserver<StreamingRawPredictResponse> responseObserver)

public default StreamObserver<StreamingRawPredictRequest> streamingRawPredict(StreamObserver<StreamingRawPredictResponse> responseObserver)

Perform a streaming online prediction request through gRPC.

Parameter
Name	Description
responseObserver	io.grpc.stub.StreamObserver<StreamingRawPredictResponse>

Returns
Type	Description
io.grpc.stub.StreamObserver<StreamingRawPredictRequest>