Cloud AI Platform v1 API - Class PredictionService.PredictionServiceBase (2.23.0)

[BindServiceMethod(typeof(PredictionService), "BindService")]
public abstract class PredictionService.PredictionServiceBase

Reference documentation and code samples for the Cloud AI Platform v1 API class PredictionService.PredictionServiceBase.

Base class for server-side implementations of PredictionService

Inheritance

object > PredictionService.PredictionServiceBase

Namespace

Google.Cloud.AIPlatform.V1

Assembly

Google.Cloud.AIPlatform.V1.dll

Methods

DirectPredict(DirectPredictRequest, ServerCallContext)

public virtual Task<DirectPredictResponse> DirectPredict(DirectPredictRequest request, ServerCallContext context)

Perform an unary online prediction request to a gRPC model server for Vertex first-party products and frameworks.

Parameters
NameDescription
requestDirectPredictRequest

The request received from the client.

contextServerCallContext

The context of the server-side call handler being invoked.

Returns
TypeDescription
TaskDirectPredictResponse

The response to send back to the client (wrapped by a task).

DirectRawPredict(DirectRawPredictRequest, ServerCallContext)

public virtual Task<DirectRawPredictResponse> DirectRawPredict(DirectRawPredictRequest request, ServerCallContext context)

Perform an unary online prediction request to a gRPC model server for custom containers.

Parameters
NameDescription
requestDirectRawPredictRequest

The request received from the client.

contextServerCallContext

The context of the server-side call handler being invoked.

Returns
TypeDescription
TaskDirectRawPredictResponse

The response to send back to the client (wrapped by a task).

Explain(ExplainRequest, ServerCallContext)

public virtual Task<ExplainResponse> Explain(ExplainRequest request, ServerCallContext context)

Perform an online explanation.

If [deployed_model_id][google.cloud.aiplatform.v1.ExplainRequest.deployed_model_id] is specified, the corresponding DeployModel must have [explanation_spec][google.cloud.aiplatform.v1.DeployedModel.explanation_spec] populated. If [deployed_model_id][google.cloud.aiplatform.v1.ExplainRequest.deployed_model_id] is not specified, all DeployedModels must have [explanation_spec][google.cloud.aiplatform.v1.DeployedModel.explanation_spec] populated.

Parameters
NameDescription
requestExplainRequest

The request received from the client.

contextServerCallContext

The context of the server-side call handler being invoked.

Returns
TypeDescription
TaskExplainResponse

The response to send back to the client (wrapped by a task).

GenerateContent(GenerateContentRequest, ServerCallContext)

public virtual Task<GenerateContentResponse> GenerateContent(GenerateContentRequest request, ServerCallContext context)

Generate content with multimodal inputs.

Parameters
NameDescription
requestGenerateContentRequest

The request received from the client.

contextServerCallContext

The context of the server-side call handler being invoked.

Returns
TypeDescription
TaskGenerateContentResponse

The response to send back to the client (wrapped by a task).

Predict(PredictRequest, ServerCallContext)

public virtual Task<PredictResponse> Predict(PredictRequest request, ServerCallContext context)

Perform an online prediction.

Parameters
NameDescription
requestPredictRequest

The request received from the client.

contextServerCallContext

The context of the server-side call handler being invoked.

Returns
TypeDescription
TaskPredictResponse

The response to send back to the client (wrapped by a task).

RawPredict(RawPredictRequest, ServerCallContext)

public virtual Task<HttpBody> RawPredict(RawPredictRequest request, ServerCallContext context)

Perform an online prediction with an arbitrary HTTP payload.

The response includes the following HTTP headers:

  • X-Vertex-AI-Endpoint-Id: ID of the [Endpoint][google.cloud.aiplatform.v1.Endpoint] that served this prediction.

  • X-Vertex-AI-Deployed-Model-Id: ID of the Endpoint's [DeployedModel][google.cloud.aiplatform.v1.DeployedModel] that served this prediction.

Parameters
NameDescription
requestRawPredictRequest

The request received from the client.

contextServerCallContext

The context of the server-side call handler being invoked.

Returns
TypeDescription
TaskHttpBody

The response to send back to the client (wrapped by a task).

ServerStreamingPredict(StreamingPredictRequest, IServerStreamWriter<StreamingPredictResponse>, ServerCallContext)

public virtual Task ServerStreamingPredict(StreamingPredictRequest request, IServerStreamWriter<StreamingPredictResponse> responseStream, ServerCallContext context)

Perform a server-side streaming online prediction request for Vertex LLM streaming.

Parameters
NameDescription
requestStreamingPredictRequest

The request received from the client.

responseStreamIServerStreamWriterStreamingPredictResponse

Used for sending responses back to the client.

contextServerCallContext

The context of the server-side call handler being invoked.

Returns
TypeDescription
Task

A task indicating completion of the handler.

StreamDirectPredict(IAsyncStreamReader<StreamDirectPredictRequest>, IServerStreamWriter<StreamDirectPredictResponse>, ServerCallContext)

public virtual Task StreamDirectPredict(IAsyncStreamReader<StreamDirectPredictRequest> requestStream, IServerStreamWriter<StreamDirectPredictResponse> responseStream, ServerCallContext context)

Perform a streaming online prediction request to a gRPC model server for Vertex first-party products and frameworks.

Parameters
NameDescription
requestStreamIAsyncStreamReaderStreamDirectPredictRequest

Used for reading requests from the client.

responseStreamIServerStreamWriterStreamDirectPredictResponse

Used for sending responses back to the client.

contextServerCallContext

The context of the server-side call handler being invoked.

Returns
TypeDescription
Task

A task indicating completion of the handler.

StreamDirectRawPredict(IAsyncStreamReader<StreamDirectRawPredictRequest>, IServerStreamWriter<StreamDirectRawPredictResponse>, ServerCallContext)

public virtual Task StreamDirectRawPredict(IAsyncStreamReader<StreamDirectRawPredictRequest> requestStream, IServerStreamWriter<StreamDirectRawPredictResponse> responseStream, ServerCallContext context)

Perform a streaming online prediction request to a gRPC model server for custom containers.

Parameters
NameDescription
requestStreamIAsyncStreamReaderStreamDirectRawPredictRequest

Used for reading requests from the client.

responseStreamIServerStreamWriterStreamDirectRawPredictResponse

Used for sending responses back to the client.

contextServerCallContext

The context of the server-side call handler being invoked.

Returns
TypeDescription
Task

A task indicating completion of the handler.

StreamGenerateContent(GenerateContentRequest, IServerStreamWriter<GenerateContentResponse>, ServerCallContext)

public virtual Task StreamGenerateContent(GenerateContentRequest request, IServerStreamWriter<GenerateContentResponse> responseStream, ServerCallContext context)

Generate content with multimodal inputs with streaming support.

Parameters
NameDescription
requestGenerateContentRequest

The request received from the client.

responseStreamIServerStreamWriterGenerateContentResponse

Used for sending responses back to the client.

contextServerCallContext

The context of the server-side call handler being invoked.

Returns
TypeDescription
Task

A task indicating completion of the handler.

StreamRawPredict(StreamRawPredictRequest, IServerStreamWriter<HttpBody>, ServerCallContext)

public virtual Task StreamRawPredict(StreamRawPredictRequest request, IServerStreamWriter<HttpBody> responseStream, ServerCallContext context)

Perform a streaming online prediction with an arbitrary HTTP payload.

Parameters
NameDescription
requestStreamRawPredictRequest

The request received from the client.

responseStreamIServerStreamWriterHttpBody

Used for sending responses back to the client.

contextServerCallContext

The context of the server-side call handler being invoked.

Returns
TypeDescription
Task

A task indicating completion of the handler.

StreamingPredict(IAsyncStreamReader<StreamingPredictRequest>, IServerStreamWriter<StreamingPredictResponse>, ServerCallContext)

public virtual Task StreamingPredict(IAsyncStreamReader<StreamingPredictRequest> requestStream, IServerStreamWriter<StreamingPredictResponse> responseStream, ServerCallContext context)

Perform a streaming online prediction request for Vertex first-party products and frameworks.

Parameters
NameDescription
requestStreamIAsyncStreamReaderStreamingPredictRequest

Used for reading requests from the client.

responseStreamIServerStreamWriterStreamingPredictResponse

Used for sending responses back to the client.

contextServerCallContext

The context of the server-side call handler being invoked.

Returns
TypeDescription
Task

A task indicating completion of the handler.

StreamingRawPredict(IAsyncStreamReader<StreamingRawPredictRequest>, IServerStreamWriter<StreamingRawPredictResponse>, ServerCallContext)

public virtual Task StreamingRawPredict(IAsyncStreamReader<StreamingRawPredictRequest> requestStream, IServerStreamWriter<StreamingRawPredictResponse> responseStream, ServerCallContext context)

Perform a streaming online prediction request through gRPC.

Parameters
NameDescription
requestStreamIAsyncStreamReaderStreamingRawPredictRequest

Used for reading requests from the client.

responseStreamIServerStreamWriterStreamingRawPredictResponse

Used for sending responses back to the client.

contextServerCallContext

The context of the server-side call handler being invoked.

Returns
TypeDescription
Task

A task indicating completion of the handler.