AI & Machine Learning

Anthropic’s Claude 3 Opus and tool use are generally available on Vertex AI

May 31, 2024

https://storage.googleapis.com/gweb-cloudblog-publish/images/image1_A1zLY0S.max-2000x2000.jpg

Nenshad Bardoliwalla

Director, Product Management, Vertex AI

In April, we made Anthropic’s most intelligent model, Claude 3 Opus, available in public preview on Vertex AI Model Garden for customers to start experimenting with. Now, we’re announcing that Claude 3 Opus is generally available on Vertex AI. We’re also excited to announce that tool use is available for all Claude 3 models on Vertex AI, helping to enable more agentic workflows, and provisioned throughput is now available for enterprise-grade reliability and performance.

The addition of Claude 3 Opus represents a milestone, with the entire Claude 3 model family — Opus, Sonnet, and Haiku — now generally available on Vertex AI.

Claude 3 Opus: Anthropic’s most capable and intelligent model yet, ideal for navigating complex tasks like in-depth analysis, research, and task automation.
Claude 3 Sonnet: Anthropic’s best combination of skills and speed, ideal for scaled AI deployments like powering intelligent virtual assistants and chatbots.
Claude 3 Haiku: Anthropic's fastest and most compact model, ideal for applications requiring rapid responses and efficient resource utilization.

Opus use cases across industries

Claude 3 Opus is designed to tackle complex tasks with remarkable comprehension and fluency. Some example applications include:

For financial institutions, Opus enables advanced market analysis, sophisticated financial modeling, automated compliance processes, and robust risk management strategies.
In the life sciences field, Opus accelerates drug discovery through literature synthesis, expedites research into novel patient treatments, and generates hypotheses for exploration.
Across internal operations, Opus facilitates the migration of entire legacy codebases, automates common workflows and tasks, and enhances productivity through seamless process automation.

Assured performance with provisioned throughput

Vertex AI now offers provisioned throughput for Anthropic's Claude 3 models, enabling a subscription-based pricing model with guaranteed performance and cost predictability. The existing pay-as-you-go option remains available. Provisioned throughput provides:

Assured capacity and performance: Reserve the required throughput (tokens/sec) with the confidence of Google-backed availability and prioritized execution for consistent low latency.
Predictable costs: Fixed pricing simplifies budgeting and forecasting for AI workloads.
Flexibility: Combine provisioned throughput for baseload capacity with pay-as-you-go for excess demand, optimizing cost and performance.

To get started with provisioned throughput, please contact your Google Cloud sales representative.

Tool use: extending Claude’s capabilities

Tool use, also known as function calling, allows foundation models to autonomously interact with external data sources, APIs, and other tools. This enables Claude to act as an autonomous agent, reasoning about which tools to leverage and executing sequences of tool interactions to accomplish complex goals that may require real-time data or computations.

For teams that want more control and flexibility in defining how Claude uses tools, you can use:

Tool use streaming: Build more engaging, human-like interactions by reducing perceived and actual wait times for end-users.
Forced tool use: Instruct Claude to reliably select the appropriate tool in various scenarios, including whether to use any tool, always use one, or use a specific tool you define.
Vision: Tools work with images. You can instruct Claude to consistently follow a predefined format when working with images, or incorporate image inputs as part of a live assistant and chat application.

Tool use, meet Google APIs

Developers can now harness the power of the Claude 3 models and combine it with external tools including the capabilities of Google's extensive collection of APIs and services. The example below shows how to provide the Google Places API as a tool to Claude, enabling the capability to search for nearby restaurants based on price and open hours.

The code sets up the AnthropicVertex client, defines the Google Places API tool with its input schema, and then sends a message to Claude instructing it to use the provided tool to find affordable and good Italian restaurants that are currently open in San Francisco.

from anthropic import AnthropicVertex

MODEL = "claude-3-opus@20240229"
PROJECT_ID = 'your-project-id'
REGION = "us-east5"

client = AnthropicVertex(region=REGION, project_id=PROJECT_ID)
response = client.messages.create(
    model = MODEL,
    max_tokens = 1024,
    tools = [
      {
        "name": "text_search_places_api",
        "description": "returns information about a set of places based on a string",
        "input_schema": {
          "type": "object",
          "properties": {
            "textQuery": {
              "type": "string",
              "description": "The text string on which to search"
            },
            "priceLevels": {
              "type": "array",
              "description": "Price levels to query places, value can be one of [PRICE_LEVEL_INEXPENSIVE, PRICE_LEVEL_MODERATE, PRICE_LEVEL_EXPENSIVE, PRICE_LEVEL_VERY_EXPENSIVE]",
            },
            "openNow": {
              "type": "boolean",
              "description": "whether those places are open for business."
            },
          },
          "required": ["textQuery"]
        }
      }
    ],
    messages = [
      {
        "role": "user",
        "content": "What are some affordable and good Italian restaurants open now in San Francisco??"
      }
    ],
)
print(response)

#Claude response
Message(
id =
  'msg_vrtx_01PVYodbvgeRibsC8mppez8m',
  content = [
    TextBlock(
      text =
      '<thinking>\nTo find affordable and good Italian restaurants open now in San Francisco, I will need to use the text_search_places_api tool. Let\'s look at the required parameters:\n\ntextQuery (required): The user\'s request provides a good text query to search for - "affordable and good Italian restaurants in San Francisco". So this parameter is satisfied.\n\nopenNow (optional): The user specified they want places open now, so I should set this optional parameter to true.\n\npriceLevels (optional): The user asked for affordable restaurants. Translating "affordable" to price levels, I think it\'s reasonable to include PRICE_LEVEL_INEXPENSIVE and PRICE_LEVEL_MODERATE. The user didn\'t ask for a specific price level, so I don\'t need to include the more expensive levels.\n\nIn summary, I have enough information to make the text_search_places_api call with the textQuery and openNow parameters, and can reasonably infer values for the optional priceLevels parameter based on the user\'s request for "affordable" restaurants. No other tools are needed to answer this query.\n</thinking>',
      type = 'text'
    ),
    ToolUseBlock(id =
      'toolu_vrtx_01AUxyh2YPF8uW6vxQx63hh4',
      input = {
        'textQuery': 'affordable and good Italian restaurants in San Francisco',
        'openNow': True,
        'priceLevels': [
          'PRICE_LEVEL_INEXPENSIVE',
          'PRICE_LEVEL_MODERATE'
        ]
      },
      name = 'text_search_places_api',
      type = 'tool_use')
  ], model = 'claude-3-opus-20240229',
  role = 'assistant', stop_reason =
  'tool_use', stop_sequence = None,
  type = 'message', usage = Usage(
    input_tokens = 727, output_tokens =
    372)
)

Build on a unified AI platform with Vertex AI

Vertex AI provides a comprehensive and enterprise-ready platform for building, deploying, and managing AI models at scale. By building on Vertex AI, you can confidently leverage the power of Claude 3 models while taking advantage of a unified AI development platform designed to support your generative AI workflows from start to finish.

Build applications with Vertex AI’s generative AI tools: Vertex AI provides an integrated environment to develop, evaluate, and deploy generative AI applications. This includes Auto SxS for model evaluation, LangChain integration for custom application building, and Colab Enterprise for iterative development.
Optimize performance and cost: Vertex AI's scalable infrastructure and flexible pricing models empower you to experiment, deploy, and scale your generative AI applications efficiently.
Maximize the power of your data with BigQuery: Seamlessly integrate your enterprise data with Claude 3's advanced capabilities, leveraging tools like BigQuery to extract valuable insights and drive informed decision-making.
Deploy responsibly with enterprise-grade security, compliance, and data governance: Leverage Google Cloud's built-in security, privacy, data governance, and compliance capabilities tailored to adhere to enterprise-level standards.

Get started with Claude 3 on Vertex AI

Visit Vertex AI Model Garden and select “Browse Model Garden”.
Navigate to the foundation models section.
Select the Claude 3 Opus, Claude 3 Sonnet, or Claude 3 Haiku tile from the available models.
Select “Enable” and follow the proceeding instructions.
You should now have API access to integrate Opus into your workflows. Deploy your AI applications and monitor their performance using Vertex AI's comprehensive tools. Use our sample notebook to get started.