[[["わかりやすい","easyToUnderstand","thumb-up"],["問題の解決に役立った","solvedMyProblem","thumb-up"],["その他","otherUp","thumb-up"]],[["わかりにくい","hardToUnderstand","thumb-down"],["情報またはサンプルコードが不正確","incorrectInformationOrSampleCode","thumb-down"],["必要な情報 / サンプルがない","missingTheInformationSamplesINeed","thumb-down"],["翻訳に関する問題","translationIssue","thumb-down"],["その他","otherDown","thumb-down"]],["最終更新日 2025-09-04 UTC。"],[[["\u003cp\u003eDataflow SQL is deprecated and will be inaccessible via the Google Cloud console after July 31, 2024, and through the Google Cloud CLI after January 31, 2025; consider alternatives like Beam SQL.\u003c/p\u003e\n"],["\u003cp\u003eCreating a Dataflow SQL job requires writing and running a Dataflow SQL query using the Dataflow SQL editor in the Google Cloud console or through the Google Cloud CLI.\u003c/p\u003e\n"],["\u003cp\u003eRunning Dataflow SQL queries turns the SQL into an Apache Beam pipeline, and jobs can be configured with pipeline options like region, machine type, and network settings.\u003c/p\u003e\n"],["\u003cp\u003eDataflow SQL jobs utilize standard Dataflow pricing for resources like vCPU, memory, and Persistent Disk, and stopping these jobs requires cancellation rather than draining.\u003c/p\u003e\n"],["\u003cp\u003eDataflow SQL queries utilize a similar syntax to that of BigQuery standard SQL, and can be extended through the use of streaming extensions.\u003c/p\u003e\n"]]],[],null,["# Use Dataflow SQL\n\n\u003cbr /\u003e\n\n| **Caution:** Dataflow SQL is [deprecated](/appengine/docs/deprecations). As of July 31, 2024, you can no longer access Dataflow SQL in the Google Cloud console. As of January 31, 2025, you can no longer use Dataflow SQL in the Google Cloud CLI. For information about alternative methods of creating SQL queries with Dataflow, see [Beam SQL and `beam_sql` magic](/dataflow/docs/guides/notebook-advanced#beam-sql).\n\n\u003cbr /\u003e\n\nThe page explains how to use Dataflow SQL and create Dataflow SQL\njobs.\n\nTo create a Dataflow SQL job, you must [write](#writing-queries) and\n[run](#running-queries) a Dataflow SQL query.\n| **Note:** To use Dataflow SQL, you might need to [enable the Data Catalog API](/apis/docs/enable-disable-apis#enabling_apis) in the Google Cloud [project](/resource-manager/docs/cloud-platform-resource-hierarchy#projects) that you're using to write and run queries. To run Dataflow SQL queries, your user account needs to have the [**Storage Admin** role](/iam/docs/understanding-roles#cloud-storage-roles) create and write to a temporary storage bucket.\n\nUse the Dataflow SQL editor\n---------------------------\n\nThe Dataflow SQL editor is a page in the Google Cloud console where you\nwrite and run queries for creating Dataflow SQL jobs.\n\nTo access the Dataflow SQL editor, follow these steps:\n\n- In the Google Cloud console, go to the **Dataflow SQL Editor** page.\n\n [Go to Dataflow SQL editor](https://console.cloud.google.com/dataflow/sqleditor)\n\nYou can also access the Dataflow SQL editor from the [Dataflow\nmonitoring interface](/dataflow/docs/guides/using-monitoring-intf) by following\nthese steps:\n\n1. In the Google Cloud console, go to the Dataflow **Jobs**\n page.\n\n [Go to Jobs](https://console.cloud.google.com/dataflow/jobs)\n2. In the Dataflow menu, click **SQL Workspace**.\n\nWrite Dataflow SQL queries\n--------------------------\n\nDataflow SQL queries use the [Dataflow SQL query syntax](/dataflow/docs/reference/sql/query-syntax).\nThe Dataflow SQL query syntax is similar to [BigQuery standard SQL](/bigquery/docs/reference/standard-sql/query-syntax).\n\nYou can use the [Dataflow SQL streaming extensions](/dataflow/docs/reference/sql/streaming-extensions)\nto aggregate data from continuously updating Dataflow sources like\nPub/Sub.\n\nFor example, the following query counts the passengers in a\nPub/Sub stream of taxi rides every minute: \n\n```sql\nSELECT\n DATETIME(tr.window_start) AS starttime,\n SUM(tr.passenger_count) AS pickup_count\nFROM TUMBLE ((SELECT * FROM pubsub.topic.`pubsub-public-data`.`taxirides-realtime`),\nDESCRIPTOR(event_timestamp), 'INTERVAL 1 MINUTE') AS tr\nWHERE\n tr.ride_status = \"pickup\"\nGROUP BY DATETIME(tr.window_start)\n```\n\nRun Dataflow SQL queries\n------------------------\n\nWhen you run a Dataflow SQL query, Dataflow turns the\nquery into an [Apache Beam pipeline](/dataflow/docs/concepts/beam-programming-model)\nand runs the pipeline.\n\nYou can run a Dataflow SQL query using the Google Cloud console or\nGoogle Cloud CLI. \n\n### Console\n\nTo run a Dataflow SQL query, use the Dataflow SQL editor:\n\n1. Go to the **Dataflow SQL Editor** page.\n\n [Go to Dataflow SQL editor](https://console.cloud.google.com/dataflow/sqleditor)\n2. Enter the Dataflow SQL query into the query editor.\n\n3. Click **Create job** to open a panel of job options.\n\n4. Optional: For **Job name**, enter a unique job name.\n\n5. For **Regional endpoint**, select a value from the menu.\n\n6. Optional: Click **Show optional parameters**, and then enter values for the\n provided Dataflow pipeline options.\n\n7. For **Destination** , select an **Output type**, and then enter\n values for the provided fields.\n\n8. Optional: In the **SQL query parameters** section, add parameters and then\n enter values in the provided fields.\n\n9. Click **Create**.\n\n\n| **Note:** Starting a Dataflow SQL job might take several minutes. You cannot update a Dataflow SQL job after creating it.\n\n\u003cbr /\u003e\n\n### gcloud\n\nTo run a Dataflow SQL query, use the [`gcloud dataflow sql query`](/sdk/gcloud/reference/dataflow/sql/query)\ncommand. The following is an example SQL query that creates \n\n```bash\ngcloud dataflow sql query \\\n --job-name=JOB_NAME \\\n --region=REGION \\\n --bigquery-table=BIGQUERY_TABLE \\\n --bigquery-dataset=BIGQUERY_DATASET \\\n --bigquery-project=BIGQUERY_PROJECT \\\n'\u003cvar translate=\"no\"\u003eSQL_QUERY\u003c/var\u003e'\n```\n\nReplace the following:\n\n- \u003cvar translate=\"no\"\u003eJOB_NAME\u003c/var\u003e: a name for your Dataflow SQL job\n- \u003cvar translate=\"no\"\u003eREGION\u003c/var\u003e: the [Dataflow location](/dataflow/docs/resources/locations) for deploying your Dataflow job\n- \u003cvar translate=\"no\"\u003eBIGQUERY_TABLE\u003c/var\u003e: the name of the BigQuery table to which you want to write the output\n- \u003cvar translate=\"no\"\u003eBIGQUERY_DATASET\u003c/var\u003e: the BigQuery dataset ID that contains the output table\n- \u003cvar translate=\"no\"\u003eBIGQUERY_PROJECT\u003c/var\u003e: the Google Cloud project ID that contains the output BigQuery table\n- \u003cvar translate=\"no\"\u003eSQL_QUERY\u003c/var\u003e: your Dataflow SQL query\n\n\n| **Note:** Starting a Dataflow SQL job might take several minutes. You cannot update a Dataflow SQL job after creating it.\n\n\u003cbr /\u003e\n\n### Set pipeline options\n\nYou can set Dataflow pipeline options for Dataflow SQL jobs.\nDataflow pipeline options are [execution parameters](/dataflow/docs/guides/setting-pipeline-options)\nthat configure how and where to run Dataflow SQL queries.\n\nTo set Dataflow pipeline options for Dataflow SQL jobs,\nspecify the following parameters when you [run a Dataflow SQL query](#running-query). \n\n### Console\n\n### gcloud\n\nFor more information, see the\n[`gcloud dataflow sql query`](/sdk/gcloud/reference/dataflow/sql/query)\ncommand reference.\n| **Note:** Dataflow SQL jobs use autoscaling and Dataflow automatically chooses the execution mode (batch or streaming). You cannot control this behavior for Dataflow SQL jobs.\n\nStop Dataflow SQL jobs\n----------------------\n\nTo stop a Dataflow SQL job, you must [cancel](/dataflow/docs/guides/stopping-a-pipeline#cancel) it.\nStopping a Dataflow SQL job with the `drain` option is not supported.\n\nPricing\n-------\n\nDataflow SQL uses the standard Dataflow pricing; it\ndoes not have separate pricing. You are billed for the resources consumed by the\nDataflow jobs that you create based on your SQL statements. The\ncharges for these resource are the standard Dataflow charges for\nvCPU, memory, Persistent Disk, Streaming Engine, and Dataflow Shuffle.\n\nA Dataflow SQL job might consume additional resources such as\nPub/Sub and BigQuery, each billed at their own\npricing.\n\nFor more information about Dataflow pricing, see\n[Dataflow pricing](https://cloud.google.com/dataflow/pricing).\n\nWhat's next\n-----------\n\n- Walk through the [Joining streaming data with Dataflow SQL](/dataflow/docs/samples/join-streaming-data-with-sql) tutorial.\n- Explore the [Google Cloud CLI for Dataflow SQL](/sdk/gcloud/reference/dataflow/sql/query)."]]
