이 페이지에서는 Cortex Framework Data Foundation 배포를 위해 외부 데이터 세트를 구성하는 선택적 단계를 설명합니다. 일부 고급 사용 사례에서는 기록의 엔터프라이즈 시스템을 보완하기 위해 외부 데이터 세트가 필요할 수 있습니다. BigQuery 공유 (이전의 Analytics Hub)에서 사용되는 외부 교환 외에도 데이터를 수집하고 보고 모델과 결합하기 위해 맞춤 또는 맞춤형 방법이 필요한 데이터 세트가 있을 수 있습니다.
다음 외부 데이터 세트를 사용 설정하려면 데이터 세트를 배포하려는 경우 k9.deployDataset를 True로 설정하세요.
다음 단계에 따라 지원되는 외부 데이터 세트의 방향성 비순환 그래프 (DAG)를 구성합니다.
메시지가 표시되면 noaa_global_forecast_system를 데이터 세트의 이름으로 유지합니다. 필요한 경우 weather_daily.sql의 FROM 절에서 데이터 세트와 테이블의 이름을 조정합니다.
데이터 세트 OpenStreetMap Public Dataset에 대한 등록정보 검색을 반복합니다.
postcode.sql에서 BigQuery-public-data.geo_openstreetmap.planet_layers을 포함하는 FROM 절을 조정합니다.
지속 가능성 및 ESG 통계: Cortex Framework는 SAP 공급업체 실적 데이터를 고급 ESG 통계와 결합하여 전 세계 운영 전반에서 배송 실적, 지속 가능성, 위험을 더 전체적으로 비교합니다. 자세한 내용은 Dun & Bradstreet 데이터 소스를 참고하세요.
일반적인 고려사항
공유는 EU 및 미국 위치에서만 지원되며 NOAA Global Forecast와 같은 일부 데이터 세트는 단일 멀티 위치에서만 제공됩니다.
필수 데이터 세트에 사용할 수 있는 위치와 다른 위치를 타겟팅하는 경우 예약된 쿼리를 만들어 공유 링크 데이터 세트에서 새 레코드를 복사한 다음 전송 서비스를 사용하여 해당 새 레코드를 나머지 배포와 동일한 위치 또는 리전에 있는 데이터 세트에 복사하는 것이 좋습니다.
그런 다음 SQL 파일을 조정해야 합니다.
이러한 DAG를 Cloud Composer에 복사하기 전에 필요한 Python 모듈을 종속 항목으로 추가하세요.
[[["이해하기 쉬움","easyToUnderstand","thumb-up"],["문제가 해결됨","solvedMyProblem","thumb-up"],["기타","otherUp","thumb-up"]],[["이해하기 어려움","hardToUnderstand","thumb-down"],["잘못된 정보 또는 샘플 코드","incorrectInformationOrSampleCode","thumb-down"],["필요한 정보/샘플이 없음","missingTheInformationSamplesINeed","thumb-down"],["번역 문제","translationIssue","thumb-down"],["기타","otherDown","thumb-down"]],["최종 업데이트: 2025-09-04(UTC)"],[[["\u003cp\u003eThis page provides instructions for configuring optional external datasets within the Cortex Framework Data Foundation deployment, which can be utilized to enhance enterprise systems of record with external data.\u003c/p\u003e\n"],["\u003cp\u003eConfiguring external datasets involves setting \u003ccode\u003ek9.deployDataset\u003c/code\u003e to \u003ccode\u003eTrue\u003c/code\u003e and setting up Directed Acyclic Graphs (DAGs) for each supported dataset like the holiday calendar, search trends, weather, and sustainability/ESG data.\u003c/p\u003e\n"],["\u003cp\u003eThe Holiday Calendar DAG retrieves special dates from PyPi Holidays, allowing customization of countries and years through the \u003ccode\u003eholiday_calendar.ini\u003c/code\u003e file.\u003c/p\u003e\n"],["\u003cp\u003eThe Trends DAG fetches "Interest Over Time" data from Google Search Trends, with configurable terms and date ranges in \u003ccode\u003etrends.ini\u003c/code\u003e, and recommends multiple copies for large term lists.\u003c/p\u003e\n"],["\u003cp\u003eThe Weather DAG uses public data from \u003ccode\u003eBigQuery-public-data.geo_openstreetmap.planet_layers\u003c/code\u003e and the \u003ccode\u003enoaa_global_forecast_system\u003c/code\u003e from Analytics Hub, both of which need to be available in the same region as other datasets.\u003c/p\u003e\n"]]],[],null,["# Configure external datasets\n===========================\n\nThis page describes an optional step to configure external datasets for\nthe Cortex Framework Data Foundation deployment. Some advanced\nuse cases might require external datasets to complement an enterprise system of\nrecord. In addition to external exchanges consumed from\n[BigQuery sharing (formerly Analytics Hub)](/bigquery/docs/analytics-hub-introduction),\nsome datasets might need custom or tailored methods to ingest data\nand join them with the reporting models.\n\nTo enable the following external datasets, set `k9.deployDataset` to `True`\nif you want Dataset to be deployed.\n\nConfigure the Directed Acyclic Graphs (DAGs) for the supported external datasets\nfollowing these steps:\n\n1. **Holiday Calendar:** This DAG retrieves the special dates from\n [PyPi Holidays](https://pypi.org/project/holidays/).\n\n | **Note:** If using sample data, keep default values.\n 1. Adjust the list of countries, the list of years, as well as other DAG parameters to retrieve holidays in [`holiday_calendar.ini`](https://github.com/GoogleCloudPlatform/cortex-data-foundation/blob/main/src/k9/src/holiday_calendar/holiday_calendar.ini).\n2. **Trends** : This DAG retrieves *Interest Over Time* for a specific set\n of terms from [Google Search trends](https://trends.google.com/trends/).\n The terms can be configured in [`trends.ini`](https://github.com/GoogleCloudPlatform/cortex-data-foundation/blob/main/src/k9/src/trends/trends.ini).\n\n 1. After an initial run, adjust the `start_date` to `'today 7-d'` in [`trends.ini`](https://github.com/GoogleCloudPlatform/cortex-data-foundation/blob/main/src/k9/src/trends/trends.ini).\n 2. Get familiarized with the results coming from the different terms to tune parameters.\n 3. We recommend partitioning large lists to multiple copies of this DAG running at different times.\n 4. For more information about the underlying library being used, see [Pytrends](https://pypi.org/project/pytrends/).\n3. **Weather** : By default, this DAG uses the publicly available\n test dataset [`BigQuery-public-data.geo_openstreetmap.planet_layers`](https://console.cloud.google.com/bigquery/analytics-hub/exchanges(analyticshub:search)?queryText=open%20street%20map).\n The query also relies on an NOAA dataset only available\n through Sharing: [`noaa_global_forecast_system`](https://console.cloud.google.com/bigquery/analytics-hub/exchanges(analyticshub:search)?queryText=noaa%20global%20forecast).\n\n **This dataset needs to be created in the same region as the other datasets prior to executing deployment**. If the datasets aren't available in your region, you can continue\n with the following instructions to transfer the data into the chosen region:\n 1. Go to the [**Sharing (Analytics Hub)**](https://console.cloud.google.com/bigquery/analytics-hub) page.\n 2. Click **Search listings**.\n 3. Search for **NOAA Global Forecast System**.\n 4. Click **Subscribe**.\n 5. When prompted, keep `noaa_global_forecast_system` as the name of the dataset. If needed, adjust the name of the dataset and table in the FROM clauses in `weather_daily.sql`.\n 6. Repeat the listing search for Dataset `OpenStreetMap Public Dataset`.\n 7. Adjust the `FROM` clauses containing: `BigQuery-public-data.geo_openstreetmap.planet_layers` in `postcode.sql`.\n4. **Sustainability and ESG insights** : Cortex Framework combines\n SAP supplier performance data with advanced ESG insights to compare\n delivery performance, sustainability and risks more holistically across\n global operations. For more information,\n see the [Dun \\& Bradstreet data source](/cortex/docs/dun-and-bradstreet).\n\nGeneral considerations\n----------------------\n\n- [Sharing](/bigquery/docs/analytics-hub-introduction)\n is only supported in EU and US locations,\n and some datasets, such as NOAA Global Forecast, are only offered\n in a single multi location.\n\n If you are targeting a location different\n from the one available for the required dataset, we recommended to create\n a [scheduled query](/bigquery/docs/scheduling-queries)\n to copy the new records from the Sharing\n linked dataset followed by a [transfer service](/bigquery/docs/dts-introduction)\n to copy those new records into a dataset located\n in the same location or region as the rest of your deployment.\n You then need to adjust the SQL files.\n- Before copying these DAGs to Cloud Composer, add the required\n python modules [as dependencies](/composer/docs/how-to/using/installing-python-dependencies#options_for_managing_python_packages):\n\n Required modules:\n pytrends~=4.9.2\n holidays"]]
