Enables the creation of a 10-million-row data warehouse with thousands of data-ingestion pipelines in just a few months
Boosts returns by providing internal teams and clients with on-demand, self-service insights into comprehensive, up-to-date market data
Minimizes cost, accuracy, and timing issues caused by manual analysis of data from internal and external silos
Unlocks data-monetization opportunities with an affordable, scalable, and easy-to-use platform that simplifies app development
New Frontier replaced its legacy analytics platform with a modern Google Cloud solution, powered by Looker and BigQuery, creating a scalable self-service data and BI platform with a new customer app.
New Frontier replaced its legacy analytics platform with a modern Google Cloud solution, powered by Looker and BigQuery, creating a scalable self-service data and BI platform with a new customer app.
The connection between food, insights, and financial trading
What struck me about BigQuery and Looker were their simplicity and cost-effectiveness. Their visual development tools make them easy to learn, and with Google Cloud, we'd get a whole ecosystem of tools with built-in AI. Google Cloud is ahead of the competition.
Kyle Hush
Data Analyst, New Frontier
Agriculture is a big business. In the US, the agriculture industry contributes more than $1.5 trillion to the nation's gross domestic product (GDP). Every day, agricultural commodity brokerages like New Frontier facilitate thousands of trade deals between crop and livestock producers, processors, and investors. Traditionally, brokerages have had to rely on expensive, hard-to-use heritage platforms to manage transactions and maintain an edge in the market by making predictions such as future pricing trends. However, those legacy platforms typically offer limited insights into third-party data that's needed to make accurate predictions. To better understand where the market has been and where it's likely going, brokers often manually download data from their platform and third-party sources, and then manually compile and analyze it in a spreadsheet.
Brokers' unique analytics models based on varied and potentially outdated data sources undermine accuracy, slow time-sensitive insights, and drive up costs.
Kyle Hush, data analyst at New Frontier, says, "Our industry is technically dated in many respects, including its reliance on spreadsheets and data silos. Instead of continuing down a path with limited growth, we decided to build our own technologically advanced and forward-looking solution that we could use, and then offer it to others as a subscription-based app." Along with delivering a more comprehensive view of industry data from one access point, New Frontier also wanted its platform to provide consistent and advanced data models — and be fast and affordable.
To build and launch its innovative solution, New Frontier evaluated leading cloud platforms, data warehouses, and BI tools as well as infrastructure and implementation strategies. The technologies it chose would have to safely ingest and analyze thousands of agricultural, government, and financial market reports from third-party sources including the Chicago Mercantile Exchange (CME Group). As a lean business with only seven employees, ease of use was also critical. New Frontier couldn't afford long learning curves at implementation, or in the future. Ultimately, the company chose to build on Google Cloud using BigQuery, Looker Embedded, and Looker Studio.
"I had meetings with every provider under the sun for BI and data warehousing tools," said Hush. "The fact that the CME uses Looker was appealing, but what struck me about BigQuery and Looker were their simplicity and cost-effectiveness. Their visual development tools make them easy to learn, and with Google Cloud, we'd get a whole ecosystem of tools with built-in AI like Vertex AI in BigQuery and Gemini in Looker. Google Cloud is ahead of the competition."
Democratized, visual access to 10 million rows of data from thousands of sources in just months
After choosing its solution's foundational technologies, New Frontier moved forward with simultaneous objectives. Hush and his team built a data warehouse with BigQuery. Using Looker Embedded, they created a semantic layer as well as data models and an internal-facing app. They built dashboards with Looker Studio. And to streamline development, they used both Vertex AI and Gemini in Looker. "As someone whose background is not in technology, having a visual language like we get in Looker makes it easy to get to work straightaway," says Hush. "And we leverage built-in AI as much as possible for coding, querying, troubleshooting, high-level analysis, and even for help developing large-scale machine learning models."
New Frontier's BigQuery data warehouse continuously ingests information from the CME, the USDA, the Federal Reserve, the US Energy Information Administration, and other third-party sources so insights are comprehensive and current.
As someone whose background is not in technology, having a visual language like we get in Looker makes it easy to get to work straightaway. And we leverage built-in AI as much as possible. In just months, we ingested close to 10 million rows of agricultural and economic data.
Kyle Hush
Data Analyst, New Frontier
"Market speculation and trading are all predicated on the volume of data, and the speed at which you can clean, store, and analyze it, which is why we chose BigQuery to be the backbone of our whole operation," says Hush. "In just months, we ingested close to 10 million rows of agricultural and economic data, and turned it into very usable formats for a lot of people. I think the value we're realizing from that speaks for itself."
The semantic layer in New Frontier's platform simplifies data modeling, governance, exploration, and visualization. Built with Looker's modeling language, LookML, the semantic layer enforces one governance policy that controls access to data in BigQuery, based on data type, user role, and even database row number. To enable one source of truth for users, Hush and his team also used LookML to quickly create standardized data, analytical, and ML models so insights and metrics are consistent for everyone. This strategy eliminates issues around efficiency and accuracy that can arise when brokers manually compile and analyze data themselves using their own models in a spreadsheet, often with stagnant data copied from differing external sources.
Increasing returns with comprehensive, on-demand market insights
Our overall goal is to monetize our data through an app that's easy to use. Looker helps us with that because we can present people with comprehensive insights in graphs and dashboards that look great and reflect up-to-date information from the CME, USDA, and other sources.
Kyle Hush
Data Analyst, New Frontier
For the company's internal app, Hush and his team embedded Looker and built hundreds of dashboards using Looker Studio, the ML models they built with Looker, and the comprehensive and accurate information in BigQuery. "Looker and BigQuery provide the scaffolding for advanced quantitative analysis," says Hush. "Without BigQuery and Looker, we wouldn't be able to effectively generate and scale accurate futures models."
From dashboards, brokers can now immediately see historic and current market trends and futures market data. Dashboard insights include supply and demand information related to crop conditions, livestock prices for specific cuts, futures predictions, and price-action predictions. Brokers can adjust dashboard visualizations and drill down into supporting data. And with Looker Studio, they can also explore all the data they have access to in BigQuery, and quickly create their own dashboards and reports.
By equipping its brokers with Looker dashboards and BI tools, New Frontier can help producers and processors negotiate more lucrative futures contracts. That's because all parties can make more informed decisions about whether, and when, to lock in the current sale or purchase price for their crops and livestock, even though they might not sell or buy their commodities for months — a process known as hedging. They can also offer speculative investments that yield returns based on industry price fluctuations.
New Frontier is now working on an external-facing, subscription-based app called Mercintel that uses Looker Embedded to surface the latest platform insights to customers. "Our overall goal is to monetize our data through an app that's easy to use," Hush explains. "Looker helps us with that because we can present people with comprehensive insights in graphs and dashboards that look great and reflect up-to-date information from the CME, USDA, and other sources." Ensuring the app is intuitive and affordable will also attract customers beyond the typical users of brokerage platforms such as agribusiness professionals and speculative investors who need insights to improve their business strategies.
The move to replace its legacy platform with its modern solution in Google Cloud gives New Frontier and its customers a competitive edge by moving past silos, manual processes, and expensive heritage licenses to holistic, on-demand views that are both affordable and intuitive. "The real advantage of our approach is timing," Hush says. "We're in a very fast-paced industry. Even when you're outside of the office, people need answers quickly. Being able to rapidly respond to a customer or message by looking at a dashboard, or using Gemini with natural language to generate a custom summary or report, that's game-changing from a service standpoint — and it's disruptive when you look at it from an industry perspective."
New Frontier is a full-service futures brokerage that provides investment options for agricultural producers and processors as well as domestic and global speculative derivatives clients.
Industry: Financial Services
Location: United States
Products: Google Cloud, BigQuery, Looker Embedded, Looker Studio
