zest HACCP: Developing a fresh approach to processing food labels with machine learning

About zest HACCP

Through its online platform, zest HACCP saves retailers both time and money by enabling them to digitize their food control plan, ensuring they meet hygiene and traceability standards.

Industries: Retail & Consumer Goods, Technology
Location: France

About Cirruseo

As a Google Cloud Premier Partner, Cirruseo offers innovative cloud-based IT solutions such as system migrations, made-to-measure apps, and analytics services.

zest HACCP built a solution to speed up data input using Cloud Vision API, helping retailers save time spent generating goods receipt information that meets hygiene and traceability standards.

Google Cloud Results

  • Saves time with automated text transcription and record entry, enabling operatives to add value in core business areas
  • Improves overall quality and accuracy of data thanks to an 80% pre-validation accuracy rate
  • Enables agile application development in just 5 weeks on Google Cloud Platform

Data input time per unit down from 30 to 15 seconds

To make sure the food on its shelves is safe for consumption, supermarkets are expected to keep detailed records about everything from how goods are obtained and when they're received, to what kind of packaging they come in, any allergens they contain, and even their temperature. To make this process faster and more accurate, zest HACCP offers retailers an online platform that simplifies their daily food safety record-keeping and helps manage the tracking of products made on-site. Founded in 2015, the Bordeaux-based company works with major French retail brands including Intermarché, Carrefour, E.Leclerc, and Match.

"Retail is moving faster and faster: the big challenge for our clients is saving as much time as possible. Using our platform already saves the average French supermarket three man-hours a day, but with new technologies, we can develop tools to optimize that even further for our clients."

Christophe André, Director, zest HACCP

"Our clients are subject to European regulations on food security, in particular directives on hygiene and traceability," explains Christophe André, Director at zest HACCP. "For example, when they receive fresh salmon from their supplier, they need to record everything about it, from where and how it was caught to its best before date." The label relating to a particular product's delivery, plus details on product tracking, have to be kept for six months. Previously, this was done by manually recording the information or storing the label in a folder, but zest HACCP wanted to find a way that was faster and more accurate.

To speed up the goods reception process, it looked for a way to automatically extract text from a food label and input the data into the correct fields in its digital records system. Working with implementation partner Cirruseo to build a solution on Google Cloud Platform (GCP), it was able to meet these goals.

"Retail is moving faster and faster: the big challenge for our clients is saving as much time as possible," says Christophe. "Using our platform already saves the average French supermarket three man-hours a day, but with new technologies, we can develop tools to optimize that even further for our clients."

Extracting and structuring the label data

Precision is important when extracting data automatically, but it's also crucial to ensure the data can be structured so it's fit for purpose. "In order to fill in a product record, we need to both extract the document text precisely, and identify which text candidate is truly the lot number, which relates to the allergen content, and so on," explains Sarah Haimiche, Head of Data and Analytics at Cirruseo.

"Because Google Cloud Vision API detects feature objects within a bounding box, we could feed the scoring tool with additional information. Using features including text position or size, we created an algorithm set for identifying which field a particular text sequence belongs to: supplier, allergen code, or batch number, for example."

Sarah Haimiche, Head of Data and Analytics, Cirruseo

To do that, Cirruseo built a solution with Cloud Vision API on GCP, using App Engine to create a logic scoring system that automatically categorizes label text. "We chose to go down the logic route because we didn't have a data set readily available for training machine learning models," explains Sarah. "Because Google Cloud Vision API detects feature objects within a bounding box, we could feed the scoring tool with additional information. Using features including text position or size, we created an algorithm set for identifying which field a particular text sequence belongs to: supplier, allergen code, or batch number, for example."

Once a label is photographed by a retail operative and uploaded to the system, Cloud Vision API extracts and transcribes the textual data as sequences of characters. The data is then transferred to Cloud Datastore via App Engine. The scoring tool, also running on App Engine, then scores each sequence against the predefined characteristics to identify the best fit for each field. The operative can then go ahead and validate the pre-completed form. The photographs are stored using Cloud Storage.

Staying ahead with fast development

To stay ahead of its competitors, zest HACCP wanted to develop its application as quickly as possible. "None of our competitors offer a similar tool, so we were keen to get it to market fast," explains Christophe. "Our clients were also eager to see the results, as we'd told them about the possibilities of this solution and they were keen to try it out."

"The process of working with Cirruseo was very straightforward," he says. "They were able to meet our expectations for a quick launch, developing the application on GCP in just five weeks."

"With our new application using Google Cloud Vision API, we've really accelerated the data entry process for our clients. Operatives perform up to 100 goods reception operations every day, so that's potentially 25 minutes saved a day, which they can use to add value in sales, promotions, merchandising, and more."

Christophe André, Director, zest HACCP

Helping food retailers to move faster

Already in production within a number stores, the tool currently has a pre-validation accuracy level of approximately 85 percent for good quality photos. It's improved data accuracy overall because unlike before, every field is automatically filled out before validation. "The data is more consistent because we've predefined the allergy codes, for example, so there's also a qualitative improvement in the records being kept, from 70 percent before, to 85 percent accuracy now," explains Sarah.

It's also helped store operatives quickly and easily process labels on products arriving in the shop, halving data input time from 30 seconds to 15 secs per unit. On top of this, zest HACCP is now able to reduce human error in order to achieve a higher quality of data. "With our new application using Google Cloud Vision API, we've really accelerated the data entry process for our clients," says Christophe. "Operatives perform up to 100 goods reception operations every day, so that's potentially 25 minutes saved a day, which they can use to add value in sales, promotions, merchandising, and more."

zest HACCP is now planning to expand its operations beyond France to other European countries, and is currently running trials in Portugal, Belgium, and Spain.

"One of the reasons we chose GCP is that we wanted to be sure our solution would work just as well in Spain as it does in France," says Christophe. "As we expand, we're confident that GCP will be able to support our customers, wherever they may be."

About zest HACCP

Through its online platform, zest HACCP saves retailers both time and money by enabling them to digitize their food control plan, ensuring they meet hygiene and traceability standards.

Industries: Retail & Consumer Goods, Technology
Location: France

About Cirruseo

As a Google Cloud Premier Partner, Cirruseo offers innovative cloud-based IT solutions such as system migrations, made-to-measure apps, and analytics services.