Heinrich Heine University: Developing natural language processing with Google Cloud
About Dialog Systems and Machine Learning Group at Heinrich Heine University Düsseldorf
The Dialog Systems and Machine Learning Group researches Natural Language Processing to aid the development of the next generation of intelligent conversational agents.
Tell us your challenge. We're here to help.
Contact us
About TWT
Düsseldorf-based TWT is a Google Cloud partner specializing in developing machine learning applications.
The Dialog Systems and Machine Learning Group at Heinrich Heine University Düsseldorf harnesses the power of Google Cloud to cost-effectively accelerate the development of natural language processing.
Google Cloud results:
- 24 hours to run large-scale machine learning experiments with the cloud, 60% quicker than the existing on-premises solution
- Eliminates downtime, boosting research productivity
- Allows several virtual machines to be run in parallel
- Saves time and offers more flexibility than the previous on-premises solution
Supercharging authentic AI conversation with Google Cloud
Natural language processing (NLP), which lies at the intersection of computer science, artificial intelligence (AI), and linguistics, aims to equip computers with the ability to process and analyze natural language just like humans can. Dialogue modeling, or conversational AI, enables human-computer interaction via natural language, as dialogue is the most natural means of communication between humans. Stimulated by the demand for smart and connected technologies, the global NLP market is booming and is expected to be worth $48.46 billion by 2026.
This is a field of study that theDialog Systems and Machine Learning Group at Heinrich Heine University Düsseldorf is making great strides in. “We are working on developing the next generation of dialog systems that are more emotionally aware to achieve more human-like conversations. We are creating spoken dialog systems that can expand their knowledge dynamically, flexibly react to the user’s needs beyond simple information-seeking dialogs, and understand the user’s behavior, as well as sentiment, and react appropriately to it,” explains Professor Milica Gašić, Chair of Dialog Systems and Machine Learning at Heinrich Heine University Düsseldorf.
“We chose Google Cloud because we are primarily funded by the taxpayer, and Google not only offered the tools, the software compatibility, the flexibility, and the ability to scale that we required, it was also the most economical option.”
—Professor Milica Gašić, Chair of Dialog Systems and Machine Learning, Heinrich Heine University DüsseldorfSeeking an economic solution for a public sector body
Gašić’s relationship with Google stretches back to 2016 when she received the Google Faculty Award, which she used to generate a large, labeled dataset, a prerequisite to enable her and the science community to study multi-domain dialog phenomena. She had also worked previously with Google Director of Engineering, Roberto Pieraccini. Being familiar with the platform and Google’s suite of machine learning products, Gašić was confident that it could deliver everything she needed for her research, but she only committed to work with Google Cloud when it proved to be the most cost-effective option for her limited education budget.
“Roberto has a lot of long-term industrial insight into what we are doing through his work on Google Assistant, and he is always available to discuss how things are working and not working,” says Gašić. “I knew that he would be a really valuable part of the Google team for us. But, ultimately, we chose Google Cloud because we are primarily funded by the taxpayer, and Google not only offered the tools, the software compatibility, the flexibility, and the ability to scale that we required, it was also the most economical option.” Google University Relations provided Professor Gašić and her team with Google Cloud credits as seed funding in the summer of 2020, which helped establish the team’s work on Google Cloud.
Designing a tailor-made contract with Google
As the Dialog Systems and Machine Learning Group was looking to create a bespoke Google Cloud solution, it enlisted Google Cloud Partner TWT to provide ongoing support and to help draw up a tailor-made contract that best suited its needs. Gašić says that TWT has taken the time to really understand what the team needs to optimize its relationship with Google and to ensure that it makes the most of everything Google Cloud has to offer. “TWT is always responsive to our questions and takes time to address our issues. It is integral to our collaboration with Google, plus it provides us with VAT invoices that comply with German payment regulations,” she says.
“We wanted to tailor all our hardware to our specific needs, so that we could be flexible in how we start, stop, and monitor our research experiments and Compute Engine and Cloud GPUs were key to this.”
—Michael Heck, Postdoctoral Researcher, Dialog Systems and Machine Learning Group,Heinrich Heine University DüsseldorfCreating bespoke virtual machines
Having signed a tailored contract with Google Cloud, the Dialog Systems and Machine Learning Group set about using this to create a bespoke platform for conducting its research. Although there are systems in place that will automatically run and process machine learning, to undertake their highly specialized research, Gašić and her team of postdocs and Ph.D. students wanted to create their own customized virtual machines (VM). Michael Heck, a postdoctoral researcher working with the group on looking at how dialog systems interact with semantics, explains why: “We wanted to tailor all our hardware to our specific needs, so that we could be flexible in how we start, stop, and monitor our research experiments and Compute Engine and Cloud GPUs were key to this.”
When creating machine learning algorithms, the team runs a series of experiments to see which work and which don’t. Compute Engine ensures that they get those results fast, and it also enables them to create a grid search to find an optimal set of parameters for each of its applications. “Compute Engine enables us to have several VMs running in parallel, rather than one VM, running many systems. This helps us to fine-tune and optimize our processes,” says Heck.
Meanwhile, to manage data and remain cost-effective, the team uses Cloud GPUs, opting, for example, for one V100 GPU that lets them input a lot of data simultaneously to speed up time-critical experiments, such as the one the team conducted for evaluating user simulators. The team wanted to compare eight user simulators. Each user simulator is used to train a dialogue policy. One policy training takes 10 hours, which meant the team needed guaranteed immediate access to 40 GPUs to finish in the allotted 10 hours. Locally, each team member has guaranteed access to three GPUs, which means that the whole experiment can take up to one week. In this case, the team was less than 24 hours away from an important publication deadline, so they ran these experiments on Google Cloud and they were completed overnight.
Typically, one VM takes care of one job and one job requires one GPU. To save the time and tedium of doing manual inputting, the gsutil tool is then used to migrate data to and from local machines and to and from Google Cloud storage buckets.
“We use a custom-coded script suite to create VM groups based on custom machine templates,” says Heck. “These job scripts are designed so that any data migration to and from the bucket is automatized and does not require manual synchronization. This way we can keep our data in a central secure place, as after job completion all files are sent automatically to the bucket, and we synchronize data from there to local backup. VMs that process a job shut down automatically after job completion, when an error is thrown, or after a customizable time-out.”
Enabling a hybrid on-premises cloud solution
Some of the group’s work is still done using on-premises machines, but now the team has the flexibility to move between local systems and the cloud. “Typically, when our research is in its infancy or if we’re not under time pressure, we begin work on a local machine, debugging code, for example. When we get to the stage when we need to test to see if we can solve the problems that we're trying to solve and we need to run a lot of experiments concurrently, we often move over to Google Cloud,” explains Gašić. “This is particularly relevant close to deadlines or during revision phases for publications that require additional experiments in a limited time span.”
Having the flexibility and scalable memory capacity of the cloud means that the group can now run large-scale experiments within a 24-hour time frame, compared to two to three days with the on-premises machines. It also enables it to operate a grid search by utilizing large quantities of GPUs, up to 40, in parallel, avoiding the bottleneck caused by the limited number of readily available GPUs, up to 20, on the on-premises setup. And, while on-premises systems are occasionally taken down for maintenance, this is not an issue with the cloud.
“Today's deployed dialog systems rely on handcrafted rules and pre-defined domain scopes limited to what the system can talk about. With the support of Google Cloud, I envisage a time when we can dynamically expand NLP services based on utilizing large unstructured knowledge bases using machine learning.”
—Professor Milica Gašić, Chair of Dialog Systems and Machine Learning, Heinrich Heine University DüsseldorfOptimizing machine learning
Google Cloud has transformed the speed at which researchers can develop their machine learning models. It vastly surpasses the computing power and memory offered by the on-premises architecture, meaning that it can now easily run experiments in parallel and reach conclusions faster. “Let me give a concrete example of this in action,” says Heck. “Dialogue state tracking maintains memory of a dialogue system and as such is an integral part of any dialogue system. A tracking model that we're currently using is built upon large transformers, and it takes several kinds of input at the same time, which it encodes and then continues processing. Training to encode several types of information requires a lot of memory, due to the large amounts of parameter updates that are required. Locally, only a few specialized cards meet our memory requirements, and they are in high demand. Timely experiments are made possible because we have undelayed access to cards on Google Cloud with such large memory. Without this ability, we would have to process each type of information sequentially, which would slow down the whole training process.”
Now that the Dialog Systems and Machine Learning Group can undertake its cutting-edge machine learning research so much faster and at a scale not possible with its previous setup, its findings will be applied to real-world scenarios faster than expected. As a result, Gašić and her team are moving increasingly closer to achieving their dream of creating an intelligent machine that humans can talk to and be made to feel that they are communicating with another person and not a machine.
“Today's deployed dialog systems rely on handcrafted rules and predefined domain scopes limiting what the system can talk about. With the support of Google Cloud, I envisage a time when we can dynamically expand NLP services based on large unstructured sources of knowledge using machine learning,“ concludes Gašić.
Tell us your challenge. We're here to help.
Contact us
About Dialog Systems and Machine Learning Group at Heinrich Heine University Düsseldorf
The Dialog Systems and Machine Learning Group researches Natural Language Processing to aid the development of the next generation of intelligent conversational agents.
About TWT
Düsseldorf-based TWT is a Google Cloud partner specializing in developing machine learning applications.