Test Cases in Dialogflow CX
Romin Irani
Developer Advocate
As a developer, you want to ensure that as you develop new features for your Dialogflow CX agent, you want to ensure that it passes a fixed number of Test Cases that you have marked as Golden Test Cases. Dialogflow CX not only provides you with a feature to manage a list of Test Cases that you can run against your agent but also understand Test coverage.
This blog post will take a sample Dialogflow Agent and will highlight how you can get started today with not just testing the agent but capturing those test runs as golden test cases, building up a test suite and also understanding test coverage.
Sample Agent : ABC Holiday Resorts
We will utilize an existing Agent that has been made available for download here. Go ahead and restore the sample Agent (ABC Holiday Resorts) into a sample Agent project following the steps given over here. Once you have restored the agent, you should be able to see the sample flow that is shown below:
The Sample agent has a single flow and the agent currently supports users in three different areas:
Get Information on current offers/deals
Get Information on current cities in which resorts are present
Reserve a vacation in one of the resorts
Given that the objective here is to highlight the Test Cases feature of Dialogflow, the Agent has been kept simple and does not have any external integrations and complex validations.
You can try out the Agent by using the Test Agent feature in Dialogflow CX console with the following phrases:
What are the current offers?
Where are the resorts?
Book a vacation
Test Cases and Coverage
The first up step is to visit the Test Cases feature in Dialogflow CX console. Go to the Manage tab and then the Test Cases section as shown below:
Since we do not have any Test Cases provided as part of the initial restore of the Agent and the fact that we have not captured any so far, the list is empty.
Click on Coverage and you will see the coverage for various Flows, Intents, Route Groups in the Agent. This is also currently not there since we do not have any Test Cases.
Capturing a Test Case
We can capture a Test Case via the Test Agent feature in the console. The steps to capture a Test Case is given below:
Assuming that your Agent is open in the Dialogflow CX console, click on the Test Agent button.
Provide the utterances as you go through the flow.
A sample test case is shown below. This exercises the option of asking the Agent about current offers/deals.
Once the flow is complete, click on the Save button.
This will ask you for the Test Cases, which we can provide as shown below and finally click on the Save button. This will save the Test Case.
Viewing Test Cases and Coverage
You can view the current list of Test Cases, via the Test Cases link in the Manage Tab, as shown below:
There are a couple of Test Cases that we added and you can also check the Coverage by clicking on the Coverage option in the screen above.
The Transition coverage is shown above and you can see the value for Yes for the two test cases that we captured. Additionally, you can also see the Intents coverage via the Intents option above. The screen is shown below:
You should go ahead and try out other flows and scenarios via the Test Agent feature. Capture those Test Cases and take a look at the overall coverage. This will ensure that you are testing the transitions and intents and if there are newer versions of the Agent, you can ensure that there is no regression by running these Test Cases at any time via the Run option.
Additionally, if you would like to look at Test Cases in detail, we suggest looking at the prebuilt Agents that are available as part of Dialogflow CX. Import one of the pre-built Agents like Travel : Car Rental Agent or Travel : Baggage Claim Agent and visit the Test Cases.
This concludes our overview of Test Cases in Dialogflow CX. Feel free to reach out to me on Twitter @iRomin for any questions or feedback. Do check out the references in the next section for learning more about Dialogflow CX and Test Cases.
