Since LUIS became generally available last year several new features have been added to make it easier to create and train robust natural language models for your applications. If you’ve used LUIS in the past, hopefully you’re pretty familiar with intents and entities, and how to train your model to use them by using utterances. In this post, we’ll go over a scenario that several customers have been struggling with, and how we used the new Patterns feature to solve it.
Scenario
Recently, a customer needed help with the following scenario. They were building a chat bot to use for a traveling aid, using LUIS to extract different location entities for their app. Here’s are two examples of a user utterance one might have, for an intent we’ll call GetDistance for a travel application:
“How far away is Seattle?”
“What is the distance to Seattle?”
If an app has between 10 and 20 utterances with different lengths of sentence, different word order, and even different words (using synonyms), LUIS may return a lower intent score than intended. In each of the sample utterances above, we only used one location (Seattle). In a travel application, we might want to add another feature involving two locations, let’s say we added another intent and called it GetDistanceFromAtoB, with a sample utterance of :
“How far is it from Tacoma to Portland?”
There are many cities, and many ways you can express the need to book a trip from one location to the next, and we can’t easily predict which intent might get resolved to the top intent, as they both use similar words and entities. Should we simply keep adding user utterances and labeling each city manually? Also, it seems rather tedious to add a new similar utterance just to label the same entity for different city names over and over again.
Thankfully, with the new Patterns feature we now have a new tool to not only increase the accuracy of our language model, but we can also use it to decrease the volume of repetitive utterances we may have needed to train our model with previously.
Patterns
Patterns are a new feature in LUIS which help you provide better targeting for intent and entity recognition for your language models. In essence, they allow you to label entities which follow a specific pattern, so that you don’t need to continuously add more examples as your application grows. Patterns are available for all LUIS applications under the ‘Improve app performance‘ section on the left menu under the BUILD tab. Moving forward, we’ve already created a LUIS app called TravelApp, and added the two intents from above (GetDistance, GetDistanceFromAtoB).
To solve for our travel app scenario, let’s create a new entity called location, and select Pattern.any as the entity type.
Note that the window includes a note it states, “a pattern entity is a variable-length placeholder used only in a pattern’s template utterance to mark where the entity begins and ends.” What does this mean? Well, within the recognized pattern, you can add an entity of any length without it affecting the accuracy of your model. For example:
Locations in the pattern: “What’s the distance to { location } ” are all recognized in bold:
- What’s the distance to the edge of the universe
- What’s the distance to the second largest city in the state of Illinois
- What’s the distance to Tacoma
Thanks to the Pattern.any entity, LUIS knows where the name of the location entity starts and ends. Now let’s actually add this pattern to our LUIS app. In the Patterns view, we select our intent of GetDistance from the dropdown, and enter a sample pattern, wrapping entities with curly { } braces.
Note: You might find that adding patterns is very similar to adding utterances to intents, and tagging the entities!
We are only going to add this one pattern, remember – this feature is supposed to help decrease how many examples we need to provide. After we train our model, we’re ready to test if our new pattern works. It is important to note that we haven’t provided our model any data regarding what our entity should map to, we haven’t provided any specific city names or real locations at all. Selecting Test on the upper right of the dashboard, let’s pass in a sample utterance for some location – “how far is it to santa fe?”
Despite having no knowledge of what ‘santa fe’ is, LUIS not only returned the proper intent (GetDistance), but with the maximum possible score, in addition to recognizing ‘santa fe‘ as a location entity. Neat right?
Entity Roles
The other scenario we needed to solve for was the intent GetDistanceFromAtoB, recall from above that the sample utterance we decided on was:
“How far is it from Tacoma to Portland?”
This phrase includes two location entities, and the problem presented is – how can we maintain the contextual information for an entity, when multiple entities of the same type are recognized in the same phrase? LUIS patterns also introduces a new easy to use feature called Roles. Roles can only be used for patterns, and are added to provide contextual information for entities of the Pattern.any type. From our utterance above, LUIS would have no way of providing the context that we seek something from a specific start location (Tacoma), to and end location (Portland). Roles will solve that for us – selecting our location pattern.any entity, we can simply add the names for the contextual roles we want to keep track of, in this case the ‘origin‘ and ‘destination‘.
After that, we head back to our Patterns view to add a new pattern for the intent GetDistanceFromAtoB. Adding a role uses the format ` { Entity : Role } `. To create a pattern to satisfy the utterance “How far is it from Tacoma to Portland?”, the pattern we input becomes:
How far is it from { location:origin } to { location:destination }
Once again training our model, we open up the test panel to verify that our new pattern works. This time, our response returned two location entities, each with a specific role.
In the raw JSON response from this query, you’ll notice that the contextual information stored as roles for our two location entities are provided as easy to consume key-value pairs for your applications. Prior to patterns with roles, LUIS would respond with a list of entities which included little to no contextual information which required much more complex application logic to consume the information.
{
"query": "how far is it from toronto to montreal?",
"topScoringIntent": {
"intent": "GetDistanceFromAtoB",
"score": 0.999993861
},
"intents": [
{
"intent": "GetDistanceFromAtoB",
"score": 0.999993861
},
{
"intent": "GetDistance",
"score": 0.00000469839051
},
{
"intent": "None",
"score": 0.00000163265042
}
],
"entities": [
{
"entity": "toronto",
"type": "location",
"startIndex": 19,
"endIndex": 25,
"role": "origin"
},
{
"entity": "montreal",
"type": "location",
"startIndex": 30,
"endIndex": 37,
"role": "destination"
}
]
}
Neat right?
Moving Forward
We hope that this has been a good guide through one of the most exciting new feature in LUIS, and we hope that patterns provides a quick and easy solutions for your natural language needs. For more powerful new features in LUIS, we’d recommend you check out Phrase Lists, another new feature aimed to improve performance for your language models.
Until next time –
Happy Coding!
Matthew Shim from the Azure Bot Service Team