In this post we’ll go over a few different strategies to manage user feedback for your bots. Instead of starting from the beginning with a new bot project, we’ll be continuing off of the demo QnA bot we created in a previous post – QnA Maker with rich cards in .NET. The demo we started from that post is not required, but it provides a good stepping stone into a great bot framework feature we’d like to showcase – Suggested Actions.
Suggested Actions can help address two challenges which often come up when designing a user experience, which are also challenges which extend to bot design:
- Ease of use
- Effectively delivering the right content to the user
These two challenges are also not mutually exclusive – a bot which is simple to use can also deliver content that a user didn’t need, or wasn’t looking for – making for a frustrating experience.
As the documentation states – “Suggested actions enable your bot to present buttons that the user can tap to provide input.” These buttons can be used to enable a user to quickly answer a question, make selections, or trigger other bot actions without needing to typing everything out in a new message to the bot. While this functionality could be handled via a card attachment, card attachments remain visible and persist, whereas a suggested action will disappear after it has been used. A great example of suggested actions is showcased with one of our local business bots –
In this restaurant scenario, we simply typed in ‘menu’. Rather than follow complicated dialog paths, a link to the restaurant’s food menu was served. Below the link to the menu are two Suggested Actions – ‘Vegetarian Options‘ and ‘Gluten-Free Option‘. Even in person with a traditional physical menu, if a user was only interested in a certain subset of the options available, they would potentially still need to look through a large portion of the menu before finding it. Using suggested actions in this user scenario, the bot is able to immediately display a subset of menu options for the user to select. Reflecting on the two challenges we listed above – easy to use, and effective content delivery, this feature is able to satisfy both.
Alright, now that we’ve introduced Suggested Actions for bots, let’s dive into the implementation!
Suggested Actions for user feedback
As mentioned above, this demo continues from the demo bot from a previous post, you can find the sample code on GitHub here. The demo bot leverages the QnA maker service, which we’ll be building on by adding a ‘thumbs up’ / ‘thumbs down’ user feedback option. As a practical use case, this will allow users to provide some sort of feedback to how your QnA service is performing, (i.e – Was the user’s question answered?) which you can use to gauge how to train that service over time. Lastly, we’ll integrate Application Insights to the bot so that this feedback can be stored and viewed.
Adding a Feedback Dialog
First, we’ll start by adding a new dialog to the bot project which we’ll call FeedbackDialog:
[Serializable]
public class FeedbackDialog : IDialog<IMessageActivity>
{
private string qnaURL;
private string userQuestion;
public FeedbackDialog(string url, string question)
{
// keep track of data associated with feedback
qnaURL = url;
userQuestion = question;
}
public async Task StartAsync(IDialogContext context)
{
context.Done<IMessageActivity>(null);
}
}
In this new dialog, we’ll add two string properties in the constructor, qnaURL and userQuestion which will be used to keep track of the data we want to associate with our feedback. Our goal is to take the user’s question, along with some data property the service responded with, and associate it with a positive or negative feedback.
In the previous demo, we created a dialog called QnaDialog which inherits the QnAMakerDialog class from the Bot.Builder.CognitiveServices package installed, shown below:
using System.Runtime.CompilerServices;
using System.Threading.Tasks;
using Microsoft.Bot.Builder.Dialogs;
using Microsoft.Bot.Connector;
namespace Microsoft.Bot.Builder.CognitiveServices.QnAMaker
{
// Summary:
// A dialog specialized to handle QnA response from QnA Maker.
public class QnAMakerDialog : IDialog<IMessageActivity>
{
protected readonly IQnAService[] services;
public QnAMakerDialog(params IQnAService[] services);
public IQnAService[] MakeServicesFromAttributes();
public Task MessageReceivedAsync(IDialogContext context, IAwaitable<IMessageActivity> argument);
protected virtual Task DefaultWaitNextMessageAsync(IDialogContext context, IMessageActivity message, QnAMakerResults result);
protected virtual bool IsConfidentAnswer(QnAMakerResults qnaMakerResults);
protected virtual Task QnAFeedbackStepAsync(IDialogContext context, QnAMakerResults qnaMakerResults);
protected virtual Task RespondFromQnAMakerResultAsync(IDialogContext context, IMessageActivity message, QnAMakerResults result);
}
}
We created an override for the RespondFromQnAMakerResultAsync method in order to intercept the response from the QnA service, and format it to render as a card to the user. This time, we need to create an override for DefaultWaitNextMessageAsync:
protected override async Task DefaultWaitNextMessageAsync(IDialogContext context, IMessageActivity message, QnAMakerResults result)
{
// get the URL
var answer = result.Answers.First().Answer;
string[] qnaAnswerData = answer.Split(';');
string qnaURL = qnaAnswerData[2];
// pass user's question
var userQuestion = (context.Activity as Activity).Text;
context.Call(new FeedbackDialog(qnaURL, userQuestion), ResumeAfterFeedback);
}
Implementing this override allows us to route the conversation to a new dialog after the QnA service responds to a user’s question. Taking the user’s question and URL property from the QnA service response, we add the new FeedbackDialog we created to the top of the dialog stack using context.Call().
Note: This override is only necessary since we’re still building with the QnA service. For custom dialogs, you can simply use context.Call() and pass whatever relevant data/properties are necessary for your bot
Adding Suggested Actions
In FeedbackDialog, we’ll add code to the StartAsync method to implement Suggested Actions as follows:
public async Task StartAsync(IDialogContext context)
{
var feedback = ((Activity)context.Activity).CreateReply("Did you find what you need?");
feedback.SuggestedActions = new SuggestedActions()
{
Actions = new List<CardAction>()
{
new CardAction(){ Title = "👍", Type=ActionTypes.PostBack, Value=$"yes-positive-feedback" },
new CardAction(){ Title = "👎", Type=ActionTypes.PostBack, Value=$"no-negative-feedback" }
}
};
await context.PostAsync(feedback);
context.Wait(this.MessageReceivedAsync);
}
In the code above we:
- Created a new Activity object which will prompt the user with a question – “Did you find what you need?” when the dialog starts
- Add a new instance of SuggestedActions() to our Activity object
- Defined new actions for our ‘thumbs up/thumbs down’ feature
You may notice that the SuggestedActions class includes the same property for ‘Actions’ just like rich cards. Before the Activity is posted back to the user, we call one more async Task method, which we define as MessageReceivedAsync, implemented per the following:
public async Task MessageReceivedAsync(IDialogContext context, IAwaitable<IMessageActivity> result)
{
var userFeedback = await result;
if(userFeedback.Text.Contains("yes-positive-feedback") || userFeedback.Text.Contains("no-negative-feedback"))
{
if(userFeedback.Text.Contains("yes-positive-feedback"))
{
// post positive feedback to DB...
}
else if(userFeedback.Text.Contains("no-negative-feedback"))
{
// post negative feedback to DB...
}
await context.PostAsync("Thanks for your feedback!");
context.Done<IMessageActivity>(null);
}
else
{
// no feedback, return to QnA dialog
context.Done<IMessageActivity>(userFeedback);
}
}
From here, you can simply post the user’s feedback to whatever database you’d like! Once this is done, we use context.Done() to resolve the current dialog (FeedbackDialog) on the stack , and return to the parent dialog – QnADialog. Let’s take a look at what this looks like now by starting up the Bot Framework Emulator –
While posting feedback to a database is a valid option, is it really necessary? After all, that database would require maintenance, and upkeep. Using a relational database like SQL, now you’ll need provision additional queries to view your data. Keeping those in mind, let’s look at a newer alternative way to do this.
Adding Application Insights
Click Here for a detailed step-by-step to add Application Insights for .NET applications. To enable Application Insights for our bot, we simply right click on the bot project, select ‘Add’ —> ‘Application Insights Telemetry…‘and follow the prompts to register the bot application.
When completed, you’ll automatically be re-directed to the Azure portal where you’ll see the dashboard for your newly deployed service, which you can access at any time from Visual Studio by clicking on ‘overview’ under ApplicationInsights.config.
Application Insights is now configured and ready, the last thing to do is to add the code to our bot necessary to track our user feedback. Back in FeedbackDialog, we have one dependency to add –
using Microsoft.ApplicationInsights;
In MessageReceivedAsync, we create a new TelemetryClient which will connect the bot to application insights. To post our feedback, we can create a new Dictionary to store whatever properties we’d like to monitor, in this cause – userQuestion, QnA data, and the user’s Vote. Finally, we use the TrackEvent method to post the information to App Insights.
public async Task MessageReceivedAsync(IDialogContext context, IAwaitable<IMessageActivity> result)
{
var userFeedback = await result;
if (userFeedback.Text.Contains("yes-positive-feedback") || userFeedback.Text.Contains("no-negative-feedback"))
{
// create telemetry client to post to Application Insights
TelemetryClient telemetry = new TelemetryClient();
if (userFeedback.Text.Contains("yes-positive-feedback"))
{
// post feedback to App Insights
var properties = new Dictionary<string, string>
{
{"Question", userQuestion },
{"URL", qnaURL },
{"Vote", "Yes" }
// add properties relevant to your bot
};
telemetry.TrackEvent("Yes-Vote", properties);
}
else if (userFeedback.Text.Contains("no-negative-feedback"))
{
// post feedback to App Insights
}
...
Back in the Application Insights dashboard, we can verify right away that the user feedback for the bot was successfully captured! You can easily configure the Metrics Explorer to view not only custom events like our user feedback, but right out of the box you get rich performance monitoring, interactive data analytics, and diagnostics to monitor your application.
The properties we added to be tracked will also be automatically added as options to filter your data! In addition to customizing the data you wish to track, you can simply pin the view to your Azure dashboard and have it readily available as soon as you login to the portal – no database queries! With this setup, we can see what types of questions users are asking our QnA service, which responses are helpful or not, and ultimately whether the users have a positive or negative experience, and as the developer, adjust your bot accordingly.
Summary
In this post, we talked about Suggested Actions, an option in the BotBuilder SDK intended to help provide richer bot experiences. We picked a user feedback scenario to implement suggested actions, based on a QnA service we created in this previous post, and briefly went over one way to maintain and monitor that feedback using Application Insights on Azure.
Click Here to view the sample code on GitHub.
Happy Making!
Matthew Shim from the Bot Framework Team