Luis Action Binding for Web Apps

In a previous post, we announced that action binding from the LUIS Service has been deprecated and must now be handled on the client side of an application, which allows you to have stricter control over conversational flow instead of relying on LUIS to do the work. In the first article, we discussed how to implement action binding for a bot using the Microsoft Bot Framework SDK. In this article, we will discuss how to use action binding for a Web application, in both ASP.NET MVC and NodeJS.

Using LUIS Action Bindings for a Web Project (C#)

The following example will discuss how action binding can be used in an ASP.NET MVC application.

NOTE: Client side LUIS Actions are agnostic from application. See LuisActions.Samples.

The key class to examine is the HomeController and its POST Index handler where it receives a custom model containing the user query, along with the action instance resolved for the user’s intent. This method is ‘re-entrant’, as it will be called until the action instance for the intent is valid. If the intent has not been obtained yet, i.e., at the first POST call, the method performs a call to LUIS service to resolve it.

[HttpPost]
public async Task<ActionResult> Index(QueryViewModel model)
{
    if (!model.HasIntent)
    {
        var luisService = new LuisService(new
        LuisModelAttribute(ConfigurationManager.AppSettings["LUIS_ModelId"], ConfigurationManager.AppSettings["LUIS_SubscriptionKey"]));
        var luisResult = await luisService.QueryAsync(model.Query, CancellationToken.None);
        var resolver = new LuisActionResolver(typeof(GetTimeInPlaceAction).Assembly);
        var action = resolver.ResolveActionFromLuisIntent(luisResult);
        model.LuisAction = action;

        // TODO: this is dangerous. This should be stored somewhere else, not in the client, or at least encrypted
        model.LuisActionType = action.GetType().AssemblyQualifiedName;
    }

    ModelState.Clear();
    var isValid = TryValidateModel(model.LuisAction);
    if (isValid)
    {
        // fulfill
        var actionResult = await model.LuisAction.FulfillAsync();
        return this.View("ActionFulfill", actionResult);
    }
    else
    {
        // not valid, continue to present form with missing/invalid parameters
        return this.View(model);
    }
}

NOTE: The sample web application uses the ASP.NET Razor view engine.

Let’s walk through the steps of the code sample:

1) The QueryViewModel is passed into the async POST method (Index). This contains a custom model that defines the user query.

2) The Index POST method checks if the model provided has an intent (bool HasIntent). If it’s false, meaning there is no intent yet, then the POST method will handle the appropriate API call to the LUIS service to retrieve an intent.

3) Create a new LuisService object, native to the Microsoft.Bot.Builder.Luis namespace, passing in the Model ID and Luis API key, and referencing the object in luisService.

4) Invoke the QueryAsync method for the new luisService, passing in the user’s query model and providing the reference in luisResult.

5) Create a new LuisActionResolver object with a type of assembly for the action GetTimeInPlaceAction, and create a reference resolver.

6) The resolver object is passed the luisResult when the call is completed. It uses the ResolveActionFromLuisIntent to determine the action to bind.

Assuming a valid model is submitted and all validations pass, the execution proceeds with the action fulfillment to update the page view. If validation fails, the same view will be returned (re-entrant calls) until the action model can be validated.

The helper class LuisActionModelBinder shown below is where an existing action (in memory) instance is ‘hydrated’ back from the model exchanged with the view in order to be available at the controller’s method when checking for its valid state.

public class LuisActionModelBinder : DefaultModelBinder
{
    public override object BindModel(ControllerContext controllerContext, ModelBindingContext context)
    {
        HttpRequestBase request = controllerContext.HttpContext.Request;
        var type = request.Form["LuisActionType"];
        if (!string.IsNullOrWhiteSpace(type))
        {
            var action = Activator.CreateInstance(Type.GetType(type));

            Func<object> modelAccessor = () => action;
            context.ModelMetadata = new ModelMetadata(
                new DataAnnotationsModelMetadataProvider(),
                context.ModelMetadata.ContainerType,
                modelAccessor,
                action.GetType(),
                context.ModelName);

            return base.BindModel(controllerContext, context);
        }

        return null;
    }
}

Using LUIS Action Bindings in a Web Application (NodeJS)

The Node.js sample code uses an Express web server.

NOTE: In the web directory from the terminal use the command npm start to run the sample web app which defaults to http://localhost3000

NOTE: The Actions used for the sample Web app imported as shown here are located two file directories above. You may notice that the Actions are agnostic with respect to which type of application you are developing (Bot/Web/Console)

Key things to examine are the Index Route and View. The GET / route returns the index view on initial load. Once the user submits a query, the POST route submits the user’s query. In this sample, it goes to the same address, index.

When the post is submitted, the tryEvaluate method is called, passing in the parameters corresponding to the user’s query per the following:

function tryEvaluate(actionModel, query, formPost) {
  if (actionModel) {
    // Populate action parameters from form data
    var action = SampleActions.find(a => a.intentName === actionModel.intentName);
    actionModel.parameters = _.pick(formPost, _.keys(action.schema));
    return LuisActions.evaluate(ModelUrl, SampleActions, actionModel);
  } else {
    return LuisActions.evaluate(ModelUrl, SampleActions, null, query);
  }
}

The tryEvaluate method acts as a wrapper for LuisAction.evaluate, from the Action Binding framework. The evaluate method returns a Promise which resolves the actionModel. The returned object includes a status field. See Status definition from the framework.

  • Status.NoActionRecognized

    No intent was detected or no matching action was found, Re-route back to the index page.

  • Status.MissingParameters

    An action was matched, but there are missing or invalid parameters. Proceed to display a form with input fields for each parameter. The input fields are built using the createFieldsViewModel helper function, along with the action (parameters) schema, the model parameter, and its errors.

  • Status.Fulfilled

    An action was matched, its parameters were validated and the action was fulfilled. The actionModel contains a result field with the action’s result. Proceed to display the fulfill view that prints the result.

Summary

The first article in this series covered the mechanisms of the Action Binding framework and how to implement Action Binding on the client in a bot. This article went over similar concepts for an ASP.NET MVC Web app in C# and a Node.js Web app using Express. In a future article, we will conclude with how to implement Binding Action in Console applications.

Happy Making!

Pablo Constantini, Ezequiel Jadib, and Matthew Shim from the Bot Framework team