Cards today
One of the many new Bot Framework features we showcased at Build 2017 was Adaptive Cards. Modern ‘cards’ as a UI feature have existed for several years now, birthed by the advent of the modern web and mobile technologies. Most of us have interacted with cards on some level, whether or not we realize it – a card is a UI container for short, grouped or related pieces of information.Consider the following examples of everyday card use:
Cards are used everywhere! Currently, large and popular platforms like twitter and Facebook provide templates for developers to create cards from. Using templates, large platforms like Facebook can tightly control third party content to ensure that users get a consistent experience (i.e Facebook always looks like Facebook), and content authors only need to decorate the meta-data in their content once.
Current card limitations
While this current way of creating cards is simple and effective, there are certainly limitations to using templates.
- For the host application, let’s say facebook, every template must be designed, implemented and documented on every platform
- For the content author, you are limited to the types of content the templating will allow, and you have no ability to customize the templates
As an alternative to the rigid nature of the template model, on the other end of the spectrum we can create cards using HTML and CSS, which of course includes the following limitations as well:
- Performance and security costs with mixing UI stacks
- Difficult to enforce consistency, and restrict what card authors can do
- Content authors must design and implement every style of card across each platform
Introducing Adaptive Cards
So, what’s the deal with adaptive cards and why are they significant? Adaptive cards provides a nice middle-ground solution between using fixed templates and HTML.
They can also improve cards by adding the following features:
- card automatically adapts to host U/X and brand guidelines, using a json payload
- schema to layout content
- actions to show other cards, urls, etc
- input to gather info from users
- speech enabled from day one
You can click here to check out the in-depth presentation from Build 2017 where we introduced Adaptive cards and it’s key features.
Demo – adding Adaptive cards to a Bot
In this example, we’ll show you how to create an Adaptive Card to display weather information by using the LUIS Action Bindings Sample’s GetWeatherInPlace Action. We will walk through setting up the LUIS Model,creating the solution, and modifying the code to display a weather card using data retrieved from the APIXU weather service.
Prerequisites:
- Free luis.ai account
- Bot Framework Emulator
- Free APIXU account
- Visual Studio 2017 or 2015 Community (update all extensions)
LUIS model setup
1) Create a new LUIS app, naming it whatever you like.
2) Click on intents on the left menu to add an Intent named WeatherInPlace. This is the intent we’ll be using from the LUIS service to bind to an action defined in the client.
Add utterances for the intent, in our model we added:
"what is the weather like in hong kong"
"what is the weather like in mexico city"
"tell me the weather in jerusalem"
"what is the weather in miami, fl"
"what temperature is it in paris, france"
"weather in dallas, texas"
"what is the weather in seattle, wa"
3) Select entities from the left menu, and add the prebuilt geography entity to the model.
4) Create a custom entity with the name Place and for Entity type select Composite. Now click Add child and select geography from the dropdown.
5) Add a phrase list for weather synonyms. Try adding ““weather”, “forecast”, “temperature”, “warm”, and “rain”. This will enable asking more varied questions, such as: “what is the temperature in Miami, FL?” or “How warm is it in Philadelphia, PA?”
6) Finally, retrieve the LUIS subscription id which can be found under my keys in the top menu, and application id which can be found under then dashboard section of the left menu. You’ll need these keys to allow your Bot to access the LUIS service.
7) Now Train and Publish the model.
The exported model should be similar to something like this:
{
"luis_schema_version": "2.1.0",
"versionId": "0.1",
"name": "WeatherInPlace",
"desc": "",
"culture": "en-us",
"intents": [
{
"name": "None"
},
{
"name": "WeatherInPlace"
}
],
"entities": [],
"composites": [
{
"name": "Place",
"children": [
"geography"
]
}
],
"closedLists": [],
"bing_entities": [
"geography"
],
"actions": [],
"model_features": [
{
"name": "Weather synonyms",
"mode": true,
"words": "weather,climate,rain,winter,temperature,forecast,warm,hot,rainy,cold,colder,snow",
"activated": true
}
],
"regex_features": [],
"utterances": [
{
"text": "what is the weather like in hong kong",
"intent": "WeatherInPlace",
"entities": [
{
"entity": "Place",
"startPos": 28,
"endPos": 36
}
]
},
{
"text": "what is the weather like in mexico city",
"intent": "WeatherInPlace",
"entities": [
{
"entity": "Place",
"startPos": 28,
"endPos": 38
}
]
},
{
"text": "tell me the weather in jerusalem",
"intent": "WeatherInPlace",
"entities": [
{
"entity": "Place",
"startPos": 23,
"endPos": 31
}
]
},
{
"text": "what is the weather in miami, fl",
"intent": "WeatherInPlace",
"entities": [
{
"entity": "Place",
"startPos": 23,
"endPos": 31
}
]
},
{
"text": "what temperature is it in paris, france",
"intent": "WeatherInPlace",
"entities": [
{
"entity": "Place",
"startPos": 26,
"endPos": 38
}
]
},
{
"text": "weather in dallas, texas",
"intent": "WeatherInPlace",
"entities": [
{
"entity": "Place",
"startPos": 11,
"endPos": 23
}
]
},
{
"text": "what is the weather in seattle, wa",
"intent": "WeatherInPlace",
"entities": [
{
"entity": "Place",
"startPos": 23,
"endPos": 33
}
]
}
]
}
C# Solution Setup
The Bot Builder Samples and APIXU C# Library are available on GitHub.
1) Use git to pull the Bot Builder Samples
https://github.com/Microsoft/BotBuilder-Samples.git
2) Do the same for the APIXU csharp library. Make sure you obtain the proper API credentials for your account from the APIXU portal.
https://github.com/apixu/apixu-csharp.git
3) Open the LuisActionBinding solution from the \BotBuilder-Samples\CSharp\Blog-LUISActionBinding\ folder.
4) Remove the LuisActions.Samples.Console, .Web and .Nuget projects, as we’re only concerned with building a bot.
5) Ensure that the .Bot project is set as the startup project.
6) Add the APIXU C# library project to the solution. It’s in the \apixu-csharp\APIXULib\ folder. Once added, reference the project in the .Bot and .Shared projects.
7) Add the Adaptive Cards Nuget package to the .Bot and .Shared projects as shown.
8) Update Microsoft.Bot.Builder to version 3.8.1 in nugget package manager. Also restore packages if you have not already.
9) Change the configuration settings in your Bot project’s web.config to be your keys obtained from LUIS and APIXU.
<add key="LUIS_SubscriptionKey" value="LUISSUbscriptionKey" /> <add key="LUIS_ModelId" value="LUISModelId" /> <add key="APIXUKey" value="APIXUKey" />
C# Code
We’ve set up our LUIS model, added the appropriate libraries, and added the necessary authorization settings for our Bot to communicate with the services we need. Now we need to define the action to create our adaptive cards.
Next, open the GetWeatherInPlaceAction.cs file in the .Shared project and change it to the following:
namespace LuisActions.Samples
{
using System;
using System.Threading.Tasks;
using Microsoft.Cognitive.LUIS.ActionBinding;
using AdaptiveCards;
using APIXULib;
using System.Configuration;
[Serializable]
[LuisActionBinding("WeatherInPlace", FriendlyName = "Get the Weather in a location")]
public class GetWeatherInPlaceAction : GetDataFromPlaceBaseAction
{
public override Task<object> FulfillAsync()
{
var result = GetCard(this.Place);
return Task.FromResult((object)result);
}
private static AdaptiveCard GetCard(string place)
{
WeatherModel model = new Repository().GetWeatherData(ConfigurationManager.AppSettings["APIXUKey"], GetBy.CityName, place, Days.Five);
// creates Adaptive Card
var card = new AdaptiveCard();
if (model != null)
{
if (model.current != null)
{
card.Speak = $"<s>Today the temperature is {model.current.temp_f}</s><s>Winds are {model.current.wind_mph} miles per hour from the {model.current.wind_dir}</s>";
}
if (model.forecast != null && model.forecast.forecastday != null)
{
AddCurrentWeather(model, card);
AddForecast(place, model, card);
return card;
}
}
return null;
}
private static void AddCurrentWeather(WeatherModel model, AdaptiveCard card)
{
var current = new ColumnSet();
card.Body.Add(current);
var currentColumn = new Column();
current.Columns.Add(currentColumn);
currentColumn.Size = "35";
var currentImage = new Image();
currentColumn.Items.Add(currentImage);
currentImage.Url = GetIconUrl(model.current.condition.icon);
var currentColumn2 = new Column();
current.Columns.Add(currentColumn2);
currentColumn2.Size = "65";
string date = DateTime.Parse(model.current.last_updated).DayOfWeek.ToString();
AddTextBlock(currentColumn2, $"{model.location.name} ({date})", TextSize.Large, false);
AddTextBlock(currentColumn2, $"{model.current.temp_f.ToString().Split('.')[0]}° F", TextSize.Large);
AddTextBlock(currentColumn2, $"{model.current.condition.text}", TextSize.Medium);
AddTextBlock(currentColumn2, $"Winds {model.current.wind_mph} mph {model.current.wind_dir}", TextSize.Medium);
}
private static void AddForecast(string place, WeatherModel model, AdaptiveCard card)
{
var forecast = new ColumnSet();
card.Body.Add(forecast);
foreach (var day in model.forecast.forecastday)
{
if (DateTime.Parse(day.date).DayOfWeek != DateTime.Parse(model.current.last_updated).DayOfWeek)
{
var column = new Column();
AddForcastColumn(forecast, column, place);
AddTextBlock(column, DateTimeOffset.Parse(day.date).DayOfWeek.ToString().Substring(0, 3), TextSize.Medium);
AddImageColumn(day, column);
AddTextBlock(column, $"{day.day.mintemp_f.ToString().Split('.')[0]}/{day.day.maxtemp_f.ToString().Split('.')[0]}", TextSize.Medium);
}
}
}
private static void AddImageColumn(Forecastday day, Column column)
{
var image = new Image();
image.Size = ImageSize.Auto;
image.Url = GetIconUrl(day.day.condition.icon);
column.Items.Add(image);
}
private static string GetIconUrl(string url)
{
if (string.IsNullOrEmpty(url))
return string.Empty;
if (url.StartsWith("http"))
return url;
//some clients do not accept \\
return "https:" + url;
}
private static void AddForcastColumn(ColumnSet forecast, Column column, string place)
{
forecast.Columns.Add(column);
column.Size = "20";
var action = new OpenUrlAction();
action.Url = $"https://www.bing.com/search?q=forecast in {place}";
column.SelectAction = action;
}
private static void AddTextBlock(Column column, string text, TextSize size, bool isSubTitle = true)
{
column.Items.Add(new TextBlock()
{
Text = text,
Size = size,
HorizontalAlignment = HorizontalAlignment.Center,
IsSubtle = isSubTitle,
Separation = SeparationStyle.None
});
}
}
}
Other than GetCard and FulfillAsync, all of the other methods included are just used to customize our adaptive card properties. Under the hood, the adaptive cards library is responsible for determining which type of channel or host app the bot is running, and adapting it accordingly to the host app’s styling.
You can check the adaptive cards schema documentation for a detailed listing of all of the different properties you can use to format an adaptive card.
Open the RootDialog in the .Bot project, and modify the WeatherInPlaceActionHandlerAsyncmethod as follows:
This change allows the bot’s IMessageActivity to handle an adaptive card.
[LuisIntent("WeatherInPlace")]
public async Task WeatherInPlaceActionHandlerAsync(IDialogContext context, object actionResult)
{
IMessageActivity message = null;
var weatherCard = (AdaptiveCards.AdaptiveCard)actionResult;
if (weatherCard == null)
{
message = context.MakeMessage();
message.Text = $"I couldn't find the weather for '{context.Activity.AsMessageActivity().Text}'. Are you sure that's a real city?";
}
else
{
message = GetMessage(context, weatherCard, "Weather card");
}
await context.PostAsync(message);
}
Add the GetMessage method as follows, which takes our newly created card and adds it as an attachment to the message which will be returned to the user:
private IMessageActivity GetMessage(IDialogContext context, AdaptiveCards.AdaptiveCard card, string cardName)
{
var message = context.MakeMessage();
if (message.Attachments == null)
message.Attachments = new List<Attachment>();
var attachment = new Attachment()
{
Content = card,
ContentType = "application/vnd.microsoft.card.adaptive",
Name = cardName
};
message.Attachments.Add(attachment);
return message;
}
Running our bot using the emulator, this is what our adaptive card looks like.
Summary
We’ve now added Adaptive Cards to one of our pre-existing sample bots (LUIS Action Binding sample). The adaptive cards library allows us to provision our bots (and other applications) with scalable components to cover most of the needs of the developer while still maintaining the styling and security of the host application.
You can check out the Adaptive Cards presentation from Build 2017 here.
At the presentation we also made the Adaptive Cards Github repo public. Although we defined the properties we want our card to display, the library is responsible for determining which channel the bot is being accessed on, and scaling those properties accordingly – or should we say – Adaptively!
Happy Making!
Eric Dahlvang and Matthew Shim from the Bot Framework team