Load testing a Bot

Posted on by The Bot Framework Team

UPDATE: 02/23/18 – A sample callback service called MockChannel has been provided in the BotFramework-Samples repo. When creating load tests, you need a service to pass as activity.ServiceURL. When your bot posts back in response to your tests, it will post it back to this mock service.

One of the most common questions and requests we receive from the bot developer community is how developers can perform load testing on a bot. Previously we hadn’t documented this end-to-end, but now we’re providing an authorization scheme to implement load testing for your bots.

This guide includes two sections: the first describes how to authenticate messages to and from your bot (which is the most complex part of constructing a load test) and the second describes how to create and parse those messages.

So what is this auth scheme and why do we need it to perform payload testing? If you’re already familiar with how the bot framework sends/receives message activity to a user, feel free to skip down to creating an access token.

Overview of Authentication in the Bot Framework

If you’re not already familiar, we highly recommend that you click here to review a detailed description of how authentication works within the bot framework.

In short, your bot validates every incoming request made from a Bot Framework Connector. When you create a load test, you must provide a token which will authorize your test to the bot. Without a valid token, your requests won’t be authenticated resulting in an HTTP 403 (Forbidden) error. You cannot create a token that impersonates one of our Connector services, but you can create a special token similar to one that the Bot Framework Emulator uses.

Creating an access token

Create an HTTP POST to –

https://login.microsoftonline.com/botframework.com/oauth2/v2.0/token

Inside the post, you must include the following x-www-form-urlencoded values:

grant_type=client_credentials
client_id= <YOUR MSA APP ID>
client_secret = <YOUR MSA APP SECRET>
scope = <YOUR MSA APP ID>/.default

A successful request will return a JSON payload with an access_token. This token you receive is what you need to pass your bot’s authentication. In production, the bot connector service provides a signed JWT token by for a user’s messages by default.

Creating a mock channel/message sink

UDATE: 02/23/18 – Click here for a sample mock channel service

In order to test your bot, you’ll need to create a custom UI/message sink to send and receive messages to your bot. This message sink will effectively act like a channel and accept HTTP POST messages with JSON-serialized bot framework activities.

Within this channel/message sink you can also define all of the the methods and procedures you need to test your bot.

Send a message to the bot

You can now create an HTTP POST to your bot’s endpoint. In the message, you’ll need to include an authorization header, and pass in the access_token you retrieved earlier:

    POST /messages/v3
    Authorization: Bearer <ACCESS_TOKEN> 
    Content-Type: application/json

Note: Depending on the platform, you may also use Content-Length.

Below the header, your message body must be JSON-serialized and include the following fields:

  • type
  • id
  • channelId
  • conversation/id
  • from/id
  • recipient/id
  • serviceUrl

Your outgoing message schema might look something like this:

{
  "type": "message",
  "id": <GUID>,
  "channelId" : "test", 
  "conversation": { "id": <CONVERSATION ID> },
  "from": { "id": <USER ID> },
  "recipient": { "id": <BOT HANDLE> },
  "serviceUrl": <SEE BELOW> 
}

The serviceUrl property here is critical to note, and needs to be set to the endpoint of your message sink/client. By default, your bot will attempt to submit a response back to the connector service. If you don’t set a serviceUrl your message sink/channel will not receive replies from your bot.

In the above example we sent a message type activity, but you can provide any of the defined activity types you wish. See Activities overview from the docs.

Now that we can pass the bot’s authorization check using the new access token,the bot will attempt to reply to the messages we send. Normally, when a bot tries to reply to a user, the process works like this:

1) Your Bot will issue a GET request to the MSA login service to retrieve a JWT token 2) The MSA login service issues a JWT token back to the bot 3) The Bot uses the JWT token in the Authorization header of the request to the connector service, eventually reaching the user

Note: The bot will still create a request to the MSA login service for a token, so you can test for token authentication from your message sink if you wish

Tips:

  • DO NOT use Direct Line to load test your bot
  • Create traffic that’s more realistic for bot users
  • Default State API is a frequent choke/bottleneck

Summary

We’ve released a new URL along with a corresponding schema to get a valid token for direct messages to pass your Bot’s authentication. There is no change to the way your bot will behave, the token just allows you to submit direct messages to your Bot without going through the connector service – this is what will allow you to perform load tests!

Happy Making!

Dan Driscoll and Matthew Shim from the Bot Framework Team.

Posted in Bot Framework