This week, we added new sample bots to our Bot Builder-Samples GitHub repository, showcasing how some Bing and Cognitive Services APIs for vision, speech and search can help developers build smarter conversational interfaces.
|Sample||Description||API used||Sample link|
|Image Caption||Takes an image as input and provides a description of the image using the popular Cognitive Services Computer Vision API.||Vision||C#|Node|
|Speech To Text||Takes an audio file as input and converts the audio stream to text using Bing’s Speech API.||Speech||C#|Node|
|Similar Products||Takes a product image as input and responds back with a list of similar products using Bing’s visually similar search capabilities. It also suggests Internet merchant locations to buy the products online.||Image Search||C#|Node|
We are committed to helping developers build the next great conversational experiences. If you have ideas for future code samples, please use this forum to post them and we will review them shortly.
Vision API: Brings advanced image and video understanding skills to your bots using the popular Microsoft Cognitive Services API. It uses tags, descriptions and domain-specific models to identify content and label it with confidence. For example, you can use it to recognize objects, people’s faces, age, gender or even feelings. In addition, it recognizes adult/racy content in images to allow automated restriction of adult content. It also identifies image types and color schemes. Try out the demo.
Speech API: Recognizes audio from a file and transcribes the content to provide a text output. The Bing Speech API enables you to build smart bots that are voice triggered. It uses built-in language and acoustic models that cover a wide range of scenarios with high accuracy. Additionally, when bots need to “talk” back to their users, the API can also be used to convert text that is generated by the bot into audio that can be played back to the user. The API uses the same speech-to-text and text-to-speech services used in Cortana. Try out the demo.
Image Search API: Searches the web for images and provides comprehensive results using Bing’s knowledge and intelligence services. Results include thumbnails, full image URLs, publishing website info, image metadata, and more. It is ideal for any bot that needs to embed search results directly into messages, or leverage them as input for other interesting applications. Besides search, the Bing API supports several intelligent features like celebrity recognition, visually similar product search (where to buy), which can be used across a broad array of scenarios. Try out the demo.