Bot Framework

PSC Tech Talk – Microsoft Bot Framework

Marky Roden Bot Framework, PSC Labs, Tech Talks , , ,

Starting in early 2016 cloud vendors started to promote the concept of bots as a cool new feature and new way for users to interact with their applications within the enterprise. Understanding the general acceptance of a bot as a user interface, the push to gain traction in the space began in earnest.  Seeing this shift in emphasis in the vendor landscape prompted PSC Labs to create an investigation team for a short-term project.

In this presentation Adam Lepley (@AdamLepley) presented the first of a number of talks (here, here and here) he has given on the MS Bot Framework, how it works, why it was created and how easy it is to use.

What is the Microsoft Bot Framework? 

The bot framework is as the name implies a framework for building “bots”. What this means as a developer is that in C# and JavaScript Microsoft has created libraries containing methods and functions for simplifying the creation of an interactive chat bot. Once the created the framework also provides the ability to publish the bot to many chat “channels” like facebook, slack, teams, skype, SMS and others.

Chat bots are not a new concept. Various web sites and chat clients have been leveraging varied forms bots for many years, but mostly with the emphasis on consumer facing applications.  Targeting chat applications also comes with the benefit of building on a platform the user already is familiar with. It also removes the friction of leaning a new application and lessens the burden on developers on creating complex custom UI.

The Plan

Within the lab we always like to learn about a new technology and then make a plan to better understand it and demonstrate capability in it. The Plan was initially to download the examples, install, learn and then expand on what we learn and make our own examples with broader applicability to PSC clients.

We looked into:

  • How to create a bot
  • How to deploy a bot to different channels
  • How to add Artificial intelligence (LUIS)
  • How can we build something applicable to our clients / What else can we play with?

What did we find?

Adam discussed and demonstrated how easy it is to create a bot using the framework. He was able to build a hello world bot in about 10 minutes and publish it to a point where we could actually interact with it in the meeting itself.

The investigation team created the three bots aimed at demonstrating increased productivity gains and enhanced user experiences:

  • Common data capture – The ability to quickly and easily view and create timesheets from a bot.
  • Predictive Analytics – Using Machine Learning techniques to return projected sales results back to users based on Product information hosted in an external database.
  • Cognitive Services – Using cognitive services and natural language processing to demonstrate free text entry in a bot to create task logging on an external site.

The common assumption is text is the primary integrations when using chat clients. This is mostly true when two humans communicate over chat, but as it relates to bots, we have a variety options Microsoft provides with its abstraction.

The bot framework supports text (plain and rich), images (up to 20 Mb), video (up to 1 mins), buttons and the following rich content cards…

In addition to the rich content cards, Microsoft has released a separate service which enables you to build more complex card content layouts which can be rendered from data coming from the bot framework. This also exposed more native platform specific custom rendering of cards.

Timesheet Bot

We set out to build a bot which would help fill out weekly timesheet for our consultants. Our bot has two main features: displaying and creating a weekly timesheet. For displaying the previous week’s timesheet, used a carousel card which can display a collocation of cards representing the days of the week. Each card also has a set of buttons which can either link to additional actions within the bot, or external links to an existing website.

Product information Bot

We created a bot demonstrating the ability to search a product database which in turn triggered an external API call to an associated Azure machine learning service. Users can interact with the bot via a series of question and answers. e.g. “What product are you searching for? Please select one of the following”. The results are then fed back to the bots in the form of a chart graphic. This bot demonstrates a powerful way to access a variety of on demand reports right within a chat client.

Productivity Bot using Natural Language interpretation

We used the Azure LUIS service (Language Understanding Intelligent Service), which is a part of Microsoft’s cognitive services and uses machine learning to help derive intent from text. Users can use an unstructured text request to “create a task” or “create new task” or “I want a new task” which the LUIS service derives the intent to be “Create a Task”. Using the secure integration with and external task tracking service (Trello) the bot is then able to ask the user the necessary questions to create a task based on user inputs.

Conclusion

Bots are being used today by startup and some commercial enterprises trying to break into the corporate enterprise space. Our time spent with the Microsoft Bot Framework has convinced us that bot are ready for the enterprise and there are use cases for them effective implementation today.

 

Calling an external service from your chat bot

Marky Roden Azure, Bot Framework , ,

In this article I will show to how integrate simple commands (intents) into your bot to then integrate with an external service.

Introduction

In previous articles we have looked at how to create a sample Azure bot and in this article we will be looking into how “intents” work. The microsoft documentation on dialogIntents can be found here (for the node bot).

URL Shortener

In an effort to learn more about running a node service in Azure, Azure SQL and to help brand some of my tweets a little better I created my own little URL shortening service running at https://marky.co. The details are not so important for this article but I took the Coligo.io article on create a node url shortner with Mongo and modified it to work with Azure SQL instead of Mongo. The service is called very simply by POSTing the appropriate parameters at the appropriate service on my website. What is returned is a shortened URL. Simple process really.

IntentDialog

“The IntentDialog class lets you listen for the user to say a specific keyword or phrase. We call this intent detection because you are attempting to determine what the user is intending to do. IntentDialogs are useful for building more open ended bots that support natural language style understanding.”

So in a sense I am looking at this as an opportunity to treat my bot as a CLI client for my own laziness. While the “intent” is to allow for natural language understanding I want to look at it as an opportunity to make my bot into a monkey butler of sorts. Do my work for me, I am lazy (or productive).

I am not a fan of CLI in the programming world. Not being a historically Unix kinda guy, never had to and I prefer a point and click tooling approach to things like git and Azure CLI. That said, this is not programming – this is bots 😉

Setting up your dialogIntent to listen for -sh 

The following code snippets is the basis for my simple bot connector – listen for the “-sh ” command then take the following argument and process it.

The http module is used for the communication with the shortener service. Once the response is returned from the service the shortened URL is sent back to the user. The process is simple and the intent of this is just to show an example of how to use a command in a bot to make it do something.

	"use strict";
	var builder = require("botbuilder");
	var botbuilder_azure = require("botbuilder-azure");
	var azure = require('azure-storage');

	var http = require('http');

	var data = "";

	var options = {
		host: 'marky.co',
		port: '80',
		path: thepath,
		method: 'POST',
		headers: {
			'secret': theSercretKeyWhichStopsBots,
			'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8'
		}
	};

	var dotenv = require('dotenv').config();

	var useEmulator = (process.env.NODE_ENV == 'development');

	var connector = useEmulator ? new builder.ChatConnector({
			appId: process.env['MicrosoftAppId'],
			appPassword: process.env['MicrosoftAppPassword']
		}) : new botbuilder_azure.BotServiceConnector({
		appId: process.env['MicrosoftAppId'],
		appPassword: process.env['MicrosoftAppPassword'],
		stateEndpoint: process.env['BotStateEndpoint'],
		openIdMetadata: process.env['BotOpenIdMetadata']
	});

	var bot = new builder.UniversalBot(connector);

	var intents = new builder.IntentDialog();
	bot.dialog('/', intents);

	intents.matches(/-sh /i, function (session, args) {
		var input = args.matched.input
		
		//input will be in the format "-sh http://www.xomino.com"
		//better validation would probably be appropriate at some point

		//split the input into "-sh" and theURL
		data="url="+input.split(" ")[1]; 
		//match the POST length to the incoming URL
		options.headers['Content-Length'] = data.length 
		session.sendTyping(); //...typing

		var req = http.request(options, function(res) {
			res.setEncoding('utf8');
			res.on('data', function (chunk) {
				session.send('Your short URL is: '+JSON.parse(chunk).shortUrl);
			});
		});

		req.on('error', function(e) {
			console.log('problem with request: ' + e.message);
		});

	// write data to request body
		req.write(data);
		req.end();
	});

	intents.onDefault(function (session, args, next) {
		session.send("I'm sorry "+session.dialogData.name+". I didn't understand.");
	});

	if (useEmulator) {
		var restify = require('restify');
		var server = restify.createServer();
		server.listen(3978, function() {
			console.log('test bot endpont at http://localhost:3978/api/messages');
		});
		server.post('/api/messages', connector.listen());    
	} else {
		module.exports = { default: connector.listen() }
	}

 

Using this code I can start to use my bot as a URL shortener rather than having to go to my website and post the URL into a form.

Note – in case you were curious this does not work in the Skype client itself because a URL is automagically transformed into something else for preview. Ah well that’s not quite the point really 🙂

Conclusion

In this article we have see that using the bot IntentDialog class we are able to listen for predetermined commands which will then allow it to take input, act on that input and return an answer. In this case a URL shortener is a simple application but it would be much more useful if I could securely look into my enterprise and extract information relating to my business……

NWCJS Meetup: Creating Bots with JavaScript and the Microsoft Bot Framework

Marky Roden Bot Framework, NWCJS , ,

Next week’s Northwest Chicago JavaScript meetup (23 March @PSCGroup) features our own Adam Lepley speaking about some of the work he has been doing on the Microsoft Bot Framework.

https://www.meetup.com/Northwest-Chicago-JavaScript/events/236993059/

“Bots, they are invading our chat apps, now it’s time to learn to develop your own. Come learn how Microsoft’s Bot Framework enables you to leverage your existing JavaScript skills to create a single bot which can be easily deployed to many channels including Skype, Slack, Facebook Messenger, SMS or embedded into your own site. I’ll also show you how to take it a step further to include natural language processing to elevate your bot’s intelligence. Walk away with tools to build a great bot that can connect with users, wherever they are.”

Setting up the sample Azure bot to work locally with the bot emulator

Marky Roden Azure, Bot Framework , ,

In this article I will demonstrate how to configure your local development environment to work with the environmental variables set up within your Azure environment in the sample bot previously discussed,

Introduction 

In the previous article we looked at how to create a sample azure bot and then how to configure it in VSTS for continuous integration. If you want to develop with this sample locally you will need to set it up to work with the local bot emulator. (What is Bot Builder for Node.js and why should I use it?). To be able to do this you will have to configure your local development environment to use the process.env variables which are picked up within the azure runtime environment.

process.env

process.env is how environmental variables are passed into a node development environment. They are especially important when it comes to keeping secret keys secret. We have to make sure that they are not included in the git repository and are not available to the end user. You can learn more about process.env in the nodejs documentation.

Using dotenv

I like to use the dotenv nodejs package to handle local env variables to just read my variables locally from a .env file. If you look at the package.json for the example project, turns out so does microsoft 😉

...
  "dependencies": {
    "azure-storage": "^1.3.2",
    "botbuilder": "^3.4.2",
    "botbuilder-azure": "0.0.1",
    "dotenv": "^4.0.0"
  },

When I clone the VSTS repo locally and run it – nothing seems to happen…..and that’s because I do not have any local env variables….


var useEmulator = (process.env.NODE_ENV == 'development');

......
if (useEmulator) {
    var restify = require('restify');
    var server = restify.createServer();
    server.listen(3978, function() {
        console.log('test bot endpont at http://localhost:3978/api/messages');
    });
    server.post('/api/messages', connector.listen());    
} else {
    module.exports = { default: connector.listen() }
}

So we can use a .env file locally in the root of the project and then require it in my code. The .env file contains nothing more that NODE_ENV = ‘development’ right now.

NOTE: To make sure this is not pushed up to the repo – add *.env to your .gitignore file.

We now have a web server running (nice typo MS 😉 ).

Bot Emulator

Once you download and install the bot emulator you can configure it as per the instructions to run a simple bot without issue. This does not work once you have a bot running in Azure. The sample I created uses multiple Azure services and you can see them being called as the environmental variables in the code.

 

var connector = useEmulator ? new builder.ChatConnector() : new botbuilder_azure.BotServiceConnector({
    appId: process.env['MicrosoftAppId'],
    appPassword: process.env['MicrosoftAppPassword'],
    stateEndpoint: process.env['BotStateEndpoint'],
    openIdMetadata: process.env['BotOpenIdMetadata']
});

Unfortunately when I load up the emulator – as you can see above “new builder.ChatConnector” does not pass in any environmental variables. I then get the following error in the console

The appId is undefined as it is not being passed into the application.

The ApplicationId and secret key were given to you when you created the bot in the first place – if you didn’t write them down, you’re going to be regretting that decision right now….

We need to modify the ChatConnector request to pass in the necessary variables, and we also need to add those variables to the .env file.

 

var connector = useEmulator ? new builder.ChatConnector({
        appId: process.env['MicrosoftAppId'], //new param
        appPassword: process.env['MicrosoftAppPassword']  //new param
    }) : new botbuilder_azure.BotServiceConnector({
    appId: process.env['MicrosoftAppId'],
    appPassword: process.env['MicrosoftAppPassword'],
    stateEndpoint: process.env['BotStateEndpoint'],
    openIdMetadata: process.env['BotOpenIdMetadata']
});

Refreshing the bot emulator, we start to get somewhere

and then it dies when we try and talk to it (doh)

Azure Storage Credentials

The problem is that this chat example uses the Azure Storage Service (to make the Azure Function part of the example work). So we have to add the AzureWebJobsStorage environmental variable to our .env file.

 

var queueSvc = azure.createQueueService(process.env.AzureWebJobsStorage);

You can find this connection key in your Azure portal > Storage Service > Access Keys

The format for the .env file entry  is as follows (Running Azure Functions Locally with the CLI and VS Code): AzureWebJobsStorage=’DefaultEndpointsProtocol=https;AccountName=storagename;AccountKey=secretKey’

Once that is in place – we have a bot running locally talking to the bot framework – unfortunately we do not get a response back from the Azure Function….

Azure Function debugging…..

The problem quite simply is that the message sent to the azure function contains a serviceURL from which it should respond – and in this case it is ” serviceUrl: ‘http://localhost:63136′,” and of course it has no idea what that is.

For the sake of this blog post that is ok – we are at least up and running with the “bot” part of this emulator working, although somewhat disappointing it can’t be fully developed in this environment.

Conclusion

As part of this process we have seen how to connect the local bot emulator to a service running in Azure and how to incorporate a connection to Azure Functions.

 

 

Adding your bot code to VSTS for source control and configuring continuous integration

Marky Roden Azure, Bot Framework , ,

In this article I will walk through the steps to enable source control and continuous integration for your bot

Introduction

In previous articles we have seen how to create a sample bot and how to link it into Skype. In this article we are going to look at how to use Visual Services Team Studio (VSTS) to manage source control and eventually continuous integration. We are going to set up is an environment such that when you check your code into the repository a code deployment is triggered to the nodejs server (our bot in this case) which then finally restarts. In short: code checks in and everything refreshes automagically.

What is going to happen is that we are taking the code out of the Azure sample development web based IDE environment and move it to something a little more robust.

Selecting your Source Control

From your bot interface select settings and scroll down to the continuous integration section.

Note: it is really important to read the instructions – if you do NOT download the zip file of your bot first you will end up with a blank look on your face and a need to rebuild your bot from scratch – take it from this idiot (facepalm).

bot1

Download your zip file and then head over to your favorite source control repo and create one.

In my case I used Visual Studio Team Services (VSTS) to set up a git repo.

bot2I unzipped the file locally and associated it with my new project in VSTS. I use JetBrains Webstorm as my JavaScript development environment but this would work just as well in Visual Studio or VS Code.

bot3

 

Setting up continuous integration

Back within the Azure bot development website click the Continuous Integration button. This will then cause you to move through a number of screens configuring the CI. For this to work you have to have configured your Azure environment to be linked with your VSTS environment. Here’s a great blog post describing how to do just that (Linking your VSTS Account to Azure)

So “Setup”

bot4Select VSTS

bot5

and then your project within VSTS

bot6

Which branch (master just to keep things simple for now)

bot7

et voila

bot8

Configuring a performance test

Following the prompts you can set up the perf test (for once the CI is complete)

bot9

bot10

 

and we now have your Azure bot being deployed from code sitting within VSTS. Cool 🙂

Example

I have my bot hooked up in Skype

bot12

I make a quick change to the message in WebStorm (my JavaScript IDE)

bot11

Commit and push

bot13

bot14

Head back to Skype and within about 60 seconds (it has to restart after all) we have a new message

bot15

What’s really interesting though is that I started my conversation before the restart was ready – but the message was queued and when the server woke up it finally responded – that was very cool

Conclusion

In this article we have looked at how to set up a VSTS source control repo of our new test bot and then configured continuous integration through the bot framework. I check in my new code and within seconds the new functionality is posted out to the live bot.

 

 

 

Adding your Azure bot framework bot into Skype

Marky Roden Azure, Bot Framework , ,

In this article I will show how simple it is to add your newly created bot into your Skype contacts.

Introduction

In the previous article we looked at how to create your first sample Azure bot which uses Azure Functions in the background. This article will continue from where that left off.

Channels

The bot framework has a number of pre-configured “Channels” which allow your bot to be easily added to other chat mediums.

If you aren’t already there, go into your bot and select Channels. Conveniently there is an “Add to Skype” button right there….(can’t imagine what that does)

bot1

Adding to Skype

You click on the link to Add to Contacts

bot2

you then “Add to contacts” – having to log in during the process (I need to get a better logo for my bot)

bot3

about 30 seconds later my bot appeared in my Skype client as a contact (bonus if you recognize the song lyrics)

bot4

Once it appeared in my contacts, I could talk to it – and while this is pretty simple stupid, this is VERY cool 🙂

bot5

I have to say though, the “response” from the Azure functions seems very slow for what should be a trivial task. I have no idea why or how it is even working right now – I’ll get to that 🙂

Conclusion

Adding your bot to the pre-configured channels is as easy as following the instructions and we now have a bot running in Skype! It’s not quite getting me a beer from the fridge yet, but we’ll work on that 🙂


How to create a sample nodejs bot which utilizes Azure Functions in 15 minutes

Marky Roden Azure, Bot Framework , , ,

In this article I will demonstrate how to create a nodejs bot hosted in Azure. The bot will be created from a sample provided from Microsoft. The example bot will use Azure Functions as the trigger for it’s communication. You will need a Microsoft Azure account to be able to follow this process for yourself. This is only an introduction on how to get up and running, more steps in the process will come later.

Introduction 

Looking at the documentation on how to create a test a nodejs bot within the Microsoft Bot Framework can be found in the Microsoft documentation (What is Bot Builder for Node.js and why should I use it?).

I found another more interesting (less CLI) way to do it through Azure portal. As I work my way through figuring out how all this works I expect there will be a number of blog posts as there is a lot to talk about. This post is focused on getting us up and running.

As of February 2017 this Bot Service is in Preview and the process may change before it goes GA later in the year !

Creating a bot service in Azure

Within your Azure portal create a new instance of a bot service from within Intelligence and Analytics.

bot1

Give it a name and resource group

bot2

and create.

Once created you will be able to access your bot from your portal resources pane.

Setting up your service

Opening your new service for the first time, you’ll be presented with the form to create an AppId and secret password for your bot. Follow the prompts and get back to the screen to create a sample bot.

bot3

We are going to select NodeJS and create an azure function bot for the sake of demo

bot4

and then we wait……Ho hum, I wonder what’s going on in the background – probably magic….

bot5

Developer interface

Once the bot has been created you will be presented with a web based IDE showing you the node code and a chat window on the right.


 bot6

You can talk to yourself!! Something I like to do all the time…..apparently….

bot9

Back in Azure Portal

In setting up all this up, Azure auto-magically created a number of new services have been created in the Azure Portal to manage this functionality(which is why we had to wait for the background magic earlier). We can now see the following services making all this happen.

bot8

How it works and how it is happening will come in future blog posts. For the moment we are up and running……and this blog post is getting long enough already.

Conclusion

This entire process took no more than about 15 minutes to complete. There is a lot more to explore around the deployment of the bot and the further development thereof, but for the moment we are up and running with a bot and Azure Functions (at least in development).

PS

I now have a nagging feeling that this is cool and all, but I really should check and understand how much those new magic services are going to cost…….