Setting up a secure, custom domain, node.js site on Azure

In this article I will demonstrate the necessary steps to set up a node.js server running https, hosted in Azure.

Introduction

This article is a combination of my own work and a conglomeration of reference point blog articles which I had to find to achieve all of this.

Creating a node.js site on Azure

If you follow the instructions on this site you should be able to create an azure site (Get started with Node.js web apps in Azure App Service)

a1

Creating a custom domain

Once you have registered your new domain (in my case marky.co) you need to go to the azure portal and follow the instructions posted here (Configuring a custom domain name for an Azure cloud service). You cannot do this on your free tier though and this where you have to chose your plan carefully. To be able to interact with Office Add-Ins I need my service to be SSL enabled.

a2 a3

Once you have selected a Basic plan you should get the following options

a4

Assign your site and as the instructions stated – you can “Bring your domain” by changing the CName within your domain name provider DNS management tools.

a5

Adding SSL

There are a number of methods for getting an SSL certificate but I have taken to doing it for free – you can use the same process I detailed here for exposing your node server to manually collect the letsencrypt certificates  (Using Let’s Encrypt to create an SSL certificate for my Bluemix hosted web site) to create the .pem files.

To turn the .pem files into .pfx files you need to follow the openssl instructions here (How To: Get LetsEncrypt working with IIS manually)

openssl pkcs12 -export -out “certificate.pfx” -inkey “privkey.pem” -in “cert.pem” -certfile chain.pem

a6

The certificate.pfx file can then be loaded into the azure portal. When you import the certificate successfully it is displayed on the main blade automatically.

a7

 

Add the SSL binding

a71

aaaah we love the cloud….

a8

a9

IMPORTANT – Restart your instance and there we go

a10

Conclusion

In this article we have seen how to deploy an instance of node.js on Azure, applied a custom domain to it, created an SSL certificate and added to an azure instance. Once this is complete you should have an SSL secured node.js  instance running which can then be used for Office Add-in deployments.

 

Enabling a node.js SSL webserver using Let’s Encrypt .pem certificates

In this article I will show a simple example of getting a node.js SSL website up and running using the .pem certificates issued from Let’s Encrypt.

Introduction

 

Once you have gone through the process of getting your Let’s Encrypt certificates you will have 4 certificates

w1

You will need to download the root certificate and an intermediate certificate from Let’s Encrypt – https://letsencrypt.org/certificates/

w2

Once you have the root.pem the actual node code to get this running is relatively simple.

 

// Startup Express App
var express = require('express');

var https = require('https');
var http = require('http');
var fs = require('fs');
var app = express();

var options = {
  key: fs.readFileSync('copperCerts/privkey1.pem'),
  cert: fs.readFileSync('copperCerts/cert1.pem'),
  ca: [
    fs.readFileSync('copperCerts/root.pem', 'utf8'),
    fs.readFileSync('copperCerts/chain.pem', 'utf8')
  ]
};

http.createServer(app).listen(8080);
https.createServer(options, app).listen(3000);

 

IBM is a platinum sponsor of the new node.js foundation

Take a look at the video from Angel Diaz IBM VP Cloud it is only 90 seconds – IBM’s now big behind Node.js

Angel also wrote this article on node explaining is some more detail about the foundation and IBM’s involvement

http://www.thoughtsoncloud.com/2015/06/node-js-embarks-on-open-governance-journey/

IBM are a platinum sponsor of the new node.js foundation.

Here’s a quote from the linux foundation press release

http://www.linuxfoundation.org/news-media/announcements/2015/06/nodejs-foundation-advances-community-collaboration-announces-new?utm_source=nodeweekly&utm_medium=email

IBM

“An independent Node.js Foundation built on open governance is a major industry wide event as it ensures the continued adoption and growth of one of the world’s most ubiquitous programming languages. The Node.js foundation will provide developers with a top development platform that when combined with the power of IBM Cloud and Mobile will accelerate time to application concept, deployment, and refinement.  As a platinum member of the foundation, IBM looks forward to continued partnership in developing and promoting Node.js and the inevitable expansion of this vibrant community,” said Angel Diaz, Vice President, Cloud Architecture & Technology, IBM. 

Passing authentication information through the Bluemix Hybrid Secure Gateway

In this article I will demonstrate a couple of the things which can be passed through a Bluemix secure gateway, allowing us to create normal web based applications.

Introduction

In the previous article I demonstrated how to create a TLS secured hybrid Bluemix application. In this article we will look at some of the web properties/headers, cookies etc which we can pass through the gateway.

The Gateway

To demonstrate what can be passed through the gateway I am using a simple notes form to display the incoming information

The Cookie, Header and username fields are all hidden if the field value is blank

b1

 

Here is my application running on node, using the secure gateway and once again accessing the domino server hosted on my laptop.

b2

 

No username, no cookie, no header.

Changing the code back within the calling application we are going to add some additional information. In the following code snippet you can see that we have added some header “Marky” information.

app.get('/secureTunnel', function(req, res) {
    tunnel.create('8888', function(){
        var options = {
            headers: {"Cookie": "", "Marky": "Hi Marky"},
            port: '8888',
            path: '/xomino/ainx.nsf/testform?readform'
        };

        var callback = function(obj){
            res.writeHead(200, {"Content-Type": "text/html"});
            res.end(obj.body);
        }
        var obj = simpleHTTP.run(options, callback);

    });
});

When we refresh the application we can see that the header has been passed through the secure gateway to the application itself:


b3

b4

If we try and log into the application (directly on the domino server) we can generate a session authentication token. These screenshots are taken directly from the domino server.


b5

b6

At this point though just because the domino window is logged in, the node app still records anonymous.
b7

Adding the cookie to the node application code, which is passed through the gateway

app.get('/secureTunnel', function(req, res) {
    tunnel.create('8888', function(){
        var cookie = "DomAuthSessId=D2BF0063D62C9138E9F723BB88C046F5";
        var options = {
            headers: {"Cookie": cookie, "Marky": "Hi Marky"},
            port: '8888',
            path: '/xomino/ainx.nsf/testform?readform'
        };

        var callback = function(obj){
            res.writeHead(200, {"Content-Type": "text/html"});
            //res.write(JSON.stringify(obj.resHeaders)+"<hr/>")
            res.end(obj.body);
        }
        var obj = simpleHTTP.run(options, callback);
    });
});

We can now see that the user is authenticated within the bounds of the hybrid application

b8

Pushing all this code up into Bluemix you can truly appreciate the authenticated hybrid app

b9

Conclusion

In this article we have seen how we can push basic header information through the gateway and pseudo-demonstrate an authenticated application. There are of course multiple hurdles to overcome between this demo and a real world application, but I hope it has given you an idea for what’s possible.

 

 

Creating a secure Bluemix hybrid app using TLS encryption

In this article I will demonstrate how to secure a hybrid IBM Bluemix application using the Secure Gateway and the Mutual TLS encryption option.

Introduction

In the previous article I demonstrated how to create a sample hybrid app which was unsecure because you could just call the gateway URL and access the application behind the firewall. While this worked well as a concept demo, it is not a production feasible set up. In this article we will look at how to set up a secure tunnel to the gateway URL and then on to our application.

The basis for this article comes from this developerworks article.

https://developer.ibm.com/bluemix/2015/04/17/securing-destinations-tls-bluemix-secure-gateway/

It took me a long time (relatively) to figure out how to make this work in my environment as I did not understand what was being accomplished by the example. I hope to provide a greater level of detail and explanation in this article.

Creating a secure gateway

Following the steps described previously we can set up a Secure Gateway within our Bluemix app. This time we are going to create a gateway which is secured with TLS encryption. As you can see from the image below, when you select the “TLS Mutual Auth” option a grey section appears underneath the form fields.

  • Select Auto Generate cert and Private key

s1

 

Click on the + Icon at the end of the fields and you will see the new gateway created

s2

Click on the gear icon at the end of the line and you will see an option to “Download Keys”. On selecting that a zip file will be downloaded. You will notice that I have not blurred out the port or Destination ID this time. This is because the point of this is that without those TLS Keys, knowing this information will be of no use to you. (You’re welcome).

Once the Keys are downloaded you need to add them to your node application (in my case in the root)

s3

 

As you can see from the image below, the .pem files are just text files which are the key files used as part of the encryption handshake when we connect to the gateway.

s4

Once the keys have been added to the project we are then able to create our basic app.

Basic node app 101

The following code creates a basic node website

var express = require('express');
var app = express();
var http = require('http');

app.get('/', function(req, res) {
    res.write("Hi I am the root")
    res.end();
});

var host = (process.env.VCAP_APP_HOST || 'localhost');
var port = (process.env.VCAP_APP_PORT || 4000);
app.listen(port, host);

 

s5

 

Secure tunnel

We are going to create a secure tunnel to the Bluemix Secure gateway. We do this using the following code which should be saved as tunnel.js.

//tunnel.js
var tls = require('tls');
var fs = require('fs');
var net = require('net');

var options = {
    host: 'cap-sg-prd-5.integration.ibmcloud.com',
    port: '15101',
    key: fs.readFileSync('Hqb17PFJ9Oe_5lm_key.pem'),
    cert: fs.readFileSync('Hqb17PFJ9Oe_5lm_cert.pem'),
    ca: fs.readFileSync('DigiCertCA2.pem')
};

var creations = 0;
var server;

//In this case the port value is the port of the tunnel created on the local server
//to the secure gateway - this is NOT the 15xxx port of the gateway
exports.create = function(port, callback) {
    if(creations == 0){
        creations++;
        //server not currently running, create one
        server = net.createServer(function (conn) {
            connectFarside(conn, function(err, socket) {
                if (err){
                    console.log(err);
                }
                socket.pipe(conn);
                conn.pipe(socket);
            });
        });
        server.listen(port, function(){
            console.log('tunnel on port: '+port);
            callback();
        });
    } else{
        //server already running
        creations++;
        callback()
    }
};

function connectFarside(conn, callback) {
    try {
        var socket = tls.connect(options, function() {
            console.log('tunnel connected');
            callback(null, socket);
        });
        socket.on('error', function(err){
            console.log('Socket error: ' + JSON.stringify(err));
        });
    } catch(err) {
        console.log(err)
        callback(err);
    }
};

exports.close = function(){
    creations--;
    if(creations == 0){
        //close the server if this was the only connections running on it
        server.close();
    }
}

 

This file needs some explanation. What it is doing is the following:

  • When we call tunnel.create we are going to create a secure tunnel from the current node server to whatever is passed in through the options object
  • The port on which the tunnel is created has NOTHING to do with the port of the secure gateway. This tunnel will connect a specific port on the current server to the port on the gateway server.
  • The tunnel itself connects to the secureGateway on port 15101 (in this case)
  • The port is created as part of the connection to the gateway. When the connection is complete the port is closed. This prevents someone from guessing the new port on the server and using it!
  • This is not the best way of doing it, it is not very flexible for a reusable, in production, service with multiple connections. The port and server should not be hard coded. They are for this example so it is easier to understand. We will look at making it generic it later.

Connecting to our backend service (simpleHTTP.js)

In this case I am demonstrating connecting to a web page, but there is no reason why you cannot connect to mysql, mongo or anything else. I have a simple http connection module which will connect to the specified webpage on the back end and return the page as a buffered string back to the original app.get(‘/secureTunnel’).

//simpleHTTP.js
var http = require('http')
exports.run = function(options, callback){
    var response = {};
    var body = "";

    var req = http.get(options, function(res) {
        // Buffer the body entirely for processing as a whole.
        var bodyChunks = [];
        res.on('data', function(chunk) {
            // You can process streamed parts here...
            bodyChunks.push(chunk);
        }).on('end', function() {
            body = Buffer.concat(bodyChunks);
            //put the response into a format which can be easily passed to the callback
            response = {'body': body, 'resHeaders': res.headers}
            callback(response)
        })
    });
}

Building out our node app

Building out the rest of the app.js it looks like this:

  • Create the route for app.get(“secureTunnel”, function()……..)
  • When called the route does two things
    • Calls tunnel.create
    • Passes the connection request to simpleHTTP
    • Closes the tunnel

 


var express = require('express');
var app = express();
var http = require('http');
var tunnel = require('./tunnel.js');      //code used to create and manage the secure tunnel
var simpleHTTP = require('./simpleHTTP'); //code used to create the request to the http service (web page)

app.get('/secureTunnel', function(req, res) {
    tunnel.create('8888', function(){

        var options = {
            //host is not necessary in this case because it binds directly this server if blank
            port: '8888',          //The tunnel port
            path: '/xomino/jQinX.nsf/Marky?readform' //the path of the test page on my laptop
        };

        var callback = function(obj){
            res.writeHead(200, {"Content-Type": "text/html"});
            res.write(JSON.stringify(obj.resHeaders)+"<hr/>")
            res.end(obj.body);
        }
        //make the http call and display the results out on the page.
        var obj = simpleCopper.run(options, callback);
        tunner.close()
    });
});

var host = (process.env.VCAP_APP_HOST || 'localhost');
// The port on the DEA for communication with the application:
var port = (process.env.VCAP_APP_PORT || 4000);
// Start server
app.listen(port, host);

Putting it all together

Here is the process for the connection laid out in bullet points:

  • Request comes in to /secureTunnel
  • Create a secure tunnel on port 8888
  • The tunnel is created on port 8888 by setting:
    • The secureGateway URL
    • The secureGateway port
    • The secure gateway keys
    • and then opening the new tunnel to the secureGateway
  • At this point the host website (bluemix) on port 8888 is not connected to the secure gateway on port 15101
  • The connection is made and the request to the service is made
    • The request to the connection is not made over port 80 or 4000 (or whatever you are using for the host), it is actually made to the newly created tunnel port 8888
    • The connection to 8888 routes out to secureGateway port 15101 – this is now permitted
    • The secure gateway in turn connects back into the hybrid environment (192.168.0.2:80) and gets the desired information from within the firewall
    • The firewalled service responds back through the secure gateway and back to the host port 8888
    • The response is packaged up and returned to the user’s screen (for this demo the HI Message)
    • The tunnel on port 8888 is then closed and cannot be accessed by anyone any more.

And we have a result locally

s6

Which is my node server locally, connecting to the secure gateway to come BACK to my local domino server

s7

This of course would look WAAAY more impressive it was Bluemix making the call. So I committed the code and pushed it up the the xominoKnox Bluemix repository…….et voilà

s8

Conclusion

What I hoped to achieve in this article is a step by step explanation of how a secure tunnel is created to facilitate the secure hybrid environment. As I mentioned this hard coded version is not ideal for production yet because the keys are hard coded to the connection. With a little effort the code could be genericised to use the connection tunnel multiple secure gateways within BlueMix.

The code for this project can be found here – https://hub.jazz.net/git/mdroden/xominoKnox but I assure you the gateway is not longer open 😉

Very Cool 🙂

 

How to add a Node.js Express route in a separate file

In this article I will show how you can manage your routes in a separate file from app.js. It also demonstrates more generally how adding modules to your applications works in node.js.

Introduction

In this article I will create a simple route in a route.js file and reference it from my app.js. This will demonstrate how to keep the code separated and easier to manage.

The example application

This is a very simple express example with only two routes – the root of the app which says “Hi I am the root” and a second one which says “I am a new route”.

The initial app is a very basic app created using express.

// Startup Express App
var express = require('express');
var app = express();
var http = require('http').Server(app);

http.listen(3000);

// handle HTTP GET request to the "/" URL
app.get('/', function(req, res) {
    res.write("Hi I am the root")
    res.end();

})

Which then produces the following simple page

n1

New routes.js

I created the following file routes.js which will display a message when going to /marky

module.exports = function(app) {

    app.get('/marky', function(req, res) {
        res.write("I am a new route")
        res.end();
    });
}

mobile.exports is node.js specific code which allows for code includes in this very manner. For more on this check out this article. Notice that (app) is passed to the function so that it properly scoped to the original code.

Back in the app.js we add a single line to require this new library and that’s it.

// Startup Express App
var express = require('express');
var app = express();
var http = require('http').Server(app);

//include other libraries
var routes = require('./routes')(app); //This is the extra line

http.listen(3000);

// handle HTTP GET request to the "/" URL
app.get('/', function(req, res) {
    res.write("Hi I am the root")
    res.end();

})

n2

Conclusion

More fundamentally than this simple example, this is the core of how node modules (including express) work. When you “require” express or http or any other module within your node application, this is how it is put together. Kinda cool 🙂

Creating a sample Hybrid Bluemix environment

In this article I will demonstrate how to create a sample Hybrid app running in IBM Bluemix but getting data from behind a company firewall.

Introduction

A couple of years ago the prevailing message from vendors was “move to the cloud !!!”. The thing the vendors found though, was that the companies do not necessarily want to move their “data” to the cloud. It is either too complicated, expensive, unnecessary or they just do not flat out trust their data to the cloud. All that said though they are interested in the ability to securely expose their data to the outside world without exposing any of their internal systems. This has been achieved for years using a DMZ style firewall architecture which exposes only the web server but not the database server to the outside world.

In the Cloud world this concept is called a Hybrid model – cloud app, on premises data. In this article I want to show one way which IBM has approached this in Bluemix.

Reference

I wish I had listened to Ryan Baxter, last year at MWLUG 2014. I heard him talk about this concept and I serious thought to myself – who would want to do that. Being ahead of your time, happens to the best of us. Anyway you can see how Ryan set up his environment at that time using Cast Iron here. This is an excellent video and gives a nice overview of cast iron – that said, it is not the way I am going to do it and not the way IBM wants you to do it any more. So enjoy but come back….

I found most of the information I am going to write about today in this video…https://www.youtube.com/watch?v=pY-FRwGQ_8Y&feature=youtu.be

(For more information on getting started with your first Bluemix application check out this NotesIn9 video)

So Bluemix

Within my Bluemix application I created a simple node application (xominoKnox) and then added the “Secure Gateway” Service.

b1

 

 

b2

 

b3

 

I then created a Jazz Hub Git site and then cloned the repository locally (See this post for more information on that)

Creating the secure gateway

So the way that the gateway works is this:

  1. Create and configure the Bluemix end of the gateway
  2. Install the gateway code on the machine within the firewall
  3. Open the connection from inside the firewall
  4. Configure the connection to access data behind the firewall
  5. Use the connection

So let’s go through those steps one by one and explain what is going on.

1. Create and configure the Bluemix end of the gateway

Click on the Secure Gateway Service from within your Dashboard app view and you will see the configuration screen to create your first Gateway

b4

b5

Click Add Gateway and then you will be prompted to name your Gateway connection

b6

Click Connect it and you will then be presented with the status screen – Not Connected

b7

 

2. Install the gateway code on the machine within the firewall

The computer that you install the gateway on, inside your firewall, does not have to be the destination machine, it does however have to have access to the destination machine. Currently (April 2015) you will need to install a docker container on the machine and then inside of that the bluemix-secure-gateway can be installed. For those people without docker already, go here to get it installed.

NOTE FOR WINDOWS USERS: I had serious issues getting this installed due to the Oracle Virtual Box which has to be installed along with it. If you find that the Virtual box does not install – use this regedit hack to fix it. https://www.virtualbox.org/ticket/11349

This fixed it for me. Follow these step by step:

  1. Uninstall Virtualbox
  2. Uninstall Any Virtual Box Network Adaptors from Device Manager
  3. Go into the registry at: HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Control\Network
  4. Change “MaxFilters” from 8 to 20 (decimal)”
  5. Reboot your PC
  6. Install VirtualBox 4.3.X (Run as Administrator)

3. Open the connection from inside the firewall

Once you have docker installed and running (Boot2Docker for windows), copy the commend below into your docker window and run it.

b8

 

You will see the Connected message appear on your Secure Gateway dashboard and the tunnel connected message in the docker window.

b9

The gateway is set up and a secure tunnel from Bluemix to my laptop (behind my office firewall) is set up.

To be clear at this point the outside world cannot access the Copper/xomino server running on my laptop from the outside world. It is just running on my laptop as normal within my development environment.

4. Configure the connection to access data behind the firewall

Next we have to create a destination (behind the firewall). Understand that the docker window is by default bridged and therefore does not know that it is running on my local computer (127.0.0.1).The IP address  I have given Bluemix is the IP address of the laptop on my network.


b10

b12

 

As you can see from the image above a Cloud Host and port has been assigned. If you do this a number of times you will see that the port changes.

You will also note that I chose not to use No TLS in the connection. This means that this is NOT PRODUCTION ready. We really need to secure this so that only my application can call that URL. More on that later – but for the  sake of this article/demo I am leaving it simple.

5. Use the connection

If we connect to the URL shown in the image above we can see a Domino server !!!


b13

 

If we go to a specific page on that server we see this. Not much to look at I grant you, more on that in a later article.

b15

But if we go to the gateway path – and add the “/xomino/ainx.nsf/testForm?readform” to the end of the URL – we get the exact same thing, from the exact same server, just displayed in a cloud app.

b14

And that is very cool! Especially as it only took about 3 hours to figure this out 🙂

Conclusion

As we have seen in this article, it is relatively simple to set up a secure connection from a computer behind a firewall, and Bluemix. The example show it not yet fully secure though as anyone could call the URL and get web page from my server.

In a future article we will look at securing the connection and what else we are able to do with it.

 

BTE102: The Demonstration application – beyondtheeveryday.com

In the previous post  I described how Mark Leusink and I are going to be speaking about Angular.js in our presentation at ConnectED later this month.

We are very proud to announce the demonstration application upon which the presentation will be based

http://beyondtheeveryday.com

This application was created by Mark Leusink and is an amazing example of how simple an Angular application can be integrated with Domino data. The application is fully responsive and is particularly nice to use on a mobile device. All credit goes to Mark for this, I am very flattered to be talking with such an astute and talented developer. You can find out more about the application from Mark’s blog – http://linqed.eu/2015/01/14/marky-marks-mobile-first-connected-sessions-demo-app/

The application is running on a Domino server.

b1

IMG_0339

During the presentation we will demonstrate how we got the application to work on a node server hosted in Bluemix and also demonstrate the application running on SharePoint, IBM Connections and as a native Mobile application.

If you are still not sure, listen to Pete Janzen talk about his recommendations for the conference 😉

Write once – run anywhere………………. !

Websockets in XPages: Improving on the automated partialRefresh interface

In this article I will further discuss how tom improve the user experience of an automated partial Refresh on an user’s XPage. Although these posts were originally about using Bluemix to host the node.js server I kinda feel that the focus has drifted onto websockets more than Bluemix. So in an attempt to make it easier to find I am going to use the Websockets in XPages title moniker for a few posts and then go back to Bluemix 🙂

Introduction

In the last article we looked at how to push a automated partialRefresh to a XPage application using websockets. In that article it was noted that the user experience was not ideal because the whole panel refreshed without the user knowing about it. For some apps that is appropriate and for others it may not be. At this point in his career Dave Leedy is impressed he gave someone else and idea and I quote: “wow! that’s fricken awesome!!!”

So, that’s not a great user experience – what if they were doing something at the time?

Yes I was thinking that too! So I believe we can improve the user experience a little and take what Dave suggested and tweak it a little. Now where have a seen something which let’s the user know there is new data changes but doesn’t refresh the page without their action……….

b4

oh yeah that.

Instead of refreshing the control automatically, we will make the message create a “refresh” icon on the page which the user can then update at their leisure.

b5

The modified code is all in what happens when the page receives the refresh socket message. I added a jQuery rotate function just for some added “je ne sais quoi“. In the function we can see that when the refresh event is detected by the socket code the refreshControl function is called. This in turn makes the hidden refreshIcon visible, adds an onClick event and then rotates it. The onClick event performed the partialRefreshGet as we saw in the previous example turning the page briefly grey. We then hide the icon and remove the click event (to avoid piling on multiple events as the page gets continually refreshed)

 

 // Function to add a message to the page
  var refreshControl = function(data) {

	  $('.refreshIcon')
	  	.css({display: 'block'})
	  	.on('click', function(){
	  	   var temp = $('[id*='+data.refreshId+']').css({background: '#CCCCCC', color: 'white'}).attr('id')
		   XSP.partialRefreshGet(temp, {})
		   $(this).css({display: 'none'}).off('click')
	  	})
	  	.rotate({
	      angle:0,
	      animateTo:360,
	      easing: function (x,t,b,c,d){        // t: current time, b: begInnIng value, c: change In value, d: duration
	          return c*(t/d)+b;
	      }
	   })
  };

  // When a refresh message is received from the server
  // action it
  socket.on('refresh', function(data) { refreshControl(data); });

The following video quickly demonstrates the new capability.

Conclusion

In this brief article we concentrated on how to improve a user experience by notifying them that changes were pending and then allowing them to determine when the changes were made.

I still don’t think this is as optimal as I would like but you get the idea. As I said a long time ago – the more DOM you are reloading the worse the user experience. With a viewPanel we are kinda limited on what we can and cannot refresh. A better option may be to architect the application just the new data and update as appropriate……….

 

XPages and Bluemix: Pushing out data changes through automated partialRefresh

In this article I will demonstrate how using targeted websockets messages we are able to refresh user data on pertinent screens within an application, and keep user’s data up to date.

Introduction

In previous articles I have discussed the creation of a nodejs websockets service within Bluemix and how we are able to send messages to specific web pages using the socket.io rooms capability. Both of those examples were proofs of concept and the messages were generated in the browser via firebug console commands. We are going to look at how we can automate these messages and begin to build a user case for using websockets within our applications.

Disclaimer: This idea for an example was the brainchild of David Leedy and in many ways it is a genius example to relate websockets to XPages functionality. And at the same time, I am absolutely disgusted that I am even talking about this because I would never dream of actually implementing this within an application. The fact that the page changes without the user’s knowledge is poor, and the fact that it refreshes the entire control when only a small piece of data changes is horrible. All that said however, this is still a demonstration of the capability and in future blog posts on the subject I will actually show example which I would be proud to actually put into one of my own applications 😉

Keeping data up to data on an XPage

Let’s say we have a sample application with a simple XPages viewPanel on it

b1

Somewhere within the application – someone else makes a change to the data

b2

The only way to see the change would be to refresh the page – and you the user would never know when.

b3

This can be pseudo-automated from a user experience perspective in a number of ways but they all involve periodic checking for updates on way or another. If you scale that over many users this is extremely inefficient.

That is where websockets comes in very nicely.

Pushing to specific rooms

In the previous article I demonstrated how to record an XPage as a “room” dynamically and in this example we will do the same thing.

  var temp = (location.href.indexOf('copper.xomino.com')>0) ? "http://localhost:3000" : 'http://xominosocket.mybluemix.net'
  var socket = io.connect(temp)

  // Send message to server that user has joined
  nickname=$('.username').text()
  var xpageName = location.href.split('.nsf/')[1]
  xpageName = xpageName.split('.xsp')[0]
  socket.emit('joinRoom', xpageName, nickname);

Automating a partialRefresh

In a similar manner to listening for a new chat message and then acting upon it, we are going to listen for a “refresh” socket event and then action it. In this case we are also going to pass in the id of the XSP control we want to refresh. For the sake of this example I am also using some CSS to make the element appear momentarily grey (see the video and all will be clear)

  // Function to add a message to the page
  var refreshControl = function(data) {
	  //Get the refreshId from the incoming data and color the control grey
	  //then get the id of the element via the is attribute
	  var temp = $('[id*='+data.refreshId+']').css({background: '#CCCCCC', color: 'white'}).attr('id')
	  //With the known id - trigger a XPages partial refresh of the control
	  XSP.partialRefreshGet(temp, {})
  };

  // When a refresh message is received from the server
  // action it by calling the refreshControl function
  socket.on('refresh', function(data) { refreshControl(data); });

Creating a new listener on the node.js server

I created a new route to post my refresh data to “xpagesRefresh”. When data is POSTed at xpagesRefresh it is parsed and send back out via websockets using the “refresh” socket event.

// Handle the form POST containing the name and , reply with the language
app.post('/xpagesRefresh', cors(corsOptions), function(req, res){
  var request_data = {};

  if (req.body){
    request_data = {
      'refreshId': req.body.refreshId,
      'nickname': req.body.nickname,
      'rt': 'text'
    };
  }

  var data = { refreshId: request_data.refreshId, nickname: request_data.nickname};
  console.log("POSTing at xpagesRrefresh")
  console.log(req.body)

  io.sockets.emit("refresh", data);
  res.send(data);
});

Automating the partial refresh via the user action

In a real application we are not going to have someone sitting on a browser pushing updates via firebug. We want to be able to create the update when the XPage is saved in the first place. To do this we transfer the code we saw in previous firebug blog example into the onComplete of a Save button. In this way when a document is updated within the application. A notification is sent out to all the people looking at the page, updating the data for them.

 

<xp:button value="Save" id="button1" rendered="#{javascript:document1.isEditable()}">
	<xp:eventHandler event="onclick" submit="true" refreshMode="partial"
		save="true" refreshId="blank">
		<xp:this.onComplete>
		<![CDATA[
			var socketServerURL = (location.href.indexOf('copper.xomino.com')>0) ? "http://copper.xomino.com:3000" : 'http://xominosocket.mybluemix.net'

			var data = {
			  refreshId: 'wrapper',
			  nickname: 'automated'
			   };

			  console.log(data)
			  $.ajax({ url:
			          socketServerURL+"/xpagesRefresh",
			      type: "POST",
			      data: data
			  }).done(function( msg ) {
			      console.log( msg);
			  });
			]]>
			</xp:this.onComplete>
	</xp:eventHandler>
	</xp:button>

Demonstration

The video below shows how we are able to trigger a partial refresh after updating a document within the application. Note the screen flickers grey as the CSS change happens before the partial refresh. It is so quick it is almost instant.

Pushing to Bluemix

As before, pushing my new code to Bluemix is as simple as checking it in to the jazz hub repository and redeploying. In the picture below we can see that we are using marky.psclistens.com as the application domain and not copper.xomino.com. As we showed before, if not copper then use Bluemix.

b6

 

We create the capability locally, tested it, proved it and deployed it seamlessly to Bluemix with a “Commit and Push” – and that is *so* cool 🙂

Conclusion
In this article we saw how to trigger core XPages functionality automatically without the user having to interact with the application. In the next article we will look at how to improve on this, frankly horrible, user experience.

Thanks Dave 🙂