Websockets in XPages: Improving on the automated partialRefresh interface

In this article I will further discuss how tom improve the user experience of an automated partial Refresh on an user’s XPage. Although these posts were originally about using Bluemix to host the node.js server I kinda feel that the focus has drifted onto websockets more than Bluemix. So in an attempt to make it easier to find I am going to use the Websockets in XPages title moniker for a few posts and then go back to Bluemix 🙂

Introduction

In the last article we looked at how to push a automated partialRefresh to a XPage application using websockets. In that article it was noted that the user experience was not ideal because the whole panel refreshed without the user knowing about it. For some apps that is appropriate and for others it may not be. At this point in his career Dave Leedy is impressed he gave someone else and idea and I quote: “wow! that’s fricken awesome!!!”

So, that’s not a great user experience – what if they were doing something at the time?

Yes I was thinking that too! So I believe we can improve the user experience a little and take what Dave suggested and tweak it a little. Now where have a seen something which let’s the user know there is new data changes but doesn’t refresh the page without their action……….

b4

oh yeah that.

Instead of refreshing the control automatically, we will make the message create a “refresh” icon on the page which the user can then update at their leisure.

b5

The modified code is all in what happens when the page receives the refresh socket message. I added a jQuery rotate function just for some added “je ne sais quoi“. In the function we can see that when the refresh event is detected by the socket code the refreshControl function is called. This in turn makes the hidden refreshIcon visible, adds an onClick event and then rotates it. The onClick event performed the partialRefreshGet as we saw in the previous example turning the page briefly grey. We then hide the icon and remove the click event (to avoid piling on multiple events as the page gets continually refreshed)

 

 // Function to add a message to the page
  var refreshControl = function(data) {

	  $('.refreshIcon')
	  	.css({display: 'block'})
	  	.on('click', function(){
	  	   var temp = $('[id*='+data.refreshId+']').css({background: '#CCCCCC', color: 'white'}).attr('id')
		   XSP.partialRefreshGet(temp, {})
		   $(this).css({display: 'none'}).off('click')
	  	})
	  	.rotate({
	      angle:0,
	      animateTo:360,
	      easing: function (x,t,b,c,d){        // t: current time, b: begInnIng value, c: change In value, d: duration
	          return c*(t/d)+b;
	      }
	   })
  };

  // When a refresh message is received from the server
  // action it
  socket.on('refresh', function(data) { refreshControl(data); });

The following video quickly demonstrates the new capability.

Conclusion

In this brief article we concentrated on how to improve a user experience by notifying them that changes were pending and then allowing them to determine when the changes were made.

I still don’t think this is as optimal as I would like but you get the idea. As I said a long time ago – the more DOM you are reloading the worse the user experience. With a viewPanel we are kinda limited on what we can and cannot refresh. A better option may be to architect the application just the new data and update as appropriate……….

 

XPages and Bluemix: Pushing out data changes through automated partialRefresh

In this article I will demonstrate how using targeted websockets messages we are able to refresh user data on pertinent screens within an application, and keep user’s data up to date.

Introduction

In previous articles I have discussed the creation of a nodejs websockets service within Bluemix and how we are able to send messages to specific web pages using the socket.io rooms capability. Both of those examples were proofs of concept and the messages were generated in the browser via firebug console commands. We are going to look at how we can automate these messages and begin to build a user case for using websockets within our applications.

Disclaimer: This idea for an example was the brainchild of David Leedy and in many ways it is a genius example to relate websockets to XPages functionality. And at the same time, I am absolutely disgusted that I am even talking about this because I would never dream of actually implementing this within an application. The fact that the page changes without the user’s knowledge is poor, and the fact that it refreshes the entire control when only a small piece of data changes is horrible. All that said however, this is still a demonstration of the capability and in future blog posts on the subject I will actually show example which I would be proud to actually put into one of my own applications 😉

Keeping data up to data on an XPage

Let’s say we have a sample application with a simple XPages viewPanel on it

b1

Somewhere within the application – someone else makes a change to the data

b2

The only way to see the change would be to refresh the page – and you the user would never know when.

b3

This can be pseudo-automated from a user experience perspective in a number of ways but they all involve periodic checking for updates on way or another. If you scale that over many users this is extremely inefficient.

That is where websockets comes in very nicely.

Pushing to specific rooms

In the previous article I demonstrated how to record an XPage as a “room” dynamically and in this example we will do the same thing.

  var temp = (location.href.indexOf('copper.xomino.com')>0) ? "http://localhost:3000" : 'http://xominosocket.mybluemix.net'
  var socket = io.connect(temp)

  // Send message to server that user has joined
  nickname=$('.username').text()
  var xpageName = location.href.split('.nsf/')[1]
  xpageName = xpageName.split('.xsp')[0]
  socket.emit('joinRoom', xpageName, nickname);

Automating a partialRefresh

In a similar manner to listening for a new chat message and then acting upon it, we are going to listen for a “refresh” socket event and then action it. In this case we are also going to pass in the id of the XSP control we want to refresh. For the sake of this example I am also using some CSS to make the element appear momentarily grey (see the video and all will be clear)

  // Function to add a message to the page
  var refreshControl = function(data) {
	  //Get the refreshId from the incoming data and color the control grey
	  //then get the id of the element via the is attribute
	  var temp = $('[id*='+data.refreshId+']').css({background: '#CCCCCC', color: 'white'}).attr('id')
	  //With the known id - trigger a XPages partial refresh of the control
	  XSP.partialRefreshGet(temp, {})
  };

  // When a refresh message is received from the server
  // action it by calling the refreshControl function
  socket.on('refresh', function(data) { refreshControl(data); });

Creating a new listener on the node.js server

I created a new route to post my refresh data to “xpagesRefresh”. When data is POSTed at xpagesRefresh it is parsed and send back out via websockets using the “refresh” socket event.

// Handle the form POST containing the name and , reply with the language
app.post('/xpagesRefresh', cors(corsOptions), function(req, res){
  var request_data = {};

  if (req.body){
    request_data = {
      'refreshId': req.body.refreshId,
      'nickname': req.body.nickname,
      'rt': 'text'
    };
  }

  var data = { refreshId: request_data.refreshId, nickname: request_data.nickname};
  console.log("POSTing at xpagesRrefresh")
  console.log(req.body)

  io.sockets.emit("refresh", data);
  res.send(data);
});

Automating the partial refresh via the user action

In a real application we are not going to have someone sitting on a browser pushing updates via firebug. We want to be able to create the update when the XPage is saved in the first place. To do this we transfer the code we saw in previous firebug blog example into the onComplete of a Save button. In this way when a document is updated within the application. A notification is sent out to all the people looking at the page, updating the data for them.

 

<xp:button value="Save" id="button1" rendered="#{javascript:document1.isEditable()}">
	<xp:eventHandler event="onclick" submit="true" refreshMode="partial"
		save="true" refreshId="blank">
		<xp:this.onComplete>
		<![CDATA[
			var socketServerURL = (location.href.indexOf('copper.xomino.com')>0) ? "http://copper.xomino.com:3000" : 'http://xominosocket.mybluemix.net'

			var data = {
			  refreshId: 'wrapper',
			  nickname: 'automated'
			   };

			  console.log(data)
			  $.ajax({ url:
			          socketServerURL+"/xpagesRefresh",
			      type: "POST",
			      data: data
			  }).done(function( msg ) {
			      console.log( msg);
			  });
			]]>
			</xp:this.onComplete>
	</xp:eventHandler>
	</xp:button>

Demonstration

The video below shows how we are able to trigger a partial refresh after updating a document within the application. Note the screen flickers grey as the CSS change happens before the partial refresh. It is so quick it is almost instant.

Pushing to Bluemix

As before, pushing my new code to Bluemix is as simple as checking it in to the jazz hub repository and redeploying. In the picture below we can see that we are using marky.psclistens.com as the application domain and not copper.xomino.com. As we showed before, if not copper then use Bluemix.

b6

 

We create the capability locally, tested it, proved it and deployed it seamlessly to Bluemix with a “Commit and Push” – and that is *so* cool 🙂

Conclusion
In this article we saw how to trigger core XPages functionality automatically without the user having to interact with the application. In the next article we will look at how to improve on this, frankly horrible, user experience.

Thanks Dave 🙂

Creating an XPages Websockets chat client using Bluemix

In this article I will demonstrate how I was able to take an example Bluemix, node.js based, websocket chat program and re-purpose it to be used in XPages.

Introduction

Earlier this year I was very excited to find the Websockets in XPages project on OpenNTF published by Mark Ambler. The concept behind that project is to be able to create a notes document in a queue which is processed and then send out to all users. As much as I promised to help out and use the project, life and a business need to learn and use Angular.js got in the way. My abject apologies to Mark for not following through on my promise to help move the project along.

With this article though, I want to start my exploration of using an external Websockets server to transmit messages to my XPages applications. One of the *nice* things about Websockets is that unlike JavaScript AJAX they are *not* restricted by CORS. This means that I can host my Websockets server anywhere. In this case it is a win win for me as I get to learn more about Bluemix, node.js, websockets and other NoSQL databases like Redis in this case.

Creating a Websockets chat example in Bluemix

I found this article on Create an HTML5 chat app on Bluemix with Node.js, Redis, and Socket.io and following it through I was able to get my own chat program up and running within an hour or so.

ws1

It is not an especially difficult example to follow but I did find that it helped to understand a little about Bluemix and how a node.js application is put together. You should be able to figure it out though just following through the example.

The code for the original example can be found here https://hub.jazz.net/project/joelennon/bluemixchat/overview?welcome= if you want to take a look at it.

Porting the “client” to an XPage

Within the original example, the interface makes a connection to a Redis database to store the last 99 entries of the chat. Within the XPages example I could do that but I am not going to at this time. So the XPages interface will lose chat history when you refresh the page. I am not really concerned about that in the big picture.

Because I had all the files locally I was able to create a new IBM Domino database and drag the files into the WebContent directory. Within node.js the “Public” directory is assigned to the root of the server, but in this case I removed the public folder as it is unnecessary.

w2s2

 

The original example uses the jade templating engine to create the web page. But in this case I felt lazy and just viewed the source of the example once it was working and then extracted all the HTML I needed.

Moving socket.io to the HEAD

Because this is XPages and because we have dojo and I am sure I have pointed out before – we have to move the files to the HTML HEAD in such a manner as they come before Dojo within the application. Socket.io is apparently one such JavaScript library.

The HTML of the XPage is relatively simple. As you can see below we a using the xp:resources tag to insert the same HTML references to the local files as they were in the original example.

<?xml version="1.0" encoding="UTF-8"?>
<xp:view xmlns:xp="http://www.ibm.com/xsp/core">

	<link rel="stylesheet" href="//cdn.jsdelivr.net/normalize/3.0.1/normalize.min.css"/>
	<link rel="stylesheet" href="stylesheets/style.css"/>

	<xp:this.resources>
		<xp:headTag tagName="script">
			<xp:this.attributes>
				<xp:parameter name="type" value="text/javascript" />
				<xp:parameter name="src" value="//cdn.jsdelivr.net/jquery/2.1.1/jquery.min.js" />
			</xp:this.attributes>
		</xp:headTag>
		<xp:headTag tagName="script">
			<xp:this.attributes>
				<xp:parameter name="type" value="text/javascript" />
				<xp:parameter name="src" value="socket.io/socket.io.js" />
			</xp:this.attributes>
		</xp:headTag>
		<xp:headTag tagName="script">
			<xp:this.attributes>
				<xp:parameter name="type" value="text/javascript" />
				<xp:parameter name="src" value="javascripts/client.js" />
			</xp:this.attributes>
		</xp:headTag>
	</xp:this.resources>

	<h1>XPages Websocket Chat using Bluemix</h1>

	<input id="msg" autocomplete="off" autofocus="autofocus"/>
	<button type="submit">Send</button>

	<ul id="messages">
	</ul>

</xp:view>

 The critical difference – the server connection

Within the original example the socket code was on the same server as the client creating the messages. In this case they are not as my XPages server is wholly independent of Bluemix.

So I had to change the initial connection to the websocket server. Within the client.js file I changed the first line from

$(document).ready(function() {
  var socket = io(), nickname, msgList = $('#messages');
  ...

to the following

$(document).ready(function() {
  var socket = io.connect('http://xominosocket.mybluemix.net') //connect to the Bluemix server
  var nickname, msgList = $('#messages');
  ...

With these changes and a couple of stupid spelling mistake corrections I was able to bring my application up within my xSocket XPage.

ws3

You can see from the Firebug console that the copper.xomino.com application is talking to the xominosocket.bluemix.net application

Mobile Compatible

Yes you have to be connected to the website (rather than OS Push notification) but websockets works on iOS7+ and Android 4.4+

(http://caniuse.com/#feat=websockets)

Screenshot_2014-11-08-11-47-45

Conclusion

There is a lot more detail which we can go into as to how this example works but in the mean time if you want to play with it.

Here is the Bluemix Page: http://xominosocket.mybluemix.net/ (with chat history)

Here is an XPage: http://demo.xomino.com/xomino/WSinX.nsf/xSocket.xsp (no history)

For the full effect – open each link in a different browser and you can talk to yourself 😉

You can find my Bluemix code on Jazz Hub – https://hub.jazz.net/project/mdroden/xominosocket/overview.

WebSockets comes to XPages!! Awesome – Awesome – Awesome

In this article I will introduce and discuss the OpenNTF WebSockets OSGI plugin by Mark Ambler. The websocket plugin posted to Mark Ambler is based on the http://java-websocket.org/ project.

I realize I am not the first person to play with a websocket server on top of a domino database. I know of at least 4 other people who have at least done their own POC for this. But this is the first time I have got my hands on it 🙂 I’m a little excited……

Brief overview of WebSockets
I already wrote about the reasons for websockets in XPages. But to recap, WebSockets allows me the developer to PUSH messages out to logged in users without them asking for it. This reduces network traffic and creates a new paradigm for fluid data update.

What I will be doing over the next few blog posts is demonstrate examples of why WebSockets fundamentally changes how we architect our web apps in the future. Yes it is that significant in my mind.

How does the Domino integration work?

  • Download the OpenNTF project, unzip and follow the instructions in the PDF file.
  • Make sure you sign the update site before deploying
  • Install the update site in Domino Designer and restart
  • Understand that this is not port 80….this is demonstration right now and not production ready.

Once you have the plugin installed and up and running application you will be able to use the chat.nsf example to demonstrate for yourself what is going on. This is only intended to be an example of what is possible, not a production app. replacing sametime 🙂

From a Domino perspective, The core of the application is the websocket.nsf database. In there if you look at the Broadcast Example Agent you will see how it works.

w1

This article was written based on code release 1.0.7

By creating a document in the database with the appropriate fields, the websocket OSGI code picks up the document from the view and routes it out to the users.

The websocket.nsf database tracks users in the vUsers view and from there we can ascertain the unique code assigned to each logged in user.

Using that websocketID listed in the view we are able to ensure that the appropriate message is transmitted to the appropriate user.

To extend this into something I can use I create an SSJS function which will look up the socketID – in this case I do not need to know the ID – just the username.

function createMessage(socketID, msgObj){

	var db:NotesDatabase = session.getDatabase("","websocket.nsf",false);
	var doc:NotesDocument = db.createDocument()
	var online = ""
	if (!socketID){
		var view = db.getView("vUsers")
		println("rec: "+msgObj.recipient)
		var user:NotesViewEntry = view.getEntryByKey(msgObj.recipient)
		if (user){
			socketID = user.getColumnValues()[1]
			online =  user.getColumnValues()[2]
			println(socketID)
		}
	}

	if (socketID && (online=="ONLINE")){
		doc.replaceItemValue("Form", "fmSocketMessage")
		doc.replaceItemValue("from", msgObj.from)
		doc.replaceItemValue("to", socketID)
		doc.replaceItemValue("sentFlag", 0)

		//The msgObj will come in as an Array of Json - we need to convert that into a multi-value notes field
		//This is planned to be fixed to apss the JSON straight through in a later release

		for (var i=0; i<msgObj.data.length; i++){
			var temp = msgObj.data[i]
			for (key in temp){
				if (i<1){
					var item:NotesItem = doc.replaceItemValue("data", key+"="+temp[key])
				} else {
					item.appendToTextList(key+"="+temp[key])
				}
			}
		}
		doc.replaceItemValue("text", "")
		doc.save()
	} else {
		println("WebSocket messgae not sent")
	}
}

I can then call this function from an XPages button (or anywhere else for that matter) and pass in the data as an object.

	<xp:button value="Send Message" id="button1">
		<xp:eventHandler event="onclick" submit="true" refreshMode="complete">
			<xp:this.action>
				<xp:executeScript>
					<xp:this.script><![CDATA[#{javascript:
						var msgObj = {}
						msgObj.recipient = context.getUser().getDistinguishedName()
						msgObj.from = "Marky demo"
						msgObj.data = [{"beer": "english"}, {"lager": "french"}]
						createMessage("", msgObj)
					}]]></xp:this.script>
				</xp:executeScript>
			</xp:this.action>
		</xp:eventHandler>
	</xp:button>

Looking at the chat example below which I have also slightly modified to console.dir the output so that it is readable we can see the message is sent to the client

w2

 

In a more advanced demonstration I added a field to the right hand form so that i can pass through my own JSON – the video below shows this best 🙂

This was recorded at 1920 resolution so it appears blurred on youtube – CHANGE THE QUALITY to 720HD and that will fix it

OpenNTF WebSockets OSGI plugin

Mark Ambler

Caveat

Please bare in mind this is not production ready code – and it may never be……

Because there are two servers running (domino and websocket) the websockets are current running over a non-standard port. I believe IHS can be used as a proxy to fix that.

As a proof of concept though and to get the information out to the community and IBM (hint hint) this is a great step forward in capabilities.

This is a great example of using and OSGI plugin and the Java code in Domino to access a Java server and transmit data – very cool !

Conclusion

There is no polling of the server by the client. There are zero unnecessary http calls

We can have any application with an appropriate update, review the users who are logged in and update their web page with the appropriate data.

THIS IS AWESOME!!!!!!!!!!!!

Awesome!!!!!

Next – more examples

I feel a series coming on…….:)

 

Update

You know a project is moving along when this blog post was already out of date by the time I am ready to post it – but rather than waste it because there is good information in here – here’s the disclaimer.

This post is based on v1.0.7 of the webshell code release – as of v1.0.8 the JSON data is being passed via rich text which allows for pre-formatting server-side before sending it out. This also allows for more complex JSON creation than multi-value text fields can handle – thanks Mark – more about that soon !!

This change was based on my request for a feature on the github site

If you want to see this improve and progress – get involve – play with the code and make requests for changes – like everything OpenNTF – the more you play with it and the more people are involved – the better it will become !